Tabular Metric Evaluator

class synthyverse.evaluation.eval.TabularMetricEvaluator(metrics, discrete_features=None, target_column='target', missing_imputation_method='drop', random_state=0)

Bases: object

Evaluator for tabular synthetic data quality metrics.

This class provides a unified interface for evaluating synthetic data quality across the dimensions fidelity, utility, and privacy, using various metrics.

Parameters:

• metrics (Union[dict, list]) – Dictionary mapping metric names to their parameters, or list of metric names (will use default parameters). Dictionaries can be used to specify metric hyperparameters, and compute different configurations of the same metric.

• discrete_features (list) – List of column names that are discrete/categorical. Default: [].

• target_column (str) – Name of the target column for supervised metrics. Default: “target”.

• missing_imputation_method (str) – Method for handling missing values. “drop” removes missing rows, other options perform imputation: “random”, “mean”, “median”, “most_frequent”, “missforest”. Default: “drop”.

• random_state (int) – Random seed for reproducibility. Default: 0.

Example

>>> import pandas as pd
>>> from synthyverse.evaluation import TabularMetricEvaluator
>>>
>>> # Prepare data
>>> X_train = pd.DataFrame(...)
>>> X_test = pd.DataFrame(...)
>>> X_syn = pd.DataFrame(...)
>>> discrete_features = ["category_col"]
>>> target_column = "target"
>>>
>>> # Define metrics
>>> metrics = ["mle"]
>>>
>>> # Compute different configurations of the same metric by adding a dash to the metric name
>>> metrics = {"mle-trts": {"train_set":"real"}, "mle-tstr": {"train_set":"synthetic", "tune":True}}
>>>
>>> # Create evaluator
>>> evaluator = TabularMetricEvaluator(
...     metrics=metrics,
...     discrete_features=discrete_features,
...     target_column=target_column
... )
>>>
>>> # Evaluate synthetic data
>>> results = evaluator.evaluate(X_train, X_test, X_syn)

evaluate(X_train, X_test, X_syn, X_val=None)

Evaluate synthetic data quality using specified metrics.

Parameters:

• X_train (DataFrame) – Training data as a pandas DataFrame.

• X_test (DataFrame) – Test data as a pandas DataFrame.

• X_syn (DataFrame) – Synthetic data as a pandas DataFrame.

• X_val (Optional[DataFrame]) – Optional validation data as a pandas DataFrame.

Returns:

Dictionary mapping metric names to their evaluation results.

Return type:

dict