AlphaPrecisionBetaRecall

class synthyverse.evaluation.fidelity.AlphaPrecisionBetaRecall(discrete_features=[], k=2)

Bases: object

Registry name: alphaprecisionbetarecall

Alpha-Precision, Beta-Recall score.

Paper: “How faithful is your synthetic data? sample-level metrics for evaluating and auditing generative models” by Alaa et al. (2022).

Parameters:

  • discrete_features (list) – List of discrete/categorical feature names. Default: [].

  • k (int) – Number of nearest neighbors to use in Beta-Recall. Default: 2.

Example

>>> import pandas as pd
>>> from synthyverse.evaluation import AlphaPrecisionBetaRecall
>>>
>>> # Prepare data
>>> X_real = pd.DataFrame(...)
>>> X_syn = pd.DataFrame(...)
>>> discrete_features = ["category_col"]
>>>
>>> # Create metric
>>> metric = AlphaPrecisionBetaRecall(
...     discrete_features=discrete_features,
...     k=2
... )
>>>
>>> # Evaluate
>>> results = metric.evaluate(X_real, X_syn)
evaluate(X_train, X_syn)

Evaluate synthetic data using alpha-precision and beta-recall.

Parameters:

  • X_train (DataFrame) – Real training data as a pandas DataFrame.

  • X_syn (DataFrame) – Synthetic data as a pandas DataFrame.

Returns:

Dictionary with keys:

  • ”alphaprecisionbetarecall.alpha_precision”: Alpha-precision score

  • ”alphaprecisionbetarecall.beta_coverage”: Beta-coverage score

Return type:

dict