ClassifierMIA

class synthyverse.evaluation.privacy.ClassifierMIA(ref_prop=0.5, model_name='randomforest', model_params=None, discrete_features=None, member_prop=1.0, repeats=1, subsample=False, random_state=0)

Bases: MIA

Registry name: mia.classifier

Classifier-based membership inference attack metric.

Trains a classifier to distinguish between the synthetic and reference distributions.

Parameters:

  • ref_prop (float) – Proportion of test set to use as attacker reference non-members. Default: 0.5.

  • model_name (str) – Model used for the attack classifier. Default: “randomforest”.

  • model_params (dict) – Optional parameters for the attack classifier.

  • discrete_features (list) – List of discrete/categorical feature names. Default: [].

  • member_prop (float) – Proportion of train set to use as members. Default: 1.0.

  • repeats (int) – Number of repeated evaluations when subsampling records. Default: 1.

  • subsample (bool) – Whether to subsample the synthetic set to the same size as the attacker reference set, and the member set to the same size as the evaluation non-member set. If False, all synthetic records and all members are used, and the metric is evaluated once. Default: False.

  • random_state (int) – Random seed for reproducibility. Default: 0.

evaluate(X_train, X_test, X_syn)

Evaluate membership inference risk.

Parameters:

  • X_train (DataFrame) – Real training data whose rows are treated as members.

  • X_test (DataFrame) – Independent real test data split into reference records and evaluation non-members.

  • X_syn (DataFrame) – Synthetic data available to the attacker.

Returns:

Dictionary with attack AUC and lift-at-k scores. Keys have the

form “<attack_name>.<score>”.

Return type:

dict