PyPi package reference¶

Fairness Metrics Toolbox for Classification https://pypi.org/project/aiverify-fairness-metrics-toolbox-for-classification/

Install package, download data and model files¶

In [1]:

!pip install aiverify-fairness-metrics-toolbox-for-classification --quiet

!wget -q https://github.com/aiverify-foundation/aiverify/blob/e2a0099bf51837e516ef09ca7115cbcbd5d8896c/stock-plugins/aiverify.stock.fairness-metrics-toolbox-for-classification/algorithms/fairness_metrics_toolbox_for_classification/tests/user_defined_files/data/sample_mc_toxic_data.sav
!wget -q https://github.com/aiverify-foundation/aiverify/blob/e2a0099bf51837e516ef09ca7115cbcbd5d8896c/stock-plugins/aiverify.stock.fairness-metrics-toolbox-for-classification/algorithms/fairness_metrics_toolbox_for_classification/tests/user_defined_files/model/sample_mc_toxic_sklearn_linear.LogisticRegression.sav

!pip install aiverify-fairness-metrics-toolbox-for-classification --quiet !wget -q https://github.com/aiverify-foundation/aiverify/blob/e2a0099bf51837e516ef09ca7115cbcbd5d8896c/stock-plugins/aiverify.stock.fairness-metrics-toolbox-for-classification/algorithms/fairness_metrics_toolbox_for_classification/tests/user_defined_files/data/sample_mc_toxic_data.sav !wget -q https://github.com/aiverify-foundation/aiverify/blob/e2a0099bf51837e516ef09ca7115cbcbd5d8896c/stock-plugins/aiverify.stock.fairness-metrics-toolbox-for-classification/algorithms/fairness_metrics_toolbox_for_classification/tests/user_defined_files/model/sample_mc_toxic_sklearn_linear.LogisticRegression.sav

     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 61.0/61.0 kB 1.3 MB/s eta 0:00:00
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 18.3/18.3 MB 63.0 MB/s eta 0:00:00
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 13.3/13.3 MB 77.8 MB/s eta 0:00:00
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 85.8/85.8 kB 5.9 MB/s eta 0:00:00
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 60.8/60.8 kB 4.4 MB/s eta 0:00:00
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 75.9/75.9 kB 4.9 MB/s eta 0:00:00
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 85.5/85.5 kB 4.3 MB/s eta 0:00:00
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 22.9/22.9 MB 22.4 MB/s eta 0:00:00
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 4.5/4.5 MB 66.7 MB/s eta 0:00:00
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.2/1.2 MB 40.5 MB/s eta 0:00:00
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.2/1.2 MB 43.0 MB/s eta 0:00:00
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
ipython 7.34.0 requires jedi>=0.16, which is not installed.
google-genai 1.11.0 requires httpx<1.0.0,>=0.28.1, but you have httpx 0.26.0 which is incompatible.
thinc 8.3.6 requires numpy<3.0.0,>=2.0.0, but you have numpy 1.26.4 which is incompatible.

Execute Fairness test¶

In [ ]:

!ls

!ls

Execute the python module by providing necessary arguments including data path, model path, ground truth path, ground truth field and model type.

In [ ]:

!python -m aiverify_fairness_metrics_toolbox_for_classification \
--data_path "sample_mc_toxic_data.sav" \
--model_path "sample_mc_toxic_sklearn_linear.LogisticRegression.sav" \
--ground_truth_path "sample_mc_toxic_data.sav" \
--ground_truth toxic \
--model_type CLASSIFICATION \
--sensitive_features_list gender

!python -m aiverify_fairness_metrics_toolbox_for_classification \ --data_path "sample_mc_toxic_data.sav" \ --model_path "sample_mc_toxic_sklearn_linear.LogisticRegression.sav" \ --ground_truth_path "sample_mc_toxic_data.sav" \ --ground_truth toxic \ --model_type CLASSIFICATION \ --sensitive_features_list gender

In [7]:

!ls

!ls

sample_data		  sample_mc_toxic_sklearn_linear.LogisticRegression.sav
sample_mc_toxic_data.sav  test.log

In [7]:

Generated result will be stored in results.json file

In [8]:

!cat /content/output/results.json

!cat /content/output/results.json

cat: /content/output/results.json: No such file or directory

Upload the result.json file to the Portal to generate report if needed. Follow the detailed guide here for reference.