# Fairness Assessment in Clinical Prediction Models

**Planimeter Kft.**  
Version 1.0.0 - 2026-09-24

This package contains the reproducible materials supporting the Planimeter methodological example **Fairness Assessment in Clinical Prediction Models**. The project shows how acceptable aggregate predictive performance can coexist with materially different subgroup error profiles, and how fairness diagnostics can be incorporated into routine model validation and governance.

## What is included

- `fairness_analysis_planimeter_v1.0.0.R` - fits logistic regression and random forest models, calculates subgroup performance, fairmodels parity diagnostics, the case-study Aggregate Parity Deviation (APD), cutoff sensitivity, and publication figures.
- `fairness_data_preparation_planimeter_v1.0.0.R` - rebuilds the compact analysis dataset from the `riskCommunicator::framingham` teaching dataset.
- `data/framingham_analysis_dataset.csv` - analysis-ready data with the fixed development train/test split.
- `reference/` - numerical outputs from the development run used for validation.
- `output/validation_checks.csv` - independent checks of supplied results.
- `output/dataset_checks.csv` - data-integrity diagnostics.
- `DATA_DICTIONARY.md` - description of released fields and reference files.
- `LICENSE_GPL-3.0.txt` - software license for Planimeter R code.

The `figures/` directory is populated when the main R analysis script is executed.

## Data and model setup

The source is the Framingham teaching dataset distributed with the R package `riskCommunicator`. After selecting the analysis variables and removing incomplete rows, the released dataset contains **11,173 observations**. A fixed seed (`20260610`) creates a 70/30 development split: **7,821 training** and **3,352 test** observations.

Two prediction models are compared:

- logistic regression;
- random forest (`ranger`, 500 trees).

Both use the same predictors and are evaluated on the same held-out test set. The classification cutoff for the primary fairness assessment is 0.50.

## Validated results

The test set contains **1,466 male** and **1,886 female** observations. Cardiovascular event rates differ substantially: **33.6% in men** and **18.3% in women**.

Overall test performance:

- Random forest AUC: **0.725**; accuracy at 0.50 cutoff: **75.0%**.
- Logistic regression AUC: **0.749**; accuracy at 0.50 cutoff: **76.4%**.

Subgroup discrimination also differs. Random-forest AUC is **0.656 in men** and **0.733 in women**; logistic-regression AUC is **0.709 in men** and **0.743 in women**.

At the 0.50 cutoff, the random forest has an APD of **3.808**. The logistic-regression parity ratios imply an APD of **4.222**. APD is defined here as the sum of absolute log deviations from parity across Accuracy Equality, Equal Opportunity, Predictive Equality, Predictive Parity and Statistical Parity.

The cutoff analysis demonstrates that fairness is not a fixed property of the fitted model. For example, the random forest has approximately the same overall accuracy at cutoffs 0.50 and 0.70 (**75.0% vs 75.0%**) while APD changes from **3.81 to 1.95**. This is a diagnostic observation, not a recommendation to use a particular cutoff: the individual error-rate and parity components must still be examined.

## Interpretation and methodological scope

This is a methodological model-audit demonstration, not a determination that either model is fair or unfair. Differences in subgroup event prevalence affect several performance measures, and parity ratios answer different questions. A value close to 1 indicates similarity for that metric relative to the illustrative privileged group, but there is no single universally correct fairness definition.

`Male` is designated as the privileged group only to reproduce the demonstration. That coding choice is not a normative statement.

APD is **not** a `fairmodels` statistic. It was introduced specifically for this case study as a compact summary of five parity ratios. Lower APD means those selected ratios are collectively closer to 1, but the summary can hide which errors changed and should not replace inspection of the component metrics.

The binary sex field and the historical Framingham teaching data are additional limitations. The fitted models are not validated clinical decision tools and should not be used for patient-level decisions.

## Running the analysis

Run from the project root:

```text
Fairness_Assessment_Planimeter_v1.0.0/
|-- fairness_analysis_planimeter_v1.0.0.R
|-- fairness_data_preparation_planimeter_v1.0.0.R
|-- README.md
|-- DATA_DICTIONARY.md
|-- LICENSE_GPL-3.0.txt
|-- data/
|   `-- framingham_analysis_dataset.csv
|-- reference/
|-- figures/
`-- output/
```

Required R packages for the main analysis are `dplyr`, `readr`, `ggplot2`, `scales`, `yardstick`, `ranger`, `DALEX`, `fairmodels`, `purrr`, and `tidyr`. The data-preparation script additionally requires `riskCommunicator`.

## Validation note

The supplied reference outputs were independently recalculated from the released CSV files during package preparation. All numerical checks in `output/validation_checks.csv` passed within numerical tolerance. The packaging environment did not contain an R runtime, so the final R scripts were reviewed structurally but were not executed there; they should receive one final local R run before publication.

## License and AI disclosure

The Planimeter R source code is released under the **GNU General Public License v3.0 or later (GPL-3.0-or-later)**.

Generative AI was used as a support tool during drafting, code development, review and validation. The analytical design, interpretation, final materials and responsibility for the release remain with Planimeter Kft.

The GNU GPL applies to the Planimeter R source code and does not replace or modify any terms applying to the source dataset distributed with `riskCommunicator`.