Specify the decision and who may be affected
Start with the real decision, the people affected, and the harms of each error. A classifier that flags transactions for review has different consequences from one that automatically denies an application. Choose meaningful slices with domain and affected-community input, subject to appropriate privacy protections and applicable requirements. Document how labels were collected: historical decisions or outcomes observed for only some people may encode past inequities or leave systematic gaps.
A lending model predicts repayment, but repayment labels exist only for applicants who previously received loans. Explain why comparing only those observed labels may not describe performance for all applicants.
Calculate group metrics from confusion matrices
Suppose group A has 32 true positives, 8 false negatives, 12 false positives, and 48 true negatives. Group B has 12, 8, 8, and 72 respectively. Both groups contain 100 examples. Recall is 32/40 = 80% for A and 12/20 = 60% for B. False positive rate is 12/60 = 20% for A and 8/80 = 10% for B. Positive prediction rates are 44% and 20%. These describe different outcomes; none alone settles whether the system is fair.
Compute the positive prediction rate and recall for each group using the counts above. Then state the denominator for false positive rate and explain why comparing raw error counts can mislead when group sizes differ.
Choose measures that fit the harm
Demographic parity compares positive prediction rates. Equality of opportunity compares true positive rates for the designated positive outcome. Equalized odds compares both true positive and false positive rates across groups. These criteria can conflict depending on the data and setting, although they can also coincide. Select measures before assessing the model, consider label quality and the meaning of the positive outcome, and review uncertainty: a small slice can produce a volatile estimate. Do not infer an individual’s experience or the cause of a disparity from an aggregate metric alone.
In the worked example, do equal positive prediction rates or equal true positive rates hold? Neither: 44% differs from 20%, and 80% differs from 60%. Identify what additional information you need before deciding how to respond.
Use explanations to investigate, then test the remedy
Feature attribution estimates how features contribute to a model output under a particular explanation method and reference. It can reveal a surprising dependence worth investigation, but it does not establish causation, prove a model is fair, or replace slice evaluation. Inspect data provenance, missingness, proxy variables, and errors; test proposed changes on a separate evaluation set. Document limitations, group sample sizes, uncertainty, review decisions, and an escalation or rollback plan for post-release issues.
A model heavily relies on postal area. List two plausible explanations, one potentially harmful proxy relationship, and a concrete follow-up analysis. Avoid concluding that removing the feature alone eliminates all proxy information.
Group A has 32 true positives and 8 false negatives; group B has 12 true positives and 8 false negatives. What are their recalls?
Keep exploring
Use the official certification page and its linked resources to validate your coverage. For guided labs, explore the Google Skills learning path. Revisit the practical exercise and explain your decision in your own words before moving on.