@jdi
Kingston, Canada – August 2026 – The John Deutsch Institute (JDI) has released a new JDI Policy Insight paper, Fairness in AI-Assisted Decisions: Five Insights for Policymakers, written by Tanvir Ahmed Khan, a PhD candidate in the Department of Economics at Queen’s University. Algorithms now inform decisions about credit, hiring, benefits, healthcare, and pretrial release. The paper argues that the question policymakers usually ask about them is the wrong one.
That question is whether the algorithm is fair. In most high-stakes settings the algorithm does not make the decision: it produces a score that a loan officer, case worker, recruiter, or judge then interprets. The outcome depends on the model and on how people and institutions use it. Canadian federal guidance already reflects this, since the Treasury Board’s Directive on Automated Decision-Making covers tools that support human decision-makers rather than replace them.
Khan draws five points from the literature. Competing definitions of fairness can conflict, and the conflict is sometimes mathematical rather than political: the dispute between ProPublica and Northpointe over the COMPAS recidivism tool arose because a score can be calibrated across racial groups while still producing different error rates, when underlying base rates differ. A more accurate model does not guarantee smaller disparities, since sharper predictions filtered through biased human cutoffs can widen gaps rather than close them. A disparity that grows after adoption does not reveal its own cause, which might be the training data, human overrides, or a shifted threshold. How fairness risks are framed changes what organizations do about them, with experimental evidence showing that managers told algorithmic bias is inescapable become more likely to abandon the tool for unguided human discretion. And evaluating a model is not the same as evaluating the decision process built around it.
Khan recommends:
- Specify the fairness objective before deployment. Decide in advance whether the priority is equal treatment of similar cases, parity in error rates, or equitable access, since post-deployment review is hard to interpret without a stated benchmark.
- Evaluate the decision process, not just the model. Assessment should trace the full chain from training data through thresholds, override rules, and exception handling to the final allocation.
- Track how human discretion changes after adoption. Log when recommendations are overridden, by whom, and why, and review how override patterns are distributed across groups.
- Treat model upgrades as governance events. Better prediction can expose institutional bias that noise previously obscured, so a material performance change should trigger renewed review of the surrounding procedures.
- Communicate fairness risks without fatalism. Internal reporting should separate identifying a risk from concluding the system should be abandoned, and state the benchmark being used.
Khan’s assessment is that Canada’s existing architecture already recognizes the decision process as the policy object, but does not yet ensure departments generate evidence about the operational margin where scores become decisions. Closing that gap is the next stage of responsible AI governance.
The full paper is available as JDI Policy Insight 26-0802 on the JDI website at jdi.queensu.ca.
About the John Deutsch Institute and the JDI Policy Insight series
The JDI at Queen’s University conducts rigorous, policy-relevant economic research to inform decision-makers in government, industry, and civil society. Through events, publications, and collaborations, the Institute fosters evidence-based dialogue on critical economic challenges facing Canada and the world. The newly launched JDI Policy Insight series provides balanced academic summaries of key policy issues, helping readers assess the benefits, costs, and risks of reforms and understand what we know and don’t know about policy.
