Question 1

Why would I want to publish a model card if it reveals weaknesses?

Accepted Answer

A model card that only disclosed strengths would be marketing, not documentation. A real model card discloses known limitations, demographic performance gaps, and failure modes. Yes, this reveals weaknesses. But revealing weaknesses is the whole point. A user who doesn't know their AI system is 15% less accurate on non-English inputs cannot make informed deployment decisions. A bank that doesn't know their lending model has 3% higher false-negative rates for applicants over 65 cannot take corrective action. A health system that doesn't know their diagnostic model fails on rare diseases cannot properly supervise it. Model cards exist to enable informed consent - the model's operator and the affected users both understand what they're getting. Hiding weaknesses doesn't make them disappear; it just makes them harder to mitigate. Transparency is harder to do than secrecy, but it builds long-term trust.

Question 2

What should a comprehensive model card include?

Accepted Answer

Google's original model card framework (2019) specifies: (1) Model details - name, version, date, developers, architecture. (2) Training data - sources, preprocessing, demographics, limitations. (3) Evaluation results - performance metrics (accuracy, precision, recall, F1, AUC) broken down by subgroup (age, gender, race, language, geography). (4) Quantitative analyses - performance gaps between groups, on out-of-distribution data, on adversarial examples. (5) Ethical considerations - potential harms, mitigation strategies, intended use cases. (6) Caveats and recommendations - who should / shouldn't use this model, how to monitor for drift. A minimal model card covers items 1 - 3. A comprehensive card covers all six plus field-specific additions (e.g., healthcare models should disclose sensitivity/specificity, fairness metrics; legal models should disclose domain coverage and precedent coverage).

Question 3

How do I get subgroup performance data if I don't have labels for protected attributes?

Accepted Answer

This is real problem. Your training data may not include explicit demographic labels (age, race, etc.). You have a few options: (1) Proxy signals - use correlated features as imperfect proxies. If you don't have explicit race labels, geographic and name data provide signals (imperfect and potentially offensive, but better than nothing). (2) External auditing - hire a third party to evaluate performance on a diverse dataset they control. (3) User testing - deploy to a diverse user group and measure differences in real-world performance. (4) Synthetic data - generate synthetic test cases representing different demographics and evaluate your model. (5) Transparency about what you don't know - in the model card, explicitly state which subgroups you haven't evaluated and why. This honesty is valuable; it tells downstream users 'we tested X and Y, but we couldn't evaluate Z due to data limitations.' BorealisMark actually scores honesty about limitations highly - incomplete evaluation that you acknowledge is better than complete evaluation that you don't disclose.

Question 4

What's the relationship between model cards and legal liability?

Accepted Answer

In most jurisdictions, a model card is not yet a legal requirement (except in specific domains like high-risk AI under the EU AI Act). However, from a liability standpoint, model cards provide evidence of due diligence. If you deploy an AI system without understanding its performance characteristics, and it causes harm, you may be liable for negligence. If you deployed it after careful analysis documented in a model card, you have evidence that you acted responsibly given available information. The inverse is also true: if you knew about a known limitation and deployed anyway without disclosing it, a model card becomes evidence of negligence. Many organizations skip model cards precisely because they're concerned about liability - they assume that documenting known problems creates legal risk. In reality, the risk comes from deploying known-broken systems without disclosure. A model card that says 'This model has a 5% error rate on X' is better legally than a model with the same error rate that you deployed without knowing or documenting it.