What is a model card?

A model card is a standardized document that accompanies a machine learning model and describes, in fixed sections, what it does, what it is meant for, who it was evaluated on and where it fails. The format was introduced by Margaret Mitchell and colleagues at Google in Model Cards for Model Reporting (2019) as an answer to a specific problem: models were shipping with an aggregate accuracy metric and little else, leaving whoever adopted them to discover on their own, in production, for whom and under what conditions that number did not hold. The paper's standard sections are model details, intended use and explicitly out-of-scope uses, relevant factors that can shift performance, evaluation metrics disaggregated by subgroup, ethical considerations and known limitations. It is not a marketing document or a plain changelog: it is an information contract between whoever trains the model and whoever will use it.

The link to hidden stratification

The most important section, and the one most often filled in poorly or missing outright from model cards published outside research standards, is the one on metrics disaggregated by subgroup. A model card that reports only an aggregate accuracy reproduces exactly the problem described by hidden stratification: a high average number hiding a systematic failure on a subgroup not explicitly labeled in the data. The evaluation section of the Mitchell et al. format exists to prevent this outcome upstream, requiring whoever documents the model to break performance down by gender, ethnicity, language, skin type or any other relevant factor named in the previous section, before the model reaches whoever has to adopt it. A complete model card does not eliminate the risk of hidden subgroups not yet identified, but it makes it verifiable, in a single document, whether the vendor at least disaggregated what it had planned to measure.

An enterprise example

A team evaluating an open-weight document classification model for a back-office pipeline receives, along with the weights, a model card that declares 94% aggregate accuracy but, in the "factors" section, explicitly states it was evaluated only on English and German documents, with a caveat noting unverified performance on non-Latin alphabets. That single piece of information, absent from a plain spec sheet carrying only the average metric, saves the team from discovering in production, on Italian documents with accented characters or scanned forms in other languages, an error rate that no aggregate test would have flagged before release.

Why it matters for decision makers

For whoever has to decide whether to adopt a third-party model, the model card is the first document to ask for and read in full, not just for the accuracy line: in AI BOM due diligence it is the primary source the other inventory entries derive from, and it is often the only place where a vendor explicitly documents where its own model was not tested. The absence of a model card, or a model card reporting only aggregate metrics with no subgroup breakdown, is itself a risk signal to treat as such before signing a contract, not after the first incident in production.