The test your trail has to pass
An audit trail is not a log file. It is the set of records that lets a person who was not in the room reconstruct a decision: what the AI was asked, what it was asked it against, what it answered, what a human did about that answer, and when the result became final. If any link in that chain is missing, the decision rests on memory and goodwill - which is exactly what an appeal, an internal audit, or a regulator will test.
1. Model identity and version, per evaluation
AI models change under you. An evaluation made in March and an evaluation made in August may be made by different model versions with different behaviour. Record the exact model identifier against every single evaluation, not as a general platform fact. When someone asks "would this candidate have scored differently today?", the honest answer starts with knowing what scored them then.
2. A configuration snapshot from scoring time
Rubrics get edited. Grading bands get adjusted. If your trail points at the current rubric rather than the rubric as it stood when the answer was scored, every past decision silently changes its meaning each time you improve your criteria. The trail must freeze the configuration - criteria, levels, weights, grading schema - at the moment of evaluation, and the frozen copy is the one an auditor sees.
3. Evidence citations, per criterion
A score with no evidence is an opinion with a number attached. For every rubric criterion, the trail should record which part of the respondent's actual answer put them at that level - quoted, not paraphrased. This is the record that turns an appeal from an argument about fairness into a conversation about specific evidence, and it is the single strongest signal that separates purpose-built assessment AI from pasting answers into a chatbot.
4. The human decision record: confirm, override, comment
Human oversight only counts if it leaves a mark. The trail needs a ledger of what each assessor did with each AI-suggested score: confirmed as-is, or overridden to a new value, by whom, when, and with what comment. The ledger serves two masters at once - it evidences oversight for the auditor, and it produces the agreement data that tells you whether the AI is actually reliable on your rubric.
5. The release record
Nothing should reach the person being assessed until someone accountable decides it does. The trail records who released each result and when - and, just as importantly, makes it structurally impossible for a result to be visible before that record exists. For certification and other consequential decisions, the platform should prompt for confirmation and record it, so the accountability chain ends at a named person, not at a model.
6. Explicit exclusions - what the AI did not score
Every real assessment has parts the AI deliberately did not touch: context-only questions, deterministically marked multiple choice, media that failed transcription, responses excluded from quality metrics. An audit trail that stays silent about exclusions invites the assumption that everything was AI-scored. State what was out of scope, per response, in the record itself.
7. Export - the trail has to leave the building
An audit trail that only exists inside a vendor's UI is a trail you lose the day you change vendors or the day an auditor wants the file. The full record - scores, evidence, ledger, configuration, release - should export in a portable format on demand, without a support ticket.
The auditor's five questions, mapped
- "How was this score produced?" - model version + configuration snapshot + evidence citations.
- "Who was accountable for it?" - the override ledger and the release record, both naming people.
- "Would the candidate have known AI was involved?" - the disclosure shown before they started, kept with the assessment record.
- "What did the AI not do?" - the explicit exclusions.
- "Can I take this away and check it?" - the export.
If your current process can answer all five without anyone reconstructing history from emails, you have an audit trail. If it cannot, that is the gap to close before scale makes it permanent - and it is worth closing whether or not a regulator ever asks, because the same records are what make your own quality conversations honest.
For how these records behave in a live product, see how scoring works and the AI transparency page.
Common questions
- Why does AI-assisted assessment need an audit trail at all?
- Because assessment decisions get challenged. A candidate appeals a score, an auditor samples a cohort, a regulator asks how a decision was made. If the answer is "the AI said so", the decision is indefensible. An audit trail turns each score into a reconstructable decision: which model, against which rubric version, citing which evidence, confirmed or overridden by which person, released when.
- What should be recorded for every AI evaluation?
- At minimum: the model and version used, a snapshot of the rubric and grading configuration as it stood at scoring time, the evidence the AI cited for each criterion, any human override with who, when and the comment, the release record, and an explicit note of anything that was deliberately not AI-scored.
- Is a released result the same as a reviewed result?
- No, and conflating them is a common audit failure. Release proves someone pressed a button; review proves someone engaged with the scores. A defensible trail records both separately, and honest agreement metrics count only genuinely reviewed responses.
- How do audit trails relate to the EU AI Act?
- The Act expects transparency towards the people being assessed and human oversight of consequential decisions, and record-keeping is how oversight is evidenced. An audit trail that captures configuration, evidence, human decisions and disclosure is the practical backbone of compliance - though the trail is worth keeping regardless of jurisdiction.