An AI audit trail is a tamper-resistant, chronological record of events across an AI system’s lifecycle - including how its training data was created, by whom, and under which controls. For high-risk systems, EU AI Act Article 12 makes automatic, traceable logging mandatory from 2 December 2027. A complete annotation audit trail records every label, edit, review decision, configuration change and access, and exports as a chain-of-custody document.
For the training-data stage: dataset ingestion events (source, hash, classification); every annotation action with actor identity and timestamp; model-assist provenance (which model pre-labeled, and who accepted or corrected it); review and adjudication decisions; guideline and ontology versions in force; configuration and permission changes; and every access to the data. Immutability matters as much as coverage - a log that can be edited is an anecdote.
Post-incident, the first questions are provenance questions: where did the training data come from, who touched it, what quality controls ran? EU AI Act Article 12 codifies this for high-risk systems; HIPAA’s audit controls, GDPR’s accountability principle and financial model-risk frameworks make the same demand in their own vocabulary.
LabelFort’s Evidence Export packages the audit trail per dataset: event log (100% coverage), annotator & verifier roster and role separation, IAA scores per cohort against acceptance criteria, guideline versions, and export manifest with hashes. It is designed to be filed - into AI Act technical documentation, a QMS, or a model-risk review - without reformatting.
Agreement between two annotators, corrected for the agreement expected by chance. κ = 1 is perfect; κ = 0 is no better than chance. Regulated annotation QA contracts above 0.90.
Read More
How consistently independent annotators label the same data. High IAA evidences that labels reflect a shared, documented standard rather than individual opinion.
Read More
Reliability for any number of annotators, tolerant of missing data, across nominal, ordinal, interval and ratio labels - the general case where kappa handles exactly two.
Read More
For high-risk systems under the EU AI Act, yes. Article 12 requires tamper resistant automatic logging, and Article 10 requires documented data governance over annotation and labeling, both from 2 December 2027 under the Annex III timeline. Sectoral rules, HIPAA and model risk guidance among them, impose parallel duties in their own terms.
Append only storage, cryptographic hashing or chaining of events, strict separation between the actor's performing actions and whoever administers the log, and exportability for independent verification.
The two aren't quite interchangeable. An audit trail is the continuous record of everything that happened. A chain of custody document is the specific export of that record for one dataset, packaged to answer who touched it, when, and under what authority. Every chain of custody export should be traceable back to the underlying audit trail, but not every audit trail entry needs to be repackaged into a standalone export until someone asks for it.
The documented trail of a training dataset's handling: sources, handlers, transformations, accesses, and exports, verifiable end to end. It underpins the EU AI Act technical documentation and sectoral audit requirements.
Hash verified ingestion, named actors under role based access, immutable action logs, versioned guidelines, and per dataset Evidence Exports that package the full custody record for your auditors.
IAA scored per cohort, audit trails on every action; evidence of exports mapped to EU AI Act Articles 10 & 12. See it on your own data in an evidence-grade PoC.
Certifications & readiness
