The vocabulary of defensible training data.

Agreement between two annotators, corrected for the agreement expected by chance. κ = 1 is perfect; κ = 0 is no better than chance. Regulated annotation QA contracts above 0.90.

How consistently independent annotators label the same data. High IAA evidences that labels reflect a shared, documented standard rather than individual opinion.

Reliability for any number of annotators, tolerant of missing data, across nominal, ordinal, interval and ratio labels - the general case where kappa handles exactly two.

A tamper-resistant record of every event in an AI system’s data and model lifecycle, kept automatically over the system’s lifetime.

The documented, verifiable trail of who handled training data, when, and what changed. It is usually the thing that decides an audit.

The policies, controls and evidence that make training data defensible - and how to operationalise them at the annotation stage rather than after the fact.

Platforms that manage risk, compliance and oversight across the AI lifecycle. Most stop at the model - the training-data layer is where behavior is actually decided.

Labeling raw data - images, video, text, audio, LiDAR, documents - so machine-learning models can learn from it.

Label Studio and CVAT are genuinely free to run, but they shift compliance, workforce, QA and audit costs onto you. When open source wins, and when it does not.

Partitioning audio into speaker-labeled segments - “who spoke when.” How DER measures quality, and why humans stay in the loop for regulated voice AI.