Evidence, engagement & findings
Evidence categories
Section titled “Evidence categories”| Category | Examples |
|---|---|
| Scope and governance | Assessment mandate, boundary, roles, thresholds and approvals |
| System | System card, architecture, capabilities, limitations, intended and prohibited uses |
| Data and model | Provenance, quality, representativeness, evaluation, drift and change records |
| Deployment | Geography, culture, language, accessibility, workflow and integration evidence |
| Interested parties | Stakeholder mapping, engagement plan, participant input and response |
| Impacts | Scenarios, metrics, complaints, incidents, research, monitoring and distribution analysis |
| Measures | Treatment design, implementation, tests, owner, success criteria and residual impact |
Affected-party engagement
Section titled “Affected-party engagement”Engagement should be proportionate, inclusive and capable of influencing the assessment.
Consider:
- People directly subject to decisions.
- People indirectly affected.
- Vulnerable and marginalised groups.
- Workers and operators.
- Domain experts and civil society.
- Suppliers and downstream organisations.
- Public authorities where relevant.
Record participant selection, accessibility, language, compensation, safeguarding, questions, feedback, disagreements and how decisions changed. Consultation theatre—collecting views after the decision is fixed—does not provide meaningful evidence.
Impact tests and analysis
Section titled “Impact tests and analysis”Use methods suited to the impact:
- Disaggregated performance evaluation.
- Accessibility and usability testing.
- Scenario and misuse analysis.
- Human-factors and oversight testing.
- Privacy and security testing.
- Contestability and explanation review.
- Monitoring and incident trend analysis.
- Environmental or resource measurement.
Define the population, benchmark, threshold and limitation before claiming effectiveness.
Findings
Section titled “Findings”Raise a finding where:
- Scope omits a material affected group or use.
- Required information is unavailable.
- Engagement is absent or non-representative.
- A material impact has no measure or owner.
- A test fails.
- Monitoring cannot detect a relevant harm.
- A high maturity score lacks evidence.
- Residual impact exceeds an approved threshold.
- N/A is being used as a gap substitute.
Treatment and residual impact
Section titled “Treatment and residual impact”Treatment may avoid, reduce, share, monitor or—in a documented accountable decision—accept residual impact. For benefits, measures may increase reach or equitable distribution.
Record whether the measure transfers burden to another group or creates a new impact. Re-evaluate the complete system, not only the corrected metric.
Evidence checklist
Section titled “Evidence checklist”- Evidence belongs to the selected system and version.
- Affected parties and operating context are represented.
- Benefits and harms are both considered.
- Contrary evidence is retained.
- Tests have thresholds and limitations.
- Findings link to impacts and decisions.
- Measures have owners and effectiveness checks.
- Sensitive engagement records are protected.