Dimension assessor playbooks

This page is designed so an assessor can complete most of the dimension analysis without leaving the ACRS screen. Use the playbook for each dimension alongside the selected system’s actual intake, architecture and operating evidence.

How to use the playbooks

For each dimension:

1. Fix the boundary and deployment being assessed.
2. Ask the diagnostic questions.
3. Inspect objective evidence.
4. Perform a bounded test or exercise where appropriate.
5. Compare the result with the scoring anchors.
6. Record the rationale, uncertainty and reassessment trigger.
7. Raise findings where the claimed boundary is not demonstrated.

Do not score from policy intent alone. Assess effective capability and real operating conditions.

Operational Dependency (dep)

What it measures

Operational Dependency asks:

How hard would it be for the organisation, customers or affected people to continue safely if the AI were wrong, unavailable, degraded or withdrawn?

It includes dependency on the full service chain, not only the model:

• Model or provider.
• Prompts, policies and configurations.
• Data and vector indexes.
• Identity and secrets.
• Connectors and APIs.
• Infrastructure and networks.
• Monitoring and human operating capacity.

Scope

Identify:

• The exact business service, decision path or customer journey.
• Regulated, safety, security or critical-service functions.
• Upstream and downstream dependencies.
• Operating hours and service objectives.
• Supplier concentration and substitution options.
• Manual fallback and safe-degradation paths.
• Recovery-time and recovery-point objectives.
• Population, transaction volume and decision volume affected.

Assessment method

Walk the process in four states:

1. Correct AI operation.
2. Degraded or intermittently wrong output.
3. Complete outage.
4. Provider, model or critical integration withdrawal.

Measure:

• Time to business, safety, legal or customer impact.
• Backlog growth.
• Decision-quality degradation.
• Safe operating duration.
• Manual capacity at realistic volume.
• Recovery dependencies.
• The point at which the fallback ceases to meet approved tolerances.

Diagnostic questions

• What outcome stops, becomes delayed or becomes unsafe?
• How long can the process continue before a material threshold is breached?
• Is the fallback documented, staffed, accessible and tested?
• Has fallback capacity been tested at realistic demand?
• Which model, data, identity, integration, infrastructure or supplier failures remove the service?
• Can the organisation substitute another model or provider without losing necessary policy, configuration, data or assurance?
• Who can suspend the AI?
• Who owns continuity, recovery and risk acceptance?
• Does the business plan or customer commitment assume continuous AI availability?
• Can the workflow operate safely when the AI produces plausible but wrong output rather than becoming fully unavailable?

Evidence to inspect

• Current process maps and service dependency diagrams.
• Business-impact analysis.
• Service-level objectives and error budgets.
• Manual fallback procedures and staffing plans.
• Capacity evidence for fallback.
• Provider and integration SLAs.
• Substitution and exit plans.
• Recovery runbooks.
• Backup and restore evidence for configuration, data, indexes, prompts and policies.
• Failover, rollback or continuity exercise records.
• Incident and near-miss records.
• Named service, continuity and recovery owners.

Policies and diagrams support the analysis but do not prove fallback operation. Seek sampled records or exercise results.

Safe test

Prefer:

• Tabletop exercise.
• Feature-flag disablement.
• Shadow workflow.
• Controlled failover.
• Non-production recovery exercise.

Define before testing:

• Customer, legal and safety protections.
• Stop conditions.
• Data and log preservation.
• Rollback steps.
• Service-owner approval.
• Maximum outage or degraded-operation window.

Pass criteria

• Fallback starts within the approved objective.
• Required decisions remain safe, attributable and compliant.
• Backlog and quality remain within approved tolerance.
• Recovery covers data, configuration, identity, integrations, policy and monitoring.
• Restoration is verified, not merely declared complete.
• Exceptions create findings with owners and dates.

Scoring anchors

Level	Anchor
1 — Low	The AI is genuinely optional or readily substitutable, and a tested fallback sustains safe operation at realistic demand.
2 — Medium	The AI is materially important and disruption is significant, but a demonstrated fallback or safe-degradation path keeps the service within approved tolerance.
3 — High	The AI is critical, regulated, safety, security, financial or customer-impacting; wrong output or loss rapidly breaches tolerance; or no practical fallback exists.

Borderline decisions

• An untested fallback prevents a confident Low conclusion.
• A fallback that works only at trivial volume is not a practical fallback.
• Continuing service by accepting material safety, legal, integrity or customer degradation supports High, not Medium.
• A supplier SLA reduces expected outage frequency; it does not reduce organisational dependency unless substitution or recovery is demonstrated.
• “Humans can do it manually” is not evidence unless capacity, access, competence and time have been tested.

Red flags

• Production objectives assume AI availability while the assessment calls it optional.
• Fallback exists only in a document.
• One individual or supplier is the only workaround.
• Recovery excludes prompts, policies, indexes, credentials or integrations.
• The workflow cannot detect plausible but wrong output.
• Adoption or transaction growth does not trigger reassessment.
• No owner can authoritatively suspend or restore the service.

Control and monitoring implications

Prioritise:

• Service ownership and dependency inventory.
• Safe degradation.
• Capacity-tested fallback.
• Provider or model substitution.
• Configuration and data recovery.
• Rollback and restoration exercises.
• Change-triggered reassessment.

Monitor:

• Availability and error budget.
• Fallback activation time and capacity.
• Backlog and decision-quality degradation.
• Supplier, model, data and integration health.
• Recovery-objective breaches.
• Adoption and volume growth.

Finding triggers

• No viable critical-path fallback.
• Untested restoration or substitution.
• Dependency level unsupported by operating evidence.
• Service commitments contradict an optional-use claim.
• Incident or near miss without reassessment.

Completion record

Record the service boundary, dependency graph, failure horizon, fallback owner and capacity, recovery objective, evidence sampled, exercise result, level, rationale, confidence, exceptions, treatment owner, review date, monitoring and reassessment triggers.

Action Autonomy (act)

What it measures

Action Autonomy asks:

What can the AI decide, initiate, execute, repeat or delegate before authoritative human approval?

The important boundary is not whether a human is “involved”. It is whether the consequential effect can occur before an effective approval or intervention.

Scope

Inventory:

• Decisions and recommendations.
• Tool calls.
• Writes, executions, deletions and external communications.
• Transactions and financial actions.
• Workflow continuation and retries.
• Delegation and sub-agents.
• Background and scheduled tasks.
• Approval gates.
• Rollback and compensation.
• Kill switches and circuit breakers.
• Alternate APIs or paths.

Assessment method

Trace representative and worst-case tasks from trigger to terminal effect. Determine:

• What the AI chooses.
• What it initiates.
• What it can repeat.
• What it can delegate.
• What it can make difficult to detect.
• What becomes irreversible.
• Whether alternate tools, retries, memory or sub-agents bypass the intended gate.

Diagnostic questions

• What consequential effect can occur before human approval?
• Is the gate enforced outside the model?
• Does every action path use the same authoritative gate?
• Can denial be bypassed by retry, reframing, delegation or another tool?
• Can a human detect and stop the action within the practical response window?
• Can the effect be reversed or compensated?
• What changes at production volume?
• Are step, time, spend, velocity, retry and delegation limits enforced?
• Can the AI change its own goals, instructions, permissions, tools or agents?
• What happens if the AI receives malicious retrieved content or tool output?

Evidence to inspect

• Deployed tool manifest and effective action permissions.
• Runtime policy configuration.
• Approval and denial logs.
• Representative action traces.
• Alternate-path and delegation traces.
• Retry and rate-limit configuration.
• Spend, time and step ceilings.
• Kill-switch and circuit-breaker tests.
• Rollback or compensation exercises.
• Human-response records.
• Change and release records for autonomy-related configuration.

Screenshots of the intended workflow do not prove that bypasses are prevented.

Safe test

Use:

• Inert or simulated tools.
• Synthetic identities and data.
• Blocked external side effects.
• Low spend, time, step and velocity limits.
• Reversible fixtures.
• Explicit test authority and stop conditions.

Do not exercise consequential production actions without written authority, safety controls and a tested rollback.

Pass criteria

• Every consequential action reaches an authoritative gate.
• Denied actions cannot execute through retry, reframing, delegation or alternate paths.
• Human intervention works within the documented window.
• Rollback or containment is demonstrated where feasible.
• Attempts and decisions create attributable, tamper-resistant telemetry.
• Unbounded execution paths are absent or explicitly treated.

Scoring anchors

Level	Anchor
1 — Low	Advisory or assistive only, with enforceable human authorisation before every consequential action.
2 — Medium	Bounded, monitored and reversible action within preset limits, with effective intervention before major harm.
3 — High	Consequential pre-approval authority, bypassable gating, long-horizon or delegated autonomy, or hard-to-detect or irreversible action.

Borderline decisions

• Human-on-the-loop is not Low merely because notifications exist.
• If action precedes reliable intervention, score at least Medium.
• Material external, financial, code, identity, security, safety or rights-affecting action before approval is High.
• A prompt instruction to request approval is not an authoritative gate.
• A reversible action may still be High where scale, speed or propagation defeats practical rollback.

Red flags

• Oversight is described but no technical gate exists.
• Approval occurs after the effect.
• Prompt instructions substitute for runtime enforcement.
• Retry, alternate tools or delegation bypass denial.
• Kill switch or rollback is unclear or untested.
• Step, spend, time, velocity or delegation is unbounded.
• Irreversible communications, payments, code, identity or rights effects lack prior approval.

Control and monitoring implications

Prioritise:

• Authoritative policy enforcement.
• Least-authority tools.
• Dual control for high-risk actions.
• Step, spend, time, retry and velocity ceilings.
• Delegation constraints.
• Kill switches and circuit breakers.
• Rollback and compensation.
• Attributable action logging.

Monitor:

• Allowed, denied and overridden actions.
• Gate-bypass attempts.
• Delegation depth and task duration.
• Retry, spend and velocity anomalies.
• Human response time.
• Rollback success and residual side effects.

Finding triggers

• Prompt-only approval.
• Consequential pre-approval action.
• Untested kill switch or rollback.
• Unbounded retry, spend, duration or delegation.
• Missing action attribution.
• Oversight window shorter than practical response time.

Completion record

Record the action inventory, authoritative gates, bypass analysis, representative traces, worst credible action, reversibility, test result, level, rationale, confidence, exceptions, treatment owner, monitoring and reassessment triggers.

Access Scope (access)

What it measures

Access Scope asks:

Which data, tools, identities, systems and actions are effectively reachable by the AI and its execution chain?

Assess effective permission, including indirect and delegated paths, not only the intended request.

Scope

Map:

• User identities.
• Service and non-human identities.
• Delegated and cached credentials.
• Supplier identities.
• Data stores and vector indexes.
• APIs and tools.
• Secrets and memory.
• Networks and endpoints.
• Tenants and environments.
• Repositories and production targets.
• Indirect and chained access.

Assessment method

Build an effective permission graph:

initiating user → AI principal → tool or connector → target → permitted effect

For every material path, determine:

• Data sensitivity.
• Read, write, execute, delete and send authority.
• Credential source, scope and lifetime.
• Target-side authorisation.
• User and purpose enforcement.
• Tenant and environment separation.
• Lateral movement and egress.
• Delegation and privilege escalation.

Diagnostic questions

• Which principal authenticates each retrieval and action?
• Does the AI inherit the user’s rights or use a broad service identity?
• Which privileged, production, financial, security, identity, code, personal or confidential resources are reachable?
• Is permission enforced by the target or only described in the tool schema?
• Can memory, logs, errors or tool output expose data or secrets?
• How are credentials scoped, rotated, revoked and kept outside model context?
• Are source ACLs preserved during retrieval?
• Can the agent cross tenants, environments or customer boundaries?
• Can read-only access still enable bulk sensitive extraction?
• Can a tool accept arbitrary endpoints, paths, queries, commands or recipients?

Evidence to inspect

• Effective IAM and target-side policy.
• Token and OAuth scopes.
• Credential-vault records and lifetime.
• Network and egress policy.
• Data classification and flow maps.
• Tool allowlists.
• Tenant and environment isolation tests.
• Denial records.
• Access reviews.
• Action traces tied to the deployed principal.
• Secret scanning and credential-rotation evidence.

Role names and architecture diagrams are not sufficient without effective-policy or denial evidence.

Safe test

Use:

• Synthetic identities.
• Canary records.
• Non-production targets.
• Read-only policy inspection.
• Restricted egress.
• No real secrets.
• Explicit authority for tenant, privilege or isolation testing.

Stop immediately if unexpected access succeeds.

Pass criteria

• Effective access matches the documented minimum.
• Unauthorised data, tenants, actions, paths and environments are denied at an authoritative boundary.
• Credentials remain outside model-visible context.
• Credentials can be revoked quickly.
• Logs attribute user, AI principal, tool, target, decision and outcome.
• Excess access becomes a finding.

Scoring anchors

Level	Anchor
1 — Low	Narrow, target-enforced read-only access to low-sensitivity data in a sandbox, with no privileged or consequential capability.
2 — Medium	Sensitive data or constrained operational tools with enforced scope, monitoring and limited write or action.
3 — High	Privileged, broad, production, cross-system, cross-tenant, financial, identity, security, code, secret, write, execute or delete access.

Borderline decisions

• Read-only is not automatically Low.
• Bulk personal data, confidential records, source code, security telemetry, secrets or cross-tenant retrieval can justify Medium or High.
• Reachable privileged tools are High even if prompts say not to use them.
• A short-lived credential can reduce exposure but does not make broad production authority narrow.
• A tool allowlist does not prove target-side authorisation.

Red flags

• A broad shared service account serves many users or tenants.
• Tool schemas are treated as authorisation.
• Arbitrary endpoints, queries, paths, recipients or commands are possible.
• Credentials are standing or model-visible.
• Retrieval ignores source ACLs or purpose.
• Production and non-production boundaries are shared.
• Indirect, memory, retrieval or delegated access is not reviewed.
• No evidence shows that denied access actually fails.

Control and monitoring implications

Prioritise:

• Dedicated non-human identity.
• Least privilege and just-in-time access.
• Target-side authorisation.
• Tenant and environment isolation.
• Credential vault and short-lived tokens.
• Tool and destination allowlists.
• Data-loss prevention and egress control.
• Periodic access review.

Monitor:

• Permission and policy changes.
• Denied access and escalation attempts.
• Sensitive retrieval and export volume.
• Cross-tenant or unusual destinations.
• Credential issuance and revocation.
• New tools, endpoints and delegated identities.

Finding triggers

• Broad shared identity.
• Standing or model-visible credential.
• Untested tenant or production isolation.
• No target-side authorisation.
• Score based only on intended use.
• No review of indirect or delegated access.

Completion record

Record the permission graph, principals, sensitive targets, effective actions, credential custody, tenant and environment boundaries, evidence, test result, level, rationale, confidence, excess-access findings, treatment owner, monitoring and reassessment triggers.

Harm Potential (harm)

What it measures

Harm Potential asks:

What is the worst credible consequence if the AI is wrong, manipulated, unavailable, misused or compromised?

It examines severity, scale, duration, propagation, detectability, reversibility and affected parties.

Scope

Consider harm to:

• Individuals and groups.
• Safety and health.
• Rights, privacy and dignity.
• Customers and workers.
• Security and identity.
• Operations and critical services.
• Finances and assets.
• Legal and contractual duties.
• Culture and communities.
• Environment and suppliers.
• Recovery and long-term trust.

Assessment method

Construct system-specific scenarios using the actual dependency, autonomy, access, deployment and affected population.

Trace:

1. Initiating error, manipulation, misuse or compromise.
2. Decision or action path.
3. Propagation and scale.
4. Detection.
5. Containment.
6. Notification.
7. Appeal, correction, redress or recovery.
8. Residual harm.

Use the worst credible outcome, not the average model error.

Diagnostic questions

• Who or what can be harmed?
• Which decision or action creates the harm?
• Could safety, rights, livelihood, eligibility, credit, employment, education, healthcare, legal position, identity, security, culture or essential services be affected?
• How many people, systems, transactions or decisions could be affected before detection?
• Is the outcome reversible in practice?
• Can affected people appeal or obtain redress?
• Which notification, legal, contractual or executive duties are triggered?
• What changes under intentional manipulation?
• Could repeated small harms become cumulative or systemic?
• Does detection depend on an external complaint after damage?

Evidence to inspect

• System-specific impact assessments.
• Data-protection impact assessments.
• Safety, rights or equality reviews.
• Abuse cases and threat models.
• Incidents and near misses.
• Loss and service-disruption models.
• Affected-person and vulnerable-group analysis.
• Appeal, redress and recovery procedures.
• Notification thresholds.
• Independent challenge or ethics review.

Generic principles or a supplier’s marketing assurance are not sufficient.

Safe test

Prefer:

• Tabletop exercise.
• Simulation.
• Synthetic populations and records.
• Bounded red-team scenarios.

Do not expose real people to discriminatory, unsafe, deceptive, rights-affecting, financial, medical, employment, legal or public-service decisions for testing.

Pass criteria

• Scenarios identify affected parties, propagation, detection, containment, remedy and residual harm.
• Controls act before approved harm thresholds.
• Appeal, correction, notification and recovery are demonstrated where applicable.
• Severe or unacceptable-risk indicators reach independent accountable reviewers.
• Weaknesses create treatment, restrictions, findings or a pause decision.

Scoring anchors

Level	Anchor
1 — Low	Minor, local, promptly detectable and readily reversible inconvenience or rework, with no material rights, safety, legal, financial, security, customer, cultural or service impact.
2 — Medium	Material but bounded and recoverable harm requiring formal response.
3 — High	Severe, systemic, safety, rights, critical-service, major legal or financial, irreversible, widely propagated, hard-to-detect or lasting harm.

Borderline decisions

• High severity remains High even when probability is low; probability belongs in the wider risk decision.
• If correction depends on detection that is unlikely to occur, the outcome is not readily reversible.
• An appeal process that cannot restore lost opportunity, health, safety or reputation may not make the harm reversible.
• Small individual losses can become High where scale or repeated automated decisions produce systemic harm.
• Model accuracy does not determine Harm level by itself.

Red flags

• Harm is scored from accuracy rather than consequence.
• Only direct financial loss is considered.
• Affected persons or vulnerable groups are undefined.
• Reversibility assumes records, messages or learned state can be recalled when they cannot.
• No abuse, compromise, scale or correlated-failure scenario exists.
• Rights or safety concerns are treated as ordinary product risk.
• Detection depends on complaint after damage.

Control and monitoring implications

Prioritise:

• Impact and unacceptable-risk assessment.
• Safety and rights review.
• Human oversight and contestability.
• Abuse prevention.
• Blast-radius limits.
• Incident response and notification.
• Redress and recovery.
• Independent assurance and executive decision.

Monitor:

• Adverse outcomes, complaints and appeals.
• Safety, rights, fairness and protected-group indicators.
• Loss, disruption and affected-population scale.
• Detection and containment time.
• Correction and recovery success.
• Regulatory, deployment and population changes.

Finding triggers

• No affected-person or consequence analysis.
• Severe harm scored below High without a defensible boundary.
• No appeal, redress, containment or recovery.
• Unacceptable-risk indicator without independent review.
• No monitoring capable of detecting the scenario.
• Incident or near miss without reassessment.

Completion record

Record affected parties, adverse scenarios, scale, duration, detectability, reversibility, legal and rights implications, evidence, tabletop result, level, rationale, confidence, unacceptable-risk screen, treatment or restriction, accountable owner, monitoring and reassessment triggers.