The decision to make
The useful governance question is not “Is AI safe?” It is: what degree of autonomy is acceptable for this use case, given its potential impact if something goes wrong?
This framework helps firms make that decision in a structured, risk-proportionate way. It does not replace the firm’s risk appetite, policy, legal analysis or governance processes.
Autonomy-risk matrix
| Consequence of failure / Autonomy | Assist | Recommend | Initiate | Execute |
|---|---|---|---|---|
| Low operational impact | Basic controls | Clear human owner | Defined workflow boundary | Restricted, low-risk automation |
| Customer impact | Human review | Outcome testing and challenge | Enhanced approvals and evidence | Exceptional approval; normally avoid unless tightly bounded |
| Financial or regulatory impact | Human-led decision | Independent challenge and strong evidence | Narrow mandate, enhanced controls | Senior approval, rigorous assurance and tested intervention |
| Material customer, market or resilience impact | Assistive use only unless exceptional case | Enhanced governance | Generally avoid without compelling safeguards | Generally prohibited or subject to exceptional governance |
Four autonomy levels
Assist: The system drafts, searches, analyses or summarises. A human decides and acts. Minimum safeguards include safe-use standards, training, appropriate data controls, quality assurance and human responsibility.
Recommend: The system produces a ranking, classification, score or suggested action that materially influences a person’s decision. Minimum safeguards include a named decision owner, evaluation, meaningful human review, outcome monitoring, challenge and ability to override.
Initiate: The system triggers a workflow step or creates a proposed action within defined rules, usually requiring later approval for consequential outcomes. Minimum safeguards include an explicit mandate, workflow limits, identity and access control, action records, monitoring and an intervention path.
Execute: The system takes a consequential action without case-by-case human approval. This requires exceptional governance, narrow scope, explicit risk appetite, strong authority limits, evidence, independent challenge, tested intervention, fallback and remediation.
Decision questions
- What is the worst plausible consequence for a customer, firm or market?
- Is the action reversible? If so, how quickly and at what cost?
- Can a human meaningfully review the action before harm occurs?
- Are authority, permissions and operating limits explicit and enforceable?
- Can the firm reconstruct what occurred and demonstrate the control state?
- Can the firm restrict, pause or revoke authority with a safe fallback?
- Is the expected benefit sufficient to justify the residual risk?
Practical use
- Proceed as assistive AI.
- Proceed with human approval and outcome monitoring.
- Proceed only within defined bounded-execution controls.
- Defer or prohibit until controls, evidence and intervention are stronger.