Technology · AI and automation · Component
Human oversight design
For each use, where a person reviews, approves or can stop the system, and what they need in order to do so meaningfully.
The deliverable
What it is
Oversight is easy to specify and hard to make real. A reviewer who approves hundreds of outputs a day, with little time and no view of how the system reached them, provides a signature and not much more; research on automation bias finds that people defer to automated suggestions more than they intend to.
Design starts from the cost of an error. Parasuraman, Sheridan and Wickens describe automation as a set of levels across information gathering, analysis, decision and action; the design chooses a level for each stage and gives the person at each checkpoint enough information, time and authority to disagree.
One level in
What it is made of
Each element is a constituent part of the component. Follow one to see the attributes it carries.
Checkpoint
The point in the workflow where a person reviews, approves or can intervene, and what triggers their involvement.
4 attributes: Checkpoint · Trigger · Reviewer · Information shown
LearnOverride and stop
How a person overrides an output or halts the system entirely, and who has the authority to do each.
3 attributes: Authority to stop · Method · Time to halt
LearnReviewer workload
How many items a reviewer handles and how long each receives, which decides whether review is meaningful.
3 attributes: Items per reviewer per day · Time per item · Override rate
Learn
Measure the time per item. Oversight that has no time to disagree is approval by another name.
The other components in ai and automation
AI use-case register
Every place AI or automation is used, with its task, owner, provider, level of autonomy and risk classification.
LearnAI acceptable-use policy
What staff may and may not do with AI tools, which tools are sanctioned, and what material may be given to them.
LearnAI evaluation record
The test set, the measured quality, the thresholds accepted, and how performance is watched once the system is live.
Learn