Skip to main content
AI systems are probabilistic. A consulting recommendation should explain what happens when the model is wrong, not only what happens when it works.

​
Identify the failure mode

For every AI use case, ask:
  • What can the system get wrong?
  • How often would that matter?
  • Who is affected?
  • Can the error be detected?
  • Can it be reversed?
  • What is the cost of a bad output?

​
Match oversight to consequence

Low-impact drafting may need simple human review. Higher-impact decisions may need:
  • explicit approval;
  • structured validation rules;
  • dual control;
  • audit logging;
  • restricted autonomy;
  • escalation to a qualified person.
Do not use the phrase “human in the loop” as a substitute for specifying who reviews what and when.

​
Define acceptance criteria

Examples:
  • generated output must contain required fields;
  • sources must be traceable where the use case requires evidence;
  • low-confidence cases route to manual review;
  • no external action occurs until named approval;
  • sensitive categories are excluded from autonomous processing.

​
Test representative cases

Use normal cases, edge cases, incomplete inputs and adversarial or ambiguous examples where relevant.

​
Monitor after launch

Track:
  • accuracy or quality rate;
  • exception rate;
  • human override;
  • failure categories;
  • user adoption;
  • drift in inputs or behaviour.

​
Explain uncertainty to clients

The system is designed to assist with the defined task, not to guarantee a correct answer in every case. The implementation therefore includes review and exception handling proportionate to the consequence of error.

​
Consultant boundary

Do not claim that a model is “safe” or “compliant” simply because a provider markets it as enterprise-ready. Risk depends on the actual use case, data, controls, users and operating environment.