> ## Documentation Index
> Fetch the complete documentation index at: https://irisdocs.prescientlabs.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Model Limitations & Human Oversight

> Design AI use cases around uncertainty, review requirements and failure modes rather than assuming model outputs are always reliable.

AI systems are probabilistic. A consulting recommendation should explain what happens when the model is wrong, not only what happens when it works.

## Identify the failure mode

For every AI use case, ask:

* What can the system get wrong?
* How often would that matter?
* Who is affected?
* Can the error be detected?
* Can it be reversed?
* What is the cost of a bad output?

## Match oversight to consequence

Low-impact drafting may need simple human review.

Higher-impact decisions may need:

* explicit approval;
* structured validation rules;
* dual control;
* audit logging;
* restricted autonomy;
* escalation to a qualified person.

Do not use the phrase “human in the loop” as a substitute for specifying who reviews what and when.

## Define acceptance criteria

Examples:

* generated output must contain required fields;
* sources must be traceable where the use case requires evidence;
* low-confidence cases route to manual review;
* no external action occurs until named approval;
* sensitive categories are excluded from autonomous processing.

## Test representative cases

Use normal cases, edge cases, incomplete inputs and adversarial or ambiguous examples where relevant.

## Monitor after launch

Track:

* accuracy or quality rate;
* exception rate;
* human override;
* failure categories;
* user adoption;
* drift in inputs or behaviour.

## Explain uncertainty to clients

> The system is designed to assist with the defined task, not to guarantee a correct answer in every case. The implementation therefore includes review and exception handling proportionate to the consequence of error.

## Consultant boundary

Do not claim that a model is “safe” or “compliant” simply because a provider markets it as enterprise-ready. Risk depends on the actual use case, data, controls, users and operating environment.