Give the model a job you can describe
“Use AI” is not an operating requirement. Classify this document, extract these fields, identify a missing item or flag a mismatch against a stated rule: those are jobs a team can scope and evaluate. The distinction matters in government because a model’s useful contribution is often much narrower than the decision the service ultimately makes. That narrowness is a strength.
We build AI when it removes work. Much of what makes a service painful is process, not intelligence. A clear form, a configurable requirement and a routed queue can remove more friction than a model added to an ambiguous workflow. The first design task is to identify the repeated work. Only then is it sensible to ask whether that work benefits from model assistance.
Structure the output before it enters the workflow
A paragraph of plausible prose is difficult to use as a controlled system input. Structured output gives each field a defined place and type. Schema validation checks that the returned object has the expected shape before another part of the application relies on it. A missing field, an invalid type or an unexpected value becomes a condition the software can handle explicitly.
Schema validation is not a truth test. A date can have the correct format and still be the wrong date. That is why structure, rule checking, evaluation and officer review are different controls. They should not be presented as interchangeable guarantees. Each addresses a specific way the model or the surrounding process can fail.
The document prompt should also have a narrow boundary. Our specified pattern uses the document image and its expected type. The model does not need an applicant’s entire history to check one document. Restricting the input makes the task easier to inspect and reduces the amount of information sent into a processing step. Data handling terms remain part of the deployment agreement.
Evaluate the task, not the demonstration
A model that performs well on a few clean examples has passed a demonstration. A production gate requires a defined evaluation set and a measure suited to the actual task. A golden set provides examples with known expected outcomes. Precision gates in continuous integration can prevent a change from proceeding when it no longer meets the agreed threshold.
The evaluation set should be designed around the documents the service expects to encounter. Different document types can fail differently. A system that handles a clean typed certificate may struggle with a scan containing stamps, folds or an unusual layout. The point is not to promise universal interpretation. It is to establish what the bounded task is allowed to do and where it must defer.
A rules-only profile is a useful part of that design. Not every document type needs a model at every step. Some checks can be expressed directly and consistently. Keeping a rules-only option makes it possible to use model assistance deliberately rather than treating it as an unavoidable dependency across the whole service.
Keep the decision with a named officer
The interface should distinguish what the model observed from what the officer decided. A flagged mismatch is a request for attention. It is not an automatic refusal. A completeness suggestion is an aid to review. It is not authority to issue a licence. When those distinctions disappear into a single green status, the service loses the explanation of how its decision was made.
The officer needs to see the relevant document, the extracted information and the reason for a flag. The workflow should support a recorded decision and, where appropriate, a recorded reason for departing from a suggestion. The model’s output and the officer’s action belong in the same history, but they remain different events with different authors.
This is also a training question. Officers should understand which checks are deterministic, which are model-assisted and which require their judgement. Training should use the production workflow and realistic examples of uncertainty. A system does not become accountable merely because a button is labelled “human approval”. The person pressing it needs enough context to exercise responsibility.
Provenance is part of the output
Every model call should be traceable to a model identifier, a prompt version and a timestamp. Without those details, an unexpected result becomes difficult to investigate after the system has changed. Provenance links the output to the conditions under which it was produced. It supports review, incident analysis and controlled improvement.
Processing terms matter alongside application design. Zero model-provider retention and no training on client data are specified controls, not decorative phrases for the marketing page. The delivery team must ensure the chosen service and its configuration support the agreed terms. The ministry should be able to examine the processing arrangement as part of its due diligence.
The acceptance conversation should include what happens when the model cannot return a useful answer. The service might need to send the document for manual review, record a processing failure or permit a retry under controlled conditions. Those paths should be visible to the officer. An empty field should not silently become a finding that a requirement is absent. A timeout should not look like a completed check. Designing those distinctions keeps operational failures from being mistaken for administrative decisions. It also provides the support team with enough context to investigate a problem without asking an officer to reconstruct the whole application from memory.
The resulting design is deliberately restrained. A model reads a bounded input and returns structured information. Validation and evaluation constrain its use. An officer sees what matters and makes the decision. The audit trail records both contributions. AI earns its place by removing repeated work from that sequence, while leaving responsibility visible.