How to automate document processing with AI without losing control

1. Choose one operation and define its boundaries

Specify a concrete transformation: extract invoice number, date, supplier, currency, line items and total, then prepare a draft for review. Do not begin with an agent that runs all purchasing. Extraction, content verification, document creation and posting are different actions with different failure costs. The first version can end at a draft without permission to alter the accounts. Make that boundary enforceable in the integration, not merely a sentence in its instructions.

List eligible document types, languages, input channels and exceptions. If multi-page invoices or handwritten annotations are outside scope, route them explicitly to manual handling. Assign someone to decide whether the output is usable. Also check whether a model is needed at all: a structured export or existing integration may solve the problem more simply. AI is useful for variable content, not as a fashionable replacement for reliable exchange of fields already available in structured form. Define the smallest useful outcome before selecting the technology.

2. Approve data handling before sending files

Inventory the information in documents: contacts, prices, payment details, commercial terms and internal identifiers. Decide what may be sent to the selected service, who approves it and what retention and access conditions are required. Do not assume providers handle information identically. Verify settings, contractual conditions and required operating modes for the specific solution before using real documents. Record the decision so future configuration changes can be reviewed against it.

Use synthetic or properly anonymized documents for early testing while preserving difficult structures: poor scans, tables spanning pages and similar identifiers. Removing a customer name alone may not remove identifying information. Define access to originals, logs, outputs and backups separately. Do not repeat full document contents in every technical notification. Retention should match an agreed need, and deletion should cover copies under the process’s control. These are design decisions, not last-day cleanup tasks. A successful extraction demo does not establish that the surrounding handling process is suitable for confidential business material.

3. Build a reference set before tuning prompts

Collect ordinary and difficult examples and manually record expected critical-field values. Include poor scans, missing currency, repeated files, unfamiliar suppliers, multiple totals and continued tables. For each, specify the expected response: accept a draft, request clarification or reject it. Test the behaviour of the whole process, not just whether the recognized text resembles the original. Include examples that should produce no automated action.

Separate tuning examples from final evaluation examples. Repeatedly adjusting instructions against the same files can make performance look better than it will be on unseen material. Version the set and preserve explanations for the expected answers. A small convenient sample is not automatically representative of future work. Initially, it should expose known failure classes and catch regressions. Add confirmed new exceptions over time without publishing confidential customer documents. When expected answers are disputed, resolve the business interpretation before scoring the model; an ambiguous reference answer cannot establish a reliable accuracy result.

4. Separate extraction from validation

After receiving a file, check allowed format, size and upload rules. The system may then recognize text or send approved content to a model, obtain structured output and validate it with ordinary code. A schema defines field names, types and permitted missing values. Valid JSON proves structural conformity, not truthful payment details. Treat the extracted values as candidates until the required checks have completed.

Separate deterministic rules check arithmetic, allowed currency, supplier existence and item mapping. If line totals disagree with the document total, do not simply ask the model to change a number until validation passes. Explain the discrepancy: discount, shipping, rounding or recognition error. Preserve a reference to the relevant original passage so a reviewer can inspect the field. Missing information should remain unknown rather than being completed with a plausible guess. This is especially important for identifiers, quantities and details that affect downstream records. Define which checks block progress and which require clarification instead of treating every warning as equivalent.

5. Measure correctness, not confident wording

A model’s self-assessment or confident phrasing does not verify accuracy. Compare outputs with agreed reference values. Measure critical fields separately and also measure whole-document correctness. In a fictional example, all mandatory fields are correct in 96 of 100 test documents. That is 96% fully correct documents in that set, not a guarantee for any future files. Report what the set contains so readers understand the scope of the result.

Measure unsafe escapes separately: incorrect results that passed without required review. Also track manual-handling rate and correction time. A solution that sends almost everything to a person may reduce one risk while delivering little operational benefit. Increasing automatic acceptance at the expense of silent defects requires an explicit risk decision, not just a better-looking throughput number. Set criteria before evaluation so success is not redefined to fit the output. Different document types and fields can require different acceptance rules. A minor description error and an incorrect payment identifier should not disappear into the same average.

6. Make human review an actual workflow

“A person will check everything” is not an operating procedure. Define who opens the queue, which fields they verify, where they see the original and what happens when information conflicts. Show the reason for review: unknown supplier, inconsistent amount or missing field. Operators should not reread an entire document merely to discover why the system stopped. Give them a clear distinction between confirmed information and a model suggestion.

Separate confirming an extracted value from authorizing a business action. Reading an invoice correctly does not mean it should be paid. Critical actions need separate permissions and agreed procedures. The worksheet below illustrates exception routes, not a complete security-control list. Measure queue age and workload, and provide backup ownership. If review is delayed, expose the state rather than repeatedly creating the same task. Preserve the reviewer and output version so the decision can be reconstructed. Test the review interface with the people who will use it under ordinary workload, not only with its designers.

7. Treat document contents as untrusted data

OWASP describes prompt injection through external material, including files that attempt to alter model behaviour. In this workflow, an invoice supplies candidate fields, not authority. An instruction inside a PDF must not authorize data sharing, account-detail changes or business operations. A prompt telling the model to ignore outside instructions is not enough to declare the system protected.

Constrain the extraction component: it does not need payment, deletion or unrestricted internal-service permissions. Validate downstream actions using separate code and access rules independent of the model’s response. Include a reference document containing an unrelated instruction and verify that it does not become an action. Layered safeguards do not justify promising zero risk. If untrusted text can expand the component’s access, fix the authority boundary rather than only editing prompt wording. Keep the test focused on the permitted workflow and record the observed outcome, including whether the system safely stopped or continued with only the allowed extraction.

8. Count the whole cost and plan recovery

Model calls are only one cost component. Include file preparation, recognition, storage, human review, corrections, support and retries. Compare the entire workflow before and after, including exceptions. Do not automatically convert saved minutes into cash savings if actual expenditure does not change. The benefit may be released team capacity or faster preparation; describe it accurately. Use measured handling time rather than assuming every automated document removes all previous work.

Define behaviour when the service is unavailable, a limit is reached or the same file arrives again. A retry must not create a duplicate ERP document. This requires agreed identifiers and outcome checks, not only matching filenames. Preserve processing state and a safe way to resume after failure. Provide a manual route for urgent documents and spending limits so a technical retry loop cannot create uncontrolled usage. Test recovery on reference data before launch. The operator should be able to distinguish waiting, failed and completed work without guessing from an empty screen or a missing response.

9. Roll out gradually and evaluate changes

NIST’s AI Risk Management Framework is a voluntary risk-management framework, not a certificate of error-free operation. A useful direction for a pilot is explicit scope, ownership, evaluation and continued risk review. Start with test data, then recommendations under agreed review. Expand automatic actions only after the criteria are met and the necessary authority has been granted. Record what remains outside scope even when the initial results are encouraging.

Version the model, instructions, schema and rules. Rerun the reference set after changes; a new configuration may improve some documents and worsen others. Monitor unfamiliar layouts, correction rates and incorrect acceptance. Decide who can stop the automatic flow and restore a previously working configuration. A pilot is complete when the team understands what the system does, where it fails and how work continues safely, not when it produces one impressive answer. Keep the evidence available to the people responsible for operating and approving the process.

• Eligible documents and permitted actions are explicit.
• Data transfer is approved for the selected service.
• Critical fields are checked independently of the model.
• Exceptions have an owner and a clear route.
• Retries, outages and version changes are tested.