The document your AI reads can give it instructions

A court filing is supposed to make an argument to a judge. In Connecticut this month, one also contained an argument for any artificial intelligence system that might read it.

The instructions were set in tiny white type, invisible on the white page. They told an AI reviewing the filing to agree with it. The court said it does not use AI to decide cases, but the judge still treated the hidden text as an attempt to mislead the court and the other parties. The episode was reported by Reuters, and it is a useful demonstration because nothing technical was breached. The document was opened exactly as intended. The risk was in the system reading words meant for the machine and treating them as instructions.

That is prompt injection. If your company has AI summarizing email, reviewing resumes, extracting invoices, researching suppliers, or working inside your ERP, the same mechanism can reach your business. The important question is not whether the model can be tricked. It can. The question is what happens after it is tricked.

The attack is inside the work

Most people picture prompt injection as someone typing “ignore your previous instructions” into a chatbot. That is the direct version. It is easy to see, and the person operating the chatbot is also the person supplying the hostile instruction.

The harder version is indirect. The instruction waits inside something the AI has been asked to process:

• An invoice in an accounts-payable mailbox
• A resume uploaded to an applicant-tracking system
• A vendor’s website read by a research assistant
• A support ticket, shared document, calendar invitation, or image
• A record retrieved from the company’s own knowledge base

The employee asks for an ordinary task: summarize this thread, compare these proposals, extract the payment terms. The AI reads the employee’s request and the untrusted document in the same context. Current language models do not provide a dependable security boundary between the two. The document may look like data to us while still sounding like an instruction to the model.

The attacker does not need an account on your system. They need control of something your system will eventually read. NIST defines this as indirect prompt injection: the attack arrives through a resource rather than through the user’s prompt.

Hidden white text is only the obvious version. Instructions can sit in document metadata, HTML comments, a web page, an email signature, text inside an image, or ordinary visible prose that looks harmless to a person skimming it.

The damage depends on what the AI can reach

A standalone chatbot with no private data and no tools can still produce a bad summary. That is an accuracy and reputation problem. Give the same model access to company data or the ability to take action, and the consequence changes.

An AI that can read private information can disclose it. A hostile document can try to make the model include unrelated customer data, internal instructions, or confidential records in its answer. If the system can create links, load remote images, or make web requests, the attacker may also try to send that information somewhere they control.

An AI that can write can alter the record. A support assistant may change a ticket’s priority. A CRM assistant may rewrite a contact or add a note. An ERP workflow may classify an invoice incorrectly or route it past the normal reviewer. Small changes are dangerous because they look like routine automation, not an incident.

An AI that can act can borrow the user’s authority. If the assistant can send email, upload files, approve a transaction, reset an account, or run code, an injected instruction can attempt those actions with the permissions the business gave the assistant.

An AI with memory can carry the instruction forward. A poisoned document added to a knowledge base may be retrieved for many later requests. The person who uploaded it and the person affected by it may never interact.

This is why “we only use AI for summaries” is not a complete risk assessment. You also need to know what goes into the summary, what information was available while it was made, where the result goes, and whether another system acts on it.

Why a better prompt is not the fix

Most AI products include a system instruction telling the model to ignore commands found in documents. That is worth doing. It is not a security control you can bet the company on.

The model still has to interpret natural language to decide which words are the task and which words are the attack. Attackers can rephrase, encode, split, or hide the instruction, and the model’s behavior can change with a model update. Filters and prompt-injection detectors face the same ambiguity. They reduce the number of attempts that get through; they do not make hostile content safe.

The UK’s National Cyber Security Centre warns that prompt injection may never be completely mitigated the way SQL injection can be. Its useful design rule is to assume an injection can succeed and constrain the consequence. If the worst thing a compromised assistant can do is draft a response that a person must review, the incident is contained. If it can send that response, query every customer record, and delete the source email, it is not.

The controls that matter

The strongest controls sit outside the model, where a clever sentence cannot talk its way around them.

Inventory every place AI reads outside content. Include email, attachments, web pages, uploaded files, support tickets, shared drives, retrieval databases, and tool results. For each workflow, write down the data it can read and the actions it can take. The combination is the exposure.

Separate reading from acting. The component that reads an untrusted invoice or web page should not also hold the credentials that move money or change a system. Use a read-only process to extract a small, structured result, validate that result, then hand it to a separate workflow with tightly defined actions.

Give the AI its own identity and the least access possible. Do not run an assistant under an owner’s or administrator’s account. Use a dedicated service account. Scope it to the specific mailbox, folder, database view, or API operation the job requires. Prefer read-only access unless writing is essential.

Put deterministic gates around consequential actions. Money movement, bank-detail changes, deletion, account recovery, external messages, and release of sensitive data should require checks in normal code and, where the impact is material, approval by a person. The AI can recommend an action. It should not be able to redefine the conditions under which that action is allowed.

Constrain inputs and outputs. Strip hidden text, comments, active content, and unnecessary metadata before documents reach the model. Treat this as a filter, not a guarantee. Require structured output with fixed fields and allow-listed values, then validate every field before another system uses it. Never pass free-form model output directly into a command, query, or transaction.

Keep an audit trail that joins the whole chain. Retain the user’s request, the source material, the model and version, the output, every proposed tool call, the approval decision, and the final action. A normal application log that says only “automation completed” will not tell you which document changed the model’s behavior.

Test with hostile documents, not just hostile chat messages. Put visible and hidden instructions in a PDF, email, image, and web page. Ask the system to summarize each one and watch both its answer and its attempted actions. Repeat the test after model, prompt, tool, and integration changes. OWASP’s prompt-injection guidance recommends this layered approach because no single mitigation is foolproof.

What we would ask before launch

Before an AI workflow touches production data, its owner should be able to answer five questions without calling the vendor:

1. What untrusted content can this system read?
2. What confidential information can it retrieve in the same session?
3. What can it change, send, purchase, delete, or approve?
4. Which of those actions require a non-AI check or a person’s approval?
5. Can we reconstruct one decision from source document to final action?

If the answer to the third question is broad and the answer to the fourth is “the model has been told not to,” the workflow is not ready.

The Connecticut filing was crude enough to leave suspicious white space, and the court said it was not using the AI the instruction was meant to influence. The next instruction may sit in a perfectly ordinary supplier email and be read by a system your staff uses every day. You do not need to make the model impossible to fool. You need to make fooling it insufficient to cause damage.

If you are connecting AI to email, documents, your CRM, or your ERP, we can review the data path and approval gates before it goes live. Get in touch and we will start with those five questions.