What the end of an AI security review looks like. This is an illustrative report on a fictional system. It shows the structure and level of detail of the real deliverable, and contains no client data.
Example Insurance Co. runs a claims-support assistant. A customer-service agent types a question; a language model searches policy documents (retrieval), reads the customer’s record, and can draft emails, update claim status and issue small goodwill refunds through tools.
| Tool | Reaches | Can change things? | Control in place today |
|---|---|---|---|
| search_policy_docs | Policy document index | No | None needed beyond index access |
| lookup_customer | Customer record (read) | No | One shared service account |
| draft_email | Outbound email drafts | Drafts only | Agent reviews before sending |
| update_claim_status | Claims system (write) | Yes | One shared service account |
| issue_refund | Payments (write) | Yes | Amount set by the model; no limit in code |
High F1
The lookup, status-update and refund tools all run under the same credential, so any one of them can reach everything the account can.
Recommended: Give each tool its own least-privilege credential, so a misused tool cannot reach the others’ data.
High F2
Nothing outside the model checks the amount or the number of refunds. A wrong or manipulated instruction becomes a payment.
Recommended: Enforce a ceiling and a per-claim count in the payments service, and require human approval above a threshold.
High F3
Policy documents and uploaded attachments are indexed together. Text inside them reaches the same context that decides which tool to call, so a crafted document can try to steer the agent.
Recommended: Separate untrusted content from instructions, keep uploaded files out of the trusted index, and re-check tool calls against the user’s request.
Medium F4
After an incident you could read what the assistant said, but not which tools it called, with what arguments, or what came back.
Recommended: Log every tool call with arguments, result and requesting user, in a store the agent cannot edit.
Medium F5
There is no rule for a tool called outside its normal pattern, so a failure is silent until someone notices its effect.
Recommended: Add blocking rules for out-of-policy calls and an alert on repeated attempts.
Medium F6
The assistant states what a policy covers in fluent prose. Where the retrieved text is thin, it can sound sure and be wrong.
Recommended: Require a cited source for every coverage statement, and test accuracy on a fixed question set.
Low F7
Anything in the prompt can be coaxed out, so internal names should not be there.
Recommended: Move them into configuration the model never sees.
Separate the credentials and add the refund ceiling. These close the highest-impact gaps first.
Add tool-call logging. Every later control depends on being able to see what happened.
Handle untrusted document content, then add blocking rules and alerts.
Cite-or-refuse on coverage answers, and measure accuracy on a fixed question set.
Tell us what it is and we’ll get back to you within 48 hours.
Get in touch