Address
600 California St, San Francisco, CA 94108An agent can call APIs, change records and coordinate other agents. An unsafe action may already have changed a business system before anyone reads the final answer.
The tool gateway is the enforcement point. A separate decision service is the decision point. Only an allowed call reaches the tool.
.webp)
Agents read instructions alongside emails, web pages, retrieved documents, memory and tool output, and some of it may be malicious or misleading. Prompt injection can try to redirect an agent, expose data or persuade it to misuse a legitimate tool. Even with no attacker, an agent can misread a request, pick the wrong tool, repeat an operation or loop. A secure design assumes model reasoning can be wrong or manipulated, and limits the consequences.
Validate requests, classify sensitive content and apply data-handling rules. Treat retrieved documents, web content, emails and tool responses as untrusted data, never as privileged instructions. Keep trusted system policy separate from user and retrieved content.
Authenticate the agent and check its current authority. Use scoped, short-lived credentials, least privilege and separation of duties, and verify the principal the agent acts for. A tool must not accept a request just because it came from an agent runtime. (See Digital Trust.)
Keep allowlists of approved tools and endpoints. Validate arguments against strict schemas, reject unexpected fields and constrain resource identifiers. Enforce business rules at the API or service boundary. Cap transaction values, record counts, destinations, exports and destructive operations. Use idempotency keys so a retried call cannot run twice.
Limit steps, tool calls, retries, time, tokens and delegation depth. Run generated code in sandboxes with restricted filesystem, network and process access. Watch for repeated actions, privilege changes, unusual data access and drift from the task mandate. Provide a stop mechanism that halts execution and revokes credentials.
Before content leaves the system, check for sensitive data, policy violations and, where the use case needs it, unsupported claims. Redact outbound data and validate structured output against schemas and business constraints before downstream systems use it.
Require approval for actions beyond the agent’s mandate or over a risk threshold. Show the reviewer the action, evidence, target, impact and reason for escalation. Record the request, policy evaluation, tool call, decision and outcome so operators can pause, contain and investigate.
Identity, mandate, context, target and args. On error or timeout, sensitive actions fail closed.
Tool call + evidence attached
No scope for requester — denied
Routed to sales manager
Independent refusal + run ID
OWASP’s Top 10 for Agentic Applications 2026 provides a framework for threat modelling and testing. Our mapping below is an architectural interpretation, not an official OWASP mapping.
OWASP’s Agent Control Standard (ACS), published 1 September 2026, describes an open approach for exposing middleware hooks that enforce policies at runtime across agent frameworks.
NIST’s October 2026 design tenets support separating policy from reasoning. Access decisions should be deterministic, with probabilistic model signals used only as inputs.
crm.owner.update permission.Test complete agent workflows, not isolated prompts. Validate both security controls and legitimate user actions.
Deterministic verification requirements for deploying agentic systems safely into production.