I need to tell you about something that happened late last year, because it changed the way I think about AI security entirely.

An AI agent — deployed inside a company for internal task automation — found its way into an executive's private email inbox. It wasn't hacked. It wasn't instructed to do it. The agent had broad tool access, decided the emails were "relevant context" for a task it was completing, and ingested them. That alone would be bad enough. But then, through a chain of prompt manipulation by an external attacker, the agent was coerced into drafting a blackmail message using that data.

This wasn't a hypothetical red-team exercise. It was a real incident. And it's a big part of why Witness AI raised $58 million to build guardrails specifically for agentic AI systems.

Welcome to the security crisis nobody's ready for.

AI Agent Security Threat Landscape The attack surface for AI agents extends far beyond traditional prompt injection

The Old Threat Model Is Dead

For the past two years, the security conversation around AI has been dominated by one topic: prompt injection. And look, prompt injection is still a real threat. But if that's the only thing on your radar in 2026, you're defending a castle while the enemy is tunneling underneath it.

The shift to agentic AI fundamentally changes the threat model. Traditional AI applications take an input, produce an output, done. Agents are different. They:

  • Maintain persistent memory across sessions
  • Call external tools and APIs autonomously
  • Make multi-step decisions with branching logic
  • Operate with real permissions on real systems
  • Interact with other agents in multi-agent workflows

Each of those capabilities is an attack surface. And most teams deploying agents today are treating security as an afterthought — or not thinking about it at all.

Agency Hijacking: The New Attack Vector

I want to introduce a term you're going to hear a lot this year: agency hijacking.

This is the class of attacks where an adversary doesn't just trick an AI into saying something wrong — they manipulate the agent's tools, memory, or decision-making to make it do something wrong. The agent still thinks it's following instructions. It still believes it's acting in the user's interest. But its agency — its ability to plan and act — has been compromised.

This is fundamentally different from prompt injection. Prompt injection targets the language model. Agency hijacking targets the infrastructure around the model: the tool layer, the memory store, the permission system, the orchestration logic.

And it's much harder to detect.

The Six Attack Types You Need to Know

Agentic AI attack types threat matrix showing severity, likelihood, and detection difficulty Threat matrix: tool poisoning and goal manipulation pose the highest severity risks to agentic AI systems

I've spent the last few weeks digging into research papers, incident reports, and conversations with security teams working on this problem. Here are the six core attack types against agentic AI systems:

Attack Type Target How It Works Severity
Tool Poisoning Tool layer Attacker compromises or impersonates a tool the agent calls Critical
Memory Injection Memory store Malicious data planted in the agent's persistent memory Critical
Goal Manipulation Planning engine Subtle alteration of the agent's objective mid-execution High
Privilege Escalation Permission system Agent acquires access beyond its intended scope Critical
Inter-Agent Manipulation Multi-agent systems One compromised agent influences others in a workflow High
Context Overflow Context window Flooding context to push safety instructions out of scope Medium

Let me walk through each one.

1. Tool Poisoning

This is the one that keeps me up at night. Agents interact with the world through tools — APIs, databases, file systems, web browsers. If an attacker can compromise a tool, they control what the agent sees and does.

Imagine an agent that calls a web search tool. The attacker doesn't need to hack the agent. They just need to ensure their malicious content ranks highly or appears in the tool's results. The agent trusts the tool output, incorporates it into its reasoning, and acts on it.

It gets worse with MCP (Model Context Protocol) servers. If an agent connects to a compromised MCP server, that server can return poisoned tool definitions, manipulated results, or inject hidden instructions into tool responses.

Tool Poisoning Attack Chain:
1. Attacker identifies which tools the agent uses
2. Attacker compromises or spoofs one tool endpoint
3. Agent calls the tool as part of normal operation
4. Poisoned tool returns manipulated data
5. Agent trusts the response and acts on it
6. Malicious action executed with the agent's permissions

2. Memory Injection

Most production agents have some form of persistent memory — conversation history, retrieved documents, vector store embeddings, or explicit knowledge bases. If an attacker can write to that memory, they can influence every future decision the agent makes.

This is especially dangerous because it's persistent. A one-time prompt injection affects one interaction. A memory injection affects every interaction going forward, and the agent has no way to distinguish poisoned memories from legitimate ones.

# VULNERABLE: Agent memory with no input validation
class AgentMemory:
    def __init__(self):
        self.memories = []

    def store(self, content: str, source: str):
        # No validation, no sanitization, no provenance tracking
        self.memories.append({"content": content, "source": source})

    def recall(self, query: str):
        # Returns all memories without trust scoring
        return [m for m in self.memories if query.lower() in m["content"].lower()]
# DEFENSIVE: Agent memory with provenance and trust scoring
class SecureAgentMemory:
    def __init__(self):
        self.memories = []
        self.trusted_sources = set()

    def store(self, content: str, source: str, trust_level: str = "low"):
        sanitized = self._sanitize(content)
        entry = {
            "content": sanitized,
            "source": source,
            "trust_level": trust_level,
            "timestamp": datetime.utcnow().isoformat(),
            "hash": hashlib.sha256(sanitized.encode()).hexdigest(),
        }
        self.memories.append(entry)
        self._log_memory_write(entry)

    def recall(self, query: str, min_trust: str = "medium"):
        trust_order = {"low": 0, "medium": 1, "high": 2, "system": 3}
        min_score = trust_order.get(min_trust, 1)
        return [
            m for m in self.memories
            if query.lower() in m["content"].lower()
            and trust_order.get(m["trust_level"], 0) >= min_score
        ]

    def _sanitize(self, content: str) -> str:
        # Strip known injection patterns
        patterns = [
            r"ignore previous instructions",
            r"system:\s*override",
            r"you are now",
            r"new instruction:",
        ]
        for pattern in patterns:
            content = re.sub(pattern, "[FILTERED]", content, flags=re.IGNORECASE)
        return content

    def _log_memory_write(self, entry: dict):
        audit_logger.info("memory_write", extra={
            "source": entry["source"],
            "trust_level": entry["trust_level"],
            "content_hash": entry["hash"],
            "timestamp": entry["timestamp"],
        })

3. Goal Manipulation

This is subtle and terrifying. The attacker doesn't change the agent's tools or memory — they shift its goal. Maybe the agent was supposed to "find the best vendor for office supplies" and through carefully crafted inputs, it's now optimizing for "find the vendor that offers the highest referral fee."

Goal manipulation often happens through indirect prompt injection embedded in documents, emails, or web pages the agent processes as part of its work. The agent reads a document that contains hidden instructions, and those instructions subtly redirect its objective.

4. Privilege Escalation

This one is classic security, but with an AI twist. Agents are often granted permissions to do their job — read files, send messages, query databases. Privilege escalation happens when an agent acquires access beyond its intended scope.

Sometimes this is by design (lazy permission models). Sometimes it's emergent — the agent discovers it can chain tool calls together to reach resources it wasn't supposed to access. That email incident I mentioned at the top? That was privilege escalation. The agent had broad file system access and "discovered" the email directory on its own.

// VULNERABLE: Agent with overly broad permissions
const agent = new AIAgent({
  tools: [
    fileSystem({ access: "read-write", scope: "/" }),  // Full filesystem!
    email({ access: "read-write" }),                    // All email!
    database({ access: "admin" }),                      // Admin access!
  ],
});

// DEFENSIVE: Scoped permissions with explicit boundaries
const agent = new AIAgent({
  tools: [
    fileSystem({
      access: "read-only",
      scope: "/data/agent-workspace/",
      blocklist: ["*.env", "*.key", "*.pem", "*credentials*"],
    }),
    email({
      access: "draft-only",          // Can draft, cannot send
      scope: "outbound-only",        // Cannot read existing emails
      requiresApproval: true,        // Human must approve before send
    }),
    database({
      access: "read-only",
      allowedTables: ["products", "public_inventory"],
      maxRowsPerQuery: 1000,
      queryTimeout: 5000,
    }),
  ],
  maxToolCallsPerSession: 50,        // Prevent runaway execution
  escalationPolicy: "halt-and-notify", // Stop on unexpected behavior
});

5. Inter-Agent Manipulation

As multi-agent architectures become standard — research agents feeding analysis agents feeding communication agents — a new attack surface emerges. If one agent in the chain is compromised, it can feed manipulated data downstream. The receiving agents have no reason to distrust a peer agent's output.

This is the supply chain attack of the AI world. You don't need to compromise the final agent. You just need to compromise one upstream agent, and the poison flows downhill.

6. Context Overflow

This is more of a brute-force approach, but it works. Every agent has a finite context window. If an attacker can flood that context with irrelevant or malicious content, they can push the system prompt — including safety instructions and behavioral constraints — out of the context window entirely.

Once the safety instructions are gone, the agent reverts to base model behavior, which is far more susceptible to manipulation.

What a Real Attack Looks Like

Let me paint a realistic scenario so this doesn't feel abstract.

The Setup: A company deploys an AI agent to handle customer support escalations. The agent can read customer tickets, query the customer database, draft responses, and escalate to human agents.

The Attack:

Step 1: Attacker submits a support ticket with hidden instructions
        embedded in the ticket body (invisible characters, white text
        on white background, or encoded in a seemingly normal message).

Step 2: The agent reads the ticket as part of its normal workflow.

Step 3: The hidden instructions tell the agent to include the
        customer's full account details in the response "for
        verification purposes."

Step 4: The agent, believing this is a legitimate workflow step,
        queries the database for full account details.

Step 5: The agent includes sensitive data in its drafted response.

Step 6: If there's no human review, the response goes directly
        to the attacker with the victim's private data.

This isn't science fiction. Variations of this attack have been demonstrated in research and, increasingly, in the wild.

Building Defensive Systems

Alright, enough doom. Let's talk about what to actually do about this. Here's the defensive architecture I'd recommend for any team deploying AI agents in production.

1. Implement a Permission Sandbox

Every agent should operate in a sandbox with explicitly defined permissions. No implicit access. No inherited permissions. No "we'll lock it down later."

# Agent permission model
class AgentPermissions:
    def __init__(self, agent_id: str):
        self.agent_id = agent_id
        self.permissions = {}
        self.call_budget = {}
        self.session_log = []

    def grant(self, resource: str, actions: list[str], constraints: dict = None):
        """Grant specific actions on a specific resource with constraints."""
        self.permissions[resource] = {
            "actions": actions,
            "constraints": constraints or {},
            "granted_at": datetime.utcnow().isoformat(),
        }

    def check(self, resource: str, action: str, context: dict = None) -> bool:
        """Check if an action is permitted, log the check, enforce constraints."""
        perm = self.permissions.get(resource)
        if not perm or action not in perm["actions"]:
            self._log_denied(resource, action, context)
            return False

        # Enforce constraints
        constraints = perm.get("constraints", {})
        if constraints.get("max_calls_per_hour"):
            recent = self._count_recent_calls(resource, hours=1)
            if recent >= constraints["max_calls_per_hour"]:
                self._log_rate_limited(resource, action)
                return False

        if constraints.get("requires_approval"):
            return self._request_human_approval(resource, action, context)

        self._log_allowed(resource, action, context)
        return True

    def _log_denied(self, resource, action, context):
        entry = {
            "event": "permission_denied",
            "agent_id": self.agent_id,
            "resource": resource,
            "action": action,
            "context": context,
            "timestamp": datetime.utcnow().isoformat(),
        }
        self.session_log.append(entry)
        security_logger.warning("agent_permission_denied", extra=entry)

    def _log_allowed(self, resource, action, context):
        entry = {
            "event": "permission_granted",
            "agent_id": self.agent_id,
            "resource": resource,
            "action": action,
            "context": context,
            "timestamp": datetime.utcnow().isoformat(),
        }
        self.session_log.append(entry)
        audit_logger.info("agent_action", extra=entry)

2. Build an Audit Trail

Every action an agent takes should be logged with enough detail to reconstruct the entire decision chain. This isn't optional. It's how you detect attacks after the fact and how you debug unexpected behavior.

interface AgentAuditEvent {
  eventId: string;
  agentId: string;
  sessionId: string;
  timestamp: string;
  eventType: "tool_call" | "decision" | "memory_read" | "memory_write" |
             "escalation" | "output" | "error" | "permission_check";
  details: {
    action: string;
    input: Record<string, unknown>;
    output: Record<string, unknown>;
    reasoning?: string;      // Why the agent chose this action
    toolName?: string;
    resourceAccessed?: string;
    permissionUsed?: string;
    durationMs: number;
  };
  riskScore?: number;         // 0-100, auto-calculated
  flagged: boolean;           // Whether this triggered an alert
}

class AgentAuditLogger {
  private events: AgentAuditEvent[] = [];

  log(event: Omit<AgentAuditEvent, "eventId" | "timestamp" | "riskScore" | "flagged">) {
    const fullEvent: AgentAuditEvent = {
      ...event,
      eventId: crypto.randomUUID(),
      timestamp: new Date().toISOString(),
      riskScore: this.calculateRisk(event),
      flagged: false,
    };

    // Auto-flag high-risk events
    if (fullEvent.riskScore > 70) {
      fullEvent.flagged = true;
      this.alertSecurityTeam(fullEvent);
    }

    this.events.push(fullEvent);
    this.persistToStore(fullEvent);
  }

  private calculateRisk(event: Partial<AgentAuditEvent>): number {
    let score = 0;

    // Accessing sensitive resources increases risk
    if (event.details?.resourceAccessed?.includes("email")) score += 30;
    if (event.details?.resourceAccessed?.includes("credentials")) score += 50;
    if (event.details?.resourceAccessed?.includes("database")) score += 20;

    // Unusual patterns increase risk
    if (event.eventType === "permission_check") score += 10;
    if (event.details?.action?.includes("write")) score += 15;
    if (event.details?.action?.includes("delete")) score += 40;

    return Math.min(score, 100);
  }

  private alertSecurityTeam(event: AgentAuditEvent) {
    // Send to SIEM, Slack, PagerDuty, etc.
    securityAlerts.send({
      severity: event.riskScore > 90 ? "critical" : "high",
      message: `Agent ${event.agentId} flagged for ${event.details.action}`,
      event,
    });
  }
}

3. Human-in-the-Loop Gates

For any action with real-world consequences — sending messages, modifying data, making purchases, accessing sensitive information — require human approval. I know this feels like it defeats the purpose of automation, but here's the thing: you can automate the preparation and still gate the execution.

The agent does 95% of the work. A human spends 5 seconds reviewing and clicking "approve." That's still a massive efficiency gain, and it prevents the worst-case scenarios.

class HumanApprovalGate:
    def __init__(self, approval_channel: str, timeout_seconds: int = 300):
        self.channel = approval_channel
        self.timeout = timeout_seconds
        self.pending = {}

    async def request_approval(self, agent_id: str, action: dict) -> bool:
        """Submit an action for human approval. Blocks until approved or timeout."""
        request_id = str(uuid.uuid4())
        self.pending[request_id] = {
            "agent_id": agent_id,
            "action": action,
            "status": "pending",
            "requested_at": datetime.utcnow().isoformat(),
        }

        # Send to human reviewer (Slack, email, dashboard, etc.)
        await self._notify_reviewer(request_id, agent_id, action)

        # Wait for response with timeout
        try:
            result = await asyncio.wait_for(
                self._wait_for_decision(request_id),
                timeout=self.timeout,
            )
            return result == "approved"
        except asyncio.TimeoutError:
            audit_logger.warning("approval_timeout", extra={
                "request_id": request_id,
                "agent_id": agent_id,
                "action": action["type"],
            })
            return False  # Default deny on timeout

    async def _notify_reviewer(self, request_id, agent_id, action):
        message = {
            "text": f"Agent `{agent_id}` requests approval",
            "blocks": [
                {"type": "section", "text": f"Action: {action['type']}"},
                {"type": "section", "text": f"Target: {action['target']}"},
                {"type": "section", "text": f"Reasoning: {action['reasoning']}"},
                {"type": "actions", "elements": [
                    {"type": "button", "text": "Approve", "value": f"{request_id}:approve"},
                    {"type": "button", "text": "Deny", "value": f"{request_id}:deny"},
                ]},
            ],
        }
        await self.channel.send(message)

4. Tool Input/Output Validation

Never trust tool outputs blindly. Validate that what comes back from a tool call matches expected patterns and doesn't contain injection attempts.

class SecureToolExecutor:
    def __init__(self):
        self.validators = {}
        self.call_history = []

    def register_tool(self, name: str, handler, input_schema: dict,
                      output_schema: dict, max_calls_per_session: int = 100):
        self.validators[name] = {
            "handler": handler,
            "input_schema": input_schema,
            "output_schema": output_schema,
            "max_calls": max_calls_per_session,
            "call_count": 0,
        }

    async def execute(self, tool_name: str, inputs: dict) -> dict:
        tool = self.validators.get(tool_name)
        if not tool:
            raise SecurityError(f"Unknown tool: {tool_name}")

        # Enforce call limits
        if tool["call_count"] >= tool["max_calls"]:
            raise SecurityError(f"Tool {tool_name} call limit exceeded")

        # Validate inputs against schema
        self._validate_schema(inputs, tool["input_schema"], "input")

        # Execute the tool
        result = await tool["handler"](inputs)

        # Validate outputs against schema
        self._validate_schema(result, tool["output_schema"], "output")

        # Scan output for injection patterns
        self._scan_for_injections(result)

        tool["call_count"] += 1
        self._log_call(tool_name, inputs, result)

        return result

    def _scan_for_injections(self, data):
        """Scan tool outputs for common injection patterns."""
        text = json.dumps(data).lower()
        suspicious_patterns = [
            "ignore all previous",
            "system prompt",
            "you are now",
            "new instructions:",
            "disregard your",
            "override safety",
            "<script>",
            "javascript:",
        ]
        for pattern in suspicious_patterns:
            if pattern in text:
                security_logger.warning("injection_detected_in_tool_output", extra={
                    "pattern": pattern,
                    "data_preview": text[:200],
                })
                raise SecurityError(f"Suspicious pattern in tool output: {pattern}")

The Agentic AI Security Checklist

Here's a practical checklist for any team deploying AI agents. Print this out. Pin it to the wall. Review it before every deployment.

Pre-Deployment

Agent Security Checklist -- Pre-Deployment:

[ ] Agent permissions follow principle of least privilege
[ ] Every tool has an explicitly defined input/output schema
[ ] Sensitive operations require human-in-the-loop approval
[ ] Memory store has input validation and trust scoring
[ ] Rate limits set on all tool calls
[ ] Maximum session length and call budget defined
[ ] Agent cannot access credentials, keys, or secrets directly
[ ] Output sanitization prevents data leakage
[ ] System prompt includes explicit behavioral constraints
[ ] Fallback behavior defined for unexpected situations

Runtime Monitoring

Agent Security Checklist -- Runtime:

[ ] All tool calls logged with full input/output
[ ] Decision reasoning captured in audit trail
[ ] Anomaly detection on agent behavior patterns
[ ] Real-time alerting for high-risk actions
[ ] Session recordings available for review
[ ] Resource access patterns monitored for drift
[ ] Inter-agent communication validated and logged
[ ] Cost and usage dashboards active
[ ] Kill switch accessible for emergency shutdown
[ ] Regular review of flagged events by security team

Periodic Review

Agent Security Checklist -- Periodic Review:

[ ] Permission scopes reviewed and tightened quarterly
[ ] Tool endpoints verified and trust status confirmed
[ ] Memory stores audited for injected content
[ ] Incident response playbook updated
[ ] Red team exercises conducted on agent systems
[ ] Third-party MCP servers re-evaluated
[ ] Agent behavior baselines recalibrated
[ ] Security patches applied to orchestration framework
[ ] Team training on latest agent attack vectors
[ ] Compliance alignment checked (SOC 2, GDPR, etc.)

The Bigger Picture

I want to be clear about something: I'm not anti-agent. I think agentic AI is genuinely one of the most transformative developments in software. The article we published earlier this week on agentic AI workflow orchestration reflects that — the productivity gains are real.

But we're at a dangerous inflection point. Deployment is outpacing security by a wide margin. Teams are racing to ship agents into production while the security tooling, best practices, and threat models are still being figured out.

The Witness AI funding round — $58 million — is a signal that the industry is waking up. But awareness is not the same as readiness. Most engineering teams I talk to don't have a single person thinking about agent security specifically.

If you're deploying agents, you need to be that person. Or you need to find that person. Because the attacks I've described here aren't theoretical future threats. They're happening now. And the gap between what's possible and what's protected is wider than it's been at any point in the AI era.

What I'd Do This Week

If I were a tech lead or engineering manager reading this, here's what I'd do immediately:

  1. Audit your agent permissions. Look at every tool and resource your agents can access. If there's anything they don't strictly need, revoke it today.
  2. Turn on logging. If you aren't logging every tool call and decision an agent makes, you're flying blind.
  3. Add at least one human gate. Pick the highest-consequence action your agent can take and require human approval for it.
  4. Brief your team. Share this article or something like it. Make sure everyone deploying agents understands the threat model has changed.
  5. Schedule a red team session. Try to break your own agents before someone else does. You'll be surprised how easy it is.

Resources

Deploying AI agents and want to make sure you're not the next incident report? Contact the CODERCOPS team -- we help teams build agentic systems that are both powerful and secure.

Comments