Tools are how the AI system actually touches the business. A model that can only read its context window is a summariser; a model that can call your APIs, write to your records, send your Slack messages, and trigger your workflows is part of your production surface. Most of the accountability questions on AI engagements are tool-layer questions in disguise.
What this layer covers
- Defining tools the agent can call (signature, semantics, side effects)
- Named ownership — who is responsible when a tool's contract changes
- Audit logging — the record of every call with the prompt that triggered it
- The side-effect boundary — what is reversible, what requires human sign-off
- Protocol choices (native tool-use, MCP, function calling)
- The tool-marketplace anti-pattern
Three rules we hold
1. Every tool has a named owner
Not a team. A person. When the API changes — and it will — we know who to call. The owner is responsible for the tool's contract, its tests, its observability, and its deprecation path. We list the owner in the tool's docstring and in the deliverable handoff document.
Tools without owners are the easiest entries on the audit review for the customer's risk team to flag. Names solve this in one line of comment.
2. Side effects are reversible or signed-off
Reads, queries, lookups — fine for the agent to call autonomously inside the boundary the IAM role allows. Writes, sends, deletes, or anything that affects a customer record or moves money — behind a human review boundary by default. The boundary moves over time as the eval harness earns the trust, but the default is conservative.
The line is "would I want to read this in the post-incident report." If yes, sign-off required.
Mutating production records through an autonomous tool call is a deliberate decision, recorded in writing, with a specific reason. The default is human-in-the-loop.
3. Every tool call is audit-logged with the prompt that triggered it
Not "the tool was called" — the prompt that triggered it. The record includes the conversation turn ID, the prompt hash, the tool name, the arguments passed, the result returned, and the timestamp. Six months later, when a customer asks "why did your system refund this order", the answer is one query against the audit table.
The same audit log is what surfaces prompt-injection attempts — a tool call that does not match the natural reading of the user's turn is a flag. See Layer 09 for the prompt-injection defenses that build on this log.
Default reference architecture
Tool as a Lambda (or Fargate task)
Each tool is its own Lambda function in the workload account, with a dedicated IAM role scoped to exactly what the tool needs. The function's docstring is the tool definition: name, description, input schema, side effects, owner, audit log destination. The orchestration layer reads this metadata to construct the tool-use schema for the model.
Functions for reversible reads (lookup customer, query order history, check policy) — straightforward Lambda. Functions for mutations (issue refund, send email, escalate ticket) — Lambda that writes to a confirmation queue, awaits human sign-off, then executes. The sign-off step is a Slack approval, an in-product modal, or an email link, depending on the customer's workflow.
Tool definition
Three things the model needs to know about each tool:
- What it does — one sentence. "Issue a refund of the specified amount to the customer's original payment method."
- The inputs — typed schema. Required fields, optional fields, enums where applicable.
- The side effects — explicitly. "Mutates the order record; emits a webhook to the customer; visible in the refund queue."
The temptation is to write expansive docstrings that "help the model decide." More text is not more help. A precise one-sentence description with an exact schema gets cleaner tool calls than a paragraph of suggestions.
Audit log schema
DynamoDB for the primary store, S3 for long-term archival. Primary key on tool call ID, secondary index on turn ID for "all calls in this conversation", tertiary on user ID for "all calls this user's session caused." KMS-encrypted with the customer-managed key. Retention per the governance regime.
Authorization at the tool boundary
The agent's tool-use is one layer; the tool's own authorization is another. A read-customer tool checks that the calling agent's session has permission for the requested customer ID, via the same identity system the rest of the application uses. "The agent called it" is never sufficient justification for a side effect; the customer's own authorization system has the last word.
The reason: in a prompt-injection scenario, the agent may be convinced to call a tool it should not. The tool's own authorization is the second line of defense. We never let the tool layer trust the orchestration layer alone.
Build vs. buy at this layer
Default: build. Tools are the integration to the customer's business. There is no vendor for "the integration to the customer's internal CRM with the customer's specific permission boundary." Build them as ordinary backend code, with named owners, audit logs, and tests. The agent layer just discovers them and calls them.
The anti-pattern is the tool marketplace — pre-built integrations for every SaaS. Fine for personal-productivity agents that operate inside one user's permission scope. Wrong shape for production, because they bypass the customer's permission model, their audit requirements, and their ability to deprecate or rotate integrations cleanly. We have audited deployments where a marketplace tool was making API calls under shared credentials nobody had documented; rotation was a multi-week project.
The five mistakes we see
1. Unowned tools
Anonymous Lambdas in a folder. When the upstream API breaks, nobody knows who is responsible. Names solve this; we ask every engagement to list the owner before the tool ships.
2. Autonomous mutations on first deploy
Building the side-effecting tools to call autonomously from day one, before the eval harness has any evidence the agent is correct on this task. The right pattern is human-in-the-loop on day one, narrow the loop as the harness earns trust, only ever remove the human boundary with a specific written decision.
3. Audit log in CloudWatch text
Tool calls logged as console.log output in
CloudWatch instead of as structured records in a queryable
store. Six months later, a customer asks for evidence; the
only path is to grep gigabytes of log text. DynamoDB + S3 from
day one.
4. Bypassing the customer's auth
The tool trusts the agent's session implicitly. The tool's own authorization should be independent — same identity, same authorization system the rest of the customer's application uses. Defense in depth at the tool boundary.
5. Marketplace integrations with shared credentials
Discussed above. The hard rotation cost shows up the day a credential leaks.
How it connects to the other layers
The tools layer is downstream of Layer 05 — the orchestration layer is what dispatches tool calls. It depends on Layer 01's IAM scoping for the Lambda roles. It is graded by Layer 07 on tool-call-success rate. Its audit log is the foundation for Layer 09's prompt-injection defense and compliance posture. The observability layer surfaces tool-call success rate as one of the four AI metrics every production system needs to track.
Related: the orchestration layer reference architecture, the governance layer reference architecture, the tooling catalog, and the concepts & standards glossary.