Reliable AI Agents: Context, Tools and Evals

An agent needs an execution contract

An assistant that describes a deployment and an agent that changes production are different systems. The latter needs an explicit contract for what it can read, which actions it can execute, how it recovers from interruptions and what counts as success.

The model is one component of that contract. A credible design also includes a runtime, tool interfaces, durable state, authorisation and evaluation. This article proposes an architecture for a hypothetical release-preparation agent. The examples are design recommendations, not claims about deployments performed for Nikba clients.

Choose a bounded workflow before increasing autonomy

For a release assistant, the initial task might be: inspect an approved commit, run a defined test suite, produce release notes and prepare a deployment proposal. The runtime can encode those stages directly. There is little benefit in asking a model to rediscover the release process on every run.

Give the model discretion where the work actually requires interpretation: selecting a relevant diagnostic, explaining a failure or deciding which permitted read-only tool to call next. Anthropic's workflow-versus-agent distinction provides useful vocabulary, while its tooling discussion dates from 2024. More recent architectures should be evaluated on their own merits.

Define a step limit, elapsed-time budget, allowed tools, retry policy and explicit terminal states. “The model stopped talking” is not a sufficient success condition. For this example, success means that the test evidence is recorded and the proposal matches the approved commit.

Separate durable state from the current context window

The working context contains the information needed for the next decision. The durable record stores what has happened: task identity, source revisions, tool requests, results, approvals and produced artifacts. These lifetimes differ. A context reset or a replaced worker should not erase the authoritative execution history.

Anthropic's April 2026 Managed Agents architecture separates the session, harness and execution environment. One useful implication is that an agent can retrieve relevant events from a persistent record instead of relying on a single growing transcript.

For the proposed release assistant, persist the approved commit hash and artifact identifiers before any worker starts. On resume, reconstruct the necessary context from those records. If the working tree or deployment target has changed, treat the mismatch as a new decision rather than silently continuing an old plan.

Engineer the context for each decision

System instructions, tool descriptions, source documents and accumulated results compete for context. Include the current objective, relevant constraints, a concise execution summary and the evidence required for the next step. Store large logs as artifacts and retrieve the relevant portion when a diagnostic needs them.

Context engineering treats this selection as an ongoing runtime concern. Summaries can help, but they are derived representations. Preserve links to the original results so that an omitted warning or changed assumption can be checked.

As a design rule for our example, retain exact revision IDs, target environments, approval scope and test exit codes. Summarise repetitive progress output. This is a concrete information policy that an implementation can test; it is more useful than an instruction to “remember everything important.”

Make tools narrow, typed and permission-aware

A tool contract should describe its inputs, scope, outputs, possible side effects and failure modes. The runtime validates the request and the caller's authority before execution. Tool results should distinguish success, a retryable transport error, invalid input and a rejected operation.

inspect_release(commit, repository) → evidence
run_checks(commit, check_profile) → test_results
prepare_release(commit, evidence_ids) → proposal
execute_release(proposal_id, approval_id, idempotency_key) → execution_record

These interfaces are illustrative. The execution tool must resolve its target and permissions from trusted application state. An arbitrary string generated by the model should not determine which production account is authorised.

MCP can standardise how clients discover and call tools, but a protocol connection does not replace the application's security design. The MCP security guidance discusses threats including confused deputies and token passthrough. Its draft-specific details should be checked against the protocol version being deployed.

Treat retrieved instructions as untrusted input

Repository files, web pages and tool responses may contain instructions intended to redirect an agent. A model can misinterpret such content as authority. Prompt injection can therefore cross from a read operation into an attempted write or data disclosure.

OWASP's prompt-injection guidance recommends layered mitigations, including least privilege, deterministic checks and controls on privileged operations. Labelling external text is useful, but it does not establish a complete defence.

In the release example, the retrieved change log can inform release notes. It cannot grant access to another repository or authorise a deployment. The runtime should reject an attempted target change that lacks a matching approval, even if the model provides a convincing explanation.

Design retries around side effects

Consider a timeout after a deployment request. The remote operation may have succeeded even though its response was lost. Blindly repeating the tool call can duplicate work. Our proposed execution interface uses an idempotency key bound to the approved proposal; the service records that key with the operation and returns the recorded status on a retry.

Persist the intent before execution, reconcile uncertain outcomes and inspect the remote state before choosing a recovery action. The exact transaction boundaries depend on the external system. An agent's statement that it has “rolled back” is not evidence that a compensating action occurred.

Scope approvals to the actual action, destination and artifact. For this design, changing the commit or environment invalidates the previous execution approval. A permission granted for a staging check should not silently authorise production writes.

Evaluate outcomes and inspect the trace

An agent can produce persuasive release notes while testing the wrong revision. Score the resulting state and artifacts, then inspect the trace to understand how it reached them. Anthropic's January 2026 evaluation guidance describes tasks, trials and graders for systems that act across multiple turns.

For our release assistant, the initial evaluation matrix would include:

• A clean release whose evidence and proposal must reference the same commit.

• A failing test that must stop the execution path.

• An instruction hidden in a source document that attempts to change the deployment target.

• A transport timeout after the external operation has started.

• An approval that expires or no longer matches the artifact.

• A process restart that must resume from the durable record without repeating completed side effects.

Run repeated trials where output variability affects the result. Separate task completion, permission violations, recovery correctness, latency and cost. A single aggregate score can conceal a rare but unacceptable execution error. Maintain human review for ambiguous cases and use deterministic checks wherever the expected state can be specified directly.

Define what can change safely after launch

Version the model, system instructions, tool schemas and harness policy together with evaluation results. Before an upgrade, rerun the failure cases that justified existing controls. Model improvements can make an old workaround unnecessary, while new tool capabilities can expand the possible consequences of an error.

For a first implementation, prefer a release-preparation agent with observable, bounded tasks and a clear handoff to execution. Expand its authority only when the product needs it and the measured behaviour supports that decision.

For retrieval architecture and permissions, read RAG versus fine-tuning. For context and generation costs, see LLM inference.