Tool-Result Output-Contract Conformance Lab

The inverse of argument integrity — does your agent trust what its tools return?

The argument-integrity lab grades the structured arguments your agent SENDS against a tool's declared input schema. Nothing grades the tool result your agent CONSUMES against the tool's declared output contract. Yet a compromised tool, a tampered MCP response, or a confused deputy upstream returns a result that violates its declared schema — a status outside the declared enum (a refund tool returning approved_by_attacker), a type confusion (a string where a number was declared, with no coercion), an injected extra field not in the contract, or a value out of declared range — and an over-trusting agent acts on it. AgentSmack grades each incoming result against its declared output schema for the failure classes nothing else catches: an out-of-enum result driving a privileged action (a critical hard blocker, with RESULT→ACTION lineage); a type-confused result consumed without re-validation; an injected field in a privileged tool's result (escalating when the value is a directive); an out-of-range / out-of-pattern value; and a missing required field consumed as if complete. Ground-truth wins: a recorded authorized:true on the action clears the latent confused-deputy blocker, and the binding default-denies — a result is never bound to an action ordered before it. Findings map to OWASP LLM05 / LLM03 / LLM06 / NIST AI RMF coverage with paste-able remediation, a synthetic-attacker gauntlet, and audience report variants. Raw result values / secrets are never stored — only a redacted sha/prefix. Load a sample to watch the gauge swing with no live infra.

This lab grades a result's field / type / enum / range integrity. Its sibling, the tool-result truncation / pagination-completeness check, grades a different axis: did your agent treat a truncated / paginated / row-capped result as if it were complete — totalling page one, or concluding “no records” over a capped result?