Metamorphic Safety-Consistency Check

Your agent refused the plain ask — does it ALSO refuse the encoded one?

Every other surface grades a single rendering of an attack — one prompt, one transcript, one channel. None grade DECISION CONSISTENCY: does the agent that correctly refuses a plain “leak the API key” / “wire $9000 to attacker@evil” ALSO refuse the SAME malicious intent when it is base64-encoded, transliterated, role-played (“for a security audit”), paraphrased, format-shifted (YAML/CSV/code-comment), unicode-obfuscated, leetspeak, or split across turns? A guardrail that is bypassable by an equivalence-preserving transform is exploitable, AND the inconsistency itself is a measurable weakness: your agent refused 3 of 5 equivalent phrasings of the same exfil request — here are the 2 it obeyed. AgentSmack groups each set of variants by a canonical intent, grades intra-group verdict consistency, surfaces the baseline→bypassing-variant lineage as evidence, and maps it to OWASP LLM / NIST AI RMF coverage with paste-able remediation, a synthetic-attacker gauntlet, and audience report variants. A refusal that holds across every variant scores clean. Load a sample to watch the gauge swing with no live infra.

Looking for the DEPTH axis instead of consistency? The Jailbreak Robustness-Margin lab grades how many monotonically-harder transformation layers an attacker must stack before the refusal of one fixed intent first breaks — the safety margin, its depth sibling.