Remediation-Efficacy Diff

Did your fix actually close the attacks?

The adversarial gauntlet generates payloads and reports which ones exploited your agent (the “before”), and the prompt-patch generator hands you paste-able hardening text — but nothing proves the proposed fix actually worked. This is that proof. Feed a before-patch gauntlet run and an after-patch run of the same per-payload classification list, and AgentSmack diffs them by payload identity (ap_<12hex>) into fixed (an attack that landed no longer lands), regressed (a previously-safe payload the patch broke), still exploited (residual risk the patch missed), plus added / dropped payloads — then folds a closed verdict (remediated · partially_remediated · ineffective · regressed · insufficient_signal) and an efficacy rate over the attacks that actually landed before. Two non-launderable properties make it CISO-grade: any new recommended regression is sticky — closing ten attacks never launders one newly-broken recommended payload — and zero attacks landed before yields an honest insufficient_signal, never a fabricated 100. delivery_failed is excluded from the denominator so a patch can’t inflate efficacy by making attacks fail to deliver. Raw payload bytes never enter the diff. Load a sample to watch it partition with no live infra.