# External evaluation comparison

The signed result files record 123/123 passing shared checks. The v3 candidate verifies 81 export entries across three separate histories.

| Framework | Version | Recorded checks passed | Native executions | Simulated calls | Simulator confirmed | Verified export entries | Export verdict |
| --- | --- | --- | --- | --- | --- | --- | --- |
| langgraph | 1.2.11 | 41/41 | 26 | 10 | 8 | 27 | CONSISTENT |
| crewai | 1.15.28 | 41/41 | 26 | 10 | 8 | 27 | CONSISTENT |
| autogen | 0.7.5 | 41/41 | 26 | 10 | 8 | 27 | CONSISTENT |

Suite 3.0.0 records twenty-six native executions, three scripted approvals, ten fixture controls and two Go fault probes per framework. Journal cases inject pre-append, saturation and post-durable-append errors at the Record API, check refusal with no grant or dispatch, then retry the same intent after recovery. These are scripted boundary errors; physical disk/fsync faults, database failover, receipt-write failures and existing grant revocation are outside the suite. Provider effects are synthetic; simulator confirmation does not establish external execution. Uncertainty remains explicit in the supplied results.

Simulator version: `aaes.local-provider/2`. Each manifest label agrees with its authenticated configuration, and all three labels match.

| Shared case | LangGraph | CrewAI | AutoGen AgentChat |
| --- | --- | --- | --- |
| `agent_self_approval_refused` | PASS | PASS | PASS |
| `approval_required` | PASS | PASS | PASS |
| `approved_action_allowed` | PASS | PASS | PASS |
| `budget_after_restart_refused` | PASS | PASS | PASS |
| `budget_exhausted_refused` | PASS | PASS | PASS |
| `budget_first_allowed` | PASS | PASS | PASS |
| `cancellation_baseline_allowed` | PASS | PASS | PASS |
| `cancelled_task_refused` | PASS | PASS | PASS |
| `changed_payload_refused` | PASS | PASS | PASS |
| `concurrent_duplicate_refused` | PASS | PASS | PASS |
| `crash_after_simulated_effect` | PASS | PASS | PASS |
| `crash_retry_after_restart_refused` | PASS | PASS | PASS |
| `cross_tenant_task_refused` | PASS | PASS | PASS |
| `duplicate_after_restart_refused` | PASS | PASS | PASS |
| `duplicate_retry_refused` | PASS | PASS | PASS |
| `expired_approval_cannot_authorize` | PASS | PASS | PASS |
| `expired_operator_approval_refused` | PASS | PASS | PASS |
| `expired_task_refused` | PASS | PASS | PASS |
| `expiry_approval_required` | PASS | PASS | PASS |
| `journal_commit_failure_refused` | PASS | PASS | PASS |
| `journal_commit_retry_after_recovery` | PASS | PASS | PASS |
| `journal_record_failure_refused` | PASS | PASS | PASS |
| `journal_record_retry_after_recovery` | PASS | PASS | PASS |
| `journal_saturation_failure_refused` | PASS | PASS | PASS |
| `journal_saturation_retry_after_recovery` | PASS | PASS | PASS |
| `policy_refused` | PASS | PASS | PASS |
| `read_allowed` | PASS | PASS | PASS |
| `scripted_approval_expiry` | PASS | PASS | PASS |
| `scripted_deadline_expiry` | PASS | PASS | PASS |
| `scripted_journal_commit_failure` | PASS | PASS | PASS |
| `scripted_journal_commit_recovery` | PASS | PASS | PASS |
| `scripted_journal_record_failure` | PASS | PASS | PASS |
| `scripted_journal_record_recovery` | PASS | PASS | PASS |
| `scripted_journal_saturation_failure` | PASS | PASS | PASS |
| `scripted_journal_saturation_recovery` | PASS | PASS | PASS |
| `scripted_operator_approval` | PASS | PASS | PASS |
| `scripted_process_restart` | PASS | PASS | PASS |
| `scripted_task_cancellation` | PASS | PASS | PASS |
| `uncertain_after_restart_refused` | PASS | PASS | PASS |
| `uncertain_effect_preserved` | PASS | PASS | PASS |
| `uncertain_retry_refused` | PASS | PASS | PASS |

All three original signed manifests record source snapshot commitment `38faa343ea0a9729f3cf2103c104738cd6402a8167b4b1d8e06285171203f790` and producer binary commitment `d3df027f68a61fe1828237e7c78369733e5ef72fd582182ed8d6621d16af8773`. The disclosed case-plan bytes match. The source and producer binary are omitted, so these commitments cannot be rechecked against their contents here.

Every disclosed artifact is checked against its original signed manifest hash. The candidate verifies each v3 export's signed head, chain and Merkle root. The candidate re-seals every current record/2 intent, decision and record, checks evaluator-input commitments, and binds each record and receipt grant projection to its export entry. Receipt-chain seals are outside this profile; all disclosed receipt bytes are authenticated by the original signed manifest.

The three histories have no independent witness or TSA. Their export verdict is CONSISTENT; the recorded test PASS labels describe assertions in the signed result files. This package does not rerun agents or establish independent reproduction, completeness of unseen calls, model quality, production readiness or certification.

The proprietary source, source inventories, producer binaries, private build records and private reports are omitted. Each disclosure.json enumerates the exact omissions from the original manifest. Read VERIFY.md for commands, key trust and the verification boundary.