Verifying an EVE Shadow Report

A shadow report (shadow_report_v2) is a signed record of evaluating a candidate policy in shadow — live or by historical replay — and measuring where it would diverge from the authoritative policy, without changing any active decision. The report lets a reviewer confirm the divergence summary is authentic and unaltered — without trusting the EVE server.

Readiness: authenticated shadow-policy evaluation and replay + signed divergence reports are PILOT_READY (authenticated pilot-ready). Shadow evaluation is structurally non-authoritative: it cannot alter the active verdict, and promotion is dry-run only.

What it proves

• Integrity + authenticity — the divergence summary (agreement/divergence counts, divergence-by-class, critical regressions) was signed by the EVE signing key (ECDSA P-384 in the production configuration; HMAC-SHA256 fallback is symmetric and not independently verifiable) over jcs-1 canonical bytes, and has not been altered.
• Binding to inputs — the report binds the source and shadow policy versions, the run id, the source input-set digest, and the selection digest, so a report cannot be silently re-pointed at a different policy or input set.
• Non-authoritative provenance — the report is produced inside a structural side-effect boundary; a shadow evaluation could not have changed the active verdict or caused an operational side effect.

What it does NOT prove

• The estimates block is labelled modeled effects from replay / sample — NOT observed production outcomes. A shadow report does not prove what the candidate policy would do in production; it reports modeled divergence over the replayed/sampled inputs.
• It does not mean a policy was promoted or activated. Promotion is dry-run only (activated: false); a shadow report never activates a production policy.
• Divergence counts describe the replayed set only; they are not a claim about traffic that was not replayed.

Required keys

shadow_report_v2 binds at least:

Verification fails closed if a required key is absent.

Canonicalization (jcs-1)

The report is signed over jcs-1, a constrained RFC 8785 profile with proven byte-for-byte parity across Python, TypeScript, and the browser for the supported value domain. Because jcs-1 fails closed on non-integer floats, agreement and divergence rates are carried as integer permille, not as floats. This is a constrained profile, not full RFC 8785 float formatting. A verifier reading a report whose canon is not jcs-1 reports unsupported canonicalization.

Verification profiles: strict vs standard

• Standard — verifies canonicalization, content hash, signature, and schema.
• Strict — additionally requires an independently verifiable ECDSA P-384 signature and rejects HMAC-fallback evidence as not independently verifiable. Use strict for third-party review with no shared secret. Both Python (verify-report) and TypeScript (verify_shadow_report.mjs) strict verifiers detect every listed tamper case.

Linked evidence

• Decision Certificate — the authoritative decision that shadow evaluation compares against (and does not alter).
• Red Team Report — a separate signed report of adversarial regression results.

Legacy shadow envelopes (json-sorted-compact-v1) remain verifiable and are not re-signed; verifiers select the scheme from the signed canon field.

Selective-disclosure / omission checks

The signature covers the full canonical payload, including the divergence-by-class breakdown and the critical_regressions set. Dropping a divergence class or a critical regression invalidates the content digest and the signature. Verify the full report before reading a subset; do not re-canonicalize a reduced object. Note also that a missing historical input is recorded explicitly as MISSING_HISTORICAL_INPUT and is never fabricated.

Failure categories

Freshness

A shadow report describes a specific run over a specific input set. It does not expire on its own; enforce a freshness policy on generated_at and on the source input-set digest so you do not act on a divergence summary computed against out-of-date inputs. Treat over-age reports as stale and re-run the shadow evaluation.

Replay behavior

A shadow report is a record, not an authorization, and shadow evaluation is structurally non-authoritative: re-presenting the report or re-running the replay never changes the active verdict and causes no operational side effect (all real side-effect kinds fail closed inside the shadow boundary). Deterministic run_id / input_set_digest make a re-run over the same inputs reproducible.

Verify it yourself

Python

# The unified verifier ships with the EVE service; the first argument is the
# evidence KIND (a jcs-1 report routes to verify_signed_report_v2). Synthetic fixture.
import json
from core.eve_sdk import verify_evidence

with open("shadow_report.json", "r", encoding="utf-8") as fh:
    report = json.load(fh)

result = verify_evidence("shadow", report, strict=True)
print(result["valid"], result["reason"])  # True, "verified"

# Pip-wheel path for content-hash + ECDSA P-384 signature only:
#   from eve_coreguard import verify_decision_record
#   r = verify_decision_record(report, public_key_pem=public_key_pem)
#   assert r.valid and r.independently_verifiable

Node (TypeScript / ESM, Node >= 18)

import { verifyDecisionRecord } from "eve-ai-governance";
import { readFileSync } from "node:fs";

const publicKeyPem = readFileSync("public_key.pem", "utf8");
const report = JSON.parse(readFileSync("shadow_report.json", "utf8"));

const result = await verifyDecisionRecord(report, { publicKeyPem });
console.log(result.valid ? "VERIFIED" : "FAILED");

CLI

# The `eve` CLI ships with the EVE service/repo. Form: verify <kind> <evidence.json>
# [--strict]; the ECDSA P-384 public key is auto-fetched from /.well-known/eve-pubkey.
python -m core.eve_sdk.cli verify shadow shadow_report.json --strict

Readiness

• Authenticated shadow evaluation and signed divergence reports are PILOT_READY (authenticated pilot-ready).
• Offline verification (Python / Node / browser) is SUPPORTED; jcs-1 is SUPPORTED (constrained RFC 8785 profile; rates carried as integer permille).
• Shadow evaluation runs in hosted / embedded (service) mode; the Python eve-coreguard client (0.2.10) reaches the service via hosted mode. TypeScript verifies offline and never signs.
• Independent verification requires the ECDSA P-384 public key; the HMAC fallback is symmetric and not independently verifiable.
• The clients are published on public registries: pip install eve-coreguard (PyPI) and npm install eve-ai-governance (npm).

Limitations

• Divergence estimates are modeled effects from replay / sample, explicitly not observed production outcomes.
• Shadow evaluation cannot alter the active verdict; promotion is dry-run only and never activates a production policy.
• In the pilot, the replay-job / separation-of-duties store is in-process (bounded, tenant-partitioned); durable cross-restart persistence is a deployment concern.
• HMAC-fallback signatures are not independently verifiable; use the ECDSA P-384 configuration and the strict profile for external review.