Redaction and purge

The tension

Immutability is the feature. Every state has a content address, every capture is forever, every reviewer can trust that the history they're reading wasn't quietly rewritten. That guarantee is what makes the rest of Heddle work.

But sometimes the bytes themselves are wrong. Someone pasted an API key into a comment. A test fixture contains real customer data. A blob got committed that violates GDPR. The bytes need to come out — yet you don't want to undermine the immutability story.

The naive answer is "amend history and re-sign everything." That answer has burned every team that's tried it. Heddle's answer is redaction: the bytes leave, the receipt stays.

The receipt model

A redaction operation does five things:

1. Resolve the blob. The operator names a (state, path) pair on the command line; Heddle looks up the blob's BLAKE3 hash at that location.
2. Build a Redaction object — the tombstone — carrying the redacted blob hash, the state, the path, the operator's reason, the redactor identity (name + email from the principal), and the timestamp.
3. Sign optionally. When --sign-with is passed, Heddle signs the canonical payload with Ed25519, RSA, or P-256 (per the key's PEM header or --sign-algo) and attaches the signature to the Redaction. Signing is the auditor's verification anchor; the redactor identity is recorded either way.
4. Store the tombstone in the redactions store. The store is keyed by blob hash, so a single declaration causes the stub to render everywhere that blob's address surfaces — across every state, every thread, every mounted view. --all-states adds an extra audit OpRecord entry for each additional (state, path) the blob occupies.
5. Append OpRecord::Redact to the oplog. The audit trail lives in the same append-only record as every other action — see the oplog.

After redact: the blob's address returns a stub on materialize, but the original bytes stay on disk. The Redaction tombstone is in the DAG forever — auditors can verify the operation by re-walking the canonical payload and checking the signature, if any.

Purge vs redact

Redaction is the soft form: bytes stay on disk; the stub plus the tombstone do the work. Purge is the hard form: the loose blob is deleted from local storage, and an OpRecord::Purge entry is appended to the oplog. The Redaction tombstone itself is unchanged in shape — only its purged_at field is set. All audit fields (redactor, reason, timestamp, signature) are preserved.

Two honest gotchas:

• Pack files. If the blob has been packed into a packfile, purge can remove the loose copy but the packed copy persists until you repack. Purge warns when this happens.
• Replicas need a signed purge to propagate. A purge ships over the wire as a Redaction carrying purged_at, and receivers fail closed — unsigned, tampered, and untrusted-key records are all refused at the receive boundary. Run heddle purge apply --sign-with <key.pem> --force so the operation can travel (--force is the irreversibility confirmation purge always requires), and make sure each receiving replica has the signer's public key in [redact] trusted_keys (manage it via heddle redact trust add --from-pem <key.pem>). An unsigned purge still applies locally; it just doesn't propagate. Note that a purge only drops the loose copy on each replica — packfile repack after wire-purge is an operator follow-up.

Purge is irreversible and refuses to run without --force. Use it only when policy requires the bytes themselves to leave disk — leaked credentials that need to be unrecoverable, certain regulated PII. For routine secret-out operations where the audit trail is enough, redact alone is the right tool.

The audit trail

Every redaction writes an OpRecord::Redact to the oplog and stores a Redaction tombstone in the DAG. The tombstone carries the redactor identity (name + email from the principal), the timestamp, the redacted blob hash, the state and path the operator named, the reason, and — when --sign-with was passed — a cryptographic signature over the canonical payload. Auditors verify the operation via heddle redact show, which reports a three-state signature status: verified, unsigned, or tampered.

Redact a leaked API key from a test fixture

bash$ heddle redact apply HEAD --path src/test-fixtures/auth.json --reason "leaked API key" --sign-with ~/.heddle/keys/ops.pemredacted src/test-fixtures/auth.json (b3a8e201) in d01a8b4e (redaction r0a1b2c3)  reason: leaked API key

Why this isn't an escape hatch

Redaction is intentionally not a general-purpose history editor. You can't redact a thread's resolution. You can't redact someone's signature. You can't redact an oplog entry. The only thing you can redact is a blob — and even then, the redaction itself appears in the oplog as a first-class event.

The promise to reviewers stays intact: history wasn't rewritten. Specific bytes were withdrawn, with a paper trail.

The commands

• heddle redact — apply a redaction, list active redactions, show a tombstone. The soft form: the bytes leave materialized views; the receipt stays for audit.
• heddle purge — harden a redaction into a physical delete. The hard form: the bytes themselves leave disk. Irreversible.

Related: attribution and signing for how the redactor's identity is recorded; the oplog for where redaction tombstones land.