Redaction is not a black rectangle

SpecSpec: ISO 32000-2, §7.11ISO 32000-2 §7.11SpecSpec: ISO 32000-2, §14.13.5ISO 32000-2 §14.13.5

At a glance

A black rectangle drawn over a name hides nothing. The page looks redacted, but the text underneath is still there, still selectable, still copyable. Real redaction does not cover the content. It deletes it.

This page explains the gap between a cosmetic overlay and true content removal, why metadata and embedded files survive a naive job, and how the NextPDF enterprise RedactionEngine refuses to pretend a region is gone when it cannot actually remove it.

Why this matters

The most quoted PDF data breaches were not clever attacks. They were black rectangles. Someone drew a filled box over a sensitive paragraph, exported the file, and shipped it. The recipient pressed the box, dragged across it, and the original words copied straight out of the document onto the clipboard.

The reason is structural. A PDF page is a set of drawing operators layered in order. Painting an opaque rectangle adds one more operator on top; it does not remove the text-showing operator beneath. To the eye the text is hidden. To the machine it is exactly where it always was. The overlay is a costume, and a costume comes off.

This is expensive because the failure is silent and final. The document looks correct, passes a visual review, and is published. The leak is discovered later by someone who selected the text, and by then every copy is already out.

The short version

• A black box is a drawing operation layered over the text, not a deletion of it. The bytes survive.
• Selectable text survives because the page content stream still carries the glyphs that draw the words.
• Document metadata survives because it lives in a separate place — the document information dictionary and the XMP packet — that no rectangle on a page can reach.
• Embedded files survive because a PDF can carry whole files inside it as embedded file streams. SpecSpec: ISO 32000-2, §7.11ISO 32000-2 §7.11 A mark on a page never touches an attachment.
• True redaction is destructive. It removes the content from the byte stream, not from the picture. What is gone is gone, and the residue surfaces are cleaned too.

How NextPDF approaches it

Cosmetic versus destructive, precisely

A cosmetic overlay changes what a page looks like. Destructive redaction changes what the document contains. The first is reversible by anyone with a text cursor. The second leaves nothing to recover.

The NextPDF enterprise RedactionEngine is built for the destructive case at the text-content level. Its redact() method takes the content, a RedactionPolicy describing what to find and how to replace it, and returns a RedactionResult. Inside, it detects the sensitive spans, coalesces any that overlap into disjoint covering ranges, and then rewrites the content string with substr_replace so each covered byte is removed exactly once. The sensitive text is not painted over. It is gone from the buffer the result carries back.

The result is honest about what happened: it records the SHA-256 hash of the original content, the rewritten redactedContent, an audit report of every finding, and a wasModified flag. You can prove the bytes changed, and you can prove which ones.

The residue you have to chase

Removing the visible text is necessary but not sufficient. The same words, and words you never saw on the page, hide in places a page-level edit never reaches.

A document carries metadata twice over: a legacy information dictionary and an XMP packet, either of which can hold an author name, a title, or a tool fingerprint. It can carry whole files as embedded file streams. SpecSpec: ISO 32000-2, §7.11ISO 32000-2 §7.11 It can bind those payloads to page structure as associated files, a related-artifact channel that the page does not render but the document still holds. SpecSpec: ISO 32000-2, §14.13.5ISO 32000-2 §14.13.5 None of these is on the page you redacted, so none of them is touched by a rectangle. The classic guidance on this — the NSA “Hidden Data and Metadata in Adobe PDF Files” report — exists precisely because so many real leaks came from the residue, not the visible page.

A redaction that deletes the words on the page but leaves the author’s name in the XMP packet and the original spreadsheet as an attachment has not redacted the document. It has redacted one view of it.

1. DetectFind every sensitive span — visible text, and the entity types a policy targets — not just the region you happen to see.
2. Delete, do not coverRewrite the content stream so the glyphs are removed from the bytes. An opaque box on top leaves the text underneath intact.
3. Clear the residueStrip the information dictionary and XMP metadata, and remove embedded and associated files that carry the same payload off-page.
4. VerifyConfirm no selectable text, metadata field, or attachment reproduces the redacted content before the file leaves the building.

What a complete destructive redaction removes, in order: detect the sensitive content, delete the bytes from the content stream rather than draw over them, then clear the residue surfaces — metadata and embedded payloads — that a page-level mark never reaches, and finally verify nothing copyable survived.

Failing closed instead of faking it

There are two ways to redact: by content (find this name, delete it) and by zone (paint out this rectangle of the page, whatever is under it). The enterprise engine does the first truthfully. It deliberately does not pretend to do the second.

The redact() method accepts a third argument, a list of ZoneRedaction coordinate regions. The text-content engine has no page-content overlay sink — no way to actually remove what sits inside a coordinate box. So when that list is non-empty, it does not draw a rectangle and call the zone “redacted”. It throws a typed RuntimeException and tells you to route the zone through a page-content redaction pipeline instead.

That refusal is the whole point of this page made into code. The engine would rather stop and name what it cannot do than return a “redacted” finding for a region it left completely untouched. A silent cosmetic overlay reported as a removal is the exact false assurance that produces a breach. Failing closed turns that breach into an error message you see before you ship.

Human-factors note:Human-factors note

The dangerous moment is when a tool looks like it succeeded. A box appears, the page reads as redacted, and a reviewer signs off — while the text underneath copies straight out. An engine that throws on a redaction it cannot truly fulfil moves the discovery from after publication, where it is a breach, to before, where it is a fixable error. The friction is the feature: you are made to use a route that actually deletes, rather than one that only appears to.

Common misconception

“I drew a black box over it, so it’s redacted.”

This is the trap, and it is the most common one in the entire format. A filled rectangle is a drawing instruction stacked on top of the text. The text-showing operators are still in the content stream, the glyphs still draw, and a select or a copy reads them out unchanged. You hid a picture; you did not change a document.

A second, subtler trap: “I deleted the text on the page, so the document is clean.” Not yet. The same content, and content you never displayed, can persist in the metadata and in embedded files. Redaction is finished only when every surface — page content, metadata, and attachments — is clear of what you set out to remove.

Limits and boundaries

The honest scope of what NextPDF does here, by edition:

• Core has no document redaction at all. It ships a privacy sanitiser for audit-log PII — keeping sensitive values out of your own diagnostic records — and that is a different thing entirely from redacting a delivered PDF. If you are on Core and you need to redact a document, the capability you want is not present. Do not draw a box and assume the engine removed anything.
• The enterprise RedactionEngine redacts text content, destructively. It rewrites the content buffer to remove the sensitive bytes and reports a hash, an audit report, and a wasModified flag so the removal is provable.
• Coordinate-zone redaction fails closed. Pass a non-empty ZoneRedaction list and the engine throws rather than fake an overlay. That is a deliberate safety property, not a missing feature: a region it cannot truly clear is one it refuses to call clean.
• Clearing every residue surface is the operator’s responsibility too. Removing visible text is one step. A complete job also strips the information dictionary, the XMP packet, and any embedded or associated files (SpecSpec: ISO 32000-2, §7.11ISO 32000-2 §7.11, SpecSpec: ISO 32000-2, §14.13.5ISO 32000-2 §14.13.5) that carry the same payload off the page.

Document redaction — edition availability

Edition	Availability
Core	○ Not in this edition. Core has no document redaction. It provides only an audit-log PII sanitiser, which keeps sensitive values out of your own diagnostic records — it does not redact a delivered PDF.
Pro	○Not in this edition
Enterprise	○ Available. The enterprise RedactionEngine performs destructive text-content redaction and fails closed — it throws a typed exception — on coordinate-zone redaction it cannot truly fulfil, rather than painting a cosmetic overlay and reporting a region as removed.

Related docs

• How PDF encryption really works — why encrypting a file is not the same as removing content from it, and where each protects you.
• Metadata and the XMP packet — the residue surface a page-level redaction never touches, in detail.
• An API that refuses to guess — the same fail-closed stance the redaction engine applies to a zone it cannot fulfil.

Glossary

• Cosmetic overlay — a drawing operation, such as a filled rectangle, layered over content to hide it visually. It does not remove the content beneath, which remains selectable and copyable.
• Destructive redaction — removal of the content from the document’s byte stream, not from the picture. Once done, there is nothing underneath to recover.
• Content stream — the ordered list of drawing operators that render a page, including the text-showing operators that draw glyphs. A box on top does not delete the text operators beneath it.
• Residue surface — a place a document holds content that the page does not display: the information dictionary, the XMP metadata packet, and embedded or associated files. A complete redaction clears all of them.
• Embedded file stream — a whole file carried inside a PDF (ISO 32000-2 §7.11). A mark on a page never touches an attachment.
• Fail closed — to stop with a typed error when an operation cannot be done truthfully, rather than produce a plausible-looking result that is wrong. The enterprise engine does this for coordinate-zone redaction it cannot fulfil.