What redaction actually removes

Every couple of years, a court filing or a government report is published with names blacked out, and within a day somebody has copied the text out from under the boxes and posted it. It happens often enough that it has stopped being news. It should have stopped being possible.

Here is why it keeps happening.

A page is a list of instructions, and a black box is one more of them

A page in a PDF is not a picture. It is a sequence of drawing instructions: set this font, move to this point, show these glyphs, fill this rectangle. A reader runs them in order, and what you see is whatever ended up on top.

Drawing a black rectangle over a name appends one instruction to that list. It does not remove the instruction that drew the name. The name is still in the file, at the same coordinates, in the same font, in the order it always had — and anything that reads the file rather than looking at it will find it there. A copy and paste will. So will a text extractor, a search index, and a screen reader.

This is not a bug in one particular tool. It is what drawing means. A tool that offers you a rectangle and calls it redaction has told you the truth about what it drew and nothing at all about what it removed.

Removing means editing the page

To actually remove the name, something has to go into the page’s content stream, find the instructions that draw those particular glyphs, and take them out — leaving everything around them exactly as it was, so the line above still sets in the same font and the paragraph below does not shift up.

A face inside a photograph is the same problem in a different encoding: pixels in an image object, which means decoding the image, clearing the area, and writing it back. Covering it achieves nothing.

Only then is the box drawn, because a reader still has to be able to see that something was taken out. The order is the whole of it: content first, box afterwards.

The copies a file keeps without telling you

A tool can edit the content stream correctly and still hand back a document that contains what you removed, because a PDF keeps more than one copy of its own words.

• The document details. The title and author fields routinely restate the thing being hidden. Interview — <the name you just redacted>.docx is a title, and a title is not on the page.
• Earlier versions of the page. A PDF can be saved incrementally: the new version is appended and the old one stays in the file, unreferenced and perfectly readable. Black out a page in a file like that, save it the same way, and you have shipped a document that contains the page both with and without the box.
• Annotations and form fields. A comment, a highlight or a filled-in field carries its own text, separately from the page, and a reader hands it back on request — wherever on the page it happens to sit.
• The tagging. A tagged PDF carries a structure tree, which is the reading order a screen reader follows. It holds a copy of the text.

Pagecraft clears all four, and then writes the file out fresh from the objects that are still referred to, so nothing that was orphaned along the way is carried forward into the result.

The part that is not optional

A claim that something is gone is worth exactly as much as the check behind it, so the claim is checked. After the new file has been written, the same search that found the text runs again over the bytes that came out. If any of it is still there, the operation fails and nothing is written: no file, no download, an error naming the pages the text survived on.

That is a deliberately annoying failure. A document that looks redacted and is not is more dangerous than one that was never touched, because somebody is about to publish it.

What it will not do

• A scan is a picture of words. There is nothing in it to search for. Recognise the text first, or give the area to clear as a rectangle.
• Drawings stay. A chart or a hand-drawn signature under a rectangle is covered by the box rather than taken out of the file. Text and images are removed; vector artwork is not.
• An area that cuts into a word takes the rest of that word. Half a name left on the page is worse than a whole one, so the boundary is rounded outwards rather than in.
• A protected document comes back in the clear. If it needs a password, protect it again afterwards.

The tool page lists the rest of the limits, and it lists them before you use it rather than afterwards.

Where this runs

On the machine you are reading this on. Opening the document, finding the text, editing the content streams, decoding and re-encoding the images and writing the new file all happen inside the browser tab, and the bytes are never sent anywhere, because there is nowhere for them to be sent.

For a document you are redacting, that is not a nicety. Sending a file somewhere to have a secret taken out of it means the secret has already been somewhere else.