[
https://issues.apache.org/jira/browse/PDFBOX-6252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18111663#comment-18111663
]
Gustavo A. commented on PDFBOX-6252:
------------------------------------
Michael, taken on both counts. The description is now a third of what it was,
with the original
attached as {{ANALYSIS-full.txt}}.
On the free lists: we are not asking that they be preserved, and that reading
is out of the
description now. It changes nothing -- the same reproducer, with the save
adding a page rather than
touching the title, reorganizes the list exactly as you describe (object 6
reused, the empty-list
entry now correct, {{/Size 7}}), and {{/Outlines 6 0 R}}, written by the
revision underneath, still
resolves to that page.
The free list was never the mechanism anyway. The writer numbers from
{{getHighestXRefObjectNumber()}}, the highest number with an *in-use* entry in
the model; in
PDFBOX-6236 the invisible number is an {{/XRef}} stream, in use in the file.
Seeding from the
previous trailer's {{/Size}} covers both.
> PDDocument.saveIncremental() writes an update section that contradicts the
> sections it extends, on a document whose cross-reference marks an object free
> --------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: PDFBOX-6252
> URL: https://issues.apache.org/jira/browse/PDFBOX-6252
> Project: PDFBox
> Issue Type: Bug
> Components: Writing
> Affects Versions: 2.0.37, 3.0.8 PDFBox, 4.0.0
> Reporter: Gustavo A.
> Priority: Major
> Attachments: ANALYSIS-full.txt, IncrementalXrefRepro.java,
> IncrementalXrefRepro20.java, XrefReuseRepro.java, XrefStreamRepro.java,
> XrefStreamTopFree.java, free-object-after-addpage.pdf, free-object-input.pdf,
> topfree-after-incremental.pdf, topfree-input.pdf
>
>
> h2. Summary
> {{PDDocument#saveIncremental}} derives the update section's trailer and
> cross-reference entries
> from the objects it holds in memory rather than from the combination of its
> own section with the
> ones already in the file. On a document whose cross-reference marks an object
> free, two things
> follow: {{/Size}} shrinks below the address space the file already claims,
> and that free number is
> handed to a new object without the writer knowing it was ever spoken for.
> No signing is involved, no cross-reference stream, and no file produced by an
> older PDFBox: the
> input is a five-object PDF written by hand. Measured identically on 3.0.6,
> 3.0.7, 3.0.8 and on the
> 3.0.9-SNAPSHOT and 4.0.0-SNAPSHOT builds of 2026-09-03 (JDK 25). Neither is a
> regression; neither
> is fixed in trunk.
> h2. Steps to reproduce
> Input ({{free-object-input.pdf}}, attached): five objects, table covering
> 0..6 in one subsection,
> {{/Size 7}}, object 6 free with the list 0 -> 6 -> end, catalog {{/Outlines 6
> 0 R}}.
> {code:java}
> try (PDDocument doc = Loader.loadPDF(input))
> {
> doc.getDocumentInformation().setTitle("a change, so the save has
> something to write");
> // doc.addPage(new PDPage()); // second run, for reading 2
> doc.saveIncremental(out);
> }
> {code}
> h2. 1. /Size shrinks (title-only save)
> {noformat}
> input /Size 7
> update /Size 6
> {noformat}
> Table 15 defines it over "the combination of the original section and all
> update sections", and
> says any object numbered above it "shall be ignored and defined to be missing
> by a conforming
> reader" -- so the catalog's {{/Outlines 6 0 R}} is instructed to be missing.
> The same happens on a
> cross-reference stream input ({{XrefStreamTopFree.java}}: 9 -> 8), where
> table 17 puts it even more
> directly: "the number one greater than the highest object number used in this
> section _or in any
> section for which this shall be an update_".
> h2. 2. The free number is handed to a new object (save that adds a page)
> The added page is written as {{6 0 obj}} -- the number the previous section
> marks free and the
> catalog still references. Read back, {{/Outlines 6 0 R}} from the earlier
> revision resolves to the
> new page, in PDFBox and in pypdf 6.16.2:
> {noformat}
> BEFORE /Outlines -> 6 0 R "Object 6 0 not defined." (dangling)
> AFTER /Outlines -> 6 0 R {'/Type': '/Page', '/MediaBox': [...]}
> {noformat}
> Reusing a free number is legal on its own. The objection is that PDFBox is
> not choosing to reuse
> anything: the number simply looks unallocated. When the earlier revision is
> signed, what it says
> changes underneath it.
> h2. The mechanism
> Both readings follow from one seed. {{COSWriter#write}} starts numbering at
> {{getHighestXRefObjectNumber()}}, which counts only numbers that have an
> *in-use* entry, and
> {{/Size}} is then written as that ceiling + 1. A number the file's own
> trailer covers but that
> carries no in-use entry is invisible to both. Trunk as of 2026-09-03:
> * {{COSWriter#write(PDDocument, SignatureInterface)}}:
> {{number = pdDocument.getDocument().getHighestXRefObjectNumber()}}
> * {{COSWriter#doWriteTrailer(COSDocument)}}: {{trailer.setLong(COSName.SIZE,
> number + 1)}}; the
> stream path does the equivalent in {{doWriteXRefInc}} with
> {{pdfxRefStream.setSize(number + 1)}}.
> h2. Suggested fix
> * Seed with {{max(getHighestXRefObjectNumber(), previous trailer /Size -
> 1)}}. The previous trailer
> already declares the address space the file claims, whether or not every
> number in it has an entry,
> so the writer needs to know nothing about free lists. On a well-formed file
> the two agree and
> nothing changes.
> * Write {{/Size}} as {{max(previous trailer /Size, highest object number
> written + 1)}}. An
> incremental update can never lower the address space of the file it extends.
> Not suggested: repairing the document before the save. Rewriting existing
> bytes breaks every
> signature already in the file, which is what an incremental save exists to
> avoid. A repair has to
> be appended as its own revision, by the caller.
> h2. Relation to PDFBOX-6236 and PDFBOX-5382
> 6236 is the same "the number looks unallocated" condition reached from a
> different direction: there
> the invisible number belonged to the increment's own {{/XRef}} stream, which
> had no entry in its
> own revision. PDFBOX-6176 fixed that entry in 3.0.8; the free-entry direction
> is not fixed. The
> first suggestion covers both.
> 5382 is the read side of the same missing information -- {{COSDocument}}'s
> cross-reference map
> holds no entries for free objects, as Michael Klink pointed out there in
> 2022. So
> {{SigUtils.checkCrossReferenceTable}} does *not* warn on either file, before
> or after the save:
> {{getXrefTable()}} comes back with keys 1..5 and object 6 is absent from the
> model. A control with
> an actual hole does produce the warning, so this is not a logging artefact.
> h2. 2.0 branch
> 2.0.37 writes the same update section as 3.0.8 for the same input, byte
> offsets included, so
> reading 1 holds there. Reading 2 does not: with the changed objects flagged,
> 2.0.37 numbers the
> added page 7 and writes the free object out as {{6 0 obj null}}.
> h2. Attachments
> {{free-object-input.pdf}} / {{free-object-after-addpage.pdf}} and
> {{topfree-input.pdf}} /
> {{topfree-after-incremental.pdf}} are the inputs and outputs of the two
> readings, a few hundred
> bytes each, synthetic, no data of any kind. The reproducers --
> {{IncrementalXrefRepro.java}},
> {{XrefReuseRepro.java}}, {{XrefStreamRepro.java}}, {{XrefStreamTopFree.java}}
> and
> {{IncrementalXrefRepro20.java}} for the 2.0 API -- build their input in
> memory and need only PDFBox
> and its runtime dependencies, plus {{log4j-api}} on 4.0.0-SNAPSHOT.
> {{ANALYSIS-full.txt}} is the long-form analysis behind the above: the full
> walkthrough of the
> clauses involved, the measurements build by build, and what the appended
> section does to the free
> list.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
