I was almost sure that there can not be multiple objects with the exact
same object and revision number, for if this were allowed, how would you
know which one to use? But sure enough, section 7.5.6 clearly states that
it's allowed (not sure how I missed that before or where I thought I saw
that they had to be unique).
I understand how marking them as deleted works as well as not marking them
as deleted but just abandoning the object in favor of a new object.
However, I want to make sure understand why the original object would not
be visible. First, let me quote the spec, then explain how I imagine a
conforming reader would work, just to make sure we're all on the same
page.
Section 7.5.6
"The update’s cross-reference section shall include a byte offset to this
new copy of the object, overriding the old byte offset contained in the
original cross-reference section. When a conforming reader reads the file,
it shall build its cross-reference information in such a way that the most
recent copy of each object shall be the one accessed from the file."
So we start at the end of the file and get the trailer. If it has a link
to the previous trailer (/Prev), we go there (recurse until we're at the
first trailer section). Parse the xref section which the first trailer
references, then return and parse the next one and so forth. If the xref
table contains a byte offset for an object/generation we already saw, we
overwrite the old byte offset with the new one. For example, if the
original xref says object 1 0 was at offset 1234, and the next xref says
it's at 2345 we overwrite the 1234 with 2345.
Sorry if I'm a little slow, but I just want to make sure I have a firm
grasp on how it is supposed to work (per the spec). This will help me
work on PDFBOX-1000 to make sure it's implemented properly. My goal is to
be able to have the conforming parser in such good shape that people can
use it to determine if a PDF conforms to the spec or not, and if not
exactly why it doesn't, what part of the spec is violated, etc. This
would be useful for us, as well as developers of other PDF libraries. It'd
be awesome to have other PDF libs have automated tests which run their out
put through our conforming parser to make sure that they're doing
everything properly, looking for regression bugs, etc. Yes, I realize
that the PDF spec is 756 pages which makes this a lofty goal, but it's my
goal none-the-less. Now if only I had more time to work on this...
----
Thanks,
Adam
From:
Thomas Chojecki<[email protected]>
To:
[email protected]
Date:
06/20/2011 16:09
Subject:
Re: [jira] [Created] (PDFBOX-1042) Wrong XRefStream order while parsing
incremental updated PDF with XRefStreams
Am 20.06.2011 19:00, schrieb [email protected]:
What is the proposed solution for this? According to the PDF spec,
there
will never be two objects with the same object number and revision.
However, this is the real world, not a world of conforming PDF
documents,
so I completely understand that this does occur. My questions is
mainly:
how do you plan on telling which object is the "right" one, and which
one
should be overwritten?
First of all, with incremental updates you can overwrite existing
objects. The revision only increase if the prev. object was marked as
deleted. This is little strange. In normal case the revision will never
increment. a good parser can mark older not used objects as garbage.
example:
you have a page with some pictures. now you don't want some of the
images but you want add some text to the rest of it. if you want update
this page incremental, you need to get the page object. change it
(remove some images and add some text). if you write the new page object
and update the xref table, the old page object isn't visible any more.
the images are still in the document and reserve the object number.
the next pdf writer doesn't know that the objects aren't use any more.
if you add another page via incremental update, you use new objects with
new object numbers. each pdf has a limit amount of object numbers. 65k
and you can update existing objects but don't replace it by other.
so if you would spare some object numbers. you can try to mark unused
object as deleted. so a good pdf writer see this objects and can use the
same number with a incremented revision.
this is the theory, in the real world no one do this :) 65k objects are
enough and if someone plan to waste so many objects, he can use xref
streams instead. ;)
-------
The normal way to overwrite an object is using the same obj no. and the
same revision. To mark the new object, only the xref-table need to be
updated. so it's very important to parse the table from old to new one.
the new one should set the prev flag with the offset of the older one.
so a parser read the newest xref-table, check the prev value and jump
into this one. It's a kind of recursive parsing the table. The first is
the newest and the last table in the line is the oldest. A conforming
parser should parse the oldest first.
But, if you parse the xref table from the beginning of the file to the
end of the file, you read the table also the right way.
If you need to handle xref streams, the easy way from beginning to the
end of the file, doesn't work for most documents. documents that use
xref streams are mostly called "weboptimized" or "linearized".
Weboptimized files are optimized for reading the document from beginning
till end. this is for big documents. if you try to load a not optimized
pdf, the application need to read the whole stream and can then handle
the file. so if you only need the first page of the pdf, you need to
parse the whole pdf. For the slow web this isn't a good way.
weboptimized documents contain all needed information right in front of
the file so the reader can read a small amount of data and knows, how
many pages a document has. where the first page is without loading the
rest of the file. try to load the pdf specification from the web. you
will see the first page and a progress bar (depending of the speed of
our connection) that load the rest of the document.
for example:
a weboptimized document can contain more xref streams. the first one
contains only a few objects, the last one the rest of it.
The patch i tried yesterday, sort the xref streams from smallest prev
offset to the biggest. but in my example file the xref streams are mixed
so the offset of the prev doesn't help. the best way is to jump direct
to the offsets and read the streams recursive. maybe i got an idea
Sry for the long text, tried to explain it for every user that has a
basic knowledge of the pdf structure.
----
Thanks,
Adam
Best regards
Thomas
- FHA 203b; 203k; HECM; VA; USDA; Conventional
- Warehouse Lines; FHA-Authorized Originators
- Lending and Servicing in over 45 States
www.swmc.com - www.simplehecmcalculator.com
Visit www.swmc.com/resources for helpful links on Training, Webinars, Lender
Alerts and Submitting Conditions
This email and any content within or attached hereto from Sun West Mortgage
Company, Inc. is confidential and/or legally privileged. The information is
intended only for the use of the individual or entity named on this email. If
you are not the intended recipient, you are hereby notified that any
disclosure, copying, distribution or taking any action in reliance on the
contents of this email information is strictly prohibited, and that the
documents should be returned to this office immediately by email. Receipt by
anyone other than the intended recipient is not a waiver of any privilege.
Please do not include your social security number, account number, or any other
personal or financial information in the content of the email. Should you have
any questions, please call (800) 453 7884.
