Re: [Podofo-users] Loading pdfs with large number of trailers / updates causes stack overflow

[F. E.]

Hello again,

today I had some time for testing whether reducing the try-catch clauses
solves the stack overflow, and it does:

I removed try-catch clauses in PdfParser::ReadNextTrailer &
PdfParser::ReadXRefContents.
Also I removed "PODOFO_RAISE_ERROR( ePdfError_NoTrailer )" in
PdfParser::ReadNextTrailer() when there's no trailer,
since the error got suppressed in subsequent try-catch clauses anyway.

With this, I could safely try to load the test document with 160 trailers,
and podofo failed to load it safely without running into the stack overflow
(tested witth Debug and Release).
I added a patch file for the changes to the mail, although its for v0.9.8.
We don't have upgraded to podofo 0.10 yet, So I can't test any changes to
this version beforehand.
But recreating the patch file for 0.10 should be fairly easy, given that
the parser code did not really change between 0.9.x and 0.10.x

Minor side-note:
Why does the PdfTokenizer::RecursionGuard throw a "ePdfError_InvalidXRef"
error when the rmax recursion depth is reached?
It would make more sense to throw a special "MaxRecursionReached" error
here.

Regards,
F.E.

Am Di., 6. Feb. 2024 um 12:40 Uhr schrieb Francesco Pretto :

> Hello, thank you for the fixture, I can reproduce in Windows. With a test
> case fixing the bug becomes less "guess work" and more fun. Unfortunately
> I'm still in a period where I have little time to spend on PoDoFo in tasks
> that are not part of my daily work. If nobody takes it in the mean time, I
> think I may have time to look at it in a couple of weeks.
>
> Regards.
> Francesco
>
> On Thu, 1 Feb 2024 at 19:07, F. E.  wrote:
>
>> Hello again,
>>
>> using a ps script, I successfully created an empty pdf with 160 trailers.
>> The trailers are all empty and do not change any objects, but reference
>> each other via /Prev, which is sufficient for the stack overflow to happen.
>>
>> Feel free to test the parsing with it :)
>>
>> Regards,
>> F.E.
>>
>> Am Do., 1. Feb. 2024 um 07:58 Uhr schrieb F. E. :
>>
>>> "think the matter is complex enough that this may undermine the chances
>>> a complete fix materializes soon. Also if you or someone else come up with
>>> a fix it would be recommendable for such SO related issue to add a unit
>>> test on an anonymized file that shows the same or very similar behavior."
>>>
>>> All thats needed is an example pdf with a massive amount of trailers /
>>> updates, it does not have to be my specific file. I know one can open a pdf
>>> in update-only mode with Podofo, I just don't know what the easiest way to
>>> apply an update would be. Maybe such a pdf could even be crafted manually.
>>> I'll look into this in the coming days, and will share a file if I'm
>>> successful.
>>>
>>> Am Mi., 31. Jan. 2024 um 14:28 Uhr schrieb Francesco Pretto <
>>> cez...@gmail.com>:
>>>
>>>> On Tue, 30 Jan 2024 at 12:39, F. E.  wrote:
>>>>
>>>>>  I sadly cannot share the file I'm testing against, since it contains
>>>>> confidential data.
>>>>>
>>>>
>>>> I think the matter is complex enough that this may undermine the
>>>> chances a complete fix materializes soon. Also if you or someone else come
>>>> up with a fix it would be recommendable for such SO related issue to add a
>>>> unit test on an anonymized file that shows the same or very similar
>>>> behavior.
>>>>
>>>> Francesco
>>>>
>>> ___
>> Podofo-users mailing list
>> Podofo-users@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/podofo-users
>>
>
--- src/podofo/base/PdfParser.cpp   2022-04-11 07:07:00.0 +0100
+++ src/podofo/base/PdfParser.cpp   2024-02-13 14:35:00.0 +0100
@@ -298,8 +298,7 @@ void PdfParser::ReadDocumentStructure()
 try {
 ReadNextTrailer();
 } catch( PdfError & e ) {
-if( e != ePdfError_NoTrailer )
-throw e;
+throw e;
 }
 }
 
@@ -518,13 +517,8 @@ void PdfParser::ReadNextTrailer()
 //if( strcmp( m_buffer.GetBuffer(), "railer" ) == 0 )
 {
 PdfParserObject trailer( m_vecObjects, m_device, m_buffer );
-try {
-// Ignore the encryption in the trailer as the trailer may not be 
encrypted
-trailer.ParseFile( NULL, true );
-} catch( PdfError & e ) {
-e.AddToCallstack( __FILE__, __LINE__, "The linearized trailer was 
found in the file, but contains errors." );
-t

Re: [Podofo-users] Loading pdfs with large number of trailers / updates causes stack overflow

[F. E.]

Hello again,

using a ps script, I successfully created an empty pdf with 160 trailers.
The trailers are all empty and do not change any objects, but reference
each other via /Prev, which is sufficient for the stack overflow to happen.

Feel free to test the parsing with it :)

Regards,
F.E.

Am Do., 1. Feb. 2024 um 07:58 Uhr schrieb F. E. :

> "think the matter is complex enough that this may undermine the chances a
> complete fix materializes soon. Also if you or someone else come up with a
> fix it would be recommendable for such SO related issue to add a unit test
> on an anonymized file that shows the same or very similar behavior."
>
> All thats needed is an example pdf with a massive amount of trailers /
> updates, it does not have to be my specific file. I know one can open a pdf
> in update-only mode with Podofo, I just don't know what the easiest way to
> apply an update would be. Maybe such a pdf could even be crafted manually.
> I'll look into this in the coming days, and will share a file if I'm
> successful.
>
> Am Mi., 31. Jan. 2024 um 14:28 Uhr schrieb Francesco Pretto <
> cez...@gmail.com>:
>
>> On Tue, 30 Jan 2024 at 12:39, F. E.  wrote:
>>
>>>  I sadly cannot share the file I'm testing against, since it contains
>>> confidential data.
>>>
>>
>> I think the matter is complex enough that this may undermine the chances
>> a complete fix materializes soon. Also if you or someone else come up with
>> a fix it would be recommendable for such SO related issue to add a unit
>> test on an anonymized file that shows the same or very similar behavior.
>>
>> Francesco
>>
>


empty160trailer.pdf
Description: Adobe PDF document
___
Podofo-users mailing list
Podofo-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/podofo-users

Re: [Podofo-users] Loading pdfs with large number of trailers / updates causes stack overflow

[F. E.]

"think the matter is complex enough that this may undermine the chances a
complete fix materializes soon. Also if you or someone else come up with a
fix it would be recommendable for such SO related issue to add a unit test
on an anonymized file that shows the same or very similar behavior."

All thats needed is an example pdf with a massive amount of trailers /
updates, it does not have to be my specific file. I know one can open a pdf
in update-only mode with Podofo, I just don't know what the easiest way to
apply an update would be. Maybe such a pdf could even be crafted manually.
I'll look into this in the coming days, and will share a file if I'm
successful.

Am Mi., 31. Jan. 2024 um 14:28 Uhr schrieb Francesco Pretto <
cez...@gmail.com>:

> On Tue, 30 Jan 2024 at 12:39, F. E.  wrote:
>
>>  I sadly cannot share the file I'm testing against, since it contains
>> confidential data.
>>
>
> I think the matter is complex enough that this may undermine the chances a
> complete fix materializes soon. Also if you or someone else come up with a
> fix it would be recommendable for such SO related issue to add a unit test
> on an anonymized file that shows the same or very similar behavior.
>
> Francesco
>
___
Podofo-users mailing list
Podofo-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/podofo-users

Re: [Podofo-users] Loading pdfs with large number of trailers / updates causes stack overflow

[F. E.]

 @zyx 
Increasing the s_maxRecursionDepth would defeat the purpose of the
RecursionGuard. The guard is already aligned to the typcial stack size of
256, so increasing the max would just open door to stack overflows the
guard is meant to prevent. If anything, the guards max value may actually
be to big, since it does not leave "wiggle room" for client code using
podofo.
You can increase the stack, thats true, but that would just be a bandaid
fix. As soon as an even bigger pdf file comes alone, it might crash again.
The parsing code should not use an recursive approach, thats the design
flaw, imo. The recursion guard is meant to mitigate that, but the whole
process seems to have its own flaw somewhere.

@Christopher
At least in this place, the caught exceptions do get rethrown, so there is
no new exception with every catch:

try {
ReadNextTrailer();
} catch( PdfError & e ) {
if( e != ePdfError_NoTrailer )
{
e.AddToCallstack( __FILE__, __LINE__ );
throw e;
}
}

But with every catch, an additional line is added to the internal call
stack (a string collection) of the error object. Maybe thats the issue
here, the error object getting larger with every catch, and eventually
busting the stack frame? I don't know, but thats the first thing I'm going
to try out, commenting out this AddToCallstack cakks and check if it still
crashes. If it does, removing the try catch-clauses altogether (in the
recursively called functions), and let the try-catch clasues in
ReadDocumentStructure handle it. If that STILL causes stack overflow during
excpetion unwinding for my file, I'm dead lost.

@Michal
Yes, I saw that XRefStm takes part in the recursion as well, but I left it
out for simplicity. Its also not relevant in my use case, the pdf I'm
testing with does not have xref tables as streams.
But for a proper rework of the parsing, that streams need to be handled as
well, sure.
I also thought about using a stack / queue for xref offsets to check, but
I'm just not familiar enough with the whole parsing process to hack
something like that myself ^^

Greetings,
F.E.

Am Di., 30. Jan. 2024 um 23:00 Uhr schrieb Michal Sudolsky <
sudols...@gmail.com>:

>
>> right? When you set the s_maxRecursionDepth to large-enough value, will
>> PoDoFo be able to open the file? Possibly also making the stack larger,
>> to accommodate the recursion.
>>
>>
> I suppose it will work fine. Using recursion there is not a very good
> idea. Also there are two keys that need to be followed: "XRefStm" and
> "Prev". @Francesco Pretto  maybe you could fix that in
> the new podofo?  Something like this could work for example (pseudocode):
>
> xrefs - vector of xrefs (or stack);
> checked - set of already checked xrefs;
> xrefs.push_back(first xref - startxref);
> while(!xrefs.empty())
> {
>   XRefType xref = xrefs.back();
>   xrefs.pop_back();
>   if(checked.contains(xref))continue; // to avoid cycles
>   checked.insert(xref);
>   ...
>   just check whether this is non-stream xref with trailer -
> xrefstream contains just single "previous xref"
>   xrefs.push_back(next "XRefStm" key from trailer);
>   xrefs.push_back(next "Prev" key from trailer);
> }
>
>
>
>> ___
>> Podofo-users mailing list
>> Podofo-users@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/podofo-users
>>
> ___
> Podofo-users mailing list
> Podofo-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/podofo-users
>
___
Podofo-users mailing list
Podofo-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/podofo-users

Re: [Podofo-users] Loading pdfs with large number of trailers / updates causes stack overflow

[F. E.]

Hello again,

I tested a little further, and learned that the stack overflow actually
happens during Exception unwinding of an exception thrown by the
RecursionGuard.

So the recursion guards detects there's something wrong, throws its
exception, that is repeatedly caught and rethrown in ReadXRefContents &
ReadNextTrailer (for adding the line to the errors call stack), until the
OS intervenes for some reason. It feels like the exception unwinding causes
stack frames to be used as well, so it eventually runs out during unwinding.

I played around with the s_maxRecursionDepth of the RecursionGuard, it
works fine for up to 97, but with 98 or higher it starts to fail. Of course
that number may be dependant on my test enviroment / case. I sadly cannot
share the file I'm testing against, since it contains confidential data.

But it begs the question, whether that repeated catch-throw is really
necessary? Its only done to add numerous lines to the PdfError callstack,
which imo is not of much value in this context. There is no contextual
information of the singular calls (e.g. which object/trailer is currently
parsed), so even if it works and does not crash, you only see the same two
functions being called again and again.

So I'm suggesting to get rid of the catch-throw blocks in the recursively
called functions, hoping that this prevents the stack overflow during
exception unwinding.

Regards,
F.E.



Am Di., 30. Jan. 2024 um 10:43 Uhr schrieb F. E. :

> Hello dear podofo-users,
>
> we currently have some issues with loading some pdf files using Podofo.
> When performing the load operation, the podofo code crashes with a stack
> overflow error.
>
> I took a closer look at the pdf file, stepping through the Podofo code. In
> doing so, I found out that that this pdf file has 160 trailers / updates!
> This gets problematic, because parsing occurs recursively here:
>
> ReadDocumentStructure()
> -> ReadXRefContents()
> -> ReadNextTrailer()
> -> ReadXRefContents()
> -> ReadNextTrailer() ...
>
> Parsing begins with ReadDocumentStructure. At the end of the function, the
> (current) Xref table is read through ReadXRefContents. After the Xref
> table, there is a trailer, which is processed at the end of the
> ReadXRefContents function by calling the ReadNextTrailer function.
>
> In such a trailer, the /Prev attribute may be present, which refers to the
> previous version (offset to an Xref table). If such a thing is found,
> ReadXRefContents is called again with the new offset, and thus,
> ReadNextTrailer is called again later. If this also has a /Prev, the
> recursion continues until a trailer has no /Prev anymore.
>
> So, there are two functions calling each other repeadetly, without
> recursion unwinding in between. Consequently, the stack for the function
> calls is eventually exhausted, leading to a StackOverflow error (in my test
> at Update 127).
>
> We are using Podofo 0.9.8, but I checked the 0.10.x code as well, the
> parsing works essentially the same.
>
> So, in order to prevent stack overflows due to a large number of trailers,
> the code needs to be restructured from a recursive to an iterative
> approach. One solution could be to pass the /Prev offset from the trailer
> back to the caller, and then using a do-while loop until the passed back
> offset is zero. But I'm not familiar enough with all the details of the
> parsing code.
>
> Is there someone who could have a look into this?
>
> Is there maybe another, easier approach for preventing the stack overflow?
>
> Greetings,
>
> F.E.
>
>
>
___
Podofo-users mailing list
Podofo-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/podofo-users

[Podofo-users] Loading pdfs with large number of trailers / updates causes stack overflow

[F. E.]

Hello dear podofo-users,

we currently have some issues with loading some pdf files using Podofo.
When performing the load operation, the podofo code crashes with a stack
overflow error.

I took a closer look at the pdf file, stepping through the Podofo code. In
doing so, I found out that that this pdf file has 160 trailers / updates!
This gets problematic, because parsing occurs recursively here:

ReadDocumentStructure()
-> ReadXRefContents()
-> ReadNextTrailer()
-> ReadXRefContents()
-> ReadNextTrailer() ...

Parsing begins with ReadDocumentStructure. At the end of the function, the
(current) Xref table is read through ReadXRefContents. After the Xref
table, there is a trailer, which is processed at the end of the
ReadXRefContents function by calling the ReadNextTrailer function.

In such a trailer, the /Prev attribute may be present, which refers to the
previous version (offset to an Xref table). If such a thing is found,
ReadXRefContents is called again with the new offset, and thus,
ReadNextTrailer is called again later. If this also has a /Prev, the
recursion continues until a trailer has no /Prev anymore.

So, there are two functions calling each other repeadetly, without
recursion unwinding in between. Consequently, the stack for the function
calls is eventually exhausted, leading to a StackOverflow error (in my test
at Update 127).

We are using Podofo 0.9.8, but I checked the 0.10.x code as well, the
parsing works essentially the same.

So, in order to prevent stack overflows due to a large number of trailers,
the code needs to be restructured from a recursive to an iterative
approach. One solution could be to pass the /Prev offset from the trailer
back to the caller, and then using a do-while loop until the passed back
offset is zero. But I'm not familiar enough with all the details of the
parsing code.

Is there someone who could have a look into this?

Is there maybe another, easier approach for preventing the stack overflow?

Greetings,

F.E.
___
Podofo-users mailing list
Podofo-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/podofo-users

[Podofo-users] Patch for ignoring "/Prev 0" in trailer

[F. E.]

Hello dear podofo users,

I was having some issues with an unloadable pdf file. Analysing the file, I
found out that Podofo does not allow a zero for the /Prev key inside the
trailer dictionary:

xref
1 5
790024 0 n
000208 0 n
789878 0 n
12 0 n
790084 0 n
trailer
<< */Prev 0* /Root 5 0 R /Size 6 >>
startxref
790136
%%EOF

When removing or renaming the "/Prev 0" key, podofo can load the file just fine.
But then the file will no longer be loadable by Adobe Viewer. Why that
happens, I don't know,
especailly since Adobe can load other pdf files without any prev key
without issue.

So to resolve this whole issue, I created a patch that ignores the
/Prev key if its value is zero.

See attached patch.

Greetings

F.E.
--- src/podofo/base/PdfParser.cpp
+++ src/podofo/base/PdfParser.cpp
@@ -586,12 +586,17 @@
 
 if( trailer.GetDictionary().HasKey( "Prev" ) )
 {
-// Whenever we read a Prev key, 
-// we know that the file was updated.
-m_nIncrementalUpdates++;
-
 try {
 pdf_long lOffset = 
static_cast(trailer.GetDictionary().GetKeyAsLong( "Prev", 0 ));
+if( 0 == lOffset )
+{
+PdfError::LogMessage( eLogSeverity_Warning, "XRef contents 
at offset %" PDF_FORMAT_INT64 " is invalid, skipping the read\n", 
static_cast( lOffset ));
+return;
+}
+
+// Whenever we read a Prev key (!= 0), 
+// we know that the file was updated.
+m_nIncrementalUpdates++;
 
 if( m_visitedXRefOffsets.find( lOffset ) == 
m_visitedXRefOffsets.end() )
 ReadXRefContents( lOffset );
___
Podofo-users mailing list
Podofo-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/podofo-users

Re: [Podofo-users] Pdf not loadable: Unsupported encryption method Version=5 Revision=6

[F. E.]

Hello Podofo users,

@Andreas: much thanks for your patch!

are there any plans to add this patch to the SVN repo? Because I usually
generate the patches from the SVN commits for our own podofo build.
Sadly I don't see any changes on the SVN since last december. Or did podofo
switch to some git or something?

Greetings,
F. E.


Am Mi., 2. Feb. 2022 um 18:14 Uhr schrieb Andreas Brzesowsky <
andreas.brzesow...@dots.de>:

> Hi,
>
> yes, it's possible to always call idn_free. Changed this with attached
> patch.
> But only in Windows it is necessary to use this function, because of the
> heap concept of Windows.
> So the same reason why use podofo_free that always calls just free in the
> context of the module.
>
> Regards
>
> Andy
>
> Am 01.02.2022 um 08:33 schrieb zyx:
>
> On Mon, 2022-01-31 at 17:59 +0100, Andreas Brzesowsky wrote:
>
> +#ifdef _WIN32
> +// In Windows the libidn.dll could use an other heap and then
> the normal free will crash.
> +idn_free(password_sasl);
> +#else
>
>   Hi,
> would it make sense to always call the right function, not only
> sometimes? If the two can differ, no matter if only sometimes or
> always, then I'd say it's better to always use the right function.
>
> Pity the stringprep_profile() man page does not specify what function
> should be used to free the output buffer.
>
> It seems the idn_free() calls just free() on Linux, thus it might be
> possible to use that function unconditionally.
>   Bye,
>   zyx
>
>
> ___
> Podofo-users mailing 
> listPodofo-users@lists.sourceforge.nethttps://lists.sourceforge.net/lists/listinfo/podofo-users
>
>
> --
>
> [image: dots Software] <http://www.dots.de/en/>
>
> Andreas Brzesowsky
> Software Developer
>
> dots Gesellschaft für Softwareentwicklung mbH
> Schlesische Str. 27, 10997 Berlin, Germany
>
> Tel: +49 (0)30 695 799-33
>
> andreas.brzesow...@dots.de
> https://www.dots.de
>
> District court | Amtsgericht: Berlin Charlottenburg HRB 65201
> Managing Director | Geschäftsführer: Katsuji Kondo
>
>
> ___
> Podofo-users mailing list
> Podofo-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/podofo-users
>
___
Podofo-users mailing list
Podofo-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/podofo-users

Re: [Podofo-users] PdfPage::DeleteAnnotation does not delete the annotation object as it should (full mail)

[F. E.]

Hello again,

I couldn't help myself and created yet another patch ^^. Apart from
comments, this patch does only change a single line:
It moves the 'pObj->GetArray().erase( it );' call out of the while loop to
the end of the function, to keep 'ref' valid until it can be safely erased
from the array.
This approach is possible, since the while loop is canceled anyway in this
case, so the iterator remains unchanged and can be used for erase at the
end.
We could also remove the bFound bool and perform its check via 'it ==
pObj->GetArray().end()', but I left that out of the patch.
Maybe my comments are a bit redundant and could be shortened.

Greetings
F.E.

Am Di., 9. Apr. 2019 um 08:09 Uhr schrieb F. E. :

> Hello again,
>
> IIRC the object wasn't deleted to allow it to be reused, but I'm not sure
>> if that
>> was really correct (the docs aren't clear about this).
>>
> Unlikely. IMO the object was clearly meant to be removed, but since the
> used PdfReference is destroyed in the process,
> that never applied. See the last two lines of the function, it is even
> stated in the comment:
>
>> // delete the PdfObject in the file
>> delete this->GetObject()->GetOwner()->RemoveObject( ref );
>>
>
> Conclusion: Please wait for my acceptance of your first patch with some
>> cosmetic
>> and comment changes (I first need to test it), I reject v2 and v3
>> herewith.
>>
> All right. I understand the reasoning behind rejecting v2 and v3. I would
> like to see v2 being implemented for v0.9.7,
> but I fear the info will be forgotten until then ^^.
>
> Greetings,
> F.E.
>
> Am Sa., 6. Apr. 2019 um 20:47 Uhr schrieb Matthew Brincke <
> ma...@mailbox.org>:
>
>> Hello F.E., hello all,
>> > On 03 April 2019 at 12:11 "F. E."  wrote:
>> >
>> > Hello again,
>> > I created two alternative patches for this issue, since the first one
>> feels
>> > rather ugly. With patch v2, I switched the PdfReference parameter to
>> > call-by-value instead of call-by-reference.
>>
>> I'd like to refrain from changing API before 0.9.7 unless it's really
>> necessary
>> to fix a bug, here it isn't (see your first patch, which I'm going to
>> accept
>> after some testing going OK, from reading it it's fine).
>> > With patch v3, I kept the parameter as it is, but passed a local copy
>> of the
>> > required PdfReference to the function.
>>
>> That looks clean (and the comment I'd like to put into the first one) but
>> there's
>> one problem: The method called is public so all users would need to
>> change their
>> calls to fix the problem, so that's clearly impractical (+ docs'd need to
>> change).
>>
>> > With either of the three patches, the annotations are deleted without
>> leftover
>> > objects, as it should be.
>>
>> IIRC the object wasn't deleted to allow it to be reused, but I'm not sure
>> if that
>> was really correct (the docs aren't clear about this).
>>
>> Conclusion: Please wait for my acceptance of your first patch with some
>> cosmetic
>> and comment changes (I first need to test it), I reject v2 and v3
>> herewith.
>>
>> > Greetings, F.E.
>> >
>>
>> Best regards, mabri
>>
>>
>> ___
>> Podofo-users mailing list
>> Podofo-users@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/podofo-users
>>
>


pdfpage_delete_annotation_fix_4.patch
Description: Binary data
___
Podofo-users mailing list
Podofo-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/podofo-users

Re: [Podofo-users] PdfPage::DeleteAnnotation does not delete the annotation object as it should (full mail)

[F. E.]

Hello again,

IIRC the object wasn't deleted to allow it to be reused, but I'm not sure
> if that
> was really correct (the docs aren't clear about this).
>
Unlikely. IMO the object was clearly meant to be removed, but since the
used PdfReference is destroyed in the process,
that never applied. See the last two lines of the function, it is even
stated in the comment:

> // delete the PdfObject in the file
> delete this->GetObject()->GetOwner()->RemoveObject( ref );
>

Conclusion: Please wait for my acceptance of your first patch with some
> cosmetic
> and comment changes (I first need to test it), I reject v2 and v3 herewith.
>
All right. I understand the reasoning behind rejecting v2 and v3. I would
like to see v2 being implemented for v0.9.7,
but I fear the info will be forgotten until then ^^.

Greetings,
F.E.

Am Sa., 6. Apr. 2019 um 20:47 Uhr schrieb Matthew Brincke :

> Hello F.E., hello all,
> > On 03 April 2019 at 12:11 "F. E."  wrote:
> >
> > Hello again,
> > I created two alternative patches for this issue, since the first one
> feels
> > rather ugly. With patch v2, I switched the PdfReference parameter to
> > call-by-value instead of call-by-reference.
>
> I'd like to refrain from changing API before 0.9.7 unless it's really
> necessary
> to fix a bug, here it isn't (see your first patch, which I'm going to
> accept
> after some testing going OK, from reading it it's fine).
> > With patch v3, I kept the parameter as it is, but passed a local copy of
> the
> > required PdfReference to the function.
>
> That looks clean (and the comment I'd like to put into the first one) but
> there's
> one problem: The method called is public so all users would need to change
> their
> calls to fix the problem, so that's clearly impractical (+ docs'd need to
> change).
>
> > With either of the three patches, the annotations are deleted without
> leftover
> > objects, as it should be.
>
> IIRC the object wasn't deleted to allow it to be reused, but I'm not sure
> if that
> was really correct (the docs aren't clear about this).
>
> Conclusion: Please wait for my acceptance of your first patch with some
> cosmetic
> and comment changes (I first need to test it), I reject v2 and v3 herewith.
>
> > Greetings, F.E.
> >
>
> Best regards, mabri
>
>
> ___
> Podofo-users mailing list
> Podofo-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/podofo-users
>
___
Podofo-users mailing list
Podofo-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/podofo-users

Re: [Podofo-users] PdfPage::DeleteAnnotation does not delete the annotation object as it should (full mail)

[F. E.]

Hello again,

I created two alternative patches for this issue, since the first one feels
rather ugly.
With patch v2, I switched the PdfReference parameter to call-by-value
instead of call-by-reference.
With patch v3, I kept the parameter as it is, but passed a local copy of
the required PdfReference to the function.

With either of the three patches, the annotations are deleted without
leftover objects, as it should be.

Greetings,
F.E.

Am Mo., 1. Apr. 2019 um 11:47 Uhr schrieb F. E. :

> Hello again,
>
> I created a simple patch for this issue. The ref parameter is copied into
> a local refCpy variable, which is later used instead of the invalidated ref
> parameter for deleting the annotation and pdf object. Tested the patch
> against a sample pdf, the annotations are now correctly removed, nothing is
> left behind.
>
> Greetings,
> F.E.
>
> Am Fr., 29. März 2019 um 13:30 Uhr schrieb F. E. :
>
>> Hello dear podofo users,
>>
>> I stumbled upon an issue with the deletion of annotations today:
>>
>> PdfPage.cpp:409ff
>>
>>> void PdfPage::DeleteAnnotation( int index )
>>> {
>>> PdfObject* pObj = this->GetAnnotationsArray( false );
>>> PdfObject* pItem;
>>> [..]
>>> pItem = &(pObj->GetArray()[index]);
>>> if( pItem->IsDictionary() )
>>> {
>>>[..]
>>> }
>>> else
>>> {
>>> this->DeleteAnnotation( pItem->GetReference() );
>>> }
>>> }
>>>
>>
>> When calling DeleteAnnotation with an index, the requested Annotation is
>> stored into pItem. In my case it is not a dictionary, so DeleteAnnotation
>> with a Reference is called:
>>
>> PdfPage.cpp:446ff
>>
>>> void PdfPage::DeleteAnnotation( const PdfReference & ref )
>>> {
>>> [..]
>>>
>> while( it != pObj->GetArray().end() )
>>> {
>>> if( (*it).IsReference() && (*it).GetReference() == ref )
>>> {
>>> pObj->GetArray().erase( it );
>>> bFound = true;
>>> break;
>>> }
>>> ++it;
>>> }
>>> [..]
>>> delete this->GetObject()->GetOwner()->RemoveObject( ref );
>>> }
>>>
>>
>> Here, the reference is removed from the array and the corresponding
>> object is deleted at the end.
>> *The problem is, the erase operation on the array destroys the reference
>> ref! *I can see with the debugger that Ref's object number changes after
>> the erase, so the subsequent RemoveObject call is in error! It either tries
>> to remove a non-existing object (best-case) or some other unrelated object
>> (worst-case), depending on what values are stored at the location now.
>> When checking the resulting PDF, I can see that the annotation objects
>> are still there, but it was removed from the page's annotation list.
>>
>> I guess the fix might be quite easy:
>> Either pass the PdfReference by value instead of by reference, or copy it
>> into a local before removing it from the array, so a valid PdfReference
>> object can be used for subsequent removal.
>> I didn't have time to test this myself today, though.
>>
>> Could someone confirm my findings, please?
>> Greetings
>> F.E.
>>
>> p.s.: Sorry for the first, uncomplete mail, didn't know gmail sends the
>> mail right away when pressing CTRL+Enter
>>
>


pdfpage_delete_annotation_fix_3.patch
Description: Binary data


pdfpage_delete_annotation_fix_2.patch
Description: Binary data
___
Podofo-users mailing list
Podofo-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/podofo-users

[Podofo-users] PdfPage::DeleteAnnotation does not delete the annotation object as it should (full mail)

[F. E.]

 Hello dear podofo users,

I stumbled upon an issue with the deletion of annotations today:

PdfPage.cpp:409ff

> void PdfPage::DeleteAnnotation( int index )
> {
> PdfObject* pObj = this->GetAnnotationsArray( false );
> PdfObject* pItem;
> [..]
> pItem = &(pObj->GetArray()[index]);
> if( pItem->IsDictionary() )
> {
>[..]
> }
> else
> {
> this->DeleteAnnotation( pItem->GetReference() );
> }
> }
>

When calling DeleteAnnotation with an index, the requested Annotation is
stored into pItem. In my case it is not a dictionary, so DeleteAnnotation
with a Reference is called:

PdfPage.cpp:446ff

> void PdfPage::DeleteAnnotation( const PdfReference & ref )
> {
> [..]
>
while( it != pObj->GetArray().end() )
> {
> if( (*it).IsReference() && (*it).GetReference() == ref )
> {
> pObj->GetArray().erase( it );
> bFound = true;
> break;
> }
> ++it;
> }
> [..]
> delete this->GetObject()->GetOwner()->RemoveObject( ref );
> }
>

Here, the reference is removed from the array and the corresponding object
is deleted at the end.
*The problem is, the erase operation on the array destroys the reference
ref! *I can see with the debugger that Ref's object number changes after
the erase, so the subsequent RemoveObject call is in error! It either tries
to remove a non-existing object (best-case) or some other unrelated object
(worst-case), depending on what values are stored at the location now.
When checking the resulting PDF, I can see that the annotation objects are
still there, but it was removed from the page's annotation list.

I guess the fix might be quite easy:
Either pass the PdfReference by value instead of by reference, or copy it
into a local before removing it from the array, so a valid PdfReference
object can be used for subsequent removal.
I didn't have time to test this myself today, though.

Could someone confirm my findings, please?
Greetings
F.E.

p.s.: Sorry for the first, uncomplete mail, didn't know gmail sends the
mail right away when pressing CTRL+Enter
___
Podofo-users mailing list
Podofo-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/podofo-users

[Podofo-users] PdfPage::DeleteAnnotation does not delete the annotation object as it should

[F. E.]

Hello dear podofo users,

I stumbled upon an issue with the deletion of annotations today:

PdfPage.cpp:409ff

> void PdfPage::DeleteAnnotation( int index )
> {
> PdfObject* pObj = this->GetAnnotationsArray( false );
> PdfObject* pItem;
> [..]
> pItem = &(pObj->GetArray()[index]);
> if( pItem->IsDictionary() )
> {
>[..]
> }
> else
> {
> this->DeleteAnnotation( pItem->GetReference() );
> }
> }
>

When calling DeleteAnnotation with an index, the requested Annotation is
stored into pItem. In my case it is not a dictionary, so DeleteAnnotation
with a Reference is called:

PdfPage.cpp:446ff

> void PdfPage::DeleteAnnotation( const PdfReference & ref )
> {
>
while( it != pObj->GetArray().end() )
> {
> if( (*it).IsReference() && (*it).GetReference() == ref )
> {
> pObj->GetArray().erase( it );
> bFound = true;
> break;
> }
> ++it;
> }
> }
>
___
Podofo-users mailing list
Podofo-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/podofo-users

Re: [Podofo-users] Fwd: Better handling of xref entries shorter than 20 chars

[F. E.]

 > I think "\n\r" combination is unnecessary.
On principal I agree. But without the patch, Podofo loads PDFs with '\n\r'
just fine and I don't want to change that without need.
Furthermore, the standard states 'if the marker is 2 characters (both a
carriage return and a line feed)'. Imo there's no restriction to the order
of CR+LF.

> This would be better: (e1 == '\r' && e2 == '\n') || (e1 == ' ' && (e2 ==
'\r' || e2 == '\n'))
I added two patches with your simplification, one with check for '\n\r'and
one without it.

Greetings,
F.E.

Am So., 10. März 2019 um 10:02 Uhr schrieb Michal Sudolsky <
sudols...@gmail.com>:

> I think "\n\r" combination is unnecessary.
>
> Also CheckEOL is too complicated there is unnecessary allocation and
> four string comparisons while it just had to compare two char values. This
> would be better:
>
> (e1 == '\r' && e2 == '\n') || (e1 == ' ' && (e2 == '\r' || e2 == '\n'))
>
> On Fri, Mar 8, 2019 at 8:16 PM F. E.  wrote:
>
>> Hello again,
>>
>> I created another patch for checking the xref size. With this iteration,
>> '\r\n', '\n\r', ' \r' and ' \n' are considered valid as EOL, anythign else
>> will fail the parsing of the current subsection.
>> I checked the patch with some of the researchers pdf files, it works
>> fine, but the error description from podofo could be a bit more clear,
>> though.
>>
>> I also checked the patch against the unit tests. The tests from the SVN,
>> although with fixes for MSCV compaired to the 0.9.6 tests, somehow produced
>> a "dereference out of range deque iterator" error. So I used the tests form
>> v0.9.6, had to fix some stuff, but tests were running through. I had one
>> failing test ('testDifferencesEncoding', last assert fails, somehow the
>> differences of the encoding are not applied), but thats totally unrelated
>> to my chances (I think).
>>
>> It would be nice if someone could check the patch again with a proper
>> test environment, mine feels very thrown together and quick-n-dirty (had to
>> dl and build cppunit as well and I'm not sure if I set everything up
>> correctly .. at least it built).
>>
>> Greetings
>> F.E.
>>
>> Am Do., 7. März 2019 um 13:13 Uhr schrieb F. E. :
>>
>>> I disagree. To much leeway only creates more attack surface for
>>>>> malicious pdfs, since parsing is alwas a delicate procedure.
>>>>>
>>>> I do not think that allowing xref entries shorter than 20 chars is
>>>> vulnerability. On the other side it could extend supported PDFs (if such
>>>> PDFs really exists).
>>>>
>>> My remark is more of a general advise. Parsers in general are often the
>>> weakest link and primary attack vector for attackers. Easy to exploit as
>>> well, just send a malicious input and a vulnerable parser will ultimately
>>> grant you RCE or what else. So the premise for writing a parser should
>>> always be, keep it as simple as possible and stick to the requirements.
>>> So i would say, if the pdf standard defines that xref entries have to be
>>> exactly 20 chacracters long, we should strictly stick to that. And if there
>>> are some PDFs with different-sized xref entries, they are to be rejected.
>>> Nevertheless, even if we would like to allow xref entries shorter than
>>> 20 chars, Podofos current parsing code still handles this wrong, so it has
>>> to be fixed regardless. And I think the easier solution right now is to
>>> just check for the correct size and fail consistantly otherwise. Changing
>>> the code to allow shorter xref entries requires a bigger change.
>>>
>>> Or there is hole in standard: https://pdfsig-collision.florz.de
>>>> In contracts with first two attacks seems SWA does not violate
>>>> standard, at least not with ByteRange key. From what I see in pdf reference
>>>> there is nothing what states that byte range must not include only
>>>> contents. And there is also not stated that there must be always exactly
>>>> two ranges. But seems for example acrobat reader does not accept more than
>>>> 2 ranges and checks whether byte range covers what it should.
>>>>
>>> I've got to read this link, looks interesting. Yeah, the SWA attack is
>>> quite a nifty trick and looks totally fine from the signature validation
>>> perspective. The ByteRange to the signature clearly states which parts to
>>> use for the check, which all adds up just fine. It may be an oversight of
>>> the standard, yeah, and what the acrobat 

Re: [Podofo-users] Fwd: Better handling of xref entries shorter than 20 chars

[F. E.]

Hello again,

I created another patch for checking the xref size. With this iteration,
'\r\n', '\n\r', ' \r' and ' \n' are considered valid as EOL, anythign else
will fail the parsing of the current subsection.
I checked the patch with some of the researchers pdf files, it works fine,
but the error description from podofo could be a bit more clear, though.

I also checked the patch against the unit tests. The tests from the SVN,
although with fixes for MSCV compaired to the 0.9.6 tests, somehow produced
a "dereference out of range deque iterator" error. So I used the tests form
v0.9.6, had to fix some stuff, but tests were running through. I had one
failing test ('testDifferencesEncoding', last assert fails, somehow the
differences of the encoding are not applied), but thats totally unrelated
to my chances (I think).

It would be nice if someone could check the patch again with a proper test
environment, mine feels very thrown together and quick-n-dirty (had to dl
and build cppunit as well and I'm not sure if I set everything up correctly
.. at least it built).

Greetings
F.E.

Am Do., 7. März 2019 um 13:13 Uhr schrieb F. E. :

> I disagree. To much leeway only creates more attack surface for malicious
>>> pdfs, since parsing is alwas a delicate procedure.
>>>
>> I do not think that allowing xref entries shorter than 20 chars is
>> vulnerability. On the other side it could extend supported PDFs (if such
>> PDFs really exists).
>>
> My remark is more of a general advise. Parsers in general are often the
> weakest link and primary attack vector for attackers. Easy to exploit as
> well, just send a malicious input and a vulnerable parser will ultimately
> grant you RCE or what else. So the premise for writing a parser should
> always be, keep it as simple as possible and stick to the requirements.
> So i would say, if the pdf standard defines that xref entries have to be
> exactly 20 chacracters long, we should strictly stick to that. And if there
> are some PDFs with different-sized xref entries, they are to be rejected.
> Nevertheless, even if we would like to allow xref entries shorter than 20
> chars, Podofos current parsing code still handles this wrong, so it has to
> be fixed regardless. And I think the easier solution right now is to just
> check for the correct size and fail consistantly otherwise. Changing the
> code to allow shorter xref entries requires a bigger change.
>
> Or there is hole in standard: https://pdfsig-collision.florz.de
>> In contracts with first two attacks seems SWA does not violate standard,
>> at least not with ByteRange key. From what I see in pdf reference there is
>> nothing what states that byte range must not include only contents. And
>> there is also not stated that there must be always exactly two ranges. But
>> seems for example acrobat reader does not accept more than 2 ranges and
>> checks whether byte range covers what it should.
>>
> I've got to read this link, looks interesting. Yeah, the SWA attack is
> quite a nifty trick and looks totally fine from the signature validation
> perspective. The ByteRange to the signature clearly states which parts to
> use for the check, which all adds up just fine. It may be an oversight of
> the standard, yeah, and what the acrobat reader is imposing sounds very
> reasonable, I think I will add something like that to my signature check as
> well.
>
> rom what I read about SWA I really do not think that too short xref entry
>> had role in that attack. I rather think that this is simply mistake or bug
>> in software which manipulated these pdf files. Only one entry is shorter
>> and there is sequence CR+LF+LF at the end of xref right before "trailer". I
>> am sending fixed version. Can you try it?
>>
> Yes, this particular file can be repaired, because it has only one shorter
> xref entry. I fixed it myself by "moving" a CR from the last line up.
> But the exploits from the researchers contain 5 genuinely different SWA
> pdf files (there are 27 swa pdfs in total, but many duplicates in different
> folders). Only the first I linked can be easily fixed, with the other four
> pdfs ALL xref entries of the forged xraf table are to short! See:
>
> https://www.pdf-insecurity.org/download/exploits/3_eXpert_PDF_12_Ultimate/siwa.pdf
> https://www.pdf-insecurity.org/download/exploits/5_Foxit_Reader/siwa.pdf
>
> https://www.pdf-insecurity.org/download/exploits/10_Nuance_Power_PDF_Standard/siwa-2.pdf
>
> https://www.pdf-insecurity.org/download/exploits/18_Perfect_PDF_10_Premium/siwa.pdf
>
> https://www.pdf-insecurity.org/download/exploits/20_Soda_PDF_Desktop/siwa.pdf
> <- Thats the one that can be repaired!
>
> Podofo is unable to load the first three, all because the

Re: [Podofo-users] Better handling of xref entries shorter than 20 chars

[F. E.]

>
> This one tiny error totally breaks the loading of the whole file, bc
> Podofo does not read the next subsection properly.
> Instead of '8 5', he reads '5 958' as next sub section!
>
The tiny error does not break the whole loading process, I've mistaken this
with other pdf files.
In this case, like stated below, it 'only' disrupts the parsing of the
following xref entries, resulting in the signature object not being parsed
at all.

Am Mi., 6. März 2019 um 16:11 Uhr schrieb F. E. :

>
> No, This is really not good idea. Probably many pdf which are working now
>> with podofo will stop working. First pdf which I checked now has SPACE+LF
>> and it opens fine in any possible pdf viewer. Also OpenOffice generates
>> SPACE+LF (version 4.1.2 on windows probably).
>>
>> Also in pdf reference is stated that if end of line marker is single
>> character then it is preceded by space, check page 94.
>>
> You are right, i didn't recognize that part. Maybe thats the reason why
> the Tests fail as well. We could easily add that to the check, though (but
> maybe with more than one line):
>
>  bool emptyValid = false;
>> if ( empty1 == '\r' ) { emptyValid = (empty2 == '\n'); }
>> else if (empty1 == '\n' ) { emptyValid = (empty2 == '\r'); } // May be
>> removed if LF+CR shall not be allowed!
>> else if (empty1 == ' ') { emptyValid = (empty2 == '\n' || empty2 == '\r'
>> ); }
>> if ( read != 5 || !emptyValid )
>>
>
> Maybe better would be to check whether are both empty1 and empty2
>> whitespaces.
>>
> Yep, that would be better, agreed.All the patch should accomplish is
> safely failing when the xref entry size is not correct. Bc right now it's a
> gamble whether podofo fails to load or is parsing nonsense.
>
> On the contrary I think podofo is too strict. Maybe would be better that
>> it actually can open these files with xref entries shorter or longer than
>> 20 chars as readers from that page do. I think some readers are implemented
>> to read xref table by lines (by using for example c++ function getline).
>>
> I disagree. To much leeway only creates more attack surface for malicious
> pdfs, since parsing is alwas a delicate procedure. A lax handling of the
> standard is the reason why the researchers had so much success with
> tinkering pdf files for circumventing the signature check. To be more
> specific, all the shown SIWA attacks use to-short xref entries. And it
> looks that this is actually a requirement for this attacks, otherwise the
> restrictions regarding offsets (secured by the signature) are much harder
> to meet.
>
>
>> I checked this file and all 3 xref tables has CR+LF
>
> Nope. Check it again, there is one entry with only LF:
> In the second xref table, right before the '8 5' subsection, the entry
> '005131 0 n' is missing it.
>
> This one tiny error totally breaks the loading of the whole file, bc
> Podofo does not read the next subsection properly.
> Instead of '8 5', he reads '5 958' as next sub section!
>
> Trust me, I checked every pdf file provided by the researchers, more than
> half of it had this error. Mostly the Load failed, but two of them could be
> loaded (e.g. the one I linked earlier). For this two pdf files, I could not
> find any siganture fields at all, bc. Podofo did not parse properly. The
> reference to the signature object was there, but it pointed to a not
> existing object. And thats the REAL issue here, when Podofo does NOT fail
> at loading, but the parsed document is insufficient.
>
> Greetings
> F.E.
>
___
Podofo-users mailing list
Podofo-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/podofo-users

[Podofo-users] Fwd: Better handling of xref entries shorter than 20 chars

[F. E.]

> No, This is really not good idea. Probably many pdf which are working now
> with podofo will stop working. First pdf which I checked now has SPACE+LF
> and it opens fine in any possible pdf viewer. Also OpenOffice generates
> SPACE+LF (version 4.1.2 on windows probably).
>
> Also in pdf reference is stated that if end of line marker is single
> character then it is preceded by space, check page 94.
>
You are right, i didn't recognize that part. Maybe thats the reason why the
Tests fail as well. We could easily add that to the check, though (but
maybe with more than one line):

 bool emptyValid = false;
> if ( empty1 == '\r' ) { emptyValid = (empty2 == '\n'); }
> else if (empty1 == '\n' ) { emptyValid = (empty2 == '\r'); } // May be
> removed if LF+CR shall not be allowed!
> else if (empty1 == ' ') { emptyValid = (empty2 == '\n' || empty2 == '\r'
> ); }
> if ( read != 5 || !emptyValid )
>

Maybe better would be to check whether are both empty1 and empty2
> whitespaces.
>
Yep, that would be better, agreed.All the patch should accomplish is safely
failing when the xref entry size is not correct. Bc right now it's a gamble
whether podofo fails to load or is parsing nonsense.

On the contrary I think podofo is too strict. Maybe would be better that it
> actually can open these files with xref entries shorter or longer than 20
> chars as readers from that page do. I think some readers are implemented to
> read xref table by lines (by using for example c++ function getline).
>
I disagree. To much leeway only creates more attack surface for malicious
pdfs, since parsing is alwas a delicate procedure. A lax handling of the
standard is the reason why the researchers had so much success with
tinkering pdf files for circumventing the signature check. To be more
specific, all the shown SIWA attacks use to-short xref entries. And it
looks that this is actually a requirement for this attacks, otherwise the
restrictions regarding offsets (secured by the signature) are much harder
to meet.


> I checked this file and all 3 xref tables has CR+LF

Nope. Check it again, there is one entry with only LF:
In the second xref table, right before the '8 5' subsection, the entry
'005131 0 n' is missing it.

This one tiny error totally breaks the loading of the whole file, bc Podofo
does not read the next subsection properly.
Instead of '8 5', he reads '5 958' as next sub section!

Trust me, I checked every pdf file provided by the researchers, more than
half of it had this error. Mostly the Load failed, but two of them could be
loaded (e.g. the one I linked earlier). For this two pdf files, I could not
find any siganture fields at all, bc. Podofo did not parse properly. The
reference to the signature object was there, but it pointed to a not
existing object. And thats the REAL issue here, when Podofo does NOT fail
at loading, but the parsed document is insufficient.

Greetings
F.E.
___
Podofo-users mailing list
Podofo-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/podofo-users

Re: [Podofo-users] Better handling of xref entries shorter than 20 chars

[F. E.]

>
> ... if I understand that correctly, then the proposed change allows
> "\r\n" and "\n\r" as a line separator, which I do not think is a good
> idea, thus I'd be rather more strict, than less

You understand correctly, and i would prefer the more strict approach as
well.
But, the current parsing without check does not distinguish between CR+LF
and LF+CR, both variants will work transparently.
So by using the less strict check, we limit the fail-return to the
instances we know are actually wrong without restricting this 'semi-legit'
LF+CR case.
That was my reasoning for the patch. Nevetheless, I added the strict patch
with this mail.

Greetings,
F.E.


Am Mo., 4. März 2019 um 17:12 Uhr schrieb zyx :

> On Mon, 2019-03-04 at 16:21 +0100, F. E. wrote:
> > empty1 and empty2 are two characters for holding CR('\r') and
> > LF('\n'). So when an xref entry
> > misses the CR (or the LF), empty2 will hold the first character of
> > the next xref entry. So why not check the two empty variables if they
> > hold the required characters and failing when they don't:
>
> Hi,
> I'm fine with the change, but...
>
> > > if ( read != 5 || ( empty1 != '\r' ) ||  ( empty2 != '\n' ) )
> > >
> >
> > Or a little bit less strict (allowing LF+CR):
> >
> > > if ( read != 5 || ( empty1 != '\r' && empty2 != 'r' ) ||  ( empty1
> > > != '\n' && empty2 != '\n' )   )
>
> ... if I understand that correctly, then the proposed change allows
> "\r\n" and "\n\r" as a line separator, which I do not think is a good
> idea, thus I'd be rather more strict, than less. There might be some
> tricks how to recover from broken XRef tables, which PoDoFo doesn't
> have many, if any at all.
>
> I do not know what others opinion is.
> Bye,
> zyx
>
>
>
> ___
> Podofo-users mailing list
> Podofo-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/podofo-users
>


podofo_xref_entry_crlf_check_strict.patch
Description: Binary data
___
Podofo-users mailing list
Podofo-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/podofo-users

[Podofo-users] Better handling of xref entries shorter than 20 chars

[F. E.]

Dear podofo users,

I'm currently investigating some forged pdf documents which try to
circumvent the signature checks of various pdf viewers. Maybe you heard
about that:
https://www.pdf-insecurity.org/signature/signature.html

When trying to load the used pdfs there, nearly all of them produced some
loading errors (which is fine, I guess). But four of them could be loaded,
so I could check them against my own signature validation code. Sadly, my
code did not find any signature fields for two of the four documents.
When debugging this and stepping through the podofo code, I learned that
there is some issue when reading the xref table. As it turned out, the
forged pdf did not respect the fixed xref entry size of 20 characters, but
used only a LF were CR+LF is needed. See:
https://www.pdf-insecurity.org/download/exploits/20_Soda_PDF_Desktop/siwa.pdf

Podofo is doing right in expecting the entry to be 20 characters long. But
when the entry is shorter, Podofo does not recognize this and continues
with the parsing, likely creating bogus (if it not entirely fails). I think
that's not a good behaviour. It would be better if the parsing fails as
soon as an entry shorter (or larger) than 20 chars is encountered. And if
I'm not mistaken, such a check is not even difficult to implement:

PdfParser.cpp:894ff

> // XRefEntry is defined in PDF spec section 7.5.4 Cross-Reference Table as
> // nn g n eol
> // nn is 10-digit offset number with max value 99 (bigger
> than 2**32 = 4GB)
> // g is a 5-digit generation number with max value 9 (smaller than
> 2**17)
> int read = sscanf( m_buffer.GetBuffer(),
> "%10" PDF_FORMAT_INT64 " %5" PDF_FORMAT_INT64
> " %c%c%c",
> , , , ,
>  );
>
> if ( read != 5 )
> {
> // part of XrefEntry is missing, or i/o error
> PODOFO_RAISE_ERROR( ePdfError_InvalidXRef );
> }
>

empty1 and empty2 are two characters for holding CR('\r') and LF('\n'). So
when an xref entry
misses the CR (or the LF), empty2 will hold the first character of the next
xref entry. So why not check the two empty variables if they hold the
required characters and failing when they don't:

if ( read != 5 || *( empty1 != '\r' ) ||  ( empty2 != '\n' )* )
>

Or a little bit less strict (allowing LF+CR):

if ( read != 5 || *( empty1 != '\r' && empty2 != 'r' ) ||  ( empty1 != '\n'
> && empty2 != '\n' )* )
>

With this proposed check, wrong xref entry sizes would be handled more
consistantly / reported straight away instead of silently continuing with a
wrong offset. We could also add an additional error code for it instead of
using ePdfError_InvalidXRef.

What do you think about that?

Greetings
F.E.


podofo_xref_entry_crlf_check.patch
Description: Binary data
___
Podofo-users mailing list
Podofo-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/podofo-users

Re: [Podofo-users] Check for preventing signatures from decryption not sufficient

[F. E.]

Hi,

I'm sorry, that's my fault. I've your previous message still marked as
> "to-be-done", but I changed my mind, I think it is risky to make such
> change just before the release. I didn't bother to let you know, for
> which I apologize.
>
Apology accepted. I just thought (besides being disgruntled), if I don't
bring up the issue again it might get totally forgotten.


> I didn't look into your PDF, neither I'd find time any time soon, I'm
> afraid. But if nobody is quicker, I'll check it in time of review &
> test of your patch.

Looking into the PDF when reviewing the patch is fine, I guess.
Don't forget the other PDF I linked to in a previous mail, in contrast to
the last PDF this one is loadable with my patch.


> Whether certain key should or should not be
> encrypted declares the PDF standard. It's just about to find out it
> there.
>
I had a brief look into the standard and could find it now, the ID should
be unencrypted:

"(Optional, but strongly recommended; PDF 1.1) An array of two byte-strings
constituting a file identifier (see Section 10.3, “File Identifiers”) for
the file. *The two bytestrings should be direct objects and should be
unencrypted.* Although this entry is optional, its absence might prevent
the file from functioning in some workflows that depend on files being
uniquely identified."

So Podofo is definitely doing something wrong there, since the ID
bytestrings of a trailer are always decrypted, as far as I can see! With
AESv2 encrpyted documents this wrong decryption does not cause errors, with
AESv3 it does, it seems. Keep in mind that the latter document (with AESv3)
is still not loadable even after the decryption of the ID is circumvented,
there's might be something wrong in general with decrpytion of AESv3
encrypted documents.

Best regards
F.E.
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
Podofo-users mailing list
Podofo-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/podofo-users

Re: [Podofo-users] Check for preventing signatures from decryption not sufficient

[F. E.]

Hello dear podofo users,

I had some time to come up with a patch for this issue. I don't think it's
the best solution available (a proper redesign might be better), but at
least the patch should have low impact and is easily conceivable.

The current situation:

PdfTokenizer.cpp:615
// 'Contents' key of a /Type/Sig dictionary is an unencrypted Hex
string
bool bIsSigContents = key == PdfName( "Contents" ) &&
dict.HasKey( "Type" ) &&
dict.GetKey( "Type" )->GetDataType() == ePdfDataType_Name &&
dict.GetKey( "Type" )->GetName() == PdfName( "Sig" );

This check is invalid when the 'Type' key is located after the 'Contents'
key in the pdf file. The boolean is set wrongly to 'false'.

PdfTokenizer.cpp:621
// Get the next variant. If there isn't one, it'll throw
UnexpectedEOF.
this->GetNextVariant( val, bIsSigContents ? NULL : pEncrypt );

Due to the invalid check, the bool is 'false' and the unecnrypted signature
content is going to be decrypted, resulting in a decryption error ('Length
not a multiple of 16').

The patch idea:

We cannot check the dictionary type during parsing, so the check has to be
moved after all dictionary keys have been processed. Since we don't know
beforehand if the 'Contents' content has to be decrypted or not and we have
to read the pdf sequentially (thats the current design), we have to read
the 'Contents' content first and process it later. This requires some
function interface changes, so we are able to get the raw, unprocessed
content.

The patch itself:

Some of the PdfTokenizer functions were expanded with an optional boolean
parameter ('getRawHex') and/or were given a return value (EPdfDataType
instead of void).
The optional parameter only has an effect if the currently read out content
is a hex string, it prevents the conversion of the content via SetHexData
(when set to true).
The return value is required for checking whether the currently read out
content was actually a hex string or something else.

// 'Contents' key of a signature dictionary is an unencrypted hex
string.
// Since '/Type/Sig' may be located after 'Contents' key, we can't
check for it here and have to postpone the check.
bool bIsContents = ( key == PdfName( "Contents" ) );

// Get the next variant. If there isn't one, it'll throw
UnexpectedEOF.
// If we have the 'Contents' key and it is a hex string, we will
get the raw, unprocessed string.
// We will process the raw string at the end, after all keys have
been read, so the check for '/Type/Sig' is valid.
EPdfDataType dataType = this->GetNextVariant( val, pEncrypt,
bIsContents );
processRawHexContents |= ( bIsContents && dataType ==
ePdfDataType_HexString );

=> When the current key is 'Contents', the optional parameter is set to
indicate we want the raw hex content. If the function then returns the hex
string data type,
we know 'val' is set with a raw hex string. The string is stored into the
dictionary and a flag is set indicating that we have to process the raw hex
string later ('processRawHexContents').

// Process raw hex contents, decrypting it if we don't have a signature
dicionary.
if ( processRawHexContents )
{
// Check if signature dictionary
bool isSigDict = dict.HasKey( "Type" ) &&
dict.GetKey( "Type" )->GetDataType() == ePdfDataType_Name &&
dict.GetKey( "Type" )->GetName() == PdfName( "Sig" );

// Read raw hex contents from dictionary.
PdfString rawContents = dict.GetKey( "Contents" )->GetString(),
processedContents;

// Process raw hex contents (decrypting if NOT sig dict)
processedContents.SetHexData( rawContents.GetString(),
rawContents.GetLength(), isSigDict ? NULL : pEncrypt );

// Overwrite raw hex content in dictionary with processed one.
dict.AddKey( "Contents", processedContents );
}

When all keys were processed, the flag is evaluated; if set, the raw hex
string is extracted from the dictionary, processed according to the
dictionary type and stored into the dictionary again.


I tested the patch with the following file:
http://www.all-ip-store.de/_uploads/user/Telekom%20All-IP/IP-Voice-Data_
Auftragsformular.pdf
Without patch, the loading fails, with the patch the file is loaded without
error. I will check if the signature can be verified tomorow, other tests I
havn't done yet.

Best regards,
F.E.

2018-02-27 11:01 GMT+01:00 F. E. <exler7...@gmail.com>:

>Hi,
>> do you really expect everyone on the list to answer to your message
>> that he/she doesn't have currently time and/or resources to look on
>> your issue? I hope not.
>>
> No, I don't expect everyone to answer. But I

[Podofo-users] Check for preventing signatures from decryption not sufficient

[F. E.]

Hello dear podofo users,

today i stumbled upon a parsing bug in PdfTokenizer::ReadDictionary. With
SVN revision 1794, a check was added to prevent a signature content stream
from being decrypted by comparing the dictionary type to "Sig". But this
requires the type to be known/parsed beforehand, otherwise the dictionary
doesn't contain the type (yet). Consequently, if you have a pdf file where
the '/Type/Sig' is declared after the signature content, the check does not
work and the signature content gets decrypted, which in my case ultimately
prevented the pdf file from being loaded successfully. When I prevented the
decryption during debugging, the pdf file was loaded successfully.

I consulted the pdf reference for this issue, but couldn't find any
indication that the dictionary type has to be specified before the
signature content occurs, my test pdf should be perfectly valid. So podofo
needs to be fixed. I tried to analyze the code and a possible solution /
workaround, but I'm not feeling confident & competent enough to come up
with a good, working patch.

Regards,
F.E.
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
Podofo-users mailing list
Podofo-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/podofo-users

Re: [Podofo-users] Check for preventing signatures from decryption not sufficient

[F. E.]

>
>Hi,
> do you really expect everyone on the list to answer to your message
> that he/she doesn't have currently time and/or resources to look on
> your issue? I hope not.
>
No, I don't expect everyone to answer. But I think it's not too much to ask
for that *at least one* acknowledges it's a problem,
or tells me it's not important enough. Just *any reaction from anyone*!
Especially when there was quite some activity on the mailing list.

Yes, I agree the problem is there.

Great!


> No, I do not know how to deal with
> it at the moment. No, I cannot promise whether I'll look on it myself.
>
No problem.


> Yes, I would like to see this fixed in the next release of PoDoFo too.
>
That would be great.


> Yes, a simple test .pdf file would help with the reproducer.
>
I can provide that, luckily one of the pdf files is openly available:
http://www.all-ip-store.de/_uploads/user/Telekom%20All-IP/IP-Voice-Data_Auftragsformular.pdf


> P.S.: if anyone replied to you during the weekend, then the SourceForge
> site had some outage of the list(s), thus it was not received. I resent
> my weekend messages only today.
>
I had some troubles, too. My initial mail didn't make it to the mailing
list, it seemed, so I had to resend it again.

Best regards,
F.E.
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
Podofo-users mailing list
Podofo-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/podofo-users

Re: [Podofo-users] Check for preventing signatures from decryption not sufficient

[F. E.]

Ping ... ?

I don't want to annoy anyone, but  I didn't receive any responses, not even
an "I don't know" or "We don't have time for that right now".
Right now I have three pdfs all with the same loading error, and all have
been created with an Adobe/Acrobat application (Adobe PDF Library 10.0.1,
Acrobat Distiller 10.1.12 (Windows), Acrobat Distiller 8.1.0 (Windows)).
Clearly Podofo fails to open some perfectly valid pdf files.

Regards,
F.E.


2018-02-22 9:55 GMT+01:00 F. E. <exler7...@gmail.com>:

> Hello dear podofo users,
>
> today i stumbled upon a parsing bug in PdfTokenizer::ReadDictionary. With
> SVN revision 1794, a check was added to prevent a signature content stream
> from being decrypted by comparing the dictionary type to "Sig". But this
> requires the type to be known/parsed beforehand, otherwise the dictionary
> doesn't contain the type (yet). Consequently, if you have a pdf file where
> the '/Type/Sig' is declared after the signature content, the check does not
> work and the signature content gets decrypted, which in my case ultimately
> prevented the pdf file from being loaded successfully. When I prevented the
> decryption during debugging, the pdf file was loaded successfully.
>
> I consulted the pdf reference for this issue, but couldn't find any
> indication that the dictionary type has to be specified before the
> signature content occurs, my test pdf should be perfectly valid. So podofo
> needs to be fixed. I tried to analyze the code and a possible solution /
> workaround, but I'm not feeling confident & competent enough to come up
> with a good, working patch.
>
> Regards,
> F.E.
>
>
>
>
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
Podofo-users mailing list
Podofo-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/podofo-users

[Podofo-users] Fwd: Check for preventing signatures from decryption not sufficient

[F. E.]

Hello dear podofo users,

today i stumbled upon a parsing bug in PdfTokenizer::ReadDictionary. With
SVN revision 1794, a check was added to prevent a signature content stream
from being decrypted by comparing the dictionary type to "Sig". But this
requires the type to be known/parsed beforehand, otherwise the dictionary
doesn't contain the type (yet). Consequently, if you have a pdf file where
the '/Type/Sig' is declared after the signature content, the check does not
work and the signature content gets decrypted, which in my case ultimately
prevented the pdf file from being loaded successfully. When I prevented the
decryption during debugging, the pdf file was loaded successfully.

I consulted the pdf reference for this issue, but couldn't find any
indication that the dictionary type has to be specified before the
signature content occurs, my test pdf should be perfectly valid. So podofo
needs to be fixed. I tried to analyze the code and a possible solution /
workaround, but I'm not feeling confident & competent enough to come up
with a good, working patch.

Regards,
F.E.
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
Podofo-users mailing list
Podofo-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/podofo-users

[Podofo-users] Regarding current state of AES-256 encryption

[F. E.]

Hello Podofo-Users,

I just want to ask the following for clarification.

As far as I known podofo currently supports AES-256 only with "PDF 1.7
Adobe Extension Level 3", which is not as secure as using AES-128.
See:
http://blogs.adobe.com/security/2008/12/acrobat_9_and_password_encrypt.html

With "PDF 1.7 Adobe Extension Level 8" (I think) a better AES-256
implementation was introduced,  but this is currently not supported in
podofo, correct?

Regards
F.E.
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
Podofo-users mailing list
Podofo-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/podofo-users

[Podofo-users] Removing a PdfArray element safely

[F. E.]

Hallo dear podofo users,


I recently wrote some podofo code to remove signature fields. I inspected
the pdf with podofobrowser and removed all objects and references
step-by-step, that worked nice.
But when I thought the removal is complete, I recognized that the AcroForm
Fields-Array still holds an element, which is null.

So I still need to remove the PdfArray element. The PdfArray has an erase
function, which takes a single iterator or a range. For my use case, there
might me multiple elements to be erased, not necessarily grouped up
together, so no range possible, I've got to erase the elements one-by-one.

Here's my solution so far:

typedef std::list PdfArrayIteratorList;
PdfArrayIteratorList elems_to_delete;

PdfArray::iterator it = pFieldsArray->begin(), end = pFieldsArray->end();
for ( ; it != end; ++it )
{
[..] // omitted, checks if field is of type '/Sig' and removes dicts,
objects and references

// Save array element for subsequent deletion
elems_to_delete.push_back( it );
}

// Remove all null array entries
PdfArrayIteratorList::const_reverse_iterator del_it =
elems_to_delete.rbegin(), del_it_end = elems_to_delete.rend();
for ( ; del_it != del_it_end; ++del_it )
{
pFieldsArray->erase( *del_it ); // Is this safe to do??
}

Erasing during an iteration is likely unsafe, since it could invalidate my
iterators.
So there's no erase in the first iteration, all iterators in
elems_to_delete should still be valid.
In the second iteration, I revers-iterate through elems_to_delete and call
erase on any saved iterator.

I use revers iteration, because I don't know how PdfArray is implemented.
It shouldn't be necessary if it's a linked-list or tree-map, that shouldn't
invalidate my iterators.
But if it's really some sort of array, I would expect some shifting after
an erase. The shifting shouldn't affect my (remaining) iterators when I'm
revers iterating.
But that's all just guessing.

So my question is, what is the right, safe way to erase multiple elements
fron a PdfArray? Is my solution ok so far or are there some pitfalls I
overlooked?


Regards,
F.E.
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
Podofo-users mailing list
Podofo-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/podofo-users

Re: [Podofo-users] Signing encrypted pdfs

[F. E.]

>
> If you know the password, then there is no need to create a new PDF (at
> least in general, not sure of PoDoFo specifics here) – you can just sign
> the encrypted PDF directly *(ensuring that all the new objects are
> properly encrypted as well).  *
>

That's the hard part, isn't it? At least it's not working with Podofo when
I'm doing that straightforward. See case 2.) of my first mail:

2.) Ok, so I changed my signing implementation to not use normal Write(),
> so I don't have to use update mode for loading. When I'm now loading the
> encrypted pdf document, it works and I'm able to place my signature. BUT,
> when I'm opening the file in a pdf viewer, it doesn't ask me for any
> password! I can see that my configured permissions are still there, so this
> part of the security settings is still there, but the password (user/owner)
> got lost.
>


2017-08-04 17:01 GMT+02:00 Leonard Rosenthol <lrose...@adobe.com>:

> If you know the password, then there is no need to create a new PDF (at
> least in general, not sure of PoDoFo specifics here) – you can just sign
> the encrypted PDF directly (ensuring that all the new objects are properly
> encrypted as well).
>
>
>
> Leonard
>
>
>
> *From: *"F. E." <exler7...@gmail.com>
> *Date: *Friday, August 4, 2017 at 9:57 AM
> *To: *Leonard Rosenthol <lrose...@adobe.com>
> *Cc: *"podofo-users@lists.sourceforge.net" <podofo-users@lists.
> sourceforge.net>
>
> *Subject: *Re: [Podofo-users] Signing encrypted pdfs
>
>
>
> >> would it work to append the pages of the encrypted pdf to an empty pdf
>
> >>
>
> > No, because you have to be able to decrypt the pages to combine them.
>
> I should've make clear that I'm assuming I can decrypt it xD.
>
>
>
> So yes, if I don't know the password, I can only add the encrypted pdf as
> an attachment to a new pdf and sign this.
>
> If I know the password, i thought I could use podofo to load the encrypted
> pdf, append its pages to a new empty pdf and sign it afterwards.
>
> Is this wishful thinking or could it work :) ?
>
>
>
> 2017-08-02 15:46 GMT+02:00 Leonard Rosenthol <lrose...@adobe.com>:
>
> > would it work to append the pages of the encrypted pdf to an empty pdf
>
> >
>
> No, because you have to be able to decrypt the pages to combine them.
>
>
>
> (in the FWIW/historical category – there are remnants of attempts to solve
> this still present in the PDF standard, in the form of Crypt Filters, but
> it turns out to just be a problem not worth solving in the long term)
>
>
>
> Leonard
>
>
>
> *From: *"F. E." <exler7...@gmail.com>
> *Date: *Wednesday, August 2, 2017 at 5:53 AM
> *To: *"podofo-users@lists.sourceforge.net" <podofo-users@lists.
> sourceforge.net>
> *Subject: *Re: [Podofo-users] Signing encrypted pdfs
>
>
>
> >Hi,
> >load for update (and using WriteUpdate()) means that you keep the file
> >as is and only add your changes at the end. Having new values
> encrypted
> >and old values not, or using different encryption method, or even
> >trying to use old encryption on the new content, doesn't sound right
> to
> >me.
>
> Ok, that explains my 1.) and the side note.
>
> > It’s not technically possible to sign an encrypted PDF – because there
> is no way to encrypt the new update section
> > with the same key as the previous sections – unless you know what it is
> (and then it’s a bit tricky still).
>
> Convinced me ^^.
>
>
>
> Some follow-up questions I'd like to ask:
>
>- For my case 2.), signing an encrypted pdf with normal loading, I
>learned that the resulting pdf seems to have no password at all.
>How does this happen? Does podofo remove the password encryption
>automatically when writing the signed file or is this some kind of error in
>the viewer?
>- Instead of adding the encrypted pdf as attachment, which is a nice
>idea I didn't think about, would it work to append the pages of the
>encrypted pdf to an empty pdf and set encryption and signature then? I know
>there's an issue with embedded files when appending, but I could extract
>the files and delete them prior to appending and add them later again.
>- I had a third question, but I don't know what it was about anymore
>:-/.
>
>
>
> Regards,
> F.E.
>
>
>
>
>
> 2017-08-02 0:45 GMT+02:00 Leonard Rosenthol via Podofo-users <
> podofo-users@lists.sourceforge.net>:
>
> It’s not technically possible to sign an encrypted PDF – because there is
> no way to encrypt the new upda

Re: [Podofo-users] Signing encrypted pdfs

[F. E.]

>> would it work to append the pages of the encrypted pdf to an empty pdf

>>

> No, because you have to be able to decrypt the pages to combine them.

I should've make clear that I'm assuming I can decrypt it xD.


So yes, if I don't know the password, I can only add the encrypted pdf as
an attachment to a new pdf and sign this.

If I know the password, i thought I could use podofo to load the encrypted
pdf, append its pages to a new empty pdf and sign it afterwards.

Is this wishful thinking or could it work :) ?

2017-08-02 15:46 GMT+02:00 Leonard Rosenthol <lrose...@adobe.com>:

> > would it work to append the pages of the encrypted pdf to an empty pdf
>
> >
>
> No, because you have to be able to decrypt the pages to combine them.
>
>
>
> (in the FWIW/historical category – there are remnants of attempts to solve
> this still present in the PDF standard, in the form of Crypt Filters, but
> it turns out to just be a problem not worth solving in the long term)
>
>
>
> Leonard
>
>
>
> *From: *"F. E." <exler7...@gmail.com>
> *Date: *Wednesday, August 2, 2017 at 5:53 AM
> *To: *"podofo-users@lists.sourceforge.net" <podofo-users@lists.
> sourceforge.net>
> *Subject: *Re: [Podofo-users] Signing encrypted pdfs
>
>
>
> >Hi,
> >load for update (and using WriteUpdate()) means that you keep the file
> >as is and only add your changes at the end. Having new values
> encrypted
> >and old values not, or using different encryption method, or even
> >trying to use old encryption on the new content, doesn't sound right
> to
> >me.
>
> Ok, that explains my 1.) and the side note.
>
> > It’s not technically possible to sign an encrypted PDF – because there
> is no way to encrypt the new update section
> > with the same key as the previous sections – unless you know what it is
> (and then it’s a bit tricky still).
>
> Convinced me ^^.
>
>
>
> Some follow-up questions I'd like to ask:
>
>- For my case 2.), signing an encrypted pdf with normal loading, I
>learned that the resulting pdf seems to have no password at all.
>How does this happen? Does podofo remove the password encryption
>automatically when writing the signed file or is this some kind of error in
>the viewer?
>- Instead of adding the encrypted pdf as attachment, which is a nice
>idea I didn't think about, would it work to append the pages of the
>encrypted pdf to an empty pdf and set encryption and signature then? I know
>there's an issue with embedded files when appending, but I could extract
>the files and delete them prior to appending and add them later again.
>- I had a third question, but I don't know what it was about anymore
>:-/.
>
>
>
> Regards,
> F.E.
>
>
>
>
>
> 2017-08-02 0:45 GMT+02:00 Leonard Rosenthol via Podofo-users <
> podofo-users@lists.sourceforge.net>:
>
> It’s not technically possible to sign an encrypted PDF – because there is
> no way to encrypt the new update section with the same key as the previous
> sections – unless you know what it is (and then it’s a bit tricky still).
>
> Better to simply create a new PDF with the encrypted PDF as an attachment
> as part of a collection and then sign the new PDF.
>
> Leonard
>
> On 8/1/17, 11:27 AM, "zyx" <z...@litepdf.cz> wrote:
>
> On Tue, 2017-08-01 at 16:38 +0200, F. E. wrote:
> > As a side note, I noticed that I cannot use load-for-update in this
> > case, because the resulting pdf has an unknown password protection.
> > Neither my specified user nor owner password work for decryption.
> > That's odd and mighht be a bug, don't know ...
>
> Hi,
> load for update (and using WriteUpdate()) means that you keep the file
> as is and only add your changes at the end. Having new values encrypted
> and old values not, or using different encryption method, or even
> trying to use old encryption on the new content, doesn't sound right to
> me.
>
> Doing sign & encrypt at the same time sounds the way to go, as you
> figured out yourself.
> Bye,
> zyx
>
> --
> https://na01.safelinks.protection.outlook.com/?url=
> http%3A%2F%2Fwww.litePDF.cz=02%7C01%7C%7C08f72d27a1eb4526088408d4d8f1
> c292%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%
> 7C636371980250993292=9YEy0XwyP2Aioj0EKQmb6J5xexlziV
> QRc3O3KnSU61U%3D=0
>  i...@litepdf.cz
>
> 
> --
> Check out the vibrant tech community on one of the world's most
> engagin

Re: [Podofo-users] Signing encrypted pdfs

[F. E.]

>Hi,
>load for update (and using WriteUpdate()) means that you keep the file
>as is and only add your changes at the end. Having new values encrypted
>and old values not, or using different encryption method, or even
>trying to use old encryption on the new content, doesn't sound right to
>me.

Ok, that explains my 1.) and the side note.

> It’s not technically possible to sign an encrypted PDF – because there is
no way to encrypt the new update section
> with the same key as the previous sections – unless you know what it is
(and then it’s a bit tricky still).

Convinced me ^^.

Some follow-up questions I'd like to ask:

   - For my case 2.), signing an encrypted pdf with normal loading, I
   learned that the resulting pdf seems to have no password at all.
   How does this happen? Does podofo remove the password encryption
   automatically when writing the signed file or is this some kind of error in
   the viewer?

   - Instead of adding the encrypted pdf as attachment, which is a nice
   idea I didn't think about, would it work to append the pages of the
   encrypted pdf to an empty pdf and set encryption and signature then? I know
   there's an issue with embedded files when appending, but I could extract
   the files and delete them prior to appending and add them later again.

   - I had a third question, but I don't know what it was about anymore :-/.


Regards,
F.E.



2017-08-02 0:45 GMT+02:00 Leonard Rosenthol via Podofo-users <
podofo-users@lists.sourceforge.net>:

> It’s not technically possible to sign an encrypted PDF – because there is
> no way to encrypt the new update section with the same key as the previous
> sections – unless you know what it is (and then it’s a bit tricky still).
>
> Better to simply create a new PDF with the encrypted PDF as an attachment
> as part of a collection and then sign the new PDF.
>
> Leonard
>
> On 8/1/17, 11:27 AM, "zyx" <z...@litepdf.cz> wrote:
>
> On Tue, 2017-08-01 at 16:38 +0200, F. E. wrote:
> > As a side note, I noticed that I cannot use load-for-update in this
> > case, because the resulting pdf has an unknown password protection.
> > Neither my specified user nor owner password work for decryption.
> > That's odd and mighht be a bug, don't know ...
>
> Hi,
> load for update (and using WriteUpdate()) means that you keep the file
> as is and only add your changes at the end. Having new values encrypted
> and old values not, or using different encryption method, or even
> trying to use old encryption on the new content, doesn't sound right to
> me.
>
> Doing sign & encrypt at the same time sounds the way to go, as you
> figured out yourself.
> Bye,
> zyx
>
> --
> https://na01.safelinks.protection.outlook.com/?url=
> http%3A%2F%2Fwww.litePDF.cz=02%7C01%7C%7C08f72d27a1eb4526088408d4d8f1
> c292%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%
> 7C636371980250993292=9YEy0XwyP2Aioj0EKQmb6J5xexlziV
> QRc3O3KnSU61U%3D=0
>  i...@litepdf.cz
>
> 
> --
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! https://na01.safelinks.
> protection.outlook.com/?url=http%3A%2F%2Fsdm.link%
> 2Fslashdot=02%7C01%7C%7C08f72d27a1eb4526088408d4d8f1c292%
> 7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C636371980250993292=z%
> 2B6nJU9jeBLfcT6Zf6fJwhOYw5JEZizBmLHvYCnsHdQ%3D=0
> ___
> Podofo-users mailing list
> Podofo-users@lists.sourceforge.net
> https://na01.safelinks.protection.outlook.com/?url=
> https%3A%2F%2Flists.sourceforge.net%2Flists%2Flistinfo%2Fpodofo-users&
> data=02%7C01%7C%7C08f72d27a1eb4526088408d4d8f1c292%
> 7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C636371980250993292=
> sWzuGkB3yg7Z3U%2F23MWWWkQUl9U9Me8A8XsD8bXOm8w%3D=0
>
>
> 
> --
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> ___
> Podofo-users mailing list
> Podofo-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/podofo-users
>
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
Podofo-users mailing list
Podofo-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/podofo-users

Re: [Podofo-users] Signing encrypted pdfs

[F. E.]

Hello again,


some follow-up:

When loading an unencrypted pdf, I can encrypt and sign it in one go. This
works, Password and Permissions are present and the pdf has a valid
signature.
Since I'm in control of the encryption as well for my use case, that's a
solution for me. But it's no solution for the general case of signing an
encrypted pdf.

As a side note, I noticed that I cannot use load-for-update in this case,
because the resulting pdf has an unknown password protection. Neither my
specified user nor owner password work for decryption. That's odd and
mighht be a bug, don't know ...

Regards,
F.E.




2017-08-01 15:00 GMT+02:00 F. E. <exler7...@gmail.com>:

> Hello dear podofo users,
>
>
> I want to sign an already encrypted pdf using podofo, but I don't get it
> to work:
>
> 1.) My existing solution for signing a pdf document utilizes loading the
> pdf in update mode with subsequent WriteUpdate. This does not work, when
> I'm trying to load the encrypted pdf document, I get
> 'ePdfError_CannotEncryptedForUpdate' as error.
>
> 2.) Ok, so I changed my signing implementation to not use normal Write(),
> so I don't have to use update mode for loading. When I'm now loading the
> encrypted pdf document, it works and I'm able to place my signature. BUT,
> when I'm opening the file in a pdf viewer, it doesn't ask me for any
> password! I can see that my configured permissions are still there, so this
> part of the security settings is still there, but the password (user/owner)
> got lost.
>
> 3.) Ok, so I thought, maybe I've got to set the security options again
> after loading the pdf. Did that, but when performing the write operation
> for signing, I got a cryptic runtime error (read access violation):
>
> The error is thrown in 'base\PdfParserObj.cpp', Line 346:
> PdfInputStream* pInput = m_pEncrypt->CreateEncryptionInputStream( 
> );
>
> The error states: "this->m_pEncrypt-> was 0xFFDF"
>
> I don't quite understand, how this can even happen. In the line above, the
> ->Operator is used on
> m_pEncrypt as well, and there's no problem with it. I can't step into the
> function, the error comes right away. But m_pEncrypt is still valid, says
> the debugger.
>
> Maybe I just can't set security options again?
>
>
> And now I'm out of ideas. Does anyone have a hint for me?
>
> Best regards,
> F.E.
>
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
Podofo-users mailing list
Podofo-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/podofo-users

[Podofo-users] Signing encrypted pdfs

[F. E.]

Hello dear podofo users,


I want to sign an already encrypted pdf using podofo, but I don't get it to
work:

1.) My existing solution for signing a pdf document utilizes loading the
pdf in update mode with subsequent WriteUpdate. This does not work, when
I'm trying to load the encrypted pdf document, I get
'ePdfError_CannotEncryptedForUpdate' as error.

2.) Ok, so I changed my signing implementation to not use normal Write(),
so I don't have to use update mode for loading. When I'm now loading the
encrypted pdf document, it works and I'm able to place my signature. BUT,
when I'm opening the file in a pdf viewer, it doesn't ask me for any
password! I can see that my configured permissions are still there, so this
part of the security settings is still there, but the password (user/owner)
got lost.

3.) Ok, so I thought, maybe I've got to set the security options again
after loading the pdf. Did that, but when performing the write operation
for signing, I got a cryptic runtime error (read access violation):

The error is thrown in 'base\PdfParserObj.cpp', Line 346:
PdfInputStream* pInput = m_pEncrypt->CreateEncryptionInputStream(  );

The error states: "this->m_pEncrypt-> was 0xFFDF"

I don't quite understand, how this can even happen. In the line above, the
->Operator is used on
m_pEncrypt as well, and there's no problem with it. I can't step into the
function, the error comes right away. But m_pEncrypt is still valid, says
the debugger.

Maybe I just can't set security options again?


And now I'm out of ideas. Does anyone have a hint for me?

Best regards,
F.E.
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
Podofo-users mailing list
Podofo-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/podofo-users

Re: [Podofo-users] Removing Embedded Attachments and File Annotations

[F. E.]

I gave the references a try every now and then and half of the time it
confused me more than it helped >.<. It's quite some heavy lecture. But now
I ordered Leonards book, hoping it helps me understanding ^^.

"By reading the specification you get all the needed information, thus also
the answers for all your questions."
I agree that reading the specification can tell me if I missed something to
be removed. But that does only count for PDF related fields and objects. I
cannot fathom whether there are some Podofo specific objects which have to
be removed as well. Furthermore I'm still not quite sure if I can just
bluntly call the Clear() functions to get the corresponding PDF elements
removed. It looks like it works, the file size is decreasing, but I don't
know if there are any pitfalls, e.g. wrong order of clears, certain objects
not to cleared and so on. These are podofo-specific questions I'm asking ;).

F.E.

2017-06-20 13:27 GMT+02:00 zyx <z...@litepdf.cz>:

> On Tue, 2017-06-20 at 12:13 +0200, F. E. wrote:
> > Do you have some example pdf files where my removal code should fail
> > in a way?
>
> Hi,
> what about reading the specifications Leonard referenced, instead of
> trying with random PDF files, which usually tends to fail at the end
> anyway? By reading the specification you get all the needed
> information, thus also the answers for all your questions.
> Bye,
> zyx
>
> --
> http://www.litePDF.cz i...@litepdf.cz
>
> 
> --
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> ___
> Podofo-users mailing list
> Podofo-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/podofo-users
>
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
Podofo-users mailing list
Podofo-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/podofo-users

Re: [Podofo-users] Removing Embedded Attachments and File Annotations

[F. E.]

Let me clarify again my use case and my solution so far:

I want to remove all files listed in the attachment list when I view the
pdf file with a pdf viewer. I encountered two different types of
attachments, embedded files located at document level, and file annotations
located on pages.

I started with removing the file annotations by using
"pPage->DeleteAnnotation( a );". That removed the attachment from the
viewers attachment list and from the page, but consulting the pdf file size
I recognized there's still the files data hidden inside the pdf. So I took
some File Annotation extraction code and bluntly added various Clear()
calls. That decreased the pdf file size roughly by the size of the removed
File Annotations. I did the same for embedded files, took some extraction
code and added Clear() calls, taking care that both attachment reference
and actual data is removed.

My question concerning the completeness / correctness of my removal code:

It seems my request was ambiguous. I originally only wanted to know whether
I missed some objects to be removed (for the given place) and if it's safe
to just call clear() on objects and dictionaries. I didn't know that there
might be more potential places for embedded attachments, which my removal
code cannot handle. If it just ignores this places, that shoulb be ok and
recognizable, if my code only deletes reference and leaves the filestreams,
that would be worse. I'll try to handle it later on. Do you have some
example pdf files where my removal code should fail in a way?

Best regards,
F.E.


2017-06-19 18:14 GMT+02:00 Leonard Rosenthol <lrose...@adobe.com>:

> As far as PDF 1.7 (ISO 32000-1) is concerned, that will handle both of the
> defined locations for embedded files – though not necessary other places
> that embedded filestreams can be used.
>
>
>
> For PDF 2.0 (ISO 32000-2), there are many more potential places that
> embedded file streams can exist.
>
>
>
> Leonard
>
>
>
> *From: *"F. E." <exler7...@gmail.com>
> *Date: *Sunday, June 18, 2017 at 12:51 PM
> *To: *"podofo-users@lists.sourceforge.net" <podofo-users@lists.
> sourceforge.net>
> *Subject: *[Podofo-users] Removing Embedded Attachments and File
> Annotations
>
>
>
> Hello Podofo Users,
>
> I'm using Podofo for removing Embedded Files and File Annotation. My code
> works fine so far, no files can be seen using a PDF Viewer, and I even
> managed to remove the dependent filestreams (otherwise files are still
> "hidden" inside the pdf and take space).
> What I'm not sure about is, whether my removal code is complete (removing
> all data) and safe (e.g. no double frees). So I'd like to commit my code
> here and ask whether you think the code is correct.
>
> On removing Embedded Files:
>
> There's code somewhere on how to extract embedded files. I refactored this
> code and added Clear instructions whenever I thought them necessary for the
> task.
>
>
> // Check if given ptr is a reference, if so, resolve ptr reference and
> store previous ptr to ref_ptr
> template 
> void CheckReference( const PdfMemDocument* doc, T* , T* _ptr )
> {
> if ( IS_NULL( ptr ) || !ptr->IsReference() ) { return; }
>
> ref_ptr = ptr;
> ptr = doc->GetObjects().GetObject( ref_ptr->GetReference() );
> }
>
> template 
> void Clear( T* ptr )
> {
> if ( ptr ) { ptr->Clear(); }
> }
>
> void RemoveEmbeddedFileData( PdfObject *pObj ) const
> {
> PdfObject *pRefObj = NULL;
> CheckReference( m_pDocument, pObj, pRefObj );
>
> if ( !IS_NULL( pObj ) && pObj->GetDataType() ==
> ePdfDataType_Dictionary )
> {
> PdfDictionary  = pObj->GetDictionary();
> if ( outerDict.HasKey("EF") )
> {
> PdfDictionary  = outerDict.GetKey("EF")->
> GetDictionary();
> if ( innerDict.HasKey("F") )
> {
> PdfObject *pStreamObj = innerDict.GetKey("F"),
> *pStreamRefObj = NULL;
> CheckReference( m_pDocument, pStreamObj, pStreamRefObj
> );
> Clear( pStreamObj );
> Clear( pStreamRefObj );
> }
> innerDict.Clear();
> }
> outerDict.Clear();
> }
>
> Clear( pObj );
> Clear( pRefObj );
> }
>
> void RemoveEmbeddedFiles()
> {
> PdfObject *pEmbFilesObj = NULL;
> {
> const PdfNamesTree *pNamesTree = m_pDocument->GetNamesTree(
> true );
> RETURN_ON_NULL( pNamesTr

[Podofo-users] Appending PDF: Embedded files of appended file not accessible in result pdf

[F. E.]

Hello dear podofo users,

recently I experimented with merging multiple pdf files into one pdf using
the Append function of PdfDocument. It works like a charm with the pdf
pages and annotations, but I recognised that attached files (at document
root) of appended pdfs are not listed in the attachment table of my pdf
viewer (Foxit 8.0, didn't check other viewers I've got to admit). When
checking the size of my result pdf, it was clear that the embedded files
are still there somewhere, but I cannot access them it seems.

Is this the expected behaviour or did I stumble over a bug / missing
feature here? I think there might be a way to recreate the necessary
internal references, but I'm lacking the knowledge to do so. I could
circumvent this issue by manually extracting the embedded files (I know how
to do that) and adding them to the result pdf (trivial via AttachFile
function), but I don't know how to remove the embeeded files before
appending the pdf file.

Thanks in advance,
Frederik
--
Announcing the Oxford Dictionaries API! The API offers world-renowned
dictionary content that is easy and intuitive to access. Sign up for an
account today to start using our lexical data to power your apps and
projects. Get started today and enter our developer competition.
http://sdm.link/oxford___
Podofo-users mailing list
Podofo-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/podofo-users

Re: [Podofo-users] Problem with loading password protected PDF

[F. E.]

Hi,

we created a patch ourselves, I'll just attach it and hope it fits :).

Best regards
F.E.


2016-05-10 20:57 GMT+02:00 zyx <z...@litepdf.cz>:

> On Tue, 2016-05-10 at 18:07 +0200, F. E. wrote:
> > Is there a reason the INCLUDE_DIRECTORIES are not set for OpenSSL or
> > is it just a bug?
>
> Hi,
> I believe it's just a bug. Could you share your change in a form of a
> patch, please? (If you are not having a svn checkout, then have a file
> before your change, then after and generate the patch using diff
> command:
>$ diff -up file.before file.after >change.patch
> There are available windows executables for the diff too. The "-up"
> parameter tells the diff to create a unified diff and reference the
> method it is diffing in (usually in C/C++ code it's that method, even
> not always).
> Thanks and bye,
> Milan
> --
> http://www.litePDF.cz i...@litepdf.cz
>
>
> --
> Mobile security can be enabling, not merely restricting. Employees who
> bring their own devices (BYOD) to work are irked by the imposition of MDM
> restrictions. Mobile Device Manager Plus allows you to control only the
> apps on BYO-devices by containerizing them, leaving personal data
> untouched!
> https://ad.doubleclick.net/ddm/clk/304595813;131938128;j
> ___
> Podofo-users mailing list
> Podofo-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/podofo-users
>


podofo_cmakelists_add_openssl_include.patch
Description: Binary data
--
Mobile security can be enabling, not merely restricting. Employees who
bring their own devices (BYOD) to work are irked by the imposition of MDM
restrictions. Mobile Device Manager Plus allows you to control only the
apps on BYO-devices by containerizing them, leaving personal data untouched!
https://ad.doubleclick.net/ddm/clk/304595813;131938128;j___
Podofo-users mailing list
Podofo-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/podofo-users

Re: [Podofo-users] Problem with loading password protected PDF

[F. E.]

Hello again,

Thanks for the last answer.

I tried running the encrypt.cpp, but encountered compile, link and runtime
errors (bad alloc). With the help of a collegue we started from scratch and
rebuilt podofo and all dependent libraries. Now it's working, my example
code as well as encrypt.cpp.
When using CMake to generate the vs project files, however, we encountered
an minor issue with the generation. Although we specified everything needed
for OpenSSL (-DCMAKE_INCLUDE_PATH / -DCMAKE_LIBRARY_PATH /
-DLIBCRYPTO_LIBRARY_NAMES_DEBUG / -DLIBCRYPTO_LIBRARY_NAMES_RELEASE) and
OpenSSL got enabled in podofo_config.h, the include directories were not
added to the projects. That resulted in a compile error (missing header in
PDFEncrypt.cpp). We checked CMakeLists.txt and found that there was no
INCLUDE_DIRECTORIES call for the OpenSSl include dir (line 322ff.), unlike
with other libraries like libjpeg. We added that call and it generated the
project files correctly. But we're not sure if that solution is correct.

Is there a reason the INCLUDE_DIRECTORIES are not set for OpenSSL or is it
just a bug?

Best regards,
F.E.


2016-05-04 19:52 GMT+02:00 zyx :

> On Wed, 2016-05-04 at 09:45 +0200, Max Müller wrote:
> > when playing around with Podofo I stumbled upon an issue that I'm not
> > able to load password protected PDFs.
>
> Hi,
> could you try to compile and run the encrypt.cpp attached at
> https://sourceforge.net/p/podofo/mailman/message/34065834/
> please? That works for me fine. It uses the same method (way of loading
> the files it created) as you outlined here. There might also be related
> whether you compiled with the openssl and the encryption/decryption
> method the PDF file is encrypted supported.
>
> With respect of PDF file sharing, if you can make a very simple and
> small example PDF which fails for you, with no confidential data, then
> there's no problem to send it to this mailing list, I believe.
> Bye,
> zyx
>
> --
> http://www.litePDF.cz i...@litepdf.cz
>
>
> --
> Find and fix application performance issues faster with Applications
> Manager
> Applications Manager provides deep performance insights into multiple
> tiers of
> your business applications. It resolves application problems quickly and
> reduces your MTTR. Get your free trial!
> https://ad.doubleclick.net/ddm/clk/302982198;130105516;z
> ___
> Podofo-users mailing list
> Podofo-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/podofo-users
>
--
Mobile security can be enabling, not merely restricting. Employees who
bring their own devices (BYOD) to work are irked by the imposition of MDM
restrictions. Mobile Device Manager Plus allows you to control only the
apps on BYO-devices by containerizing them, leaving personal data untouched!
https://ad.doubleclick.net/ddm/clk/304595813;131938128;j___
Podofo-users mailing list
Podofo-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/podofo-users