Martin Mirchev created PDFBOX-6316:
--------------------------------------

             Summary: ArrayIndexOutOfBoundsException when an RC4 encryption 
dictionary has a negative /Length
                 Key: PDFBOX-6316
                 URL: https://issues.apache.org/jira/browse/PDFBOX-6316
             Project: PDFBox
          Issue Type: Bug
          Components: Crypto
    Affects Versions: 4.0.0, 3.0.8 PDFBox
            Reporter: Martin Mirchev
         Attachments: rc4-length-negative.pdf

{{StandardSecurityHandler}} uses {{/Length / 8}} as the key length without 
checking it, so a negative
{{/Length}} reaches {{MessageDigest.update(digest, 0, length)}} in 
{{computeRC4key()}} and an unchecked
exception escapes {{Loader.loadPDF()}} instead of an {{IOException}}. Attached 
{{rc4-length-negative.pdf}} is a
PDFBox-saved RC4-128 file ({{StandardProtectionPolicy}}, owner password 
"owner", empty user password)
with {{/Length 128}} changed to {{/Length -64}}:

{code}
java.lang.ArrayIndexOutOfBoundsException
  at 
java.base/sun.security.provider.DigestBase.engineUpdate(DigestBase.java:109)
  ...
  at 
org.apache.pdfbox.pdmodel.encryption.StandardSecurityHandler.computeRC4key(StandardSecurityHandler.java:963)
  at 
org.apache.pdfbox.pdmodel.encryption.StandardSecurityHandler.getUserPassword234(StandardSecurityHandler.java:678)
  at 
org.apache.pdfbox.pdmodel.encryption.StandardSecurityHandler.isOwnerPassword234(StandardSecurityHandler.java:615)
  at 
org.apache.pdfbox.pdmodel.encryption.StandardSecurityHandler.prepareForDecryption(StandardSecurityHandler.java:251)
{code}

The unmodified file loads and renders. A too-large value ({{/Length 999}}) is 
already reported as an
{{IOException}} ("Input buffer too short"). Rejecting a /Length outside 40..128 
(or < 40 bits) for
R2-R4 with an {{IOException}} would cover both.

This issue was found by an LLM-assisted tool and verified manually.




--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to