Davis-Zhang-Onehouse opened a new pull request, #20711:
URL: https://github.com/apache/hudi/pull/20711

   ### Describe the issue this Pull Request addresses
   
   issue: #20710
   Part of #20683, RFC-118 (#20684).
   Stacked on #20709: review only the last commit. Until the PRs below it 
merge, this PR also shows their commits.
   
   Extraction limits and isolation for PDF decoding: budgets for decoded page 
contents, fonts, content-string bodies and graphics depth, a structure scanner 
that refuses undecodable or oversized streams before PDFBox decodes them, CMap 
and Type 1 font checks, bounded scratch disk, and failure codes that record 
which limit a file hit.
   
   ### Summary and Changelog
   
   - Decode guard in front of PDFBox: a stream structure scanner (numeric 
tokens, inline images, CMap range sections), budgets for decoded page contents, 
page fonts, content-string bodies and graphics nesting depth, and refusal of 
undecodable content streams.
   - Bounded scratch disk for PDF decoding and handle release on every exit 
path.
   - `PdfFailureCode`: each refused or failed file records which limit it hit; 
it lands in the error table instead of failing the round.
   - 21 test classes (17 new), with small PDF fixtures built in the tests.
   
   Only files added earlier in this chain are modified. No code was copied from 
other projects.
   
   ### Impact
   
   PDF preparation refuses files that exceed its decode budgets and records 
why, instead of using unbounded memory or disk. No change outside the embedding 
stream.
   
   ### Risk Level
   
   low: new limits in the PDF path of the new source only. Verified on JDK 11 / 
Spark 3.5 / Scala 2.12 with 346 tests (none skipped), module compilation, 
checkstyle and RAT.
   
   ### Documentation Update
   
   New PDF limits carry their config documentation; a website page follows once 
the chain lands.
   
   ### Contributor's checklist
   
   - [x] Read through [contributor's 
guide](https://hudi.apache.org/contribute/how-to-contribute)
   - [x] Enough context is provided in the sections above
   - [x] Adequate tests were added if applicable
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

  • [PR] feat(ingest): RFC-118: ... Davis-Zhang-Onehouse (via GitHub) via commits

Reply via email to