rahil-c opened a new pull request, #19678: URL: https://github.com/apache/hudi/pull/19678
### Describe the issue this Pull Request addresses Closes #19676 Stacked on #19677, review that one first. Until it merges this PR's diff includes its commits. ### Summary and Changelog `tika-parsers-standard-package` now ships in `hudi-utilities-bundle`, so unstructured ingestion extracts text with no runtime flags. Previously only `tika-core` shipped, and `tika-core` registers no parser at all, so every file produced `parse_status=EMPTY`, empty text, no chunks and a null vector while the job exited successfully. commons-compress is relocated, which makes Tika's references follow the bundled copy rather than resolving whatever the runtime provides. It is pinned to 1.27.1 because a transitive 1.9 otherwise wins and predates the `ArchiveStreamFactory.detect(InputStream)` that Tika's zip container detection calls. Bouncycastle's multi-release classes are filtered out, being built for JDKs newer than the shade plugin can parse while the base classes are what runs. `TikaDocumentParser` now parses a small in-memory plain-text document on `init` and fails when no text comes back. `init` runs only when parsing is enabled, so reaching it means text extraction was asked for, and a deployment that can extract none is a misconfiguration rather than a per-file problem. Blob-only ingestion remains available through `parse.enabled=false`. The slim bundle is deliberately left alone. Its users get the new error naming the dependency to add. ### Impact `hudi-utilities-bundle` grows from 90 MB to 141 MB. Users of that bundle who enable parsing get working text extraction where they previously got silently empty results; users who want blobs only should set `hoodie.streamer.source.unstructured.parse.enabled=false`, which also skips the probe. ### Risk Level medium The bundle gains around 85 artifacts, so the risk is dependency collision rather than logic. Mitigated by keeping jackson, slf4j and commons-io out of the added set and letting the runtime provide them, and by relocating commons-compress so the one known conflict cannot recur. Verified by ingesting a 15,774-file corpus through HoodieStreamer with the bundle alone, no `--packages`, no `--jars` and no `extraClassPath`: SUCCESS 4988, EMPTY 5011 matching exactly the binary files, FAILED 1, and no `NoSuchMethodError`. ### Documentation Update The website should state that document parsing works out of the box with the utilities bundle, and that the slim bundle needs `org.apache.tika:tika-parsers-standard-package` supplied separately. Happy to follow up with that change once the approach here is settled. ### Contributor's checklist - [x] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute) - [x] Enough context is provided in the sections above - [x] Adequate tests were added if applicable -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
