yihua opened a new pull request, #19254: URL: https://github.com/apache/hudi/pull/19254
### Describe the issue this Pull Request addresses The Lance and Vortex base writers duplicate ~85-90% of their structure (Arrow buffering, dual flush triggers, allocator lifecycle, bloom filter support, empty-file close handling): 362 and 345 lines that must be maintained in lockstep. This PR extracts the shared boilerplate so the next Arrow-native format only implements its FFI-specific hooks. Writer-side only; reader-side extraction is a follow-up. ### Summary and Changelog - Add abstract `HoodieBaseArrowWriter` (`org.apache.hudi.io.arrow`, hudi-hadoop-common, ungated): owns the Arrow child-allocator lifecycle, `VectorSchemaRoot` management, row buffering, the dual flush trigger (row-count batch size or byte watermark), bloom-filter and min/max key support, `getDataSize`, empty-file-on-close handling, and close ordering with exception suppression. Imports only Arrow and Hudi types (no `dev.vortex`, no Lance native types), so it compiles on the Java 8 target. - `HoodieBaseLanceWriter` shrinks 362 to 161 lines and `HoodieBaseVortexWriter` 345 to 184, each keeping only format-specific hooks (Lance native writer calls; Vortex C Data Interface export and the dedicated native allocator with its tolerated one-time schema-export retention). The Vortex writer stays in the JDK-17-gated source root. - Format hooks: `initializeFormatWriter`, `writeBatch`, `closeFormatWriter`, `getFormatName`, `isFormatWriterInitialized`, plus optional `finalizeFooterMetadata` (Lance footer), `closeFormatResources`, and `closeAfterAllocator` (Vortex native allocator). - No public or protected API changes: constructors, `addFooterMetadata`, `getWrittenRecordCount`, `getDataSize` are intact, the Spark/Flink subclasses are unmodified, and the reflective `HoodieSparkVortexWriter.newWriter` factory contract is untouched. ### Impact Behavior-preserving deduplication of the Arrow writer path. Validated: Lance tests (TestLanceDataSource 61, TestHoodieSparkLanceWriter 23, TestHoodieSparkLanceReader 16) and the Vortex round-trip (2) all pass on JDK 17; the Java 11 Spark chain and Flink client compile; checkstyle clean. ### Risk Level medium: touches the write path of two formats, mitigated by the full Lance and Vortex test runs above and an adversarial equivalence review of flush triggers, close ordering, and allocator lifecycle. ### Documentation Update none ### Contributor's checklist - [x] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute) - [x] Enough context is provided in the sections above - [x] Adequate tests were added if applicable -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
