zjw1111 opened a new pull request, #210: URL: https://github.com/apache/paimon-cpp/pull/210
### Purpose Linked issue: N/A Add the File Index write path and initially support Bitmap and Range Bitmap indexes. - Add the `FileIndexFormat::Writer` API and Java-compatible V1 index serialization. - Build per-column indexes while writing append-only, key-value, and shredding data files. - Embed small indexes in `DataFileMeta` and publish larger indexes as extra `.index` files. - Share data-writer completion, metadata finalization, index publication, and abort bookkeeping through `DataFileWriterBase`. - Add a pool-backed `ByteArrayOutputStream` on top of `MemorySegmentOutputStream`. - Parse File Index options from core options and expose the manifest threshold through `CoreOptions`. Bloom Filter and BSI writers, plus the cleanup and validation follow-ups found during review, are intentionally deferred and recorded in `file-index-write-interface-design.md`. ### Tests - `cmake --build build --target unittest -j "$(nproc)"` built all unit and integration test targets. The initial run passed 26 of 27 test executables and exposed a null-buffer validation issue in the new `ByteArrayOutputStream` test; the issue was fixed. - `./build/debug/paimon-common-test` passed all 1439 tests after the fix. - `./build/debug/paimon-core-test --gtest_filter='DataFileIndexWriterTest.*:CoreOptionsTest.*:AppendOnlyWriterTest.*'` passed all 56 selected tests after relinking the final library. - `pre-commit run --from-ref upstream/main --to-ref HEAD` passed. - `git diff --check upstream/main...HEAD` passed. ### API and Format Yes. This adds the public `FileIndexFormat::Writer` API and writes the Java-compatible V1 File Index container format. Existing read formats and existing public APIs are not changed incompatibly. ### Documentation Added `file-index-write-interface-design.md`, including the current design and prioritized follow-up work for later PRs. ### Generative AI tooling Generated-by: OpenAI Codex (GPT-5) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
