lxy-9602 opened a new pull request, #238: URL: https://github.com/apache/paimon-cpp/pull/238
<!-- PR titles must follow Conventional Commits: <type>(<optional-scope>): <description> --> ### Purpose <!-- Linking this pull request to the issue --> Linked issue: #236 <!-- What is the purpose of the change --> Add initial Apache Paimon Mosaic file format support through the Mosaic 0.2.0 Rust FFI. This change: - adds optional Mosaic build integration guarded by `PAIMON_ENABLE_MOSAIC`, which is disabled by default; - downloads and builds the Mosaic Rust FFI through `ThirdpartyToolchain.cmake`; - registers the Mosaic file format factory; - supports reading and writing through the Arrow C Data Interface; - supports projection and configurable read batch sizes; - supports Zstandard and uncompressed Mosaic files; - supports `file.compression.zstd-level`, `file.block-size`, and `mosaic.num-buckets`; - applies the Java-compatible `file.block-size` option to Mosaic row groups, ORC stripes, and Parquet row groups; - makes `file.block-size` override the corresponding format-specific ORC and Parquet options, matching Java behavior; - uses the Paimon-backed Arrow memory pool for Arrow-side allocations and offset normalization. The initial implementation does not support prefetch, statistics extraction, predicate or selection-bitmap pushdown, or writer metadata. ### Tests Added Mosaic write/read tests covering: - multiple read batch sizes; - `file.block-size` and `mosaic.num-buckets`; - boolean, integer, floating-point, string, binary, date, decimal, and timestamp types; - millisecond, microsecond, and nanosecond timestamps, with and without timezone; - list, map, nested list, list-of-map, and map-of-list types; - explicit rejection of unsupported second-precision timestamps and generic struct types. The tests currently generate Mosaic files in C++ and do not include Java-generated interoperability fixtures. <!-- List UT and IT cases to verify this change --> ### API and Format This change adds the public `Options::FILE_BLOCK_SIZE` option. It adds support for reading and writing the existing Apache Paimon Mosaic file format through the upstream Mosaic Rust FFI. It does not modify the Mosaic binary format. For ORC and Parquet, this change only adds Java-compatible mapping of file.block-size to the existing stripe and row-group size writer settings. <!-- Does this change affect API in include dir or storage format or protocol --> ### Documentation <!-- Does this change introduce a new feature --> ### Generative AI tooling Generated-by: OpenAI Codex (GPT-5) <!-- If generative AI tooling has been used in the process of authoring this patch, please include the phrase: 'Generated-by: ' followed by the name of the tool and its version. If no, write 'No'. Please refer to the [ASF Generative Tooling Guidance](https://www.apache.org/legal/generative-tooling.html) for details. --> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
