JunRuiLee opened a new pull request, #412:
URL: https://github.com/apache/paimon-rust/pull/412
**PR 2** of reading Paimon `VECTOR` columns in paimon-rust, tracked in #410.
Enables reading vector columns stored inline in ordinary parquet data files.
> Stacked on **#411 (PR 1)** — depends on the `VectorType` /
`DataType::Vector` introduced there. Review/merge after #411. The diff shown
here includes #411's commits until it merges.
Roadmap (#410): PR 1 type system (#411) → **PR 2 (this)** Arrow conversion +
parquet inline read → PR 3 dedicated `.vector.` files.
## Changes
- `VectorType` ↔ Arrow `FixedSizeList` conversion in `arrow/mod.rs` (both
directions), replacing PR 1's `Unsupported`: forward `Vector` →
`FixedSizeList(Field("element", elem, element_nullable), length)`; reverse
guards non-positive sizes with `u32::try_from` and routes through
`VectorType::try_new` (rejecting length 0 and invalid element types).
- The read path builds the target Arrow schema from `paimon_type_to_arrow`,
so an inline `FixedSizeList` vector column now materializes directly.
Unchanged boundaries (vectors still not filter-pushable / not cast / not
partition or BTree keys / no `binary_row` datum); writing and dedicated
`.vector.` files remain out of scope.
## Testing
`cargo test -p paimon --lib` 901 passed; `cargo fmt --all -- --check` clean;
`cargo clippy -p paimon --all-targets --features fulltext,vortex,mosaic -- -D
warnings` clean. Adds 5 Arrow conversion unit tests + an end-to-end parquet
test reading a `FixedSizeList<Float32>` column with a null row (downcast
asserts `value_length`, child values, null bitmap).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
