liujiwen-up opened a new pull request, #321:
URL: https://github.com/apache/paimon-rust/pull/321
### Purpose
Linked issue: close #xxx
Add a minimal DataFusion SQL end-to-end test for Vortex file format tables.
This verifies that `paimon-datafusion` can enable the core `paimon/vortex`
feature, create a table with `file.format = 'vortex'`, insert rows through SQL,
and read them back correctly.
### Brief change log
- Add `vortex = ["paimon/vortex"]` feature passthrough to
`paimon-datafusion`.
- Add a feature-gated DataFusion integration test for:
- `CREATE TABLE ... WITH ('file.format' = 'vortex')`
- `INSERT INTO`
- `SELECT ... ORDER BY`
- simple `WHERE id = 2` query
- Assert that the SQL flow writes an actual `.vortex` data file.
- Make Vortex reader convert arrays using the target Arrow schema instead of
the preferred Arrow representation, so string columns are returned as
schema-compatible `Utf8`.
- Add a core-level Vortex reader test covering `STRING` / `Utf8` schema
compatibility.
### Tests
- `cargo fmt --all -- --check`
- `cargo check -p paimon-datafusion --features vortex`
- `cargo test -p paimon-datafusion --test vortex_tables --features vortex
test_vortex_file_format_sql_e2e`
- `cargo test -p paimon --features vortex
test_vortex_reader_returns_utf8_for_string_schema`
- `cargo test -p paimon --features vortex arrow::format::vortex`
### API and Format
No public API change.
No storage format change. This only exposes the existing optional Vortex
support through the DataFusion integration feature and fixes Arrow schema
compatibility when reading Vortex data.
### Documentation
No documentation update required. The change adds test coverage for existing
Vortex support rather than introducing a new user-facing feature.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
