yihua opened a new pull request, #878:
URL: https://github.com/apache/hudi-rs/pull/878
## Description
closes #858
The Python package exposed only the read APIs, so `pip install hudi` could
not create or change a table. `HudiTable` gains `create` (wrapping
`TableCreateBuilder`, forwarding only the arguments a caller passes so the Rust
defaults stay the defaults), `append`, `upsert` (with optional `update_columns`
on copy-on-write), `update`, `delete`, `delete_keys`, `overwrite` and
`dynamic_partition_overwrite`, each returning a `HudiWriteResult`. Every write
releases the GIL and maps errors to Python exceptions.
Write input is any `pyarrow.Table`, `RecordBatch` or `RecordBatchReader`, a
list of batches, or any object implementing the Arrow PyCapsule stream
interface, which covers Polars and pandas 2.2+ without adding either as a
dependency. View, large and dictionary types are normalized to the types the
writer accepts (moving that into core is #864), and unnamed pandas index
columns are dropped. `update` takes a dict of column values that must fit the
column's type, checked at every nesting level, or a one-row batch. The README's
"Writing Tables" section shows the Python call next to each Rust call.
The diff is over the 1000-line guideline mostly because of
`python/tests/test_table_write.py`. One test is a strict `xfail`: upserting a
Polars `list` column needs #868 and should be unmarked when that merges.
## How are the changes test-covered
- [ ] N/A
- [x] Automated tests (unit and/or integration tests)
- [ ] Manual tests
- [ ] Details are described below
`test_table_write.py` covers every verb on copy-on-write and merge-on-read,
each accepted input type (pyarrow, Polars, pandas, PyCapsule-only objects),
nested type normalization, `update` value checks (numeric, nested, temporal,
struct keys), and the error paths. Rust unit tests cover the type
normalization. `make check` and `make test-python` pass.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
