yihua opened a new issue, #958:
URL: https://github.com/apache/hudi-rs/issues/958
### Feature Description
The public Rust API of `hudi-core` (re-exported through `hudi`) should not
carry `Hoodie` naming. The Python, C++, and FFI bindings already expose no
`Hoodie` names, so this is about the Rust surface only. Internal (`pub(crate)`
or private) names are out of scope here.
Items that already existed on the read path and shipped in 0.5.0 stay as
they are:
- `metadata::commit::{HoodieCommitMetadata, HoodieWriteStat}` (#477)
- `metadata::replace_commit::HoodieReplaceCommitMetadata` (#477)
- `metadata::table::records::HoodieMetadataFileInfo` (#492)
Avro field names are on-disk format and also stay, including the public
fields that mirror them (`CompactionPlan::preserve_hoodie_metadata`,
`RestoreMetadata::hoodie_restore_metadata`).
The items below came in with the write path (#666, #931) and have not been
released yet. The names and approaches are a proposal, open for discussion.
| Current | Where it is used outside `hudi-core` src | Proposal |
|---|---|---|
| `index::HoodieKey` (also re-exported at the crate root and in `table`) |
`Table::delete_keys`, the Python bridge (internal only, the Python API is
unchanged), integration tests | Rename, e.g. `RecordKey` to pair with
`RecordLocation`, or `HudiKey` |
| `index::HoodieIndex` trait | integration tests | Rename, e.g. `Index`, or
`HudiIndex` |
| `metadata::replace_commit::HoodieRequestedReplaceMetadata` | integration
test | Rename, e.g. `RequestedReplaceMetadata` |
| `metadata::rollback::{HoodieRollbackRequest,
HoodieRollbackPartitionMetadata}` | rollback tests | Drop the prefix; no name
collision |
| `metadata::rollback::{HoodieInstantInfo, HoodieRollbackPlan,
HoodieRollbackMetadata}` | rollback tests | See below; dropping the prefix
collides with existing read-side types |
| `schema::avsc::hoodie_*_schema()` (7 functions),
`metadata::table::encode::hoodie_metadata_schema_json()` | none | Narrow to
`pub(crate)`, or rename |
#### Rollback type collision
`metadata::rollback` has two sets of types for the same Avro records:
- Read side (#706, released): `InstantInfo`, `RollbackPlan`,
`RollbackMetadata`. Subset structs that deserialize only the fields the
metadata table reader needs.
- Write side (#666): `HoodieInstantInfo`, `HoodieRollbackPlan`,
`HoodieRollbackMetadata`. The full shapes, serialized by the writer.
Options:
1. Merge each pair into one type that keeps the released read-side name and
takes the full write-side shape. This removes the duplication, but it is not a
pure rename: the read-side structs fail on a missing field today, while the
write-side ones fill defaults (`#[serde(default)]`), so reader error behavior
changes. Roughly 100-150 lines across `metadata/rollback.rs`,
`write/rollback.rs`, the metadata table rollback reader, and tests.
2. Narrow the write-side types to `pub(crate)` and move their
integration-test uses into unit tests. Small, no behavior change, leaves the
duplication.
3. Give the write-side types distinct non-`Hoodie` names for now and merge
later.
### Additional context
None of this blocks reads or writes; it is API hygiene before the write path
ships in a release. Renaming `HoodieKey` changes the Rust signature of
`Table::delete_keys`, so it should land before the first release that includes
the write API.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
