yihua opened a new pull request, #871:
URL: https://github.com/apache/hudi-rs/pull/871
## Description
closes #870
The native write path accepted only a single `Utf8` record key field, so
tables keyed on several fields or on an integer could not be written. Record
keys are now built the way Java's `KeyGenerator.constructRecordKey` builds them
(`field:value` pairs joined by commas, `__null__` and `__empty__` placeholders,
the same all-null error) for string, boolean, integer, date and decimal key
fields. The writer, `SimpleIndex` and the virtual-key reader share one encoder
(`keygen/record_key.rs`) and one key generator resolver, so they cannot
disagree.
Timestamp and floating-point key fields stay unsupported: Spark's Avro path,
Spark's row writer, Flink and the writer's JDK each store them differently, and
an encoding that does not match the table's would duplicate rows. A delete or
update filter on one field of a composite key scans instead of using the index,
and an `IN` list on `_hoodie_record_key` is refused for composite keys, since
their values contain commas.
## How are the changes test-covered
- [ ] N/A
- [x] Automated tests (unit and/or integration tests)
- [ ] Manual tests
- [ ] Details are described below
`table_write_composite_keys_tests` runs a composite-key lifecycle (append,
upsert, delete by key, delete by one key field, the refused `IN` form) on
copy-on-write and merge-on-read, each with the record index and with
`SimpleIndex`, plus partitioned complex keys and a single `Int64` key end to
end. Unit tests pin the encoding against Java's output for each accepted type,
the placeholders and the all-null message, and the rejection of timestamp and
float keys. The new integration tests fail on `main`.
https://claude.ai/code/session_01PpvixgKyY2VqVo8UAdtUeF
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
