yihua opened a new pull request, #19028:
URL: https://github.com/apache/hudi/pull/19028
### Change Logs
Adds a write config
`hoodie.write.schema.allow.timestamp.precision.evolution` (default `false`)
that, when `true`, lets the internal-schema reconcile path relabel a column
between `timestamp-millis` and `timestamp-micros` (and between the
`local-timestamp` variants), and attach a missing `local-timestamp` logical
type on top of a bare `long`. Default `false` keeps the existing strict
rejection.
The non-reconcile write path was already lenient via Avro reader/writer
compatibility (both logical types share the same Avro `long` primitive). The
internal-schema reconcile path, triggered when
`hoodie.write.set.null.for.missing.columns=true`, instead rejected the relabel
through `SchemaChangeUtils.isTypeUpdateAllow`. This closes the parity gap and
enables forward-fixing tables that earlier writer versions persisted with a
`timestamp-micros` label while writers produced `timestamp-millis` values, or
that dropped the `local-timestamp` logical type entirely and stored the column
as bare `long`.
### Impact
- New advanced write config, opt-in. Default preserves the prior strict
behavior; no existing caller sees a change.
- One new overload on `AvroSchemaEvolutionUtils.reconcileSchema`, one new
factory on `TableChanges.ColumnUpdateChange.get`. Pre-existing overloads kept
as delegates.
- Config threaded through `HoodieSchemaUtils`, `BaseHoodieWriteClient`,
`HoodieMergeHelper`, `FileGroupReaderBasedMergeHandle`.
### Risk level (write none, low medium or high below)
low
The gate is opt-in and the default branch preserves prior behavior. Test
coverage adds positive variants on the existing v6/v8/CURRENT logical-repair
fixtures and a negative variant that asserts `SchemaCompatibilityException`
when the reconcile path is on with the gate closed.
### Documentation Update
New config documented inline on
`HoodieCommonConfig.ALLOW_TIMESTAMP_PRECISION_EVOLUTION` with
`sinceVersion("1.3.0")`.
### Contributor's checklist
- [x] Read through [contributor's
guide](https://hudi.apache.org/contribute/how-to-contribute)
- [x] Change Logs and Impact were stated clearly
- [x] Adequate tests were added if applicable
- [ ] CI passed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
