jackylee-ch opened a new pull request, #10193:
URL: https://github.com/apache/paimon/pull/10193
## Purpose
A Paimon `TIMESTAMP(0)` column becomes a pyarrow timestamp with unit `s`, and
`to_avro_type` raised `Avro does not support pyarrow timestamp with unit s`.
So
an Avro table with a `TIMESTAMP(0)` column could not be written, even though
Parquet and ORC accept it. Java `AvroSchemaConverter` maps `precision<=3` to
`timestamp-millis` (0 included), and millis holds second granularity
losslessly.
## Change
- Map the `s` unit to `timestamp-millis` / `local-timestamp-millis` alongside
`ms`. Values are written from pyarrow via `to_pydict` (Python datetimes),
so
fastavro emits correct millis — no seconds/millis confusion.
- The `ns` unit (`TIMESTAMP(7-9)`) stays rejected: Avro has no nanos logical
type, matching Java's `precision>6` rejection; widening to micros would
drop
precision silently.
## Tests
- `to_avro_type` maps `s` to `(local-)timestamp-millis` and still rejects
`ns`;
a second-granularity value round-trips through `write_avro` unchanged.
Verified non-vacuous (without the fix the `s` case raises).
Written with Claude Code; verification is mine.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
