jackylee-ch opened a new pull request, #983: URL: https://github.com/apache/paimon-rust/pull/983
The Variant binary stores timestamps as microseconds (encoding type codes 12 and 13 are `TIMESTAMP(MICROS)` / `TIMESTAMP_NTZ(MICROS)`), and the shredding spec requires the shredded `typed_value` column to use microseconds. A configured shredding schema, however, had its declared type copied through verbatim, so declaring `TIMESTAMP(3)` produced a millisecond-typed `typed_value` column that still held the microsecond count. Rust round-trips this losslessly because it ignores the unit on both sides, but the file violates the spec and any precision-aware reader (Spark, DuckDB, Arrow, Java `Timestamp.fromMicros`) reads it 1000x off. Pin the shredded timestamp `typed_value` to microseconds so the column is annotated consistently; other scalars are unchanged. A test asserts this holds even when the configured schema declares another precision. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
