ump45nose commented on PR #7227:
URL: 
https://github.com/apache/datafusion-comet/pull/7227#issuecomment-6895055172

   @andygrove thanks for tracing that. I went with the first option in 
`172e6b10`: the runtime check now rejects `Timestamp(Second, _) -> Timestamp` 
as well. A second-unit timestamp can only come from an `ARROW:schema` hint over 
an unannotated INT64 leaf, so it now follows the same `INT64 -> Timestamp` rule 
the footer check applies after `mark_parquet_leaves` rewrites it to `Int64`. 
Both paths reject the read, as Spark does. `Second -> bigint` and `Second -> 
decimal(20,0)` are unchanged, so the earlier `canReadAsDecimal` case still 
reads.
   
   New test `parquet_arrow_writer_timestamp_second_read_as_timestamp_errors` 
writes `Timestamp(Second, UTC)` with ArrowWriter and reads it as `timestamp` 
and `timestamp_ntz`, with row filters off and on. All four reads are rejected 
naming the column. Without the change, the row-filters-off read succeeds.
   
   Local: `cargo test -p datafusion-comet --lib -- parquet::` 335 passed (1 
ignored), `cargo clippy -p datafusion-comet --all-targets -- -D warnings` 
clean. The commit also applies `cargo fmt` to two spots from my earlier commit 
that the current stable rustfmt formats differently.
   
   (AI-assisted change and reply.)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to