JingsongLi commented on code in PR #884:
URL: https://github.com/apache/paimon-rust/pull/884#discussion_r4056354807
##########
crates/paimon/src/arrow/residual.rs:
##########
@@ -1114,7 +1114,15 @@ fn timestamp_scalar(
timezone: Option<&'static str>,
) -> crate::Result<Option<ArrayRef>> {
let array: ArrayRef = match precision {
- 0..=3 => {
+ 0 => {
+ let value = millis.div_euclid(1_000);
+ let array = TimestampSecondArray::new_scalar(value).into_inner();
Review Comment:
Fixed in 531c5a06. Exact timestamp predicates now compare the decoded Arrow
value and Datum literal in lossless i128 epoch nanoseconds, independently of
whether the physical array is seconds, milliseconds, microseconds, or
nanoseconds. Added coverage for existing millisecond-encoded TIMESTAMP(0) and
TIMESTAMP_LTZ(0) Parquet files and for the complete DataFileReader path.
##########
crates/paimon/src/arrow/mod.rs:
##########
@@ -149,7 +149,8 @@ pub fn paimon_type_to_arrow(dt: &PaimonDataType) ->
crate::Result<ArrowDataType>
fn timestamp_time_unit(precision: u32) -> crate::Result<TimeUnit> {
match precision {
- 0..=3 => Ok(TimeUnit::Millisecond),
+ 0 => Ok(TimeUnit::Second),
Review Comment:
Fixed in 531c5a06. The Parquet writer now keeps the public input schema as
Timestamp(Second) but recursively normalizes second timestamps to
Timestamp(Millisecond) for the physical writer and casts each batch at that
boundary. The regression test asserts INT64 + TIMESTAMP_MILLIS (including LTZ
adjusted-to-UTC), millisecond min/max statistics, a successful filtered read,
and the final DataFileReader output returning Timestamp(Second).
##########
crates/paimon/src/arrow/residual.rs:
##########
@@ -1114,7 +1114,15 @@ fn timestamp_scalar(
timezone: Option<&'static str>,
) -> crate::Result<Option<ArrayRef>> {
let array: ArrayRef = match precision {
- 0..=3 => {
+ 0 => {
+ let value = millis.div_euclid(1_000);
Review Comment:
Fixed in 531c5a06. Timestamp literals are no longer divided to the column
precision. The exact evaluator preserves millis plus nanos-of-millisecond and
compares in i128 epoch nanoseconds. The focused regression covers equality and
greater-than-or-equal for a 1.5-second literal against [1s, 2s] for both
TIMESTAMP(0) and TIMESTAMP_LTZ(0).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
