adarshomrey commented on issue #18479:
URL: https://github.com/apache/iceberg/issues/18479#issuecomment-6382676793
I can reproduce this on main (JDK 21) with a unit test that doesn't need a
table.
Schema: optional timestamp_ns "ts", spec: identity("ts").
Expression projected = Projections.inclusive(spec)
.project(Expressions.in("ts", "2026-01-01T00:00:00.000000001",
"2025-01-01T00:00:00.000000001"));
Binder.bind(spec.partitionType(), projected, true);
This throws ArithmeticException: long overflow:
DateTimeUtil.microsToNanos
<- Literals$TimestampLiteral.to (Literals.java:452)
<- Literals$LongLiteral.to (Literals.java:311)
<- UnboundPredicate.bindInOperation
For values near the epoch there is no exception but the bound set is wrong:
[1, 86400000000001] becomes [1000, 86400000000001000].
Cause: Identity.projectStrict handles set predicates by passing the raw Long
values
from asSetPredicate().literalSet() to Expressions.predicate(op, name,
Iterable<T>),
which wraps them in LongLiteral. At bind time LongLiteral.to(TIMESTAMP_NANO)
assumes micros and multiplies by 1000. Single-value predicates avoid this
because
Identity passes the typed Literal.
I'd like to work on this. The tricky part is that there is no public way to
build
an UnboundPredicate for a set from typed literals (that constructor is
private and
the literal classes are package-private). Which direction would you prefer?
(1) a new Expressions/UnboundPredicate factory that takes typed literals,
used by
Identity,
(2) changing how LongLiteral converts to timestamp_ns, or
(3) something else?
I have failing tests for the inclusive projection and can add strict
projection
(and NOT IN) as well.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
