jiwen624 commented on issue #18479:
URL: https://github.com/apache/iceberg/issues/18479#issuecomment-6387604104
> 3\. I can reproduce this on main (JDK 21) with a unit test that doesn't
need a table.
> Schema: optional timestamp_ns "ts", spec: identity("ts").
> Expression projected = Projections.inclusive(spec)
> .project(Expressions.in("ts", "2026-01-01T00:00:00.000000001",
> "2025-01-01T00:00:00.000000001"));
> Binder.bind(spec.partitionType(), projected, true);
> This throws ArithmeticException: long overflow:
> DateTimeUtil.microsToNanos
> <- Literals$TimestampLiteral.to (Literals.java:452)
> <- Literals$LongLiteral.to (Literals.java:311)
> <- UnboundPredicate.bindInOperation
> For values near the epoch there is no exception but the bound set is
wrong:
> [1, 86400000000001] becomes [1000, 86400000000001000].
> Cause: Identity.projectStrict handles set predicates by passing the
raw Long values
> from asSetPredicate().literalSet() to Expressions.predicate(op, name,
Iterable),
> which wraps them in LongLiteral. At bind time
LongLiteral.to(TIMESTAMP_NANO)
> assumes micros and multiplies by 1000. Single-value predicates avoid
this because
> Identity passes the typed Literal.
> I'd like to work on this. The tricky part is that there is no public
way to build
> an UnboundPredicate for a set from typed literals (that constructor is
private and
> the literal classes are package-private). Which direction would you
prefer?
> (1) a new Expressions/UnboundPredicate factory that takes typed
literals, used by
> Identity,
> (2) changing how LongLiteral converts to timestamp_ns, or
> (3) something else?
> I have failing tests for the inclusive projection and can add strict
projection
> (and NOT IN) as well.
Thanks for confirming the repro! There's already a fix in #18478. It wraps
the IN/NOT IN values with Expressions.nanos(...) in Identity, so the existing
public API covers it without a new factory or a change to LongLiteral. It has
tests for both inclusive IN and strict NOT IN. Reviews welcome there.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
