mbutrovich commented on issue #3141: URL: https://github.com/apache/iceberg-rust/issues/3141#issuecomment-6589456819
> Whether Java's overflow is desirable is beside the point It is the point. The Iceberg spec is the reference for every implementation, and Java is one implementation of it. Java settles only what the spec leaves open. The spec doesn't leave this open. The [truncate transform details](https://github.com/apache/iceberg/blob/f5704ee94b62839d79d9caf38045a3275fd3b4f0/format/spec.md?plain=1#L653-L661) define `int` truncation as `v - (v % W)` and say the remainder must be positive. For `W > 2^30`, Java's `(v % W) + W` overflows, the remainder comes out negative, and the partition value violates the spec. Java treats it as a bug in apache/iceberg#18111, with a fix in apache/iceberg#18112. apache/iceberg-cpp#1042 and PyIceberg already produce the spec value. Once #18112 merges, Java writes the spec value too, so copying today's Java would leave iceberg-rust as the outlier. Copying it also makes iceberg-rust's own scan planning prune files that match the filter ([repro](https://github.com/apache/iceberg-rust/pull/3573#discussion_r4448234423)). Problem 1 is not a bug in iceberg-rust. `rem_euclid` gave the spec value. Problem 2 is a real bug. At the integer minimum the result doesn't fit in the type and the spec is silent, so wrapping as Java does is the right fix for the debug panic. Could we retitle the issue to the debug panic and update the expected behavior to match? Any issue that reports iceberg-rust diverging from Java should quote what the spec says about that behavior, with a permalink. If the spec settles it, the spec is the expected behavior, whatever Java does. If Java contradicts the spec, that's a bug report for apache/iceberg. Java's behavior is the expected behavior here only where the spec is silent. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
