viirya commented on PR #2643: URL: https://github.com/apache/arrow-rs/pull/2643#issuecomment-1236163002
> I'm not a massive fan of forcing users to choose between slow but correct or fast but may have inconsistent behaviour, especially as having parallel kernels increases the likelihood of further divergent behaviour... I think that you're talking about `divide_checked`. Another thought is, I guess the non-simd one should be optimized by the compiler? Not sure how much performance difference between them. I was thinking if possibly to do same thing on `simd_checked_divide_op`. But seems simd integers (packed_simd2) don't provide similar wrapping/checked APIs. > Taking a step back I wonder if we could just define the overflow behaviour as wrapping, and use explicit wrapping_op to avoid signed overflow panics in non-release builds. This avoids runtime penalties, is consistent with how Rust handles overflow (unlike C++ signed integer overflow is actually defined, the debug panics are just "helpful"), and is what I at least would expect to occur. Hmm, is that something we want to have? Actually it may cause more difficulty for us to use this crate. As I mentioned below, we actually need two variants: overflow-checking (currently it could be by setting overflow-checks cargo flag) and overflow-as-null. I don't think defining the overflow behavior as wrapping is good idea. It sounds like a regression from current status. Users cannot choose overflow-checking behavior after that. > I'm not sure what SQL says on the topic of overflow, if anything, which may be relevant here? Perhaps @alamb knows? This is the next think we want to do. Actually it is more important to us. In Spark, once configured, it is allowed to have overflow. Overflowing value will be represented as NULL. That's being said, we can skip this change (overflow-checking variant/non-overflow-checking variant) if it cannot reach consensus. I just thought to have overflow/non-overflow variants like C++ is a good idea. We actually need an overflow-checking variant and an overflow-as-null variant. And the current arithmetic kernels are overflow-checking variant already (if overflow-checks is enabled by users). We just need to add an overflow-as-null variant. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
