jorisvandenbossche commented on issue #34901: URL: https://github.com/apache/arrow/issues/34901#issuecomment-1503439087
> For pyarrow, we should probably: > > 1. Allow both safe and unsafe conversion options for scalar APIs (feature) > 2. Default to safe conversion for scalars, which appears is not happening (bug) Yes, fully agreed, I opened a separate issue for specifically this aspect: https://github.com/apache/arrow/issues/35040 > 3. Look into allowing safe conversion from int <-> float for valid numbers larger than 2^53 (feature) For checking the safety of casting int to float, we indeed use this fixed range: https://github.com/apache/arrow/blob/e488942cd552ac36a46d40477c1b0326a626ed98/cpp/src/arrow/compute/kernels/scalar_cast_numeric.cc#L171-L250 I am not fully sure this is something we should change. First, I think this is a lot simpler in implementation to just check for values within the range, compared to checking for certain integers that can still be represented as float outside of that range. But also for the user this seems easier to understand and gives more consistent behaviour? (just everything outside of that range will fail with the default `safe=True`, and not depending on the exact value) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
