shehabgamin commented on issue #15916: URL: https://github.com/apache/datafusion/issues/15916#issuecomment-3752099364
@carlosahs Great questions! > For the integer types, we set the precision to the minimum power of 10 that contains all of the bits, right? To be honest, I didn't think about this and looked at the behavior from the Spark codebase. > So, Int8 => Ok((3, 0)) because 10^3 = 1000, 2^8 = 256 and 1000 >= 256. But then we have UInt8 mapped to Ok((5, 0)), is there a particular reason why powers of 10 for unsigned integers need to match with the powers of 10 for signed integers that double their bits, i.e., UInt8 and Int16 share 5 exponent, UInt16 and Int32 share 10 exponent. Spark doesn't support unsigned integers so Sail will try to cast an unsigned integer to a signed integer. So for example, `UInt8 -> Int16`. You can use `fn coerce_types` to do this casting however. And I'm not sure why I did `UInt64 -> Int64`, `UInt64` just shouldn't be supported. > I'm a little bit lost on mapping for Float32 and Float64. Are precision and scale values computed based on [f32::DIGITS](https://doc.rust-lang.org/std/primitive.f32.html#associatedconstant.DIGITS) and [f64::DIGITS](https://doc.rust-lang.org/std/primitive.f64.html#associatedconstant.DIGITS)? I looked at Spark's behavior replicated it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
