leoyvens opened a new pull request, #13590: URL: https://github.com/apache/datafusion/pull/13590
## Which issue does this PR close? My motivation was to improve DF testing of float outputs, even at the least-significant digits. The situation in #13569 seemed a bit uncomfortable, and generally the SLTs were formatting floats and decimal in complicated ways. So I took a shot at tackling it "the hard way", which involved changing virtually all SLTs that print out floats. ## Rationale for this change The rationale is that this makes the SLT test output closer to the output a DataFusion user would typically see, in `datafusion-cli`, when writing float outputs to CSV or when using `arrow_cast` to convert a float to string. Tests do become more sensitive to minor changes in the handling of floats by DataFusion. But if the user will have to deal with it, then the tests should also have to deal with it. The interaction with pg_compat tests is an interesting one. The Postgres `avg` over integers returns a `numeric`, while the DataFusion `avg` over integers returns a `Float64`. The SLTs previously dealt with this by rounding everything to 12 decimal digits. This PR deals with it by testing the value within an epsilon, with the justification of allowing us to entirely remove the dependency on the `bigdecimal` crate. ## What changes are included in this PR? This PR does non-trivial code changes only to `sqllogictest/src/engines/conversion.rs`. But the fallout to `.slt` files makes the diff large. - SLTs now formats floats using `ryu` which is what arrow-rs uses, improving consistency with user-visible outputs. - pg_compat tests now test `avg` of integers within an epsilon, rather than relying on implicit truncation by the SLT engine. - Dependency on `bigdecimal` is removed (this was a test-only dependency). - Dependency area on `rust_bigdecimal` is reduced (this is a test-only dependency). ## Are these changes tested? The changes are to the tests. Who tests the tests? 😄 ## Are there any user-facing changes? No. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
