shehabgamin commented on PR #16409: URL: https://github.com/apache/datafusion/pull/16409#issuecomment-2973551244
> Got it @shehabgamin > > I'm seeing a lot of `slt` tests like > > ``` > #S > > > #E > #query > #L > ``` > > which not very explanatory. For testing `spark` integration it would probably be interesting something we did in `Comet` https://github.com/apache/datafusion-comet/blob/6bf80b107cc1574cb7f259719d0aa203e387efc4/spark/src/test/scala/org/apache/comet/CometExpressionCoverageSuite.scala#L48 > > So the test takes recent samples from internal Spark examples and run it, comparing the actual(Spark provided) and expected values @comphead I fixed the script, nice catch! > https://github.com/apache/datafusion-comet/blob/6bf80b107cc1574cb7f259719d0aa203e387efc4/spark/src/test/scala/org/apache/comet/CometExpressionCoverageSuite.scala#L182 > > This is how to get Spark examples from their internal repo Sail generates Gold data tests using tests extracted directly from the Spark code base. However, we cannot directly port the exact SQL queries from Sail because DataFusion and Spark interpret SQL data types differently (Sail speaks Spark SQL). So, when creating the `slt` tests in DataFusion, explicit type casts are required. We found that this was not straightforward for contributors to do, which is why we created this script to generate the `slt` tests with the necessary type casts as a basic starting point. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
