SparkApplicationMaster commented on code in PR #16946: URL: https://github.com/apache/datafusion/pull/16946#discussion_r2296299288
########## datafusion/sqllogictest/test_files/spark/conditional/if.slt: ########## @@ -21,7 +21,135 @@ # For more information, please see: # https://github.com/apache/datafusion/issues/15914 -## Original Query: SELECT if(1 < 2, 'a', 'b'); -## PySpark 3.5.5 Result: {'(IF((1 < 2), a, b))': 'a', 'typeof((IF((1 < 2), a, b)))': 'string', 'typeof((1 < 2))': 'boolean', 'typeof(a)': 'string', 'typeof(b)': 'string'} -#query -#SELECT if((1 < 2)::boolean, 'a'::string, 'b'::string); +## Basic IF function tests + +# Test basic true condition +query T +SELECT if(true, 'yes', 'no'); Review Comment: please also check if when(...).otherwise(`other`) used here: https://github.com/apache/datafusion/pull/16946/files#diff-8184a681e2d4b84411030426011f4e80cc4a79e2debd39f6290d0159d83a63a5R97 does not eagerly calculate `other`. because I faced that it is calculating `other` when using `when` expression: https://github.com/lakehq/sail/issues/648 and the fix was just removing else_expr from when expr and placing it to the last branch of when with true predicate: https://github.com/lakehq/sail/pull/649/files also checked this now by adding in datafusion if.slt: ``` query I SELECT case when true then 1 / 1 else 1 / 0 end; ---- 1 ``` and got this: ``` 1. query failed: DataFusion error: Arrow error: Divide by zero error [SQL] SELECT case when true then 1 / 1 else 1 / 0 end; ``` Not sure if eager evaluation of else expression is OK in datafusion, but in spark it's definitely NOT OK, and there is a doctest checking this: https://github.com/apache/spark/blob/326052ec8280d8bf8ee1904504be3b62a72d3d29/python/pyspark/sql/column.py#L1418-L1427 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
