comphead opened a new issue, #26341: URL: https://github.com/apache/datafusion/issues/26341
### Describe the bug Spark's `ParseUrl` has `failOnError = SQLConf.get.ansiEnabled` ([urlExpressions.scala#L211-L219](https://github.com/apache/spark/blob/v4.0.0/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/urlExpressions.scala#L211-L219)). A malformed URL returns NULL unless ANSI mode is on, and then it raises `INVALID_URL`. `try_parse_url` never raises. `parse_url` in `datafusion-spark` always raises the ANSI style error ([parse_url.rs#L110-L113](https://github.com/apache/datafusion/blob/9dec77584ea39d93aa01bba7f880b318b37d2c24/datafusion/spark/src/function/url/parse_url.rs#L110-L113)) and does not read `datafusion.execution.enable_ansi_mode`. With ANSI mode off, which is the DataFusion default, the result differs from Spark. ### To Reproduce Spark 4.0.0: ```sql SELECT parse_url('inva lid://spark.apache.org/path?query=1', 'QUERY') -- NULL (spark.sql.ansi.enabled=false) -- [INVALID_URL] The url is invalid: inva lid://spark.apache.org/path?query=1. -- Use `try_parse_url` to tolerate invalid URL and return NULL instead. (spark.sql.ansi.enabled=true) ``` DataFusion: [parse_url.slt#L346-L347](https://github.com/apache/datafusion/blob/9dec77584ea39d93aa01bba7f880b318b37d2c24/datafusion/sqllogictest/test_files/spark/url/parse_url.slt#L346-L347) expects the error without enabling ANSI mode. ### Expected behavior Read `enable_ansi_mode` like the other ANSI aware functions in the crate (for example `round`): return NULL for a malformed URL when it is off and raise `INVALID_URL` when it is on. This is a small part of #23929. ### Additional context Found by running the queries in `datafusion/sqllogictest/test_files/spark` on a local Spark and comparing the results with the expected values in the files, which is the kind of check proposed in #17045. Spark results come from Apple's build of Spark 4.0.0 (`4.0.0.65-apple`) with a UTC session and `spark.sql.ansi.enabled=false` unless stated. The Spark source links point at the `v4.0.0` tag. DataFusion results are the sqllogictest expectations at 9dec77584, which CI enforces. Anything marked as not run was derived from reading the code. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
