andygrove opened a new issue, #347: URL: https://github.com/apache/datafusion-spark/issues/347
### Describe the bug Spark's `contains` implicitly casts a floating-point, decimal or timestamp argument to a string. This crate's `contains` rejects them at planning with ``Unsupported Data Type: Spark `contains` function expects STRING or BINARY arguments``. Rejecting them is deliberate for now, because Arrow's cast to string formats these types differently from Spark. Spark's decimal-to-string output also depends on ANSI mode. ### To Reproduce On Spark 4.2.0: | Query | ANSI off | ANSI on | |---|---|---| | `SELECT contains(1.5D, '.')` | `true` | `true` | | `SELECT contains(TIMESTAMP '2024-01-01 00:00:00', ' ')` | `true` | `true` | | `SELECT contains(CAST(0.0000001 AS DECIMAL(10,8)), 'E')` | `true` | `false` | Here all three raise the planning error above. ### Expected behavior Cast these arguments to a string the way Spark does, then compare. This needs Spark-compatible casts to string from #88, so this issue can close once those exist. Spark gets this from `ImplicitCastInputTypes`: `Contains` declares string inputs, and the implicit cast to STRING applies to these types. #346 proposes declaring input types that way and coercing them in one place. With that, `contains` would declare its inputs instead of choosing accepted types in `coerce_types`. ### Additional context `string/contains.slt` records Spark's results as commented-out queries that cite this issue. Raised in review of #328. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
