andygrove opened a new issue, #347:
URL: https://github.com/apache/datafusion-spark/issues/347

   ### Describe the bug
   
   Spark's `contains` implicitly casts a floating-point, decimal or timestamp 
argument to a string. This crate's `contains` rejects them at planning with 
``Unsupported Data Type: Spark `contains` function expects STRING or BINARY 
arguments``.
   
   Rejecting them is deliberate for now, because Arrow's cast to string formats 
these types differently from Spark. Spark's decimal-to-string output also 
depends on ANSI mode.
   
   ### To Reproduce
   
   On Spark 4.2.0:
   
   | Query | ANSI off | ANSI on |
   |---|---|---|
   | `SELECT contains(1.5D, '.')` | `true` | `true` |
   | `SELECT contains(TIMESTAMP '2024-01-01 00:00:00', ' ')` | `true` | `true` |
   | `SELECT contains(CAST(0.0000001 AS DECIMAL(10,8)), 'E')` | `true` | 
`false` |
   
   Here all three raise the planning error above.
   
   ### Expected behavior
   
   Cast these arguments to a string the way Spark does, then compare. This 
needs Spark-compatible casts to string from #88, so this issue can close once 
those exist.
   
   Spark gets this from `ImplicitCastInputTypes`: `Contains` declares string 
inputs, and the implicit cast to STRING applies to these types. #346 proposes 
declaring input types that way and coercing them in one place. With that, 
`contains` would declare its inputs instead of choosing accepted types in 
`coerce_types`.
   
   ### Additional context
   
   `string/contains.slt` records Spark's results as commented-out queries that 
cite this issue. Raised in review of #328.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to