andygrove opened a new pull request, #356:
URL: https://github.com/apache/datafusion-spark/pull/356

   ## Which issue does this PR close?
   
   - Part of #353.
   
   ## Rationale for this change
   
   Spark declares `crc32`'s input type as BINARY, and casts a STRING argument 
to it. `crc32` already accepts the same types, but it rejects the others with 
DataFusion's errors instead of Spark's:
   
   | Query | Before | This PR |
   |---|---|---|
   | `crc32(1::INT)` | `Function 'crc32' requires Binary, but received Int32` 
... | `` [DATATYPE_MISMATCH.UNEXPECTED_INPUT_TYPE] The first parameter of 
`crc32` requires the "BINARY" type, however it has the type "INT". SQLSTATE: 
42K09 `` |
   | `crc32('a', 'b')` | `Function 'crc32' expects 1 arguments but received 2` 
... | `` [WRONG_NUM_ARGS.WITHOUT_SUGGESTION] The `crc32` requires 1 parameters 
but the actual number is 2. ... `` |
   
   ## What changes are included in this PR?
   
   `crc32` declares Spark's `Crc32.inputTypes` with `SparkInputTypes` from 
#350, the same way `sha2` does.
   
   ## Are these changes tested?
   
   Yes. New records in `hash/crc32.slt` cover a dictionary-encoded string, a 
column with NULLs, the errors for other types and for the wrong number of 
arguments, and ANSI mode. The error records fail without this change. I checked 
`crc32.slt` against Spark 4.2.0 with the PySpark runner, and every record 
matches. The runner does not compare error messages, so I checked them against 
Spark by hand. Spark's message also names the argument's SQL text, as noted in 
#350.
   
   ## Are there any user-facing changes?
   
   Only the error messages above. The results do not change.
   
   This PR was prepared with AI assistance (Claude Code) and reviewed by me.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to