andygrove opened a new pull request, #356:
URL: https://github.com/apache/datafusion-spark/pull/356
## Which issue does this PR close?
- Part of #353.
## Rationale for this change
Spark declares `crc32`'s input type as BINARY, and casts a STRING argument
to it. `crc32` already accepts the same types, but it rejects the others with
DataFusion's errors instead of Spark's:
| Query | Before | This PR |
|---|---|---|
| `crc32(1::INT)` | `Function 'crc32' requires Binary, but received Int32`
... | `` [DATATYPE_MISMATCH.UNEXPECTED_INPUT_TYPE] The first parameter of
`crc32` requires the "BINARY" type, however it has the type "INT". SQLSTATE:
42K09 `` |
| `crc32('a', 'b')` | `Function 'crc32' expects 1 arguments but received 2`
... | `` [WRONG_NUM_ARGS.WITHOUT_SUGGESTION] The `crc32` requires 1 parameters
but the actual number is 2. ... `` |
## What changes are included in this PR?
`crc32` declares Spark's `Crc32.inputTypes` with `SparkInputTypes` from
#350, the same way `sha2` does.
## Are these changes tested?
Yes. New records in `hash/crc32.slt` cover a dictionary-encoded string, a
column with NULLs, the errors for other types and for the wrong number of
arguments, and ANSI mode. The error records fail without this change. I checked
`crc32.slt` against Spark 4.2.0 with the PySpark runner, and every record
matches. The runner does not compare error messages, so I checked them against
Spark by hand. Spark's message also names the argument's SQL text, as noted in
#350.
## Are there any user-facing changes?
Only the error messages above. The results do not change.
This PR was prepared with AI assistance (Claude Code) and reviewed by me.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]