[
https://issues.apache.org/jira/browse/SPARK-46397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ruifeng Zheng resolved SPARK-46397.
-----------------------------------
Fix Version/s: 4.0.0
Resolution: Fixed
Issue resolved by pull request 44529
[https://github.com/apache/spark/pull/44529]
> sha2(df.a, 1024) throws a different exception in Spark Connect
> --------------------------------------------------------------
>
> Key: SPARK-46397
> URL: https://issues.apache.org/jira/browse/SPARK-46397
> Project: Spark
> Issue Type: Bug
> Components: Connect
> Affects Versions: 4.0.0
> Reporter: Hyukjin Kwon
> Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
>
> {code}
> from pyspark.sql import functions as sf
> spark.range(1).select(sf.sha2(sf.col("id"), 1024)).collect()
> {code}
> Non-connect:
> {code}
> ...
> pyspark.errors.exceptions.captured.IllegalArgumentException: requirement
> failed: numBits 1024 is not in the permitted values (0, 224, 256, 384, 512)
> {code}
> Connect:
> {code}
> ...
> pyspark.errors.exceptions.connect.AnalysisException:
> [DATATYPE_MISMATCH.UNEXPECTED_INPUT_TYPE] Cannot resolve "sha2(id, 1024)" due
> to data type mismatch: Parameter 1 requires the "BINARY" type, however "id"
> has the type "BIGINT". SQLSTATE: 42K09;
> 'Project [unresolvedalias(sha2(id#1L, 1024))]
> +- Range (0, 1, step=1, splits=Some(1))
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
