ueshin opened a new pull request, #49219:
URL: https://github.com/apache/spark/pull/49219
### What changes were proposed in this pull request?
Fixes `transpose` to show a proper error message when invalid index columns
are specified.
### Why are the changes needed?
When invalid index columns are specified, it shows `INTERNAL_ERROR`.
```py
>>> df = spark.range(10).transpose(sf.col("id") + 1)
Traceback (most recent call last):
...
py4j.protocol.Py4JJavaError: An error occurred while calling o40.transpose.
: org.apache.spark.SparkException: [INTERNAL_ERROR] Found the unresolved
operator: 'UnresolvedTranspose [unresolvedalias((id#0L + cast(1 as bigint)))]
SQLSTATE: XX000
...
```
### Does this PR introduce _any_ user-facing change?
Yes, the proper error message will be shown.
```py
>>> df = spark.range(10).transpose(sf.col("id") + 1)
Traceback (most recent call last):
...
pyspark.errors.exceptions.captured.AnalysisException:
[TRANSPOSE_INVALID_INDEX_COLUMN] Invalid index column for TRANSPOSE because:
Index column must be an atomic attribute SQLSTATE: 42804
```
### How was this patch tested?
Added the related tests.
### Was this patch authored or co-authored using generative AI tooling?
No.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
