cloud-fan commented on code in PR #58549:
URL: https://github.com/apache/spark/pull/58549#discussion_r4210484727
##########
python/pyspark/sql/udf.py:
##########
@@ -321,6 +328,7 @@ def _conf_is_true(key: str, default: Optional[str] = None)
-> bool:
@staticmethod
def _check_return_type(returnType: DataType, evalType: int) -> None:
+ _check_no_char_varchar(returnType, "Python UDF return types")
Review Comment:
**Non-blocking (P2):** For a Classic deterministic UDF whose return type was
supplied as DDL, self.returnType is evaluated inside the optional transpilation
try. A DDL type such as char(3) now raises the new structured error at this
line, but the surrounding broad except Exception treats it as a transpilation
failure and emits an unrelated fallback warning. Under warnings-as-errors,
callers receive UserWarning instead of CHAR_VARCHAR_NOT_SUPPORTED_IN_PYTHON;
with ordinary warnings they get a misleading transpilation warning before the
later structured error.
**Verification:**
- **Regression:** Exercise a deterministic batched UDF with a CHAR or
VARCHAR DDL return type while experimental transpilation and ANSI mode are
enabled, and assert the exact structured condition under both ordinary warning
handling and warnings-as-errors.
- **Compatibility:** Exercise a non-CHAR/VARCHAR optional transpilation
failure and verify that UDF construction still emits its diagnostic warning,
clears transpiled options, and preserves interpreted execution fallback.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
