guan404ming commented on code in PR #59580:
URL: https://github.com/apache/spark/pull/59580#discussion_r4344532603
##########
python/pyspark/sql/transpile.py:
##########
@@ -928,6 +928,13 @@ def _get_transpilers(session: "SparkSession") ->
List[AbstractTranspiler]:
if not configured_transpilers:
return []
transpiler_names = configured_transpilers.split(",")
+ unknown_names = [name for name in transpiler_names if name not in
AbstractTranspiler.varieties]
+ if unknown_names:
+ warnings.warn(
+ f"Ignoring unknown transpilers {unknown_names} in "
+ "spark.sql.experimental.optimizer.pyTranspilers; registered
transpilers are "
+ f"{sorted(AbstractTranspiler.varieties)}"
+ )
return [
AbstractTranspiler.varieties[name]()
for name in transpiler_names
Review Comment:
Thanks for the review! The comprehension keeps only registered names via `if
name in AbstractTranspiler.varieties`, so it warns and continues. Please
correct me if I misunderstood.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
