[
https://issues.apache.org/jira/browse/SPARK-38981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon updated SPARK-38981:
---------------------------------
Labels: (was: beginner)
> Unexpected commutative property of udf/pandas_udf and filters
> -------------------------------------------------------------
>
> Key: SPARK-38981
> URL: https://issues.apache.org/jira/browse/SPARK-38981
> Project: Spark
> Issue Type: Bug
> Components: Optimizer
> Affects Versions: 3.2.1
> Reporter: Maximilian Sackel
> Priority: Major
> Attachments: optimization_udf_filter.html, screenshot-1.png,
> screenshot-2.png
>
>
> Hello all,
> When running the attached minmal working example in the attachments, the
> order of the filter and the UDF is swapped by the optimizer. This can lead to
> errors, which are difficult to debug. In the documentation I have found no
> reference to such behavior.
> Is this a bug or a functionality which is poorly documented?
> With kind regards,
> Max
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
