Vinod KC created SPARK-58578:
--------------------------------
Summary: Mark ApplyFunctionExpression stateful to prevent
reusedRow data races under concurrent evaluation
Key: SPARK-58578
URL: https://issues.apache.org/jira/browse/SPARK-58578
Project: Spark
Issue Type: Sub-task
Components: SQL
Affects Versions: 4.2.0, 4.0.0, 4.1.0, 3.5.0, 4.3.0
Reporter: Vinod KC
`ApplyFunctionExpression` holds a shared mutable row buffer (reusedRow) on the
expression instance. When sibling QueryExecutions derived from the same base
DataFrame run concurrently, they share the same expression instance and race on
this buffer, silently producing wrong results.
Because stateful defaults to false, the expression is never cloned before
concurrent execution. Need to override def stateful: Boolean = true, which
causes each concurrent execution to get its own independent instance.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
