[
https://issues.apache.org/jira/browse/SPARK-51499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18063662#comment-18063662
]
Holden Karau commented on SPARK-51499:
--------------------------------------
Where is this?
> why copy child RDD in EvalPythonExec? it may lead into risk of oom
> ------------------------------------------------------------------
>
> Key: SPARK-51499
> URL: https://issues.apache.org/jira/browse/SPARK-51499
> Project: Spark
> Issue Type: Question
> Components: PySpark, SQL
> Affects Versions: 3.2.0
> Reporter: fengtong
> Priority: Major
>
> I found
> val inputRDD = child.execute().map(_.copy())
> in "doExecute()" method of trait EvalPythonExec.
> I'm concerning about the reason why a copy leveraged here, which may lead
> into risk of oom
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
