[
https://issues.apache.org/jira/browse/SPARK-26660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
liupengcheng updated SPARK-26660:
---------------------------------
Attachment: screenshot-1.png
> Add warning logs for large taskBinary size
> ------------------------------------------
>
> Key: SPARK-26660
> URL: https://issues.apache.org/jira/browse/SPARK-26660
> Project: Spark
> Issue Type: Improvement
> Components: Spark Core
> Affects Versions: 2.1.0, 2.4.0
> Environment: Spark2.1
> Reporter: liupengcheng
> Priority: Minor
> Attachments: screenshot-1.png
>
>
> In our production environment, we encountered a OOM problem in ML
> application. After carefully investigation, it was found that some ML library
> may generate large ml model, thus causing executor may not able to
> deserialize it and result in OOM failures.
> !screenshot-1.png!
> In order to facilitate the debuging of memory problem caused by large
> taskBinary broadcast, we may should add same warning logs for it.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
