[
https://issues.apache.org/jira/browse/SPARK-12803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15096058#comment-15096058
]
Sean Owen commented on SPARK-12803:
-----------------------------------
What do you mean here by profiling -- generating heap dumps or hprof output? or
connecting with a profiler? I don't see a problem with configuring all
executors to allow profiling at the time you need to do it.
> Consider adding ability to profile specific instances of executors in spark
> ---------------------------------------------------------------------------
>
> Key: SPARK-12803
> URL: https://issues.apache.org/jira/browse/SPARK-12803
> Project: Spark
> Issue Type: Improvement
> Components: Java API
> Reporter: Rajesh Balamohan
>
> It would be useful to profile specific instances of executors as opposed to
> adding profiler details to all executors via
> "spark.executor.extraJavaOptions".
> Setting the number of executors to just 1 and profiling wouldn't be much
> useful (in some cases, most of the time with single executor mode would be
> spent in terms of reading data from remote node). At the same time, setting
> profiling option to all executors could just create too many number of
> snapshots; making it harder to analyze.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
