GitHub user sarutak opened a pull request:
https://github.com/apache/spark/pull/5437
[SPARK-2673][CORE][WEBUI] Enable to attach Debugger to Executors easily
In current implementation, it's difficult to attach debugger to each
Executor in the cluster.
There are some reasons as follows.
1) Multi Executors can run on the same machine so each executor should open
individual debug ports.
2) Even if we can open unique debug port to each Executors running on the
same machine, it's a bother to check debug port of each executor.
To solve those problem, I think following 2 improvement is needed.
1) Enable executor to open unique debug port on a machine by setting debug
port to 0.
2) Expand WebUI to be able to show debug ports opening in each executor.
Following image is how Web UI(ExecutorsPage) show the debug ports for each
Executor.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/sarutak/spark debug-port-feature
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/5437.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #5437
----
commit a2d75722a13088d94c6cc986d753d944adf7ab10
Author: Kousuke Saruta <[email protected]>
Date: 2015-04-06T11:29:45Z
Initial commit
commit 5c4982c9413c9ed6985e8e8f614730c5cd2c1023
Author: Kousuke Saruta <[email protected]>
Date: 2015-04-07T09:08:41Z
Improved code
commit e87175319c0eedb3d48b825db9d8628946e3e303
Author: Kousuke Saruta <[email protected]>
Date: 2015-04-08T01:05:12Z
Merge branch 'master' of git://git.apache.org/spark into debug-port-feature
commit 602c335f7c382b1db78549386216adf2307ee040
Author: Kousuke Saruta <[email protected]>
Date: 2015-04-08T08:27:49Z
Added a test case to YarnClusterSuite for the debug port feature
commit 4c16e9cb4f0d84927494279650b5b75295f61478
Author: Kousuke Saruta <[email protected]>
Date: 2015-04-08T09:05:54Z
Temporary commit
commit 654500d89ffdb94f9e2c83bc18c7e92793fb524b
Author: Kousuke Saruta <[email protected]>
Date: 2015-04-09T07:11:11Z
Added test cases to UISeleniumSuite
commit dbf69f39b0443dc7eff49cf985b2ac9cc658c954
Author: Kousuke Saruta <[email protected]>
Date: 2015-04-09T07:11:36Z
Added ExecutorDebugPortsuite
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
