This is an automated email from the ASF dual-hosted git repository.
wenchen pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/spark-website.git
The following commit(s) were added to refs/heads/asf-site by this push:
new 2be8d58 Add remote debug guidance (#259)
2be8d58 is described below
commit 2be8d58af1fb84c35aebd4086593449be33327e0
Author: wuyi <ngone_5...@163.com>
AuthorDate: Wed Feb 12 20:00:16 2020 +0800
Add remote debug guidance (#259)
* add remote debug guidance
* revert unnecessary change
* address comment
* address comments
---
.gitignore | 1 +
developer-tools.md | 46 +++++++++++++++++++++
images/intellij_remote_debug_configuration.png | Bin 0 -> 225674 bytes
images/intellij_start_remote_debug.png | Bin 0 -> 363476 bytes
site/developer-tools.html | 44 ++++++++++++++++++++
.../images/intellij_remote_debug_configuration.png | Bin 0 -> 225674 bytes
site/images/intellij_start_remote_debug.png | Bin 0 -> 363476 bytes
7 files changed, 91 insertions(+)
diff --git a/.gitignore b/.gitignore
index da8a864..e102d47 100644
--- a/.gitignore
+++ b/.gitignore
@@ -1,4 +1,5 @@
.idea/
target/
.DS_Store
+.jekyll-cache/
.jekyll-metadata
diff --git a/developer-tools.md b/developer-tools.md
index 30206a7..c664dfc 100644
--- a/developer-tools.md
+++ b/developer-tools.md
@@ -435,6 +435,52 @@ Error:(147, 9) value q is not a member of StringContext
q"""
^
```
+<h4>Debug Spark Remotely</h4>
+This part will show you how to debug Spark remotely with IntelliJ.
+
+<h5>Set up Remote Debug Configuration</h5>
+Follow <i>Run > Edit Configurations > + > Remote</i> to open a default Remote
Configuration template:
+<img src="/images/intellij_remote_debug_configuration.png" style="width: 75%;
max-width: 660px;" />
+
+Normally, the default values should be good enough to use. Make sure that you
choose <b>Listen to remote JVM</b>
+as <i>Debugger mode</i> and select the right JDK version to generate proper
<i>Command line arguments for remote JVM</i>.
+
+Once you finish configuration and save it. You can follow <i>Run > Run >
Your_Remote_Debug_Name > Debug</i> to start remote debug
+process and wait for SBT console to connect:
+
+<img src="/images/intellij_start_remote_debug.png" style="width: 75%;
max-width: 660px;" />
+
+<h5>Trigger the remote debugging</h5>
+
+In general, there are 2 steps:
+1. Set JVM options using the <i>Command line arguments for remote JVM</i>
generated in the last step.
+2. Start the Spark execution (SBT test, pyspark test, spark-shell, etc.)
+
+The following is an example of how to trigger the remote debugging using SBT
unit tests.
+
+Enter in SBT console
+```
+./build/sbt
+```
+Switch to project where the target test locates, e.g.:
+```
+sbt > project core
+```
+Copy pasting the <i>Command line arguments for remote JVM</i>
+```
+sbt > set javaOptions in Test +=
"-agentlib:jdwp=transport=dt_socket,server=n,suspend=n,address=localhost:5005"
+```
+Set breakpoints with IntelliJ and run the test with SBT, e.g.:
+```
+sbt > testOnly *SparkContextSuite -- -t "Only one SparkContext may be active
at a time"
+```
+
+It should be successfully connected to IntelliJ when you see "Connected to the
target VM,
+address: 'localhost:5005', transport: 'socket'" in IntelliJ console. And then,
you can start
+debug in IntelliJ as usual.
+
+To exit remote debug mode (so that you don't have to keep starting the remote
debugger),
+type "session clear" in SBT console while you're in a project.
<h4>Eclipse</h4>
diff --git a/images/intellij_remote_debug_configuration.png
b/images/intellij_remote_debug_configuration.png
new file mode 100644
index 0000000..adb346f
Binary files /dev/null and b/images/intellij_remote_debug_configuration.png
differ
diff --git a/images/intellij_start_remote_debug.png
b/images/intellij_start_remote_debug.png
new file mode 100644
index 0000000..32152b3
Binary files /dev/null and b/images/intellij_start_remote_debug.png differ
diff --git a/site/developer-tools.html b/site/developer-tools.html
index 1d3f94d..b4d8f9a 100644
--- a/site/developer-tools.html
+++ b/site/developer-tools.html
@@ -612,6 +612,50 @@ Error:(147, 9) value q is not a member of StringContext
</code></pre></div> </div>
</li>
</ul>
+<h4>Debug Spark Remotely</h4>
+<p>This part will show you how to debug Spark remotely with IntelliJ.</p>
+
+<h5>Set up Remote Debug Configuration</h5>
+<p>Follow <i>Run > Edit Configurations > + > Remote</i> to open a
default Remote Configuration template:
+<img src="/images/intellij_remote_debug_configuration.png" style="width: 75%;
max-width: 660px;" /></p>
+
+<p>Normally, the default values should be good enough to use. Make sure that
you choose <b>Listen to remote JVM</b>
+as <i>Debugger mode</i> and select the right JDK version to generate proper
<i>Command line arguments for remote JVM</i>.</p>
+
+<p>Once you finish configuration and save it. You can follow <i>Run > Run
> Your_Remote_Debug_Name > Debug</i> to start remote debug
+process and wait for SBT console to connect:</p>
+
+<p><img src="/images/intellij_start_remote_debug.png" style="width: 75%;
max-width: 660px;" /></p>
+
+<h5>Trigger the remote debugging</h5>
+
+<p>In general, there are 2 steps:</p>
+<ol>
+ <li>Set JVM options using the <i>Command line arguments for remote JVM</i>
generated in the last step.</li>
+ <li>Start the Spark execution (SBT test, pyspark test, spark-shell,
etc.)</li>
+</ol>
+
+<p>The following is an example of how to trigger the remote debugging using
SBT unit tests.</p>
+
+<p>Enter in SBT console</p>
+<div class="highlighter-rouge"><div class="highlight"><pre
class="highlight"><code>./build/sbt
+</code></pre></div></div>
+<p>Switch to project where the target test locates, e.g.:</p>
+<div class="highlighter-rouge"><div class="highlight"><pre
class="highlight"><code>sbt > project core
+</code></pre></div></div>
+<p>Copy pasting the <i>Command line arguments for remote JVM</i></p>
+<div class="highlighter-rouge"><div class="highlight"><pre
class="highlight"><code>sbt > set javaOptions in Test +=
"-agentlib:jdwp=transport=dt_socket,server=n,suspend=n,address=localhost:5005"
+</code></pre></div></div>
+<p>Set breakpoints with IntelliJ and run the test with SBT, e.g.:</p>
+<div class="highlighter-rouge"><div class="highlight"><pre
class="highlight"><code>sbt > testOnly *SparkContextSuite -- -t "Only one
SparkContext may be active at a time"
+</code></pre></div></div>
+
+<p>It should be successfully connected to IntelliJ when you see
“Connected to the target VM,
+address: ‘localhost:5005’, transport: ‘socket’”
in IntelliJ console. And then, you can start
+debug in IntelliJ as usual.</p>
+
+<p>To exit remote debug mode (so that you don’t have to keep starting
the remote debugger),
+type “session clear” in SBT console while you’re in a
project.</p>
<h4>Eclipse</h4>
diff --git a/site/images/intellij_remote_debug_configuration.png
b/site/images/intellij_remote_debug_configuration.png
new file mode 100644
index 0000000..adb346f
Binary files /dev/null and
b/site/images/intellij_remote_debug_configuration.png differ
diff --git a/site/images/intellij_start_remote_debug.png
b/site/images/intellij_start_remote_debug.png
new file mode 100644
index 0000000..32152b3
Binary files /dev/null and b/site/images/intellij_start_remote_debug.png differ
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org
