Pass that debug string to your executor like this: --conf spark.executor.extraJavaOptions="-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address= 7761". When your executor is launched it will send debug information on port 7761. When you attach the Eclipse debugger, you need to have the IP/host of that executor as the remote process to attach to. This is pretty standard "remote debugging" procedure .
In the interest of sanity I'd recommend you run with just 1 executor when you're debugging...makes life easier (of course the easiest thing is if you can do the debugging in local[*] mode but some bugs don't manifest that way) On Thu, Jun 25, 2015 at 1:17 AM, nitinkalra2000 <[email protected]> wrote: > I am trying to debug Spark application running on eclipse in > clustered/distributed environment but not able to succeed. Application is > java based and I am running it through Eclipse. Configurations to spark for > Master/worker is provided through Java only. > > Though I can debug the code on driver side but as the code flow moves in > Spark(i.e call to .map(..)), the debugger doesn't stop. Because that code > is > running in Workers JVM. > > Is there anyway I can achieve this ? > > I have tried giving following configurations in Tomcat through Eclipse : > -Xdebug -Xrunjdwp:server=y,transport=dt_socket,address=7761,suspend=n > > and setting respective port in Debug->remote java application. > > But after these settings I get the error: Failed to connect to remote VM. > Connection Refused > > Note: I have tried this on Windows as well as on Linux(CentOs) environment. > > If anybody has any solution to this, please help. > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/Debugging-Apache-Spark-clustered-application-from-Eclipse-tp23483.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > >
