I think as long as you set the master in the code as the correct cluster URL, everything should work as expected. So you should be able to place breakpoints in IntelliJ as if running in local mode.
Get Outlook for iOS<https://aka.ms/o0ukef> ________________________________ From: Pat Ferrel <p...@occamsmachete.com> Sent: Tuesday, April 7, 2020 10:58:39 PM To: Luiz Camargo <camar...@gmail.com>; Dennis Suhari <d.suh...@icloud.com.invalid> Cc: zahidr1...@gmail.com <zahidr1...@gmail.com>; user@spark.apache.org <user@spark.apache.org>; yeikel valdes <em...@yeikel.com> Subject: Re: IDE suitable for Spark IntelliJ Scala works well when debugging master=local. Has anyone used it for remote/cluster debugging? I’ve heard it is possible... From: Luiz Camargo <camar...@gmail.com><mailto:camar...@gmail.com> Reply: Luiz Camargo <camar...@gmail.com><mailto:camar...@gmail.com> Date: April 7, 2020 at 10:26:35 AM To: Dennis Suhari <d.suh...@icloud.com.invalid><mailto:d.suh...@icloud.com.invalid> Cc: yeikel valdes <em...@yeikel.com><mailto:em...@yeikel.com>, zahidr1...@gmail.com <zahidr1...@gmail.com><mailto:zahidr1...@gmail.com>, user@spark.apache.org<user@spark.apache.org><mailto:user@spark.apache.org> Subject: Re: IDE suitable for Spark I have used IntelliJ Spark/Scala with the sbt tool On Tue, Apr 7, 2020 at 1:18 PM Dennis Suhari <d.suh...@icloud.com.invalid> wrote: We are using Pycharm resp. R Studio with Spark libraries to submit Spark Jobs. Von meinem iPhone gesendet Am 07.04.2020 um 18:10 schrieb yeikel valdes <em...@yeikel.com<mailto:em...@yeikel.com>>: Zeppelin is not an IDE but a notebook. It is helpful to experiment but it is missing a lot of the features that we expect from an IDE. Thanks for sharing though. ---- On Tue, 07 Apr 2020 04:45:33 -0400 zahidr1...@gmail.com<mailto:zahidr1...@gmail.com>wrote ---- When I first logged on I asked if there was a suitable IDE for Spark. I did get a couple of responses. Thanks. I did actually find one which is suitable IDE for spark. That is Apache Zeppelin. One of many reasons it is suitable for Apache Spark is. The up and running Stage which involves typing bin/zeppelin-daemon.sh start Go to browser and type http://localhost:8080 That's it! Then to Hit the ground running There are also ready to go Apache Spark examples showing off the type of functionality one will be using in real life production. Zeppelin comes with embedded Apache Spark and scala as default interpreter with 20 + interpreters. I have gone on to discover there are a number of other advantages for real time production environment with Zeppelin offered up by other Apache Products. Backbutton.co.uk<http://Backbutton.co.uk> ¯_(ツ)_/¯ ♡۶Java♡۶RMI ♡۶ Make Use Method {MUM} makeuse.org<http://makeuse.org> <http://www.backbutton.co.uk> -- Prof. Luiz Camargo Educador - Computação
