Re: Using inteliJ for spark development

[Eran Witkon]

Thanks, so based on that article, should I use sbt or maven? Or either?
Eran
On Wed, 23 Dec 2015 at 13:05 Akhil Das  wrote:

> You will have to point to your spark-assembly.jar since spark has a lot of
> dependencies. You can read the answers discussed over here to have a better
> understanding
> http://stackoverflow.com/questions/3589562/why-maven-what-are-the-benefits
>
> Thanks
> Best Regards
>
> On Wed, Dec 23, 2015 at 4:27 PM, Eran Witkon  wrote:
>
>> Thanks, all of these examples shows how to link to spark source and build
>> it as part of my project. why should I do that? why not point directly to
>> my spark.jar?
>> Am I missing something?
>> Eran
>>
>> On Wed, Dec 23, 2015 at 9:59 AM Akhil Das 
>> wrote:
>>
>>> 1. Install sbt plugin on IntelliJ
>>> 2. Create a new project/Import an sbt project like Dean suggested
>>> 3. Happy Debugging.
>>>
>>> You can also refer to this article for more information
>>> https://cwiki.apache.org/confluence/display/SPARK/Useful+Developer+Tools#UsefulDeveloperTools-IntelliJ
>>>
>>> Thanks
>>> Best Regards
>>>
>>> On Mon, Dec 21, 2015 at 10:51 PM, Eran Witkon 
>>> wrote:
>>>
 Any pointers how to use InteliJ for spark development?
 Any way to use scala worksheet run like spark- shell?

>>>
>>>
>

Re: Using inteliJ for spark development

[Akhil Das]

Both are similar, give both a go and choose the one you like.
On Dec 23, 2015 7:55 PM, "Eran Witkon"  wrote:

> Thanks, so based on that article, should I use sbt or maven? Or either?
> Eran
> On Wed, 23 Dec 2015 at 13:05 Akhil Das  wrote:
>
>> You will have to point to your spark-assembly.jar since spark has a lot
>> of dependencies. You can read the answers discussed over here to have a
>> better understanding
>> http://stackoverflow.com/questions/3589562/why-maven-what-are-the-benefits
>>
>> Thanks
>> Best Regards
>>
>> On Wed, Dec 23, 2015 at 4:27 PM, Eran Witkon 
>> wrote:
>>
>>> Thanks, all of these examples shows how to link to spark source and
>>> build it as part of my project. why should I do that? why not point
>>> directly to my spark.jar?
>>> Am I missing something?
>>> Eran
>>>
>>> On Wed, Dec 23, 2015 at 9:59 AM Akhil Das 
>>> wrote:
>>>
 1. Install sbt plugin on IntelliJ
 2. Create a new project/Import an sbt project like Dean suggested
 3. Happy Debugging.

 You can also refer to this article for more information
 https://cwiki.apache.org/confluence/display/SPARK/Useful+Developer+Tools#UsefulDeveloperTools-IntelliJ

 Thanks
 Best Regards

 On Mon, Dec 21, 2015 at 10:51 PM, Eran Witkon 
 wrote:

> Any pointers how to use InteliJ for spark development?
> Any way to use scala worksheet run like spark- shell?
>


>>

Re: Using inteliJ for spark development

[Eran Witkon]

Thanks, all of these examples shows how to link to spark source and build
it as part of my project. why should I do that? why not point directly to
my spark.jar?
Am I missing something?
Eran

On Wed, Dec 23, 2015 at 9:59 AM Akhil Das 
wrote:

> 1. Install sbt plugin on IntelliJ
> 2. Create a new project/Import an sbt project like Dean suggested
> 3. Happy Debugging.
>
> You can also refer to this article for more information
> https://cwiki.apache.org/confluence/display/SPARK/Useful+Developer+Tools#UsefulDeveloperTools-IntelliJ
>
> Thanks
> Best Regards
>
> On Mon, Dec 21, 2015 at 10:51 PM, Eran Witkon 
> wrote:
>
>> Any pointers how to use InteliJ for spark development?
>> Any way to use scala worksheet run like spark- shell?
>>
>
>

Re: Using inteliJ for spark development

[Akhil Das]

You will have to point to your spark-assembly.jar since spark has a lot of
dependencies. You can read the answers discussed over here to have a better
understanding
http://stackoverflow.com/questions/3589562/why-maven-what-are-the-benefits

Thanks
Best Regards

On Wed, Dec 23, 2015 at 4:27 PM, Eran Witkon  wrote:

> Thanks, all of these examples shows how to link to spark source and build
> it as part of my project. why should I do that? why not point directly to
> my spark.jar?
> Am I missing something?
> Eran
>
> On Wed, Dec 23, 2015 at 9:59 AM Akhil Das 
> wrote:
>
>> 1. Install sbt plugin on IntelliJ
>> 2. Create a new project/Import an sbt project like Dean suggested
>> 3. Happy Debugging.
>>
>> You can also refer to this article for more information
>> https://cwiki.apache.org/confluence/display/SPARK/Useful+Developer+Tools#UsefulDeveloperTools-IntelliJ
>>
>> Thanks
>> Best Regards
>>
>> On Mon, Dec 21, 2015 at 10:51 PM, Eran Witkon 
>> wrote:
>>
>>> Any pointers how to use InteliJ for spark development?
>>> Any way to use scala worksheet run like spark- shell?
>>>
>>
>>

Re: Using inteliJ for spark development

[Akhil Das]

1. Install sbt plugin on IntelliJ
2. Create a new project/Import an sbt project like Dean suggested
3. Happy Debugging.

You can also refer to this article for more information
https://cwiki.apache.org/confluence/display/SPARK/Useful+Developer+Tools#UsefulDeveloperTools-IntelliJ

Thanks
Best Regards

On Mon, Dec 21, 2015 at 10:51 PM, Eran Witkon  wrote:

> Any pointers how to use InteliJ for spark development?
> Any way to use scala worksheet run like spark- shell?
>

Using inteliJ for spark development

[Eran Witkon]

Any pointers how to use InteliJ for spark development?
Any way to use scala worksheet run like spark- shell?