Hi Gourav, Hi Alex, you can try this
https://github.com/zalando/spark-appliance
this docker image (registry.opensource.zalan.do/bi/spark:1.6.0-1) is
integrated with Jupyter notebook, plugins (kernels) for spark and R are
installed, some python libs, like NumPy,SciPy and matplotlib are already
installed.
you can try locally with
docker run -d --net=host \
-e START_MASTER="true" \
-e START_WORKER="true" \
-e START_WEBAPP="true" \
-e START_NOTEBOOK="true" \
registry.opensource.zalan.do/bi/spark:1.6.0-1
then access the notebook: http://localhost:8888/
using docker env variables EXT_JARS and PYTHON_LIBS you can even install
extra python libraries, or attaching jars into spark cluster, such as:
docker run -d --net=host \
-e START_MASTER="true" \
-e START_WORKER="true" \
-e START_WEBAPP="true" \
-e START_NOTEBOOK="true" \
-e PYTHON_LIBS="pandas,scikit-learn"
-e
EXT_JARS="s3://zalando-spark/spark-csv_2.11-1.3.0.jar,s3://zalando-spark/json-serde-1.3.6-jar-with-dependencies.jar"
registry.opensource.zalan.do/bi/spark:1.6.0-1
and it is also easy to be deployed on AWS.
have fun :)
2016-02-16 15:39 GMT+01:00 Gourav Sengupta <gourav.sengu...@gmail.com>:
> take a look here as well http://zeppelin-project.org/ it executes Scala
> and Python and Markup document in the same notebook and draws beautiful
> visualisations as well. It comes built in AWS EMR as well.
>
>
> Regards,
> Gourav
>
> On Tue, Feb 16, 2016 at 12:43 PM, Aleksandr Modestov <
> aleksandrmodes...@gmail.com> wrote:
>
>> Thank you!
>> I will test Spark Notebook.
>>
>> On Tue, Feb 16, 2016 at 3:37 PM, andy petrella <andy.petre...@gmail.com>
>> wrote:
>>
>>> Hello Alex!
>>>
>>> Rajeev is right, come over the spark notebook gitter room, you'll be
>>> helped by many experienced people if you have some troubles:
>>> https://gitter.im/andypetrella/spark-notebook
>>>
>>> The spark notebook has many integrated, reactive (scala) and extendable
>>> (scala) plotting capabilities.
>>>
>>> cheers and have fun!
>>> andy
>>>
>>> On Tue, Feb 16, 2016 at 1:04 PM Rajeev Reddy <rajeev.redd...@gmail.com>
>>> wrote:
>>>
>>>> Hello,
>>>>
>>>> Let me understand your query correctly.
>>>>
>>>> Case 1. You have a jupyter installation for python and you want to use
>>>> it for scala.
>>>> Solution: You can install kernels other than python Ref
>>>> <https://github.com/ipython/ipython/wiki/IPython-kernels-for-other-languages>
>>>>
>>>> Case 2. You want to use spark scala
>>>> Solution: You can create a notebook config in which you create spark
>>>> context and inject it back to your notebook OR Install other kind of
>>>> notebooks like spark-notebook
>>>> <https://github.com/andypetrella/spark-notebook/> or apache zeppelin
>>>> <https://zeppelin.incubator.apache.org/>
>>>>
>>>>
>>>> According to my experience for case 2. I have been using and prefer
>>>> spark notebook over zeppelin
>>>>
>>>>
>>>> On Tue, Feb 16, 2016 at 4:49 PM, AlexModestov <
>>>> aleksandrmodes...@gmail.com> wrote:
>>>>
>>>>> Hello!
>>>>> I want to use Scala from Jupyter (or may be something else if you could
>>>>> recomend anything. I mean an IDE). Does anyone know how I can do this?
>>>>> Thank you!
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> View this message in context:
>>>>> http://apache-spark-user-list.1001560.n3.nabble.com/Scala-from-Jupyter-tp26234.html
>>>>> Sent from the Apache Spark User List mailing list archive at
>>>>> Nabble.com.
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
>>>>> For additional commands, e-mail: user-h...@spark.apache.org
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Thanks,
>>>> Rajeev Reddy
>>>> Software Development Engineer 1
>>>> IXP - Information Intelligence (I2) Team
>>>> Flipkart Internet Pvt. Ltd (flipkart.com)
>>>> http://rajeev-reddy.com
>>>> +91-8001618957
>>>>
>>> --
>>> andy
>>>
>>
>>
>
