JL and Moon,
Thank you for your responses below! I would second JL's observation.
Creating a new interpreter via the GUI allows this customization but it
would be nice to be able to just point at a configuration file (the same
configuration file used to run spark-shell) and maybe add that to the
default interpreter.
Thank you,
Markus
On 02/22/2015 10:09 PM, Jongyoul Lee wrote:
Hi Markus,
I also cannot find to pass spark configurations to zeppelin easily,
but you can do it with JAVA_OPTS. In your case,
JAVA_OPTS="-Dspark.serializer=org.apache.spark.serializer.KryoSerializer
-Dspark.kryo.registrator=com.uebercomputing.mailrecord.MailRecordRegistrator"
Hi Moon,
Zeppelin needs to read spark-default.conf, adds a configuration option
to set spark-default.conf, or allow set spark.* to zeppelin-site.xml.
Please give us your opinion.
Regards,
JL
On Mon, Feb 23, 2015 at 10:24 AM, moon soo Lee <[email protected]
<mailto:[email protected]>> wrote:
Hi Markus,
I think following directories
conf/ (basic configurations, interpreter configurations)
notebook/ (notebooks)
interpreter/ (if you have any 3rd party interpreter installed)
are things to take care when move to next Zeppelin version.
Thanks
moon
On Fri, Feb 20, 2015 at 12:57 AM, M. Dale <[email protected]
<mailto:[email protected]>> wrote:
Anthony,
That did the trick! Thank you so much for your quick reply.
It seems that conf/interpreter.json got changed and now saves
my new interpreter. When I move to the next version of
Zeppelin, is that what I need to copy to keep that interpreter
as an option?
Thanks again for your help,
Markus
On 02/19/2015 10:46 AM, Anthony Corbacho wrote:
hello Markus,
in Zeppelin you can configure those option through the
interpreters.
basically, you will have to create a new interpreter and set
the serialization options (you will see that you can add
custom options to spark interpreter) and then activate this
interpreter in your notebook.
hope it help.
On Feb 20, 2015 12:41 AM, "M. Dale" <[email protected]
<mailto:[email protected]>> wrote:
I would like to use Spark with Kryo serialization.
From the command line spark-shell I would add:
--conf
spark.serializer=org.apache.spark.serializer.KryoSerializer \
--conf
spark.kryo.registrator=com.uebercomputing.mailrecord.MailRecordRegistrator
\
Or I can add those to conf/spark-defaults.conf:
spark.serializer=org.apache.spark.serializer.KryoSerializer
spark.kryo.registrator=com.uebercomputing.mailrecord.MailRecordRegistrator
How can I set these properties for the Zeppelin-provided
spark context?
Thanks for your help,
Markus
--
이종열, Jongyoul Lee, 李宗烈
http://madeng.net
