Re: Equivalent of Jupyter %run

2020-01-11 Thread Dave Boyd
ucas. [1] https://spark.apache.org/docs/latest/api/python/pyspark.html?highlight=addpyfile#pyspark.SparkContext.addPyFile -Original Message- From: Dave Boyd mailto:db...@incadencecorp.com>> Sent: 09 January 2020 17:44 To: users@zeppelin.apache.org<mailto:users@zeppelin.apache.org

Equivalent of Jupyter %run

2020-01-09 Thread Dave Boyd
I have googled this but don't see a solution. We are working on a project where we want to have some common python functions shared between notes. In Jupyter we would just do a %run.  Is there an equivelent in Zeppelin? Is there a way to store files as .py files that zeppelin can find for impor

Re: Adding jars to Spark 2.4.0 under yarn cluster mode in Zeppelin 0.8.1

2019-04-07 Thread Dave Boyd
From the connection refused message I wonder if it is an SSL error. I note none of the information for SSL (truststore, keystore, etc.) I would think the YARN cluster requires some form of authentication. On 4/7/19 9:27 AM, Jeff Zhang wrote: It looks like the interpreter process can not connect

Re: Spark issue moving from local to yarn-client

2019-03-14 Thread Dave Boyd
Ok, this had more information: > INFO [2019-03-15 02:00:46,364] ({pool-2-thread-3} > Logging.scala[logInfo]:54) - Logging events to > hdfs:///var/log/spark/applicationHistory/application_1551287663522_0145 > ERROR [2019-03-15 02:00:46,366] ({SparkListenerBus} > Logging.scala[logError]:91) - unc

Re: Spark issue moving from local to yarn-client

2019-03-14 Thread Dave Boyd
If sparksql is running without problems, check the zeppelin and spark on yarn issues. Also, what do you use for zeppelin-0.7.4? zeppelin-0.8.2? Is it a branch that you maintain yourself? 在 2019年3月15日,上午6:31,Dave Boyd mailto:db...@incadencecorp.com>> 写道: All: I have some code t

Spark issue moving from local to yarn-client

2019-03-14 Thread Dave Boyd
All: I have some code that worked fine in Zeppelin 0.7.4 but I am having issues in 0.8.2 when going from spark master of local to yarn-client. Yarn client worked in 0.7.4. When my master is set to local[*] it runs just fine. However, as soon as I switch to yarn-client I get the Cannot cal