Re: Error about PySpark

2017-02-02 Thread Jianfeng (Jeff) Zhang
Please try to install numpy Best Regard, Jeff Zhang From: mingda li > Reply-To: "users@zeppelin.apache.org" > Date: Friday, February 3, 2017 at

Re: Connection refused when trying to run livy.spark interpterer in kerberized HDP

2017-01-28 Thread Jianfeng (Jeff) Zhang
>>> But when I enable user impersonation on ivy.spark interpterer Do you mean you enable the option of user impersonation on livy interpreter setting of zeppelin ? You don't need to do that as impersonation is done by livy, the option there is a little confusing, it should not be displayed.

Re: 0.6.1 and spark 2.0.0

2016-09-16 Thread Jianfeng (Jeff) Zhang
This is a known issue. There’s 2 solutions. 1. Use the spark 2.0 of HDP 2.5 2. Disable timeline service in yarn-site.xml Best Regard, Jeff Zhang From: Herman Yu > Reply-To: "users@zeppelin.apache.org"

Re: No active SparkContext black hole

2016-10-07 Thread Jianfeng (Jeff) Zhang
Could you paste the log ? Best Regard, Jeff Zhang From: Mark Libucha > Reply-To: "users@zeppelin.apache.org" > Date: Friday, October 7, 2016 at 12:11 AM

Re: Having issues with Hive RuntimeException in runing Zeppelin notebook applicaiton

2016-11-16 Thread Jianfeng (Jeff) Zhang
>>> at /home/asif/zeppelin-0.6.2-bin-all/metastore_db has an incompatible >>> format with the current version of the software. The database was created >>> by or upgraded by version 10.11. Try to delete this folder and rerun it again. Best Regard, Jeff Zhang From: moon soo Lee

Re: Unable to print("Hello World") using %python :(

2016-12-10 Thread Jianfeng (Jeff) Zhang
This exception is thrown from spark side. I suspect you set them. Please grep them in SPARK_CONF_DIR grep -nr SPARK_CLASSPATH SPARK_CONF_DIR grep -nr spark.driver.extraClassPath SPARK_CONF_DIR Best Regard, Jeff Zhang From: Russell Jurney

Re: Zeppeline becomes too slow after adding artifacts

2016-12-06 Thread Jianfeng (Jeff) Zhang
Maybe spark interpreter is downloading the dependency. Could rerun the paragraph again ? Best Regard, Jeff Zhang On 12/7/16, 4:38 AM, "Nabajyoti Dash" wrote: >Hi, >I am using zeppelin to visualize my hbase data. >I had a sample zeppelin notebook, which was

Re: Limit on multiple concurrent interpreters / isolated notebooks?

2016-12-14 Thread Jianfeng (Jeff) Zhang
Yes, this would be a critical performance issue for multiple user case. Because currently zeppelin only support yarn-client mode which means the driver JVM is in the same host of zeppelin server. So regarding the concurrent users, it depends on the memory you configure for the driver and how many

Re: Unable to print("Hello World") using %python :(

2016-12-11 Thread Jianfeng (Jeff) Zhang
apache.org>> Subject: Re: Unable to print("Hello World") using %python :( Thanks, I've already gone through and checked my spark config. SPARK_CLASSPATH isn't set, and I spark-submit all the time. On Sat, Dec 10, 2016 at 5:06 PM Jianfeng (Jeff) Zhang <jzh...@hortonworks.com<mai

Re: Job failed: Implementing class

2016-12-07 Thread Jianfeng (Jeff) Zhang
Do you use zeppelin binary distribution or build it by yourself ? Best Regard, Jeff Zhang From: Facundo Bianco > Reply-To: "users@zeppelin.apache.org"

Re: Zeppelin out of memory issue - (GC overhead limit exceeded)

2017-03-26 Thread Jianfeng (Jeff) Zhang
This is a bug of zeppelin. spark.driver.memory won't take effect. As for now it isn't passed to spark through -conf parameter. See https://issues.apache.org/jira/browse/ZEPPELIN-1263 The workaround is to specify SPARK_DRIVER_MEMORY in interpreter setting page. Best Regard, Jeff Zhang From:

Re: Zeppelin out of memory issue - (GC overhead limit exceeded)

2017-03-26 Thread Jianfeng (Jeff) Zhang
have read that GC overhead limit exceeded error occurs when the heap memory is insufficient. So How can I increase the heap memory. Please correct me if I am wrong as I am still trying to learn these things. Reagrds, Rushikesh Raut On Sun, Mar 26, 2017 at 4:25 PM, Jianfeng (Jeff) Zhang <jzh..

Re: Setting Zeppelin to work with multiple Hadoop clusters when running Spark.

2017-03-26 Thread Jianfeng (Jeff) Zhang
"users@zeppelin.apache.org<mailto:users@zeppelin.apache.org>" <users@zeppelin.apache.org<mailto:users@zeppelin.apache.org>> Subject: Re: Setting Zeppelin to work with multiple Hadoop clusters when running Spark. I know it, thanks, but it's non reliable solution. 2017-03-26 5:23

Re: Should zeppelin.pyspark.python be used on the worker nodes ?

2017-03-20 Thread Jianfeng (Jeff) Zhang
It is dynamic, you can set enviroment variable in interpreter setting page. Best Regard, Jeff Zhang From: Ruslan Dautkhanov > Reply-To: "users@zeppelin.apache.org"

Re: Roadmap for 0.8.0

2017-03-20 Thread Jianfeng (Jeff) Zhang
Strongly +1 for adding system test for different interpreter modes and focus on bug fixing than new features. I do heard from some users complain about the bugs of zeppelin major release. A stabilized release is very necessary for community. Best Regard, Jeff Zhang From: moon soo Lee

Re: sqlContext not avilable as hiveContext in notebook

2017-04-04 Thread Jianfeng (Jeff) Zhang
I could get hiveContext correctly in zeppelin master, could you try 0.7.1 which is released recently ? Best Regard, Jeff Zhang From: Meethu Mathew > Reply-To: "users@zeppelin.apache.org"

Re: java.lang.NullpointerException

2017-04-23 Thread Jianfeng (Jeff) Zhang
The message is clear that you are setting both spark.driver.extraJavaOptions and SPARK_JAVA_OPTS. Please check spark-defaults.conf and interpreter setting. Best Regard, Jeff Zhang From: kant kodali > Reply-To:

Re: Release on 0.7.1 and 0.7.2

2017-03-14 Thread Jianfeng (Jeff) Zhang
+1 Best Regard, Jeff Zhang From: Jun Kim > Reply-To: "users@zeppelin.apache.org" > Date: Tuesday, March 14, 2017 at 11:38 AM To:

Re: java.lang.ClassNotFoundException: $anonfun$1

2017-03-07 Thread Jianfeng (Jeff) Zhang
>>> It appears that during execution time on the yarn hosts, the native CDH >>> spark1.5 jars are loaded before the new spark2 jars. I've tried using >>> spark.yarn.archive to specify the spark2 jars in hdfs as well as using >>> other spark options, none of which seems to make a difference.

Re: Zeppelin postgres interpreter throws NPE

2017-08-03 Thread Jianfeng (Jeff) Zhang
Agree, you can create a ticket for better exception message. And the error is better to be displayed in frontend Best Regard, Jeff Zhang From: Richard Xin > Reply-To: "users@zeppelin.apache.org"

Re: can Zeppelin runParagraph from different notebooks?

2017-06-26 Thread Jianfeng (Jeff) Zhang
elin runParagraph from different notebooks? %spark run("2CN3UDXMZ", "20170609-233658_1498522009"); (which is spark script written in scala) :288: error: not found: value run run("2CN3UDXMZ", "20170609-233658_1498522009"); ^ Zeppelin Version 0.7.1

Re: InvalidClassException using Zeppelin (master) and spark-2.1 on a standalone spark cluster

2017-06-27 Thread Jianfeng (Jeff) Zhang
It is fixed in https://issues.apache.org/jira/browse/ZEPPELIN-1977 Best Regard, Jeff Zhang On 6/27/17, 12:46 PM, "David Howell" wrote: >Hi, >I know this issue is resolved for reading from json, and tested for that >use >case, but I'm seeing the exact same

Re: unable to restart zeppelin on AWS EMR

2017-06-04 Thread Jianfeng (Jeff) Zhang
Please check the zeppelin log Best Regard, Jeff Zhang From: shyla deshpande > Reply-To: "users@zeppelin.apache.org" > Date: Sunday, June 4,

Re: Unable to run Zeppelin Spark on YARN

2017-05-04 Thread Jianfeng (Jeff) Zhang
Could you try set yarn-client in interpreter setting page ? Best Regard, Jeff Zhang From: Yeshwanth Jagini > Reply-To: "users@zeppelin.apache.org"

Re: InvalidClassException using Zeppelin (master) and spark-2.1 on a standalone spark cluster

2017-06-27 Thread Jianfeng (Jeff) Zhang
;I am running Zeppelin 0.7.1 , and yes it is fixed for reading from json, >but it still throws an error for writing to json. > >See my repro example. > > >-----Original Message- >From: Jianfeng (Jeff) Zhang [mailto:jzh...@hortonworks.com] >Sent: Tuesday, 27

Re: Zeppelin Port Configuration

2017-09-13 Thread Jianfeng (Jeff) Zhang
What do you see in logs ? Best Regard, Jeff Zhang From: Carlos Andres Zambrano Barrera > Reply-To: "users@zeppelin.apache.org" > Date: Thursday, September

Re: shell interpreter variables

2017-09-22 Thread Jianfeng (Jeff) Zhang
This is due to the implementation of shell interpreter. Each paragraph will launch a shell process, which means each paragraph are in separated shell sessions. Best Regard, Jeff Zhang From: Mohit Jaggi > Reply-To:

Re: Zeppelin Port Configuration

2017-09-13 Thread Jianfeng (Jeff) Zhang
aws.com/mailtrack-crx/icon-signature.png]<https://mailtrack.io/>Sent with Mailtrack<https://mailtrack.io/install?source=signature=en=cza...@gmail.com=22> 2017-09-13 14:45 GMT-05:00 Jianfeng (Jeff) Zhang <jzh...@hortonworks.com<mailto:jzh...@hortonworks.com>>: What do you see in logs ?

Re: Implementing run all paragraphs sequentially

2017-10-06 Thread Jianfeng (Jeff) Zhang
Since almost everyone agree on to run serial by default. We could implement it first. Regarding the parallel mode, we could leave it in future although personally I prefer to define DAG for note. Best Regard, Jeff Zhang From: Michael Segel

Re: Is any limitation of maximum interpreter processes?

2017-10-02 Thread Jianfeng (Jeff) Zhang
Which interpreter is pending ? It is possible that spark interpreter pending due to yarn resource capacity if you run it in yarn client mode If it is pending, you can check the log first. Best Regard, Jeff Zhang From: Belousov Maksim Eduardovich

Re: ZeppelinContext run method runs a paragraph as anonymous user

2017-09-13 Thread Jianfeng (Jeff) Zhang
Could you create a ticket for it first ? I suspect it is due to we didn't pass user name in InterpreterContextRunner, but needs more investigation. Best Regard, Jeff Zhang From: Deenar Toraskar > Reply-To:

Re: Zeppelin Interpreter Page Not Showing

2017-09-27 Thread Jianfeng (Jeff) Zhang
Which version of zeppelin do you use ? And could you check the zeppelin server log ? Best Regard, Jeff Zhang From: "Tan, Jialiang" > Reply-To: "users@zeppelin.apache.org"

Re: How to execute spark-submit on Note

2017-10-03 Thread Jianfeng (Jeff) Zhang
I am surprised why would you use %spark-submit, there’s no document about %spark-submit. If you want to use spark-submit in zeppelin, then you could use %sh Best Regard, Jeff Zhang From: 小野圭二 > Reply-To:

Re: Error in combining data from Tajo and MariaDB with Spark and Zeppelin

2017-08-31 Thread Jianfeng (Jeff) Zhang
Have tried that in spark-shell ? Best Regard, Jeff Zhang From: Cinyoung Hur > Reply-To: "users@zeppelin.apache.org" > Date: Friday, September 1,

Re: Trying to 0.7.3 running with Spark

2017-10-07 Thread Jianfeng (Jeff) Zhang
Could you check the log again ? There should be another exception above the exception you pasted. Most likely SparkContext is failed to create. Best Regard, Jeff Zhang From: Terry Healy > Reply-To:

Re: Sequential processing disabled?

2017-11-30 Thread Jianfeng (Jeff) Zhang
:users@zeppelin.apache.org>> Subject: RE: Sequential processing disabled? Jeff, OK makes sense. Is this a global setting? Can it be set per user, or per Notebook? Chris From: Jianfeng (Jeff) Zhang [mailto:jzh...@hortonworks.com] Sent: Thursday, November 30, 2017 9:08 AM To: users@zeppelin.a

Re: Sequential processing disabled?

2017-11-29 Thread Jianfeng (Jeff) Zhang
You can configure zeppelin.jdbc.concurrent.use as false to make jdbc interpreter run sequentially Best Regard, Jeff Zhang From: "Geiss, Chris" > Reply-To: "users@zeppelin.apache.org"

Re: Sequential processing disabled?

2017-11-30 Thread Jianfeng (Jeff) Zhang
l processing disabled? Hi Jeff, Thank you for the reply. So that will result in Impala paragraphs running sequentially? Is that because Impala goes through JDBC? Chris From: Jianfeng (Jeff) Zhang [mailto:jzh...@hortonworks.com] Sent: Wednesday, November 29, 2017 7:03 PM To: users@zeppelin.apache.org

Re: Livy Manager - Web UI for Managing Apache Livy Sessions

2017-12-07 Thread Jianfeng (Jeff) Zhang
Great work, @Keiji, Livy also provide a webUI for managing livy sessions. And welcome to contribute your work to livy. http://livy.incubator.apache.org/ https://github.com/apache/incubator-livy Best Regard, Jeff Zhang From: Keiji Yoshida >

Re: Sequential processing disabled?

2017-12-01 Thread Jianfeng (Jeff) Zhang
> Subject: RE: Sequential processing disabled? Hi Jeff, Does that mean we can have two Impala interpreters configured? One with sequential processing and one with parallel processing? Chris From: Jianfeng (Jeff) Zhang [mailto:jzh...@hortonworks.com] Sent: Thursday, November 30, 2017 7:08

Re: All PySpark jobs are canceled when one user cancel his PySpark paragraph (job)

2018-06-12 Thread Jianfeng (Jeff) Zhang
Which version do you use ? Best Regard, Jeff Zhang From: Jhon Anderson Cardenas Diaz mailto:jhonderson2...@gmail.com>> Reply-To: "users@zeppelin.apache.org" mailto:users@zeppelin.apache.org>> Date: Friday, June 8, 2018 at 11:08 PM To:

Re: Zeppelin 0.8

2018-06-07 Thread Jianfeng (Jeff) Zhang
I am doing the release, the latest RC4 is canceled, I will start RC5 in the next few days. Best Regard, Jeff Zhang From: Benjamin Kim mailto:bbuil...@gmail.com>> Reply-To: "users@zeppelin.apache.org" mailto:users@zeppelin.apache.org>> Date: Thursday, June

Re: Spark Error replayPlot(x): could not open file

2018-04-18 Thread Jianfeng (Jeff) Zhang
Do you mind to try zeppelin 0.8 branch ? You need to build it from source Best Regard, Jeff Zhang From: "Joe W. Byers" > Reply-To: "users@zeppelin.apache.org"

Re: [ANNOUNCE] Apache Zeppelin 0.8.0 released

2018-06-28 Thread Jianfeng (Jeff) Zhang
Hi Patrick, Which link is broken ? I can access all the links. Best Regard, Jeff Zhang From: Patrick Maroney mailto:pmaro...@wapacklabs.com>> Reply-To: mailto:users@zeppelin.apache.org>> Date: Friday, June 29, 2018 at 4:59 AM To: mailto:users@zeppelin.apache.org>> Cc: dev