Re: Zepplin 0.7.3 DEBUG console output, how do I switch it off!!! Spark 2.2.0-bin-hadoop2.7

[Jeff Zhang]

I just suspect it is due to different spark configuration in zeppelin spark
interpreter and spark-shell. Please make sure they are using the same
configuration.


Amer Sheikh 于2017年10月14日周六 下午1:44写道：

> Hi Jeff
>
> Thanks for your reply. The zeppelin console output is continuous when my
> scala code (spark sql) is running.
> The same code runs very from a separate spark-shell, so im suspecting that
> Zeppelin is executing the code in DEBUG mode.
>
> When I installed Zeppelin, I did not change any settings.  My main concern
> is the very slow performance.
>
> Kind Regards
>
> Amer
>
> On Sat, Oct 14, 2017 at 2:16 AM, Jianfeng (Jeff) Zhang <
> jzh...@hortonworks.com> wrote:
>
>>
>> These are zeppelin server log which is controlled by log4.properties
>> under ZEPPELIN_CONF_DIR. And I don’t understand why would you say these
>> log would affect the performance.
>>
>> Could you explain more ?
>>
>>
>>
>>
>> Best Regard,
>> Jeff Zhang
>>
>>
>> From: Amer Sheikh 
>> Reply-To: "users@zeppelin.apache.org" 
>> Date: Saturday, October 14, 2017 at 12:15 AM
>> To: "users@zeppelin.apache.org" 
>> Subject: Re: Zepplin 0.7.3 DEBUG console output, how do I switch it
>> off!!! Spark 2.2.0-bin-hadoop2.7
>>
>> Hi there
>>
>> Please see the console output
>>
>>
>>
>> On Thu, Oct 12, 2017 at 1:25 PM, Jeff Zhang  wrote:
>>
>>>
>>> What kind of DEBUG log you see ? Can you paste it ?
>>>
>>>
>>> amersh...@gmail.com 于2017年10月12日周四 下午5:44写道：
>>>


 On 2017-10-12 10:41, "amersh...@gmail.com" wrote:
 > Hello there
 >
 > I have installed Spark 2.20 and from the spark-shell, my scala code
 works perfectly and performs well.
 >
 > The same scala executed from Zeppelin causes lots of DEBUG statements
 to be printed in the console (almost non-stop) and the performance is very
 slow.
 >
 > I have just installed binaries and haven't changed any
 configurations, but it seems Zeppelin is running in debug mode...
 >
 > Help..
 >
 > I use bin\zeppelin.cmd to start zeppelin. I'm on Windows Server 2012
 >
 > Please help...
 >
 >
 > PS

 The scala works perfectly within Zeppelin... Just very slow performance
 and lots of Debug output...

>>>
>>
>

Re: run zeppelin notebook from outside Zeppelin UI or programmatically

[Jeff Zhang]

Check notebook rest api

https://zeppelin.apache.org/docs/0.8.0-SNAPSHOT/usage/rest_api/notebook.html


Richard Xin 于2017年10月14日周六 上午8:08写道：

> Hi,
> is there a recommended way to run zeppelin notebook from outside Zeppelin
> UI or command-line or programmatically?
>
> I found one post, but looks pretty hacky to me.
>
> How to run zeppelin notebook from command line (automatically)
> 
>
> How to run zeppelin notebook from command line (automatically)
>
> How do we run the notebook from command line? Further to 1, how would I
> pass command line arguments into the not...
>
> 
>
>
> Thanks,
>
>

run zeppelin notebook from outside Zeppelin UI or programmatically

[Richard Xin]

Hi, is there a recommended way to run zeppelin notebook from outside Zeppelin 
UI or command-line or programmatically? 
I found one post, but looks pretty hacky to me. 
How to run zeppelin notebook from command line (automatically)

| 
| 
| 
|  |  |

 |

 |
| 
|  | 
How to run zeppelin notebook from command line (automatically)

How do we run the notebook from command line? Further to 1, how would I pass 
command line arguments into the not...
 |

 |

 |



Thanks,

Error when trying to build from source?

[Michael Segel]

I’ve been trying to build zeppelin from the source. 
Ran in to the following error:
[INFO] Compiling 6 source files to 
/opt/zeppelin-0.7.3/zeppelin-display/target/classes at 1507925417280
[ERROR] 
/opt/zeppelin-0.7.3/zeppelin-display/src/main/scala/org/apache/zeppelin/display/angular/AbstractAngularElem.scala:25:
 error: object xml is not a member of package scala
[INFO] import scala.xml._
[INFO]  ^
[ERROR] 
/opt/zeppelin-0.7.3/zeppelin-display/src/main/scala/org/apache/zeppelin/display/angular/AbstractAngularElem.scala:39:
 error: not found: type Elem
[INFO]   extends Elem(prefix, label, attributes1, scope, minimizeEmpty, 
child:_*) {
[INFO]   ^

My build command :  mvn -Pmapr51 -Pyarn -Pbuild-distr -Pspark-2.1 -Phadoop-2.7 
-Ppyspark package -DskipTests 

Any ideas? 

Thx in advance…. 

Re: sql paragraph doesn't see my 3rd party jars

[moon soo Lee]

Could you share little more about how did you add 3rd party jars?

ie. through %spark.dep, Interpreter setting menu on GUI, environment
variable, and so on.

Thanks,
moon

On Sat, Oct 7, 2017 at 3:47 AM Serega Sheypak 
wrote:

> Hi, I'm trying to use spark and sql paragraphs with 3rd party jars added
> to spark interpreter configuration.
>
> My spark code works fine.
>
>
> My sql paragraph fails with class not found exception
> %sql
> create external table MY_TABLE row format serde 'com.my.MyAvroSerde'
> with serdeproperties ('serialization.class'='com.my.ContainerProto')
> stored as inputformat 'com.my.ProtoAvroFileFormat'
> LOCATION 'hdfs://my/data'
>
> Exception:
> MetaException(message:org.apache.hadoop.hive.serde2.SerDeException
> java.lang.ClassNotFoundException: Class com.my.ContainerProto not found)
>
>
> It's confusing since spark paragraph works well with
> code
>
> import com.my.ContainerProto
> // bla-bla
> rdd.map{bytes => ContainerProto.fromBytes(bytes)}
>
> Code executed and result produced. Why sql paragraph doesn't see my 3rd
> party jars?
>
>
>
>

Re: question about how to update zeppelin interpreters.

[moon soo Lee]

Hi,

I think id value of interpreter is just randomly generated ID [1]. You can
find ID generation routine here [2].

Thanks,
moon

[1]
https://github.com/apache/zeppelin/blob/v0.7.3/zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/InterpreterSetting.java#L106
[2]
https://github.com/apache/zeppelin/blob/v0.7.3/zeppelin-zengine/src/main/java/org/apache/zeppelin/notebook/utility/IdHashes.java#L74

On Tue, Oct 3, 2017 at 9:46 PM Jeffrey Rodriguez 
wrote:

> Hi folks,
>I would like to upgrade the zeppelin interpreter properties
> programmatically. I found two ways.
> 1. Update the conf/interpreters.json
> 2. User the interpreter REST API.
>
> My question is, should users/developers update the interpreter directly?
>
> id values for interpreters are like "2CVTZCCU4'", which seems to me not
> very ramdom and more like a signature or class hash.
>
> Is the prefer way to maintain consistency, the REST API??
>
> Regards,
>Jeff Rodriguez
>

Re: Zepplin 0.7.3 DEBUG console output, how do I switch it off!!! Spark 2.2.0-bin-hadoop2.7

[Amer Sheikh]

Hi there

Please see the console output



On Thu, Oct 12, 2017 at 1:25 PM, Jeff Zhang  wrote:

>
> What kind of DEBUG log you see ? Can you paste it ?
>
>
> amersh...@gmail.com 于2017年10月12日周四 下午5:44写道：
>
>>
>>
>> On 2017-10-12 10:41, "amersh...@gmail.com" wrote:
>> > Hello there
>> >
>> > I have installed Spark 2.20 and from the spark-shell, my scala code
>> works perfectly and performs well.
>> >
>> > The same scala executed from Zeppelin causes lots of DEBUG statements
>> to be printed in the console (almost non-stop) and the performance is very
>> slow.
>> >
>> > I have just installed binaries and haven't changed any configurations,
>> but it seems Zeppelin is running in debug mode...
>> >
>> > Help..
>> >
>> > I use bin\zeppelin.cmd to start zeppelin. I'm on Windows Server 2012
>> >
>> > Please help...
>> >
>> >
>> > PS
>>
>> The scala works perfectly within Zeppelin... Just very slow performance
>> and lots of Debug output...
>>
>

RE: Zeppelin Stops Loading Notes

[Belousov Maksim Eduardovich]

Paul, Ben, Fabian,
please share your workload at time when notes are not loading.

How much interpreters were started at that moment?

You can find all started interpreters in linux command line with:
ps -ef | grep ZeppelinServer | grep -v grep | awk "NR==1" | awk -F' ' '{print 
$2}' | xargs ps -f --ppid | wc -l

And spark started interpreters:
ps -ef | grep ZeppelinServer | grep -v grep | awk "NR==1" | awk -F' ' '{print 
$2}' | xargs ps -f --ppid | grep spark | wc -l



Максим Белоусов
Архитектор
Отдел отчетности и витрин данных
Управление хранилищ данных и отчетности
Тел.: +7 495 648-10-00, доб. 2271

From: Paul Brenner [mailto:pbren...@placeiq.com]
Sent: Thursday, October 12, 2017 7:45 PM
To: Geoffrey Cheng ; users@zeppelin.apache.org
Subject: Re: Zeppelin Stops Loading Notes

[https://share.polymail.io/v2/z/a/NTlkZjliYjE3MmFm/GBrDc33gW4GiZNTZBS_xV13grE-GeWLFw0ScDUHMwFNKw_JiWRWwck1mDOMthhhFuKERg2yT-GFU9L4aTwWhM7XWYY-r6DJGKBzEDe3VE2RfpzddnNRkp_Zpnb_TXVItyGxSxTvIer3-RVJ3GroFVEMhWGg=.png]
Does this issue need a Jira ticket? The problem is that I have no idea how to 
reproduce and I’m not sure if there is anything in the logs that is relevant.

Any ideas how we can produce an actionable Jira ticket out of this?

[https://ci3.googleusercontent.com/proxy/tFn1I-GEOnccUtv8DHHEc49-6g3x3CbuQKzbfl2Z1BObEy0Qz6QebJimpP96TK3Za5MXwXTuwBZaobKp22nYAG3NdxAC0Q=s0-d-e1-ft#https://marketing.placeiq.net/images/placeiq.png]

Paul Brenner

[https://ci4.googleusercontent.com/proxy/490PXYv9O6OiIp_DL4vuabJqVn53fMon5xNYZdftCVea9ySR2LcFDHe6Cdntb2G68uDAuA6FgLny8wKWLFWpsrPAt_FtLaE=s0-d-e1-ft#https://marketing.placeiq.net/images/twitter1.png]

[https://ci3.googleusercontent.com/proxy/fztHf1lRKLQYcAxebqfp2PYXCwVap3GobHVIbyp0j3NcuJOY16bUAZBibVOFf-fd1GsiuhrOfYy6dSwhlCwWU8ZUlw9OX5I=s0-d-e1-ft#https://marketing.placeiq.net/images/facebook.png]

[https://ci5.googleusercontent.com/proxy/H26ThD7R6DOqxoLTgzi6k5SMrHoF2Tj44xI_7XlD9KfOIiGwe1WIMc5iQBxUBA9EuIyJMdaRXrhZTOrnkrn8O9Rf1FP9UQU=s0-d-e1-ft#https://marketing.placeiq.net/images/linkedin.png]

DATA SCIENTIST

(217) 390-3033



[PlaceIQ:Landmark by 
PlaceIQ]


On Thu, Oct 12, 2017 at 8:31 AM Geoffrey Cheng mailto:geoffrey%20cheng%20%3cgeoffrey.ch...@gmail.com%3e> > wrote:
we have the same issue.  usually when multiple ppl using it, only header loads.

we tried couldn't find solution, so we restart every single time.   in fact ,  
we have to restart daily at least.

On Oct 12, 2017 2:40 AM, "Fabian Böhnlein" 
> wrote:
Hi Paul, Ben,

we also see this happen regularly. It's more likely to happen when a handful of 
people are using it.

We mostly run one spark interpreter per person. We also don't observe anything 
in the logs. The 'header' that you mentioned is actually still in the cache.

Sometimes it's specific notes that don't load.
Sometimes there's a hanging Spark interpreter, once it's killed notes load 
again.

We're pretty clueless about it.

Any front-end related logs we could enable to find out more?

On Sat, 19 Aug 2017 at 20:19 Ben Vogan 
> wrote:
I have seen Zeppelin get into this state once.  I restarted it without 
investigating the logs however so I don't have anything useful to go on as to 
why.

--Ben

On Sat, Aug 19, 2017 at 8:17 AM, Paul Brenner 
> wrote:
You were correct. We had "export ZEPPELIN_SSL_PORT=false” in our 
zeppelin-env.sh. I’m going to comment that out. I suspect it is actually 
unrelated to the behavior we are seeing where pages stop loading though. Anyone 
else see this happen?

I’ll report back if that happens again after the fix.



Paul Brenner





DATA SCIENTIST

(217) 390-3033  





 
On Fri, Aug 18, 2017 at 6:37 PM moon soo Lee  
wrote:
Hi,
 
One of configuration value in your conf/zeppelin-env.sh or 
conf/zeppelin-site.xml seems "false" which expected to be to a 
number.
 
Do you have any environment variable or property set to "false" for the 
configurations below?
 
ZEPPELIN_PORT, zeppelin.server.port
ZEPPELIN_SSL_PORT, zeppelin.server.ssl.port
ZEPPELIN_INTERPRETER_CONNECT_TIMEOUT, 
zeppelin.interpreter.connect.timeout