Re: [DISCUSS] Zeppelin community sync up meeting

[Felix Cheung]

+1


From: Eric Pugh 
Sent: Tuesday, January 26, 2021 5:28:59 AM
To: users 
Subject: Re: [DISCUSS] Zeppelin community sync up meeting

I’d advocate for a regular quarterly run well versus a monthly that ends up 
sparsely attended and a scramble to setup.   The general idea is fantastic.

On Jan 25, 2021, at 10:37 PM, moon soo Lee 
mailto:m...@apache.org>> wrote:

+1  

Looking forward to it!

On Mon, Jan 25, 2021 at 6:20 PM Jeff Zhang 
mailto:zjf...@gmail.com>> wrote:
Hi Bill,

That's right, it has been more than one year we haven't do community sync up. 
I'd like to continue that to bring the community together for more wide 
discussion around zeppelin.

Bill Brooks mailto:wwbro...@gmail.com>> 于2021年1月26日周二 
上午10:11写道：
This is a great idea, but I thought that Mei Long was already putting
these events together and prior to the pandemic, they were happening.

Bill

On Mon, Jan 25, 2021 at 3:09 PM Jeff Zhang 
mailto:zjf...@gmail.com>> wrote:
>
> Hi folks (users & developers),
>
> I'd like to raise this discussion about regular community sync up meeting to 
> discuss various things of Zeppelin. You may know me that I have been active 
> in the Zeppelin community for the last 4 years. I believe Zeppelin is a very 
> promising and important project in the data science and big data area. But it 
> needs more community involvement to improve it ,no matter whether you are 
> zeppelin users or zeppelin developers. I hope this kind of sync up meeting 
> can gather people who are interested in zeppelin together, and let's discuss 
> what we can do to make zeppelin standout in the data science and big data 
> area.
>
> I hope it will be a regular sync up meeting, maybe one time per month. Let us 
> know whether you are interested in such kind of regular sync up meeting and 
> what kind of things do you like to discuss.
>
> --
> Best Regards
>
> Jeff Zhang


--
Best Regards

Jeff Zhang

___
Eric Pugh | Founder & CEO | OpenSource Connections, LLC | 434.466.1467 | 
http://www.opensourceconnections.com | 
My Free/Busy
Co-Author: Apache Solr Enterprise Search Server, 3rd 
Ed
This e-mail and all contents, including attachments, is considered to be 
Company Confidential unless explicitly stated otherwise, regardless of whether 
attachments are marked as such.

Re: [VOTE] Release Apache Zeppelin 0.9.0-preview2 (RC2)

[Felix Cheung]

+1


From: Jeff Zhang 
Sent: Sunday, July 26, 2020 7:29:22 AM
To: users 
Cc: dev 
Subject: Re: [VOTE] Release Apache Zeppelin 0.9.0-preview2 (RC2)

+1

Tested spark interpreter tutorial, flink interpreter tutorial.

Prabhjyot Singh mailto:prabhjyotsi...@gmail.com>> 
于2020年7月25日周六 下午12:26写道：
+1

On Fri, 24 Jul 2020 at 21:23, moon soo Lee 
mailto:m...@apache.org>> wrote:
+1

I tested

 - build from source
 - src package license file
 - bin package license file
 - new ui access

On Fri, Jul 24, 2020 at 12:36 AM Alex Ott 
mailto:alex...@gmail.com>> wrote:
+1 from me. What's done:

- checked the checksum
- run spark samples
- tested cassandra interpreter
- installed some plugins from helium registry


On Thu, Jul 23, 2020 at 5:04 PM Jeff Zhang 
mailto:zjf...@gmail.com>> wrote:


Hi folks,

I propose the following RC to be released for the Apache Zeppelin 
0.9.0-preview2 release.


The commit id is 31b9ed51f946fed934885d8fbb29e9c183552e70 :
https://gitbox.apache.org/repos/asf?p=zeppelin.git;a=commit;h=31b9ed51f946fed934885d8fbb29e9c183552e70

This corresponds to the tag: v0.9.0-preview2-rc2 :
https://gitbox.apache.org/repos/asf?p=zeppelin.git;a=shortlog;h=refs/tags/v0.9.0-preview2-rc2

The release archives (tgz), signature, and checksums are here
https://dist.apache.org/repos/dist/dev/zeppelin/zeppelin-0.9.0-preview2-rc2/

The release candidate consists of the following source distribution archive
zeppelin-v0.9.0-preview2.tgz

In addition, the following supplementary binary distributions are provided
for user convenience at the same location
zeppelin-0.9.0-preview2-bin-all.tgz


The maven artifacts are here
https://repository.apache.org/content/repositories/orgapachezeppelin-1283/org/apache/zeppelin/

You can find the KEYS file here:
https://dist.apache.org/repos/dist/release/zeppelin/KEYS

Release notes available at
https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12342692==12316221

Vote will be open for next 72 hours (close at 8am 26/July PDT).

[ ] +1 approve
[ ] 0 no opinion
[ ] -1 disapprove (and reason why)


--
Best Regards

Jeff Zhang


--
With best wishes,Alex Ott
http://alexott.net/
Twitter: alexott_en (English), alexott (Russian)


--
Regards,
Prabhjyot Singh


--
Best Regards

Jeff Zhang

Re: [Announce] New Zeppelin Committer: Philipp Dallig

[Felix Cheung]

Congrats and welcome!


From: Xun Liu 
Sent: Saturday, June 27, 2020 8:58:34 AM
To: dev 
Cc: users ; philipp.dal...@gmail.com 

Subject: Re: [Announce] New Zeppelin Committer: Philipp Dallig

That's great news!
Welcome aboard Philipp!
:-)

On Sat, Jun 27, 2020 at 8:43 AM Yadong Xie 
mailto:vthink...@gmail.com>> wrote:
welcome!

On Saturday, June 27, 2020, Alex Ott 
mailto:alex...@gmail.com>> wrote:

> That's great news! Welcome aboard Philipp!
>
> On Fri, Jun 26, 2020 at 8:23 AM Jeff Zhang 
> mailto:zjf...@gmail.com>> wrote:
>
> >
> > The Project Management Committee (PMC) for Apache Zeppelin
> > has invited Philipp Dallig to become a committer and we are very pleased
> > to announce that he has accepted.
> >
> > We greatly appreciate all of Philipp Dallig's hard work and generous
> > contributions to the project. We look forward to continued involvement in
> > the project.
> >
> > Congratulations & Welcome aboard Philipp Dallig !
> >
> >
> > --
> > Best Regards
> >
> > Jeff Zhang
> >
>
>
> --
> With best wishes,Alex Ott
> http://alexott.net/
> Twitter: alexott_en (English), alexott (Russian)
>

Re: Intellij announce Big Data Tools – Integration with Zeppelin

[Felix Cheung]

Cool!


From: Jeff Zhang 
Sent: Wednesday, October 16, 2019 6:52:02 AM
To: users 
Subject: Intellij announce Big Data Tools – Integration with Zeppelin


Here's the related details

https://blog.jetbrains.com/scala/2019/10/16/meet-big-data-tools-spark-integration-and-zeppelin-notebooks-in-intellij-idea/

https://plugins.jetbrains.com/plugin/12494-big-data-tools?_ga=2.41180706.1434705875.1571232785-884153734.1558949232

--
Best Regards

Jeff Zhang

Re: Zeppelin Studio Proposal

[Felix Cheung]

This is cool!


From: CHALLA 
Sent: Wednesday, August 28, 2019 4:39:46 AM
To: users@zeppelin.apache.org 
Subject: Re: Zeppelin Studio Proposal

 great.

On Wed, Aug 28, 2019 at 4:04 PM Ivan Shapovalov 
mailto:shapovalov.iva...@gmail.com>> wrote:
Hey All,

The idea of refactoring UI sounds great!
Ever considered theia (https://github.com/theia-ide/theia)? May save a couple 
of ages for community.

Regards,
Ivan

ср, 28 авг. 2019 г. в 11:31, Jongyoul Lee 
mailto:jongy...@gmail.com>>:
Sounds great!!

On Tue, Aug 27, 2019 at 1:31 AM ieglonewolf ieglonewolf 
mailto:ieglonew...@gmail.com>> wrote:
I would like to add

Our key motive for us to start ZEPPELIN-4138
 here is to build a UI
system on top of zeppelin service which is intuitive and very much
functional.
Moreover doing it the right way is the key to scalability. I would like to
request each and every member of this community to help us develop this.

Please provide your valuable feedback

Thanks folks!

On Mon, Aug 26, 2019 at 9:24 PM Xun Liu 
mailto:neliu...@163.com>> wrote:

> Zeppelin is very much in need of a front end developed using VUE.js
> front-end technology.
> Thank you for your contribution. :-)
>
> Xun Liu
> Best Regards
>
> On Aug 26, 2019, at 9:21 PM, Jeff Zhang 
> mailto:zjf...@gmail.com>> wrote:
>
> + user mail list
>
>
> Thanks Malay for the proposal. The zeppelin frontend do needs some rework.
>
> Overall, your proposal make sense to me. User experience and performance
> are the 2 key things we need to improve in the frontend.
> I left some comments in the design doc.
>
>
> Malay Majithia mailto:malay.majit...@gmail.com>> 
> 于2019年8月23日周五 下午6:10写道：
>
>> Hey Folks,
>>
>> Regarding ZEPPELIN-4138
>> , we have come up
>> with the design document(draft) and the task list for the same:
>>
>>
>>  Zeppelin Studio - Design Document
>> 
>>
>>  Zeppelin Studio - Task list
>> 
>>
>> POC code is available on GitHub
>> .
>>
>> Sneak peek of the proposed interface:
>> 
>>
>>
>> Please review it and provide your valuable feedback.
>>
>> Best Regards
>> Malay Majithia
>>
>>
>
> --
> Best Regards
>
> Jeff Zhang
>
>
>


--
이종열, Jongyoul Lee, 李宗烈
http://madeng.net


--
Ivan Shapovalov
Kharkov, Ukraine

Re: Zeppelin log files in windows

[Felix Cheung]

Thanks for the report!

Other projects have testing setup on the like of AppVeyor. We would greatly 
appreciate any contribution in this area to fix the issue and enable continuous 
testing coverage!


From: Jeff Zhang 
Sent: Tuesday, May 28, 2019 6:26:43 AM
To: users
Subject: Re: Zeppelin log files in windows

Hi Ravi,

Sorry for the inconvenience. The community has no bandwidth to keep the 
stability of zeppelin on windows. I would recommend you to install zeppelin in 
linux if that works for you.


Ravi Pullareddy 
mailto:ravi.pullare...@minlog.com.au>> 
于2019年5月28日周二 上午9:24写道：
Hi Folks

Windows version of Zeppelin 0.8.1 has a typo error on line 76 of command.cmd. 
There is a curly brace ‘}’ in place of closed bracket ‘)’ . I request you to 
correct this and publish it. Apart from this trivial error,  Zeppelin logs to 
console but does not create a log file in Windows.  My log4j.properties file is 
as below. Please check and let me know if I am missing something.

log4j.rootLogger = INFO, stdout, dailyfile

log4j.appender.stdout = org.apache.log4j.ConsoleAppender
log4j.appender.stdout.layout = org.apache.log4j.PatternLayout
log4j.appender.stdout.layout.ConversionPattern=%5p [%d] ({%t} %F[%M]:%L) - %m%n

log4j.appender.dailyfile.DatePattern=.-MM-dd
log4j.appender.dailyfile.Threshold = INFO
log4j.appender.dailyfile = org.apache.log4j.DailyRollingFileAppender
#log4j.appender.dailyfile.File = ${zeppelin.log.file}
log4j.appender.dailyfile.File = D:\\zeppelin-0.8.1-bin-all\\logs\\zep.log
log4j.appender.dailyfile.layout = org.apache.log4j.PatternLayout
log4j.appender.dailyfile.layout.ConversionPattern=%5p [%d] ({%t} %F[%M]:%L) - 
%m%n

Thanks
Ravi


--
Best Regards

Jeff Zhang

Call for Presentations (CFP) is now open for ApacheCon North America in Las Vegas, September 9-13th

[Felix Cheung]

See the Big Data track or the Machine Learning track!

We’re delighted to announce that the Call for Presentations (CFP) is now
open for ApacheCon North America in Las Vegas, September 9-13th! As the
official conference series of the ASF, ApacheCon North America will
feature over a dozen Apache project summits, including Cassandra,
Cloudstack, Tomcat, Traffic Control, and more. We’re looking for talks
in a wide variety of categories -- anything related to ASF projects and
the Apache development process. The CFP closes at midnight on May 26th.
In addition, the ASF will be celebrating its 20th Anniversary during the
event. For more details and to submit a proposal for the CFP, visit
https://apachecon.com/acna19/ . Registration will be opening soon.

Re: Windows Supports

[Felix Cheung]

Ok but was the point about appveyer as CI. It’s not hard to setup.



From: Jongyoul Lee 
Sent: Tuesday, February 26, 2019 11:12 PM
To: users
Subject: Re: Windows Supports

@Felix
What I meant was the case that running Zeppelin in the Windows environment 
natively without a docker and a virtual Linux environment. People could run 
Zeppelin through these kinds of ways but in the case where they run Zeppelin 
natively, we didn't test that case and couldn't know the potential problems as 
well. We could guide to use a docker container or virtual Linux environment by 
default for Windows users instead of using native scripts.

@Jeff,
I also thought of running interpreters in Windows. I don't think it's easy to 
set up CI for windows.

Basically, I agree with that the best way is to support Windows well. But the 
more important thing is to keep Window users' UX. WDYT?

On Wed, Feb 27, 2019 at 12:53 AM Jeff Zhang 
mailto:zjf...@gmail.com>> wrote:
I think the issue is about running spark interpreter in windows. This is due to 
some script changes in interpreter launch script interpreter.sh, but it is not 
applied in interpreter.cmd. We could still support windows by fixing this 
issue, but I don't have time on this right now. I would be very appreciated if 
someone else can help on this, and also set up CI in appveyer

Thomas Bernhardt mailto:bernhardt...@yahoo.com>> 
于2019年2月26日周二 下午8:12写道：
We had no trouble running 0.8.0 on Windows 10 professional. We even set up 
authentication. Maybe our case is special however since we don't use any of the 
provided interpreters and only have an own interpreter.
-Tom

On Monday, February 25, 2019, 9:29:14 PM EST, Jongyoul Lee 
mailto:jongy...@gmail.com>> wrote:


Hi Dev and Users,

Recently, personally, I've got reports that Z couldn't run under Windows' 
environments.

I think we need to discuss how to handle issues supporting windows.

AFAIK, there are not many resources to test Z under Windows by committers or 
contributors. If we couldn't support Windows well, how about removing bin/*.cmd 
and focusing on alternatives like dockers.

WDYT?

JL

--
이종열, Jongyoul Lee, 李宗烈
http://madeng.net


--
Best Regards

Jeff Zhang


--
이종열, Jongyoul Lee, 李宗烈
http://madeng.net

Re: Windows Supports

[Felix Cheung]

Testing on windows can also be done as CI on appveyer.

I don’t completely get your comment on .cmd file though. Are you suggesting we 
don’t support windows and users can “run on windows” by basically running Linux 
in a virtual environment? Docker is one and there is Linux on Windows 
https://docs.microsoft.com/en-us/windows/wsl/install-win10



From: Jongyoul Lee 
Sent: Monday, February 25, 2019 6:29 PM
To: dev; users
Subject: Windows Supports

Hi Dev and Users,

Recently, personally, I've got reports that Z couldn't run under Windows'
environments.

I think we need to discuss how to handle issues supporting windows.

AFAIK, there are not many resources to test Z under Windows by committers
or contributors. If we couldn't support Windows well, how about removing
bin/*.cmd and focusing on alternatives like dockers.

WDYT?

JL

--
이종열, Jongyoul Lee, 李宗烈
http://madeng.net

Re: [DISCUSS] Deprecate support for Spark 2.2.x and earlier version

[Felix Cheung]

+1



From: Jeff Zhang 
Sent: Thursday, February 14, 2019 10:28 PM
To: users
Subject: [DISCUSS] Deprecate support for Spark 2.2.x and earlier version

Hi Folks,

Spark 2.2.x will be EOL[1] from January of 2019. So I am considering to 
deprecate support for spark 2.2.x and earlier version in Zeppelin 0.9.0. 
Deprecation means that from Zeppelin 0.9 user is still able to run spark 2.2.x 
and earlier version, but will see a warning message in frontend about this 
deprecation. And in the next major version(maybe 0.10, or 1.0), we would 
totally remove support for spark 2.2.x and earlier version. The impact for 
users is the deprecation message in frontend. It may cause issues for users if 
they use rest api of zeppelin to run paragraph, then fetch and parse the result.

Let me know your concern about this. Thanks

[1] https://spark.apache.org/versioning-policy.html


--
Best Regards

Jeff Zhang

Re: [Discuss] 0.8.1 Release

[Felix Cheung]

+1 thanks!



From: Jeff Zhang 
Sent: Tuesday, January 15, 2019 5:39 PM
To: users
Subject: Re: [Discuss] 0.8.1 Release

Hi Folks,

I will start the 0.8.1 release since there's no concerns on this.

Jeff Zhang mailto:zjf...@gmail.com>> 于2019年1月14日周一 下午8:50写道：
Hi Everyone,

Sorry for replying this mail so late, I am busy with other stuff. I plan to 
start the 0.8.1 release this week, does anyone have any concerns ?



Jeff Zhang mailto:zjf...@gmail.com>> 于2018年10月10日周三 上午8:31写道：
Community is working on that.

Paul Brenner mailto:pbren...@placeiq.com>>于2018年10月10日周三 
上午12:35写道：
I would second this if it doesn’t hold up the release too much. We would love 
to see this implemented.


[https://marketing.placeiq.net/images/placeiq.png]
   Paul Brenner  
[https://marketing.placeiq.net/images/twitter1.png] 
    
[https://marketing.placeiq.net/images/facebook.png] 
   
 
[https://marketing.placeiq.net/images/linkedin.png] 

SR. DATA SCIENTIST
(217) 390-3033


[PlaceIQ:CES
 
2018]

On Oct 9, 2018, 12:33 PM -0400, Павел Мяснов 
mailto:glowf...@gmail.com>>, wrote:
I know it is marked as improvement, not a bug; but is it possible to include 
ticket https://jira.apache.org/jira/browse/ZEPPELIN-3307 to this minor release? 
I hit this problem quite often and it would be really nice to see this problem 
solved soon.

Pavel Myasnov

On 2018/09/27 07:08:31, Jeff Zhang http://gmail.com>> wrote:
> Hi folks,>
>
> It has been a while for 0.8.0 release, and we got many feedback for that,>
> so I think it is time for us to make 0.8.1 release for fix the bugs of>
> 0.8.0. Here's the umbrella tickets for 0.8.1 release>
> https://jira.apache.org/jira/browse/ZEPPELIN-3629>
>
> If you find any ticket that is necessary for 0.8.1 but not under this>
> umbrella ticket, feel free to link that. I will start the 0.8.1 release at>
> the beginning of Oct.>
>


--
Best Regards

Jeff Zhang


--
Best Regards

Jeff Zhang

Re: [DISCUSS] ZEPPELIN-2619. Save note in [Title].zpln instead of [NOTEID]/note.json

[Felix Cheung]

Perhaps one concern is users having characters in note name that are invalid 
for file name/file path?



From: Mohit Jaggi 
Sent: Sunday, August 12, 2018 6:02 PM
To: users@zeppelin.apache.org
Cc: dev
Subject: Re: [DISCUSS] ZEPPELIN-2619. Save note in [Title].zpln instead of 
[NOTEID]/note.json

sounds like a good idea!

On Sun, Aug 12, 2018 at 5:34 PM Jeff Zhang 
mailto:zjf...@gmail.com>> wrote:
Motivation

   The motivation of ZEPPELIN-2619 is to change the notes storage structure. 
Previously we store it using {noteId}/note.json, we’d like to change it into 
{note_name}_{note_id}.zpln. There are several reasons for this change.


  1.  {noteId}/note.json is not scalable. We put all notes in one root folder 
in flat structure. And when zeppelin server starts, we need to read all 
note.json to get the note file name and build the note folder structure 
(Because we need to get the note name which is stored in note.json to build the 
notebook menu). This would be a nightmare when you have large amounts of notes.

  2.  {noteId}/note.json is not maintainable. It is difficult for a 
developer/administrator to find note file based on note name.

  3.  {noteId}/note.json has no folder structure. Currently zeppelin have to 
build the folder structure internally in memory according note name which is a 
big overhead.

New Approach

   As I mentioned above, I propose to change the note storage structure to 
{note_name}_{note_id}.zpln.  note_name could contains folders, e.g. 
folder_1/mynote_abcd.zpln

This kind of note storage structure could bring several benefits.

  1.  We don’t need to load all notes when zeppelin starts. We just need to 
list each folder to get the note name and note_id.

  2.  It is much maintainable so that it is easy to find the note file based on 
note name.

  3.  It has the folder structure already. That can be mapped to the note 
folder structure.

Side Effect

This approach only works for file system storage, so that means we have to drop 
support for MongoNotebookRepo. I think it is ok because I didn’t see any users 
talk about this in community, so I assume no one is using it.


This is overall design, welcome any comments and feedback. Thanks.


Here's the google docs, you can also comment it here.

https://docs.google.com/document/d/126egAQmhQOL4ynxJ3AQJQRBBLdW8TATYcGkDL1DNZoE/edit?usp=sharing

Re: [ANNOUNCE] Apache Zeppelin 0.8.0 released

[Felix Cheung]

Congrats and thanks for putting together the release


From: Miquel Angel Andreu Febrer 
Sent: Wednesday, June 27, 2018 11:02:20 PM
To: d...@zeppelin.apache.org
Cc: users@zeppelin.apache.org
Subject: Re: [ANNOUNCE] Apache Zeppelin 0.8.0 released

Great news,

It has been a hard work to announce this release

Thank you very much Jeff for your work and your patience





El jue., 28 jun. 2018 6:05, Sanjay Dasgupta 
escribió:

> This is really a great milestone.
>
> Thanks to those behind the grand effort.
>
> On Thu, Jun 28, 2018 at 8:51 AM, Prabhjyot Singh  >
> wrote:
>
> > Awesome! congratulations team.
> >
> >
> >
> > On Thu 28 Jun, 2018, 8:39 AM Taejun Kim,  wrote:
> >
> >> Awesome! Thanks for your great work :)
> >>
> >> 2018년 6월 28일 (목) 오후 12:07, Jeff Zhang 님이 작성:
> >>
> >>> The Apache Zeppelin community is pleased to announce the availability
> of
> >>> the 0.8.0 release.
> >>>
> >>> Zeppelin is a collaborative data analytics and visualization tool for
> >>> distributed, general-purpose data processing system such as Apache
> Spark,
> >>> Apache Flink, etc.
> >>>
> >>> This is another major release after the last minor release 0.7.3.
> >>> The community put significant effort into improving Apache Zeppelin
> since
> >>> the last release. 122 contributors fixed totally 602 issues. Lots of
> >>> new features are introduced, such as inline configuration, ipython
> >>> interpreter, yarn-cluster mode support , interpreter lifecycle manager
> >>> and etc.
> >>>
> >>> We encourage you to download the latest release
> >>> fromhttp://zeppelin.apache.org/download.html
> >>>
> >>> Release note is available
> >>> athttp://zeppelin.apache.org/releases/zeppelin-release-0.8.0.html
> >>>
> >>> We welcome your help and feedback. For more information on the project
> >>> and
> >>> how to get involved, visit our website at http://zeppelin.apache.org/
> >>>
> >>> Thank you all users and contributors who have helped to improve Apache
> >>> Zeppelin.
> >>>
> >>> Regards,
> >>> The Apache Zeppelin community
> >>>
> >> --
> >> Taejun Kim
> >>
> >> Data Mining Lab.
> >> School of Electrical and Computer Engineering
> >> University of Seoul
> >>
> >
>

Re: zeppelin 0.8 tar file

[Felix Cheung]

0.8 is not released yet.


From: Soheil Pourbafrani 
Sent: Sunday, April 29, 2018 9:18:10 AM
To: users@zeppelin.apache.org
Subject: zeppelin 0.8 tar file

Is there any pre-compiled tar file of Zeppelin 0.8 to download?

Re: [Julia] Does Spark.jl work in Zeppelin's existing Spark/livy.spark interpreters?

[Felix Cheung]

Actually, I’m not sure we support Julia as a language in the Spark interpreter.

As far as I understand this, this is Julia -> Spark so we would need support 
for this added to enable

Java (Zeppelin) -> Julia -> Spark



From: Jongyoul Lee 
Sent: Saturday, April 21, 2018 11:53:12 PM
To: users@zeppelin.apache.org
Subject: Re: [Julia] Does Spark.jl work in Zeppelin's existing Spark/livy.spark 
interpreters?

Hello,

AFAIK, there is no issue.

Regards
JL

On Wed, 18 Apr 2018 at 2:22 AM Josh Goldsborough 
> wrote:
Wondering if anyone has had success using the 
Spark.jl library for Spark to support Julia 
using one of Zeppelin's spark interpreters.

Thanks!
-Josh
--
이종열, Jongyoul Lee, 李宗烈
http://madeng.net

Re: "IPython is available, use IPython for PySparkInterpreter"

[Felix Cheung]

I think that's a good point - perhaps this shouldn't be a warning.


From: Ruslan Dautkhanov 
Sent: Monday, March 19, 2018 11:10:48 AM
To: users
Subject: "IPython is available, use IPython for PySparkInterpreter"

We're getting " IPython is available, use IPython for PySparkInterpreter "
warning each time we start %pyspark notebooks.

Although there is no difference between %pyspark and %ipyspark afaik.
At least we can use all ipython magic commands etc.
(maybe becase we have zeppelin.pyspark.useIPython=true?)

If that's the case, how we can disable "IPython is available, use IPython for 
PySparkInterpreter" warning ?


--
Ruslan Dautkhanov

Re: NullPointerException in paragraph when getting batched TableEnvironment

[Felix Cheung]

Does it work within the Flink Scala Shell?


From: André Schütz 
Sent: Wednesday, February 14, 2018 4:02:30 AM
To: us...@zeppelin.incubator.apache.org
Subject: NullPointerException in paragraph when getting batched TableEnvironment

Hi,

within the Flink Interpreter context, we try to get a Batch
TableEnvironment with the following code.

[code]
import org.apache.flink.table.api._
import org.apache.flink.table.api.scala._
import org.apache.flink.table.sources._

val batchEnvironment  = benv
val batchTableEnvironment = TableEnvironment.getTableEnvironment
(batchEnvironment) [/code]

When executing the paragraph, we get the following error.

[error]
Caused by: java.lang.ExceptionInInitializerError:
java.lang.NullPointerException
Caused by: java.lang.NullPointerException
  at org.apache.flink.table.api.scala.BatchTableEnvironment.
(BatchTableEnvironment.scala:47) at
org.apache.flink.table.api.TableEnvironment$.getTableEnvironment
(TableEnvironment.scala:1049)
[/error]

Any ideas why there is the NullPointerException?

I am grateful for any ideas.

Kind regards,
Andre

--
Andre Schütz
COO / Founder - Wegtam GmbH
an...@wegtam.com | P: +49 (0) 381-80 699 041 | M: +49 (0) 176-218 02 604
www.wegtam.com | 
www.tensei-data.com | 
www.wegtam.net

Re: Issue with multiple users running Spark

[Felix Cheung]

Should we have some doc on this? I think this could be a common problem


From: Austin Heyne 
Sent: Monday, January 15, 2018 6:59:55 AM
To: users@zeppelin.apache.org
Subject: Re: Issue with multiple users running Spark


Thanks Jeff and Michael for the help. We're seeing good success just disabling 
'zeppelin.spark.useHiveContext'.

-Austin

On 01/12/2018 07:56 PM, Jeff Zhang wrote:

There're 2 options for you:

1. Disable hiveContext in spark via setting zeppelin.spark.useHiveContext to 
false in spark's interpreter setting
2. Connect to hive metastore service instead of single derby instance. You can 
configure that in your hive-site.xml



Michael Segel 
>于2018年1月13日周六 
上午2:40写道：
Hi,

Quick response… unless you tell Derby to set up as a networked service (this is 
going back to SilverCloud days), its a single user instance. So it won’t work.
Were you using MySQL or something… you would have better luck…


I think if you go back in to Derby’s docs and see how to start this as a 
networked server (multi-user) , you could try it.
Most people don’t do this because not many people know Derby and I don’t know 
how well that portion of the code has been maintained over the years.


HTH

-Mike

> On Jan 12, 2018, at 12:35 PM, Austin Heyne 
> > wrote:
>
> Hi everyone,
>
> I'm currently running Zeppelin on a spark master node using the AWS provided 
> Zeppelin install. I'm trying to get the notebook setup so multiple devs can 
> use it (and the spark cluster) concurrently. I have the spark interpreter set 
> to instantiate 'Per Note' in 'isolated' processes. I also have 
> 'spark.dynamicAllocation.enabled' set to 'true' so the multiple spark 
> contexts can share the cluster.
>
> The problem I'm seeing is when the second spark context tries to instantiate 
> hive starts throwing errors because the Derby database has already been 
> booted (by the other context). Full stack trace is available here [1]. How do 
> I go about working around this? Is there a way to have it use another 
> database or is this a limitation?
>
> Thanks for any help!
>
> [1] https://gist.github.com/aheyne/8d84eaedefb997f248b6e88c1b9e1e34
>
> --
> Austin L. Heyne
>



--
Austin L. Heyne

Re: Use Bokeh in Apache Zeppelin

[Felix Cheung]

Nice!
Get this in 
https://github.com/apache/zeppelin/blob/master/docs/interpreter/python.md?


From: Partridge, Lucas (GE Aviation) 
Sent: Wednesday, January 10, 2018 2:27:19 AM
To: Jeff Zhang
Cc: users@zeppelin.apache.org
Subject: Use Bokeh in Apache Zeppelin

Thanks Jeff! I can confirm the following resulted in an inline graph when using 
a notebook bound to the Spark interpreter group in plain Zeppelin 0.7.0:

%pyspark
from bokeh.plotting import figure
from bokeh.io import show,output_notebook
import bkzep
output_notebook(notebook_type='zeppelin')

f = figure()
f.line(x=[1,2],y=[3,4])
show(f)

From: Jeff Zhang [mailto:zjf...@gmail.com]
Sent: 09 January 2018 23:24
To: Partridge, Lucas (GE Aviation) 
Cc: users@zeppelin.apache.org
Subject: EXT: Re: Use Bokeh in Apache Zeppelin


Awesome. Glad to see you can use bokeh in zeppelin. From bokeh after 0.12.7, 
you need bkzep. You can check the README here. https://github.com/zjffdu/bkzep

Actually you just need to import bkzep. You don't need to call 
install_notebook_hook explicitly.



Partridge, Lucas (GE Aviation) 
>于2018年1月10日周三 上午12:35写道：
Hi Jeff,

I eventually managed to get Bokeh running in Zeppelin 0.7.0 after finding your 
code at https://pypkg.com/pypi/bkzep/f/bkzep/__init__.py . So I did ‘pip 
install bkzep’ and restarted Zeppelin. Then if I pasted this code of yours…

from bokeh.io import install_notebook_hook
from bkzep.io import load_notebook, 
_show_zeppelin_app_with_state, _show_zeppelin_doc_with_state

install_notebook_hook('zeppelin', load_notebook,
  _show_zeppelin_doc_with_state, 
_show_zeppelin_app_with_state, overwrite=True)

…into a notebook paragraph before using Bokeh then I could see my plots 
directly within Zeppelin:).
Thanks, Lucas.

From: Partridge, Lucas (GE Aviation)
Sent: 09 January 2018 15:01

To: users@zeppelin.apache.org
Cc: zjf...@gmail.com
Subject: EXT: RE: Use Bokeh in Apache Zeppelin

I forgot to say I’m using Bokeh 0.12.13.

From: Partridge, Lucas (GE Aviation)
Sent: 09 January 2018 13:24
To: users@zeppelin.apache.org
Cc: zjf...@gmail.com
Subject: EXT: RE: Use Bokeh in Apache Zeppelin

Hi Jeff,

Adding support for Bokeh in Zeppelin is great! At 
https://www.zepl.com/viewer/notebooks/bm90ZTovL3pqZmZkdS90ZXN0L2VhMGI0ODQ0MzNhYjQxNjZhODg5MjI1ZjAxZWVjMjdiL25vdGUuanNvbg
 it says:

“If you want to use bokeh in spark interpreter. You need HDP 2.6.0 (Zeppelin 
0.7.0) or afterwards”

I’m not using HDP but I am using Zeppelin 0.7.0 (zeppelin-0.7.0-bin-all.tgz) in 
ubuntu 16.04. And when I do this in a notebook bound to the Spark interpreter 
group:

%pyspark
from bokeh.io import output_notebook
output_notebook(notebook_type='zeppelin')

I get this error:

Traceback (most recent call last):
  File "/tmp/zeppelin_pyspark-8411751233295366188.py", line 346, in 
raise Exception(traceback.format_exc())
Exception: Traceback (most recent call last):
  File "/tmp/zeppelin_pyspark-8411751233295366188.py", line 339, in 
exec(code)
  File "", line 2, in 
  File "/home/lucas/.local/lib/python2.7/site-packages/bokeh/util/api.py", line 
190, in wrapper
return obj(*args, **kw)
  File "/home/lucas/.local/lib/python2.7/site-packages/bokeh/io/output.py", 
line 114, in output_notebook
run_notebook_hook(notebook_type, 'load', resources, verbose, hide_banner, 
load_timeout)
  File "/home/lucas/.local/lib/python2.7/site-packages/bokeh/util/api.py", line 
190, in wrapper
return obj(*args, **kw)
  File "/home/lucas/.local/lib/python2.7/site-packages/bokeh/io/notebook.py", 
line 286, in run_notebook_hook
raise RuntimeError("no display hook installed for notebook type %r" % 
notebook_type)
RuntimeError: no display hook installed for notebook type 'zeppelin'

Can you confirm Bokeh does work with the %pyspark interpreter in Zeppelin 
0.7.0? Or should I move to a later version of Zeppelin? I’d rather stick with 
0.7.0 for now if possible.

Thanks, Lucas.

From: Jeff Zhang [mailto:zjf...@gmail.com]
Sent: 02 July 2017 00:01
To: users >
Subject: EXT: Use Bokeh in Apache Zeppelin


I write a tutorial on using bokeh in apache zeppelin. If you are interested in 
data visualization in zeppelin notebook, bokeh would be a very good library for 
you. And you can take a look at the tutorial here.

https://community.hortonworks.com/articles/109837/use-bokeh-in-apache-zeppelin.html

Re: [DISCUSS] Change some default settings for avoiding unintended usages

[Felix Cheung]

Authentication by default is good but we should avoid having well known user / 
password by default - it’s security risk.


From: Belousov Maksim Eduardovich <m.belou...@tinkoff.ru>
Sent: Thursday, December 21, 2017 12:30:57 AM
To: users@zeppelin.apache.org
Cc: d...@zeppelin.apache.org
Subject: RE: [DISCUSS] Change some default settings for avoiding unintended 
usages

The authentication by default isn't big deal, it's could be enabled.
It's nice to use another account by default: guest/guest, for example.


Thanks,

Maksim Belousov

From: Jongyoul Lee [mailto:jongy...@gmail.com]
Sent: Monday, December 18, 2017 6:07 AM
To: users <users@zeppelin.apache.org>
Cc: d...@zeppelin.apache.org
Subject: Re: [DISCUSS] Change some default settings for avoiding unintended 
usages

Agreed. Supporting container services must be good and I like this idea, but I 
don't think it's the part of this issue directly. Let's talk about this issue 
with another email.

I want to talk about enabling authentication by default. If it's enabled, we 
should login admin/password1 at the beginning. How do you think of it?

On Sat, Dec 2, 2017 at 1:57 AM, Felix Cheung 
<felixcheun...@hotmail.com<mailto:felixcheun...@hotmail.com>> wrote:
I’d +1 docker or container support (mesos, dc/os, k8s)

But I think that they are separate things. If users are authenticated and 
interpreter is impersonating each user, the risk of system disruption should be 
low. This is typically how to secure things in a system, through user directory 
(eg LDAP) and access control (normal user can’t sudo and delete everything).

Thought?

_
From: Jeff Zhang <zjf...@gmail.com<mailto:zjf...@gmail.com>>
Sent: Thursday, November 30, 2017 11:51 PM

Subject: Re: [DISCUSS] Change some default settings for avoiding unintended 
usages
To: <d...@zeppelin.apache.org<mailto:d...@zeppelin.apache.org>>
Cc: users <users@zeppelin.apache.org<mailto:users@zeppelin.apache.org>>


+1 for running interpreter process in docker container.



Jongyoul Lee <jongy...@gmail.com<mailto:jongy...@gmail.com>>于2017年12月1日周五 
下午3:36写道：
Yes, exactly, this is not only the shell interpreter problem, all can run
any script through python and Scala. Shell is just an example.

Using docker looks good but it cannot avoid unindented usage of resources
like mining coin.

On Fri, Dec 1, 2017 at 2:36 PM, Felix Cheung 
<felixcheun...@hotmail.com<mailto:felixcheun...@hotmail.com>>
wrote:

> I don’t think that’s limited to the shell interpreter.
>
> You can run any arbitrary program or script from python or Scala (or java)
> as well.
>
> _
> From: Jeff Zhang <zjf...@gmail.com<mailto:zjf...@gmail.com>>
> Sent: Wednesday, November 29, 2017 4:00 PM
> Subject: Re: [DISCUSS] Change some default settings for avoiding
> unintended usages
> To: <d...@zeppelin.apache.org<mailto:d...@zeppelin.apache.org>>
> Cc: users <users@zeppelin.apache.org<mailto:users@zeppelin.apache.org>>
>
>
>
> Shell interpreter is a black hole for security, usually we don't recommend
> or allow user to use shell.
>
> We may need to refactor the shell interpreter, running under zeppelin user
> is too dangerous.
>
>
>
>
>
> Jongyoul Lee <jongy...@gmail.com<mailto:jongy...@gmail.com>>于2017年11月29日周三 
> 下午11:44写道：
>
> > Hi, users and dev,
> >
> > Recently, I've got an issue about the abnormal usage of some
> interpreters.
> > Zeppelin's users can access shell by shell and python interpreters. It
> > means all users can run or execute what they want even if it harms the
> > system. Thus I agree that we need to change some default settings to
> > prevent this kind of abusing situation. Before we proceed to do it, I
> want
> > to listen to others' opinions.
> >
> > Feel free to reply this email
> >
> > Regards,
> > Jongyoul
> >
> > --
> > 이종열, Jongyoul Lee, 李宗烈
> > http://madeng.net
> >
>
>
>


--
이종열, Jongyoul Lee, 李宗烈
http://madeng.net




--
이종열, Jongyoul Lee, 李宗烈
http://madeng.net

Re: Accessing Spark UI from Zeppelin

[Felix Cheung]

You could set to replace http://masternode with your custom http hostname.

Perhaps you want that to be set to a known, public (and authenticated?) IP/url? 
If you do have it it can be set to the Zeppelin config before Zeppelin starts.


From: ankit jain 
Sent: Thursday, December 14, 2017 2:35:32 PM
To: users@zeppelin.apache.org
Cc: Esteban de Jesus Hernandez; Jhon; Emmanuel; Oliver; Phil
Subject: Accessing Spark UI from Zeppelin


Hi Zeppelin users,

We are following https://issues.apache.org/jira/browse/ZEPPELIN-2949 to launch 
spark ui.

Our Zeppelin instance is deployed on AWS EMR master node and setting 
zeppelin.spark.uiWebUrl to a url which elb maps to 
https://masternode:4040.

When user clicks on spark url within Zeppelin it redirects him to Yarn RM( 
something like http://masternode:20888/proxy/application_1511906080313_0023/) 
which fails to load.

Usually to access EMR Web interfaces requires to setup a SSH tunnel and change 
proxy settings in the browser - 
http://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-spark-history.html

Is there a way we can avoid users having to setup ssh tunnel and allow direct 
access to Spark UI?

Ideally, we will implement a filter which does Authentication on the user and 
then redirect to Spark UI – right now not sure what the redirect URL should be?

--
Thanks & Regards,
Anki

Re: zeppelin build fails with DependencyConvergence error

[Felix Cheung]

Instead of exclusion, would it be better to use the version in the cloudera 
repo?

Please do consider contributing these changes back to Zeppelin source. Thanks!

_
From: Ruslan Dautkhanov 
Sent: Monday, December 11, 2017 3:42 PM
Subject: Re: zeppelin build fails with DependencyConvergence error
To: Zeppelin Users 


Looks like master branch of Zeppelin still has compatibility issue with 
Cloudera dependencies.

When built using


mvn clean package -DskipTests -Pspark-2.2 -Dhadoop.version=2.6.0-cdh5.12.1 
-Phadoop-2.6 -Pvendor-repo -pl '!...list of excluded packages' -e

maven fails on jackson convergence error - see below email for more details.
Looks like there was a change in Zeppelin that upgraded Jackson's version?
So now it conflicts with older jackson library as referenced by cloudera repo.

workaround: Zeppelin builds fine with pom change [1] - the question is now
would somebody expect Zeppelin would still be functioning correctly with these 
exclusions?



[1]

--- a/zeppelin-zengine/pom.xml
+++ b/zeppelin-zengine/pom.xml
@@ -364,6 +364,30 @@
   com.google.guava
   guava
 
+
+  com.fasterxml.jackson.core
+  jackson-core
+
+
+  com.fasterxml.jackson.core
+  jackson-annotations
+
+
+  com.fasterxml.jackson.core
+  jackson-databind
+
+
+  org.codehaus.jackson
+  jackson-mapper-asl
+
+
+  org.codehaus.jackson
+  jackson-core-asl
+
+
+  org.apache.zookeeper
+  zookeeper
+
   
 




On Sun, Aug 27, 2017 at 2:25 PM, Ruslan Dautkhanov 
> wrote:
Building from a current Zeppelin snapshot fails with
zeppelin build fails with 
org.apache.maven.plugins.enforcer.DependencyConvergence
see details below.

Build command
/opt/maven/maven-latest/bin/mvn clean package -DskipTests -Pspark-2.2 
-Dhadoop.version=2.6.0-cdh5.12.0 -Phadoop-2.6 -Pvendor-repo -Pscala-2.10 
-Psparkr -pl '!..excluded certain modules..' -e

maven 3.5.0
jdk 1.8.0_141
RHEL 7.3
npm.x86_64   1:3.10.10-1.6.11.1.1.el7
nodejs.x86_641:6.11.1-1.el7 @epel
latest zeppelin snapshot

Any ideas? It's my first attempt to build on rhel7/jdk8 .. never seen this 
problem before.

Thanks,
Ruslan



[INFO] Scanning for projects...
[WARNING]
[WARNING] Some problems were encountered while building the effective model for 
org.apache.zeppelin:zeppelin-spark-dependencies_2.10:jar:0.8.0-SNAPSHOT
[WARNING] 'build.plugins.plugin.(groupId:artifactId)' must be unique but found 
duplicate declaration of plugin 
com.googlecode.maven-download-plugin:download-maven-plugin @ line 940, column 15
[WARNING] 'build.plugins.plugin.(groupId:artifactId)' must be unique but found 
duplicate declaration of plugin 
com.googlecode.maven-download-plugin:download-maven-plugin @ line 997, column 15
[WARNING]
[WARNING] Some problems were encountered while building the effective model for 
org.apache.zeppelin:zeppelin-spark_2.10:jar:0.8.0-SNAPSHOT
[WARNING] 'build.plugins.plugin.(groupId:artifactId)' must be unique but found 
duplicate declaration of plugin org.scala-tools:maven-scala-plugin @ line 467, 
column 15
[WARNING] 'build.plugins.plugin.(groupId:artifactId)' must be unique but found 
duplicate declaration of plugin org.apache.maven.plugins:maven-surefire-plugin 
@ line 475, column 15
[WARNING] 'build.plugins.plugin.(groupId:artifactId)' must be unique but found 
duplicate declaration of plugin org.apache.maven.plugins:maven-compiler-plugin 
@ line 486, column 15
[WARNING] 'build.plugins.plugin.(groupId:artifactId)' must be unique but found 
duplicate declaration of plugin org.scala-tools:maven-scala-plugin @ line 496, 
column 15
[WARNING] 'build.plugins.plugin.(groupId:artifactId)' must be unique but found 
duplicate declaration of plugin org.apache.maven.plugins:maven-surefire-plugin 
@ line 504, column 15
[WARNING]
[WARNING] It is highly recommended to fix these problems because they threaten 
the stability of your build.
[WARNING]
[WARNING] For this reason, future Maven versions might no longer support 
building such malformed projects.
[WARNING]
[WARNING] The project org.apache.zeppelin:zeppelin-web:war:0.8.0-SNAPSHOT uses 
prerequisites which is only intended for maven-plugin projects but not for non 
maven-plugin projects. For such purposes you should use the 
maven-enforcer-plugin. See 
https://maven.apache.org/enforcer/enforcer-rules/requireMavenVersion.html


... [skip]

[INFO] 
[INFO] Building Zeppelin: Zengine 0.8.0-SNAPSHOT
[INFO] 
[INFO]
[INFO] --- maven-clean-plugin:2.6.1:clean (default-clean) @ zeppelin-zengine ---
[INFO]

Re: [DISCUSS] Change some default settings for avoiding unintended usages

[Felix Cheung]

I’d +1 docker or container support (mesos, dc/os, k8s)

But I think that they are separate things. If users are authenticated and 
interpreter is impersonating each user, the risk of system disruption should be 
low. This is typically how to secure things in a system, through user directory 
(eg LDAP) and access control (normal user can’t sudo and delete everything).

Thought?

_
From: Jeff Zhang <zjf...@gmail.com>
Sent: Thursday, November 30, 2017 11:51 PM
Subject: Re: [DISCUSS] Change some default settings for avoiding unintended 
usages
To: <d...@zeppelin.apache.org>
Cc: users <users@zeppelin.apache.org>



+1 for running interpreter process in docker container.



Jongyoul Lee <jongy...@gmail.com<mailto:jongy...@gmail.com>>于2017年12月1日周五 
下午3:36写道：
Yes, exactly, this is not only the shell interpreter problem, all can run
any script through python and Scala. Shell is just an example.

Using docker looks good but it cannot avoid unindented usage of resources
like mining coin.

On Fri, Dec 1, 2017 at 2:36 PM, Felix Cheung 
<felixcheun...@hotmail.com<mailto:felixcheun...@hotmail.com>>
wrote:

> I don’t think that’s limited to the shell interpreter.
>
> You can run any arbitrary program or script from python or Scala (or java)
> as well.
>
> _
> From: Jeff Zhang <zjf...@gmail.com<mailto:zjf...@gmail.com>>
> Sent: Wednesday, November 29, 2017 4:00 PM
> Subject: Re: [DISCUSS] Change some default settings for avoiding
> unintended usages
> To: <d...@zeppelin.apache.org<mailto:d...@zeppelin.apache.org>>
> Cc: users <users@zeppelin.apache.org<mailto:users@zeppelin.apache.org>>
>
>
>
> Shell interpreter is a black hole for security, usually we don't recommend
> or allow user to use shell.
>
> We may need to refactor the shell interpreter, running under zeppelin user
> is too dangerous.
>
>
>
>
>
> Jongyoul Lee <jongy...@gmail.com<mailto:jongy...@gmail.com>>于2017年11月29日周三 
> 下午11:44写道：
>
> > Hi, users and dev,
> >
> > Recently, I've got an issue about the abnormal usage of some
> interpreters.
> > Zeppelin's users can access shell by shell and python interpreters. It
> > means all users can run or execute what they want even if it harms the
> > system. Thus I agree that we need to change some default settings to
> > prevent this kind of abusing situation. Before we proceed to do it, I
> want
> > to listen to others' opinions.
> >
> > Feel free to reply this email
> >
> > Regards,
> > Jongyoul
> >
> > --
> > 이종열, Jongyoul Lee, 李宗烈
> > http://madeng.net
> >
>
>
>


--
이종열, Jongyoul Lee, 李宗烈
http://madeng.net

Re: [DISCUSS] Change some default settings for avoiding unintended usages

[Felix Cheung]

I don’t think that’s limited to the shell interpreter.

You can run any arbitrary program or script from python or Scala (or java) as 
well.

_
From: Jeff Zhang 
Sent: Wednesday, November 29, 2017 4:00 PM
Subject: Re: [DISCUSS] Change some default settings for avoiding unintended 
usages
To: 
Cc: users 


Shell interpreter is a black hole for security, usually we don't recommend
or allow user to use shell.

We may need to refactor the shell interpreter, running under zeppelin user
is too dangerous.





Jongyoul Lee 于2017年11月29日周三 下午11:44写道：

> Hi, users and dev,
>
> Recently, I've got an issue about the abnormal usage of some interpreters.
> Zeppelin's users can access shell by shell and python interpreters. It
> means all users can run or execute what they want even if it harms the
> system. Thus I agree that we need to change some default settings to
> prevent this kind of abusing situation. Before we proceed to do it, I want
> to listen to others' opinions.
>
> Feel free to reply this email
>
> Regards,
> Jongyoul
>
> --
> 이종열, Jongyoul Lee, 李宗烈
> http://madeng.net
>

Re: Zeppelin 0.7.2 integration with Presto 0.184

[Felix Cheung]

Great. Could someone open a JIRA on this? Unless the policy is changing, this 
can be a blocker for Presto for 0.8 release

Re: Cloudera Data Science Workbench and Zeppelin

[Felix Cheung]

Having used it myself, it looks to be a different technology at several 
different levels, so I'm not sure it is based on Zeppelin in any way.



From: Mich Talebzadeh 
Sent: Sunday, September 10, 2017 1:07:55 AM
To: users@zeppelin.apache.org
Subject: Re: Cloudera Data Science Workbench and Zeppelin

I was under the impression that it was based on zeppelin or originally started 
based on it.

it would be interesting to see how it was developed as it is a relatively new 
comer.

HTH


Dr Mich Talebzadeh



LinkedIn  
https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw



http://talebzadehmich.wordpress.com


Disclaimer: Use it at your own risk. Any and all responsibility for any loss, 
damage or destruction of data or any other property which may arise from 
relying on this email's technical content is explicitly disclaimed. The author 
will in no case be liable for any monetary damages arising from such loss, 
damage or destruction.



On 10 September 2017 at 08:54, Jeff Zhang 
> wrote:

I haven't use Cloudera Data Science workbench, this is the first time I heard 
that Cloudera Data Science workbench is using zeppelin. Are you sure about that 
?



Mich Talebzadeh 
>于2017年9月10日周日 
下午3:13写道：
Hi,

As I understood Cloudera Data Science Workbench was based on Zeppelin. Zeppelin 
as open source was supported by Hortonworks and Cloudera took Zeppelin and 
created the Workbench.

As anyone within Zeppelin community come across this Workbench and how do you 
rate it?

Thanks



Dr Mich Talebzadeh



LinkedIn  
https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw



http://talebzadehmich.wordpress.com


Disclaimer: Use it at your own risk. Any and all responsibility for any loss, 
damage or destruction of data or any other property which may arise from 
relying on this email's technical content is explicitly disclaimed. The author 
will in no case be liable for any monetary damages arising from such loss, 
damage or destruction.

Re: [DISCUSS] Reducing default interpreters while building and releasing Zeppelin

[Felix Cheung]

Sure - I think it will be important to discuss what criteria to use to decide 
what is included vs what will be released separately.

_
From: Jongyoul Lee <jongy...@gmail.com<mailto:jongy...@gmail.com>>
Sent: Sunday, June 4, 2017 9:47 AM
Subject: Re: [DISCUSS] Reducing default interpreters while building and 
releasing Zeppelin
To: dev <d...@zeppelin.apache.org<mailto:d...@zeppelin.apache.org>>
Cc: <users@zeppelin.apache.org<mailto:users@zeppelin.apache.org>>


It means we release with some interpreters and deploy all interpreters into
maven separately. We already had a install-interpreter script inside it. If
someone wants to install specific interpreter not included in default
release package, they can use that script to install specific one.

On Sun, Jun 4, 2017 at 9:11 AM, Felix Cheung 
<felixcheun...@hotmail.com<mailto:felixcheun...@hotmail.com>>
wrote:

> Are we proposing some interpreters to be built and released separately?
>
> Is this going to be separate packaging? Or separate release pipeline?
>
>
> _
> From: Jongyoul Lee 
> <jongy...@gmail.com<mailto:jongy...@gmail.com><mailto:jongy...@gmail.com>>
> Sent: Friday, June 2, 2017 11:04 PM
> Subject: [DISCUSS] Reducing default interpreters while building and
> releasing Zeppelin
> To: dev 
> <d...@zeppelin.apache.org<mailto:d...@zeppelin.apache.org><mailto:d...@zeppelin.apache.org>>,
>  <
> users@zeppelin.apache.org<mailto:users@zeppelin.apache.org><mailto:users@zeppelin.apache.org>>
>
>
> Hi dev and users,
>
> Recently, 
> zeppelin.apache.org<http://zeppelin.apache.org><http://zeppelin.apache.org> 
> is being
> changed for increasing user experiences and convenience. I like this kind
> of changes. I, however, saw some arguments that which interpreters we will
> locate in the first page. I'd like to expand its argument to the package we
> release.
>
> Current zeppelin packages exceed 700MB with default option because
> Zeppelin tried to include all interpreters by default. It was good at the
> early age but, nowadays, Zeppelin community suffer from the size because
> ASF infra allows the package size under 500MB. So I'd like to reduce the
> package size by reducing default packages.
>
> In case of rebuilding homepage, community proposed some criteria including
> mailing list and # of question in stackoverflow. I think we can adapt same
> criteria into release version of Zeppelin.
>
> To handle this kind of issue, I think consensus of community is the most
> important factor. If someone wants to have an idea to deal with it, please
> feel free to talk about it.
>
> Thanks,
> Jongyoul Lee
>
> --
> 이종열, Jongyoul Lee, 李宗烈
> http://madeng.net
>
>
>


--
이종열, Jongyoul Lee, 李宗烈
http://madeng.net

Re: [DISCUSS] Reducing default interpreters while building and releasing Zeppelin

[Felix Cheung]

Are we proposing some interpreters to be built and released separately?

Is this going to be separate packaging? Or separate release pipeline?


_
From: Jongyoul Lee >
Sent: Friday, June 2, 2017 11:04 PM
Subject: [DISCUSS] Reducing default interpreters while building and releasing 
Zeppelin
To: dev >, 
>


Hi dev and users,

Recently, zeppelin.apache.org is being changed for 
increasing user experiences and convenience. I like this kind of changes. I, 
however, saw some arguments that which interpreters we will locate in the first 
page. I'd like to expand its argument to the package we release.

Current zeppelin packages exceed 700MB with default option because Zeppelin 
tried to include all interpreters by default. It was good at the early age but, 
nowadays, Zeppelin community suffer from the size because ASF infra allows the 
package size under 500MB. So I'd like to reduce the package size by reducing 
default packages.

In case of rebuilding homepage, community proposed some criteria including 
mailing list and # of question in stackoverflow. I think we can adapt same 
criteria into release version of Zeppelin.

To handle this kind of issue, I think consensus of community is the most 
important factor. If someone wants to have an idea to deal with it, please feel 
free to talk about it.

Thanks,
Jongyoul Lee

--
이종열, Jongyoul Lee, 李宗烈
http://madeng.net

Re: Livy - add external libraries from additional maven repo

[Felix Cheung]

How are you setting this:
When setting my local maven repo and the "groupId:artifactId:version" in the 
interpreter settings


From: Theofilos Kakantousis <t...@kth.se>
Sent: Wednesday, May 31, 2017 1:56:07 AM
To: users@zeppelin.apache.org
Subject: Re: Livy - add external libraries from additional maven repo

Thanks everyone for the feedback!

Indeed %dep works only for Spark interpreter, just mentioned it to show the 
interpreter behavior I expected with Livy.
When setting my local maven repo and the "groupId:artifactId:version" in the 
interpreter settings, I can see the dependency (i.e. a jar file) being 
downloaded to the local OS under "local-repo" directory but the dependency is 
not deployed with the Spark application in YARN.

Cheers,
Theo

On 2017-05-31 01:19, Ben Vogan wrote:
For what it's worth I have successfully added jar files and maven packages to 
sessions using zeppelin & livy 0.3 - although not using %dep.  In the 
interpreter settings I set the livy.spark.jars setting for jars that are on my 
HDFS cluster, and livy.spark.jars.packages for maven packages - although only 
using maven central and not a local repo.

--Ben

On Tue, May 30, 2017 at 12:36 PM, Felix Cheung 
<felixcheun...@hotmail.com<mailto:felixcheun...@hotmail.com>> wrote:
To add, this might be an issue with Livy.

I'm seeing something similar as well.

If you can get a repo with calling the Livy REST API directly it will be 
worthwhile to follow up with the Livy community separately.



From: Felix Cheung <felixcheun...@hotmail.com<mailto:felixcheun...@hotmail.com>>
Sent: Tuesday, May 30, 2017 11:34:31 AM
To: users@zeppelin.apache.org<mailto:users@zeppelin.apache.org>; 
users@zeppelin.apache.org<mailto:users@zeppelin.apache.org>
Subject: Re: Livy - add external libraries from additional maven repo

if I recall, %dep only works with the built in Spark interpreter and not the 
Livy interpreter.

To manage dependency win Livy you will need to set Spark conf with Livy.


From: Theofilos Kakantousis <t...@kth.se<mailto:t...@kth.se>>
Sent: Tuesday, May 30, 2017 9:05:15 AM
To: users@zeppelin.apache.org<mailto:users@zeppelin.apache.org>
Subject: Livy - add external libraries from additional maven repo

Hi everyone,

I'm using Zeppelin with Livy 0.4 and trying to add external libraries from an 
additional maven repo to my application according to the documentation 
available 
here<https://zeppelin.apache.org/docs/0.7.0/interpreter/livy.html#adding-external-libraries>.
 The example works fine, but when I set the livy.spark.jars.packages to my 
library the interpreter throws an unresolved dependency error.

I have added the additional maven repository in the interpreter settings and 
have also tried setting livy.spark.jars.ivy but without luck. However, if I use 
the Spark interpreter with the following code it works fine.

"%dep
z.reset();
z.addRepo("my repo").url("http://myrepo;<http://myrepo>).snapshot
z.load("mygroup:myartifact:myversion");

Has anyone managed to do that with Livy? Thanks!

Cheers,
Theo



--
BENJAMIN VOGAN | Data Platform Team Lead

[https://docs.google.com/uc?export=download=0B2SW57jgJhgaOGNLSXhCaUNoMzQ=0B2SW57jgJhgaNm1lM1Q4NXlrN1VkcmQ5MG1TNWs1L0pOdkZBPQ]<http://www.shopkick.com/>
[https://docs.google.com/uc?export=download=0B2SW57jgJhgaU0o1Zy1YWmhNN0k=0B2SW57jgJhgaMG1hZXJ0b0hQYzNvVnhyRlhFV3R0aDh6eEhVPQ]<https://www.facebook.com/shopkick>[https://docs.google.com/uc?export=download=0B2SW57jgJhgaNjNwelg2Y1lwdkE=0B2SW57jgJhgaR0xZWG5xajFGQk5VRWtTNC9yV09lbmVQeDRRPQ]<https://www.instagram.com/shopkick/>[https://docs.google.com/uc?export=download=0B2SW57jgJhgaTU1MQjdpaG96N0E=0B2SW57jgJhgaazkyTkU1VlF5UFBrYmFuZklhMFA1T3NqZklvPQ]<https://www.pinterest.com/shopkick/>[https://docs.google.com/uc?export=download=0B2SW57jgJhgaQVJzbkc2TUVqOEU=0B2SW57jgJhgaU2Jpb3hWQlZoVWFsQjl0aHBrVVZpMXp0VFYwPQ]<https://twitter.com/shopkickbiz>[https://docs.google.com/uc?export=download=0B2SW57jgJhgaMHBINExPa1NLTVE=0B2SW57jgJhgacUlnaGRqNVJDNlhCTVlodHloeUR5aXNoZlp3PQ]<https://www.linkedin.com/company-beta/831240/?pathWildcard=831240>

Re: Livy - add external libraries from additional maven repo

[Felix Cheung]

To add, this might be an issue with Livy.

I'm seeing something similar as well.

If you can get a repo with calling the Livy REST API directly it will be 
worthwhile to follow up with the Livy community separately.



From: Felix Cheung <felixcheun...@hotmail.com>
Sent: Tuesday, May 30, 2017 11:34:31 AM
To: users@zeppelin.apache.org; users@zeppelin.apache.org
Subject: Re: Livy - add external libraries from additional maven repo

if I recall, %dep only works with the built in Spark interpreter and not the 
Livy interpreter.

To manage dependency win Livy you will need to set Spark conf with Livy.


From: Theofilos Kakantousis <t...@kth.se>
Sent: Tuesday, May 30, 2017 9:05:15 AM
To: users@zeppelin.apache.org
Subject: Livy - add external libraries from additional maven repo

Hi everyone,

I'm using Zeppelin with Livy 0.4 and trying to add external libraries from an 
additional maven repo to my application according to the documentation 
available 
here<https://zeppelin.apache.org/docs/0.7.0/interpreter/livy.html#adding-external-libraries>.
 The example works fine, but when I set the livy.spark.jars.packages to my 
library the interpreter throws an unresolved dependency error.

I have added the additional maven repository in the interpreter settings and 
have also tried setting livy.spark.jars.ivy but without luck. However, if I use 
the Spark interpreter with the following code it works fine.

"%dep
z.reset();
z.addRepo("my repo").url("http://myrepo;<http://myrepo>).snapshot
z.load("mygroup:myartifact:myversion");

Has anyone managed to do that with Livy? Thanks!

Cheers,
Theo

Re: Livy - add external libraries from additional maven repo

[Felix Cheung]

if I recall, %dep only works with the built in Spark interpreter and not the 
Livy interpreter.

To manage dependency win Livy you will need to set Spark conf with Livy.


From: Theofilos Kakantousis 
Sent: Tuesday, May 30, 2017 9:05:15 AM
To: users@zeppelin.apache.org
Subject: Livy - add external libraries from additional maven repo

Hi everyone,

I'm using Zeppelin with Livy 0.4 and trying to add external libraries from an 
additional maven repo to my application according to the documentation 
available 
here.
 The example works fine, but when I set the livy.spark.jars.packages to my 
library the interpreter throws an unresolved dependency error.

I have added the additional maven repository in the interpreter settings and 
have also tried setting livy.spark.jars.ivy but without luck. However, if I use 
the Spark interpreter with the following code it works fine.

"%dep
z.reset();
z.addRepo("my repo").url("http://myrepo;).snapshot
z.load("mygroup:myartifact:myversion");

Has anyone managed to do that with Livy? Thanks!

Cheers,
Theo

Re: Permission denied: user=zeppelin while using %spark.pyspark interpreter in AWS EMR cluster

[Felix Cheung]

Seems to be with hdfs ACL - does the service user Zeppelin have access to your 
storage?


From: BigData Consultant 
Sent: Friday, May 26, 2017 10:56:31 PM
To: d...@zeppelin.apache.org; users@zeppelin.apache.org
Subject: Permission denied: user=zeppelin while using %spark.pyspark 
interpreter in AWS EMR cluster

Hi Team,

I have created pyspark structure streaming program and trying to execute in
the Zeppelin notebook, I am getting the following error:

Py4JJavaError: An error occurred while calling o191.start.
: org.apache.hadoop.security.AccessControlException: Permission denied:
user=zeppelin, access=WRITE,
inode="/mnt/tmp/temporary-e0cf0f09-a6f4-44d6-9a72-324660085608/metadata":hdfs:hadoop:drwxr-xr-x


I am using Zeppelin Notebook Version 0.7.1 in AWS EMR cluster.

Help would be much appreciated.

*Full stacktrace:*

Traceback (most recent call last):
File "/tmp/zeppelin_pyspark-8165971491474576109.py", line 349, in 
raise Exception(traceback.format_exc())
Exception: Traceback (most recent call last):
File "/tmp/zeppelin_pyspark-8165971491474576109.py", line 342, in 
exec(code)
File "", line 5, in 
File "/usr/lib/spark/python/pyspark/sql/streaming.py", line 816, in start
return self._sq(self._jwrite.start())
File "/usr/lib/spark/python/lib/py4j-0.10.4-src.zip/py4j/java_gateway.py",
line 1133, in __call__
answer, self.gateway_client, self.target_id, self.name)
File "/usr/lib/spark/python/pyspark/sql/utils.py", line 63, in deco
return f(*a, **kw)
File "/usr/lib/spark/python/lib/py4j-0.10.4-src.zip/py4j/protocol.py", line
319, in get_return_value
format(target_id, ".", name), value)
Py4JJavaError: An error occurred while calling o191.start.
: org.apache.hadoop.security.AccessControlException: Permission denied:
user=zeppelin, access=WRITE,
inode="/mnt/tmp/temporary-e0cf0f09-a6f4-44d6-9a72-324660085608/metadata":hdfs:hadoop:drwxr-xr-x
at
org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:320)
at
org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:292)
at
org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:213)
at
org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:190)
at
org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:1728)
at
org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:1712)
at
org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkAncestorAccess(FSDirectory.java:1695)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFileInternal(FSNamesystem.java:2515)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFileInt(FSNamesystem.java:2450)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFile(FSNamesystem.java:2334)
at
org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.create(NameNodeRpcServer.java:624)
at
org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.create(ClientNamenodeProtocolServerSideTranslatorPB.java:397)
at
org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
at
org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:616)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:982)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2049)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2045)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2045)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at
org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:106)
at
org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:73)
at
org.apache.hadoop.hdfs.DFSOutputStream.newStreamForCreate(DFSOutputStream.java:1653)
at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:1689)
at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:1624)
at
org.apache.hadoop.hdfs.DistributedFileSystem$7.doCall(DistributedFileSystem.java:448)
at
org.apache.hadoop.hdfs.DistributedFileSystem$7.doCall(DistributedFileSystem.java:444)
at
org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
at
org.apache.hadoop.hdfs.DistributedFileSystem.create(DistributedFileSystem.java:459)
at

Re: Hive Reserve Keyword support

[Felix Cheung]

I think you can put backticks around the name date

https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL


From: Jongyoul Lee 
Sent: Tuesday, May 9, 2017 10:33:50 AM
To: users@zeppelin.apache.org
Subject: Re: Hive Reserve Keyword support

If it's possible for you to pass that properties when you create a connection, 
you can passes it by setting it into interpreter setting

On Sat, Apr 29, 2017 at 4:25 PM, Dibyendu Bhattacharya 
> wrote:
Hi,

I have a Hive Table which has a column named date. When I tried to query using 
Zeppelin %jdbc interpreter , I got bellow error.


Error while compiling statement: FAILED: ParseException line 1:312 Failed to 
recognize predicate 'date'. Failed rule: 'identifier' in expression 
specification
class org.apache.hive.service.cli.HiveSQLException
org.apache.hive.jdbc.Utils.verifySuccess(Utils.java:231)
org.apache.hive.jdbc.Utils.verifySuccessWithInfo(Utils.java:217)
org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:254)
org.apache.zeppelin.jdbc.JDBCInterpreter.executeSql(JDBCInterpreter.java:322)
org.apache.zeppelin.jdbc.JDBCInterpreter.interpret(JDBCInterpreter.java:408)
org.apache.zeppelin.interpreter.LazyOpenInterpreter.interpret(LazyOpenInterpreter.java:94)
org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:341)
org.apache.zeppelin.scheduler.Job.run(Job.java:176)
org.apache.zeppelin.scheduler.ParallelScheduler$JobRunner.run(ParallelScheduler.java:162)


My query looks like this :

select x,y,z from mytable where date = '2017-04-28"

I believe it is failing because date is reserve keyword . Is there anyway I can 
set  hive.support.sql11.reserved.keywords=false in Zeppelin ?

regards,
Dibyendu






--
이종열, Jongyoul Lee, 李宗烈
http://madeng.net

Re: UnicodeDecodeError in zeppelin 0.7.1

[Felix Cheung]

And are they running with the same Python version? What is the Python version?

_
From: moon soo Lee >
Sent: Thursday, April 20, 2017 11:53 AM
Subject: Re: UnicodeDecodeError in zeppelin 0.7.1
To: >


Hi,

0.7.1 didn't changed any encoding type as far as i know.
One difference is 0.7.1 official artifact has been built with JDK8 while 0.7.0 
built with JDK7 (we'll use JDK7 to build upcoming 0.7.2 binary). But i'm not 
sure that can make pyspark and spark encoding type changes.

Do you have exactly the same interpreter setting in 0.7.1 and 0.7.0?

Thanks,
moon

On Wed, Apr 19, 2017 at 5:30 AM Meethu Mathew 
> wrote:
Hi,

I just migrated from zeppelin 0.7.0 to zeppelin 0.7.1 and I am facing this 
error while creating an RDD(in pyspark).

UnicodeDecodeError: 'utf8' codec can't decode byte 0x80 in position 0: invalid 
start byte

I was able to create the RDD without any error after adding use_unicode=False 
as follows
sc.textFile("file.csv",use_unicode=False)

​But it fails when I try to stem the text. I am getting similar error when 
trying to apply stemming to the text using python interpreter.

UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 4: ordinal 
not in range(128)

All these code is working in 0.7.0 version. There is no change in the dataset 
and code. ​Is there any change in the encoding type in the new version of 
zeppelin?


Regards,

Meethu Mathew

Re: Roadmap for 0.8.0

[Felix Cheung]

+1 on quality and stabilization.

I'm not sure if releasing as preview or calling it unstable fits with the ASF 
release process though.

Other projects have code freeze, RC (and longer RC iteration time) etc. - do we 
think those will help improve quality when the release is finally cut?


_
From: Jianfeng (Jeff) Zhang 
<jzh...@hortonworks.com<mailto:jzh...@hortonworks.com>>
Sent: Monday, March 20, 2017 6:13 PM
Subject: Re: Roadmap for 0.8.0
To: <users@zeppelin.apache.org<mailto:users@zeppelin.apache.org>>, dev 
<d...@zeppelin.apache.org<mailto:d...@zeppelin.apache.org>>



Strongly +1 for adding system test for different interpreter modes and focus on 
bug fixing than new features. I do heard from some users complain about the 
bugs of zeppelin major release. A stabilized release is very necessary for 
community.




Best Regard,
Jeff Zhang


From: moon soo Lee 
<m...@apache.org<mailto:m...@apache.org><mailto:m...@apache.org>>
Reply-To: 
"users@zeppelin.apache.org<mailto:users@zeppelin.apache.org><mailto:users@zeppelin.apache.org>"
 
<users@zeppelin.apache.org<mailto:users@zeppelin.apache.org><mailto:users@zeppelin.apache.org>>
Date: Tuesday, March 21, 2017 at 4:10 AM
To: 
"users@zeppelin.apache.org<mailto:users@zeppelin.apache.org><mailto:users@zeppelin.apache.org>"
 
<users@zeppelin.apache.org<mailto:users@zeppelin.apache.org><mailto:users@zeppelin.apache.org>>,
 dev 
<d...@zeppelin.apache.org<mailto:d...@zeppelin.apache.org><mailto:d...@zeppelin.apache.org>>
Subject: Re: Roadmap for 0.8.0

Great to see discussion for 0.8.0.
List of features for 0.8.0 looks really good.

Interpreter factory refactoring
Interpreter layer supports various behavior depends on combination of 
PerNote,PerUser / Shared,Scoped,Isolated. We'll need strong test cases for each 
combination as a first step.
Otherwise, any pullrequest will silently break one of behavior at any time no 
matter we refactor or not. And fixing and testing this behavior is so hard.
Once we have complete test cases, not only guarantee the behavior but also make 
refactoring much easier.


0.8.0 release
I'd like to suggest improvements on how we release a new version.

In the past, 0.6.0 and 0.7.0 release with some critical problems. (took 3 
months to stabilize 0.6 and we're working on stabilizing 0.7.0 for 2 months)

I think the same thing will happen again with 0.8.0, while we're going to make 
lots of changes and add many new features.
After we released 0.8.0, while 'Stabilizing' the new release, user who tried 
the new release may get wrong impression of the quality. Which is very bad and 
we already repeated the mistake in 0.6.0 and 0.7.0.

So from 0.8.0 release, I'd suggest we improve way we release new version to 
give user proper expectation. I think there're several ways of doing it.

1. Release 0.8.0-preview officially and then release 0.8.0.
2. Release 0.8.0 with 'beta' or 'unstable' label. And keep 0.7.x as a 'stable' 
release in the download page. Once 0.8.x release becomes stable enough make 
0.8.x release as a 'stable' and move 0.7.x to 'old' releases.


After 0.8.0,
Since Zeppelin projects starts, project went through some major milestone, like

- project gets first users and first contributor
- project went into Apache Incubator
- project became TLP.

And I think it's time to think about hitting another major milestone.

Considering features we already have, features we're planning on 0.8, wide 
adoption of Zeppelin in the industry, I think it's time to focus on make 
project more mature and make a 1.0 release. Which i think big milestone for the 
project.

After 0.8.0 release, I suggest we more focus on bug fixes, stability 
improvement, optimizing user experience than adding new features. And with 
subsequent minor release, 0.8.1, 0.8.2 ... moment we feel confident about the 
quality, release it as a 1.0.0 instead of 0.8.x.

Once we have 1.0.0 released, then I think we can make larger, experimental 
changes on 2.0.0 branch aggressively, while we keep maintaining 1.0.x branch.


Thanks,
moon

On Mon, Mar 20, 2017 at 8:55 AM Felix Cheung 
<felixcheun...@hotmail.com<mailto:felixcheun...@hotmail.com><mailto:felixcheun...@hotmail.com>>
 wrote:
There are several pending visualization improvements/PRs that would be very 
good to get them in as well.



From: Jongyoul Lee 
<jongy...@gmail.com<mailto:jongy...@gmail.com><mailto:jongy...@gmail.com>>
Sent: Sunday, March 19, 2017 9:03:24 PM
To: dev; 
users@zeppelin.apache.org<mailto:users@zeppelin.apache.org><mailto:users@zeppelin.apache.org>
Subject: Roadmap for 0.8.0

Hi dev & users,

Recently, community submits very new features for Apache Zeppelin. I think it's 
very positive signals to improve Apache Zeppelin and its community. But

Re: Roadmap for 0.8.0

[Felix Cheung]

There are several pending visualization improvements/PRs that would be very 
good to get them in as well.



From: Jongyoul Lee 
Sent: Sunday, March 19, 2017 9:03:24 PM
To: dev; users@zeppelin.apache.org
Subject: Roadmap for 0.8.0

Hi dev & users,

Recently, community submits very new features for Apache Zeppelin. I think it's 
very positive signals to improve Apache Zeppelin and its community. But in 
another aspect, we should focus on what the next release includes. I think we 
need to summarize and prioritize them. Here is what I know:

* Cluster management
* Admin feature
* Replace some context to separate users
* Helium online

Feel free to talk if you want to add more things. I think we need to choose 
which features will be included in 0.8.0, too.

Regards,
Jongyoul Lee

--
이종열, Jongyoul Lee, 李宗烈
http://madeng.net

Re: Tensorflow not working in Zeppelin 0.7.0

[Felix Cheung]

Right that addresses the error. Thanks!!

My original point was that in addition to the uninitialized variable, we could 
also enhance interactive mode detection - as of now interactive mode is always 
off for Zeppelin.


_
From: moon soo Lee <m...@apache.org<mailto:m...@apache.org>>
Sent: Friday, March 17, 2017 2:07 PM
Subject: Re: Tensorflow not working in Zeppelin 0.7.0
To: <users@zeppelin.apache.org<mailto:users@zeppelin.apache.org>>


Saw patch [1] has been made in Tensorflow.
Thanks Sriram for taking care of this issue!

Best,
moon

[1] https://github.com/tensorflow/tensorflow/pull/8420

On Wed, Mar 15, 2017 at 8:32 PM Sriram Narayanamoorthy 
<nsrira...@gmail.com<mailto:nsrira...@gmail.com>> wrote:
We fixed the issue on tensorflow by a simply initializing the _interactive 
variable in the logging file. And I tested it out on both Zeppelin and Jupyter 
and it seems to work. Do you have a better version of this in mind?
On Tue, Mar 14, 2017 at 8:45 PM Felix Cheung 
<felixcheun...@hotmail.com<mailto:felixcheun...@hotmail.com>> wrote:
I realize there is a discussion on be Tensorflow issue, aside from the thought 
of initializing the variables,  I think they should broaden their detection of 
interactive shell to include Zeppelin, Jupyter, etc.



From: Sriram Narayanamoorthy <nsrira...@gmail.com<mailto:nsrira...@gmail.com>>
Sent: Tuesday, March 14, 2017 4:34:13 PM
To: users@zeppelin.apache.org<mailto:users@zeppelin.apache.org>

Subject: Re: Tensorflow not working in Zeppelin 0.7.0
Python 3.5 and Tensorflow 1.0

On Tue, Mar 14, 2017 at 4:21 PM, Jianfeng (Jeff) 
Zhang<jzh...@hortonworks.com<mailto:jzh...@hortonworks.com>> wrote:
Which version of tensorflow and python do you use ?  I believe I use it before 
and works for me.


Best Regard,
Jeff Zhang


From: Sriram Narayanamoorthy <nsrira...@gmail.com<mailto:nsrira...@gmail.com>>
Reply-To: "users@zeppelin.apache.org<mailto:users@zeppelin.apache.org>" 
<users@zeppelin.apache.org<mailto:users@zeppelin.apache.org>>
Date: Wednesday, March 15, 2017 at 2:19 AM
To: "users@zeppelin.apache.org<mailto:users@zeppelin.apache.org>" 
<users@zeppelin.apache.org<mailto:users@zeppelin.apache.org>>
Subject: Re: Tensorflow not working in Zeppelin 0.7.0

Thanks.

I have submitted a Jira ticket: 
https://issues.apache.org/jira/browse/ZEPPELIN-2259
Also, opened an issue on tensorflow: 
https://github.com/tensorflow/tensorflow/issues/8401


On Tue, Mar 14, 2017 at 8:38 AM, Felix Cheung 
<felixcheun...@hotmail.com<mailto:felixcheun...@hotmail.com>> wrote:
This sounds like more an issue with Tensorflow when working in notebook 
environment in general.

Would you please open a JIRA. Also feel free to open an issue on Tensorflow as 
well.



From: Sriram Narayanamoorthy <nsrira...@gmail.com<mailto:nsrira...@gmail.com>>
Sent: Tuesday, March 14, 2017 7:28:12 AM
To: users@zeppelin.apache.org<mailto:users@zeppelin.apache.org>
Subject: Tensorflow not working in Zeppelin 0.7.0

Hi all,

There seems to be some issue using Tensorflow in Zeppelin 0.7.0 and it throws 
this error:
NameError: name '_interactive' is not defined

Another user reported the same 
issue<http://stackoverflow.com/questions/42757433/tensorflow-can-not-work-with-zeppelin>
 on SO. The fix suggested there seems to be really hacky.

The issue seems to be that in tf_logging.py file 
[.\Anaconda3\Lib\site-packages\tensorflow\python\platform\tf_logging.py] the 
_interactive variable is not triggering for Zeppelin.

# Determine whether we are in an interactive environment try: # This is only 
defined in interactive shells if _sys.ps1: _interactive = True except 
AttributeError: # Even now, we may be in an interactive shell with `python -i`. 
_interactive = _sys.flags.interactive

Any thoughts why? I am using Zeppelin on Windows.

Best,
-Sriram

--
Sriram Narayanamoorthy



--
Sriram Narayanamoorthy



--
Sriram Narayanamoorthy

Re: Tensorflow not working in Zeppelin 0.7.0

[Felix Cheung]

I realize there is a discussion on be Tensorflow issue, aside from the thought 
of initializing the variables,  I think they should broaden their detection of 
interactive shell to include Zeppelin, Jupyter, etc.



From: Sriram Narayanamoorthy <nsrira...@gmail.com>
Sent: Tuesday, March 14, 2017 4:34:13 PM
To: users@zeppelin.apache.org
Subject: Re: Tensorflow not working in Zeppelin 0.7.0

Python 3.5 and Tensorflow 1.0

On Tue, Mar 14, 2017 at 4:21 PM, Jianfeng (Jeff) Zhang 
<jzh...@hortonworks.com<mailto:jzh...@hortonworks.com>> wrote:
Which version of tensorflow and python do you use ?  I believe I use it before 
and works for me.


Best Regard,
Jeff Zhang


From: Sriram Narayanamoorthy <nsrira...@gmail.com<mailto:nsrira...@gmail.com>>
Reply-To: "users@zeppelin.apache.org<mailto:users@zeppelin.apache.org>" 
<users@zeppelin.apache.org<mailto:users@zeppelin.apache.org>>
Date: Wednesday, March 15, 2017 at 2:19 AM
To: "users@zeppelin.apache.org<mailto:users@zeppelin.apache.org>" 
<users@zeppelin.apache.org<mailto:users@zeppelin.apache.org>>
Subject: Re: Tensorflow not working in Zeppelin 0.7.0

Thanks.

I have submitted a Jira ticket: 
https://issues.apache.org/jira/browse/ZEPPELIN-2259
Also, opened an issue on tensorflow: 
https://github.com/tensorflow/tensorflow/issues/8401


On Tue, Mar 14, 2017 at 8:38 AM, Felix Cheung 
<felixcheun...@hotmail.com<mailto:felixcheun...@hotmail.com>> wrote:
This sounds like more an issue with Tensorflow when working in notebook 
environment in general.

Would you please open a JIRA. Also feel free to open an issue on Tensorflow as 
well.



From: Sriram Narayanamoorthy <nsrira...@gmail.com<mailto:nsrira...@gmail.com>>
Sent: Tuesday, March 14, 2017 7:28:12 AM
To: users@zeppelin.apache.org<mailto:users@zeppelin.apache.org>
Subject: Tensorflow not working in Zeppelin 0.7.0

Hi all,

There seems to be some issue using Tensorflow in Zeppelin 0.7.0 and it throws 
this error:
NameError: name '_interactive' is not defined

Another user reported the same 
issue<http://stackoverflow.com/questions/42757433/tensorflow-can-not-work-with-zeppelin>
 on SO. The fix suggested there seems to be really hacky.

The issue seems to be that in tf_logging.py file 
[.\Anaconda3\Lib\site-packages\tensorflow\python\platform\tf_logging.py] the 
_interactive variable is not triggering for Zeppelin.

# Determine whether we are in an interactive environment try: # This is only 
defined in interactive shells if _sys.ps1: _interactive = True except 
AttributeError: # Even now, we may be in an interactive shell with `python -i`. 
_interactive = _sys.flags.interactive

Any thoughts why? I am using Zeppelin on Windows.

Best,
-Sriram

--
Sriram Narayanamoorthy



--
Sriram Narayanamoorthy



--
Sriram Narayanamoorthy

Re: Tensorflow not working in Zeppelin 0.7.0

[Felix Cheung]

This sounds like more an issue with Tensorflow when working in notebook 
environment in general.

Would you please open a JIRA. Also feel free to open an issue on Tensorflow as 
well.



From: Sriram Narayanamoorthy 
Sent: Tuesday, March 14, 2017 7:28:12 AM
To: users@zeppelin.apache.org
Subject: Tensorflow not working in Zeppelin 0.7.0

Hi all,

There seems to be some issue using Tensorflow in Zeppelin 0.7.0 and it throws 
this error:
NameError: name '_interactive' is not defined

Another user reported the same 
issue
 on SO. The fix suggested there seems to be really hacky.

The issue seems to be that in tf_logging.py file 
[.\Anaconda3\Lib\site-packages\tensorflow\python\platform\tf_logging.py] the 
_interactive variable is not triggering for Zeppelin.

# Determine whether we are in an interactive environment try: # This is only 
defined in interactive shells if _sys.ps1: _interactive = True except 
AttributeError: # Even now, we may be in an interactive shell with `python -i`. 
_interactive = _sys.flags.interactive

Any thoughts why? I am using Zeppelin on Windows.

Best,
-Sriram

--
Sriram Narayanamoorthy

Re: How can i convert below DataFrame array to string

[Felix Cheung]

Explode?

https://spark.apache.org/docs/latest/api/python/pyspark.sql.html?highlight=explode#pyspark.sql.functions.explode


For Spark questions like this, try stackoverflow

spark.apache.org/community.html




From: Suresh Ayyavoo 
Sent: Sunday, March 12, 2017 8:53:52 PM
To: users@zeppelin.apache.org
Subject: How can i convert below DataFrame array to string

Hi All,

How can i convert below DataFrame [accountID: array, completed_at: 
array, created_at: array] to
[accountID: string, completed_at: string, created_at: string]


root

 |-- accountID: array (nullable = true)

 ||-- element: string (containsNull = true)

 |-- completed_at: array (nullable = true)

 ||-- element: string (containsNull = true)

 |-- created_at: array (nullable = true)

 ||-- element: string (containsNull = true)


root
|-- accountID: string (containsNull =true)
|-- completed_at: string (containsNull =true)
|--create_at : string (containsNull =true)


Thanks

Re: Another instance of Derby already booted the database .../metastore_db

[Felix Cheung]

The issue is since you are running Spark interpreter in isolated mode there are 
two running Spark driver process trying to access derby metastore at the same 
location, at the same time.

There are some config in Spark that could redirect metastore to different 
location (try spark.warehouse.dir) but ideally you want to be able to share 
temp view and so on and would need to have a shared metastore. In such case it 
would be best to use the external Hive metastore server instead - making sure 
hive-site.xml is available would be the way to go.


_
From: Paul Brenner >
Sent: Friday, January 27, 2017 8:01 AM
Subject: Another instance of Derby already booted the database .../metastore_db
To: >



We had this same problem. Copying or symlinking hive-site.xml to the zeppelin 
conf directory and then restarting solved it for us.

[https://ci3.googleusercontent.com/proxy/tFn1I-GEOnccUtv8DHHEc49-6g3x3CbuQKzbfl2Z1BObEy0Qz6QebJimpP96TK3Za5MXwXTuwBZaobKp22nYAG3NdxAC0Q=s0-d-e1-ft#https://marketing.placeiq.net/images/placeiq.png]
 Paul Brenner  
[https://ci4.googleusercontent.com/proxy/490PXYv9O6OiIp_DL4vuabJqVn53fMon5xNYZdftCVea9ySR2LcFDHe6Cdntb2G68uDAuA6FgLny8wKWLFWpsrPAt_FtLaE=s0-d-e1-ft#https://marketing.placeiq.net/images/twitter1.png]
  
[https://ci3.googleusercontent.com/proxy/fztHf1lRKLQYcAxebqfp2PYXCwVap3GobHVIbyp0j3NcuJOY16bUAZBibVOFf-fd1GsiuhrOfYy6dSwhlCwWU8ZUlw9OX5I=s0-d-e1-ft#https://marketing.placeiq.net/images/facebook.png]
 
 
[https://ci5.googleusercontent.com/proxy/H26ThD7R6DOqxoLTgzi6k5SMrHoF2Tj44xI_7XlD9KfOIiGwe1WIMc5iQBxUBA9EuIyJMdaRXrhZTOrnkrn8O9Rf1FP9UQU=s0-d-e1-ft#https://marketing.placeiq.net/images/linkedin.png]
 
DATA SCIENTIST
(217) 390-3033


[PlaceIQ:Location
 Data 
Accuracy]


On Fri, Jan 27, 2017 at 9:51 AM Jan Rasehorn mailto:Jan%20Rasehorn%20>> wrote:
Hello,

I was running into an issue with a query on a temp table and to test it I 
opened another notebook in another tab. When I execute this new notebook, I got 
an error:

Caused by: java.sql.SQLException: Failed to start database 'metastore_db' with 
class loader 
org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@1ec187c, see the 
next exception for details.
at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown 
Source)
at 
org.apache.derby.impl.jdbc.SQLExceptionFactory40.wrapArgsForTransportAcrossDRDA(Unknown
 Source)
... 132 more
Caused by: java.sql.SQLException: Another instance of Derby may have already 
booted the database /opt/zeppelin-0.6.2/bin/metastore_db

I'm running Spark in isolated mode.

Does anybody know how to fix that?

Re: [VOTE] Release Apache Zeppelin 0.7.0 (RC2)

[Felix Cheung]

I'm seeing the same behavior with the marital status drop down box on the intro 
notebook on netinst build.

I think this could be a bigger problem.


_
From: Khalid Huseynov >
Sent: Sunday, January 22, 2017 8:08 PM
Subject: Re: [VOTE] Release Apache Zeppelin 0.7.0 (RC2)
To: >
Cc: >


I've just tried and was running fine when changing age field in report mode. 
However didn't run on change of marital status neither in report nor in default 
mode. Hopefully more people could verify it. Also as far as i know mentioned 
log isn't related to run feature.

On Sun, Jan 22, 2017 at 1:46 PM, Vinay Shukla 
> wrote:
Downloaded
https://dist.apache.org/repos/dist/dev/zeppelin/zeppelin-0.7.0-rc2/

Then ran Basic Feature tutorial successfully and went to *report* mode and
it does not re-run the paragraph when the marital or age field value is
changed (even after shift +  enter)


INFO [2017-01-22 13:36:21,924] ({qtp1622006612-76}
NotebookServer.java[sendNote]:704) - New operation from 0:0:0:0:0:0:0:1 :
62990 : anonymous : GET_NOTE : 2A94M5J1Z

 WARN [2017-01-22 13:36:21,926] ({qtp1622006612-76}
GitNotebookRepo.java[revisionHistory]:157) - No Head found for 2A94M5J1Z,
No HEAD exists and no explicit starting revision was specified

 INFO [2017-01-22 13:36:26,223] ({qtp1622006612-14}
NotebookServer.java[sendNote]:704) - New operation from 0:0:0:0:0:0:0:1 :
62990 : anonymous : GET_NOTE : 2A94M5J1Z

 WARN [2017-01-22 13:36:26,224] ({qtp1622006612-14}
GitNotebookRepo.java[revisionHistory]:157) - No Head found for 2A94M5J1Z,
No HEAD exists and no explicit starting revision was specified

Did anyone else try this?

On Sun, Jan 22, 2017 at 2:04 AM, Hyung Sung Shim 
> wrote:

> +1
> 2017? 1? 22? (?) ?? 12:54, Ahyoung Ryu 
> >?? ??:
>
>> +1
>>
>> On Sun, Jan 22, 2017 at 5:52 PM, Jeff Zhang 
>> > wrote:
>>
>> > +1
>> >
>> >
>> >
>> > DuyHai Doan 
>> > >?2017?1?22??? ??4:01???
>> >
>> >> +1
>> >>
>> >> On Sun, Jan 22, 2017 at 8:59 AM, Mina Lee 
>> >> > wrote:
>> >>
>> >> I propose the following RC to be released for the Apache Zeppelin 0.7.0
>> >> release.
>> >>
>> >> The commit id is d0ddf8fdff75b4229543f7ebfd5eb001f685382d which is
>> >> corresponds to the tag v0.7.0-rc2:
>> >> *https://git-wip-us.apache.org/repos/asf?p=zeppelin.git;a=commit;h=
>> d0ddf8fdff75b4229543f7ebfd5eb001f685382d
>> >> > d0ddf8fdff75b4229543f7ebfd5eb001f685382d>*
>> >>
>> >> The release archives (tgz), signature, and checksums are here
>> >> https://dist.apache.org/repos/dist/dev/zeppelin/zeppelin-0.7.0-rc2/
>> >>
>> >> The release candidate consists of the following source distribution
>> >> archive
>> >> zeppelin-0.7.0.tgz
>> >>
>> >> In addition, the following supplementary binary distributions are
>> provided
>> >> for user convenience at the same location
>> >> zeppelin-0.7.0-bin-all.tgz
>> >> zeppelin-0.7.0-bin-netinst.tgz
>> >>
>> >> The maven artifacts are here
>> >> https://repository.apache.org/content/repositories/
>> orgapachezeppelin-1023
>> >>
>> >> You can find the KEYS file here:
>> >> https://dist.apache.org/repos/dist/release/zeppelin/KEYS
>> >>
>> >> Release notes available at
>> >> https://issues.apache.org/jira/secure/ReleaseNote.jspa?
>> >> version=12336544=12316221
>> >>
>> >> Vote will be open for next 72 hours (close at 23:59 24/Jan PST).
>> >>
>> >> [ ] +1 approve
>> >> [ ] 0 no opinion
>> >> [ ] -1 disapprove (and reason why)
>> >>
>> >>
>> >>
>>
>

Re: [VOTE] Release Apache Zeppelin 0.7.0 (RC1)

[Felix Cheung]

Thanks. I'd incline to agree having latest matplotlib broken with python 
(python interpreter) seems like a big inconvenience.

Is there a workaround? Let's discuss a bit in the JIRA and bring this back to 
the vote thread.

Also we should really try to mark JIRA too to be included in RC builds, to 
Alex's (bzz) point.

I'll merge this PR now.


_
From: Goodman, Alexander (398K) 
>
Sent: Saturday, January 21, 2017 3:33 PM
Subject: Re: [VOTE] Release Apache Zeppelin 0.7.0 (RC1)
To: >
Cc: >


I also think ZEPPELIN-1980 should be included in 0.7.0

https://issues.apache.org/jira/browse/ZEPPELIN-1980

The reason being, built-in matplotlib integration is a major new feature that 
is slated to be included in 0.7.0, but without this PR, it will be broken for 
users with matplotlib 2.0.0, which was just recently released. The relevant PR 
should be merged shortly.

On Sat, Jan 21, 2017 at 2:34 PM, Alexander Bezzubov 
> wrote:
Hi Jeff,

thank you for the notice, it is a good catch!

But the issue that you mention does not have FixFor version assigned - if
you want it to part of the release and even conside it as important enough
to block the relese vote, could you please update that?

This would simplify drastically the work of release manager.

Also, as you know, the release branch has been unnounced a while a go, but
for this issue as far as I can tell, there was no any indication that it
was intended to be part of it, so seeing -1 is a bit surprising. Do you see
what I mean?

Release Manager can not read minds, as well as volonteer infinite ammount
of time preparing a new Relese Candidate after each issue beeing solved.

Cutting a release is a lot of work for Release Manger, so going further
lets try to simplify life of the brave volonteer helping with release
management! :)

Hope it does make sense! Please, let us know what you think.

--
Alex

On Sat, Jan 21, 2017, 21:55 Jeff Zhang 
> wrote:

> -1,  Can we include ZEPPELIN-1985
>  ? This issue cause
> only user1 can see the pig tutorial note which is a very bad user
> experience.
>
>
>
> Prabhjyot Singh 
> >于2017年1月21日周六 
> 下午7:25写道：
>
> > +1
> >
> > On Jan 21, 2017 4:52 PM, "Jun Kim" 
> > > wrote:
> >
> > +1
> > 2017년 1월 21일 (토) 오후 8:21, CloverHearts 
> > >님이 작성:
> >
> > +1
> >
> >
> >
> > The Zeppelin 0.7 release includes some amazing features.
> >
> > In particular, a major bug fix for the remote interpreter.
> >
> > I am actively in favor of this release.
> >
> >
> >
> >
> >
> > *출발**: *Mina Lee >
> > *회신* *대상**:* >
> > *날짜**: *2017년 1월 21일 토요일 오전 3:17
> > *받는* *사람**: *>, 
> > >
> > *주제**: *[VOTE] Release Apache Zeppelin 0.7.0 (RC1)
> >
> >
> >
> > Hi folks,
> >
> > I propose the following RC to be released for the Apache Zeppelin 0.7.0
> > release.
> >
> > The commit id is bcbe0855175f55eddd10b918303806eb5e5db091 which is
> > corresponds to the tag v0.7.0-rc1:
> >
> >
> >
> https://git-wip-us.apache.org/repos/asf?p=zeppelin.git;a=commit;h=bcbe0855175f55eddd10b918303806eb5e5db091
> >
> > The release archives (tgz), signature, and checksums are here
> > https://dist.apache.org/repos/dist/dev/zeppelin/zeppelin-0.7.0-rc1/
> >
> > The release candidate consists of the following source distribution
> > archive
> >
> > zeppelin-0.7.0.tgz
> >
> > In addition, the following supplementary binary distributions are
> provided
> > for user convenience at the same location
> >
> > zeppelin-0.7.0-bin-all.tgz
> > zeppelin-0.7.0-bin-netinst.tgz
> >
> >
> > The maven artifacts are here
> >
> https://repository.apache.org/content/repositories/orgapachezeppelin-1022
> >
> > You can find the KEYS file here:
> > https://dist.apache.org/repos/dist/release/zeppelin/KEYS
> >
> > Release notes available at
> >
> >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12336544=12316221
> >
> > Vote will be open for next 72 hours (close at 3:30 am 24/Jan PST).
> >
> > [ ] +1 approve
> > [ ] 0 no opinion
> > [ ] -1 disapprove (and reason why)
> >
> > --
> > Taejun Kim
> >
> > Data Mining Lab.
> > School of Electrical and Computer Engineering
> > University of Seoul
> >
> >
>



--
Alex Goodman
Data Scientist I
Science Data Modeling and Computing (398K)
Jet Propulsion Laboratory
California Institute of Technology
Tel: 

Re: Mailing List

[Felix Cheung]

Right, there isn't a lot of traffic on the user list.
Perhaps you can resend your question?



From: Meeraj Kunnumpurath <mee...@servicesymphony.com>
Sent: Tuesday, January 10, 2017 12:05:46 PM
To: users@zeppelin.apache.org
Subject: Re: Mailing List

Thanks Felix. I saw a couple on the interpreter REST API, that was it.

Sent from my iPhone

On Jan 10, 2017, at 10:06 PM, Felix Cheung 
<felixcheun...@hotmail.com<mailto:felixcheun...@hotmail.com>> wrote:

There was a few email yesterday - do you not get them?



From: Meeraj Kunnumpurath 
<mee...@servicesymphony.com<mailto:mee...@servicesymphony.com>>
Sent: Tuesday, January 10, 2017 9:09:52 AM
To: users@zeppelin.apache.org<mailto:users@zeppelin.apache.org>
Subject: Mailing List

Hello

Is this mailing list active? I see hardly any traffic here, just seen
couple of mails over the past three days. I had asked a few questions for
which there has been no response.

Regards

--
Meeraj Kunnumpurath
Director and Executive Principal
Service Symphony Ltd
00 44 7702 693597
00 971 50 409 0169
mee...@servicesymphony.com<mailto:mee...@servicesymphony.com>

Re: Mailing List

[Felix Cheung]

There was a few email yesterday - do you not get them?



From: Meeraj Kunnumpurath 
Sent: Tuesday, January 10, 2017 9:09:52 AM
To: users@zeppelin.apache.org
Subject: Mailing List

Hello

Is this mailing list active? I see hardly any traffic here, just seen
couple of mails over the past three days. I had asked a few questions for
which there has been no response.

Regards

--
Meeraj Kunnumpurath
Director and Executive Principal
Service Symphony Ltd
00 44 7702 693597
00 971 50 409 0169
mee...@servicesymphony.com

Re: py4j doesn't work with python interpreter

[Felix Cheung]

You said you ran

sudo pip install pyplot

I think you need to pip install py4j instead?

_
From: Jianfeng (Jeff) Zhang 
>
Sent: Monday, January 2, 2017 4:24 PM
Subject: Re: py4j doesn't work with python interpreter
To: >


Could you attache the interpreter log ?


Best Regard,
Jeff Zhang


From: Dominique Fabre 
>
Reply-To: "users@zeppelin.apache.org" 
>
Date: Monday, January 2, 2017 at 9:09 PM
To: "users@zeppelin.apache.org" 
>
Subject: py4j doesn't work with python interpreter

Hi all,

I installed py4j with "pipe install" command: sudo pip install pyplot.

When i try to use dynamic form with python intrepeter in a paragraph:

%python
print (z.input("f1","defaultValue"))

i have this result:
You must install py4j Python module (pip install py4j) to use Zeppelin dynamic 
forms features

Is there any other configuration or installation required to run this paragraph?

Thanks

Dominique

Re: Unable to connect with Spark Interpreter

[Felix Cheung]

Hmm possibly with the classpath. These might be Windows specific issues. We 
probably need to debug to fix these.



From: Jan Botorek 
Sent: Tuesday, November 29, 2016 4:01:43 AM
To: users@zeppelin.apache.org
Subject: RE: Unable to connect with Spark Interpreter

Your last advice helped me to progress a little bit:

-  I started spark interpreter manually

o   c:\zepp\\bin\interpreter.cmd, -d, c:\zepp\interpreter\spark\, -p, 61176, 
-l, c:\zepp\/local-repo/2C2ZNEH5W

o   I needed to add a ‚\‘ into the –d attributte and make the path shorter --> 
moved to c:\zepp

-  Then, in Zeppelin web environment I setup the spark interpret to 
„connect to existing process“ (localhost/61176)

-  After that, when I execute any command, in interpreter cmd window 
appears this exception:

o   Exception in thread "pool-1-thread-2" java.lang.NoClassDefFoundError: 
scala/Option

o   at java.lang.Class.forName0(Native Method)

o   at java.lang.Class.forName(Class.java:264)

o   at 
org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer.createInterpreter(RemoteInterpreterServer.java:148)

o   at 
org.apache.zeppelin.interpreter.thrift.RemoteInterpreterService$Processor$createInterpreter.getResult(RemoteInterpreterService.java:1409)

o   at 
org.apache.zeppelin.interpreter.thrift.RemoteInterpreterService$Processor$createInterpreter.getResult(RemoteInterpreterService.java:1394)

o   at 
org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)

o   at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)

o   at 
org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:285)

o   at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)

o   at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)

o   at java.lang.Thread.run(Thread.java:745)

o   Caused by: java.lang.ClassNotFoundException: scala.Option

o   at java.net.URLClassLoader.findClass(URLClassLoader.java:381)

o   at java.lang.ClassLoader.loadClass(ClassLoader.java:424)

o   at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)

o   at java.lang.ClassLoader.loadClass(ClassLoader.java:357)

o   ... 11 more

Is this of any help, please?

Regards,
Jan



From: Jan Botorek [mailto:jan.boto...@infor.com]
Sent: Tuesday, November 29, 2016 12:13 PM
To: users@zeppelin.apache.org
Subject: RE: Unable to connect with Spark Interpreter

I am sorry, but the directory local-repo is not presented in the zeppelin 
folder. I use this (https://zeppelin.apache.org/download.html) newest binary 
version.

Unfortunately, in the 0.6 version downloaded and built from github, also the 
folder local-repo doesn’t exist


From: Jeff Zhang [mailto:zjf...@gmail.com]
Sent: Tuesday, November 29, 2016 10:45 AM
To: users@zeppelin.apache.org
Subject: Re: Unable to connect with Spark Interpreter

I still don't see much useful info. Could you try run the following interpreter 
command directly ?

c:\_libs\zeppelin-0.6.2-bin-all\\bin\interpreter.cmd  -d 
c:\_libs\zeppelin-0.6.2-bin-all\interpreter\spark -p 53099 -l 
c:\_libs\zeppelin-0.6.2-bin-all\/local-repo/2C2ZNEH5W


Jan Botorek >于2016年11月29日周二 
下午5:26写道：
I attach the log file after debugging turned on.

From: Jeff Zhang [mailto:zjf...@gmail.com]
Sent: Tuesday, November 29, 2016 10:04 AM

To: users@zeppelin.apache.org
Subject: Re: Unable to connect with Spark Interpreter

Then I guess the spark process is failed to start so no logs for spark 
interpreter.

Can you use the following log4.properties ? This log4j properties file print 
more error info for further diagnose.

log4j.rootLogger = INFO, dailyfile

log4j.appender.stdout = org.apache.log4j.ConsoleAppender
log4j.appender.stdout.layout = org.apache.log4j.PatternLayout
log4j.appender.stdout.layout.ConversionPattern=%5p [%d] ({%t} %F[%M]:%L) - %m%n

log4j.appender.dailyfile.DatePattern=.-MM-dd
log4j.appender.dailyfile.Threshold = DEBUG
log4j.appender.dailyfile = org.apache.log4j.DailyRollingFileAppender
log4j.appender.dailyfile.File = ${zeppelin.log.file}
log4j.appender.dailyfile.layout = org.apache.log4j.PatternLayout
log4j.appender.dailyfile.layout.ConversionPattern=%5p [%d] ({%t} %F[%M]:%L) - 
%m%n


log4j.logger.org.apache.zeppelin.notebook.Paragraph=DEBUG
log4j.logger.org.apache.zeppelin.scheduler=DEBUG
log4j.logger.org.apache.zeppelin.livy=DEBUG
log4j.logger.org.apache.zeppelin.flink=DEBUG
log4j.logger.org.apache.zeppelin.interpreter.remote=DEBUG
log4j.logger.org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer=DEBUG



Jan Botorek >于2016年11月29日周二 
下午4:57写道：
If I 

Re: Zepelin problem in HA HDFS

[Felix Cheung]

Quite possibly since Spark is talking to HDFS.

Does it work in your environment when HA switch over with a long running spark 
shell session?



From: Ruslan Dautkhanov 
Sent: Sunday, November 20, 2016 5:27:54 PM
To: users@zeppelin.apache.org
Subject: Re: Zepelin problem in HA HDFS

When I failed over HDFS HA nameservice to another namenode, Zeppelin now has 
the same
error stack *but* for the other namenode, which now became standby.

Not sure if has something to do with Spark 2.0..



--
Ruslan Dautkhanov

On Sun, Nov 20, 2016 at 4:59 PM, Ruslan Dautkhanov 
> wrote:
Running into issues with Zeppelin in a cluster that runs HA HDFS.
See complete exception stack [1].
"pc1udatahad01.x.y/10.20.32.54:8020...
category READ is not supported in state standby"
Yes, pc1udatahad01 is a current standby, why Spark/HMS/doesn't switch over to 
the active one?
hdfs-site.xml that exists in zeppelin home/conf has a symlink
hdfs-site.xml -> /etc/hive/conf/hdfs-site.xml
and hdfs config properly points to a HA HDFS namespace.

Thoughts?

Interesting side effect is that HMS switches to a local Derby database (I sent 
email on this last week in a separate email chain). See [1] stack - it seems 
Hive/HMS tries to talk to HDFS and fails over to a local Derby database.



Zeppelin 0.6.2
Spark 2.0.2
Hive 1.1
RHEL 6.6
Java 7



[1]

 INFO [2016-11-20 16:47:21,044] ({Thread-40} 
RetryInvocationHandler.java[invoke]:148) - Exception while invoking getFileInfo 
of class ClientNamenodeProtocolTranslatorPB over 
pc1udatahad01.x.y/10.20.32.54:8020. Trying to fail 
over immediately.
org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.ipc.StandbyException): 
Operation category READ is not supported in state standby. Visit 
https://s.apache.org/sbnn-error
at 
org.apache.hadoop.hdfs.server.namenode.ha.StandbyState.checkOperation(StandbyState.java:88)
at 
org.apache.hadoop.hdfs.server.namenode.NameNode$NameNodeHAContext.checkOperation(NameNode.java:1831)
at 
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkOperation(FSNamesystem.java:1449)
at 
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getFileInfo(FSNamesystem.java:4271)
at 
org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getFileInfo(NameNodeRpcServer.java:897)
at 
org.apache.hadoop.hdfs.server.namenode.AuthorizationProviderProxyClientProtocol.getFileInfo(AuthorizationProviderProxyClientProtocol.java:528)
at 
org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getFileInfo(ClientNamenodeProtocolServerSideTranslatorPB.java:829)
at 
org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
at 
org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:617)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1073)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2086)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2082)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1709)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2080)

at org.apache.hadoop.ipc.Client.call(Client.java:1472)
at org.apache.hadoop.ipc.Client.call(Client.java:1409)
at 
org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:230)
at com.sun.proxy.$Proxy16.getFileInfo(Unknown Source)
at 
org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getFileInfo(ClientNamenodeProtocolTranslatorPB.java:762)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at 
org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:256)
at 
org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:104)
at com.sun.proxy.$Proxy17.getFileInfo(Unknown Source)
at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:2121)
at 
org.apache.hadoop.hdfs.DistributedFileSystem$19.doCall(DistributedFileSystem.java:1215)
at 
org.apache.hadoop.hdfs.DistributedFileSystem$19.doCall(DistributedFileSystem.java:1211)
at 
org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
at 

Re: "You must build Spark with Hive. Export 'SPARK_HIVE=true'"

[Felix Cheung]

Hmm, SPARK_HOME is set it should pick up the right Spark.

Does this work with the Scala Spark interpreter instead of pyspark? If it 
doesn't, is there more info in the log?



From: Ruslan Dautkhanov 
Sent: Monday, November 21, 2016 1:52:36 PM
To: users@zeppelin.apache.org
Subject: "You must build Spark with Hive. Export 'SPARK_HIVE=true'"

Getting
You must build Spark with Hive. Export 'SPARK_HIVE=true'
See full stack [2] below.

I'm using Spark 1.6 that comes with CDH 5.8.3.
So it's definitely compiled with Hive.
We use Jupyter notebooks without problems in the same environment.

Using Zeppelin 0.6.2, downloaded as zeppelin-0.6.2-bin-all.tgz from from 
apache.org

Is Zeppelin compiled with Hive too? I guess so.
Not sure what else is missing.

Tried to play with ZEPPELIN_SPARK_USEHIVECONTEXT but it does not make 
difference.


[1]
$ cat zeppelin-env.sh
export JAVA_HOME=/usr/java/java7
export SPARK_HOME=/opt/cloudera/parcels/CDH/lib/spark
export SPARK_SUBMIT_OPTIONS="--principal  --keytab yyy --conf 
spark.driver.memory=7g --conf spark.executor.cores=2 --conf 
spark.executor.memory=8g"
export SPARK_APP_NAME="Zeppelin notebook"
export HADOOP_CONF_DIR=/etc/hadoop/conf
export HIVE_CONF_DIR=/etc/hive/conf
export HIVE_HOME=/opt/cloudera/parcels/CDH/lib/hive
export PYSPARK_PYTHON="/opt/cloudera/parcels/Anaconda/bin/python2"
export 
PYTHONPATH="/opt/cloudera/parcels/CDH/lib/spark/python:/opt/cloudera/parcels/CDH/lib/spark/python/lib/py4j-0.9-src.zip"
export MASTER="yarn-client"
export ZEPPELIN_SPARK_USEHIVECONTEXT=true




[2]

You must build Spark with Hive. Export 'SPARK_HIVE=true' and run build/sbt 
assembly
Traceback (most recent call last):
File "/tmp/zeppelin_pyspark-9143637669637506477.py", line 267, in 
raise Exception(traceback.format_exc())
Exception: Traceback (most recent call last):
File "/tmp/zeppelin_pyspark-9143637669637506477.py", line 265, in 
exec(code)
File "", line 9, in 
File "/opt/cloudera/parcels/CDH/lib/spark/python/pyspark/sql/context.py", line 
580, in sql

[3]
Also have correct symlinks in zeppelin_home/conf for
- hive-site.xml
- hdfs-site.xml
- core-site.xml
- yarn-site.xml



Thank you,
Ruslan Dautkhanov

Re: Is it possible to run Java code on Zeppelin Notebook?

[Felix Cheung]

I think you will need to convert Java code into Scala syntax? But Scala can 
call into Java libraries and so on.

I don't think we have an interpreter for Java since it does not come with a 
REPL until Java 9?



From: Abhisar Mohapatra 
Sent: Thursday, November 17, 2016 9:23:24 AM
To: users@zeppelin.apache.org; Muhammad Rezaul Karim
Subject: Re: Is it possible to run Java code on Zeppelin Notebook?

Yes it will. I guess there are some implementations too

On Thu, Nov 17, 2016 at 10:41 PM, Muhammad Rezaul Karim 
> wrote:
Hi All,

I am a new user of Zeppelin and got to know that Apache Zeppelin is using Spark 
as the backend interpreter.

Till date, I have run some codes written in Scala on the Zeppelin notebook. 
However, I am pretty familiar with writing Spark application using Java.
Now my question: is it possible to run Java code on Zeppelin Notebook?



Thanks and Regards,
-
Md. Rezaul Karim
PhD Researcher, Insight Centre for Data Analytics
National University of Ireland Galway
E-mail: rezaul.ka...@insight-centre.org
Web: www.insight-centre.org
Phone: +353892311519


_
The information contained in this communication is intended solely for the use 
of the individual or entity to whom it is addressed and others authorized to 
receive it. It may contain confidential or legally privileged information. If 
you are not the intended recipient you are hereby notified that any disclosure, 
copying, distribution or taking any action in reliance on the contents of this 
information is strictly prohibited and may be unlawful. If you have received 
this communication in error, please notify us immediately by responding to this 
email and then delete it from your system. The firm is neither liable for the 
proper and complete transmission of the information contained in this 
communication nor for any delay in its receipt.

Re: Binary for CDH 5.8.0

[Felix Cheung]

We don't have Hadoop distribution specific binary releases - compiling from 
source with switches would be the best route.





On Mon, Sep 19, 2016 at 12:22 PM -0700, "Abhi Basu" 
<9000r...@gmail.com> wrote:

Is there a specific binary for CDH 5.8.0, hadoop 2.6. and Spark 1.6?

Or, is the best method to compile the source code with appropriate switches?

Thanks,

Abhi

--
Abhi Basu

Re: Hbase configuration storage without data

[Felix Cheung]

I like that approach - though you should be able to clear result output before 
exporting the note, if all you want is the config? The should remove all output 
data, keeping it smaller?


_
From: Mohit Jaggi >
Sent: Monday, September 12, 2016 10:38 AM
Subject: Re: Hbase configuration storage without data
To: >


one option is to keep the data in separate files. notes.json can contain the 
code and the data can be a pointer to /path/to/file. import/export can choose 
to include or exclude the data. when it is included the data files are added to 
a tgz file containing notes.json otherwise you just export notes.json



On Mon, Sep 12, 2016 at 10:33 AM, moon soo Lee 
> wrote:
Right big note.json file is a problem.
But at the same time, i think having data in the note.json helps make 
import/export simple and make notebook renderable without run it.

So far, i didn't see much discussion about this subject on mailing list or on 
the issue tracker.

If there's an good idea that can handle large data while keeping import/export 
simple and ability to render without run, that would be a great starting point 
of the discussions.

Thanks,
moon

On Wed, Sep 7, 2016 at 9:40 PM Vikash Kumar 
> wrote:
Hi moon,
Yes that was the way that I was using. But is there any plan for future 
releases to removing the data from note and storing only configuration?
Because storing the configuration with data when there is no max result limit 
will create a big note.json file.

Thanks & Regards,
Vikash Kumar
From: moon soo Lee [mailto:m...@apache.org]
Sent: Wednesday, September 7, 2016 8:39 PM
To: users@zeppelin.apache.org
Subject: Re: Hbase configuration storage without data

Hi,

For now, code and result data are mixed in note.json, which is represented by 
'class Note' [1]. And every Notebook storage layer need to implement 
'NotebookRepo.get()' [2] to read note.json from underlying storage and convert 
it into 'class Note'.

As you see the related API and class definition, NotebookRepo actually doesn't 
have any restriction how 'class Note' is serialized and saved in the storage.

So you can event new format, you can exclude result data from saving, and so on.

Hop this helps.

Thanks,
moon

[1] 
https://github.com/apache/zeppelin/blob/master/zeppelin-zengine/src/main/java/org/apache/zeppelin/notebook/Note.java
[2] 
https://github.com/apache/zeppelin/blob/master/zeppelin-zengine/src/main/java/org/apache/zeppelin/notebook/repo/NotebookRepo.java#L47

On Wed, Sep 7, 2016 at 3:47 AM Vikash Kumar 
> wrote:
Hi all,
We are storing the note.json configuration into hbase as it is 
stored into File system. As default behavior in note.json the query data is 
stored along with configuration. But we want to store the configurations only 
and when user loading its note then query should get executed and data 
generated. This feature we are using for phoenix interpreter. So how can we 
remove the data from note.json? Is there any plan for that?


Thanks & Regards,
Vikash Kumar

Re: Matplotlib uses tkinter instead of Agg

[Felix Cheung]

And
matplotlib.use('Agg')

Would only work before matplotlib is first used so you would need to restart 
the interpreter. From error stack below it looks like something might be 
setting the default backend in matplotlib to TkAgg though.

Are you using the Python interpreter or PySpark interpreter? Also how you are 
calling matplotlib like Moon asks?

_
From: moon soo Lee >
Sent: Tuesday, September 13, 2016 2:34 PM
Subject: Re: Matplotlib uses tkinter instead of Agg
To: >


Hi,

Thanks for sharing the problem.
Could you share which version of Zeppelin are you using and how did you try 
matplotlib inside of Zeppelin? Are you trying matplotlib with z.show() ?

Thanks,
moon

On Tue, Sep 13, 2016 at 1:56 AM Xi Shen 
> wrote:
Hi,

I want to build a Zeppelin docker image for my self. The docker image is based 
on ubuntu:wily, and has openjdk-8-jre and python3 installed. I also installed 
other packages that I need.

After started Zeppelin in the docker, I am able to access the webapp from my 
local browser. I tried to execute some simple Python script, and it works fine. 
But when I try to run the matplotlib example, I got error saying that tkinter 
cannot find the $DISPLAY.

Traceback (most recent call last):
File "", line 1, in 
File "/usr/local/lib/python3.4/dist-packages/matplotlib/pyplot.py", line 535, 
in figure
**kwargs)
File 
"/usr/local/lib/python3.4/dist-packages/matplotlib/backends/backend_tkagg.py", 
line 84, in new_figure_manager
return new_figure_manager_given_figure(num, figure)
File 
"/usr/local/lib/python3.4/dist-packages/matplotlib/backends/backend_tkagg.py", 
line 92, in new_figure_manager_given_figure
window = Tk.Tk()
File "/usr/lib/python3.4/tkinter/__init__.py", line 1859, in __init__
self.tk = _tkinter.create(screenName, baseName, className, 
interactive, wantobjects, useTk, sync, use)
_tkinter.TclError: no display name and no $DISPLAY environment variable

Some people on the Internet suggested adding matplotlib.use('Agg') at the 
beginning of the notebook, but it still does not work for me.

--

Thanks,
David S.

Re: spark.jars option for Zeppelin over Livy

[Felix Cheung]

Probably livy.spark.jars then as Spark Packages is a different list?

http://spark.apache.org/docs/latest/submitting-applications.html#advanced-dependency-management


_
From: Vinay Shukla <vinayshu...@gmail.com<mailto:vinayshu...@gmail.com>>
Sent: Tuesday, August 2, 2016 1:51 PM
Subject: Re: spark.jars option for Zeppelin over Livy
To: <users@zeppelin.apache.org<mailto:users@zeppelin.apache.org>>


In recent Zeppelin builds with Livy interpreter you can set spark-packages (e.g 
Databricks csv)

as value of property "livy.spark.jars.packages" on Livy interpreter properties.

Please see the attached screenshot

On Tue, Aug 2, 2016 at 1:43 PM, Felix Cheung 
<felixcheun...@hotmail.com<mailto:felixcheun...@hotmail.com>> wrote:
Have you tried setting it in the Interpreter menu under Livy?





On Tue, Aug 2, 2016 at 11:04 AM -0700, "Chen Song" 
<chen.song...@gmail.com<mailto:chen.song...@gmail.com>> wrote:

When using Zeppelin over Livy, how would I set the spark.jars option?

I try to set it in my spark-defaults.conf and Livy doesn't respect it at all. 
Other properties in spark-default.conf seem to be picked up propertly.

Chen

Re: Passing command line params when cell is evaluated?

[Felix Cheung]

This should be captured in a log file under /logs of Zeppelin binaries.

This specific warning, in believe, might be coming from the way Zeppelin is 
calling spark-submit?





On Wed, Jul 6, 2016 at 5:24 PM -0700, "Shayne Hodge" 
> wrote:

When I execute a few lines of Spark/Scala code, I get back the following:

warning: there were 1 deprecation warning(s); re-run with -deprecation for 
details

(I think this occurs in some of the sample notebooks, too).  I'm guessing this 
would be what I would do if passing the code on the command line to Spark.  How 
can I see the deprecation warning within Zeppelin?

Thanks,



Shayne Hodge

Data Scientist
sho...@snaplogic.com
SnapLogic.com







SnapLogic Inc | 2 W Fifth Avenue Fourth Floor | San Mateo | 94402 | California