Re: [VOTE] Release Apache Zeppelin 0.7.0 (RC1)

2017-01-21 Thread Jun Kim
+1 2017년 1월 21일 (토) 오후 8:21, CloverHearts 님이 작성: > +1 > > > > The Zeppelin 0.7 release includes some amazing features. > > In particular, a major bug fix for the remote interpreter. > > I am actively in favor of this release. > > > > > > *출발**: *Mina Lee

Re: [VOTE] Release Apache Zeppelin 0.7.0 (RC1)

2017-01-21 Thread CloverHearts
+1 The Zeppelin 0.7 release includes some amazing features. In particular, a major bug fix for the remote interpreter. I am actively in favor of this release. 출발: Mina Lee 회신 대상: 날짜: 2017년 1월 21일 토요일 오전 3:17 받는 사람:

Re: [VOTE] Release Apache Zeppelin 0.7.0 (RC1)

2017-01-21 Thread Hyung Sung Shim
+1 2017년 1월 21일 (토) 오전 4:55, Jeff Zhang 님이 작성: > -1, Can we include ZEPPELIN-1985 > ? This issue cause > only user1 can see the pig tutorial note which is a very bad user > experience. > > > > Prabhjyot Singh

Re: [VOTE] Release Apache Zeppelin 0.7.0 (RC1)

2017-01-21 Thread Ahyoung Ryu
+1 On Sun, Jan 22, 2017 at 2:54 AM, Hyung Sung Shim wrote: > +1 > 2017년 1월 21일 (토) 오전 4:55, Jeff Zhang 님이 작성: > >> -1, Can we include ZEPPELIN-1985 >> ? This issue cause >> only user1 can see the pig

Re: [DISCUSS] Remove PostgresqlInterpreter

2017-01-23 Thread Jongyoul Lee
I created https://issues.apache.org/jira/browse/ZEPPELIN-2003 On Mon, Jan 23, 2017 at 10:56 PM, Jongyoul Lee wrote: > Thanks for replying it. I'll make a PR for doing it. > > On Mon, Jan 23, 2017 at 4:09 PM, Prabhjyot Singh < > prabhjyotsi...@apache.org> wrote: > >> +1. >>

Re: [DISCUSS] Remove PostgresqlInterpreter

2017-01-23 Thread Jongyoul Lee
Thanks for replying it. I'll make a PR for doing it. On Mon, Jan 23, 2017 at 4:09 PM, Prabhjyot Singh wrote: > +1. > > Yes, agreed I too think its overhead, and all the features can be achieved > via JDBCInterpreter. > > On 23 January 2017 at 10:45, Jongyoul Lee

zeppelin memory usage

2017-01-24 Thread Paul Brenner
We are using zeppelin with multiple users on the same server and often run out of memory on the dedicated VM that zeppelin runs on. It isn’t uncommon to run out of memory on the VM zeppelin is running on when just 3-4 users are using zeppelin. Is this normal behavior? Does each user’s spark

[CANCEL][VOTE] Release Apache Zeppelin 0.7.0 (RC2)

2017-01-23 Thread Mina Lee
Thank you Vinay for reporting. The PR#1647 [1] that disables run on select option change is merged intentionally, but seems like it didn't consider report view. I made a PR#1932 [2] to make it runnable by hitting enter after selection so it can be run in report view too. I am canceling rc2 vote,

Re: [VOTE] Release Apache Zeppelin 0.7.0 (RC3)

2017-01-24 Thread Sora Lee
+1 On Tue, Jan 24, 2017 at 3:09 PM Jongyoul Lee wrote: > +1 (binding) > > On Tue, Jan 24, 2017 at 2:43 PM, Mina Lee wrote: > > I propose the following RC to be released for the Apache Zeppelin 0.7.0 > release. > > The commit id is

Re: zeppelin memory usage

2017-01-24 Thread Paul Brenner
It looks like one of our big problems is that zeppelin doesn’t always kill all completed processes. Is there an accepted way to kill a spark instance? Most of the time executing sys.exit in a paragraph will kill the spark instance in yarn and I believe also kill the corresponding zeppelin

[VOTE] Release Apache Zeppelin 0.7.0 (RC3)

2017-01-23 Thread Mina Lee
I propose the following RC to be released for the Apache Zeppelin 0.7.0 release. The commit id is 48ad70e8c62975bdb00779bed5919eaca98c5b5d which is corresponds to the tag v0.7.0-rc3: *https://git-wip-us.apache.org/repos/asf?p=zeppelin.git;a=commit;h=48ad70e8c62975bdb00779bed5919eaca98c5b5d

Re: [VOTE] Release Apache Zeppelin 0.7.0 (RC3)

2017-01-23 Thread Jongyoul Lee
+1 (binding) On Tue, Jan 24, 2017 at 2:43 PM, Mina Lee wrote: > I propose the following RC to be released for the Apache Zeppelin 0.7.0 > release. > > The commit id is 48ad70e8c62975bdb00779bed5919eaca98c5b5d which is > corresponds to the tag v0.7.0-rc3: >

Re: [VOTE] Release Apache Zeppelin 0.7.0 (RC3)

2017-01-24 Thread Windy Qin
+1 On 2017-01-24 17:13 (+0800), Sora Lee wrote: > +1 > > On Tue, Jan 24, 2017 at 3:09 PM Jongyoul Lee wrote: > > > +1 (binding) > > > > On Tue, Jan 24, 2017 at 2:43 PM, Mina Lee wrote: > > > > I propose the following RC to be

Re: [VOTE] Release Apache Zeppelin 0.7.0 (RC3)

2017-01-24 Thread Hyung Sung Shim
+1 2017년 1월 24일 (화) 오전 1:36, Windy Qin 님이 작성: > > +1 > On 2017-01-24 17:13 (+0800), Sora Lee wrote: > > +1 > > > > On Tue, Jan 24, 2017 at 3:09 PM Jongyoul Lee wrote: > > > > > +1 (binding) > > > > > > On Tue, Jan 24, 2017 at 2:43

Re: [VOTE] Release Apache Zeppelin 0.7.0 (RC3)

2017-01-24 Thread rohit choudhary
+1 On 24-Jan-2017 6:11 PM, "Jeff Zhang" wrote: +1 Hyung Sung Shim 于2017年1月24日周二 下午7:19写道: > +1 > 2017년 1월 24일 (화) 오전 1:36, Windy Qin 님이 작성: > > > +1 > On 2017-01-24 17:13 (+0800), Sora Lee wrote: > > +1 > > > >

Re: [VOTE] Release Apache Zeppelin 0.7.0 (RC3)

2017-01-24 Thread Ahyoung Ryu
+1 On Tue, Jan 24, 2017 at 8:19 PM, Hyung Sung Shim wrote: > +1 > 2017년 1월 24일 (화) 오전 1:36, Windy Qin 님이 작성: > >> >> +1 >> On 2017-01-24 17:13 (+0800), Sora Lee wrote: >> > +1 >> > >> > On Tue, Jan 24, 2017 at 3:09 PM Jongyoul Lee

Monitoring Zeppelin Health Via Rest API

2017-01-24 Thread Rob Anderson
Hello, We're running Zeppelin 0.6.2 and authenticating against Active Directory via Shiro. Everything is working pretty well, however, we do occasionally have issues, which is leading to a bad user experience, as operationally we're unaware of a problem. We'd like to monitor the health of

Notebooks & Notes

2017-01-24 Thread darren
Hi Just a quick thought. Instead of calling them 'paragraphs' in a notebook. Why not call them 'notes'? They look like post-it notes already. Just a thought. Get Outlook for Android

Re: Notebooks & Notes

2017-01-24 Thread Mohit Jaggi
haha...the collection of paras is called a note :) On Tue, Jan 24, 2017 at 4:54 PM, wrote: > Hi > > Just a quick thought. Instead of calling them 'paragraphs' in a notebook. > Why not call them 'notes'? > > They look like post-it notes already. > > Just a thought. > > Get

Re: Notebooks & Notes

2017-01-24 Thread Jun Kim
And the collection of notes is a notebook :) But some codes inside Zeppelin may still use the term "notebook" as note. Plz modify it if u find it! 2017년 1월 25일 (수) 오전 9:58, Mohit Jaggi 님이 작성: > haha...the collection of paras is called a note :) > > On Tue, Jan 24, 2017 at

Re: Notebooks & Notes

2017-01-24 Thread Mohit Jaggi
if a para becomes a note, note will become notebook and notebook will become a backpack or briefcase :) that can work too! On Tue, Jan 24, 2017 at 5:06 PM, Jun Kim wrote: > And the collection of notes is a notebook :) > > But some codes inside Zeppelin may still use the term

Re: [VOTE] Release Apache Zeppelin 0.7.0 (RC3)

2017-01-24 Thread Jeff Zhang
And seems ZEPPELIN-1982 also need to be included in 0.7 Jeff Zhang 于2017年1月25日周三 下午1:38写道: > Please help also merge ZEPPELIN-2006 ( > https://github.com/apache/zeppelin/pull/1938) If there's another RC > > > moon soo Lee 于2017年1月25日周三 上午11:22写道: > > Currently,

Re: Monitoring Zeppelin Health Via Rest API

2017-01-24 Thread Jongyoul Lee
AFAIK, Zeppelin doesn't have it for now. We have to develop that function. Regards, Jongyoul On Wed, Jan 25, 2017 at 3:18 AM, Rob Anderson wrote: > Hello, > > We're running Zeppelin 0.6.2 and authenticating against Active Directory > via Shiro. Everything is

Re: [VOTE] Release Apache Zeppelin 0.7.0 (RC3)

2017-01-24 Thread Jeff Zhang
Please help also merge ZEPPELIN-2006 ( https://github.com/apache/zeppelin/pull/1938) If there's another RC moon soo Lee 于2017年1月25日周三 上午11:22写道: > Currently, cron scheduling feature is broken [1] and patch is available at > [2]. > Shell we include this patch in 0.7.0 release? >

Re: [CANCEL][VOTE] Release Apache Zeppelin 0.7.0 (RC3)

2017-01-26 Thread Andreas Weise
As there seems to be RC4, any chance to also get PR 1804 [1] merged for 0.7.0. It fixes an important issue [2] when customizing the zeppelin start page. Without the bugfix, it is not possible to use a custom homepage which includes the notebook list. Thanks. Andreas [1]

Re: Another instance of Derby already booted the database .../metastore_db

2017-01-27 Thread Felix Cheung
The issue is since you are running Spark interpreter in isolated mode there are two running Spark driver process trying to access derby metastore at the same location, at the same time. There are some config in Spark that could redirect metastore to different location (try spark.warehouse.dir)

[no subject]

2017-01-30 Thread Michał Kabocik
Thank you for the reply. I know that impersonation is done by livy, but when I have user impersonation disabled on livy.spark interpreter, then livy impersonates zeppelin user, not my user. On ranger audit and in yarn history manager I see zeppelin via livy-session. What I want to achieve is

Re: "output disappears" after running a paragraph - on recent master snapshot

2017-01-30 Thread Ruslan Dautkhanov
Thanks for the follow up Moon. I just happened to witness this blinking paragraph's output on his PC. It's interesting that it only happens in Chrome for him. (I use the same version of Chrome as him, and can't reproduce it) We also checked that it happens for different interpreters - %sh and

SSL with Elasticsearch / Shield

2017-01-28 Thread Jonathan Greenleaf
I have been able to start the server on 8443, but have not been able to make a connection to my Elasticsearch data node over SSL. I verified I can curl (curl --insecure -v -u user:pwd https:xx.xx.xx.xx:9200/...) the ES box with the Shield credentials. Within the interpreter I have name ->

Re: Monitoring Zeppelin Health Via Rest API

2017-01-25 Thread Jongyoul Lee
I think it's good idea and we need add more beans to be monitored, too. On Thu, Jan 26, 2017 at 9:45 AM, Vinay Shukla wrote: > How about launching Zeppelin JVM with JConsole related config and trying > to monitor JVM level stats? > > On Wed, Jan 25, 2017 at 9:12 AM, Rob

notebook interpreter restart

2017-01-25 Thread Paul Brenner
Alec, The way we use zeppelin at our company is to set our interpreters to “isolated”. That way each notebook gets it’s own application on yarn.  This mostly works well. The one downside is that if you stop a notebook (e.g. by calling sys.exit in a cell of a spark notebook) it does stop the

Re: notebook interpreter restart

2017-01-25 Thread Paul Brenner
Did you try setting your interpreter to “Isolated” mode? Is it currently in “shared” mode?  If you haven’t played with this setting before then:  1. Open the interpreters page 2. Find your interpreter and click the edit button in the top right corner 3. Beneath the word “option” at the top

[CANCEL][VOTE] Release Apache Zeppelin 0.7.0 (RC3)

2017-01-25 Thread Mina Lee
There are several critical issues found during the rc3 verification. I am canceling vote to include below bug fixes: - Cron scheduling doesn't work after certain number of running [1] - Table is not rendered in certain condition [2] - Livy not working in anonymous mode [3] -

Re: notebook interpreter restart

2017-01-25 Thread Alec Lee
Hello, Paul Thank you so much for prompt reply. Our understanding is each interpreter takes one application on YARN, if multiple notebooks share same interpreter will run through same YARN application. As you mentioned this “… which will kill any other YARN application associated with that

notebook interpreter restart

2017-01-25 Thread Alec Lee
Hi, all Currently we are exploring feature of zeppelin, now the situation we are using YARN to manage spark jobs. In terms of the experiments, we conclude that one interpreter is corresponding to An application in YARN cluster, that means all the notebooks from zeppelin with same interpreter

Re: notebook interpreter restart

2017-01-25 Thread Alec Lee
Thanks a lot, we miss that setting, will do the experiments right away, :) best AL > On Jan 25, 2017, at 6:17 PM, Paul Brenner wrote: > > > Did you try setting your interpreter to “Isolated” mode? Is it currently in > “shared” mode? > > If you haven’t played with

Re: [Discuss] Move some interpreters out of zeppelin project

2017-01-20 Thread Jongyoul Lee
Hi Jeff, Thanks for starting this issue. It increases flexibility of improving interpreters itself but it can also decreases stability of interpreters. I'm worried about this side-effect. As you mentioned, it's hard for me to review new interpreter that I didn't use but it couldn't be a reason

Re: [VOTE] Release Apache Zeppelin 0.7.0 (RC1)

2017-01-21 Thread Alexander Bezzubov
Hi Jeff, thank you for the notice, it is a good catch! But the issue that you mention does not have FixFor version assigned - if you want it to part of the release and even conside it as important enough to block the relese vote, could you please update that? This would simplify drastically the

Re: [VOTE] Release Apache Zeppelin 0.7.0 (RC1)

2017-01-21 Thread Jeff Zhang
Hi Alex & Mina, Sorry for didn't fill the fix version (maybe my misunderstanding, usually I use fix version for when it is fixed and target version as when it will be fixed). It is fine for me to without this ticket in 0.7, I would cancel my -1. Alexander Bezzubov

Re: [VOTE] Release Apache Zeppelin 0.7.0 (RC2)

2017-01-22 Thread DuyHai Doan
+1 On Sun, Jan 22, 2017 at 8:59 AM, Mina Lee wrote: > I propose the following RC to be released for the Apache Zeppelin 0.7.0 > release. > > The commit id is d0ddf8fdff75b4229543f7ebfd5eb001f685382d which is > corresponds to the tag v0.7.0-rc2: >

Re: [VOTE] Release Apache Zeppelin 0.7.0 (RC1)

2017-01-21 Thread Ray
+1 On Sat, Jan 21, 2017 at 7:17 PM, Mina Lee wrote: > Hi folks, > > I propose the following RC to be released for the Apache Zeppelin 0.7.0 > release. > > The commit id is bcbe0855175f55eddd10b918303806eb5e5db091 which is > corresponds to the tag v0.7.0-rc1: >

Re: [VOTE] Release Apache Zeppelin 0.7.0 (RC1)

2017-01-21 Thread Felix Cheung
Thanks. I'd incline to agree having latest matplotlib broken with python (python interpreter) seems like a big inconvenience. Is there a workaround? Let's discuss a bit in the JIRA and bring this back to the vote thread. Also we should really try to mark JIRA too to be included in RC builds,

Re: [VOTE] Release Apache Zeppelin 0.7.0 (RC2)

2017-01-22 Thread Jeff Zhang
+1 DuyHai Doan 于2017年1月22日周日 下午4:01写道: > +1 > > On Sun, Jan 22, 2017 at 8:59 AM, Mina Lee wrote: > > I propose the following RC to be released for the Apache Zeppelin 0.7.0 > release. > > The commit id is d0ddf8fdff75b4229543f7ebfd5eb001f685382d which

Re: InvalidClassException using Zeppelin (master) and spark-2.1 on a standalone spark cluster

2017-01-20 Thread Jonathan Kelly
Hi, Antoine, this issue was being tracked in https://issues.apache.org/jira/browse/ZEPPELIN-1977, but it is now resolved as of yesterday (looks like about 18 hours ago). Maybe you need to pull from master again and rebuild? ~ Jonathan On Fri, Jan 20, 2017 at 1:19 PM Antoine

InvalidClassException using Zeppelin (master) and spark-2.1 on a standalone spark cluster

2017-01-20 Thread Antoine
Hei, I'm trying to use Zeppelin from the master git branch with spark-2.1 and I get an invalid class exeption when I use a standalone spark cluster. java.io.InvalidClassException: org.apache.commons.lang3.time.FastDateParser; local class incompatible: stream classdesc serialVersionUID = 2, local

Re: [VOTE] Release Apache Zeppelin 0.7.0 (RC2)

2017-01-22 Thread Hyung Sung Shim
+1 2017년 1월 22일 (일) 오전 12:54, Ahyoung Ryu 님이 작성: > +1 > > On Sun, Jan 22, 2017 at 5:52 PM, Jeff Zhang wrote: > > > +1 > > > > > > > > DuyHai Doan 于2017年1月22日周日 下午4:01写道: > > > >> +1 > >> > >> On Sun, Jan 22, 2017 at 8:59 AM, Mina Lee

Re: [DISCUSS] Release package size

2017-01-20 Thread Mina Lee
Decision making taking more time than I expected and I think this shouldn't be blocker for 0.7.0. We can take more time deciding which interpreters can be included or excluded. Until then, I am just going to go with our current one: zeppelin-bin-all, zeppelin-bin-netinst. Moon's suggestion looks

Save button on interpreter is not working

2017-01-26 Thread Nabajyoti Dash
Hi, I am using zeppelin 6.2 . When I am accessing my local zeppelin everything is working fine. But when I am trying to set it up in staging it's working in a weird manner. Like sometimes the save button of the interpreter is not working.The interpreters are getting refreshed each time.Deleting a

How group based authentication works with shiro?

2017-01-27 Thread Michał Kabocik
Dears, I'm trying to configure shiro to authenticate users from AD and to limit access to login to webui for specific group. Here's my shiro.ini config: activeDirectoryRealm = org.apache.zeppelin.server.ActiveDirectoryGroupRealm activeDirectoryRealm.systemUsername = user

Another instance of Derby already booted the database .../metastore_db

2017-01-27 Thread Jan Rasehorn
Hello, I was running into an issue with a query on a temp table and to test it I opened another notebook in another tab. When I execute this new notebook, I got an error: Caused by: java.sql.SQLException: Failed to start database 'metastore_db' with class loader

Connection refused when trying to run livy.spark interpterer in kerberized HDP

2017-01-27 Thread Michał Kabocik
Dears, I'm running Zeppelin 0.6.2 on HDP 2.5.3. My Zeppelin has also configured shiro to authenticate users from AD. Now I'm trying to configure livy.spark interpreter with user impersonation. I used livy 0.2 and 0.3 but in both cases I cannot start livy.spark interpterer. Here's the config of

Another instance of Derby already booted the database .../metastore_db

2017-01-27 Thread Paul Brenner
We had this same problem. Copying or symlinking hive-site.xml to the zeppelin conf directory and then restarting solved it for us.  http://www.placeiq.com/ http://www.placeiq.com/ http://www.placeiq.com/ Paul Brenner https://twitter.com/placeiq https://twitter.com/placeiq

Issue while building zeppelin ERROR: spawn npm ENOENT

2017-01-27 Thread Rajnish Garg
I am using the following command to build zeppelin 0.8 on ubuntu mvn clean package -Pbuild-distr -Pyarn -Pspark-1.6 -Dspark.version=1.6.0 -Phadoop-2.6 -Dhadoop.version=2.6.0-cdh5.9.0 -DskipTests -Pvendor-repo I am getting following error: [WARNING] warning Unmet peer dependency

Re: [VOTE] Release Apache Zeppelin 0.7.0 (RC3)

2017-01-24 Thread moon soo Lee
Currently, cron scheduling feature is broken [1] and patch is available at [2]. Shell we include this patch in 0.7.0 release? [1] http://issues.apache.org/jira/browse/ZEPPELIN-2009 [2] https://github.com/apache/zeppelin/pull/1941 On Tue, Jan 24, 2017 at 6:45 PM Prabhjyot Singh

Re: [VOTE] Release Apache Zeppelin 0.7.0 (RC3)

2017-01-24 Thread Prabhjyot Singh
+1 On Jan 24, 2017 6:25 PM, "Ahyoung Ryu" wrote: > +1 > > On Tue, Jan 24, 2017 at 8:19 PM, Hyung Sung Shim > wrote: > >> +1 >> 2017년 1월 24일 (화) 오전 1:36, Windy Qin 님이 작성: >> >>> >>> +1 >>> On 2017-01-24 17:13 (+0800), Sora Lee

Re: Connection refused when trying to run livy.spark interpterer in kerberized HDP

2017-01-28 Thread Jianfeng (Jeff) Zhang
>>> But when I enable user impersonation on ivy.spark interpterer Do you mean you enable the option of user impersonation on livy interpreter setting of zeppelin ? You don't need to do that as impersonation is done by livy, the option there is a little confusing, it should not be displayed.

Binaries for a setup behind corp firewall and proxy

2017-02-21 Thread Mathieu D
Hello Zeppelin folks, I'm currently deploying Zeppelin for a customer with corporate firewalls and proxies everywhere. Zeppelin fails at startup because of npm download. It seems that proxy parameter at system level (http_proxy env variable) is not used when downloading npm. Essentially, I would

Re: Zeppelin Books

2017-02-15 Thread Jongyoul Lee
There's no book for Apache Zeppelin yet. On Tue, Feb 14, 2017 at 11:44 PM, Muhammad Rezaul Karim < reza_cse...@yahoo.com> wrote: > > Hi All, > > Could anyone suggest me some recent books on Apache Zeppelin? > > > Kind regards, > Reza > > > -- 이종열, Jongyoul Lee, 李宗烈 http://madeng.net

Re: Binaries for a setup behind corp firewall and proxy

2017-02-21 Thread moon soo Lee
Hi Mathieu, Thanks for reporting the problem. I see no issue tracking this problem in our JIRA [1]. Do you mind add one? We're preparing 0.7.1 release and hope we can address this problem in upcoming release. Thanks, moon [1] https://issues.apache.org/jira/browse/ZEPPELIN On Tue, Feb 21,

[DISCUSS] Behaviour of Interpreter when it is restarted

2017-02-22 Thread Prabhjyot Singh
This is WRT the PR that I've created https://github.com/apache/zeppelin/pull/2034. The issue that I want to discuss over here is how should an Interpreter behave when it is; - restarted from notebook - restarted from Interpreter setting page - edited from Interpreter setting page - deleted

zeppelin ActiveDirectory Authentication won't work as expected

2017-02-22 Thread Yaar Reuveni
Hey, No response on previous times I've asked this, trying again. I configured Shiro authentication using Active directory, I have checked this both on version 0.6 and 0.7 and it doesn't work in both. I have a specific group in our active directory, and I wish to grant access to users *only form

Re: [DISCUSS] Behaviour of Interpreter when it is restarted

2017-02-22 Thread Jeff Zhang
I think we can combine scenario 2 and 3 if user click yes button on the popup window of whether you want to restart interpreter in scenario 3. Regarding the restarting scenario of 1,2,3, IMHO I think we don't need to differentiate them. Otherwise it might confuse users that restarting in

Re: zeppelin ActiveDirectory Authentication won't work as expected

2017-02-22 Thread Michał Kabocik
Hello Yaar, I have identical case as you have and also was asking for help on this matter. Unfortunately, based on my tests and assumptions, there is no such functionality yet. All, please correct me if I'm wrong. King regards, Michał 22.02.2017 10:19 "Yaar Reuveni"

Re: Zeppelin unable to respond after some time

2017-02-18 Thread Shane Lee
My environment is on windows and I see this well using sparkr. It appears the R Session somehow is killed after period of inactivity and it can't be found using task manager. I would need to restart the interpreter in order to get another R Session. > On Feb 17, 2017, at 9:11 PM, RUSHIKESH

Re: Re: Zeppelin unable to respond after some time

2017-02-17 Thread xyun...@simuwell.com
Within the Scala REPL everything is working fine. Even you application session is down but you run the same code again you will see there is a new job getting created. But zeppelin has such a problem. you can do a test. run a notebook and get the job finished at the end of the notebook. Then

Re: Re: Zeppelin unable to respond after some time

2017-02-17 Thread Paul Brenner
I don’t believe that this explains my issue. Running the Scala REPL also keeps a session alive for as long as the REPL is running. I’ve had REPLs open for days (sh don’t tell anyone) that have correspondingly kept sessions alive for the same period of time with no problem. I only see this

Re: Re: Zeppelin unable to respond after some time

2017-02-17 Thread xyun...@simuwell.com
The problem could be not only the resource, but the session. If you run a chunk of spark code and you should see the a running application in the spark UI, but in your code if you shut it down after the job is finished, then on the spark UI you will see the hob is finished. Within zeppelin,

Re: Re: Zeppelin unable to respond after some time

2017-02-17 Thread Paul Brenner
I’ve definitely had this problem with jobs that don’t take all the resources on the cluster. Also, my experience matches what others have reported: just restarting zeppelin and re-runing the stuck paragraph solves the issue.  I’ve also experienced this problem with for loops. Some for loops

Re: Re: Zeppelin unable to respond after some time

2017-02-17 Thread xyun...@simuwell.com
I have solved the similar issue before. You should check on spark UI and probably you will see your single job is taking all the resources. Therefore further job that submitting to the same cluster will just hang on there. When you restart zeppelin then the old job is killed and all the

speed up zeppelin-web build

2017-02-17 Thread Иван Шаповалов
Hi all, Got build time up to 30 mins. Most of this time is front-end build. Are there ways to speed it up ? Any suggestions appreciated Thanks in advance -- Ivan

Re: NoSuchMethodException in SQL

2017-02-16 Thread Jeff Zhang
zeppelin will create SparkContext implicitly for users, so it might be too late to set it after interpreter is opened. You can try to set that in interpreter setting. Muhammad Rezaul Karim 于2017年2月16日周四 下午11:52写道: > Hi Lee, > > Thanks for the info that really helped. I

Re: NoSuchMethodException in SQL

2017-02-16 Thread Md. Rezaul Karim
How could I do that? Could you please send me any reference link? On Feb 17, 2017 1:17 AM, Jeff Zhang wrote:zeppelin will create SparkContext implicitly for users, so it might be too late to set it after interpreter is opened. You can try to set that in interpreter

Re: Re: Zeppelin unable to respond after some time

2017-02-17 Thread moon soo Lee
Hi, Download 0.7.0 -> Run R tutorial notebook repeatedly will reproduce the problem? Otherwise, can someone clarify instruction to reproduce the problem? Thanks, moon On Sat, Feb 18, 2017 at 5:45 AM xyun...@simuwell.com wrote: > Within the Scala REPL everything is

Re: Re: Zeppelin unable to respond after some time

2017-02-17 Thread RUSHIKESH RAUT
Yes, it will reproduce the problem. Try running all the tutorials notebook at random, you will see it is in hanging state. In my experience it occurs more often with R and sparkR code, even if there is enough memory in the cluster. Just leave the spark session running and after an hour or so it

Re: NoSuchMethodException in SQL

2017-02-16 Thread Jongyoul Lee
Which version of Spark do you use? I've found the current version supports "lz4" by default, then it looks like you don't have to set anything. If you want to another type of compression, you can set the configuration settings into interpreter tab instead of using "conf.set'. Actually, that

Re: [0.7.0] z.show frequently fails to render tables or charts

2017-02-18 Thread moon soo Lee
Hi, ZEPPELIN-2084 [1] addresses the problem. Patch [2] is available and merged to master branch. Thanks, moon [1] https://issues.apache.org/jira/browse/ZEPPELIN-2084 [2] https://github.com/apache/zeppelin/pull/2005 On Sat, Feb 18, 2017 at 10:51 PM Xiaohui Liu wrote: > Hi,

[0.7.0] z.show frequently fails to render tables or charts

2017-02-18 Thread Xiaohui Liu
Hi, After deploying 0.7.0, z.show often fails to render tables and charts. When that happens, if we refresh the page, the tables and charts will show up. In browser side, we observed this error: TypeError: Cannot read property 'type' of undefined at cells (handsonHelper.js:57) at

Re: [0.7.0] z.show frequently fails to render tables or charts

2017-02-18 Thread moon soo Lee
The patch is available in latest branch-0.7 as well. On Sat, Feb 18, 2017 at 11:51 PM moon soo Lee wrote: > Hi, > > ZEPPELIN-2084 [1] addresses the problem. > Patch [2] is available and merged to master branch. > > Thanks, > moon > > [1]

Re: [0.7.0] z.show frequently fails to render tables or charts

2017-02-18 Thread Xiaohui Liu
Cool! Thanks for the update! On Sat, 18 Feb 2017 at 11:00 PM, moon soo Lee wrote: > The patch is available in latest branch-0.7 as well. > > > On Sat, Feb 18, 2017 at 11:51 PM moon soo Lee wrote: > > Hi, > > ZEPPELIN-2084 [1] addresses the problem. > Patch [2]

Alert when paragraph finishes

2017-02-22 Thread Paul Brenner
I know it is a long shot, but is the ability to be alerted when a paragraph finishes already implemented somewhere? I found this unfinished pull request so I assume it just isn’t a high priority right now, but wanted to double check:  https://github.com/apache/zeppelin/pull/560

PySpark: Dependencies and scripts through Zeppelin

2017-02-23 Thread William Markito Oliveira
What's the right way to package scripts and distribute dependencies (conda environments) with Zeppelin ? I'm currently using the variable zeppelin.pyspark.python from the Spark interpreter set to my conda environment python, but still when I submit the jobs and they start executing I do get "No

Re: Real-time Updated Plots with Kafka

2017-02-23 Thread Mich Talebzadeh
you can use zeppelin scheduler to execute it every few seconds assuming that zeppelin has sufficient to execute every instance of the code Dr Mich Talebzadeh LinkedIn * https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw

How can I use pyspark in zeppelin?

2017-02-23 Thread Beth Lee
I installed spark-2.1.0-bin-hadoop2.7.tgz and zeppelin-0.7.0-bin-all.tgz in ubuntu. I set the zeppelin-env.sh like below. export PYTHONPATH=/usr/bin/python export PYSPARK_PYTHON=/home/jin/spark/python So I try to use pyspark in the zeppelin notebook. %spark.pyspark print(2+2)

Re: How can I use pyspark in zeppelin?

2017-02-24 Thread Hyung Sung Shim
hello. Could you set spark-2.1.0-bin-hadoop2.7 path as SPARK_HOME ? You can refer to http://zeppelin.apache.org/docs/0.7.0/interpreter/spark.html#1-export-spark_home . 2017-02-24 15:44 GMT+09:00 Beth Lee : > I installed spark-2.1.0-bin-hadoop2.7.tgz and

Issue with matplotlib in Zeppelin when rendering time series plots

2017-02-24 Thread Bruno Aranda
Hi, I am trying to use matplotlib to render some plots using the plot_date function in PySpark, where the X axis are datetimes. With a sample code like this one: // %pyspark import matplotlib.pyplot as plt import matplotlib.dates as mdates import pandas as pd import numpy as np sx =

Re: How can I use pyspark in zeppelin?

2017-02-24 Thread Beth Lee
Yes, I already registered it. But the result is same. Thanks, Jin 2017-02-24 18:05 GMT+09:00 Hyung Sung Shim : > hello. > Could you set spark-2.1.0-bin-hadoop2.7 path as SPARK_HOME ? > You can refer to http://zeppelin.apache.org/docs/0.7.0/interpreter/spark. >

Re: Issue with matplotlib in Zeppelin when rendering time series plots

2017-02-24 Thread Bruno Aranda
Hi Alex, Excellent, thanks! Using ipython is a solution that works for us. Cheers, Bruno On Fri, 24 Feb 2017 at 10:54 Goodman, Alexander (398K) < alexander.good...@jpl.nasa.gov> wrote: > Hi Bruno, > > This is a known issue with matplotlib that's probably out of our control. > Essentially,

Re: After upgrading to 0.7.0 behaviour changed for REPL

2017-02-19 Thread Xiaohui Liu
z.getInterpreterContext.out.clear is really neat to control what we want to show in the output box. Is there a similar interface in python interpreter? Regards On Tue, Feb 14, 2017 at 8:20 AM, moon soo Lee wrote: > Hi, > > The behavior is changed since Zeppelin support

Real-time Updated Plots with Kafka

2017-02-22 Thread Chaoran Yu
Hello guys, I'm working on visualization based on Zeppelin that displays data coming from Kafka. I'm wondering if it's possible to make my plots update in real time as data keep coming in from Kafka. For example, consider a simple program shown below: %spark import

Re: [DISCUSS] Behaviour of Interpreter when it is restarted

2017-02-22 Thread Jongyoul Lee
Basically, I agree that we need to keep same behavior even though users restart in any place. I don't have any preference between restarting all processes and starting user's process but currently Zeppelin doesn't have any "admin" feature and no concept like admin by default. Thus we must not

Re: How can I use anaconda in zeppelin?

2017-02-23 Thread Beth Lee
Thank you I try using it. Thanks, Jin 2017-02-23 15:55 GMT+09:00 moon soo Lee : > If you're using '%python', not '%pyspark', you can try %python.conda to > change your environment. > > Run > > %python.conda help > > in the notebook will display available command. This allow

Re: How can I use anaconda in zeppelin?

2017-02-22 Thread moon soo Lee
If you're using '%python', not '%pyspark', you can try %python.conda to change your environment. Run %python.conda help in the notebook will display available command. This allow dynamically configure conda environment. Hope this helps. Thanks, moon On Thu, Feb 23, 2017 at 3:23 PM Beth Lee

Apply a search filter using active directory ldap

2017-02-22 Thread Corbella Riccardo Gianpaolo
Hi, I’m using the active directory in order to control the access to zeppelin 0.7.0. My configuration of the shiro.ini is the following: activeDirectoryRealm = org.apache.zeppelin.realm.ActiveDirectoryGroupRealm activeDirectoryRealm.systemUsername = foo activeDirectoryRealm.systemPassword =

Re: [DISCUSS] Behaviour of Interpreter when it is restarted

2017-02-22 Thread Jongyoul Lee
Yes, we can set some roles by Shiro but what I mean is we didn't implement role-based functions like hiding or showing some menu. On Thu, Feb 23, 2017 at 12:17 PM, Prabhjyot Singh wrote: > Yes, agreed we need some components to manage lifecycle of interpreters. > >

Re: Real-time Updated Plots with Kafka

2017-02-22 Thread Jongyoul Lee
It's interesting. :-) On Thu, Feb 23, 2017 at 11:52 AM, Chaoran Yu wrote: > Hello guys, > >I'm working on visualization based on Zeppelin that displays data > coming from Kafka. I'm wondering if it's possible to make my plots update > in real time as data keep

[DISCUSS] Admin feature

2017-02-22 Thread Jongyoul Lee
Hi folks, Recently, I've heard some new feature assumed that it needed the admin account or similar role. But Apache Zeppelin doesn't have any admin feature like hiding/showing menu and settings. I want to know how community thinks about that feature. My first concern is that we have to consider

plot multiple locations points in map

2017-02-22 Thread Alec Lee
Hi all My pyspark code is able to generate the list of locations (longitude, latitude), which I want to plot on the Map integrate in notebook. Any China map API that I can call? A sample code is preferred. thanks AL

How can I use anaconda in zeppelin?

2017-02-22 Thread Beth Lee
I want to use anaconda in zeppelin. So I modify the configure file in /zeppelin/conf/zeppelin-env.sh like below. export SPARK_HOME=/home/jin/spark export PYTHONPATH=/home/jin/anaconda3/bin/python export PYSPARK_PYTHON=/home/jin/spark/python Because I think when I modified the PYTHONPATH then I

Re: [DISCUSS] Admin feature

2017-02-22 Thread Jeff Zhang
I think in authenticated mode, admin is necessary. What admin should take charge can be in the following 4 areas: 1. Interpreter setting. Admin can control a template for each interpreter, and each user's template extend this base template but can also customize his own setting. Besides that,

Re: [DISCUSS] Admin feature

2017-02-22 Thread Alec Lee
We have multiple uses in our organization share same anonymous account, which will potentially cause the problems, we hope to have true authenticated mode. thanks AL > On Feb 22, 2017, at 9:14 PM, Jongyoul Lee wrote: > > Hi folks, > > Recently, I've heard some new

<    1   2   3   4   5   6   7   8   9   10   >