Re: [DISCUSS] Zeppelin 0.6.2 release

[Mina Lee]

Thank you for the opinions! I am going to create v0.6.2-rc1 tag at 1am 10
Oct in PDT.

On Sun, Oct 9, 2016 at 1:53 AM Felix Cheung 
wrote:

> Seems like we have a couple of good fixes.
>
>
>
> +1 for another release
>
>
>
>
>
>
>
>
>
>
>
> On Fri, Oct 7, 2016 at 11:53 PM -0700, "Ahyoung Ryu" <
> ahyoung...@apache.org> wrote:
>
>
>
> +1
>
>
>
> On Sat, Oct 8, 2016 at 1:45 PM, Prabhjyot Singh  >
>
> wrote:
>
>
>
> > +1
>
> >
>
> > On 8 Oct 2016 5:42 a.m., "Hyung Sung Shim"  wrote:
>
> >
>
> > > +1
>
> > >
>
> > > 2016년 10월 8일 토요일, Victor Manuel Garcia님이 작성한
>
> > 메시지:
>
> > >
>
> > > > +1
>
> > > >
>
> > > > El 7/10/2016 23:16, "moon soo Lee" >
>
> > > > escribió:
>
> > > >
>
> > > > > Although 0.6.1 supports Spark 2.0, there were some bugs around it.
>
> > > > > Current 0.6 branch resolves many annoying problems and i think
> it's a
>
> > > lot
>
> > > > > more stable than previous release, 0.6.1. Let's start release
> process
>
> > > for
>
> > > > > 0.6.2.
>
> > > > >
>
> > > > > Thanks,
>
> > > > > moon
>
> > > > >
>
> > > > > On Sat, Oct 8, 2016 at 3:33 AM DuyHai Doan 
> > > > > wrote:
>
> > > > >
>
> > > > > > +1
>
> > > > > >
>
> > > > > > On Fri, Oct 7, 2016 at 4:19 PM, Renjith Kamath <
>
> > > > renjith.kam...@gmail.com 
>
> > > > > >
>
> > > > > > wrote:
>
> > > > > >
>
> > > > > > > +1
>
> > > > > > >
>
> > > > > > > Thanks & Regards,
>
> > > > > > > RK
>
> > > > > > >
>
> > > > > > > On Fri, Oct 7, 2016 at 4:19 PM, Anthony Corbacho <
>
> > > > > > > anthonycorba...@apache.org 
>
> > > > > > > > wrote:
>
> > > > > > >
>
> > > > > > > > I also +1
>
> > > > > > > >
>
> > > > > > > > On Friday, 7 October 2016, Jeff Zhang 
> > > > > wrote:
>
> > > > > > > >
>
> > > > > > > > > +1 for it.
>
> > > > > > > > >
>
> > > > > > > > > On Fri, Oct 7, 2016 at 5:37 PM, Jongyoul Lee <
>
> > > jongy...@gmail.com
>
> > > > 
>
> > > > > > > > > > wrote:
>
> > > > > > > > >
>
> > > > > > > > > > Great job, Mina.
>
> > > > > > > > > >
>
> > > > > > > > > > I think it's enough to release 0.6.2.
>
> > > > > > > > > >
>
> > > > > > > > > > On Fri, Oct 7, 2016 at 4:39 PM, Mina Lee <
>
> > mina...@apache.org
>
> > > > 
>
> > > > > > > > > > wrote:
>
> > > > > > > > > >
>
> > > > > > > > > > > Hi devs,
>
> > > > > > > > > > >
>
> > > > > > > > > > > Release process has been delayed longer than we planned
>
> > > > > > > > > > > since several critical issues have been reported
>
> > > > continuously.
>
> > > > > > > > > > > [1] All issues that were blocking 0.6.2 have been
>
> > resolved
>
> > > > > > finally,
>
> > > > > > > > > > > so I want to hear thoughts about 0.6.2 release.
>
> > > > > > > > > > >
>
> > > > > > > > > > > Regards,
>
> > > > > > > > > > > Mina
>
> > > > > > > > > > >
>
> > > > > > > > > > > [1]
> https://issues.apache.org/jira/browse/ZEPPELIN-1347
>
> > > > > > > > > > >
>
> > > > > > > > > >
>
> > > > > > > > > >
>
> > > > > > > > > >
>
> > > > > > > > > > --
>
> > > > > > > > > > 이종열, Jongyoul Lee, 李宗烈
>
> > > > > > > > > > http://madeng.net
>
> > > > > > > > > >
>
> > > > > > > > >
>
> > > > > > > > >
>
> > > > > > > > >
>
> > > > > > > > > --
>
> > > > > > > > > Best Regards
>
> > > > > > > > >
>
> > > > > > > > > Jeff Zhang
>
> > > > > > > > >
>
> > > > > > > >
>
> > > > > > >
>
> > > > > >
>
> > > > >
>
> > > >
>
> > >
>
> >
>
>

[GitHub] zeppelin issue #1476: [WIP] ZEPPELIN-335. Pig Interpreter

[felixcheung]

Github user felixcheung commented on the issue:

https://github.com/apache/zeppelin/pull/1476
  
Thanks, comment replied, in particular, 
[this](https://github.com/apache/zeppelin/pull/1476#discussion_r82544875)


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] zeppelin issue #1426: ZEPPELIN-1430. Display appId and webui link in LivyInt...

[felixcheung]

Github user felixcheung commented on the issue:

https://github.com/apache/zeppelin/pull/1426
  
LGTM except for one question


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] zeppelin issue #1462: ZEPPELIN-1477. Add Integration Test for LivyInterprete...

[prabhjyotsingh]

Github user prabhjyotsingh commented on the issue:

https://github.com/apache/zeppelin/pull/1462
  
I too share same thought as @felixcheung, if we can avoid jar's, can we use 
`org.mockito.Mock` instead ?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] zeppelin issue #1499: [ZEPPELIN-1528] Add progress information in addition t...

[sadikovi]

Github user sadikovi commented on the issue:

https://github.com/apache/zeppelin/pull/1499
  
@Leemoonsoo @AhyoungRyu @felixcheung could you have a look at this PR? I am 
not sure about how good overall implementation is, and whether or not you guys 
want this feature in Zeppelin. Would appreciate the feedback, thanks.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] zeppelin pull request #1499: [ZEPPELIN-1528] Add progress information in add...

[sadikovi]

GitHub user sadikovi opened a pull request:

https://github.com/apache/zeppelin/pull/1499

[ZEPPELIN-1528] Add progress information in addition to progress percentage

### What is this PR for?
PR adds progress information in addition to progress percentage, allowing 
to provide links to Spark UI, for example. Currently done by adding 
`getProgressInfo()` method to the interpreter API that returns list of 
`InterpreterProgressInfo`, similar to completion API in this sense.
Main changes are here:
- 
`zeppelin-interpreter/src/main/java/org/apache/zeppelin/interpreter/Interpreter.java
 `
- 
`zeppelin-interpreter/src/main/java/org/apache/zeppelin/scheduler/Job.java`
- `zeppelin-interpreter/src/main/thrift/RemoteInterpreterService.thrift`
- 
`spark/src/main/scala/org/apache/zeppelin/spark/SparkJobDetailsListener.scala`
- `zeppelin-web/src/app/notebook/paragraph/paragraph.controller.js`
- `zeppelin-web/src/app/notebook/paragraph/paragraph-progressInfo.html`

### What type of PR is it?
Feature

### Todos
* [ ] - Add unittests for `SparkJobDetailsListener`

### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-1528

### How should this be tested?
Currently tested manually, by building distribution and launching simple 
Spark jobs in paragraph. 
Also inspecting that progress info is updated as jobs get executed and 
final result is returned correctly in stdout in paragraph (initially had 
problems with this since there is a difference in displaying between 0.6.x and 
0.7.x). 

### Screenshots (if appropriate)
![screenshot from 2016-10-10 17 38 
02](https://cloud.githubusercontent.com/assets/7788766/19227090/50c181ca-8f11-11e6-93e7-60dcc3805e63.png)

### Questions:
* Does the licenses files need update? No.
* Is there breaking changes for older versions? Possibly.
* Does this needs documentation? I am not sure, I added javadoc for the 
method.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/sadikovi/zeppelin spark-progress-info

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/zeppelin/pull/1499.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1499


commit 7b09074c5a13393a398b4eae6a7e438f8928952f
Author: Ivan Sadikov 
Date:   2016-10-04T19:59:36Z

update thrift

commit b50441260f59542df47ed719f806f9b763bf65c2
Author: Ivan Sadikov 
Date:   2016-10-05T03:55:48Z

add spark job details

commit 0bb8c4f9fe1fc2622420afffbfe524a7e41932cf
Author: Ivan Sadikov 
Date:   2016-10-05T20:45:20Z

add progress info api

commit dfef21ee022e03ab2c0c08f69445fdb1520aadd2
Author: Ivan Sadikov 
Date:   2016-10-06T04:11:00Z

update job details listener

commit 50b822dfb403c5c3e24db4a5ebbce16434c79bff
Author: Ivan Sadikov 
Date:   2016-10-06T04:44:25Z

fix case match

commit 7edd759c67b4e9dd74043029b8d3409e1cd156f4
Author: sadikovi 
Date:   2016-10-06T18:19:14Z

Merge remote-tracking branch 'origin/master' into spark-progress-info

commit 1157fbfd125c4eedb7b504fe5ce56c26fed6bdf0
Author: Ivan Sadikov 
Date:   2016-10-09T19:39:39Z

add final progress info update

commit 71aa17cb1359ed1e2cac5384d8f7e57f27c6fcb8
Author: Ivan Sadikov 
Date:   2016-10-10T04:36:06Z

improve final progress info update




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: [DISCUSS] Zeppelin 0.7 Roadmap

[Jeff Zhang]

+1 for adding impersonation. Consider most of subtasks of ZEPPELIN-1337
(multiple user support) will involve large changes, I am afraid it may be a
little risky to release 0.7 in Nov, and I expect we would introduce new
bugs since the changes are big.

On Mon, Oct 10, 2016 at 10:30 AM, Corneau Damien 
wrote:

> We don't have many routes, and so far no need for subroutes, so
> angular-ui-router isn't really needed.
> However lazy loading could be nice
>
> On Mon, Oct 10, 2016 at 10:53 AM, Dennis Jung  wrote:
>
> > Hi~
> > In fronted part, how about update router to ui-router with lazy load
> > module?
> > ui-router is 3rd party module, but it is more powerful than ngRoute. And
> > using lazy-loading will help resource loading performance.
> >
> > http://stackoverflow.com/questions/21023763/angularjs-
> > difference-between-angular-route-and-angular-ui-router
> >
> > How do you think? If it seems okay, I could help on this.
> > Thanks.
> >
> > 2016-10-10 0:22 GMT+09:00 DuyHai Doan :
> >
> > > I'd also add in the Enterprise section a topic about monitoring (e.g.
> at
> > > least exposing JMX metrics as a 1st step). I guess we cannot pretend to
> > be
> > > enterprise ready without at least some means to monitor the Zeppelin
> > server
> > > state.
> > >
> > > Hopefully the monitoring topic will be covered by the big refactoring
> > JIRA
> > > epic: https://issues.apache.org/jira/browse/ZEPPELIN-1525
> > >
> > > On Sun, Oct 9, 2016 at 4:53 PM, Jongyoul Lee 
> wrote:
> > >
> > > > Thanks for the sharing this roadmap.
> > > >
> > > > I'd like to add impersonation for Spark/JDBC interpreter.
> > > >
> > > > How do you think of ti?
> > > >
> > > > Regards,
> > > > Jongyoul
> > > >
> > > > On Sat, Oct 8, 2016 at 6:46 AM, moon soo Lee 
> wrote:
> > > >
> > > >> Hi, Apache Zeppelin users and developers,
> > > >>
> > > >> We're about to start a release process for 0.6.2 and i think it's
> good
> > > >> time to discuss about future release, 0.7.0. There were great
> > discussion
> > > >> about roadmap [1] and we updated wiki page [2], but 0.7.0 section on
> > the
> > > >> roadmap wiki is empty at the moment. Having a 0.7.0 section on a
> wiki
> > > page,
> > > >> i think, doesn't mean neither rejecting other subjects nor 100%
> > > guarantee
> > > >> of them. However it's more for helping Zeppelin users and developers
> > > have
> > > >> reasonable expectations for the next release and helps community
> focus
> > > on
> > > >> main theme.
> > > >>
> > > >> Multi-tenancy related feature (interpreter authorization,
> > impersonation,
> > > >> per user interpreter, and so on) are the most active subject in the
> > > >> community at the moment i think. And we have a new menu, "Job" in
> the
> > > >> master branch is another big change. I think "Enterprise ready"
> > section
> > > on
> > > >> [2] can be main topic for 0.7 release.
> > > >>
> > > >> And there're many improvements around generic JDBC interpreter and
> > > Python
> > > >> support (matplotlib integration and so on). They can be another
> > > important
> > > >> subject with new Interpreters.
> > > >>
> > > >> Besides that, I've seen people struggle with front-end performance
> and
> > > we
> > > >> can address that on 0.7, that would be great.
> > > >> Also i'd like to keep working on pluggability for visualization,
> which
> > > >> was subject from 0.6 release.
> > > >> Therefore, i would suggest following draft as a roadmap for 0.7.0
> > > >>
> > > >> * Enterprise support
> > > >>   - Multi user support (ZEPPELIN-1337)
> > > >>   - Job management
> > > >> * Interpreter
> > > >>   - Improve JDBC / Python interpreter
> > > >>   - New interpreters
> > > >> * Front end performance improvement
> > > >> * Pluggable visualization
> > > >>
> > > >> Regarding timeline, although we're keep making series of 0.6.x
> > release,
> > > >> it's already been 3 months since 0.6 release. And i think many items
> > are
> > > >> already been addressed on master branch or patches are available.
> So i
> > > >> think we can target near future, like November.
> > > >>
> > > >> What do you think? And feedback would be appreciated.
> > > >>
> > > >> Thanks,
> > > >> moon
> > > >>
> > > >> [1] http://apache-zeppelin-users-incubating-mailing-li
> > > >>  > > x6.nabble.com/DISCUSS-Update-Roadmap-tp2452.html>
> > > >> st.75479.x6.nabble.com/DISCUSS-Update-Roadmap-tp2452.html
> > > >>  > > x6.nabble.com/DISCUSS-Update-Roadmap-tp2452.html>
> > > >> [2] https://cwiki.apache.org/confluence/display/ZEPPELIN/
> > > Zeppelin+Roadmap
> > > >>
> > > >
> > > >
> > > >
> > > > --
> > > > 이종열, Jongyoul Lee, 李宗烈
> > > > http://madeng.net
> > > >
> > >
> >
>



-- 
Best Regards

Jeff Zhang

[jira] [Created] (ZEPPELIN-1529) Confusing Error Message for livy interpreter

[Jeff Zhang (JIRA)]

Jeff Zhang created ZEPPELIN-1529:


 Summary: Confusing Error Message for livy interpreter
 Key: ZEPPELIN-1529
 URL: https://issues.apache.org/jira/browse/ZEPPELIN-1529
 Project: Zeppelin
  Issue Type: Bug
Affects Versions: 0.6.1
Reporter: Jeff Zhang


The following code in livy interpreter will get very confusing error message.
{code}
sc.parallelize(1 to 10).
{code}

{noformat}
:29: error: value print is not a member of 
org.apache.spark.rdd.RDD[Int]
possible cause: maybe a semicolon is missing before `value print'?
  print("")
  ^
{noformat}

The cause is that we add print statement at the end of statement, not sure why 
we do that.

{code}
  String[] lines = stringLines.split("\n");
  String[] linesToRun = new String[lines.length + 1];
  for (int i = 0; i < lines.length; i++) {
linesToRun[i] = lines[i];
  }
  linesToRun[lines.length] = "print(\"\")";
{code}
\cc [~prabhjot]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] zeppelin issue #1426: ZEPPELIN-1430. Display appId and webui link in LivyInt...

[zjffdu]

Github user zjffdu commented on the issue:

https://github.com/apache/zeppelin/pull/1426
  
@prabhjyotsingh @felixcheung @Leemoonsoo I just update the PR to add a flag 
to display appInfo. By default it is false. 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] zeppelin pull request #1498: [DOCS] Fix dead link in docs/datasource_authori...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/zeppelin/pull/1498


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: [DISCUSS] Zeppelin 0.7 Roadmap

[Dennis Jung]

Hi~
In fronted part, how about update router to ui-router with lazy load module?
ui-router is 3rd party module, but it is more powerful than ngRoute. And
using lazy-loading will help resource loading performance.

http://stackoverflow.com/questions/21023763/angularjs-difference-between-angular-route-and-angular-ui-router

How do you think? If it seems okay, I could help on this.
Thanks.

2016-10-10 0:22 GMT+09:00 DuyHai Doan :

> I'd also add in the Enterprise section a topic about monitoring (e.g. at
> least exposing JMX metrics as a 1st step). I guess we cannot pretend to be
> enterprise ready without at least some means to monitor the Zeppelin server
> state.
>
> Hopefully the monitoring topic will be covered by the big refactoring JIRA
> epic: https://issues.apache.org/jira/browse/ZEPPELIN-1525
>
> On Sun, Oct 9, 2016 at 4:53 PM, Jongyoul Lee  wrote:
>
> > Thanks for the sharing this roadmap.
> >
> > I'd like to add impersonation for Spark/JDBC interpreter.
> >
> > How do you think of ti?
> >
> > Regards,
> > Jongyoul
> >
> > On Sat, Oct 8, 2016 at 6:46 AM, moon soo Lee  wrote:
> >
> >> Hi, Apache Zeppelin users and developers,
> >>
> >> We're about to start a release process for 0.6.2 and i think it's good
> >> time to discuss about future release, 0.7.0. There were great discussion
> >> about roadmap [1] and we updated wiki page [2], but 0.7.0 section on the
> >> roadmap wiki is empty at the moment. Having a 0.7.0 section on a wiki
> page,
> >> i think, doesn't mean neither rejecting other subjects nor 100%
> guarantee
> >> of them. However it's more for helping Zeppelin users and developers
> have
> >> reasonable expectations for the next release and helps community focus
> on
> >> main theme.
> >>
> >> Multi-tenancy related feature (interpreter authorization, impersonation,
> >> per user interpreter, and so on) are the most active subject in the
> >> community at the moment i think. And we have a new menu, "Job" in the
> >> master branch is another big change. I think "Enterprise ready" section
> on
> >> [2] can be main topic for 0.7 release.
> >>
> >> And there're many improvements around generic JDBC interpreter and
> Python
> >> support (matplotlib integration and so on). They can be another
> important
> >> subject with new Interpreters.
> >>
> >> Besides that, I've seen people struggle with front-end performance and
> we
> >> can address that on 0.7, that would be great.
> >> Also i'd like to keep working on pluggability for visualization, which
> >> was subject from 0.6 release.
> >> Therefore, i would suggest following draft as a roadmap for 0.7.0
> >>
> >> * Enterprise support
> >>   - Multi user support (ZEPPELIN-1337)
> >>   - Job management
> >> * Interpreter
> >>   - Improve JDBC / Python interpreter
> >>   - New interpreters
> >> * Front end performance improvement
> >> * Pluggable visualization
> >>
> >> Regarding timeline, although we're keep making series of 0.6.x release,
> >> it's already been 3 months since 0.6 release. And i think many items are
> >> already been addressed on master branch or patches are available. So i
> >> think we can target near future, like November.
> >>
> >> What do you think? And feedback would be appreciated.
> >>
> >> Thanks,
> >> moon
> >>
> >> [1] http://apache-zeppelin-users-incubating-mailing-li
> >>  x6.nabble.com/DISCUSS-Update-Roadmap-tp2452.html>
> >> st.75479.x6.nabble.com/DISCUSS-Update-Roadmap-tp2452.html
> >>  x6.nabble.com/DISCUSS-Update-Roadmap-tp2452.html>
> >> [2] https://cwiki.apache.org/confluence/display/ZEPPELIN/
> Zeppelin+Roadmap
> >>
> >
> >
> >
> > --
> > 이종열, Jongyoul Lee, 李宗烈
> > http://madeng.net
> >
>

[GitHub] zeppelin issue #1494: [ZEPPELIN-451] Save codes and messages as multi-line

[sixmen]

Github user sixmen commented on the issue:

https://github.com/apache/zeppelin/pull/1494
  
It supports old notebook format.
Old code can not read new notebook format, but it can not be helped any way.
So I think it is an endurable breaking change.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] zeppelin issue #1494: [ZEPPELIN-451] Save codes and messages as multi-line

[anthonycorbacho]

Github user anthonycorbacho commented on the issue:

https://github.com/apache/zeppelin/pull/1494
  
@sixmen thank you for your contribution, this feature looks AWESOME.
I didnt dive into the code, but one question came in my mind;

This change is kinda *breaking change* since the notebook format is now 
different from what we have. So what can be the best way in your opinion to 
support *old* notebook format?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] zeppelin issue #1476: [WIP] ZEPPELIN-335. Pig Interpreter

[zjffdu]

Github user zjffdu commented on the issue:

https://github.com/apache/zeppelin/pull/1476
  
@felixcheung Thanks for the review, license is added and comments are 
addressed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] zeppelin issue #1496: [Zeppelin-907] Apply new mechanism to AngularInterpret...

[vectorijk]

Github user vectorijk commented on the issue:

https://github.com/apache/zeppelin/pull/1496
  
@jongyoul Thanks!


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] zeppelin issue #1378: [ZEPPELIN-1358] Add support to display Pandas DataFram...

[bustios]

Github user bustios commented on the issue:

https://github.com/apache/zeppelin/pull/1378
  
Sorry for the late response @bzz. Of course it's possible. I already made 
the changes to make it optional.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] zeppelin issue #1476: [WIP] ZEPPELIN-335. Pig Interpreter

[felixcheung]

Github user felixcheung commented on the issue:

https://github.com/apache/zeppelin/pull/1476
  
how about LICENSE file?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (ZEPPELIN-1528) Add progress information in addition to progress percentage

[Ivan Sadikov (JIRA)]

Ivan Sadikov created ZEPPELIN-1528:
--

 Summary: Add progress information in addition to progress 
percentage
 Key: ZEPPELIN-1528
 URL: https://issues.apache.org/jira/browse/ZEPPELIN-1528
 Project: Zeppelin
  Issue Type: Improvement
  Components: GUI, zeppelin-interpreter
Reporter: Ivan Sadikov
Assignee: Ivan Sadikov
Priority: Minor


Add richer progress information for running paragraph, e.g. what jobs have been 
executed so far and link to Spark UI. Currently only progress percentage is 
shown.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] zeppelin issue #1497: [ZEPPELIN-1233] Support Redis interpreter

[felixcheung]

Github user felixcheung commented on the issue:

https://github.com/apache/zeppelin/pull/1497
  
Hi - thank you for your contribution. Could you see the list of steps here:

https://zeppelin.apache.org/docs/0.6.1/development/writingzeppelininterpreter.html#contributing-a-new-interpreter-to-zeppelin-releases

Notably, could you add:
- tests
- documentation
- list licenses for dependencies



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] zeppelin issue #1498: [DOCS] Fix dead link in docs/datasource_authorization....

[felixcheung]

Github user felixcheung commented on the issue:

https://github.com/apache/zeppelin/pull/1498
  
LGTM


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: [DISCUSS] Zeppelin 0.7 Roadmap

[DuyHai Doan]

I'd also add in the Enterprise section a topic about monitoring (e.g. at
least exposing JMX metrics as a 1st step). I guess we cannot pretend to be
enterprise ready without at least some means to monitor the Zeppelin server
state.

Hopefully the monitoring topic will be covered by the big refactoring JIRA
epic: https://issues.apache.org/jira/browse/ZEPPELIN-1525

On Sun, Oct 9, 2016 at 4:53 PM, Jongyoul Lee  wrote:

> Thanks for the sharing this roadmap.
>
> I'd like to add impersonation for Spark/JDBC interpreter.
>
> How do you think of ti?
>
> Regards,
> Jongyoul
>
> On Sat, Oct 8, 2016 at 6:46 AM, moon soo Lee  wrote:
>
>> Hi, Apache Zeppelin users and developers,
>>
>> We're about to start a release process for 0.6.2 and i think it's good
>> time to discuss about future release, 0.7.0. There were great discussion
>> about roadmap [1] and we updated wiki page [2], but 0.7.0 section on the
>> roadmap wiki is empty at the moment. Having a 0.7.0 section on a wiki page,
>> i think, doesn't mean neither rejecting other subjects nor 100% guarantee
>> of them. However it's more for helping Zeppelin users and developers have
>> reasonable expectations for the next release and helps community focus on
>> main theme.
>>
>> Multi-tenancy related feature (interpreter authorization, impersonation,
>> per user interpreter, and so on) are the most active subject in the
>> community at the moment i think. And we have a new menu, "Job" in the
>> master branch is another big change. I think "Enterprise ready" section on
>> [2] can be main topic for 0.7 release.
>>
>> And there're many improvements around generic JDBC interpreter and Python
>> support (matplotlib integration and so on). They can be another important
>> subject with new Interpreters.
>>
>> Besides that, I've seen people struggle with front-end performance and we
>> can address that on 0.7, that would be great.
>> Also i'd like to keep working on pluggability for visualization, which
>> was subject from 0.6 release.
>> Therefore, i would suggest following draft as a roadmap for 0.7.0
>>
>> * Enterprise support
>>   - Multi user support (ZEPPELIN-1337)
>>   - Job management
>> * Interpreter
>>   - Improve JDBC / Python interpreter
>>   - New interpreters
>> * Front end performance improvement
>> * Pluggable visualization
>>
>> Regarding timeline, although we're keep making series of 0.6.x release,
>> it's already been 3 months since 0.6 release. And i think many items are
>> already been addressed on master branch or patches are available. So i
>> think we can target near future, like November.
>>
>> What do you think? And feedback would be appreciated.
>>
>> Thanks,
>> moon
>>
>> [1] http://apache-zeppelin-users-incubating-mailing-li
>> 
>> st.75479.x6.nabble.com/DISCUSS-Update-Roadmap-tp2452.html
>> 
>> [2] https://cwiki.apache.org/confluence/display/ZEPPELIN/Zeppelin+Roadmap
>>
>
>
>
> --
> 이종열, Jongyoul Lee, 李宗烈
> http://madeng.net
>

[GitHub] zeppelin pull request #1496: [Zeppelin-907] Apply new mechanism to AngularIn...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/zeppelin/pull/1496


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] zeppelin issue #1369: [ZEPPELIN-1376] Add proxy credentials for dependency r...

[doanduyhai]

Github user doanduyhai commented on the issue:

https://github.com/apache/zeppelin/pull/1369
  
OK I'll merge if no more remarks


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] zeppelin issue #1487: [ZEPPELIN-1520] Change test environment to use the loc...

[jongyoul]

Github user jongyoul commented on the issue:

https://github.com/apache/zeppelin/pull/1487
  
@zjffdu I've missed remove some scripts in `testing`. I'll fix it. 
Concerning the modes, in an early stage of Spark, there're some different 
behaviours of different modes, thus we need to test Zeppelin with Spark in a 
standalone cluster, but for now, Spark become stable and if different modes 
have different behaviours, it's definitely Spark's problem and Spark should fix 
it. Thus I don't think we need to test Zeppelin a cluster mode. Furthermore, 
personally, I don't know what benefit we have in a different test environment 
between local and CI. In case of Python, I've experienced that some tests only 
failed in CI, not in a local with different setting between them. I think we 
should try to keep both environments equally.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: How Apache Zeppelin runs a paragraph

[Jongyoul Lee]

Thanks you Jeff for the comments,

1. You're right, so some contribution already is submitted and merged.
Tagging these kind of bugs as `newbie` and `beginner` would be helpful.
2. I'll plan to write a post about Interpreters including this topic.

Thanks,
Jongyoul

On Sun, Oct 9, 2016 at 9:29 AM, Jeff Zhang  wrote:

> Thanks Jougyoul for the effort. This is really helpful for new
> contributors.
>
> Several comments.
>
> 1. Seems we didn't use noteId/notebookId correctly in code, do we have
> ticket to track that ? Because the sample code is conflicted with the
> definition of note/notebook in the post, it might confuse users.
>
> @Path("job/{notebookId}/{paragraphId}")
> @ZeppelinApi
> public Response runParagraph(@PathParam("notebookId") String notebookId,
> @PathParam("paragraphId") String paragraphId, String message)
> throws IOException, IllegalArgumentException {
>   LOG.info("run paragraph job asynchronously {} {} {}", notebookId, 
> paragraphId, message);
>
>   Note note = notebook.getNote(notebookId);
>
>
> 2. It might be better to use diagram to visualize the differences between 
> shared/scoped/shared
>
>
>
>
> On Sun, Oct 9, 2016 at 12:38 AM, Felix Cheung 
> wrote:
>
>> Great post!
>>
>>
>>
>>
>>
>> On Tue, Oct 4, 2016 at 8:56 PM -0700, "Jongyoul Lee" 
>> wrote:
>>
>> Hello DuyHai,
>>
>> Thanks for the fixing the typo. I've fixed it.
>>
>> Concerning the debugging, I think writing posts or updating wiki would be
>> better. I'm willing to write a simple post for that remote debugging.
>>
>> On Tue, Oct 4, 2016 at 5:53 PM, DuyHai Doan  wrote:
>>
>>> Hello Jongyoul
>>>
>>> Excellent write up to demystify the execution life-cycle of a paragraph.
>>>
>>> There is a small typo in the blog: "At last, it has two words separated
>>> by comma, *getInterpreter* handles it as “%{group_name}.{interpreter_na
>>> me}”"
>>>
>>> Instead of comma, shouldn't it be DOT (.) ?
>>>
>>> There is also another unknown details that may be interesting to expose:
>>> how are interpreter JVM launched ?
>>>
>>> InterpreterFactory
>>>
>>> private Interpreter createRemoteRepl(String interpreterPath, String noteId, 
>>> String className,
>>> Properties property, String interpreterSettingId) {
>>>   int connectTimeout = 
>>> conf.getInt(ConfVars.ZEPPELIN_INTERPRETER_CONNECT_TIMEOUT);
>>>   String localRepoPath = conf.getInterpreterLocalRepoPath() + "/" + 
>>> interpreterSettingId;
>>>   int maxPoolSize = 
>>> conf.getInt(ConfVars.ZEPPELIN_INTERPRETER_MAX_POOL_SIZE);
>>>
>>>   RemoteInterpreter remoteInterpreter =
>>>   new RemoteInterpreter(property, noteId, className, 
>>> conf.getInterpreterRemoteRunnerPath(),
>>>   interpreterPath, localRepoPath, connectTimeout, maxPoolSize,
>>>   remoteInterpreterProcessListener, appEventListener);
>>>   remoteInterpreter.addEnv(env);
>>>
>>>   return new LazyOpenInterpreter(remoteInterpreter);
>>> }
>>>
>>>
>>> The code will call the bin/intepreter.sh (or interpreter.cmd if on
>>> Windows) to launch a new JVM for the interpreter.
>>>
>>> And depending on the type of interpreter, the command will be different:
>>>
>>> if [[ -n "${SPARK_SUBMIT}" ]]; then
>>> ${SPARK_SUBMIT} --class ${ZEPPELIN_SERVER} --driver-class-path
>>> "${ZEPPELIN_INTP_CLASSPATH_OVERRIDES}:${CLASSPATH}"
>>> --driver-java-options "${JAVA_INTP_OPTS}" ${SPARK_SUBMIT_OPTIONS}
>>> ${SPARK_APP_JAR} ${PORT} &
>>> else
>>> ${ZEPPELIN_RUNNER} ${JAVA_INTP_OPTS} ${ZEPPELIN_INTP_MEM} -cp
>>> ${ZEPPELIN_INTP_CLASSPATH_OVERRIDES}:${CLASSPATH} ${ZEPPELIN_SERVER}
>>> ${PORT} &
>>> fi
>>>
>>> There is a special case for Spark because we execute a SparkSubmit.
>>>
>>> This detail is important for contributors because when they need to
>>> DEBUG an interpreter code, they have to modify the interpreter.sh file to
>>> add an "-Xdebug -Xrunjdwp:transport=dt_socket,
>>> server=y,suspend=n,address=xxx"
>>>
>>> Most of people naively add this debug argument to the zeppelin.sh file
>>> but it only debugs the Zeppelin server itself, not the interpreter.
>>>
>>> Do you think we may create a page in the official doc for this ?
>>>
>>>
>>> On Tue, Oct 4, 2016 at 9:00 AM, Jongyoul Lee  wrote:
>>>
 Hello devs and users,

 I've submitted a post into Apache Zeppelin Stories on medium.com:
 https://medium.com/apache-zeppelin-stories/how-a
 pache-zeppelin-runs-a-paragraph-783a0a612ba9#.x9ym8fovv

 I hope all users and contributors understand Apache Zeppelin deeply.

 Regards,
 Jongyoul

 --
 이종열, Jongyoul Lee, 李宗烈
 http://madeng.net

>>>
>>>
>>
>>
>> --
>> 이종열, Jongyoul Lee, 李宗烈
>> http://madeng.net
>>
>
>
>
> --
> Best Regards
>
> Jeff Zhang
>



-- 
이종열, Jongyoul Lee, 李宗烈
http://madeng.net

[GitHub] zeppelin issue #1369: [ZEPPELIN-1376] Add proxy credentials for dependency r...

[AhyoungRyu]

Github user AhyoungRyu commented on the issue:

https://github.com/apache/zeppelin/pull/1369
  
@doanduyhai Checked again and it works well! After removing the wrong 
dependency, the error message doesn't show up anymore.



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] zeppelin issue #1476: [WIP] ZEPPELIN-335. Pig Interpreter

[zjffdu]

Github user zjffdu commented on the issue:

https://github.com/apache/zeppelin/pull/1476
  
Thanks @AhyoungRyu for the help, merged.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] zeppelin issue #1472: Add message of localhost:port on script

[AhyoungRyu]

Github user AhyoungRyu commented on the issue:

https://github.com/apache/zeppelin/pull/1472
  
@djKooks Yeah you're right. I left the comment like that at the first time, 
I realized that it's different change after checking it again. Sorry for any 
confusion. 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] zeppelin issue #1472: Add message of localhost:port on script

[djKooks]

Github user djKooks commented on the issue:

https://github.com/apache/zeppelin/pull/1472
  
@AhyoungRyu It looks different. The one in #1399 records ip address in log, 
but I want to show this during script is running. It seems ok to keep working 
on.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] zeppelin issue #1369: [ZEPPELIN-1376] Add proxy credentials for dependency r...

[AhyoungRyu]

Github user AhyoungRyu commented on the issue:

https://github.com/apache/zeppelin/pull/1369
  
@doanduyhai Yeah your screenshot is saying so. Let me test it again. Thanks 
for your quick response! 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] zeppelin issue #1476: [WIP] ZEPPELIN-335. Pig Interpreter

[AhyoungRyu]

Github user AhyoungRyu commented on the issue:

https://github.com/apache/zeppelin/pull/1476
  
@zjffdu Thanks for your great work! 👍 
While I was quickly looking through `pig.md`, found some need-to 
be-changed(?) parts. So I've just pushed my commits to your Zeppelin repo 
instead of leaving multiple review comments one by one in here. Could you check 
[https://github.com/zjffdu/zeppelin/pull/2](https://github.com/zjffdu/zeppelin/pull/2)
 out? :)


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] zeppelin issue #1210: [ZEPPELIN-1220] Add geographical map as visualization ...

[Leemoonsoo]

Github user Leemoonsoo commented on the issue:

https://github.com/apache/zeppelin/pull/1210
  
Dear @mwpenny and Zeppelin PMC.

I have reviewed the license of ArcGIS Online 
https://developers.arcgis.com/terms/ and and 
http://www.esri.com/legal/software-license.
Although it's client javascript library is Apache 2, client does not 
functioning without ArcGIS online service, so the ArcGIS Online license does 
matter. However, the license does not look like type of license we can use. For 
example

> Generally, if you are not planning to monetize your app, say by charging 
a fee or by embedding revenue generating advertising in your app, you can make 
your app available to just about anyone.

and

> If you want to charge for access to apps you build using ArcGIS Online 
you have to have a paid subscription.

Apache 2 License allow monetize Apache Zeppelin. However ArcGIS Online 
license has incompatible restriction.

Also the client is loading arbitrary code (javascript) dynamically from 
their server on runtime and we can not guarantee what kind of code will be 
executed on our user's machine. 
see 
https://github.com/apache/zeppelin/blob/master/zeppelin-web/src/app/notebook/paragraph/paragraph.controller.js#L1392

Therefore, unfortunately, we might need to remove this changes from our 
source tree.



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] zeppelin issue #1480: [ZEPPELIN-1489] Remove jdbc-like interpreter example p...

[AhyoungRyu]

Github user AhyoungRyu commented on the issue:

https://github.com/apache/zeppelin/pull/1480
  
@Leemoonsoo Thanks!


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (ZEPPELIN-1527) Support Redis Cluster Interpreter

[darion yaphet (JIRA)]

darion yaphet created ZEPPELIN-1527:
---

 Summary: Support Redis Cluster Interpreter
 Key: ZEPPELIN-1527
 URL: https://issues.apache.org/jira/browse/ZEPPELIN-1527
 Project: Zeppelin
  Issue Type: Sub-task
Reporter: darion yaphet






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (ZEPPELIN-1526) Drop box to select running type

[darion yaphet (JIRA)]

darion yaphet created ZEPPELIN-1526:
---

 Summary: Drop box to select running type
 Key: ZEPPELIN-1526
 URL: https://issues.apache.org/jira/browse/ZEPPELIN-1526
 Project: Zeppelin
  Issue Type: Improvement
  Components: GUI
Reporter: darion yaphet


The running type such as alluxio ,  hbase and sql which we are specified use a 
drop box to select it maybe better .



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Refactor time !

[moon soo Lee]

Thanks for taking care!

On Sun, Oct 9, 2016 at 4:05 PM DuyHai Doan  wrote:

> I have created a JIRA epic to track down all the task:
> https://issues.apache.org/jira/browse/ZEPPELIN-1525
>
> I think I would start by the synchronize blocks and then move onto Eric's
> PR for Guice DI.
>
> After we have a DI mechanism, it will be much easier to inject thread pools
> for thread management and also to create JMX monitoring
>
> Any objection before I start coding ?
>
>
>
> On Sat, Oct 8, 2016 at 2:05 PM, Eric Charles  wrote:
>
> > On 04/10/16 12:54, Anthony Corbacho wrote:
> >
> >> You made my day, this is the kind of email i really like !!
> >>
> >> I think its a great idea and i am willing to spend sometime on it.
> >>
> >> I also want to move to a DI (guice) architecture , let me know what you
> >> think about it.
> >>
> >
> > A PR is opened for Guice DI. If someone jumps for review, I can rebase
> >
> > https://github.com/apache/zeppelin/pull/1361
> >
> >
> >
> >
> >> On Tuesday, 4 October 2016, DuyHai Doan  wrote:
> >>
> >> Hello devs
> >>>
> >>> The code base of Zeppelin has grown very fast in the last 12 months and
> >>> it's great. It means that we have more and more contributors.
> >>>
> >>> However, to make the project maintainable at long term, we need regular
> >>> code refactoring.
> >>>
> >>> I have some ideas to share with you
> >>>
> >>> 1) Use Java 8 to benefit from Lambda & streams.
> >>>
> >>>   Now that Java 8 is well established, it is a good time to upgrade the
> >>> project. I believe some interpreters also need Java 8. Cassandra
> >>> interpreter right now does not have unit tests for the latest features
> >>> because the Embedded Cassandra server used for testing requires Java 8.
> >>>
> >>>  It would also be a good opportunity to go through the code base and
> >>> replace some boilerplate for() loop with manual filtering by the stream
> >>> shortcut :  list.stream().filter(..).map(). It would improve greatly
> >>> code
> >>> readability
> >>>
> >>> 2) Multi threading
> >>>
> >>>  I've seen the usage of synchronize block at a few places in the code
> >>> base.
> >>> Although perfectly valid, it has a cost at runtime and since more and
> >>> more
> >>> people are asking for multi-tenancy or using a single Zeppelin instance
> >>> to
> >>> server multiple users, I guess the synchronized blocks has a huge cost.
> >>>
> >>> There are some solid alternatives:
> >>>
> >>>  - ConcurrentHashMap if we synchronized on a map
> >>>  - CopyOnWriteArrayList if we synchronized on a list.
> >>>
> >>> Of cours each sychronize block should be taken carefully not to
> introduce
> >>> regression
> >>>
> >>> 3) Thread management
> >>>
> >>> I've seen some usage of new Thread() {...}.run(); it may be a good time
> >>> to
> >>> introduce ThreadPool and pass them along (inside context objects for
> >>> example) to have a more centralized thread management
> >>>
> >>> The advantage of having thread pool is that we can manage them in a
> >>> single
> >>> place, monitor them and expose the info through JMX and also control
> >>> system
> >>> resource by defining max thread number and thread pool queue
> >>>
> >>> 4) Server monitoring
> >>> I hear many users on the field complain about the fact that they have
> to
> >>> restart Zeppelin server regularly because it "hangs" after running a
> long
> >>> time.
> >>>
> >>> If we can expose some system metrics through JMX, it would help people
> >>> monitor the state of Zeppelin server and take appropriate actions
> >>>
> >>> Right now we may only focus on monitoring the server itself, not the
> >>> interpreter JVMs processes. It can be done in a 2nd step
> >>>
> >>>
> >>> What do you think about the ideas ?
> >>>
> >>>
> >>
>

[GitHub] zeppelin issue #1369: [ZEPPELIN-1376] Add proxy credentials for dependency r...

[doanduyhai]

Github user doanduyhai commented on the issue:

https://github.com/apache/zeppelin/pull/1369
  
@AhyoungRyu @astroshim 

Please tell me if it's ok so that I can merge. We need to merge this PR 
before I can move on the refactoring task (of changing `synchronized` blocks by 
appropriate datastructures)


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: Refactor time !

[DuyHai Doan]

I have created a JIRA epic to track down all the task:
https://issues.apache.org/jira/browse/ZEPPELIN-1525

I think I would start by the synchronize blocks and then move onto Eric's
PR for Guice DI.

After we have a DI mechanism, it will be much easier to inject thread pools
for thread management and also to create JMX monitoring

Any objection before I start coding ?



On Sat, Oct 8, 2016 at 2:05 PM, Eric Charles  wrote:

> On 04/10/16 12:54, Anthony Corbacho wrote:
>
>> You made my day, this is the kind of email i really like !!
>>
>> I think its a great idea and i am willing to spend sometime on it.
>>
>> I also want to move to a DI (guice) architecture , let me know what you
>> think about it.
>>
>
> A PR is opened for Guice DI. If someone jumps for review, I can rebase
>
> https://github.com/apache/zeppelin/pull/1361
>
>
>
>
>> On Tuesday, 4 October 2016, DuyHai Doan  wrote:
>>
>> Hello devs
>>>
>>> The code base of Zeppelin has grown very fast in the last 12 months and
>>> it's great. It means that we have more and more contributors.
>>>
>>> However, to make the project maintainable at long term, we need regular
>>> code refactoring.
>>>
>>> I have some ideas to share with you
>>>
>>> 1) Use Java 8 to benefit from Lambda & streams.
>>>
>>>   Now that Java 8 is well established, it is a good time to upgrade the
>>> project. I believe some interpreters also need Java 8. Cassandra
>>> interpreter right now does not have unit tests for the latest features
>>> because the Embedded Cassandra server used for testing requires Java 8.
>>>
>>>  It would also be a good opportunity to go through the code base and
>>> replace some boilerplate for() loop with manual filtering by the stream
>>> shortcut :  list.stream().filter(..).map(). It would improve greatly
>>> code
>>> readability
>>>
>>> 2) Multi threading
>>>
>>>  I've seen the usage of synchronize block at a few places in the code
>>> base.
>>> Although perfectly valid, it has a cost at runtime and since more and
>>> more
>>> people are asking for multi-tenancy or using a single Zeppelin instance
>>> to
>>> server multiple users, I guess the synchronized blocks has a huge cost.
>>>
>>> There are some solid alternatives:
>>>
>>>  - ConcurrentHashMap if we synchronized on a map
>>>  - CopyOnWriteArrayList if we synchronized on a list.
>>>
>>> Of cours each sychronize block should be taken carefully not to introduce
>>> regression
>>>
>>> 3) Thread management
>>>
>>> I've seen some usage of new Thread() {...}.run(); it may be a good time
>>> to
>>> introduce ThreadPool and pass them along (inside context objects for
>>> example) to have a more centralized thread management
>>>
>>> The advantage of having thread pool is that we can manage them in a
>>> single
>>> place, monitor them and expose the info through JMX and also control
>>> system
>>> resource by defining max thread number and thread pool queue
>>>
>>> 4) Server monitoring
>>> I hear many users on the field complain about the fact that they have to
>>> restart Zeppelin server regularly because it "hangs" after running a long
>>> time.
>>>
>>> If we can expose some system metrics through JMX, it would help people
>>> monitor the state of Zeppelin server and take appropriate actions
>>>
>>> Right now we may only focus on monitoring the server itself, not the
>>> interpreter JVMs processes. It can be done in a 2nd step
>>>
>>>
>>> What do you think about the ideas ?
>>>
>>>
>>

[jira] [Created] (ZEPPELIN-1525) Refactoring round 1 Epic

[DOAN DuyHai (JIRA)]

DOAN DuyHai created ZEPPELIN-1525:
-

 Summary: Refactoring round 1 Epic
 Key: ZEPPELIN-1525
 URL: https://issues.apache.org/jira/browse/ZEPPELIN-1525
 Project: Zeppelin
  Issue Type: Improvement
  Components: Core, zeppelin-interpreter, zeppelin-server, 
zeppelin-zengine
Affects Versions: 0.7.0
Reporter: DOAN DuyHai
Assignee: DOAN DuyHai


This is the parent ticket for internal refactoring round 1



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] zeppelin issue #133: [ZEPPELIN-127] Fix issue raised by ZEPPELIN-127 related...

[doanduyhai]

Github user doanduyhai commented on the issue:

https://github.com/apache/zeppelin/pull/133
  
@Leemoonsoo 

The fix is using pretty standard Scala (`object xxx extends Serializable`) 
and not impacted by Scala 2.11 breaking changes


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] zeppelin issue #1480: [ZEPPELIN-1489] Remove jdbc-like interpreter example p...

[Leemoonsoo]

Github user Leemoonsoo commented on the issue:

https://github.com/apache/zeppelin/pull/1480
  
@AhyoungRyu Great work! It definitely helps using JDBC datasource in 
Zeppelin.
CI failure is not related.

LGTM and merge if there're no further comments.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---