Re: [VOTE] Release Apache Zeppelin 0.10.1 RC2

[moon soo Lee]

+1

On Thu, Feb 24, 2022 at 9:31 AM Prabhjyot Singh 
wrote:

> +1
>
> On Thu, Feb 24, 2022 at 1:58 PM Jeff Zhang  wrote:
>
>> Hi folks,
>>
>> I propose the following RC to be released for the Apache Zeppelin
>> 0.10.1 release.
>>
>>
>> The commit id is 1654ce3847f4ebce7b228f54671340569d25a32e :
>>
>> https://git-wip-us.apache.org/repos/asf?p=zeppelin.git;a=commit;h=1654ce3847f4ebce7b228f54671340569d25a32e
>>
>>
>>
>> This corresponds to the tag: v0.10.1-rc2 :
>>
>> https://git-wip-us.apache.org/repos/asf?p=zeppelin.git;a=shortlog;h=refs/tags/v0.10.1-rc2
>>
>> The release archives (tgz), signature, and checksums are here
>> https://dist.apache.org/repos/dist/dev/zeppelin/zeppelin-0.10.1-rc2/
>>
>> The release candidate consists of the following source distribution
>> archive
>> zeppelin-0.10.1.tgz
>>
>> In addition, the following supplementary binary distributions are provided
>> for user convenience at the same location
>> zeppelin-0.10.1-bin-all.tgz
>>
>>
>> The maven artifacts are here
>>
>> https://repository.apache.org/content/repositories/orgapachezeppelin-1323/org/apache/zeppelin/
>>
>> You can find the KEYS file here:
>> https://dist.apache.org/repos/dist/release/zeppelin/KEYS
>>
>> Release notes available at
>>
>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12316221=12350543
>>
>>
>> Vote will be open for next 72 hours (close at 12PM 26/Feb PDT).
>>
>> [ ] +1 approve
>> [ ] 0 no opinion
>>
>> [ ] -1 disapprove (and reason why)
>>
>>
>>
>> --
>> Best Regards
>>
>> Jeff Zhang
>>
>
>
> --
> Regards,
> Prabhjyot Singh
>

Re: [VOTE] Release Apache Zeppelin 0.10.0 RC2

[moon soo Lee]

+1

Thank Jeff for preparing the RC.


On Thu, Aug 19, 2021 at 7:24 PM Jeff Zhang  wrote:

> +1，  Verified most of the tutorial notes
>
> Jongyoul Lee  于2021年8月19日周四 下午12:20写道：
>
>> +1
>>
>> I've checked the basic example about this RC
>>
>> 2021년 8월 18일 (수) 오후 3:28, 님이 작성:
>>
>>> +1
>>>
>>>
>>>
>>> *From:* Philipp Dallig 
>>> *Sent:* Wednesday, August 18, 2021 3:24 PM
>>> *To:* us...@zeppelin.apache.org
>>> *Subject:* Re: [VOTE] Release Apache Zeppelin 0.10.0 RC2
>>>
>>>
>>>
>>> +1
>>>
>>> Am 18.08.21 um 08:01 schrieb Francois Reynald:
>>>
>>> +1
>>>
>>>
>>>
>>> On 18 Aug 2021, at 05:52, Jeff Zhang  wrote:
>>>
>>>
>>>
>>>
>>> Hi folks,
>>>
>>>
>>>
>>> I propose the following RC to be released for the Apache Zeppelin 0.10.0 
>>> release.
>>>
>>>
>>>
>>>
>>>
>>> The commit id is 8f85476c40df3536718c5aa2c07b519b6bb6607e :
>>>
>>> https://git-wip-us.apache.org/repos/asf?p=zeppelin.git;a=commit;h=8f85476c40df3536718c5aa2c07b519b6bb6607e
>>>
>>>
>>>
>>> This corresponds to the tag: v0.10.0-rc2 :
>>>
>>> https://git-wip-us.apache.org/repos/asf?p=zeppelin.git;a=shortlog;h=refs/tags/v0.10.0-rc2
>>>
>>> The release archives (tgz), signature, and checksums are here
>>>
>>> https://dist.apache.org/repos/dist/dev/zeppelin/zeppelin-0.10.0-rc2/
>>>
>>>
>>>
>>> The release candidate consists of the following source distribution archive
>>>
>>> zeppelin-0.10.0.tgz
>>>
>>>
>>>
>>> In addition, the following supplementary binary distributions are provided
>>>
>>> for user convenience at the same location
>>>
>>> zeppelin-0.10.0-bin-all.tgz
>>>
>>>
>>>
>>>
>>>
>>> The maven artifacts are here
>>>
>>> https://repository.apache.org/content/repositories/orgapachezeppelin-1302/org/apache/zeppelin/
>>>
>>>
>>>
>>> You can find the KEYS file here:
>>>
>>> https://dist.apache.org/repos/dist/release/zeppelin/KEYS
>>>
>>>
>>>
>>> Release notes available at 
>>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12349536==12316221
>>>
>>>
>>>
>>> Vote will be open for next 72 hours (close at 10PM 20/Aug PDT).
>>>
>>>
>>>
>>> [ ] +1 approve
>>>
>>> [ ] 0 no opinion
>>>
>>> [ ] -1 disapprove (and reason why)
>>>
>>>
>>>
>>>
>>>
>>> --
>>>
>>> Best Regards
>>>
>>> Jeff Zhang
>>>
>>>
>>>
>>> *François Reynald | Educational Services*
>>> t. +33 (0) 624 200 125
>>> cloudera.com 
>>>
>>>
>>>
>>>
>>
>> --
>> 이종열, Jongyoul Lee, 李宗烈
>> http://madeng.net
>>
>
>
> --
> Best Regards
>
> Jeff Zhang
>

Re: CI plattform

[moon soo Lee]

Before, I tried to migrate to Github Actions. ZEPPELIN-4385
.
Zeppelin project repository was not configured to trigger build (INFRA-19249

).

However, it looks like other repositories under Apache GitHub organization
can leverage Github Actions. e.g.
https://github.com/apache/incubator-tvm/actions. So it looks like there's a
way to use Github Actions.

I also agree that the CI platform that just works for a new user without
any setup is pretty important.

Thanks,
moon



On Mon, Nov 23, 2020 at 6:11 PM Jeff Zhang  wrote:

> Thanks for bring this up @Philipp Dallig 
> This is also an issue for me, but I don't know what other better options we
> have, maybe we can use apache's jenkins platform, but it may be very slow.
> Or we can check what other apache project use for CI
>
> Philipp Dallig  于2020年11月21日周六 下午9:13写道：
>
> > Hi Folks,
> >
> > I think that travis-ci.org and travis-ci.com can no longer be used for
> > our CI tests. Due to a restriction for open source builds
> > (https://twitter.com/geerlingguy/status/1324480667984498689) I have
> > migrated my Zeppelin fork from travis-ci.org to travis-ci.com.
> >
> > Last week I received 1 credits for starting travis-ci.com build
> > jobs. Unfortunately these credits are used up very quickly due to the
> > many and long tests. I used up these 1 points within one day.
> >
> > Some facts from the timeline:
> >   - In May 2018 Travis CI announces open source project migration to
> > travis-ci.com
> > (
> >
> https://blog.travis-ci.com/2018-05-02-open-source-projects-on-travis-ci-com-with-github-apps
> > )
> >   - On Nov 2, 2020: Travis CI announces new pricing model, effectively
> > ends generous open source offering.
> > (https://blog.travis-ci.com/2020-11-02-travis-ci-new-billing)
> >   - December 31, 2020: Travis-ci.org will be put into read-only mode.
> > (
> >
> https://docs.travis-ci.com/user/migrate/open-source-repository-migration/#frequently-asked-questions
> > )
> >
> >
> > Do we have a plan for migration, and if so, what is our plan?
> >
> > Is anything planned by the Apache Foundation?
> >
> > I would prefer a CI platform where a new user with a pull request does
> > not have to do a CI setup or a setup with only a few steps. Of course, a
> > new CI platform should be as transparent as possible.
> >
> > With kind regards
> > Philipp Dallig
> >
> >
>
> --
> Best Regards
>
> Jeff Zhang
>

Re: [Draft] Zeppelin Board report Oct 2020

[moon soo Lee]

Thank Jeff, Philipp, Felix for the quick review!

On Wed, Oct 14, 2020 at 10:49 PM Felix Cheung 
wrote:

> Looks good
>
> On Wed, Oct 14, 2020 at 9:01 PM Jeff Zhang  wrote:
>
>> +1, Thanks moon
>>
>> moon soo Lee  于2020年10月15日周四 上午11:58写道：
>>
>>> Hi Folks,
>>>
>>> Since the report due date is 14/Oct, I made a draft of it.
>>> Please review this.
>>>
>>> ## Description:
>>>
>>>  -  Apache Zeppelin is a collaborative data analytics and visualization
>>> tool
>>> for general-purpose data processing systems.
>>>
>>> ## Activity:
>>>
>>>  - 0.9.0-preview2 has released. Now we are planning the release of 0.9.0.
>>>
>>> ## Health report:
>>>
>>>  - +6 new code contributors since last report. 329 total.
>>>
>>> ## PMC changes:
>>>
>>>  - There are currently 22 committers and 11 PMC members in this project.
>>> The Committer-to-PMC ratio is 2:1.
>>>  - Last PMC addition was Jeff Zhang on Thu Jan 25 2018
>>>  - We invited a new PMC member, but wait for his response.
>>>
>>> ## Committer base changes:
>>>
>>>  - Currently 22 committers.
>>>  - Last committer addition was Philipp Dallig on 2020-06-24
>>>
>>> ## Releases:
>>>
>>>  - 0.9.0-preview2 was released on July 27, 2020
>>>  - 0.9.0-preview1 was released on March 26, 2020
>>>  - 0.8.2 was released on Wed Sep 29 2019
>>>  - 0.8.1 was released on Wed Jan 23 2019
>>>  - 0.8.0 was released on Wed Jul 18 2018
>>>
>>>
>>> ## Mailing list activity:
>>>
>>>  - us...@zeppelin.apache.org:
>>> - 92 emails sent to list ( 173 in previous quarter)
>>>
>>> - dev@zeppelin.apache.org:
>>> - 777 emails sent to list ( 774 in previous quarter)
>>>
>>> ## JIRA activity:
>>>
>>>  - 132 JIRA tickets created in the last 3 months
>>>  - 88 JIRA tickets are closed in last 3 months.
>>>
>>
>>
>> --
>> Best Regards
>>
>> Jeff Zhang
>>
>

[Draft] Zeppelin Board report Oct 2020

[moon soo Lee]

Hi Folks,

Since the report due date is 14/Oct, I made a draft of it.
Please review this.

## Description:

 -  Apache Zeppelin is a collaborative data analytics and visualization tool
for general-purpose data processing systems.

## Activity:

 - 0.9.0-preview2 has released. Now we are planning the release of 0.9.0.

## Health report:

 - +6 new code contributors since last report. 329 total.

## PMC changes:

 - There are currently 22 committers and 11 PMC members in this project.
The Committer-to-PMC ratio is 2:1.
 - Last PMC addition was Jeff Zhang on Thu Jan 25 2018
 - We invited a new PMC member, but wait for his response.

## Committer base changes:

 - Currently 22 committers.
 - Last committer addition was Philipp Dallig on 2020-06-24

## Releases:

 - 0.9.0-preview2 was released on July 27, 2020
 - 0.9.0-preview1 was released on March 26, 2020
 - 0.8.2 was released on Wed Sep 29 2019
 - 0.8.1 was released on Wed Jan 23 2019
 - 0.8.0 was released on Wed Jul 18 2018


## Mailing list activity:

 - us...@zeppelin.apache.org:
- 92 emails sent to list ( 173 in previous quarter)

- dev@zeppelin.apache.org:
- 777 emails sent to list ( 774 in previous quarter)

## JIRA activity:

 - 132 JIRA tickets created in the last 3 months
 - 88 JIRA tickets are closed in last 3 months.

Re: Supported interpreters

[moon soo Lee]

I think you can just set up a PR in this case. But opening a JIRA issue is
always a good idea :)


On Wed, Sep 30, 2020 at 5:38 AM Andrea Santurbano  wrote:

> Thank you so much Moon,
> do I need to open a JIRA issue for it?
> Or I can just set up the PR?
>
> Il giorno mar 29 set 2020 alle ore 19:10 moon soo Lee <
> leemoon...@gmail.com>
> ha scritto:
>
> > Hi,
> >
> > You can find it from 'gh-pages' branch.
> >
> https://github.com/apache/zeppelin/blob/gh-pages/supported_interpreters.md
> >
> > 'gh-pages' keeps the source code of the website.
> > Each version of Zeppelin documentation (which you can find source code
> > under /docs dir of master branch) is added to the website after the
> > release, under "Docs" menu.
> >
> >
> >
> > On Tue, Sep 29, 2020 at 4:41 AM Andrea Santurbano 
> > wrote:
> >
> > > Hi guys,
> > > the list here:
> > > https://zeppelin.apache.org/supported_interpreters.html
> > > Is missing the Neo4j interpreter, how can I add it?
> > > I looked into the github repo but I didn't find anything.
> > >
> >
>

Re: Supported interpreters

[moon soo Lee]

Hi,

You can find it from 'gh-pages' branch.
https://github.com/apache/zeppelin/blob/gh-pages/supported_interpreters.md

'gh-pages' keeps the source code of the website.
Each version of Zeppelin documentation (which you can find source code
under /docs dir of master branch) is added to the website after the
release, under "Docs" menu.



On Tue, Sep 29, 2020 at 4:41 AM Andrea Santurbano  wrote:

> Hi guys,
> the list here:
> https://zeppelin.apache.org/supported_interpreters.html
> Is missing the Neo4j interpreter, how can I add it?
> I looked into the github repo but I didn't find anything.
>

Re: [Proposal] Zeppelin Client API (Zeppelin SDK)

[moon soo Lee]

Looks really good!
Do you think to have a CLI on top of high-level API be also useful?


On Fri, Jul 31, 2020 at 7:13 AM Jeff Zhang  wrote:

> Thanks for your feedback, I have created ticket
> https://issues.apache.org/jira/browse/ZEPPELIN-4981
>
> Just like @Alex Ott   mentioned, this api may need
> several iterations, so I plan to make it as an experimental feature
> first, and then refine the api based on user feedback.
> .
>
> Alex Ott  于2020年7月31日周五 下午7:48写道：
>
>> Idea is very good - I think that we'll need to get several iterations of
>> API refinement, but current approach looks promising.
>>
>> On Fri, Jul 24, 2020 at 5:54 PM Jeff Zhang  wrote:
>>
>> > Hi Folks,
>> >
>> > I'd like to discuss this proposal with you about the zeppelin client api
>> > (zeppelin sdk).
>> > The background is that now Zeppelin’s main usage scenario is interactive
>> > data analysis. Although it provides rest api, it is not easy for an
>> > external system (e.g. scheduler system) to integrate Zeppelin for the
>> > scenario where zeppelin is used as a backend job service. So I propose
>> to
>> > introduce a new module: Zeppelin client api (Zeppelin SDK), whose
>> purpose
>> > is to provide easy api for external systems to integrate zeppelin.
>> >
>> > I have created a google doc for the details, welcome any comments and
>> > feedback.
>> >
>> >
>> >
>> https://docs.google.com/document/d/1bLLKKxleZlZpP9EFJlLLkJKwDBps-RNvzNwh3LFZWZ4/edit?usp=sharing
>> >
>> >
>> > --
>> > Best Regards
>> >
>> > Jeff Zhang
>> >
>>
>>
>> --
>> With best wishes,Alex Ott
>> http://alexott.net/
>> Twitter: alexott_en (English), alexott (Russian)
>>
>
>
> --
> Best Regards
>
> Jeff Zhang
>

Re: K8s docker images

[moon soo Lee]

Sounds like a plan!

On Wed, Jul 29, 2020 at 4:19 AM Alex Ott  wrote:

> Hi Philipp
>
> That would be nice to do finally :-)
> from my side I'm ok with having only the latest one.
> Thank you for moving it forward!
>
>
> On Wed, Jul 29, 2020 at 1:11 PM Philipp Dallig 
> wrote:
>
> > Hi Zeppelin developer,
> >
> > I would like to move forward the provision of docker images for
> Kubernetes.
> >
> > Right now we have three open PR:
> >
> >   - https://github.com/apache/zeppelin/pull/3380
> >   - https://github.com/apache/zeppelin/pull/3769
> >   - https://github.com/apache/zeppelin/pull/3859
> >
> > I would like to close PR 3380 and PR 3769 in favour for PR 3859. I also
> > want to close ZEPPELIN-4154 and create a new JIRA ticket for PR 3859.
> >
> > As written in PR 3859 I agree with Alex Ott that multiple interpreter
> > images are difficult to handle.
> >
> > Can I move forward in this matter, what are your thoughts?
> >
> > Best Regards
> >
> > Philipp Dallig
> >
> >
> >
>
> --
> With best wishes,Alex Ott
> http://alexott.net/
> Twitter: alexott_en (English), alexott (Russian)
>

Re: [VOTE] Release Apache Zeppelin 0.9.0-preview2 (RC2)

[moon soo Lee]

+1

I tested

 - build from source
 - src package license file
 - bin package license file
 - new ui access

On Fri, Jul 24, 2020 at 12:36 AM Alex Ott  wrote:

> +1 from me. What's done:
>
> - checked the checksum
> - run spark samples
> - tested cassandra interpreter
> - installed some plugins from helium registry
>
>
> On Thu, Jul 23, 2020 at 5:04 PM Jeff Zhang  wrote:
>
>>
>> Hi folks,
>>
>> I propose the following RC to be released for the Apache Zeppelin 
>> 0.9.0-preview2 release.
>>
>>
>> The commit id is 31b9ed51f946fed934885d8fbb29e9c183552e70 : 
>> https://gitbox.apache.org/repos/asf?p=zeppelin.git;a=commit;h=31b9ed51f946fed934885d8fbb29e9c183552e70
>>
>> This corresponds to the tag: v0.9.0-preview2-rc2 : 
>> https://gitbox.apache.org/repos/asf?p=zeppelin.git;a=shortlog;h=refs/tags/v0.9.0-preview2-rc2
>>
>> The release archives (tgz), signature, and checksums are here 
>> https://dist.apache.org/repos/dist/dev/zeppelin/zeppelin-0.9.0-preview2-rc2/
>>
>> The release candidate consists of the following source distribution archive
>> zeppelin-v0.9.0-preview2.tgz
>>
>> In addition, the following supplementary binary distributions are provided
>> for user convenience at the same location
>> zeppelin-0.9.0-preview2-bin-all.tgz
>>
>>
>> The maven artifacts are here 
>> https://repository.apache.org/content/repositories/orgapachezeppelin-1283/org/apache/zeppelin/
>>
>> You can find the KEYS file here: 
>> https://dist.apache.org/repos/dist/release/zeppelin/KEYS
>>
>> Release notes available at 
>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12342692==12316221
>>
>> Vote will be open for next 72 hours (close at 8am 26/July PDT).
>>
>> [ ] +1 approve
>> [ ] 0 no opinion
>> [ ] -1 disapprove (and reason why)
>>
>>
>> --
>> Best Regards
>>
>> Jeff Zhang
>>
>
>
> --
> With best wishes,Alex Ott
> http://alexott.net/
> Twitter: alexott_en (English), alexott (Russian)
>

Re: [Draft] Zeppelin Board Report

[moon soo Lee]

Thank Jeff for writing a draft.
It looks good to me :)

On Mon, Jul 6, 2020 at 7:53 AM Jeff Zhang  wrote:

>
> Hi Folks,
>
> Here's the draft for this month zeppelin board report, let me know what
> you think.
>
>
> ## Description:
>
>  -  Apache Zeppelin is a collaborative data analytics and visualization
> tool
>
> for general-purpose data processing systems.
>
> ## Activity:
>
>  - 0.9.0-preview1 got many feedback from community and we make a lot of
> imporvements. Now We are planning the release of 0.9.0-preview2.
>
>  - A series of flink on zeppelin videos are published on Zeppelin's
> youtube channel.
>
>  -  2 security issues are still unresolved due to no enough
> knowledge, we are working with security team to resolve them asap.
> of security.
> zeppelin: Notebook permission bypass vulnerability
> 2018-08-20
> zeppelin: Apache Zeppelin LDAP Injection
> CVE-2019-10096.
>
>
>
> ## Health report:
>
>  - +16 new code contributors since last report. 323 total.
>
>
> ## PMC changes:
>
>  - There are currently 22 committers and 11 PMC members in this project.
> The Committer-to-PMC ratio is 2:1.
>
>  - Last PMC addition was Jeff Zhang on Thu Jan 25 2018
>
>  - We invited a new PMC member, but wait for his response.
>
> ## Committer base changes:
>
>  - Currently 22 committers.
>
>  - Last committer addition was Philipp Dallig on 2020-06-24
>
> ## Releases:
>
>  - 0.9.0-preview1 was released on March 26, 2020
>
>  - 0.8.2 was released on Wed Sep 29 2019
>
>  - 0.8.1 was released on Wed Jan 23 2019
>
>  - 0.8.0 was released on Wed Jul 18 2018
>
>
> ## Mailing list activity:
>
>  - us...@zeppelin.apache.org:
>
> - 108 emails sent to list ( 122 in previous quarter)
>
> - dev@zeppelin.apache.org:
>
> - 863 emails sent to list ( 934 in previous quarter)
>
> ## JIRA activity:
>
>  - 228 JIRA tickets created in the last 3 months
>  - 136 JIRA tickets are closed in last 3 months.
>
>
> --
> Best Regards
>
> Jeff Zhang
>

Re: [DISCUSS] Release 0.9.0 preview2

[moon soo Lee]

I don't see a particular blocker and looks like it's time to have preview2!
Shell we create an umbrella issue in Jira for 0.9.0 preview2 release
anyway, so people can track the progress of the release?

Thanks,
moon


On Tue, Jun 30, 2020 at 7:49 PM Jeff Zhang  wrote:

> Hi Folks,
>
> We discuss about releasing 0.9.0 preview2 several weeks ago, I think it is
> time to do that, let me know if you have any blocker issues that you would
> like to include in 0.9.0 preview2.
>
> --
> Best Regards
>
> Jeff Zhang
>

Re: [Announce] New Zeppelin Committer: Philipp Dallig

[moon soo Lee]

It's exciting news because I was always impressed by high-quality code
contributions from Philipp Dalling.
Congratulations! Welcome aboard!

On Thu, Jun 25, 2020 at 11:23 PM Jeff Zhang  wrote:

>
> The Project Management Committee (PMC) for Apache Zeppelin
> has invited Philipp Dallig to become a committer and we are very pleased
> to announce that he has accepted.
>
> We greatly appreciate all of Philipp Dallig's hard work and generous
> contributions to the project. We look forward to continued involvement in
> the project.
>
> Congratulations & Welcome aboard Philipp Dallig !
>
>
> --
> Best Regards
>
> Jeff Zhang
>

Re: Update Zeppelin Interpreter only leveragin Helium

[moon soo Lee]

What Helium does for the Interpreter is just downloading binaries from
maven repository.
*bin/install-interpreter.sh* is the CLI, which runs InstallInterpreter

class.

So, if multiple versions of interpreters are published to a maven
repository (with different neo4j driver versions),
Helium can help users to download different versions and install.

The interpreter maven artifacts are predefined in conf/interpreter-list
,
however, *bin/install-interpreter.sh* can download any artifact using '-t'
flag.

Thanks,
moon


On Sun, Jun 21, 2020 at 7:51 AM Jeff Zhang  wrote:

> I don't know much about helium, so I can not help on that.
> But if the api is incompatible, you can also consider using reflection, you
> can refer to the spark/spark-shims module.
>
>
> Andrea Santurbano  于2020年6月21日周日 下午10:11写道：
>
> > Unfortunatelly no because there are different classes so we cannot use
> the
> > same approach.
> > Do you think it's something feasible use helium for that?
> >
> > Il giorno dom 21 giu 2020 alle 09:54 Jeff Zhang  ha
> > scritto:
> >
> > > If the api is compatible between different versions of Neo4j driver,
> then
> > > you can just don't ship Neo4j driver into zeppelin, let users provide
> > that.
> > > e.g. currently spark interpreter supports multiple versions of spark,
> > > Zeppelin doesn't' ship with any spark jars, it is user's responsibility
> > to
> > > provide that via specifying SPARK_HOME
> > >
> > >
> > >
> > > Andrea Santurbano  于2020年6月21日周日 下午3:35写道：
> > >
> > > > Hi guys,
> > > > If I want to update a zeppelin interpreter only there is a way to do
> > it?
> > > > Let me explain the use case, I want to use Zeppelin 0.8.2 with Neo4j
> > 4.0
> > > > and I need to update the driver version because the version bundled
> > with
> > > > the Neo4j interpreter is not compatible with Neo4j 4.0. Can I
> leverage
> > > > Helium for this? In case so how can I do it?
> > > > I have to do a PR for updating the Neo4j Interpreter in the master
> > branch
> > > > but I also want to provide support to the old version and I don't
> want
> > to
> > > > wait for a new release :D
> > > > Thanks a lot
> > > > Andrea
> > > >
> > >
> > >
> > > --
> > > Best Regards
> > >
> > > Jeff Zhang
> > >
> >
>
>
> --
> Best Regards
>
> Jeff Zhang
>

Re: Apache Zeppelin in The State of Developer Ecosystem survey 2020

[moon soo Lee]

Curious if the result in [2] was chosen from a list provided with the
question, or people typed their tool without a given list.
Anyway, a very interesting survey!
Thank Alex for sharing this.

On Thu, Jun 18, 2020 at 8:09 PM Jeff Zhang  wrote:

> Thanks for the sharing, Alex
>
> Alex  于2020年6月19日周五 上午2:41写道：
>
> > Hey,
> >
> > I just noticed that the recent Developer Ecosystem survey 2020 [1]
> includes
> > interesting data on Apache Zeppelin adoption in the "Big Data analytics
> > platform" section [2].
> >
> >  1. https://www.jetbrains.com/lp/devecosystem-2020/
> >  2. https://imgur.com/SJbMSmE
> >
> >
> > --
> > Alex
> >
>
>
> --
> Best Regards
>
> Jeff Zhang
>

Re: [ANNOUNCE] Welcome Yadong Xie, Hsuan Lee as New Committer

[moon soo Lee]

Excited to see great work with new UI. Welcome Yadong Xie, Hsuan Lee.

On Wed, Apr 29, 2020 at 7:03 AM Jeff Zhang  wrote:

> The Project Management Committee (PMC) for Apache Zeppelin
> has invited Yadong Xie and Hsuan Lee  to become a committer and we are very
> pleased
> to announce that they have accepted.
>
> We greatly appreciate all of Yadong Xie and Hsuan Lee's hard work and
> generous contributions to the project, especially on the new angular UI
> implementation. We look forward to continued involvement in the project.
>
> Congratulations & Welcome aboard Yadong Xie and Hsuan Lee !
>
> --
> Best Regards
>
> Jeff Zhang
>

Re: Logging doesn't appear to work with Docker version of 0.9.0-preview1

[moon soo Lee]

When I build my own Docker image based on master branch, I experience the
same problem. But in my case /logs is not empty.

On Tue, Mar 31, 2020 at 1:50 PM Eric Pugh 
wrote:

> I’ve tried with building my own Docker image of Zeppelin, and using the
> apache/zeppelin:0.9.0 image:
>
> docker run -p 8080:8080 --rm -v $PWD/logs:/logs -v $PWD/notebook:/notebook
> -e ZEPPELIN_LOG_DIR='/logs' -e ZEPPELIN_NOTEBOOK_DIR='/notebook' --name
> zeppelin apache/zeppelin:0.9.0
>
>
> The /logs directory remains stubbornly empty!   I wanted to confirm with
> others that they see the same issue before opening a JIRA.
>
> Thanks
>
>
> ___
> Eric Pugh | Founder & CEO | OpenSource Connections, LLC | 434.466.1467 |
> http://www.opensourceconnections.com <
> http://www.opensourceconnections.com/> | My Free/Busy <
> http://tinyurl.com/eric-cal>
> Co-Author: Apache Solr Enterprise Search Server, 3rd Ed <
> https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw>
>
> This e-mail and all contents, including attachments, is considered to be
> Company Confidential unless explicitly stated otherwise, regardless of
> whether attachments are marked as such.
>
>

Re: Zeppelin Draft board report for April 2020

[moon soo Lee]

119 closed in the last 3 months

On Mon, Apr 6, 2020 at 10:46 AM Alex Ott  wrote:

> Looks great for me - do we have statistics on closed JIRA tickets?
>
> moon soo Lee  at "Mon, 6 Apr 2020 09:21:26 -0700" wrote:
>  msL> Thank Jeff for the draft.
>  msL> Looks great to me.
>
>  msL> On Mon, Apr 6, 2020 at 9:08 AM Jeff Zhang  wrote:
>
>  >> Hi Folks,
>  >>
>  >>
>  >> Here's draft of Zeppelin Report, let me know if you have concerns.
> Thanks
>  >>
>  >>
>  >>
>  >>
>  >> ## Description:
>  >>
>  >>  -  Apache Zeppelin is a collaborative data analytics and visualization
>  >> tool
>  >>
>  >> for general-purpose data processing systems.
>  >>
>  >>
>  >>
>  >> ## Activity:
>  >>
>  >>  - Zeppelin 0.9.0-preview1 is released in March, this is first time we
> have
>  >> preview major release. Because based on our experience, it is always
> not
>  >> possible do enough test and verification in 3 day's voting period
> because
>  >> of the complexity of zeppelin. So we would like to make preview
> release for
>  >> each major release.
>  >>
>  >>
>  >>
>  >>
>  >> ## Health report:
>  >>
>  >>  - +12 new code contributors since last report. 307 total
>  >>
>  >> ## PMC changes:
>  >>
>  >>  - Currently 11 PMC members.
>  >>
>  >>  - Last PMC addition was Jeff Zhang on Thu Jan 25 2018
>  >>
>  >> ## Committer base changes:
>  >>
>  >>  - Currently 19 committers.
>  >>
>  >>  - Last committer addition was Alex Ott at Jan 1 2020
>  >>
>  >> ## Releases:
>  >>
>  >> - 0.9.0-preview1 was released on March 26, 2020
>  >>
>  >>  - 0.8.2 was released on Wed Sep 29 2019
>  >>
>  >>  - 0.8.1 was released on Wed Jan 23 2019
>  >>
>  >>  - 0.8.0 was released on Wed Jul 18 2018
>  >>
>  >>  - 0.7.3 was released on Wed Sep 20 2017
>  >>
>  >>  - 0.7.2 was released on Mon Jun 12 2017
>  >>
>  >>  - 0.7.1 was released on Fri Mar 31 2017
>  >>
>  >>  - 0.7.0 was released on Sun Feb 05 2017
>  >>
>  >>  - 0.6.2 was released on Fri Oct 14 2016
>  >>
>  >>  - 0.6.1 was released on Aug 15 2016
>  >>
>  >>  - 0.6.0 was released on Jul 02 2016
>  >>
>  >>  - 0.5.6-incubating was released on Jan 22 2016
>  >>
>  >>  - 0.5.5-incubating was released on Nov 18 2015
>  >>
>  >>  - 0.5.0-incubating was released on Jul 23 2015
>  >>
>  >> ## Mailing list activity:
>  >>
>  >>  - us...@zeppelin.apache.org:
>  >>
>  >> -  122 emails sent to list ( 151 in previous quarter)
>  >>
>  >> - dev@zeppelin.apache.org:
>  >>
>  >> - 934 emails sent to list ( 923 in previous quarter)
>  >>
>  >> ## JIRA activity:
>  >>
>  >>  - 175 JIRA tickets created in the last 3 months
>  >>
>  >>
>  >>
>  >> --
>  >> Best Regards
>  >>
>  >> Jeff Zhang
>  >>
>
>
> --
> With best wishes,Alex Ott
> http://alexott.net/
> Twitter: alexott_en (English), alexott (Russian)
>

Re: Zeppelin Draft board report for April 2020

[moon soo Lee]

Thank Jeff for the draft.
Looks great to me.

On Mon, Apr 6, 2020 at 9:08 AM Jeff Zhang  wrote:

> Hi Folks,
>
>
> Here's draft of Zeppelin Report, let me know if you have concerns. Thanks
>
>
>
>
> ## Description:
>
>  -  Apache Zeppelin is a collaborative data analytics and visualization
> tool
>
> for general-purpose data processing systems.
>
>
>
> ## Activity:
>
>  - Zeppelin 0.9.0-preview1 is released in March, this is first time we have
> preview major release. Because based on our experience, it is always not
> possible do enough test and verification in 3 day's voting period because
> of the complexity of zeppelin. So we would like to make preview release for
> each major release.
>
>
>
>
> ## Health report:
>
>  - +12 new code contributors since last report. 307 total
>
> ## PMC changes:
>
>  - Currently 11 PMC members.
>
>  - Last PMC addition was Jeff Zhang on Thu Jan 25 2018
>
> ## Committer base changes:
>
>  - Currently 19 committers.
>
>  - Last committer addition was Alex Ott at Jan 1 2020
>
> ## Releases:
>
> - 0.9.0-preview1 was released on March 26, 2020
>
>  - 0.8.2 was released on Wed Sep 29 2019
>
>  - 0.8.1 was released on Wed Jan 23 2019
>
>  - 0.8.0 was released on Wed Jul 18 2018
>
>  - 0.7.3 was released on Wed Sep 20 2017
>
>  - 0.7.2 was released on Mon Jun 12 2017
>
>  - 0.7.1 was released on Fri Mar 31 2017
>
>  - 0.7.0 was released on Sun Feb 05 2017
>
>  - 0.6.2 was released on Fri Oct 14 2016
>
>  - 0.6.1 was released on Aug 15 2016
>
>  - 0.6.0 was released on Jul 02 2016
>
>  - 0.5.6-incubating was released on Jan 22 2016
>
>  - 0.5.5-incubating was released on Nov 18 2015
>
>  - 0.5.0-incubating was released on Jul 23 2015
>
> ## Mailing list activity:
>
>  - us...@zeppelin.apache.org:
>
> -  122 emails sent to list ( 151 in previous quarter)
>
> - dev@zeppelin.apache.org:
>
> - 934 emails sent to list ( 923 in previous quarter)
>
> ## JIRA activity:
>
>  - 175 JIRA tickets created in the last 3 months
>
>
>
> --
> Best Regards
>
> Jeff Zhang
>

Re: [VOTE] Release Apache Zeppelin 0.9.0-preview1 (RC1)

[moon soo Lee]

+1

Tested
 - The binary package can run tutorial notebook out of the box
 - Can build docker image
- scripts/docker/zeppelin/bin/Dockerfile give me some error on R
dependency installation. But I think it's not a blocker. We can address it
on next RC.
 - Zeppelin on Kubernetes
- Everything but Spark-2.4.5 does not run out of the box. I think it is
not a blocker. We can address it on next RC.




On Wed, Mar 25, 2020 at 11:21 AM Prabhjyot Singh 
wrote:

> +1
>
> On Wed, 25 Mar 2020 at 21:12, Eric Pugh 
> wrote:
>
> > Out of curiosity, is there a Docker version of 0.9.0 preview1 available?
> > I tend to run Zeppelin as a Docker image, and I know I had some
> challenges
> > with the 0.8.x line of Docker images from
> > https://hub.docker.com/r/apache/zeppelin.
> >
> > I did modify the ./scripts/docker/zeppelin/Dockerfile to load the
> > v0.0.0-preview1-rc1.bin.tgz distribution.
> >
> > Then I followed the steps in docs/setup/deployment/docker.md to try out
> > Zeppelin and it all appears to work as a Docker image!
> >
> > One thing to note was that it didn’t auto upgrade my old .json files into
> > the new .zpln files.  I had to manually upload them.  Is there any auto
> > upgrade feature for existing notebook files?
> >
> > So non binding +1 ;-)
> >
> > Eric
> >
> >
> >
> > On Mar 25, 2020, at 1:54 AM, Jongyoul Lee  wrote:
> >
> > +1
> >
> > 2020년 3월 24일 (화) 오전 12:15, Jeff Zhang 님이 작성:
> >
> >> Hi folks,
> >>
> >> I propose the following RC to be released for the Apache Zeppelin
> >> 0.9.0-preview1 release.
> >>
> >>
> >> The commit id is a74365c0813b451db1bc78def7d1ad1279429224 :
> >>
> >>
> https://gitbox.apache.org/repos/asf?p=zeppelin.git;a=commit;h=a74365c0813b451db1bc78def7d1ad1279429224
> >>
> >> This corresponds to the tag: v0.9.0-preview1-rc1 :
> >>
> >>
> https://gitbox.apache.org/repos/asf?p=zeppelin.git;a=shortlog;h=refs/tags/v0.9.0-preview1-rc1
> >>
> >> The release archives (tgz), signature, and checksums are here
> >>
> >>
> https://dist.apache.org/repos/dist/dev/zeppelin/zeppelin-0.9.0-preview1-rc1/
> >>
> >> The release candidate consists of the following source distribution
> >> archive
> >> zeppelin-v0.9.0-preview1.tgz
> >>
> >> In addition, the following supplementary binary distributions are
> provided
> >> for user convenience at the same location
> >> zeppelin-0.9.0-preview1-bin-all.tgz
> >>
> >>
> >> The maven artifacts are here
> >>
> >>
> https://repository.apache.org/content/repositories/orgapachezeppelin-1276/org/apache/zeppelin/
> >>
> >> You can find the KEYS file here:
> >> https://dist.apache.org/repos/dist/release/zeppelin/KEYS
> >>
> >> Release notes available at
> >>
> >>
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12342692==12316221
> >>
> >> Vote will be open for next 72 hours (close at 7am 26/March PDT).
> >>
> >> [ ] +1 approve
> >> [ ] 0 no opinion
> >> [ ] -1 disapprove (and reason why)
> >>
> >>
> >> --
> >> Best Regards
> >>
> >> Jeff Zhang
> >>
> >
> >
> > --
> > 이종열, Jongyoul Lee, 李宗烈
> > http://madeng.net
> >
> >
> > ___
> > *Eric Pugh **| *Founder & CEO | OpenSource Connections, LLC |
> 434.466.1467
> > | http://www.opensourceconnections.com | My Free/Busy
> > 
> > Co-Author: Apache Solr Enterprise Search Server, 3rd Ed
> > <
> https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw
> >
> > This e-mail and all contents, including attachments, is considered to be
> > Company Confidential unless explicitly stated otherwise, regardless
> > of whether attachments are marked as such.
> >
> >
>
> --
>
> Warm Regards,
>
> Prabhjyot Singh
>

Re: [DISCUSS] Release for 0.9.0-preview-1

[moon soo Lee]

Definitely +1

On Mon, Mar 16, 2020 at 8:02 AM Prabhjyot Singh 
wrote:

> Yes, +1, we have put in a lot of features. We should do it soon.
>
> On Mon, Mar 16, 2020, 8:28 PM Yadong Xie  wrote:
>
>> Big +1
>>
>> Eric Pugh  于2020年3月16日周一 下午10:54写道：
>>
>> > Big +1 from me.
>> >
>> >
>> > > On Mar 16, 2020, at 10:45 AM, Jeff Zhang  wrote:
>> > >
>> > > Hi Folks,
>> > >
>> > > I'd like to start the discussion of releasing of 0.9.0-preview-1. As
>> we
>> > discussed before releasing a preview version first is suitable for a
>> major
>> > release based on our previous experience[1]. The last major release
>> 0.8.0
>> > is in 2018, I do believe we should release 0.9 to allow more people try
>> > that and gather feedback. What do you think ?
>> > >
>> > >
>> > > [1]
>> >
>> https://lists.apache.org/thread.html/3812fd7dab53e1fed4140c826fc69fa5b718636371b6c7c677b4ae88%40%3Cdev.zeppelin.apache.org%3E
>> > <
>> >
>> https://lists.apache.org/thread.html/3812fd7dab53e1fed4140c826fc69fa5b718636371b6c7c677b4ae88%40%3Cdev.zeppelin.apache.org%3E
>> > >
>> > >
>> > > --
>> > > Best Regards
>> > >
>> > > Jeff Zhang
>> >
>> > ___
>> > Eric Pugh | Founder & CEO | OpenSource Connections, LLC | 434.466.1467 |
>> > http://www.opensourceconnections.com <
>> > http://www.opensourceconnections.com/> | My Free/Busy <
>> > http://tinyurl.com/eric-cal>
>> > Co-Author: Apache Solr Enterprise Search Server, 3rd Ed <
>> >
>> https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw
>> >
>> >
>> > This e-mail and all contents, including attachments, is considered to be
>> > Company Confidential unless explicitly stated otherwise, regardless of
>> > whether attachments are marked as such.
>> >
>> >
>>
>

Re: Autolink references in Github repo?

[moon soo Lee]

Cool! Thanks Alex!

On Tue, Mar 10, 2020 at 12:07 PM Alex Ott  wrote:

> the auto-linking was configured, and works now - you can see it at comment
> to https://github.com/apache/zeppelin/pull/3634 - unfortunately it works
> only for newly created/edited PRs
>
> On Sat, Mar 7, 2020 at 12:12 AM Jeff Zhang  wrote:
>
> > Thanks Alex
> >
> > Alex Ott  于2020年3月7日周六 上午3:01写道：
> >
> >> I've created https://issues.apache.org/jira/browse/INFRA-19934 for that
> >>
> >> On Mon, Mar 2, 2020 at 6:31 AM Alex Ott  wrote:
> >>
> >>> Ok. Thank you - I'll file Infra ticket
> >>>
> >>> On Mon, 2 Mar 2020, 03:37 moon soo Lee,  wrote:
> >>>
> >>>> Hi,
> >>>>
> >>>> I don't have admin rights to GitHub repo but ASF Infra does.
> >>>> Usually, configuration change on Github repo can be done by filing
> >>>> issues
> >>>> to ASF infra https://issues.apache.org/jira/projects/INFRA.
> >>>>
> >>>> Let me know if anything else I can help here.
> >>>>
> >>>> Thanks!
> >>>> moon
> >>>>
> >>>>
> >>>>
> >>>> On Sun, Mar 1, 2020 at 6:19 PM Jeff Zhang  wrote:
> >>>>
> >>>> > + Moon
> >>>> >
> >>>> > Moon should have the admin right
> >>>> >
> >>>> > Alex Ott  于2020年3月2日周一 上午1:39写道：
> >>>> >
> >>>> >> Hi all
> >>>> >>
> >>>> >> one question - who owns the Zeppelin repository at Github, or have
> >>>> >> administrative rights? It would be nice to configure Autolink
> >>>> references
> >>>> >> for Zeppelin repo to automatically create links to Jira for PRs,
> >>>> etc. I
> >>>> >> have very good experience of using them for my private projects...
> >>>> >>
> >>>> >> --
> >>>> >> With best wishes,Alex Ott
> >>>> >> http://alexott.net/
> >>>> >> Twitter: alexott_en (English), alexott (Russian)
> >>>> >>
> >>>> >
> >>>> >
> >>>> > --
> >>>> > Best Regards
> >>>> >
> >>>> > Jeff Zhang
> >>>> >
> >>>>
> >>>
> >>
> >> --
> >> With best wishes,Alex Ott
> >> http://alexott.net/
> >> Twitter: alexott_en (English), alexott (Russian)
> >>
> >
> >
> > --
> > Best Regards
> >
> > Jeff Zhang
> >
>
>
> --
> With best wishes,Alex Ott
> http://alexott.net/
> Twitter: alexott_en (English), alexott (Russian)
>

Re: Autolink references in Github repo?

[moon soo Lee]

Hi,

I don't have admin rights to GitHub repo but ASF Infra does.
Usually, configuration change on Github repo can be done by filing issues
to ASF infra https://issues.apache.org/jira/projects/INFRA.

Let me know if anything else I can help here.

Thanks!
moon



On Sun, Mar 1, 2020 at 6:19 PM Jeff Zhang  wrote:

> + Moon
>
> Moon should have the admin right
>
> Alex Ott  于2020年3月2日周一 上午1:39写道：
>
>> Hi all
>>
>> one question - who owns the Zeppelin repository at Github, or have
>> administrative rights? It would be nice to configure Autolink references
>> for Zeppelin repo to automatically create links to Jira for PRs, etc. I
>> have very good experience of using them for my private projects...
>>
>> --
>> With best wishes,Alex Ott
>> http://alexott.net/
>> Twitter: alexott_en (English), alexott (Russian)
>>
>
>
> --
> Best Regards
>
> Jeff Zhang
>

Re: [ANNOUNCE] Welcome Alex Ott as New Committer

[moon soo Lee]

little but late for the welcoming party here.
I couldn’t appreciate more about starting 2020 with this great news!
Welcome Alex Ott and appreciate your involvement and contributions!

Regards,
moon


On Sat, Jan 11, 2020 at 02:21 Alex Ott  wrote:

> Thank
>
> Thank you for introduction Jeff, and thank you all for greetings!
>
> I'm looking forward to make more contribution to this very interesting
> project!
>
> Prabhjyot Singh  at "Sat, 11 Jan 2020 14:17:57 +0530" wrote:
>  PS> Hello Alex,
>
>  PS> Hearty Greetings and a very warm welcome to you.
>
>  PS> On Sat, Jan 11, 2020, 2:16 PM Xun Liu  wrote:
>
>  PS> Hi, Alex Ott.
>  PS>
>  PS> Congratulations on becoming a committer of Apache Zeppelin !!
>  PS> :-)
>  PS>
>  PS> On Sat, Jan 11, 2020 at 4:38 PM Jongyoul Lee 
> wrote:
>  PS>
>  PS> That's really great!!
>  PS>
>  PS> I appreciate his activities against Apache Zeppelin and I'm
> really happy as
>  PS> I have already known how difficult to contribute an
> opensource continuously.
>  PS>
>  PS> Please keep helping make Apache Zeppelin healthy.
>  PS>
>  PS> Regards,
>  PS> JL
>  PS>
>  PS> 2020년 1월 7일 (화) 오후 6:48, Jeff Zhang 님이 작성:
>  PS>
>  >> The Project Management Committee (PMC) for Apache Zeppelin
>  >> has invited Alex Ott to become a committer and we are very pleased
>  >> to announce that he has accepted.
>  >>
>  >> We greatly appreciate all of Alex's hard work and generous
> contributions to
>  >> the project. We look forward to continued involvement in the project.
>  >>
>  >> Congratulations & Welcome aboard Alex !
>  >>
>  >>
>  >> --
>  >> Best Regards
>  >>
>  >> Jeff Zhang
>  >>
>
>  PS> --
>  PS> 이종열, Jongyoul Lee, 李宗烈
>  PS> http://madeng.net
>
>
>
> --
> With best wishes,Alex Ott
> http://alexott.net/
> Twitter: alexott_en (English), alexott (Russian)
>

Re: [DISCUSS] Create a preview for Zeppelin 0.9.0 before making RC

[moon soo Lee]

Right, 3 days for each RC for a major release is too short.
+1 for creating a preview and docker image as well :)

On Mon, Dec 16, 2019 at 7:40 AM Andrea Santurbano  wrote:

> Yes please!
> Would be awesome have a Docker container in order test it :)
>
> Il giorno lun 16 dic 2019 alle ore 16:24 Jeff Zhang  ha
> scritto:
>
> > Hi Folks,
> >
> > We have already done lots of works on 0.9.0, I think we should consider
> > about make a release for 0.9.0. But since Zeppelin is a very complicated
> > project considering its rich features, the regular process of testing it
> > for 3 days for each RC is too short in my opinion, so I suggest to
> create a
> > preview for each major release. Then the community has more time to test
> it
> > and we have more time to fix bugs and ensure the release quality. What do
> > you think ?
> >
> >
> > --
> > Best Regards
> >
> > Jeff Zhang
> >
>

Recognizing contributors

[moon soo Lee]

Hi,

I'd like to have a discussion about recognizing contributors.
Many people contribute Apache Zeppelin in many different ways,

One way via code contribution, another way via non-code contributions like
creating videos, writing articles, UI design, providing talks, and so on.

I think code contributors are recognized in release notes [1]. Even GitHub
provides a contributor page [2] that anyone has access to. However,
non-code contributors are not well recognized today, IMO.

I believe, recognizing/appreciating all types of contributions really help
make the community healthy.

How about have a contributor list in 'Contributors' page [3] on website and
let contributors add themselves? Maybe it can be looks like the 'Current
Committers' page [4] on Spark project, but for contributors. Or we can also
try list contributors in release note [1].

Or do you have any better ideas and opinions? Please share your thoughts
about the right way of recognizing contributors.

Thanks,
moon

[1]
http://zeppelin.apache.org/releases/zeppelin-release-0.8.2.html#contributors
[2] https://github.com/apache/zeppelin/graphs/contributors
[3] http://zeppelin.apache.org/community.html
[4] https://spark.apache.org/committers.html

Re: Automatic update of the Helium registry doesn't work?

[moon soo Lee]

cc ahyoung. Could you take a look?? And, any good idea how we can we move
helium.json and related code zeppelin project source tree?


On Sun, Nov 3, 2019 at 06:01 Jeff Zhang  wrote:

> + Moon
>
> I think helium.json on S3 is maintained by ZEPL, Moon should know the
> details.
>
>
> Alex Ott  于2019年11月3日周日 下午6:52写道：
>
> > Hi all
> >
> > I've noticed that Helium registry (
> > https://s3.amazonaws.com/helium-package/helium.json) is still listing
> > zeppelin-leaflet 1.0.4 as the latest, although 1.0.5 was released 5
> months
> > ago (https://www.npmjs.com/package/zeppelin-leaflet) with some
> > improvements
> > regarding omitting the points that have null value for one of the
> > coordinates.
> >
> > As I understood from documentation, registry should be regenerated
> > automatically on publish of the new versions, but this doesn't happen.
> >
> > How we can enforce it?
> >
> > --
> > With best wishes,Alex Ott
> > http://alexott.net/
> > Twitter: alexott_en (English), alexott (Russian)
> >
>
>
> --
> Best Regards
>
> Jeff Zhang
>

Re: Why not all Pull-Request runs Jenkins?

[moon soo Lee]

Just filed an infra issue https://issues.apache.org/jira/browse/INFRA-19249

Thanks,
moon



On Wed, Oct 9, 2019 at 8:25 PM Jeff Zhang  wrote:

> I also notice this issue, My PR will run jenkins job, but lots of
> contributors' PR don't trigger jenkins job.
>
>
>
> moon soo Lee  于2019年10月10日周四 上午8:32写道：
>
> > Hi,
> >
> > I can see some Pull-Request does not run Jenkins, like below
> >
> > https://github.com/apache/zeppelin/pull/3480
> > [image: image.png]
> >
> > And some does run Jenkins, like below
> >
> > https://github.com/apache/zeppelin/pull/3479
> >
> > [image: image.png]
> >
> > It looks like Jenkins is triggered only when github apache organization
> > member creates pull-request. Is it true?
> >
> > Does anyone have idea why Jenkins is not running for all pullrequests?
> >
> > Thanks,
> > moon
> >
>
>
> --
> Best Regards
>
> Jeff Zhang
>

Why not all Pull-Request runs Jenkins?

[moon soo Lee]

Hi,

I can see some Pull-Request does not run Jenkins, like below

https://github.com/apache/zeppelin/pull/3480
[image: image.png]

And some does run Jenkins, like below

https://github.com/apache/zeppelin/pull/3479

[image: image.png]

It looks like Jenkins is triggered only when github apache organization
member creates pull-request. Is it true?

Does anyone have idea why Jenkins is not running for all pullrequests?

Thanks,
moon

Re: [ANNOUNCE] Apache Zeppelin 0.8.2 released

[moon soo Lee]

Great work!

On Sun, Sep 29, 2019 at 9:46 PM Prabhjyot Singh 
wrote:

> Awesome!
>
> Thank you so much Jeff for taking care of this.
>
> Congratulations community.
>
> On Sun, 29 Sep 2019, 2:15 pm Jeff Zhang,  wrote:
>
>> The Apache Zeppelin community is pleased to announce the availability of
>> the 0.8.2 release.
>>
>> Zeppelin is a collaborative data analytics and visualization tool for
>> distributed, general-purpose data processing system such as Apache Spark,
>> Apache Flink, etc.
>>
>> This is another minor release after the last minor release 0.8.1
>> The community put significant effort into improving Apache Zeppelin since 
>> the last release. 26 patches for improvements and bug fixes.
>>
>> We encourage you to download the latest release 
>> fromhttp://zeppelin.apache.org/download.html
>> Release note is available 
>> athttp://zeppelin.apache.org/releases/zeppelin-release-0.8.2.html
>>
>> We welcome your help and feedback. For more information on the project and
>> how to get involved, visit our website at http://zeppelin.apache.org/
>>
>> Thank you all users and contributors who have helped to improve Apache
>> Zeppelin.
>>
>> Regards,
>> The Apache Zeppelin community
>>
>>

Re: [DISCUSS] Release 0.8.2

[moon soo Lee]

+1

On Mon, Sep 23, 2019 at 8:16 PM Prabhjyot Singh 
wrote:

> +1
>
> I see a lot of issues are fixed in the past few months. This sounds a
> good intermediate step before 0.9 release.
>
> On Tue, 24 Sep 2019 at 00:23, Eric Pugh 
> wrote:
>
>> The iPython related fixes would address some issues I’ve run into!
>>
>> On Sep 22, 2019, at 11:31 PM, Jeff Zhang  wrote:
>>
>> Hi Folks,
>>
>> 0.8.1 is released half year ago (Jan 2019), I'd to start the discussion
>> about releasing of 0.8.2 which fixed several critical issues. Let me know
>> your thoughts, thanks.
>>
>>
>> https://jira.apache.org/jira/issues/?jql=project%20%3D%20ZEPPELIN%20AND%20fixVersion%20%3D%200.8.2
>>
>>
>>
>> --
>> Best Regards
>>
>> Jeff Zhang
>>
>>
>> ___
>> *Eric Pugh **| *Founder & CEO | OpenSource Connections, LLC | 434.466.1467
>> | http://www.opensourceconnections.com | My Free/Busy
>> 
>> Co-Author: Apache Solr Enterprise Search Server, 3rd Ed
>> 
>> This e-mail and all contents, including attachments, is considered to be
>> Company Confidential unless explicitly stated otherwise, regardless
>> of whether attachments are marked as such.
>>
>>
>
> --
> Regards,
> Prabhjyot Singh
>

Re: [VOTE] Creating feature branch for zeppelin new ui initiative

[moon soo Lee]

+1
Looking forward to

On Thu, Sep 5, 2019 at 10:48 PM Xun Liu  wrote:

> +1
> Very looking forward to
>
> Xun Liu
> Best Regards
>
> > On Sep 6, 2019, at 11:18 AM, Jongyoul Lee  wrote:
> >
> > +1
> >
> > On Thu, Sep 5, 2019 at 10:17 PM Jeff Zhang  wrote:
> >
> >> Just to be clear that this thread is about voting of creating separated
> >> feature branch. Regarding merging this into master, we can discuss it
> >> later. Because previously we also thought about moving frontend code
> out of
> >> zeppelin project into subproject. So that there could be multiple
> frontend
> >> implementations, each of them in different subproject.
> >>
> >>
> >> Jeff Zhang  于2019年9月5日周四 上午10:33写道：
> >>
> >>>
> >>> Hi all,
> >>>
> >>> There's one thread [1] discussing about the new zeppelin ui initiative.
> >> It
> >>> looks like the community have reach consensus on this.
> >>>
> >>> Considering this is a large effort and most of work is on frontend, I
> >>> would suggest to create a new feature branch for this initiative. So
> that
> >>> the development of this initiative can move fast and without breaking
> the
> >>> current master branch. Later at proper time, we can consider to merge
> it
> >>> back into master branch, since the most of the work is in frontend, I
> >>> believe the merge won't be too difficult. Let's vote for creating
> feature
> >>> branch for this. Here's my +1
> >>>
> >>>
> >>> [1]
> >>>
> >>
> https://lists.apache.org/thread.html/6ba4722d6eab70366d11bc43d727ca51a4539446e34d24ed7578b00c@%3Cusers.zeppelin.apache.org%3E
> >>>
> >>>
> >>>
> >>> --
> >>> Best Regards
> >>>
> >>> Jeff Zhang
> >>>
> >>
> >>
> >> --
> >> Best Regards
> >>
> >> Jeff Zhang
> >>
> >
> >
> > --
> > 이종열, Jongyoul Lee, 李宗烈
> > http://madeng.net
>
>
>

Re: [DISCUSS] Notebook serving

[moon soo Lee]

Hi,

Although https://github.com/apache/zeppelin/pull/3356/files implements the
basic functionality of the design,
few key features are not yet implemented and many things can be improved.
Also we can even expand the scope, like,

 - Scaling (or autoscaling) serving is not yet implemented. Need to design
how we want to scale (manually by chaning # of replica or using autoscaling
using horizontal autoscaler, or maybe both?)
 - Routing table generation is based on periodic polling of services. Can
be improved by using watch api in kubernetes.
 - Notebook serving design take care of Testing of notebook because we can
not think serving notebook in production without test. But what about Code
review? do we need? if yes, how do we want to handle this for notebooks?
 - We have Test task and Serving task. Do we also need Training for Machine
learning use case?
 - Serving task runs at least one ZeppelinServer and an Interpreter JVM
process. Will there be a way to reduce memory footprint? like a
using graalvm in the container. So hundreds and thousands of small models
can be deployed without much overhead.
 - Every component, TestTask, ServingTask, ContextStorage, MetricStorage
are pluggable. Good! And do we need additional implementation for them?
currently Kubernetes environment is default implementation for all of the
component, but how about integrate with other popular software frameworks?
like Kubeflow, TensroflowServing, etc?

I think there are a lot of interesting topics beyond the pullrequest I
made. So, hope this be part of GSoC.

Thanks,
moon

On Fri, Apr 26, 2019 at 1:50 AM Dragos Dublea 
wrote:

> Hello,
>
> Happy Day!
>
> It is great to follow the improvements on this topic in this Pull Request
> <https://github.com/apache/zeppelin/pull/3356/files>.
> Is this project not going to be the part of GSoC any further? Is there a
> scope for FE as a part of GSoC?
>
> Thanks,
>
>
>
> On Tue, 16 Apr 2019 at 23:06, moon soo Lee  wrote:
>
> > Hi,
> >
> > You're right. I joined the program as a mentor.
> > Thanks again for the interest to the project and to this topic.
> >
> > Thanks,
> > moon
> >
> > On Sat, Apr 13, 2019 at 8:50 AM Dragos Dublea <
> dubleadragos2...@gmail.com>
> > wrote:
> >
> >> Hello,
> >>
> >> I will be very glad to take up any subtasks or participate in a
> >> discussion about this project if you get time for the discussion.
> >>
> >> As I will begin with my vacation soon, I am excited to work on this
> >> project with the Zeppelin community.
> >>
> >> Thank you
> >>
> >> On Wed, Apr 10, 2019, 4:03 PM Dragos Dublea  >
> >> wrote:
> >>
> >>> Hello,
> >>>
> >>> Thank you so much for your reply. AFAIK, Only the student signup period
> >>> is over. Mentors can still join the program. They will have to receive
> an
> >>> invite from the organization admins. Here, in this case, Apache
> Software
> >>> Foundation org admins will have to send the invite link to enable your
> >>> signup.
> >>>
> >>> Thanks again
> >>>
> >>>
> >>>
> >>> On Wed, 10 Apr 2019 at 13:23, moon soo Lee  wrote:
> >>>
> >>>> Hi,
> >>>>
> >>>> Thanks for the interest in this topic. I realized mentor sign up
> period
> >>>> is
> >>>> finished.
> >>>> Let me see if there's a way to add myself as a GsoC mentor or any
> other
> >>>> alternative.
> >>>>
> >>>> Regards,
> >>>> moon
> >>>>
> >>>> On Tue, Apr 9, 2019 at 9:55 PM Dragos Dublea <
> >>>> dubleadragos2...@gmail.com>
> >>>> wrote:
> >>>>
> >>>> > Hello,
> >>>> >
> >>>> > This is a very interesting topic. I did go through the design doc.
> >>>> Can you
> >>>> > please mentor me to implement this? I am very much interested in
> >>>> taking it
> >>>> > up.
> >>>> >
> >>>> > Thanks
> >>>> >
> >>>> > On 2019/03/26 21:31:16, moon soo Lee  wrote:
> >>>> > > Hi,>
> >>>> > >
> >>>> > > There're some challenges bringing a model inside notebook to a
> >>>> > production>
> >>>> >
> >>>> > > environment.>
> >>>> > > Many many organizations, the most common practice I see today is
> >&

Re: New committer: Liu Xun

[moon soo Lee]

Congrats Liu Xun and welcome!


On Wed, Apr 17, 2019 at 8:17 PM Jeff Zhang  wrote:

> The Project Management Committee (PMC) for Apache Zeppelin
> has invited Liu Xun to become a committer and we are pleased
> to announce that he has accepted.
>
> Thanks Liu Xun's contribution to apache zeppelin.
>
> Being a committer enables easier contribution to the
> project since there is no need to go via the patch
> submission process. This should enable better productivity.
> Being a PMC member enables assistance with the management
> and to guide the direction of the project.
>
>
> --
> Best Regards
>
> Jeff Zhang
>

Re: [DISCUSS] Notebook serving

[moon soo Lee]

Hi,

You're right. I joined the program as a mentor.
Thanks again for the interest to the project and to this topic.

Thanks,
moon

On Sat, Apr 13, 2019 at 8:50 AM Dragos Dublea 
wrote:

> Hello,
>
> I will be very glad to take up any subtasks or participate in a discussion
> about this project if you get time for the discussion.
>
> As I will begin with my vacation soon, I am excited to work on this
> project with the Zeppelin community.
>
> Thank you
>
> On Wed, Apr 10, 2019, 4:03 PM Dragos Dublea 
> wrote:
>
>> Hello,
>>
>> Thank you so much for your reply. AFAIK, Only the student signup period
>> is over. Mentors can still join the program. They will have to receive an
>> invite from the organization admins. Here, in this case, Apache Software
>> Foundation org admins will have to send the invite link to enable your
>> signup.
>>
>> Thanks again
>>
>>
>>
>> On Wed, 10 Apr 2019 at 13:23, moon soo Lee  wrote:
>>
>>> Hi,
>>>
>>> Thanks for the interest in this topic. I realized mentor sign up period
>>> is
>>> finished.
>>> Let me see if there's a way to add myself as a GsoC mentor or any other
>>> alternative.
>>>
>>> Regards,
>>> moon
>>>
>>> On Tue, Apr 9, 2019 at 9:55 PM Dragos Dublea >> >
>>> wrote:
>>>
>>> > Hello,
>>> >
>>> > This is a very interesting topic. I did go through the design doc. Can
>>> you
>>> > please mentor me to implement this? I am very much interested in
>>> taking it
>>> > up.
>>> >
>>> > Thanks
>>> >
>>> > On 2019/03/26 21:31:16, moon soo Lee  wrote:
>>> > > Hi,>
>>> > >
>>> > > There're some challenges bringing a model inside notebook to a
>>> > production>
>>> >
>>> > > environment.>
>>> > > Many many organizations, the most common practice I see today is
>>> > something>
>>> > > like>
>>> > >
>>> > > 1. Data scientist develop a model in a data science notebook.>
>>> > > 2. SW engineer rewrites the model, to meet the production
>>> requirements.>
>>> > >
>>> > > In other words, data scientists do not have self-service capability.
>>> And>
>>> > > the organization is spending a lot of time for reimplementing model
>>> for>
>>> > > production.>
>>> > >
>>> > > I tried to identify the gaps between data science notebook and
>>> > production>
>>> >
>>> > > environment, and what can possibly address them. So models that
>>> created
>>> > by>
>>> > > data scientists in the notebook can go production with minimum
>>> efforts.>
>>> > >
>>> > > I made a proposal to solve this problem. Please review and comment.
>>> Any>
>>> > > ideas and feedbacks are welcome. You can make a modification if
>>> needed.>
>>> > >
>>> >
>>> >
>>> https://docs.google.com/document/d/1YA6q8W9yO8a88xzLDYs9zv_fKu2_cnB58rmQbakxi1I/edit?usp=sharing
>>> > >
>>> > >
>>> > > This document is linked from>
>>> > > https://issues.apache.org/jira/browse/ZEPPELIN-3994>
>>> > >
>>> > > Thanks,>
>>> > > moon>
>>> > >
>>> >
>>>
>>

Re: [DISCUSS] Notebook serving

[moon soo Lee]

Hi,

Thanks for the interest in this topic. I realized mentor sign up period is
finished.
Let me see if there's a way to add myself as a GsoC mentor or any other
alternative.

Regards,
moon

On Tue, Apr 9, 2019 at 9:55 PM Dragos Dublea 
wrote:

> Hello,
>
> This is a very interesting topic. I did go through the design doc. Can you
> please mentor me to implement this? I am very much interested in taking it
> up.
>
> Thanks
>
> On 2019/03/26 21:31:16, moon soo Lee  wrote:
> > Hi,>
> >
> > There're some challenges bringing a model inside notebook to a
> production>
>
> > environment.>
> > Many many organizations, the most common practice I see today is
> something>
> > like>
> >
> > 1. Data scientist develop a model in a data science notebook.>
> > 2. SW engineer rewrites the model, to meet the production requirements.>
> >
> > In other words, data scientists do not have self-service capability. And>
> > the organization is spending a lot of time for reimplementing model for>
> > production.>
> >
> > I tried to identify the gaps between data science notebook and
> production>
>
> > environment, and what can possibly address them. So models that created
> by>
> > data scientists in the notebook can go production with minimum efforts.>
> >
> > I made a proposal to solve this problem. Please review and comment. Any>
> > ideas and feedbacks are welcome. You can make a modification if needed.>
> >
>
> https://docs.google.com/document/d/1YA6q8W9yO8a88xzLDYs9zv_fKu2_cnB58rmQbakxi1I/edit?usp=sharing
> >
> >
> > This document is linked from>
> > https://issues.apache.org/jira/browse/ZEPPELIN-3994>
> >
> > Thanks,>
> > moon>
> >
>

[DISCUSS] Notebook serving

[moon soo Lee]

Hi,

There're some challenges bringing a model inside notebook to a production
environment.
Many many organizations, the most common practice I see today is something
like

1. Data scientist develop a model in a data science notebook.
2. SW engineer rewrites the model, to meet the production requirements.

In other words, data scientists do not have self-service capability. And
the organization is spending a lot of time for reimplementing model for
production.

I tried to identify the gaps between data science notebook and production
environment, and what can possibly address them. So models that created by
data scientists in the notebook can go production with minimum efforts.

I made a proposal to solve this problem. Please review and comment. Any
ideas and feedbacks are welcome. You can make a modification if needed.
https://docs.google.com/document/d/1YA6q8W9yO8a88xzLDYs9zv_fKu2_cnB58rmQbakxi1I/edit?usp=sharing

This document is linked from
https://issues.apache.org/jira/browse/ZEPPELIN-3994

Thanks,
moon

Re: Zeppelin in GSOC 2019

[moon soo Lee]

Hi,

Great to see your interest to project. Thanks!
Looks like we need volunteers for a mentor and some backend subject for
GSoC2019.
Any ideas?

Best,
moon




On Mon, Mar 4, 2019 at 3:05 PM Vasiliy Morkovkin 
wrote:

> Hi everyone, I'm pursuing bachelor degree at Moscow institute of physics
> and technology and eager to contribute to Zeppelin in context of GSOC
> 2019. I've become a real fan of Zeppelin over the past couple of months,
> using it at my job. But I have found out only one ticket (front-end
> task) with label of GSOC 2019 on your Jira. Perhaps you may have any
> ideas for new features or improvements in Zeppelin, but you don't have
> enough hands on them. It would be wonderful if anyone agreed to mentor
> these ideas within GSOC :)
> Currently I am in a position of Scala developer (back-end) for 1.5 year.
> I also can write in Java or Python without any problems if necessary.
> Really fond of databases and highload. Also I have experience with some
> other great Apache projects like Cassandra, Kafka and Spark.
>
> Best regards, Basil Morkovkin.
>
>

Re: [VOTE] Release Apache Zeppelin 0.8.0 (RC6)

[moon soo Lee]

+1

Best,
moon

On Wed, Jun 27, 2018 at 12:40 AM NineQS CloverHearts <
cloverhearts...@gmail.com> wrote:

> +1
> 
> From: Belousov Maksim Eduardovich 
> Sent: Tuesday, June 26, 2018 10:27:49 PM
> To: dev@zeppelin.apache.org
> Subject: RE: [VOTE] Release Apache Zeppelin 0.8.0 (RC6)
>
> +1
>
>
> Regards,
>
> Maksim Belousov
>
> -Original Message-
> From: Prabhjyot Singh [mailto:prabhjyotsi...@apache.org]
> Sent: Monday, June 25, 2018 11:33 AM
> To: dev 
> Subject: Re: [VOTE] Release Apache Zeppelin 0.8.0 (RC6)
>
> +1
>
> On Mon, 25 Jun 2018 at 06:05, Jeff Zhang  wrote:
>
> > Here's my +1.
> >
> > I build it from source successfully and ran spark tutorial in local,
> > yarn-client, yarn-cluster mode successfully.
> >
> >
> >
> > Jeff Zhang 于2018年6月25日周一 上午8:34写道：
> >
> > > Hi folks,
> > >
> > > I propose the following RC to be released for the Apache Zeppelin
> > > 0.8.0 release.
> > >
> > > The commit id is 603cc492f5e322c73915501e8cf65970d0c61870
> > >
> > >  *
> > https://git-wip-us.apache.org/repos/asf?p=zeppelin.git;a=commit;h=603c
> > c492f5e322c73915501e8cf65970d0c61870
> > <
> > https://git-wip-us.apache.org/repos/asf?p=zeppelin.git;a=commit;h=603c
> > c492f5e322c73915501e8cf65970d0c61870
> > >*
> > >
> > >
> > > This corresponds to the tag: v0.8.0-rc6
> > >
> > > The release archives (tgz), signature, and checksums are here *
> > https://dist.apache.org/repos/dist/dev/zeppelin/zeppelin-0.8.0-rc6/ <
> > https://dist.apache.org/repos/dist/dev/zeppelin/zeppelin-0.8.0-rc6/>*
> > >
> > >
> > > The release candidate consists of the following source distribution
> > archive zeppelin-0.8.0.tgz
> > >
> > > In addition, the following supplementary binary distributions are
> > provided
> > > for user convenience at the same location zeppelin-0.8.0-bin-all.tgz
> > >
> > > zeppelin-0.8.0-bin-netinst.tgz
> > >
> > > The maven artifacts are here *
> > https://repository.apache.org/content/repositories/orgapachezeppelin-1
> > 252
> > > <
> > https://repository.apache.org/content/repositories/orgapachezeppelin-1
> > 252
> > >*
> > >
> > > You can find the KEYS file here:
> > https://dist.apache.org/repos/dist/release/zeppelin/KEYS
> > >
> > > Release notes available at
> > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12316
> > 221=12339144
> > >
> > > Vote will be open for next 72 hours (close at 6PM 27/June PDT).
> > >
> > > [ ] +1 approve
> > > [ ] 0 no opinion
> > > [ ] -1 disapprove (and reason why)
> > >
> > >
> >
> > --
> >
> > Warm Regards,
> >
> > Prabhjyot Singh
> >
> >
>

Re: [VOTE] Release Apache Zeppelin 0.8.0 (RC5)

[moon soo Lee]

+1

Thanks,
moon

On Fri, Jun 22, 2018 at 17:08 Belousov Maksim Eduardovich
 wrote:

> +1
>
>
> Regards,
>
> Maksim Belousov
>
> -Original Message-
> From: Jeff Zhang [mailto:zjf...@gmail.com]
> Sent: Thursday, June 21, 2018 10:18 AM
> To: dev@zeppelin.apache.org
> Subject: Re: [VOTE] Release Apache Zeppelin 0.8.0 (RC5)
>
> Here's my +1
>
> I build it from source code successfully, and run zeppelin tutorial note
> successfully in local, yarn-client, yarn-cluster mode under spark-2.3
>
>
>
> Miquel Angel Andreu Febrer 于2018年6月20日周三
> 下午1:58写道：
>
> > Thanks Jeff,
> >
> > LGTM
> >
> > +1
> >
> >
> > El mié., 20 jun. 2018 7:56, Prabhjyot Singh
> > 
> > escribió:
> >
> > > +1
> > >
> > > On Wed, 20 Jun 2018 at 10:48, CloverHearts
> > > 
> > > wrote:
> > >
> > > > Awesome +1
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > On Wed, Jun 20, 2018 at 11:41 AM +0900, "Jeff Zhang" <
> > zjf...@apache.org>
> > > > wrote:
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > Hi folks,
> > > >
> > > > I propose the following RC to be released for the Apache Zeppelin
> > > > 0.8.0 release.
> > > >
> > > > The commit id is adc2e63c1ff1f7690ba7b3c10984c8024e379a31
> > > >
> > > >  *
> > > >
> > >
> > https://git-wip-us.apache.org/repos/asf?p=zeppelin.git;a=commit;h=adc2
> > e63c1ff1f7690ba7b3c10984c8024e379a31
> > > > *
> > > >
> > > >
> > > >
> > > > This corresponds to the tag: v0.8.0-rc5
> > > >
> > > > The release archives (tgz), signature, and checksums are here
> > > > *https://dist.apache.org/repos/dist/dev/zeppelin/zeppelin-0.8.0-rc
> > > > 5/
> > > > *
> > > >
> > > >
> > > > The release candidate consists of the following source
> > > > distribution archive zeppelin-0.8.0.tgz
> > > >
> > > > In addition, the following supplementary binary distributions are
> > > provided
> > > > for user convenience at the same location
> > > > zeppelin-0.8.0-bin-all.tgz
> > > >
> > > > zeppelin-0.8.0-bin-netinst.tgz
> > > >
> > > > The maven artifacts are here
> > > > *
> > > >
> > >
> > https://repository.apache.org/content/repositories/orgapachezeppelin-1
> > 242/
> > > > *
> > > >
> > > > You can find the KEYS file here:
> > > > https://dist.apache.org/repos/dist/release/zeppelin/KEYS
> > > >
> > > > Release notes available at
> > > >
> > > >
> > >
> > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12316
> > 221=12339144
> > > >
> > > > Vote will be open for next 72 hours (close at 8PM 22/June PDT).
> > > >
> > > > [ ] +1 approve
> > > > [ ] 0 no opinion
> > > > [ ] -1 disapprove (and reason why)
> > > >
> > > > --
> > > >
> > > > Warm Regards,
> > > >
> > > > Prabhjyot Singh
> > > >
> > > >
> > >
> >
>

Re: [VOTE] Release Apache Zeppelin 0.8.0 (RC4)

[moon soo Lee]

+1
Verified

 - Signature
 - License exists
 - Build from source
 - Run tutorial notebooks
 - Install helium visualizaitons
 - Works with multiple spark version

Thanks Jeff for preparing the release.

On Wed, May 30, 2018 at 8:54 AM Miquel Angel Andreu Febrer <
miquelangeland...@gmail.com> wrote:

> +1
>
> Test the principal interpretes and works successfuly
>
>
> El mié., 30 may. 2018 16:12, Jeff Zhang  escribió:
>
> > I will start with my +1
> >
> > I build source code successfully and can run spark tutorial note.
> >
> > Try zeppelin-0.8.0-bin-all and run spark tutorial note successfully under
> > local, yarn-client and yarn-cluster mode. also run it under livy
> > interpreter and impersonation mode
> > Try zeppelin-0.8.0-bin-netinst and run spark tutorial note successfully
> > under local, yarn-client and yarn-cluster mode.
> >
> > Jeff Zhang 于2018年5月30日周三 下午10:08写道：
> >
> > > Hi folks,
> > >
> > > I propose the following RC to be released for the Apache Zeppelin
> > > 0.8.0 release.
> > >
> > >
> > > The commit id is 41d9e0ac3f908da2eb4167984271aced78a8bfc6
> > >
> > >  *
> >
> https://git-wip-us.apache.org/repos/asf?p=zeppelin.git;a=commit;h=41d9e0ac3f908da2eb4167984271aced78a8bfc6
> > <
> >
> https://git-wip-us.apache.org/repos/asf?p=zeppelin.git;a=commit;h=41d9e0ac3f908da2eb4167984271aced78a8bfc6
> > >*
> > >
> > >
> > >
> > > This corresponds to the tag: v0.8.0-rc4
> > >
> > > The release archives (tgz), signature, and checksums are here *
> > https://dist.apache.org/repos/dist/dev/zeppelin/zeppelin-0.8.0-rc4/ <
> > https://dist.apache.org/repos/dist/dev/zeppelin/zeppelin-0.8.0-rc4/>*
> > >
> > >
> > > The release candidate consists of the following source distribution
> > archive zeppelin-0.8.0.tgz
> > >
> > > In addition, the following supplementary binary distributions are
> > provided
> > > for user convenience at the same location zeppelin-0.8.0-bin-all.tgz
> > >
> > > zeppelin-0.8.0-bin-netinst.tgz
> > >
> > > The maven artifacts are here *
> >
> https://repository.apache.org/content/repositories/orgapachezeppelin-1195/
> > > <
> >
> https://repository.apache.org/content/repositories/orgapachezeppelin-1195/
> > >*
> > >
> > > You can find the KEYS file here:
> > https://dist.apache.org/repos/dist/release/zeppelin/KEYS
> > >
> > > Release notes available at
> >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12316221=12339144
> > >
> > > Vote will be open for next 72 hours (close at 7AM 2/June PDT).
> > >
> > > [ ] +1 approve
> > > [ ] 0 no opinion
> > > [ ] -1 disapprove (and reason why)
> > >
> > >
> >
>

Re: [VOTE] Release Apache Zeppelin 0.8.0 (RC2)

[moon soo Lee]

Thanks Jeff preparing the release.

+1

 - License exists in source/binary package
 - Signature
 - Build from source
 - Run tutorial notebooks

I think ./bin/install-interpreter.sh does not work because 0.8.0 artifacts
are not published yet. If ./bin/install-interpreter.sh command have a
capability to add staging repo (
https://repository.apache.org/content/repositories/orgapachezeppelin-1124/)
that'll be helpful verifying RC.

On Tue, May 15, 2018 at 10:42 AM Sanjay Dasgupta 
wrote:

> -1
>
> I downloaded zeppelin-0.8.0-bin-netinst.tgz, and then tried to install the
> shell and jdbc interpreters. Both attempts failed (Ubuntu 16.04, Oracle JDK
> 1.8.0_171).
>
> The error while trying to install the shell interpreter is shown below:
>
> $ ./bin/install-interpreter.sh --name shell
>
> Java HotSpot(TM) 64-Bit Server VM warning: ignoring option
> MaxPermSize=512m; support was removed in 8.0
> SLF4J: Class path contains multiple SLF4J bindings.
> SLF4J: Found binding in
>
> [jar:file:/home/yajnas/Desktop/Data-Analytics-AI/zeppelin/zeppelin-0.8.0-rc2-bin-netinst/lib/interpreter/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in
>
> [jar:file:/home/yajnas/Desktop/Data-Analytics-AI/zeppelin/zeppelin-0.8.0-rc2-bin-netinst/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
> explanation.
> SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
> Install shell(org.apache.zeppelin:zeppelin-shell:0.8.0) to
>
> /home/yajnas/Desktop/Data-Analytics-AI/zeppelin/zeppelin-0.8.0-rc2-bin-netinst/interpreter/shell
> ...
> org.sonatype.aether.RepositoryException: Cannot fetch dependencies for
> org.apache.zeppelin:zeppelin-shell:0.8.0
> at
>
> org.apache.zeppelin.dep.DependencyResolver.getArtifactsWithDep(DependencyResolver.java:179)
> at
>
> org.apache.zeppelin.dep.DependencyResolver.loadFromMvn(DependencyResolver.java:128)
> at
> org.apache.zeppelin.dep.DependencyResolver.load(DependencyResolver.java:76)
> at
> org.apache.zeppelin.dep.DependencyResolver.load(DependencyResolver.java:93)
> at
> org.apache.zeppelin.dep.DependencyResolver.load(DependencyResolver.java:85)
> at
>
> org.apache.zeppelin.interpreter.install.InstallInterpreter.install(InstallInterpreter.java:170)
> at
>
> org.apache.zeppelin.interpreter.install.InstallInterpreter.install(InstallInterpreter.java:134)
> at
>
> org.apache.zeppelin.interpreter.install.InstallInterpreter.install(InstallInterpreter.java:126)
> at
>
> org.apache.zeppelin.interpreter.install.InstallInterpreter.main(InstallInterpreter.java:278)
> Caused by: org.sonatype.aether.resolution.DependencyResolutionException:
> Could not find artifact org.apache.zeppelin:zeppelin-shell:jar:0.8.0 in
> central (http://repo1.maven.org/maven2/)
> at
>
> org.sonatype.aether.impl.internal.DefaultRepositorySystem.resolveDependencies(DefaultRepositorySystem.java:375)
> at
>
> org.apache.zeppelin.dep.DependencyResolver.getArtifactsWithDep(DependencyResolver.java:176)
> ... 8 more
> Caused by: org.sonatype.aether.resolution.ArtifactResolutionException:
> Could not find artifact org.apache.zeppelin:zeppelin-shell:jar:0.8.0 in
> central (http://repo1.maven.org/maven2/)
> at
>
> org.sonatype.aether.impl.internal.DefaultArtifactResolver.resolve(DefaultArtifactResolver.java:537)
> at
>
> org.sonatype.aether.impl.internal.DefaultArtifactResolver.resolveArtifacts(DefaultArtifactResolver.java:216)
> at
>
> org.sonatype.aether.impl.internal.DefaultRepositorySystem.resolveDependencies(DefaultRepositorySystem.java:358)
> ... 9 more
> Caused by: org.sonatype.aether.transfer.ArtifactNotFoundException: Could
> not find artifact org.apache.zeppelin:zeppelin-shell:jar:0.8.0 in central (
> http://repo1.maven.org/maven2/)
> at
>
> org.sonatype.aether.connector.wagon.WagonRepositoryConnector$4.wrap(WagonRepositoryConnector.java:971)
> at
>
> org.sonatype.aether.connector.wagon.WagonRepositoryConnector$4.wrap(WagonRepositoryConnector.java:966)
> at
>
> org.sonatype.aether.connector.wagon.WagonRepositoryConnector$GetTask.flush(WagonRepositoryConnector.java:707)
> at
>
> org.sonatype.aether.connector.wagon.WagonRepositoryConnector$GetTask.flush(WagonRepositoryConnector.java:701)
> at
>
> org.sonatype.aether.connector.wagon.WagonRepositoryConnector.get(WagonRepositoryConnector.java:452)
> at
>
> org.sonatype.aether.impl.internal.DefaultArtifactResolver.resolve(DefaultArtifactResolver.java:456)
> ... 11 more
> $
>
>
> On Tue, May 15, 2018 at 9:34 PM, Renjith Kamath 
> wrote:
>
> > +1
> > verified both
> >
> > Thanks & Regards,
> > RK
> >
> > On Tue, May 15, 2018 at 3:14 PM, Miquel Angel Andreu Febrer <
> > miquelangeland...@gmail.com> wrote:
> >
> > > +1
> > >
> > > Launched the packages with docker and works well
> > >
> > > El mar., 15 may. 2018 11:06, Jeff Zhang  escribió:
> > >
> > > > Here's my +1
> > > >
> > > > I 

Re: [Discussion] 0.8.0 Release

[moon soo Lee]

Cool. Thanks for the volunteer.

Why don't we update issue link of
https://issues.apache.org/jira/browse/ZEPPELIN-2385? I think issue link is
bit out dated.
We can remove issue links if they're not expected in 0.8.0, add issue links
if they must be included in 0.8.0.
So everyone can track the progress of 0.8.0 in the same place.

Thanks,
moon


On Wed, Dec 27, 2017 at 8:17 AM Belousov Maksim Eduardovich <
m.belou...@tinkoff.ru> wrote:

> This is great news.
>
> Our team will take part in testing and bug fixing.
>
>
> Thanks,
>
> Maksim Belousov
>
>
> -Original Message-
> From: Miquel Angel Andreu Febrer [mailto:miquelangeland...@gmail.com]
> Sent: Wednesday, December 27, 2017 3:46 PM
> To: dev@zeppelin.apache.org
> Subject: Re: [Discussion] 0.8.0 Release
>
> Hello everyone,
>
> I think it's a good idea, I do not know if we are pending some important
> issues for the 0.8.0, we should see in jira. I can help with the management
> of the 0.8.0 release if the new release will be launched at the end of
> January
>
> Regards
>
> El 27 dic. 2017 13:35, "Jeff Zhang"  escribió:
>
> > Hi folks,
> >
> > It is a long time since our last release, and many new features are
> > added into 0.8.0, so I think we could think about to prepare the
> > release of 0.8.0, if no volunteer to be the release manager, I
> > volunteer to be the release manager of this version. My current
> > thinking is to make the release at the end of January. Any thinking and
> comments ?
> >
>

Re: [DISCUSS] Review process

[moon soo Lee]

Hi,

Current review process[1] does require either at least a +1 from committer
or 24 hours for lazy consensus.

Pullrequest can be open for 1 or 2 days for additional review, but i think
they're not hard requirements. (e.g. Hotfixes are already being merged
without waiting additional review)

So, technically, current policy allows any committer can start review, mark
+1 and merge immediately without any delay if necessary.

Thanks,
moon

[1]
http://zeppelin.apache.org/contribution/contributions.html#the-review-process


On Mon, Dec 18, 2017 at 2:13 AM Belousov Maksim Eduardovich <
m.belou...@tinkoff.ru> wrote:

> +1 for non-delay merging.
> Our team have opened approved PR [1] for 5 days.
>
> I didn't find any pages with `consensus how to review and merge
> contributions`.
> It would be nice to write a check list for reviewer.
>
> The development of Zeppelin is very important for us and we want to review
> new commits.
>
>
> [1] https://github.com/apache/zeppelin/pull/2697
>
>
> Thanks,
> Maksim Belousov
>
> -Original Message-
> From: Jongyoul Lee [mailto:jongy...@gmail.com]
> Sent: Monday, December 18, 2017 12:12 PM
> To: dev 
> Subject: Re: [DISCUSS] Review process
>
> Thank you for the replying it. I think so
>
> On Mon, Dec 18, 2017 at 3:15 PM, Miquel Angel Andreu Febrer <
> miquelangeland...@gmail.com> wrote:
>
> > I agree, ig is necessary to have no delay afternoon merging. I think
> > it will help speed up processes and help contributors
> >
> > El 18 dic. 2017 4:33, "Jongyoul Lee"  escribió:
> >
> > Hi committers,
> >
> > I want to suggest one thing about our reviewing process. We have the
> > policy to wait for one-day before merging some PRs. AFAIK, It's
> > because we reduce mistakes and prevent abuses from committing by owner
> > without reviewing it concretely. I would like to change this policy to
> > remove delay after merging it. We, recently, don't have much reviewers
> > and committers who can merge continuously, and in my case, I,
> > sometimes, forget some PRs that I have to merge. And I also believe
> > all committers have consensus how to review and merge contributions.
> >
> > How do you think of it?
> >
> > JL
> >
> > --
> > 이종열, Jongyoul Lee, 李宗烈
> > http://madeng.net
> >
>
>
>
> --
> 이종열, Jongyoul Lee, 李宗烈
> http://madeng.net
>

Re: Disable cron runs

[moon soo Lee]

Hi,

Currently, there's no easy way to disable cron.
I think it's okay to add configuration parameter that disables all cron
run, while the default value of the parameter enable the cron (for backward
compatibility).

Let me know if you need any help on this.

Thanks,
moon

On Wed, Oct 4, 2017 at 2:35 AM Belousov Maksim Eduardovich <
m.belou...@tinkoff.ru> wrote:

> Hello, dev!
>
> Our team have about 600 notes and 35 of them are scheduled. Some scheduled
> notes create tables in DB and send e-mails.
> We want to hide testing activities when we test notes on another server.
> Now we remove cron parameters in json files. It is not cool.
>
> Are there any workarounds to disable running of notes with cron?
> Can it be a good decision to add a new server parameter that will disable
> all cron runs?
>
>
> Thanks,
>
> Maksim Belousov
>
>

Re: Building a query service using Zeppelin

[moon soo Lee]

Hi Saksham,

Thanks for sharing your plan.
Programmatic universal query execution platform sounds interesting.

I saw a lot of users use Zeppelin with various SQL backends.
Your plan definitely looks like help many other users, too. If i can add
few more idea

* Search available table, columns
* Preview table contents
* Table level, column level, row level access control

Those are some feedbacks and requests from Zeppelin users i met, who have
SQL backends. Technical point of view, query syntax are different in each
SQL backends. So to provide features above, need to leverage ORM (e.g.
hibernate), probably.

Thanks,
moon

On Mon, Oct 9, 2017 at 7:15 AM Saksham Srivastava 
wrote:

> Hi,
>
>
> We have been evaluating Zeppelin to develop a programmatic(API-driven)
> universal query execution platform.
>
> A platform which can:
>
>   *   Accept and validate any query (spark-sql, sql, hive, elastic-search
> etc)
>   *   Run the query
>   *   Track the status
>   *   Stop the query.
>   *   Also, support multiple queries and workflow.
>
> We evaluated quite a few frameworks and then finally chose Zeppelin over
> others considering:
>
>   *   Zeppelin is an ASF project.
>   *   The community is lively and helpful.
>   *   The API driven approach of Zeppelin which will help mashup more APIs
> and create pipelines.
>
> We would like to contribute more to Zeppelin by contributing code, raising
> and fixing issues and adding more APIs and interpreters.
>
> What does the community think of this, please feel forward to reply with
> suggestions or questions.
>
>
> Thanks,
>
> Saksham
>

Re: Issue with Paragraph Status API

[moon soo Lee]

Hi Saksham,

Thanks for sharing the problem.
I think that API shouldn't wants for execution to complete.
I think it's not a known issue. If you can file an jira issue, it'll help
track this problem.

Thanks,
moon

On Fri, Sep 22, 2017 at 6:42 AM Saksham Srivastava 
wrote:

> Hi,
>
>
> I see there is an issue with the Paragraph status API, in Zeppelin 0.7.2
>
>
>
> https://zeppelin.apache.org/docs/0.7.2/rest-api/rest-notebook.html#get-the-status-of-a-single-paragraph
>
> Apache Zeppelin Notebook REST API - Apache Software Foundation<
> https://zeppelin.apache.org/docs/0.7.2/rest-api/rest-notebook.html#get-the-status-of-a-single-paragraph
> >
> zeppelin.apache.org
> This page contains Apache Zeppelin Notebook REST API information.
>
> If the paragraph is still in execution, the API call just waits for the
> execution to complete.
>
> I guess the ideal implementation should be to return the status
> synchronously as "running" or something else.
>
>
> I am testing a simple spark-sql query.
>
>
> Is this a known issue?
>
>
> Thanks,
>
> Saksham
>
>
>
>
>

Re: [VOTE] Release Apache Zeppelin 0.7.3 (RC5)

[moon soo Lee]

+1 for releasing this RC

Verified

- Build from source
- Binary package runs tutorial notebook
- License file exists
- Signature

Thanks,
moon


On Wed, Sep 20, 2017 at 3:17 AM Krishna Pandey 
wrote:

> +1
>
> Ran PySpark and Spark use-cases successfully.
> Also verified PGP, SHA-512 and MD5 signatures for all binaries.
>
> Regards,
> Krishna Pandey
>
> On Mon, Sep 18, 2017 at 10:35 PM, Mina Lee  wrote:
>
> > I propose the following RC to be released for the Apache Zeppelin 0.7.3
> > release.
> >
> > The commit id is 5d246c96b054dcb8854e2beb924fc2c89b767fc1 which is
> > corresponds to the tag v0.7.3-rc5:
> > https://git-wip-us.apache.org/repos/asf?p=zeppelin.git;h=
> > refs/tags/v0.7.3-rc5
> >
> > The release archives (tgz), signature, and checksums are here
> > https://dist.apache.org/repos/dist/dev/zeppelin/zeppelin-0.7.3-rc5/
> >
> > The release candidate consists of the following source distribution
> archive
> > zeppelin-0.7.3.tgz
> >
> > In addition, the following supplementary binary distributions are
> provided
> > for user convenience at the same location
> > zeppelin-0.7.3-bin-all.tgz
> > zeppelin-0.7.3-bin-netinst.tgz
> >
> > The maven artifacts are here
> >
> https://repository.apache.org/content/repositories/orgapachezeppelin-1047
> >
> > You can find the KEYS file here:
> > https://dist.apache.org/repos/dist/release/zeppelin/KEYS
> >
> > Release notes available at
> > https://issues.apache.org/jira/secure/ReleaseNote.jspa?versi
> > on=12341035=12316221
> >
> > Vote will be open for next 72 hours (close at 10:30 21/Sep PDT).
> >
> > [ ] +1 approve
> > [ ] 0 no opinion
> > [ ] -1 disapprove (and reason why)
> >
>

Re: Official Docker image build and release process

[moon soo Lee]

I think tag push automatically builds docker image.
see https://issues.apache.org/jira/browse/INFRA-12781

Thans,
moon

On Fri, Sep 15, 2017 at 2:31 PM Felix Cheung 
wrote:

> Anyone?
>
> 
> From: Felix Cheung 
> Sent: Thursday, September 14, 2017 8:45:07 AM
> To: dev@zeppelin.apache.org
> Subject: Official Docker image build and release process
>
> Hi!
>
> Where do we have information on this?
>
> Thanks
>
>

Re: Introduction

[moon soo Lee]

Hi Saksham,

Thanks for the interest to the project and community.

Interpreter binding mode is explained here [1].
For execution and parallelism, you can check Interpreter.getScheduler()
method [2]. Each interpreter can override this method and implement it's
own scheduler to achieve sequential/parallel or any other possible
execution policy. [3] is an example of returning different schedulers
depends on condition.

Hope this helps.

Thanks,
moon

[1]
http://zeppelin.apache.org/docs/snapshot/usage/interpreter/interpreter_binding_mode.html
[2]
https://github.com/apache/zeppelin/blob/v0.7.2/zeppelin-interpreter/src/main/java/org/apache/zeppelin/interpreter/Interpreter.java
[3]
https://github.com/apache/zeppelin/blob/v0.7.2/spark/src/main/java/org/apache/zeppelin/spark/SparkSqlInterpreter.java#L156


On Thu, Aug 24, 2017 at 4:30 AM Saksham Srivastava 
wrote:

> Hi,
>
>
> I am Saksham and work as a developer. We have recently started using
> Apache Zeppelin as an interface to submit queries on our clusters. It looks
> like a very exciting project. Earlier, I have contributed to Apache
> CloudStack too.
>
>
> Can someone be kind enough to guide me to documentation which talks about:
>
>
>   *   Notebooks, notes, paragraphs (the hierarchy)
>   *   interpreter bindings to notes/paragraphs,
>   *   details of (sequential/parallel) execution of paragraphs inside a
> notebook
>   *   parallelism within notes/paragraphs
>   *   achieving concurrent execution
>
> I am looking forward to be a part of this exciting community.
>
> Thanks,
> Saksham
>

Re: API to get/download paragraph result

[moon soo Lee]

Hi Saksham,

As you mentioned, there's an API that returns paragraph result in JSON
format [1].
Types of result can be TEXT, HTML, TABLE, IMG etc [2].

I assume you meant download TABLE data as csv/tsv format programmatically,
right?
One reason we don't have such API is that the csv/tsv conversion is done on
front-end side [3].

To provide an API, we'll need to move these routines from front-end to
back-end side.

Thanks,
moon

[1]
https://github.com/apache/zeppelin/blob/master/zeppelin-server/src/main/java/org/apache/zeppelin/rest/NotebookRestApi.java#L486
[2]
https://github.com/apache/zeppelin/blob/master/zeppelin-interpreter/src/main/java/org/apache/zeppelin/interpreter/InterpreterResult.java#L50
[3]
https://github.com/apache/zeppelin/blob/master/zeppelin-web/src/app/notebook/paragraph/result/result.controller.js#L749

On Wed, Aug 30, 2017 at 12:19 AM Saksham Srivastava 
wrote:

> Hi,
>
>
> My previous mail did not get much love so I am trying again :)
>
>
> Is there any API to download the response of a paragraph?
>
>
> I know there is a synchronous run API which will wait for the completion
> of execution and will return the result.
>
>
> http://zeppelin.apache.org/docs/0.7.2/rest-api/rest-notebook.html#run-a-paragraph-synchronously
>
>
> But in case a user is trying to run a paragraph asynchronously, he/she
> cannot get the results through the API
>
> I know there is status API to check the status of paragraph (it will not
> have the result set info AFAIK)
>
>
> https://zeppelin.apache.org/docs/0.7.2/rest-api/rest-notebook.html#get-the-status-of-a-single-paragraph
>
>
> And finally, there is paragraph-information API
>
>
> https://zeppelin.apache.org/docs/0.7.2/rest-api/rest-notebook.html#get-a-paragraph-information
>
>
> The result set can be viewed in the response JSON of this as
> body.results.msg.data
>
>
> My expectation is to have an API which can return the result set which is
> seen in the notebook itself. We have the option to view/download the
> results in multiple formats in Notebook but there is no such way through
> APIs.
>
> We have APIs to clear the result too.
>
>
> Why do we need such an API?
>
> To allow the user to get/download the result set programmatically.
>
>
> Is this a relevant requirement?
>
>
> Thanks,
>
> Saksham
>

Re: Re: [DISCUSS] Release 0.7.3

[moon soo Lee]

+1

On Tue, Aug 29, 2017 at 5:05 AM Windy Qin  wrote:

> +1
>
>
>
> Thanks
> WindyQin
>
> From: Krishna Pandey
> Date: 2017-08-29 20:01
> To: dev
> Subject: Re: [DISCUSS] Release 0.7.3
> +1
>
> On Tue, Aug 29, 2017 at 12:28 PM, Prabhjyot Singh <
> prabhjyotsi...@apache.org
> > wrote:
>
> > Hi,
> >
> > Since 0.7.2 release, we have resolved 22 issues and that includes most of
> > the critical issues in 0.7.3 umbrella issue [1].
> >
> > With these bugs fixes, it makes Zeppelin moves towards stability, and I'd
> > like to proceed release process for 0.7.3 at this point and provide
> stable
> > release.
> >
> > Remaining issue can be resolved in 0.8.0 release (or 0.7.4 if another
> > maintenance release is necessary).
> >
> > Shell we proceed to 0.7.3?
> >
> > Thanks and Regards,
> > Prabhjyot Singh
> >
> > [1] https://issues.apache.org/jira/browse/ZEPPELIN-2864
> >
> > --
> >
> > Warm Regards,
> >
> > Prabhjyot Singh
> >
>

Re: What about the release of Zeppelin 0.7.3

[moon soo Lee]

Hi,

0.7.3 is a maintenance release, so in terms of new feature, I prefer to
ship it in 0.8.0, so we can keep 0.7.3 as a stable release.

We have a discussion thread for release 0.8.0 [1]. And I think we can
discuss there, more about 0.8.0 release which includes all the new
functionalities.

Does it make sense for you?

Thanks,
moon

[1]
https://lists.apache.org/thread.html/bb860fb22bbb473f3e784db8461e80f651c62ba793921f82f52d06ff@%3Cdev.zeppelin.apache.org%3E

On Thu, Aug 17, 2017 at 7:35 AM Polyakov Valeriy <v.polja...@tinkoff.ru>
wrote:

> Hi, Moon!
>
>
>
> Is there any possibility to include in 0.7.3 some critical functionality
> (for Tinkoff) out of security scope? It will help us to upgrade our
> platform in more stable mode. Now we are forced to use unstable version
> built from development snapshot in production.
>
>
>
>
>
>
>
> On 2017-08-16 18:22, moon soo Lee <m...@apache.org<mailto:m...@apache.org>>
> wrote:
>
> > Hi,>
>
> >
>
> > An issue for Release 0.7.3 [1] has created and some security issues are>
>
> > linked. Please take a look and let us know if you have another security>
>
> > issue need to be included.>
>
> >
>
> > Thanks,>
>
> > moon>
>
> >
>
> > [1] https://issues.apache.org/jira/browse/ZEPPELIN-2864>
>
> >
>
> > On Mon, Aug 7, 2017 at 1:58 AM Yanbo Liang <yb...@gmail.com yb...@gmail.com>> wrote:>
>
> >
>
> > > Hi All,>
>
> > >>
>
> > > Recently we sent some security features/fixes to Apache Zeppelin. Do
> you>
>
> > > have plan to release a new version in near future? Then we can upgrade
> our>
>
> > > Zeppelin to use these new features.>
>
> > >>
>
> > > Thanks>
>
> > > Yanbo>
>
> > >>
>
> >
>

Re: What about the release of Zeppelin 0.7.3

[moon soo Lee]

Hi,

An issue for Release 0.7.3 [1] has created and some security issues are
linked. Please take a look and let us know if you have another security
issue need to be included.

Thanks,
moon

[1] https://issues.apache.org/jira/browse/ZEPPELIN-2864

On Mon, Aug 7, 2017 at 1:58 AM Yanbo Liang  wrote:

> Hi All,
>
> Recently we sent some security features/fixes to Apache Zeppelin. Do you
> have plan to release a new version in near future? Then we can upgrade our
> Zeppelin to use these new features.
>
> Thanks
> Yanbo
>

Re: Broken link on Apache Zeppelin website

[moon soo Lee]

Thanks for reporting the problem

Made a patch https://github.com/apache/zeppelin/pull/2494

Will deploy soon.

Thanks,
moon

On Sat, Jul 15, 2017 at 12:54 AM Krishna Pandey 
wrote:

> Hi All,
>
> Whenever we raise a Pull Request for Apache Zeppelin, we are presented with
> a helping Questionnaire. One such question is below which points to
> Contribution guide.
>
> ### What is this PR for?
> A few sentences describing the overall goals of the pull request's commits.
> First time? Check out the contributing guide -
> https://zeppelin.apache.org/
> contribution/contributions.html
> 
>
>
> When you land on the webpage following the above link and click Community
> drop-down list on top menu and then click "Contributors" link, it throws a
> HTTP 404 error "The requested URL /contribution/community.html was not
> found on this server."
>
> Thanks,
> Krishna Pandey
>

New PMC member: Ahyoung Ryu

[moon soo Lee]

The Project Management Committee (PMC) for Apache Zeppelin has invited
Ahyoung Ryu to become a PMC and we are pleased to announce that has
accepted.

Please join us to welcome.

The Apache Zeppelin PMC

Re: Add version 0.7.3 in jira

[moon soo Lee]

Done!

On Thu, Jun 29, 2017 at 3:19 PM Jeff Zhang  wrote:

> Could anyone create 0.7.3 in zeppelin's jira since 0.7.2 has been released.
>

New committer: Park Hoon

[moon soo Lee]

The Project Management Committee (PMC) for Apache Zeppelin has invited Park
Hoon (1ambda) to become a committer and we are pleased to announce that has
accepted.

Please join us to welcome.

The Apache Zeppelin PMC

Re: Official Docker Image for Apache Zeppelin 0.7.2

[moon soo Lee]

Great work!

Tested basic features and it's working well.

I think it'll help user a lot when
http://zeppelin.apache.org/download.html have
some guide (or link to guide) for docker image.

Thanks,
moon


On Thu, Jun 15, 2017 at 10:24 PM Park Hoon <1am...@gmail.com> wrote:

> Hi, devs.
>
>
> A few months ago the community started to create official docker image for
> Zeppelin. As of 0.7.2, release process includes building docker image.
> Thus, every release can ship its own docker image.
>
> You can test the docker image for 0.7.2 with this command.
>
> *docker run -p 8080:8080 --rm --name zeppelin apache/zeppelin:0.7.2*
>
> or you can set docker volume options to persist notebooks and logs like
>
> *docker run -p 8080:8080 --rm -v $PWD/logs:/logs -v
> $PWD/notebook:/notebook -e ZEPPELIN_LOG_DIR='/logs' -e
> ZEPPELIN_NOTEBOOK_DIR='/notebook' --name zeppelin apache/zeppelin:0.7.2*
>
> And here are few links which might be helpful
>
> - https://hub.docker.com/r/apache/zeppelin/
> - https://github.com/apache/zeppelin/blob/master/docs/install/docker.md
>
> Since this is an initial version, it might include some bugs or be not
> easy to configure some options.
>
> Hope that we can improve the docker file and deliver flexible images to
> the community for various environments and use cases.
>
> Thanks.
>
>
> FYI, if you are having trouble with access localhost:8080, try to clear
> browser caches like
>
> [image: Inline image 1]
>
> On Fri, Jun 16, 2017 at 2:22 PM, Park Hoon <1am...@gmail.com> wrote:
>
>> Hi, users.
>>
>> A few months ago the community started to create official docker image
>> for Zeppelin. As of 0.7.2, release process includes building docker image.
>> Thus, every release can ship its own docker image.
>>
>> You can test the docker image for 0.7.2 with this command.
>>
>> *docker run -p 8080:8080 --rm --name zeppelin apache/zeppelin:0.7.2*
>>
>> or you can set docker volume options to persist notebooks and logs like
>>
>> *docker run -p 8080:8080 --rm -v $PWD/logs:/logs -v
>> $PWD/notebook:/notebook -e ZEPPELIN_LOG_DIR='/logs' -e
>> ZEPPELIN_NOTEBOOK_DIR='/notebook' --name zeppelin apache/zeppelin:0.7.2*
>>
>> And here are few links which might be helpful
>>
>> - https://hub.docker.com/r/apache/zeppelin/
>> - https://github.com/apache/zeppelin/blob/master/docs/install/docker.md
>>
>> Since this is an initial version, it might include some bugs or be not
>> easy to configure some options.
>>
>> Hope that we can improve the docker file and deliver flexible images to
>> the community for various environments and use cases.
>>
>> Thanks.
>>
>>
>> FYI, if you are having trouble with access localhost:8080, try to clear
>> browser caches like
>>
>> [image: Inline image 1]
>>
>
>

Re: [VOTE] Release Apache Zeppelin 0.7.2 (RC1)

[moon soo Lee]

Verified

  - Build from source
  - Binary package runs tutorial notebook
  - Build with JDK1.7
  - Signature

+1 for releasing this RC.

Thanks,
moon

On Fri, Jun 9, 2017 at 12:14 AM Mina Lee  wrote:

> I propose the following RC to be released for the Apache Zeppelin 0.7.2
> release.
>
> The commit id is 467ed12cc87100ce2f2bc5a94199fabf57f6bb2a which is
> corresponds to the tag v0.7.2-rc1:
>
> https://git-wip-us.apache.org/repos/asf?p=zeppelin.git;h=refs/tags/v0.7.2-rc1
>
> The release archives (tgz), signature, and checksums are here
> https://dist.apache.org/repos/dist/dev/zeppelin/zeppelin-0.7.2-rc1/
>
> The release candidate consists of the following source distribution archive
> zeppelin-0.7.2.tgz
>
> In addition, the following supplementary binary distributions are provided
> for user convenience at the same location
> zeppelin-0.7.2-bin-all.tgz
> zeppelin-0.7.2-bin-netinst.tgz
>
> The maven artifacts are here
> https://repository.apache.org/content/repositories/orgapachezeppelin-1042
>
> You can find the KEYS file here:
> https://dist.apache.org/repos/dist/release/zeppelin/KEYS
>
> Release notes available at
>
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12340231=12316221
>
> Vote will be open for next 72 hours (close at 01:00 12/Jun PDT).
>
> [ ] +1 approve
> [ ] 0 no opinion
> [ ] -1 disapprove (and reason why)
>

Re: How to change the python installation location for Python interpreter?

[moon soo Lee]

Hi,

You can set your python executable path in 'zeppelin.python' property,
Python interpreter setting in the interpreter menu.

If your system has conda installed, then you can manage environment using
%python.conda. Please take a look
http://zeppelin.apache.org/docs/latest/interpreter/python.html#conda

Thanks,
moon

On Sat, May 27, 2017 at 1:59 PM BigData Consultant <
bigdata.consultant2...@gmail.com> wrote:

> Hi Team,
>
> Where is the python installation location specified for Python interpreter?
>
> How to change the python installation location for Python interpreter with
> new python installation directory?
>
> Help would be much appreciated.
>
> Thanks.
>

Re: What is the user case for remote of interpreter option is false

[moon soo Lee]

The option is legacy and being used in few unit tests as far as i remember.

I think we can either try completely remove this code (to keep code base
simple) or we can try re-introduce this feature (to optimize resource
usage, less restriction on data sharing between interpreters).

I think interpreters like markdown, jdbc will have no big problem with
remote=false while Spark interpreter require separate JVM and remote=true
is required.

remote=false will provide some chances to save some system resource because
it does not run separate JVM process for interpreter.

Also communication between interpreter (through ResourcePool) can be much
faster and have less restricted on object serialization.

So remote=false option definitely provides some advantages.
However, keeping code base for remote=false and remote=true introduces more
work.

What do you think?

Thanks,
moon

On Mon, May 8, 2017 at 8:13 PM Jeff Zhang  wrote:

> I see all the interpreter setting set remote of interpreter option as true,
> and didn't find any false case in interpreter-setting.json, does anyone
> know what is the use case of remote of interpreter option is false. Is this
> a legacy mode that should be removed ?
>
>
> https://github.com/apache/zeppelin/blob/master/zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/InterpreterSettingManager.java#L183
>

Re: Windows Build Support

[moon soo Lee]

Okay, then let me file an INFRA issue to turn off appveyor on pullrequest
build, but keeping it for branch build.

On Tue, May 2, 2017 at 9:39 PM Jeff Zhang <zjf...@gmail.com> wrote:

> Make sense for option 2
>
>
> moon soo Lee <m...@apache.org>于2017年5月3日周三 上午9:21写道：
>
> > Moving thread to dev@
> >
> > Currently CI on pullrequest is
> >
> >  - Individual contributor runs Travis - CI
> >  - ASF's Jenkins connected to Apache Zeppelin github repo, and get
> > information from each contributor's Travis build.
> >  - ASF's Appveyor build each pullrequest.
> >
> > CI on branch (master, branch-0.7, ...) is
> >  - ASF's Jenkins build new commits on branch
> >  - ASF's Appveyor build new commits on branch
> >
> >
> > Since Appveyor is enabled, I can see Zeppelin PR build on appveyor waits
> > very long time in the queue. I think we can have several options on it.
> >
> > 1. Keep it as is,
> >
> > Pros - Each pullrequest will be verified on windows environment before
> > merge
> > Cons - Long wait to get a slot in Appveyor
> >
> > 2. Disable Appveyor on Pullreuqest build, keep enable Appveyor on branch
> > build.
> >
> > Pros - Each pullrequest will not wait long time for Appveyor finish the
> > job.
> > Cons - If somethings broken on windows, we'll know after PR is merged.
> >
> > 3. Let individual contributor setup Appveyor and make Jenkins check it,
> > like we do it with Travis
> >
> > Pros - Both Travis, Appveyor test can be finished quickly on each PR
> > Cons - Each individual contributor need to setup both Travis, Appveyor.
> >
> >
> > To me, 2) looks reasonable because most of PR are not related to the
> > windows environment, so recognizing some PR break windows support after
> > merge will be rare event. And of course we can just submit another patch
> to
> > fix that.
> >
> > 3) is little bit too much for me, asking all contributors setup both
> travis
> > and appveyor.
> >
> > What do you guys think?
> >
> > Thanks,
> > moon
> >
> > On Wed, Apr 26, 2017 at 6:31 PM moon soo Lee <m...@apache.org> wrote:
> >
> > > Filed an infra issue to enable appveyor support.
> > > https://issues.apache.org/jira/browse/INFRA-14019
> > >
> > > Thanks,
> > > moon
> > >
> > >
> > > On Fri, Apr 21, 2017 at 2:22 AM Park Hoon <1am...@gmail.com> wrote:
> > >
> > >> Thanks for reply @moon.
> > >>
> > >>
> > >> Additionally, i create umbrella issue for windows support
> > >>
> > >> - https://issues.apache.org/jira/browse/ZEPPELIN-2440
> > >>
> > >>
> > >> Regard,
> > >>
> > >> Hoon Park
> > >>
> > >> On Thu, Apr 20, 2017 at 12:28 PM, moon soo Lee <m...@apache.org>
> wrote:
> > >>
> > >>> Hi,
> > >>>
> > >>> Thanks for staring the thread.
> > >>>
> > >>> I've seen almost every month people report build broken on windows.
> > >>> Windows CI env will definitely help solving the problem.
> > >>>
> > >>> I'm +1 on setting up appveyor for the project.
> > >>>
> > >>> Due to very high usage on ASF's travis account, Zeppelin project
> guide
> > >>> each contributor setup travis account and check success/fail using
> > Jenkins.
> > >>>
> > >>> I don't know how busy ASF's appveyor account. But I think we can
> first
> > >>> try just turn on appveyor with ASF's account and see how busy it is.
> > >>>
> > >>> Thanks,
> > >>> moon
> > >>>
> > >>>
> > >>> On Wed, Apr 19, 2017 at 5:15 AM Park Hoon <1am...@gmail.com> wrote:
> > >>>
> > >>>> Hi all,
> > >>>>
> > >>>> Some people are struggling with build Zeppelin in windows.
> > >>>>
> > >>>> -
> > >>>>
> >
> http://stackoverflow.com/questions/40498638/install-zeppelin-on-windows/42560442#42560442
> > >>>>
> > >>>> IMO, it's enough to verify that zeppelin works well on windows.
> > >>>>
> > >>>> But it's not that hard to setup CI for windows and it's possible to
> > >>>> execute integration tests on windows, it's useful. As you know, we
> > have
> > >>>> some issues related with windows env
> > >>>>
> > >>>> -
> > >>>>
> >
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20ZEPPELIN%20and%20text%20~%20windows
> > >>>>
> > >>>> I found that some apache projects already use appveyor to run CI on
> > >>>> windows. (sadly, travis doesn't support)
> > >>>>
> > >>>> - (thrift)
> https://github.com/apache/thrift/blob/master/appveyor.yml
> > >>>> - (arrow) https://issues.apache.org/jira/browse/INFRA-12967
> > >>>>
> > >>>>
> > >>>>
> > >>>> Thanks.
> > >>>>
> > >>>>
> > >>>>
> > >>
> >
>

Re: Windows Build Support

[moon soo Lee]

Moving thread to dev@

Currently CI on pullrequest is

 - Individual contributor runs Travis - CI
 - ASF's Jenkins connected to Apache Zeppelin github repo, and get
information from each contributor's Travis build.
 - ASF's Appveyor build each pullrequest.

CI on branch (master, branch-0.7, ...) is
 - ASF's Jenkins build new commits on branch
 - ASF's Appveyor build new commits on branch


Since Appveyor is enabled, I can see Zeppelin PR build on appveyor waits
very long time in the queue. I think we can have several options on it.

1. Keep it as is,

Pros - Each pullrequest will be verified on windows environment before merge
Cons - Long wait to get a slot in Appveyor

2. Disable Appveyor on Pullreuqest build, keep enable Appveyor on branch
build.

Pros - Each pullrequest will not wait long time for Appveyor finish the job.
Cons - If somethings broken on windows, we'll know after PR is merged.

3. Let individual contributor setup Appveyor and make Jenkins check it,
like we do it with Travis

Pros - Both Travis, Appveyor test can be finished quickly on each PR
Cons - Each individual contributor need to setup both Travis, Appveyor.


To me, 2) looks reasonable because most of PR are not related to the
windows environment, so recognizing some PR break windows support after
merge will be rare event. And of course we can just submit another patch to
fix that.

3) is little bit too much for me, asking all contributors setup both travis
and appveyor.

What do you guys think?

Thanks,
moon

On Wed, Apr 26, 2017 at 6:31 PM moon soo Lee <m...@apache.org> wrote:

> Filed an infra issue to enable appveyor support.
> https://issues.apache.org/jira/browse/INFRA-14019
>
> Thanks,
> moon
>
>
> On Fri, Apr 21, 2017 at 2:22 AM Park Hoon <1am...@gmail.com> wrote:
>
>> Thanks for reply @moon.
>>
>>
>> Additionally, i create umbrella issue for windows support
>>
>> - https://issues.apache.org/jira/browse/ZEPPELIN-2440
>>
>>
>> Regard,
>>
>> Hoon Park
>>
>> On Thu, Apr 20, 2017 at 12:28 PM, moon soo Lee <m...@apache.org> wrote:
>>
>>> Hi,
>>>
>>> Thanks for staring the thread.
>>>
>>> I've seen almost every month people report build broken on windows.
>>> Windows CI env will definitely help solving the problem.
>>>
>>> I'm +1 on setting up appveyor for the project.
>>>
>>> Due to very high usage on ASF's travis account, Zeppelin project guide
>>> each contributor setup travis account and check success/fail using Jenkins.
>>>
>>> I don't know how busy ASF's appveyor account. But I think we can first
>>> try just turn on appveyor with ASF's account and see how busy it is.
>>>
>>> Thanks,
>>> moon
>>>
>>>
>>> On Wed, Apr 19, 2017 at 5:15 AM Park Hoon <1am...@gmail.com> wrote:
>>>
>>>> Hi all,
>>>>
>>>> Some people are struggling with build Zeppelin in windows.
>>>>
>>>> -
>>>> http://stackoverflow.com/questions/40498638/install-zeppelin-on-windows/42560442#42560442
>>>>
>>>> IMO, it's enough to verify that zeppelin works well on windows.
>>>>
>>>> But it's not that hard to setup CI for windows and it's possible to
>>>> execute integration tests on windows, it's useful. As you know, we have
>>>> some issues related with windows env
>>>>
>>>> -
>>>> https://issues.apache.org/jira/issues/?jql=project%20%3D%20ZEPPELIN%20and%20text%20~%20windows
>>>>
>>>> I found that some apache projects already use appveyor to run CI on
>>>> windows. (sadly, travis doesn't support)
>>>>
>>>> - (thrift) https://github.com/apache/thrift/blob/master/appveyor.yml
>>>> - (arrow) https://issues.apache.org/jira/browse/INFRA-12967
>>>>
>>>>
>>>>
>>>> Thanks.
>>>>
>>>>
>>>>
>>

Re: Travis jobs

[moon soo Lee]

I think it's more beneficial to include core tests with 6 test disabled,
compare to all test disabled. Later we can fix failed test one by one.

Thanks,
moon

On Thu, Apr 27, 2017 at 8:18 AM Alexander Shoshin <
alexander_shos...@epam.com> wrote:

> The problem is that some core tests fail at the moment. I had disabled 6
> test classes when I was trying to receive "green" travis build after
> enabling core tests. So we can't just enable all tests - it will brake the
> travis job. And fixing all tests errors can take a lot of time.
>
> Regards,
> Alexander
>
> -----Original Message-
> From: moon soo Lee [mailto:m...@apache.org]
> Sent: Thursday, April 27, 2017 12:07 AM
> To: dev@zeppelin.apache.org
> Subject: Re: Travis jobs
>
> Right, seconds travis job seems doing nothing.
> Second travis job supposed to test all core unit tests plus integration
> test (except for spark related test).
> Could you help fix this problem?
>
> Best,
> moon
>
> On Tue, Apr 25, 2017 at 1:04 AM Alexander Shoshin <
> alexander_shos...@epam.com> wrote:
>
> > Hi devs!
> >
> > Does anybody know what is the purpose of the second Travis job?
> >
> > I thought it should run all core unit tests but I found that it
> > doesn't run anything. Actually it runs 5 integration tests from
> > org.apache.zeppelin.integration package but all of them are skipped
> > because of TEST_SELENIUM=false. As a result we have zeppelin-server
> > unit tests which don't used at all (in any of Travis jobs).
> >
> > I guess that it is because of the exclamation mark in "-Dtest" property:
> > -Dtest='!ZeppelinSparkClusterTest,!org.apache.zeppelin.spark.*'
> > which is not supported by maven-surefire-plugin of version 2.17 (which
> > is used in Zeppelin). Exclamation mark is supported started from 2.19
> > but still does not work properly. Instead of this we can use surefire
> "exclude"
> > property in pom.xml by passing parameter from command line:
> >
> > .travis.yaml:
> > -Dtests.to.exclude='ZeppelinSparkClusterTest,org.apache.zeppelin.spark.*'
> > pom.xml: ${tests.to.exclude}
> >
> > So should we run all core unit tests in the second Travis job or not?
> >
> > Regards,
> > Alexander
> >
>

Re: Travis jobs

[moon soo Lee]

Right, seconds travis job seems doing nothing.
Second travis job supposed to test all core unit tests plus integration
test (except for spark related test).
Could you help fix this problem?

Best,
moon

On Tue, Apr 25, 2017 at 1:04 AM Alexander Shoshin <
alexander_shos...@epam.com> wrote:

> Hi devs!
>
> Does anybody know what is the purpose of the second Travis job?
>
> I thought it should run all core unit tests but I found that it doesn't
> run anything. Actually it runs 5 integration tests from
> org.apache.zeppelin.integration package but all of them are skipped because
> of TEST_SELENIUM=false. As a result we have zeppelin-server unit tests
> which don't used at all (in any of Travis jobs).
>
> I guess that it is because of the exclamation mark in "-Dtest" property:
> -Dtest='!ZeppelinSparkClusterTest,!org.apache.zeppelin.spark.*'
> which is not supported by maven-surefire-plugin of version 2.17 (which is
> used in Zeppelin). Exclamation mark is supported started from 2.19 but
> still does not work properly. Instead of this we can use surefire "exclude"
> property in pom.xml by passing parameter from command line:
>
> .travis.yaml:
> -Dtests.to.exclude='ZeppelinSparkClusterTest,org.apache.zeppelin.spark.*'
> pom.xml: ${tests.to.exclude}
>
> So should we run all core unit tests in the second Travis job or not?
>
> Regards,
> Alexander
>

Re: CI status is lost in PR page

[moon soo Lee]

Yes, somehow, PR page does not display CI status. I don't know exact
reason. I filed an INFRA issue
https://issues.apache.org/jira/browse/INFRA-14005.

On Tue, Apr 25, 2017 at 10:42 PM Jeff Zhang  wrote:

> I didn't see the CI status in the PR page recently, I need to check travis
> manually. Does anyone experience the same thing ?
>
>
>
> [image: image.png]
>

Re: shiro.ini : Zeppelin interpreter authorization: lock to one user

[moon soo Lee]

Hi,

https://issues.apache.org/jira/browse/ZEPPELIN-2236 tracks restricting
interpreter setting to admin group. And we're trying to solve this problem
in 0.8.0 [1]. Please keep an eye on this issue and feel free to make any
contributions.

Thanks,
moon

[1]
https://lists.apache.org/thread.html/f4be5635e6e72d2a9dc23771ceeaef8a8e5933bfb14db7cab303bc22@%3Cdev.zeppelin.apache.org%3E


On Wed, Apr 12, 2017 at 6:48 PM Arun Khetarpal  wrote:

> Hi -
>
> I think the ask is that currently it seems that activeDirectoryGroupRealm
> maps each active directory group to a shiro role. Later we can authorize
> urls with roles in our urls section of shiro file.
>
> Our problem is that we don't really have an admin group handy in our AD -
> due to which there is a bit of a problem in setting the correct
> authorization of some urls. We were able to play around with
> ActiveDirectoryGroupRealm and modify in such a way that an internal role is
> created for systemUser and then authorize urls for him based on the
> internal role.
>
> Later we placed this modified ActiveDirectoryGroupRealm in Class path and
> everything works as expected. So my question is -
>
> a. Is there a better way provided by zeppelin for users to provide their
> own implementation of ActiveDirectoryGroupRealm ?
> b. Is the community interested in modifying ActiveDirectoryGroupRealm with
> our tweaks ?
>
>
> Regards,
> Arun
>
>
>
>
> On 12 April 2017 at 15:55, Vartul Gupta 
> wrote:
>
> > Hi Team,
> >
> > I am trying to setup authentication and authorization for Zeppelin using
> > Shiro file.
> > Now my question is How we can set Zeppelin interpreter settings to be
> > restricted to a particular AD admin user.
> >
> > Here is the sample of my shiro file .
> >
> > [main]
> > activeDirectoryRealm =
> org.apache.zeppelin.realm.ActiveDirectoryGroupRealm
> > activeDirectoryRealm.systemUsername = adminuser1
> > activeDirectoryRealm.systemPassword = ^^
> > #activeDirectoryRealm.hadoopSecurityCredentialPath =
> > jceks://user/zeppelin/zeppelin.jceks
> > activeDirectoryRealm.searchBase = "OU=AADDC
> Users,DC=team2testdomain,DC=testabc
> > ,DC=com"
> > activeDirectoryRealm.url = ldaps://team2testdomain.testabc.com:636
> > activeDirectoryRealm.principalSuffix = @TEAM2TESTDOMAIN.testabc.COM
> > activeDirectoryRealm.groupRolesMap = "OU=AADDC
> > Users,DC=team2testdomain,DC= testabc,DC=com":"admin"
> > activeDirectoryRealm.authorizationCachingEnabled = true
> > sessionManager =
> org.apache.shiro.web.session.mgt.DefaultWebSessionManager
> > cacheManager = org.apache.shiro.cache.MemoryConstrainedCacheManager
> > securityManager.cacheManager = $cacheManager
> > securityManager.sessionManager = $sessionManager
> > # 86,400,000 milliseconds = 24 hour
> > securityManager.sessionManager.globalSessionTimeout = 8640
> > shiro.loginUrl = /api/login
> >
> > [urls]
> > # anon means the access is anonymous.
> > # authcBasic means Basic Auth Security
> > # To enfore security, comment the line below and uncomment the next one
> > #/** = anon
> > /** = authc
> >
> > Thanks
> > Vartul
> >
>

Re: Access to Spark Context from a Custom Interpreter

[moon soo Lee]

Hi Michael,

Interpreter does not close / open between paragraphs. That means you can
just keep any object as member variable in your interpreter implementation.

For example SparkInterpreter [1] keeps IMain as a member variable.

Interpreter shipped in Zeppelin codebase is developed with the same API
that custom interpreter uses. They have no differences.

Hope this helps!

Thanks,
moon

[1]
https://github.com/apache/zeppelin/blob/v0.7.1/spark/src/main/java/org/apache/zeppelin/spark/SparkInterpreter.java#L102

On Thu, Apr 20, 2017 at 9:42 AM Michael Marconi  wrote:

> Hi – I'd like to know if it's possible to access the SparkContext from a
> Custom Interpreter I am developing.  I need access to DataFrames created in
> earlier paragraphs, so I can transform them and make the results available
> to subsequent paragraphs.
>
> I was expecting to find the Spark Context on the Interpreter Context that
> is handed to my Custom Interpreter but this is not the case.
>
> I've come across this similar request StackOverflow but it doesn't answer
> the question:
> http://stackoverflow.com/questions/37099590/zeppelin-how-to-create-dataframe-from-within-custom-interpreter
>
> Can you advise whether this is possible with an externally-contributed
> Custom Interpreter or would I need to fork the Zeppelin codebase and pull a
> new interpreter into the Spark package?
>
> Many thanks,
> Michael
>

Re: [VOTE] Release Apache Zeppelin 0.7.1 (RC3)

[moon soo Lee]

Verified

 - signature
 - no unexpected binary in source package
 - build from source
 - basic features on binary package

I've found a problem that Note is refreshing when NoteList broadcast (e.g.
Note creation, deletion, rename). Explained the problem in the issue
https://issues.apache.org/jira/browse/ZEPPELIN-2336.

Hope this can be fixed in 0.7.2.

+1.

On Thu, Mar 30, 2017 at 9:35 PM Felix Cheung 
wrote:

> +1
>
> Tested bin, checked signature, ran through a bunch of tests, checked
> report mode, UI etc
>
> I don't think it's a blocker:
> When I search I see a list of note names but with empty white boxes under
> each of them. Isn't it suppose to show where the match is found?
>
> 
> From: Khalid Huseynov 
> Sent: Thursday, March 30, 2017 9:27:01 PM
> To: dev@zeppelin.apache.org
> Subject: Re: Re: [VOTE] Release Apache Zeppelin 0.7.1 (RC3)
>
> +1
>
> On Fri, Mar 31, 2017 at 12:32 PM, Ahyoung Ryu  wrote:
>
> > +1
> > tested binary package
> >  - Spark and Python tutorial note
> >  - Zombie process issue
> >
> > and could successfully build Zeppelin from source.
> > Thanks always Mina for your effort.
> >
> >
> > On Fri, Mar 31, 2017 at 12:20 PM, Anthony Corbacho <
> > anthonycorba...@apache.org> wrote:
> >
> > > +1
> > >
> > > Tested with my usual cases.
> > >
> > > Thank you mina!!!
> > >
> > > On Fri, Mar 31, 2017 at 12:13 PM, Jun Kim  wrote:
> > >
> > > > +1
> > > > 2017년 3월 31일 (금) 오후 12:13, Jongyoul Lee 님이 작성:
> > > >
> > > > > +1
> > > > >
> > > > > tested about zombie processes
> > > > >
> > > > >
> > > > > On Fri, 31 Mar 2017 at 12:03 CloverHeartsDev <
> > > cloverhearts...@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > +1
> > > > > >
> > > > > > -Original Message-
> > > > > > From: Windy Qin [mailto:windyqinchaof...@163.com]
> > > > > > Sent: Friday, March 31, 2017 11:56 AM
> > > > > > To: dev 
> > > > > > Subject: Re: Re: [VOTE] Release Apache Zeppelin 0.7.1 (RC3)
> > > > > >
> > > > > > +1
> > > > > >
> > > > > >
> > > > > >
> > > > > > Thanks
> > > > > > WindyQin
> > > > > >
> > > > > > From: Mina Lee
> > > > > > Date: 2017-03-31 09:46
> > > > > > To: dev
> > > > > > Subject: Re: [VOTE] Release Apache Zeppelin 0.7.1 (RC3)
> > > > > > +1
> > > > > >
> > > > > > On Wed, Mar 29, 2017 at 4:39 PM Hyung Sung Shim <
> hss...@nflabs.com
> > >
> > > > > wrote:
> > > > > >
> > > > > > > +1
> > > > > > > Tested tutorials of zeppelin-0.7.1-bin-all on CentOS7.
> > > > > > > Thank you for your effort mina!
> > > > > > >
> > > > > > > 2017-03-29 16:04 GMT+09:00 DuyHai Doan :
> > > > > > >
> > > > > > > > +1
> > > > > > > > Le 29 mars 2017 07:06, "Jeff Zhang"  a
> > écrit :
> > > > > > > >
> > > > > > > > > Thanks Jongyoul for explanation, let's improve it later.
> > > > > > > > >
> > > > > > > > > +1.
> > > > > > > > >
> > > > > > > > > * Build zeppelin from source successfully
> > > > > > > > > * Run pig tutorial using tez mode successfully
> > > > > > > > > * Run spark tutorial using livy interpreter successfully.
> > > > > > > > > * Verify interpreter fail error can be propagated to
> > frontend.
> > > > > > > > > * Verify the restart button works in several scenarios.
> Works
> > > > > > properly.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > Jongyoul Lee 于2017年3月29日周三 上午11:37写道：
> > > > > > > > >
> > > > > > > > > > Jeff,
> > > > > > > > > >
> > > > > > > > > > Moon and I discuss the behaviors of clicking restart
> > buttons
> > > > > > > > > > from
> > > > > > > > > notebook
> > > > > > > > > > page and interpreter page. In your case, it's intended
> > fully.
> > > > > > > > > > For
> > > > > > > more
> > > > > > > > > > detail, In case of "per user", we can do the same thing
> in
> > > note
> > > > > > > > > > and interpreter pages, but in case of "per note", we
> cannot
> > > get
> > > > > > > > > > the exact
> > > > > > > > > note
> > > > > > > > > > when users click restart button in interpreter tab. Thus
> > Moon
> > > > > > > > > > and I
> > > > > > > > > decided
> > > > > > > > > > not to change original behaviors of interpreter page. It
> > > would
> > > > > > > > > > be
> > > > > > > > changed
> > > > > > > > > > when we add admin feature.
> > > > > > > > > >
> > > > > > > > > > Thanks for testing it.
> > > > > > > > > >
> > > > > > > > > > Regards,
> > > > > > > > > > Jongyoul
> > > > > > > > > >
> > > > > > > > > > On Wed, Mar 29, 2017 at 11:47 AM, Jeff Zhang <
> > > zjf...@gmail.com
> > > > >
> > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > Might find another bug about restarting button. The
> > > > restarting
> > > > > > > button
> > > > > > > > > on
> > > > > > > > > > > note page behaves differently from that of interpreter
> > > page.
> > > > > > > > > > > This
> > > > > > > > > might
> > > > > 

Re: Roadmap for 0.8.0

[moon soo Lee]

And if i suggest simplest way for us to set quality expectation to user,
which will be labeling release in download page.

Currently releases are divided into 2 categories in download page. 'Latest
release' and 'Old releases'. I think we can treat 'Latest' as unstable and
add one more category 'Stable release'.

For example, once 0.8.0 is released,

Latest release : 0.8.0
Stable release : 0.7.1
Old release : 0.6.2, 0.6.1 

Once we feel confident about the stability of latest release, we can just
change label from latest to stable in the download page. (and previous
stable goes to old releases)
We can even include formal vote for moving release from 'latest' to
'stable' in our release process, if it is necessary.

Thanks,
moon

On Tue, Mar 21, 2017 at 6:59 AM moon soo Lee <m...@apache.org> wrote:

> Yes, having longer RC period will help.
>
> But if i recall 0.7.0 release, although 21 people participated verifying
> through 4 RC for 15days, it wasn't enough to catch all critical problems
> during the release process. After the release, we've got much more number
> of bug reports, in next few days.
>
> Basically, verifying RC is limited to people who subscribe mailing list +
> willing to contribute time to verify RC, which is much smaller number of
> people who download release from download page. So having longer RC period
> will definitely help and i think we should do, but I think it's still not
> enough to make sure the quality, considering past history.
>
> AFAIK, releasing 0.8.0-preview, calling it unstable is up to the project.
> ASF release process defines how to release source code, but it does not
> really restrict what kind of 'version' the project should have releases.
> For example, spark released spark-2.0.0-preview[1] before spark-2.0.0.
>
> Thanks,
> moon
>
> [1] http://spark.apache.org/news/spark-2.0.0-preview.html
>
>
> On Mon, Mar 20, 2017 at 11:31 PM Jongyoul Lee <jongy...@gmail.com> wrote:
>
> I agree that it will help prolong RC period and use it actually. And also
> we need code freeze for the new features and spend time to stabilize RC.
>
> On Tue, Mar 21, 2017 at 1:25 PM, Felix Cheung <felixcheun...@hotmail.com>
> wrote:
>
> +1 on quality and stabilization.
>
> I'm not sure if releasing as preview or calling it unstable fits with the
> ASF release process though.
>
> Other projects have code freeze, RC (and longer RC iteration time) etc. -
> do we think those will help improve quality when the release is finally cut?
>
>
> _
> From: Jianfeng (Jeff) Zhang <jzh...@hortonworks.com>
> Sent: Monday, March 20, 2017 6:13 PM
> Subject: Re: Roadmap for 0.8.0
> To: <us...@zeppelin.apache.org>, dev <dev@zeppelin.apache.org>
>
>
>
> Strongly +1 for adding system test for different interpreter modes and
> focus on bug fixing than new features. I do heard from some users complain
> about the bugs of zeppelin major release. A stabilized release is very
> necessary for community.
>
>
>
>
> Best Regard,
> Jeff Zhang
>
>
> From: moon soo Lee <m...@apache.org<mailto:m...@apache.org
> <m...@apache.org>>>
> Reply-To: "us...@zeppelin.apache.org<mailto:us...@zeppelin.apache.org
> <us...@zeppelin.apache.org>>" <us...@zeppelin.apache.org<
> mailto:us...@zeppelin.apache.org <us...@zeppelin.apache.org>>>
> Date: Tuesday, March 21, 2017 at 4:10 AM
> To: "us...@zeppelin.apache.org<mailto:us...@zeppelin.apache.org
> <us...@zeppelin.apache.org>>" <us...@zeppelin.apache.org<
> mailto:us...@zeppelin.apache.org <us...@zeppelin.apache.org>>>, dev <
> dev@zeppelin.apache.org<mailto:dev@zeppelin.apache.org
> <dev@zeppelin.apache.org>>>
>
> Subject: Re: Roadmap for 0.8.0
>
> Great to see discussion for 0.8.0.
> List of features for 0.8.0 looks really good.
>
> Interpreter factory refactoring
> Interpreter layer supports various behavior depends on combination of
> PerNote,PerUser / Shared,Scoped,Isolated. We'll need strong test cases for
> each combination as a first step.
> Otherwise, any pullrequest will silently break one of behavior at any time
> no matter we refactor or not. And fixing and testing this behavior is so
> hard.
> Once we have complete test cases, not only guarantee the behavior but also
> make refactoring much easier.
>
>
> 0.8.0 release
> I'd like to suggest improvements on how we release a new version.
>
> In the past, 0.6.0 and 0.7.0 release with some critical problems. (took 3
> months to stabilize 0.6 and we're working on stabilizing 0.7.0 for 2 months)
>
> I think the same thing will happen again with 0.8.0, while we're going to

Re: Roadmap for 0.8.0

[moon soo Lee]

Yes, having longer RC period will help.

But if i recall 0.7.0 release, although 21 people participated verifying
through 4 RC for 15days, it wasn't enough to catch all critical problems
during the release process. After the release, we've got much more number
of bug reports, in next few days.

Basically, verifying RC is limited to people who subscribe mailing list +
willing to contribute time to verify RC, which is much smaller number of
people who download release from download page. So having longer RC period
will definitely help and i think we should do, but I think it's still not
enough to make sure the quality, considering past history.

AFAIK, releasing 0.8.0-preview, calling it unstable is up to the project.
ASF release process defines how to release source code, but it does not
really restrict what kind of 'version' the project should have releases.
For example, spark released spark-2.0.0-preview[1] before spark-2.0.0.

Thanks,
moon

[1] http://spark.apache.org/news/spark-2.0.0-preview.html


On Mon, Mar 20, 2017 at 11:31 PM Jongyoul Lee <jongy...@gmail.com> wrote:

> I agree that it will help prolong RC period and use it actually. And also
> we need code freeze for the new features and spend time to stabilize RC.
>
> On Tue, Mar 21, 2017 at 1:25 PM, Felix Cheung <felixcheun...@hotmail.com>
> wrote:
>
> +1 on quality and stabilization.
>
> I'm not sure if releasing as preview or calling it unstable fits with the
> ASF release process though.
>
> Other projects have code freeze, RC (and longer RC iteration time) etc. -
> do we think those will help improve quality when the release is finally cut?
>
>
> _
> From: Jianfeng (Jeff) Zhang <jzh...@hortonworks.com>
> Sent: Monday, March 20, 2017 6:13 PM
> Subject: Re: Roadmap for 0.8.0
> To: <us...@zeppelin.apache.org>, dev <dev@zeppelin.apache.org>
>
>
>
> Strongly +1 for adding system test for different interpreter modes and
> focus on bug fixing than new features. I do heard from some users complain
> about the bugs of zeppelin major release. A stabilized release is very
> necessary for community.
>
>
>
>
> Best Regard,
> Jeff Zhang
>
>
> From: moon soo Lee <m...@apache.org<mailto:m...@apache.org
> <m...@apache.org>>>
> Reply-To: "us...@zeppelin.apache.org<mailto:us...@zeppelin.apache.org
> <us...@zeppelin.apache.org>>" <us...@zeppelin.apache.org<
> mailto:us...@zeppelin.apache.org <us...@zeppelin.apache.org>>>
> Date: Tuesday, March 21, 2017 at 4:10 AM
> To: "us...@zeppelin.apache.org<mailto:us...@zeppelin.apache.org
> <us...@zeppelin.apache.org>>" <us...@zeppelin.apache.org<
> mailto:us...@zeppelin.apache.org <us...@zeppelin.apache.org>>>, dev <
> dev@zeppelin.apache.org<mailto:dev@zeppelin.apache.org
> <dev@zeppelin.apache.org>>>
>
> Subject: Re: Roadmap for 0.8.0
>
> Great to see discussion for 0.8.0.
> List of features for 0.8.0 looks really good.
>
> Interpreter factory refactoring
> Interpreter layer supports various behavior depends on combination of
> PerNote,PerUser / Shared,Scoped,Isolated. We'll need strong test cases for
> each combination as a first step.
> Otherwise, any pullrequest will silently break one of behavior at any time
> no matter we refactor or not. And fixing and testing this behavior is so
> hard.
> Once we have complete test cases, not only guarantee the behavior but also
> make refactoring much easier.
>
>
> 0.8.0 release
> I'd like to suggest improvements on how we release a new version.
>
> In the past, 0.6.0 and 0.7.0 release with some critical problems. (took 3
> months to stabilize 0.6 and we're working on stabilizing 0.7.0 for 2 months)
>
> I think the same thing will happen again with 0.8.0, while we're going to
> make lots of changes and add many new features.
> After we released 0.8.0, while 'Stabilizing' the new release, user who
> tried the new release may get wrong impression of the quality. Which is
> very bad and we already repeated the mistake in 0.6.0 and 0.7.0.
>
> So from 0.8.0 release, I'd suggest we improve way we release new version
> to give user proper expectation. I think there're several ways of doing it.
>
> 1. Release 0.8.0-preview officially and then release 0.8.0.
> 2. Release 0.8.0 with 'beta' or 'unstable' label. And keep 0.7.x as a
> 'stable' release in the download page. Once 0.8.x release becomes stable
> enough make 0.8.x release as a 'stable' and move 0.7.x to 'old' releases.
>
>
> After 0.8.0,
> Since Zeppelin projects starts, project went through some major milestone,
> like
>
> - project gets first users and first contributor
> - project went

Re: Roadmap for 0.8.0

[moon soo Lee]

Great to see discussion for 0.8.0.
List of features for 0.8.0 looks really good.

*Interpreter factory refactoring*
Interpreter layer supports various behavior depends on combination of
PerNote,PerUser / Shared,Scoped,Isolated. We'll need strong test cases for
each combination as a first step.
Otherwise, any pullrequest will silently break one of behavior at any time
no matter we refactor or not. And fixing and testing this behavior is so
hard.
Once we have complete test cases, not only guarantee the behavior but also
make refactoring much easier.


*0.8.0 release*
I'd like to suggest improvements on how we release a new version.

In the past, 0.6.0 and 0.7.0 release with some critical problems. (took 3
months to stabilize 0.6 and we're working on stabilizing 0.7.0 for 2 months)

I think the same thing will happen again with 0.8.0, while we're going to
make lots of changes and add many new features.
After we released 0.8.0, while 'Stabilizing' the new release, user who
tried the new release may get wrong impression of the quality. Which is
very bad and we already repeated the mistake in 0.6.0 and 0.7.0.

So from 0.8.0 release, I'd suggest we improve way we release new version to
give user proper expectation. I think there're several ways of doing it.

1. Release 0.8.0-preview officially and then release 0.8.0.
2. Release 0.8.0 with 'beta' or 'unstable' label. And keep 0.7.x as a
'stable' release in the download page. Once 0.8.x release becomes stable
enough make 0.8.x release as a 'stable' and move 0.7.x to 'old' releases.


*After 0.8.0, *
Since Zeppelin projects starts, project went through some major milestone,
like

- project gets first users and first contributor
- project went into Apache Incubator
- project became TLP.

And I think it's time to think about hitting another major milestone.

Considering features we already have, features we're planning on 0.8, wide
adoption of Zeppelin in the industry, I think it's time to focus on make
project more mature and make a 1.0 release. Which i think big milestone for
the project.

After 0.8.0 release, I suggest we more focus on bug fixes, stability
improvement, optimizing user experience than adding new features. And with
subsequent minor release, 0.8.1, 0.8.2 ... moment we feel confident about
the quality, release it as a 1.0.0 instead of 0.8.x.

Once we have 1.0.0 released, then I think we can make larger, experimental
changes on 2.0.0 branch aggressively, while we keep maintaining 1.0.x
branch.


Thanks,
moon

On Mon, Mar 20, 2017 at 8:55 AM Felix Cheung 
wrote:

> There are several pending visualization improvements/PRs that would be
> very good to get them in as well.
>
>
> --
> *From:* Jongyoul Lee 
> *Sent:* Sunday, March 19, 2017 9:03:24 PM
> *To:* dev; us...@zeppelin.apache.org
> *Subject:* Roadmap for 0.8.0
>
> Hi dev & users,
>
> Recently, community submits very new features for Apache Zeppelin. I think
> it's very positive signals to improve Apache Zeppelin and its community.
> But in another aspect, we should focus on what the next release includes. I
> think we need to summarize and prioritize them. Here is what I know:
>
> * Cluster management
> * Admin feature
> * Replace some context to separate users
> * Helium online
>
> Feel free to talk if you want to add more things. I think we need to
> choose which features will be included in 0.8.0, too.
>
> Regards,
> Jongyoul Lee
>
> --
> 이종열, Jongyoul Lee, 李宗烈
> http://madeng.net
>

Re: [CI] second job exceeds travis.org max job time

[moon soo Lee]

Nowadays, the second and the third profile takes around 40~45min.
Which is very close to limit of travis 50min.

I've seen sometimes second or the third profile just fails because of it
takes longer than usual (by slow network download, etc).

I think we need to split second/third profile so the build can be finished
in 30 min, which gives enough margin for travis build and adding more tests.

Thanks,
moon

On Fri, Mar 17, 2017 at 11:07 PM Jongyoul Lee  wrote:

> I found same issue. In my case, Zeppelin server won't run. I'm not sure why
> it happens.
>
> On Sat, Mar 18, 2017 at 2:17 AM, Remy Gayet  wrote:
>
> > nevermind. One of my test is hanging because of a bug somewhere in my
> code.
> >
> > On Fri, Mar 17, 2017 at 12:15 PM, Remy Gayet 
> wrote:
> >
> > > Hello guys,
> > >
> > > the second build configuration (env below) is now taking too much time
> (
> > >
> > > 49min) and gets killed, as per travis.org policy (
> > > https://docs.travis-ci.com/user/customizing-the-build#Build-Timeouts).
> > >
> > > SCALA_VER=\"2.11\" SPARK_VER=\"2.1.0\" HADOOP_VER=\"2.6\"
> > LIVY_VER=\"0.3.0\" PROFILE=\"-Pspark-2.1 -Phadoop-2.6 -Ppyspark -Psparkr
> > -Pscalding -Phelium-dev -Pexamples -Pscala-2.11 -Plivy-0.3\"
> > BUILD_FLAG=\"package -Pbuild-distr -DskipRat\" TEST_FLAG=\"verify
> > -Pusing-packaged-distr -DskipRat\" TEST_PROJECTS=\"\""
> > >
> > > Am I the only one experiencing this? If not, any thought on what to
> > remove
> > > to reduce this build duration?
> > >
> > > Cheers,
> > >
> > > Remy (Remilito)
> > >
> > >
> >
> >
> > --
> > Remy GAYET,
> > +33 6 798 798 62.
> >
>
>
>
> --
> 이종열, Jongyoul Lee, 李宗烈
> http://madeng.net
>

Re: Notebook Storage in Azure

[moon soo Lee]

Hi,

Thanks for sharing the problem.
Do you mind create an JIRA issue?
https://issues.apache.org/jira/browse/ZEPPELIN
That might help track this problem.

Thanks,
moon



On Sat, Feb 11, 2017 at 8:09 AM Gopikrishnan K. Rajaram 
wrote:

> Hello: - I have done the settings in zeppelin-site.xml as described in the
> support documentation. After restarting zeppelin, I tried to create a new
> notebook; it was still stored in the local unix file system and no copy was
> found in Azure storage account. I found this warning message in the log
> file:
>
> WARN [2017-02-10 20:30:46,654] ({main} NotebookRepoSync.java[]:88) -
> Failed to initialize org.apache.zeppelin.notebook.repo.AzureNotebookRepo
> notebook storage class
> at
> org.apache.zeppelin.notebook.repo.AzureNotebookRepo.(AzureNotebookRepo.java:76)
>
> I installed Zeppelin 0.7.0 on the master node of a 3 node Spark (2.1.0)
> cluster on Azure VM's.
>
> Thanks
>
> Krish
>

Re: [DISCUSS] Scaling out CI capability

[moon soo Lee]

ZEPPELIN-2123 is resolved.

Pull requests build capacity is not limited by 5 concurrent build any more.
Now Zeppelin project leverages each individual contributors travis-ci, so
we've got virtually unlimited concurrent build capacity for pull request.

Only thing the contributor should aware is setup and switch on travis-ci
for Zeppelin folk, explained here
http://zeppelin.apache.org/contribution/contributions.html#continuous-integration
.

Thanks,
moon

On Fri, Feb 17, 2017 at 5:30 PM moon soo Lee <m...@apache.org> wrote:

> Thanks for the feedbacks!
>
> Then let me merge https://github.com/apache/zeppelin/pull/2026 and
> configure Jenkins for building PR.
>
> Worst case, If something does not work, then we can just come back to
> travis build PR with almost no effort.
>
> I'll update progress in the issue comment
> https://issues.apache.org/jira/browse/ZEPPELIN-2123
>
> Thanks,
> moon
>
> On Thu, Feb 16, 2017 at 11:39 PM Jongyoul Lee <jongy...@gmail.com> wrote:
>
> I think it's good idea. :-)
>
> On Thu, Feb 16, 2017 at 7:54 PM, Jeff Zhang <zjf...@gmail.com> wrote:
>
> > Great, can't wait to try this.
> >
> >
> >
> > moon soo Lee <m...@apache.org>于2017年2月16日周四 下午6:35写道：
> >
> > > Hi Apache Zeppelin contributors!
> > >
> > > Recently, it's not very difficult to see pullrequest is waiting more
> than
> > > 10 hours to get travis-ci build slot.
> > >
> > > It's because of all ASF project shares the same travis account and the
> > > account's capability is limited. Although ASF assigned 5 concurrent
> build
> > > capacity to Zeppelin, it's not dedicated slot, it's more like max
> > > concurrent build limit, so when ASF travis is busy, Zeppelin
> pullrequest
> > > should wait.
> > >
> > > But each contributor can get 5 concurrent build capacity for opensource
> > > project. if we can leverage that, Zeppelin will no longer suffer from
> CI
> > > build capacity.
> > >
> > > I created https://issues.apache.org/jira/browse/ZEPPELIN-2123 and it
> > > includes proposed way, how Zeppelin can leverage individual
> contributors
> > > travis account.
> > > And created required script
> https://github.com/apache/zeppelin/pull/2026
> > .
> > >
> > > Please take a look the proposal. Any feedback would be really
> > appreciated!
> > >
> > > Thanks,
> > > moon
> > >
> >
>
>
>
> --
> 이종열, Jongyoul Lee, 李宗烈
> http://madeng.net
>
>

[DISCUSS] 0.7.1 release

[moon soo Lee]

Hi developers,

Since 0.7.0 release, some critical problems reported.
I think we need 0.7.1 release as soon as possible with those critical
problems fixed.

I created issue for 0.7.1 release
https://issues.apache.org/jira/browse/ZEPPELIN-2134 and link critical
issues with "blocked by", that they must be included in 0.7.1.

There're more issues reported for 0.7.0 release here [1], we can try best
include 0.7.1 but they're not blocker for the release i think.

I suggest focus we start release process for 0.7.1 as soon as all issues
linked from ZEPPELIN-2134 are resolved. Feel free to link critical issue to
ZEPPELIN-2134 so we can make sure 0.7.1 includes the fix.

Thanks,
moon

[1]
https://issues.apache.org/jira/browse/ZEPPELIN-2132?jql=project%20%3D%20ZEPPELIN%20and%20status%20%3D%20open%20and%20(affectedVersion%20%3D%200.7.0%20or%20fixversion%20%3D%200.7.1)

Re: [DISCUSS] Scaling out CI capability

[moon soo Lee]

Thanks for the feedbacks!

Then let me merge https://github.com/apache/zeppelin/pull/2026 and
configure Jenkins for building PR.

Worst case, If something does not work, then we can just come back to
travis build PR with almost no effort.

I'll update progress in the issue comment
https://issues.apache.org/jira/browse/ZEPPELIN-2123

Thanks,
moon

On Thu, Feb 16, 2017 at 11:39 PM Jongyoul Lee <jongy...@gmail.com> wrote:

> I think it's good idea. :-)
>
> On Thu, Feb 16, 2017 at 7:54 PM, Jeff Zhang <zjf...@gmail.com> wrote:
>
> > Great, can't wait to try this.
> >
> >
> >
> > moon soo Lee <m...@apache.org>于2017年2月16日周四 下午6:35写道：
> >
> > > Hi Apache Zeppelin contributors!
> > >
> > > Recently, it's not very difficult to see pullrequest is waiting more
> than
> > > 10 hours to get travis-ci build slot.
> > >
> > > It's because of all ASF project shares the same travis account and the
> > > account's capability is limited. Although ASF assigned 5 concurrent
> build
> > > capacity to Zeppelin, it's not dedicated slot, it's more like max
> > > concurrent build limit, so when ASF travis is busy, Zeppelin
> pullrequest
> > > should wait.
> > >
> > > But each contributor can get 5 concurrent build capacity for opensource
> > > project. if we can leverage that, Zeppelin will no longer suffer from
> CI
> > > build capacity.
> > >
> > > I created https://issues.apache.org/jira/browse/ZEPPELIN-2123 and it
> > > includes proposed way, how Zeppelin can leverage individual
> contributors
> > > travis account.
> > > And created required script
> https://github.com/apache/zeppelin/pull/2026
> > .
> > >
> > > Please take a look the proposal. Any feedback would be really
> > appreciated!
> > >
> > > Thanks,
> > > moon
> > >
> >
>
>
>
> --
> 이종열, Jongyoul Lee, 李宗烈
> http://madeng.net
>

[DISCUSS] Scaling out CI capability

[moon soo Lee]

Hi Apache Zeppelin contributors!

Recently, it's not very difficult to see pullrequest is waiting more than
10 hours to get travis-ci build slot.

It's because of all ASF project shares the same travis account and the
account's capability is limited. Although ASF assigned 5 concurrent build
capacity to Zeppelin, it's not dedicated slot, it's more like max
concurrent build limit, so when ASF travis is busy, Zeppelin pullrequest
should wait.

But each contributor can get 5 concurrent build capacity for opensource
project. if we can leverage that, Zeppelin will no longer suffer from CI
build capacity.

I created https://issues.apache.org/jira/browse/ZEPPELIN-2123 and it
includes proposed way, how Zeppelin can leverage individual contributors
travis account.
And created required script https://github.com/apache/zeppelin/pull/2026.

Please take a look the proposal. Any feedback would be really appreciated!

Thanks,
moon

Re: Feedback on ZEPPELIN-1764

[moon soo Lee]

Thanks for the interest to the project!
Just left a comment. let me know how you think.

Thanks,
moon

On Fri, Feb 10, 2017 at 6:55 PM vipul modi  wrote:

> Hi
>
> I would like to take up
> https://issues.apache.org/jira/browse/ZEPPELIN-1764
> I have added a proposed approach in the JIRA, wanted to check if it is a
> good way to implement it before coding it up.
>
> --
> Vipul Modi
> Software Development Engineer 2
> Flipkart Internet Pvt. Ltd.
>

Re: Require wiki edit permission

[moon soo Lee]

Your cwiki id is zjffdu, right?

On Fri, Feb 10, 2017 at 11:05 AM Jeff Zhang  wrote:

> Hi I am working on PIG-5126 to add document about running pig in zeppelin.
> But it is not good practise to include corporate link in apache project. So
> I would like to copy the content of below link to zeppelin wiki page and
> refer the zeppelin wiki in pig doc.
>
>
> https://community.hortonworks.com/articles/79183/how-to-use-pig-in-zeppelin.html
>

Re: Is there any way to cancel travis build in queue ?

[moon soo Lee]

AFAIK, no one has permission to cancel the job except for ASF-INFRA.

On Fri, Feb 10, 2017 at 9:56 AM Jeff Zhang  wrote:

> There's lots of pending build job in travis build queue, I can cancel some
> of my build as I would push a new commit. But I didn't find way to cancel
> that. Does anyone know how to cancel it ? Thanks
>

Re: ZEPPELIN-1856

[moon soo Lee]

Yes, it looks like we can ignore latecomer OUTPUT_APPEND and OUTPUT_UPDATE,
as well.

Thanks!
moon

On Tue, Feb 7, 2017 at 8:57 PM Alexander Shoshin <alexander_shos...@epam.com>
wrote:

> Thanks, I got it )
>
> Then I'm going to follow the way you suggested - to ignore an
> OUTPUT_UPDATE_ALL event after a job is finished, because we can't guarantee
> that all events will be delivered before 'interpret(..)' method ends - we
> don't know how many events we should receive from the concrete interpreter.
>
> Should we ignore latecomer OUTPUT_APPEND and OUTPUT_UPDATE events as well?
>
> Regards,
> Alexander
>
> -----Original Message-
> From: moon soo Lee [mailto:m...@apache.org]
> Sent: Tuesday, February 07, 2017 8:28 AM
> To: dev@zeppelin.apache.org
> Subject: Re: ZEPPELIN-1856
>
> onOutputClear() method is connected with InterpreterOutput.clear() method.
> clear() method is designed for the use case inside of
> Interpreter.interpret() like,
>
> InterpreterOutput.write("something"); // notebook displays "something"
> InterpreterOutput.clear(); // notebook clears "something"
> InterpreterOutput.write("something else"); // notebook displays "something
> else"
>
> For example, it can be useful when some interpreter want to print progress
> information (like logs, etc) while it's running, and when it finishes,
> clear output and display result.
>
> Will there be a way to fix flaky test ZEPPELIN-1856 while keeping this
> ability?
>
> Thanks,
> moon
>
> On Mon, Feb 6, 2017 at 6:08 PM Alexander Shoshin <
> alexander_shos...@epam.com>
> wrote:
>
> > Hi moon,
> > Thanks for the answer.
> >
> > A agree that one of possible ways to solve the problem is to
> > synchronize OUTPUT_UPDATE_ALL event delivery and 'interpret()' result
> return.
> >
> > But I still don't understand why we need to clear 'Paragraph.results'
> > from 'onOutputClear()' method. I thought that the main goal of this
> > method is to clear results on UI through 'broadcastParagraph(..)'. And
> > if we also try to clear 'Paragraph.results' from this method we can
> > find that it is either already cleared (by 'InterpreterContext'
> > initialization in
> > 'Paragraph.jobRun()') or filled with 'interpret()' result. After
> > ignoring OUTPUT_UPDATE_ALL event in case of FINISHED status we will
> > come in situation where we always clear an empty 'Paragraph.results'.
> > Does it make sense?
> >
> > Regards,
> > Alexander
> >
> > -Original Message-
> > From: moon soo Lee [mailto:m...@apache.org]
> > Sent: Saturday, February 04, 2017 4:30 AM
> > To: dev@zeppelin.apache.org
> > Subject: Re: ZEPPELIN-1856
> >
> > Hi,
> >
> > Thanks for working on issue ZEPPELIN-1856.
> > I think removing 'note.clearParagraphOutput(paragraphId);' from
> > 'onOutputClear()' is not the best solution, because without the line
> > 'onOutputClear()' does not do anything useful.
> >
> > InterpreterContext is constructed before call
> > RemoteInterpreterServer.interpret(). So OUTPUT_UPDATE_ALL supposed to
> > arrive before RemoteInterpreterServer.interpret() returns. Apparently,
> > it looks like it doesn't. So i think we need to tackle this problem.
> >
> > OUTPUT_UPDATE_ALL event is delivered via
> > RemoteInterpreterEventClient/RemoteInterpreterEventPoller, while
> > RemoteInterpreterServer.interpret() does not, and
> > RemoteInterpreterEventClient/RemoteInterpreterEventPoller does not
> > guarantee message deliver before interpret() returns. That's origin of
> > problem i think.
> >
> > Therefore, proper way solving this problem is add some mechanism that
> > guarantee RemoteInterpreterEventClient/RemoteInterpreterEventPoller
> > deliver the OUTPUT_UPDATE_ALL event before interpret() returns. Or
> > simple workaround could be ignore OUTPUT_UPDATE_ALL event when
> > paragraph becomes FINISHED status (after interpret() returns).
> >
> > Thanks,
> > moon
> >
> > On Sat, Feb 4, 2017 at 12:02 AM Alexander Shoshin <
> > alexander_shos...@epam.com> wrote:
> >
> > > Hi,
> > >
> > > I'm working on issue
> > > https://issues.apache.org/jira/browse/ZEPPELIN-1856
> > > and I found that we receive a NullPointerException sometimes because
> > > a paragraph result is cleared twice when we run a job. First
> > > Paragraph.result is cleared just before running
> > > RemoteInterpreter.interpret(..) and this is ok. But then we receive
> > > an OUTPUT_UPDATE_ALL event fro

Re: ZEPPELIN-1856

[moon soo Lee]

onOutputClear() method is connected with InterpreterOutput.clear() method.
clear() method is designed for the use case inside of
Interpreter.interpret() like,

InterpreterOutput.write("something"); // notebook displays "something"
InterpreterOutput.clear(); // notebook clears "something"
InterpreterOutput.write("something else"); // notebook displays "something
else"

For example, it can be useful when some interpreter want to print progress
information (like logs, etc) while it's running, and when it finishes,
clear output and display result.

Will there be a way to fix flaky test ZEPPELIN-1856 while keeping this
ability?

Thanks,
moon

On Mon, Feb 6, 2017 at 6:08 PM Alexander Shoshin <alexander_shos...@epam.com>
wrote:

> Hi moon,
> Thanks for the answer.
>
> A agree that one of possible ways to solve the problem is to synchronize
> OUTPUT_UPDATE_ALL event delivery and 'interpret()' result return.
>
> But I still don't understand why we need to clear 'Paragraph.results' from
> 'onOutputClear()' method. I thought that the main goal of this method is to
> clear results on UI through 'broadcastParagraph(..)'. And if we also try to
> clear 'Paragraph.results' from this method we can find that it is either
> already cleared (by 'InterpreterContext' initialization in
> 'Paragraph.jobRun()') or filled with 'interpret()' result. After ignoring
> OUTPUT_UPDATE_ALL event in case of FINISHED status we will come in
> situation where we always clear an empty 'Paragraph.results'. Does it make
> sense?
>
> Regards,
> Alexander
>
> -Original Message-
> From: moon soo Lee [mailto:m...@apache.org]
> Sent: Saturday, February 04, 2017 4:30 AM
> To: dev@zeppelin.apache.org
> Subject: Re: ZEPPELIN-1856
>
> Hi,
>
> Thanks for working on issue ZEPPELIN-1856.
> I think removing 'note.clearParagraphOutput(paragraphId);' from
> 'onOutputClear()' is not the best solution, because without the line
> 'onOutputClear()' does not do anything useful.
>
> InterpreterContext is constructed before call
> RemoteInterpreterServer.interpret(). So OUTPUT_UPDATE_ALL supposed to
> arrive before RemoteInterpreterServer.interpret() returns. Apparently, it
> looks like it doesn't. So i think we need to tackle this problem.
>
> OUTPUT_UPDATE_ALL event is delivered via
> RemoteInterpreterEventClient/RemoteInterpreterEventPoller, while
> RemoteInterpreterServer.interpret() does not, and
> RemoteInterpreterEventClient/RemoteInterpreterEventPoller does not
> guarantee message deliver before interpret() returns. That's origin of
> problem i think.
>
> Therefore, proper way solving this problem is add some mechanism that
> guarantee RemoteInterpreterEventClient/RemoteInterpreterEventPoller deliver
> the OUTPUT_UPDATE_ALL event before interpret() returns. Or simple
> workaround could be ignore OUTPUT_UPDATE_ALL event when paragraph becomes
> FINISHED status (after interpret() returns).
>
> Thanks,
> moon
>
> On Sat, Feb 4, 2017 at 12:02 AM Alexander Shoshin <
> alexander_shos...@epam.com> wrote:
>
> > Hi,
> >
> > I'm working on issue
> > https://issues.apache.org/jira/browse/ZEPPELIN-1856
> > and I found that we receive a NullPointerException sometimes because a
> > paragraph result is cleared twice when we run a job. First
> > Paragraph.result is cleared just before running
> > RemoteInterpreter.interpret(..) and this is ok. But then we receive an
> > OUTPUT_UPDATE_ALL event from the RemoteInterpreterServer and set
> > Paragraph.result to null again that may lead to a NullPointerException
> > if Paragraph.result was already filled by
> > RemoteInterpreter.interpret(..) responce.
> >
> > To resolve this problem we need to remove
> > note.clearParagraphOutput(paragraphId) line from the onOutputClear()
> > method in NotebookServer.java class:
> >
> > @Override
> > public void onOutputClear(String noteId, String paragraphId) {
> > Notebook notebook = notebook();
> > final Note note = notebook.getNote(noteId);
> > note.clearParagraphOutput(paragraphId);// this line seems to be
> > wrong
> > Paragraph paragraph = note.getParagraph(paragraphId);
> > broadcastParagraph(note, paragraph); }
> >
> > This method is called only when RemoteInterpreterServer.interpret(..)
> > initializes an InterpreterContext and sends an OUTPUT_UPDATE_ALL event.
> > Is it safe to remove this line or I miss something? It was added by
> > https://github.com/apache/zeppelin/pull/1658.
> >
> > Thanks,
> > Alexander
> >
>

Re: [VOTE] Release Apache Zeppelin 0.7.0 (RC4)

[moon soo Lee]

Verified
 - checksum and signature for release artifacts
 - build from source
 - LICENSE source/binary release
 - source release does not have unexpected binary
 - binary package functioning

+1 (binding)

On Fri, Feb 3, 2017 at 1:28 PM Hyung Sung Shim  wrote:

> +1
> Thanks mina for your effort.
>
> 2017-02-03 11:58 GMT+09:00 Renjith Kamath :
>
> +1
>
> On Fri, Feb 3, 2017 at 8:01 AM, Prabhjyot Singh  >
> wrote:
>
> > +1
> >
> > On Feb 2, 2017 8:25 PM, "Alexander Bezzubov"  wrote:
> >
> >> +1,
> >>
> >> and thank you for an awesome work Mina!
> >> Your persistence in making RCs and incorporating feedback is admirable.
> >>
> >> Verified:
> >>  - checksums, signatures + keys for sources and bin-all
> >>  - bin-all can run all Spark Zeppelin Tutorial in local mode
> >>  - sources do compile, but only without tests.
> >>Build \w tests fails on zeppelin-zengine for me
> >>
> >> 1) Failed tests:
> >>   NotebookTest.testSchedulePoolUsage:397 expected: but
> >> was:
> >>
> >> 2) frontend build on Linux also failed mysteriously executing yarn
> >> command, most probably due to local env configuration.
> >>
> >>
> >> --
> >> Alex
> >>
> >> On Thu, Feb 2, 2017 at 5:28 PM, Jeff Zhang  wrote:
> >>
> >>> +1
> >>>
> >>> Ahyoung Ryu 于2017年2月2日周四 下午9:36写道：
> >>>
> >>> +1
> >>>
> >>> On Thu, Feb 2, 2017 at 10:07 PM, Jun Kim  wrote:
> >>>
> >>> +1
> >>> 2017년 2월 2일 (목) 오후 9:49, Sora Lee 님이 작성:
> >>>
> >>> +1
> >>>
> >>> On Thu, Feb 2, 2017 at 9:40 PM Khalid Huseynov 
> >>> wrote:
> >>>
> >>> > +1
> >>> >
> >>> > On Thu, Feb 2, 2017 at 9:21 PM, DuyHai Doan 
> >>> wrote:
> >>> >
> >>> > +1
> >>> >
> >>> > On Thu, Feb 2, 2017 at 9:56 AM, Mina Lee  wrote:
> >>> >
> >>> > > I propose the following RC to be released for the Apache Zeppelin
> >>> 0.7.0
> >>> > > release.
> >>> > >
> >>> > > The commit id is df007f2284a09caa7c8b35f8b59d5f1993fe8b64 which is
> >>> > > corresponds to the tag v0.7.0-rc4:
> >>> > > *
>
> >>> > https://git-wip-us.apache.org/repos/asf?p=zeppelin.git;a=sho
> >>> rtlog;h=refs/tags/v0.7.0-rc4
> >>> > > <
> >>> > https://git-wip-us.apache.org/repos/asf?p=zeppelin.git;a=sho
>
> >>> rtlog;h=refs/tags/v0.7.0-rc4
> >>>
> >>> > >*
> >>> > >
> >>> > > The release archives (tgz), signature, and checksums are here
> >>> > >
> https://dist.apache.org/repos/dist/dev/zeppelin/zeppelin-0.7.0-rc4/
> >>> > >
> >>> > > The release candidate consists of the following source distribution
> >>> > > archive
> >>> > > zeppelin-0.7.0.tgz
> >>> > >
> >>> > > In addition, the following supplementary binary distributions are
> >>> > provided
> >>> > > for user convenience at the same location
> >>> > > zeppelin-0.7.0-bin-all.tgz
> >>> > > zeppelin-0.7.0-bin-netinst.tgz
> >>> > >
> >>> > > The maven artifacts are here
> >>> > >
> >>> > https://repository.apache.org/content/repositories/orgapache
> >>> zeppelin-1027
> >>> > >
> >>> > > You can find the KEYS file here:
> >>> > > https://dist.apache.org/repos/dist/release/zeppelin/KEYS
> >>> > >
> >>> > > Release notes available at
> >>> > > https://issues.apache.org/jira/secure/ReleaseNote.jspa?
> >>> > > version=12336544=12316221
> >>> > >
> >>> > > Vote will be open for next 72 hours (close at 01:00 5/Feb PST).
> >>> > >
> >>> > > [ ] +1 approve
> >>> > > [ ] 0 no opinion
> >>> > > [ ] -1 disapprove (and reason why)
> >>> > >
> >>> >
> >>> >
> >>> >
> >>>
> >>> --
> >>> Taejun Kim
> >>>
> >>> Data Mining Lab.
> >>> School of Electrical and Computer Engineering
> >>> University of Seoul
> >>>
> >>>
> >>>
> >>
>
>

Re: ZEPPELIN-1856

[moon soo Lee]

Hi,

Thanks for working on issue ZEPPELIN-1856.
I think removing 'note.clearParagraphOutput(paragraphId);' from
'onOutputClear()' is not the best solution, because without the line
'onOutputClear()' does not do anything useful.

InterpreterContext is constructed before call
RemoteInterpreterServer.interpret(). So OUTPUT_UPDATE_ALL supposed to
arrive before RemoteInterpreterServer.interpret() returns. Apparently, it
looks like it doesn't. So i think we need to tackle this problem.

OUTPUT_UPDATE_ALL event is delivered via
RemoteInterpreterEventClient/RemoteInterpreterEventPoller, while
RemoteInterpreterServer.interpret() does not, and
RemoteInterpreterEventClient/RemoteInterpreterEventPoller does not
guarantee message deliver before interpret() returns. That's origin of
problem i think.

Therefore, proper way solving this problem is add some mechanism that
guarantee RemoteInterpreterEventClient/RemoteInterpreterEventPoller deliver
the OUTPUT_UPDATE_ALL event before interpret() returns. Or simple
workaround could be ignore OUTPUT_UPDATE_ALL event when paragraph becomes
FINISHED status (after interpret() returns).

Thanks,
moon

On Sat, Feb 4, 2017 at 12:02 AM Alexander Shoshin <
alexander_shos...@epam.com> wrote:

> Hi,
>
> I'm working on issue https://issues.apache.org/jira/browse/ZEPPELIN-1856
> and I found that we receive a NullPointerException sometimes because a
> paragraph result is cleared twice when we run a job. First Paragraph.result
> is cleared just before running RemoteInterpreter.interpret(..) and this is
> ok. But then we receive an OUTPUT_UPDATE_ALL event from the
> RemoteInterpreterServer and set Paragraph.result to null again that may
> lead to a NullPointerException if Paragraph.result was already filled by
> RemoteInterpreter.interpret(..) responce.
>
> To resolve this problem we need to remove
> note.clearParagraphOutput(paragraphId) line from the onOutputClear() method
> in NotebookServer.java class:
>
> @Override
> public void onOutputClear(String noteId, String paragraphId) {
> Notebook notebook = notebook();
> final Note note = notebook.getNote(noteId);
> note.clearParagraphOutput(paragraphId);// this line seems to be
> wrong
> Paragraph paragraph = note.getParagraph(paragraphId);
> broadcastParagraph(note, paragraph);
> }
>
> This method is called only when RemoteInterpreterServer.interpret(..)
> initializes an InterpreterContext and sends an OUTPUT_UPDATE_ALL event.
> Is it safe to remove this line or I miss something? It was added by
> https://github.com/apache/zeppelin/pull/1658.
>
> Thanks,
> Alexander
>

Re: Beginner issue

[moon soo Lee]

Done!

Thanks for contribution!

Regards,
moon

On Thu, Feb 2, 2017 at 5:31 PM Viktor Boginskii 
wrote:

> Hello,
>
> I would like to start contribute to Zeppelin.
>
> Could anyone assign issue
> https://issues.apache.org/jira/browse/ZEPPELIN-2014 to me (vboginskii) in
> jira?
>
> Viktor Boginskii
> Senior Software Engineer
>
> Office: +380 44 390 5457 <+380%2044%20390%205457>
> x 66644   Cell: +380 67 144 6117 
> <+380%2067%20144%206117>
>  Email: viktor_bogins...@epam.com
> Kyiv, Ukraine (GMT+2)   epam.com
>
> CONFIDENTIALITY CAUTION AND DISCLAIMER
> This message is intended only for the use of the individual(s) or
> entity(ies) to which it is addressed and contains information that is
> legally privileged and confidential. If you are not the intended recipient,
> or the person responsible for delivering the message to the intended
> recipient, you are hereby notified that any dissemination, distribution or
> copying of this communication is strictly prohibited. All unintended
> recipients are obliged to delete this message and destroy any printed
> copies.
>
>

Re: Unstable travis CI recently

[moon soo Lee]

Hi,

I fixed some CI issues today.

https://issues.apache.org/jira/browse/ZEPPELIN-1583
https://issues.apache.org/jira/browse/ZEPPELIN-1857
https://issues.apache.org/jira/browse/ZEPPELIN-1986
https://issues.apache.org/jira/browse/ZEPPELIN-1749

Now, making CI green on PR wouldn't be too difficult.

But we still have many flaky tests
https://issues.apache.org/jira/browse/ZEPPELIN-1981.
To keep CI stable, I think all committers should take care

 - Merge PR after pass the CI.
 - When CI fails with known flaky test, make sure each test profile pass at
least once in multiple CI build attempts.

Otherwise, new PR might break CI even though it looks perfectly okay in
code review. Because there're cases that very difficult to catch in code
review.
For example,

 - New feature prints some log, and amount of log breaks CI. (CI log limit)
 - New feature increase start up time, and timeout in existing testing code
all affected
 - New test consumes some memory to be run, which exceed memory of build
machine.
 - 

Thanks,
moon

On Fri, Jan 20, 2017 at 12:20 AM Jongyoul Lee  wrote:

> I can change some fields and types but cannot convert all tickets to
> sub-task. I'll change them manually.
>
> On Thu, Jan 19, 2017 at 4:20 PM, Jeff Zhang  wrote:
>
> > Hi Jongyoul,
> >
> > Could you move all these tickets under this umbrella ticket ?
> >
> > You can do bulk change by clicking the gear tool button on top right of
> > this page. ( I don't' have privilege to do that)
> >
> >
> > https://issues.apache.org/jira/issues/?jql=project%20%
> > 3D%20ZEPPELIN%20AND%20text%20~%20flaky%20and%20status%20!%
> > 3D%20RESOLVED%20ORDER%20%20BY%20status%20ASC%20
> >
> >
> >
> >
> > Jongyoul Lee 于2017年1月19日周四 下午12:11写道：
> >
> > > Thanks Jeff,
> > >
> > > I made a new PR for handling this issue.
> > >
> > > https://issues.apache.org/jira/browse/ZEPPELIN-1981
> > >
> > > On Thu, Jan 19, 2017 at 9:47 AM, Jianfeng (Jeff) Zhang <
> > > jzh...@hortonworks.com> wrote:
> > >
> > > > +1
> > > >
> > > > Best Regard,
> > > > Jeff Zhang
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > On 1/19/17, 1:03 AM, "Jongyoul Lee"  wrote:
> > > >
> > > > >I also agree that ppl don't care of the result of CI anymore even
> it's
> > > > >real
> > > > >failure. One possible solution is making umbrella ticket, grabbing
> > flaky
> > > > >tests, disabling at first and enabling when it solves. but it
> assumes
> > we
> > > > >need to do our best to fix the flaky tests. Otherwise, we will lose
> > some
> > > > >tests...
> > > > >
> > > > >How do you guys think of it?
> > > > >
> > > > >On Thu, Jan 19, 2017 at 1:56 AM, Felix Cheung <
> > > felixcheun...@hotmail.com>
> > > > >wrote:
> > > > >
> > > > >> I'd agree. Is there a course of actions you can propose? Disable
> all
> > > > >>these
> > > > >> tests is a not a long term solution, right?
> > > > >>
> > > > >>
> > > > >> _
> > > > >> From: Jeff Zhang >
> > > > >> Sent: Tuesday, January 17, 2017 6:01 PM
> > > > >> Subject: Re: Unstable travis CI recently
> > > > >> To: >
> > > > >>
> > > > >>
> > > > >>
> > > > >> Should we disable these flaky test now ? CI seems become more
> > unstable
> > > > >> recently. It is almost useless for me, I never see a success CI
> > > > >>recently.
> > > > >> Here's one screenshot of recent closed PRs. Most of them has CI
> > > failure.
> > > > >> IMO, this is pretty bad, especially for new contributors.
> > > > >>
> > > > >>
> > > > >>
> > > > >> [pasted1]
> > > > >>
> > > > >>
> > > > >>
> > > > >>
> > > > >>
> > > > >> Jun Kim  >>?2016?12?13???
> > > > >> ??11:27???
> > > > >> @Hoon Thanks for your information :-) I should use that next time!
> > > > >>
> > > > >> 2016? 12? 13? (?) ?? 8:23, Park Hoon <1am...@gmail.com 1amb4
> > > > >> a...@gmail.com>>?? ??:
> > > > >>
> > > > >> > I totally agree with your opinions. I will work on
> ZEPPELIN-1739,
> > > > >> > ZEPPELIN-1749 first i reported before.
> > > > >> >
> > > > >> > @Jun Kim. So true. We have to wait long time. FYI, we can use
> our
> > > own
> > > > >> > travis CI containers to test (I recently learned also!) by
> > > configuring
> > > > >> > your-github-id/zeppelin-repo in travis CI
> > > > >> >
> > > > >> > Thanks!
> > > > >> >
> > > > >> > On Tue, Dec 13, 2016 at 8:19 PM, Jun Kim
> > > > >> > > > >> j...@gmail.com>> wrote:
> > > > >> >
> > > > >> > > I definitely agree with you!
> > > > >> > >
> > > > >> > > I reopened my PR twice recently to pass CI and it wasn't
> because
> > > of
> > > > >>me.
> > > > >> > >
> > > > >> > > CI takes about ~40min for a test, so I had to wait 1h and
> 20min
> > to
> > > > >> write
> > > > >> > a
> > > > >> > > comment after passing CI T_T
> > > > >> > >
> > > > >> > > And the worst of it is that I don't 

Re: [Discuss] Move some interpreters out of zeppelin project

[moon soo Lee]

Thanks Jeff for staring the thread.
Here's my thoughts

1. Do we need to do this
yes.

2. If the answer is yes, which interpreters should be moved out
If Zeppelin community has no problem maintaining certain interpreter, then
no reason to remove contribution from community.
However, if Zeppelin community can not maintain well (e.g. not catching up
target system version update, bug report is not taken care, etc), then we
can consider move out non-maintainable code from community.

3. How do we integrate these interpreters into zeppelin
Helium package description [1] already reserved package type 'INTERPRETER'
for it. And i hope 'helium' becomes a place
finding/installing/uninstalling/upgrading all pluggable modules in
Zeppelin. I can make pullrequest quickly to support INTERPRETER
installation through helium gui menu.

4. How does zeppelin work with these third party interpreters
In the point of view of encouraging 3rd party interpreter,
after 3) is done, Zeppelin-netinst package will display community managed
interpreters and 3rd party interpreters together in helium menu.
And their installation procedure will be exactly the same. (click 'enable'
button and click 'ok' on confirm dialog).

So, user will not see any difference between using community managed
interpreter and using 3rd party interpreter.
And this encourage develop more 3rd party interpreters than community
managed interpreters, i think.

Thanks,
moon

[1]
https://github.com/apache/zeppelin/blob/master/zeppelin-interpreter/src/main/java/org/apache/zeppelin/helium/HeliumPackage.java#L40


On Fri, Jan 20, 2017 at 6:39 AM Jongyoul Lee  wrote:

> Hi Jeff,
>
> Thanks for starting this issue.
>
> It increases flexibility of improving interpreters itself but it can also
> decreases stability of interpreters. I'm worried about this side-effect. As
> you mentioned, it's hard for me to review new interpreter that I didn't use
> but it couldn't be a reason why we divide some code from Zeppelin. We have
> to make more ppl as committers to review various interpreters. Thus I don't
> want some interpreters out of Zeppelin.
>
> But I, totally, agree about #3, #4. If we deploy minimum package of
> Zeppelin, we have to provide GUI for install/uninstall. If it's done,
> bin-all-pkg is meaningless and bin-min-pkg is enough.
>
> On Fri, Jan 20, 2017 at 7:14 PM, Jeff Zhang  wrote:
>
> > As we talk in another thread [1] about moving some interpreters out of
> > zeppelin project. I open this thread to discuss it in more details. I'd
> > like to raise 4 questions for this.
> >
> > 1. Do we need to do this
> > 2. If the answer is yes, which interpreters should be moved out
> > 3. How do we integrate these interpreters into zeppelin
> > 4. How does zeppelin work with these third party interpreters
> >
> > I will first give my inputs on this.
> >
> > *1. Do we need to do this ?*
> > Personally, I strongly +1 on this. Several reasons:
> >
> >- Keep the zeppelin project much smaller
> >- Each interpreter's improvements won't be blocked by the release of
> >zeppelin. Interpreters can has its own release cycle as long as
> >zeppelin-interpreter doesn't break the compatibility.
> >- Zeppelin developer don't have the knowledge of all interpreters.
> >Sometimes it is very difficult for zeppelin committers to review a new
> >interpreter that he doesn't know.
> >
> >
> > 2. Which interpreters should be moved out ?
> > We can discuss it  in another thread about the min package.
> >
> > 3. How do we integrate these interpreters into zeppelin
> > Currently, user can install third party interpreter by running script (
> > http://zeppelin.apache.org/docs/0.7.0-SNAPSHOT/manual/
> > interpreterinstallation.html#3rd-party-interpreters), but this is not
> > convienient, and it is hard to let every user to be aware of this
> feature.
> > So I think we should do that in zeppelin UI. We should allow user to
> > install/uninstall/upgrade/downgrade third party interpreters in the
> > interpreter page.
> >
> > 4. How does zeppelin work with these third party interpreters
> > Besides the interface zeppelin expose to the third party interpreter to
> be
> > install/uninstall/upgrade/downgrade, it is third party interpreter's own
> > responsibility to develop and make new release.
> >
> > Please help comment on these 4 questions and feel free to add any things
> > that I miss.
> >
> >
> > [1] https://lists.apache.org/thread.html/69f606409790d7ba11422e8c6df941
> > a75c5dfae0aca63eccf2f840bf@%3Cusers.zeppelin.apache.org%3E
> >
>
>
>
> --
> 이종열, Jongyoul Lee, 李宗烈
> http://madeng.net
>

Re: Notebook example

[moon soo Lee]

Althought it's not mendentory, creating issue helps track the changes such
as when code has chaged, which version of zeppelin includes the changes,
etc. So, please create if you don't mind.

Thanks,
moon
On Fri, 9 Dec 2016 at 11:05 AM Alexander Shoshin <alexander_shos...@epam.com>
wrote:

> Thanks for reply.
>
> And do I need to create an issue first? Or I can just make a pull request
> for adding a note to tutorial examples?
>
> Thanks,
> Alexander
>
> > -Original Message-
> > From: moon soo Lee [mailto:m...@apache.org]
> > Sent: Thursday, December 08, 2016 11:30 PM
> > To: dev@zeppelin.apache.org
> > Subject: Re: Notebook example
> >
> > Hi,
> >
> > Appreciate for sharing a notebook and considering contribution.
> > You can contribute your note as a tutorial under
> https://github.com/apache/zeppelin/tree/master/notebook by making
> pullrequest. So Zeppelin distribution can include notebook. Just sharing on
> the mailing list greatly helps as well if you prefer.
> >
> > Don't hesitate making pullrequest or asking questions.
> >
> > Thanks,
> > moon
> >
> > On Thu, Dec 8, 2016 at 10:26 AM Alexander Shoshin <
> alexander_shos...@epam.com> wrote:
> >
> > > Hi everyone.
> > >
> > > I want to become a Zeppelin contributor. Before starting to fix issues
> > > I decided to make an example of a notebook to become understand
> > > Zeppelin better :). I chose Apache Flink as a main Interpreter for my
> > > example because there are not so many examples of "Zeppelin on Flink"
> > > usage. I think it also may be helpful for your community according to
> > > https://issues.apache.org/jira/browse/ZEPPELIN-684, so here is the
> link:
> > > https://github.com/AlexanderShoshin/zeppelin-example
> > >
> > > I am ready to make any kind of corrections in this example.
> > >
> > > Regards,
> > > Alexander
> > >
>

Re: Notebook example

[moon soo Lee]

Hi,

Appreciate for sharing a notebook and considering contribution.
You can contribute your note as a tutorial under
https://github.com/apache/zeppelin/tree/master/notebook by making
pullrequest. So Zeppelin distribution can include notebook. Just sharing on
the mailing list greatly helps as well if you prefer.

Don't hesitate making pullrequest or asking questions.

Thanks,
moon

On Thu, Dec 8, 2016 at 10:26 AM Alexander Shoshin <
alexander_shos...@epam.com> wrote:

> Hi everyone.
>
> I want to become a Zeppelin contributor. Before starting to fix issues I
> decided to make an example of a notebook to become understand Zeppelin
> better :). I chose Apache Flink as a main Interpreter for my example
> because there are not so many examples of "Zeppelin on Flink" usage. I
> think it also may be helpful for your community according to
> https://issues.apache.org/jira/browse/ZEPPELIN-684, so here is the link:
> https://github.com/AlexanderShoshin/zeppelin-example
>
> I am ready to make any kind of corrections in this example.
>
> Regards,
> Alexander
>

Re: [DISCUSS] Zeppelin 0.7 Roadmap

[moon soo Lee]

Hi

I updated the wiki.

https://cwiki.apache.org/confluence/display/ZEPPELIN/Zeppelin+Roadmap

Please feel free to give more suggestions about the roadmap any time.

I think roadmap doesn't mean the community will reject items not on the
roadmap, it's more like showing where the primary focus of the community
goes.

Although there were a lot of progresses since 0.6.x, not all the issues
planned 0.7 are resolved yet. So let's see how much we can get close to the
roadmap at the end of Nov.

Thanks,
moon

On Sun, Oct 9, 2016 at 7:35 PM Jeff Zhang <zjf...@gmail.com> wrote:

> +1 for adding impersonation. Consider most of subtasks of ZEPPELIN-1337
> (multiple user support) will involve large changes, I am afraid it may be a
> little risky to release 0.7 in Nov, and I expect we would introduce new
> bugs since the changes are big.
>
> On Mon, Oct 10, 2016 at 10:30 AM, Corneau Damien <cornead...@gmail.com>
> wrote:
>
> > We don't have many routes, and so far no need for subroutes, so
> > angular-ui-router isn't really needed.
> > However lazy loading could be nice
> >
> > On Mon, Oct 10, 2016 at 10:53 AM, Dennis Jung <inylov...@gmail.com>
> wrote:
> >
> > > Hi~
> > > In fronted part, how about update router to ui-router with lazy load
> > > module?
> > > ui-router is 3rd party module, but it is more powerful than ngRoute.
> And
> > > using lazy-loading will help resource loading performance.
> > >
> > > http://stackoverflow.com/questions/21023763/angularjs-
> > > difference-between-angular-route-and-angular-ui-router
> > >
> > > How do you think? If it seems okay, I could help on this.
> > > Thanks.
> > >
> > > 2016-10-10 0:22 GMT+09:00 DuyHai Doan <doanduy...@gmail.com>:
> > >
> > > > I'd also add in the Enterprise section a topic about monitoring (e.g.
> > at
> > > > least exposing JMX metrics as a 1st step). I guess we cannot pretend
> to
> > > be
> > > > enterprise ready without at least some means to monitor the Zeppelin
> > > server
> > > > state.
> > > >
> > > > Hopefully the monitoring topic will be covered by the big refactoring
> > > JIRA
> > > > epic: https://issues.apache.org/jira/browse/ZEPPELIN-1525
> > > >
> > > > On Sun, Oct 9, 2016 at 4:53 PM, Jongyoul Lee <jongy...@gmail.com>
> > wrote:
> > > >
> > > > > Thanks for the sharing this roadmap.
> > > > >
> > > > > I'd like to add impersonation for Spark/JDBC interpreter.
> > > > >
> > > > > How do you think of ti?
> > > > >
> > > > > Regards,
> > > > > Jongyoul
> > > > >
> > > > > On Sat, Oct 8, 2016 at 6:46 AM, moon soo Lee <m...@apache.org>
> > wrote:
> > > > >
> > > > >> Hi, Apache Zeppelin users and developers,
> > > > >>
> > > > >> We're about to start a release process for 0.6.2 and i think it's
> > good
> > > > >> time to discuss about future release, 0.7.0. There were great
> > > discussion
> > > > >> about roadmap [1] and we updated wiki page [2], but 0.7.0 section
> on
> > > the
> > > > >> roadmap wiki is empty at the moment. Having a 0.7.0 section on a
> > wiki
> > > > page,
> > > > >> i think, doesn't mean neither rejecting other subjects nor 100%
> > > > guarantee
> > > > >> of them. However it's more for helping Zeppelin users and
> developers
> > > > have
> > > > >> reasonable expectations for the next release and helps community
> > focus
> > > > on
> > > > >> main theme.
> > > > >>
> > > > >> Multi-tenancy related feature (interpreter authorization,
> > > impersonation,
> > > > >> per user interpreter, and so on) are the most active subject in
> the
> > > > >> community at the moment i think. And we have a new menu, "Job" in
> > the
> > > > >> master branch is another big change. I think "Enterprise ready"
> > > section
> > > > on
> > > > >> [2] can be main topic for 0.7 release.
> > > > >>
> > > > >> And there're many improvements around generic JDBC interpreter and
> > > > Python
> > > > >> support (matplotlib integration and so on). They can be another
> > >

Re: Disabling "run all paragraphs" button on notebook

[moon soo Lee]

Currently there's not.
You'll need to modify source code to disable it.

Thanks,
moon

On Tue, Nov 8, 2016 at 9:32 AM Sachin Janani  wrote:

> Hi All,
> I was finding a way or property to disable 'run all paragraphs' button on
> notebook but didnt find any can someone please help me and let me know if
> there is any way to disable that.
>
> Regards,
> Sachin Janani
>

Re: Build error

[moon soo Lee]

Hi,

A patch available for the error you have.
https://github.com/apache/zeppelin/pull/1603

By the way, if you just want to build and use Zeppelin, you can try skip
test by adding -DskipTests in your maven command.

Thanks,
moon

On Mon, Nov 7, 2016 at 9:51 PM Andrii Pelekh  wrote:

> Hi all,
>
> I'm new to the project and I'm trying to build Zeppelin from source. I
> followed all the instructions and double checked all the software required
> multiple times but I keep getting errors. Please help. My log file is
> attached.
>
> $ git clone https://github.com/apache/zeppelin.git
> $ cd zeppelin/
> $ ./dev/change_scala_version.sh 2.11
> $ mvn clean package -Pspark-2.0 -Phadoop-2.4 -Pyarn -Ppyspark -Psparkr
> -Pscala-2.11
>
>
> System details:
> Ubuntu 16.04.1 LTS
> Open jdk 7
> Apache Maven 3.3.9
>
> Thanks,
> Andrii
>

Re: Max text message size for Websocket connection

[moon soo Lee]

Hi,

ZeppelinServer [1] sets Websocket text message size with default value
defined in ZeppelinConfiguration [2].

Hope this helps.

Thanks,
moon

[1]
https://github.com/apache/zeppelin/blob/master/zeppelin-server/src/main/java/org/apache/zeppelin/server/ZeppelinServer.java#L214
[2]
https://github.com/apache/zeppelin/blob/master/zeppelin-zengine/src/main/java/org/apache/zeppelin/conf/ZeppelinConfiguration.java#L576


On Thu, Oct 13, 2016 at 3:53 AM Sandeep More  wrote:

> Hello,
>
>
>
> I am planning to add proxy support for Zeppelin in Knox and would like to
>
> know what are the default values for the following properties:
>
>
>
> 1) Max text message size for Websocket connection
>
> 2) Max binary message size for Websocket connection
>
>
>
> Also, are these properties configurable.
>
>
>
> Thanks !
>
> Sandeep
>
>

Re: [VOTE] Release Apache Zeppelin 0.6.2 (rc1)

[moon soo Lee]

Tested pr-1505 [1] and thanks for quick patch.
While patch is already available i think it make sense to cancel this vote
and prepare rc2 for 0.6.2 release.

Thanks,
moon

[1]  https://github.com/apache/zeppelin/pull/1505

On Tue, Oct 11, 2016 at 7:38 PM Mina Lee <mina...@apache.org> wrote:

> Moon, I could reproduce the error. In my opinion this is quite serious
>
> issue which will cause really bad user experience.
>
> I'd like to cancel this vote and prepare 0.6.2-rc2 after we fix this one. I
>
> just made a [1] PR to fix it.
>
> It would be appreciated if you share your opinion about this.
>
>
>
> And my bad for KEY location, I will use the correct url from the next
>
> release vote.
>
>
>
> Thanks,
>
> Mina
>
>
>
> [1] https://github.com/apache/zeppelin/pull/1505
>
>
>
>
>
>
>
> On Tue, Oct 11, 2016 at 6:22 PM moon soo Lee <m...@apache.org> wrote:
>
>
>
> > Thanks Mina for preparing the release.
>
> >
>
> >
>
> >
>
> > Verified,
>
> >
>
> >
>
> >
>
> >  - Build from source
>
> >
>
> >  - LICENSE and NOTICE in the source and binary packages
>
> >
>
> >  - Signature (KEYS file need to be along side with release artifacts. see
>
> >
>
> > [1]. I think keys location need to be
>
> >
>
> > https://dist.apache.org/repos/dist/release/zeppelin/KEYS)
>
> >
>
> >  - Maven artifacts
>
> >
>
> >
>
> >
>
> > However, couldn't run tutorial notebook. SparkInterpreter didn't worked
>
> >
>
> > with zero configuration, under Java 1.7.
>
> >
>
> >
>
> >
>
> > Thanks,
>
> >
>
> > moon
>
> >
>
> >
>
> >
>
> > [1] http://www.apache.org/dev/release-signing.html#keys-policy
>
> >
>
> >
>
> >
>
> >
>
> >
>
> > On Tue, Oct 11, 2016 at 5:26 PM Prabhjyot Singh <
> prabhjyotsi...@apache.org
>
> > >
>
> >
>
> > wrote:
>
> >
>
> >
>
> >
>
> > > +1
>
> >
>
> > >
>
> >
>
> > >
>
> >
>
> > >
>
> >
>
> > > On 11 Oct 2016 1:21 p.m., "Hyung Sung Shim" <hss...@nflabs.com> wrote:
>
> >
>
> > >
>
> >
>
> > >
>
> >
>
> > >
>
> >
>
> > > > +1 and Mina. Thank you for your effort.
>
> >
>
> > >
>
> >
>
> > > > 2016년 10월 11일 (화) 오후 4:38, Victor Manuel Garcia <
>
> > victor.gar...@beeva.com
>
> >
>
> > >
>
> >
>
> > > > >님이
>
> >
>
> > >
>
> >
>
> > > > 작성:
>
> >
>
> > >
>
> >
>
> > > >
>
> >
>
> > >
>
> >
>
> > > > > +1
>
> >
>
> > >
>
> >
>
> > > > >
>
> >
>
> > >
>
> >
>
> > > > > 2016-10-11 3:36 GMT+02:00 Ahyoung Ryu <ahyoung...@apache.org>:
>
> >
>
> > >
>
> >
>
> > > > >
>
> >
>
> > >
>
> >
>
> > > > > > +1
>
> >
>
> > >
>
> >
>
> > > > > >
>
> >
>
> > >
>
> >
>
> > > > > >
>
> >
>
> > >
>
> >
>
> > > > > >
>
> >
>
> > >
>
> >
>
> > > > > > On Tue, Oct 11, 2016 at 10:24 AM, Jeff Zhang <zjf...@gmail.com>
>
> >
>
> > > wrote:
>
> >
>
> > >
>
> >
>
> > > > > >
>
> >
>
> > >
>
> >
>
> > > > > > > +1, teste with spark 2.0 with basic notes,  ZeppelinContext and
>
> > UDF
>
> >
>
> > >
>
> >
>
> > > > > works
>
> >
>
> > >
>
> >
>
> > > > > > > correctly.
>
> >
>
> > >
>
> >
>
> > > > > > >
>
> >
>
> > >
>
> >
>
> > > > > > > On Tue, Oct 11, 2016 at 8:41 AM, Anthony Corbacho <
>
> >
>
> > >
>
> >
>
> > > > > > > corbacho.anth...@gmail.com> wrote:
>
> >
>
> > >
>
> >
>
> > > > > > >
>
> >
>
>

Re: [VOTE] Release Apache Zeppelin 0.6.2 (rc1)

[moon soo Lee]

Thanks Mina for preparing the release.

Verified,

 - Build from source
 - LICENSE and NOTICE in the source and binary packages
 - Signature (KEYS file need to be along side with release artifacts. see
[1]. I think keys location need to be
https://dist.apache.org/repos/dist/release/zeppelin/KEYS)
 - Maven artifacts

However, couldn't run tutorial notebook. SparkInterpreter didn't worked
with zero configuration, under Java 1.7.

Thanks,
moon

[1] http://www.apache.org/dev/release-signing.html#keys-policy


On Tue, Oct 11, 2016 at 5:26 PM Prabhjyot Singh 
wrote:

> +1
>
>
>
> On 11 Oct 2016 1:21 p.m., "Hyung Sung Shim"  wrote:
>
>
>
> > +1 and Mina. Thank you for your effort.
>
> > 2016년 10월 11일 (화) 오후 4:38, Victor Manuel Garcia 
> > >님이
>
> > 작성:
>
> >
>
> > > +1
>
> > >
>
> > > 2016-10-11 3:36 GMT+02:00 Ahyoung Ryu :
>
> > >
>
> > > > +1
>
> > > >
>
> > > >
>
> > > >
>
> > > > On Tue, Oct 11, 2016 at 10:24 AM, Jeff Zhang 
> wrote:
>
> > > >
>
> > > > > +1, teste with spark 2.0 with basic notes,  ZeppelinContext and UDF
>
> > > works
>
> > > > > correctly.
>
> > > > >
>
> > > > > On Tue, Oct 11, 2016 at 8:41 AM, Anthony Corbacho <
>
> > > > > corbacho.anth...@gmail.com> wrote:
>
> > > > >
>
> > > > > > +1
>
> > > > > >
>
> > > > > > I tested with my notebooks works as expected.
>
> > > > > > Great job!
>
> > > > > >
>
> > > > > > On Mon, Oct 10, 2016 at 9:34 PM, Mina Lee 
>
> > > wrote:
>
> > > > > >
>
> > > > > > > Hi folks,
>
> > > > > > >
>
> > > > > > > I propose the following RC to be released for the Apache
> Zeppelin
>
> > > > > > > 0.6.2 release.
>
> > > > > > >
>
> > > > > > > The commit id is 9a44590841d5d83da1a297d507f6e2e67784ba1f which
>
> > is
>
> > > > > > > corresponds to the tag v0.6.2-rc1:
>
> > > > > > > https://git-wip-us.apache.org/repos/asf?p=zeppelin.git;a=
>
> > commit;h=
>
> > > > > > > 9a44590841d5d83da1a297d507f6e2e67784ba1f
>
> > > > > > >
>
> > > > > > > The release archives (tgz), signature, and checksums are here
>
> > > > > > >
>
> > > https://dist.apache.org/repos/dist/dev/zeppelin/zeppelin-0.6.2-rc1/
>
> > > > > > >
>
> > > > > > > The release candidate consists of the following source
>
> > distribution
>
> > > > > > archive
>
> > > > > > > zeppelin-0.6.2.tgz
>
> > > > > > >
>
> > > > > > > In addition, the following supplementary binary distributions
> are
>
> > > > > > provided
>
> > > > > > > for user convenience at the same location
>
> > > > > > > zeppelin-0.6.2-bin-all.tgz
>
> > > > > > > zeppelin-0.6.2-netinst-all.tgz
>
> > > > > > >
>
> > > > > > > The maven artifacts are here
>
> > > > > > > https://repository.apache.org/content/repositories/
>
> > > > > > orgapachezeppelin-1019
>
> > > > > > >
>
> > > > > > > You can find the KEYS file here:
>
> > > > > > > https://people.apache.org/keys/committer/minalee.asc
>
> > > > > > >
>
> > > > > > > Release notes available at
>
> > > > > > > https://issues.apache.org/jira/secure/ReleaseNote.jspa?
>
> > > > > > > projectId=12316221=12338135
>
> > > > > > >
>
> > > > > > > Vote will be open for next 72 hours (close at 6am 13/Oct PDT).
>
> > > > > > >
>
> > > > > > > [ ] +1 approve
>
> > > > > > > [ ] 0 no opinion
>
> > > > > > > [ ] -1 disapprove (and reason why)
>
> > > > > > >
>
> > > > > >
>
> > > > >
>
> > > > >
>
> > > > >
>
> > > > > --
>
> > > > > Best Regards
>
> > > > >
>
> > > > > Jeff Zhang
>
> > > > >
>
> > > >
>
> > >
>
> > >
>
> > >
>
> > > --
>
> > > *Victor Manuel Garcia Martinez*
>
> > > *Technical Architect*
>
> > > <
>
> > > https://www.linkedin.com/profile/view?id=AAMAABFzLAsBCDe_qh9oo-
>
> > ENueO999zvw0zkhXQ=hp-identity-headline
>
> > > >
>
> > >
>
> > >
>
> > > *+34 672104297 <+34%20672%2010%2042%2097>  | victor.gar...@beeva.com <
> victor.gar...@beeva.com>*
>
> > >  *  | victormanuel.garcia.marti...@bbva.com
>
> > > *
>
> > >
>
> > > 
>
> > >
>
> >
>
>

Re: Refactor time !

[moon soo Lee]

Thanks for taking care!

On Sun, Oct 9, 2016 at 4:05 PM DuyHai Doan  wrote:

> I have created a JIRA epic to track down all the task:
> https://issues.apache.org/jira/browse/ZEPPELIN-1525
>
> I think I would start by the synchronize blocks and then move onto Eric's
> PR for Guice DI.
>
> After we have a DI mechanism, it will be much easier to inject thread pools
> for thread management and also to create JMX monitoring
>
> Any objection before I start coding ?
>
>
>
> On Sat, Oct 8, 2016 at 2:05 PM, Eric Charles  wrote:
>
> > On 04/10/16 12:54, Anthony Corbacho wrote:
> >
> >> You made my day, this is the kind of email i really like !!
> >>
> >> I think its a great idea and i am willing to spend sometime on it.
> >>
> >> I also want to move to a DI (guice) architecture , let me know what you
> >> think about it.
> >>
> >
> > A PR is opened for Guice DI. If someone jumps for review, I can rebase
> >
> > https://github.com/apache/zeppelin/pull/1361
> >
> >
> >
> >
> >> On Tuesday, 4 October 2016, DuyHai Doan  wrote:
> >>
> >> Hello devs
> >>>
> >>> The code base of Zeppelin has grown very fast in the last 12 months and
> >>> it's great. It means that we have more and more contributors.
> >>>
> >>> However, to make the project maintainable at long term, we need regular
> >>> code refactoring.
> >>>
> >>> I have some ideas to share with you
> >>>
> >>> 1) Use Java 8 to benefit from Lambda & streams.
> >>>
> >>>   Now that Java 8 is well established, it is a good time to upgrade the
> >>> project. I believe some interpreters also need Java 8. Cassandra
> >>> interpreter right now does not have unit tests for the latest features
> >>> because the Embedded Cassandra server used for testing requires Java 8.
> >>>
> >>>  It would also be a good opportunity to go through the code base and
> >>> replace some boilerplate for() loop with manual filtering by the stream
> >>> shortcut :  list.stream().filter(..).map(). It would improve greatly
> >>> code
> >>> readability
> >>>
> >>> 2) Multi threading
> >>>
> >>>  I've seen the usage of synchronize block at a few places in the code
> >>> base.
> >>> Although perfectly valid, it has a cost at runtime and since more and
> >>> more
> >>> people are asking for multi-tenancy or using a single Zeppelin instance
> >>> to
> >>> server multiple users, I guess the synchronized blocks has a huge cost.
> >>>
> >>> There are some solid alternatives:
> >>>
> >>>  - ConcurrentHashMap if we synchronized on a map
> >>>  - CopyOnWriteArrayList if we synchronized on a list.
> >>>
> >>> Of cours each sychronize block should be taken carefully not to
> introduce
> >>> regression
> >>>
> >>> 3) Thread management
> >>>
> >>> I've seen some usage of new Thread() {...}.run(); it may be a good time
> >>> to
> >>> introduce ThreadPool and pass them along (inside context objects for
> >>> example) to have a more centralized thread management
> >>>
> >>> The advantage of having thread pool is that we can manage them in a
> >>> single
> >>> place, monitor them and expose the info through JMX and also control
> >>> system
> >>> resource by defining max thread number and thread pool queue
> >>>
> >>> 4) Server monitoring
> >>> I hear many users on the field complain about the fact that they have
> to
> >>> restart Zeppelin server regularly because it "hangs" after running a
> long
> >>> time.
> >>>
> >>> If we can expose some system metrics through JMX, it would help people
> >>> monitor the state of Zeppelin server and take appropriate actions
> >>>
> >>> Right now we may only focus on monitoring the server itself, not the
> >>> interpreter JVMs processes. It can be done in a 2nd step
> >>>
> >>>
> >>> What do you think about the ideas ?
> >>>
> >>>
> >>
>

[DISCUSS] Zeppelin 0.7 Roadmap

[moon soo Lee]

Hi, Apache Zeppelin users and developers,

We're about to start a release process for 0.6.2 and i think it's good time
to discuss about future release, 0.7.0. There were great discussion about
roadmap [1] and we updated wiki page [2], but 0.7.0 section on the roadmap
wiki is empty at the moment. Having a 0.7.0 section on a wiki page, i
think, doesn't mean neither rejecting other subjects nor 100% guarantee of
them. However it's more for helping Zeppelin users and developers have
reasonable expectations for the next release and helps community focus on
main theme.

Multi-tenancy related feature (interpreter authorization, impersonation,
per user interpreter, and so on) are the most active subject in the
community at the moment i think. And we have a new menu, "Job" in the
master branch is another big change. I think "Enterprise ready" section on
[2] can be main topic for 0.7 release.

And there're many improvements around generic JDBC interpreter and Python
support (matplotlib integration and so on). They can be another important
subject with new Interpreters.

Besides that, I've seen people struggle with front-end performance and we
can address that on 0.7, that would be great.
Also i'd like to keep working on pluggability for visualization, which was
subject from 0.6 release.
Therefore, i would suggest following draft as a roadmap for 0.7.0

* Enterprise support
  - Multi user support (ZEPPELIN-1337)
  - Job management
* Interpreter
  - Improve JDBC / Python interpreter
  - New interpreters
* Front end performance improvement
* Pluggable visualization

Regarding timeline, although we're keep making series of 0.6.x release,
it's already been 3 months since 0.6 release. And i think many items are
already been addressed on master branch or patches are available. So i
think we can target near future, like November.

What do you think? And feedback would be appreciated.

Thanks,
moon

[1] http://apache-zeppelin-users-incubating-mailing-li

st.75479.x6.nabble.com/DISCUSS-Update-Roadmap-tp2452.html

[2] https://cwiki.apache.org/confluence/display/ZEPPELIN/Zeppelin+Roadmap

Re: [DISCUSS] Zeppelin 0.6.2 release

[moon soo Lee]

Although 0.6.1 supports Spark 2.0, there were some bugs around it.
Current 0.6 branch resolves many annoying problems and i think it's a lot
more stable than previous release, 0.6.1. Let's start release process for
0.6.2.

Thanks,
moon

On Sat, Oct 8, 2016 at 3:33 AM DuyHai Doan  wrote:

> +1
>
> On Fri, Oct 7, 2016 at 4:19 PM, Renjith Kamath 
> wrote:
>
> > +1
> >
> > Thanks & Regards,
> > RK
> >
> > On Fri, Oct 7, 2016 at 4:19 PM, Anthony Corbacho <
> > anthonycorba...@apache.org
> > > wrote:
> >
> > > I also +1
> > >
> > > On Friday, 7 October 2016, Jeff Zhang  wrote:
> > >
> > > > +1 for it.
> > > >
> > > > On Fri, Oct 7, 2016 at 5:37 PM, Jongyoul Lee  > > > > wrote:
> > > >
> > > > > Great job, Mina.
> > > > >
> > > > > I think it's enough to release 0.6.2.
> > > > >
> > > > > On Fri, Oct 7, 2016 at 4:39 PM, Mina Lee  > > > > wrote:
> > > > >
> > > > > > Hi devs,
> > > > > >
> > > > > > Release process has been delayed longer than we planned
> > > > > > since several critical issues have been reported continuously.
> > > > > > [1] All issues that were blocking 0.6.2 have been resolved
> finally,
> > > > > > so I want to hear thoughts about 0.6.2 release.
> > > > > >
> > > > > > Regards,
> > > > > > Mina
> > > > > >
> > > > > > [1] https://issues.apache.org/jira/browse/ZEPPELIN-1347
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > 이종열, Jongyoul Lee, 李宗烈
> > > > > http://madeng.net
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Best Regards
> > > >
> > > > Jeff Zhang
> > > >
> > >
> >
>