[GitHub] zeppelin issue #1837: ZEPPELIN-1901. Output schema might be null for PigQuer...

[zjffdu]

Github user zjffdu commented on the issue:

https://github.com/apache/zeppelin/pull/1837
  
Ping @felixcheung , CI failure is not relevant.

```
AngularElem
- should provide onclick method *** FAILED ***
  The code passed to eventually never returned normally. Attempted 1 times 
over 192.484891 milliseconds. Last failure message: 0 was not equal to 1. 
(AbstractAngularElemTest.scala:64)
```





---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] zeppelin issue #1844: [ZEPPELIN-1825] Use versioned notebook storage by defa...

[khalidhuseynov]

Github user khalidhuseynov commented on the issue:

https://github.com/apache/zeppelin/pull/1844
  
@felixcheung i didn't see much places to update docs, but maybe @AhyoungRyu 
knows if there's something to be updated?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] zeppelin issue #1796: ZEPPELIN-1852. Use multiple InterpreterResult for disp...

[felixcheung]

Github user felixcheung commented on the issue:

https://github.com/apache/zeppelin/pull/1796
  
LGTM - let's check CI.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] zeppelin issue #1856: remove org.apache.spark.sql.hive.HiveSharedState class...

[felixcheung]

Github user felixcheung commented on the issue:

https://github.com/apache/zeppelin/pull/1856
  
looks good, why doesn't CI tests for Spark 2.1 fails for this?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] zeppelin issue #1844: [ZEPPELIN-1825] Use versioned notebook storage by defa...

[felixcheung]

Github user felixcheung commented on the issue:

https://github.com/apache/zeppelin/pull/1844
  
LGTM
any documentation to update?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] zeppelin issue #1846: ZEPPELIN-1770. Restart only the client user's interpre...

[felixcheung]

Github user felixcheung commented on the issue:

https://github.com/apache/zeppelin/pull/1846
  
oh gosh :)
is it possible to have test for this?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] zeppelin issue #1851: [ZEPPELIN-1908] Invalid Typo Method name

[felixcheung]

Github user felixcheung commented on the issue:

https://github.com/apache/zeppelin/pull/1851
  
LGTM


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] zeppelin issue #1853: [ZEPPELIN-1911] Adding how to install evaluate package...

[felixcheung]

Github user felixcheung commented on the issue:

https://github.com/apache/zeppelin/pull/1853
  
`knitr` should install `evaluate` as a dependency actually?
do you know if that is not the case?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] zeppelin issue #1854: [ZEPPELIN-1912] fix: Improve perosnalized mode tooltip...

[felixcheung]

Github user felixcheung commented on the issue:

https://github.com/apache/zeppelin/pull/1854
  
LGTM


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] zeppelin issue #1855: [MINOR][ZEPPELIN-1913] Separate "Apache Zeppelin Confi...

[felixcheung]

Github user felixcheung commented on the issue:

https://github.com/apache/zeppelin/pull/1855
  
This is great!


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] zeppelin issue #1848: [ZEPPELIN-1812] Ace editor show/hide problem.

[prabhjyotsingh]

Github user prabhjyotsingh commented on the issue:

https://github.com/apache/zeppelin/pull/1848
  
Tested, LGTM!


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] zeppelin pull request #1848: [ZEPPELIN-1812] Ace editor show/hide problem.

[cloverhearts]

Github user cloverhearts closed the pull request at:

https://github.com/apache/zeppelin/pull/1848


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] zeppelin pull request #1848: [ZEPPELIN-1812] Ace editor show/hide problem.

[cloverhearts]

GitHub user cloverhearts reopened a pull request:

https://github.com/apache/zeppelin/pull/1848

[ZEPPELIN-1812] Ace editor show/hide problem.

### What is this PR for?
When editor hide is on and page refresh then editor is gone.
I was solve this problem.

### What type of PR is it?
Bug Fix

### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-1812

### How should this be tested?
1. hide to editor mode on in paragraph configure.
2. page refresh (f5)
3. enable to show editor mode in paragraph configure.
4. if show editor then is correct.
### Screenshots (if appropriate)
 before

![badeditor](https://cloud.githubusercontent.com/assets/10525473/21677078/62136de2-d2ed-11e6-9418-0be4453fd839.gif)

 after

![goodeditor](https://cloud.githubusercontent.com/assets/10525473/21677092/6c8d97e8-d2ed-11e6-8966-a3b7af60c231.gif)


### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? no


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/cloverhearts/zeppelin ZEPPELIN-1812

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/zeppelin/pull/1848.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1848


commit b7cd700eccfd2ed51f29c17285bad36579c02208
Author: cloverhearts 
Date:   2017-01-05T10:09:38Z

fixed ace editor hide problem

commit 037993d23257b31836ad59d8805540d30dd1c420
Author: cloverhearts 
Date:   2017-01-06T04:09:33Z

fixed test case

commit c3efea2b93ce2defcf822ef380b5f66cc349c81a
Author: cloverhearts 
Date:   2017-01-06T05:25:26Z

Revert "fixed test case"

This reverts commit 037993d23257b31836ad59d8805540d30dd1c420.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] zeppelin pull request #1855: [ZEPPELIN-1913] Separate "Apache Zeppelin Confi...

[AhyoungRyu]

GitHub user AhyoungRyu opened a pull request:

https://github.com/apache/zeppelin/pull/1855

[ZEPPELIN-1913] Separate "Apache Zeppelin Configuration" from quickstart 
page

### What is this PR for?
I think [Quick 
Start](https://zeppelin.apache.org/docs/0.7.0-SNAPSHOT/install/install.html) 
page should contain minimum contents to get start Zeppelin for beginners. If 
not, this can be making the entrance barrier high. But after "SSL 
configuration" contents were added, configuration section came so long. So I 
created new page `configuration.md` and separated the all configuration related 
contents from Quick Start page.

### What type of PR is it?
Documentation

### What is the Jira issue?
[ZEPPELIN-1913](https://issues.apache.org/jira/browse/ZEPPELIN-1913)

### How should this be tested?
Outline the steps to test the PR here.

### Screenshots (if appropriate)
 - before (quickstart guide + Zeppelin configuration in one page)

![before](https://cloud.githubusercontent.com/assets/10060731/21708853/b7cae9ee-d41f-11e6-867a-3870ecd42efa.gif)

 - after (separated two page)

![after](https://cloud.githubusercontent.com/assets/10060731/21708865/d0cf7ca2-d41f-11e6-9ca2-0ea9649c896e.gif)


### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? no


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/AhyoungRyu/zeppelin separate/confContents

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/zeppelin/pull/1855.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1855


commit 9d0cb197e7971ebf644dee3506b551a8976ad089
Author: AhyoungRyu 
Date:   2017-01-06T05:31:44Z

Separate 'Zeppelin Configuration' section from quickstart

commit 54f9ad9fb50fe00f18ca0c8163e0abfe51097cc7
Author: AhyoungRyu 
Date:   2017-01-06T05:35:40Z

Update page description




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (ZEPPELIN-1913) Too long Zeppelin config contents are included in quickstart page

[Ahyoung Ryu (JIRA)]

Ahyoung Ryu created ZEPPELIN-1913:
-

 Summary: Too long Zeppelin config contents are included in 
quickstart page
 Key: ZEPPELIN-1913
 URL: https://issues.apache.org/jira/browse/ZEPPELIN-1913
 Project: Zeppelin
  Issue Type: Task
  Components: documentation
Reporter: Ahyoung Ryu
Assignee: Ahyoung Ryu
Priority: Minor
 Fix For: 0.7.0


After "Apache Zeppelin Configuration to enable SSL" contents were added, too 
many contents are included in [Quickstart 
page|https://zeppelin.apache.org/docs/0.7.0-SNAPSHOT/install/install.html] 
under ["Apache Zeppelin 
Configuration"|https://zeppelin.apache.org/docs/0.7.0-SNAPSHOT/install/install.html#apache-zeppelin-configuration]
 section. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Windows support ?

[Felix Cheung]

No, Travis does not support Windows
https://github.com/travis-ci/travis-ci/issues/2104

AppVeyor does...
https://www.appveyor.com/docs/installed-software/#operating-system

From: Jeff Zhang 
Sent: Thursday, January 5, 2017 7:04:18 PM
To: dev@zeppelin.apache.org
Subject: Re: Windows support ?

I think most of the community development is on *nix. Although zeppelin
support windows,  it would be very probable to break the windows support as
zeppelin is evolving fast and very few developer use windows. Not sure
whether travis support windows, if yes, we can add one windows build to
prevent the break on windows.



Sourav Mazumder 于2017年1月6日周五 上午10:53写道：

> I ported successfully a previous version of Zeppelin.
> If needed we can start with that as baseline after making necessary
> changes for the latest version.
>
> Regards,
> Sourav
>
> > On Jan 6, 2017, at 3:23 AM, Luciano Resende 
> wrote:
> >
> > I see a few sporadic issues/questions related to windows, which seems to
> > get lower priority and I was trying the building Zeppelin in windows
> > environment and there are some clear issues.
> >
> > - What is the official word around Zeppelin supporting Windows?
> >
> > - Is Windows a supported development environment for Zeppelin (e.g.
> > building)?
> >
> >
> > --
> > Luciano Resende
> > http://twitter.com/lresende1975
> > http://lresende.blogspot.com/
>

[GitHub] zeppelin issue #1812: [ZEPPELIN-1850] Use yarn instead of npm to reduce buil...

[1ambda]

Github user 1ambda commented on the issue:

https://github.com/apache/zeppelin/pull/1812
  
@Leemoonsoo :)

- Now user doens't need to care about `yarn.lock` and can use npm commands.
- CI failure is irrelevant








---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] zeppelin pull request #1854: [ZEPPELIN-1912] fix: Improve perosnalized mode ...

[1ambda]

GitHub user 1ambda opened a pull request:

https://github.com/apache/zeppelin/pull/1854

[ZEPPELIN-1912] fix: Improve perosnalized mode tooltip text

### What is this PR for?

Currently the tooltip for personalized mode is quite confusing. Since

- tooltip text is {{Collaboration mode}}
- and when i click the icon, dialog tell me {{Do you want to personalized 
your analysis?}}

**Tooltips should tell us what we can do instead of just stating current 
state as other tooltips do**

https://cloud.githubusercontent.com/assets/4968473/21708295/6ae778e0-d41a-11e6-8aa6-a30a3cebec74.png;>

https://cloud.githubusercontent.com/assets/4968473/21708297/6e30f01c-d41a-11e6-991e-504aadfa5dc9.png;>


### What type of PR is it?
[Improvement]

### Todos

Nothing

### What is the Jira issue?

[ZEPPELIN-1912](https://issues.apache.org/jira/browse/ZEPPELIN-1912)

### How should this be tested?

1. Enable shiro and login
2. Check the tooltip text while switching modes

### Screenshots (if appropriate)

 After

https://cloud.githubusercontent.com/assets/4968473/21708341/d869d6c4-d41a-11e6-808f-5eadd6f7727a.png;>

https://cloud.githubusercontent.com/assets/4968473/21708305/7c5cef24-d41a-11e6-8524-993a880300f2.png;>


### Questions:
* Does the licenses files need update? - NO
* Is there breaking changes for older versions? - NO
* Does this needs documentation? - NO


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/1ambda/zeppelin 
ZEPPELIN-1912/enhance-personalized-tooltip

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/zeppelin/pull/1854.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1854


commit 9aee7472f257d086dddc84d32b061e3aa7437d14
Author: 1ambda <1am...@gmail.com>
Date:   2017-01-06T05:12:24Z

fix: Improve perosnalized mode tooltip text




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (ZEPPELIN-1912) Better tooltip description in personalized mode

[Hoon Park (JIRA)]

Hoon Park created ZEPPELIN-1912:
---

 Summary: Better tooltip description in personalized mode
 Key: ZEPPELIN-1912
 URL: https://issues.apache.org/jira/browse/ZEPPELIN-1912
 Project: Zeppelin
  Issue Type: Improvement
  Components: front-end
Reporter: Hoon Park
 Attachments: confusing-1.png, confusing-2.png

Currently the tooltip for personalized mode is quite confusing. Since

- tooltip text is {{Collaboration mode}}
- and when i click the icon, dialog tell me {{Do you want to personalized your 
analysis?}}

Tooltips should tell us what we can do instead of just stating current state. 





--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Windows support ?

[Jeff Zhang]

I think most of the community development is on *nix. Although zeppelin
support windows,  it would be very probable to break the windows support as
zeppelin is evolving fast and very few developer use windows. Not sure
whether travis support windows, if yes, we can add one windows build to
prevent the break on windows.



Sourav Mazumder 于2017年1月6日周五 上午10:53写道：

> I ported successfully a previous version of Zeppelin.
> If needed we can start with that as baseline after making necessary
> changes for the latest version.
>
> Regards,
> Sourav
>
> > On Jan 6, 2017, at 3:23 AM, Luciano Resende 
> wrote:
> >
> > I see a few sporadic issues/questions related to windows, which seems to
> > get lower priority and I was trying the building Zeppelin in windows
> > environment and there are some clear issues.
> >
> > - What is the official word around Zeppelin supporting Windows?
> >
> > - Is Windows a supported development environment for Zeppelin (e.g.
> > building)?
> >
> >
> > --
> > Luciano Resende
> > http://twitter.com/lresende1975
> > http://lresende.blogspot.com/
>

[GitHub] zeppelin pull request #1849: [ZEPPELIN-1902] Fix blank default intp select b...

[AhyoungRyu]

Github user AhyoungRyu closed the pull request at:

https://github.com/apache/zeppelin/pull/1849


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] zeppelin pull request #1849: [ZEPPELIN-1902] Fix blank default intp select b...

[AhyoungRyu]

GitHub user AhyoungRyu reopened a pull request:

https://github.com/apache/zeppelin/pull/1849

[ZEPPELIN-1902] Fix blank default intp select box after Zeppelin started 
when Shiro on

### What is this PR for?
After #1520 we can set default interpreter when we create new note. 
Currently this feature is implemented to get `interpreterSetting` list when 
Zeppelin is initiated (can be checked in dev console). And it needs user's 
info. So there can be two cases like below.

 - Shiro off 
It tries to get interpreter setting lists as `anonymous` user

![shirooff](https://cloud.githubusercontent.com/assets/10060731/21676394/cce513d8-d378-11e6-9645-99255882b664.png)
In this case, the problem is not happened. If we don't turn Shiro on, it 
doesn't need user's info and Zeppelin automatically creates fake user(?) info 
as `anonymous`. So there is no problem to get the interpreter setting lists. 

 - Shiro on
tries to get interpreter setting lists as '  ' user since we're not 
loggined yet. 

![shiroon](https://cloud.githubusercontent.com/assets/10060731/21676414/db03515a-d378-11e6-99c9-34775e07a2e8.png)
 
So it's failed to get interpreter setting lists since it doesn't have any 
user info. It needs reloading to get it. 

![whenshiroon](https://cloud.githubusercontent.com/assets/10060731/21676657/e08ee03e-d379-11e6-8844-6baa673e0308.gif)


Actually we don't need to send `GET_INTERPRETER_SETTINGS` msg when Zeppelin 
started. It would be fine when we click "Create new note" button. So I removed 
`init()` (it's used only for getting interpreter settings) and made 
`getInterpreterSettings` called only when user clicks "Create new note" in both 
Zeppelin home and Navbar. 


### What type of PR is it?
Bug Fix

### What is the Jira issue?
[ZEPPELIN-1902](https://issues.apache.org/jira/browse/ZEPPELIN-1902)

### How should this be tested?
In master (not this branch)
1. [Turn Shiro 
on](https://zeppelin.apache.org/docs/0.7.0-SNAPSHOT/security/shiroauthentication.html#1-enable-shiro)
 and start(restart) Zeppelin 
2. Click "Create new note" in home (or Navbar)
3. the default interpreter select box will be like 
![screen shot 2017-01-05 at 7 16 08 
pm](https://cloud.githubusercontent.com/assets/10060731/21676986/7a005a80-d37b-11e6-80d9-1597914c4b05.png)

but it'll be okay after reloading. It shouldn't be like that with this 
patch. 


### Screenshots (if appropriate)
after fixing this problem

![whenshiroonafter](https://cloud.githubusercontent.com/assets/10060731/21676678/08a2360c-d37a-11e6-9a9b-03457b88a4fb.gif)
 

### Questions:
* Does the licenses files need update? no 
* Is there breaking changes for older versions? no
* Does this needs documentation? no


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/AhyoungRyu/zeppelin ZEPPELIN-1902

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/zeppelin/pull/1849.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1849


commit 30909338de44eef0171a00b4377b54a2acd94852
Author: AhyoungRyu 
Date:   2017-01-05T10:38:21Z

Fix blank default intp select box after Zeppelin started when Shiro on




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] zeppelin issue #1848: [ZEPPELIN-1812] Ace editor show/hide problem.

[minahlee]

Github user minahlee commented on the issue:

https://github.com/apache/zeppelin/pull/1848
  
@cloverhearts There are some lines that need to be modified in test.


https://github.com/apache/zeppelin/blob/master/zeppelin-server/src/test/java/org/apache/zeppelin/integration/ParagraphActionsIT.java#L466


https://github.com/apache/zeppelin/blob/master/zeppelin-server/src/test/java/org/apache/zeppelin/integration/ParagraphActionsIT.java#L480

Could you change these lines? Once we see the selenium test pass, I will 
merge it.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] zeppelin issue #1851: [ZEPPELIN-1908] Invalid Typo Method name

[AhyoungRyu]

Github user AhyoungRyu commented on the issue:

https://github.com/apache/zeppelin/pull/1851
  
LGTM 
CI failed only in Selenium test and it looks irrelevant to this change
```
Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 55.46 sec 
<<< FAILURE! - in org.apache.zeppelin.integration.InterpreterIT

testShowDescriptionOnInterpreterCreate(org.apache.zeppelin.integration.InterpreterIT)
  Time elapsed: 55.197 sec  <<< ERROR!
org.openqa.selenium.ElementNotVisibleException: Element is not currently 
visible and so may not be interacted with
```



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: Windows support ?

[Sourav Mazumder]

I ported successfully a previous version of Zeppelin. 
If needed we can start with that as baseline after making necessary changes for 
the latest version.

Regards,
Sourav

> On Jan 6, 2017, at 3:23 AM, Luciano Resende  wrote:
> 
> I see a few sporadic issues/questions related to windows, which seems to
> get lower priority and I was trying the building Zeppelin in windows
> environment and there are some clear issues.
> 
> - What is the official word around Zeppelin supporting Windows?
> 
> - Is Windows a supported development environment for Zeppelin (e.g.
> building)?
> 
> 
> -- 
> Luciano Resende
> http://twitter.com/lresende1975
> http://lresende.blogspot.com/

Re: Exporting Spark paragraphs as Spark Applications

[Jeff Zhang]

Hi Luciano, maybe I am wrong, just my two cents for your consideration.



Jeff Zhang 于2017年1月6日周五 上午8:32写道：

>
> Thanks Luciano.  I am not saying the community don't feel this is a good
> idea. It's just my personal opinion (maybe with some bias, I didn't talk
> with many customers as you)  I just feel maybe you can spend time on
> improving zeppelin to make zeppelin to do the job rather than exporting the
> jar and leverage other tools to deploy the jar.  Because I don't want you
> to waste time that maybe finally you find out customer are happy to do that
> in one central place: zeppelin.  Anyway, this is just my personal thinking,
> you can talk with your customers to hear their feedback.
>
>
> Luciano Resende 于2017年1月6日周五 上午5:01写道：
>
> Hi Jeff,
>
> While I agree with you that what you mentioned is completely acceptable for
> some users, particularly regarding the data science personas. Having said
> that, while talking with multiple enterprise companies, that have their own
> scheduler infrastructure with different quality of service or just want to
> deploy this as an app into their production environment which will have
> much more resources for running these apps with complete data sets, and
> currently they finish the experiment/development of the application in an
> interactive environment and them move their final code into a native spark
> application.
>
> Zeppelin is evolving quickly in this area, and I think that export as an
> application might be a good option for users that want to actually deploy
> their notebooks as native applications into their own Spark cluster.
>
> Having said that, if the community feels that this is not a required
> function in Zeppelin anymore, then I can continue with the development of
> the tool as a standalone command line tool. I was even thinking about
> expanding the functionality and implementing what is described in
> ZEPPELIN-1793.
>
> Thoughts ?
>
> On Thu, Jan 5, 2017 at 12:38 AM, Jeff Zhang  wrote:
>
> > Thanks Luciano. IIRC, what user want is to run the whole spark app, but
> > they don't care about whether it is in zeppelin or through a standard
> spark
> > app jar. I know zeppelin currently doesn't do well in converting note to
> > production spark app as Lee mentioned. But exporting note as jar seems a
> > short term solution, not a long term solution. I just feel when zeppelin
> > improve in this field, user might abandon this solution and transit to
> > zeppelin again. Here's some disadvantages I can see of this approach.
> >
> > 1.  If user want to change the code in iterative development , they have
> to
> > repeat the whole pipeline (change code in zeppelin -> export it to spark
> > jar -> redeploy this jar). This process is painful and wasting time.
> > 2.  Hard to debug and diagnose as code is changed/restructured when
> > exporting to jar
> > 3.  User have to leverage several distinct tools for the whole
> development
> > cycle (zeppelin, spark job server, and maybe cron job)
> >
> > Besides,  the OOM issue of Spark REPL Lee mentioned might not be a
> problem.
> > Because we can shutdown the app (close interpreter) after the app is
> done.
> >
> >
> >
> >
> >
> > Luciano Resende 于2017年1月5日周四 下午3:59写道：
> >
> > Some use cases discussed earlier on this thread:
> >
> > https://www.mail-archive.com/dev@zeppelin.apache.org/msg06323.html
> >
> > https://www.mail-archive.com/dev@zeppelin.apache.org/msg06332.html
> >
> > On Wed, Jan 4, 2017 at 4:51 PM, Jianfeng (Jeff) Zhang <
> > jzh...@hortonworks.com> wrote:
> >
> > >
> > > I don¹t understand why user want to export zeppelin note as spark
> > > application.
> > >
> > > If they want to trigger the running of spark app, why not use
> zeppelin¹s
> > > rest api for that. Even user export it as spark application, most of
> time
> > > in reality, they need to submit it through spark job server, so why not
> > > use zeppelin as a spark job server.
> > > And if the spark app fails, it is pretty hard to debug it, because the
> > > exporting tool has changed/restructured the source code.
> > >
> > >
> > > If this is a pretty large and complicated spark application, I don¹t
> > think
> > > zeppelin is a proper tool for that, they¹d better to use IDE for that
> > > project.
> > >
> > > BTW, After https://github.com/apache/zeppelin/pull/1799, user can
> define
> > > the dependency between paragraphs, and they can run one whole note
> which
> > > contains different interpreters.
> > >
> > >
> > >
> > > Best Regard,
> > > Jeff Zhang
> > >
> > >
> > >
> > >
> > >
> > > On 1/5/17, 2:25 AM, "Luciano Resende"  wrote:
> > >
> > > >I have made some progress with a tool to handle the points discussed
> in
> > > >this thread. It's currently a command line tool and given a Zeppelin
> > > >notebook (note.json) it generates a Spark scala application, compiles
> it
> > > >using the compiler embedded in the scala sdk and 

[GitHub] zeppelin pull request #1835: [MINOR][DOCS] Add troubleshooting section to sp...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/zeppelin/pull/1835


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] zeppelin pull request #1853: [ZEPPELIN-1911] Adding how to install evaluate ...

[astroshim]

GitHub user astroshim opened a pull request:

https://github.com/apache/zeppelin/pull/1853

[ZEPPELIN-1911] Adding how to install evaluate package to build R.

### What is this PR for?
This PR inform how to avoid build failing with R.


### What type of PR is it?
Documentation


### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-1911


### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? no


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/astroshim/zeppelin ZEPPELIN-1911

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/zeppelin/pull/1853.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1853


commit 2ca82573e497ebcdca373c2be1048b1b7a1b4afa
Author: astroshim 
Date:   2017-01-06T01:56:16Z

add doc how to install evaluate package




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (ZEPPELIN-1911) dependency ‘evaluate’ is not available for package ‘rzeppelin’

[Shim HyungSung (JIRA)]

Shim HyungSung created ZEPPELIN-1911:


 Summary: dependency ‘evaluate’ is not available for package 
‘rzeppelin’
 Key: ZEPPELIN-1911
 URL: https://issues.apache.org/jira/browse/ZEPPELIN-1911
 Project: Zeppelin
  Issue Type: Bug
Reporter: Shim HyungSung


I got following error when i build the zeppelin with R.

+++ dirname R/install-dev.sh
++ cd R
++ pwd
+ FWDIR=/Users/shim/zeppelin/r/R
+ LIB_DIR=/Users/shim/zeppelin/r/R/../../R/lib
+ mkdir -p /Users/shim/zeppelin/r/R/../../R/lib
+ pushd /Users/shim/zeppelin/r/R
+ R CMD INSTALL --library=/Users/shim/zeppelin/r/R/../../R/lib 
/Users/shim/zeppelin/r/R/rzeppelin/
ERROR: dependency ‘evaluate’ is not available for package ‘rzeppelin’
* removing ‘/Users/shim/zeppelin/r/lib/rzeppelin’
[INFO] 
[INFO] Reactor Summary:
[INFO]
[INFO] Zeppelin ... SUCCESS [  3.676 s]
[INFO] Zeppelin: Interpreter .. SUCCESS [  9.502 s]
[INFO] Zeppelin: Zengine .. SUCCESS [  6.573 s]
[INFO] Zeppelin: Display system apis .. SUCCESS [ 13.944 s]
[INFO] Zeppelin: Spark dependencies ... SUCCESS [ 41.658 s]
[INFO] Zeppelin: Spark  SUCCESS [ 19.214 s]
[INFO] Zeppelin: Markdown interpreter . SUCCESS [  1.040 s]
[INFO] Zeppelin: Angular interpreter .. SUCCESS [  0.332 s]
[INFO] Zeppelin: Shell interpreter  SUCCESS [  0.462 s]
[INFO] Zeppelin: Livy interpreter . SUCCESS [  6.015 s]
[INFO] Zeppelin: HBase interpreter  SUCCESS [  4.141 s]
[INFO] Zeppelin: Apache Pig Interpreter ... SUCCESS [  3.624 s]
[INFO] Zeppelin: PostgreSQL interpreter ... SUCCESS [  0.662 s]
[INFO] Zeppelin: JDBC interpreter . SUCCESS [  1.169 s]
[INFO] Zeppelin: File System Interpreters . SUCCESS [  1.374 s]
[INFO] Zeppelin: Flink  SUCCESS [  5.809 s]
[INFO] Zeppelin: Apache Ignite interpreter  SUCCESS [  0.922 s]
[INFO] Zeppelin: Kylin interpreter  SUCCESS [  0.470 s]
[INFO] Zeppelin: Python interpreter ... SUCCESS [  0.560 s]
[INFO] Zeppelin: Lens interpreter . SUCCESS [  3.323 s]
[INFO] Zeppelin: Apache Cassandra interpreter . SUCCESS [ 46.931 s]
[INFO] Zeppelin: Elasticsearch interpreter  SUCCESS [  2.890 s]
[INFO] Zeppelin: BigQuery interpreter . SUCCESS [  1.297 s]
[INFO] Zeppelin: Alluxio interpreter .. SUCCESS [  2.158 s]
[INFO] Zeppelin: Scio . SUCCESS [ 32.122 s]
[INFO] Zeppelin: web Application .. SUCCESS [04:01 min]
[INFO] Zeppelin: Server ... SUCCESS [ 15.204 s]
[INFO] Zeppelin: Packaging distribution ... SUCCESS [  3.915 s]
[INFO] Zeppelin: R Interpreter  FAILURE [ 15.748 s]
[INFO] 
[INFO] BUILD FAILURE
[INFO] 
[INFO] Total time: 08:07 min
[INFO] Finished at: 2017-01-05T17:38:12-08:00
[INFO] Final Memory: 225M/793M
[INFO] 
[ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.2.1:exec 
(default) on project zeppelin-zrinterpreter_2.10: Command execution failed. 
Process exited with an error: 1 (Exit value: 1) -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e 
switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please 
read the following articles:
[ERROR] [Help 1] 
http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn  -rf :zeppelin-zrinterpreter_2.10




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] zeppelin pull request #1852: [ZEPPELIN-1910] fix: Multiple dialog when don't...

[1ambda]

GitHub user 1ambda opened a pull request:

https://github.com/apache/zeppelin/pull/1852

[ZEPPELIN-1910] fix: Multiple dialog when don't have permission for run all 
para

### What is this PR for?

See multiple same dialog when user doesn't have permission for **Run all 
paragraphs** inside a note

 Implementation details

- Introduce new websocket message `RUN_ALL_PARAGRAPHS` since we need to 
broadcast *bringing dialog* message only once
- Refactor `NotebookServer.runParagraph` to avoid duplication
- Add necessary functions to backend and frontend

### What type of PR is it?
[Bug Fix]

### Todos

Fixed at once

### What is the Jira issue?
[ZEPPELIN-1910](https://issues.apache.org/jira/browse/ZEPPELIN-1910)

### How should this be tested?

1. Set permission to a note
2. Enable shiro and login as an user **who doens't have write permission**
3. Click *Run all paragraphs* button

### Screenshots (if appropriate)

 1. Before


![bug-multiple-dialog](https://cloud.githubusercontent.com/assets/4968473/21704503/440a9774-d3fd-11e6-8e41-fcad71c5c9e7.gif)

 2. After


![run-all-after-fixed](https://cloud.githubusercontent.com/assets/4968473/21704488/304fd578-d3fd-11e6-9f6e-d64c82c508df.gif)

### Questions:
* Does the licenses files need update? - NO
* Is there breaking changes for older versions? - NO
* Does this needs documentation? - NO


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/1ambda/zeppelin 
ZEPPELIN-1910/run-all-para-bring-multi-dialog

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/zeppelin/pull/1852.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1852


commit 02f325d18581e1d0bf129306b8dfb10408881706
Author: 1ambda <1am...@gmail.com>
Date:   2017-01-06T01:41:22Z

fix: Multiple dialog when don't have permission for run all para




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (ZEPPELIN-1910) BUG: "run all paragraphs" brings the same dialog multiple times when user dones't have permission

[Hoon Park (JIRA)]

Hoon Park created ZEPPELIN-1910:
---

 Summary: BUG: "run all paragraphs" brings the same dialog multiple 
times when user dones't have permission
 Key: ZEPPELIN-1910
 URL: https://issues.apache.org/jira/browse/ZEPPELIN-1910
 Project: Zeppelin
  Issue Type: Bug
  Components: front-end
Reporter: Hoon Park
 Fix For: 0.7.0


Run all paragraphs brings the same dialog multiple times when user dones't have 
permission



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (ZEPPELIN-1909) spark session with hive support can't be created correctly

[lichenglin (JIRA)]

lichenglin created ZEPPELIN-1909:


 Summary: spark session with hive support can't be created correctly
 Key: ZEPPELIN-1909
 URL: https://issues.apache.org/jira/browse/ZEPPELIN-1909
 Project: Zeppelin
  Issue Type: Bug
  Components: Interpreters
Affects Versions: 0.7.0
Reporter: lichenglin
Priority: Blocker
 Fix For: 0.7.0


{code}

if (useHiveContext()) {
  if (hiveClassesArePresent()) {
Utils.invokeMethod(builder, "enableHiveSupport");
sparkSession = Utils.invokeMethod(builder, "getOrCreate");
logger.info("Created Spark session with Hive support");
  } else {
Utils.invokeMethod(builder, "config",
new Class[]{ String.class, String.class},
new Object[]{ "spark.sql.catalogImplementation", "in-memory"});
sparkSession = Utils.invokeMethod(builder, "getOrCreate");
logger.info("Created Spark session with Hive support");
  }
} 
{code}

{code}
 /**
   * See org.apache.spark.sql.SparkSession.hiveClassesArePresent
   * @return
   */
  private boolean hiveClassesArePresent() {
try {
  this.getClass().forName("org.apache.spark.sql.hive.HiveSessionState");
  this.getClass().forName("org.apache.spark.sql.hive.HiveSharedState");
  this.getClass().forName("org.apache.hadoop.hive.conf.HiveConf");
  return true;
} catch (ClassNotFoundException | NoClassDefFoundError e) {
  return false;
}
  }
{code}

since "org.apache.spark.sql.hive.HiveSharedState" has been removed from 
spark2.1.x 
So hiveClassesArePresent will always return false.
This will cause spark session with hive support would be created with 
"spark.sql.catalogImplementation=in-memory"

May be just remove 
{code}
  this.getClass().forName("org.apache.spark.sql.hive.HiveSharedState");
{code}

is OK?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] zeppelin issue #1840: [ZEPPELIN-1730, 1587] add spark impersonation through ...

[zjffdu]

Github user zjffdu commented on the issue:

https://github.com/apache/zeppelin/pull/1840
  
```
user configures export ZEPPELIN_IMPERSONATE_CMD in here with kinit 
@ -k -t  and then it's run before spark-submit
```
One concern is that this requires all the interpreters of one user share 
the same keytab/principal. e.g. spark interpreter may affect shell interpreter 
if they use different keytab/principal for the same user. For the long term, we 
may need to put security related settings in one central place rather than in 
each interpreter setting. 



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: Exporting Spark paragraphs as Spark Applications

[Jeff Zhang]

Thanks Luciano.  I am not saying the community don't feel this is a good
idea. It's just my personal opinion (maybe with some bias, I didn't talk
with many customers as you)  I just feel maybe you can spend time on
improving zeppelin to make zeppelin to do the job rather than exporting the
jar and leverage other tools to deploy the jar.  Because I don't want you
to waste time that maybe finally you find out customer are happy to do that
in one central place: zeppelin.  Anyway, this is just my personal thinking,
you can talk with your customers to hear their feedback.


Luciano Resende 于2017年1月6日周五 上午5:01写道：

> Hi Jeff,
>
> While I agree with you that what you mentioned is completely acceptable for
> some users, particularly regarding the data science personas. Having said
> that, while talking with multiple enterprise companies, that have their own
> scheduler infrastructure with different quality of service or just want to
> deploy this as an app into their production environment which will have
> much more resources for running these apps with complete data sets, and
> currently they finish the experiment/development of the application in an
> interactive environment and them move their final code into a native spark
> application.
>
> Zeppelin is evolving quickly in this area, and I think that export as an
> application might be a good option for users that want to actually deploy
> their notebooks as native applications into their own Spark cluster.
>
> Having said that, if the community feels that this is not a required
> function in Zeppelin anymore, then I can continue with the development of
> the tool as a standalone command line tool. I was even thinking about
> expanding the functionality and implementing what is described in
> ZEPPELIN-1793.
>
> Thoughts ?
>
> On Thu, Jan 5, 2017 at 12:38 AM, Jeff Zhang  wrote:
>
> > Thanks Luciano. IIRC, what user want is to run the whole spark app, but
> > they don't care about whether it is in zeppelin or through a standard
> spark
> > app jar. I know zeppelin currently doesn't do well in converting note to
> > production spark app as Lee mentioned. But exporting note as jar seems a
> > short term solution, not a long term solution. I just feel when zeppelin
> > improve in this field, user might abandon this solution and transit to
> > zeppelin again. Here's some disadvantages I can see of this approach.
> >
> > 1.  If user want to change the code in iterative development , they have
> to
> > repeat the whole pipeline (change code in zeppelin -> export it to spark
> > jar -> redeploy this jar). This process is painful and wasting time.
> > 2.  Hard to debug and diagnose as code is changed/restructured when
> > exporting to jar
> > 3.  User have to leverage several distinct tools for the whole
> development
> > cycle (zeppelin, spark job server, and maybe cron job)
> >
> > Besides,  the OOM issue of Spark REPL Lee mentioned might not be a
> problem.
> > Because we can shutdown the app (close interpreter) after the app is
> done.
> >
> >
> >
> >
> >
> > Luciano Resende 于2017年1月5日周四 下午3:59写道：
> >
> > Some use cases discussed earlier on this thread:
> >
> > https://www.mail-archive.com/dev@zeppelin.apache.org/msg06323.html
> >
> > https://www.mail-archive.com/dev@zeppelin.apache.org/msg06332.html
> >
> > On Wed, Jan 4, 2017 at 4:51 PM, Jianfeng (Jeff) Zhang <
> > jzh...@hortonworks.com> wrote:
> >
> > >
> > > I don¹t understand why user want to export zeppelin note as spark
> > > application.
> > >
> > > If they want to trigger the running of spark app, why not use
> zeppelin¹s
> > > rest api for that. Even user export it as spark application, most of
> time
> > > in reality, they need to submit it through spark job server, so why not
> > > use zeppelin as a spark job server.
> > > And if the spark app fails, it is pretty hard to debug it, because the
> > > exporting tool has changed/restructured the source code.
> > >
> > >
> > > If this is a pretty large and complicated spark application, I don¹t
> > think
> > > zeppelin is a proper tool for that, they¹d better to use IDE for that
> > > project.
> > >
> > > BTW, After https://github.com/apache/zeppelin/pull/1799, user can
> define
> > > the dependency between paragraphs, and they can run one whole note
> which
> > > contains different interpreters.
> > >
> > >
> > >
> > > Best Regard,
> > > Jeff Zhang
> > >
> > >
> > >
> > >
> > >
> > > On 1/5/17, 2:25 AM, "Luciano Resende"  wrote:
> > >
> > > >I have made some progress with a tool to handle the points discussed
> in
> > > >this thread. It's currently a command line tool and given a Zeppelin
> > > >notebook (note.json) it generates a Spark scala application, compiles
> it
> > > >using the compiler embedded in the scala sdk and then package all
> these
> > > >resources into a jar that works with spark-submit command.
> > > >
> > > >I would like to start prototyping the 

Windows support ?

[Luciano Resende]

I see a few sporadic issues/questions related to windows, which seems to
get lower priority and I was trying the building Zeppelin in windows
environment and there are some clear issues.

- What is the official word around Zeppelin supporting Windows?

- Is Windows a supported development environment for Zeppelin (e.g.
building)?


-- 
Luciano Resende
http://twitter.com/lresende1975
http://lresende.blogspot.com/

Re: Exporting Spark paragraphs as Spark Applications

[Luciano Resende]

Hi Jeff,

While I agree with you that what you mentioned is completely acceptable for
some users, particularly regarding the data science personas. Having said
that, while talking with multiple enterprise companies, that have their own
scheduler infrastructure with different quality of service or just want to
deploy this as an app into their production environment which will have
much more resources for running these apps with complete data sets, and
currently they finish the experiment/development of the application in an
interactive environment and them move their final code into a native spark
application.

Zeppelin is evolving quickly in this area, and I think that export as an
application might be a good option for users that want to actually deploy
their notebooks as native applications into their own Spark cluster.

Having said that, if the community feels that this is not a required
function in Zeppelin anymore, then I can continue with the development of
the tool as a standalone command line tool. I was even thinking about
expanding the functionality and implementing what is described in
ZEPPELIN-1793.

Thoughts ?

On Thu, Jan 5, 2017 at 12:38 AM, Jeff Zhang  wrote:

> Thanks Luciano. IIRC, what user want is to run the whole spark app, but
> they don't care about whether it is in zeppelin or through a standard spark
> app jar. I know zeppelin currently doesn't do well in converting note to
> production spark app as Lee mentioned. But exporting note as jar seems a
> short term solution, not a long term solution. I just feel when zeppelin
> improve in this field, user might abandon this solution and transit to
> zeppelin again. Here's some disadvantages I can see of this approach.
>
> 1.  If user want to change the code in iterative development , they have to
> repeat the whole pipeline (change code in zeppelin -> export it to spark
> jar -> redeploy this jar). This process is painful and wasting time.
> 2.  Hard to debug and diagnose as code is changed/restructured when
> exporting to jar
> 3.  User have to leverage several distinct tools for the whole development
> cycle (zeppelin, spark job server, and maybe cron job)
>
> Besides,  the OOM issue of Spark REPL Lee mentioned might not be a problem.
> Because we can shutdown the app (close interpreter) after the app is done.
>
>
>
>
>
> Luciano Resende 于2017年1月5日周四 下午3:59写道：
>
> Some use cases discussed earlier on this thread:
>
> https://www.mail-archive.com/dev@zeppelin.apache.org/msg06323.html
>
> https://www.mail-archive.com/dev@zeppelin.apache.org/msg06332.html
>
> On Wed, Jan 4, 2017 at 4:51 PM, Jianfeng (Jeff) Zhang <
> jzh...@hortonworks.com> wrote:
>
> >
> > I don¹t understand why user want to export zeppelin note as spark
> > application.
> >
> > If they want to trigger the running of spark app, why not use zeppelin¹s
> > rest api for that. Even user export it as spark application, most of time
> > in reality, they need to submit it through spark job server, so why not
> > use zeppelin as a spark job server.
> > And if the spark app fails, it is pretty hard to debug it, because the
> > exporting tool has changed/restructured the source code.
> >
> >
> > If this is a pretty large and complicated spark application, I don¹t
> think
> > zeppelin is a proper tool for that, they¹d better to use IDE for that
> > project.
> >
> > BTW, After https://github.com/apache/zeppelin/pull/1799, user can define
> > the dependency between paragraphs, and they can run one whole note which
> > contains different interpreters.
> >
> >
> >
> > Best Regard,
> > Jeff Zhang
> >
> >
> >
> >
> >
> > On 1/5/17, 2:25 AM, "Luciano Resende"  wrote:
> >
> > >I have made some progress with a tool to handle the points discussed in
> > >this thread. It's currently a command line tool and given a Zeppelin
> > >notebook (note.json) it generates a Spark scala application, compiles it
> > >using the compiler embedded in the scala sdk and then package all these
> > >resources into a jar that works with spark-submit command.
> > >
> > >I would like to start prototyping the integration into the Zeppelin UI
> and
> > >I was wondering if it would be ok to use the above jar as a dependency
> > >(e.g. from a maven release) and integrate into zeppelin...
> > >
> > >Thoughts ?
> > >
> > >
> > >On Mon, Sep 19, 2016 at 7:47 AM, Sourav Mazumder <
> > >sourav.mazumde...@gmail.com> wrote:
> > >
> > >> To Moon's point, This is what my vision is around this feature -
> > >>
> > >> 1. Use should be able to package 1, more than one, all of the
> > >>paragraphs in
> > >> a Notebook to create a Jar file which can be used with Spark-Submit.
> > >>
> > >> 2. The tool should automatically remove the all the interactive
> > >>statements
> > >> like print, show etc.
> > >>
> > >> 3. The tool should automatically create a Main class in addition to
> the
> > >>jar
> > >> file(s) which will internally call the respective jar. User can then
> > 

[GitHub] zeppelin issue #1848: [ZEPPELIN-1812] Ace editor show/hide problem.

[astroshim]

Github user astroshim commented on the issue:

https://github.com/apache/zeppelin/pull/1848
  
LGTM


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] zeppelin issue #1840: [ZEPPELIN-1730, 1587] add spark impersonation through ...

[Tagar]

Github user Tagar commented on the issue:

https://github.com/apache/zeppelin/pull/1840
  
Thank you @khalidhuseynov .
On 

> user configures export ZEPPELIN_IMPERSONATE_CMD in here with kinit 
@ -k -t  and then it's run before spark-submit

The only problem I see with this option is that Kerberos tickets will not 
be renewed automatically, and will expire at some point.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] zeppelin issue #1840: [ZEPPELIN-1730, 1587] add spark impersonation through ...

[khalidhuseynov]

Github user khalidhuseynov commented on the issue:

https://github.com/apache/zeppelin/pull/1840
  
@zjffdu i didn't try secured cluster mode yet, but as i checked spark 
documentation, they indeed don't allow using `--principal` & `--keytab` for 
spark-submit alongside with `--proxy-user` because of security issue on 
exposing keytab. Then possible solutions could be: 
1. user configures `export ZEPPELIN_IMPERSONATE_CMD` in 
[here](https://github.com/apache/zeppelin/blob/d1fc86b7b2d2012c0323345166c98cc02886e9f1/conf/zeppelin-env.sh.template#L83)
 with `kinit @ -k -t ` and then it's run before 
`spark-submit`
2. don't use `--proxy-user` in cluster mode
3. other suggestions



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] zeppelin pull request #1844: [ZEPPELIN-1825] Use versioned notebook storage ...

[khalidhuseynov]

GitHub user khalidhuseynov reopened a pull request:

https://github.com/apache/zeppelin/pull/1844

[ZEPPELIN-1825] Use versioned notebook storage by default 

### What is this PR for?
Due to recent work on versioned notes under ZEPPELIN-1044, we can use by 
default Git storage which extends local storage. so the behaviour will be same 
as before with additional ability to use versioning out of box without 
configuring it.


### What type of PR is it?
Improvement |  Refactoring

### Todos
* [x] - change VFS to Git

### What is the Jira issue?
[ZEPPELIN-1825](https://issues.apache.org/jira/browse/ZEPPELIN-1825)

### How should this be tested?
start zeppelin and by default be able to use version control (if git client 
installed)

### Screenshots (if appropriate)

### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? no


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/khalidhuseynov/incubator-zeppelin 
default-git-repo

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/zeppelin/pull/1844.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1844


commit cb9e9fb394851ae6028003bdf6d9f7af6a7b4ec1
Author: Khalid Huseynov 
Date:   2017-01-05T05:24:50Z

modify default storage to git repo

commit 5edd5e0a321c10b94100ff9dcdca67e35bf5ac2c
Author: Khalid Huseynov 
Date:   2017-01-05T05:32:01Z

fix in zeppelin-site.xml




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] zeppelin pull request #1844: [ZEPPELIN-1825] Use versioned notebook storage ...

[khalidhuseynov]

Github user khalidhuseynov closed the pull request at:

https://github.com/apache/zeppelin/pull/1844


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] zeppelin issue #1845: [ZEPPELIN-1906] Use multiple InterpreterResult for dis...

[prabhjyotsingh]

Github user prabhjyotsingh commented on the issue:

https://github.com/apache/zeppelin/pull/1845
  
@zjffdu replaced regex with slightly better logic, let me know what you 
think about this.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] zeppelin issue #1851: [ZEPPELIN-1908] Invalid Typo Method name

[soralee]

Github user soralee commented on the issue:

https://github.com/apache/zeppelin/pull/1851
  
Looks good to me 👍 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] zeppelin pull request #1851: [ZEPPELIN-1908] Invalid Typo Method name

[cloverhearts]

GitHub user cloverhearts opened a pull request:

https://github.com/apache/zeppelin/pull/1851

[ZEPPELIN-1908] Invalid Typo Method name

### What is this PR for?
Invalid Typo Method name

closeAndRmoveAllInterpreterGroups -> closeAndRemoveAllInterpreterGroups

### What type of PR is it?
Bug Fix

### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-1908

### Questions: no
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? no


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/cloverhearts/zeppelin ZEPPELIN-1908

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/zeppelin/pull/1851.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1851


commit 0d2bbca742891aa14e4be17f97dfa79551d97c70
Author: cloverhearts 
Date:   2017-01-05T13:02:53Z

fixed method typo




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (ZEPPELIN-1908) method name typo for closeAndRemoveAllInterpreterGroups

[CloverHearts (JIRA)]

CloverHearts created ZEPPELIN-1908:
--

 Summary: method name typo for closeAndRemoveAllInterpreterGroups
 Key: ZEPPELIN-1908
 URL: https://issues.apache.org/jira/browse/ZEPPELIN-1908
 Project: Zeppelin
  Issue Type: Bug
Reporter: CloverHearts


invalid method name closeAndRemoveAllInterpreterGroups in Interpreter 
setting.java and interpreterGroup.java

closeAndRmoveAllInterpreterGroups -> closeAndRemoveAllInterpreterGroups



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] zeppelin issue #1848: [ZEPPELIN-1812] Ace editor show/hide problem.

[minahlee]

Github user minahlee commented on the issue:

https://github.com/apache/zeppelin/pull/1848
  
Tested, LGTM


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] zeppelin issue #1838: [ZEPPELIN-1832] Fixed a bug in zombie process when Zep...

[cloverhearts]

Github user cloverhearts commented on the issue:

https://github.com/apache/zeppelin/pull/1838
  
But actually this is a temporary resolved.
As a result, Zombie Process issues have been mitigated.

Zomebie Process is very important and risky issue.
So, I felt the need to take minimal steps.

This fix minimizes the potential for Zombie.
I forced the reference count to zero at shutdown to terminate the process.

The exact problem with the Zombie Process is:
1. Interpreter reference count management failure
2. Close failed while working on RemoteInterpreter
3. Close the RemoteInterpreter Unknown stop.
4. Termination timing issues not yet discovered ..

Therefore, I feel it is necessary to take temporary measures through this 
PR, to register detailed issues and gradually resolve them.

What do you think?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] zeppelin pull request #1850: [ZEPPELIN-1731] Fix unexpected EditOnDoubleClic...

[minahlee]

GitHub user minahlee opened a pull request:

https://github.com/apache/zeppelin/pull/1850

[ZEPPELIN-1731] Fix unexpected EditOnDoubleClick

### What is this PR for?
This PR fixes wrong `EditOnDoubleClick` value in editor setting when 
default bound interpreter has changed.

### What type of PR is it?
Bug Fix

### Todos
* [ ] - Task

### What is the Jira issue?
[ZEPPELIN-1731](https://issues.apache.org/jira/browse/ZEPPELIN-1731)

### How should this be tested?
1. Set default interpreter to `md`.
2. Write below code in first paragraph and execute the paragraph. The 
editor should be hidden when you run.
```
## abc
pwd
```
3. Double click the result to make editor to be shown.
4. Change default interpreter to `sh`. At this point syntax highlight on 
`## abc` should be changed from blue to green.
5. Execute paragraph and see if editor stays displayed.

### Screenshots (if appropriate)
**Before**
![jan-05-2017 
21-39-46](https://cloud.githubusercontent.com/assets/8503346/21681060/c69c68f6-d390-11e6-87fe-aebebb5244d7.gif)


**After**
![jan-05-2017 
21-38-57](https://cloud.githubusercontent.com/assets/8503346/21681101/fbcd19ee-d390-11e6-98ff-296eb679677e.gif)




### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? no


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/minahlee/zeppelin ZEPPELIN-1731

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/zeppelin/pull/1850.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1850


commit 6e627088b94325efeaebe21dacf421ea26bd0997
Author: Mina Lee 
Date:   2017-01-05T12:18:43Z

Fix unexpected EditOnDoubleClick




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] zeppelin issue #1845: [ZEPPELIN-1906] Use multiple InterpreterResult for dis...

[r-kamath]

Github user r-kamath commented on the issue:

https://github.com/apache/zeppelin/pull/1845
  
Tested. LGTM.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] zeppelin issue #1848: [ZEPPELIN-1812] Ace editor show/hide problem.

[cloverhearts]

Github user cloverhearts commented on the issue:

https://github.com/apache/zeppelin/pull/1848
  
ci reopen


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] zeppelin pull request #1848: [ZEPPELIN-1812] Ace editor show/hide problem.

[cloverhearts]

GitHub user cloverhearts reopened a pull request:

https://github.com/apache/zeppelin/pull/1848

[ZEPPELIN-1812] Ace editor show/hide problem.

### What is this PR for?
When editor hide is on and page refresh then editor is gone.
I was solve this problem.

### What type of PR is it?
Bug Fix

### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-1812

### How should this be tested?
1. hide to editor mode on in paragraph configure.
2. page refresh (f5)
3. enable to show editor mode in paragraph configure.
4. if show editor then is correct.
### Screenshots (if appropriate)
 before

![badeditor](https://cloud.githubusercontent.com/assets/10525473/21677078/62136de2-d2ed-11e6-9418-0be4453fd839.gif)

 after

![goodeditor](https://cloud.githubusercontent.com/assets/10525473/21677092/6c8d97e8-d2ed-11e6-8966-a3b7af60c231.gif)


### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? no


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/cloverhearts/zeppelin ZEPPELIN-1812

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/zeppelin/pull/1848.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1848


commit b7cd700eccfd2ed51f29c17285bad36579c02208
Author: cloverhearts 
Date:   2017-01-05T10:09:38Z

fixed ace editor hide problem




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] zeppelin issue #1814: [ZEPPELIN-1869] changed the API response to generate t...

[cloverhearts]

Github user cloverhearts commented on the issue:

https://github.com/apache/zeppelin/pull/1814
  
@Leemoonsoo Ah! sorry, i will fix!


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] zeppelin pull request #1849: [ZEPPELIN-1902] Fix blank default intp select b...

[AhyoungRyu]

GitHub user AhyoungRyu opened a pull request:

https://github.com/apache/zeppelin/pull/1849

[ZEPPELIN-1902] Fix blank default intp select box after Zeppelin started 
when Shiro on

### What is this PR for?
After #1520 we can set default interpreter when we create new note. 
Currently this feature is implemented to get `interpreterSetting` list when 
Zeppelin is initiated (can be checked in dev console). And it needs user's 
info. So there can be two cases like below.

 - Shiro off 
It tries to get interpreter setting lists as `anonymous` user

![shirooff](https://cloud.githubusercontent.com/assets/10060731/21676394/cce513d8-d378-11e6-9645-99255882b664.png)
In this case, the problem is not happened. If we don't turn Shiro on, it 
doesn't need user's info and Zeppelin automatically creates fake user(?) info 
as `anonymous`. So there is no problem to get the interpreter setting lists. 

 - Shiro on
tries to get interpreter setting lists as '  ' user since we're not 
loggined yet. 

![shiroon](https://cloud.githubusercontent.com/assets/10060731/21676414/db03515a-d378-11e6-99c9-34775e07a2e8.png)
 
So it's failed to get interpreter setting lists since it doesn't have any 
user info. It needs reloading to get it. 

![whenshiroon](https://cloud.githubusercontent.com/assets/10060731/21676657/e08ee03e-d379-11e6-8844-6baa673e0308.gif)


Actually we don't need to send `GET_INTERPRETER_SETTINGS` msg when Zeppelin 
started. It would be fine when we click "Create new note" button. So I removed 
`init()` (it's used only for getting interpreter settings) and made 
`getInterpreterSettings` called only when user clicks "Create new note" in both 
Zeppelin home and Navbar. 


### What type of PR is it?
Bug Fix

### What is the Jira issue?
[ZEPPELIN-1902](https://issues.apache.org/jira/browse/ZEPPELIN-1902)

### How should this be tested?
In master (not this branch)
1. [Turn Shiro 
on](https://zeppelin.apache.org/docs/0.7.0-SNAPSHOT/security/shiroauthentication.html#1-enable-shiro)
 and start(restart) Zeppelin 
2. Click "Create new note" in home (or Navbar)
3. the default interpreter select box will be like 
![screen shot 2017-01-05 at 7 16 08 
pm](https://cloud.githubusercontent.com/assets/10060731/21676986/7a005a80-d37b-11e6-80d9-1597914c4b05.png)

but it'll be okay after reloading. It shouldn't be like that with this 
patch. 


### Screenshots (if appropriate)
after fixing this problem

![whenshiroonafter](https://cloud.githubusercontent.com/assets/10060731/21676678/08a2360c-d37a-11e6-9a9b-03457b88a4fb.gif)
 

### Questions:
* Does the licenses files need update? no 
* Is there breaking changes for older versions? no
* Does this needs documentation? no


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/AhyoungRyu/zeppelin ZEPPELIN-1902

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/zeppelin/pull/1849.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1849


commit a321ba72958d61c3f7694913b48c80f2f797475d
Author: AhyoungRyu 
Date:   2017-01-05T09:40:23Z

Fix interpreterSettings list showing up issue when Shiro off




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] zeppelin pull request #1848: [ZEPPELIN-1812] Ace editor show/hide problem.

[cloverhearts]

GitHub user cloverhearts opened a pull request:

https://github.com/apache/zeppelin/pull/1848

[ZEPPELIN-1812] Ace editor show/hide problem.

### What is this PR for?
When editor hide is on and page refresh then editor is gone.
I was solve this problem.

### What type of PR is it?
Bug Fix

### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-1812

### How should this be tested?
1. hide to editor mode on in paragraph configure.
2. page refresh (f5)
3. enable to show editor mode in paragraph configure.
4. if show editor then is correct.
### Screenshots (if appropriate)
 before

![badeditor](https://cloud.githubusercontent.com/assets/10525473/21677078/62136de2-d2ed-11e6-9418-0be4453fd839.gif)

 after

![goodeditor](https://cloud.githubusercontent.com/assets/10525473/21677092/6c8d97e8-d2ed-11e6-8966-a3b7af60c231.gif)


### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? no


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/cloverhearts/zeppelin ZEPPELIN-1812

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/zeppelin/pull/1848.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1848


commit b7cd700eccfd2ed51f29c17285bad36579c02208
Author: cloverhearts 
Date:   2017-01-05T10:09:38Z

fixed ace editor hide problem




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] zeppelin pull request #1847: [MINOR] Restore paragraph control button margin...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/zeppelin/pull/1847


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (ZEPPELIN-1907) Shell Interpreter does not renew ticket on secure cluster

[Prabhu Joseph (JIRA)]

Prabhu Joseph created ZEPPELIN-1907:
---

 Summary: Shell Interpreter does not renew ticket on secure cluster
 Key: ZEPPELIN-1907
 URL: https://issues.apache.org/jira/browse/ZEPPELIN-1907
 Project: Zeppelin
  Issue Type: Bug
Reporter: Prabhu Joseph


Kerberos ticket and renew lifetime are set to 1 hour. On accessing secure 
hadoop from shell interpreter, it does kinit and returns result successfully 
but after 1 hour, the ticket gets expired and hadoop list fails with below 
exception.

{code}
%sh
hadoop fs -ls /

17/01/05 09:29:45 WARN ipc.Client: Exception encountered while connecting to 
the server : 
javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: 
No valid credentials provided (Mechanism level: Failed to find any Kerberos 
tgt)]
at 
com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:211)
at 
org.apache.hadoop.security.SaslRpcClient.saslConnect(SaslRpcClient.java:413)
at 
org.apache.hadoop.ipc.Client$Connection.setupSaslConnection(Client.java:595)
at org.apache.hadoop.ipc.Client$Connection.access$2000(Client.java:397)
at org.apache.hadoop.ipc.Client$Connection$2.run(Client.java:762)
at org.apache.hadoop.ipc.Client$Connection$2.run(Client.java:758)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724)
at 
org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:757)
at org.apache.hadoop.ipc.Client$Connection.access$3200(Client.java:397)
at org.apache.hadoop.ipc.Client.getConnection(Client.java:1618)
at org.apache.hadoop.ipc.Client.call(Client.java:1449)
at org.apache.hadoop.ipc.Client.call(Client.java:1396)
at 
org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:233)
at com.sun.proxy.$Proxy10.getFileInfo(Unknown Source)
at 
org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getFileInfo(ClientNamenodeProtocolTranslatorPB.java:816)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:278)
at 
org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:194)
at 
org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:176)
at com.sun.proxy.$Proxy11.getFileInfo(Unknown Source)
at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:2158)
at 
org.apache.hadoop.hdfs.DistributedFileSystem$25.doCall(DistributedFileSystem.java:1423)
at 
org.apache.hadoop.hdfs.DistributedFileSystem$25.doCall(DistributedFileSystem.java:1419)
at 
org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
at 
org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1419)
at org.apache.hadoop.fs.Globber.getFileStatus(Globber.java:57)
at org.apache.hadoop.fs.Globber.glob(Globber.java:265)
at org.apache.hadoop.fs.FileSystem.globStatus(FileSystem.java:1674)
at org.apache.hadoop.fs.shell.PathData.expandAsGlob(PathData.java:326)
at org.apache.hadoop.fs.shell.Command.expandArgument(Command.java:235)
at org.apache.hadoop.fs.shell.Command.expandArguments(Command.java:218)
at 
org.apache.hadoop.fs.shell.FsCommand.processRawArguments(FsCommand.java:103)
at org.apache.hadoop.fs.shell.Command.run(Command.java:165)
at org.apache.hadoop.fs.FsShell.run(FsShell.java:297)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:90)
at org.apache.hadoop.fs.FsShell.main(FsShell.java:350)
Caused by: GSSException: No valid credentials provided (Mechanism level: Failed 
to find any Kerberos tgt)
at 
sun.security.jgss.krb5.Krb5InitCredential.getInstance(Krb5InitCredential.java:147)
at 
sun.security.jgss.krb5.Krb5MechFactory.getCredentialElement(Krb5MechFactory.java:122)
at 
sun.security.jgss.krb5.Krb5MechFactory.getMechanismContext(Krb5MechFactory.java:187)
at 
sun.security.jgss.GSSManagerImpl.getMechanismContext(GSSManagerImpl.java:224)
at 
sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:212)
at 
sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179)
at 

[GitHub] zeppelin issue #1847: [MINOR] Restore paragraph control button margin style

[soralee]

Github user soralee commented on the issue:

https://github.com/apache/zeppelin/pull/1847
  
LGTM :+1: 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: Exporting Spark paragraphs as Spark Applications

[Jeff Zhang]

Thanks Luciano. IIRC, what user want is to run the whole spark app, but
they don't care about whether it is in zeppelin or through a standard spark
app jar. I know zeppelin currently doesn't do well in converting note to
production spark app as Lee mentioned. But exporting note as jar seems a
short term solution, not a long term solution. I just feel when zeppelin
improve in this field, user might abandon this solution and transit to
zeppelin again. Here's some disadvantages I can see of this approach.

1.  If user want to change the code in iterative development , they have to
repeat the whole pipeline (change code in zeppelin -> export it to spark
jar -> redeploy this jar). This process is painful and wasting time.
2.  Hard to debug and diagnose as code is changed/restructured when
exporting to jar
3.  User have to leverage several distinct tools for the whole development
cycle (zeppelin, spark job server, and maybe cron job)

Besides,  the OOM issue of Spark REPL Lee mentioned might not be a problem.
Because we can shutdown the app (close interpreter) after the app is done.





Luciano Resende 于2017年1月5日周四 下午3:59写道：

Some use cases discussed earlier on this thread:

https://www.mail-archive.com/dev@zeppelin.apache.org/msg06323.html

https://www.mail-archive.com/dev@zeppelin.apache.org/msg06332.html

On Wed, Jan 4, 2017 at 4:51 PM, Jianfeng (Jeff) Zhang <
jzh...@hortonworks.com> wrote:

>
> I don¹t understand why user want to export zeppelin note as spark
> application.
>
> If they want to trigger the running of spark app, why not use zeppelin¹s
> rest api for that. Even user export it as spark application, most of time
> in reality, they need to submit it through spark job server, so why not
> use zeppelin as a spark job server.
> And if the spark app fails, it is pretty hard to debug it, because the
> exporting tool has changed/restructured the source code.
>
>
> If this is a pretty large and complicated spark application, I don¹t think
> zeppelin is a proper tool for that, they¹d better to use IDE for that
> project.
>
> BTW, After https://github.com/apache/zeppelin/pull/1799, user can define
> the dependency between paragraphs, and they can run one whole note which
> contains different interpreters.
>
>
>
> Best Regard,
> Jeff Zhang
>
>
>
>
>
> On 1/5/17, 2:25 AM, "Luciano Resende"  wrote:
>
> >I have made some progress with a tool to handle the points discussed in
> >this thread. It's currently a command line tool and given a Zeppelin
> >notebook (note.json) it generates a Spark scala application, compiles it
> >using the compiler embedded in the scala sdk and then package all these
> >resources into a jar that works with spark-submit command.
> >
> >I would like to start prototyping the integration into the Zeppelin UI
and
> >I was wondering if it would be ok to use the above jar as a dependency
> >(e.g. from a maven release) and integrate into zeppelin...
> >
> >Thoughts ?
> >
> >
> >On Mon, Sep 19, 2016 at 7:47 AM, Sourav Mazumder <
> >sourav.mazumde...@gmail.com> wrote:
> >
> >> To Moon's point, This is what my vision is around this feature -
> >>
> >> 1. Use should be able to package 1, more than one, all of the
> >>paragraphs in
> >> a Notebook to create a Jar file which can be used with Spark-Submit.
> >>
> >> 2. The tool should automatically remove the all the interactive
> >>statements
> >> like print, show etc.
> >>
> >> 3. The tool should automatically create a Main class in addition to the
> >>jar
> >> file(s) which will internally call the respective jar. User can then
> >>change
> >> this main class if needed for parameterization through Args.
> >>
> >> Regards,
> >> Sourav
> >>
> >> On Mon, Sep 19, 2016 at 7:33 AM, Sourav Mazumder <
> >> sourav.mazumde...@gmail.com> wrote:
> >>
> >> > I am also pretty much for this.
> >> >
> >> > I have got the similar request from each and every people/group who I
> >> > showcased Zeppelin.Regards,
> >> > Sourav
> >> >
> >> > On Fri, Sep 16, 2016 at 8:06 PM, moon soo Lee 
> wrote:
> >> >
> >> >> Hi Luciano,
> >> >>
> >> >> I've also got a lot of questions about "Productize the notebook"
> >>every
> >> >> time
> >> >> i meet users use Zeppelin in their work.
> >> >>
> >> >> I think it's actually about two different problems that Zeppelin
> >>need to
> >> >> address.
> >> >>
> >> >> *1) Provide way that interactive notebook becomes part of production
> >> data
> >> >> pipeline.*
> >> >>
> >> >> Although Zeppelin does have quite convenient cron-like scheduler for
> >> each
> >> >> Note, built-in cron scheduler is not ready for serious use in the
> >> >> production. Because it lacks some features like actions after
> >> >> success/fail,
> >> >> fault-tolerance, history, and so on. I think community is working on
> >> >> improving it, and it's going to take some time.
> >> >>  Meanwhile, any external enterprise level job scheduler can run Note
> >>or
> >> >> Paragraph via REST api. But we 

[GitHub] zeppelin issue #1847: [MINOR] Restore paragraph control button margin style

[AhyoungRyu]

Github user AhyoungRyu commented on the issue:

https://github.com/apache/zeppelin/pull/1847
  
Didn't realized that. Good catch. Looks good 👍 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---