[GitHub] zeppelin pull request #1743: add hadoop-2.7 to available hadoop version

[hayashidac]

GitHub user hayashidac opened a pull request:

https://github.com/apache/zeppelin/pull/1743

add hadoop-2.7 to available hadoop version

### What is this PR for?
we can use hadoop 2.7 for spark-dependency, but building document doesn't 
contain that.
hadoop 2.7 should be contained to available hadoop version at document.


### What type of PR is it?
[Documentation]

### Todos


### What is the Jira issue?
ZEPPELIN-1783


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/hayashidac/zeppelin ZEPPELIN-1783

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/zeppelin/pull/1743.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1743


commit c84509e4c363853e8e6ce32c452703c5091fc940
Author: chie8842 
Date:   2016-12-10T07:46:54Z

add hadoop-2.7 to available hadoop version




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (ZEPPELIN-1783) Contain hadoop 2.7 for available hadoop version at building document

[chie hayashida (JIRA)]

chie hayashida created ZEPPELIN-1783:


 Summary: Contain hadoop 2.7 for available hadoop version at 
building document
 Key: ZEPPELIN-1783
 URL: https://issues.apache.org/jira/browse/ZEPPELIN-1783
 Project: Zeppelin
  Issue Type: Improvement
  Components: documentation
Reporter: chie hayashida
Priority: Minor


we can build with hadoop 2.7 at latest zeppelin, but document doesn't contain.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Adding Angular API to non-Spark interpreters

[Goodman, Alexander (398K)]

Hi Randy,

The main thing you are probably looking for is the AngularObjectRegistry
class (https://github.com/apache/zeppelin/blob/master/zeppelin-int
erpreter/src/main/java/org/apache/zeppelin/display/AngularOb
jectRegistry.java). One of the fields of InterpreterContext is an instance
of this class, so you could in theory just place the interpreter result
into the registry here:

https://github.com/apache/zeppelin/blob/master/zeppelin-
interpreter/src/main/java/org/apache/zeppelin/interpreter/remote/
RemoteInterpreterServer.java#L474

While I agree that we should try to better expose the angular API to more
interpreters, there are a few problems with your approach.

First, you may noticed that a recent change was made to interpreter results
(https://github.com/apache/zeppelin/pull/1658). This ultimately makes the
feature you are asking for considerably more complicated to implement
correctly since you have just one paragraphID but potentially multiple
results with different types, and you'll only want a certain part of the
result to get bound to the registry.

Also, I am not sure if enforcing this automatically would be a good idea
considering that only certain types of results are useful to store as
AngularObjects. If this gets applied to every paragraph automatically I
could see this potentially leading to some churn within the system, but I
can't say for sure. Either way, I think this should be something like a per
paragraph feature which can be toggled via a UI option.

I think this is a good discussion to have, since I do agree that leveraging
this feature to other interpreters would lead to great advances in
Zeppelin's visualization capabilities. Perhaps we should take some time to
discuss other ideas for making this happen?

Thanks,
Alex

On Fri, Dec 9, 2016 at 12:27 PM, Randy Gelhausen  wrote:

> I created https://issues.apache.org/jira/browse/ZEPPELIN-1782 with
> additional detail for this feature request.
>
> I looked into the InterpreterContext classes, but am not sure how to
> implement this. Any suggestions from others?
>
> Thanks in advance,
> -Randy
>
> On 2016-11-24 18:31 (-0500), Randy Gelhausen  wrote:
> > I often need to visualize data in other ways than supported by the out
> of>
> > the box charts. I can do this with Angular notes and custom JavaScript,
> but>
> > Spark is the only interpreter with capability to set Angular variables.>
> >
> > Could it be possible to automatically expose all interpreter results as>
> > Angular variables using the paragraph ID as the variable name?>
> >
> > This would greatly improve the custom viz capabilities for Zeppelin
> without>
> > requiring developers of each interpreter to integrate with Zeppelin's>
> > Angular API.>
> >
>



-- 
Alex Goodman
Data Scientist I
Science Data Modeling and Computing (398K)
Jet Propulsion Laboratory
California Institute of Technology
Tel: +1-818-354-6012

[GitHub] zeppelin issue #1612: ZEPPELIN-1607. Refactor Livy Interpreter to adapt scop...

[Leemoonsoo]

Github user Leemoonsoo commented on the issue:

https://github.com/apache/zeppelin/pull/1612
  
LGTM and merge to master if there're no more comment.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] zeppelin issue #1739: Branch 0.5.6 al

[Leemoonsoo]

Github user Leemoonsoo commented on the issue:

https://github.com/apache/zeppelin/pull/1739
  
@andreistankevich could you close this PR if you accidentally created?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] zeppelin pull request #1728: [ZEPPELIN-1754] PING request stacking on websoc...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/zeppelin/pull/1728


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: Adding Angular API to non-Spark interpreters

[Randy Gelhausen]

I created https://issues.apache.org/jira/browse/ZEPPELIN-1782 with
additional detail for this feature request.

I looked into the InterpreterContext classes, but am not sure how to
implement this. Any suggestions from others?

Thanks in advance,
-Randy

On 2016-11-24 18:31 (-0500), Randy Gelhausen  wrote:
> I often need to visualize data in other ways than supported by the out
of>
> the box charts. I can do this with Angular notes and custom JavaScript,
but>
> Spark is the only interpreter with capability to set Angular variables.>
>
> Could it be possible to automatically expose all interpreter results as>
> Angular variables using the paragraph ID as the variable name?>
>
> This would greatly improve the custom viz capabilities for Zeppelin
without>
> requiring developers of each interpreter to integrate with Zeppelin's>
> Angular API.>
>

[jira] [Created] (ZEPPELIN-1782) Expose all paragraph output as Angular variables

[Randy Gelhausen (JIRA)]

Randy Gelhausen created ZEPPELIN-1782:
-

 Summary: Expose all paragraph output as Angular variables
 Key: ZEPPELIN-1782
 URL: https://issues.apache.org/jira/browse/ZEPPELIN-1782
 Project: Zeppelin
  Issue Type: Improvement
  Components: front-end, Interpreters
Reporter: Randy Gelhausen


I need to visualize data in different ways than the existing D3 graphs. I can 
do this with Angular paragraphs with custom JavaScript, but
Spark is the only interpreter with capability to set Angular variables.

It's possible to write code for each Interpreter to set Angular variables, but 
this is a large task.

A very nice new feature would be to expose each paragraph's text output to an 
Angular variable with the ${paragraph_id}-output or similar. This way JDBC, or 
Flink, or Hive, etc. results can easily be visualized with custom Javascript.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Notebook example

[moon soo Lee]

Althought it's not mendentory, creating issue helps track the changes such
as when code has chaged, which version of zeppelin includes the changes,
etc. So, please create if you don't mind.

Thanks,
moon
On Fri, 9 Dec 2016 at 11:05 AM Alexander Shoshin 
wrote:

> Thanks for reply.
>
> And do I need to create an issue first? Or I can just make a pull request
> for adding a note to tutorial examples?
>
> Thanks,
> Alexander
>
> > -Original Message-
> > From: moon soo Lee [mailto:m...@apache.org]
> > Sent: Thursday, December 08, 2016 11:30 PM
> > To: dev@zeppelin.apache.org
> > Subject: Re: Notebook example
> >
> > Hi,
> >
> > Appreciate for sharing a notebook and considering contribution.
> > You can contribute your note as a tutorial under
> https://github.com/apache/zeppelin/tree/master/notebook by making
> pullrequest. So Zeppelin distribution can include notebook. Just sharing on
> the mailing list greatly helps as well if you prefer.
> >
> > Don't hesitate making pullrequest or asking questions.
> >
> > Thanks,
> > moon
> >
> > On Thu, Dec 8, 2016 at 10:26 AM Alexander Shoshin <
> alexander_shos...@epam.com> wrote:
> >
> > > Hi everyone.
> > >
> > > I want to become a Zeppelin contributor. Before starting to fix issues
> > > I decided to make an example of a notebook to become understand
> > > Zeppelin better :). I chose Apache Flink as a main Interpreter for my
> > > example because there are not so many examples of "Zeppelin on Flink"
> > > usage. I think it also may be helpful for your community according to
> > > https://issues.apache.org/jira/browse/ZEPPELIN-684, so here is the
> link:
> > > https://github.com/AlexanderShoshin/zeppelin-example
> > >
> > > I am ready to make any kind of corrections in this example.
> > >
> > > Regards,
> > > Alexander
> > >
>

[GitHub] zeppelin issue #1733: [ZEPPELIN-1753] Fix blank notename issue

[marchpig]

Github user marchpig commented on the issue:

https://github.com/apache/zeppelin/pull/1733
  
@cuspymd Thank you for review!
Maybe ZEPPELIN-1763 should be fixed by another PR. :smile: 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] zeppelin issue #1730: [ZEPPELIN-1736] Introduce trash & enable removing fold...

[marchpig]

Github user marchpig commented on the issue:

https://github.com/apache/zeppelin/pull/1730
  
@tae-jun It works well! Thank you very much for accepting my opinions. 
:smiley: 
I think that handling `~Trash` prefix in the future is fine to me because 
it is not a bug but just my opinion :)

However, I found a minor problem.
When a folder name has a trailing space, it is not moved to trash. For 
example, "My folder /My note".
I think that you can handle this corner case in the future work!

Thanks again for considering my opinions. :+1: 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

RE: Notebook example

[Alexander Shoshin]

Thanks for reply.

And do I need to create an issue first? Or I can just make a pull request for 
adding a note to tutorial examples?

Thanks,
Alexander

> -Original Message-
> From: moon soo Lee [mailto:m...@apache.org] 
> Sent: Thursday, December 08, 2016 11:30 PM
> To: dev@zeppelin.apache.org
> Subject: Re: Notebook example
>
> Hi,
>
> Appreciate for sharing a notebook and considering contribution.
> You can contribute your note as a tutorial under 
> https://github.com/apache/zeppelin/tree/master/notebook by making 
> pullrequest. So Zeppelin distribution can include notebook. Just sharing on 
> the mailing list greatly helps as well if you prefer.
> 
> Don't hesitate making pullrequest or asking questions.
>
> Thanks,
> moon
>
> On Thu, Dec 8, 2016 at 10:26 AM Alexander Shoshin < 
> alexander_shos...@epam.com> wrote:
>
> > Hi everyone.
> >
> > I want to become a Zeppelin contributor. Before starting to fix issues 
> > I decided to make an example of a notebook to become understand 
> > Zeppelin better :). I chose Apache Flink as a main Interpreter for my 
> > example because there are not so many examples of "Zeppelin on Flink" 
> > usage. I think it also may be helpful for your community according to 
> > https://issues.apache.org/jira/browse/ZEPPELIN-684, so here is the link:
> > https://github.com/AlexanderShoshin/zeppelin-example
> >
> > I am ready to make any kind of corrections in this example.
> >
> > Regards,
> > Alexander
> >

[GitHub] zeppelin pull request #1742: [ZEPPELIN-1781] Failed to run the script 'test_...

[cuspymd]

GitHub user cuspymd opened a pull request:

https://github.com/apache/zeppelin/pull/1742

[ZEPPELIN-1781] Failed to run the script 'test_zeppelin_pr.py' on Windows 10

### What is this PR for?
Fix failure of script execution on Windows 10


### What type of PR is it?
[Bug Fix]

### Todos

### What is the Jira issue?
[ZEPPELIN-1781](https://issues.apache.org/jira/browse/ZEPPELIN-1781)

### How should this be tested?
Run 'test_zeppelin_pr.py' on Windows 10.

### Screenshots (if appropriate)

### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/cuspymd/zeppelin test-pr

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/zeppelin/pull/1742.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1742


commit eb4b5670ab3c2f31d188e5ee806290d666f26bb8
Author: Myoungdo Park 
Date:   2016-12-09T14:50:20Z

Fix failure of script execution on Windows 10




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (ZEPPELIN-1781) Failed to run the script 'test_zeppelin_pr.py' on Windows 10

[Myoungdo Park (JIRA)]

Myoungdo Park created ZEPPELIN-1781:
---

 Summary: Failed to run the script 'test_zeppelin_pr.py' on Windows 
10
 Key: ZEPPELIN-1781
 URL: https://issues.apache.org/jira/browse/ZEPPELIN-1781
 Project: Zeppelin
  Issue Type: Bug
  Components: build
Affects Versions: 0.7.0
 Environment: windows 10
Reporter: Myoungdo Park
Priority: Minor


The script 'test_zeppelin_pr.py' execution fails on Windows 10.
I think it did not consider Windows 10.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] zeppelin issue #1733: [ZEPPELIN-1753] Fix blank notename issue

[cuspymd]

Github user cuspymd commented on the issue:

https://github.com/apache/zeppelin/pull/1733
  
@marchpig I checked it works nicely ~.
But this PR dosen't resolve another note name issue 
[ZEPPELIN-1763](https://issues.apache.org/jira/browse/ZEPPELIN-1763).
I reproduce ZEPPELIN-1763 after applying this PR.
It still occurs When changing note names repeatedly. 
anyway, LGTM!!


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] zeppelin issue #1741: [HOTFIX][ZEPPELIN-1779]Flaky test: PySparkInterpreterM...

[zjffdu]

Github user zjffdu commented on the issue:

https://github.com/apache/zeppelin/pull/1741
  
Thanks @agoodm lgtm


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] zeppelin pull request #1741: [HOTFIX][ZEPPELIN-1779]Flaky test: PySparkInter...

[agoodm]

GitHub user agoodm opened a pull request:

https://github.com/apache/zeppelin/pull/1741

[HOTFIX][ZEPPELIN-1779]Flaky test: PySparkInterpreterMatplotlibTest

### What is this PR for?
The matplotlib tests have stalled as a result of conda downloading version 
2.0.0b1. While this was an old beta version and the issue has been fixed 
upstream (https://github.com/conda-forge/matplotlib-feedstock/issues/86), we 
still have not tested the inline backend for 2.0.0 which will be a release with 
a lot of major changes, hence this PR should be anyway. I am also partially 
undoing one of the changes made in #1709 which suppressed the output of the 
dependency installation from conda. This change was very puzzling to me since 
dependency info isn't that verbose. I ended up determining the cause of this 
issue because I reverted this change and was then able to see the dependency 
versions in the log. I could see suppressing this might being necessary for the 
R packages in case they aren't cached and a wall of compiler output get printed 
out, but I see no real justification to mute it for python packages, especially 
given that they will only be installed for two short profiles.

### What type of PR is it?
Hot Fix 

### What is the Jira issue?
[ZEPPELIN-1779](https://issues.apache.org/jira/browse/ZEPPELIN-1779)

### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/agoodm/zeppelin ZEPPELIN-1779

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/zeppelin/pull/1741.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1741


commit 1211a2f9e037ab0214aa73776afa3e32b85762a0
Author: Alex Goodman 
Date:   2016-12-09T10:29:31Z

Ensure matplotlib version is 1.5.3




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] zeppelin issue #1735: [ZEPPELIN-1767] Blank paragraph should not be run.

[AhyoungRyu]

Github user AhyoungRyu commented on the issue:

https://github.com/apache/zeppelin/pull/1735
  
@astroshim Good catch! Tested and it was working well as expected. 
AFAIK there are 3 way to run a paragraph in Zeppelin note like below 

1. \w keyboard shortcut `Shift + Enter` : we can't run the blank paragraph 
by default.
2. \w icon button: your PR disable this one :)
3. \w RestApi [like 
this](https://zeppelin.apache.org/docs/0.7.0-SNAPSHOT/rest-api/rest-notebook.html#run-a-paragraph-asynchronously)
 : I was able to run the blank paragraph 

I think we need to block 3rd case as well. What do you think? 



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (ZEPPELIN-1778) Potential security issue for passing user credential to interpreter process

[Jeff Zhang (JIRA)]

Jeff Zhang created ZEPPELIN-1778:


 Summary: Potential security issue for passing user credential to 
interpreter process
 Key: ZEPPELIN-1778
 URL: https://issues.apache.org/jira/browse/ZEPPELIN-1778
 Project: Zeppelin
  Issue Type: Improvement
Affects Versions: 0.6.2, 0.7.0
Reporter: Jeff Zhang
Priority: Critical


Currently zeppelin-server will pass user credential info to interpreter process 
through thrift. This would cause potential security issue as I think the thrift 
protocol we used for now is not secured. One solution is to enable SSL for 
thrift.

Besides, there're 2 other problems:
* credential info will be save in conf/credentials.json in plain text. 
* credential info be passed to all the interpreters no matter whether this 
interpreter need this. 

\cc [~moon] [~vinayshu...@gmail.com] [~prasadwagle]




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)