GitHub user prabhjyotsingh opened a pull request:
https://github.com/apache/incubator-zeppelin/pull/827
ZEPPELIN-773 : Livy interperter
### What is this PR for?
As Zeppelin evolves its notebook, for large scale data analysis, multiple
zeppelin users are expected to use and connect to the same set of data
repositories within an enterprise. Since Zeppelin notebooks could affect data,
state and its lineage, it is important to have separation of users, provide
them with appropriate sandboxes, in addition to capturing the right audit
details. Further, the IT within the organization would prefer to support fewer
Zeppelin instances (preferably one) to support its customers. Therefore, the
objectives of creating a multi-tenant zeppelin are:
â Supporting workloads of multiple customers
â Supporting multiple LOBs (lines of business), on a single data systems
â Support fine grained audits
As a natural evolution of Zeppelin Authentication and Authorization design,
partly user awareness in downstream data systems such as Spark/Hive and others,
is essential to achieve the above stated objectives.
### What type of PR is it?
Feature
### Todos
* [ ] - Test case
* [ ] - Review Comments
* [ ] - Documentation
### What is the Jira issue?
ZEPPELIN-773
### How should this be tested?
- Install Livy by following steps on https://github.com/cloudera/livy
- Start the Livy server
- Now by using Zeppelin-Livy interpreter, run any of the spark, pyspark or
R commands.
### Screenshots (if appropriate)
<img width="1436" alt="screen shot 2016-04-11 at 12 41 35 pm"
src="https://cloud.githubusercontent.com/assets/674497/14419479/b514979c-ffe3-11e5-9dea-df9854d8409c.png";>
<img width="1434" alt="screen shot 2016-04-11 at 12 41 59 pm"
src="https://cloud.githubusercontent.com/assets/674497/14419478/b514922e-ffe3-11e5-9c98-93c5b99de106.png";>
<img width="1440" alt="screen shot 2016-04-11 at 12 48 13 pm"
src="https://cloud.githubusercontent.com/assets/674497/14419480/b515d8c8-ffe3-11e5-8c20-4c988c621f51.png";>
### Questions:
* Does the licenses files need update? n/a
* Is there breaking changes for older versions? n/a
* Does this needs documentation? yes
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/prabhjyotsingh/incubator-zeppelin
livyInterperter
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-zeppelin/pull/827.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #827
----
commit fadc6d92ffabc0dc4cb2079702ad292df3fd070a
Author: Prabhjyot Singh <prabhjyotsi...@gmail.com>
Date: 2016-02-08T16:29:13Z
userName to be present in InterpreterContext/RemoteInterpreterContext
commit d928203cf6d69b3c6427aa35f047b25a399b3198
Author: Prabhjyot Singh <prabhjyotsi...@gmail.com>
Date: 2016-02-08T16:48:37Z
revert shiri.ini
if fromMessage.principal.equals("anonymous") then set user as null
commit 320790cf4581ba14dfa6791220818058ba3bc7ba
Author: Prabhjyot Singh <prabhjyotsi...@gmail.com>
Date: 2016-02-09T09:00:40Z
fix for CI, missing change signature
commit 57ca5771f968ed93fcc566b24cfa699aaf7fb301
Author: Prabhjyot Singh <prabhjyotsi...@gmail.com>
Date: 2016-02-16T16:23:32Z
review change create such class AuthenticationInfo, and pass it into
InterpreterContext
commit ba91da4121668f5082aa6e0d73a69ecd2b35fa32
Author: Prabhjyot Singh <prabhjyotsi...@gmail.com>
Date: 2016-02-16T16:23:53Z
Merge remote-tracking branch 'origin/master' into UserInInterpreterContext
commit 34dcc32035128cc7ecd0b4c2ed8ad7342b034ecd
Author: Prabhjyot Singh <prabhjyotsi...@gmail.com>
Date: 2016-02-16T16:44:06Z
instead of null pass "new AuthenticationInfo()"
commit a5a991d2df197d6242d96e2451b4ea0112a7c56e
Author: Prabhjyot Singh <prabhjyotsi...@gmail.com>
Date: 2016-02-16T18:38:10Z
check for selenium
commit 95e7c13086de56df000a6cd014fd02d475bd977d
Author: Prabhjyot Singh <prabhjyotsi...@gmail.com>
Date: 2016-02-16T18:38:55Z
test for selenium
commit 0709b9c51c6bc7aaa4dafa5c6d52a0eb4530fefe
Author: Prabhjyot Singh <prabhjyotsi...@gmail.com>
Date: 2016-02-16T18:58:37Z
moving AuthenticationInfo to org.apache.zeppelin.display.AuthenticationInfo
commit 40534974ae1ce813214d6e13dc5d5558a4815f07
Author: Prabhjyot Singh <prabhjyotsi...@gmail.com>
Date: 2016-02-17T19:45:52Z
test livy
commit ace28a8000a8f8a65c66344bccec1b868d9a85be
Author: Prabhjyot Singh <prabhjyotsi...@gmail.com>
Date: 2016-02-17T21:35:26Z
working livy
commit d0519d5a02ff5979a37dec9b8c9976db57db400b
Author: Prabhjyot Singh <prabhjyotsi...@gmail.com>
Date: 2016-02-19T13:05:43Z
Working livy interpreter with spark, pyspark, R
commit 10311d3bafa2234852e84ca5f75424646c909a4e
Author: Prabhjyot Singh <prabhjyotsi...@gmail.com>
Date: 2016-02-20T06:43:00Z
This works in all cases
commit 07f0846d789213200ddedd54352c35d3e446902a
Author: Prabhjyot Singh <prabhjyotsi...@gmail.com>
Date: 2016-02-20T18:51:35Z
with lspark sql
commit 8f4ec479d8fbbbc938e9bbfa8eb3bf6be6ed9fe5
Author: Prabhjyot Singh <prabhjyotsi...@gmail.com>
Date: 2016-02-20T18:54:08Z
removing unrequired logs
commit 9cb081993dd964d87f91e15864c3a5fcd998a6c0
Author: Prabhjyot Singh <prabhjyotsi...@gmail.com>
Date: 2016-02-21T06:29:14Z
Fix for 1st request failing
commit de2fd3cd60811d891234c88030c1bf80378ee553
Author: Prabhjyot Singh <prabhjyotsi...@gmail.com>
Date: 2016-02-22T14:43:35Z
have spark streaming
commit b53fd8bfb7da935fb511444f4c70e63b6490e616
Author: Rohit Choudhary <rconl...@gmail.com>
Date: 2016-02-23T03:19:13Z
Don't need so many interpreters
commit 1f9a111188bcfc67c0d02826963d92d49ba8ef1d
Author: Rohit Choudhary <rconl...@gmail.com>
Date: 2016-02-23T04:29:33Z
remove references to LivyInterpreters
commit ee2dcebe3551b09f5986cb6a3414680d4929762e
Author: Prabhjyot Singh <prabhjyotsi...@gmail.com>
Date: 2016-02-23T06:27:44Z
working spark sql
commit 68f438de8b4d76d1019f261dfb9a47b8d9c7b45b
Author: Prabhjyot Singh <prabhjyotsi...@gmail.com>
Date: 2016-02-23T06:46:13Z
Merge remote-tracking branch 'origin/master' into livyInterperter
commit 426bbe80c318621afd7157278fc4b07448cf8e6c
Author: Rohit Choudhary <rconl...@gmail.com>
Date: 2016-02-23T17:54:32Z
Merge branch 'livyInterperter' of
https://github.com/prabhjyotsingh/incubator-zeppelin into livyInterperter
commit 4f513a5b5aa797e9dc8a7fd72dbed17c70a182c7
Author: Prabhjyot Singh <prabhjyotsi...@gmail.com>
Date: 2016-02-24T04:03:14Z
site.xml
commit 948615a3ebb8309cc4c44208fbffd213d5e4a6b9
Author: Prabhjyot Singh <prabhjyotsi...@gmail.com>
Date: 2016-02-24T05:28:36Z
Merge remote-tracking branch 'origin/master' into livyInterperter
# Conflicts:
# testing/startSparkCluster.sh
commit 9d89b0d235855636d93a37bcd74f4951ced39d3c
Author: Prabhjyot Singh <prabhjyotsi...@gmail.com>
Date: 2016-02-26T05:47:03Z
Merge remote-tracking branch 'origin/master' into livyInterperter
commit b8e17794d957a51a63eebf16a5338b3db1c6651e
Author: Prabhjyot Singh <prabhjyotsi...@gmail.com>
Date: 2016-02-26T05:59:33Z
adding back LivySparkRInterpreter.java , pyspark
commit dadc2571956abd40af48cb1ad3ee5d82382843c5
Author: Prabhjyot Singh <prabhjyotsi...@gmail.com>
Date: 2016-03-02T18:55:21Z
reverting zeppelinConfiguration
commit 0fbb74bcf6fc4532f15656d6e679d1166d55fa5a
Author: Prabhjyot Singh <prabhjyotsi...@gmail.com>
Date: 2016-03-07T11:22:52Z
Merge remote-tracking branch 'origin/master' into livyInterperter
# Conflicts:
# conf/zeppelin-site.xml.template
commit ea05fe9a1ab8f8a5fcd49354d916edf9bc52a84a
Author: Prabhjyot Singh <prabhjyotsi...@gmail.com>
Date: 2016-03-08T06:00:58Z
Merge remote-tracking branch 'origin/master' into livyInterperter
commit ee1c9f4cfb79c2d34d9c0bc34e81d48fac1d8908
Author: Prabhjyot Singh <prabhjyotsi...@gmail.com>
Date: 2016-03-09T05:12:14Z
Merge remote-tracking branch 'origin/master' into livyInterperter
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
![https://cloud.githubusercontent.com/assets/674497/14419479/b514979c-ffe3-11e5-9dea-df9854d8409c.png"](https://cloud.githubusercontent.com/assets/674497/14419479/b514979c-ffe3-11e5-9dea-df9854d8409c.png"){kind=link}
![https://cloud.githubusercontent.com/assets/674497/14419478/b514922e-ffe3-11e5-9c98-93c5b99de106.png"](https://cloud.githubusercontent.com/assets/674497/14419478/b514922e-ffe3-11e5-9c98-93c5b99de106.png"){kind=link}
![https://cloud.githubusercontent.com/assets/674497/14419480/b515d8c8-ffe3-11e5-8c20-4c988c621f51.png"](https://cloud.githubusercontent.com/assets/674497/14419480/b515d8c8-ffe3-11e5-8c20-4c988c621f51.png"){kind=link}