[GitHub] zeppelin issue #3238: ZEPPELIN-3870. Configuration in spark-defaults.conf do...

[weand]

Github user weand commented on the issue:

https://github.com/apache/zeppelin/pull/3238
  
@zjffdu 

i tried to build the latest branch-0.8 where this PR has been merged 
recently.

build fails with `RemoteInterpreterServer.java:[22,32] package 
org.apache.commons.lang3 does not exist`

```
[ERROR] Failed to execute goal 
org.apache.maven.plugins:maven-compiler-plugin:3.1:compile (default-compile) on 
project zeppelin-interpreter: Compilation failure
[ERROR] 
/home/zeppelin/zeppelin-distro/work/zeppelin-repo/zeppelin-interpreter/src/main/java/org/apache/zeppelin/interpreter/remote/RemoteInterpreterServer.java:[22,32]
 package org.apache.commons.lang3 does not exist
```


---

[GitHub] zeppelin pull request #3202: [ZEPPELIN-3528] [branch-0.8] Ordering by interp...

[weand]

Github user weand closed the pull request at:

https://github.com/apache/zeppelin/pull/3202


---

[GitHub] zeppelin pull request #3202: [ZEPPELIN-3528] [branch-0.8] Ordering by interp...

[weand]

GitHub user weand opened a pull request:

https://github.com/apache/zeppelin/pull/3202

[ZEPPELIN-3528] [branch-0.8] Ordering by interpreter name in same 
interpreter group

### What is this PR for?
Merge for branch-0.8.

In 'Create new note' diagram, 'Default Interpreter' is ordered by only 
interpreter groups.

But interpreter is not ordered in same interpreter group.
(It seems to be ordered by hashcode of 'InterpreterSetting')

When there are many interpreter in same group and this group is first,
user cannot predict what interpreter is showed first.

So I think that interpreter need to be ordered by name.

### What type of PR is it?
[Improvement]

### Todos
* [x] - Modify Code

### What is the Jira issue?
* https://issues.apache.org/jira/browse/ZEPPELIN-3528

### How should this be tested?
* Click 'create new note'

### Screenshots (if appropriate)

### Questions:
* Does the licenses files need update?
* No
* Is there breaking changes for older versions?
* No
* Does this needs documentation?
* No

Author: Jaehoon Jeong 

Closes #3005 from woowahan-jaehoon/ZEPPELIN-3528 and squashes the following 
commits:

5cb709ce3 [Jaehoon Jeong] [ZEPPELIN-3528] Ordering by interpreter name in 
same interpreter group

### What is this PR for?
A few sentences describing the overall goals of the pull request's commits.
First time? Check out the contributing guide - 
https://zeppelin.apache.org/contribution/contributions.html


### What type of PR is it?
[Bug Fix | Improvement | Feature | Documentation | Hot Fix | Refactoring]

### Todos
* [ ] - Task

### What is the Jira issue?
* Open an issue on Jira https://issues.apache.org/jira/browse/ZEPPELIN/
* Put link here, and add [ZEPPELIN-*Jira number*] in PR title, eg. 
[ZEPPELIN-533]

### How should this be tested?
* First time? Setup Travis CI as described on 
https://zeppelin.apache.org/contribution/contributions.html#continuous-integration
* Strongly recommended: add automated unit tests for any new or changed 
behavior
* Outline any manual steps to test the PR here.

### Screenshots (if appropriate)

### Questions:
* Does the licenses files need update?
* Is there breaking changes for older versions?
* Does this needs documentation?


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/weand/zeppelin branch-0.8

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/zeppelin/pull/3202.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3202


commit 00af514ea29dee5b1df692ea8fd88ea6aad436c2
Author: Jaehoon Jeong 
Date:   2018-06-05T11:39:55Z

[ZEPPELIN-3528] Ordering by interpreter name in same interpreter group

### What is this PR for?
In 'Create new note' diagram, 'Default Interpreter' is ordered by only 
interpreter groups.

But interpreter is not ordered in same interpreter group.
(It seems to be ordered by hashcode of 'InterpreterSetting')

When there are many interpreter in same group and this group is first,
user cannot predict what interpreter is showed first.

So I think that interpreter need to be ordered by name.

### What type of PR is it?
[Improvement]

### Todos
* [x] - Modify Code

### What is the Jira issue?
* https://issues.apache.org/jira/browse/ZEPPELIN-3528

### How should this be tested?
* Click 'create new note'

### Screenshots (if appropriate)

### Questions:
* Does the licenses files need update?
* No
* Is there breaking changes for older versions?
* No
* Does this needs documentation?
* No

Author: Jaehoon Jeong 

Closes #3005 from woowahan-jaehoon/ZEPPELIN-3528 and squashes the following 
commits:

5cb709ce3 [Jaehoon Jeong] [ZEPPELIN-3528] Ordering by interpreter name in 
same interpreter group




---

[GitHub] zeppelin issue #2951: ZEPPELIN-3435. Interpreter timeout lifecycle leads to ...

[weand]

Github user weand commented on the issue:

https://github.com/apache/zeppelin/pull/2951
  
:+1:  no more orphans were spotted since applying changes of this PR.


---

[GitHub] zeppelin issue #2951: ZEPPELIN-3435. Interpreter timeout lifecycle leads to ...

[weand]

Github user weand commented on the issue:

https://github.com/apache/zeppelin/pull/2951
  
... but we are now trying this in production.


---

[GitHub] zeppelin issue #2951: ZEPPELIN-3435. Interpreter timeout lifecycle leads to ...

[weand]

Github user weand commented on the issue:

https://github.com/apache/zeppelin/pull/2951
  
I can't confirm that. LifecycleManager tried to close the interpreter group 
at 13:06 (see log pasted in the Jira) - 60 minutes after the last job was 
finished. The same interpreter group was started again at 13:17 - from 
Interpreter Log:
```
 INFO [2018-04-27 12:06:40,066] ({dag-scheduler-event-loop} 
Logging.scala[logInfo]:54) - ResultStage 113 (toPandas at 
:23) finished in 0.038 s
 INFO [2018-04-27 12:06:40,066] ({Thread-19} Logging.scala[logInfo]:54) - 
Job 28 finished: toPandas at :23, took 0.082673 s
 INFO [2018-04-27 12:06:40,101] ({pool-2-thread-10} 
SchedulerFactory.java[jobFinished]:115) - Job 20180215-131521_501880463 
finished by scheduler interpreter_101669487
 INFO [2018-04-27 12:09:40,123] ({spark-dynamic-executor-allocation} 
Logging.scala[logInfo]:54) - Request to remove executorIds: 1, 0
 INFO [2018-04-27 12:09:40,125] ({spark-dynamic-executor-allocation} 
Logging.scala[logInfo]:54) - Requesting to kill executor(s) 1, 0
 INFO [2018-04-27 12:09:40,128] ({spark-dynamic-executor-allocation} 
Logging.scala[logInfo]:54) - Actual list of executor(s) to be killed is 1, 0
 INFO [2018-04-27 12:09:40,135] ({spark-dynamic-executor-allocation} 
Logging.scala[logInfo]:54) - Removing executor 1 because it has been idle for 
180 seconds (new desired total will be 1)
 INFO [2018-04-27 12:09:40,135] ({spark-dynamic-executor-allocation} 
Logging.scala[logInfo]:54) - Removing executor 0 because it has been idle for 
180 seconds (new desired total will be 0)
 INFO [2018-04-27 12:09:40,547] ({dispatcher-event-loop-4} 
Logging.scala[logInfo]:54) - Executor 0 on 172.16.16.63 killed by driver.
 INFO [2018-04-27 12:09:40,551] ({dispatcher-event-loop-4} 
Logging.scala[logInfo]:54) - Executor 1 on 172.16.16.63 killed by driver.
 INFO [2018-04-27 12:09:40,552] ({dag-scheduler-event-loop} 
Logging.scala[logInfo]:54) - Executor lost: 0 (epoch 56)
 INFO [2018-04-27 12:09:40,553] ({dispatcher-event-loop-2} 
Logging.scala[logInfo]:54) - Trying to remove executor 0 from 
BlockManagerMaster.
 INFO [2018-04-27 12:09:40,555] ({dispatcher-event-loop-2} 
Logging.scala[logInfo]:54) - Removing block manager BlockManagerId(0, 
172.16.16.63, 33086, None)
 INFO [2018-04-27 12:09:40,556] ({SparkListenerBus} 
Logging.scala[logInfo]:54) - Existing executor 0 has been removed (new total is 
1)
 INFO [2018-04-27 12:09:40,556] ({dag-scheduler-event-loop} 
Logging.scala[logInfo]:54) - Removed 0 successfully in removeExecutor
 INFO [2018-04-27 12:09:40,557] ({dag-scheduler-event-loop} 
Logging.scala[logInfo]:54) - Executor lost: 1 (epoch 56)
 INFO [2018-04-27 12:09:40,557] ({dispatcher-event-loop-6} 
Logging.scala[logInfo]:54) - Trying to remove executor 1 from 
BlockManagerMaster.
 INFO [2018-04-27 12:09:40,557] ({dispatcher-event-loop-6} 
Logging.scala[logInfo]:54) - Removing block manager BlockManagerId(1, 
172.16.16.63, 34862, None)
 INFO [2018-04-27 12:09:40,557] ({SparkListenerBus} 
Logging.scala[logInfo]:54) - Existing executor 1 has been removed (new total is 
0)
 INFO [2018-04-27 12:09:40,557] ({dag-scheduler-event-loop} 
Logging.scala[logInfo]:54) - Removed 1 successfully in removeExecutor
 INFO [2018-04-27 12:15:25,523] ({Spark Context Cleaner} 
Logging.scala[logInfo]:54) - Cleaned accumulator 1428
 INFO [2018-04-27 12:15:25,531] ({Spark Context Cleaner} 
Logging.scala[logInfo]:54) - Cleaned accumulator 1422
 INFO [2018-04-27 12:15:25,531] ({Spark Context Cleaner} 
Logging.scala[logInfo]:54) - Cleaned shuffle 30
 INFO [2018-04-27 12:15:25,531] ({Spark Context Cleaner} 
Logging.scala[logInfo]:54) - Cleaned accumulator 1433
 INFO [2018-04-27 12:15:25,531] ({Spark Context Cleaner} 
Logging.scala[logInfo]:54) - Cleaned accumulator 1431
 WARN [2018-04-27 13:17:26,528] ({main} NativeCodeLoader.java[]:62) 
- Unable to load native-hadoop library for your platform... using builtin-java 
classes where applicable
 INFO [2018-04-27 13:17:26,619] ({main} 
RemoteInterpreterServer.java[main]:260) - 
URL:jar:file:/opt/zeppelin/zeppelin-0.8.1.20180426202048/lib/interpreter/zeppelin-interpreter-0.8.1-SNAPSHOT.jar!/org/apache/zeppelin/interpreter/remote/RemoteInterpreterServer.class
 INFO [2018-04-27 13:17:26,654] ({main} 
RemoteInterpreterServer.java[]:161) - Launching ThriftServer at 
172.16.16.65:46586
 INFO [2018-04-27 13:17:26,657] ({main} 
RemoteInterpreterServer.java[]:165) - Starting remote interpreter server 
on port 46586
 INFO [2018-04-27 13:17:26,659] ({Thread-2} 
RemoteInterpreterServer.java[run]:202) - Starting remote interpreter server on 
port 46586
 INFO [2018-04-27 13:17:27,665] ({Thread-3} 
RemoteInterpreterUtils.java[registerInterpreter]:165) - callbackHost: 
172.16.16.65, callbackPort: 39299, callbackInfo: 
CallbackInfo(host:172.16.16.65, port:46586)
 INFO [2018-04-27 13:17:27,772] ({pool-1

[GitHub] zeppelin issue #2925: ZEPPELIN-3404. Fail to run cronjob when user doesn't r...

[weand]

Github user weand commented on the issue:

https://github.com/apache/zeppelin/pull/2925
  
great. LGTM :+1: please merge to master and branch-0.8


---

[GitHub] zeppelin issue #2942: [HOTFIX] ZEPPELIN-3405 fix permission checks with role...

[weand]

Github user weand commented on the issue:

https://github.com/apache/zeppelin/pull/2942
  
works


---

[GitHub] zeppelin issue #2937: ZEPPELIN-3401. Deadlock while restarting interpreter

[weand]

Github user weand commented on the issue:

https://github.com/apache/zeppelin/pull/2937
  
so far no more issues. LGTM :+1: 


---

[GitHub] zeppelin issue #2935: ZEPPELIN-3411 Long running logic inside synchronized b...

[weand]

Github user weand commented on the issue:

https://github.com/apache/zeppelin/pull/2935
  
are you going to merge that into branch-0.8 as well?


---

[GitHub] zeppelin issue #2937: ZEPPELIN-3401. Deadlock while restarting interpreter

[weand]

Github user weand commented on the issue:

https://github.com/apache/zeppelin/pull/2937
  
Today we had that bug again (seems to occur daily) without the patch added 
to PROD.
Now just added the patch to PROD. Will wait another 48h from now before I 
tell you that it's working.


---

[GitHub] zeppelin issue #2925: ZEPPELIN-3404. Fail to run cronjob when user doesn't r...

[weand]

Github user weand commented on the issue:

https://github.com/apache/zeppelin/pull/2925
  
@zjffdu uhm, I can't confirm it. Today after checking out this PR from 
scratch my first point now works correctly. sorry for the spam.

created issue 
[ZEPPELIN-3427](https://issues.apache.org/jira/browse/ZEPPELIN-3427) for the 
second point. is that really so complex to wait for 0.8.1 or 0.9.0 ?

Part of that change would be:
1) moving/adding logic of setting note.config.cronExecutionUser + 
cronExecutionRoles to 
org.apache.zeppelin.socket.NotebookServer.isCronUpdated(Map<String, Object>, 
Map<String, Object>) when _cronUpdated = true_.
2) Then evaluate those config properties in 
org.apache.zeppelin.notebook.Note.runAll() when authenticationInfo is created




---

[GitHub] zeppelin issue #2925: ZEPPELIN-3404. Fail to run cronjob when user doesn't r...

[weand]

Github user weand commented on the issue:

https://github.com/apache/zeppelin/pull/2925
  
The notebook now runs successfully after restart with the user who enabled 
the scheduler.

2 remaining issues:

1) user1 schedules the cron. then the note is executed via cron with 
'user1' only as long as nobody else runs the note. as soon as 'user2' runs the 
note manually, the cron will also use 'user2' from that time on. Endusers would 
expect the cron will always run as 'user1'.

2) One more valid but yet uncovered use case left in my mind (dealing with 
permissions):
- interpreter permissions have been set only for groups (e.g. DepartmentA 
should be allowed to run spark interpreter)
- user created note with %spark interpreter and schedules a cron
- now when cron runs, **only username is added to authentication info. 
groups of the user (e.g. DepartmentA in that example) are omitted** and the 
notebook fails with an error, that **user 'xyz' has no permission for running 
the interpreter**.

any chance to fix that case as well? 
I think of something like introducing cronExecutingRoles (in addition to  
cronExecutingUser) and adding all roles of the user who enables the cron to 
that config property. And then when a cron runs, all that roles are added to 
the corresponding authentication info.


---

[GitHub] zeppelin issue #2925: ZEPPELIN-3404. Fail to run cronjob when user doesn't r...

[weand]

Github user weand commented on the issue:

https://github.com/apache/zeppelin/pull/2925
  
The notebook now runs successfully after restart with the user who enabled 
the scheduler.

There is one more valid use case left in my mind (dealing with permissions):
- interpreter permissions have been set only for groups (e.g. DepartmentA 
should be allowed to run spark interpreter)
- user created note with %spark interpreter and schedules a cron
- now when cron runs, only username is added to authentication info. groups 
of the user (e.g. DepartmentA in that example) are omitted and the notebook 
fails with an error, that user 'xyz' has no permission for running spark 
interpreter.

any chance to fix that case as well? 
I think of something like introducing cronExecutingRoles (in addition to  
cronExecutingUser) and adding all roles of the user who enables the cron to 
that config property. And then when a cron runs, all that roles are added to 
the corresponding authentication info.


---

[GitHub] zeppelin issue #2925: ZEPPELIN-3404. Fail to run cronjob when user doesn't r...

[weand]

Github user weand commented on the issue:

https://github.com/apache/zeppelin/pull/2925
  
@mebelousov I fully agree that running document as the group is not what a 
user would expect.

That's why I'm proposing another approach again:
1) prior to #2914 the cronExecutionUser config property was added to 
org.apache.zeppelin.notebook.Note.config when activating the cron and entering 
a username
2) that property is not set anymore. this PR here only sets the 
authentication info at running time to something (a principal) from the 
permission tab (which can be user or groups, which can not be determined 
properly)
3) so why not again set that cronExecutionUser config property from 1) 
(under the hood): with the user who activates the cron schedule expression. 
when only the owner can enable cron expressions, it will always be the username 
regardless of what principals exactly are configured in the permissions tab.


---

[GitHub] zeppelin issue #2925: ZEPPELIN-3404. Fail to run cronjob when user doesn't r...

[weand]

Github user weand commented on the issue:

https://github.com/apache/zeppelin/pull/2925
  
and optionally (or alternatively) introduce a config parameter for a static 
cronExecutionUser (kind of service user) to be defined by ops team. when this 
config parameter is set, cron schedules will always use that user.


---

[GitHub] zeppelin issue #2925: ZEPPELIN-3404. Fail to run cronjob when user doesn't r...

[weand]

Github user weand commented on the issue:

https://github.com/apache/zeppelin/pull/2925
  
@zjffdu is there a chance to set the cronExecutionUser implictly to the 
user who activates the cron schedule. 
and in addtion only allow owners (which can be defined by username or 
groupnames) to activate the cron schedule.


---

[GitHub] zeppelin issue #2925: ZEPPELIN-3404. Fail to run cronjob when user doesn't r...

[weand]

Github user weand commented on the issue:

https://github.com/apache/zeppelin/pull/2925
  
so running via cron will use the last user that executed a paragraph. that 
does not fit exactly to the docu change from original PR:
> Cron executing user (It is removed from 0.8 where it **enforces the cron 
execution user to be the note owner** for security purpose)

[https://github.com/apache/zeppelin/pull/2914/files#diff-bed1be4fb1f7f6c8e3bd6db18d72dc44](https://github.com/apache/zeppelin/pull/2914/files#diff-bed1be4fb1f7f6c8e3bd6db18d72dc44)


---

[GitHub] zeppelin issue #2914: [ZEPPELIN-3350] Don't allow set cronExecutionUser

[weand]

Github user weand commented on the issue:

https://github.com/apache/zeppelin/pull/2914
  
not directly when logged out. instead the NPE happens when Zeppelin gets 
restartet and next cron runs before the note was run manually, which is a 
frequent use case IMHO.
> ERROR [2018-04-12 21:11:00,024] ({DefaultQuartzScheduler_Worker-1} 
QuartzScheduler.java[schedulerError]:2425) - Job (note.2DBFZ9GGT threw an 
exception.
org.quartz.SchedulerException: Job threw an unhandled exception. [See 
nested exception: java.lang.NullPointerException: while trying to invoke the 
method org.apache.zeppelin.user.AuthenticationInfo.getUser() of a null object 
loaded from local variable 'authenticationInfo']
at org.quartz.core.JobRunShell.run(JobRunShell.java:213)
at 
org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:573)
Caused by: java.lang.NullPointerException: while trying to invoke the 
method org.apache.zeppelin.user.AuthenticationInfo.getUser() of a null object 
loaded from local variable 'authenticationInfo'
at 
org.apache.zeppelin.notebook.Paragraph.setAuthenticationInfo(Paragraph.java:209)
at org.apache.zeppelin.notebook.Paragraph.execute(Paragraph.java:353)
at org.apache.zeppelin.notebook.Note.run(Note.java:678)
at org.apache.zeppelin.notebook.Note.runAll(Note.java:659)
at org.apache.zeppelin.notebook.Note.runAll(Note.java:648)
at 
org.apache.zeppelin.notebook.Notebook$CronJob.execute(Notebook.java:936)
at org.quartz.core.JobRunShell.run(JobRunShell.java:202)
... 1 more


---

[GitHub] zeppelin issue #2914: [ZEPPELIN-3350] Don't allow set cronExecutionUser

[weand]

Github user weand commented on the issue:

https://github.com/apache/zeppelin/pull/2914
  
@zjffdu are you sure this works when logged out? i followed the call stack 
and I'm wondering if there would happen NPE if no AuthenticationInfo is present:
org.apache.zeppelin.notebook.Note.runAll(AuthenticationInfo, boolean)
> org.apache.zeppelin.notebook.Paragraph.execute(boolean) : Line 353
> 
org.apache.zeppelin.notebook.Paragraph.setAuthenticationInfo(AuthenticationInfo)
 : Line 209 << NPE


---

[GitHub] zeppelin issue #2749: [ZEPPELIN-3180] BUGFIX - save operation on interpreter...

[weand]

Github user weand commented on the issue:

https://github.com/apache/zeppelin/pull/2749
  
*no more comment*, we really would like to see this feature in master and 
branch-0.8 branches


---

[GitHub] zeppelin pull request #2842: [ZEPPELIN-3300] branch-0.8 broken due to 0.9.0 ...

[weand]

Github user weand closed the pull request at:

https://github.com/apache/zeppelin/pull/2842


---

[GitHub] zeppelin issue #2835: [ZEPPELIN-3289] Table not using full height after para...

[weand]

Github user weand commented on the issue:

https://github.com/apache/zeppelin/pull/2835
  
Opened another table issue with table option 'useFilter' - 
[ZEPPELIN-3301](https://issues.apache.org/jira/browse/ZEPPELIN-3301). 

Probably related to this issue here.


---

[GitHub] zeppelin pull request #2842: [ZEPPELIN-3300] branch-0.8 broken due to 0.9.0 ...

[weand]

GitHub user weand opened a pull request:

https://github.com/apache/zeppelin/pull/2842

[ZEPPELIN-3300] branch-0.8 broken due to 0.9.0 parent dependency in sap 
interpreter

### What is this PR for?
Hotfix PR for branch-0.8 build


### What type of PR is it?
[Hot Fix]

### Todos
* [ ] - Task

### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-3300

### How should this be tested?
CI pass

### Screenshots (if appropriate)

### Questions:
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? No


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/weand/zeppelin ZEPPELIN-3300

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/zeppelin/pull/2842.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2842


commit 73e99fc3e802e09b77b9905cdd93251714207e4d
Author: Andreas Weise <a.weise@...>
Date:   2018-03-06T21:09:44Z

fix 0.9.0-SNAPSHOT dependency in sap/pom.xml




---

[GitHub] zeppelin issue #2835: [ZEPPELIN-3289] Table not using full height after para...

[weand]

Github user weand commented on the issue:

https://github.com/apache/zeppelin/pull/2835
  
LGTM. no errors in console and ZEPPELIN-3292 data duplication fixed as well.


---

[GitHub] zeppelin issue #2808: [ZEPPELIN-3243] fix IndexOutOfBoundsException when 'No...

[weand]

Github user weand commented on the issue:

https://github.com/apache/zeppelin/pull/2808
  
We did not face any other IndexOutOfBoundException so far. So I would 
propose to merge that first.


---

[GitHub] zeppelin issue #2835: [ZEPPELIN-3289] Table not using full height after para...

[weand]

Github user weand commented on the issue:

https://github.com/apache/zeppelin/pull/2835
  
height issue is solved. thanks.

but now got the bahvior, that results are duplicated:
[https://ibin.co/3trkDDqHas5m.gif](https://ibin.co/3trkDDqHas5m.gif)

and the following is logged into console:
`result.controller.js?55cc:685` 
```
Graph drawing error TypeError: Cannot read property 'core' of undefined
at TableVisualization.refresh (eval at  
(app.bundle.js:1968), :172:26)
at TableVisualization.activate (eval at  
(app.bundle.js:1962), :108:14)
at afterLoaded (eval at  (app.bundle.js:2172), 
:699:31)
at retry (eval at  (app.bundle.js:2172), :235:7)
at angular.js:19374
at completeOutstandingRequest (angular.js:5955)
at angular.js:6234
```


---

[GitHub] zeppelin issue #2809: [ZEPPELIN-3249] Add support for streaming table

[weand]

Github user weand commented on the issue:

https://github.com/apache/zeppelin/pull/2809
  
seems like it has changed table behavior when height of paragraph has been 
changed.
https://issues.apache.org/jira/browse/ZEPPELIN-3289
@prabhjyotsingh can you fix that bug?


---

[GitHub] zeppelin issue #2824: ZEPPELIN-3278. Avoid duplicated interpreter setting

[weand]

Github user weand commented on the issue:

https://github.com/apache/zeppelin/pull/2824
  
I can confirm that it works - tested on branch-0.8


---

[GitHub] zeppelin issue #2819: ZEPPELIN-3265. DevInterpreter doesn't work

[weand]

Github user weand commented on the issue:

https://github.com/apache/zeppelin/pull/2819
  
@zjffdu this PR creates duplicate interpreters in interpreter.json.

after upgrading to this commit, all installed interpreters are duplicated, 
e.g. %angular, %md, %spark, %python, ... .

Is this the intended behavior?


---

[GitHub] zeppelin pull request #2808: [ZEPPELIN-3243] fix IndexOutOfBoundsException w...

[weand]

Github user weand closed the pull request at:

https://github.com/apache/zeppelin/pull/2808


---

[GitHub] zeppelin pull request #2808: [ZEPPELIN-3243] fix IndexOutOfBoundsException w...

[weand]

GitHub user weand reopened a pull request:

https://github.com/apache/zeppelin/pull/2808

[ZEPPELIN-3243] fix IndexOutOfBoundsException when 'No interpreter is 
binded to this note'

### What is this PR for?
I observed IndexOutOfBoundsException when using zeppelin locally on an 
existing notebook repository and with fresh interpreter.json file.
The default Zeppelin behavior when using existing notebooks is the manual 
binding of the interpreter settings, which works fine. This PR simply avoids 
the IndexOutOfBoundsException.

### What type of PR is it?
Bug Fix

### Todos
-

### What is the Jira issue?

[https://issues.apache.org/jira/browse/ZEPPELIN-3243](https://issues.apache.org/jira/browse/ZEPPELIN-3243)

### How should this be tested?
- manual

### Screenshots (if appropriate)

### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? no


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/weand/zeppelin ZEPPELIN-3243

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/zeppelin/pull/2808.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2808


commit b97a38eeed89573e64059dc862ab9bc05700ce7d
Author: Andreas Weise <a.weise@...>
Date:   2018-02-18T11:55:32Z

ZEPPELIN-3243 fix IndexOutOfBoundsException when 'No interpreter is
binded to this note'




---

[GitHub] zeppelin issue #2439: [ZEPPELIN-2680] allow opening notebook as a reader

[weand]

Github user weand commented on the issue:

https://github.com/apache/zeppelin/pull/2439
  
@herval: I merged latest master: 
[https://github.com/weand/zeppelin/tree/ZEPPELIN-2680](https://github.com/weand/zeppelin/tree/ZEPPELIN-2680)

But I also face continuous requests against helium endpoint (without having 
any helium configured at all). Same behavior like @khalidhuseynov. I recognized 
this only by having a look at the Network Communication in the browsers 
developer tools.

I'm not into the angular stuff so much. Could you have a look?


---

[GitHub] zeppelin issue #2749: [ZEPPELIN-3180] BUGFIX - save operation on interpreter...

[weand]

Github user weand commented on the issue:

https://github.com/apache/zeppelin/pull/2749
  
LGTM @felixcheung what do you think?


---

[GitHub] zeppelin issue #2439: [ZEPPELIN-2680] allow opening notebook as a reader

[weand]

Github user weand commented on the issue:

https://github.com/apache/zeppelin/pull/2439
  
@herval would you mind merging again on latest master?

@soralee @zjffdu as we face the same issue, and we are using helium, I will 
also test it.


---

[GitHub] zeppelin pull request #2808: [ZEPPELIN-3243] fix IndexOutOfBoundsException w...

[weand]

GitHub user weand opened a pull request:

https://github.com/apache/zeppelin/pull/2808

[ZEPPELIN-3243] fix IndexOutOfBoundsException when 'No interpreter is 
binded to this note'

### What is this PR for?
I observed IndexOutOfBoundsException when using zeppelin locally on an 
existing notebook repository and with fresh interpreter.json file.
The default Zeppelin behavior when using existing notebooks is the manual 
binding of the interpreter settings, which works fine. This PR simply avoids 
the IndexOutOfBoundsException.

### What type of PR is it?
Bug Fix

### Todos
-

### What is the Jira issue?

[https://issues.apache.org/jira/browse/ZEPPELIN-3243](https://issues.apache.org/jira/browse/ZEPPELIN-3243)

### How should this be tested?
- manual

### Screenshots (if appropriate)

### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? no


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/weand/zeppelin ZEPPELIN-3243

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/zeppelin/pull/2808.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2808


commit b97a38eeed89573e64059dc862ab9bc05700ce7d
Author: Andreas Weise <a.weise@...>
Date:   2018-02-18T11:55:32Z

ZEPPELIN-3243 fix IndexOutOfBoundsException when 'No interpreter is
binded to this note'




---

[GitHub] zeppelin issue #2749: [ZEPPELIN-3180] BUGFIX - save operation on interpreter...

[weand]

Github user weand commented on the issue:

https://github.com/apache/zeppelin/pull/2749
  
any progress here?


---

[GitHub] zeppelin pull request #2806: [ZEPPELIN-3241] fix NPE when restarting Interpr...

[weand]

GitHub user weand opened a pull request:

https://github.com/apache/zeppelin/pull/2806

[ZEPPELIN-3241] fix NPE when restarting Interpreter

### What is this PR for?
We often (but not always) face an NPE when restarting Spark interpreter. 
Our Spark interpreter run in scoped mode per note. The restart then is not 
possible, only restart of whole zeppelin daemon helps out.

This PR fixes this NPE.

### What type of PR is it?
Bug Fix

### Todos

### What is the Jira issue?

[https://issues.apache.org/jira/browse/ZEPPELIN-3241](https://issues.apache.org/jira/browse/ZEPPELIN-3241)

### How should this be tested?
- manual

### Screenshots (if appropriate)

### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? no


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/weand/zeppelin ZEPPELIN-3241

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/zeppelin/pull/2806.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2806


commit 85c04481011acede3e7292d2b260ddf28084cb61
Author: Andreas Weise <a.weise@...>
Date:   2018-02-16T21:45:59Z

ZEPPELIN-3241: NPE when restarting Interpreter




---

[GitHub] zeppelin pull request #2706: ZEPPELIN-3105 Notebook not running via REST API...

[weand]

Github user weand closed the pull request at:

https://github.com/apache/zeppelin/pull/2706


---

[GitHub] zeppelin pull request #2706: ZEPPELIN-3105 Notebook not running via REST API...

[weand]

GitHub user weand reopened a pull request:

https://github.com/apache/zeppelin/pull/2706

ZEPPELIN-3105 Notebook not running via REST API after Zeppelin Restart

### What is this PR for?
Fixes regression introduced with 
[ZEPPELIN-3013](https://issues.apache.org/jira/browse/ZEPPELIN-3013). Executing 
Notebook and
Paragraph via Rest API was not possible anymore, because some parsing
was introduced paragraph.setText(...). The setter was not invoked after
deserialization, thus transient variables paragraph.intpText and 
paragraph.scriptText were
not set.

This PR fixes that by invoking paragraph.parseText() after deserialization.

Also a test was added, and AbstractTestRestApi.shutdown() method was
overloaded to support the use case of not deleting the interpreter.json
during server restart within one test case.

### What type of PR is it?
[Bug Fix]

### Todos
-

### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-3105

### How should this be tested?
Refer to the test case added,
org.apache.zeppelin.rest.NotebookRestApiTest.testRunWithServerRestart()

### Screenshots (if appropriate)

### Questions:
* Does the licenses files need update? - no
* Is there breaking changes for older versions? - no
* Does this needs documentation? - no

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/weand/zeppelin ZEPPELIN-3105

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/zeppelin/pull/2706.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2706


commit 591ec8da28dac68bc67d21bb25a504563a311210
Author: Andreas Weise <a.weise@...>
Date:   2017-12-17T22:35:42Z

ZEPPELIN-3105 Notebook not running via REST API after Zeppelin Restart

commit ea8f3ead61f1b6c283ef7e75230c1cbd0c80e0b0
Author: Andreas Weise <a.weise@...>
Date:   2017-12-18T22:14:06Z

ZEPPELIN-3105 Notebook not running via REST API after Zeppelin Restart

parseText within fromJson Method.

commit ff4398313a55ae5ddd238083de264c1c31230e9a
Author: Andreas Weise <a.weise@...>
Date:   2017-12-19T20:14:22Z

Merge branch 'master' of https://github.com/apache/zeppelin.git into
ZEPPELIN-3105

commit 02e1f08bef36436252ce104e550b7b3186981455
Author: Andreas Weise <a.weise@...>
Date:   2017-12-21T21:24:52Z

Merge branch 'master' of https://github.com/apache/zeppelin.git into 
ZEPPELIN-3105

commit d46182f789bb227f14aa0a5e54eb0e5d5d7c382a
Author: Andreas Weise <a.weise@...>
Date:   2017-12-22T19:10:11Z

Merge branch 'master' of https://github.com/apache/zeppelin.git into
ZEPPELIN-3105

Conflicts:

zeppelin-server/src/test/java/org/apache/zeppelin/rest/AbstractTestRestApi.java




---

[GitHub] zeppelin issue #2706: ZEPPELIN-3105 Notebook not running via REST API after ...

[weand]

Github user weand commented on the issue:

https://github.com/apache/zeppelin/pull/2706
  
@zjffdu As this fixes a regression from 
[ZEPPELIN-3013](https://issues.apache.org/jira/browse/ZEPPELIN-3013) would you 
mind helping here out?


---

[GitHub] zeppelin issue #2706: ZEPPELIN-3105 Notebook not running via REST API after ...

[weand]

Github user weand commented on the issue:

https://github.com/apache/zeppelin/pull/2706
  
Don't get the Selenium tests running properly.

Instead of the error from Travis CI, I face other errors, that number of 
python processes is wrong.

```
21:05:52,088  INFO org.apache.zeppelin.CommandExecutor:46 - Sending command 
"bash -c ps aux | grep 'zeppelin_ipython' | grep -v 'grep' | wc -l" to localhost
21:05:52,290  INFO org.apache.zeppelin.CommandExecutor:46 - Sending command 
"bash -c ps aux | grep 'zeppelin/interpreter/python/*' | sed -E 
'/grep|local-repo/d' | wc -l" to localhost
21:05:56,226  INFO org.apache.zeppelin.CommandExecutor:46 - Sending command 
"bash -c ../bin/zeppelin-daemon.sh restart" to localhost
Tests run: 11, Failures: 11, Errors: 0, Skipped: 0, Time elapsed: 471.138 
sec <<< FAILURE! - in org.apache.zeppelin.integration.InterpreterModeActionsIT

testPerUserIsolatedAction(org.apache.zeppelin.integration.InterpreterModeActionsIT)
  Time elapsed: 186.266 sec  <<< FAILURE!
java.lang.AssertionError: The number of python process is
Expected: "1"
 but: was "0"
at org.hamcrest
```

I already have ipython + jupyter installed. Also Firefox 31 with Auto 
Update disabled.

When starting zeppelin and a python notebook (created by the tests) 
manually, I get proper results inside Web GUI. I also see various zeppelin 
processes via ps aux. But the grep pipes used in the test doesn't cover them.

Could anyone please assist, who can run the test properly. @soralee ?


---

[GitHub] zeppelin pull request #2706: ZEPPELIN-3105 Notebook not running via REST API...

[weand]

GitHub user weand reopened a pull request:

https://github.com/apache/zeppelin/pull/2706

ZEPPELIN-3105 Notebook not running via REST API after Zeppelin Restart

### What is this PR for?
Fixes regression introduced with 
[ZEPPELIN-3013](https://issues.apache.org/jira/browse/ZEPPELIN-3013). Executing 
Notebook and
Paragraph via Rest API was not possible anymore, because some parsing
was introduced paragraph.setText(...). The setter was not invoked after
deserialization, thus transient variables paragraph.intpText and 
paragraph.scriptText were
not set.

This PR fixes that by invoking paragraph.parseText() after deserialization.

Also a test was added, and AbstractTestRestApi.shutdown() method was
overloaded to support the use case of not deleting the interpreter.json
during server restart within one test case.

### What type of PR is it?
[Bug Fix]

### Todos
-

### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-3105

### How should this be tested?
Refer to the test case added,
org.apache.zeppelin.rest.NotebookRestApiTest.testRunWithServerRestart()

### Screenshots (if appropriate)

### Questions:
* Does the licenses files need update? - no
* Is there breaking changes for older versions? - no
* Does this needs documentation? - no

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/weand/zeppelin ZEPPELIN-3105

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/zeppelin/pull/2706.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2706


commit 591ec8da28dac68bc67d21bb25a504563a311210
Author: Andreas Weise <a.weise@...>
Date:   2017-12-17T22:35:42Z

ZEPPELIN-3105 Notebook not running via REST API after Zeppelin Restart

commit ea8f3ead61f1b6c283ef7e75230c1cbd0c80e0b0
Author: Andreas Weise <a.weise@...>
Date:   2017-12-18T22:14:06Z

ZEPPELIN-3105 Notebook not running via REST API after Zeppelin Restart

parseText within fromJson Method.

commit ff4398313a55ae5ddd238083de264c1c31230e9a
Author: Andreas Weise <a.weise@...>
Date:   2017-12-19T20:14:22Z

Merge branch 'master' of https://github.com/apache/zeppelin.git into
ZEPPELIN-3105

commit 02e1f08bef36436252ce104e550b7b3186981455
Author: Andreas Weise <a.weise@...>
Date:   2017-12-21T21:24:52Z

Merge branch 'master' of https://github.com/apache/zeppelin.git into 
ZEPPELIN-3105

commit d46182f789bb227f14aa0a5e54eb0e5d5d7c382a
Author: Andreas Weise <a.weise@...>
Date:   2017-12-22T19:10:11Z

Merge branch 'master' of https://github.com/apache/zeppelin.git into
ZEPPELIN-3105

Conflicts:

zeppelin-server/src/test/java/org/apache/zeppelin/rest/AbstractTestRestApi.java




---

[GitHub] zeppelin pull request #2706: ZEPPELIN-3105 Notebook not running via REST API...

[weand]

Github user weand closed the pull request at:

https://github.com/apache/zeppelin/pull/2706


---

[GitHub] zeppelin issue #2706: ZEPPELIN-3105 Notebook not running via REST API after ...

[weand]

Github user weand commented on the issue:

https://github.com/apache/zeppelin/pull/2706
  
This time it failed here:

org.apache.zeppelin.integration.InterpreterModeActionsIT.testPerUserIsolatedAction(InterpreterModeActionsIT.java:729)

the last time it failed at another line - 660:

org.apache.zeppelin.integration.InterpreterModeActionsIT.testPerUserIsolatedAction(InterpreterModeActionsIT.java:660)

this seems quite flaky. Can not see, why the changes here break this test. 
Do you have any idea?


---

[GitHub] zeppelin issue #2706: ZEPPELIN-3105 Notebook not running via REST API after ...

[weand]

Github user weand commented on the issue:

https://github.com/apache/zeppelin/pull/2706
  
close and reopen PR does NOT trigger the CI.

now merged latest master again, and awaiting test results.


---

[GitHub] zeppelin pull request #2706: ZEPPELIN-3105 Notebook not running via REST API...

[weand]

Github user weand closed the pull request at:

https://github.com/apache/zeppelin/pull/2706


---

[GitHub] zeppelin pull request #2706: ZEPPELIN-3105 Notebook not running via REST API...

[weand]

GitHub user weand reopened a pull request:

https://github.com/apache/zeppelin/pull/2706

ZEPPELIN-3105 Notebook not running via REST API after Zeppelin Restart

### What is this PR for?
Fixes regression introduced with 
[ZEPPELIN-3013](https://issues.apache.org/jira/browse/ZEPPELIN-3013). Executing 
Notebook and
Paragraph via Rest API was not possible anymore, because some parsing
was introduced paragraph.setText(...). The setter was not invoked after
deserialization, thus transient variables paragraph.intpText and 
paragraph.scriptText were
not set.

This PR fixes that by invoking paragraph.parseText() after deserialization.

Also a test was added, and AbstractTestRestApi.shutdown() method was
overloaded to support the use case of not deleting the interpreter.json
during server restart within one test case.

### What type of PR is it?
[Bug Fix]

### Todos
-

### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-3105

### How should this be tested?
Refer to the test case added,
org.apache.zeppelin.rest.NotebookRestApiTest.testRunWithServerRestart()

### Screenshots (if appropriate)

### Questions:
* Does the licenses files need update? - no
* Is there breaking changes for older versions? - no
* Does this needs documentation? - no

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/weand/zeppelin ZEPPELIN-3105

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/zeppelin/pull/2706.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2706


commit 591ec8da28dac68bc67d21bb25a504563a311210
Author: Andreas Weise <a.weise@...>
Date:   2017-12-17T22:35:42Z

ZEPPELIN-3105 Notebook not running via REST API after Zeppelin Restart

commit ea8f3ead61f1b6c283ef7e75230c1cbd0c80e0b0
Author: Andreas Weise <a.weise@...>
Date:   2017-12-18T22:14:06Z

ZEPPELIN-3105 Notebook not running via REST API after Zeppelin Restart

parseText within fromJson Method.




---

[GitHub] zeppelin issue #2706: ZEPPELIN-3105 Notebook not running via REST API after ...

[weand]

Github user weand commented on the issue:

https://github.com/apache/zeppelin/pull/2706
  
@zjffdu can u confirm it's a flaky test?


---

[GitHub] zeppelin pull request #2706: ZEPPELIN-3105 Notebook not running via REST API...

[weand]

Github user weand closed the pull request at:

https://github.com/apache/zeppelin/pull/2706


---

[GitHub] zeppelin pull request #2706: ZEPPELIN-3105 Notebook not running via REST API...

[weand]

GitHub user weand reopened a pull request:

https://github.com/apache/zeppelin/pull/2706

ZEPPELIN-3105 Notebook not running via REST API after Zeppelin Restart

### What is this PR for?
Fixes regression introduced with 
[ZEPPELIN-3013](https://issues.apache.org/jira/browse/ZEPPELIN-3013). Executing 
Notebook and
Paragraph via Rest API was not possible anymore, because some parsing
was introduced paragraph.setText(...). The setter was not invoked after
deserialization, thus transient variables paragraph.intpText and 
paragraph.scriptText were
not set.

This PR fixes that by invoking paragraph.parseText() after deserialization.

Also a test was added, and AbstractTestRestApi.shutdown() method was
overloaded to support the use case of not deleting the interpreter.json
during server restart within one test case.

### What type of PR is it?
[Bug Fix]

### Todos
-

### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-3105

### How should this be tested?
Refer to the test case added,
org.apache.zeppelin.rest.NotebookRestApiTest.testRunWithServerRestart()

### Screenshots (if appropriate)

### Questions:
* Does the licenses files need update? - no
* Is there breaking changes for older versions? - no
* Does this needs documentation? - no

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/weand/zeppelin ZEPPELIN-3105

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/zeppelin/pull/2706.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2706


commit 591ec8da28dac68bc67d21bb25a504563a311210
Author: Andreas Weise <a.weise@...>
Date:   2017-12-17T22:35:42Z

ZEPPELIN-3105 Notebook not running via REST API after Zeppelin Restart

commit ea8f3ead61f1b6c283ef7e75230c1cbd0c80e0b0
Author: Andreas Weise <a.weise@...>
Date:   2017-12-18T22:14:06Z

ZEPPELIN-3105 Notebook not running via REST API after Zeppelin Restart

parseText within fromJson Method.




---

[GitHub] zeppelin pull request #2706: ZEPPELIN-3105 Notebook not running via REST API...

[weand]

GitHub user weand reopened a pull request:

https://github.com/apache/zeppelin/pull/2706

ZEPPELIN-3105 Notebook not running via REST API after Zeppelin Restart

### What is this PR for?
Fixes regression introduced with 
[ZEPPELIN-3013](https://issues.apache.org/jira/browse/ZEPPELIN-3013). Executing 
Notebook and
Paragraph via Rest API was not possible anymore, because some parsing
was introduced paragraph.setText(...). The setter was not invoked after
deserialization, thus transient variables paragraph.intpText and 
paragraph.scriptText were
not set.

This PR fixes that by invoking paragraph.parseText() after deserialization.

Also a test was added, and AbstractTestRestApi.shutdown() method was
overloaded to support the use case of not deleting the interpreter.json
during server restart within one test case.

### What type of PR is it?
[Bug Fix]

### Todos
-

### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-3105

### How should this be tested?
Refer to the test case added,
org.apache.zeppelin.rest.NotebookRestApiTest.testRunWithServerRestart()

### Screenshots (if appropriate)

### Questions:
* Does the licenses files need update? - no
* Is there breaking changes for older versions? - no
* Does this needs documentation? - no

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/weand/zeppelin ZEPPELIN-3105

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/zeppelin/pull/2706.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2706


commit 591ec8da28dac68bc67d21bb25a504563a311210
Author: Andreas Weise <a.weise@...>
Date:   2017-12-17T22:35:42Z

ZEPPELIN-3105 Notebook not running via REST API after Zeppelin Restart

commit ea8f3ead61f1b6c283ef7e75230c1cbd0c80e0b0
Author: Andreas Weise <a.weise@...>
Date:   2017-12-18T22:14:06Z

ZEPPELIN-3105 Notebook not running via REST API after Zeppelin Restart

parseText within fromJson Method.




---

[GitHub] zeppelin pull request #2706: ZEPPELIN-3105 Notebook not running via REST API...

[weand]

Github user weand closed the pull request at:

https://github.com/apache/zeppelin/pull/2706


---

[GitHub] zeppelin issue #2706: ZEPPELIN-3105 Notebook not running via REST API after ...

[weand]

Github user weand commented on the issue:

https://github.com/apache/zeppelin/pull/2706
  
Tests run: 3, Failures: 0, Errors: 3, Skipped: 0, Time elapsed: 164.418 sec 
<<< FAILURE! - in org.apache.zeppelin.integration.InterpreterModeActionsIT

seems to be a flaky test, see log 
[here](https://travis-ci.org/weand/zeppelin/jobs/318812306)


---

[GitHub] zeppelin issue #2706: ZEPPELIN-3105 Notebook not running via REST API after ...

[weand]

Github user weand commented on the issue:

https://github.com/apache/zeppelin/pull/2706
  
how can I retrigger CI? 


---

[GitHub] zeppelin pull request #2706: ZEPPELIN-3105 Notebook not running via REST API...

[weand]

GitHub user weand opened a pull request:

https://github.com/apache/zeppelin/pull/2706

ZEPPELIN-3105 Notebook not running via REST API after Zeppelin Restart

### What is this PR for?
Fixes regression introduced with 
[ZEPPELIN-3013](https://issues.apache.org/jira/browse/ZEPPELIN-3013). Executing 
Notebook and
Paragraph via Rest API was not possible anymore, because some parsing
was introduced paragraph.setText(...). The setter was not invoked after
deserialization, thus paragraph.intpText and paragraph.scriptText was
not set.

This PR fixes that by introducing a PostJsonDeserProcessable interface,
that can be used on any class implementing JsonSerializable, for 
implementing some routines to be invoked after deserialization. For
enabling the interface, a corresponding TypeAdapterFactory must be
registered on the GsonBuilder.

Paragraph class now implements PostJsonDeserProcessable and parses
intpText and scriptText variables properly after deserialization.

Also a test was added, and AbstractTestRestApi.shutdown() method was
overloaded to support the use case of not deleting the interpreter.json
during server restart within one test case.

### What type of PR is it?
[Bug Fix]

### Todos
-

### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-3105

### How should this be tested?
Refer to the test case added,
org.apache.zeppelin.rest.NotebookRestApiTest.testRunWithServerRestart()

### Screenshots (if appropriate)

### Questions:
* Does the licenses files need update? - no
* Is there breaking changes for older versions? - no
* Does this needs documentation? - no

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/weand/zeppelin ZEPPELIN-3105

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/zeppelin/pull/2706.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2706


commit 0785ea5e85a43043c852b4f62270594f663ea977
Author: Andreas Weise <a.we...@avm.de>
Date:   2017-12-17T22:35:42Z

ZEPPELIN-3105 Notebook not running via REST API after Zeppelin Restart:
skip to run blank paragraph

### What is this PR for?
Fixes regression introduced with ZEPPELIN-3013. Executing Notebook and
Paragraph via Rest API was not possible anymore, because some parsing
was introduced paragraph.setText(...). The setter was not invoked after
deserialization, thus paragraph.intpText and paragraph.scriptText was
not set.

This PR fixes that by introducing a PostJsonDeserProcessable interface,
that can be used on any class implementing JsonSerializable. For
enabling the interface, a corresponding TypeAdapterFactory must be
registered on the GsonBuilder.

Paragraph class now implements PostJsonDeserProcessable and parses
intpText and scriptText variables properly.

Also a test was added, and AbstractTestRestApi.shutdown() method was
overloaded to support the use case of not deleting the interpreter.json
during server restart within one test case.

### What type of PR is it?
[Bug Fix]

### Todos
-

### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-3105

### How should this be tested?
Refer to the test case added,
org.apache.zeppelin.rest.NotebookRestApiTest.testRunWithServerRestart()

### Screenshots (if appropriate)

### Questions:
* Does the licenses files need update? - no
* Is there breaking changes for older versions? - no
* Does this needs documentation? - no




---

[GitHub] zeppelin pull request #2675: [ZEPPELIN-3060] Unable to use interpreter names...

[weand]

Github user weand closed the pull request at:

https://github.com/apache/zeppelin/pull/2675


---

[GitHub] zeppelin issue #2675: [ZEPPELIN-3060] Unable to use interpreter names with s...

[weand]

Github user weand commented on the issue:

https://github.com/apache/zeppelin/pull/2675
  
Closed and created https://issues.apache.org/jira/browse/ZEPPELIN-3083 
instead.


---

[GitHub] zeppelin issue #2675: [ZEPPELIN-3060] Unable to use interpreter names with s...

[weand]

Github user weand commented on the issue:

https://github.com/apache/zeppelin/pull/2675
  
Well, the current implementation does not enforce any naming pattern when 
creating a new interpreter.

Before ZEPPELIN-3013 interpreters with any characters e.g. - could be 
created and executed.

If that's not supported anymore, there should be a failure when creating 
such an interpreter.

The current behavior is quite misleading, as the interpreter name is not 
even display correctly in the error message. See the corresponding Jira ticket 
of this PR.


---

[GitHub] zeppelin pull request #2675: [ZEPPELIN-3060] Unable to use interpreter names...

[weand]

GitHub user weand opened a pull request:

https://github.com/apache/zeppelin/pull/2675

[ZEPPELIN-3060] Unable to use interpreter names with special characters 
(e.g. dash or underscore)

[ZEPPELIN-3060] Unable to use interpreter names with special characters
(e.g. dash or underscore)

### What is this PR for?
This fixes a regression which was introduced with ZEPPELIN-3013. There
is no restriction regarding characters when creating new interpreters.
But when running a note the interpreter name was parsed using the regex
%([\w\.]+).*, which only considered the word character class and dots.
Interpreter names with special characters (e.g. _ or -) were not matched
correctly. This change enables usage of interpreter names that use other
characters by changing the regex group to %([^\s\(]+).*, which means:
parse the interpreter name from % until any whitespace or open
parenthesis.

### What type of PR is it?
Bug Fix

### Todos
-

### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-3060

### How should this be tested?
Create an interpreter with a name containing one or more special
characters, e.g. dash or underscore. Try running that interpreter.

### Screenshots (if appropriate)

### Questions:
* Does the licenses files need update? no 
* Is there breaking changes for older versions? no
* Does this needs documentation? no

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/weand/zeppelin ZEPPELIN-3060

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/zeppelin/pull/2675.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2675


commit 7d715a0fcc8ee37059890926b282b163391118e1
Author: Andreas Weise <a.we...@avm.de>
Date:   2017-11-16T20:09:36Z

[ZEPPELIN-3060] Unable to use interpreter names with special characters
(e.g. dash or underscore)

### What is this PR for?
This fixes a regression which was introduced with ZEPPELIN-3013. There
is no restriction regarding characters when creating new interpreters.
But when running a note the interpreter name was parsed using the regex
%([\w\.]+).*, which only considered the word character class and dots.
Interpreter names with special characters (e.g. _ or -) were not matched
correctly. This change enables usage of interpreter names that use other
characters by changing the regex group to %([^\s\(]+).*, which means:
parse the interpreter name from % until any whitespace or open
parenthesis.

### What type of PR is it?
Bug Fix

### Todos
-

### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-3060

### How should this be tested?
Create an interpreter with a name containing one or more special
characters, e.g. dash or underscore. Try running that interpreter.

### Screenshots (if appropriate)

### Questions:
* Does the licenses files need update? no 
* Is there breaking changes for older versions? no
* Does this needs documentation? no

commit 54c69ca1686338420797409b7ea6f79f609bef8d
Author: Andreas Weise <a.we...@avm.de>
Date:   2017-11-16T20:15:07Z

Merge remote-tracking branch 'origin/master'




---

[GitHub] zeppelin issue #2647: [ZEPPELIN-3012] Interpreter Permissions not working pr...

[weand]

Github user weand commented on the issue:

https://github.com/apache/zeppelin/pull/2647
  
Tests added added. Can you merge that into master?


---

[GitHub] zeppelin pull request #2647: [ZEPPELIN-3012] Interpreter Permissions not wor...

[weand]

GitHub user weand opened a pull request:

https://github.com/apache/zeppelin/pull/2647

[ZEPPELIN-3012] Interpreter Permissions not working properly for groups

### What is this PR for?
This fixes issues when using Zeppelin with permissions for users belonging 
to more than one roles / groups. Constructor of 
org.apache.zeppelin.user.AuthenticationInfo didn't consider that the roles 
String will contain spaces, in case there is more than one role, e.g. 'role1, 
role2'.

This change fixes the issue, by invoking trim() on each role.

### What type of PR is it?
[Bug Fix]

### Todos

### What is the Jira issue?
https://issues.apache.org/jira/browse/ZEPPELIN-3012

### How should this be tested?
* Enable shiro.ini, including 'admin' user
* Login with 'admin' and enable Interpreter Permissions for any 
Interpreter, granting access to 'role2'
* Try to use this Interpreter with 'user1', who belongs to role2.

### Screenshots (if appropriate)

### Questions:
* Does the licenses files need update? no
* Is there breaking changes for older versions? no
* Does this needs documentation? no


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/weand/zeppelin ZEPPELIN-3012

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/zeppelin/pull/2647.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2647


commit 1e5d2b72491289ca7572966d74c7bf6bc7ac81ec
Author: Andreas Weise <a.we...@avm.de>
Date:   2017-11-02T23:11:53Z

[ZEPPELIN-3012] Interpreter Permissions not working properly for groups




---

[GitHub] zeppelin issue #2062: ZEPPELIN-2161 Nested Group Support in LdapRealm for AD

[weand]

Github user weand commented on the issue:

https://github.com/apache/zeppelin/pull/2062
  
Rebased and green now. Thanks for your help.
Would you mind merging that into branch-0.7 as well?

Regarding docu:
The LDAP Realm section in the docu 
[shiroauthentication.md](https://github.com/apache/zeppelin/blob/master/docs/security/shiroauthentication.md#ldap)
 only mentions LdapGroupRealm implementation. This change is related to the 
undocumented LdapRealm, which was introduced in 0.7.0.

Should we document two LDAP Realms, or should we mark the old one 
deprecated? 

Because of that, I would prefer an separate Issue for the missing piece of 
documentation.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] zeppelin issue #2062: ZEPPELIN-2161 Nested Group Support in LdapRealm for AD

[weand]

Github user weand commented on the issue:

https://github.com/apache/zeppelin/pull/2062
  
@prabhjyotsingh There isn't any documentation for 
org.apache.zeppelin.realm.LdapRealm yet.

@Leemoonsoo Any special reason why LdapRealm wasn't documented in 0.7.0 
yet? 

Can anyone help me why the build is failing. I don't see, why these tests 
now fail, they have no obvious reference to changes of this PR:
  NotebookTest.testPublicPrivateNewNote:1173 expected:<2C[CJ4Y6RZ]> but 
was:<2C[ADXZJ8N]>
  NotebookTest.testAuthorizationRoles:759 expected: but was:


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] zeppelin issue #2062: ZEPPELIN-2161 Nested Group Support in LdapRealm for AD

[weand]

Github user weand commented on the issue:

https://github.com/apache/zeppelin/pull/2062
  
any idea why the build has failed? I don't get the error.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] zeppelin pull request #2062: ZEPPELIN-2161 Nested Group Support in LdapRealm...

[weand]

GitHub user weand opened a pull request:

https://github.com/apache/zeppelin/pull/2062

ZEPPELIN-2161 Nested Group Support in LdapRealm for AD

### What is this PR for?
A common use case in LDAP/AD setup is the hierarchical structuring of
groups - a.k.a. adding groups to other groups. Such nesting groups can
help reduce the number of roles that need to be managed.

Current zeppelin realm implementations doesn't have support for looking
up memberships throughout nested group structures.

E.g. consider the following nested group scenario:
```
acme_employees
 \__department_a
 \__sub_department_x
```
User 'bob' is in Group 'sub_department_x'.
Notebook 'note1' has a Reader Role assignment for 'department_a' or
'acme_employees'.
Then access must be granted for 'bob' on 'note1'.

In AD enviroments this scenarios can be efficiently implemented using
the so called LDAP_MATCHING_RULE_IN_CHAIN operator
'1.2.840.113556.1.4.1941'.

This PR introduces a property 'groupSearchEnableMatchingRuleInChain' to
org.apache.zeppelin.realm.LdapRealm which defaults to false. If enabled,
all roles of potential nested group hierarchies will be resolved using
the LDAP_MATCHING_RULE_IN_CHAIN operator.

### What type of PR is it?
Improvement

### Todos
-

### What is the Jira issue?
[ZEPPELIN-2161]

### How should this be tested?
Set groupSearchEnableMatchingRuleInChain = true for the ldap realm.

### Screenshots (if appropriate)

### Questions:
* Does the licenses files need update? n
* Is there breaking changes for older versions? n
* Does this needs documentation? y

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/weand/zeppelin ZEPPELIN-2161

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/zeppelin/pull/2062.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2062






---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] zeppelin pull request #2061: ZEPPELIN-2161 Nested Group Support in LdapRealm...

[weand]

Github user weand closed the pull request at:

https://github.com/apache/zeppelin/pull/2061


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] zeppelin issue #2061: ZEPPELIN-2161 Nested Group Support in LdapRealm for AD

[weand]

Github user weand commented on the issue:

https://github.com/apache/zeppelin/pull/2061
  
any idea why the build has failed?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] zeppelin pull request #2061: ZEPPELIN-2161 Nested Group Support in LdapRealm...

[weand]

GitHub user weand opened a pull request:

https://github.com/apache/zeppelin/pull/2061

ZEPPELIN-2161 Nested Group Support in LdapRealm for AD

### What is this PR for?
A common use case in LDAP/AD setup is the hierarchical structuring of
groups - a.k.a. adding groups to other groups. Such nesting groups can
help reduce the number of roles that need to be managed.

Current zeppelin realm implementations doesn't have support for looking
up memberships throughout nested group structures.

E.g. consider the following nested group scenario:
```
acme_employees
 \__department_a
 \__sub_department_x
```
User 'bob' is in Group 'sub_department_x'.
Notebook 'note1' has a Reader Role assignment for 'department_a' or
'acme_employees'.
Then access must be granted for 'bob' on 'note1'.

In AD enviroments this scenarios can be efficiently implemented using
the so called LDAP_MATCHING_RULE_IN_CHAIN operator
'1.2.840.113556.1.4.1941'.

This PR introduces a property 'groupSearchEnableMatchingRuleInChain' to
org.apache.zeppelin.realm.LdapRealm which defaults to false. If enabled,
all roles of potential nested group hierarchies will be resolved using
the LDAP_MATCHING_RULE_IN_CHAIN operator.

### What type of PR is it?
Improvement

### Todos
-

### What is the Jira issue?
[ZEPPELIN-2161]

### How should this be tested?
Set groupSearchEnableMatchingRuleInChain = true for the ldap realm.

### Screenshots (if appropriate)

### Questions:
* Does the licenses files need update? n
* Is there breaking changes for older versions? n
* Does this needs documentation? y

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/weand/zeppelin master

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/zeppelin/pull/2061.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2061


commit f312f1134d40c7f9a28a1235c952bd85393bbb76
Author: Andreas Weise <a.we...@avm.de>
Date:   2017-02-23T21:55:06Z

ZEPPELIN-2161 Nested Group Support in LdapRealm for AD

### What is this PR for?
A common use case in LDAP/AD setup is the hierarchical structuring of
groups - a.k.a. adding groups to other groups. Such nesting groups can
help reduce the number of roles that need to be managed.
Current zeppelin realm implementations doesn't have support for looking
up memberships throughout nested group structures.
E.g. consider the following nested group scenario:
acme_employees
 \__department_a
 \__sub_department_x
User 'bob' is in Group 'sub_department_x'.
Notebook 'note1' has a Reader Role assignment for 'department_a' or
'acme_employees'.
Then access must be granted for 'bob' on 'note1'.
In AD enviroments this scenarios can be efficiently implemented using
the so called LDAP_MATCHING_RULE_IN_CHAIN operator
'1.2.840.113556.1.4.1941'.

This PR introduces a property 'groupSearchEnableMatchingRuleInChain' to
org.apache.zeppelin.realm.LdapRealm which defaults to false. If enabled,
all roles of potential nested group hierarchies will be resolved using
the LDAP_MATCHING_RULE_IN_CHAIN operator.

### What type of PR is it?
Improvement

### Todos
-

### What is the Jira issue?
[ZEPPELIN-2161]

### How should this be tested?
Set groupSearchEnableMatchingRuleInChain = true for the ldap realm.

### Screenshots (if appropriate)

### Questions:
* Does the licenses files need update? n
* Is there breaking changes for older versions? n
* Does this needs documentation? y




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---