[GitHub] [zeppelin] liuxunorg commented on issue #3342: [ZEPPELIN-4031] Fixed Unable to detect that the interpreter process was killed manually
liuxunorg commented on issue #3342: [ZEPPELIN-4031] Fixed Unable to detect that the interpreter process was killed manually URL: https://github.com/apache/zeppelin/pull/3342#issuecomment-477419405 CI Pass https://travis-ci.org/liuxunorg/zeppelin @zjffdu , @felixcheung , Please help me review code. In the second commit, I adjusted the way to recreate the invalid remote interpreter process. It is now very elegant to fix remote invalid interpreters, It can be used immediately without any manual intervention. See the GIF effect: https://github.com/apache/zeppelin/pull/3342#issuecomment-477046336 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [zeppelin] liuxunorg edited a comment on issue #3342: [ZEPPELIN-4031] Fixed Unable to detect that the interpreter process was killed manually
liuxunorg edited a comment on issue #3342: [ZEPPELIN-4031] Fixed Unable to detect that the interpreter process was killed manually URL: https://github.com/apache/zeppelin/pull/3342#issuecomment-477419405 CI Pass https://travis-ci.org/liuxunorg/zeppelin/builds/512089147 @zjffdu , @felixcheung , Please help me review code. In the second commit, I adjusted the way to recreate the invalid remote interpreter process. It is now very elegant to fix remote invalid interpreters, It can be used immediately without any manual intervention. See the GIF effect: https://github.com/apache/zeppelin/pull/3342#issuecomment-477046336 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [zeppelin] zjffdu commented on issue #3316: [ZEPPELIN-3985]. Move note permission from notebook-authorization.json to note file
zjffdu commented on issue #3316: [ZEPPELIN-3985]. Move note permission from notebook-authorization.json to note file URL: https://github.com/apache/zeppelin/pull/3316#issuecomment-477418606 Will merge if no more comments This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [zeppelin] liuxunorg commented on a change in pull request #3342: [ZEPPELIN-4031] Fixed Unable to detect that the interpreter process was killed manually
liuxunorg commented on a change in pull request #3342: [ZEPPELIN-4031] Fixed
Unable to detect that the interpreter process was killed manually
URL: https://github.com/apache/zeppelin/pull/3342#discussion_r269836270
##
File path:
zeppelin-zengine/src/main/java/org/apache/zeppelin/interpreter/remote/RemoteInterpreter.java
##
@@ -199,13 +199,31 @@ public Void call(Client client) throws Exception {
}
}
+ // Detecting an invalid interpreter process,
+ // Clean up session, Recreate the interpreter process
+ private void resurrectionInvalidIntpProcess() throws InterpreterException {
+if (interpreterProcess != null && !interpreterProcess.isRunning() &&
isOpened) {
+ LOGGER.info("Check whether the InterpreterProcess has been shutdown.");
+ // clean invalid session and dirty data of interpreterSetting
+ ManagedInterpreterGroup intpGroup = getInterpreterGroup();
+ intpGroup.close();
+ // recreate RemoteInterpreterProcess
+ isOpened = false;
+ isCreated = false;
+ open();
+}
+ }
+
@Override
public InterpreterResult interpret(final String st, final InterpreterContext
context)
throws InterpreterException {
if (LOGGER.isDebugEnabled()) {
LOGGER.debug("st:\n{}", st);
}
+// Detecting an invalid interpreter process
+resurrectionInvalidIntpProcess();
Review comment:
Every time the user executes a note, Called
`RemoteInterpreter.java::interpret(final String st, final InterpreterContext
context)` function,
Check if the remote interpreter process is valid, If the remote interpreter
is invalid,
In the `RemoteInterpreter.java`, It is very easy to perform session
recycling for the interpreter process,
And recreate the interpreter. Conforms to zeppelin's original process of
creating an interpreter.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [zeppelin] asfgit closed pull request #3338: ZEPPELIN-4081. when the python process is killed, the task state is still running
asfgit closed pull request #3338: ZEPPELIN-4081. when the python process is killed,the task state is still running URL: https://github.com/apache/zeppelin/pull/3338 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [zeppelin] asfgit closed pull request #3343: [ZEPPELIN-4063] Don't include noteId for constructing Interpreter GroupId when under isolated per user mode
asfgit closed pull request #3343: [ZEPPELIN-4063] Don't include noteId for constructing Interpreter GroupId when under isolated per user mode URL: https://github.com/apache/zeppelin/pull/3343 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [zeppelin] asfgit closed pull request #3331: [Zeppelin-4049] Hadoop Submarine (Machine Learning) interpreter
asfgit closed pull request #3331: [Zeppelin-4049] Hadoop Submarine (Machine Learning) interpreter URL: https://github.com/apache/zeppelin/pull/3331 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [zeppelin] liuxunorg commented on issue #3331: [Zeppelin-4049] Hadoop Submarine (Machine Learning) interpreter
liuxunorg commented on issue #3331: [Zeppelin-4049] Hadoop Submarine (Machine Learning) interpreter URL: https://github.com/apache/zeppelin/pull/3331#issuecomment-477409909 Thanks @zjffdu , @felixcheung review code, **[Hadoop submarine project (https://hadoop.apache.org/submarine)](https://hadoop.apache.org/submarine)** is developing new features support `Pytroch`, `MXNet`, such as `model publish`, `model incremental update` and , etc. These features are ideal for integration in zeppelin. I will continue to upgrade and update the hadoop submarine interpreter. :-) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
0.8.2-SNAPSHOT: WebSocket Write EOF 1006
Hello, I have created a docker container based on Python 3.7 with matplotlib and some other libraries. Here is my Dockerfile: RUN apt-get update RUN pip install --upgrade pip RUN pip install numpy pandas scipy matplotlib RUN apt-get install -y libaio-dev RUN apt-get clean I build the image: docker build --build-arg http_proxy=http://host:port --build-arg https_proxy=http:// host:port -t custompy37 . Then I activate my docker container: %python.docker activate custompy37 Then I run the attached notebook multiple times. Eventually I get the error below: NotebookServer.java[onClose]:372) - Closed connection to 10.39.227.226 : 58950. (1006) WebSocket Write EOF Questions: 1) Is it a known issue 2) What causes that? Thanks, Jim Python Note.json Description: application/json
[GitHub] [zeppelin] zjffdu commented on issue #3331: [Zeppelin-4049] Hadoop Submarine (Machine Learning) interpreter
zjffdu commented on issue #3331: [Zeppelin-4049] Hadoop Submarine (Machine Learning) interpreter URL: https://github.com/apache/zeppelin/pull/3331#issuecomment-477149884 Thanks @liuxunorg for this big contribution, I will merge it if no more comments. Since this PR is independent on other components, we can fix issues in other PRs later. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Created] (ZEPPELIN-4095) Only Readable Zeppelin Server
Jeff Zhang created ZEPPELIN-4095: Summary: Only Readable Zeppelin Server Key: ZEPPELIN-4095 URL: https://issues.apache.org/jira/browse/ZEPPELIN-4095 Project: Zeppelin Issue Type: New Feature Reporter: Jeff Zhang -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (ZEPPELIN-4094) Pyspark on Windows missing PYTHONPATH
Carsten Braess created ZEPPELIN-4094: Summary: Pyspark on Windows missing PYTHONPATH Key: ZEPPELIN-4094 URL: https://issues.apache.org/jira/browse/ZEPPELIN-4094 Project: Zeppelin Issue Type: Bug Components: Interpreters, pySpark Affects Versions: 0.8.1 Reporter: Carsten Braess Using default setup will result in exception when starting local pyspark remoteinterpreter due to pyspark module not found. In interpreter.cmd there is a set command for PYTHONPATH to include %ZEPPLIN_HOME%\interpreter\spark\pyspark\pyspark.zip and proper %ZEPPLIN_HOME%\interpreter\spark\pyspark\py4j-*.zip, which does work (verified by debug output), but somehow the spawned python process does not inherit it. Workaround: Set PYTHONPATH in windows enviroment variables to include both files. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] [zeppelin] liuxunorg commented on issue #3342: [ZEPPELIN-4031] Fixed Unable to detect that the interpreter process was killed manually
liuxunorg commented on issue #3342: [ZEPPELIN-4031] Fixed Unable to detect that the interpreter process was killed manually URL: https://github.com/apache/zeppelin/pull/3342#issuecomment-477046336 ### No need to click the second time to execute successfully. Restore the interpreter process immediately  This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [zeppelin] jongyoul commented on issue #3335: [MINOR] Refactor CronJob class
jongyoul commented on issue #3335: [MINOR] Refactor CronJob class URL: https://github.com/apache/zeppelin/pull/3335#issuecomment-477038163 CI passed: https://travis-ci.org/jongyoul/zeppelin/builds/511850669 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
