Great post!
On Tue, Oct 4, 2016 at 8:56 PM -0700, "Jongyoul Lee" <jongy...@gmail.com<mailto:jongy...@gmail.com>> wrote: Hello DuyHai, Thanks for the fixing the typo. I've fixed it. Concerning the debugging, I think writing posts or updating wiki would be better. I'm willing to write a simple post for that remote debugging. On Tue, Oct 4, 2016 at 5:53 PM, DuyHai Doan <doanduy...@gmail.com<mailto:doanduy...@gmail.com>> wrote: Hello Jongyoul Excellent write up to demystify the execution life-cycle of a paragraph. There is a small typo in the blog: "At last, it has two words separated by comma, getInterpreter handles it as "%{group_name}.{interpreter_name}"" Instead of comma, shouldn't it be DOT (.) ? There is also another unknown details that may be interesting to expose: how are interpreter JVM launched ? InterpreterFactory private Interpreter createRemoteRepl(String interpreterPath, String noteId, String className, Properties property, String interpreterSettingId) { int connectTimeout = conf.getInt(ConfVars.ZEPPELIN_INTERPRETER_CONNECT_TIMEOUT); String localRepoPath = conf.getInterpreterLocalRepoPath() + "/" + interpreterSettingId; int maxPoolSize = conf.getInt(ConfVars.ZEPPELIN_INTERPRETER_MAX_POOL_SIZE); RemoteInterpreter remoteInterpreter = new RemoteInterpreter(property, noteId, className, conf.getInterpreterRemoteRunnerPath(), interpreterPath, localRepoPath, connectTimeout, maxPoolSize, remoteInterpreterProcessListener, appEventListener); remoteInterpreter.addEnv(env); return new LazyOpenInterpreter(remoteInterpreter); } The code will call the bin/intepreter.sh (or interpreter.cmd if on Windows) to launch a new JVM for the interpreter. And depending on the type of interpreter, the command will be different: if [[ -n "${SPARK_SUBMIT}" ]]; then ${SPARK_SUBMIT} --class ${ZEPPELIN_SERVER} --driver-class-path "${ZEPPELIN_INTP_CLASSPATH_OVERRIDES}:${CLASSPATH}" --driver-java-options "${JAVA_INTP_OPTS}" ${SPARK_SUBMIT_OPTIONS} ${SPARK_APP_JAR} ${PORT} & else ${ZEPPELIN_RUNNER} ${JAVA_INTP_OPTS} ${ZEPPELIN_INTP_MEM} -cp ${ZEPPELIN_INTP_CLASSPATH_OVERRIDES}:${CLASSPATH} ${ZEPPELIN_SERVER} ${PORT} & fi There is a special case for Spark because we execute a SparkSubmit. This detail is important for contributors because when they need to DEBUG an interpreter code, they have to modify the interpreter.sh file to add an "-Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=xxx" Most of people naively add this debug argument to the zeppelin.sh file but it only debugs the Zeppelin server itself, not the interpreter. Do you think we may create a page in the official doc for this ? On Tue, Oct 4, 2016 at 9:00 AM, Jongyoul Lee <jongy...@gmail.com<mailto:jongy...@gmail.com>> wrote: Hello devs and users, I've submitted a post into Apache Zeppelin Stories on medium.com<http://medium.com>: https://medium.com/apache-zeppelin-stories/how-apache-zeppelin-runs-a-paragraph-783a0a612ba9#.x9ym8fovv I hope all users and contributors understand Apache Zeppelin deeply. Regards, Jongyoul -- ???, Jongyoul Lee, ??? http://madeng.net -- ???, Jongyoul Lee, ??? http://madeng.net
