Thanks Jougyoul for the effort. This is really helpful for new contributors.
Several comments.
1. Seems we didn't use noteId/notebookId correctly in code, do we have
ticket to track that ? Because the sample code is conflicted with the
definition of note/notebook in the post, it might confuse users.
@Path("job/{notebookId}/{paragraphId}")
@ZeppelinApi
public Response runParagraph(@PathParam("notebookId") String notebookId,
@PathParam("paragraphId") String paragraphId, String message)
throws IOException, IllegalArgumentException {
LOG.info("run paragraph job asynchronously {} {} {}", notebookId,
paragraphId, message);
Note note = notebook.getNote(notebookId);
2. It might be better to use diagram to visualize the differences
between shared/scoped/shared
On Sun, Oct 9, 2016 at 12:38 AM, Felix Cheung <[email protected]>
wrote:
> Great post!
>
>
>
>
>
> On Tue, Oct 4, 2016 at 8:56 PM -0700, "Jongyoul Lee" <[email protected]>
> wrote:
>
> Hello DuyHai,
>
> Thanks for the fixing the typo. I've fixed it.
>
> Concerning the debugging, I think writing posts or updating wiki would be
> better. I'm willing to write a simple post for that remote debugging.
>
> On Tue, Oct 4, 2016 at 5:53 PM, DuyHai Doan <[email protected]> wrote:
>
>> Hello Jongyoul
>>
>> Excellent write up to demystify the execution life-cycle of a paragraph.
>>
>> There is a small typo in the blog: "At last, it has two words separated
>> by comma, *getInterpreter* handles it as “%{group_name}.{interpreter_na
>> me}”"
>>
>> Instead of comma, shouldn't it be DOT (.) ?
>>
>> There is also another unknown details that may be interesting to expose:
>> how are interpreter JVM launched ?
>>
>> InterpreterFactory
>>
>> private Interpreter createRemoteRepl(String interpreterPath, String noteId,
>> String className,
>> Properties property, String interpreterSettingId) {
>> int connectTimeout =
>> conf.getInt(ConfVars.ZEPPELIN_INTERPRETER_CONNECT_TIMEOUT);
>> String localRepoPath = conf.getInterpreterLocalRepoPath() + "/" +
>> interpreterSettingId;
>> int maxPoolSize = conf.getInt(ConfVars.ZEPPELIN_INTERPRETER_MAX_POOL_SIZE);
>>
>> RemoteInterpreter remoteInterpreter =
>> new RemoteInterpreter(property, noteId, className,
>> conf.getInterpreterRemoteRunnerPath(),
>> interpreterPath, localRepoPath, connectTimeout, maxPoolSize,
>> remoteInterpreterProcessListener, appEventListener);
>> remoteInterpreter.addEnv(env);
>>
>> return new LazyOpenInterpreter(remoteInterpreter);
>> }
>>
>>
>> The code will call the bin/intepreter.sh (or interpreter.cmd if on
>> Windows) to launch a new JVM for the interpreter.
>>
>> And depending on the type of interpreter, the command will be different:
>>
>> if [[ -n "${SPARK_SUBMIT}" ]]; then
>> ${SPARK_SUBMIT} --class ${ZEPPELIN_SERVER} --driver-class-path
>> "${ZEPPELIN_INTP_CLASSPATH_OVERRIDES}:${CLASSPATH}"
>> --driver-java-options "${JAVA_INTP_OPTS}" ${SPARK_SUBMIT_OPTIONS}
>> ${SPARK_APP_JAR} ${PORT} &
>> else
>> ${ZEPPELIN_RUNNER} ${JAVA_INTP_OPTS} ${ZEPPELIN_INTP_MEM} -cp
>> ${ZEPPELIN_INTP_CLASSPATH_OVERRIDES}:${CLASSPATH} ${ZEPPELIN_SERVER}
>> ${PORT} &
>> fi
>>
>> There is a special case for Spark because we execute a SparkSubmit.
>>
>> This detail is important for contributors because when they need to DEBUG
>> an interpreter code, they have to modify the interpreter.sh file to add an
>> "-Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=xxx"
>>
>> Most of people naively add this debug argument to the zeppelin.sh file
>> but it only debugs the Zeppelin server itself, not the interpreter.
>>
>> Do you think we may create a page in the official doc for this ?
>>
>>
>> On Tue, Oct 4, 2016 at 9:00 AM, Jongyoul Lee <[email protected]> wrote:
>>
>>> Hello devs and users,
>>>
>>> I've submitted a post into Apache Zeppelin Stories on medium.com:
>>> https://medium.com/apache-zeppelin-stories/how-a
>>> pache-zeppelin-runs-a-paragraph-783a0a612ba9#.x9ym8fovv
>>>
>>> I hope all users and contributors understand Apache Zeppelin deeply.
>>>
>>> Regards,
>>> Jongyoul
>>>
>>> --
>>> 이종열, Jongyoul Lee, 李宗烈
>>> http://madeng.net
>>>
>>
>>
>
>
> --
> 이종열, Jongyoul Lee, 李宗烈
> http://madeng.net
>
--
Best Regards
Jeff Zhang
