What is the expected day for v0.6?
From: moon soo Lee <leemoon...@gmail.com> To: users@zeppelin.apache.org Date: 2016/06/30 11:36 AM Subject: Re: spark interpreter Hi Ben, Livy interpreter is included in 0.6.0. If it is not listed when you create interpreter setting, could you check if your 'zeppelin.interpreters' property list Livy interpreter classes? (conf/zeppelin-site.xml) Thanks, moon On Wed, Jun 29, 2016 at 11:52 AM Benjamin Kim <bbuil...@gmail.com> wrote: On a side note… Has anyone got the Livy interpreter to be added as an interpreter in the latest build of Zeppelin 0.6.0? By the way, I have Shiro authentication on. Could this interfere? Thanks, Ben On Jun 29, 2016, at 11:18 AM, moon soo Lee <m...@apache.org> wrote: Livy interpreter internally creates multiple sessions for each user, independently from 3 binding modes supported in Zeppelin. Therefore, 'shared' mode, Livy interpreter will create sessions per each user, 'scoped' or 'isolated' mode will result create sessions per notebook, per user. Notebook is shared among users, they always use the same interpreter instance/process, for now. I think supporting per user interpreter instance/process would be future work. Thanks, moon On Wed, Jun 29, 2016 at 7:57 AM Chen Song <chen.song...@gmail.com> wrote: Thanks for your explanation, Moon. Following up on this, I can see the difference in terms of single or multiple interpreter processes. With respect to spark drivers, since each interpreter spawns a separate Spark driver in regular Spark interpreter setting, it is clear to me the different implications of the 3 binding modes. However, when it comes to Livy server with impersonation turned on, I am a bit confused. Will Livy interpreter always create a new Spark driver (along with a Spark Context instance) for each user session, regardless of the binding mode of Livy interpreter? I am not very familiar with Livy, but from what I could tell, I see no difference between different binding modes for Livy on as far as how Spark drivers are concerned. Last question, when a notebook is shared among users, will they always use the same interpreter instance/process already created? Thanks Chen On Fri, Jun 24, 2016 at 11:51 AM moon soo Lee <m...@apache.org> wrote: Hi, Thanks for asking question. It's not dumb question at all, Zeppelin docs does not explain very well. Spark Interpreter, 'shared' mode, a spark interpreter setting spawn a interpreter process to serve all notebooks which binded to this interpreter setting. 'scoped' mode, a spark interpreter setting spawn multiple interpreter processes per notebook which binded to this interpreter setting. Using Livy interpreter, Zeppelin propagate current user information to Livy interpreter. And Livy interpreter creates different session per user via Livy Server. Hope this helps. Thanks, moon On Tue, Jun 21, 2016 at 6:41 PM Chen Song <chen.song...@gmail.com> wrote: Zeppelin provides 3 binding modes for each interpreter. With `scoped` or `shared` Spark interpreter, every user share the same SparkContext. Sorry for the dumb question, how does it differ from Spark via Ivy Server? -- Chen Song
