Re: Query about the high availability of Zeppelin

[Alexander Filipchik]

Honestly,  HA requires more than just active stand by.
It should be able to scale without major surgeries, which is not possible
right now. For example, if you start too many interpreters, zeppelin box
will simply run out of memory.

Alex

On Thu, Jun 29, 2017 at 10:59 PM, wenxing zheng 
wrote:

> at first, I would think GIT storage is a good option and we can push and
> pull the changes regularly.
>
> With multiple zeppelin instances, maybe we need a new component or service
> to act as a distributed scheduler: dispatch the Job to and manage the Jobs
> on the Zeppelin instances.
>
> On Fri, Jun 30, 2017 at 1:26 PM, Vinay Shukla 
> wrote:
>
>> Here is what I think should be part of HA consideration:
>>
>>1. Have multiple Zeppelin Instances
>>2. Have the notebooks storage backed by something like an NFS so all
>>notebooks are visible across all Zeppelin instances
>>3. Put multiple load balancers infront of Zeppelin to route requests.
>>
>> Consider that HA needs scalability, which depends on which interpreter
>> you plan to use. So you might need to consider HA at both Zeppelin and
>> interpreter level. For example if you were using Z + Livy + Spark, you will
>> need to consider scalability + HA needs of Z + Livy interpreter + Livy
>> Server + Spark (on Cluster manager).
>>
>> On Thu, Jun 29, 2017 at 10:04 PM, wenxing zheng 
>> wrote:
>>
>>> and do we have any architecture doc for reference? Because we need to
>>> add the HA capability as soon as possible, hope we can figure it out.
>>>
>>> On Fri, Jun 30, 2017 at 12:33 PM, wenxing zheng >> > wrote:
>>>
 Thanks to Jeff and Moon.

 So currently the active-active model doesn't work on GIT storage, am I
 right?

 On Fri, Jun 30, 2017 at 12:16 PM, moon soo Lee  wrote:

> Basically active-(hot)standby model would work.
> Two or more Zeppelin instance can be started and pointing the same
> notebook storage, if only one Zeppelin instance (active) change notebook 
> at
> any given time.
>
> In case of the active instance fails, one of rest instance can take
> over the role by refreshing notebook list and start make change.
>
> But all these fail over is not provided by Zeppelin and need to
> depends on external script or HA software (like Heartbeat).
>
> Like Jeff mentioned, community does not have concrete plan for having
> HA built-in at this moment.
>
> Hope this helps,
>
> Thanks,
> moon
>
> On Fri, Jun 30, 2017 at 1:01 PM Jeff Zhang  wrote:
>
>>
>> No concrete plan for that. There're other higher priority things need
>> to be done. At least it would not be available in 0.8, maybe after 1.0
>>
>>
>> wenxing zheng 于2017年6月30日周五 上午11:47写道：
>>
>>> Thanks to Jianfeng.
>>>
>>> Do you  know any plan on this?
>>>
>>>
>>> On Fri, Jun 30, 2017 at 11:32 AM, Jianfeng (Jeff) Zhang <
>>> jzh...@hortonworks.com> wrote:
>>>
 HA is not supported, there’s still  lots of configuration files
 stored in local file system.



 Best Regard,
 Jeff Zhang


 From: wenxing zheng 
 Reply-To: "users@zeppelin.apache.org" 
 Date: Friday, June 30, 2017 at 9:40 AM
 To: "users@zeppelin.apache.org" 
 Subject: Query about the high availability of Zeppelin

 Hello all,

 I still didn't find any docs on this topic? Appreciated if anyone
 can shed some lights on how to get the Zeppelin into a cluster with
 shared/centralized storage

 Regards, Wenxing

>>>
>>>

>>>
>>
>

Re: Is there a way to close interpreter after inactivity

[Alexander Filipchik]

Nice! I hacked it together for 0.6 but porting it to 0.7 is challenging.



On Wed, Mar 1, 2017 at 12:18 PM, Ruslan Dautkhanov <dautkha...@gmail.com>
wrote:

> https://issues.apache.org/jira/browse/ZEPPELIN-2197
>
> This was created just yesterday :-)
>
>
> On Wed, Mar 1, 2017 at 12:54 PM Alexander Filipchik <afilipc...@gmail.com>
> wrote:
>
>> Hi,
>>
>> Is there any way to close an isolated interpreter after some timeout?
>> Let's say set an inactivity timeout of 30 mins (user input or job output)
>> and then return all the resources and close everything.
>>
>> Thank you,
>> Alex
>>
>

Is there a way to close interpreter after inactivity

[Alexander Filipchik]

Hi,

Is there any way to close an isolated interpreter after some timeout?
Let's say set an inactivity timeout of 30 mins (user input or job output)
and then return all the resources and close everything.

Thank you,
Alex