Re: Query about the high availability of Zeppelin
gt;>>>>>> Here is what I think should be part of HA consideration: >>>>>>> >>>>>>>1. Have multiple Zeppelin Instances >>>>>>>2. Have the notebooks storage backed by something like an NFS so >>>>>>>all notebooks are visible across all Zeppelin instances >>>>>>>3. Put multiple load balancers infront of Zeppelin to route >>>>>>>requests. >>>>>>> >>>>>>> Consider that HA needs scalability, which depends on which >>>>>>> interpreter you plan to use. So you might need to consider HA at both >>>>>>> Zeppelin and interpreter level. For example if you were using Z + Livy + >>>>>>> Spark, you will need to consider scalability + HA needs of Z + Livy >>>>>>> interpreter + Livy Server + Spark (on Cluster manager). >>>>>>> >>>>>>> On Thu, Jun 29, 2017 at 10:04 PM, wenxing zheng < >>>>>>> wenxing.zh...@gmail.com> wrote: >>>>>>> >>>>>>>> and do we have any architecture doc for reference? Because we need >>>>>>>> to add the HA capability as soon as possible, hope we can figure it >>>>>>>> out. >>>>>>>> >>>>>>>> On Fri, Jun 30, 2017 at 12:33 PM, wenxing zheng < >>>>>>>> wenxing.zh...@gmail.com> wrote: >>>>>>>> >>>>>>>>> Thanks to Jeff and Moon. >>>>>>>>> >>>>>>>>> So currently the active-active model doesn't work on GIT storage, >>>>>>>>> am I right? >>>>>>>>> >>>>>>>>> On Fri, Jun 30, 2017 at 12:16 PM, moon soo Lee <m...@apache.org> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> Basically active-(hot)standby model would work. >>>>>>>>>> Two or more Zeppelin instance can be started and pointing the >>>>>>>>>> same notebook storage, if only one Zeppelin instance (active) change >>>>>>>>>> notebook at any given time. >>>>>>>>>> >>>>>>>>>> In case of the active instance fails, one of rest instance can >>>>>>>>>> take over the role by refreshing notebook list and start make change. >>>>>>>>>> >>>>>>>>>> But all these fail over is not provided by Zeppelin and need to >>>>>>>>>> depends on external script or HA software (like Heartbeat). >>>>>>>>>> >>>>>>>>>> Like Jeff mentioned, community does not have concrete plan for >>>>>>>>>> having HA built-in at this moment. >>>>>>>>>> >>>>>>>>>> Hope this helps, >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> moon >>>>>>>>>> >>>>>>>>>> On Fri, Jun 30, 2017 at 1:01 PM Jeff Zhang <zjf...@gmail.com> >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> No concrete plan for that. There're other higher priority things >>>>>>>>>>> need to be done. At least it would not be available in 0.8, maybe >>>>>>>>>>> after 1.0 >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> wenxing zheng <wenxing.zh...@gmail.com>于2017年6月30日周五 上午11:47写道： >>>>>>>>>>> >>>>>>>>>>>> Thanks to Jianfeng. >>>>>>>>>>>> >>>>>>>>>>>> Do you know any plan on this? >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On Fri, Jun 30, 2017 at 11:32 AM, Jianfeng (Jeff) Zhang < >>>>>>>>>>>> jzh...@hortonworks.com> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> HA is not supported, there’s still lots of configuration >>>>>>>>>>>>> files stored in local file system. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Best Regard, >>>>>>>>>>>>> Jeff Zhang >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> From: wenxing zheng <wenxing.zh...@gmail.com> >>>>>>>>>>>>> Reply-To: "users@zeppelin.apache.org" < >>>>>>>>>>>>> users@zeppelin.apache.org> >>>>>>>>>>>>> Date: Friday, June 30, 2017 at 9:40 AM >>>>>>>>>>>>> To: "users@zeppelin.apache.org" <users@zeppelin.apache.org> >>>>>>>>>>>>> Subject: Query about the high availability of Zeppelin >>>>>>>>>>>>> >>>>>>>>>>>>> Hello all, >>>>>>>>>>>>> >>>>>>>>>>>>> I still didn't find any docs on this topic? Appreciated if >>>>>>>>>>>>> anyone can shed some lights on how to get the Zeppelin into a >>>>>>>>>>>>> cluster with >>>>>>>>>>>>> shared/centralized storage >>>>>>>>>>>>> >>>>>>>>>>>>> Regards, Wenxing >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>
Re: Query about the high availability of Zeppelin
Basically Zeppelin HA require 2 major things: 1. Shared storage (storage for notebook, interpreter setting, zeppelin-site.xml, zeppelin-env.sh, shiro.ini, credentials.json) 2. Recover running interpreter.The standby zeppelin instance don't know where are the running interpreter (host:port), so it can not recover the running interpreters when standby zeppelin become active. Maybe we can store the runtime info in zookeeper, anyway it requires more design and discussion. Ruslan Dautkhanov <dautkha...@gmail.com>于2017年7月1日周六 上午8:07写道： > I think if you have a shared storage for notebooks (for example, NFS > mounted from a third server), > and a load-balancer that supports sticky sessions (like F5) on top, it > should be possible to have HA without > any code change in Zeppelin. Am I missing something? > > > > -- > Ruslan Dautkhanov > > On Fri, Jun 30, 2017 at 5:54 PM, Alexander Filipchik <afilipc...@gmail.com > > wrote: > >> Honestly, HA requires more than just active stand by. >> It should be able to scale without major surgeries, which is not possible >> right now. For example, if you start too many interpreters, zeppelin box >> will simply run out of memory. >> >> Alex >> >> On Thu, Jun 29, 2017 at 10:59 PM, wenxing zheng <wenxing.zh...@gmail.com> >> wrote: >> >>> at first, I would think GIT storage is a good option and we can push and >>> pull the changes regularly. >>> >>> With multiple zeppelin instances, maybe we need a new component or >>> service to act as a distributed scheduler: dispatch the Job to and manage >>> the Jobs on the Zeppelin instances. >>> >>> On Fri, Jun 30, 2017 at 1:26 PM, Vinay Shukla <vinayshu...@gmail.com> >>> wrote: >>> >>>> Here is what I think should be part of HA consideration: >>>> >>>>1. Have multiple Zeppelin Instances >>>>2. Have the notebooks storage backed by something like an NFS so >>>>all notebooks are visible across all Zeppelin instances >>>>3. Put multiple load balancers infront of Zeppelin to route >>>>requests. >>>> >>>> Consider that HA needs scalability, which depends on which interpreter >>>> you plan to use. So you might need to consider HA at both Zeppelin and >>>> interpreter level. For example if you were using Z + Livy + Spark, you will >>>> need to consider scalability + HA needs of Z + Livy interpreter + Livy >>>> Server + Spark (on Cluster manager). >>>> >>>> On Thu, Jun 29, 2017 at 10:04 PM, wenxing zheng < >>>> wenxing.zh...@gmail.com> wrote: >>>> >>>>> and do we have any architecture doc for reference? Because we need to >>>>> add the HA capability as soon as possible, hope we can figure it out. >>>>> >>>>> On Fri, Jun 30, 2017 at 12:33 PM, wenxing zheng < >>>>> wenxing.zh...@gmail.com> wrote: >>>>> >>>>>> Thanks to Jeff and Moon. >>>>>> >>>>>> So currently the active-active model doesn't work on GIT storage, am >>>>>> I right? >>>>>> >>>>>> On Fri, Jun 30, 2017 at 12:16 PM, moon soo Lee <m...@apache.org> >>>>>> wrote: >>>>>> >>>>>>> Basically active-(hot)standby model would work. >>>>>>> Two or more Zeppelin instance can be started and pointing the same >>>>>>> notebook storage, if only one Zeppelin instance (active) change >>>>>>> notebook at >>>>>>> any given time. >>>>>>> >>>>>>> In case of the active instance fails, one of rest instance can take >>>>>>> over the role by refreshing notebook list and start make change. >>>>>>> >>>>>>> But all these fail over is not provided by Zeppelin and need to >>>>>>> depends on external script or HA software (like Heartbeat). >>>>>>> >>>>>>> Like Jeff mentioned, community does not have concrete plan for >>>>>>> having HA built-in at this moment. >>>>>>> >>>>>>> Hope this helps, >>>>>>> >>>>>>> Thanks, >>>>>>> moon >>>>>>> >>>>>>> On Fri, Jun 30, 2017 at 1:01 PM Jeff Zhang <zjf...@gmail.com> wrote: >>>>>>> >>>>>>>> >>>>>&
Re: Query about the high availability of Zeppelin
I think if you have a shared storage for notebooks (for example, NFS mounted from a third server), and a load-balancer that supports sticky sessions (like F5) on top, it should be possible to have HA without any code change in Zeppelin. Am I missing something? -- Ruslan Dautkhanov On Fri, Jun 30, 2017 at 5:54 PM, Alexander Filipchik <afilipc...@gmail.com> wrote: > Honestly, HA requires more than just active stand by. > It should be able to scale without major surgeries, which is not possible > right now. For example, if you start too many interpreters, zeppelin box > will simply run out of memory. > > Alex > > On Thu, Jun 29, 2017 at 10:59 PM, wenxing zheng <wenxing.zh...@gmail.com> > wrote: > >> at first, I would think GIT storage is a good option and we can push and >> pull the changes regularly. >> >> With multiple zeppelin instances, maybe we need a new component or >> service to act as a distributed scheduler: dispatch the Job to and manage >> the Jobs on the Zeppelin instances. >> >> On Fri, Jun 30, 2017 at 1:26 PM, Vinay Shukla <vinayshu...@gmail.com> >> wrote: >> >>> Here is what I think should be part of HA consideration: >>> >>>1. Have multiple Zeppelin Instances >>>2. Have the notebooks storage backed by something like an NFS so all >>>notebooks are visible across all Zeppelin instances >>>3. Put multiple load balancers infront of Zeppelin to route requests. >>> >>> Consider that HA needs scalability, which depends on which interpreter >>> you plan to use. So you might need to consider HA at both Zeppelin and >>> interpreter level. For example if you were using Z + Livy + Spark, you will >>> need to consider scalability + HA needs of Z + Livy interpreter + Livy >>> Server + Spark (on Cluster manager). >>> >>> On Thu, Jun 29, 2017 at 10:04 PM, wenxing zheng <wenxing.zh...@gmail.com >>> > wrote: >>> >>>> and do we have any architecture doc for reference? Because we need to >>>> add the HA capability as soon as possible, hope we can figure it out. >>>> >>>> On Fri, Jun 30, 2017 at 12:33 PM, wenxing zheng < >>>> wenxing.zh...@gmail.com> wrote: >>>> >>>>> Thanks to Jeff and Moon. >>>>> >>>>> So currently the active-active model doesn't work on GIT storage, am I >>>>> right? >>>>> >>>>> On Fri, Jun 30, 2017 at 12:16 PM, moon soo Lee <m...@apache.org> >>>>> wrote: >>>>> >>>>>> Basically active-(hot)standby model would work. >>>>>> Two or more Zeppelin instance can be started and pointing the same >>>>>> notebook storage, if only one Zeppelin instance (active) change notebook >>>>>> at >>>>>> any given time. >>>>>> >>>>>> In case of the active instance fails, one of rest instance can take >>>>>> over the role by refreshing notebook list and start make change. >>>>>> >>>>>> But all these fail over is not provided by Zeppelin and need to >>>>>> depends on external script or HA software (like Heartbeat). >>>>>> >>>>>> Like Jeff mentioned, community does not have concrete plan for having >>>>>> HA built-in at this moment. >>>>>> >>>>>> Hope this helps, >>>>>> >>>>>> Thanks, >>>>>> moon >>>>>> >>>>>> On Fri, Jun 30, 2017 at 1:01 PM Jeff Zhang <zjf...@gmail.com> wrote: >>>>>> >>>>>>> >>>>>>> No concrete plan for that. There're other higher priority things >>>>>>> need to be done. At least it would not be available in 0.8, maybe after >>>>>>> 1.0 >>>>>>> >>>>>>> >>>>>>> wenxing zheng <wenxing.zh...@gmail.com>于2017年6月30日周五 上午11:47写道： >>>>>>> >>>>>>>> Thanks to Jianfeng. >>>>>>>> >>>>>>>> Do you know any plan on this? >>>>>>>> >>>>>>>> >>>>>>>> On Fri, Jun 30, 2017 at 11:32 AM, Jianfeng (Jeff) Zhang < >>>>>>>> jzh...@hortonworks.com> wrote: >>>>>>>> >>>>>>>>> HA is not supported, there’s still lots of configuration files >>>>>>>>> stored in local file system. >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> Best Regard, >>>>>>>>> Jeff Zhang >>>>>>>>> >>>>>>>>> >>>>>>>>> From: wenxing zheng <wenxing.zh...@gmail.com> >>>>>>>>> Reply-To: "users@zeppelin.apache.org" <users@zeppelin.apache.org> >>>>>>>>> Date: Friday, June 30, 2017 at 9:40 AM >>>>>>>>> To: "users@zeppelin.apache.org" <users@zeppelin.apache.org> >>>>>>>>> Subject: Query about the high availability of Zeppelin >>>>>>>>> >>>>>>>>> Hello all, >>>>>>>>> >>>>>>>>> I still didn't find any docs on this topic? Appreciated if anyone >>>>>>>>> can shed some lights on how to get the Zeppelin into a cluster with >>>>>>>>> shared/centralized storage >>>>>>>>> >>>>>>>>> Regards, Wenxing >>>>>>>>> >>>>>>>> >>>>>>>> >>>>> >>>> >>> >> >
Re: Query about the high availability of Zeppelin
Honestly, HA requires more than just active stand by. It should be able to scale without major surgeries, which is not possible right now. For example, if you start too many interpreters, zeppelin box will simply run out of memory. Alex On Thu, Jun 29, 2017 at 10:59 PM, wenxing zheng <wenxing.zh...@gmail.com> wrote: > at first, I would think GIT storage is a good option and we can push and > pull the changes regularly. > > With multiple zeppelin instances, maybe we need a new component or service > to act as a distributed scheduler: dispatch the Job to and manage the Jobs > on the Zeppelin instances. > > On Fri, Jun 30, 2017 at 1:26 PM, Vinay Shukla <vinayshu...@gmail.com> > wrote: > >> Here is what I think should be part of HA consideration: >> >>1. Have multiple Zeppelin Instances >>2. Have the notebooks storage backed by something like an NFS so all >>notebooks are visible across all Zeppelin instances >>3. Put multiple load balancers infront of Zeppelin to route requests. >> >> Consider that HA needs scalability, which depends on which interpreter >> you plan to use. So you might need to consider HA at both Zeppelin and >> interpreter level. For example if you were using Z + Livy + Spark, you will >> need to consider scalability + HA needs of Z + Livy interpreter + Livy >> Server + Spark (on Cluster manager). >> >> On Thu, Jun 29, 2017 at 10:04 PM, wenxing zheng <wenxing.zh...@gmail.com> >> wrote: >> >>> and do we have any architecture doc for reference? Because we need to >>> add the HA capability as soon as possible, hope we can figure it out. >>> >>> On Fri, Jun 30, 2017 at 12:33 PM, wenxing zheng <wenxing.zh...@gmail.com >>> > wrote: >>> >>>> Thanks to Jeff and Moon. >>>> >>>> So currently the active-active model doesn't work on GIT storage, am I >>>> right? >>>> >>>> On Fri, Jun 30, 2017 at 12:16 PM, moon soo Lee <m...@apache.org> wrote: >>>> >>>>> Basically active-(hot)standby model would work. >>>>> Two or more Zeppelin instance can be started and pointing the same >>>>> notebook storage, if only one Zeppelin instance (active) change notebook >>>>> at >>>>> any given time. >>>>> >>>>> In case of the active instance fails, one of rest instance can take >>>>> over the role by refreshing notebook list and start make change. >>>>> >>>>> But all these fail over is not provided by Zeppelin and need to >>>>> depends on external script or HA software (like Heartbeat). >>>>> >>>>> Like Jeff mentioned, community does not have concrete plan for having >>>>> HA built-in at this moment. >>>>> >>>>> Hope this helps, >>>>> >>>>> Thanks, >>>>> moon >>>>> >>>>> On Fri, Jun 30, 2017 at 1:01 PM Jeff Zhang <zjf...@gmail.com> wrote: >>>>> >>>>>> >>>>>> No concrete plan for that. There're other higher priority things need >>>>>> to be done. At least it would not be available in 0.8, maybe after 1.0 >>>>>> >>>>>> >>>>>> wenxing zheng <wenxing.zh...@gmail.com>于2017年6月30日周五 上午11:47写道： >>>>>> >>>>>>> Thanks to Jianfeng. >>>>>>> >>>>>>> Do you know any plan on this? >>>>>>> >>>>>>> >>>>>>> On Fri, Jun 30, 2017 at 11:32 AM, Jianfeng (Jeff) Zhang < >>>>>>> jzh...@hortonworks.com> wrote: >>>>>>> >>>>>>>> HA is not supported, there’s still lots of configuration files >>>>>>>> stored in local file system. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Best Regard, >>>>>>>> Jeff Zhang >>>>>>>> >>>>>>>> >>>>>>>> From: wenxing zheng <wenxing.zh...@gmail.com> >>>>>>>> Reply-To: "users@zeppelin.apache.org" <users@zeppelin.apache.org> >>>>>>>> Date: Friday, June 30, 2017 at 9:40 AM >>>>>>>> To: "users@zeppelin.apache.org" <users@zeppelin.apache.org> >>>>>>>> Subject: Query about the high availability of Zeppelin >>>>>>>> >>>>>>>> Hello all, >>>>>>>> >>>>>>>> I still didn't find any docs on this topic? Appreciated if anyone >>>>>>>> can shed some lights on how to get the Zeppelin into a cluster with >>>>>>>> shared/centralized storage >>>>>>>> >>>>>>>> Regards, Wenxing >>>>>>>> >>>>>>> >>>>>>> >>>> >>> >> >
Re: Query about the high availability of Zeppelin
at first, I would think GIT storage is a good option and we can push and pull the changes regularly. With multiple zeppelin instances, maybe we need a new component or service to act as a distributed scheduler: dispatch the Job to and manage the Jobs on the Zeppelin instances. On Fri, Jun 30, 2017 at 1:26 PM, Vinay Shukla <vinayshu...@gmail.com> wrote: > Here is what I think should be part of HA consideration: > >1. Have multiple Zeppelin Instances >2. Have the notebooks storage backed by something like an NFS so all >notebooks are visible across all Zeppelin instances >3. Put multiple load balancers infront of Zeppelin to route requests. > > Consider that HA needs scalability, which depends on which interpreter you > plan to use. So you might need to consider HA at both Zeppelin and > interpreter level. For example if you were using Z + Livy + Spark, you will > need to consider scalability + HA needs of Z + Livy interpreter + Livy > Server + Spark (on Cluster manager). > > On Thu, Jun 29, 2017 at 10:04 PM, wenxing zheng <wenxing.zh...@gmail.com> > wrote: > >> and do we have any architecture doc for reference? Because we need to add >> the HA capability as soon as possible, hope we can figure it out. >> >> On Fri, Jun 30, 2017 at 12:33 PM, wenxing zheng <wenxing.zh...@gmail.com> >> wrote: >> >>> Thanks to Jeff and Moon. >>> >>> So currently the active-active model doesn't work on GIT storage, am I >>> right? >>> >>> On Fri, Jun 30, 2017 at 12:16 PM, moon soo Lee <m...@apache.org> wrote: >>> >>>> Basically active-(hot)standby model would work. >>>> Two or more Zeppelin instance can be started and pointing the same >>>> notebook storage, if only one Zeppelin instance (active) change notebook at >>>> any given time. >>>> >>>> In case of the active instance fails, one of rest instance can take >>>> over the role by refreshing notebook list and start make change. >>>> >>>> But all these fail over is not provided by Zeppelin and need to depends >>>> on external script or HA software (like Heartbeat). >>>> >>>> Like Jeff mentioned, community does not have concrete plan for having >>>> HA built-in at this moment. >>>> >>>> Hope this helps, >>>> >>>> Thanks, >>>> moon >>>> >>>> On Fri, Jun 30, 2017 at 1:01 PM Jeff Zhang <zjf...@gmail.com> wrote: >>>> >>>>> >>>>> No concrete plan for that. There're other higher priority things need >>>>> to be done. At least it would not be available in 0.8, maybe after 1.0 >>>>> >>>>> >>>>> wenxing zheng <wenxing.zh...@gmail.com>于2017年6月30日周五 上午11:47写道： >>>>> >>>>>> Thanks to Jianfeng. >>>>>> >>>>>> Do you know any plan on this? >>>>>> >>>>>> >>>>>> On Fri, Jun 30, 2017 at 11:32 AM, Jianfeng (Jeff) Zhang < >>>>>> jzh...@hortonworks.com> wrote: >>>>>> >>>>>>> HA is not supported, there’s still lots of configuration files >>>>>>> stored in local file system. >>>>>>> >>>>>>> >>>>>>> >>>>>>> Best Regard, >>>>>>> Jeff Zhang >>>>>>> >>>>>>> >>>>>>> From: wenxing zheng <wenxing.zh...@gmail.com> >>>>>>> Reply-To: "users@zeppelin.apache.org" <users@zeppelin.apache.org> >>>>>>> Date: Friday, June 30, 2017 at 9:40 AM >>>>>>> To: "users@zeppelin.apache.org" <users@zeppelin.apache.org> >>>>>>> Subject: Query about the high availability of Zeppelin >>>>>>> >>>>>>> Hello all, >>>>>>> >>>>>>> I still didn't find any docs on this topic? Appreciated if anyone >>>>>>> can shed some lights on how to get the Zeppelin into a cluster with >>>>>>> shared/centralized storage >>>>>>> >>>>>>> Regards, Wenxing >>>>>>> >>>>>> >>>>>> >>> >> >
Re: Query about the high availability of Zeppelin
and do we have any architecture doc for reference? Because we need to add the HA capability as soon as possible, hope we can figure it out. On Fri, Jun 30, 2017 at 12:33 PM, wenxing zheng <wenxing.zh...@gmail.com> wrote: > Thanks to Jeff and Moon. > > So currently the active-active model doesn't work on GIT storage, am I > right? > > On Fri, Jun 30, 2017 at 12:16 PM, moon soo Lee <m...@apache.org> wrote: > >> Basically active-(hot)standby model would work. >> Two or more Zeppelin instance can be started and pointing the same >> notebook storage, if only one Zeppelin instance (active) change notebook at >> any given time. >> >> In case of the active instance fails, one of rest instance can take over >> the role by refreshing notebook list and start make change. >> >> But all these fail over is not provided by Zeppelin and need to depends >> on external script or HA software (like Heartbeat). >> >> Like Jeff mentioned, community does not have concrete plan for having HA >> built-in at this moment. >> >> Hope this helps, >> >> Thanks, >> moon >> >> On Fri, Jun 30, 2017 at 1:01 PM Jeff Zhang <zjf...@gmail.com> wrote: >> >>> >>> No concrete plan for that. There're other higher priority things need to >>> be done. At least it would not be available in 0.8, maybe after 1.0 >>> >>> >>> wenxing zheng <wenxing.zh...@gmail.com>于2017年6月30日周五 上午11:47写道： >>> >>>> Thanks to Jianfeng. >>>> >>>> Do you know any plan on this? >>>> >>>> >>>> On Fri, Jun 30, 2017 at 11:32 AM, Jianfeng (Jeff) Zhang < >>>> jzh...@hortonworks.com> wrote: >>>> >>>>> HA is not supported, there’s still lots of configuration files stored >>>>> in local file system. >>>>> >>>>> >>>>> >>>>> Best Regard, >>>>> Jeff Zhang >>>>> >>>>> >>>>> From: wenxing zheng <wenxing.zh...@gmail.com> >>>>> Reply-To: "users@zeppelin.apache.org" <users@zeppelin.apache.org> >>>>> Date: Friday, June 30, 2017 at 9:40 AM >>>>> To: "users@zeppelin.apache.org" <users@zeppelin.apache.org> >>>>> Subject: Query about the high availability of Zeppelin >>>>> >>>>> Hello all, >>>>> >>>>> I still didn't find any docs on this topic? Appreciated if anyone can >>>>> shed some lights on how to get the Zeppelin into a cluster with >>>>> shared/centralized storage >>>>> >>>>> Regards, Wenxing >>>>> >>>> >>>> >
Re: Query about the high availability of Zeppelin
Thanks to Jeff and Moon. So currently the active-active model doesn't work on GIT storage, am I right? On Fri, Jun 30, 2017 at 12:16 PM, moon soo Lee <m...@apache.org> wrote: > Basically active-(hot)standby model would work. > Two or more Zeppelin instance can be started and pointing the same > notebook storage, if only one Zeppelin instance (active) change notebook at > any given time. > > In case of the active instance fails, one of rest instance can take over > the role by refreshing notebook list and start make change. > > But all these fail over is not provided by Zeppelin and need to depends on > external script or HA software (like Heartbeat). > > Like Jeff mentioned, community does not have concrete plan for having HA > built-in at this moment. > > Hope this helps, > > Thanks, > moon > > On Fri, Jun 30, 2017 at 1:01 PM Jeff Zhang <zjf...@gmail.com> wrote: > >> >> No concrete plan for that. There're other higher priority things need to >> be done. At least it would not be available in 0.8, maybe after 1.0 >> >> >> wenxing zheng <wenxing.zh...@gmail.com>于2017年6月30日周五 上午11:47写道： >> >>> Thanks to Jianfeng. >>> >>> Do you know any plan on this? >>> >>> >>> On Fri, Jun 30, 2017 at 11:32 AM, Jianfeng (Jeff) Zhang < >>> jzh...@hortonworks.com> wrote: >>> >>>> HA is not supported, there’s still lots of configuration files stored >>>> in local file system. >>>> >>>> >>>> >>>> Best Regard, >>>> Jeff Zhang >>>> >>>> >>>> From: wenxing zheng <wenxing.zh...@gmail.com> >>>> Reply-To: "users@zeppelin.apache.org" <users@zeppelin.apache.org> >>>> Date: Friday, June 30, 2017 at 9:40 AM >>>> To: "users@zeppelin.apache.org" <users@zeppelin.apache.org> >>>> Subject: Query about the high availability of Zeppelin >>>> >>>> Hello all, >>>> >>>> I still didn't find any docs on this topic? Appreciated if anyone can >>>> shed some lights on how to get the Zeppelin into a cluster with >>>> shared/centralized storage >>>> >>>> Regards, Wenxing >>>> >>> >>>
Re: Query about the high availability of Zeppelin
Basically active-(hot)standby model would work. Two or more Zeppelin instance can be started and pointing the same notebook storage, if only one Zeppelin instance (active) change notebook at any given time. In case of the active instance fails, one of rest instance can take over the role by refreshing notebook list and start make change. But all these fail over is not provided by Zeppelin and need to depends on external script or HA software (like Heartbeat). Like Jeff mentioned, community does not have concrete plan for having HA built-in at this moment. Hope this helps, Thanks, moon On Fri, Jun 30, 2017 at 1:01 PM Jeff Zhang <zjf...@gmail.com> wrote: > > No concrete plan for that. There're other higher priority things need to > be done. At least it would not be available in 0.8, maybe after 1.0 > > > wenxing zheng <wenxing.zh...@gmail.com>于2017年6月30日周五 上午11:47写道： > >> Thanks to Jianfeng. >> >> Do you know any plan on this? >> >> >> On Fri, Jun 30, 2017 at 11:32 AM, Jianfeng (Jeff) Zhang < >> jzh...@hortonworks.com> wrote: >> >>> HA is not supported, there’s still lots of configuration files stored >>> in local file system. >>> >>> >>> >>> Best Regard, >>> Jeff Zhang >>> >>> >>> From: wenxing zheng <wenxing.zh...@gmail.com> >>> Reply-To: "users@zeppelin.apache.org" <users@zeppelin.apache.org> >>> Date: Friday, June 30, 2017 at 9:40 AM >>> To: "users@zeppelin.apache.org" <users@zeppelin.apache.org> >>> Subject: Query about the high availability of Zeppelin >>> >>> Hello all, >>> >>> I still didn't find any docs on this topic? Appreciated if anyone can >>> shed some lights on how to get the Zeppelin into a cluster with >>> shared/centralized storage >>> >>> Regards, Wenxing >>> >> >>
Re: Query about the high availability of Zeppelin
No concrete plan for that. There're other higher priority things need to be done. At least it would not be available in 0.8, maybe after 1.0 wenxing zheng <wenxing.zh...@gmail.com>于2017年6月30日周五 上午11:47写道： > Thanks to Jianfeng. > > Do you know any plan on this? > > > On Fri, Jun 30, 2017 at 11:32 AM, Jianfeng (Jeff) Zhang < > jzh...@hortonworks.com> wrote: > >> HA is not supported, there’s still lots of configuration files stored in >> local file system. >> >> >> >> Best Regard, >> Jeff Zhang >> >> >> From: wenxing zheng <wenxing.zh...@gmail.com> >> Reply-To: "users@zeppelin.apache.org" <users@zeppelin.apache.org> >> Date: Friday, June 30, 2017 at 9:40 AM >> To: "users@zeppelin.apache.org" <users@zeppelin.apache.org> >> Subject: Query about the high availability of Zeppelin >> >> Hello all, >> >> I still didn't find any docs on this topic? Appreciated if anyone can >> shed some lights on how to get the Zeppelin into a cluster with >> shared/centralized storage >> >> Regards, Wenxing >> > >
Query about the high availability of Zeppelin
Hello all, I still didn't find any docs on this topic? Appreciated if anyone can shed some lights on how to get the Zeppelin into a cluster with shared/centralized storage Regards, Wenxing
