You mean standalone service which runs Procedure V2 ?
Not sure how much work is involved.

Is this concerning the stability of Master where backup / restore
procedures run ?

To my understanding, errors in one procedure are isolated, not having
adverse impact on Master's stability.

On Thu, Sep 22, 2016 at 8:32 PM, 张铎 <palomino...@gmail.com> wrote:

> So what about a standalone service other than master? You can use your own
> procedure store in that service?
>
> 2016-09-23 11:28 GMT+08:00 Ted Yu <yuzhih...@gmail.com>:
>
> > An earlier implementation was client driven.
> >
> > But with that approach, it is hard to resume if there is error midway.
> > Using Procedure V2 makes the backup / restore more robust.
> >
> > Another consideration is for security. It is hard to enforce security (to
> > be implemented) for client driven actions.
> >
> > Cheers
> >
> > > On Sep 22, 2016, at 8:15 PM, Andrew Purtell <andrew.purt...@gmail.com>
> > wrote:
> > >
> > > No, this misses Matteo's finer point, which is "shelling out" from the
> > master directly to run MR is a first. Why not drive this with a utility
> > derived from Tool?
> > >
> > > On Sep 22, 2016, at 7:57 PM, Vladimir Rodionov <vladrodio...@gmail.com
> >
> > wrote:
> > >
> > >>>> In our production cluster,  it is a common case we just have HDFS
> and
> > >>>> HBase deployed.
> > >>>> If our Master/RS depend on MR framework (especially some features we
> > >>>> have not used at all),  it introduced another cost for maintain.  I
> > >>>> don't think it is a good idea.
> > >>
> > >> So , you are not backup users in this case. Many our customers have
> full
> > >> stack deployed and
> > >> want see backup to be a standard feature. Besides this, nothing will
> > happen
> > >> in your cluster
> > >> if you won't be doing backups.
> > >>
> > >> This discussion (we do not want see M/R dependency) goes to nowhere.
> We
> > >> asked already, at least twice, to suggest another framework (other
> than
> > M/R)
> > >> for bulk data copy with *conversion*. Still waiting for suggestions.
> > >>
> > >> -Vlad
> > >>
> > >>
> > >>
> > >>
> > >>> On Thu, Sep 22, 2016 at 7:49 PM, Ted Yu <yuzhih...@gmail.com> wrote:
> > >>>
> > >>> If MR framework is not deployed in the cluster, hbase still functions
> > >>> normally (post merge).
> > >>>
> > >>> In terms of build time dependency, we have long been depending on
> > >>> mapreduce. Take a look at ExportSnapshot.
> > >>>
> > >>> Cheers
> > >>>
> > >>> On Thu, Sep 22, 2016 at 7:42 PM, Heng Chen <heng.chen.1...@gmail.com
> >
> > >>> wrote:
> > >>>
> > >>>> In our production cluster,  it is a common case we just have HDFS
> and
> > >>>> HBase deployed.
> > >>>> If our Master/RS depend on MR framework (especially some features we
> > >>>> have not used at all),  it introduced another cost for maintain.  I
> > >>>> don't think it is a good idea.
> > >>>>
> > >>>> 2016-09-23 10:28 GMT+08:00 张铎 <palomino...@gmail.com>:
> > >>>>> To be specific, for example, our nice Backup/Restore feature, if we
> > >>> think
> > >>>>> this is not a core feature of HBase, then we could make it depend
> on
> > >>> MR,
> > >>>>> and start a standalone BackupManager instance that submits MR jobs
> to
> > >>> do
> > >>>>> periodical maintenance job. And if we think this is a core feature
> > that
> > >>>>> everyone should use it, then we'd better implement it without MR
> > >>>>> dependency, like DLS.
> > >>>>>
> > >>>>> Thanks.
> > >>>>>
> > >>>>> 2016-09-23 10:11 GMT+08:00 张铎 <palomino...@gmail.com>:
> > >>>>>
> > >>>>>> I‘m -1 on let master or rs launch MR jobs. It is OK that some of
> our
> > >>>>>> features depend on MR but I think the bottom line is that we
> should
> > >>>> launch
> > >>>>>> the jobs from outside manually or by other services.
> > >>>>>>
> > >>>>>> 2016-09-23 9:47 GMT+08:00 Andrew Purtell <
> andrew.purt...@gmail.com
> > >:
> > >>>>>>
> > >>>>>>> Ok, got it. Well "shelling out" is on the line I think, so a fair
> > >>>>>>> question.
> > >>>>>>>
> > >>>>>>> Can this be driven by a utility derived from Tool like our other
> MR
> > >>>> apps?
> > >>>>>>> The issue is needing the AccessController to decide if allowed?
> But
> > >>>> nothing
> > >>>>>>> prevents the user from running the job manually/independently,
> > right?
> > >>>>>>>
> > >>>>>>>> On Sep 22, 2016, at 3:44 PM, Matteo Bertozzi <
> > >>>> theo.berto...@gmail.com>
> > >>>>>>> wrote:
> > >>>>>>>>
> > >>>>>>>> just a remark. my query was not about tools using MR (everyone i
> > >>>> think
> > >>>>>>> is
> > >>>>>>>> ok with those).
> > >>>>>>>> the topic was about: "are we ok with running MR jobs from Master
> > >>> and
> > >>>> RSs
> > >>>>>>>> code?" since this will be the first time we do this
> > >>>>>>>>
> > >>>>>>>> Matteo
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>>> On Thu, Sep 22, 2016 at 2:49 PM, Devaraj Das <
> > >>> d...@hortonworks.com>
> > >>>>>>> wrote:
> > >>>>>>>>>
> > >>>>>>>>> Very much agree; for tools like ExportSnapshot / Backup /
> > Restore,
> > >>>> it's
> > >>>>>>>>> fine to be dependent on MR. MR is the right framework for such.
> > We
> > >>>>>>> should
> > >>>>>>>>> also do compactions using MR (just saying :) )
> > >>>>>>>>> ________________________________________
> > >>>>>>>>> From: Ted Yu <yuzhih...@gmail.com>
> > >>>>>>>>> Sent: Thursday, September 22, 2016 2:00 PM
> > >>>>>>>>> To: dev@hbase.apache.org
> > >>>>>>>>> Subject: Re: [DISCUSSION] MR jobs started by Master or RS
> > >>>>>>>>>
> > >>>>>>>>> I agree - backup / restore is in the same category as import /
> > >>>> export.
> > >>>>>>>>>
> > >>>>>>>>> On Thu, Sep 22, 2016 at 1:58 PM, Andrew Purtell <
> > >>>>>>> andrew.purt...@gmail.com>
> > >>>>>>>>> wrote:
> > >>>>>>>>>
> > >>>>>>>>>> Backup is extra tooling around core in my opinion. Like import
> > or
> > >>>>>>> export.
> > >>>>>>>>>> Or the optional MOB tool. It's fine.
> > >>>>>>>>>>
> > >>>>>>>>>>> On Sep 22, 2016, at 1:50 PM, Matteo Bertozzi <
> > >>>> mberto...@apache.org>
> > >>>>>>>>>> wrote:
> > >>>>>>>>>>>
> > >>>>>>>>>>> What's the latest opinion around running MR jobs from hbase
> > >>>> (Master
> > >>>>>>> or
> > >>>>>>>>>> RS)?
> > >>>>>>>>>>>
> > >>>>>>>>>>> I remember in the past that there was discussion about not
> > >>> having
> > >>>> MR
> > >>>>>>>>> has
> > >>>>>>>>>>> direct dependency of hbase.
> > >>>>>>>>>>>
> > >>>>>>>>>>> I think some of discussion where around MOB that had a MR job
> > to
> > >>>>>>>>> compact,
> > >>>>>>>>>>> that later was transformed in a non-MR job to be merged, I
> > think
> > >>>> we
> > >>>>>>>>> had a
> > >>>>>>>>>>> similar discussion for log split/replay.
> > >>>>>>>>>>>
> > >>>>>>>>>>> the latest is the new Backup feature (HBASE-7912), that runs
> a
> > >>> MR
> > >>>> job
> > >>>>>>>>>> from
> > >>>>>>>>>>> the master to copy data or restore data.
> > >>>>>>>>>>> (backup is also "not really core" as in.. if you don't use
> > >>> backup
> > >>>>>>>>> you'll
> > >>>>>>>>>>> not end up running MR jobs, but this was probably true for
> MOB
> > >>> as
> > >>>> in
> > >>>>>>>>> "if
> > >>>>>>>>>>> you don't enable MOB you don't need MR")
> > >>>>>>>>>>>
> > >>>>>>>>>>> any thoughts? do we a rule that says "we don't want to have
> > >>> hbase
> > >>>> run
> > >>>>>>>>> MR
> > >>>>>>>>>>> jobs, only tool started manually by the user can do that". or
> > >>> can
> > >>>> we
> > >>>>>>>>>> start
> > >>>>>>>>>>> adding MR calls around without problems?
> > >>>
> >
>

Reply via email to