An earlier implementation was client driven. 

But with that approach, it is hard to resume if there is error midway. 
Using Procedure V2 makes the backup / restore more robust. 

Another consideration is for security. It is hard to enforce security (to be 
implemented) for client driven actions. 

Cheers

> On Sep 22, 2016, at 8:15 PM, Andrew Purtell <andrew.purt...@gmail.com> wrote:
> 
> No, this misses Matteo's finer point, which is "shelling out" from the master 
> directly to run MR is a first. Why not drive this with a utility derived from 
> Tool?
> 
> On Sep 22, 2016, at 7:57 PM, Vladimir Rodionov <vladrodio...@gmail.com> wrote:
> 
>>>> In our production cluster,  it is a common case we just have HDFS and
>>>> HBase deployed.
>>>> If our Master/RS depend on MR framework (especially some features we
>>>> have not used at all),  it introduced another cost for maintain.  I
>>>> don't think it is a good idea.
>> 
>> So , you are not backup users in this case. Many our customers have full
>> stack deployed and
>> want see backup to be a standard feature. Besides this, nothing will happen
>> in your cluster
>> if you won't be doing backups.
>> 
>> This discussion (we do not want see M/R dependency) goes to nowhere. We
>> asked already, at least twice, to suggest another framework (other than M/R)
>> for bulk data copy with *conversion*. Still waiting for suggestions.
>> 
>> -Vlad
>> 
>> 
>> 
>> 
>>> On Thu, Sep 22, 2016 at 7:49 PM, Ted Yu <yuzhih...@gmail.com> wrote:
>>> 
>>> If MR framework is not deployed in the cluster, hbase still functions
>>> normally (post merge).
>>> 
>>> In terms of build time dependency, we have long been depending on
>>> mapreduce. Take a look at ExportSnapshot.
>>> 
>>> Cheers
>>> 
>>> On Thu, Sep 22, 2016 at 7:42 PM, Heng Chen <heng.chen.1...@gmail.com>
>>> wrote:
>>> 
>>>> In our production cluster,  it is a common case we just have HDFS and
>>>> HBase deployed.
>>>> If our Master/RS depend on MR framework (especially some features we
>>>> have not used at all),  it introduced another cost for maintain.  I
>>>> don't think it is a good idea.
>>>> 
>>>> 2016-09-23 10:28 GMT+08:00 张铎 <palomino...@gmail.com>:
>>>>> To be specific, for example, our nice Backup/Restore feature, if we
>>> think
>>>>> this is not a core feature of HBase, then we could make it depend on
>>> MR,
>>>>> and start a standalone BackupManager instance that submits MR jobs to
>>> do
>>>>> periodical maintenance job. And if we think this is a core feature that
>>>>> everyone should use it, then we'd better implement it without MR
>>>>> dependency, like DLS.
>>>>> 
>>>>> Thanks.
>>>>> 
>>>>> 2016-09-23 10:11 GMT+08:00 张铎 <palomino...@gmail.com>:
>>>>> 
>>>>>> I‘m -1 on let master or rs launch MR jobs. It is OK that some of our
>>>>>> features depend on MR but I think the bottom line is that we should
>>>> launch
>>>>>> the jobs from outside manually or by other services.
>>>>>> 
>>>>>> 2016-09-23 9:47 GMT+08:00 Andrew Purtell <andrew.purt...@gmail.com>:
>>>>>> 
>>>>>>> Ok, got it. Well "shelling out" is on the line I think, so a fair
>>>>>>> question.
>>>>>>> 
>>>>>>> Can this be driven by a utility derived from Tool like our other MR
>>>> apps?
>>>>>>> The issue is needing the AccessController to decide if allowed? But
>>>> nothing
>>>>>>> prevents the user from running the job manually/independently, right?
>>>>>>> 
>>>>>>>> On Sep 22, 2016, at 3:44 PM, Matteo Bertozzi <
>>>> theo.berto...@gmail.com>
>>>>>>> wrote:
>>>>>>>> 
>>>>>>>> just a remark. my query was not about tools using MR (everyone i
>>>> think
>>>>>>> is
>>>>>>>> ok with those).
>>>>>>>> the topic was about: "are we ok with running MR jobs from Master
>>> and
>>>> RSs
>>>>>>>> code?" since this will be the first time we do this
>>>>>>>> 
>>>>>>>> Matteo
>>>>>>>> 
>>>>>>>> 
>>>>>>>>> On Thu, Sep 22, 2016 at 2:49 PM, Devaraj Das <
>>> d...@hortonworks.com>
>>>>>>> wrote:
>>>>>>>>> 
>>>>>>>>> Very much agree; for tools like ExportSnapshot / Backup / Restore,
>>>> it's
>>>>>>>>> fine to be dependent on MR. MR is the right framework for such. We
>>>>>>> should
>>>>>>>>> also do compactions using MR (just saying :) )
>>>>>>>>> ________________________________________
>>>>>>>>> From: Ted Yu <yuzhih...@gmail.com>
>>>>>>>>> Sent: Thursday, September 22, 2016 2:00 PM
>>>>>>>>> To: dev@hbase.apache.org
>>>>>>>>> Subject: Re: [DISCUSSION] MR jobs started by Master or RS
>>>>>>>>> 
>>>>>>>>> I agree - backup / restore is in the same category as import /
>>>> export.
>>>>>>>>> 
>>>>>>>>> On Thu, Sep 22, 2016 at 1:58 PM, Andrew Purtell <
>>>>>>> andrew.purt...@gmail.com>
>>>>>>>>> wrote:
>>>>>>>>> 
>>>>>>>>>> Backup is extra tooling around core in my opinion. Like import or
>>>>>>> export.
>>>>>>>>>> Or the optional MOB tool. It's fine.
>>>>>>>>>> 
>>>>>>>>>>> On Sep 22, 2016, at 1:50 PM, Matteo Bertozzi <
>>>> mberto...@apache.org>
>>>>>>>>>> wrote:
>>>>>>>>>>> 
>>>>>>>>>>> What's the latest opinion around running MR jobs from hbase
>>>> (Master
>>>>>>> or
>>>>>>>>>> RS)?
>>>>>>>>>>> 
>>>>>>>>>>> I remember in the past that there was discussion about not
>>> having
>>>> MR
>>>>>>>>> has
>>>>>>>>>>> direct dependency of hbase.
>>>>>>>>>>> 
>>>>>>>>>>> I think some of discussion where around MOB that had a MR job to
>>>>>>>>> compact,
>>>>>>>>>>> that later was transformed in a non-MR job to be merged, I think
>>>> we
>>>>>>>>> had a
>>>>>>>>>>> similar discussion for log split/replay.
>>>>>>>>>>> 
>>>>>>>>>>> the latest is the new Backup feature (HBASE-7912), that runs a
>>> MR
>>>> job
>>>>>>>>>> from
>>>>>>>>>>> the master to copy data or restore data.
>>>>>>>>>>> (backup is also "not really core" as in.. if you don't use
>>> backup
>>>>>>>>> you'll
>>>>>>>>>>> not end up running MR jobs, but this was probably true for MOB
>>> as
>>>> in
>>>>>>>>> "if
>>>>>>>>>>> you don't enable MOB you don't need MR")
>>>>>>>>>>> 
>>>>>>>>>>> any thoughts? do we a rule that says "we don't want to have
>>> hbase
>>>> run
>>>>>>>>> MR
>>>>>>>>>>> jobs, only tool started manually by the user can do that". or
>>> can
>>>> we
>>>>>>>>>> start
>>>>>>>>>>> adding MR calls around without problems?
>>> 

Reply via email to