To be specific, for example, our nice Backup/Restore feature, if we think
this is not a core feature of HBase, then we could make it depend on MR,
and start a standalone BackupManager instance that submits MR jobs to do
periodical maintenance job. And if we think this is a core feature that
everyone should use it, then we'd better implement it without MR
dependency, like DLS.

Thanks.

2016-09-23 10:11 GMT+08:00 张铎 <palomino...@gmail.com>:

> I‘m -1 on let master or rs launch MR jobs. It is OK that some of our
> features depend on MR but I think the bottom line is that we should launch
> the jobs from outside manually or by other services.
>
> 2016-09-23 9:47 GMT+08:00 Andrew Purtell <andrew.purt...@gmail.com>:
>
>> Ok, got it. Well "shelling out" is on the line I think, so a fair
>> question.
>>
>> Can this be driven by a utility derived from Tool like our other MR apps?
>> The issue is needing the AccessController to decide if allowed? But nothing
>> prevents the user from running the job manually/independently, right?
>>
>> > On Sep 22, 2016, at 3:44 PM, Matteo Bertozzi <theo.berto...@gmail.com>
>> wrote:
>> >
>> > just a remark. my query was not about tools using MR (everyone i think
>> is
>> > ok with those).
>> > the topic was about: "are we ok with running MR jobs from Master and RSs
>> > code?" since this will be the first time we do this
>> >
>> > Matteo
>> >
>> >
>> >> On Thu, Sep 22, 2016 at 2:49 PM, Devaraj Das <d...@hortonworks.com>
>> wrote:
>> >>
>> >> Very much agree; for tools like ExportSnapshot / Backup / Restore, it's
>> >> fine to be dependent on MR. MR is the right framework for such. We
>> should
>> >> also do compactions using MR (just saying :) )
>> >> ________________________________________
>> >> From: Ted Yu <yuzhih...@gmail.com>
>> >> Sent: Thursday, September 22, 2016 2:00 PM
>> >> To: dev@hbase.apache.org
>> >> Subject: Re: [DISCUSSION] MR jobs started by Master or RS
>> >>
>> >> I agree - backup / restore is in the same category as import / export.
>> >>
>> >> On Thu, Sep 22, 2016 at 1:58 PM, Andrew Purtell <
>> andrew.purt...@gmail.com>
>> >> wrote:
>> >>
>> >>> Backup is extra tooling around core in my opinion. Like import or
>> export.
>> >>> Or the optional MOB tool. It's fine.
>> >>>
>> >>>> On Sep 22, 2016, at 1:50 PM, Matteo Bertozzi <mberto...@apache.org>
>> >>> wrote:
>> >>>>
>> >>>> What's the latest opinion around running MR jobs from hbase (Master
>> or
>> >>> RS)?
>> >>>>
>> >>>> I remember in the past that there was discussion about not having MR
>> >> has
>> >>>> direct dependency of hbase.
>> >>>>
>> >>>> I think some of discussion where around MOB that had a MR job to
>> >> compact,
>> >>>> that later was transformed in a non-MR job to be merged, I think we
>> >> had a
>> >>>> similar discussion for log split/replay.
>> >>>>
>> >>>> the latest is the new Backup feature (HBASE-7912), that runs a MR job
>> >>> from
>> >>>> the master to copy data or restore data.
>> >>>> (backup is also "not really core" as in.. if you don't use backup
>> >> you'll
>> >>>> not end up running MR jobs, but this was probably true for MOB as in
>> >> "if
>> >>>> you don't enable MOB you don't need MR")
>> >>>>
>> >>>> any thoughts? do we a rule that says "we don't want to have hbase run
>> >> MR
>> >>>> jobs, only tool started manually by the user can do that". or can we
>> >>> start
>> >>>> adding MR calls around without problems?
>> >>>
>> >>
>>
>
>

Reply via email to