If MR framework is not deployed in the cluster, hbase still functions normally (post merge).
In terms of build time dependency, we have long been depending on mapreduce. Take a look at ExportSnapshot. Cheers On Thu, Sep 22, 2016 at 7:42 PM, Heng Chen <heng.chen.1...@gmail.com> wrote: > In our production cluster, it is a common case we just have HDFS and > HBase deployed. > If our Master/RS depend on MR framework (especially some features we > have not used at all), it introduced another cost for maintain. I > don't think it is a good idea. > > 2016-09-23 10:28 GMT+08:00 张铎 <palomino...@gmail.com>: > > To be specific, for example, our nice Backup/Restore feature, if we think > > this is not a core feature of HBase, then we could make it depend on MR, > > and start a standalone BackupManager instance that submits MR jobs to do > > periodical maintenance job. And if we think this is a core feature that > > everyone should use it, then we'd better implement it without MR > > dependency, like DLS. > > > > Thanks. > > > > 2016-09-23 10:11 GMT+08:00 张铎 <palomino...@gmail.com>: > > > >> I‘m -1 on let master or rs launch MR jobs. It is OK that some of our > >> features depend on MR but I think the bottom line is that we should > launch > >> the jobs from outside manually or by other services. > >> > >> 2016-09-23 9:47 GMT+08:00 Andrew Purtell <andrew.purt...@gmail.com>: > >> > >>> Ok, got it. Well "shelling out" is on the line I think, so a fair > >>> question. > >>> > >>> Can this be driven by a utility derived from Tool like our other MR > apps? > >>> The issue is needing the AccessController to decide if allowed? But > nothing > >>> prevents the user from running the job manually/independently, right? > >>> > >>> > On Sep 22, 2016, at 3:44 PM, Matteo Bertozzi < > theo.berto...@gmail.com> > >>> wrote: > >>> > > >>> > just a remark. my query was not about tools using MR (everyone i > think > >>> is > >>> > ok with those). > >>> > the topic was about: "are we ok with running MR jobs from Master and > RSs > >>> > code?" since this will be the first time we do this > >>> > > >>> > Matteo > >>> > > >>> > > >>> >> On Thu, Sep 22, 2016 at 2:49 PM, Devaraj Das <d...@hortonworks.com> > >>> wrote: > >>> >> > >>> >> Very much agree; for tools like ExportSnapshot / Backup / Restore, > it's > >>> >> fine to be dependent on MR. MR is the right framework for such. We > >>> should > >>> >> also do compactions using MR (just saying :) ) > >>> >> ________________________________________ > >>> >> From: Ted Yu <yuzhih...@gmail.com> > >>> >> Sent: Thursday, September 22, 2016 2:00 PM > >>> >> To: dev@hbase.apache.org > >>> >> Subject: Re: [DISCUSSION] MR jobs started by Master or RS > >>> >> > >>> >> I agree - backup / restore is in the same category as import / > export. > >>> >> > >>> >> On Thu, Sep 22, 2016 at 1:58 PM, Andrew Purtell < > >>> andrew.purt...@gmail.com> > >>> >> wrote: > >>> >> > >>> >>> Backup is extra tooling around core in my opinion. Like import or > >>> export. > >>> >>> Or the optional MOB tool. It's fine. > >>> >>> > >>> >>>> On Sep 22, 2016, at 1:50 PM, Matteo Bertozzi < > mberto...@apache.org> > >>> >>> wrote: > >>> >>>> > >>> >>>> What's the latest opinion around running MR jobs from hbase > (Master > >>> or > >>> >>> RS)? > >>> >>>> > >>> >>>> I remember in the past that there was discussion about not having > MR > >>> >> has > >>> >>>> direct dependency of hbase. > >>> >>>> > >>> >>>> I think some of discussion where around MOB that had a MR job to > >>> >> compact, > >>> >>>> that later was transformed in a non-MR job to be merged, I think > we > >>> >> had a > >>> >>>> similar discussion for log split/replay. > >>> >>>> > >>> >>>> the latest is the new Backup feature (HBASE-7912), that runs a MR > job > >>> >>> from > >>> >>>> the master to copy data or restore data. > >>> >>>> (backup is also "not really core" as in.. if you don't use backup > >>> >> you'll > >>> >>>> not end up running MR jobs, but this was probably true for MOB as > in > >>> >> "if > >>> >>>> you don't enable MOB you don't need MR") > >>> >>>> > >>> >>>> any thoughts? do we a rule that says "we don't want to have hbase > run > >>> >> MR > >>> >>>> jobs, only tool started manually by the user can do that". or can > we > >>> >>> start > >>> >>>> adding MR calls around without problems? > >>> >>> > >>> >> > >>> > >> > >> >
