(Back with a sore throat.) Also for what it is worth - it may well be that the attempt to bolt containers-as-executors to YARN is too little too late and coordination of container based services and applications (such as distributed map-reduce workflows or more likely Spark) will be handled by the native management tooling we are all using to build container based infrastructure. It's unclear to me how long HBase will live into this new regime but I'd optimistically wager long enough so even having YARN around is a suspect assumption.
> On Sep 22, 2016, at 6:58 PM, Andrew Purtell <andrew.purt...@gmail.com> wrote: > > > We should also do compactions using MR (just saying :) > > No, never. It's not a good idea to wed any of our core function to something > that independently evolves, that some of us don't have commit rights on (and > never will), and has varying degrees of utility depending on deploy. Like JM > says in some places not having the MR runtime around is virtuous. > > (Runs away screaming.) > > On Sep 22, 2016, at 2:49 PM, Devaraj Das <d...@hortonworks.com> wrote: > >> Very much agree; for tools like ExportSnapshot / Backup / Restore, it's fine >> to be dependent on MR. MR is the right framework for such. We should also do >> compactions using MR (just saying :) ) >> ________________________________________ >> From: Ted Yu <yuzhih...@gmail.com> >> Sent: Thursday, September 22, 2016 2:00 PM >> To: dev@hbase.apache.org >> Subject: Re: [DISCUSSION] MR jobs started by Master or RS >> >> I agree - backup / restore is in the same category as import / export. >> >> On Thu, Sep 22, 2016 at 1:58 PM, Andrew Purtell <andrew.purt...@gmail.com> >> wrote: >> >>> Backup is extra tooling around core in my opinion. Like import or export. >>> Or the optional MOB tool. It's fine. >>> >>>> On Sep 22, 2016, at 1:50 PM, Matteo Bertozzi <mberto...@apache.org> >>> wrote: >>>> >>>> What's the latest opinion around running MR jobs from hbase (Master or >>> RS)? >>>> >>>> I remember in the past that there was discussion about not having MR has >>>> direct dependency of hbase. >>>> >>>> I think some of discussion where around MOB that had a MR job to compact, >>>> that later was transformed in a non-MR job to be merged, I think we had a >>>> similar discussion for log split/replay. >>>> >>>> the latest is the new Backup feature (HBASE-7912), that runs a MR job >>> from >>>> the master to copy data or restore data. >>>> (backup is also "not really core" as in.. if you don't use backup you'll >>>> not end up running MR jobs, but this was probably true for MOB as in "if >>>> you don't enable MOB you don't need MR") >>>> >>>> any thoughts? do we a rule that says "we don't want to have hbase run MR >>>> jobs, only tool started manually by the user can do that". or can we >>> start >>>> adding MR calls around without problems? >>>
