Forgot WALPlayer :) -Vlad
On Thu, Sep 22, 2016 at 4:21 PM, Vladimir Rodionov <vladrodio...@gmail.com> wrote: > >> and > >> backups too, but don't want to bother having to install and configure > YARN > >> just for that, as well as removing resources from HBase to give it to > > Any suggestions on how to do bulk data move with transformation from/to > HBase cluster w/o MapReduce? > > Opposition to M/R does not make sense imo, as since we have a lot of tools > in HBase which depend on MapReduce: > > CountRows > CountCells > Import > Export > ImportTsv > ExportTsv > CopyTable > VerifyReplication > ExportSnapshot > > and new backup create/restore of course. > > > -Vlad > > > > > On Thu, Sep 22, 2016 at 4:15 PM, Jean-Marc Spaggiari < > jean-m...@spaggiari.org> wrote: > >> My 2¢: I have a strong preference for NOT having a dependency on MR >> anywhere :( I run my HBase cluste without YARN. Just HBase and HDFS. I >> like >> all the features that we built. Would love to be able to use MOBs and >> backups too, but don't want to bother having to install and configure YARN >> just for that, as well as removing resources from HBase to give it to >> yarn.... >> >> JMS >> >> 2016-09-22 18:44 GMT-04:00 Matteo Bertozzi <theo.berto...@gmail.com>: >> >> > just a remark. my query was not about tools using MR (everyone i think >> is >> > ok with those). >> > the topic was about: "are we ok with running MR jobs from Master and RSs >> > code?" since this will be the first time we do this >> > >> > Matteo >> > >> > >> > On Thu, Sep 22, 2016 at 2:49 PM, Devaraj Das <d...@hortonworks.com> >> wrote: >> > >> > > Very much agree; for tools like ExportSnapshot / Backup / Restore, >> it's >> > > fine to be dependent on MR. MR is the right framework for such. We >> should >> > > also do compactions using MR (just saying :) ) >> > > ________________________________________ >> > > From: Ted Yu <yuzhih...@gmail.com> >> > > Sent: Thursday, September 22, 2016 2:00 PM >> > > To: dev@hbase.apache.org >> > > Subject: Re: [DISCUSSION] MR jobs started by Master or RS >> > > >> > > I agree - backup / restore is in the same category as import / export. >> > > >> > > On Thu, Sep 22, 2016 at 1:58 PM, Andrew Purtell < >> > andrew.purt...@gmail.com> >> > > wrote: >> > > >> > > > Backup is extra tooling around core in my opinion. Like import or >> > export. >> > > > Or the optional MOB tool. It's fine. >> > > > >> > > > > On Sep 22, 2016, at 1:50 PM, Matteo Bertozzi < >> mberto...@apache.org> >> > > > wrote: >> > > > > >> > > > > What's the latest opinion around running MR jobs from hbase >> (Master >> > or >> > > > RS)? >> > > > > >> > > > > I remember in the past that there was discussion about not having >> MR >> > > has >> > > > > direct dependency of hbase. >> > > > > >> > > > > I think some of discussion where around MOB that had a MR job to >> > > compact, >> > > > > that later was transformed in a non-MR job to be merged, I think >> we >> > > had a >> > > > > similar discussion for log split/replay. >> > > > > >> > > > > the latest is the new Backup feature (HBASE-7912), that runs a MR >> job >> > > > from >> > > > > the master to copy data or restore data. >> > > > > (backup is also "not really core" as in.. if you don't use backup >> > > you'll >> > > > > not end up running MR jobs, but this was probably true for MOB as >> in >> > > "if >> > > > > you don't enable MOB you don't need MR") >> > > > > >> > > > > any thoughts? do we a rule that says "we don't want to have hbase >> run >> > > MR >> > > > > jobs, only tool started manually by the user can do that". or can >> we >> > > > start >> > > > > adding MR calls around without problems? >> > > > >> > > >> > >> > >
