I'm also eager to see the online merge in HBase. I have many plans for it. In the meantime, I will try to get this small improvment in 0.94. I will keep the code in the Merge class since it will most probably be totally re-do when the online will be there.
Thanks, JM 2013/3/21 Ted Yu <[email protected]>: > I agree that we should put effort behind online merge in 0.95 and beyond. > > On Thu, Mar 21, 2013 at 5:03 PM, Enis Söztutar <[email protected]> wrote: > >> I agree that for 0.94 we would still need offline merge, but if we can get >> away with it, I would rather support only one tool. >> >> Enis >> >> >> On Thu, Mar 21, 2013 at 4:31 PM, Ted Yu <[email protected]> wrote: >> >> > Changing the subject to reflect current discussion. >> > >> > I think Jean-Marc's work on offline merge would most likely be useful in >> > the context of 0.94.x code base. >> > >> > Cheers >> > >> > On Thu, Mar 21, 2013 at 4:17 PM, Enis Söztutar <[email protected]> >> wrote: >> > >> > > Thanks J-M. >> > > >> > > What I am trying to understand is that whether we should cut the cord >> for >> > > offline merge once online is working. If you think about it, there >> should >> > > not be a need to merge offline tables. >> > > >> > > Enis >> > > >> > > >> > > On Thu, Mar 21, 2013 at 3:16 PM, Jean-Marc Spaggiari < >> > > [email protected]> wrote: >> > > >> > > > Offline merge is already there and working fine. >> > > > >> > > > The usecase here was to retreive all the regions for a given table to >> > > > merge them 2 by 2, offline. >> > > > >> > > > It's working fine, but since the Meta rework it's not working anymore >> > > > and I'm trying to rebase the patch. >> > > > >> > > > Like J-D is saying, yes, it's used only in the offline merge... And >> > > > since the online merge is coming, I think it's cleaner to keep the >> > > > code in the offline merge since it will disapear soon, but in the >> > > > meantime, at least, we will have the offline one. >> > > > >> > > > JM >> > > > >> > > > 2013/3/21 Enis Söztutar <[email protected]>: >> > > > > What is the use case behind offline merge? Is it because we cannot >> do >> > > > > online merge yet? If we can get HBASE-7403 in, is there still need >> to >> > > > > support offline merge? >> > > > > >> > > > > Enis >> > > > > >> > > > > >> > > > > On Thu, Mar 21, 2013 at 2:56 PM, Jean-Daniel Cryans < >> > > [email protected] >> > > > >wrote: >> > > > > >> > > > >> As far as I can tell, only the merge code uses MetaUtils to do >> > offline >> > > > >> work. If this is the code you are in then pull it back into >> > MetaUtils >> > > > >> I think. >> > > > >> >> > > > >> J-D >> > > > >> >> > > > >> On Thu, Mar 21, 2013 at 2:44 PM, Jean-Marc Spaggiari >> > > > >> <[email protected]> wrote: >> > > > >> > Vector is because of a very old bad habit ;) I will change that >> to >> > > > >> ArrayList. >> > > > >> > >> > > > >> > So far I have inlined the scanMetaRegion feature into the Merge, >> > but >> > > > >> > maybe it should be cleaner to put it back in? >> > > > >> > >> > > > >> > Anyway, I will keep the inlined one until everything is cleaned. >> > > > >> > >> > > > >> > 2013/3/21 Enis Söztutar <[email protected]>: >> > > > >> >> The problem around current META scanning is that there is more >> > than >> > > > one >> > > > >> way >> > > > >> >> to do these, and the META layout is exposed. We should refrain >> > from >> > > > >> >> exposing the META details. >> > > > >> >> AFAIK, these do the same thing: >> > > > >> >> MetaReader.Visitor >> > > > >> >> MetaScanner.MetaScannerVisitor, and >> > > > >> >> MetaUtils.ScannerListener >> > > > >> >> >> > > > >> >> More concerning is that the code for managing META is spread >> over >> > > > >> >> MetaEditor, MetaReader, MetaScanner, MetaUtils, HRegionInfo >> (and >> > > > maybe >> > > > >> >> more). There are a couple of issues to rework these interfaces, >> > > but I >> > > > >> did >> > > > >> >> not get the chance to work on those. >> > > > >> >> >> > > > >> >> >> > > > >> >> Enis >> > > > >> >> >> > > > >> >> >> > > > >> >> >> > > > >> >> >> > > > >> >> >> > > > >> >> >> > > > >> >> >> > > > >> >> >> > > > >> >> On Thu, Mar 21, 2013 at 2:18 PM, Jean-Daniel Cryans < >> > > > >> [email protected]>wrote: >> > > > >> >> >> > > > >> >>> Mmmm I may have been trigger happy. You could pull back the >> > > > >> >>> ScannerListener and scanMetaRegion. >> > > > >> >>> >> > > > >> >>> BTW, why are you using a Vector instead of ArrayList? >> > > > >> >>> >> > > > >> >>> J-D >> > > > >> >>> >> > > > >> >>> On Thu, Mar 21, 2013 at 2:05 PM, Jean-Marc Spaggiari >> > > > >> >>> <[email protected]> wrote: >> > > > >> >>> > Hi, >> > > > >> >>> > >> > > > >> >>> > In trunk, since HBASE-3171 (Drop ROOT and instead store META >> > > > >> >>> > location(s) directly in ZooKeeper ) there is no more >> > > > >> >>> > MetaUtils.ScannerListener. >> > > > >> >>> > >> > > > >> >>> > In the merge, I used it to retreive all the regions >> belonging >> > > to a >> > > > >> >>> > specific table, from the META. kind of scan. >> > > > >> >>> > >> > > > >> >>> > // Retrieve the list of regions for this table. >> > > > >> >>> > final List<HRegionInfo> regions = new >> > > > >> Vector<HRegionInfo>(); >> > > > >> >>> > >> > > utils.scanMetaRegion(HRegionInfo.FIRST_META_REGIONINFO, >> > > > new >> > > > >> >>> > MetaUtils.ScannerListener() { >> > > > >> >>> > public boolean processRow(HRegionInfo info) { >> > > > >> >>> > if ((info != null) && >> > > > >> >>> > (Bytes.compareTo(info.getTableName(), tableName) == 0)) { >> > > > >> >>> > regions.add(info); >> > > > >> >>> > } >> > > > >> >>> > return true; >> > > > >> >>> > } >> > > > >> >>> > }); >> > > > >> >>> > >> > > > >> >>> > >> > > > >> >>> > Is there a recommanded way to replace that? The Merge is >> > running >> > > > >> >>> > offline, so I can't do a scan. >> > > > >> >>> > >> > > > >> >>> > Thanks, >> > > > >> >>> > >> > > > >> >>> > JM >> > > > >> >>> >> > > > >> >> > > > >> > > >> > >>
