Hi Shahab, We issue the splits on the table and right away the major compact of the table. Then we connect to the HRegionServer info server at http://host:60030 and we check html output for the task to be completed using an HttpClient. So we actually wait for major compact and not for split. We've been using it since 0.89 but we believe there must be a bettter approach than that.
Thanks, IVan 2015-07-16 15:50 GMT+02:00 Shahab Yunus <[email protected]>: > Ivan, > > Is it possible that you poll for number of regions (e.g. in a loop) after > invoking split or merge to confirm that the action has been performed? I > know it is a crude way but maybe something can be done in these lines. Or > are you already doing this when you said 'look for Region Info'? > > Regards, > Shahab > > On Thu, Jul 16, 2015 at 9:46 AM, Ivan Brondino <[email protected]> > wrote: > > > Hi, > > > > I can confirm that it was what Jean Marc was saying. Thanks JM! > > > > Regarding question #1, any help? I am rephrasing it in case was not > clear. > > > > 1) Since both split and merge are async invocations on the client side, > we > > have a naîve > > workaround to make it synchronous that is to look at the RS Info Server > via > > an HttpClient checking > > for pending tasks to be finished. Once all tasks are finished in all RS > we > > issue the next split/merge for the same table. Is there a better way to > > check if major compact has finished? > > > > Thanks again, > > > > Iván > > > > > > 2015-07-15 17:49 GMT+02:00 Ivan Brondino <[email protected]>: > > > > > Hi Ted, Jean Marc, > > > > > > Thank you guys for your answer. > > > > > > I think Jean Marc spotted the issue, although we do slightly different. > > > > > > Let's say we have regions A and B. We split both A into A1 and A2 and > B > > > into B1 and B2. > > > Then we merge A2 and B1. Note that A and B are adjacent and so A1 with > > A2, > > > A2 with B1 and B1 with B2. > > > > > > We were not waiting for the major compact after the splits. Thus it > could > > > be the reason. > > > > > > We will give it a try and come back to you. > > > > > > Thanks, > > > > > > Ivan > > > > > > > > > > > > > > > > > > 2015-07-15 16:54 GMT+02:00 Jean-Marc Spaggiari < > [email protected] > > >: > > > > > >> Hi Ivan, > > >> > > >> > > >> For #1, let's consider regions A, B, C and D all in server S. > > >> > > >> If you merge A and B togher and C and D togher, still in S, you will > > have > > >> A' and C' regions only on S. > > >> > > >> Now, if you try to merge again A' and C' into a new bigger region, > this > > >> will fail silently until A' and C' are major compacted. > > >> > > >> Indeed, before major compaction, A' and C' will both still contain > > >> references to A+B and C+D files/regions. So to be able to merge A' and > > C' > > >> you need this to be cleaned and so you need to major compact them. > > >> > > >> HTH, > > >> > > >> JM > > >> > > >> 2015-07-15 10:43 GMT-04:00 Ted Yu <[email protected]>: > > >> > > >> > bq. that some times merge just does not work. > > >> > > > >> > Can you identify under what scenario the merge doesn't work (through > > >> closer > > >> > inspection of the region server log - assuming you have DEBUG > logging > > >> > turned on) ? > > >> > > > >> > bq. Are there minimum requirements for two regions to be merged? > > >> > > > >> > If the two adjacent regions to be merged are on different servers, > > HBase > > >> > would move one of them to the other region's server. > > >> > > > >> > bq. If those requirements are not met, does the merge process exit > > >> > silently? > > >> > > > >> > Nothing I am aware of. > > >> > > > >> > Cheers > > >> > > > >> > On Wed, Jul 15, 2015 at 3:08 AM, Ivan Brondino <[email protected] > > > > >> > wrote: > > >> > > > >> > > Hi, > > >> > > > > >> > > We are using Hbase 0.98.6 and Hadoop 2.5.0 - cdh 5.3.5. > > >> > > > > >> > > We have a couple of questions regarding splits and merge: > > >> > > > > >> > > 1) Since both split and merge are async invocations on the client > > >> side, > > >> > we > > >> > > have a naîve > > >> > > workaround to make it synchronous that is to look at the RS Info > > >> Server > > >> > via > > >> > > an HttpClient checking > > >> > > for pending tasks to be finished. Once all tasks are finished in > all > > >> RS > > >> > we > > >> > > issue the next split/merge for the same table. Is there a better > way > > >> to > > >> > do > > >> > > it? > > >> > > > > >> > > 2) Regarding merges, we've noticed that some times merge just does > > not > > >> > > work. We see on the master log that merge request is issued and it > > >> > forwards > > >> > > it to the corresponding RS but then on the RS nothing happens. We > > >> always > > >> > > merge two regions on the same region server. > > >> > > Are there minimum requirements for two regions to be merged? If > > those > > >> > > requirements are not met, does the merge process exit silently? > > >> > > > > >> > > Thanks in advance, > > >> > > > > >> > > Iván > > >> > > > > >> > > > >> > > > > > > > > >
