Thanks Ted. That should work for us. Regards,
Ivan 2015-07-16 16:06 GMT+02:00 Ted Yu <[email protected]>: > Ivan: > You can utilize the following method in HBaseAdmin to retrieve compaction > status: > > public CompactionState getCompactionState(final TableName tableName) > > This API is available in 0.98+ > > FYI > > On Thu, Jul 16, 2015 at 6:58 AM, Ivan Brondino <[email protected]> > wrote: > > > Hi Shahab, > > > > We issue the splits on the table and right away the major compact of the > > table. > > Then we connect to the HRegionServer info server at http://host:60030 > and > > we check html output for the task to be completed using an HttpClient. > > So we actually wait for major compact and not for split. > > We've been using it since 0.89 but we believe there must be a bettter > > approach than that. > > > > Thanks, > > > > IVan > > > > > > > > 2015-07-16 15:50 GMT+02:00 Shahab Yunus <[email protected]>: > > > > > Ivan, > > > > > > Is it possible that you poll for number of regions (e.g. in a loop) > after > > > invoking split or merge to confirm that the action has been performed? > I > > > know it is a crude way but maybe something can be done in these lines. > Or > > > are you already doing this when you said 'look for Region Info'? > > > > > > Regards, > > > Shahab > > > > > > On Thu, Jul 16, 2015 at 9:46 AM, Ivan Brondino <[email protected]> > > > wrote: > > > > > > > Hi, > > > > > > > > I can confirm that it was what Jean Marc was saying. Thanks JM! > > > > > > > > Regarding question #1, any help? I am rephrasing it in case was not > > > clear. > > > > > > > > 1) Since both split and merge are async invocations on the client > side, > > > we > > > > have a naîve > > > > workaround to make it synchronous that is to look at the RS Info > Server > > > via > > > > an HttpClient checking > > > > for pending tasks to be finished. Once all tasks are finished in all > RS > > > we > > > > issue the next split/merge for the same table. Is there a better way > to > > > > check if major compact has finished? > > > > > > > > Thanks again, > > > > > > > > Iván > > > > > > > > > > > > 2015-07-15 17:49 GMT+02:00 Ivan Brondino <[email protected]>: > > > > > > > > > Hi Ted, Jean Marc, > > > > > > > > > > Thank you guys for your answer. > > > > > > > > > > I think Jean Marc spotted the issue, although we do slightly > > different. > > > > > > > > > > Let's say we have regions A and B. We split both A into A1 and A2 > > and > > > B > > > > > into B1 and B2. > > > > > Then we merge A2 and B1. Note that A and B are adjacent and so A1 > > with > > > > A2, > > > > > A2 with B1 and B1 with B2. > > > > > > > > > > We were not waiting for the major compact after the splits. Thus it > > > could > > > > > be the reason. > > > > > > > > > > We will give it a try and come back to you. > > > > > > > > > > Thanks, > > > > > > > > > > Ivan > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > 2015-07-15 16:54 GMT+02:00 Jean-Marc Spaggiari < > > > [email protected] > > > > >: > > > > > > > > > >> Hi Ivan, > > > > >> > > > > >> > > > > >> For #1, let's consider regions A, B, C and D all in server S. > > > > >> > > > > >> If you merge A and B togher and C and D togher, still in S, you > will > > > > have > > > > >> A' and C' regions only on S. > > > > >> > > > > >> Now, if you try to merge again A' and C' into a new bigger region, > > > this > > > > >> will fail silently until A' and C' are major compacted. > > > > >> > > > > >> Indeed, before major compaction, A' and C' will both still contain > > > > >> references to A+B and C+D files/regions. So to be able to merge A' > > and > > > > C' > > > > >> you need this to be cleaned and so you need to major compact them. > > > > >> > > > > >> HTH, > > > > >> > > > > >> JM > > > > >> > > > > >> 2015-07-15 10:43 GMT-04:00 Ted Yu <[email protected]>: > > > > >> > > > > >> > bq. that some times merge just does not work. > > > > >> > > > > > >> > Can you identify under what scenario the merge doesn't work > > (through > > > > >> closer > > > > >> > inspection of the region server log - assuming you have DEBUG > > > logging > > > > >> > turned on) ? > > > > >> > > > > > >> > bq. Are there minimum requirements for two regions to be merged? > > > > >> > > > > > >> > If the two adjacent regions to be merged are on different > servers, > > > > HBase > > > > >> > would move one of them to the other region's server. > > > > >> > > > > > >> > bq. If those requirements are not met, does the merge process > exit > > > > >> > silently? > > > > >> > > > > > >> > Nothing I am aware of. > > > > >> > > > > > >> > Cheers > > > > >> > > > > > >> > On Wed, Jul 15, 2015 at 3:08 AM, Ivan Brondino < > > [email protected] > > > > > > > > >> > wrote: > > > > >> > > > > > >> > > Hi, > > > > >> > > > > > > >> > > We are using Hbase 0.98.6 and Hadoop 2.5.0 - cdh 5.3.5. > > > > >> > > > > > > >> > > We have a couple of questions regarding splits and merge: > > > > >> > > > > > > >> > > 1) Since both split and merge are async invocations on the > > client > > > > >> side, > > > > >> > we > > > > >> > > have a naîve > > > > >> > > workaround to make it synchronous that is to look at the RS > Info > > > > >> Server > > > > >> > via > > > > >> > > an HttpClient checking > > > > >> > > for pending tasks to be finished. Once all tasks are finished > in > > > all > > > > >> RS > > > > >> > we > > > > >> > > issue the next split/merge for the same table. Is there a > better > > > way > > > > >> to > > > > >> > do > > > > >> > > it? > > > > >> > > > > > > >> > > 2) Regarding merges, we've noticed that some times merge just > > does > > > > not > > > > >> > > work. We see on the master log that merge request is issued > and > > it > > > > >> > forwards > > > > >> > > it to the corresponding RS but then on the RS nothing happens. > > We > > > > >> always > > > > >> > > merge two regions on the same region server. > > > > >> > > Are there minimum requirements for two regions to be merged? > If > > > > those > > > > >> > > requirements are not met, does the merge process exit > silently? > > > > >> > > > > > > >> > > Thanks in advance, > > > > >> > > > > > > >> > > Iván > > > > >> > > > > > > >> > > > > > >> > > > > > > > > > > > > > > > > > > > >
