I had a similar thought, that this could be removed in 0.21. - Andy
________________________________ From: stack <[email protected]> To: [email protected] Sent: Wednesday, September 2, 2009 8:13:06 PM Subject: Re: Compactions no longer limited Andrew: Maybe now that in 0.20.0 we only run compaction on open IFF the region has references, may be this facility is no longer needed? St.Ack On Wed, Sep 2, 2009 at 6:05 PM, Andrew Purtell <[email protected]> wrote: > Hi Ken, > > Compactions are serviced by a thread which sleeps for a configurable > interval and then wakes to do work. As compaction requests are raised, they > are queued and the thread is signaled and wakes early. When a region server > first starts up, a limit is imposed on how many compaction requests can be > serviced from the queue before the thread goes back to sleep. The limit is > gradually relaxed. The net effect is to draw out compactions over an initial > "safe mode". The reason this is done is to limit load on DFS when the > cluster is starting. Compactions during this period often process flushes > from the last shutdown and can well trigger splits. Around the 0.19 > timeframe I had a 25 node test cluster pushing 1000 regions per node. > Startup compaction/split activity would crush DFS and prevent successful > (re)starts. The mechanism described here solved that issue. > > - Andy > > > > > > ________________________________ > From: Ken Weiner <[email protected]> > To: [email protected] > Sent: Wednesday, September 2, 2009 5:50:53 PM > Subject: Compactions no longer limited > > What does it mean when the following INFO message appears in the > RegionServer logs? I generally see that message show up on all > RegionServers at the same time. > > org.apache.hadoop.hbase.regionserver.HRegionServer: compactions no longer > limited > > Thanks, Ken > > > > >
