Agreed. I don't think we need this anymore given that we don't compact everything on open.
JG On Wed, September 2, 2009 10:18 pm, Andrew Purtell wrote: > I had a similar thought, that this could be removed in 0.21. > > > - Andy > > > > > > ________________________________ > From: stack <[email protected]> > To: [email protected] > Sent: Wednesday, September 2, 2009 8:13:06 PM > Subject: Re: Compactions no longer limited > > > Andrew: > > > Maybe now that in 0.20.0 we only run compaction on open IFF the region > has references, may be this facility is no longer needed? > > St.Ack > > > On Wed, Sep 2, 2009 at 6:05 PM, Andrew Purtell <[email protected]> > wrote: > > >> Hi Ken, >> >> >> Compactions are serviced by a thread which sleeps for a configurable >> interval and then wakes to do work. As compaction requests are raised, >> they are queued and the thread is signaled and wakes early. When a >> region server first starts up, a limit is imposed on how many compaction >> requests can be serviced from the queue before the thread goes back to >> sleep. The limit is gradually relaxed. The net effect is to draw out >> compactions over an initial "safe mode". The reason this is done is to >> limit load on DFS when the cluster is starting. Compactions during this >> period often process flushes from the last shutdown and can well trigger >> splits. Around the 0.19 timeframe I had a 25 node test cluster pushing >> 1000 regions per node. >> Startup compaction/split activity would crush DFS and prevent successful >> (re)starts. The mechanism described here solved that issue. >> >> >> - Andy >> >> >> >> >> >> >> ________________________________ >> From: Ken Weiner <[email protected]> >> To: [email protected] >> Sent: Wednesday, September 2, 2009 5:50:53 PM >> Subject: Compactions no longer limited >> >> >> What does it mean when the following INFO message appears in the >> RegionServer logs? I generally see that message show up on all >> RegionServers at the same time. >> >> >> org.apache.hadoop.hbase.regionserver.HRegionServer: compactions no >> longer limited >> >> Thanks, Ken >> >> >> >> >> >> > > > >
