On Wed, Jun 8, 2011 at 9:14 PM, Zhoushuaifeng <[email protected]> wrote: > //zhou: I agree regions with more files should take higher priority, but > there are other factors important should be considered. In our test case, we > found some regions sent to the queue by major compact checker hunging in the > queue for more than 2 days! Some scanners on these regions cannot get > availably data for a long time and lease expired. > I think set these regions priority to hbase.hstore.blockingStoreFiles - > hbase.hstore.compactionThreshold -1 as default may be a good way to solve > this problem. If regions have less than 3 files, it's priority is lower than > outdated regions, but if it has more than 4 files, it's priority will be > higher. This settings can both solve the outdated problem and will not block > flush and put. >
I would not build on hbase.hstore.blockingStoreFiles. That should be going away. What else would you suggest? > Maybe the compaction algorithm should consider age of compactions too? > If a compaction has been hanging the queue a good while, its priority > gets bumped a level? > //zhou: This is good, I totally agree. This is another good way to solve the > problem. But by now, I don't know how to make the patch, may be we can dig > more on it. > Want to file an issue on this Zhou? St.Ack
