[
https://issues.apache.org/jira/browse/HBASE-3969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13050237#comment-13050237
]
zhoushuaifeng commented on HBASE-3969:
--------------------------------------
I have done a test on hbase use ycsb. 3 region servers and 30,000 insert per
second. With no patch solution1, we can see from ganglia that after 3 days,
about 50 regions stay in the compact queue, and out dated data can not be
cleaned intime. And after added patch solution1, out dated data can be cleaned
in time.
Logs show that how the patch works:(our setings: blockingStoreFiles is 10,
compactionThreshold is 3, so the priority is 6.)
2011-06-16 13:06:31,667 DEBUG
org.apache.hadoop.hbase.regionserver.CompactSplitThread: Compaction requested
for
ufdr1,0201586139100003#4523000,1308096774090.080ad1468ece2c12efeb2480be632871.
because regionserver60020.majorCompactionChecker requests major compaction;
priority=6, compaction queue size=25
2011-06-16 13:06:31,670 DEBUG
org.apache.hadoop.hbase.regionserver.CompactSplitThread: Compaction requested
for ufdr2,00380,1308118259184.75a7c5da1c2ffadd043f7f6eea2b22b5. because
regionserver60020.majorCompactionChecker requests major compaction; priority=6,
compaction queue size=26
2011-06-16 13:06:31,675 DEBUG
org.apache.hadoop.hbase.regionserver.CompactSplitThread: Compaction requested
for
ufdr1,0198486139199963#0404000,1308093001292.50c82e3de192aa621d3d68465f3f9530.
because regionserver60020.majorCompactionChecker requests major compaction;
priority=6, compaction queue size=27
> Outdated data can not be cleaned in time
> ----------------------------------------
>
> Key: HBASE-3969
> URL: https://issues.apache.org/jira/browse/HBASE-3969
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
> Affects Versions: 0.90.1, 0.90.2, 0.90.3
> Reporter: zhoushuaifeng
> Fix For: 0.90.4
>
> Attachments: HBASE-3969-solution1.patch
>
>
> Compaction checker will send regions to the compact queue to do compact. But
> the priority of these regions is too low if these regions have only a few
> storefiles. When there is large through output, and the compact queue will
> aways have some regions with higher priority. This may causing the major
> compact be delayed for a long time(even a few days), and outdated data
> cleaning will also be delayed.
> In our test case, we found some regions sent to the queue by major compact
> checker hunging in the queue for more than 2 days! Some scanners on these
> regions cannot get availably data for a long time and lease expired.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
