[
https://issues.apache.org/jira/browse/HBASE-1083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12660960#action_12660960
]
stack commented on HBASE-1083:
------------------------------
I'm going to apply above patch. Here is how it looks now for an individual
region 335809914:
{code}
trunk/logs/hbase-stack-regionserver-aa0-000-13.u.powerset.com.log:2009-01-05
19:34:46,721 [regionserver/0:0:0:0:0:0:0:0:60020.worker] DEBUG
org.apache.hadoop.hbase.regionserver.HRegion: Opening region
TestTable,0004338266,1231183799438/335809914
trunk/logs/hbase-stack-regionserver-aa0-000-13.u.powerset.com.log:2009-01-05
19:34:46,770 [regionserver/0:0:0:0:0:0:0:0:60020.worker] DEBUG
org.apache.hadoop.hbase.regionserver.HStore: loaded
/hbasetrunk2/TestTable/335809914/info/info/680545570196928204,
isReference=false, sequence id=2548117, length=145155802, majorCompaction=false
trunk/logs/hbase-stack-regionserver-aa0-000-13.u.powerset.com.log:2009-01-05
19:34:46,772 [regionserver/0:0:0:0:0:0:0:0:60020.worker] DEBUG
org.apache.hadoop.hbase.regionserver.HStore: Loaded 1 file(s) in hstore
335809914/info, max sequence id 2548117
trunk/logs/hbase-stack-regionserver-aa0-000-13.u.powerset.com.log:2009-01-05
19:34:46,797 [regionserver/0:0:0:0:0:0:0:0:60020.worker] INFO
org.apache.hadoop.hbase.regionserver.HRegion: region
TestTable,0004338266,1231183799438/335809914 available
trunk/logs/hbase-stack-regionserver-aa0-000-13.u.powerset.com.log:2009-01-05
19:34:46,797 [regionserver/0:0:0:0:0:0:0:0:60020.worker] DEBUG
org.apache.hadoop.hbase.regionserver.CompactSplitThread: Compaction requested
for region TestTable,0004338266,1231183799438/335809914 because: Region open
check
trunk/logs/hbase-stack-regionserver-aa0-000-13.u.powerset.com.log:2009-01-05
20:11:32,815 [regionserver/0:0:0:0:0:0:0:0:60020.majorCompactionChecker] DEBUG
org.apache.hadoop.hbase.regionserver.HStore: Major compaction triggered on
store 335809914/info; time since last major compaction 2486624ms
trunk/logs/hbase-stack-regionserver-aa0-000-13.u.powerset.com.log:2009-01-05
20:11:32,815 [regionserver/0:0:0:0:0:0:0:0:60020.majorCompactionChecker] DEBUG
org.apache.hadoop.hbase.regionserver.CompactSplitThread: Compaction requested
for region TestTable,0004338266,1231183799438/335809914 because:
regionserver/0:0:0:0:0:0:0:0:60020.majorCompactionChecker requests major
compaction
trunk/logs/hbase-stack-regionserver-aa0-000-13.u.powerset.com.log:2009-01-05
20:11:39,546 [regionserver/0:0:0:0:0:0:0:0:60020.compactor] DEBUG
org.apache.hadoop.hbase.regionserver.HStore: Major compaction triggered on
store 335809914/info; time since last major compaction 2493355ms
trunk/logs/hbase-stack-regionserver-aa0-000-13.u.powerset.com.log:2009-01-05
20:11:39,729 [regionserver/0:0:0:0:0:0:0:0:60020.compactor] DEBUG
org.apache.hadoop.hbase.regionserver.HStore: Started compaction of 1 file(s)
into
/hbasetrunk2/TestTable/compaction.dir/335809914/info/mapfiles/3632008531127598620
trunk/logs/hbase-stack-regionserver-aa0-000-13.u.powerset.com.log:2009-01-05
20:12:23,329 [regionserver/0:0:0:0:0:0:0:0:60020.compactor] DEBUG
org.apache.hadoop.hbase.regionserver.HStore: moving
/hbasetrunk2/TestTable/compaction.dir/335809914/info/mapfiles/3632008531127598620
to /hbasetrunk2/TestTable/335809914/info/mapfiles/7200075435675545045
trunk/logs/hbase-stack-regionserver-aa0-000-13.u.powerset.com.log:2009-01-05
20:12:23,366 [regionserver/0:0:0:0:0:0:0:0:60020.compactor] DEBUG
org.apache.hadoop.hbase.regionserver.HStore: Completed major compaction of
335809914/info store size is 138.4m
trunk/logs/hbase-stack-regionserver-aa0-000-13.u.powerset.com.log:2009-01-05
20:44:52,844 [regionserver/0:0:0:0:0:0:0:0:60020.majorCompactionChecker] DEBUG
org.apache.hadoop.hbase.regionserver.HStore: Skipping major compaction of
335809914/info because one (major) compacted file only and elapsedTime
1993205ms is < ttl=-1
trunk/logs/hbase-stack-regionserver-aa0-000-13.u.powerset.com.log:2009-01-05
21:01:32,844 [regionserver/0:0:0:0:0:0:0:0:60020.majorCompactionChecker] DEBUG
org.apache.hadoop.hbase.regionserver.HStore: Skipping major compaction of
335809914/info because one (major) compacted file only and elapsedTime
2993205ms is < ttl=-1
trunk/logs/hbase-stack-regionserver-aa0-000-13.u.powerset.com.log:2009-01-05
21:18:12,844 [regionserver/0:0:0:0:0:0:0:0:60020.majorCompactionChecker] DEBUG
org.apache.hadoop.hbase.regionserver.HStore: Skipping major compaction of
335809914/info because one (major) compacted file only and elapsedTime
3993205ms is < ttl=-1
trunk/logs/hbase-stack-regionserver-aa0-000-13.u.powerset.com.log:2009-01-05
21:34:52,854 [regionserver/0:0:0:0:0:0:0:0:60020.majorCompactionChecker] DEBUG
org.apache.hadoop.hbase.regionserver.HStore: Skipping major compaction of
335809914/info because one (major) compacted file only and elapsedTime
4993215ms is < ttl=-1
trunk/logs/hbase-stack-regionserver-aa0-000-13.u.powerset.com.log:2009-01-05
21:51:32,864 [regionserver/0:0:0:0:0:0:0:0:60020.majorCompactionChecker] DEBUG
org.apache.hadoop.hbase.regionserver.HStore: Skipping major compaction of
335809914/info because one (major) compacted file only and elapsedTime
5993225ms is < ttl=-1
trunk/logs/hbase-stack-regionserver-aa0-000-13.u.powerset.com.log:2009-01-05
22:08:12,884 [regionserver/0:0:0:0:0:0:0:0:60020.majorCompactionChecker] DEBUG
org.apache.hadoop.hbase.regionserver.HStore: Skipping major compaction of
335809914/info because one (major) compacted file only and elapsedTime
6993245ms is < ttl=-1
trunk/logs/hbase-stack-regionserver-aa0-000-13.u.powerset.com.log:2009-01-05
22:24:52,894 [regionserver/0:0:0:0:0:0:0:0:60020.majorCompactionChecker] DEBUG
org.apache.hadoop.hbase.regionserver.HStore: Skipping major compaction of
335809914/info because one (major) compacted file only and elapsedTime
7993255ms is < ttl=-1
{code}
Main difference is that we don't schedule a major compaction every time through
and then when we go to run it, find that its not needed after all. Instead, as
in above, each major compaction interval, we check and will not run a major if
last one was result of a major.
> Will keep scheduling major compactions if last time one ran, we didn't.
> -----------------------------------------------------------------------
>
> Key: HBASE-1083
> URL: https://issues.apache.org/jira/browse/HBASE-1083
> Project: Hadoop HBase
> Issue Type: Bug
> Reporter: stack
> Assignee: stack
> Fix For: 0.19.0
>
> Attachments: 1083.patch
>
>
> Excerpt below has major compactions scheduled ever few hours but nothing to
> compact each time through -- last run was a major compaction -- and times are
> off. Need to touch files if nothing to compact so major doesn't get
> scheduled again.
> {code}
> 2008-12-20 02:41:03,933 DEBUG org.apache.hadoop.hbase.regionserver.HStore:
> Major compaction triggered on store: 170018656/content. Time since last major
> compaction: 249999 seconds
> 2008-12-20 05:27:43,933 DEBUG org.apache.hadoop.hbase.regionserver.HStore:
> Major compaction triggered on store: 170018656/content. Time since last major
> compaction: 259999 seconds
> 2008-12-20 08:14:23,932 DEBUG org.apache.hadoop.hbase.regionserver.HStore:
> Major compaction triggered on store: 170018656/content. Time since last major
> compaction: 269999 seconds
> 2008-12-20 11:01:03,932 DEBUG org.apache.hadoop.hbase.regionserver.HStore:
> Major compaction triggered on store: 170018656/content. Time since last major
> compaction: 279999 seconds
> 2008-12-20 13:47:43,932 DEBUG org.apache.hadoop.hbase.regionserver.HStore:
> Major compaction triggered on store: 170018656/content. Time since last major
> compaction: 289999 seconds
> 2008-12-20 16:34:23,932 DEBUG org.apache.hadoop.hbase.regionserver.HStore:
> Major compaction triggered on store: 170018656/content. Time since last major
> compaction: 299999 seconds
> 2008-12-20 19:21:03,933 DEBUG org.apache.hadoop.hbase.regionserver.HStore:
> Major compaction triggered on store: 170018656/content. Time since last major
> compaction: 309999 seconds
> 2008-12-20 22:07:43,932 DEBUG org.apache.hadoop.hbase.regionserver.HStore:
> Major compaction triggered on store: 170018656/content. Time since last major
> compaction: 319999 seconds
> {code}
> Then it says:
> {code}
> 2008-12-20 13:47:43,935 DEBUG org.apache.hadoop.hbase.regionserver.HStore:
> Skipping major compaction because only one (major) compacted file only and
> elapsedTime 289999825 is < ttl=-1
> {code}
> I think whats happening is that we're not compacting because we just did but
> major compactions keep getting triggered because we are not updating the file
> timestamp. Look into it. At least fix up logging so its clearer whats
> happening.. Here is snippet:
> {code}
> 2008-12-20 11:01:04,026 DEBUG org.apache.hadoop.hbase.regionserver.HStore:
> Skipping major compaction because only one (major) compacted file only and
> elapsedTime 149999850 is < ttl=-1
> 2008-12-20 11:01:04,026 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> compaction completed on region assigners,,1229364037757 in 0sec
> 2008-12-20 13:47:43,932 DEBUG org.apache.hadoop.hbase.regionserver.HStore:
> Major compaction triggered on store: 170018656/content. Time since last major
> compaction: 289999 seconds
> 2008-12-20 13:47:43,932 DEBUG
> org.apache.hadoop.hbase.regionserver.CompactSplitThread: Compaction requested
> for region streamitems,^...@^@^...@^@^C�r�,1229427582771 because:
> regionserver/0:0:0:0:0:0:0:0:60020.majorCompactionChecker requests major
> compaction
> 2008-12-20 13:47:43,932 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> starting compaction on region streamitems,^...@^@^...@^@^C�r�,1229427582771
> 2008-12-20 13:47:43,934 DEBUG org.apache.hadoop.hbase.regionserver.HStore:
> Major compaction triggered on store: 65151897/content. Time since last major
> compaction: 259999 seconds
> 2008-12-20 13:47:43,934 DEBUG
> org.apache.hadoop.hbase.regionserver.CompactSplitThread: Compaction requested
> for region streamitems,^...@^@^...@^@^C3=�,1229459727659 because:
> regionserver/0:0:0:0:0:0:0:0:60020.majorCompactionChecker requests major
> compaction
> 2008-12-20 13:47:43,935 DEBUG org.apache.hadoop.hbase.regionserver.HStore:
> Major compaction triggered on store: 1579534353/content. Time since last
> major compaction: 259983 seconds
> 2008-12-20 13:47:43,935 DEBUG
> org.apache.hadoop.hbase.regionserver.CompactSplitThread: Compaction requested
> for region streamitems,^...@^@^...@^@^AV^Ds,1229459695202 because:
> regionserver/0:0:0:0:0:0:0:0:60020.majorCompactionChecker requests major
> compaction
> 2008-12-20 13:47:43,935 DEBUG org.apache.hadoop.hbase.regionserver.HStore:
> Skipping major compaction because only one (major) compacted file only and
> elapsedTime 289999825 is < ttl=-1
> 2008-12-20 13:47:43,936 DEBUG org.apache.hadoop.hbase.regionserver.HStore:
> Major compaction triggered on store: 1277711984/content. Time since last
> major compaction: 339999 seconds
> ...
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
