[
https://issues.apache.org/jira/browse/HBASE-1083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12659002#action_12659002
]
stack commented on HBASE-1083:
------------------------------
The major compactor thread runs every 10000 seconds by default to check for
major compactions which explains the 10k increments in logs above.
The thread looks for oldest file in region. Does this by looking at
modification time. Mod time does not change so, each time thread runs a major
compaction is scheduled because file is older than major compaction interval.
Then the major compaction goes to run and notices that only one file and that
it was made by a major compaction and then we output the skipping message.
So, a major compaction is scheduled every 10k seconds though we may have asked
that the region only be majorly compacted once a day.
Let me think about how to fix. In hdfs there are two times, mod time and
access time. You can update them but its a trip to the namenode. Access time
is changed everytime blocks of a file are asked for which in our case would be
anytime a reader is opened. We could change the modtime but thats a little
perverse. Add it to the meta file? We already do here to ask if file was
result of major compaction. Should store timestamp rather than yes/no.
> Will keep scheduling major compactions if last time one ran, we didn't.
> -----------------------------------------------------------------------
>
> Key: HBASE-1083
> URL: https://issues.apache.org/jira/browse/HBASE-1083
> Project: Hadoop HBase
> Issue Type: Bug
> Reporter: stack
> Fix For: 0.19.0
>
>
> Excerpt below has major compactions scheduled ever few hours but nothing to
> compact each time through -- last run was a major compaction -- and times are
> off. Need to touch files if nothing to compact so major doesn't get
> scheduled again.
> {code}
> 2008-12-20 02:41:03,933 DEBUG org.apache.hadoop.hbase.regionserver.HStore:
> Major compaction triggered on store: 170018656/content. Time since last major
> compaction: 249999 seconds
> 2008-12-20 05:27:43,933 DEBUG org.apache.hadoop.hbase.regionserver.HStore:
> Major compaction triggered on store: 170018656/content. Time since last major
> compaction: 259999 seconds
> 2008-12-20 08:14:23,932 DEBUG org.apache.hadoop.hbase.regionserver.HStore:
> Major compaction triggered on store: 170018656/content. Time since last major
> compaction: 269999 seconds
> 2008-12-20 11:01:03,932 DEBUG org.apache.hadoop.hbase.regionserver.HStore:
> Major compaction triggered on store: 170018656/content. Time since last major
> compaction: 279999 seconds
> 2008-12-20 13:47:43,932 DEBUG org.apache.hadoop.hbase.regionserver.HStore:
> Major compaction triggered on store: 170018656/content. Time since last major
> compaction: 289999 seconds
> 2008-12-20 16:34:23,932 DEBUG org.apache.hadoop.hbase.regionserver.HStore:
> Major compaction triggered on store: 170018656/content. Time since last major
> compaction: 299999 seconds
> 2008-12-20 19:21:03,933 DEBUG org.apache.hadoop.hbase.regionserver.HStore:
> Major compaction triggered on store: 170018656/content. Time since last major
> compaction: 309999 seconds
> 2008-12-20 22:07:43,932 DEBUG org.apache.hadoop.hbase.regionserver.HStore:
> Major compaction triggered on store: 170018656/content. Time since last major
> compaction: 319999 seconds
> {code}
> Then it says:
> {code}
> 2008-12-20 13:47:43,935 DEBUG org.apache.hadoop.hbase.regionserver.HStore:
> Skipping major compaction because only one (major) compacted file only and
> elapsedTime 289999825 is < ttl=-1
> {code}
> I think whats happening is that we're not compacting because we just did but
> major compactions keep getting triggered because we are not updating the file
> timestamp. Look into it. At least fix up logging so its clearer whats
> happening.. Here is snippet:
> {code}
> 2008-12-20 11:01:04,026 DEBUG org.apache.hadoop.hbase.regionserver.HStore:
> Skipping major compaction because only one (major) compacted file only and
> elapsedTime 149999850 is < ttl=-1
> 2008-12-20 11:01:04,026 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> compaction completed on region assigners,,1229364037757 in 0sec
> 2008-12-20 13:47:43,932 DEBUG org.apache.hadoop.hbase.regionserver.HStore:
> Major compaction triggered on store: 170018656/content. Time since last major
> compaction: 289999 seconds
> 2008-12-20 13:47:43,932 DEBUG
> org.apache.hadoop.hbase.regionserver.CompactSplitThread: Compaction requested
> for region streamitems,^...@^@^...@^@^C�r�,1229427582771 because:
> regionserver/0:0:0:0:0:0:0:0:60020.majorCompactionChecker requests major
> compaction
> 2008-12-20 13:47:43,932 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> starting compaction on region streamitems,^...@^@^...@^@^C�r�,1229427582771
> 2008-12-20 13:47:43,934 DEBUG org.apache.hadoop.hbase.regionserver.HStore:
> Major compaction triggered on store: 65151897/content. Time since last major
> compaction: 259999 seconds
> 2008-12-20 13:47:43,934 DEBUG
> org.apache.hadoop.hbase.regionserver.CompactSplitThread: Compaction requested
> for region streamitems,^...@^@^...@^@^C3=�,1229459727659 because:
> regionserver/0:0:0:0:0:0:0:0:60020.majorCompactionChecker requests major
> compaction
> 2008-12-20 13:47:43,935 DEBUG org.apache.hadoop.hbase.regionserver.HStore:
> Major compaction triggered on store: 1579534353/content. Time since last
> major compaction: 259983 seconds
> 2008-12-20 13:47:43,935 DEBUG
> org.apache.hadoop.hbase.regionserver.CompactSplitThread: Compaction requested
> for region streamitems,^...@^@^...@^@^AV^Ds,1229459695202 because:
> regionserver/0:0:0:0:0:0:0:0:60020.majorCompactionChecker requests major
> compaction
> 2008-12-20 13:47:43,935 DEBUG org.apache.hadoop.hbase.regionserver.HStore:
> Skipping major compaction because only one (major) compacted file only and
> elapsedTime 289999825 is < ttl=-1
> 2008-12-20 13:47:43,936 DEBUG org.apache.hadoop.hbase.regionserver.HStore:
> Major compaction triggered on store: 1277711984/content. Time since last
> major compaction: 339999 seconds
> ...
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
