[
https://issues.apache.org/jira/browse/HBASE-8676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13679759#comment-13679759
]
Sergey Shelukhin commented on HBASE-8676:
-----------------------------------------
I think the original intent was to exclude based on the first file under the
assumption that it will be the largest one. It's the same (or even worse given
how selection is done) on trunk now
> hbase.hstore.compaction.max.size parameter has no effect in minor compaction
> ----------------------------------------------------------------------------
>
> Key: HBASE-8676
> URL: https://issues.apache.org/jira/browse/HBASE-8676
> Project: HBase
> Issue Type: Bug
> Components: regionserver
> Affects Versions: 0.94.7
> Reporter: Calvin Qu
>
> In hbase-site.xml,I set(10G)
> <property>
> <name>hbase.hstore.compaction.max.size</name>
> <value>10737418240</value>
> </property>
> but in regionserver's log,a minor compaction deals with 10 Hfiles,include a
> 220G HFile larger than hbase.hstore.compaction.max.size=10G.
> 2013-06-03 10:27:51,147 DEBUG org.apache.hadoop.hbase.regionserver.Store:
> 8b6a4d4aae3099730b353183cf754ec4 - t: Initiating minorcompaction
> 2013-06-03 10:27:51,149 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> Starting compaction on t in region
> transdb,1651763699103243149223370692144515807,1369372490168.8b6a4d4aae3099730b353183cf754ec4.
> 2013-06-03 10:27:51,150 DEBUG
> org.apache.hadoop.hbase.regionserver.CompactSplitThread: Large Compaction
> requested:
> regionName=transdb,1651763699103243149223370692144515807,1369372490168.8b6a4d4aae3099730b353183cf754ec4.,
> storeName=t, fileCount=10, fileSize=234.4g (106.0m, 2.0g, 2.1g, 10.7m, 1.7g,
> 2.1g, 1.9g, 2.0g, 932.8m, 221.5g), priority=-110, time=2671513455305347;
> Because: Opening Region; compaction_queue=(0:0), split_queue=0
> 2013-06-03 10:27:51,151 INFO org.apache.hadoop.hbase.regionserver.Store:
> Starting compaction of 10 file(s) in t of
> transdb,1651763699103243149223370692144515807,1369372490168.8b6a4d4aae3099730b353183cf754ec4.
> into
> tmpdir=hdfs://namenode01:9000/hbase/transdb/8b6a4d4aae3099730b353183cf754ec4/.tmp,
> seqid=0, totalSize=234.4g
> 2013-06-03 10:27:51,152 DEBUG org.apache.hadoop.hbase.regionserver.Compactor:
> Compacting
> hdfs://namenode01:9000/hbase/transdb/8b6a4d4aae3099730b353183cf754ec4/t/0a5fb3caf6be490ba739b77fcaa4b274.c7ce1266ef1f8696ede82f08fceb28ff-hdfs://namenode01:9000/hbase/transdb/c7ce1266ef1f8696ede82f08fceb28ff/t/0a5fb3caf6be490ba739b77fcaa4b274-top,
> keycount=7948080, bloomtype=NONE, size=106.0m, encoding=NONE
> 2013-06-03 10:27:51,153 DEBUG org.apache.hadoop.hbase.regionserver.Compactor:
> Compacting
> hdfs://namenode01:9000/hbase/transdb/8b6a4d4aae3099730b353183cf754ec4/t/6bd849c93e3a45d4a4c8a5ab691f9b22.c7ce1266ef1f8696ede82f08fceb28ff-hdfs://namenode01:9000/hbase/transdb/c7ce1266ef1f8696ede82f08fceb28ff/t/6bd849c93e3a45d4a4c8a5ab691f9b22-top,
> keycount=153911753, bloomtype=NONE, size=2.0g, encoding=NONE
> 2013-06-03 10:27:51,153 DEBUG org.apache.hadoop.hbase.regionserver.Compactor:
> Compacting
> hdfs://namenode01:9000/hbase/transdb/8b6a4d4aae3099730b353183cf754ec4/t/191ef6d1076341e1b7fb5c775d0f9fdb.c7ce1266ef1f8696ede82f08fceb28ff-hdfs://namenode01:9000/hbase/transdb/c7ce1266ef1f8696ede82f08fceb28ff/t/191ef6d1076341e1b7fb5c775d0f9fdb-top,
> keycount=159372725, bloomtype=NONE, size=2.1g, encoding=NONE
> 2013-06-03 10:27:51,154 DEBUG org.apache.hadoop.hbase.regionserver.Compactor:
> Compacting
> hdfs://namenode01:9000/hbase/transdb/8b6a4d4aae3099730b353183cf754ec4/t/d907a7dca7d8466f843f126429f05665.c7ce1266ef1f8696ede82f08fceb28ff-hdfs://namenode01:9000/hbase/transdb/c7ce1266ef1f8696ede82f08fceb28ff/t/d907a7dca7d8466f843f126429f05665-top,
> keycount=808800, bloomtype=NONE, size=10.7m, encoding=NONE
> 2013-06-03 10:27:51,154 DEBUG org.apache.hadoop.hbase.regionserver.Compactor:
> Compacting
> hdfs://namenode01:9000/hbase/transdb/8b6a4d4aae3099730b353183cf754ec4/t/35399e73e3764d0b88251c7285cf5406.c7ce1266ef1f8696ede82f08fceb28ff-hdfs://namenode01:9000/hbase/transdb/c7ce1266ef1f8696ede82f08fceb28ff/t/35399e73e3764d0b88251c7285cf5406-top,
> keycount=133146091, bloomtype=NONE, size=1.7g, encoding=NONE
> 2013-06-03 10:27:51,154 DEBUG org.apache.hadoop.hbase.regionserver.Compactor:
> Compacting
> hdfs://namenode01:9000/hbase/transdb/8b6a4d4aae3099730b353183cf754ec4/t/9d5583ffde334bc79a692d400af7bae0.c7ce1266ef1f8696ede82f08fceb28ff-hdfs://namenode01:9000/hbase/transdb/c7ce1266ef1f8696ede82f08fceb28ff/t/9d5583ffde334bc79a692d400af7bae0-top,
> keycount=159372708, bloomtype=NONE, size=2.1g, encoding=NONE
> 2013-06-03 10:27:51,155 DEBUG org.apache.hadoop.hbase.regionserver.Compactor:
> Compacting
> hdfs://namenode01:9000/hbase/transdb/8b6a4d4aae3099730b353183cf754ec4/t/506254eac1bb455283f578e71af23153.c7ce1266ef1f8696ede82f08fceb28ff-hdfs://namenode01:9000/hbase/transdb/c7ce1266ef1f8696ede82f08fceb28ff/t/506254eac1bb455283f578e71af23153-top,
> keycount=152578349, bloomtype=NONE, size=1.9g, encoding=NONE
> 2013-06-03 10:27:51,155 DEBUG org.apache.hadoop.hbase.regionserver.Compactor:
> Compacting
> hdfs://namenode01:9000/hbase/transdb/8b6a4d4aae3099730b353183cf754ec4/t/72db7a39a2694bf58782e95992b012ae.c7ce1266ef1f8696ede82f08fceb28ff-hdfs://namenode01:9000/hbase/transdb/c7ce1266ef1f8696ede82f08fceb28ff/t/72db7a39a2694bf58782e95992b012ae-top,
> keycount=159369079, bloomtype=NONE, size=2.0g, encoding=NONE
> 2013-06-03 10:27:51,155 DEBUG org.apache.hadoop.hbase.regionserver.Compactor:
> Compacting
> hdfs://namenode01:9000/hbase/transdb/8b6a4d4aae3099730b353183cf754ec4/t/128f5cebbf074129b7c59de3d436b57d.c7ce1266ef1f8696ede82f08fceb28ff-hdfs://namenode01:9000/hbase/transdb/c7ce1266ef1f8696ede82f08fceb28ff/t/128f5cebbf074129b7c59de3d436b57d-top,
> keycount=72366990, bloomtype=NONE, size=932.8m, encoding=NONE
> 2013-06-03 10:27:51,155 DEBUG org.apache.hadoop.hbase.regionserver.Compactor:
> Compacting
> hdfs://namenode01:9000/hbase/transdb/8b6a4d4aae3099730b353183cf754ec4/t/df7ac534daa645589e5e2a6fe1a1a871,
> keycount=17218754243, bloomtype=NONE, size=221.5g, encoding=NONE
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
