[ https://issues.apache.org/jira/browse/HBASE-3690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13148228#comment-13148228 ]
Hudson commented on HBASE-3690: ------------------------------- Integrated in HBase-TRUNK #2427 (See [https://builds.apache.org/job/HBase-TRUNK/2427/]) HBASE-3690 Option to Exclude Bulk Import Files from Minor Compaction Summary: We ran an incremental scrape with HFileOutputFormat and encountered major compaction storms. This is caused by the bug in HBASE-3404. The permanent fix is a little tricky without HBASE-2856. We realized that a quicker solution for avoiding these compaction storms is to simply exclude bulk import files from minor compactions and let them only be handled by time-based major compactions. Add with functionality along with a config option to enable it. Rewrote this feature to be done on a per-bulkload basis. Test Plan: - mvn test -Dtest=TestHFileOutputFormat DiffCamp Revision: Reviewers: stack, Kannan, JIRA, dhruba Reviewed By: stack CC: dhruba, lhofhansl, nspiegelberg, stack Differential Revision: 357 nspiegelberg : Files : * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/mapreduce/HFileOutputFormat.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHFileOutputFormat.java > Option to Exclude Bulk Import Files from Minor Compaction > --------------------------------------------------------- > > Key: HBASE-3690 > URL: https://issues.apache.org/jira/browse/HBASE-3690 > Project: HBase > Issue Type: Bug > Components: regionserver > Reporter: Nicolas Spiegelberg > Assignee: Nicolas Spiegelberg > Priority: Minor > Fix For: 0.94.0 > > Attachments: D357.1.patch, HBASE-3690.patch > > > We ran an incremental scrape with HFileOutputFormat and encountered major > compaction storms. This is caused by the bug in HBASE-3404. The permanent > fix is a little tricky without HBASE-2856. We realized that a quicker > solution for avoiding these compaction storms is to simply exclude bulk > import files from minor compactions and let them only be handled by > time-based major compactions. Add with functionality along with a config > option to enable it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira