[jira] [Updated] (HBASE-5332) Deterministic Compaction Jitter
[ https://issues.apache.org/jira/browse/HBASE-5332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg updated HBASE-5332: --- Status: Patch Available (was: Open) resubmitting after creating trunk-specific patch file > Deterministic Compaction Jitter > --- > > Key: HBASE-5332 > URL: https://issues.apache.org/jira/browse/HBASE-5332 > Project: HBase > Issue Type: Improvement >Reporter: Nicolas Spiegelberg >Assignee: Nicolas Spiegelberg >Priority: Minor > Fix For: 0.94.0 > > Attachments: D1785.1.patch, D1785.2.patch, D1785.3.patch, > HBASE-5332.patch > > > Currently, we add jitter to a compaction using "delay + jitter*(1 - > 2*Math.random())". Since this is non-deterministic, we can get major > compaction storms on server restart as half the Stores that were set to > "delay + jitter" will now be set to "delay - jitter". We need a more > deterministic way to jitter major compactions so this information can persist > across server restarts. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5332) Deterministic Compaction Jitter
[ https://issues.apache.org/jira/browse/HBASE-5332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg updated HBASE-5332: --- Resolution: Fixed Status: Resolved (was: Patch Available) > Deterministic Compaction Jitter > --- > > Key: HBASE-5332 > URL: https://issues.apache.org/jira/browse/HBASE-5332 > Project: HBase > Issue Type: Improvement >Reporter: Nicolas Spiegelberg >Assignee: Nicolas Spiegelberg >Priority: Minor > Fix For: 0.94.0 > > Attachments: D1785.1.patch, D1785.2.patch, D1785.3.patch, > HBASE-5332.patch > > > Currently, we add jitter to a compaction using "delay + jitter*(1 - > 2*Math.random())". Since this is non-deterministic, we can get major > compaction storms on server restart as half the Stores that were set to > "delay + jitter" will now be set to "delay - jitter". We need a more > deterministic way to jitter major compactions so this information can persist > across server restarts. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5332) Deterministic Compaction Jitter
[ https://issues.apache.org/jira/browse/HBASE-5332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg updated HBASE-5332: --- Attachment: HBASE-5332.patch trunk patch > Deterministic Compaction Jitter > --- > > Key: HBASE-5332 > URL: https://issues.apache.org/jira/browse/HBASE-5332 > Project: HBase > Issue Type: Improvement >Reporter: Nicolas Spiegelberg >Assignee: Nicolas Spiegelberg >Priority: Minor > Fix For: 0.94.0 > > Attachments: D1785.1.patch, D1785.2.patch, D1785.3.patch, > HBASE-5332.patch > > > Currently, we add jitter to a compaction using "delay + jitter*(1 - > 2*Math.random())". Since this is non-deterministic, we can get major > compaction storms on server restart as half the Stores that were set to > "delay + jitter" will now be set to "delay - jitter". We need a more > deterministic way to jitter major compactions so this information can persist > across server restarts. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5332) Deterministic Compaction Jitter
[ https://issues.apache.org/jira/browse/HBASE-5332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg updated HBASE-5332: --- Status: Open (was: Patch Available) > Deterministic Compaction Jitter > --- > > Key: HBASE-5332 > URL: https://issues.apache.org/jira/browse/HBASE-5332 > Project: HBase > Issue Type: Improvement >Reporter: Nicolas Spiegelberg >Assignee: Nicolas Spiegelberg >Priority: Minor > Fix For: 0.94.0 > > Attachments: D1785.1.patch, D1785.2.patch, D1785.3.patch, > HBASE-5332.patch > > > Currently, we add jitter to a compaction using "delay + jitter*(1 - > 2*Math.random())". Since this is non-deterministic, we can get major > compaction storms on server restart as half the Stores that were set to > "delay + jitter" will now be set to "delay - jitter". We need a more > deterministic way to jitter major compactions so this information can persist > across server restarts. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5332) Deterministic Compaction Jitter
[ https://issues.apache.org/jira/browse/HBASE-5332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg updated HBASE-5332: --- Fix Version/s: 0.94.0 Status: Patch Available (was: Open) > Deterministic Compaction Jitter > --- > > Key: HBASE-5332 > URL: https://issues.apache.org/jira/browse/HBASE-5332 > Project: HBase > Issue Type: Improvement >Reporter: Nicolas Spiegelberg >Assignee: Nicolas Spiegelberg >Priority: Minor > Fix For: 0.94.0 > > Attachments: D1785.1.patch, D1785.2.patch, D1785.3.patch > > > Currently, we add jitter to a compaction using "delay + jitter*(1 - > 2*Math.random())". Since this is non-deterministic, we can get major > compaction storms on server restart as half the Stores that were set to > "delay + jitter" will now be set to "delay - jitter". We need a more > deterministic way to jitter major compactions so this information can persist > across server restarts. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5332) Deterministic Compaction Jitter
[ https://issues.apache.org/jira/browse/HBASE-5332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phabricator updated HBASE-5332: --- Attachment: D1785.3.patch nspiegelberg updated the revision "[jira] [HBASE-5332] Deterministic Compaction Jitter". Reviewers: JIRA, Kannan, aaiyer, stack Addressed Kannan's peer review. Raised compaction delay because of false positives when the jitter brought it to 3 sec. REVISION DETAIL https://reviews.facebook.net/D1785 AFFECTED FILES src/main/java/org/apache/hadoop/hbase/regionserver/Store.java src/test/java/org/apache/hadoop/hbase/regionserver/TestCompactSelection.java src/test/java/org/apache/hadoop/hbase/regionserver/TestCompaction.java > Deterministic Compaction Jitter > --- > > Key: HBASE-5332 > URL: https://issues.apache.org/jira/browse/HBASE-5332 > Project: HBase > Issue Type: Improvement >Reporter: Nicolas Spiegelberg >Assignee: Nicolas Spiegelberg >Priority: Minor > Attachments: D1785.1.patch, D1785.2.patch, D1785.3.patch > > > Currently, we add jitter to a compaction using "delay + jitter*(1 - > 2*Math.random())". Since this is non-deterministic, we can get major > compaction storms on server restart as half the Stores that were set to > "delay + jitter" will now be set to "delay - jitter". We need a more > deterministic way to jitter major compactions so this information can persist > across server restarts. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5332) Deterministic Compaction Jitter
[ https://issues.apache.org/jira/browse/HBASE-5332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phabricator updated HBASE-5332: --- Attachment: D1785.2.patch nspiegelberg updated the revision "[jira] [HBASE-5332] Deterministic Compaction Jitter". Reviewers: JIRA, Kannan, aaiyer, stack Remove unnecessary logs added during debug REVISION DETAIL https://reviews.facebook.net/D1785 AFFECTED FILES src/main/java/org/apache/hadoop/hbase/regionserver/Store.java src/test/java/org/apache/hadoop/hbase/regionserver/TestCompactSelection.java src/test/java/org/apache/hadoop/hbase/regionserver/TestCompaction.java > Deterministic Compaction Jitter > --- > > Key: HBASE-5332 > URL: https://issues.apache.org/jira/browse/HBASE-5332 > Project: HBase > Issue Type: Improvement >Reporter: Nicolas Spiegelberg >Assignee: Nicolas Spiegelberg >Priority: Minor > Attachments: D1785.1.patch, D1785.2.patch > > > Currently, we add jitter to a compaction using "delay + jitter*(1 - > 2*Math.random())". Since this is non-deterministic, we can get major > compaction storms on server restart as half the Stores that were set to > "delay + jitter" will now be set to "delay - jitter". We need a more > deterministic way to jitter major compactions so this information can persist > across server restarts. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5332) Deterministic Compaction Jitter
[ https://issues.apache.org/jira/browse/HBASE-5332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phabricator updated HBASE-5332: --- Attachment: D1785.1.patch nspiegelberg requested code review of "[jira] [HBASE-5332] Deterministic Compaction Jitter". Reviewers: JIRA, Kannan, aaiyer, stack Changing the jitter in major compactions to be deterministic, so reboots don't cause a time-based compaction storm. This implementation seeds a random number generator with HDFS data for persistence. TEST PLAN - mvn test -Dtest=TestCompaction,TestCompactSelection,TestHeapSize REVISION DETAIL https://reviews.facebook.net/D1785 AFFECTED FILES src/main/java/org/apache/hadoop/hbase/regionserver/Store.java src/test/java/org/apache/hadoop/hbase/regionserver/TestCompactSelection.java src/test/java/org/apache/hadoop/hbase/regionserver/TestCompaction.java MANAGE HERALD DIFFERENTIAL RULES https://reviews.facebook.net/herald/view/differential/ WHY DID I GET THIS EMAIL? https://reviews.facebook.net/herald/transcript/3807/ Tip: use the X-Herald-Rules header to filter Herald messages in your client. > Deterministic Compaction Jitter > --- > > Key: HBASE-5332 > URL: https://issues.apache.org/jira/browse/HBASE-5332 > Project: HBase > Issue Type: Improvement >Reporter: Nicolas Spiegelberg >Assignee: Nicolas Spiegelberg >Priority: Minor > Attachments: D1785.1.patch > > > Currently, we add jitter to a compaction using "delay + jitter*(1 - > 2*Math.random())". Since this is non-deterministic, we can get major > compaction storms on server restart as half the Stores that were set to > "delay + jitter" will now be set to "delay - jitter". We need a more > deterministic way to jitter major compactions so this information can persist > across server restarts. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
