[jira] [Updated] (HBASE-5335) Dynamic Schema Configurations
[ https://issues.apache.org/jira/browse/HBASE-5335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg updated HBASE-5335: --- Comment: was deleted (was: Status update 1. Fixing the 3 unit test failures, caused by new split code between 89fb - trunk. 2. Fixed some ruby display/parsing problems found during manual testing. We need a way to unit test the Ruby shell code easily. 3. Taking the time to fix an issue that constantly annoys me: you should be able to run 'describe TABLE', copy the output, and paste it directly into 'create' or 'alter' ) Dynamic Schema Configurations - Key: HBASE-5335 URL: https://issues.apache.org/jira/browse/HBASE-5335 Project: HBase Issue Type: New Feature Reporter: Nicolas Spiegelberg Assignee: Nicolas Spiegelberg Labels: configuration, schema Fix For: 0.96.0 Attachments: D2247.1.patch, D2247.2.patch, D2247.3.patch, D2247.4.patch, D2247.5.patch, D2247.6.patch, D2247.7.patch, D2247.8.patch, HBASE-5335-trunk-2.patch, HBASE-5335-trunk.patch Currently, the ability for a core developer to add per-table per-CF configuration settings is very heavyweight. You need to add a reserved keyword all the way up the stack you have to support this variable long-term if you're going to expose it explicitly to the user. This has ended up with using Configuration.get() a lot because it is lightweight and you can tweak settings while you're trying to understand system behavior [since there are many config params that may never need to be tuned]. We need to add the ability to put read arbitrary KV settings in the HBase schema. Combined with online schema change, this will allow us to safely iterate on configuration settings. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5335) Dynamic Schema Configurations
[ https://issues.apache.org/jira/browse/HBASE-5335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg updated HBASE-5335: --- Status: Open (was: Patch Available) Dynamic Schema Configurations - Key: HBASE-5335 URL: https://issues.apache.org/jira/browse/HBASE-5335 Project: HBase Issue Type: New Feature Reporter: Nicolas Spiegelberg Assignee: Nicolas Spiegelberg Labels: configuration, schema Fix For: 0.96.0 Attachments: D2247.1.patch, D2247.2.patch, D2247.3.patch, D2247.4.patch, D2247.5.patch, D2247.6.patch, D2247.7.patch, D2247.8.patch, HBASE-5335-trunk-2.patch, HBASE-5335-trunk-3.patch, HBASE-5335-trunk.patch Currently, the ability for a core developer to add per-table per-CF configuration settings is very heavyweight. You need to add a reserved keyword all the way up the stack you have to support this variable long-term if you're going to expose it explicitly to the user. This has ended up with using Configuration.get() a lot because it is lightweight and you can tweak settings while you're trying to understand system behavior [since there are many config params that may never need to be tuned]. We need to add the ability to put read arbitrary KV settings in the HBase schema. Combined with online schema change, this will allow us to safely iterate on configuration settings. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5335) Dynamic Schema Configurations
[ https://issues.apache.org/jira/browse/HBASE-5335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg updated HBASE-5335: --- Status: Patch Available (was: Open) Dynamic Schema Configurations - Key: HBASE-5335 URL: https://issues.apache.org/jira/browse/HBASE-5335 Project: HBase Issue Type: New Feature Reporter: Nicolas Spiegelberg Assignee: Nicolas Spiegelberg Labels: configuration, schema Fix For: 0.96.0 Attachments: D2247.1.patch, D2247.2.patch, D2247.3.patch, D2247.4.patch, D2247.5.patch, D2247.6.patch, D2247.7.patch, D2247.8.patch, HBASE-5335-trunk-2.patch, HBASE-5335-trunk-3.patch, HBASE-5335-trunk.patch Currently, the ability for a core developer to add per-table per-CF configuration settings is very heavyweight. You need to add a reserved keyword all the way up the stack you have to support this variable long-term if you're going to expose it explicitly to the user. This has ended up with using Configuration.get() a lot because it is lightweight and you can tweak settings while you're trying to understand system behavior [since there are many config params that may never need to be tuned]. We need to add the ability to put read arbitrary KV settings in the HBase schema. Combined with online schema change, this will allow us to safely iterate on configuration settings. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5335) Dynamic Schema Configurations
[ https://issues.apache.org/jira/browse/HBASE-5335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg updated HBASE-5335: --- Status: Open (was: Patch Available) f*** Hadoop QA needs to learn patch -p1 Dynamic Schema Configurations - Key: HBASE-5335 URL: https://issues.apache.org/jira/browse/HBASE-5335 Project: HBase Issue Type: New Feature Reporter: Nicolas Spiegelberg Assignee: Nicolas Spiegelberg Labels: configuration, schema Fix For: 0.96.0 Attachments: D2247.1.patch, D2247.2.patch, D2247.3.patch, D2247.4.patch, D2247.5.patch, D2247.6.patch, D2247.7.patch, D2247.8.patch, HBASE-5335-trunk-2.patch, HBASE-5335-trunk-3.patch, HBASE-5335-trunk-3.patch, HBASE-5335-trunk.patch Currently, the ability for a core developer to add per-table per-CF configuration settings is very heavyweight. You need to add a reserved keyword all the way up the stack you have to support this variable long-term if you're going to expose it explicitly to the user. This has ended up with using Configuration.get() a lot because it is lightweight and you can tweak settings while you're trying to understand system behavior [since there are many config params that may never need to be tuned]. We need to add the ability to put read arbitrary KV settings in the HBase schema. Combined with online schema change, this will allow us to safely iterate on configuration settings. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5335) Dynamic Schema Configurations
[ https://issues.apache.org/jira/browse/HBASE-5335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg updated HBASE-5335: --- Attachment: HBASE-5335-trunk-3.patch Dynamic Schema Configurations - Key: HBASE-5335 URL: https://issues.apache.org/jira/browse/HBASE-5335 Project: HBase Issue Type: New Feature Reporter: Nicolas Spiegelberg Assignee: Nicolas Spiegelberg Labels: configuration, schema Fix For: 0.96.0 Attachments: D2247.1.patch, D2247.2.patch, D2247.3.patch, D2247.4.patch, D2247.5.patch, D2247.6.patch, D2247.7.patch, D2247.8.patch, HBASE-5335-trunk-2.patch, HBASE-5335-trunk-3.patch, HBASE-5335-trunk-3.patch, HBASE-5335-trunk.patch Currently, the ability for a core developer to add per-table per-CF configuration settings is very heavyweight. You need to add a reserved keyword all the way up the stack you have to support this variable long-term if you're going to expose it explicitly to the user. This has ended up with using Configuration.get() a lot because it is lightweight and you can tweak settings while you're trying to understand system behavior [since there are many config params that may never need to be tuned]. We need to add the ability to put read arbitrary KV settings in the HBase schema. Combined with online schema change, this will allow us to safely iterate on configuration settings. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5335) Dynamic Schema Configurations
[ https://issues.apache.org/jira/browse/HBASE-5335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg updated HBASE-5335: --- Status: Patch Available (was: Open) Dynamic Schema Configurations - Key: HBASE-5335 URL: https://issues.apache.org/jira/browse/HBASE-5335 Project: HBase Issue Type: New Feature Reporter: Nicolas Spiegelberg Assignee: Nicolas Spiegelberg Labels: configuration, schema Fix For: 0.96.0 Attachments: D2247.1.patch, D2247.2.patch, D2247.3.patch, D2247.4.patch, D2247.5.patch, D2247.6.patch, D2247.7.patch, D2247.8.patch, HBASE-5335-trunk-2.patch, HBASE-5335-trunk-3.patch, HBASE-5335-trunk-3.patch, HBASE-5335-trunk.patch Currently, the ability for a core developer to add per-table per-CF configuration settings is very heavyweight. You need to add a reserved keyword all the way up the stack you have to support this variable long-term if you're going to expose it explicitly to the user. This has ended up with using Configuration.get() a lot because it is lightweight and you can tweak settings while you're trying to understand system behavior [since there are many config params that may never need to be tuned]. We need to add the ability to put read arbitrary KV settings in the HBase schema. Combined with online schema change, this will allow us to safely iterate on configuration settings. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5743) Support GIT patches
[ https://issues.apache.org/jira/browse/HBASE-5743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg updated HBASE-5743: --- Attachment: HBASE-5743.patch Test plan: 1. ./dev-support/smart-apply-patch.sh HBASE-5335-trunk-3.patch 2. git show 5335-trunk | PATCH=echo dev-support/smart-apply-patch.sh - Support GIT patches --- Key: HBASE-5743 URL: https://issues.apache.org/jira/browse/HBASE-5743 Project: HBase Issue Type: Bug Reporter: Nicolas Spiegelberg Assignee: Nicolas Spiegelberg Attachments: HBASE-5743.patch Need to import HADOOP-7384 into our version of Hadoop QA to allow test-patch to be more flexible about patch format -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5743) Support GIT patches
[ https://issues.apache.org/jira/browse/HBASE-5743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg updated HBASE-5743: --- Attachment: HBASE-5743-2.patch using smart-apply-patch file from hadoop trunk instead of the older version from HADOOP-7384. This runs 'patch' in dry-run mode to determine prefix level, which is a far more fault-tolerant way of solving the problem. Support GIT patches --- Key: HBASE-5743 URL: https://issues.apache.org/jira/browse/HBASE-5743 Project: HBase Issue Type: Bug Reporter: Nicolas Spiegelberg Assignee: Nicolas Spiegelberg Attachments: HBASE-5743-2.patch, HBASE-5743.patch Need to import HADOOP-7384 into our version of Hadoop QA to allow test-patch to be more flexible about patch format -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5335) Dynamic Schema Configurations
[ https://issues.apache.org/jira/browse/HBASE-5335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg updated HBASE-5335: --- Status: Open (was: Patch Available) Dynamic Schema Configurations - Key: HBASE-5335 URL: https://issues.apache.org/jira/browse/HBASE-5335 Project: HBase Issue Type: New Feature Reporter: Nicolas Spiegelberg Assignee: Nicolas Spiegelberg Labels: configuration, schema Fix For: 0.96.0 Attachments: D2247.1.patch, D2247.2.patch, D2247.3.patch, D2247.4.patch, D2247.5.patch, D2247.6.patch, D2247.7.patch, D2247.8.patch, HBASE-5335-trunk-2.patch, HBASE-5335-trunk-3.patch, HBASE-5335-trunk-3.patch, HBASE-5335-trunk-4.patch, HBASE-5335-trunk.patch Currently, the ability for a core developer to add per-table per-CF configuration settings is very heavyweight. You need to add a reserved keyword all the way up the stack you have to support this variable long-term if you're going to expose it explicitly to the user. This has ended up with using Configuration.get() a lot because it is lightweight and you can tweak settings while you're trying to understand system behavior [since there are many config params that may never need to be tuned]. We need to add the ability to put read arbitrary KV settings in the HBase schema. Combined with online schema change, this will allow us to safely iterate on configuration settings. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5335) Dynamic Schema Configurations
[ https://issues.apache.org/jira/browse/HBASE-5335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg updated HBASE-5335: --- Attachment: HBASE-5335-trunk-4.patch fixed unit test failure. also fixed HBASE-5743, so the git version of this patch should apply in Hadoop QA Dynamic Schema Configurations - Key: HBASE-5335 URL: https://issues.apache.org/jira/browse/HBASE-5335 Project: HBase Issue Type: New Feature Reporter: Nicolas Spiegelberg Assignee: Nicolas Spiegelberg Labels: configuration, schema Fix For: 0.96.0 Attachments: D2247.1.patch, D2247.2.patch, D2247.3.patch, D2247.4.patch, D2247.5.patch, D2247.6.patch, D2247.7.patch, D2247.8.patch, HBASE-5335-trunk-2.patch, HBASE-5335-trunk-3.patch, HBASE-5335-trunk-3.patch, HBASE-5335-trunk-4.patch, HBASE-5335-trunk.patch Currently, the ability for a core developer to add per-table per-CF configuration settings is very heavyweight. You need to add a reserved keyword all the way up the stack you have to support this variable long-term if you're going to expose it explicitly to the user. This has ended up with using Configuration.get() a lot because it is lightweight and you can tweak settings while you're trying to understand system behavior [since there are many config params that may never need to be tuned]. We need to add the ability to put read arbitrary KV settings in the HBase schema. Combined with online schema change, this will allow us to safely iterate on configuration settings. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5335) Dynamic Schema Configurations
[ https://issues.apache.org/jira/browse/HBASE-5335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg updated HBASE-5335: --- Status: Patch Available (was: Open) Dynamic Schema Configurations - Key: HBASE-5335 URL: https://issues.apache.org/jira/browse/HBASE-5335 Project: HBase Issue Type: New Feature Reporter: Nicolas Spiegelberg Assignee: Nicolas Spiegelberg Labels: configuration, schema Fix For: 0.96.0 Attachments: D2247.1.patch, D2247.2.patch, D2247.3.patch, D2247.4.patch, D2247.5.patch, D2247.6.patch, D2247.7.patch, D2247.8.patch, HBASE-5335-trunk-2.patch, HBASE-5335-trunk-3.patch, HBASE-5335-trunk-3.patch, HBASE-5335-trunk-4.patch, HBASE-5335-trunk.patch Currently, the ability for a core developer to add per-table per-CF configuration settings is very heavyweight. You need to add a reserved keyword all the way up the stack you have to support this variable long-term if you're going to expose it explicitly to the user. This has ended up with using Configuration.get() a lot because it is lightweight and you can tweak settings while you're trying to understand system behavior [since there are many config params that may never need to be tuned]. We need to add the ability to put read arbitrary KV settings in the HBase schema. Combined with online schema change, this will allow us to safely iterate on configuration settings. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5359) Alter in the shell can be too quick and return before the table is altered
[ https://issues.apache.org/jira/browse/HBASE-5359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg updated HBASE-5359: --- Attachment: HBASE-5359.patch This is a initial patch that compiles/works/fixes HBASE-5335, but is not extensively tested. Note that this can also be extended to support addColumn and other TableEventHandlers. We should probably also employ a similar technique for enableTable/disableTable. Feel free to run with this or ask questions. It's pretty simple: just add a conditional variable to wait for ZK persist (or Exception). Alter in the shell can be too quick and return before the table is altered -- Key: HBASE-5359 URL: https://issues.apache.org/jira/browse/HBASE-5359 Project: HBase Issue Type: Bug Affects Versions: 0.92.0 Reporter: Jean-Daniel Cryans Fix For: 0.92.2 Attachments: HBASE-5359.patch This seems to be a recent change in behavior but I'm still not sure where it's coming from. The shell is able to call HMaster.getAlterStatus before the TableEventHandler is able call AM.setRegionsToReopen so that the returned status shows no pending regions. It means that the alter seems instantaneous although it's far from completed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5359) Alter in the shell can be too quick and return before the table is altered
[ https://issues.apache.org/jira/browse/HBASE-5359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg updated HBASE-5359: --- Fix Version/s: (was: 0.92.2) 0.96.0 Status: Patch Available (was: Open) only tested on trunk, should be trivial to backport to 0.94 Alter in the shell can be too quick and return before the table is altered -- Key: HBASE-5359 URL: https://issues.apache.org/jira/browse/HBASE-5359 Project: HBase Issue Type: Bug Affects Versions: 0.92.0 Reporter: Jean-Daniel Cryans Fix For: 0.96.0 Attachments: HBASE-5359.patch This seems to be a recent change in behavior but I'm still not sure where it's coming from. The shell is able to call HMaster.getAlterStatus before the TableEventHandler is able call AM.setRegionsToReopen so that the returned status shows no pending regions. It means that the alter seems instantaneous although it's far from completed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5335) Dynamic Schema Configurations
[ https://issues.apache.org/jira/browse/HBASE-5335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg updated HBASE-5335: --- Status: Open (was: Patch Available) Dynamic Schema Configurations - Key: HBASE-5335 URL: https://issues.apache.org/jira/browse/HBASE-5335 Project: HBase Issue Type: New Feature Reporter: Nicolas Spiegelberg Assignee: Nicolas Spiegelberg Labels: configuration, schema Fix For: 0.96.0 Attachments: D2247.1.patch, D2247.2.patch, D2247.3.patch, D2247.4.patch, D2247.5.patch, D2247.6.patch, D2247.7.patch, HBASE-5335-trunk.patch Currently, the ability for a core developer to add per-table per-CF configuration settings is very heavyweight. You need to add a reserved keyword all the way up the stack you have to support this variable long-term if you're going to expose it explicitly to the user. This has ended up with using Configuration.get() a lot because it is lightweight and you can tweak settings while you're trying to understand system behavior [since there are many config params that may never need to be tuned]. We need to add the ability to put read arbitrary KV settings in the HBase schema. Combined with online schema change, this will allow us to safely iterate on configuration settings. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5335) Dynamic Schema Configurations
[ https://issues.apache.org/jira/browse/HBASE-5335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg updated HBASE-5335: --- Attachment: HBASE-5335-trunk-2.patch Version 2 of the trunk patch. Major changes: 1) Use the keyword CONFIG instead of ADVANCED. This should be a little more straightforward to understand. 2) Move CompoundConfiguration into the regionserver namespace and make it a package private class with a private interface audience. This will prevent other people from using this transitory class and avoid documentation. This patch will fail on TestFromClientSide3 until HBASE-5359 is committed. Dynamic Schema Configurations - Key: HBASE-5335 URL: https://issues.apache.org/jira/browse/HBASE-5335 Project: HBase Issue Type: New Feature Reporter: Nicolas Spiegelberg Assignee: Nicolas Spiegelberg Labels: configuration, schema Fix For: 0.96.0 Attachments: D2247.1.patch, D2247.2.patch, D2247.3.patch, D2247.4.patch, D2247.5.patch, D2247.6.patch, D2247.7.patch, HBASE-5335-trunk-2.patch, HBASE-5335-trunk.patch Currently, the ability for a core developer to add per-table per-CF configuration settings is very heavyweight. You need to add a reserved keyword all the way up the stack you have to support this variable long-term if you're going to expose it explicitly to the user. This has ended up with using Configuration.get() a lot because it is lightweight and you can tweak settings while you're trying to understand system behavior [since there are many config params that may never need to be tuned]. We need to add the ability to put read arbitrary KV settings in the HBase schema. Combined with online schema change, this will allow us to safely iterate on configuration settings. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5335) Dynamic Schema Configurations
[ https://issues.apache.org/jira/browse/HBASE-5335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg updated HBASE-5335: --- Status: Patch Available (was: In Progress) Dynamic Schema Configurations - Key: HBASE-5335 URL: https://issues.apache.org/jira/browse/HBASE-5335 Project: HBase Issue Type: New Feature Reporter: Nicolas Spiegelberg Assignee: Nicolas Spiegelberg Labels: configuration, schema Fix For: 0.96.0 Attachments: D2247.1.patch, D2247.2.patch, D2247.3.patch, D2247.4.patch, D2247.5.patch, D2247.6.patch, D2247.7.patch, HBASE-5335-trunk-2.patch, HBASE-5335-trunk.patch Currently, the ability for a core developer to add per-table per-CF configuration settings is very heavyweight. You need to add a reserved keyword all the way up the stack you have to support this variable long-term if you're going to expose it explicitly to the user. This has ended up with using Configuration.get() a lot because it is lightweight and you can tweak settings while you're trying to understand system behavior [since there are many config params that may never need to be tuned]. We need to add the ability to put read arbitrary KV settings in the HBase schema. Combined with online schema change, this will allow us to safely iterate on configuration settings. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5359) Alter in the shell can be too quick and return before the table is altered
[ https://issues.apache.org/jira/browse/HBASE-5359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg updated HBASE-5359: --- Resolution: Fixed Assignee: Nicolas Spiegelberg Status: Resolved (was: Patch Available) fixed on trunk. should probably port to 94 Alter in the shell can be too quick and return before the table is altered -- Key: HBASE-5359 URL: https://issues.apache.org/jira/browse/HBASE-5359 Project: HBase Issue Type: Bug Affects Versions: 0.92.0 Reporter: Jean-Daniel Cryans Assignee: Nicolas Spiegelberg Fix For: 0.96.0 Attachments: HBASE-5359.patch This seems to be a recent change in behavior but I'm still not sure where it's coming from. The shell is able to call HMaster.getAlterStatus before the TableEventHandler is able call AM.setRegionsToReopen so that the returned status shows no pending regions. It means that the alter seems instantaneous although it's far from completed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5626) Compactions simulator tool for proofing algorithms
[ https://issues.apache.org/jira/browse/HBASE-5626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg updated HBASE-5626: --- Attachment: cf_compact.py Attached the current python script that I use to emulate compactions given different params. Compactions simulator tool for proofing algorithms -- Key: HBASE-5626 URL: https://issues.apache.org/jira/browse/HBASE-5626 Project: HBase Issue Type: Task Reporter: stack Priority: Minor Labels: noob Attachments: cf_compact.py A tool to run compaction simulations would be a nice to have. We could use it to see how well an algo ran under different circumstances loaded w/ different value types with different rates of flushes and splits, etc. HBASE-2462 had one (see in patch). Or we could try doing it using something like this: http://en.wikipedia.org/wiki/Discrete_event_simulation -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5335) Dynamic Schema Configurations
[ https://issues.apache.org/jira/browse/HBASE-5335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg updated HBASE-5335: --- Attachment: HBASE-5335-trunk.patch Initial patch for trunk. I think I should change 'ADVANCED' to config and a couple other minor things, so please don't commit. This is for hadoop QA plus for you to fiddle with. Note that TestFromClientSide3 should fail until HBASE-5359 is committed. I made a preliminary patch for HBASE-5359 and verified that it fixed the test. Dynamic Schema Configurations - Key: HBASE-5335 URL: https://issues.apache.org/jira/browse/HBASE-5335 Project: HBase Issue Type: New Feature Reporter: Nicolas Spiegelberg Assignee: Nicolas Spiegelberg Labels: configuration, schema Attachments: D2247.1.patch, D2247.2.patch, D2247.3.patch, D2247.4.patch, D2247.5.patch, D2247.6.patch, D2247.7.patch, HBASE-5335-trunk.patch Currently, the ability for a core developer to add per-table per-CF configuration settings is very heavyweight. You need to add a reserved keyword all the way up the stack you have to support this variable long-term if you're going to expose it explicitly to the user. This has ended up with using Configuration.get() a lot because it is lightweight and you can tweak settings while you're trying to understand system behavior [since there are many config params that may never need to be tuned]. We need to add the ability to put read arbitrary KV settings in the HBase schema. Combined with online schema change, this will allow us to safely iterate on configuration settings. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5335) Dynamic Schema Configurations
[ https://issues.apache.org/jira/browse/HBASE-5335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg updated HBASE-5335: --- Fix Version/s: 0.96.0 Status: Patch Available (was: Open) Dynamic Schema Configurations - Key: HBASE-5335 URL: https://issues.apache.org/jira/browse/HBASE-5335 Project: HBase Issue Type: New Feature Reporter: Nicolas Spiegelberg Assignee: Nicolas Spiegelberg Labels: configuration, schema Fix For: 0.96.0 Attachments: D2247.1.patch, D2247.2.patch, D2247.3.patch, D2247.4.patch, D2247.5.patch, D2247.6.patch, D2247.7.patch, HBASE-5335-trunk.patch Currently, the ability for a core developer to add per-table per-CF configuration settings is very heavyweight. You need to add a reserved keyword all the way up the stack you have to support this variable long-term if you're going to expose it explicitly to the user. This has ended up with using Configuration.get() a lot because it is lightweight and you can tweak settings while you're trying to understand system behavior [since there are many config params that may never need to be tuned]. We need to add the ability to put read arbitrary KV settings in the HBase schema. Combined with online schema change, this will allow us to safely iterate on configuration settings. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5332) Deterministic Compaction Jitter
[ https://issues.apache.org/jira/browse/HBASE-5332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg updated HBASE-5332: --- Fix Version/s: 0.94.0 Status: Patch Available (was: Open) Deterministic Compaction Jitter --- Key: HBASE-5332 URL: https://issues.apache.org/jira/browse/HBASE-5332 Project: HBase Issue Type: Improvement Reporter: Nicolas Spiegelberg Assignee: Nicolas Spiegelberg Priority: Minor Fix For: 0.94.0 Attachments: D1785.1.patch, D1785.2.patch, D1785.3.patch Currently, we add jitter to a compaction using delay + jitter*(1 - 2*Math.random()). Since this is non-deterministic, we can get major compaction storms on server restart as half the Stores that were set to delay + jitter will now be set to delay - jitter. We need a more deterministic way to jitter major compactions so this information can persist across server restarts. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5332) Deterministic Compaction Jitter
[ https://issues.apache.org/jira/browse/HBASE-5332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg updated HBASE-5332: --- Status: Open (was: Patch Available) Deterministic Compaction Jitter --- Key: HBASE-5332 URL: https://issues.apache.org/jira/browse/HBASE-5332 Project: HBase Issue Type: Improvement Reporter: Nicolas Spiegelberg Assignee: Nicolas Spiegelberg Priority: Minor Fix For: 0.94.0 Attachments: D1785.1.patch, D1785.2.patch, D1785.3.patch, HBASE-5332.patch Currently, we add jitter to a compaction using delay + jitter*(1 - 2*Math.random()). Since this is non-deterministic, we can get major compaction storms on server restart as half the Stores that were set to delay + jitter will now be set to delay - jitter. We need a more deterministic way to jitter major compactions so this information can persist across server restarts. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5332) Deterministic Compaction Jitter
[ https://issues.apache.org/jira/browse/HBASE-5332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg updated HBASE-5332: --- Attachment: HBASE-5332.patch trunk patch Deterministic Compaction Jitter --- Key: HBASE-5332 URL: https://issues.apache.org/jira/browse/HBASE-5332 Project: HBase Issue Type: Improvement Reporter: Nicolas Spiegelberg Assignee: Nicolas Spiegelberg Priority: Minor Fix For: 0.94.0 Attachments: D1785.1.patch, D1785.2.patch, D1785.3.patch, HBASE-5332.patch Currently, we add jitter to a compaction using delay + jitter*(1 - 2*Math.random()). Since this is non-deterministic, we can get major compaction storms on server restart as half the Stores that were set to delay + jitter will now be set to delay - jitter. We need a more deterministic way to jitter major compactions so this information can persist across server restarts. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5332) Deterministic Compaction Jitter
[ https://issues.apache.org/jira/browse/HBASE-5332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg updated HBASE-5332: --- Status: Patch Available (was: Open) resubmitting after creating trunk-specific patch file Deterministic Compaction Jitter --- Key: HBASE-5332 URL: https://issues.apache.org/jira/browse/HBASE-5332 Project: HBase Issue Type: Improvement Reporter: Nicolas Spiegelberg Assignee: Nicolas Spiegelberg Priority: Minor Fix For: 0.94.0 Attachments: D1785.1.patch, D1785.2.patch, D1785.3.patch, HBASE-5332.patch Currently, we add jitter to a compaction using delay + jitter*(1 - 2*Math.random()). Since this is non-deterministic, we can get major compaction storms on server restart as half the Stores that were set to delay + jitter will now be set to delay - jitter. We need a more deterministic way to jitter major compactions so this information can persist across server restarts. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5332) Deterministic Compaction Jitter
[ https://issues.apache.org/jira/browse/HBASE-5332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg updated HBASE-5332: --- Resolution: Fixed Status: Resolved (was: Patch Available) Deterministic Compaction Jitter --- Key: HBASE-5332 URL: https://issues.apache.org/jira/browse/HBASE-5332 Project: HBase Issue Type: Improvement Reporter: Nicolas Spiegelberg Assignee: Nicolas Spiegelberg Priority: Minor Fix For: 0.94.0 Attachments: D1785.1.patch, D1785.2.patch, D1785.3.patch, HBASE-5332.patch Currently, we add jitter to a compaction using delay + jitter*(1 - 2*Math.random()). Since this is non-deterministic, we can get major compaction storms on server restart as half the Stores that were set to delay + jitter will now be set to delay - jitter. We need a more deterministic way to jitter major compactions so this information can persist across server restarts. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4916) LoadTest MR Job
[ https://issues.apache.org/jira/browse/HBASE-4916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg updated HBASE-4916: --- Status: Patch Available (was: Open) changing status to patch available for trunk LoadTest MR Job --- Key: HBASE-4916 URL: https://issues.apache.org/jira/browse/HBASE-4916 Project: HBase Issue Type: Sub-task Components: client, regionserver Reporter: Nicolas Spiegelberg Assignee: Karthik Ranganathan Fix For: 0.94.0 Attachments: HBASE-4916.D741.1.patch Add a script to start a streaming map-reduce job where each map tasks runs an instance of the load tester for a partition of the key-space. Ensure that the load tester takes a parameter indicating the start key for write operations. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5072) Support Max Value for Per-Store Metrics
[ https://issues.apache.org/jira/browse/HBASE-5072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg updated HBASE-5072: --- Attachment: HBASE-5072.patch note: patch applies cleanly to both 89-fb trunk Support Max Value for Per-Store Metrics --- Key: HBASE-5072 URL: https://issues.apache.org/jira/browse/HBASE-5072 Project: HBase Issue Type: Improvement Components: metrics, regionserver Reporter: Nicolas Spiegelberg Assignee: Nicolas Spiegelberg Priority: Minor Fix For: 0.94.0 Attachments: D945.1.patch, D945.2.patch, HBASE-5072.patch We were bit in our multi-tenant cluster because one of our Stores encountered a bug and grew its StoreFile count. We didn't notice this because the StoreFile count currently reported by the RegionServer is an average of all Stores in the region. For the per-Store metrics, we should also record the max so we can notice outliers. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5072) Support Max Value for Per-Store Metrics
[ https://issues.apache.org/jira/browse/HBASE-5072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg updated HBASE-5072: --- Status: Patch Available (was: Open) Support Max Value for Per-Store Metrics --- Key: HBASE-5072 URL: https://issues.apache.org/jira/browse/HBASE-5072 Project: HBase Issue Type: Improvement Components: metrics, regionserver Reporter: Nicolas Spiegelberg Assignee: Nicolas Spiegelberg Priority: Minor Fix For: 0.94.0 Attachments: D945.1.patch, D945.2.patch, HBASE-5072.patch We were bit in our multi-tenant cluster because one of our Stores encountered a bug and grew its StoreFile count. We didn't notice this because the StoreFile count currently reported by the RegionServer is an average of all Stores in the region. For the per-Store metrics, we should also record the max so we can notice outliers. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5072) Support Max Value for Per-Store Metrics
[ https://issues.apache.org/jira/browse/HBASE-5072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg updated HBASE-5072: --- Resolution: Fixed Status: Resolved (was: Patch Available) Support Max Value for Per-Store Metrics --- Key: HBASE-5072 URL: https://issues.apache.org/jira/browse/HBASE-5072 Project: HBase Issue Type: Improvement Components: metrics, regionserver Reporter: Nicolas Spiegelberg Assignee: Nicolas Spiegelberg Priority: Minor Fix For: 0.94.0 Attachments: D945.1.patch, D945.2.patch, HBASE-5072.patch We were bit in our multi-tenant cluster because one of our Stores encountered a bug and grew its StoreFile count. We didn't notice this because the StoreFile count currently reported by the RegionServer is an average of all Stores in the region. For the per-Store metrics, we should also record the max so we can notice outliers. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5021) Enforce upper bound on timestamp
[ https://issues.apache.org/jira/browse/HBASE-5021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg updated HBASE-5021: --- Status: Patch Available (was: Open) Enforce upper bound on timestamp Key: HBASE-5021 URL: https://issues.apache.org/jira/browse/HBASE-5021 Project: HBase Issue Type: Improvement Reporter: Nicolas Spiegelberg Assignee: Nicolas Spiegelberg Priority: Critical Fix For: 0.94.0 Attachments: D849.1.patch, D849.2.patch, D849.3.patch, HBASE-5021-trunk.patch We have been getting hit with performance problems on our time-series database due to invalid timestamps being inserted by the timestamp. We are working on adding proper checks to app server, but production performance could be severely impacted with significant recovery time if something slips past. Since timestamps are considered a fundamental part of the HBase schema multiple optimizations use timestamp information, we should allow the option to sanity check the upper bound on the server-side in HBase. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5021) Enforce upper bound on timestamp
[ https://issues.apache.org/jira/browse/HBASE-5021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg updated HBASE-5021: --- Resolution: Fixed Status: Resolved (was: Patch Available) Enforce upper bound on timestamp Key: HBASE-5021 URL: https://issues.apache.org/jira/browse/HBASE-5021 Project: HBase Issue Type: Improvement Reporter: Nicolas Spiegelberg Assignee: Nicolas Spiegelberg Priority: Critical Fix For: 0.94.0 Attachments: D849.1.patch, D849.2.patch, D849.3.patch, HBASE-5021-trunk.patch We have been getting hit with performance problems on our time-series database due to invalid timestamps being inserted by the timestamp. We are working on adding proper checks to app server, but production performance could be severely impacted with significant recovery time if something slips past. Since timestamps are considered a fundamental part of the HBase schema multiple optimizations use timestamp information, we should allow the option to sanity check the upper bound on the server-side in HBase. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4908) HBase cluster test tool (port from 0.89-fb)
[ https://issues.apache.org/jira/browse/HBASE-4908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg updated HBASE-4908: --- Resolution: Fixed Fix Version/s: 0.94.0 Status: Resolved (was: Patch Available) HBase cluster test tool (port from 0.89-fb) --- Key: HBASE-4908 URL: https://issues.apache.org/jira/browse/HBASE-4908 Project: HBase Issue Type: Sub-task Reporter: Mikhail Bautin Assignee: Mikhail Bautin Fix For: 0.94.0 Attachments: 0001-HBase-cluster-test-tool.patch, 0002-HBase-cluster-test-tool.patch, 0003-HBase-cluster-test-tool.patch, D549.1.patch, D549.2.patch, D549.3.patch, D549.4.patch, D549.5.patch, D549.6.patch, D549.7.patch, D549.8.patch, D549.9.patch Porting one of our HBase cluster test tools (a single-process multi-threaded load generator and verifier) from 0.89-fb to trunk. I cleaned up the code a bit compared to what's in 0.89-fb, and discovered that it has some features that I have not tried yet (some kind of a kill test, and some way to run HBase as multiple processes on one machine). The main utility of this piece of code for us has been the HBaseClusterTest command-line tool (called HBaseTest in 0.89-fb), which we usually invoke as a load test in our five-node dev cluster testing, e.g.: hbase org.apache.hadoop.hbase.manual.HBaseTest -load 10:50:100:20 -tn load_test -read 1:10:50:20 -zk zk_quorum -bloom ROWCOL -compression GZIP I will be using this code to load-test the delta encoding patch and making fixes, but I am submitting the patch for early feedback. I will probably try out its other functionality and comment on how it works. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4933) Ability to calculate the blockcache hit ratio for the last few minutes
[ https://issues.apache.org/jira/browse/HBASE-4933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg updated HBASE-4933: --- Fix Version/s: 0.94.0 Status: Patch Available (was: Open) Ability to calculate the blockcache hit ratio for the last few minutes -- Key: HBASE-4933 URL: https://issues.apache.org/jira/browse/HBASE-4933 Project: HBase Issue Type: Improvement Components: metrics Reporter: dhruba borthakur Assignee: dhruba borthakur Fix For: 0.94.0 Attachments: HBASE-4933.patch The metric blockcacheHitRatio is since the beginning of time. It would be nice to calculate the block cache hit ratio for the past few minutes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4933) Ability to calculate the blockcache hit ratio for the last few minutes
[ https://issues.apache.org/jira/browse/HBASE-4933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg updated HBASE-4933: --- Resolution: Fixed Status: Resolved (was: Patch Available) Ability to calculate the blockcache hit ratio for the last few minutes -- Key: HBASE-4933 URL: https://issues.apache.org/jira/browse/HBASE-4933 Project: HBase Issue Type: Improvement Components: metrics Reporter: dhruba borthakur Assignee: dhruba borthakur Fix For: 0.94.0 Attachments: HBASE-4933.patch The metric blockcacheHitRatio is since the beginning of time. It would be nice to calculate the block cache hit ratio for the past few minutes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4933) Ability to calculate the blockcache hit ratio for the last few minutes
[ https://issues.apache.org/jira/browse/HBASE-4933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg updated HBASE-4933: --- Attachment: HBASE-4933.patch uploading patch from Phabricator Ability to calculate the blockcache hit ratio for the last few minutes -- Key: HBASE-4933 URL: https://issues.apache.org/jira/browse/HBASE-4933 Project: HBase Issue Type: Improvement Components: metrics Reporter: dhruba borthakur Assignee: dhruba borthakur Fix For: 0.94.0 Attachments: HBASE-4933.patch The metric blockcacheHitRatio is since the beginning of time. It would be nice to calculate the block cache hit ratio for the past few minutes. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4860) RegionSplitter Should Allow NSFRE during logical split verification
[ https://issues.apache.org/jira/browse/HBASE-4860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg updated HBASE-4860: --- Resolution: Fixed Fix Version/s: 0.94.0 Status: Resolved (was: Patch Available) resolving issue in 92 trunk. RegionSplitter Should Allow NSFRE during logical split verification --- Key: HBASE-4860 URL: https://issues.apache.org/jira/browse/HBASE-4860 Project: HBase Issue Type: Bug Reporter: Nicolas Spiegelberg Assignee: Nicolas Spiegelberg Priority: Minor Fix For: 0.94.0, 0.92.1 Attachments: HBASE-4860.patch When verifying that a split request has been logically completed, the client can throw a NoServerForRegionException if the parent daughter regions are in flux at this time. We should catch that error and retry later when the daughter regions should have been brought online. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-3967) Support deletes in HFileOutputFormat based bulk import mechanism
[ https://issues.apache.org/jira/browse/HBASE-3967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg updated HBASE-3967: --- Issue Type: Sub-task (was: Improvement) Parent: HBASE-4907 Support deletes in HFileOutputFormat based bulk import mechanism Key: HBASE-3967 URL: https://issues.apache.org/jira/browse/HBASE-3967 Project: HBase Issue Type: Sub-task Reporter: Kannan Muthukkaruppan Assignee: Bogdan-Alexandru Matican Attachments: diff.patch During bulk imports, it'll be useful to be able to do delete mutations (either to delete data that already exists in HBase or was inserted earlier during this run of the import). For example, we have a use case, where we are processing a log of data which may have both inserts and deletes in the mix and we want to upload that into HBase using the bulk import mechanism. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-2842) Support BloomFilter error rate on a per-family basis
[ https://issues.apache.org/jira/browse/HBASE-2842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg updated HBASE-2842: --- Issue Type: Sub-task (was: Improvement) Parent: HBASE-4907 Support BloomFilter error rate on a per-family basis Key: HBASE-2842 URL: https://issues.apache.org/jira/browse/HBASE-2842 Project: HBase Issue Type: Sub-task Components: filters, ipc, regionserver, rest, thrift Reporter: Nicolas Spiegelberg Assignee: Nicolas Spiegelberg Priority: Minor The error rate for bloom filters is currently set by the io.hfile.bloom.error.rate global variable. Todd suggested at the last HUG that it would be nice to have per-family config options instead. Trace the Bloom Type code to implement this. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4908) HBase cluster test tool (port from 0.89-fb)
[ https://issues.apache.org/jira/browse/HBASE-4908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg updated HBASE-4908: --- Issue Type: Sub-task (was: Test) Parent: HBASE-4907 HBase cluster test tool (port from 0.89-fb) --- Key: HBASE-4908 URL: https://issues.apache.org/jira/browse/HBASE-4908 Project: HBase Issue Type: Sub-task Reporter: Mikhail Bautin Assignee: Mikhail Bautin Attachments: D549.1.patch Porting one of our HBase cluster test tools (a single-process multi-threaded load generator and verifier) from 0.89-fb to trunk. I cleaned up the code a bit compared to what's in 0.89-fb, and discovered that it has some features that I have not tried yet (some kind of a kill test, and some way to run HBase as multiple processes on one machine). The main utility of this piece of code for us has been the HBaseClusterTest command-line tool (called HBaseTest in 0.89-fb), which we usually invoke as a load test in our five-node dev cluster testing, e.g.: hbase org.apache.hadoop.hbase.manual.HBaseTest -load 10:50:100:20 -tn load_test -read 1:10:50:20 -zk zk_quorum -bloom ROWCOL -compression GZIP I will be using this code to load-test the delta encoding patch and making fixes, but I am submitting the patch for early feedback. I will probably try out its other functionality and comment on how it works. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4910) thrift scannerstopwithfilter not honoring stop row
[ https://issues.apache.org/jira/browse/HBASE-4910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg updated HBASE-4910: --- Summary: thrift scannerstopwithfilter not honoring stop row (was: stopWithFilter bug) thrift scannerstopwithfilter not honoring stop row -- Key: HBASE-4910 URL: https://issues.apache.org/jira/browse/HBASE-4910 Project: HBase Issue Type: Sub-task Components: client, regionserver Reporter: Nicolas Spiegelberg Fix For: 0.94.0 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4914) Enhance MapReduce TableInputFormat to Support N-mappers per Region
[ https://issues.apache.org/jira/browse/HBASE-4914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg updated HBASE-4914: --- Description: Current TableInputFormat based MR jobs create exactly one mapper per region where each mapper sets one Scan with appropriate start/stop row keys. This change allows jobs to be run with any number of mappers per region, so that when a mapper fails, there will be less data to be reprocessed. Priority: Minor (was: Major) Summary: Enhance MapReduce TableInputFormat to Support N-mappers per Region (was: Support N-mappers per Region) Need to port from 89-fb. See SVN #1181607 Enhance MapReduce TableInputFormat to Support N-mappers per Region -- Key: HBASE-4914 URL: https://issues.apache.org/jira/browse/HBASE-4914 Project: HBase Issue Type: Sub-task Components: client, regionserver Reporter: Nicolas Spiegelberg Priority: Minor Fix For: 0.94.0 Current TableInputFormat based MR jobs create exactly one mapper per region where each mapper sets one Scan with appropriate start/stop row keys. This change allows jobs to be run with any number of mappers per region, so that when a mapper fails, there will be less data to be reprocessed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4783) Improve RowCounter to count rows in a specific key range.
[ https://issues.apache.org/jira/browse/HBASE-4783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg updated HBASE-4783: --- Resolution: Fixed Status: Resolved (was: Patch Available) Improve RowCounter to count rows in a specific key range. - Key: HBASE-4783 URL: https://issues.apache.org/jira/browse/HBASE-4783 Project: HBase Issue Type: Improvement Reporter: Nicolas Spiegelberg Assignee: Nicolas Spiegelberg Priority: Trivial Fix For: 0.94.0 Attachments: 4783.txt, HBASE-4783.patch Currently RowCounter in MR package is a very simple map only job that does a full scan of a table. Enhance the utility to let the user specify a key range and count the number of rows in this range. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4785) Improve recovery time of HBase client when a region server dies.
[ https://issues.apache.org/jira/browse/HBASE-4785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg updated HBASE-4785: --- Attachment: HBASE-4785.patch Fixes SoftValueSortedMap. Internal comments: Currently SoftValueSortedMap.entrySet() tries to iteraate through the entry set of the underlying map, and put all the values (SoftValueK,V) to a newly created TreeSetEntryK,V. The entry set of SortedMap is already sorted, so it's not necessary to have a TreeSet to sort those entries again upon adding. This gets rid of the runtime class cast exception because it does not require comparing anymore. Improve recovery time of HBase client when a region server dies. Key: HBASE-4785 URL: https://issues.apache.org/jira/browse/HBASE-4785 Project: HBase Issue Type: Improvement Reporter: Nicolas Spiegelberg Assignee: Nicolas Spiegelberg Priority: Minor Fix For: 0.94.0 Attachments: HBASE-4785.patch, HBASE-4785.patch When a region server dies, the HBase client waits until the RPC timesout before learning that it needs to check META to find the new location of the region. And it incurs this *timeout* cost for every region being served by the dead region server. Remove this overhead by clearing the entries in cache that have the dead region server as their values. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4785) Improve recovery time of HBase client when a region server dies.
[ https://issues.apache.org/jira/browse/HBASE-4785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg updated HBASE-4785: --- Resolution: Fixed Status: Resolved (was: Patch Available) Improve recovery time of HBase client when a region server dies. Key: HBASE-4785 URL: https://issues.apache.org/jira/browse/HBASE-4785 Project: HBase Issue Type: Improvement Reporter: Nicolas Spiegelberg Assignee: Nicolas Spiegelberg Priority: Minor Fix For: 0.94.0 Attachments: HBASE-4785.patch, HBASE-4785.patch When a region server dies, the HBase client waits until the RPC timesout before learning that it needs to check META to find the new location of the region. And it incurs this *timeout* cost for every region being served by the dead region server. Remove this overhead by clearing the entries in cache that have the dead region server as their values. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4859) Correctly PreWarm HBCK ThreadPool
[ https://issues.apache.org/jira/browse/HBASE-4859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg updated HBASE-4859: --- Attachment: HBASE-4859.patch patch applies on 0.92 0.94 Correctly PreWarm HBCK ThreadPool - Key: HBASE-4859 URL: https://issues.apache.org/jira/browse/HBASE-4859 Project: HBase Issue Type: Bug Reporter: Nicolas Spiegelberg Assignee: Nicolas Spiegelberg Fix For: 0.92.0, 0.94.0 Attachments: HBASE-4859.patch See description at HBASE-3553. We had a patch ready for this in HBASE-3620 but never applied it publicly. Testing showed massive speedup in HBCK, especially when RegionServers were down or had long response times. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4859) Correctly PreWarm HBCK ThreadPool
[ https://issues.apache.org/jira/browse/HBASE-4859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg updated HBASE-4859: --- Attachment: HBASE-4859.patch version 2 of the patch, modeled after HBASE-4859 behavior Correctly PreWarm HBCK ThreadPool - Key: HBASE-4859 URL: https://issues.apache.org/jira/browse/HBASE-4859 Project: HBase Issue Type: Bug Reporter: Nicolas Spiegelberg Assignee: Nicolas Spiegelberg Fix For: 0.92.0, 0.94.0 Attachments: HBASE-4859.patch, HBASE-4859.patch See description at HBASE-3553. We had a patch ready for this in HBASE-3620 but never applied it publicly. Testing showed massive speedup in HBCK, especially when RegionServers were down or had long response times. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4859) Correctly PreWarm HBCK ThreadPool
[ https://issues.apache.org/jira/browse/HBASE-4859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg updated HBASE-4859: --- Status: Patch Available (was: Open) Correctly PreWarm HBCK ThreadPool - Key: HBASE-4859 URL: https://issues.apache.org/jira/browse/HBASE-4859 Project: HBase Issue Type: Bug Reporter: Nicolas Spiegelberg Assignee: Nicolas Spiegelberg Fix For: 0.92.0, 0.94.0 Attachments: HBASE-4859.patch, HBASE-4859.patch See description at HBASE-3553. We had a patch ready for this in HBASE-3620 but never applied it publicly. Testing showed massive speedup in HBCK, especially when RegionServers were down or had long response times. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4860) RegionSplitter Should Allow NSFRE during logical split verification
[ https://issues.apache.org/jira/browse/HBASE-4860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg updated HBASE-4860: --- Attachment: HBASE-4860.patch RegionSplitter Should Allow NSFRE during logical split verification --- Key: HBASE-4860 URL: https://issues.apache.org/jira/browse/HBASE-4860 Project: HBase Issue Type: Bug Reporter: Nicolas Spiegelberg Assignee: Nicolas Spiegelberg Priority: Minor Fix For: 0.92.0, 0.94.0 Attachments: HBASE-4860.patch When verifying that a split request has been logically completed, the client can throw a NoServerForRegionException if the parent daughter regions are in flux at this time. We should catch that error and retry later when the daughter regions should have been brought online. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4860) RegionSplitter Should Allow NSFRE during logical split verification
[ https://issues.apache.org/jira/browse/HBASE-4860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg updated HBASE-4860: --- Status: Patch Available (was: Open) RegionSplitter Should Allow NSFRE during logical split verification --- Key: HBASE-4860 URL: https://issues.apache.org/jira/browse/HBASE-4860 Project: HBase Issue Type: Bug Reporter: Nicolas Spiegelberg Assignee: Nicolas Spiegelberg Priority: Minor Fix For: 0.92.0, 0.94.0 Attachments: HBASE-4860.patch When verifying that a split request has been logically completed, the client can throw a NoServerForRegionException if the parent daughter regions are in flux at this time. We should catch that error and retry later when the daughter regions should have been brought online. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4811) Support reverse Scan
[ https://issues.apache.org/jira/browse/HBASE-4811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg updated HBASE-4811: --- Comment: was deleted (was: Yeah, I'm not that familiar with the codebase, but I'd assume that in order to get forward scans you'd have to have the data sorted. And from what I understand it is internally stored as sstables or HFiles. If you have it sorted to scan in one direction, it seems pretty easy to go the other direction. LevelDb uses ssTables and supports reverse ranges. The only thing that I could think of from the design (from a high level) that might make it difficult to do reverse ranges is dealing with splitting ranges when moving ranges from one region server to another. Just from a quick look at MemStore that you mention, it uses a KeyValueSkipListSet under the covers that is a NavigableSet and supports descendingSet and descendingIterator. -jc On Tue, Nov 22, 2011 at 12:52 PM, stack (Commented) (JIRA) ) Support reverse Scan Key: HBASE-4811 URL: https://issues.apache.org/jira/browse/HBASE-4811 Project: HBase Issue Type: Improvement Components: client Affects Versions: 0.20.6 Reporter: John Carrino All the documentation I find about HBase says that if you want forward and reverse scans you should just build 2 tables and one be ascending and one descending. Is there a fundamental reason that HBase only supports forward Scan? It seems like a lot of extra space overhead and coding overhead (to keep them in sync) to support 2 tables. I am assuming this has been discussed before, but I can't find the discussions anywhere about it or why it would be infeasible. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4809) Per-CF set RPC metrics
[ https://issues.apache.org/jira/browse/HBASE-4809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg updated HBASE-4809: --- Attachment: HBASE-4809.patch Per-CF set RPC metrics -- Key: HBASE-4809 URL: https://issues.apache.org/jira/browse/HBASE-4809 Project: HBase Issue Type: Improvement Reporter: Mikhail Bautin Assignee: Mikhail Bautin Priority: Minor Attachments: D483.1.patch, D483.2.patch, D483.3.patch, D483.4.patch, D483.5.patch, HBASE-4809.patch, HBASE-4809_Per_CF_set_RPC_metrics.patch Porting per-CF set metrics for RPC times and response sizes from 0.89-fb to trunk. For each mutation signature (a set of column families involved in an RPC request) we increment several metrics, allowing to monitor access patterns. We deal with guarding against an explosion of the number of metrics in HBASE-4638 (which might even be implemented as part of this JIRA). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4809) Per-CF set RPC metrics
[ https://issues.apache.org/jira/browse/HBASE-4809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg updated HBASE-4809: --- Resolution: Fixed Fix Version/s: 0.94.0 Status: Resolved (was: Patch Available) Per-CF set RPC metrics -- Key: HBASE-4809 URL: https://issues.apache.org/jira/browse/HBASE-4809 Project: HBase Issue Type: Improvement Reporter: Mikhail Bautin Assignee: Mikhail Bautin Priority: Minor Fix For: 0.94.0 Attachments: D483.1.patch, D483.2.patch, D483.3.patch, D483.4.patch, D483.5.patch, HBASE-4809.patch, HBASE-4809_Per_CF_set_RPC_metrics.patch Porting per-CF set metrics for RPC times and response sizes from 0.89-fb to trunk. For each mutation signature (a set of column families involved in an RPC request) we increment several metrics, allowing to monitor access patterns. We deal with guarding against an explosion of the number of metrics in HBASE-4638 (which might even be implemented as part of this JIRA). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4808) Test to Ensure Expired Deletes Don't Override Puts
[ https://issues.apache.org/jira/browse/HBASE-4808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg updated HBASE-4808: --- Resolution: Fixed Status: Resolved (was: Patch Available) Test to Ensure Expired Deletes Don't Override Puts -- Key: HBASE-4808 URL: https://issues.apache.org/jira/browse/HBASE-4808 Project: HBase Issue Type: Test Reporter: Nicolas Spiegelberg Assignee: Prakash Khemani Priority: Trivial Fix For: 0.94.0 Attachments: HBASE-4808.patch We originally thought we had a bug where expired delete markers would early-out valid puts. It ended up being a false alarm, but we added a unit test to ensure that this behavior is correctly maintained. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4772) Utility to Create StoreFiles
[ https://issues.apache.org/jira/browse/HBASE-4772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg updated HBASE-4772: --- Attachment: HBASE-4772-B.patch Utility to Create StoreFiles Key: HBASE-4772 URL: https://issues.apache.org/jira/browse/HBASE-4772 Project: HBase Issue Type: Test Affects Versions: 0.94.0 Reporter: Nicolas Spiegelberg Assignee: Mikhail Bautin Priority: Minor Attachments: HBASE-4772-B.patch, HBASE-4772.patch Add a tool to create a StoreFile with the specified number of key/value pairs, with the specified compression and Bloom filter type. This is useful for creating HFileV1 HFileV2 store files for testing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4801) alter_status shell prints sensible message at completion
[ https://issues.apache.org/jira/browse/HBASE-4801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg updated HBASE-4801: --- Attachment: HBASE-4801.patch alter_status shell prints sensible message at completion Key: HBASE-4801 URL: https://issues.apache.org/jira/browse/HBASE-4801 Project: HBase Issue Type: Bug Affects Versions: 0.94.0 Reporter: Nicolas Spiegelberg Assignee: Nicolas Spiegelberg Priority: Trivial Attachments: HBASE-4801.patch The alter_status command used to print 0/0 once an alter operation had completed and its progress was no longer available. Now it instad indicates that all regions were updated. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4802) Disable show table metrics in bulk loader
[ https://issues.apache.org/jira/browse/HBASE-4802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg updated HBASE-4802: --- Attachment: HBASE-4802.patch Disable show table metrics in bulk loader - Key: HBASE-4802 URL: https://issues.apache.org/jira/browse/HBASE-4802 Project: HBase Issue Type: Bug Reporter: Nicolas Spiegelberg Assignee: Nicolas Spiegelberg Priority: Trivial Fix For: 0.94.0 Attachments: HBASE-4802.patch During bulk load, the Configuration object may be set to null. This caused an NPE in per-CF metrics because it consults the Configuration to determine whether to show the Table name. Need to add simple change to allow the conf to be null not specify table name in that instance. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4802) Disable show table metrics in bulk loader
[ https://issues.apache.org/jira/browse/HBASE-4802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg updated HBASE-4802: --- Assignee: Liyin Tang (was: Nicolas Spiegelberg) Status: Patch Available (was: Open) Disable show table metrics in bulk loader - Key: HBASE-4802 URL: https://issues.apache.org/jira/browse/HBASE-4802 Project: HBase Issue Type: Bug Reporter: Nicolas Spiegelberg Assignee: Liyin Tang Priority: Trivial Fix For: 0.94.0 Attachments: HBASE-4802.patch During bulk load, the Configuration object may be set to null. This caused an NPE in per-CF metrics because it consults the Configuration to determine whether to show the Table name. Need to add simple change to allow the conf to be null not specify table name in that instance. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4801) alter_status shell prints sensible message at completion
[ https://issues.apache.org/jira/browse/HBASE-4801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg updated HBASE-4801: --- Status: Patch Available (was: Open) patch created by one of our interns: Charles Gist. alter_status shell prints sensible message at completion Key: HBASE-4801 URL: https://issues.apache.org/jira/browse/HBASE-4801 Project: HBase Issue Type: Bug Affects Versions: 0.94.0 Reporter: Nicolas Spiegelberg Assignee: Nicolas Spiegelberg Priority: Trivial Attachments: HBASE-4801.patch The alter_status command used to print 0/0 once an alter operation had completed and its progress was no longer available. Now it instad indicates that all regions were updated. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4803) Split log worker should terminate properly when waiting for znode
[ https://issues.apache.org/jira/browse/HBASE-4803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg updated HBASE-4803: --- Status: Patch Available (was: Open) Split log worker should terminate properly when waiting for znode - Key: HBASE-4803 URL: https://issues.apache.org/jira/browse/HBASE-4803 Project: HBase Issue Type: Bug Reporter: Nicolas Spiegelberg Assignee: Prakash Khemani Priority: Minor Fix For: 0.94.0 Attachments: HBASE-4803.patch This is an attempt to fix the fact that SplitLogWorker threads were not being terminated properly in some multi-master unit tests. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4803) Split log worker should terminate properly when waiting for znode
[ https://issues.apache.org/jira/browse/HBASE-4803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg updated HBASE-4803: --- Attachment: HBASE-4803.patch Split log worker should terminate properly when waiting for znode - Key: HBASE-4803 URL: https://issues.apache.org/jira/browse/HBASE-4803 Project: HBase Issue Type: Bug Reporter: Nicolas Spiegelberg Assignee: Prakash Khemani Priority: Minor Fix For: 0.94.0 Attachments: HBASE-4803.patch This is an attempt to fix the fact that SplitLogWorker threads were not being terminated properly in some multi-master unit tests. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4808) Test to Ensure Expired Deletes Don't Override Puts
[ https://issues.apache.org/jira/browse/HBASE-4808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg updated HBASE-4808: --- Attachment: HBASE-4808.patch Test to Ensure Expired Deletes Don't Override Puts -- Key: HBASE-4808 URL: https://issues.apache.org/jira/browse/HBASE-4808 Project: HBase Issue Type: Test Reporter: Nicolas Spiegelberg Assignee: Prakash Khemani Priority: Trivial Fix For: 0.94.0 Attachments: HBASE-4808.patch We originally thought we had a bug where expired delete markers would early-out valid puts. It ended up being a false alarm, but we added a unit test to ensure that this behavior is correctly maintained. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4628) Enhance Table Create Presplit Functionality within the HBase Shell
[ https://issues.apache.org/jira/browse/HBASE-4628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg updated HBASE-4628: --- Comment: was deleted (was: nspiegelberg has accepted the revision HBASE-4628 [jira] Enhance Table Create Presplit Functionality within the HBase Shell. lgtm! Thanks for the contribution REVISION DETAIL https://reviews.facebook.net/D417 ) Enhance Table Create Presplit Functionality within the HBase Shell -- Key: HBASE-4628 URL: https://issues.apache.org/jira/browse/HBASE-4628 Project: HBase Issue Type: Improvement Reporter: Nicolas Spiegelberg Assignee: Nicolas Spiegelberg Attachments: HBASE-4628.D411.1.patch, HBASE-4628.D417.1.patch, HBASE-4628.D417.2.patch, HBASE-4628.D429.1.patch Currently, we allow the user to presplit in the HBase shell by explicitly listing the startkey of all the region shards that they want. Instead, we should provide the RegionSplitter functionality of choosing a split algorithm, followed by the number of splits that they want. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4628) Enhance Table Create Presplit Functionality within the HBase Shell
[ https://issues.apache.org/jira/browse/HBASE-4628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg updated HBASE-4628: --- Comment: was deleted (was: cgist has commented on the revision HBASE-4628 [jira] Enhance Table Create Presplit Functionality within the HBase Shell. This diff was only intended for 89-fb. There is a separate diff for apache trunk D429. REVISION DETAIL https://reviews.facebook.net/D417 ) Enhance Table Create Presplit Functionality within the HBase Shell -- Key: HBASE-4628 URL: https://issues.apache.org/jira/browse/HBASE-4628 Project: HBase Issue Type: Improvement Reporter: Nicolas Spiegelberg Assignee: Nicolas Spiegelberg Attachments: HBASE-4628.D411.1.patch, HBASE-4628.D417.1.patch, HBASE-4628.D417.2.patch, HBASE-4628.D429.1.patch Currently, we allow the user to presplit in the HBase shell by explicitly listing the startkey of all the region shards that they want. Instead, we should provide the RegionSplitter functionality of choosing a split algorithm, followed by the number of splits that they want. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4628) Enhance Table Create Presplit Functionality within the HBase Shell
[ https://issues.apache.org/jira/browse/HBASE-4628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg updated HBASE-4628: --- Comment: was deleted (was: cgist requested code review of HBASE-4628 [jira] Enhance Table Create Presplit Functionality within the HBase Shell. Reviewers: nspiegelberg, JIRA [89-fb] HBase shell can pre-split a new table at creation THis change adds optional arguments to the HBase shell's create command to split a tabel into a specified number of regions using a specified splitting algorithm as defined by RegionSplitter. Currently, we allow the user to presplit in the HBase shell by explicitly listing the startkey of all the region shards that they want. Instead, we should provide the RegionSplitter functionality of choosing a split algorithm, followed by the number of splits that they want. TEST PLAN Created tables with and without splits using the shell; also attempted to give incorrect arguments to shell create command. REVISION DETAIL https://reviews.facebook.net/D417 AFFECTED FILES src/main/ruby/hbase.rb src/main/ruby/hbase/admin.rb src/main/ruby/shell/commands/create.rb MANAGE HERALD DIFFERENTIAL RULES https://reviews.facebook.net/herald/view/differential/ WHY DID I GET THIS EMAIL? https://reviews.facebook.net/herald/transcript/837/ Tip: use the X-Herald-Rules header to filter Herald messages in your client. ) Enhance Table Create Presplit Functionality within the HBase Shell -- Key: HBASE-4628 URL: https://issues.apache.org/jira/browse/HBASE-4628 Project: HBase Issue Type: Improvement Reporter: Nicolas Spiegelberg Assignee: Nicolas Spiegelberg Attachments: HBASE-4628.D411.1.patch, HBASE-4628.D417.1.patch, HBASE-4628.D417.2.patch, HBASE-4628.D429.1.patch Currently, we allow the user to presplit in the HBase shell by explicitly listing the startkey of all the region shards that they want. Instead, we should provide the RegionSplitter functionality of choosing a split algorithm, followed by the number of splits that they want. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4778) Don't ignore corrupt StoreFiles when opening a region
[ https://issues.apache.org/jira/browse/HBASE-4778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg updated HBASE-4778: --- Resolution: Fixed Status: Resolved (was: Patch Available) Don't ignore corrupt StoreFiles when opening a region - Key: HBASE-4778 URL: https://issues.apache.org/jira/browse/HBASE-4778 Project: HBase Issue Type: Bug Reporter: Nicolas Spiegelberg Assignee: Mikhail Bautin Fix For: 0.94.0 Attachments: HBASE-4778.patch We used to ignore StoreFiles that failed to open, which led to a situation when only a subset of regions was opened, and HBase did not return results to clients for the affected set of keys. This change makes sure we propagate IOExceptions coming from an attempt to open a StoreFile all the way up to HRegionServer.openRegion, where it will lead to a failure to open the whole region. This way we can avoid returning corrupt data to the application. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4782) Consistency Check Utility for META Schemas
[ https://issues.apache.org/jira/browse/HBASE-4782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg updated HBASE-4782: --- Attachment: HBASE-4782.patch Consistency Check Utility for META Schemas -- Key: HBASE-4782 URL: https://issues.apache.org/jira/browse/HBASE-4782 Project: HBase Issue Type: Improvement Reporter: Nicolas Spiegelberg Assignee: Nicolas Spiegelberg Priority: Trivial Fix For: 0.94.0 Attachments: HBASE-4782.patch Adding a script to check if table descriptors for each region in META for a given table are consistent. The script compares the table descriptors in META for all regions of a particular table to the table's descriptor (which is just the descriptor for the first region). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4782) Consistency Check Utility for META Schemas
[ https://issues.apache.org/jira/browse/HBASE-4782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg updated HBASE-4782: --- Status: Patch Available (was: Open) Consistency Check Utility for META Schemas -- Key: HBASE-4782 URL: https://issues.apache.org/jira/browse/HBASE-4782 Project: HBase Issue Type: Improvement Reporter: Nicolas Spiegelberg Assignee: Madhuwanti Vaidya Priority: Trivial Fix For: 0.94.0 Attachments: HBASE-4782.patch Adding a script to check if table descriptors for each region in META for a given table are consistent. The script compares the table descriptors in META for all regions of a particular table to the table's descriptor (which is just the descriptor for the first region). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4783) Improve RowCounter to count rows in a specific key range.
[ https://issues.apache.org/jira/browse/HBASE-4783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg updated HBASE-4783: --- Attachment: HBASE-4783.patch Improve RowCounter to count rows in a specific key range. - Key: HBASE-4783 URL: https://issues.apache.org/jira/browse/HBASE-4783 Project: HBase Issue Type: Improvement Reporter: Nicolas Spiegelberg Assignee: Nicolas Spiegelberg Priority: Trivial Fix For: 0.94.0 Attachments: HBASE-4783.patch Currently RowCounter in MR package is a very simple map only job that does a full scan of a table. Enhance the utility to let the user specify a key range and count the number of rows in this range. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4783) Improve RowCounter to count rows in a specific key range.
[ https://issues.apache.org/jira/browse/HBASE-4783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg updated HBASE-4783: --- Fix Version/s: 0.94.0 Status: Patch Available (was: Open) Improve RowCounter to count rows in a specific key range. - Key: HBASE-4783 URL: https://issues.apache.org/jira/browse/HBASE-4783 Project: HBase Issue Type: Improvement Reporter: Nicolas Spiegelberg Assignee: Nicolas Spiegelberg Priority: Trivial Fix For: 0.94.0 Attachments: HBASE-4783.patch Currently RowCounter in MR package is a very simple map only job that does a full scan of a table. Enhance the utility to let the user specify a key range and count the number of rows in this range. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4785) Improve recovery time of HBase client when a region server dies.
[ https://issues.apache.org/jira/browse/HBASE-4785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg updated HBASE-4785: --- Attachment: HBASE-4785.patch Improve recovery time of HBase client when a region server dies. Key: HBASE-4785 URL: https://issues.apache.org/jira/browse/HBASE-4785 Project: HBase Issue Type: Improvement Reporter: Nicolas Spiegelberg Assignee: Nicolas Spiegelberg Priority: Minor Fix For: 0.94.0 Attachments: HBASE-4785.patch When a region server dies, the HBase client waits until the RPC timesout before learning that it needs to check META to find the new location of the region. And it incurs this *timeout* cost for every region being served by the dead region server. Remove this overhead by clearing the entries in cache that have the dead region server as their values. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4785) Improve recovery time of HBase client when a region server dies.
[ https://issues.apache.org/jira/browse/HBASE-4785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg updated HBASE-4785: --- Status: Patch Available (was: Open) Improve recovery time of HBase client when a region server dies. Key: HBASE-4785 URL: https://issues.apache.org/jira/browse/HBASE-4785 Project: HBase Issue Type: Improvement Reporter: Nicolas Spiegelberg Assignee: Nicolas Spiegelberg Priority: Minor Fix For: 0.94.0 Attachments: HBASE-4785.patch When a region server dies, the HBase client waits until the RPC timesout before learning that it needs to check META to find the new location of the region. And it incurs this *timeout* cost for every region being served by the dead region server. Remove this overhead by clearing the entries in cache that have the dead region server as their values. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4768) Per-(table, columnFamily) metrics with configurable table name inclusion
[ https://issues.apache.org/jira/browse/HBASE-4768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg updated HBASE-4768: --- Resolution: Fixed Fix Version/s: 0.94.0 Status: Resolved (was: Patch Available) Per-(table, columnFamily) metrics with configurable table name inclusion Key: HBASE-4768 URL: https://issues.apache.org/jira/browse/HBASE-4768 Project: HBase Issue Type: New Feature Reporter: Mikhail Bautin Assignee: Mikhail Bautin Fix For: 0.94.0 Attachments: D363.1.patch, D363.2.patch, D363.3.patch, D363.4.patch, D363.5.patch As we kept adding more granular block read and block cache usage statistics, a combinatorial explosion of various cases to monitor started to happen, especially when we wanted both per-table/column family/block type statistics and aggregate statistics on various subsets of these dimensions. Here, we un-clutters HFile readers, LruBlockCache, StoreFile, etc. by creating a centralized class that knows how to update all kinds of per-table/CF/block type counters. Table name and column family configuration have been pushed to a base class, SchemaConfigured. This is convenient as many of existing classes that have these properties (HFile readers/writers, HFile blocks, etc.) did not have a base class. Whether to collect per-(table, columnFamily) or per-columnFamily only metrics can be configured with the hbase.metrics.showTableName configuration key. We don't expect this configuration to change at runtime, so we cache the setting statically and log a warning when an attempt is made to flip it once already set. This way we don't have to pass configuration to a lot more places, e.g. everywhere an HFile reader is instantiated. Thanks to Liyin for his initial version of per-table metrics patch and a lot of valuable feedback. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4787) Make corePool as a configurable parameter in HTable
[ https://issues.apache.org/jira/browse/HBASE-4787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg updated HBASE-4787: --- Attachment: HBASE-4787.patch Make corePool as a configurable parameter in HTable --- Key: HBASE-4787 URL: https://issues.apache.org/jira/browse/HBASE-4787 Project: HBase Issue Type: Improvement Reporter: Nicolas Spiegelberg Priority: Trivial Attachments: HBASE-4787.patch Make the corePool a configurable parameter in HTable. So we can tune this parameter in the config file. While at it, change the core pool name so we can distinguish it from other AppServer pools. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4628) Enhance Table Create Presplit Functionality within the HBase Shell
[ https://issues.apache.org/jira/browse/HBASE-4628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg updated HBASE-4628: --- Comment: was deleted (was: cgist requested code review of HBASE-4628 [jira] Enhance Table Create Presplit Functionality within the HBase Shell. Reviewers: nspiegelberg, JIRA [89-fb] Ported RegionSplitter and TestRegionSplitter from trunk The RegionSplitter is useful for creating a table pre-split into many regions, including two different algorithms for splitting keys. This also includes the patch for HBASE-4627. Currently, we allow the user to presplit in the HBase shell by explicitly listing the startkey of all the region shards that they want. Instead, we should provide the RegionSplitter functionality of choosing a split algorithm, followed by the number of splits that they want. TEST PLAN The TestRegionSplitter test suite. REVISION DETAIL https://reviews.facebook.net/D411 AFFECTED FILES src/main/java/org/apache/hadoop/hbase/util/Bytes.java src/main/java/org/apache/hadoop/hbase/util/RegionSplitter.java src/test/java/org/apache/hadoop/hbase/util/TestRegionSplitter.java MANAGE HERALD DIFFERENTIAL RULES https://reviews.facebook.net/herald/view/differential/ WHY DID I GET THIS EMAIL? https://reviews.facebook.net/herald/transcript/831/ Tip: use the X-Herald-Rules header to filter Herald messages in your client. ) Enhance Table Create Presplit Functionality within the HBase Shell -- Key: HBASE-4628 URL: https://issues.apache.org/jira/browse/HBASE-4628 Project: HBase Issue Type: Improvement Reporter: Nicolas Spiegelberg Assignee: Nicolas Spiegelberg Attachments: HBASE-4628.D411.1.patch, HBASE-4628.D417.1.patch Currently, we allow the user to presplit in the HBase shell by explicitly listing the startkey of all the region shards that they want. Instead, we should provide the RegionSplitter functionality of choosing a split algorithm, followed by the number of splits that they want. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4628) Enhance Table Create Presplit Functionality within the HBase Shell
[ https://issues.apache.org/jira/browse/HBASE-4628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg updated HBASE-4628: --- Comment: was deleted (was: nspiegelberg has commented on the revision HBASE-4628 [jira] Enhance Table Create Presplit Functionality within the HBase Shell. Since this patch has been accepted into apache trunk, you don't need to make a patch for 89-fb. I'll automatically pull it in. We're just stalled on a previous commit. Go ahead and abandone this review. REVISION DETAIL https://reviews.facebook.net/D411 ) Enhance Table Create Presplit Functionality within the HBase Shell -- Key: HBASE-4628 URL: https://issues.apache.org/jira/browse/HBASE-4628 Project: HBase Issue Type: Improvement Reporter: Nicolas Spiegelberg Assignee: Nicolas Spiegelberg Attachments: HBASE-4628.D411.1.patch, HBASE-4628.D417.1.patch Currently, we allow the user to presplit in the HBase shell by explicitly listing the startkey of all the region shards that they want. Instead, we should provide the RegionSplitter functionality of choosing a split algorithm, followed by the number of splits that they want. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4628) Enhance Table Create Presplit Functionality within the HBase Shell
[ https://issues.apache.org/jira/browse/HBASE-4628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg updated HBASE-4628: --- Comment: was deleted (was: cgist has abandoned the revision HBASE-4628 [jira] Enhance Table Create Presplit Functionality within the HBase Shell. REVISION DETAIL https://reviews.facebook.net/D411 ) Enhance Table Create Presplit Functionality within the HBase Shell -- Key: HBASE-4628 URL: https://issues.apache.org/jira/browse/HBASE-4628 Project: HBase Issue Type: Improvement Reporter: Nicolas Spiegelberg Assignee: Nicolas Spiegelberg Attachments: HBASE-4628.D411.1.patch, HBASE-4628.D417.1.patch Currently, we allow the user to presplit in the HBase shell by explicitly listing the startkey of all the region shards that they want. Instead, we should provide the RegionSplitter functionality of choosing a split algorithm, followed by the number of splits that they want. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4776) HLog.closed should be checked inside of updateLock
[ https://issues.apache.org/jira/browse/HBASE-4776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg updated HBASE-4776: --- Status: Patch Available (was: Open) HLog.closed should be checked inside of updateLock -- Key: HBASE-4776 URL: https://issues.apache.org/jira/browse/HBASE-4776 Project: HBase Issue Type: Bug Reporter: Nicolas Spiegelberg Assignee: Nicolas Spiegelberg Attachments: HBASE-4776.patch Concurrency issue: HLog.closed is set inside the updateLock but not checked inside the lock. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4776) HLog.closed should be checked inside of updateLock
[ https://issues.apache.org/jira/browse/HBASE-4776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg updated HBASE-4776: --- Attachment: HBASE-4776.patch HLog.closed should be checked inside of updateLock -- Key: HBASE-4776 URL: https://issues.apache.org/jira/browse/HBASE-4776 Project: HBase Issue Type: Bug Reporter: Nicolas Spiegelberg Assignee: Nicolas Spiegelberg Attachments: HBASE-4776.patch Concurrency issue: HLog.closed is set inside the updateLock but not checked inside the lock. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4778) Don't ignore corrupt StoreFiles when opening a region
[ https://issues.apache.org/jira/browse/HBASE-4778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg updated HBASE-4778: --- Attachment: HBASE-4778.patch Don't ignore corrupt StoreFiles when opening a region - Key: HBASE-4778 URL: https://issues.apache.org/jira/browse/HBASE-4778 Project: HBase Issue Type: Bug Reporter: Nicolas Spiegelberg Assignee: Mikhail Bautin Fix For: 0.94.0 Attachments: HBASE-4778.patch We used to ignore StoreFiles that failed to open, which led to a situation when only a subset of regions was opened, and HBase did not return results to clients for the affected set of keys. This change makes sure we propagate IOExceptions coming from an attempt to open a StoreFile all the way up to HRegionServer.openRegion, where it will lead to a failure to open the whole region. This way we can avoid returning corrupt data to the application. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4778) Don't ignore corrupt StoreFiles when opening a region
[ https://issues.apache.org/jira/browse/HBASE-4778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg updated HBASE-4778: --- Status: Patch Available (was: Open) Don't ignore corrupt StoreFiles when opening a region - Key: HBASE-4778 URL: https://issues.apache.org/jira/browse/HBASE-4778 Project: HBase Issue Type: Bug Reporter: Nicolas Spiegelberg Assignee: Mikhail Bautin Fix For: 0.94.0 Attachments: HBASE-4778.patch We used to ignore StoreFiles that failed to open, which led to a situation when only a subset of regions was opened, and HBase did not return results to clients for the affected set of keys. This change makes sure we propagate IOExceptions coming from an attempt to open a StoreFile all the way up to HRegionServer.openRegion, where it will lead to a failure to open the whole region. This way we can avoid returning corrupt data to the application. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4769) Abort RegionServer Immediately on OOME
[ https://issues.apache.org/jira/browse/HBASE-4769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg updated HBASE-4769: --- Attachment: HBASE-4769.patch Abort RegionServer Immediately on OOME -- Key: HBASE-4769 URL: https://issues.apache.org/jira/browse/HBASE-4769 Project: HBase Issue Type: Bug Affects Versions: 0.94.0 Reporter: Nicolas Spiegelberg Assignee: Nicolas Spiegelberg Attachments: HBASE-4769.patch Currently, when the HRegionServer runs out of the memory, it will call master, which will cause more heap allocations and throw a second exception that it's run out of memory again. The easiest safest way to avoid this OOME storm is to abort the RegionServer immediately when it hits the memory boundary. Part of the 89-fb to trunk port. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4769) Abort RegionServer Immediately on OOME
[ https://issues.apache.org/jira/browse/HBASE-4769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg updated HBASE-4769: --- Status: Patch Available (was: Open) Abort RegionServer Immediately on OOME -- Key: HBASE-4769 URL: https://issues.apache.org/jira/browse/HBASE-4769 Project: HBase Issue Type: Bug Affects Versions: 0.94.0 Reporter: Nicolas Spiegelberg Assignee: Nicolas Spiegelberg Attachments: HBASE-4769.patch Currently, when the HRegionServer runs out of the memory, it will call master, which will cause more heap allocations and throw a second exception that it's run out of memory again. The easiest safest way to avoid this OOME storm is to abort the RegionServer immediately when it hits the memory boundary. Part of the 89-fb to trunk port. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4765) Enhance HLog metrics
[ https://issues.apache.org/jira/browse/HBASE-4765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg updated HBASE-4765: --- Resolution: Fixed Status: Resolved (was: Patch Available) Enhance HLog metrics Key: HBASE-4765 URL: https://issues.apache.org/jira/browse/HBASE-4765 Project: HBase Issue Type: Improvement Affects Versions: 0.94.0 Reporter: Nicolas Spiegelberg Assignee: Nicolas Spiegelberg Attachments: HBASE-4765-1.patch Part of 89-fb to 92 porting. Refactor HLog metrics for increased visibility. 1. Add writeSize to capture average write size 2. Decouple HLog write metrics from HFile 3. Give actual min/max outlier data for hlog latencies -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-3690) Option to Exclude Bulk Import Files from Minor Compaction
[ https://issues.apache.org/jira/browse/HBASE-3690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg updated HBASE-3690: --- Resolution: Fixed Fix Version/s: 0.94.0 Status: Resolved (was: Patch Available) Option to Exclude Bulk Import Files from Minor Compaction - Key: HBASE-3690 URL: https://issues.apache.org/jira/browse/HBASE-3690 Project: HBase Issue Type: Bug Components: regionserver Reporter: Nicolas Spiegelberg Assignee: Nicolas Spiegelberg Priority: Minor Fix For: 0.94.0 Attachments: D357.1.patch, HBASE-3690.patch We ran an incremental scrape with HFileOutputFormat and encountered major compaction storms. This is caused by the bug in HBASE-3404. The permanent fix is a little tricky without HBASE-2856. We realized that a quicker solution for avoiding these compaction storms is to simply exclude bulk import files from minor compactions and let them only be handled by time-based major compactions. Add with functionality along with a config option to enable it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4769) Abort RegionServer Immediately on OOME
[ https://issues.apache.org/jira/browse/HBASE-4769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg updated HBASE-4769: --- Attachment: HBASE-4769.patch Abort RegionServer Immediately on OOME -- Key: HBASE-4769 URL: https://issues.apache.org/jira/browse/HBASE-4769 Project: HBase Issue Type: Bug Affects Versions: 0.94.0 Reporter: Nicolas Spiegelberg Assignee: Nicolas Spiegelberg Fix For: 0.94.0 Attachments: HBASE-4769.patch, HBASE-4769.patch Currently, when the HRegionServer runs out of the memory, it will call master, which will cause more heap allocations and throw a second exception that it's run out of memory again. The easiest safest way to avoid this OOME storm is to abort the RegionServer immediately when it hits the memory boundary. Part of the 89-fb to trunk port. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4769) Abort RegionServer Immediately on OOME
[ https://issues.apache.org/jira/browse/HBASE-4769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg updated HBASE-4769: --- Resolution: Fixed Fix Version/s: 0.94.0 Status: Resolved (was: Patch Available) maybe we should put this in 0.92 as well? Abort RegionServer Immediately on OOME -- Key: HBASE-4769 URL: https://issues.apache.org/jira/browse/HBASE-4769 Project: HBase Issue Type: Bug Affects Versions: 0.94.0 Reporter: Nicolas Spiegelberg Assignee: Nicolas Spiegelberg Fix For: 0.94.0 Attachments: HBASE-4769.patch, HBASE-4769.patch Currently, when the HRegionServer runs out of the memory, it will call master, which will cause more heap allocations and throw a second exception that it's run out of memory again. The easiest safest way to avoid this OOME storm is to abort the RegionServer immediately when it hits the memory boundary. Part of the 89-fb to trunk port. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4772) Utility to Create StoreFiles
[ https://issues.apache.org/jira/browse/HBASE-4772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg updated HBASE-4772: --- Attachment: HBASE-4772.patch Utility to Create StoreFiles Key: HBASE-4772 URL: https://issues.apache.org/jira/browse/HBASE-4772 Project: HBase Issue Type: Test Affects Versions: 0.94.0 Reporter: Nicolas Spiegelberg Assignee: Mikhail Bautin Priority: Minor Attachments: HBASE-4772.patch Add a tool to create a StoreFile with the specified number of key/value pairs, with the specified compression and Bloom filter type. This is useful for creating HFileV1 HFileV2 store files for testing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4772) Utility to Create StoreFiles
[ https://issues.apache.org/jira/browse/HBASE-4772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg updated HBASE-4772: --- Status: Patch Available (was: Open) Utility to Create StoreFiles Key: HBASE-4772 URL: https://issues.apache.org/jira/browse/HBASE-4772 Project: HBase Issue Type: Test Affects Versions: 0.94.0 Reporter: Nicolas Spiegelberg Assignee: Mikhail Bautin Priority: Minor Attachments: HBASE-4772.patch Add a tool to create a StoreFile with the specified number of key/value pairs, with the specified compression and Bloom filter type. This is useful for creating HFileV1 HFileV2 store files for testing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4765) Enhance HLog metrics
[ https://issues.apache.org/jira/browse/HBASE-4765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg updated HBASE-4765: --- Attachment: HBASE-4765-1.patch Enhance HLog metrics Key: HBASE-4765 URL: https://issues.apache.org/jira/browse/HBASE-4765 Project: HBase Issue Type: Improvement Affects Versions: 0.94.0 Reporter: Nicolas Spiegelberg Assignee: Nicolas Spiegelberg Attachments: HBASE-4765-1.patch Part of 89-fb to 92 porting. Refactor HLog metrics for increased visibility. 1. Add writeSize to capture average write size 2. Decouple HLog write metrics from HFile 3. Give actual min/max outlier data for hlog latencies -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4765) Enhance HLog metrics
[ https://issues.apache.org/jira/browse/HBASE-4765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg updated HBASE-4765: --- Status: Patch Available (was: Open) Enhance HLog metrics Key: HBASE-4765 URL: https://issues.apache.org/jira/browse/HBASE-4765 Project: HBase Issue Type: Improvement Affects Versions: 0.94.0 Reporter: Nicolas Spiegelberg Assignee: Nicolas Spiegelberg Attachments: HBASE-4765-1.patch Part of 89-fb to 92 porting. Refactor HLog metrics for increased visibility. 1. Add writeSize to capture average write size 2. Decouple HLog write metrics from HFile 3. Give actual min/max outlier data for hlog latencies -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4766) Add Instrumentation for Put Latencies at Critical Paths
[ https://issues.apache.org/jira/browse/HBASE-4766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg updated HBASE-4766: --- Assignee: (was: Nicolas Spiegelberg) looks like jgray redid the flow for the critical path. going to punt on this one let someone else with new critical path knowledge work on it or revisit later. Add Instrumentation for Put Latencies at Critical Paths --- Key: HBASE-4766 URL: https://issues.apache.org/jira/browse/HBASE-4766 Project: HBase Issue Type: Improvement Affects Versions: 0.94.0 Reporter: Nicolas Spiegelberg Part of 89-fb to 92 port. Add instrumentation to collect: - memstore insert time - RWCC wait time (the time we wait after an write operation to, for RWCC read point to advance). - row lock acquisition time -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-2312) Possible data loss when RS goes into GC pause while rolling HLog
[ https://issues.apache.org/jira/browse/HBASE-2312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg updated HBASE-2312: --- Attachment: HBASE-2312.patch forgot to attach jira with the apache license grant Possible data loss when RS goes into GC pause while rolling HLog Key: HBASE-2312 URL: https://issues.apache.org/jira/browse/HBASE-2312 Project: HBase Issue Type: Bug Components: master, regionserver Affects Versions: 0.90.0 Reporter: Karthik Ranganathan Assignee: Nicolas Spiegelberg Priority: Critical Fix For: 0.92.0 Attachments: D99.1.patch, D99.2.patch, D99.3.patch, HBASE-2312.patch There is a very corner case when bad things could happen(ie data loss): 1)RS #1 is going to roll its HLog - not yet created the new one, old one will get no more writes 2)RS #1 enters GC Pause of Death 3)Master lists HLog files of RS#1 that is has to split as RS#1 is dead, starts splitting 4)RS #1 wakes up, created the new HLog (previous one was rolled) and appends an edit - which is lost The following seems like a possible solution: 1)Master detects RS#1 is dead 2)The master renames the /hbase/.logs/regionserver name directory to something else (say /hbase/.logs/regionserver name-dead) 3)Add mkdir support (as opposed to mkdirs) to HDFS - so that a file create fails if the directory doesn't exist. Dhruba tells me this is very doable. 4)RS#1 comes back up and is not able create the new hlog. It restarts itself. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4761) Add Developer Debug Options to HBase Config
[ https://issues.apache.org/jira/browse/HBASE-4761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg updated HBASE-4761: --- Attachment: HBASE-4761.patch Add Developer Debug Options to HBase Config --- Key: HBASE-4761 URL: https://issues.apache.org/jira/browse/HBASE-4761 Project: HBase Issue Type: Improvement Affects Versions: 0.94.0 Reporter: Nicolas Spiegelberg Assignee: Nicolas Spiegelberg Priority: Minor Attachments: HBASE-4761.patch Add in optional HBase configuration options that core developers will commonly use: an option to enable JDWP debugging an option to use a separate logfile for GC information. (Part of the effort to move 89-fb features over to trunk) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4627) Ability to specify a custom start/end to RegionSplitter
[ https://issues.apache.org/jira/browse/HBASE-4627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg updated HBASE-4627: --- Comment: was deleted (was: nspiegelberg requested code review of [jira] [HBASE-4627] Ability to specify a custom start/end to RegionSplitter. Reviewers: Karthik, JIRA [HBASE-4627] added a custom start/end row to RegionSplitter. Also solved an off-by-one error because the end row is prefix-inclusive and not exclusive. a href=https://issues.apache.org/jira/browse/HBASE-4489; title=Better key splitting in RegionSplitterdelHBASE-4489/del/a changed the default endKey on HexStringSplit from 7FFF... to ... While this is correct, existing users of 0.90 RegionSplitter have 7FFF as the end key in their schema and the last region will not split properly under this new code. We need to let the user specify a custom start/end key range for when situations like this arise. Optimally, we should also write the start/end key in META so we could figure this out implicitly instead of requiring the user to explicitly specify it. TEST PLAN - mvn test -Dtest=TestRegionSplitter REVISION DETAIL https://reviews.facebook.net/D39 AFFECTED FILES src/main/java/org/apache/hadoop/hbase/util/Bytes.java src/main/java/org/apache/hadoop/hbase/util/RegionSplitter.java src/test/java/org/apache/hadoop/hbase/util/TestRegionSplitter.java ) Ability to specify a custom start/end to RegionSplitter --- Key: HBASE-4627 URL: https://issues.apache.org/jira/browse/HBASE-4627 Project: HBase Issue Type: Improvement Affects Versions: 0.94.0 Reporter: Nicolas Spiegelberg Assignee: Nicolas Spiegelberg Attachments: D39.1.patch, D39.1.patch, D39.1.patch HBASE-4489 changed the default endKey on HexStringSplit from 7FFF... to ... While this is correct, existing users of 0.90 RegionSplitter have 7FFF as the end key in their schema and the last region will not split properly under this new code. We need to let the user specify a custom start/end key range for when situations like this arise. Optimally, we should also write the start/end key in META so we could figure this out implicitly instead of requiring the user to explicitly specify it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4627) Ability to specify a custom start/end to RegionSplitter
[ https://issues.apache.org/jira/browse/HBASE-4627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg updated HBASE-4627: --- Comment: was deleted (was: Karthik has commented on the revision [jira] [HBASE-4627] Ability to specify a custom start/end to RegionSplitter. INLINE COMMENTS src/main/java/org/apache/hadoop/hbase/util/Bytes.java:1417 I thought of a simpler case which it seemed to not work for: Let's take the range [0,10] and request 5 splits. If exclusive, we would do diffBI = (10 - 0)/5 = 2, so we get (0-2, 2-4, 4-6, 6-8, 8-10). If inclusive, diffBI would become 3, so we would get (0-3, 3-6, 6-9, 9-10) - which is 4 regions. But if we added the 1 to the stopBI, the we would get: diffBI = (11 - 0)/5 = 2.2, which would give us 5 regions: (0-2.2, 2.2-4.4, 4.4-6.6, 6.6-8.8, 8.8-10) So, would it not be better to do: startBI = new BigInteger(0x00) stopBI = new BigInteger(0xFF) + 1 diff BI = 255.subtract(0) REVISION DETAIL https://reviews.facebook.net/D39 COMMITS https://reviews.facebook.net/rHBASE1196256 https://reviews.facebook.net/rHBASE1196772 ) Ability to specify a custom start/end to RegionSplitter --- Key: HBASE-4627 URL: https://issues.apache.org/jira/browse/HBASE-4627 Project: HBase Issue Type: Improvement Affects Versions: 0.94.0 Reporter: Nicolas Spiegelberg Assignee: Nicolas Spiegelberg Attachments: D39.1.patch, D39.1.patch, D39.1.patch HBASE-4489 changed the default endKey on HexStringSplit from 7FFF... to ... While this is correct, existing users of 0.90 RegionSplitter have 7FFF as the end key in their schema and the last region will not split properly under this new code. We need to let the user specify a custom start/end key range for when situations like this arise. Optimally, we should also write the start/end key in META so we could figure this out implicitly instead of requiring the user to explicitly specify it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4627) Ability to specify a custom start/end to RegionSplitter
[ https://issues.apache.org/jira/browse/HBASE-4627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg updated HBASE-4627: --- Comment: was deleted (was: Karthik has commented on the revision [jira] [HBASE-4627] Ability to specify a custom start/end to RegionSplitter. INLINE COMMENTS src/main/java/org/apache/hadoop/hbase/util/Bytes.java:1417 I thought of a simpler case which it seemed to not work for: Let's take the range [0,10] and request 5 splits. If exclusive, we would do diffBI = (10 - 0)/5 = 2, so we get (0-2, 2-4, 4-6, 6-8, 8-10). If inclusive, diffBI would become 3, so we would get (0-3, 3-6, 6-9, 9-10) - which is 4 regions. But if we added the 1 to the stopBI, the we would get: diffBI = (11 - 0)/5 = 2.2, which would give us 5 regions: (0-2.2, 2.2-4.4, 4.4-6.6, 6.6-8.8, 8.8-10) So, would it not be better to do: startBI = new BigInteger(0x00) stopBI = new BigInteger(0xFF) + 1 diff BI = 255.subtract(0) REVISION DETAIL https://reviews.facebook.net/D39 COMMITS https://reviews.facebook.net/rHBASE1196256 https://reviews.facebook.net/rHBASE1196772 ) Ability to specify a custom start/end to RegionSplitter --- Key: HBASE-4627 URL: https://issues.apache.org/jira/browse/HBASE-4627 Project: HBase Issue Type: Improvement Affects Versions: 0.94.0 Reporter: Nicolas Spiegelberg Assignee: Nicolas Spiegelberg Attachments: D39.1.patch, D39.1.patch, D39.1.patch HBASE-4489 changed the default endKey on HexStringSplit from 7FFF... to ... While this is correct, existing users of 0.90 RegionSplitter have 7FFF as the end key in their schema and the last region will not split properly under this new code. We need to let the user specify a custom start/end key range for when situations like this arise. Optimally, we should also write the start/end key in META so we could figure this out implicitly instead of requiring the user to explicitly specify it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4627) Ability to specify a custom start/end to RegionSplitter
[ https://issues.apache.org/jira/browse/HBASE-4627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg updated HBASE-4627: --- Comment: was deleted (was: Karthik has commented on the revision [jira] [HBASE-4627] Ability to specify a custom start/end to RegionSplitter. INLINE COMMENTS src/main/java/org/apache/hadoop/hbase/util/Bytes.java:1371 Formatting nit: if() return null; src/main/java/org/apache/hadoop/hbase/util/Bytes.java:1417 Trying to reason with this one - so to make it an inclusive range, we are making the delta [ (end - start)/numSplits + 1 ]. Wondering if that would always give the required number of uniform sized regions. REVISION DETAIL https://reviews.facebook.net/D39 COMMITS https://reviews.facebook.net/rHBASE1196256 https://reviews.facebook.net/rHBASE1196772 ) Ability to specify a custom start/end to RegionSplitter --- Key: HBASE-4627 URL: https://issues.apache.org/jira/browse/HBASE-4627 Project: HBase Issue Type: Improvement Affects Versions: 0.94.0 Reporter: Nicolas Spiegelberg Assignee: Nicolas Spiegelberg Attachments: D39.1.patch, D39.1.patch HBASE-4489 changed the default endKey on HexStringSplit from 7FFF... to ... While this is correct, existing users of 0.90 RegionSplitter have 7FFF as the end key in their schema and the last region will not split properly under this new code. We need to let the user specify a custom start/end key range for when situations like this arise. Optimally, we should also write the start/end key in META so we could figure this out implicitly instead of requiring the user to explicitly specify it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4627) Ability to specify a custom start/end to RegionSplitter
[
https://issues.apache.org/jira/browse/HBASE-4627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nicolas Spiegelberg updated HBASE-4627:
---
Comment: was deleted
(was: nspiegelberg has commented on the revision [jira] [HBASE-4627] Ability
to specify a custom start/end to RegionSplitter.
INLINE COMMENTS
src/main/java/org/apache/hadoop/hbase/util/Bytes.java:1417 this part was very
tricky to me as well. the problem is that the region is exclusive [start,end)
but the last row of the table is prefix-exclusive [start,prefix(end)]. you can
reason about this best with an 8-bit HexStringSplit use case
row = itoa(hash(row) % 256) + row
table range = [00, FF]
row = FFnic -- should be accepted
row = FGnic -- should FAIL
memcmp(FF,FFnic) 0
memcmp(FF, FFnic, sizeof(FF)) == 0
In this case, you can split into 256 regions optimally. So you should do:
startBI = new BigInteger(0x00) ==0
stopBI = new BigInteger(0xFF) == 255
diffBI = 255.substract(0)
if (inclusive) { diffBI = 255 + 1 }
REVISION DETAIL
https://reviews.facebook.net/D39
COMMITS
https://reviews.facebook.net/rHBASE1196256
https://reviews.facebook.net/rHBASE1196772
)
Ability to specify a custom start/end to RegionSplitter
---
Key: HBASE-4627
URL: https://issues.apache.org/jira/browse/HBASE-4627
Project: HBase
Issue Type: Improvement
Affects Versions: 0.94.0
Reporter: Nicolas Spiegelberg
Assignee: Nicolas Spiegelberg
Attachments: D39.1.patch, D39.1.patch
HBASE-4489 changed the default endKey on HexStringSplit from 7FFF... to
... While this is correct, existing users of 0.90 RegionSplitter have
7FFF as the end key in their schema and the last region will not split
properly under this new code. We need to let the user specify a custom
start/end key range for when situations like this arise. Optimally, we
should also write the start/end key in META so we could figure this out
implicitly instead of requiring the user to explicitly specify it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4627) Ability to specify a custom start/end to RegionSplitter
[
https://issues.apache.org/jira/browse/HBASE-4627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nicolas Spiegelberg updated HBASE-4627:
---
Comment: was deleted
(was: nspiegelberg has commented on the revision [jira] [HBASE-4627] Ability
to specify a custom start/end to RegionSplitter.
INLINE COMMENTS
src/main/java/org/apache/hadoop/hbase/util/Bytes.java:1417 this part was very
tricky to me as well. the problem is that the region is exclusive [start,end)
but the last row of the table is prefix-exclusive [start,prefix(end)]. you can
reason about this best with an 8-bit HexStringSplit use case
row = itoa(hash(row) % 256) + row
table range = [00, FF]
row = FFnic -- should be accepted
row = FGnic -- should FAIL
memcmp(FF,FFnic) 0
memcmp(FF, FFnic, sizeof(FF)) == 0
In this case, you can split into 256 regions optimally. So you should do:
startBI = new BigInteger(0x00) ==0
stopBI = new BigInteger(0xFF) == 255
diffBI = 255.substract(0)
if (inclusive) { diffBI = 255 + 1 }
REVISION DETAIL
https://reviews.facebook.net/D39
COMMITS
https://reviews.facebook.net/rHBASE1196256
https://reviews.facebook.net/rHBASE1196772
)
Ability to specify a custom start/end to RegionSplitter
---
Key: HBASE-4627
URL: https://issues.apache.org/jira/browse/HBASE-4627
Project: HBase
Issue Type: Improvement
Affects Versions: 0.94.0
Reporter: Nicolas Spiegelberg
Assignee: Nicolas Spiegelberg
Attachments: D39.1.patch, D39.1.patch
HBASE-4489 changed the default endKey on HexStringSplit from 7FFF... to
... While this is correct, existing users of 0.90 RegionSplitter have
7FFF as the end key in their schema and the last region will not split
properly under this new code. We need to let the user specify a custom
start/end key range for when situations like this arise. Optimally, we
should also write the start/end key in META so we could figure this out
implicitly instead of requiring the user to explicitly specify it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4611) Add support for Phabricator/Differential as an alternative code review tool
[ https://issues.apache.org/jira/browse/HBASE-4611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Spiegelberg updated HBASE-4611: --- Comment: was deleted (was: nspiegelberg requested code review of [jira] [HBASE-4611] Add support for Phabricator/Differential as an alternative code review tool. Reviewers: DUMMY_REVIEWER Adding more awesome From a href=http://phabricator.org/;http://phabricator.org//a : Phabricator is a open source collection of web applications which make it easier to write, review, and share source code. It is currently available as an early release. Phabricator was developed at Facebook. It's open source so pretty much anyone could host an instance of this software. To begin with, there will be a public-facing instance located at a href=http://reviews.facebook.net;http://reviews.facebook.net/a (sponsored by Facebook and hosted by the OSUOSL a href=http://osuosl.org;http://osuosl.org/a). We will use this JIRA to deal with adding (and ensuring) Apache-friendly support that will allow us to do code reviews with Phabricator for HBase. TEST PLAN EMPTY REVISION DETAIL http://reviews.facebook.net/D21 AFFECTED FILES CHANGES.txt MANAGE HERALD DIFFERENTIAL RULES http://reviews.facebook.net/herald/view/differential/ WHY DID I GET THIS EMAIL? http://reviews.facebook.net/herald/transcript/51/ Tip: use the X-Herald-Rules header to filter Herald messages in your client. ) Add support for Phabricator/Differential as an alternative code review tool --- Key: HBASE-4611 URL: https://issues.apache.org/jira/browse/HBASE-4611 Project: HBase Issue Type: Task Reporter: Jonathan Gray Attachments: D153.1.patch, D21.1.patch, D21.1.patch From http://phabricator.org/ : Phabricator is a open source collection of web applications which make it easier to write, review, and share source code. It is currently available as an early release. Phabricator was developed at Facebook. It's open source so pretty much anyone could host an instance of this software. To begin with, there will be a public-facing instance located at http://reviews.facebook.net (sponsored by Facebook and hosted by the OSUOSL http://osuosl.org). We will use this JIRA to deal with adding (and ensuring) Apache-friendly support that will allow us to do code reviews with Phabricator for HBase. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
