[jira] [Updated] (HBASE-8348) Polish the migration to 0.96
[ https://issues.apache.org/jira/browse/HBASE-8348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-8348: - Resolution: Fixed Release Note: Added upgrade script that must be run moving a 0.92.x or 0.94.x era hbase dataset to 0.96.x Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Resolving as done. I just added the last minor but of doc on what it looks like when old client connects to new cluster. Thanks [~himan...@cloudera.com] and [~rajesh23] for the great work done in here. Polish the migration to 0.96 Key: HBASE-8348 URL: https://issues.apache.org/jira/browse/HBASE-8348 Project: HBase Issue Type: Bug Affects Versions: 0.95.0 Reporter: Jean-Daniel Cryans Assignee: rajeshbabu Priority: Critical Fix For: 0.96.0 Attachments: 8348v5.txt, 8348v5.txt, HBASE-8348-approach-2.patch, HBASE-8348-approach-2-v2.1.patch, HBASE-8348-approach-2-v2.2.patch, HBASE-8348-approach-2-v2.3.patch, HBASE-8348-approach-2-v2.4.patch, HBASE-8348-approach-3.patch, HBASE-8348_trunk.patch, HBASE-8348_trunk_v2.patch, HBASE-8348_trunk_v3.patch, log, log-2, Upgradeto96.docx, Upgradeto96.pdf Currently, migration works but there's still a couple of rough edges: - HBASE-8045 finished the .META. migration but didn't remove ROOT, so it's still on the filesystem. - Data in ZK needs to be removed manually. Either we fix up the data in ZK or we delete it ourselves. - TestMetaMigrationRemovingHTD has a testMetaUpdatedFlagInROOT method, but ROOT is gone now. Elliott was also mentioning that we could have hbase migrate do the HFileV1 checks, clear ZK, remove ROOT, etc. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8348) Polish the migration to 0.96
[ https://issues.apache.org/jira/browse/HBASE-8348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-8348: - Priority: Critical (was: Blocker) Downing to critical. I see only doc outstanding in this issue. Polish the migration to 0.96 Key: HBASE-8348 URL: https://issues.apache.org/jira/browse/HBASE-8348 Project: HBase Issue Type: Bug Affects Versions: 0.95.0 Reporter: Jean-Daniel Cryans Assignee: rajeshbabu Priority: Critical Fix For: 0.96.0 Attachments: 8348v5.txt, 8348v5.txt, HBASE-8348-approach-2.patch, HBASE-8348-approach-2-v2.1.patch, HBASE-8348-approach-2-v2.2.patch, HBASE-8348-approach-2-v2.3.patch, HBASE-8348-approach-2-v2.4.patch, HBASE-8348-approach-3.patch, HBASE-8348_trunk.patch, HBASE-8348_trunk_v2.patch, HBASE-8348_trunk_v3.patch, log, log-2, Upgradeto96.docx, Upgradeto96.pdf Currently, migration works but there's still a couple of rough edges: - HBASE-8045 finished the .META. migration but didn't remove ROOT, so it's still on the filesystem. - Data in ZK needs to be removed manually. Either we fix up the data in ZK or we delete it ourselves. - TestMetaMigrationRemovingHTD has a testMetaUpdatedFlagInROOT method, but ROOT is gone now. Elliott was also mentioning that we could have hbase migrate do the HFileV1 checks, clear ZK, remove ROOT, etc. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8348) Polish the migration to 0.96
[ https://issues.apache.org/jira/browse/HBASE-8348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Himanshu Vashishtha updated HBASE-8348: --- Attachment: Upgradeto96.docx Upgradeto96.pdf Attached is a help section on migration. It is in doc format; let me know if it is okay, and I will do a doc patch for it. Thanks. Polish the migration to 0.96 Key: HBASE-8348 URL: https://issues.apache.org/jira/browse/HBASE-8348 Project: HBase Issue Type: Bug Affects Versions: 0.95.0 Reporter: Jean-Daniel Cryans Assignee: rajeshbabu Priority: Blocker Fix For: 0.96.0 Attachments: 8348v5.txt, 8348v5.txt, HBASE-8348-approach-2.patch, HBASE-8348-approach-2-v2.1.patch, HBASE-8348-approach-2-v2.2.patch, HBASE-8348-approach-2-v2.3.patch, HBASE-8348-approach-2-v2.4.patch, HBASE-8348-approach-3.patch, HBASE-8348_trunk.patch, HBASE-8348_trunk_v2.patch, HBASE-8348_trunk_v3.patch, log, log-2, Upgradeto96.docx, Upgradeto96.pdf Currently, migration works but there's still a couple of rough edges: - HBASE-8045 finished the .META. migration but didn't remove ROOT, so it's still on the filesystem. - Data in ZK needs to be removed manually. Either we fix up the data in ZK or we delete it ourselves. - TestMetaMigrationRemovingHTD has a testMetaUpdatedFlagInROOT method, but ROOT is gone now. Elliott was also mentioning that we could have hbase migrate do the HFileV1 checks, clear ZK, remove ROOT, etc. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8348) Polish the migration to 0.96
[ https://issues.apache.org/jira/browse/HBASE-8348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Himanshu Vashishtha updated HBASE-8348: --- Attachment: log Attached is a log file, that contains sample runs of the tool. Polish the migration to 0.96 Key: HBASE-8348 URL: https://issues.apache.org/jira/browse/HBASE-8348 Project: HBase Issue Type: Bug Affects Versions: 0.95.0 Reporter: Jean-Daniel Cryans Assignee: rajeshbabu Priority: Blocker Fix For: 0.96.0 Attachments: HBASE-8348-approach-2.patch, HBASE-8348-approach-2-v2.1.patch, HBASE-8348-approach-2-v2.2.patch, HBASE-8348-approach-2-v2.3.patch, HBASE-8348-approach-3.patch, HBASE-8348_trunk.patch, HBASE-8348_trunk_v2.patch, HBASE-8348_trunk_v3.patch, log Currently, migration works but there's still a couple of rough edges: - HBASE-8045 finished the .META. migration but didn't remove ROOT, so it's still on the filesystem. - Data in ZK needs to be removed manually. Either we fix up the data in ZK or we delete it ourselves. - TestMetaMigrationRemovingHTD has a testMetaUpdatedFlagInROOT method, but ROOT is gone now. Elliott was also mentioning that we could have hbase migrate do the HFileV1 checks, clear ZK, remove ROOT, etc. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8348) Polish the migration to 0.96
[ https://issues.apache.org/jira/browse/HBASE-8348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Himanshu Vashishtha updated HBASE-8348: --- Attachment: HBASE-8348-approach-2-v2.3.patch Polish the migration to 0.96 Key: HBASE-8348 URL: https://issues.apache.org/jira/browse/HBASE-8348 Project: HBase Issue Type: Bug Affects Versions: 0.95.0 Reporter: Jean-Daniel Cryans Assignee: rajeshbabu Priority: Blocker Fix For: 0.96.0 Attachments: HBASE-8348-approach-2.patch, HBASE-8348-approach-2-v2.1.patch, HBASE-8348-approach-2-v2.2.patch, HBASE-8348-approach-2-v2.3.patch, HBASE-8348-approach-3.patch, HBASE-8348_trunk.patch, HBASE-8348_trunk_v2.patch, HBASE-8348_trunk_v3.patch, log Currently, migration works but there's still a couple of rough edges: - HBASE-8045 finished the .META. migration but didn't remove ROOT, so it's still on the filesystem. - Data in ZK needs to be removed manually. Either we fix up the data in ZK or we delete it ourselves. - TestMetaMigrationRemovingHTD has a testMetaUpdatedFlagInROOT method, but ROOT is gone now. Elliott was also mentioning that we could have hbase migrate do the HFileV1 checks, clear ZK, remove ROOT, etc. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8348) Polish the migration to 0.96
[ https://issues.apache.org/jira/browse/HBASE-8348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Himanshu Vashishtha updated HBASE-8348: --- Attachment: (was: HBASE-8348-approach-2-v2.3.patch) Polish the migration to 0.96 Key: HBASE-8348 URL: https://issues.apache.org/jira/browse/HBASE-8348 Project: HBase Issue Type: Bug Affects Versions: 0.95.0 Reporter: Jean-Daniel Cryans Assignee: rajeshbabu Priority: Blocker Fix For: 0.96.0 Attachments: HBASE-8348-approach-2.patch, HBASE-8348-approach-2-v2.1.patch, HBASE-8348-approach-2-v2.2.patch, HBASE-8348-approach-2-v2.3.patch, HBASE-8348-approach-3.patch, HBASE-8348_trunk.patch, HBASE-8348_trunk_v2.patch, HBASE-8348_trunk_v3.patch, log Currently, migration works but there's still a couple of rough edges: - HBASE-8045 finished the .META. migration but didn't remove ROOT, so it's still on the filesystem. - Data in ZK needs to be removed manually. Either we fix up the data in ZK or we delete it ourselves. - TestMetaMigrationRemovingHTD has a testMetaUpdatedFlagInROOT method, but ROOT is gone now. Elliott was also mentioning that we could have hbase migrate do the HFileV1 checks, clear ZK, remove ROOT, etc. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8348) Polish the migration to 0.96
[ https://issues.apache.org/jira/browse/HBASE-8348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Himanshu Vashishtha updated HBASE-8348: --- Attachment: HBASE-8348-approach-2-v2.3.patch Re-attaching to let qa bot run. Polish the migration to 0.96 Key: HBASE-8348 URL: https://issues.apache.org/jira/browse/HBASE-8348 Project: HBase Issue Type: Bug Affects Versions: 0.95.0 Reporter: Jean-Daniel Cryans Assignee: rajeshbabu Priority: Blocker Fix For: 0.96.0 Attachments: HBASE-8348-approach-2.patch, HBASE-8348-approach-2-v2.1.patch, HBASE-8348-approach-2-v2.2.patch, HBASE-8348-approach-2-v2.3.patch, HBASE-8348-approach-3.patch, HBASE-8348_trunk.patch, HBASE-8348_trunk_v2.patch, HBASE-8348_trunk_v3.patch, log Currently, migration works but there's still a couple of rough edges: - HBASE-8045 finished the .META. migration but didn't remove ROOT, so it's still on the filesystem. - Data in ZK needs to be removed manually. Either we fix up the data in ZK or we delete it ourselves. - TestMetaMigrationRemovingHTD has a testMetaUpdatedFlagInROOT method, but ROOT is gone now. Elliott was also mentioning that we could have hbase migrate do the HFileV1 checks, clear ZK, remove ROOT, etc. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8348) Polish the migration to 0.96
[ https://issues.apache.org/jira/browse/HBASE-8348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Himanshu Vashishtha updated HBASE-8348: --- Attachment: log-2 Polish the migration to 0.96 Key: HBASE-8348 URL: https://issues.apache.org/jira/browse/HBASE-8348 Project: HBase Issue Type: Bug Affects Versions: 0.95.0 Reporter: Jean-Daniel Cryans Assignee: rajeshbabu Priority: Blocker Fix For: 0.96.0 Attachments: HBASE-8348-approach-2.patch, HBASE-8348-approach-2-v2.1.patch, HBASE-8348-approach-2-v2.2.patch, HBASE-8348-approach-2-v2.3.patch, HBASE-8348-approach-3.patch, HBASE-8348_trunk.patch, HBASE-8348_trunk_v2.patch, HBASE-8348_trunk_v3.patch, log, log-2 Currently, migration works but there's still a couple of rough edges: - HBASE-8045 finished the .META. migration but didn't remove ROOT, so it's still on the filesystem. - Data in ZK needs to be removed manually. Either we fix up the data in ZK or we delete it ourselves. - TestMetaMigrationRemovingHTD has a testMetaUpdatedFlagInROOT method, but ROOT is gone now. Elliott was also mentioning that we could have hbase migrate do the HFileV1 checks, clear ZK, remove ROOT, etc. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8348) Polish the migration to 0.96
[ https://issues.apache.org/jira/browse/HBASE-8348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Himanshu Vashishtha updated HBASE-8348: --- Attachment: HBASE-8348-approach-2-v2.4.patch Polish the migration to 0.96 Key: HBASE-8348 URL: https://issues.apache.org/jira/browse/HBASE-8348 Project: HBase Issue Type: Bug Affects Versions: 0.95.0 Reporter: Jean-Daniel Cryans Assignee: rajeshbabu Priority: Blocker Fix For: 0.96.0 Attachments: HBASE-8348-approach-2.patch, HBASE-8348-approach-2-v2.1.patch, HBASE-8348-approach-2-v2.2.patch, HBASE-8348-approach-2-v2.3.patch, HBASE-8348-approach-2-v2.4.patch, HBASE-8348-approach-3.patch, HBASE-8348_trunk.patch, HBASE-8348_trunk_v2.patch, HBASE-8348_trunk_v3.patch, log, log-2 Currently, migration works but there's still a couple of rough edges: - HBASE-8045 finished the .META. migration but didn't remove ROOT, so it's still on the filesystem. - Data in ZK needs to be removed manually. Either we fix up the data in ZK or we delete it ourselves. - TestMetaMigrationRemovingHTD has a testMetaUpdatedFlagInROOT method, but ROOT is gone now. Elliott was also mentioning that we could have hbase migrate do the HFileV1 checks, clear ZK, remove ROOT, etc. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8348) Polish the migration to 0.96
[ https://issues.apache.org/jira/browse/HBASE-8348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-8348: - Attachment: 8348v5.txt Here are some mods to the last version of the patch making usage better reflect the mutual exclusivity, moving upgrade up from package to commands, made usage consistent so has '-' everywhere instead of '--' and then '-' sometimes in the help, removed alternative version so commands -- alternatives too long. Polish the migration to 0.96 Key: HBASE-8348 URL: https://issues.apache.org/jira/browse/HBASE-8348 Project: HBase Issue Type: Bug Affects Versions: 0.95.0 Reporter: Jean-Daniel Cryans Assignee: rajeshbabu Priority: Blocker Fix For: 0.96.0 Attachments: 8348v5.txt, HBASE-8348-approach-2.patch, HBASE-8348-approach-2-v2.1.patch, HBASE-8348-approach-2-v2.2.patch, HBASE-8348-approach-2-v2.3.patch, HBASE-8348-approach-2-v2.4.patch, HBASE-8348-approach-3.patch, HBASE-8348_trunk.patch, HBASE-8348_trunk_v2.patch, HBASE-8348_trunk_v3.patch, log, log-2 Currently, migration works but there's still a couple of rough edges: - HBASE-8045 finished the .META. migration but didn't remove ROOT, so it's still on the filesystem. - Data in ZK needs to be removed manually. Either we fix up the data in ZK or we delete it ourselves. - TestMetaMigrationRemovingHTD has a testMetaUpdatedFlagInROOT method, but ROOT is gone now. Elliott was also mentioning that we could have hbase migrate do the HFileV1 checks, clear ZK, remove ROOT, etc. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8348) Polish the migration to 0.96
[ https://issues.apache.org/jira/browse/HBASE-8348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-8348: - Attachment: 8348v5.txt include new files. Polish the migration to 0.96 Key: HBASE-8348 URL: https://issues.apache.org/jira/browse/HBASE-8348 Project: HBase Issue Type: Bug Affects Versions: 0.95.0 Reporter: Jean-Daniel Cryans Assignee: rajeshbabu Priority: Blocker Fix For: 0.96.0 Attachments: 8348v5.txt, 8348v5.txt, HBASE-8348-approach-2.patch, HBASE-8348-approach-2-v2.1.patch, HBASE-8348-approach-2-v2.2.patch, HBASE-8348-approach-2-v2.3.patch, HBASE-8348-approach-2-v2.4.patch, HBASE-8348-approach-3.patch, HBASE-8348_trunk.patch, HBASE-8348_trunk_v2.patch, HBASE-8348_trunk_v3.patch, log, log-2 Currently, migration works but there's still a couple of rough edges: - HBASE-8045 finished the .META. migration but didn't remove ROOT, so it's still on the filesystem. - Data in ZK needs to be removed manually. Either we fix up the data in ZK or we delete it ourselves. - TestMetaMigrationRemovingHTD has a testMetaUpdatedFlagInROOT method, but ROOT is gone now. Elliott was also mentioning that we could have hbase migrate do the HFileV1 checks, clear ZK, remove ROOT, etc. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8348) Polish the migration to 0.96
[ https://issues.apache.org/jira/browse/HBASE-8348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-8348: - Priority: Blocker (was: Critical) Polish the migration to 0.96 Key: HBASE-8348 URL: https://issues.apache.org/jira/browse/HBASE-8348 Project: HBase Issue Type: Bug Affects Versions: 0.95.0 Reporter: Jean-Daniel Cryans Assignee: rajeshbabu Priority: Blocker Fix For: 0.96.0 Attachments: HBASE-8348-approach-2.patch, HBASE-8348-approach-2-v2.1.patch, HBASE-8348-approach-2-v2.2.patch, HBASE-8348-approach-3.patch, HBASE-8348_trunk.patch, HBASE-8348_trunk_v2.patch, HBASE-8348_trunk_v3.patch Currently, migration works but there's still a couple of rough edges: - HBASE-8045 finished the .META. migration but didn't remove ROOT, so it's still on the filesystem. - Data in ZK needs to be removed manually. Either we fix up the data in ZK or we delete it ourselves. - TestMetaMigrationRemovingHTD has a testMetaUpdatedFlagInROOT method, but ROOT is gone now. Elliott was also mentioning that we could have hbase migrate do the HFileV1 checks, clear ZK, remove ROOT, etc. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8348) Polish the migration to 0.96
[ https://issues.apache.org/jira/browse/HBASE-8348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Himanshu Vashishtha updated HBASE-8348: --- Attachment: HBASE-8348-approach-2-v2.2.patch Attaching patch which does: a) correctly set fs (using hbase.rootdir uri). b) minor refactoring in UpgradeTo96 (initially, we were checking for live processes even if user doesn't set execute option). Rest of the patch is similar to old one. Thanks. Polish the migration to 0.96 Key: HBASE-8348 URL: https://issues.apache.org/jira/browse/HBASE-8348 Project: HBase Issue Type: Bug Affects Versions: 0.95.0 Reporter: Jean-Daniel Cryans Assignee: rajeshbabu Priority: Critical Fix For: 0.96.0 Attachments: HBASE-8348-approach-2.patch, HBASE-8348-approach-2-v2.1.patch, HBASE-8348-approach-2-v2.2.patch, HBASE-8348-approach-3.patch, HBASE-8348_trunk.patch, HBASE-8348_trunk_v2.patch, HBASE-8348_trunk_v3.patch Currently, migration works but there's still a couple of rough edges: - HBASE-8045 finished the .META. migration but didn't remove ROOT, so it's still on the filesystem. - Data in ZK needs to be removed manually. Either we fix up the data in ZK or we delete it ourselves. - TestMetaMigrationRemovingHTD has a testMetaUpdatedFlagInROOT method, but ROOT is gone now. Elliott was also mentioning that we could have hbase migrate do the HFileV1 checks, clear ZK, remove ROOT, etc. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8348) Polish the migration to 0.96
[ https://issues.apache.org/jira/browse/HBASE-8348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Himanshu Vashishtha updated HBASE-8348: --- Attachment: HBASE-8348-approach-2-v2.1.patch Polish the migration to 0.96 Key: HBASE-8348 URL: https://issues.apache.org/jira/browse/HBASE-8348 Project: HBase Issue Type: Bug Affects Versions: 0.95.0 Reporter: Jean-Daniel Cryans Assignee: rajeshbabu Priority: Critical Fix For: 0.96.0 Attachments: HBASE-8348-approach-2.patch, HBASE-8348-approach-2-v2.1.patch, HBASE-8348-approach-3.patch, HBASE-8348_trunk.patch, HBASE-8348_trunk_v2.patch, HBASE-8348_trunk_v3.patch Currently, migration works but there's still a couple of rough edges: - HBASE-8045 finished the .META. migration but didn't remove ROOT, so it's still on the filesystem. - Data in ZK needs to be removed manually. Either we fix up the data in ZK or we delete it ourselves. - TestMetaMigrationRemovingHTD has a testMetaUpdatedFlagInROOT method, but ROOT is gone now. Elliott was also mentioning that we could have hbase migrate do the HFileV1 checks, clear ZK, remove ROOT, etc. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8348) Polish the migration to 0.96
[ https://issues.apache.org/jira/browse/HBASE-8348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] rajeshbabu updated HBASE-8348: -- Attachment: HBASE-8348-approach-3.patch This is really superb Himanshu. Only thing missing is checking for no live processes exist before upgrade.In current patch I have added that check and skipping upgrade if any live processes exists. Tested in the cluster. Its fine as well. Removed TestMetaMigrationRemovingHTD#testMetaUpdatedFlagInROOT test. The reason for TestUpgradeTo96#testHFileV1Detector failure is because RootKeyComparator class not exists so not able deserialize trailer propery. It got removed recently as part of HBASE-9196(Remove dead code related to KeyValue) We need to add it back or skip root table hfile v1 check. Polish the migration to 0.96 Key: HBASE-8348 URL: https://issues.apache.org/jira/browse/HBASE-8348 Project: HBase Issue Type: Bug Affects Versions: 0.95.0 Reporter: Jean-Daniel Cryans Assignee: rajeshbabu Priority: Critical Fix For: 0.96.0 Attachments: HBASE-8348-approach-2.patch, HBASE-8348-approach-3.patch, HBASE-8348_trunk.patch, HBASE-8348_trunk_v2.patch, HBASE-8348_trunk_v3.patch Currently, migration works but there's still a couple of rough edges: - HBASE-8045 finished the .META. migration but didn't remove ROOT, so it's still on the filesystem. - Data in ZK needs to be removed manually. Either we fix up the data in ZK or we delete it ourselves. - TestMetaMigrationRemovingHTD has a testMetaUpdatedFlagInROOT method, but ROOT is gone now. Elliott was also mentioning that we could have hbase migrate do the HFileV1 checks, clear ZK, remove ROOT, etc. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8348) Polish the migration to 0.96
[ https://issues.apache.org/jira/browse/HBASE-8348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Himanshu Vashishtha updated HBASE-8348: --- Attachment: HBASE-8348-approach-2.patch Polish the migration to 0.96 Key: HBASE-8348 URL: https://issues.apache.org/jira/browse/HBASE-8348 Project: HBase Issue Type: Bug Affects Versions: 0.95.0 Reporter: Jean-Daniel Cryans Assignee: rajeshbabu Priority: Critical Fix For: 0.96.0 Attachments: HBASE-8348-approach-2.patch, HBASE-8348_trunk.patch, HBASE-8348_trunk_v2.patch, HBASE-8348_trunk_v3.patch Currently, migration works but there's still a couple of rough edges: - HBASE-8045 finished the .META. migration but didn't remove ROOT, so it's still on the filesystem. - Data in ZK needs to be removed manually. Either we fix up the data in ZK or we delete it ourselves. - TestMetaMigrationRemovingHTD has a testMetaUpdatedFlagInROOT method, but ROOT is gone now. Elliott was also mentioning that we could have hbase migrate do the HFileV1 checks, clear ZK, remove ROOT, etc. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8348) Polish the migration to 0.96
[ https://issues.apache.org/jira/browse/HBASE-8348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Himanshu Vashishtha updated HBASE-8348: --- Attachment: (was: HBASE-8348-approach-2.patch) Polish the migration to 0.96 Key: HBASE-8348 URL: https://issues.apache.org/jira/browse/HBASE-8348 Project: HBase Issue Type: Bug Affects Versions: 0.95.0 Reporter: Jean-Daniel Cryans Assignee: rajeshbabu Priority: Critical Fix For: 0.96.0 Attachments: HBASE-8348_trunk.patch, HBASE-8348_trunk_v2.patch, HBASE-8348_trunk_v3.patch Currently, migration works but there's still a couple of rough edges: - HBASE-8045 finished the .META. migration but didn't remove ROOT, so it's still on the filesystem. - Data in ZK needs to be removed manually. Either we fix up the data in ZK or we delete it ourselves. - TestMetaMigrationRemovingHTD has a testMetaUpdatedFlagInROOT method, but ROOT is gone now. Elliott was also mentioning that we could have hbase migrate do the HFileV1 checks, clear ZK, remove ROOT, etc. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8348) Polish the migration to 0.96
[ https://issues.apache.org/jira/browse/HBASE-8348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Himanshu Vashishtha updated HBASE-8348: --- Attachment: HBASE-8348-approach-2.patch Polish the migration to 0.96 Key: HBASE-8348 URL: https://issues.apache.org/jira/browse/HBASE-8348 Project: HBase Issue Type: Bug Affects Versions: 0.95.0 Reporter: Jean-Daniel Cryans Assignee: rajeshbabu Priority: Critical Fix For: 0.96.0 Attachments: HBASE-8348-approach-2.patch, HBASE-8348_trunk.patch, HBASE-8348_trunk_v2.patch, HBASE-8348_trunk_v3.patch Currently, migration works but there's still a couple of rough edges: - HBASE-8045 finished the .META. migration but didn't remove ROOT, so it's still on the filesystem. - Data in ZK needs to be removed manually. Either we fix up the data in ZK or we delete it ourselves. - TestMetaMigrationRemovingHTD has a testMetaUpdatedFlagInROOT method, but ROOT is gone now. Elliott was also mentioning that we could have hbase migrate do the HFileV1 checks, clear ZK, remove ROOT, etc. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8348) Polish the migration to 0.96
[ https://issues.apache.org/jira/browse/HBASE-8348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-8348: - Fix Version/s: (was: 0.95.2) 0.96.0 Polish the migration to 0.96 Key: HBASE-8348 URL: https://issues.apache.org/jira/browse/HBASE-8348 Project: HBase Issue Type: Bug Affects Versions: 0.95.0 Reporter: Jean-Daniel Cryans Assignee: rajeshbabu Priority: Critical Fix For: 0.96.0 Attachments: HBASE-8348_trunk.patch, HBASE-8348_trunk_v2.patch, HBASE-8348_trunk_v3.patch Currently, migration works but there's still a couple of rough edges: - HBASE-8045 finished the .META. migration but didn't remove ROOT, so it's still on the filesystem. - Data in ZK needs to be removed manually. Either we fix up the data in ZK or we delete it ourselves. - TestMetaMigrationRemovingHTD has a testMetaUpdatedFlagInROOT method, but ROOT is gone now. Elliott was also mentioning that we could have hbase migrate do the HFileV1 checks, clear ZK, remove ROOT, etc. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8348) Polish the migration to 0.96
[ https://issues.apache.org/jira/browse/HBASE-8348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] rajeshbabu updated HBASE-8348: -- Attachment: HBASE-8348_trunk_v3.patch Here is the patch includes migrations script Migration steps include: 1) running NamespaceUpgrade tool - migrates hdfs data 2) running ZK data migrator tool - migrates zookeeper data. current patch migrating replication znodes as well. 3) removing hbase.id (to avoid HBASE-9177) 4) running HFileV1Detector tool - to identify v1 hfiles. Tested the patch in the cluster === 1) installed two clusters with 0.94.11 version hbase. Enabled replication. i) master cluster ii) slave cluster 2) created table with replication scope 1 and done some puts and deletes. replication in progress 3) stopped two clusters. 4) ran migrate script 5) restarted two clusters with 0.96.0 version hbase. 6) done some puts and replication also happening as usual. Everything fine. TODO: We can use replication backup tool once its ready Polish the migration to 0.96 Key: HBASE-8348 URL: https://issues.apache.org/jira/browse/HBASE-8348 Project: HBase Issue Type: Bug Affects Versions: 0.95.0 Reporter: Jean-Daniel Cryans Assignee: rajeshbabu Priority: Critical Fix For: 0.95.2 Attachments: HBASE-8348_trunk.patch, HBASE-8348_trunk_v2.patch, HBASE-8348_trunk_v3.patch Currently, migration works but there's still a couple of rough edges: - HBASE-8045 finished the .META. migration but didn't remove ROOT, so it's still on the filesystem. - Data in ZK needs to be removed manually. Either we fix up the data in ZK or we delete it ourselves. - TestMetaMigrationRemovingHTD has a testMetaUpdatedFlagInROOT method, but ROOT is gone now. Elliott was also mentioning that we could have hbase migrate do the HFileV1 checks, clear ZK, remove ROOT, etc. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8348) Polish the migration to 0.96
[ https://issues.apache.org/jira/browse/HBASE-8348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] rajeshbabu updated HBASE-8348: -- Attachment: HBASE-8348_trunk.patch Here is the patch for trunk which includes migration tool to 1) Migrate zookeeper data if no hbase process is alive ■ Find .META region server from -ROOT- and create meta-region-server ■ Migrate table states to PB(solve HBASE-7766 and helps to find exact state of a table otherwise even disabled tables also be will enabled during master startup.) ■ clean the znodes which are not used in 0.95.0 or not needed during clean cluster startup(znodes like rs,root-region-server,unassigned,splitlog,hbase-id,shutdown are removed). 2) remove -ROOT- table from filesystem 3) Identify HFiles of version 1(using HFileV1Detector tool). and also removed TestMetaMigrationRemovingHTD#testMetaUpdatedFlagInROOT [~saint@gmail.com] bq. Also ..test that 0.95 hlogs and hfiles are readable (from Matteo) Able to read hfiles and user table hlogs of older hbase version properly but not able to read META table hlogs because we are writing edits of it into a separate hlog suffixed by -meta. So not able to find it from hlogs od older version. I think It would be better to flush -ROOT- and .META. tables atleast before stopping the cluster. Please suggest some solution for this scenario. Please review the patch and provide any suggestions/comments. Polish the migration to 0.96 Key: HBASE-8348 URL: https://issues.apache.org/jira/browse/HBASE-8348 Project: HBase Issue Type: Bug Affects Versions: 0.95.0 Reporter: Jean-Daniel Cryans Priority: Critical Fix For: 0.95.2 Attachments: HBASE-8348_trunk.patch Currently, migration works but there's still a couple of rough edges: - HBASE-8045 finished the .META. migration but didn't remove ROOT, so it's still on the filesystem. - Data in ZK needs to be removed manually. Either we fix up the data in ZK or we delete it ourselves. - TestMetaMigrationRemovingHTD has a testMetaUpdatedFlagInROOT method, but ROOT is gone now. Elliott was also mentioning that we could have hbase migrate do the HFileV1 checks, clear ZK, remove ROOT, etc. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8348) Polish the migration to 0.96
[ https://issues.apache.org/jira/browse/HBASE-8348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-8348: -- Status: Patch Available (was: Open) Polish the migration to 0.96 Key: HBASE-8348 URL: https://issues.apache.org/jira/browse/HBASE-8348 Project: HBase Issue Type: Bug Affects Versions: 0.95.0 Reporter: Jean-Daniel Cryans Priority: Critical Fix For: 0.95.2 Attachments: HBASE-8348_trunk.patch Currently, migration works but there's still a couple of rough edges: - HBASE-8045 finished the .META. migration but didn't remove ROOT, so it's still on the filesystem. - Data in ZK needs to be removed manually. Either we fix up the data in ZK or we delete it ourselves. - TestMetaMigrationRemovingHTD has a testMetaUpdatedFlagInROOT method, but ROOT is gone now. Elliott was also mentioning that we could have hbase migrate do the HFileV1 checks, clear ZK, remove ROOT, etc. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8348) Polish the migration to 0.96
[ https://issues.apache.org/jira/browse/HBASE-8348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] rajeshbabu updated HBASE-8348: -- Attachment: HBASE-8348_trunk_v2.patch setting default fs in HFileV1Detector tool. Polish the migration to 0.96 Key: HBASE-8348 URL: https://issues.apache.org/jira/browse/HBASE-8348 Project: HBase Issue Type: Bug Affects Versions: 0.95.0 Reporter: Jean-Daniel Cryans Assignee: rajeshbabu Priority: Critical Fix For: 0.95.2 Attachments: HBASE-8348_trunk.patch, HBASE-8348_trunk_v2.patch Currently, migration works but there's still a couple of rough edges: - HBASE-8045 finished the .META. migration but didn't remove ROOT, so it's still on the filesystem. - Data in ZK needs to be removed manually. Either we fix up the data in ZK or we delete it ourselves. - TestMetaMigrationRemovingHTD has a testMetaUpdatedFlagInROOT method, but ROOT is gone now. Elliott was also mentioning that we could have hbase migrate do the HFileV1 checks, clear ZK, remove ROOT, etc. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8348) Polish the migration to 0.96
[ https://issues.apache.org/jira/browse/HBASE-8348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-8348: - Priority: Critical (was: Major) Polish the migration to 0.96 Key: HBASE-8348 URL: https://issues.apache.org/jira/browse/HBASE-8348 Project: HBase Issue Type: Bug Affects Versions: 0.95.0 Reporter: Jean-Daniel Cryans Priority: Critical Fix For: 0.95.2 Currently, migration works but there's still a couple of rough edges: - HBASE-8045 finished the .META. migration but didn't remove ROOT, so it's still on the filesystem. - Data in ZK needs to be removed manually. Either we fix up the data in ZK or we delete it ourselves. - TestMetaMigrationRemovingHTD has a testMetaUpdatedFlagInROOT method, but ROOT is gone now. Elliott was also mentioning that we could have hbase migrate do the HFileV1 checks, clear ZK, remove ROOT, etc. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
