[jira] [Updated] (HBASE-7245) Recovery on failed snapshot restore
[ https://issues.apache.org/jira/browse/HBASE-7245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-7245: - Fix Version/s: (was: 2.0.0) > Recovery on failed snapshot restore > --- > > Key: HBASE-7245 > URL: https://issues.apache.org/jira/browse/HBASE-7245 > Project: HBase > Issue Type: Bug > Components: Client, master, regionserver, snapshots, Zookeeper >Reporter: Jonathan Hsieh >Priority: Major > > Restore will do updates to the file system and to meta. it seems that an > inopportune failure before meta is completely updated could result in an > inconsistent state that would require hbck to fix. > We should define what the semantics are for recovering from this. Some > suggestions: > 1) Fail Forward (see some log saying restore's meta edits not completed, then > gather information necessary to build it all from fs, and complete meta > edits.). > 2) Fail backwards (see some log saying restore's meta edits not completed, > delete incomplete snapshot region entries from meta.) > I think I prefer 1 -- if two processes end somehow updating (somehow the > original master didn't die, and a new one started up) they would be > idempotent. If we used 2, we could still have a race and still be in a bad > place. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HBASE-7245) Recovery on failed snapshot restore
[ https://issues.apache.org/jira/browse/HBASE-7245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar updated HBASE-7245: - Fix Version/s: (was: 0.99.1) 2.0.0 Recovery on failed snapshot restore --- Key: HBASE-7245 URL: https://issues.apache.org/jira/browse/HBASE-7245 Project: HBase Issue Type: Bug Components: Client, master, regionserver, snapshots, Zookeeper Reporter: Jonathan Hsieh Assignee: Matteo Bertozzi Fix For: 2.0.0 Restore will do updates to the file system and to meta. it seems that an inopportune failure before meta is completely updated could result in an inconsistent state that would require hbck to fix. We should define what the semantics are for recovering from this. Some suggestions: 1) Fail Forward (see some log saying restore's meta edits not completed, then gather information necessary to build it all from fs, and complete meta edits.). 2) Fail backwards (see some log saying restore's meta edits not completed, delete incomplete snapshot region entries from meta.) I think I prefer 1 -- if two processes end somehow updating (somehow the original master didn't die, and a new one started up) they would be idempotent. If we used 2, we could still have a race and still be in a bad place. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-7245) Recovery on failed snapshot restore
[ https://issues.apache.org/jira/browse/HBASE-7245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar updated HBASE-7245: - Fix Version/s: (was: 0.99.0) 0.99.1 Recovery on failed snapshot restore --- Key: HBASE-7245 URL: https://issues.apache.org/jira/browse/HBASE-7245 Project: HBase Issue Type: Bug Components: Client, master, regionserver, snapshots, Zookeeper Reporter: Jonathan Hsieh Assignee: Matteo Bertozzi Fix For: 0.99.1 Restore will do updates to the file system and to meta. it seems that an inopportune failure before meta is completely updated could result in an inconsistent state that would require hbck to fix. We should define what the semantics are for recovering from this. Some suggestions: 1) Fail Forward (see some log saying restore's meta edits not completed, then gather information necessary to build it all from fs, and complete meta edits.). 2) Fail backwards (see some log saying restore's meta edits not completed, delete incomplete snapshot region entries from meta.) I think I prefer 1 -- if two processes end somehow updating (somehow the original master didn't die, and a new one started up) they would be idempotent. If we used 2, we could still have a race and still be in a bad place. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-7245) Recovery on failed snapshot restore
[ https://issues.apache.org/jira/browse/HBASE-7245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-7245: - Fix Version/s: (was: 0.96.1) 0.99.0 Recovery on failed snapshot restore --- Key: HBASE-7245 URL: https://issues.apache.org/jira/browse/HBASE-7245 Project: HBase Issue Type: Bug Components: Client, master, regionserver, snapshots, Zookeeper Reporter: Jonathan Hsieh Assignee: Matteo Bertozzi Fix For: 0.99.0 Restore will do updates to the file system and to meta. it seems that an inopportune failure before meta is completely updated could result in an inconsistent state that would require hbck to fix. We should define what the semantics are for recovering from this. Some suggestions: 1) Fail Forward (see some log saying restore's meta edits not completed, then gather information necessary to build it all from fs, and complete meta edits.). 2) Fail backwards (see some log saying restore's meta edits not completed, delete incomplete snapshot region entries from meta.) I think I prefer 1 -- if two processes end somehow updating (somehow the original master didn't die, and a new one started up) they would be idempotent. If we used 2, we could still have a race and still be in a bad place. -- This message was sent by Atlassian JIRA (v6.1.4#6159)
[jira] [Updated] (HBASE-7245) Recovery on failed snapshot restore
[ https://issues.apache.org/jira/browse/HBASE-7245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-7245: - Fix Version/s: (was: 0.96.0) 0.96.1 Recovery on failed snapshot restore --- Key: HBASE-7245 URL: https://issues.apache.org/jira/browse/HBASE-7245 Project: HBase Issue Type: Bug Components: Client, master, regionserver, snapshots, Zookeeper Reporter: Jonathan Hsieh Assignee: Matteo Bertozzi Fix For: 0.96.1 Restore will do updates to the file system and to meta. it seems that an inopportune failure before meta is completely updated could result in an inconsistent state that would require hbck to fix. We should define what the semantics are for recovering from this. Some suggestions: 1) Fail Forward (see some log saying restore's meta edits not completed, then gather information necessary to build it all from fs, and complete meta edits.). 2) Fail backwards (see some log saying restore's meta edits not completed, delete incomplete snapshot region entries from meta.) I think I prefer 1 -- if two processes end somehow updating (somehow the original master didn't die, and a new one started up) they would be idempotent. If we used 2, we could still have a race and still be in a bad place. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Updated] (HBASE-7245) Recovery on failed snapshot restore
[ https://issues.apache.org/jira/browse/HBASE-7245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-7245: - Fix Version/s: (was: 0.95.2) 0.96.0 Recovery on failed snapshot restore --- Key: HBASE-7245 URL: https://issues.apache.org/jira/browse/HBASE-7245 Project: HBase Issue Type: Bug Components: Client, master, regionserver, snapshots, Zookeeper Reporter: Jonathan Hsieh Assignee: Matteo Bertozzi Fix For: 0.96.0 Restore will do updates to the file system and to meta. it seems that an inopportune failure before meta is completely updated could result in an inconsistent state that would require hbck to fix. We should define what the semantics are for recovering from this. Some suggestions: 1) Fail Forward (see some log saying restore's meta edits not completed, then gather information necessary to build it all from fs, and complete meta edits.). 2) Fail backwards (see some log saying restore's meta edits not completed, delete incomplete snapshot region entries from meta.) I think I prefer 1 -- if two processes end somehow updating (somehow the original master didn't die, and a new one started up) they would be idempotent. If we used 2, we could still have a race and still be in a bad place. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7245) Recovery on failed snapshot restore
[ https://issues.apache.org/jira/browse/HBASE-7245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-7245: - Fix Version/s: (was: 0.95.1) 0.95.2 Recovery on failed snapshot restore --- Key: HBASE-7245 URL: https://issues.apache.org/jira/browse/HBASE-7245 Project: HBase Issue Type: Bug Components: Client, master, regionserver, snapshots, Zookeeper Reporter: Jonathan Hsieh Assignee: Matteo Bertozzi Fix For: 0.95.2 Restore will do updates to the file system and to meta. it seems that an inopportune failure before meta is completely updated could result in an inconsistent state that would require hbck to fix. We should define what the semantics are for recovering from this. Some suggestions: 1) Fail Forward (see some log saying restore's meta edits not completed, then gather information necessary to build it all from fs, and complete meta edits.). 2) Fail backwards (see some log saying restore's meta edits not completed, delete incomplete snapshot region entries from meta.) I think I prefer 1 -- if two processes end somehow updating (somehow the original master didn't die, and a new one started up) they would be idempotent. If we used 2, we could still have a race and still be in a bad place. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7245) Recovery on failed snapshot restore
[ https://issues.apache.org/jira/browse/HBASE-7245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-7245: -- Issue Type: Bug (was: Sub-task) Parent: (was: HBASE-6055) Recovery on failed snapshot restore --- Key: HBASE-7245 URL: https://issues.apache.org/jira/browse/HBASE-7245 Project: HBase Issue Type: Bug Components: Client, master, regionserver, snapshots, Zookeeper Reporter: Jonathan Hsieh Assignee: Matteo Bertozzi Fix For: hbase-6055, 0.96.0 Restore will do updates to the file system and to meta. it seems that an inopportune failure before meta is completely updated could result in an inconsistent state that would require hbck to fix. We should define what the semantics are for recovering from this. Some suggestions: 1) Fail Forward (see some log saying restore's meta edits not completed, then gather information necessary to build it all from fs, and complete meta edits.). 2) Fail backwards (see some log saying restore's meta edits not completed, delete incomplete snapshot region entries from meta.) I think I prefer 1 -- if two processes end somehow updating (somehow the original master didn't die, and a new one started up) they would be idempotent. If we used 2, we could still have a race and still be in a bad place. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7245) Recovery on failed snapshot restore
[ https://issues.apache.org/jira/browse/HBASE-7245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-7245: -- Summary: Recovery on failed snapshot restore (was: Recovery on failed restore.) The subject of this JIRA is w.r.t. snapshot restore. So the first two scenarios above can be handled in separate JIRA. The operation directive file can be created in zookeeper. Recovery on failed snapshot restore --- Key: HBASE-7245 URL: https://issues.apache.org/jira/browse/HBASE-7245 Project: HBase Issue Type: Sub-task Components: Client, master, regionserver, snapshots, Zookeeper Reporter: Jonathan Hsieh Assignee: Matteo Bertozzi Fix For: hbase-6055, 0.96.0 Restore will do updates to the file system and to meta. it seems that an inopportune failure before meta is completely updated could result in an inconsistent state that would require hbck to fix. We should define what the semantics are for recovering from this. Some suggestions: 1) Fail Forward (see some log saying restore's meta edits not completed, then gather information necessary to build it all from fs, and complete meta edits.). 2) Fail backwards (see some log saying restore's meta edits not completed, delete incomplete snapshot region entries from meta.) I think I prefer 1 -- if two processes end somehow updating (somehow the original master didn't die, and a new one started up) they would be idempotent. If we used 2, we could still have a race and still be in a bad place. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
