[jira] [Updated] (HBASE-16554) Procedure V2 - handle corruption of WAL trailer
[ https://issues.apache.org/jira/browse/HBASE-16554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy updated HBASE-16554: - Fix Version/s: 2.0.0 > Procedure V2 - handle corruption of WAL trailer > --- > > Key: HBASE-16554 > URL: https://issues.apache.org/jira/browse/HBASE-16554 > Project: HBase > Issue Type: Sub-task >Reporter: Appy >Assignee: Appy > Fix For: 2.0.0 > > Attachments: HBASE-16554.master.001.patch, > HBASE-16554.master.002.patch, tracker-rebuild.patch > > > If the last wal was closed cleanly, the global tracker will be the last wal > tracker (no rebuild needed) > if the last wal does not have a tracker (corrupted/master-killed). on load() > we will rebuild the global tracker. > To compute quickly which files should be deleted, we also want the tracker of > each file. > if the wal was closed properly and has a tracker we are good, if not we need > to rebuild the tracker for that file. > each file tracker contains a bitmap about what is in the wal (the updated > bitmap), which is easy to compute just by reading each entry of the wal. and > a bitmap that keeps track of the "running procedures" up to that wal (the > deleted bitmap). The delete bitmap requires a bit of post read-all-wals work. > and it will basically require to AND the deleted bitmap of wal\(i\) and > wal(i-1) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16554) Procedure V2 - handle corruption of WAL trailer
[ https://issues.apache.org/jira/browse/HBASE-16554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy updated HBASE-16554: - Attachment: HBASE-16554.master.002.patch > Procedure V2 - handle corruption of WAL trailer > --- > > Key: HBASE-16554 > URL: https://issues.apache.org/jira/browse/HBASE-16554 > Project: HBase > Issue Type: Sub-task >Reporter: Appy >Assignee: Appy > Attachments: HBASE-16554.master.001.patch, > HBASE-16554.master.002.patch, tracker-rebuild.patch > > > If the last wal was closed cleanly, the global tracker will be the last wal > tracker (no rebuild needed) > if the last wal does not have a tracker (corrupted/master-killed). on load() > we will rebuild the global tracker. > To compute quickly which files should be deleted, we also want the tracker of > each file. > if the wal was closed properly and has a tracker we are good, if not we need > to rebuild the tracker for that file. > each file tracker contains a bitmap about what is in the wal (the updated > bitmap), which is easy to compute just by reading each entry of the wal. and > a bitmap that keeps track of the "running procedures" up to that wal (the > deleted bitmap). The delete bitmap requires a bit of post read-all-wals work. > and it will basically require to AND the deleted bitmap of wal\(i\) and > wal(i-1) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16554) Procedure V2 - handle corruption of WAL trailer
[ https://issues.apache.org/jira/browse/HBASE-16554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy updated HBASE-16554: - Attachment: HBASE-16554.master.001.patch > Procedure V2 - handle corruption of WAL trailer > --- > > Key: HBASE-16554 > URL: https://issues.apache.org/jira/browse/HBASE-16554 > Project: HBase > Issue Type: Sub-task >Reporter: Appy >Assignee: Appy > Attachments: HBASE-16554.master.001.patch, tracker-rebuild.patch > > > If the last wal was closed cleanly, the global tracker will be the last wal > tracker (no rebuild needed) > if the last wal does not have a tracker (corrupted/master-killed). on load() > we will rebuild the global tracker. > To compute quickly which files should be deleted, we also want the tracker of > each file. > if the wal was closed properly and has a tracker we are good, if not we need > to rebuild the tracker for that file. > each file tracker contains a bitmap about what is in the wal (the updated > bitmap), which is easy to compute just by reading each entry of the wal. and > a bitmap that keeps track of the "running procedures" up to that wal (the > deleted bitmap). The delete bitmap requires a bit of post read-all-wals work. > and it will basically require to AND the deleted bitmap of wal\(i\) and > wal(i-1) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16554) Procedure V2 - handle corruption of WAL trailer
[ https://issues.apache.org/jira/browse/HBASE-16554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy updated HBASE-16554: - Status: Patch Available (was: Open) > Procedure V2 - handle corruption of WAL trailer > --- > > Key: HBASE-16554 > URL: https://issues.apache.org/jira/browse/HBASE-16554 > Project: HBase > Issue Type: Sub-task >Reporter: Appy >Assignee: Appy > Attachments: HBASE-16554.master.001.patch, tracker-rebuild.patch > > > If the last wal was closed cleanly, the global tracker will be the last wal > tracker (no rebuild needed) > if the last wal does not have a tracker (corrupted/master-killed). on load() > we will rebuild the global tracker. > To compute quickly which files should be deleted, we also want the tracker of > each file. > if the wal was closed properly and has a tracker we are good, if not we need > to rebuild the tracker for that file. > each file tracker contains a bitmap about what is in the wal (the updated > bitmap), which is easy to compute just by reading each entry of the wal. and > a bitmap that keeps track of the "running procedures" up to that wal (the > deleted bitmap). The delete bitmap requires a bit of post read-all-wals work. > and it will basically require to AND the deleted bitmap of wal\(i\) and > wal(i-1) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16554) Procedure V2 - handle corruption of WAL trailer
[ https://issues.apache.org/jira/browse/HBASE-16554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy updated HBASE-16554: - Description: If the last wal was closed cleanly, the global tracker will be the last wal tracker (no rebuild needed) if the last wal does not have a tracker (corrupted/master-killed). on load() we will rebuild the global tracker. To compute quickly which files should be deleted, we also want the tracker of each file. if the wal was closed properly and has a tracker we are good, if not we need to rebuild the tracker for that file. each file tracker contains a bitmap about what is in the wal (the updated bitmap), which is easy to compute just by reading each entry of the wal. and a bitmap that keeps track of the "running procedures" up to that wal (the deleted bitmap). The delete bitmap requires a bit of post read-all-wals work. and it will basically require to AND the deleted bitmap of wal\(i\) and wal(i-1) was: If the last wal was closed cleanly, the global tracker will be the last wal tracker (no rebuild needed) if the last wal does not have a tracker (corrupted/master-killed). on load() we will rebuild the global tracker. To compute quickly which files should be deleted, we also want the tracker of each file. if the wal was closed properly and has a tracker we are good, if not we need to rebuild the tracker for that file. each file tracker contains a bitmap about what is in the wal (the updated bitmap), which is easy to compute just by reading each entry of the wal. and a bitmap that keeps track of the "running procedures" up to that wal (the deleted bitmap). The delete bitmap requires a bit of post read-all-wals work. and it will basically require to AND the deleted bitmap of wal(i) and wal(i-1) > Procedure V2 - handle corruption of WAL trailer > --- > > Key: HBASE-16554 > URL: https://issues.apache.org/jira/browse/HBASE-16554 > Project: HBase > Issue Type: Sub-task >Reporter: Appy >Assignee: Appy > Attachments: tracker-rebuild.patch > > > If the last wal was closed cleanly, the global tracker will be the last wal > tracker (no rebuild needed) > if the last wal does not have a tracker (corrupted/master-killed). on load() > we will rebuild the global tracker. > To compute quickly which files should be deleted, we also want the tracker of > each file. > if the wal was closed properly and has a tracker we are good, if not we need > to rebuild the tracker for that file. > each file tracker contains a bitmap about what is in the wal (the updated > bitmap), which is easy to compute just by reading each entry of the wal. and > a bitmap that keeps track of the "running procedures" up to that wal (the > deleted bitmap). The delete bitmap requires a bit of post read-all-wals work. > and it will basically require to AND the deleted bitmap of wal\(i\) and > wal(i-1) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16554) Procedure V2 - handle corruption of WAL trailer
[ https://issues.apache.org/jira/browse/HBASE-16554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matteo Bertozzi updated HBASE-16554: Description: If the last wal was closed cleanly, the global tracker will be the last wal tracker (no rebuild needed) if the last wal does not have a tracker (corrupted/master-killed). on load() we will rebuild the global tracker. To compute quickly which files should be deleted, we also want the tracker of each file. if the wal was closed properly and has a tracker we are good, if not we need to rebuild the tracker for that file. each file tracker contains a bitmap about what is in the wal (the updated bitmap), which is easy to compute just by reading each entry of the wal. and a bitmap that keeps track of the "running procedures" up to that wal (the deleted bitmap). The delete bitmap requires a bit of post read-all-wals work. and it will basically require to AND the deleted bitmap of was:if trailer is corrupted, build tracker for each WAL when loading, then use these trackers to build global tracker. > Procedure V2 - handle corruption of WAL trailer > --- > > Key: HBASE-16554 > URL: https://issues.apache.org/jira/browse/HBASE-16554 > Project: HBase > Issue Type: Sub-task >Reporter: Appy >Assignee: Appy > Attachments: tracker-rebuild.patch > > > If the last wal was closed cleanly, the global tracker will be the last wal > tracker (no rebuild needed) > if the last wal does not have a tracker (corrupted/master-killed). on load() > we will rebuild the global tracker. > To compute quickly which files should be deleted, we also want the tracker of > each file. > if the wal was closed properly and has a tracker we are good, if not we need > to rebuild the tracker for that file. > each file tracker contains a bitmap about what is in the wal (the updated > bitmap), which is easy to compute just by reading each entry of the wal. and > a bitmap that keeps track of the "running procedures" up to that wal (the > deleted bitmap). The delete bitmap requires a bit of post read-all-wals work. > and it will basically require to AND the deleted bitmap of -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16554) Procedure V2 - handle corruption of WAL trailer
[ https://issues.apache.org/jira/browse/HBASE-16554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matteo Bertozzi updated HBASE-16554: Description: If the last wal was closed cleanly, the global tracker will be the last wal tracker (no rebuild needed) if the last wal does not have a tracker (corrupted/master-killed). on load() we will rebuild the global tracker. To compute quickly which files should be deleted, we also want the tracker of each file. if the wal was closed properly and has a tracker we are good, if not we need to rebuild the tracker for that file. each file tracker contains a bitmap about what is in the wal (the updated bitmap), which is easy to compute just by reading each entry of the wal. and a bitmap that keeps track of the "running procedures" up to that wal (the deleted bitmap). The delete bitmap requires a bit of post read-all-wals work. and it will basically require to AND the deleted bitmap of wal(i) and wal(i-1) was: If the last wal was closed cleanly, the global tracker will be the last wal tracker (no rebuild needed) if the last wal does not have a tracker (corrupted/master-killed). on load() we will rebuild the global tracker. To compute quickly which files should be deleted, we also want the tracker of each file. if the wal was closed properly and has a tracker we are good, if not we need to rebuild the tracker for that file. each file tracker contains a bitmap about what is in the wal (the updated bitmap), which is easy to compute just by reading each entry of the wal. and a bitmap that keeps track of the "running procedures" up to that wal (the deleted bitmap). The delete bitmap requires a bit of post read-all-wals work. and it will basically require to AND the deleted bitmap of > Procedure V2 - handle corruption of WAL trailer > --- > > Key: HBASE-16554 > URL: https://issues.apache.org/jira/browse/HBASE-16554 > Project: HBase > Issue Type: Sub-task >Reporter: Appy >Assignee: Appy > Attachments: tracker-rebuild.patch > > > If the last wal was closed cleanly, the global tracker will be the last wal > tracker (no rebuild needed) > if the last wal does not have a tracker (corrupted/master-killed). on load() > we will rebuild the global tracker. > To compute quickly which files should be deleted, we also want the tracker of > each file. > if the wal was closed properly and has a tracker we are good, if not we need > to rebuild the tracker for that file. > each file tracker contains a bitmap about what is in the wal (the updated > bitmap), which is easy to compute just by reading each entry of the wal. and > a bitmap that keeps track of the "running procedures" up to that wal (the > deleted bitmap). The delete bitmap requires a bit of post read-all-wals work. > and it will basically require to AND the deleted bitmap of wal(i) and wal(i-1) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-16554) Procedure v2 - handle corruption of WAL trailer
[ https://issues.apache.org/jira/browse/HBASE-16554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy updated HBASE-16554: - Issue Type: Sub-task (was: Bug) Parent: HBASE-14350 > Procedure v2 - handle corruption of WAL trailer > --- > > Key: HBASE-16554 > URL: https://issues.apache.org/jira/browse/HBASE-16554 > Project: HBase > Issue Type: Sub-task >Reporter: Appy >Assignee: Appy > Attachments: tracker-rebuild.patch > > > if trailer is corrupted, build tracker for each WAL when loading, then use > these trackers to build global tracker. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
