[jira] [Updated] (HDFS-15524) Add edit log entry for Snapshot deletion GC thread snapshot deletion
[ https://issues.apache.org/jira/browse/HDFS-15524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shilun Fan updated HDFS-15524: -- Hadoop Flags: Reviewed > Add edit log entry for Snapshot deletion GC thread snapshot deletion > > > Key: HDFS-15524 > URL: https://issues.apache.org/jira/browse/HDFS-15524 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: snapshots >Affects Versions: 3.4.0 >Reporter: Shashikant Banerjee >Assignee: Shashikant Banerjee >Priority: Major > Fix For: 3.4.0 > > > Currently, Snapshot deletion Gc thread doesn't create an edit log transaction > when the actual snapshot is garbage collected. In cases as such, what might > happen is, if the gc thread deletes snapshots and then namenode is > restarted, snapshots which were garbage collected by the snapshot gc thread > prior restart will reapper till the gc thread again picks them up for garbage > collection as the edits were not captured for actual garbage collection and > at the same time data might have already been deleted from the datanodes > which may lead to too many spurious missing block alerts. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-15524) Add edit log entry for Snapshot deletion GC thread snapshot deletion
[ https://issues.apache.org/jira/browse/HDFS-15524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shilun Fan updated HDFS-15524: -- Affects Version/s: 3.4.0 > Add edit log entry for Snapshot deletion GC thread snapshot deletion > > > Key: HDFS-15524 > URL: https://issues.apache.org/jira/browse/HDFS-15524 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: snapshots >Affects Versions: 3.4.0 >Reporter: Shashikant Banerjee >Assignee: Shashikant Banerjee >Priority: Major > Fix For: 3.4.0 > > > Currently, Snapshot deletion Gc thread doesn't create an edit log transaction > when the actual snapshot is garbage collected. In cases as such, what might > happen is, if the gc thread deletes snapshots and then namenode is > restarted, snapshots which were garbage collected by the snapshot gc thread > prior restart will reapper till the gc thread again picks them up for garbage > collection as the edits were not captured for actual garbage collection and > at the same time data might have already been deleted from the datanodes > which may lead to too many spurious missing block alerts. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-15524) Add edit log entry for Snapshot deletion GC thread snapshot deletion
[ https://issues.apache.org/jira/browse/HDFS-15524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shashikant Banerjee updated HDFS-15524: --- Fix Version/s: 3.4.0 Resolution: Fixed Status: Resolved (was: Patch Available) > Add edit log entry for Snapshot deletion GC thread snapshot deletion > > > Key: HDFS-15524 > URL: https://issues.apache.org/jira/browse/HDFS-15524 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: snapshots >Reporter: Shashikant Banerjee >Assignee: Shashikant Banerjee >Priority: Major > Fix For: 3.4.0 > > > Currently, Snapshot deletion Gc thread doesn't create an edit log transaction > when the actual snapshot is garbage collected. In cases as such, what might > happen is, if the gc thread deletes snapshots and then namenode is > restarted, snapshots which were garbage collected by the snapshot gc thread > prior restart will reapper till the gc thread again picks them up for garbage > collection as the edits were not captured for actual garbage collection and > at the same time data might have already been deleted from the datanodes > which may lead to too many spurious missing block alerts. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-15524) Add edit log entry for Snapshot deletion GC thread snapshot deletion
[ https://issues.apache.org/jira/browse/HDFS-15524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shashikant Banerjee updated HDFS-15524: --- Status: Patch Available (was: In Progress) > Add edit log entry for Snapshot deletion GC thread snapshot deletion > > > Key: HDFS-15524 > URL: https://issues.apache.org/jira/browse/HDFS-15524 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: snapshots >Reporter: Shashikant Banerjee >Assignee: Shashikant Banerjee >Priority: Major > > Currently, Snapshot deletion Gc thread doesn't create an edit log transaction > when the actual snapshot is garbage collected. In cases as such, what might > happen is, if the gc thread deletes snapshots and then namenode is > restarted, snapshots which were garbage collected by the snapshot gc thread > prior restart will reapper till the gc thread again picks them up for garbage > collection as the edits were not captured for actual garbage collection and > at the same time data might have already been deleted from the datanodes > which may lead to too many spurious missing block alerts. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
