[jira] [Updated] (HDFS-14758) Decrease lease hard limit
[ https://issues.apache.org/jira/browse/HDFS-14758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-14758: -- Fix Version/s: 2.9.3 2.8.6 > Decrease lease hard limit > - > > Key: HDFS-14758 > URL: https://issues.apache.org/jira/browse/HDFS-14758 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Eric Payne >Assignee: hemanthboyina >Priority: Minor > Fix For: 3.3.0, 2.8.6, 2.9.3, 3.1.4, 3.2.2, 2.10.1 > > Attachments: HDFS-14758.001.patch, HDFS-14758.002.patch, > HDFS-14758.003.patch, HDFS-14758.004.patch, HDFS-14758.005.patch, > HDFS-14758.005.patch, HDFS-14758.006.patch > > > The hard limit is currently hard-coded to be 1 hour. This also determines the > NN automatic lease recovery interval. Something like 20 min will make more > sense. > After the 5 min soft limit, other clients can recover the lease. If no one > else takes the lease away, the original client still can renew the lease > within the hard limit. So even after a NN full GC of 8 minutes, leases can be > still valid. > However, there is one risk in reducing the hard limit. E.g. Reduced to 20 > min. If the NN crashes and the manual failover takes more than 20 minutes, > clients will abort. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-14758) Decrease lease hard limit
[ https://issues.apache.org/jira/browse/HDFS-14758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-14758: -- Fix Version/s: 2.10.1 3.2.2 3.1.4 3.3.0 Hadoop Flags: Reviewed Resolution: Fixed Status: Resolved (was: Patch Available) I've committed this to trunk, branch-3.2, branch-3.1 and branch-2.10. Ran all lease related tests. Thanks for working on this, [~hemanthboyina]. > Decrease lease hard limit > - > > Key: HDFS-14758 > URL: https://issues.apache.org/jira/browse/HDFS-14758 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Eric Payne >Assignee: hemanthboyina >Priority: Minor > Fix For: 3.3.0, 3.1.4, 3.2.2, 2.10.1 > > Attachments: HDFS-14758.001.patch, HDFS-14758.002.patch, > HDFS-14758.003.patch, HDFS-14758.004.patch, HDFS-14758.005.patch, > HDFS-14758.005.patch, HDFS-14758.006.patch > > > The hard limit is currently hard-coded to be 1 hour. This also determines the > NN automatic lease recovery interval. Something like 20 min will make more > sense. > After the 5 min soft limit, other clients can recover the lease. If no one > else takes the lease away, the original client still can renew the lease > within the hard limit. So even after a NN full GC of 8 minutes, leases can be > still valid. > However, there is one risk in reducing the hard limit. E.g. Reduced to 20 > min. If the NN crashes and the manual failover takes more than 20 minutes, > clients will abort. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-14758) Decrease lease hard limit
[ https://issues.apache.org/jira/browse/HDFS-14758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hemanthboyina updated HDFS-14758: - Attachment: HDFS-14758.006.patch > Decrease lease hard limit > - > > Key: HDFS-14758 > URL: https://issues.apache.org/jira/browse/HDFS-14758 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Eric Payne >Assignee: hemanthboyina >Priority: Minor > Attachments: HDFS-14758.001.patch, HDFS-14758.002.patch, > HDFS-14758.003.patch, HDFS-14758.004.patch, HDFS-14758.005.patch, > HDFS-14758.005.patch, HDFS-14758.006.patch > > > The hard limit is currently hard-coded to be 1 hour. This also determines the > NN automatic lease recovery interval. Something like 20 min will make more > sense. > After the 5 min soft limit, other clients can recover the lease. If no one > else takes the lease away, the original client still can renew the lease > within the hard limit. So even after a NN full GC of 8 minutes, leases can be > still valid. > However, there is one risk in reducing the hard limit. E.g. Reduced to 20 > min. If the NN crashes and the manual failover takes more than 20 minutes, > clients will abort. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-14758) Decrease lease hard limit
[ https://issues.apache.org/jira/browse/HDFS-14758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-14758: -- Attachment: HDFS-14758.005.patch > Decrease lease hard limit > - > > Key: HDFS-14758 > URL: https://issues.apache.org/jira/browse/HDFS-14758 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Eric Payne >Assignee: hemanthboyina >Priority: Minor > Attachments: HDFS-14758.001.patch, HDFS-14758.002.patch, > HDFS-14758.003.patch, HDFS-14758.004.patch, HDFS-14758.005.patch, > HDFS-14758.005.patch > > > The hard limit is currently hard-coded to be 1 hour. This also determines the > NN automatic lease recovery interval. Something like 20 min will make more > sense. > After the 5 min soft limit, other clients can recover the lease. If no one > else takes the lease away, the original client still can renew the lease > within the hard limit. So even after a NN full GC of 8 minutes, leases can be > still valid. > However, there is one risk in reducing the hard limit. E.g. Reduced to 20 > min. If the NN crashes and the manual failover takes more than 20 minutes, > clients will abort. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-14758) Decrease lease hard limit
[ https://issues.apache.org/jira/browse/HDFS-14758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hemanthboyina updated HDFS-14758: - Attachment: HDFS-14758.005.patch > Decrease lease hard limit > - > > Key: HDFS-14758 > URL: https://issues.apache.org/jira/browse/HDFS-14758 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Eric Payne >Assignee: hemanthboyina >Priority: Minor > Attachments: HDFS-14758.001.patch, HDFS-14758.002.patch, > HDFS-14758.003.patch, HDFS-14758.004.patch, HDFS-14758.005.patch > > > The hard limit is currently hard-coded to be 1 hour. This also determines the > NN automatic lease recovery interval. Something like 20 min will make more > sense. > After the 5 min soft limit, other clients can recover the lease. If no one > else takes the lease away, the original client still can renew the lease > within the hard limit. So even after a NN full GC of 8 minutes, leases can be > still valid. > However, there is one risk in reducing the hard limit. E.g. Reduced to 20 > min. If the NN crashes and the manual failover takes more than 20 minutes, > clients will abort. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-14758) Decrease lease hard limit
[ https://issues.apache.org/jira/browse/HDFS-14758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hemanthboyina updated HDFS-14758: - Attachment: HDFS-14758.004.patch > Decrease lease hard limit > - > > Key: HDFS-14758 > URL: https://issues.apache.org/jira/browse/HDFS-14758 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Eric Payne >Assignee: hemanthboyina >Priority: Minor > Attachments: HDFS-14758.001.patch, HDFS-14758.002.patch, > HDFS-14758.003.patch, HDFS-14758.004.patch > > > The hard limit is currently hard-coded to be 1 hour. This also determines the > NN automatic lease recovery interval. Something like 20 min will make more > sense. > After the 5 min soft limit, other clients can recover the lease. If no one > else takes the lease away, the original client still can renew the lease > within the hard limit. So even after a NN full GC of 8 minutes, leases can be > still valid. > However, there is one risk in reducing the hard limit. E.g. Reduced to 20 > min. If the NN crashes and the manual failover takes more than 20 minutes, > clients will abort. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-14758) Decrease lease hard limit
[ https://issues.apache.org/jira/browse/HDFS-14758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hemanthboyina updated HDFS-14758: - Attachment: HDFS-14758.003.patch > Decrease lease hard limit > - > > Key: HDFS-14758 > URL: https://issues.apache.org/jira/browse/HDFS-14758 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Eric Payne >Assignee: hemanthboyina >Priority: Minor > Attachments: HDFS-14758.001.patch, HDFS-14758.002.patch, > HDFS-14758.003.patch > > > The hard limit is currently hard-coded to be 1 hour. This also determines the > NN automatic lease recovery interval. Something like 20 min will make more > sense. > After the 5 min soft limit, other clients can recover the lease. If no one > else takes the lease away, the original client still can renew the lease > within the hard limit. So even after a NN full GC of 8 minutes, leases can be > still valid. > However, there is one risk in reducing the hard limit. E.g. Reduced to 20 > min. If the NN crashes and the manual failover takes more than 20 minutes, > clients will abort. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-14758) Decrease lease hard limit
[ https://issues.apache.org/jira/browse/HDFS-14758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hemanthboyina updated HDFS-14758: - Attachment: HDFS-14758.002.patch > Decrease lease hard limit > - > > Key: HDFS-14758 > URL: https://issues.apache.org/jira/browse/HDFS-14758 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Eric Payne >Assignee: hemanthboyina >Priority: Minor > Attachments: HDFS-14758.001.patch, HDFS-14758.002.patch > > > The hard limit is currently hard-coded to be 1 hour. This also determines the > NN automatic lease recovery interval. Something like 20 min will make more > sense. > After the 5 min soft limit, other clients can recover the lease. If no one > else takes the lease away, the original client still can renew the lease > within the hard limit. So even after a NN full GC of 8 minutes, leases can be > still valid. > However, there is one risk in reducing the hard limit. E.g. Reduced to 20 > min. If the NN crashes and the manual failover takes more than 20 minutes, > clients will abort. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-14758) Decrease lease hard limit
[ https://issues.apache.org/jira/browse/HDFS-14758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hemanthboyina updated HDFS-14758: - Attachment: HDFS-14758.001.patch Status: Patch Available (was: Open) > Decrease lease hard limit > - > > Key: HDFS-14758 > URL: https://issues.apache.org/jira/browse/HDFS-14758 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Eric Payne >Assignee: hemanthboyina >Priority: Minor > Attachments: HDFS-14758.001.patch > > > The hard limit is currently hard-coded to be 1 hour. This also determines the > NN automatic lease recovery interval. Something like 20 min will make more > sense. > After the 5 min soft limit, other clients can recover the lease. If no one > else takes the lease away, the original client still can renew the lease > within the hard limit. So even after a NN full GC of 8 minutes, leases can be > still valid. > However, there is one risk in reducing the hard limit. E.g. Reduced to 20 > min. If the NN crashes and the manual failover takes more than 20 minutes, > clients will abort. -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
