[jira] [Updated] (HDFS-12518) Re-encryption should handle task cancellation and progress better
[ https://issues.apache.org/jira/browse/HDFS-12518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen updated HDFS-12518: - Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.0.0 Status: Resolved (was: Patch Available) Committed to trunk and branch-3.0. Thanks for the reviews Wei-Chiu! > Re-encryption should handle task cancellation and progress better > - > > Key: HDFS-12518 > URL: https://issues.apache.org/jira/browse/HDFS-12518 > Project: Hadoop HDFS > Issue Type: Bug > Components: encryption >Affects Versions: 3.0.0-beta1 >Reporter: Xiao Chen >Assignee: Xiao Chen > Fix For: 3.0.0 > > Attachments: HDFS-12518.01.patch, HDFS-12518.02.patch, > HDFS-12518.03.patch > > > Re-encryption should handle task cancellation and progress tracking better in > general. > In a recent internal report, a canceled re-encryption could lead to the > progress of the zone being 'Processing' forever. Sending a new cancel command > would make it complete, but new re-encryptions for the same zone wouldn't > work because the canceled future is not removed. > This jira proposes to fix that, and enhance the currently handling so new > command would start from a clean state. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12518) Re-encryption should handle task cancellation and progress better
[ https://issues.apache.org/jira/browse/HDFS-12518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen updated HDFS-12518: - Attachment: HDFS-12518.03.patch patch 3 to fix checkstyle > Re-encryption should handle task cancellation and progress better > - > > Key: HDFS-12518 > URL: https://issues.apache.org/jira/browse/HDFS-12518 > Project: Hadoop HDFS > Issue Type: Bug > Components: encryption >Affects Versions: 3.0.0-beta1 >Reporter: Xiao Chen >Assignee: Xiao Chen > Attachments: HDFS-12518.01.patch, HDFS-12518.02.patch, > HDFS-12518.03.patch > > > Re-encryption should handle task cancellation and progress tracking better in > general. > In a recent internal report, a canceled re-encryption could lead to the > progress of the zone being 'Processing' forever. Sending a new cancel command > would make it complete, but new re-encryptions for the same zone wouldn't > work because the canceled future is not removed. > This jira proposes to fix that, and enhance the currently handling so new > command would start from a clean state. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12518) Re-encryption should handle task cancellation and progress better
[ https://issues.apache.org/jira/browse/HDFS-12518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen updated HDFS-12518: - Attachment: HDFS-12518.02.patch Thanks for the review [~jojochuang]! Patch 2 addressed first 2 comments. For synchronization, I'd prefer to keep things simple and easier to maintain. So kept the current (all {{submissions}} protected by {{synchronized}}) way. > Re-encryption should handle task cancellation and progress better > - > > Key: HDFS-12518 > URL: https://issues.apache.org/jira/browse/HDFS-12518 > Project: Hadoop HDFS > Issue Type: Bug > Components: encryption >Affects Versions: 3.0.0-beta1 >Reporter: Xiao Chen >Assignee: Xiao Chen > Attachments: HDFS-12518.01.patch, HDFS-12518.02.patch > > > Re-encryption should handle task cancellation and progress tracking better in > general. > In a recent internal report, a canceled re-encryption could lead to the > progress of the zone being 'Processing' forever. Sending a new cancel command > would make it complete, but new re-encryptions for the same zone wouldn't > work because the canceled future is not removed. > This jira proposes to fix that, and enhance the currently handling so new > command would start from a clean state. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12518) Re-encryption should handle task cancellation and progress better
[ https://issues.apache.org/jira/browse/HDFS-12518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen updated HDFS-12518: - Status: Patch Available (was: Open) > Re-encryption should handle task cancellation and progress better > - > > Key: HDFS-12518 > URL: https://issues.apache.org/jira/browse/HDFS-12518 > Project: Hadoop HDFS > Issue Type: Bug > Components: encryption >Affects Versions: 3.0.0-beta1 >Reporter: Xiao Chen >Assignee: Xiao Chen > Attachments: HDFS-12518.01.patch > > > Re-encryption should handle task cancellation and progress tracking better in > general. > In a recent internal report, a canceled re-encryption could lead to the > progress of the zone being 'Processing' forever. Sending a new cancel command > would make it complete, but new re-encryptions for the same zone wouldn't > work because the canceled future is not removed. > This jira proposes to fix that, and enhance the currently handling so new > command would start from a clean state. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12518) Re-encryption should handle task cancellation and progress better
[ https://issues.apache.org/jira/browse/HDFS-12518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen updated HDFS-12518: - Attachment: HDFS-12518.01.patch Patch 1: - Added the fix in {{ReencryptionUpdater}} to handle canceled future better - Modified the way {{submissions}} in {{ReencryptionHandler}} is managed: on -cancel and valid new -start, the zone submission tracker is reset. This means even if somehow an re-encryption is ended up in an erroneous state, it can still be reset from crypto CLI. - Fixed / improved a few related places: protect {{submissions}} with the handler's object lock, logging fix, missed assert. - Enhanced {{TestReencryption}} to ensure fault injector behave correctly when multi-threaded (had a 1% failure rate on dist-test > Re-encryption should handle task cancellation and progress better > - > > Key: HDFS-12518 > URL: https://issues.apache.org/jira/browse/HDFS-12518 > Project: Hadoop HDFS > Issue Type: Bug > Components: encryption >Affects Versions: 3.0.0-beta1 >Reporter: Xiao Chen >Assignee: Xiao Chen > Attachments: HDFS-12518.01.patch > > > Re-encryption should handle task cancellation and progress tracking better in > general. > In a recent internal report, a canceled re-encryption could lead to the > progress of the zone being 'Processing' forever. Sending a new cancel command > would make it complete, but new re-encryptions for the same zone wouldn't > work because the canceled future is not removed. > This jira proposes to fix that, and enhance the currently handling so new > command would start from a clean state. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org