[jira] [Commented] (HDFS-12383) Re-encryption updater should handle canceled tasks better
[ https://issues.apache.org/jira/browse/HDFS-12383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16450970#comment-16450970 ] Hudson commented on HDFS-12383: --- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #14057 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/14057/]) HDFS-12383. Re-encryption updater should handle canceled tasks better. (xyao: rev 8b2235b367298e494e58e43dc9fc7cdee1ae79ae) * (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/ReencryptionUpdater.java * (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestReencryption.java > Re-encryption updater should handle canceled tasks better > - > > Key: HDFS-12383 > URL: https://issues.apache.org/jira/browse/HDFS-12383 > Project: Hadoop HDFS > Issue Type: Bug > Components: encryption >Affects Versions: 3.0.0-beta1 >Reporter: Xiao Chen >Assignee: Xiao Chen >Priority: Major > Fix For: 3.0.0-beta1 > > Attachments: HDFS-12383.01.patch, HDFS-12383.02.patch > > > Seen an instance where the re-encryption updater exited due to an exception, > and later tasks no longer executes. Logs below: > {noformat} > 2017-08-31 09:54:08,104 INFO > org.apache.hadoop.hdfs.server.namenode.EncryptionZoneManager: Zone > /tmp/encryption-zone-3(16819) is submitted for re-encryption. > 2017-08-31 09:54:08,104 INFO > org.apache.hadoop.hdfs.server.namenode.ReencryptionHandler: Executing > re-encrypt commands on zone 16819. Current zones:[zone:16787 state:Completed > lastProcessed:null filesReencrypted:1 fileReencryptionFailures:0][zone:16813 > state:Completed lastProcessed:null filesReencrypted:1 > fileReencryptionFailures:0][zone:16819 state:Submitted lastProcessed:null > filesReencrypted:0 fileReencryptionFailures:0] > 2017-08-31 09:54:08,105 INFO > org.apache.hadoop.hdfs.protocol.ReencryptionStatus: Zone 16819 starts > re-encryption processing > 2017-08-31 09:54:08,105 INFO > org.apache.hadoop.hdfs.server.namenode.ReencryptionHandler: Re-encrypting > zone /tmp/encryption-zone-3(id=16819) > 2017-08-31 09:54:08,105 INFO > org.apache.hadoop.hdfs.server.namenode.ReencryptionHandler: Submitted batch > (start:/tmp/encryption-zone-3/data1, size:1) of zone 16819 to re-encrypt. > 2017-08-31 09:54:08,105 INFO > org.apache.hadoop.hdfs.server.namenode.ReencryptionHandler: Submission > completed of zone 16819 for re-encryption. > 2017-08-31 09:54:08,105 INFO > org.apache.hadoop.hdfs.server.namenode.ReencryptionHandler: Processing > batched re-encryption for zone 16819, batch size 1, > start:/tmp/encryption-zone-3/data1 > 2017-08-31 09:54:08,979 INFO BlockStateChange: BLOCK* BlockManager: ask > 172.26.1.71:20002 to delete [blk_1073742291_1467] > 2017-08-31 09:54:18,295 INFO > org.apache.hadoop.hdfs.server.namenode.ReencryptionUpdater: Cancelling 1 > re-encryption tasks > 2017-08-31 09:54:18,295 INFO > org.apache.hadoop.hdfs.server.namenode.EncryptionZoneManager: Cancelled zone > /tmp/encryption-zone-3(16819) for re-encryption. > 2017-08-31 09:54:18,295 INFO > org.apache.hadoop.hdfs.protocol.ReencryptionStatus: Zone 16819 completed > re-encryption. > 2017-08-31 09:54:18,296 INFO > org.apache.hadoop.hdfs.server.namenode.ReencryptionHandler: Completed > re-encrypting one batch of 1 edeks from KMS, time consumed: 10.19 s, start: > /tmp/encryption-zone-3/data1. > 2017-08-31 09:54:18,296 ERROR > org.apache.hadoop.hdfs.server.namenode.ReencryptionUpdater: Re-encryption > updater thread exiting. > java.util.concurrent.CancellationException > at java.util.concurrent.FutureTask.report(FutureTask.java:121) > at java.util.concurrent.FutureTask.get(FutureTask.java:192) > at > org.apache.hadoop.hdfs.server.namenode.ReencryptionUpdater.takeAndProcessTasks(ReencryptionUpdater.java:404) > at > org.apache.hadoop.hdfs.server.namenode.ReencryptionUpdater.run(ReencryptionUpdater.java:250) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:745) > {noformat} > Updater should be fixed to handle canceled tasks better. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12383) Re-encryption updater should handle canceled tasks better
[ https://issues.apache.org/jira/browse/HDFS-12383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16157685#comment-16157685 ] Hudson commented on HDFS-12383: --- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #12811 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/12811/]) HDFS-12383. Re-encryption updater should handle canceled tasks better. (xiao: rev 633c1ea4554cae6cd684f60848497817d4ed3d82) * (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestReencryption.java * (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/ReencryptionUpdater.java > Re-encryption updater should handle canceled tasks better > - > > Key: HDFS-12383 > URL: https://issues.apache.org/jira/browse/HDFS-12383 > Project: Hadoop HDFS > Issue Type: Bug > Components: encryption >Affects Versions: 3.0.0-beta1 >Reporter: Xiao Chen >Assignee: Xiao Chen > Fix For: 3.0.0-beta1 > > Attachments: HDFS-12383.01.patch, HDFS-12383.02.patch > > > Seen an instance where the re-encryption updater exited due to an exception, > and later tasks no longer executes. Logs below: > {noformat} > 2017-08-31 09:54:08,104 INFO > org.apache.hadoop.hdfs.server.namenode.EncryptionZoneManager: Zone > /tmp/encryption-zone-3(16819) is submitted for re-encryption. > 2017-08-31 09:54:08,104 INFO > org.apache.hadoop.hdfs.server.namenode.ReencryptionHandler: Executing > re-encrypt commands on zone 16819. Current zones:[zone:16787 state:Completed > lastProcessed:null filesReencrypted:1 fileReencryptionFailures:0][zone:16813 > state:Completed lastProcessed:null filesReencrypted:1 > fileReencryptionFailures:0][zone:16819 state:Submitted lastProcessed:null > filesReencrypted:0 fileReencryptionFailures:0] > 2017-08-31 09:54:08,105 INFO > org.apache.hadoop.hdfs.protocol.ReencryptionStatus: Zone 16819 starts > re-encryption processing > 2017-08-31 09:54:08,105 INFO > org.apache.hadoop.hdfs.server.namenode.ReencryptionHandler: Re-encrypting > zone /tmp/encryption-zone-3(id=16819) > 2017-08-31 09:54:08,105 INFO > org.apache.hadoop.hdfs.server.namenode.ReencryptionHandler: Submitted batch > (start:/tmp/encryption-zone-3/data1, size:1) of zone 16819 to re-encrypt. > 2017-08-31 09:54:08,105 INFO > org.apache.hadoop.hdfs.server.namenode.ReencryptionHandler: Submission > completed of zone 16819 for re-encryption. > 2017-08-31 09:54:08,105 INFO > org.apache.hadoop.hdfs.server.namenode.ReencryptionHandler: Processing > batched re-encryption for zone 16819, batch size 1, > start:/tmp/encryption-zone-3/data1 > 2017-08-31 09:54:08,979 INFO BlockStateChange: BLOCK* BlockManager: ask > 172.26.1.71:20002 to delete [blk_1073742291_1467] > 2017-08-31 09:54:18,295 INFO > org.apache.hadoop.hdfs.server.namenode.ReencryptionUpdater: Cancelling 1 > re-encryption tasks > 2017-08-31 09:54:18,295 INFO > org.apache.hadoop.hdfs.server.namenode.EncryptionZoneManager: Cancelled zone > /tmp/encryption-zone-3(16819) for re-encryption. > 2017-08-31 09:54:18,295 INFO > org.apache.hadoop.hdfs.protocol.ReencryptionStatus: Zone 16819 completed > re-encryption. > 2017-08-31 09:54:18,296 INFO > org.apache.hadoop.hdfs.server.namenode.ReencryptionHandler: Completed > re-encrypting one batch of 1 edeks from KMS, time consumed: 10.19 s, start: > /tmp/encryption-zone-3/data1. > 2017-08-31 09:54:18,296 ERROR > org.apache.hadoop.hdfs.server.namenode.ReencryptionUpdater: Re-encryption > updater thread exiting. > java.util.concurrent.CancellationException > at java.util.concurrent.FutureTask.report(FutureTask.java:121) > at java.util.concurrent.FutureTask.get(FutureTask.java:192) > at > org.apache.hadoop.hdfs.server.namenode.ReencryptionUpdater.takeAndProcessTasks(ReencryptionUpdater.java:404) > at > org.apache.hadoop.hdfs.server.namenode.ReencryptionUpdater.run(ReencryptionUpdater.java:250) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:745) > {noformat} > Updater should be fixed to handle canceled tasks better. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12383) Re-encryption updater should handle canceled tasks better
[ https://issues.apache.org/jira/browse/HDFS-12383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16154328#comment-16154328 ] Andrew Wang commented on HDFS-12383: Cherry-picked this back to branch-3.0 for beta1 as well, thanks folks. > Re-encryption updater should handle canceled tasks better > - > > Key: HDFS-12383 > URL: https://issues.apache.org/jira/browse/HDFS-12383 > Project: Hadoop HDFS > Issue Type: Bug > Components: encryption >Affects Versions: 3.0.0-beta1 >Reporter: Xiao Chen >Assignee: Xiao Chen > Fix For: 3.0.0-beta1 > > Attachments: HDFS-12383.01.patch, HDFS-12383.02.patch > > > Seen an instance where the re-encryption updater exited due to an exception, > and later tasks no longer executes. Logs below: > {noformat} > 2017-08-31 09:54:08,104 INFO > org.apache.hadoop.hdfs.server.namenode.EncryptionZoneManager: Zone > /tmp/encryption-zone-3(16819) is submitted for re-encryption. > 2017-08-31 09:54:08,104 INFO > org.apache.hadoop.hdfs.server.namenode.ReencryptionHandler: Executing > re-encrypt commands on zone 16819. Current zones:[zone:16787 state:Completed > lastProcessed:null filesReencrypted:1 fileReencryptionFailures:0][zone:16813 > state:Completed lastProcessed:null filesReencrypted:1 > fileReencryptionFailures:0][zone:16819 state:Submitted lastProcessed:null > filesReencrypted:0 fileReencryptionFailures:0] > 2017-08-31 09:54:08,105 INFO > org.apache.hadoop.hdfs.protocol.ReencryptionStatus: Zone 16819 starts > re-encryption processing > 2017-08-31 09:54:08,105 INFO > org.apache.hadoop.hdfs.server.namenode.ReencryptionHandler: Re-encrypting > zone /tmp/encryption-zone-3(id=16819) > 2017-08-31 09:54:08,105 INFO > org.apache.hadoop.hdfs.server.namenode.ReencryptionHandler: Submitted batch > (start:/tmp/encryption-zone-3/data1, size:1) of zone 16819 to re-encrypt. > 2017-08-31 09:54:08,105 INFO > org.apache.hadoop.hdfs.server.namenode.ReencryptionHandler: Submission > completed of zone 16819 for re-encryption. > 2017-08-31 09:54:08,105 INFO > org.apache.hadoop.hdfs.server.namenode.ReencryptionHandler: Processing > batched re-encryption for zone 16819, batch size 1, > start:/tmp/encryption-zone-3/data1 > 2017-08-31 09:54:08,979 INFO BlockStateChange: BLOCK* BlockManager: ask > 172.26.1.71:20002 to delete [blk_1073742291_1467] > 2017-08-31 09:54:18,295 INFO > org.apache.hadoop.hdfs.server.namenode.ReencryptionUpdater: Cancelling 1 > re-encryption tasks > 2017-08-31 09:54:18,295 INFO > org.apache.hadoop.hdfs.server.namenode.EncryptionZoneManager: Cancelled zone > /tmp/encryption-zone-3(16819) for re-encryption. > 2017-08-31 09:54:18,295 INFO > org.apache.hadoop.hdfs.protocol.ReencryptionStatus: Zone 16819 completed > re-encryption. > 2017-08-31 09:54:18,296 INFO > org.apache.hadoop.hdfs.server.namenode.ReencryptionHandler: Completed > re-encrypting one batch of 1 edeks from KMS, time consumed: 10.19 s, start: > /tmp/encryption-zone-3/data1. > 2017-08-31 09:54:18,296 ERROR > org.apache.hadoop.hdfs.server.namenode.ReencryptionUpdater: Re-encryption > updater thread exiting. > java.util.concurrent.CancellationException > at java.util.concurrent.FutureTask.report(FutureTask.java:121) > at java.util.concurrent.FutureTask.get(FutureTask.java:192) > at > org.apache.hadoop.hdfs.server.namenode.ReencryptionUpdater.takeAndProcessTasks(ReencryptionUpdater.java:404) > at > org.apache.hadoop.hdfs.server.namenode.ReencryptionUpdater.run(ReencryptionUpdater.java:250) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:745) > {noformat} > Updater should be fixed to handle canceled tasks better. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12383) Re-encryption updater should handle canceled tasks better
[ https://issues.apache.org/jira/browse/HDFS-12383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16152059#comment-16152059 ] Xiao Chen commented on HDFS-12383: -- Thanks Rushabh for the comment. Updater thread should only be terminated from handler's {{stopThreads}} by interruption. Only added the {{isRunning}} param for a way to unit test. Committing this to trunk based on Wei-Chiu's +1, feel free to comment / raise improvement jiras. Thanks! > Re-encryption updater should handle canceled tasks better > - > > Key: HDFS-12383 > URL: https://issues.apache.org/jira/browse/HDFS-12383 > Project: Hadoop HDFS > Issue Type: Bug > Components: encryption >Affects Versions: 3.0.0-beta1 >Reporter: Xiao Chen >Assignee: Xiao Chen > Attachments: HDFS-12383.01.patch, HDFS-12383.02.patch > > > Seen an instance where the re-encryption updater exited due to an exception, > and later tasks no longer executes. Logs below: > {noformat} > 2017-08-31 09:54:08,104 INFO > org.apache.hadoop.hdfs.server.namenode.EncryptionZoneManager: Zone > /tmp/encryption-zone-3(16819) is submitted for re-encryption. > 2017-08-31 09:54:08,104 INFO > org.apache.hadoop.hdfs.server.namenode.ReencryptionHandler: Executing > re-encrypt commands on zone 16819. Current zones:[zone:16787 state:Completed > lastProcessed:null filesReencrypted:1 fileReencryptionFailures:0][zone:16813 > state:Completed lastProcessed:null filesReencrypted:1 > fileReencryptionFailures:0][zone:16819 state:Submitted lastProcessed:null > filesReencrypted:0 fileReencryptionFailures:0] > 2017-08-31 09:54:08,105 INFO > org.apache.hadoop.hdfs.protocol.ReencryptionStatus: Zone 16819 starts > re-encryption processing > 2017-08-31 09:54:08,105 INFO > org.apache.hadoop.hdfs.server.namenode.ReencryptionHandler: Re-encrypting > zone /tmp/encryption-zone-3(id=16819) > 2017-08-31 09:54:08,105 INFO > org.apache.hadoop.hdfs.server.namenode.ReencryptionHandler: Submitted batch > (start:/tmp/encryption-zone-3/data1, size:1) of zone 16819 to re-encrypt. > 2017-08-31 09:54:08,105 INFO > org.apache.hadoop.hdfs.server.namenode.ReencryptionHandler: Submission > completed of zone 16819 for re-encryption. > 2017-08-31 09:54:08,105 INFO > org.apache.hadoop.hdfs.server.namenode.ReencryptionHandler: Processing > batched re-encryption for zone 16819, batch size 1, > start:/tmp/encryption-zone-3/data1 > 2017-08-31 09:54:08,979 INFO BlockStateChange: BLOCK* BlockManager: ask > 172.26.1.71:20002 to delete [blk_1073742291_1467] > 2017-08-31 09:54:18,295 INFO > org.apache.hadoop.hdfs.server.namenode.ReencryptionUpdater: Cancelling 1 > re-encryption tasks > 2017-08-31 09:54:18,295 INFO > org.apache.hadoop.hdfs.server.namenode.EncryptionZoneManager: Cancelled zone > /tmp/encryption-zone-3(16819) for re-encryption. > 2017-08-31 09:54:18,295 INFO > org.apache.hadoop.hdfs.protocol.ReencryptionStatus: Zone 16819 completed > re-encryption. > 2017-08-31 09:54:18,296 INFO > org.apache.hadoop.hdfs.server.namenode.ReencryptionHandler: Completed > re-encrypting one batch of 1 edeks from KMS, time consumed: 10.19 s, start: > /tmp/encryption-zone-3/data1. > 2017-08-31 09:54:18,296 ERROR > org.apache.hadoop.hdfs.server.namenode.ReencryptionUpdater: Re-encryption > updater thread exiting. > java.util.concurrent.CancellationException > at java.util.concurrent.FutureTask.report(FutureTask.java:121) > at java.util.concurrent.FutureTask.get(FutureTask.java:192) > at > org.apache.hadoop.hdfs.server.namenode.ReencryptionUpdater.takeAndProcessTasks(ReencryptionUpdater.java:404) > at > org.apache.hadoop.hdfs.server.namenode.ReencryptionUpdater.run(ReencryptionUpdater.java:250) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:745) > {noformat} > Updater should be fixed to handle canceled tasks better. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12383) Re-encryption updater should handle canceled tasks better
[ https://issues.apache.org/jira/browse/HDFS-12383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16150572#comment-16150572 ] Rushabh S Shah commented on HDFS-12383: --- We can change the while loop exit condition to use {{isRunning}} flag instead of {{while(true)}} So when we want to exit, we can just set the {{isRunning}} to false. > Re-encryption updater should handle canceled tasks better > - > > Key: HDFS-12383 > URL: https://issues.apache.org/jira/browse/HDFS-12383 > Project: Hadoop HDFS > Issue Type: Bug > Components: encryption >Affects Versions: 3.0.0-beta1 >Reporter: Xiao Chen >Assignee: Xiao Chen > Attachments: HDFS-12383.01.patch, HDFS-12383.02.patch > > > Seen an instance where the re-encryption updater exited due to an exception, > and later tasks no longer executes. Logs below: > {noformat} > 2017-08-31 09:54:08,104 INFO > org.apache.hadoop.hdfs.server.namenode.EncryptionZoneManager: Zone > /tmp/encryption-zone-3(16819) is submitted for re-encryption. > 2017-08-31 09:54:08,104 INFO > org.apache.hadoop.hdfs.server.namenode.ReencryptionHandler: Executing > re-encrypt commands on zone 16819. Current zones:[zone:16787 state:Completed > lastProcessed:null filesReencrypted:1 fileReencryptionFailures:0][zone:16813 > state:Completed lastProcessed:null filesReencrypted:1 > fileReencryptionFailures:0][zone:16819 state:Submitted lastProcessed:null > filesReencrypted:0 fileReencryptionFailures:0] > 2017-08-31 09:54:08,105 INFO > org.apache.hadoop.hdfs.protocol.ReencryptionStatus: Zone 16819 starts > re-encryption processing > 2017-08-31 09:54:08,105 INFO > org.apache.hadoop.hdfs.server.namenode.ReencryptionHandler: Re-encrypting > zone /tmp/encryption-zone-3(id=16819) > 2017-08-31 09:54:08,105 INFO > org.apache.hadoop.hdfs.server.namenode.ReencryptionHandler: Submitted batch > (start:/tmp/encryption-zone-3/data1, size:1) of zone 16819 to re-encrypt. > 2017-08-31 09:54:08,105 INFO > org.apache.hadoop.hdfs.server.namenode.ReencryptionHandler: Submission > completed of zone 16819 for re-encryption. > 2017-08-31 09:54:08,105 INFO > org.apache.hadoop.hdfs.server.namenode.ReencryptionHandler: Processing > batched re-encryption for zone 16819, batch size 1, > start:/tmp/encryption-zone-3/data1 > 2017-08-31 09:54:08,979 INFO BlockStateChange: BLOCK* BlockManager: ask > 172.26.1.71:20002 to delete [blk_1073742291_1467] > 2017-08-31 09:54:18,295 INFO > org.apache.hadoop.hdfs.server.namenode.ReencryptionUpdater: Cancelling 1 > re-encryption tasks > 2017-08-31 09:54:18,295 INFO > org.apache.hadoop.hdfs.server.namenode.EncryptionZoneManager: Cancelled zone > /tmp/encryption-zone-3(16819) for re-encryption. > 2017-08-31 09:54:18,295 INFO > org.apache.hadoop.hdfs.protocol.ReencryptionStatus: Zone 16819 completed > re-encryption. > 2017-08-31 09:54:18,296 INFO > org.apache.hadoop.hdfs.server.namenode.ReencryptionHandler: Completed > re-encrypting one batch of 1 edeks from KMS, time consumed: 10.19 s, start: > /tmp/encryption-zone-3/data1. > 2017-08-31 09:54:18,296 ERROR > org.apache.hadoop.hdfs.server.namenode.ReencryptionUpdater: Re-encryption > updater thread exiting. > java.util.concurrent.CancellationException > at java.util.concurrent.FutureTask.report(FutureTask.java:121) > at java.util.concurrent.FutureTask.get(FutureTask.java:192) > at > org.apache.hadoop.hdfs.server.namenode.ReencryptionUpdater.takeAndProcessTasks(ReencryptionUpdater.java:404) > at > org.apache.hadoop.hdfs.server.namenode.ReencryptionUpdater.run(ReencryptionUpdater.java:250) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:745) > {noformat} > Updater should be fixed to handle canceled tasks better. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12383) Re-encryption updater should handle canceled tasks better
[ https://issues.apache.org/jira/browse/HDFS-12383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16150108#comment-16150108 ] Xiao Chen commented on HDFS-12383: -- Test failures not related to this patch. Not sure what's going on with pre-commit these days, we seem to get a whole lot of failures. Will commit this in 24 hours. Thanks [~jojochuang] for reviewing! > Re-encryption updater should handle canceled tasks better > - > > Key: HDFS-12383 > URL: https://issues.apache.org/jira/browse/HDFS-12383 > Project: Hadoop HDFS > Issue Type: Bug > Components: encryption >Affects Versions: 3.0.0-beta1 >Reporter: Xiao Chen >Assignee: Xiao Chen > Attachments: HDFS-12383.01.patch, HDFS-12383.02.patch > > > Seen an instance where the re-encryption updater exited due to an exception, > and later tasks no longer executes. Logs below: > {noformat} > 2017-08-31 09:54:08,104 INFO > org.apache.hadoop.hdfs.server.namenode.EncryptionZoneManager: Zone > /tmp/encryption-zone-3(16819) is submitted for re-encryption. > 2017-08-31 09:54:08,104 INFO > org.apache.hadoop.hdfs.server.namenode.ReencryptionHandler: Executing > re-encrypt commands on zone 16819. Current zones:[zone:16787 state:Completed > lastProcessed:null filesReencrypted:1 fileReencryptionFailures:0][zone:16813 > state:Completed lastProcessed:null filesReencrypted:1 > fileReencryptionFailures:0][zone:16819 state:Submitted lastProcessed:null > filesReencrypted:0 fileReencryptionFailures:0] > 2017-08-31 09:54:08,105 INFO > org.apache.hadoop.hdfs.protocol.ReencryptionStatus: Zone 16819 starts > re-encryption processing > 2017-08-31 09:54:08,105 INFO > org.apache.hadoop.hdfs.server.namenode.ReencryptionHandler: Re-encrypting > zone /tmp/encryption-zone-3(id=16819) > 2017-08-31 09:54:08,105 INFO > org.apache.hadoop.hdfs.server.namenode.ReencryptionHandler: Submitted batch > (start:/tmp/encryption-zone-3/data1, size:1) of zone 16819 to re-encrypt. > 2017-08-31 09:54:08,105 INFO > org.apache.hadoop.hdfs.server.namenode.ReencryptionHandler: Submission > completed of zone 16819 for re-encryption. > 2017-08-31 09:54:08,105 INFO > org.apache.hadoop.hdfs.server.namenode.ReencryptionHandler: Processing > batched re-encryption for zone 16819, batch size 1, > start:/tmp/encryption-zone-3/data1 > 2017-08-31 09:54:08,979 INFO BlockStateChange: BLOCK* BlockManager: ask > 172.26.1.71:20002 to delete [blk_1073742291_1467] > 2017-08-31 09:54:18,295 INFO > org.apache.hadoop.hdfs.server.namenode.ReencryptionUpdater: Cancelling 1 > re-encryption tasks > 2017-08-31 09:54:18,295 INFO > org.apache.hadoop.hdfs.server.namenode.EncryptionZoneManager: Cancelled zone > /tmp/encryption-zone-3(16819) for re-encryption. > 2017-08-31 09:54:18,295 INFO > org.apache.hadoop.hdfs.protocol.ReencryptionStatus: Zone 16819 completed > re-encryption. > 2017-08-31 09:54:18,296 INFO > org.apache.hadoop.hdfs.server.namenode.ReencryptionHandler: Completed > re-encrypting one batch of 1 edeks from KMS, time consumed: 10.19 s, start: > /tmp/encryption-zone-3/data1. > 2017-08-31 09:54:18,296 ERROR > org.apache.hadoop.hdfs.server.namenode.ReencryptionUpdater: Re-encryption > updater thread exiting. > java.util.concurrent.CancellationException > at java.util.concurrent.FutureTask.report(FutureTask.java:121) > at java.util.concurrent.FutureTask.get(FutureTask.java:192) > at > org.apache.hadoop.hdfs.server.namenode.ReencryptionUpdater.takeAndProcessTasks(ReencryptionUpdater.java:404) > at > org.apache.hadoop.hdfs.server.namenode.ReencryptionUpdater.run(ReencryptionUpdater.java:250) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:745) > {noformat} > Updater should be fixed to handle canceled tasks better. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12383) Re-encryption updater should handle canceled tasks better
[ https://issues.apache.org/jira/browse/HDFS-12383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16150098#comment-16150098 ] Hadoop QA commented on HDFS-12383: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 14s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 1 new or modified test files. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 15m 7s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 58s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 41s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 3s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 49s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 44s{color} | {color:green} trunk passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 0s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 57s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 57s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 34s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 2s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 57s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 40s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:red}-1{color} | {color:red} unit {color} | {color:red}125m 1s{color} | {color:red} hadoop-hdfs in the patch failed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 21s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black}153m 35s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | Failed junit tests | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure060 | | | hadoop.hdfs.TestDFSStripedOutputStreamWithFailureWithRandomECPolicy | | | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure100 | | | hadoop.hdfs.TestReadStripedFileWithMissingBlocks | | | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure160 | | | hadoop.hdfs.TestClientProtocolForPipelineRecovery | | | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure130 | | | hadoop.hdfs.TestEncryptedTransfer | | | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure000 | | | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure180 | | | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure050 | | | hadoop.hdfs.server.blockmanagement.TestReconstructStripedBlocksWithRackAwareness | | | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure110 | | | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure140 | | | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure190 | | | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure070 | | | hadoop.hdfs.TestDFSInputStream | | | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure170 | | | hadoop.hdfs.TestQuota | | | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure | | | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure020 | | | hadoop.hdfs.TestLeaseRecoveryStriped | | | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure080 | | | hadoop.hdfs.server.datanode.TestDirectoryScanner | | | hadoop.hdfs.server.datanode.TestNNHandlesBlockReportPerStorage | | | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure200 | | | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure040 | | | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure120 | | |
[jira] [Commented] (HDFS-12383) Re-encryption updater should handle canceled tasks better
[ https://issues.apache.org/jira/browse/HDFS-12383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16150082#comment-16150082 ] Wei-Chiu Chuang commented on HDFS-12383: +1 pending Jenkins. thanks! > Re-encryption updater should handle canceled tasks better > - > > Key: HDFS-12383 > URL: https://issues.apache.org/jira/browse/HDFS-12383 > Project: Hadoop HDFS > Issue Type: Bug > Components: encryption >Affects Versions: 3.0.0-beta1 >Reporter: Xiao Chen >Assignee: Xiao Chen > Attachments: HDFS-12383.01.patch, HDFS-12383.02.patch > > > Seen an instance where the re-encryption updater exited due to an exception, > and later tasks no longer executes. Logs below: > {noformat} > 2017-08-31 09:54:08,104 INFO > org.apache.hadoop.hdfs.server.namenode.EncryptionZoneManager: Zone > /tmp/encryption-zone-3(16819) is submitted for re-encryption. > 2017-08-31 09:54:08,104 INFO > org.apache.hadoop.hdfs.server.namenode.ReencryptionHandler: Executing > re-encrypt commands on zone 16819. Current zones:[zone:16787 state:Completed > lastProcessed:null filesReencrypted:1 fileReencryptionFailures:0][zone:16813 > state:Completed lastProcessed:null filesReencrypted:1 > fileReencryptionFailures:0][zone:16819 state:Submitted lastProcessed:null > filesReencrypted:0 fileReencryptionFailures:0] > 2017-08-31 09:54:08,105 INFO > org.apache.hadoop.hdfs.protocol.ReencryptionStatus: Zone 16819 starts > re-encryption processing > 2017-08-31 09:54:08,105 INFO > org.apache.hadoop.hdfs.server.namenode.ReencryptionHandler: Re-encrypting > zone /tmp/encryption-zone-3(id=16819) > 2017-08-31 09:54:08,105 INFO > org.apache.hadoop.hdfs.server.namenode.ReencryptionHandler: Submitted batch > (start:/tmp/encryption-zone-3/data1, size:1) of zone 16819 to re-encrypt. > 2017-08-31 09:54:08,105 INFO > org.apache.hadoop.hdfs.server.namenode.ReencryptionHandler: Submission > completed of zone 16819 for re-encryption. > 2017-08-31 09:54:08,105 INFO > org.apache.hadoop.hdfs.server.namenode.ReencryptionHandler: Processing > batched re-encryption for zone 16819, batch size 1, > start:/tmp/encryption-zone-3/data1 > 2017-08-31 09:54:08,979 INFO BlockStateChange: BLOCK* BlockManager: ask > 172.26.1.71:20002 to delete [blk_1073742291_1467] > 2017-08-31 09:54:18,295 INFO > org.apache.hadoop.hdfs.server.namenode.ReencryptionUpdater: Cancelling 1 > re-encryption tasks > 2017-08-31 09:54:18,295 INFO > org.apache.hadoop.hdfs.server.namenode.EncryptionZoneManager: Cancelled zone > /tmp/encryption-zone-3(16819) for re-encryption. > 2017-08-31 09:54:18,295 INFO > org.apache.hadoop.hdfs.protocol.ReencryptionStatus: Zone 16819 completed > re-encryption. > 2017-08-31 09:54:18,296 INFO > org.apache.hadoop.hdfs.server.namenode.ReencryptionHandler: Completed > re-encrypting one batch of 1 edeks from KMS, time consumed: 10.19 s, start: > /tmp/encryption-zone-3/data1. > 2017-08-31 09:54:18,296 ERROR > org.apache.hadoop.hdfs.server.namenode.ReencryptionUpdater: Re-encryption > updater thread exiting. > java.util.concurrent.CancellationException > at java.util.concurrent.FutureTask.report(FutureTask.java:121) > at java.util.concurrent.FutureTask.get(FutureTask.java:192) > at > org.apache.hadoop.hdfs.server.namenode.ReencryptionUpdater.takeAndProcessTasks(ReencryptionUpdater.java:404) > at > org.apache.hadoop.hdfs.server.namenode.ReencryptionUpdater.run(ReencryptionUpdater.java:250) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:745) > {noformat} > Updater should be fixed to handle canceled tasks better. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12383) Re-encryption updater should handle canceled tasks better
[ https://issues.apache.org/jira/browse/HDFS-12383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16149993#comment-16149993 ] Xiao Chen commented on HDFS-12383: -- Test failure unrelated, just triggered another run in case. > Re-encryption updater should handle canceled tasks better > - > > Key: HDFS-12383 > URL: https://issues.apache.org/jira/browse/HDFS-12383 > Project: Hadoop HDFS > Issue Type: Bug > Components: encryption >Affects Versions: 3.0.0-beta1 >Reporter: Xiao Chen >Assignee: Xiao Chen > Attachments: HDFS-12383.01.patch, HDFS-12383.02.patch > > > Seen an instance where the re-encryption updater exited due to an exception, > and later tasks no longer executes. Logs below: > {noformat} > 2017-08-31 09:54:08,104 INFO > org.apache.hadoop.hdfs.server.namenode.EncryptionZoneManager: Zone > /tmp/encryption-zone-3(16819) is submitted for re-encryption. > 2017-08-31 09:54:08,104 INFO > org.apache.hadoop.hdfs.server.namenode.ReencryptionHandler: Executing > re-encrypt commands on zone 16819. Current zones:[zone:16787 state:Completed > lastProcessed:null filesReencrypted:1 fileReencryptionFailures:0][zone:16813 > state:Completed lastProcessed:null filesReencrypted:1 > fileReencryptionFailures:0][zone:16819 state:Submitted lastProcessed:null > filesReencrypted:0 fileReencryptionFailures:0] > 2017-08-31 09:54:08,105 INFO > org.apache.hadoop.hdfs.protocol.ReencryptionStatus: Zone 16819 starts > re-encryption processing > 2017-08-31 09:54:08,105 INFO > org.apache.hadoop.hdfs.server.namenode.ReencryptionHandler: Re-encrypting > zone /tmp/encryption-zone-3(id=16819) > 2017-08-31 09:54:08,105 INFO > org.apache.hadoop.hdfs.server.namenode.ReencryptionHandler: Submitted batch > (start:/tmp/encryption-zone-3/data1, size:1) of zone 16819 to re-encrypt. > 2017-08-31 09:54:08,105 INFO > org.apache.hadoop.hdfs.server.namenode.ReencryptionHandler: Submission > completed of zone 16819 for re-encryption. > 2017-08-31 09:54:08,105 INFO > org.apache.hadoop.hdfs.server.namenode.ReencryptionHandler: Processing > batched re-encryption for zone 16819, batch size 1, > start:/tmp/encryption-zone-3/data1 > 2017-08-31 09:54:08,979 INFO BlockStateChange: BLOCK* BlockManager: ask > 172.26.1.71:20002 to delete [blk_1073742291_1467] > 2017-08-31 09:54:18,295 INFO > org.apache.hadoop.hdfs.server.namenode.ReencryptionUpdater: Cancelling 1 > re-encryption tasks > 2017-08-31 09:54:18,295 INFO > org.apache.hadoop.hdfs.server.namenode.EncryptionZoneManager: Cancelled zone > /tmp/encryption-zone-3(16819) for re-encryption. > 2017-08-31 09:54:18,295 INFO > org.apache.hadoop.hdfs.protocol.ReencryptionStatus: Zone 16819 completed > re-encryption. > 2017-08-31 09:54:18,296 INFO > org.apache.hadoop.hdfs.server.namenode.ReencryptionHandler: Completed > re-encrypting one batch of 1 edeks from KMS, time consumed: 10.19 s, start: > /tmp/encryption-zone-3/data1. > 2017-08-31 09:54:18,296 ERROR > org.apache.hadoop.hdfs.server.namenode.ReencryptionUpdater: Re-encryption > updater thread exiting. > java.util.concurrent.CancellationException > at java.util.concurrent.FutureTask.report(FutureTask.java:121) > at java.util.concurrent.FutureTask.get(FutureTask.java:192) > at > org.apache.hadoop.hdfs.server.namenode.ReencryptionUpdater.takeAndProcessTasks(ReencryptionUpdater.java:404) > at > org.apache.hadoop.hdfs.server.namenode.ReencryptionUpdater.run(ReencryptionUpdater.java:250) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:745) > {noformat} > Updater should be fixed to handle canceled tasks better. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12383) Re-encryption updater should handle canceled tasks better
[ https://issues.apache.org/jira/browse/HDFS-12383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16149936#comment-16149936 ] Hadoop QA commented on HDFS-12383: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 21s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 1 new or modified test files. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 17m 12s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 1s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 41s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 8s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 1s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 47s{color} | {color:green} trunk passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 1s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 58s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 58s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 38s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 5s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 7s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 43s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:red}-1{color} | {color:red} unit {color} | {color:red}120m 11s{color} | {color:red} hadoop-hdfs in the patch failed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 19s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black}152m 6s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | Failed junit tests | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure090 | | | hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting | | | hadoop.hdfs.TestReadStripedFileWithMissingBlocks | | | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure020 | | | hadoop.hdfs.server.namenode.ha.TestPipelinesFailover | | | hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureToleration | | | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure170 | | | hadoop.hdfs.TestClientProtocolForPipelineRecovery | | | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure120 | | | hadoop.hdfs.TestLeaseRecoveryStriped | | | hadoop.hdfs.TestReconstructStripedFile | | | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure150 | | | hadoop.cli.TestCryptoAdminCLI | | | hadoop.hdfs.TestEncryptionZonesWithHA | | | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure100 | | | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure080 | | | hadoop.hdfs.TestFileConcurrentReader | | | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure | | | hadoop.hdfs.TestEncryptionZones | | | hadoop.hdfs.TestMaintenanceState | | | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure140 | | Timed out junit tests | org.apache.hadoop.hdfs.TestWriteReadStripedFile | | | org.apache.hadoop.hdfs.TestReadStripedFileWithDecoding | \\ \\ || Subsystem || Report/Notes || | Docker | Image:yetus/hadoop:71bbb86 | | JIRA Issue | HDFS-12383 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12884827/HDFS-12383.02.patch | | Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle | | uname | Linux 5c8d0924a636 3.13.0-117-generic #164-Ubuntu
[jira] [Commented] (HDFS-12383) Re-encryption updater should handle canceled tasks better
[ https://issues.apache.org/jira/browse/HDFS-12383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16149790#comment-16149790 ] Wei-Chiu Chuang commented on HDFS-12383: ReencryptionUpdater does not capture CancellationException so the thread would terminate upon CANCEL reencryption command, and reencryption can't be resumed as a result. The 002 patch looks mostly good. {code} if (completed.isCancelled()) { LOG.debug("Skipped canceled re-encryption task for zone {}, last: {}", task.zoneId, task.lastFile); return; } {code} How come I missed it? Sorry about that. {code} private boolean isRunning = false; {code} should it be a volatile variable? {code} try { } finally { LOG.info("Re-encrypted callable running = {} ", callableRunning.get()); } {code} I feel this try block is not needed. But either way is good for me. > Re-encryption updater should handle canceled tasks better > - > > Key: HDFS-12383 > URL: https://issues.apache.org/jira/browse/HDFS-12383 > Project: Hadoop HDFS > Issue Type: Bug > Components: encryption >Affects Versions: 3.0.0-beta1 >Reporter: Xiao Chen >Assignee: Xiao Chen > Attachments: HDFS-12383.01.patch > > > Seen an instance where the re-encryption updater exited due to an exception, > and later tasks no longer executes. Logs below: > {noformat} > 2017-08-31 09:54:08,104 INFO > org.apache.hadoop.hdfs.server.namenode.EncryptionZoneManager: Zone > /tmp/encryption-zone-3(16819) is submitted for re-encryption. > 2017-08-31 09:54:08,104 INFO > org.apache.hadoop.hdfs.server.namenode.ReencryptionHandler: Executing > re-encrypt commands on zone 16819. Current zones:[zone:16787 state:Completed > lastProcessed:null filesReencrypted:1 fileReencryptionFailures:0][zone:16813 > state:Completed lastProcessed:null filesReencrypted:1 > fileReencryptionFailures:0][zone:16819 state:Submitted lastProcessed:null > filesReencrypted:0 fileReencryptionFailures:0] > 2017-08-31 09:54:08,105 INFO > org.apache.hadoop.hdfs.protocol.ReencryptionStatus: Zone 16819 starts > re-encryption processing > 2017-08-31 09:54:08,105 INFO > org.apache.hadoop.hdfs.server.namenode.ReencryptionHandler: Re-encrypting > zone /tmp/encryption-zone-3(id=16819) > 2017-08-31 09:54:08,105 INFO > org.apache.hadoop.hdfs.server.namenode.ReencryptionHandler: Submitted batch > (start:/tmp/encryption-zone-3/data1, size:1) of zone 16819 to re-encrypt. > 2017-08-31 09:54:08,105 INFO > org.apache.hadoop.hdfs.server.namenode.ReencryptionHandler: Submission > completed of zone 16819 for re-encryption. > 2017-08-31 09:54:08,105 INFO > org.apache.hadoop.hdfs.server.namenode.ReencryptionHandler: Processing > batched re-encryption for zone 16819, batch size 1, > start:/tmp/encryption-zone-3/data1 > 2017-08-31 09:54:08,979 INFO BlockStateChange: BLOCK* BlockManager: ask > 172.26.1.71:20002 to delete [blk_1073742291_1467] > 2017-08-31 09:54:18,295 INFO > org.apache.hadoop.hdfs.server.namenode.ReencryptionUpdater: Cancelling 1 > re-encryption tasks > 2017-08-31 09:54:18,295 INFO > org.apache.hadoop.hdfs.server.namenode.EncryptionZoneManager: Cancelled zone > /tmp/encryption-zone-3(16819) for re-encryption. > 2017-08-31 09:54:18,295 INFO > org.apache.hadoop.hdfs.protocol.ReencryptionStatus: Zone 16819 completed > re-encryption. > 2017-08-31 09:54:18,296 INFO > org.apache.hadoop.hdfs.server.namenode.ReencryptionHandler: Completed > re-encrypting one batch of 1 edeks from KMS, time consumed: 10.19 s, start: > /tmp/encryption-zone-3/data1. > 2017-08-31 09:54:18,296 ERROR > org.apache.hadoop.hdfs.server.namenode.ReencryptionUpdater: Re-encryption > updater thread exiting. > java.util.concurrent.CancellationException > at java.util.concurrent.FutureTask.report(FutureTask.java:121) > at java.util.concurrent.FutureTask.get(FutureTask.java:192) > at > org.apache.hadoop.hdfs.server.namenode.ReencryptionUpdater.takeAndProcessTasks(ReencryptionUpdater.java:404) > at > org.apache.hadoop.hdfs.server.namenode.ReencryptionUpdater.run(ReencryptionUpdater.java:250) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:745) > {noformat} > Updater should be fixed to handle canceled tasks better. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12383) Re-encryption updater should handle canceled tasks better
[ https://issues.apache.org/jira/browse/HDFS-12383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16149788#comment-16149788 ] Hadoop QA commented on HDFS-12383: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 17m 13s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 1 new or modified test files. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 16m 35s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 5s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 40s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 2s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 3s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 44s{color} | {color:green} trunk passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 54s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 51s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 51s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 36s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 0s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 56s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 44s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:red}-1{color} | {color:red} unit {color} | {color:red} 99m 27s{color} | {color:red} hadoop-hdfs in the patch failed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 25s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black}146m 52s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | Failed junit tests | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure060 | | | hadoop.hdfs.TestDFSStripedOutputStreamWithFailureWithRandomECPolicy | | | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure160 | | | hadoop.hdfs.TestClientProtocolForPipelineRecovery | | | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure130 | | | hadoop.hdfs.TestEncryptedTransfer | | | hadoop.hdfs.TestFileAppendRestart | | | hadoop.hdfs.TestDFSStripedInputStreamWithRandomECPolicy | | | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure000 | | | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure050 | | | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure110 | | | hadoop.hdfs.server.namenode.web.resources.TestWebHdfsDataLocality | | | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure190 | | | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure210 | | | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure070 | | | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure170 | | | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure | | | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure090 | | | hadoop.hdfs.TestLeaseRecoveryStriped | | | hadoop.hdfs.TestFileCreationEmpty | | | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure200 | | | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure040 | | | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure120 | | | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure010 | | | hadoop.hdfs.server.datanode.TestDataNodeErasureCodingMetrics | | | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure150 | | | hadoop.hdfs.TestReconstructStripedFile | | | hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistReplicaRecovery | | Timed