[jira] [Commented] (HDFS-13330) ShortCircuitCache#fetchOrCreate never retries
[ https://issues.apache.org/jira/browse/HDFS-13330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16437658#comment-16437658 ] genericqa commented on HDFS-13330: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s{color} | {color:blue} Docker mode activated. {color} | | {color:red}-1{color} | {color:red} docker {color} | {color:red} 1m 7s{color} | {color:red} Docker failed to build yetus/hadoop:f667ef1. {color} | \\ \\ || Subsystem || Report/Notes || | JIRA Issue | HDFS-13330 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12918984/HDFS-13330.branch-2.patch | | Console output | https://builds.apache.org/job/PreCommit-HDFS-Build/23930/console | | Powered by | Apache Yetus 0.8.0-SNAPSHOT http://yetus.apache.org | This message was automatically generated. > ShortCircuitCache#fetchOrCreate never retries > - > > Key: HDFS-13330 > URL: https://issues.apache.org/jira/browse/HDFS-13330 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Wei-Chiu Chuang >Assignee: Gabor Bota >Priority: Major > Fix For: 3.2.0 > > Attachments: HDFS-13330.001.patch, HDFS-13330.002.patch, > HDFS-13330.004.patch, HDFS-13330.005.patch, HDFS-13330.branch-2.patch > > > The follow do .. while(false) loop seems useless to me. The code intended to > retry but it never worked. Let's fix it. > {code:java:title=ShortCircuitCache#fetchOrCreate} > ShortCircuitReplicaInfo info = null; > do { > if (closed) { > LOG.trace("{}: can't fethchOrCreate {} because the cache is closed.", > this, key); > return null; > } > Waitable waitable = replicaInfoMap.get(key); > if (waitable != null) { > try { > info = fetch(key, waitable); > } catch (RetriableException e) { > LOG.debug("{}: retrying {}", this, e.getMessage()); > } > } > } while (false);{code} -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13330) ShortCircuitCache#fetchOrCreate never retries
[ https://issues.apache.org/jira/browse/HDFS-13330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16437644#comment-16437644 ] Wei-Chiu Chuang commented on HDFS-13330: Attach a branch-2 patch. There's a very trivial change in the test code. {code:java} .fetch(Mockito.eq(extendedBlockId), Mockito.any(Waitable.class)); {code} > ShortCircuitCache#fetchOrCreate never retries > - > > Key: HDFS-13330 > URL: https://issues.apache.org/jira/browse/HDFS-13330 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Wei-Chiu Chuang >Assignee: Gabor Bota >Priority: Major > Fix For: 3.2.0 > > Attachments: HDFS-13330.001.patch, HDFS-13330.002.patch, > HDFS-13330.004.patch, HDFS-13330.005.patch, HDFS-13330.branch-2.patch > > > The follow do .. while(false) loop seems useless to me. The code intended to > retry but it never worked. Let's fix it. > {code:java:title=ShortCircuitCache#fetchOrCreate} > ShortCircuitReplicaInfo info = null; > do { > if (closed) { > LOG.trace("{}: can't fethchOrCreate {} because the cache is closed.", > this, key); > return null; > } > Waitable waitable = replicaInfoMap.get(key); > if (waitable != null) { > try { > info = fetch(key, waitable); > } catch (RetriableException e) { > LOG.debug("{}: retrying {}", this, e.getMessage()); > } > } > } while (false);{code} -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13330) ShortCircuitCache#fetchOrCreate never retries
[ https://issues.apache.org/jira/browse/HDFS-13330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16437560#comment-16437560 ] Hudson commented on HDFS-13330: --- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #13994 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/13994/]) HDFS-13330. ShortCircuitCache#fetchOrCreate never retries. Contributed (weichiu: rev e66e287efe2b43e710137a628f03c7df3ebdf498) * (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/shortcircuit/TestShortCircuitCache.java * (edit) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/shortcircuit/ShortCircuitCache.java > ShortCircuitCache#fetchOrCreate never retries > - > > Key: HDFS-13330 > URL: https://issues.apache.org/jira/browse/HDFS-13330 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Wei-Chiu Chuang >Assignee: Gabor Bota >Priority: Major > Fix For: 3.2.0 > > Attachments: HDFS-13330.001.patch, HDFS-13330.002.patch, > HDFS-13330.004.patch, HDFS-13330.005.patch > > > The follow do .. while(false) loop seems useless to me. The code intended to > retry but it never worked. Let's fix it. > {code:java:title=ShortCircuitCache#fetchOrCreate} > ShortCircuitReplicaInfo info = null; > do { > if (closed) { > LOG.trace("{}: can't fethchOrCreate {} because the cache is closed.", > this, key); > return null; > } > Waitable waitable = replicaInfoMap.get(key); > if (waitable != null) { > try { > info = fetch(key, waitable); > } catch (RetriableException e) { > LOG.debug("{}: retrying {}", this, e.getMessage()); > } > } > } while (false);{code} -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13330) ShortCircuitCache#fetchOrCreate never retries
[ https://issues.apache.org/jira/browse/HDFS-13330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16437414#comment-16437414 ] Gabor Bota commented on HDFS-13330: --- Thank you [~jojochuang]! > ShortCircuitCache#fetchOrCreate never retries > - > > Key: HDFS-13330 > URL: https://issues.apache.org/jira/browse/HDFS-13330 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Wei-Chiu Chuang >Assignee: Gabor Bota >Priority: Major > Attachments: HDFS-13330.001.patch, HDFS-13330.002.patch, > HDFS-13330.004.patch, HDFS-13330.005.patch > > > The follow do .. while(false) loop seems useless to me. The code intended to > retry but it never worked. Let's fix it. > {code:java:title=ShortCircuitCache#fetchOrCreate} > ShortCircuitReplicaInfo info = null; > do { > if (closed) { > LOG.trace("{}: can't fethchOrCreate {} because the cache is closed.", > this, key); > return null; > } > Waitable waitable = replicaInfoMap.get(key); > if (waitable != null) { > try { > info = fetch(key, waitable); > } catch (RetriableException e) { > LOG.debug("{}: retrying {}", this, e.getMessage()); > } > } > } while (false);{code} -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13330) ShortCircuitCache#fetchOrCreate never retries
[ https://issues.apache.org/jira/browse/HDFS-13330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16432860#comment-16432860 ] Wei-Chiu Chuang commented on HDFS-13330: +1. test failures are unrelated. > ShortCircuitCache#fetchOrCreate never retries > - > > Key: HDFS-13330 > URL: https://issues.apache.org/jira/browse/HDFS-13330 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Wei-Chiu Chuang >Assignee: Gabor Bota >Priority: Major > Attachments: HDFS-13330.001.patch, HDFS-13330.002.patch, > HDFS-13330.004.patch, HDFS-13330.005.patch > > > The follow do .. while(false) loop seems useless to me. The code intended to > retry but it never worked. Let's fix it. > {code:java:title=ShortCircuitCache#fetchOrCreate} > ShortCircuitReplicaInfo info = null; > do { > if (closed) { > LOG.trace("{}: can't fethchOrCreate {} because the cache is closed.", > this, key); > return null; > } > Waitable waitable = replicaInfoMap.get(key); > if (waitable != null) { > try { > info = fetch(key, waitable); > } catch (RetriableException e) { > LOG.debug("{}: retrying {}", this, e.getMessage()); > } > } > } while (false);{code} -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13330) ShortCircuitCache#fetchOrCreate never retries
[ https://issues.apache.org/jira/browse/HDFS-13330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16426922#comment-16426922 ] genericqa commented on HDFS-13330: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 19s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 1 new or modified test files. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 27s{color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 25m 36s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 16m 49s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m 3s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 54s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 13m 7s{color} | {color:green} branch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m 35s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 14s{color} | {color:green} trunk passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 10s{color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 42s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 16m 38s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 16m 38s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 58s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 39s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 11m 12s{color} | {color:green} patch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m 52s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 13s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 28s{color} | {color:green} hadoop-hdfs-client in the patch passed. {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red}107m 33s{color} | {color:red} hadoop-hdfs in the patch failed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 30s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black}209m 53s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | Failed junit tests | hadoop.hdfs.TestDFSStripedOutputStreamWithFailureWithRandomECPolicy | | | hadoop.hdfs.server.namenode.ha.TestRetryCacheWithHA | | | hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting | \\ \\ || Subsystem || Report/Notes || | Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:8620d2b | | JIRA Issue | HDFS-13330 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12917660/HDFS-13330.005.patch | | Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux 152672f74bbf 3.13.0-143-generic #192-Ubuntu SMP Tue Feb 27 10:45:36 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/patchprocess/precommit/personality/provided.sh | | git revision | trunk / e52539b | | maven | version: Apache
[jira] [Commented] (HDFS-13330) ShortCircuitCache#fetchOrCreate never retries
[ https://issues.apache.org/jira/browse/HDFS-13330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16426685#comment-16426685 ] Gabor Bota commented on HDFS-13330: --- Thanks [~jojochuang]! Good idea to move the value to a static final field, and I'll use try-with-resource as well. > ShortCircuitCache#fetchOrCreate never retries > - > > Key: HDFS-13330 > URL: https://issues.apache.org/jira/browse/HDFS-13330 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Wei-Chiu Chuang >Assignee: Gabor Bota >Priority: Major > Attachments: HDFS-13330.001.patch, HDFS-13330.002.patch, > HDFS-13330.004.patch > > > The follow do .. while(false) loop seems useless to me. The code intended to > retry but it never worked. Let's fix it. > {code:java:title=ShortCircuitCache#fetchOrCreate} > ShortCircuitReplicaInfo info = null; > do { > if (closed) { > LOG.trace("{}: can't fethchOrCreate {} because the cache is closed.", > this, key); > return null; > } > Waitable waitable = replicaInfoMap.get(key); > if (waitable != null) { > try { > info = fetch(key, waitable); > } catch (RetriableException e) { > LOG.debug("{}: retrying {}", this, e.getMessage()); > } > } > } while (false);{code} -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13330) ShortCircuitCache#fetchOrCreate never retries
[ https://issues.apache.org/jira/browse/HDFS-13330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16426340#comment-16426340 ] Wei-Chiu Chuang commented on HDFS-13330: Hi [~gabor.bota] thanks for the patch. Overall looks good with a little suggestion for further improvement. {code:java} for (int i = 0; i < 3; i++){ {code} It would be better to make the number 3 a static final variable. {code:java} // Teardown cache.close(); {code} You might want to consider using the try with resource statement to ensure it is closed properly. > ShortCircuitCache#fetchOrCreate never retries > - > > Key: HDFS-13330 > URL: https://issues.apache.org/jira/browse/HDFS-13330 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Wei-Chiu Chuang >Assignee: Gabor Bota >Priority: Major > Attachments: HDFS-13330.001.patch, HDFS-13330.002.patch, > HDFS-13330.004.patch > > > The follow do .. while(false) loop seems useless to me. The code intended to > retry but it never worked. Let's fix it. > {code:java:title=ShortCircuitCache#fetchOrCreate} > ShortCircuitReplicaInfo info = null; > do { > if (closed) { > LOG.trace("{}: can't fethchOrCreate {} because the cache is closed.", > this, key); > return null; > } > Waitable waitable = replicaInfoMap.get(key); > if (waitable != null) { > try { > info = fetch(key, waitable); > } catch (RetriableException e) { > LOG.debug("{}: retrying {}", this, e.getMessage()); > } > } > } while (false);{code} -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13330) ShortCircuitCache#fetchOrCreate never retries
[ https://issues.apache.org/jira/browse/HDFS-13330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16423654#comment-16423654 ] Gabor Bota commented on HDFS-13330: --- I've tested TestDataNodeVolumeFailure#testUnderReplicationAfterVolFailure today, and it passed for trunk, and with this patch too. > ShortCircuitCache#fetchOrCreate never retries > - > > Key: HDFS-13330 > URL: https://issues.apache.org/jira/browse/HDFS-13330 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Wei-Chiu Chuang >Assignee: Gabor Bota >Priority: Major > Attachments: HDFS-13330.001.patch, HDFS-13330.002.patch, > HDFS-13330.004.patch > > > The follow do .. while(false) loop seems useless to me. The code intended to > retry but it never worked. Let's fix it. > {code:java:title=ShortCircuitCache#fetchOrCreate} > ShortCircuitReplicaInfo info = null; > do { > if (closed) { > LOG.trace("{}: can't fethchOrCreate {} because the cache is closed.", > this, key); > return null; > } > Waitable waitable = replicaInfoMap.get(key); > if (waitable != null) { > try { > info = fetch(key, waitable); > } catch (RetriableException e) { > LOG.debug("{}: retrying {}", this, e.getMessage()); > } > } > } while (false);{code} -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13330) ShortCircuitCache#fetchOrCreate never retries
[ https://issues.apache.org/jira/browse/HDFS-13330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16421597#comment-16421597 ] Gabor Bota commented on HDFS-13330: --- org.apache.hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure#testUnderReplicationAfterVolFailure failed for me locally *with and without this patch* (on trunk), but I was not able to reproduce any other failed JUnit test fails. Maybe there's a known issue with testUnderReplicationAfterVolFailure, or that would be a new find? > ShortCircuitCache#fetchOrCreate never retries > - > > Key: HDFS-13330 > URL: https://issues.apache.org/jira/browse/HDFS-13330 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Wei-Chiu Chuang >Assignee: Gabor Bota >Priority: Major > Attachments: HDFS-13330.001.patch, HDFS-13330.002.patch, > HDFS-13330.004.patch > > > The follow do .. while(false) loop seems useless to me. The code intended to > retry but it never worked. Let's fix it. > {code:java:title=ShortCircuitCache#fetchOrCreate} > ShortCircuitReplicaInfo info = null; > do { > if (closed) { > LOG.trace("{}: can't fethchOrCreate {} because the cache is closed.", > this, key); > return null; > } > Waitable waitable = replicaInfoMap.get(key); > if (waitable != null) { > try { > info = fetch(key, waitable); > } catch (RetriableException e) { > LOG.debug("{}: retrying {}", this, e.getMessage()); > } > } > } while (false);{code} -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13330) ShortCircuitCache#fetchOrCreate never retries
[ https://issues.apache.org/jira/browse/HDFS-13330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16421547#comment-16421547 ] genericqa commented on HDFS-13330: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 18s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 1 new or modified test files. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 26s{color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 26m 47s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 16m 38s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m 1s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 44s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 13m 8s{color} | {color:green} branch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m 27s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 14s{color} | {color:green} trunk passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 10s{color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 41s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 16m 35s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 16m 35s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 58s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 37s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 11m 23s{color} | {color:green} patch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m 9s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 19s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 34s{color} | {color:green} hadoop-hdfs-client in the patch passed. {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red}107m 52s{color} | {color:red} hadoop-hdfs in the patch failed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 27s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black}211m 48s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | Failed junit tests | hadoop.hdfs.server.namenode.ha.TestRetryCacheWithHA | | | hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure | | | hadoop.hdfs.TestMaintenanceState | \\ \\ || Subsystem || Report/Notes || | Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:8620d2b | | JIRA Issue | HDFS-13330 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12917121/HDFS-13330.004.patch | | Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux 9a35a9ae5151 3.13.0-143-generic #192-Ubuntu SMP Tue Feb 27 10:45:36 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/patchprocess/precommit/personality/provided.sh | | git revision | trunk / dc8e343 | | maven | version: Apache Maven 3.3.9 | | Default Java | 1.8.0_151 | |
[jira] [Commented] (HDFS-13330) ShortCircuitCache#fetchOrCreate never retries
[ https://issues.apache.org/jira/browse/HDFS-13330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16421545#comment-16421545 ] genericqa commented on HDFS-13330: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 17s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 1 new or modified test files. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 25s{color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 26m 26s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 16m 43s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m 2s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 45s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 13m 19s{color} | {color:green} branch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m 30s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 14s{color} | {color:green} trunk passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 10s{color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 41s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 16m 49s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 16m 49s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 57s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 37s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 11m 10s{color} | {color:green} patch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m 39s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 12s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 26s{color} | {color:green} hadoop-hdfs-client in the patch passed. {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red}108m 6s{color} | {color:red} hadoop-hdfs in the patch failed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 35s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black}211m 15s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | Failed junit tests | hadoop.hdfs.server.namenode.ha.TestRetryCacheWithHA | | | hadoop.hdfs.TestSafeModeWithStripedFileWithRandomECPolicy | | | hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting | \\ \\ || Subsystem || Report/Notes || | Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:8620d2b | | JIRA Issue | HDFS-13330 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12917120/HDFS-13330.003.patch | | Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux c9aab21294e6 3.13.0-143-generic #192-Ubuntu SMP Tue Feb 27 10:45:36 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/patchprocess/precommit/personality/provided.sh | | git revision | trunk / dc8e343 | | maven | version: Apache Maven 3.3.9 |
[jira] [Commented] (HDFS-13330) ShortCircuitCache#fetchOrCreate never retries
[ https://issues.apache.org/jira/browse/HDFS-13330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16418209#comment-16418209 ] Chris Douglas commented on HDFS-13330: -- bq. Could you also fix the findbugs error? In this case, the findbugs warning is an artifact of the infinite loop. It infers (correctly) that the only way to exit the loop is for {{info}} to be assigned a non-null value. As you point out, it should also exit the loop before assigning {{info}} when {{replicaInfoMap.get(key)}} returns null, as [above|https://issues.apache.org/jira/browse/HDFS-13330?focusedCommentId=16412164=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16412164]. Since this retry code has never been enabled, this functionality is either unused, reliable "enough", or some other layer implements the retry logic. Changing this to a {{for}} loop with a low, fixed number of retries (i.e., 2, 3 at the most) is probably sufficient. We'd like to know if/how it's used in applications, but absent that analysis, fixing it to work "as designed" is probably the best we're going to do. bq. In addition, a test case for this method is greatly appreciated. +1 The retry semantics of the committed code were clearly untested. > ShortCircuitCache#fetchOrCreate never retries > - > > Key: HDFS-13330 > URL: https://issues.apache.org/jira/browse/HDFS-13330 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Wei-Chiu Chuang >Assignee: Gabor Bota >Priority: Major > Attachments: HDFS-13330.001.patch, HDFS-13330.002.patch > > > The follow do .. while(false) loop seems useless to me. The code intended to > retry but it never worked. Let's fix it. > {code:java:title=ShortCircuitCache#fetchOrCreate} > ShortCircuitReplicaInfo info = null; > do { > if (closed) { > LOG.trace("{}: can't fethchOrCreate {} because the cache is closed.", > this, key); > return null; > } > Waitable waitable = replicaInfoMap.get(key); > if (waitable != null) { > try { > info = fetch(key, waitable); > } catch (RetriableException e) { > LOG.debug("{}: retrying {}", this, e.getMessage()); > } > } > } while (false);{code} -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13330) ShortCircuitCache#fetchOrCreate never retries
[ https://issues.apache.org/jira/browse/HDFS-13330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16417880#comment-16417880 ] Wei-Chiu Chuang commented on HDFS-13330: Hi [~gabor.bota] thanks for the patch. I think this bug is more involved than it seems. An infinite loop looks even scarier than a loop that never retries. Suppose there's a bug somewhere else and replicaInfoMap.get(key) return null, this loop will never end, because it runs inside a lock, and no one will be able to update it. Could you also fix the findbugs error? In addition, a test case for this method is greatly appreciated. > ShortCircuitCache#fetchOrCreate never retries > - > > Key: HDFS-13330 > URL: https://issues.apache.org/jira/browse/HDFS-13330 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Wei-Chiu Chuang >Assignee: Gabor Bota >Priority: Minor > Attachments: HDFS-13330.001.patch, HDFS-13330.002.patch > > > The follow do .. while(false) loop seems useless to me. The code intended to > retry but it never worked. Let's fix it. > {code:java:title=ShortCircuitCache#fetchOrCreate} > ShortCircuitReplicaInfo info = null; > do { > if (closed) { > LOG.trace("{}: can't fethchOrCreate {} because the cache is closed.", > this, key); > return null; > } > Waitable waitable = replicaInfoMap.get(key); > if (waitable != null) { > try { > info = fetch(key, waitable); > } catch (RetriableException e) { > LOG.debug("{}: retrying {}", this, e.getMessage()); > } > } > } while (false);{code} -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13330) ShortCircuitCache#fetchOrCreate never retries
[ https://issues.apache.org/jira/browse/HDFS-13330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16417169#comment-16417169 ] genericqa commented on HDFS-13330: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 42s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s{color} | {color:red} The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 23m 35s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 33s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 17s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 35s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 9m 25s{color} | {color:green} branch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 26s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 24s{color} | {color:green} trunk passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 37s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 30s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 30s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 14s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 33s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 9m 37s{color} | {color:green} patch has no errors when building and testing our client artifacts. {color} | | {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m 34s{color} | {color:red} hadoop-hdfs-project/hadoop-hdfs-client generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 22s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 26s{color} | {color:green} hadoop-hdfs-client in the patch passed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 18s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 52m 35s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | FindBugs | module:hadoop-hdfs-project/hadoop-hdfs-client | | | Redundant nullcheck of info, which is known to be non-null in org.apache.hadoop.hdfs.shortcircuit.ShortCircuitCache.fetchOrCreate(ExtendedBlockId, ShortCircuitCache$ShortCircuitReplicaCreator) Redundant null check at ShortCircuitCache.java:is known to be non-null in org.apache.hadoop.hdfs.shortcircuit.ShortCircuitCache.fetchOrCreate(ExtendedBlockId, ShortCircuitCache$ShortCircuitReplicaCreator) Redundant null check at ShortCircuitCache.java:[line 701] | \\ \\ || Subsystem || Report/Notes || | Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:8620d2b | | JIRA Issue | HDFS-13330 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12916583/HDFS-13330.002.patch | | Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux 7210b6b488e7 4.4.0-64-generic #85-Ubuntu SMP Mon Feb 20 11:50:30 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality |