[jira] [Assigned] (HDFS-16640) RBF: router datanode UI show ip list when click dn histogram
[ https://issues.apache.org/jira/browse/HDFS-16640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangzhaohui reassigned HDFS-16640: -- Assignee: wangzhaohui > RBF: router datanode UI show ip list when click dn histogram > > > Key: HDFS-16640 > URL: https://issues.apache.org/jira/browse/HDFS-16640 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: wangzhaohui >Assignee: wangzhaohui >Priority: Minor > -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Created] (HDFS-16640) RBF: router datanode UI show ip list when click dn histogram
wangzhaohui created HDFS-16640: -- Summary: RBF: router datanode UI show ip list when click dn histogram Key: HDFS-16640 URL: https://issues.apache.org/jira/browse/HDFS-16640 Project: Hadoop HDFS Issue Type: Improvement Reporter: wangzhaohui -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Created] (HDFS-16529) Remove unnecessary setObserverRead in TestConsistentReadsObserver
wangzhaohui created HDFS-16529: -- Summary: Remove unnecessary setObserverRead in TestConsistentReadsObserver Key: HDFS-16529 URL: https://issues.apache.org/jira/browse/HDFS-16529 Project: Hadoop HDFS Issue Type: Improvement Reporter: wangzhaohui -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Created] (HDFS-16391) Avoid evaluation of LOG.debug statement in NameNodeHeartbeatService
wangzhaohui created HDFS-16391: -- Summary: Avoid evaluation of LOG.debug statement in NameNodeHeartbeatService Key: HDFS-16391 URL: https://issues.apache.org/jira/browse/HDFS-16391 Project: Hadoop HDFS Issue Type: Improvement Reporter: wangzhaohui -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Assigned] (HDFS-16364) Remove unnecessary brackets in NameNodeRpcServer#L453
[ https://issues.apache.org/jira/browse/HDFS-16364?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangzhaohui reassigned HDFS-16364: -- Assignee: wangzhaohui > Remove unnecessary brackets in NameNodeRpcServer#L453 > - > > Key: HDFS-16364 > URL: https://issues.apache.org/jira/browse/HDFS-16364 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: wangzhaohui >Assignee: wangzhaohui >Priority: Trivial > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Created] (HDFS-16364) Remove unnecessary brackets in NameNodeRpcServer#L453
wangzhaohui created HDFS-16364: -- Summary: Remove unnecessary brackets in NameNodeRpcServer#L453 Key: HDFS-16364 URL: https://issues.apache.org/jira/browse/HDFS-16364 Project: Hadoop HDFS Issue Type: Improvement Reporter: wangzhaohui -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-16183) RBF: Delete unnecessary calls of proxyOpComplete in routerRpcClient
[ https://issues.apache.org/jira/browse/HDFS-16183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangzhaohui updated HDFS-16183: --- Summary: RBF: Delete unnecessary calls of proxyOpComplete in routerRpcClient (was: RBF: Delete unnecessary metric of proxyOpComplete in routerRpcClient) > RBF: Delete unnecessary calls of proxyOpComplete in routerRpcClient > > > Key: HDFS-16183 > URL: https://issues.apache.org/jira/browse/HDFS-16183 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: wangzhaohui >Assignee: wangzhaohui >Priority: Minor > Labels: pull-request-available > Time Spent: 1.5h > Remaining Estimate: 0h > > In routerRpcClient, this.rpcMonitor.proxyOpComplete(false) do nothing, so > maybe it doesn't need to exist. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-16183) RBF: Delete unnecessary metric of proxyOpComplete in routerRpcClient
[ https://issues.apache.org/jira/browse/HDFS-16183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangzhaohui updated HDFS-16183: --- Description: In routerRpcClient, this.rpcMonitor.proxyOpComplete(false) do nothing, so maybe it doesn't need to exist. > RBF: Delete unnecessary metric of proxyOpComplete in routerRpcClient > - > > Key: HDFS-16183 > URL: https://issues.apache.org/jira/browse/HDFS-16183 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: wangzhaohui >Assignee: wangzhaohui >Priority: Minor > > In routerRpcClient, this.rpcMonitor.proxyOpComplete(false) do nothing, so > maybe it doesn't need to exist. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Created] (HDFS-16183) RBF: Delete unnecessary metric of proxyOpComplete in routerRpcClient
wangzhaohui created HDFS-16183: -- Summary: RBF: Delete unnecessary metric of proxyOpComplete in routerRpcClient Key: HDFS-16183 URL: https://issues.apache.org/jira/browse/HDFS-16183 Project: Hadoop HDFS Issue Type: Improvement Reporter: wangzhaohui -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Assigned] (HDFS-16183) RBF: Delete unnecessary metric of proxyOpComplete in routerRpcClient
[ https://issues.apache.org/jira/browse/HDFS-16183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangzhaohui reassigned HDFS-16183: -- Assignee: wangzhaohui > RBF: Delete unnecessary metric of proxyOpComplete in routerRpcClient > - > > Key: HDFS-16183 > URL: https://issues.apache.org/jira/browse/HDFS-16183 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: wangzhaohui >Assignee: wangzhaohui >Priority: Minor > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-16181) [SBN Read] Fix metric of RpcRequestCacheMissAmount can't display when tailEditLog form JN
[ https://issues.apache.org/jira/browse/HDFS-16181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangzhaohui updated HDFS-16181: --- Summary: [SBN Read] Fix metric of RpcRequestCacheMissAmount can't display when tailEditLog form JN (was: [SBN] Fix metric of RpcRequestCacheMissAmount can't display when tailEditLog form JN) > [SBN Read] Fix metric of RpcRequestCacheMissAmount can't display when > tailEditLog form JN > - > > Key: HDFS-16181 > URL: https://issues.apache.org/jira/browse/HDFS-16181 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: wangzhaohui >Assignee: wangzhaohui >Priority: Critical > Labels: pull-request-available > Attachments: after.jpg, before.jpg > > Time Spent: 20m > Remaining Estimate: 0h > > I found the JN turn on edit cache, but the metric of > rpcRequestCacheMissAmount can not display. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-16181) [SBN] Fix metric of RpcRequestCacheMissAmount can't display when tailEditLog form JN
[ https://issues.apache.org/jira/browse/HDFS-16181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangzhaohui updated HDFS-16181: --- Attachment: after.jpg > [SBN] Fix metric of RpcRequestCacheMissAmount can't display when tailEditLog > form JN > > > Key: HDFS-16181 > URL: https://issues.apache.org/jira/browse/HDFS-16181 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: wangzhaohui >Assignee: wangzhaohui >Priority: Critical > Labels: pull-request-available > Attachments: after.jpg, before.jpg > > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-16181) [SBN] Fix metric of RpcRequestCacheMissAmount can't display when tailEditLog form JN
[ https://issues.apache.org/jira/browse/HDFS-16181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangzhaohui updated HDFS-16181: --- Description: I found the JN turn on edit cache, but the metric of rpcRequestCacheMissAmount can not display. > [SBN] Fix metric of RpcRequestCacheMissAmount can't display when tailEditLog > form JN > > > Key: HDFS-16181 > URL: https://issues.apache.org/jira/browse/HDFS-16181 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: wangzhaohui >Assignee: wangzhaohui >Priority: Critical > Labels: pull-request-available > Attachments: after.jpg, before.jpg > > Time Spent: 10m > Remaining Estimate: 0h > > I found the JN turn on edit cache, but the metric of > rpcRequestCacheMissAmount can not display. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-16181) [SBN] Fix metric of RpcRequestCacheMissAmount can't display when tailEditLog form JN
[ https://issues.apache.org/jira/browse/HDFS-16181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangzhaohui updated HDFS-16181: --- Attachment: before.jpg > [SBN] Fix metric of RpcRequestCacheMissAmount can't display when tailEditLog > form JN > > > Key: HDFS-16181 > URL: https://issues.apache.org/jira/browse/HDFS-16181 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: wangzhaohui >Assignee: wangzhaohui >Priority: Critical > Labels: pull-request-available > Attachments: before.jpg > > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Assigned] (HDFS-16181) [SBN] Fix metric of RpcRequestCacheMissAmount can't display when tailEditLog form JN
[ https://issues.apache.org/jira/browse/HDFS-16181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangzhaohui reassigned HDFS-16181: -- Assignee: wangzhaohui > [SBN] Fix metric of RpcRequestCacheMissAmount can't display when tailEditLog > form JN > > > Key: HDFS-16181 > URL: https://issues.apache.org/jira/browse/HDFS-16181 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: wangzhaohui >Assignee: wangzhaohui >Priority: Critical > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Created] (HDFS-16181) [SBN] Fix metric of RpcRequestCacheMissAmount can't display when tailEditLog form JN
wangzhaohui created HDFS-16181: -- Summary: [SBN] Fix metric of RpcRequestCacheMissAmount can't display when tailEditLog form JN Key: HDFS-16181 URL: https://issues.apache.org/jira/browse/HDFS-16181 Project: Hadoop HDFS Issue Type: Bug Reporter: wangzhaohui -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-16154) TestMiniJournalCluster failing intermittently because of not reseting UserGroupInformation completely
[
https://issues.apache.org/jira/browse/HDFS-16154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
wangzhaohui updated HDFS-16154:
---
Description:
When I run the UT of org.apache.hadoop.hdfs.qjournal together at IDEA, there
are many failed UT.
I found they has the same reason ,
{code:java}
java.io.IOException: Running in secure mode, but config doesn't have a
keytabjava.io.IOException: Running in secure mode, but config doesn't have a
keytab
at org.apache.hadoop.security.SecurityUtil.login(SecurityUtil.java:308) at
org.apache.hadoop.hdfs.qjournal.server.JournalNode.start(JournalNode.java:230)
at
org.apache.hadoop.hdfs.qjournal.MiniJournalCluster.(MiniJournalCluster.java:121)
at
org.apache.hadoop.hdfs.qjournal.MiniJournalCluster.(MiniJournalCluster.java:48)
at
org.apache.hadoop.hdfs.qjournal.MiniJournalCluster$Builder.build(MiniJournalCluster.java:80)
at
org.apache.hadoop.hdfs.qjournal.TestMiniJournalCluster.testStartStop(TestMiniJournalCluster.java:38)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498) at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325) at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) at
org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) at
org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288) at
org.junit.runners.ParentRunner.access$000(ParentRunner.java:58) at
org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268) at
org.junit.runners.ParentRunner.run(ParentRunner.java:363) at
org.junit.runner.JUnitCore.run(JUnitCore.java:137) at
org.junit.runner.JUnitCore.run(JUnitCore.java:115) at
org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:40)
at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184) at
java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193) at
java.util.Iterator.forEachRemaining(Iterator.java:116) at
java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482) at
java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472) at
java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
at
java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) at
java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418) at
org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80)
at
org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:71)
at
org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:229)
at
org.junit.platform.launcher.core.DefaultLauncher.lambda$execute$6(DefaultLauncher.java:197)
at
org.junit.platform.launcher.core.DefaultLauncher.withInterceptedStreams(DefaultLauncher.java:211)
at
org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:191)
at
org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:128)
at
com.intellij.junit5.JUnit5IdeaTestRunner.startRunnerWithArgs(JUnit5IdeaTestRunner.java:71)
at
com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:33)
at
com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:220)
at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:53)
{code}
The TestSecureNNWithQJM not reset the UGI, this affected the other unit tests
threw {{IOException.}}
was:
When I run the UT of org.apache.hadoop.hdfs.qjournal together at IDEA, there
are many failed UT.
I found they has the same reason ,
{code:java}
java.io.IOException: Running in secure mode, but config doesn't have a
keytabjava.io.IOException: Running in secure mode, but config doesn't have a
keytab
at org.apache.hadoop.security.SecurityUtil.login(SecurityUtil.java:308) at
org.apache.hadoop.hdfs.qjournal.server.JournalNode.start(JournalNode.java:230)
at
org.apache.hadoop.hdfs.qjournal.MiniJournalCluster.(MiniJournalCluster.java:121)
at
[jira] [Updated] (HDFS-16154) TestMiniJournalCluster failing intermittently because of not reseting UserGroupInformation completely
[
https://issues.apache.org/jira/browse/HDFS-16154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
wangzhaohui updated HDFS-16154:
---
Description:
When I run the UT of org.apache.hadoop.hdfs.qjournal together at IDEA, there
are many failed UT.
I found they has the same reason ,
{code:java}
java.io.IOException: Running in secure mode, but config doesn't have a
keytabjava.io.IOException: Running in secure mode, but config doesn't have a
keytab
at org.apache.hadoop.security.SecurityUtil.login(SecurityUtil.java:308) at
org.apache.hadoop.hdfs.qjournal.server.JournalNode.start(JournalNode.java:230)
at
org.apache.hadoop.hdfs.qjournal.MiniJournalCluster.(MiniJournalCluster.java:121)
at
org.apache.hadoop.hdfs.qjournal.MiniJournalCluster.(MiniJournalCluster.java:48)
at
org.apache.hadoop.hdfs.qjournal.MiniJournalCluster$Builder.build(MiniJournalCluster.java:80)
at
org.apache.hadoop.hdfs.qjournal.TestMiniJournalCluster.testStartStop(TestMiniJournalCluster.java:38)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498) at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325) at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) at
org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) at
org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288) at
org.junit.runners.ParentRunner.access$000(ParentRunner.java:58) at
org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268) at
org.junit.runners.ParentRunner.run(ParentRunner.java:363) at
org.junit.runner.JUnitCore.run(JUnitCore.java:137) at
org.junit.runner.JUnitCore.run(JUnitCore.java:115) at
org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:40)
at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184) at
java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193) at
java.util.Iterator.forEachRemaining(Iterator.java:116) at
java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482) at
java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472) at
java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
at
java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) at
java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418) at
org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80)
at
org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:71)
at
org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:229)
at
org.junit.platform.launcher.core.DefaultLauncher.lambda$execute$6(DefaultLauncher.java:197)
at
org.junit.platform.launcher.core.DefaultLauncher.withInterceptedStreams(DefaultLauncher.java:211)
at
org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:191)
at
org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:128)
at
com.intellij.junit5.JUnit5IdeaTestRunner.startRunnerWithArgs(JUnit5IdeaTestRunner.java:71)
at
com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:33)
at
com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:220)
at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:53)
{code}
> TestMiniJournalCluster failing intermittently because of not reseting
> UserGroupInformation completely
> -
>
> Key: HDFS-16154
> URL: https://issues.apache.org/jira/browse/HDFS-16154
> Project: Hadoop HDFS
> Issue Type: Improvement
>Reporter: wangzhaohui
>Assignee: wangzhaohui
>Priority: Minor
> Labels: patch-available
> Attachments: HDFS-16154-001.patch
>
>
> When I run the UT of org.apache.hadoop.hdfs.qjournal together at IDEA, there
> are many failed UT.
> I found they has the same
[jira] [Assigned] (HDFS-16154) TestMiniJournalCluster failing intermittently because of not reseting UserGroupInformation completely
[ https://issues.apache.org/jira/browse/HDFS-16154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangzhaohui reassigned HDFS-16154: -- Assignee: wangzhaohui > TestMiniJournalCluster failing intermittently because of not reseting > UserGroupInformation completely > - > > Key: HDFS-16154 > URL: https://issues.apache.org/jira/browse/HDFS-16154 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: wangzhaohui >Assignee: wangzhaohui >Priority: Minor > Labels: patch-available > Attachments: HDFS-16154-001.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-16154) TestMiniJournalCluster failing intermittently because of not reseting UserGroupInformation completely
[ https://issues.apache.org/jira/browse/HDFS-16154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangzhaohui updated HDFS-16154: --- Attachment: HDFS-16154-001.patch > TestMiniJournalCluster failing intermittently because of not reseting > UserGroupInformation completely > - > > Key: HDFS-16154 > URL: https://issues.apache.org/jira/browse/HDFS-16154 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: wangzhaohui >Priority: Minor > Attachments: HDFS-16154-001.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-16154) TestMiniJournalCluster failing intermittently because of not reseting UserGroupInformation completely
[ https://issues.apache.org/jira/browse/HDFS-16154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangzhaohui updated HDFS-16154: --- Labels: patch-available (was: ) > TestMiniJournalCluster failing intermittently because of not reseting > UserGroupInformation completely > - > > Key: HDFS-16154 > URL: https://issues.apache.org/jira/browse/HDFS-16154 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: wangzhaohui >Priority: Minor > Labels: patch-available > Attachments: HDFS-16154-001.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Created] (HDFS-16154) TestMiniJournalCluster failing intermittently because of not reseting UserGroupInformation completely
wangzhaohui created HDFS-16154: -- Summary: TestMiniJournalCluster failing intermittently because of not reseting UserGroupInformation completely Key: HDFS-16154 URL: https://issues.apache.org/jira/browse/HDFS-16154 Project: Hadoop HDFS Issue Type: Improvement Reporter: wangzhaohui -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-16153) Avoid evaluation of LOG.debug statement in QuorumJournalManager
[ https://issues.apache.org/jira/browse/HDFS-16153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17393719#comment-17393719 ] wangzhaohui commented on HDFS-16153: [~ferhui] Thanks for review! I will submit a PR later. > Avoid evaluation of LOG.debug statement in QuorumJournalManager > --- > > Key: HDFS-16153 > URL: https://issues.apache.org/jira/browse/HDFS-16153 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: wangzhaohui >Assignee: wangzhaohui >Priority: Trivial > Labels: patch-available, pull-request-available > Attachments: HDFS-16153-001_patch.patch > > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-16153) Avoid evaluation of LOG.debug statement in QuorumJournalManager
[ https://issues.apache.org/jira/browse/HDFS-16153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangzhaohui updated HDFS-16153: --- Labels: patch-available (was: ) > Avoid evaluation of LOG.debug statement in QuorumJournalManager > --- > > Key: HDFS-16153 > URL: https://issues.apache.org/jira/browse/HDFS-16153 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: wangzhaohui >Assignee: wangzhaohui >Priority: Trivial > Labels: patch-available > Attachments: HDFS-16153-001_patch.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-16153) Avoid evaluation of LOG.debug statement in QuorumJournalManager
[ https://issues.apache.org/jira/browse/HDFS-16153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangzhaohui updated HDFS-16153: --- Attachment: HDFS-16153-001_patch.patch > Avoid evaluation of LOG.debug statement in QuorumJournalManager > --- > > Key: HDFS-16153 > URL: https://issues.apache.org/jira/browse/HDFS-16153 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: wangzhaohui >Assignee: wangzhaohui >Priority: Trivial > Attachments: HDFS-16153-001_patch.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Created] (HDFS-16153) Avoid evaluation of LOG.debug statement in QuorumJournalManager
wangzhaohui created HDFS-16153: -- Summary: Avoid evaluation of LOG.debug statement in QuorumJournalManager Key: HDFS-16153 URL: https://issues.apache.org/jira/browse/HDFS-16153 Project: Hadoop HDFS Issue Type: Improvement Reporter: wangzhaohui Assignee: wangzhaohui -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Issue Comment Deleted] (HDFS-16053) Make the way of get heartbeat interval from conf consistent between Balancer and TestBalancer
[ https://issues.apache.org/jira/browse/HDFS-16053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangzhaohui updated HDFS-16053: --- Comment: was deleted (was: MR: https://github.com/apache/hadoop/pull/3066) > Make the way of get heartbeat interval from conf consistent between Balancer > and TestBalancer > - > > Key: HDFS-16053 > URL: https://issues.apache.org/jira/browse/HDFS-16053 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: wangzhaohui >Assignee: wangzhaohui >Priority: Minor > Labels: pull-request-available > Attachments: HDFS-16053.patch > > Time Spent: 10m > Remaining Estimate: 0h > > In HDFS-14346, add new getTimeDuration, allowing return unit and default unit > to be specified independently. > The way to get the heartbeat interval from conf is modified in Balancer.java, > maybe we can also modify it in TestBalancer.java to keep the two consistent -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-16053) Make the way of get heartbeat interval from conf consistent between Balancer and TestBalancer
[ https://issues.apache.org/jira/browse/HDFS-16053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangzhaohui updated HDFS-16053: --- Status: Patch Available (was: Open) > Make the way of get heartbeat interval from conf consistent between Balancer > and TestBalancer > - > > Key: HDFS-16053 > URL: https://issues.apache.org/jira/browse/HDFS-16053 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: wangzhaohui >Assignee: wangzhaohui >Priority: Minor > Labels: patch-available, pull-request-available > Attachments: HDFS-16053.patch > > Time Spent: 10m > Remaining Estimate: 0h > > In HDFS-14346, add new getTimeDuration, allowing return unit and default unit > to be specified independently. > The way to get the heartbeat interval from conf is modified in Balancer.java, > maybe we can also modify it in TestBalancer.java to keep the two consistent -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-16053) Make the way of get heartbeat interval from conf consistent between Balancer and TestBalancer
[ https://issues.apache.org/jira/browse/HDFS-16053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangzhaohui updated HDFS-16053: --- Labels: pull-request-available (was: patch-available pull-request-available) > Make the way of get heartbeat interval from conf consistent between Balancer > and TestBalancer > - > > Key: HDFS-16053 > URL: https://issues.apache.org/jira/browse/HDFS-16053 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: wangzhaohui >Assignee: wangzhaohui >Priority: Minor > Labels: pull-request-available > Attachments: HDFS-16053.patch > > Time Spent: 10m > Remaining Estimate: 0h > > In HDFS-14346, add new getTimeDuration, allowing return unit and default unit > to be specified independently. > The way to get the heartbeat interval from conf is modified in Balancer.java, > maybe we can also modify it in TestBalancer.java to keep the two consistent -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-16053) Make the way of get heartbeat interval from conf consistent between Balancer and TestBalancer
[ https://issues.apache.org/jira/browse/HDFS-16053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangzhaohui updated HDFS-16053: --- Attachment: HDFS-16053.patch > Make the way of get heartbeat interval from conf consistent between Balancer > and TestBalancer > - > > Key: HDFS-16053 > URL: https://issues.apache.org/jira/browse/HDFS-16053 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: wangzhaohui >Assignee: wangzhaohui >Priority: Minor > Attachments: HDFS-16053.patch > > > In HDFS-14346, add new getTimeDuration, allowing return unit and default unit > to be specified independently. > The way to get the heartbeat interval from conf is modified in Balancer.java, > maybe we can also modify it in TestBalancer.java to keep the two consistent -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-16053) Make the way of get heartbeat interval from conf consistent between Balancer and TestBalancer
[ https://issues.apache.org/jira/browse/HDFS-16053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangzhaohui updated HDFS-16053: --- Labels: patch-available (was: ) > Make the way of get heartbeat interval from conf consistent between Balancer > and TestBalancer > - > > Key: HDFS-16053 > URL: https://issues.apache.org/jira/browse/HDFS-16053 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: wangzhaohui >Assignee: wangzhaohui >Priority: Minor > Labels: patch-available > Attachments: HDFS-16053.patch > > > In HDFS-14346, add new getTimeDuration, allowing return unit and default unit > to be specified independently. > The way to get the heartbeat interval from conf is modified in Balancer.java, > maybe we can also modify it in TestBalancer.java to keep the two consistent -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-16053) Make the way of get heartbeat interval from conf consistent between Balancer and TestBalancer
[ https://issues.apache.org/jira/browse/HDFS-16053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17355596#comment-17355596 ] wangzhaohui commented on HDFS-16053: MR: https://github.com/apache/hadoop/pull/3066 > Make the way of get heartbeat interval from conf consistent between Balancer > and TestBalancer > - > > Key: HDFS-16053 > URL: https://issues.apache.org/jira/browse/HDFS-16053 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: wangzhaohui >Assignee: wangzhaohui >Priority: Minor > > In HDFS-14346, add new getTimeDuration, allowing return unit and default unit > to be specified independently. > The way to get the heartbeat interval from conf is modified in Balancer.java, > maybe we can also modify it in TestBalancer.java to keep the two consistent -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Created] (HDFS-16053) Make the way of get heartbeat interval from conf consistent between Balancer and TestBalancer
wangzhaohui created HDFS-16053: -- Summary: Make the way of get heartbeat interval from conf consistent between Balancer and TestBalancer Key: HDFS-16053 URL: https://issues.apache.org/jira/browse/HDFS-16053 Project: Hadoop HDFS Issue Type: Improvement Reporter: wangzhaohui Assignee: wangzhaohui In HDFS-14346, add new getTimeDuration, allowing return unit and default unit to be specified independently. The way to get the heartbeat interval from conf is modified in Balancer.java, maybe we can also modify it in TestBalancer.java to keep the two consistent -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-15780) RBF: Delete unnecessary var in stateStoreService
[
https://issues.apache.org/jira/browse/HDFS-15780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17267615#comment-17267615
]
wangzhaohui commented on HDFS-15780:
you are right[~elgoiri] ,thanks for review[~ayushtkn] [~hexiaoqiao],i will
close this.
> RBF: Delete unnecessary var in stateStoreService
> -
>
> Key: HDFS-15780
> URL: https://issues.apache.org/jira/browse/HDFS-15780
> Project: Hadoop HDFS
> Issue Type: Improvement
>Reporter: wangzhaohui
>Assignee: wangzhaohui
>Priority: Trivial
> Attachments: HDFS-15780.patch
>
>
> {code:java}
> public void refreshCaches(boolean force) {
> boolean success = true;
> if (isDriverReady()) {
> List cachesToUpdate = new LinkedList<>();
> cachesToUpdate.addAll(cachesToUpdateInternal);
> cachesToUpdate.addAll(cachesToUpdateExternal);
> for (StateStoreCache cachedStore : cachesToUpdate) {
> String cacheName = cachedStore.getClass().getSimpleName();
> boolean result = false;
> try {
> result = cachedStore.loadCache(force);
> } catch (IOException e) {
> LOG.error("Error updating cache for {}", cacheName, e);
> result = false;
> }
> if (!result) {
> success = false;
> LOG.error("Cache update failed for cache {}", cacheName);
> }
> }
> } else {
> success = false;
> LOG.info("Skipping State Store cache update, driver is not ready.");
> }
> if (success) {
> // Uses local time, not driver time.
> this.cacheLastUpdateTime = Time.now();
> }
> }
> {code}
> The "success" may be unnecessary，we can use "result" instead of it, this can
> reduce the use of unnecessary variables.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-15780) RBF: Delete unnecessary var in stateStoreService
[
https://issues.apache.org/jira/browse/HDFS-15780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
wangzhaohui updated HDFS-15780:
---
Issue Type: Improvement (was: Bug)
> RBF: Delete unnecessary var in stateStoreService
> -
>
> Key: HDFS-15780
> URL: https://issues.apache.org/jira/browse/HDFS-15780
> Project: Hadoop HDFS
> Issue Type: Improvement
>Reporter: wangzhaohui
>Priority: Major
> Attachments: HDFS-15780.patch
>
>
> {code:java}
> public void refreshCaches(boolean force) {
> boolean success = true;
> if (isDriverReady()) {
> List cachesToUpdate = new LinkedList<>();
> cachesToUpdate.addAll(cachesToUpdateInternal);
> cachesToUpdate.addAll(cachesToUpdateExternal);
> for (StateStoreCache cachedStore : cachesToUpdate) {
> String cacheName = cachedStore.getClass().getSimpleName();
> boolean result = false;
> try {
> result = cachedStore.loadCache(force);
> } catch (IOException e) {
> LOG.error("Error updating cache for {}", cacheName, e);
> result = false;
> }
> if (!result) {
> success = false;
> LOG.error("Cache update failed for cache {}", cacheName);
> }
> }
> } else {
> success = false;
> LOG.info("Skipping State Store cache update, driver is not ready.");
> }
> if (success) {
> // Uses local time, not driver time.
> this.cacheLastUpdateTime = Time.now();
> }
> }
> {code}
> The "success" may be unnecessary，we can use "result" instead of it, this can
> reduce the use of unnecessary variables.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-15780) RBF: Delete unnecessary var in stateStoreService
[
https://issues.apache.org/jira/browse/HDFS-15780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
wangzhaohui updated HDFS-15780:
---
Attachment: HDFS-15780.patch
> RBF: Delete unnecessary var in stateStoreService
> -
>
> Key: HDFS-15780
> URL: https://issues.apache.org/jira/browse/HDFS-15780
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: wangzhaohui
>Priority: Major
> Attachments: HDFS-15780.patch
>
>
> {code:java}
> public void refreshCaches(boolean force) {
> boolean success = true;
> if (isDriverReady()) {
> List cachesToUpdate = new LinkedList<>();
> cachesToUpdate.addAll(cachesToUpdateInternal);
> cachesToUpdate.addAll(cachesToUpdateExternal);
> for (StateStoreCache cachedStore : cachesToUpdate) {
> String cacheName = cachedStore.getClass().getSimpleName();
> boolean result = false;
> try {
> result = cachedStore.loadCache(force);
> } catch (IOException e) {
> LOG.error("Error updating cache for {}", cacheName, e);
> result = false;
> }
> if (!result) {
> success = false;
> LOG.error("Cache update failed for cache {}", cacheName);
> }
> }
> } else {
> success = false;
> LOG.info("Skipping State Store cache update, driver is not ready.");
> }
> if (success) {
> // Uses local time, not driver time.
> this.cacheLastUpdateTime = Time.now();
> }
> }
> {code}
> The "success" may be unnecessary，we can use "result" instead of it, this can
> reduce the use of unnecessary variables.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Created] (HDFS-15780) RBF: Delete unnecessary var in stateStoreService
wangzhaohui created HDFS-15780:
--
Summary: RBF: Delete unnecessary var in stateStoreService
Key: HDFS-15780
URL: https://issues.apache.org/jira/browse/HDFS-15780
Project: Hadoop HDFS
Issue Type: Bug
Reporter: wangzhaohui
{code:java}
public void refreshCaches(boolean force) {
boolean success = true;
if (isDriverReady()) {
List cachesToUpdate = new LinkedList<>();
cachesToUpdate.addAll(cachesToUpdateInternal);
cachesToUpdate.addAll(cachesToUpdateExternal);
for (StateStoreCache cachedStore : cachesToUpdate) {
String cacheName = cachedStore.getClass().getSimpleName();
boolean result = false;
try {
result = cachedStore.loadCache(force);
} catch (IOException e) {
LOG.error("Error updating cache for {}", cacheName, e);
result = false;
}
if (!result) {
success = false;
LOG.error("Cache update failed for cache {}", cacheName);
}
}
} else {
success = false;
LOG.info("Skipping State Store cache update, driver is not ready.");
}
if (success) {
// Uses local time, not driver time.
this.cacheLastUpdateTime = Time.now();
}
}
{code}
The "success" may be unnecessary，we can use "result" instead of it, this can
reduce the use of unnecessary variables.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-15603) RBF: Fix getLocationsForPath twice in create operation
[ https://issues.apache.org/jira/browse/HDFS-15603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangzhaohui updated HDFS-15603: --- Status: Patch Available (was: Open) > RBF: Fix getLocationsForPath twice in create operation > -- > > Key: HDFS-15603 > URL: https://issues.apache.org/jira/browse/HDFS-15603 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: wangzhaohui >Assignee: wangzhaohui >Priority: Major > Attachments: HDFS-15603-001.patch > > > getLocationsForPath in create(), but getLocationsForPath again in > getCreateLocation(),is not necessary. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-15603) RBF: Fix getLocationsForPath twice in create operation
[ https://issues.apache.org/jira/browse/HDFS-15603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangzhaohui updated HDFS-15603: --- Description: getLocationsForPath in create(), but getLocationsForPath again in getCreateLocation(),is not necessary. (was: getLocationsForPath in create(), but getLocationsForPath again in getCreateLocation(),is not necessary) > RBF: Fix getLocationsForPath twice in create operation > -- > > Key: HDFS-15603 > URL: https://issues.apache.org/jira/browse/HDFS-15603 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: wangzhaohui >Assignee: wangzhaohui >Priority: Major > Attachments: HDFS-15603-001.patch > > > getLocationsForPath in create(), but getLocationsForPath again in > getCreateLocation(),is not necessary. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-15603) RBF: Fix getLocationsForPath twice in create operation
[ https://issues.apache.org/jira/browse/HDFS-15603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangzhaohui updated HDFS-15603: --- Description: getLocationsForPath in create(), but getLocationsForPath again in getCreateLocation(),is not necessary > RBF: Fix getLocationsForPath twice in create operation > -- > > Key: HDFS-15603 > URL: https://issues.apache.org/jira/browse/HDFS-15603 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: wangzhaohui >Assignee: wangzhaohui >Priority: Major > Attachments: HDFS-15603-001.patch > > > getLocationsForPath in create(), but getLocationsForPath again in > getCreateLocation(),is not necessary -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-15603) RBF: Fix getLocationsForPath twice in create operation
[ https://issues.apache.org/jira/browse/HDFS-15603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangzhaohui updated HDFS-15603: --- Attachment: HDFS-15603-001.patch > RBF: Fix getLocationsForPath twice in create operation > -- > > Key: HDFS-15603 > URL: https://issues.apache.org/jira/browse/HDFS-15603 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: wangzhaohui >Assignee: wangzhaohui >Priority: Major > Attachments: HDFS-15603-001.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Created] (HDFS-15603) RBF: Fix getLocationsForPath twice in create operation
wangzhaohui created HDFS-15603: -- Summary: RBF: Fix getLocationsForPath twice in create operation Key: HDFS-15603 URL: https://issues.apache.org/jira/browse/HDFS-15603 Project: Hadoop HDFS Issue Type: Improvement Reporter: wangzhaohui -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Assigned] (HDFS-15603) RBF: Fix getLocationsForPath twice in create operation
[ https://issues.apache.org/jira/browse/HDFS-15603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangzhaohui reassigned HDFS-15603: -- Assignee: wangzhaohui > RBF: Fix getLocationsForPath twice in create operation > -- > > Key: HDFS-15603 > URL: https://issues.apache.org/jira/browse/HDFS-15603 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: wangzhaohui >Assignee: wangzhaohui >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-15591) RBF: Fix webHdfs file display error
[ https://issues.apache.org/jira/browse/HDFS-15591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangzhaohui updated HDFS-15591: --- Attachment: HDFS-15591-004.patch > RBF: Fix webHdfs file display error > --- > > Key: HDFS-15591 > URL: https://issues.apache.org/jira/browse/HDFS-15591 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: wangzhaohui >Assignee: wangzhaohui >Priority: Major > Attachments: HDFS-15591-001.patch, HDFS-15591-002.patch, > HDFS-15591-003.patch, HDFS-15591-004.patch, RBF_Browse_Directory.png, > after-1.jpg, after-2.jpg, before-1.jpg, before-2.jpg > > > The path mounted by the router does not exist on NN，router will create > virtual folder with the mount name, but the "browse the file syaytem" display > on http is wrong. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-15591) RBF: Fix webHdfs file display error
[ https://issues.apache.org/jira/browse/HDFS-15591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17202965#comment-17202965 ] wangzhaohui commented on HDFS-15591: thanks [~ayushtkn] for review. in patch-004, fixed it. > RBF: Fix webHdfs file display error > --- > > Key: HDFS-15591 > URL: https://issues.apache.org/jira/browse/HDFS-15591 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: wangzhaohui >Assignee: wangzhaohui >Priority: Major > Attachments: HDFS-15591-001.patch, HDFS-15591-002.patch, > HDFS-15591-003.patch, HDFS-15591-004.patch, RBF_Browse_Directory.png, > after-1.jpg, after-2.jpg, before-1.jpg, before-2.jpg > > > The path mounted by the router does not exist on NN，router will create > virtual folder with the mount name, but the "browse the file syaytem" display > on http is wrong. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-15591) RBF: Fix webHdfs file display error
[ https://issues.apache.org/jira/browse/HDFS-15591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangzhaohui updated HDFS-15591: --- Attachment: (was: HDFS-15591-003.patch) > RBF: Fix webHdfs file display error > --- > > Key: HDFS-15591 > URL: https://issues.apache.org/jira/browse/HDFS-15591 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: wangzhaohui >Assignee: wangzhaohui >Priority: Major > Attachments: HDFS-15591-001.patch, HDFS-15591-002.patch, > HDFS-15591-003.patch, RBF_Browse_Directory.png, after-1.jpg, after-2.jpg, > before-1.jpg, before-2.jpg > > > The path mounted by the router does not exist on NN，router will create > virtual folder with the mount name, but the "browse the file syaytem" display > on http is wrong. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-15591) RBF: Fix webHdfs file display error
[ https://issues.apache.org/jira/browse/HDFS-15591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangzhaohui updated HDFS-15591: --- Attachment: HDFS-15591-003.patch > RBF: Fix webHdfs file display error > --- > > Key: HDFS-15591 > URL: https://issues.apache.org/jira/browse/HDFS-15591 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: wangzhaohui >Assignee: wangzhaohui >Priority: Major > Attachments: HDFS-15591-001.patch, HDFS-15591-002.patch, > HDFS-15591-003.patch, RBF_Browse_Directory.png, after-1.jpg, after-2.jpg, > before-1.jpg, before-2.jpg > > > The path mounted by the router does not exist on NN，router will create > virtual folder with the mount name, but the "browse the file syaytem" display > on http is wrong. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-15591) RBF: Fix webHdfs file display error
[
https://issues.apache.org/jira/browse/HDFS-15591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17202754#comment-17202754
]
wangzhaohui commented on HDFS-15591:
[~ayushtkn],[~elgoiri]
thanks for review.The web returned the default parameters is slash. When the
virtual path is created for the first time,
{code:java}
Path path = new Path(src, child){code}
the parameters returned to the web are also accompanied by slash,such as /test,
so there will be problems when access the subPath of this virtual path.
src=/test. child=123
New Path(src, child) will return //test123 to web.
In patch-003 I changed the uPath for the getMountStatusPoints returned
> RBF: Fix webHdfs file display error
> ---
>
> Key: HDFS-15591
> URL: https://issues.apache.org/jira/browse/HDFS-15591
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: wangzhaohui
>Assignee: wangzhaohui
>Priority: Major
> Attachments: HDFS-15591-001.patch, HDFS-15591-002.patch,
> HDFS-15591-003.patch, RBF_Browse_Directory.png, after-1.jpg, after-2.jpg,
> before-1.jpg, before-2.jpg
>
>
> The path mounted by the router does not exist on NN，router will create
> virtual folder with the mount name, but the "browse the file syaytem" display
> on http is wrong.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-15591) RBF: Fix webHdfs file display error
[ https://issues.apache.org/jira/browse/HDFS-15591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangzhaohui updated HDFS-15591: --- Attachment: HDFS-15591-003.patch > RBF: Fix webHdfs file display error > --- > > Key: HDFS-15591 > URL: https://issues.apache.org/jira/browse/HDFS-15591 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: wangzhaohui >Assignee: wangzhaohui >Priority: Major > Attachments: HDFS-15591-001.patch, HDFS-15591-002.patch, > HDFS-15591-003.patch, RBF_Browse_Directory.png, after-1.jpg, after-2.jpg, > before-1.jpg, before-2.jpg > > > The path mounted by the router does not exist on NN，router will create > virtual folder with the mount name, but the "browse the file syaytem" display > on http is wrong. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-15591) RBF: Fix webHdfs file display error
[ https://issues.apache.org/jira/browse/HDFS-15591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangzhaohui updated HDFS-15591: --- Attachment: HDFS-15591-002.patch > RBF: Fix webHdfs file display error > --- > > Key: HDFS-15591 > URL: https://issues.apache.org/jira/browse/HDFS-15591 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: wangzhaohui >Assignee: wangzhaohui >Priority: Major > Attachments: HDFS-15591-001.patch, HDFS-15591-002.patch, after-1.jpg, > after-2.jpg, before-1.jpg, before-2.jpg > > > The path mounted by the router does not exist on NN，router will create > virtual folder with the mount name, but the "browse the file syaytem" display > on http is wrong. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-15591) RBF: Fix webHdfs file display error
[ https://issues.apache.org/jira/browse/HDFS-15591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangzhaohui updated HDFS-15591: --- Attachment: (was: HDFS-15591-002.patch) > RBF: Fix webHdfs file display error > --- > > Key: HDFS-15591 > URL: https://issues.apache.org/jira/browse/HDFS-15591 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: wangzhaohui >Assignee: wangzhaohui >Priority: Major > Attachments: HDFS-15591-001.patch, after-1.jpg, after-2.jpg, > before-1.jpg, before-2.jpg > > > The path mounted by the router does not exist on NN，router will create > virtual folder with the mount name, but the "browse the file syaytem" display > on http is wrong. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-15591) RBF: Fix webHdfs file display error
[ https://issues.apache.org/jira/browse/HDFS-15591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangzhaohui updated HDFS-15591: --- Attachment: HDFS-15591-002.patch > RBF: Fix webHdfs file display error > --- > > Key: HDFS-15591 > URL: https://issues.apache.org/jira/browse/HDFS-15591 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: wangzhaohui >Assignee: wangzhaohui >Priority: Major > Attachments: HDFS-15591-001.patch, HDFS-15591-002.patch, after-1.jpg, > after-2.jpg, before-1.jpg, before-2.jpg > > > The path mounted by the router does not exist on NN，router will create > virtual folder with the mount name, but the "browse the file syaytem" display > on http is wrong. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work started] (HDFS-15591) RBF: Fix webHdfs file display error
[ https://issues.apache.org/jira/browse/HDFS-15591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HDFS-15591 started by wangzhaohui. -- > RBF: Fix webHdfs file display error > --- > > Key: HDFS-15591 > URL: https://issues.apache.org/jira/browse/HDFS-15591 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: wangzhaohui >Assignee: wangzhaohui >Priority: Major > Attachments: HDFS-15591-001.patch, after-1.jpg, after-2.jpg, > before-1.jpg, before-2.jpg > > > The path mounted by the router does not exist on NN，router will create > virtual folder with the mount name, but the "browse the file syaytem" display > on http is wrong. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-15591) RBF: Fix webHdfs file display error
[ https://issues.apache.org/jira/browse/HDFS-15591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangzhaohui updated HDFS-15591: --- Status: Open (was: Patch Available) > RBF: Fix webHdfs file display error > --- > > Key: HDFS-15591 > URL: https://issues.apache.org/jira/browse/HDFS-15591 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: wangzhaohui >Assignee: wangzhaohui >Priority: Major > Attachments: HDFS-15591-001.patch, after-1.jpg, after-2.jpg, > before-1.jpg, before-2.jpg > > > The path mounted by the router does not exist on NN，router will create > virtual folder with the mount name, but the "browse the file syaytem" display > on http is wrong. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-15591) RBF: Fix webHdfs file display error
[ https://issues.apache.org/jira/browse/HDFS-15591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangzhaohui updated HDFS-15591: --- Status: Patch Available (was: In Progress) > RBF: Fix webHdfs file display error > --- > > Key: HDFS-15591 > URL: https://issues.apache.org/jira/browse/HDFS-15591 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: wangzhaohui >Assignee: wangzhaohui >Priority: Major > Attachments: HDFS-15591-001.patch, after-1.jpg, after-2.jpg, > before-1.jpg, before-2.jpg > > > The path mounted by the router does not exist on NN，router will create > virtual folder with the mount name, but the "browse the file syaytem" display > on http is wrong. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-15591) RBF: Fix webHdfs file display error
[ https://issues.apache.org/jira/browse/HDFS-15591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangzhaohui updated HDFS-15591: --- Status: Patch Available (was: In Progress) > RBF: Fix webHdfs file display error > --- > > Key: HDFS-15591 > URL: https://issues.apache.org/jira/browse/HDFS-15591 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: wangzhaohui >Assignee: wangzhaohui >Priority: Major > Attachments: HDFS-15591-001.patch, after-1.jpg, after-2.jpg, > before-1.jpg, before-2.jpg > > > The path mounted by the router does not exist on NN，router will create > virtual folder with the mount name, but the "browse the file syaytem" display > on http is wrong. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work started] (HDFS-15591) RBF: Fix webHdfs file display error
[ https://issues.apache.org/jira/browse/HDFS-15591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HDFS-15591 started by wangzhaohui. -- > RBF: Fix webHdfs file display error > --- > > Key: HDFS-15591 > URL: https://issues.apache.org/jira/browse/HDFS-15591 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: wangzhaohui >Assignee: wangzhaohui >Priority: Major > Attachments: HDFS-15591-001.patch, after-1.jpg, after-2.jpg, > before-1.jpg, before-2.jpg > > > The path mounted by the router does not exist on NN，router will create > virtual folder with the mount name, but the "browse the file syaytem" display > on http is wrong. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-15591) RBF: Fix webHdfs file display error
[ https://issues.apache.org/jira/browse/HDFS-15591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangzhaohui updated HDFS-15591: --- Attachment: before-2.jpg > RBF: Fix webHdfs file display error > --- > > Key: HDFS-15591 > URL: https://issues.apache.org/jira/browse/HDFS-15591 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: wangzhaohui >Assignee: wangzhaohui >Priority: Major > Attachments: HDFS-15591-001.patch, after-1.jpg, after-2.jpg, > before-1.jpg, before-2.jpg > > > The path mounted by the router does not exist on NN，router will create > virtual folder with the mount name, but the "browse the file syaytem" display > on http is wrong. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-15591) RBF: Fix webHdfs file display error
[ https://issues.apache.org/jira/browse/HDFS-15591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangzhaohui updated HDFS-15591: --- Attachment: after-2.jpg > RBF: Fix webHdfs file display error > --- > > Key: HDFS-15591 > URL: https://issues.apache.org/jira/browse/HDFS-15591 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: wangzhaohui >Assignee: wangzhaohui >Priority: Major > Attachments: HDFS-15591-001.patch, after-1.jpg, after-2.jpg, > before-1.jpg, before-2.jpg > > > The path mounted by the router does not exist on NN，router will create > virtual folder with the mount name, but the "browse the file syaytem" display > on http is wrong. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-15591) RBF: Fix webHdfs file display error
[ https://issues.apache.org/jira/browse/HDFS-15591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangzhaohui updated HDFS-15591: --- Attachment: HDFS-15591-001.patch > RBF: Fix webHdfs file display error > --- > > Key: HDFS-15591 > URL: https://issues.apache.org/jira/browse/HDFS-15591 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: wangzhaohui >Assignee: wangzhaohui >Priority: Major > Attachments: HDFS-15591-001.patch, after-1.jpg, after-2.jpg, > before-1.jpg, before-2.jpg > > > The path mounted by the router does not exist on NN，router will create > virtual folder with the mount name, but the "browse the file syaytem" display > on http is wrong. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-15591) RBF: Fix webHdfs file display error
[ https://issues.apache.org/jira/browse/HDFS-15591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangzhaohui updated HDFS-15591: --- Attachment: before-1.jpg > RBF: Fix webHdfs file display error > --- > > Key: HDFS-15591 > URL: https://issues.apache.org/jira/browse/HDFS-15591 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: wangzhaohui >Assignee: wangzhaohui >Priority: Major > Attachments: HDFS-15591-001.patch, after-1.jpg, after-2.jpg, > before-1.jpg, before-2.jpg > > > The path mounted by the router does not exist on NN，router will create > virtual folder with the mount name, but the "browse the file syaytem" display > on http is wrong. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-15591) RBF: Fix webHdfs file display error
[ https://issues.apache.org/jira/browse/HDFS-15591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangzhaohui updated HDFS-15591: --- Attachment: after-1.jpg > RBF: Fix webHdfs file display error > --- > > Key: HDFS-15591 > URL: https://issues.apache.org/jira/browse/HDFS-15591 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: wangzhaohui >Assignee: wangzhaohui >Priority: Major > Attachments: HDFS-15591-001.patch, after-1.jpg, after-2.jpg, > before-1.jpg, before-2.jpg > > > The path mounted by the router does not exist on NN，router will create > virtual folder with the mount name, but the "browse the file syaytem" display > on http is wrong. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Created] (HDFS-15591) RBF: Fix webHdfs file display error
wangzhaohui created HDFS-15591: -- Summary: RBF: Fix webHdfs file display error Key: HDFS-15591 URL: https://issues.apache.org/jira/browse/HDFS-15591 Project: Hadoop HDFS Issue Type: Bug Reporter: wangzhaohui Assignee: wangzhaohui The path mounted by the router does not exist on NN，router will create virtual folder with the mount name, but the "browse the file syaytem" display on http is wrong. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-14744) RBF: Non secured routers should not log in error mode when UGI is default.
[
https://issues.apache.org/jira/browse/HDFS-14744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
wangzhaohui updated HDFS-14744:
---
Summary: RBF: Non secured routers should not log in error mode when UGI is
default. (was: cc)
> RBF: Non secured routers should not log in error mode when UGI is default.
> --
>
> Key: HDFS-14744
> URL: https://issues.apache.org/jira/browse/HDFS-14744
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: CR Hota
>Assignee: CR Hota
>Priority: Major
> Fix For: 3.3.0
>
> Attachments: HDFS-14744.001.patch
>
>
> RouterClientProtocol#getMountPointStatus logs error when groups are not found
> for default web user dr.who. The line should be logged in "error" mode for
> secured cluster, for unsecured clusters, we may want to just specify "debug"
> or else logs are filled up with this non-critical line
> {{ERROR org.apache.hadoop.hdfs.server.federation.router.RouterRpcServer:
> Cannot get the remote user: There is no primary group for UGI dr.who
> (auth:SIMPLE)}}
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-14744) cc
[
https://issues.apache.org/jira/browse/HDFS-14744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
wangzhaohui updated HDFS-14744:
---
Summary: cc (was: RBF: Non secured routers should not log in error mode
when UGI is default.)
> cc
> --
>
> Key: HDFS-14744
> URL: https://issues.apache.org/jira/browse/HDFS-14744
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: CR Hota
>Assignee: CR Hota
>Priority: Major
> Fix For: 3.3.0
>
> Attachments: HDFS-14744.001.patch
>
>
> RouterClientProtocol#getMountPointStatus logs error when groups are not found
> for default web user dr.who. The line should be logged in "error" mode for
> secured cluster, for unsecured clusters, we may want to just specify "debug"
> or else logs are filled up with this non-critical line
> {{ERROR org.apache.hadoop.hdfs.server.federation.router.RouterRpcServer:
> Cannot get the remote user: There is no primary group for UGI dr.who
> (auth:SIMPLE)}}
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Assigned] (HDFS-15425) Review Logging of DFSClient
[ https://issues.apache.org/jira/browse/HDFS-15425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangzhaohui reassigned HDFS-15425: -- Assignee: (was: Hongbing Wang) > Review Logging of DFSClient > --- > > Key: HDFS-15425 > URL: https://issues.apache.org/jira/browse/HDFS-15425 > Project: Hadoop HDFS > Issue Type: Improvement > Components: dfsclient >Reporter: Hongbing Wang >Priority: Minor > Fix For: 3.4.0 > > Attachments: HDFS-15425.001.patch, HDFS-15425.002.patch, > HDFS-15425.003.patch > > > Review use of SLF4J for DFSClient.LOG. > Make the code more concise and readable. > Less is more ! -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-14674) [SBN read] Got an unexpected txid when tail editlog
[
https://issues.apache.org/jira/browse/HDFS-14674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
wangzhaohui updated HDFS-14674:
---
Attachment: image-2019-08-22-16-24-06-518.png
> [SBN read] Got an unexpected txid when tail editlog
> ---
>
> Key: HDFS-14674
> URL: https://issues.apache.org/jira/browse/HDFS-14674
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: wangzhaohui
>Assignee: wangzhaohui
>Priority: Blocker
> Attachments: HDFS-14674-001.patch, HDFS-14674-003.patch,
> HDFS-14674-004.patch, HDFS-14674-005.patch, HDFS-14674-006.patch,
> HDFS-14674-007.patch, HDFS-14674-008.patch, HDFS-14674-009.patch,
> HDFS-14674-010.patch, HDFS-14674-011.patch,
> image-2019-08-22-16-24-06-518.png, image.png
>
>
> Add the following configuration
> !image-2019-07-26-11-34-23-405.png!
> error:
> {code:java}
> //
> [2019-07-17T11:50:21.048+08:00] [INFO] [Edit log tailer] : replaying edit
> log: 1/20512836 transactions completed. (0%) [2019-07-17T11:50:21.059+08:00]
> [INFO] [Edit log tailer] : Edits file
> http://ip/getJournal?jid=ns1003=232056426162=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?ipjid=ns1003=232056426162=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?ipjid=ns1003=232056426162=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH
> of size 3126782311 edits # 500 loaded in 3 seconds
> [2019-07-17T11:50:21.059+08:00] [INFO] [Edit log tailer] : Reading
> org.apache.hadoop.hdfs.server.namenode.RedundantEditLogInputStream@51ceb7bc
> expecting start txid #232056752162 [2019-07-17T11:50:21.059+08:00] [INFO]
> [Edit log tailer] : Start loading edits file
> http://ip/getJournal?ipjid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?ipjid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?ipjid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH
> maxTxnipsToRead = 500 [2019-07-17T11:50:21.059+08:00] [INFO] [Edit log
> tailer] : Fast-forwarding stream
> 'http://ip/getJournal?jid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?ipjid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?ipjid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH'
> to transaction ID 232056751662 [2019-07-17T11:50:21.059+08:00] [INFO] [Edit
> log tailer] ip: Fast-forwarding stream
> 'http://ip/getJournal?jid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH'
> to transaction ID 232056751662 [2019-07-17T11:50:21.061+08:00] [ERROR] [Edit
> log tailer] : Unknown error encountered while tailing edits. Shutting down
> standby NN. java.io.IOException: There appears to be a gap in the edit log.
> We expected txid 232056752162, but got txid 232077264498. at
> org.apache.hadoop.hdfs.server.namenode.MetaRecoveryContext.editLogLoaderPrompt(MetaRecoveryContext.java:94)
> at
> org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadEditRecords(FSEditLogLoader.java:239)
> at
> org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadFSEdits(FSEditLogLoader.java:161)
> at
> org.apache.hadoop.hdfs.server.namenode.FSImage.loadEdits(FSImage.java:895) at
> org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer.doTailEdits(EditLogTailer.java:321)
> at
> org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread.doWork(EditLogTailer.java:460)
> at
> org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread.access$400(EditLogTailer.java:410)
> at
> org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread$1.run(EditLogTailer.java:427)
> at
> org.apache.hadoop.security.SecurityUtil.doAsLoginUserOrFatal(SecurityUtil.java:414)
> at
> org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread.run(EditLogTailer.java:423)
> [2019-07-17T11:50:21.064+08:00] [INFO] [Edit log tailer] : Exiting with
> status 1 [2019-07-17T11:50:21.066+08:00] [INFO] [Thread-1] : SHUTDOWN_MSG:
> / SHUTDOWN_MSG:
> Shutting down NameNode at ip
> /
> {code}
>
> if dfs.ha.tail-edits.max-txns-per-lock value is 500,when the namenode load
> the editlog util 500,the current namenode will load the next editlog,but
> editlog more than 500.So,namenode got an unexpected txid when tail editlog.
>
>
> {code:java}
> //
> [2019-07-17T11:50:21.059+08:00] [INFO] [Edit log tailer] : Edits file
> http://ip/getJournal?jid=ns1003=232056426162=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
[jira] [Updated] (HDFS-14674) [SBN read] Got an unexpected txid when tail editlog
[
https://issues.apache.org/jira/browse/HDFS-14674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
wangzhaohui updated HDFS-14674:
---
Description:
Add the following configuration
!image-2019-08-22-16-24-06-518.png|width=451,height=80!
error:
{code:java}
//
[2019-07-17T11:50:21.048+08:00] [INFO] [Edit log tailer] : replaying edit log:
1/20512836 transactions completed. (0%) [2019-07-17T11:50:21.059+08:00] [INFO]
[Edit log tailer] : Edits file
http://ip/getJournal?jid=ns1003=232056426162=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
http://ip/getJournal?ipjid=ns1003=232056426162=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
http://ip/getJournal?ipjid=ns1003=232056426162=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH
of size 3126782311 edits # 500 loaded in 3 seconds
[2019-07-17T11:50:21.059+08:00] [INFO] [Edit log tailer] : Reading
org.apache.hadoop.hdfs.server.namenode.RedundantEditLogInputStream@51ceb7bc
expecting start txid #232056752162 [2019-07-17T11:50:21.059+08:00] [INFO] [Edit
log tailer] : Start loading edits file
http://ip/getJournal?ipjid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
http://ip/getJournal?ipjid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
http://ip/getJournal?ipjid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH
maxTxnipsToRead = 500 [2019-07-17T11:50:21.059+08:00] [INFO] [Edit log tailer]
: Fast-forwarding stream
'http://ip/getJournal?jid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
http://ip/getJournal?ipjid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
http://ip/getJournal?ipjid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH'
to transaction ID 232056751662 [2019-07-17T11:50:21.059+08:00] [INFO] [Edit
log tailer] ip: Fast-forwarding stream
'http://ip/getJournal?jid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH'
to transaction ID 232056751662 [2019-07-17T11:50:21.061+08:00] [ERROR] [Edit
log tailer] : Unknown error encountered while tailing edits. Shutting down
standby NN. java.io.IOException: There appears to be a gap in the edit log. We
expected txid 232056752162, but got txid 232077264498. at
org.apache.hadoop.hdfs.server.namenode.MetaRecoveryContext.editLogLoaderPrompt(MetaRecoveryContext.java:94)
at
org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadEditRecords(FSEditLogLoader.java:239)
at
org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadFSEdits(FSEditLogLoader.java:161)
at org.apache.hadoop.hdfs.server.namenode.FSImage.loadEdits(FSImage.java:895)
at
org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer.doTailEdits(EditLogTailer.java:321)
at
org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread.doWork(EditLogTailer.java:460)
at
org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread.access$400(EditLogTailer.java:410)
at
org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread$1.run(EditLogTailer.java:427)
at
org.apache.hadoop.security.SecurityUtil.doAsLoginUserOrFatal(SecurityUtil.java:414)
at
org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread.run(EditLogTailer.java:423)
[2019-07-17T11:50:21.064+08:00] [INFO] [Edit log tailer] : Exiting with status
1 [2019-07-17T11:50:21.066+08:00] [INFO] [Thread-1] : SHUTDOWN_MSG:
/ SHUTDOWN_MSG:
Shutting down NameNode at ip
/
{code}
if dfs.ha.tail-edits.max-txns-per-lock value is 500,when the namenode load the
editlog util 500,the current namenode will load the next editlog,but editlog
more than 500.So,namenode got an unexpected txid when tail editlog.
{code:java}
//
[2019-07-17T11:50:21.059+08:00] [INFO] [Edit log tailer] : Edits file
http://ip/getJournal?jid=ns1003=232056426162=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
http://ip/getJournal?jid=ns1003=232056426162=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
http://ip/getJournal?jid=ns1003=232056426162=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH
of size 3126782311 edits # 500 loaded in 3 seconds
[2019-07-17T11:50:21.059+08:00] [INFO] [Edit log tailer] : Reading
org.apache.hadoop.hdfs.server.namenode.RedundantEditLogInputStream@51ceb7bc
expecting start txid #232056752162
[2019-07-17T11:50:21.059+08:00] [INFO] [Edit log tailer] : Start loading edits
file
http://ip/getJournal?jid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
http://ip/getJournal?jid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
http://ip/getJournal?jid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH
[jira] [Commented] (HDFS-14674) [SBN read] Got an unexpected txid when tail editlog
[
https://issues.apache.org/jira/browse/HDFS-14674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16910902#comment-16910902
]
wangzhaohui commented on HDFS-14674:
[~shv] thanks,i have fixed the unit test,the wrong order imports may be due
to the use of shortcuts in writing unit test,in short,thanks for your review.
> [SBN read] Got an unexpected txid when tail editlog
> ---
>
> Key: HDFS-14674
> URL: https://issues.apache.org/jira/browse/HDFS-14674
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: wangzhaohui
>Assignee: wangzhaohui
>Priority: Blocker
> Attachments: HDFS-14674-001.patch, HDFS-14674-003.patch,
> HDFS-14674-004.patch, HDFS-14674-005.patch, HDFS-14674-006.patch,
> HDFS-14674-007.patch, HDFS-14674-008.patch, HDFS-14674-009.patch,
> HDFS-14674-010.patch, HDFS-14674-011.patch, image.png
>
>
> Add the following configuration
> !image-2019-07-26-11-34-23-405.png!
> error:
> {code:java}
> //
> [2019-07-17T11:50:21.048+08:00] [INFO] [Edit log tailer] : replaying edit
> log: 1/20512836 transactions completed. (0%) [2019-07-17T11:50:21.059+08:00]
> [INFO] [Edit log tailer] : Edits file
> http://ip/getJournal?jid=ns1003=232056426162=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?ipjid=ns1003=232056426162=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?ipjid=ns1003=232056426162=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH
> of size 3126782311 edits # 500 loaded in 3 seconds
> [2019-07-17T11:50:21.059+08:00] [INFO] [Edit log tailer] : Reading
> org.apache.hadoop.hdfs.server.namenode.RedundantEditLogInputStream@51ceb7bc
> expecting start txid #232056752162 [2019-07-17T11:50:21.059+08:00] [INFO]
> [Edit log tailer] : Start loading edits file
> http://ip/getJournal?ipjid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?ipjid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?ipjid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH
> maxTxnipsToRead = 500 [2019-07-17T11:50:21.059+08:00] [INFO] [Edit log
> tailer] : Fast-forwarding stream
> 'http://ip/getJournal?jid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?ipjid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?ipjid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH'
> to transaction ID 232056751662 [2019-07-17T11:50:21.059+08:00] [INFO] [Edit
> log tailer] ip: Fast-forwarding stream
> 'http://ip/getJournal?jid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH'
> to transaction ID 232056751662 [2019-07-17T11:50:21.061+08:00] [ERROR] [Edit
> log tailer] : Unknown error encountered while tailing edits. Shutting down
> standby NN. java.io.IOException: There appears to be a gap in the edit log.
> We expected txid 232056752162, but got txid 232077264498. at
> org.apache.hadoop.hdfs.server.namenode.MetaRecoveryContext.editLogLoaderPrompt(MetaRecoveryContext.java:94)
> at
> org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadEditRecords(FSEditLogLoader.java:239)
> at
> org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadFSEdits(FSEditLogLoader.java:161)
> at
> org.apache.hadoop.hdfs.server.namenode.FSImage.loadEdits(FSImage.java:895) at
> org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer.doTailEdits(EditLogTailer.java:321)
> at
> org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread.doWork(EditLogTailer.java:460)
> at
> org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread.access$400(EditLogTailer.java:410)
> at
> org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread$1.run(EditLogTailer.java:427)
> at
> org.apache.hadoop.security.SecurityUtil.doAsLoginUserOrFatal(SecurityUtil.java:414)
> at
> org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread.run(EditLogTailer.java:423)
> [2019-07-17T11:50:21.064+08:00] [INFO] [Edit log tailer] : Exiting with
> status 1 [2019-07-17T11:50:21.066+08:00] [INFO] [Thread-1] : SHUTDOWN_MSG:
> / SHUTDOWN_MSG:
> Shutting down NameNode at ip
> /
> {code}
>
> if dfs.ha.tail-edits.max-txns-per-lock value is 500,when the namenode load
> the editlog util 500,the current namenode will load the next editlog,but
> editlog more than 500.So,namenode got an unexpected txid when tail editlog.
>
>
> {code:java}
> //
> [2019-07-17T11:50:21.059+08:00] [INFO] [Edit log tailer] : Edits
[jira] [Updated] (HDFS-14674) [SBN read] Got an unexpected txid when tail editlog
[
https://issues.apache.org/jira/browse/HDFS-14674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
wangzhaohui updated HDFS-14674:
---
Attachment: HDFS-14674-011.patch
> [SBN read] Got an unexpected txid when tail editlog
> ---
>
> Key: HDFS-14674
> URL: https://issues.apache.org/jira/browse/HDFS-14674
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: wangzhaohui
>Assignee: wangzhaohui
>Priority: Blocker
> Attachments: HDFS-14674-001.patch, HDFS-14674-003.patch,
> HDFS-14674-004.patch, HDFS-14674-005.patch, HDFS-14674-006.patch,
> HDFS-14674-007.patch, HDFS-14674-008.patch, HDFS-14674-009.patch,
> HDFS-14674-010.patch, HDFS-14674-011.patch, image.png
>
>
> Add the following configuration
> !image-2019-07-26-11-34-23-405.png!
> error:
> {code:java}
> //
> [2019-07-17T11:50:21.048+08:00] [INFO] [Edit log tailer] : replaying edit
> log: 1/20512836 transactions completed. (0%) [2019-07-17T11:50:21.059+08:00]
> [INFO] [Edit log tailer] : Edits file
> http://ip/getJournal?jid=ns1003=232056426162=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?ipjid=ns1003=232056426162=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?ipjid=ns1003=232056426162=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH
> of size 3126782311 edits # 500 loaded in 3 seconds
> [2019-07-17T11:50:21.059+08:00] [INFO] [Edit log tailer] : Reading
> org.apache.hadoop.hdfs.server.namenode.RedundantEditLogInputStream@51ceb7bc
> expecting start txid #232056752162 [2019-07-17T11:50:21.059+08:00] [INFO]
> [Edit log tailer] : Start loading edits file
> http://ip/getJournal?ipjid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?ipjid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?ipjid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH
> maxTxnipsToRead = 500 [2019-07-17T11:50:21.059+08:00] [INFO] [Edit log
> tailer] : Fast-forwarding stream
> 'http://ip/getJournal?jid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?ipjid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?ipjid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH'
> to transaction ID 232056751662 [2019-07-17T11:50:21.059+08:00] [INFO] [Edit
> log tailer] ip: Fast-forwarding stream
> 'http://ip/getJournal?jid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH'
> to transaction ID 232056751662 [2019-07-17T11:50:21.061+08:00] [ERROR] [Edit
> log tailer] : Unknown error encountered while tailing edits. Shutting down
> standby NN. java.io.IOException: There appears to be a gap in the edit log.
> We expected txid 232056752162, but got txid 232077264498. at
> org.apache.hadoop.hdfs.server.namenode.MetaRecoveryContext.editLogLoaderPrompt(MetaRecoveryContext.java:94)
> at
> org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadEditRecords(FSEditLogLoader.java:239)
> at
> org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadFSEdits(FSEditLogLoader.java:161)
> at
> org.apache.hadoop.hdfs.server.namenode.FSImage.loadEdits(FSImage.java:895) at
> org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer.doTailEdits(EditLogTailer.java:321)
> at
> org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread.doWork(EditLogTailer.java:460)
> at
> org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread.access$400(EditLogTailer.java:410)
> at
> org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread$1.run(EditLogTailer.java:427)
> at
> org.apache.hadoop.security.SecurityUtil.doAsLoginUserOrFatal(SecurityUtil.java:414)
> at
> org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread.run(EditLogTailer.java:423)
> [2019-07-17T11:50:21.064+08:00] [INFO] [Edit log tailer] : Exiting with
> status 1 [2019-07-17T11:50:21.066+08:00] [INFO] [Thread-1] : SHUTDOWN_MSG:
> / SHUTDOWN_MSG:
> Shutting down NameNode at ip
> /
> {code}
>
> if dfs.ha.tail-edits.max-txns-per-lock value is 500,when the namenode load
> the editlog util 500,the current namenode will load the next editlog,but
> editlog more than 500.So,namenode got an unexpected txid when tail editlog.
>
>
> {code:java}
> //
> [2019-07-17T11:50:21.059+08:00] [INFO] [Edit log tailer] : Edits file
> http://ip/getJournal?jid=ns1003=232056426162=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
>
[jira] [Commented] (HDFS-14674) [SBN read] Got an unexpected txid when tail editlog
[
https://issues.apache.org/jira/browse/HDFS-14674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16909026#comment-16909026
]
wangzhaohui commented on HDFS-14674:
[~shv] [~csun] fixed the unit test,pls review.
> [SBN read] Got an unexpected txid when tail editlog
> ---
>
> Key: HDFS-14674
> URL: https://issues.apache.org/jira/browse/HDFS-14674
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: wangzhaohui
>Assignee: wangzhaohui
>Priority: Blocker
> Attachments: HDFS-14674-001.patch, HDFS-14674-003.patch,
> HDFS-14674-004.patch, HDFS-14674-005.patch, HDFS-14674-006.patch,
> HDFS-14674-007.patch, HDFS-14674-008.patch, HDFS-14674-009.patch,
> HDFS-14674-010.patch, image.png
>
>
> Add the following configuration
> !image-2019-07-26-11-34-23-405.png!
> error:
> {code:java}
> //
> [2019-07-17T11:50:21.048+08:00] [INFO] [Edit log tailer] : replaying edit
> log: 1/20512836 transactions completed. (0%) [2019-07-17T11:50:21.059+08:00]
> [INFO] [Edit log tailer] : Edits file
> http://ip/getJournal?jid=ns1003=232056426162=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?ipjid=ns1003=232056426162=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?ipjid=ns1003=232056426162=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH
> of size 3126782311 edits # 500 loaded in 3 seconds
> [2019-07-17T11:50:21.059+08:00] [INFO] [Edit log tailer] : Reading
> org.apache.hadoop.hdfs.server.namenode.RedundantEditLogInputStream@51ceb7bc
> expecting start txid #232056752162 [2019-07-17T11:50:21.059+08:00] [INFO]
> [Edit log tailer] : Start loading edits file
> http://ip/getJournal?ipjid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?ipjid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?ipjid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH
> maxTxnipsToRead = 500 [2019-07-17T11:50:21.059+08:00] [INFO] [Edit log
> tailer] : Fast-forwarding stream
> 'http://ip/getJournal?jid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?ipjid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?ipjid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH'
> to transaction ID 232056751662 [2019-07-17T11:50:21.059+08:00] [INFO] [Edit
> log tailer] ip: Fast-forwarding stream
> 'http://ip/getJournal?jid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH'
> to transaction ID 232056751662 [2019-07-17T11:50:21.061+08:00] [ERROR] [Edit
> log tailer] : Unknown error encountered while tailing edits. Shutting down
> standby NN. java.io.IOException: There appears to be a gap in the edit log.
> We expected txid 232056752162, but got txid 232077264498. at
> org.apache.hadoop.hdfs.server.namenode.MetaRecoveryContext.editLogLoaderPrompt(MetaRecoveryContext.java:94)
> at
> org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadEditRecords(FSEditLogLoader.java:239)
> at
> org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadFSEdits(FSEditLogLoader.java:161)
> at
> org.apache.hadoop.hdfs.server.namenode.FSImage.loadEdits(FSImage.java:895) at
> org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer.doTailEdits(EditLogTailer.java:321)
> at
> org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread.doWork(EditLogTailer.java:460)
> at
> org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread.access$400(EditLogTailer.java:410)
> at
> org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread$1.run(EditLogTailer.java:427)
> at
> org.apache.hadoop.security.SecurityUtil.doAsLoginUserOrFatal(SecurityUtil.java:414)
> at
> org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread.run(EditLogTailer.java:423)
> [2019-07-17T11:50:21.064+08:00] [INFO] [Edit log tailer] : Exiting with
> status 1 [2019-07-17T11:50:21.066+08:00] [INFO] [Thread-1] : SHUTDOWN_MSG:
> / SHUTDOWN_MSG:
> Shutting down NameNode at ip
> /
> {code}
>
> if dfs.ha.tail-edits.max-txns-per-lock value is 500,when the namenode load
> the editlog util 500,the current namenode will load the next editlog,but
> editlog more than 500.So,namenode got an unexpected txid when tail editlog.
>
>
> {code:java}
> //
> [2019-07-17T11:50:21.059+08:00] [INFO] [Edit log tailer] : Edits file
> http://ip/getJournal?jid=ns1003=232056426162=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
>
[jira] [Commented] (HDFS-14674) [SBN read] Got an unexpected txid when tail editlog
[
https://issues.apache.org/jira/browse/HDFS-14674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16908722#comment-16908722
]
wangzhaohui commented on HDFS-14674:
[~shv] ok,i will fix the unit test,thanks for your review.
> [SBN read] Got an unexpected txid when tail editlog
> ---
>
> Key: HDFS-14674
> URL: https://issues.apache.org/jira/browse/HDFS-14674
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: wangzhaohui
>Assignee: wangzhaohui
>Priority: Blocker
> Attachments: HDFS-14674-001.patch, HDFS-14674-003.patch,
> HDFS-14674-004.patch, HDFS-14674-005.patch, HDFS-14674-006.patch,
> HDFS-14674-007.patch, HDFS-14674-008.patch, HDFS-14674-009.patch,
> HDFS-14674-010.patch, image.png
>
>
> Add the following configuration
> !image-2019-07-26-11-34-23-405.png!
> error:
> {code:java}
> //
> [2019-07-17T11:50:21.048+08:00] [INFO] [Edit log tailer] : replaying edit
> log: 1/20512836 transactions completed. (0%) [2019-07-17T11:50:21.059+08:00]
> [INFO] [Edit log tailer] : Edits file
> http://ip/getJournal?jid=ns1003=232056426162=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?ipjid=ns1003=232056426162=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?ipjid=ns1003=232056426162=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH
> of size 3126782311 edits # 500 loaded in 3 seconds
> [2019-07-17T11:50:21.059+08:00] [INFO] [Edit log tailer] : Reading
> org.apache.hadoop.hdfs.server.namenode.RedundantEditLogInputStream@51ceb7bc
> expecting start txid #232056752162 [2019-07-17T11:50:21.059+08:00] [INFO]
> [Edit log tailer] : Start loading edits file
> http://ip/getJournal?ipjid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?ipjid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?ipjid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH
> maxTxnipsToRead = 500 [2019-07-17T11:50:21.059+08:00] [INFO] [Edit log
> tailer] : Fast-forwarding stream
> 'http://ip/getJournal?jid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?ipjid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?ipjid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH'
> to transaction ID 232056751662 [2019-07-17T11:50:21.059+08:00] [INFO] [Edit
> log tailer] ip: Fast-forwarding stream
> 'http://ip/getJournal?jid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH'
> to transaction ID 232056751662 [2019-07-17T11:50:21.061+08:00] [ERROR] [Edit
> log tailer] : Unknown error encountered while tailing edits. Shutting down
> standby NN. java.io.IOException: There appears to be a gap in the edit log.
> We expected txid 232056752162, but got txid 232077264498. at
> org.apache.hadoop.hdfs.server.namenode.MetaRecoveryContext.editLogLoaderPrompt(MetaRecoveryContext.java:94)
> at
> org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadEditRecords(FSEditLogLoader.java:239)
> at
> org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadFSEdits(FSEditLogLoader.java:161)
> at
> org.apache.hadoop.hdfs.server.namenode.FSImage.loadEdits(FSImage.java:895) at
> org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer.doTailEdits(EditLogTailer.java:321)
> at
> org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread.doWork(EditLogTailer.java:460)
> at
> org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread.access$400(EditLogTailer.java:410)
> at
> org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread$1.run(EditLogTailer.java:427)
> at
> org.apache.hadoop.security.SecurityUtil.doAsLoginUserOrFatal(SecurityUtil.java:414)
> at
> org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread.run(EditLogTailer.java:423)
> [2019-07-17T11:50:21.064+08:00] [INFO] [Edit log tailer] : Exiting with
> status 1 [2019-07-17T11:50:21.066+08:00] [INFO] [Thread-1] : SHUTDOWN_MSG:
> / SHUTDOWN_MSG:
> Shutting down NameNode at ip
> /
> {code}
>
> if dfs.ha.tail-edits.max-txns-per-lock value is 500,when the namenode load
> the editlog util 500,the current namenode will load the next editlog,but
> editlog more than 500.So,namenode got an unexpected txid when tail editlog.
>
>
> {code:java}
> //
> [2019-07-17T11:50:21.059+08:00] [INFO] [Edit log tailer] : Edits file
> http://ip/getJournal?jid=ns1003=232056426162=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
>
[jira] [Updated] (HDFS-14674) [SBN read] Got an unexpected txid when tail editlog
[
https://issues.apache.org/jira/browse/HDFS-14674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
wangzhaohui updated HDFS-14674:
---
Attachment: HDFS-14674-010.patch
> [SBN read] Got an unexpected txid when tail editlog
> ---
>
> Key: HDFS-14674
> URL: https://issues.apache.org/jira/browse/HDFS-14674
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: wangzhaohui
>Assignee: wangzhaohui
>Priority: Blocker
> Attachments: HDFS-14674-001.patch, HDFS-14674-003.patch,
> HDFS-14674-004.patch, HDFS-14674-005.patch, HDFS-14674-006.patch,
> HDFS-14674-007.patch, HDFS-14674-008.patch, HDFS-14674-009.patch,
> HDFS-14674-010.patch, image.png
>
>
> Add the following configuration
> !image-2019-07-26-11-34-23-405.png!
> error:
> {code:java}
> //
> [2019-07-17T11:50:21.048+08:00] [INFO] [Edit log tailer] : replaying edit
> log: 1/20512836 transactions completed. (0%) [2019-07-17T11:50:21.059+08:00]
> [INFO] [Edit log tailer] : Edits file
> http://ip/getJournal?jid=ns1003=232056426162=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?ipjid=ns1003=232056426162=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?ipjid=ns1003=232056426162=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH
> of size 3126782311 edits # 500 loaded in 3 seconds
> [2019-07-17T11:50:21.059+08:00] [INFO] [Edit log tailer] : Reading
> org.apache.hadoop.hdfs.server.namenode.RedundantEditLogInputStream@51ceb7bc
> expecting start txid #232056752162 [2019-07-17T11:50:21.059+08:00] [INFO]
> [Edit log tailer] : Start loading edits file
> http://ip/getJournal?ipjid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?ipjid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?ipjid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH
> maxTxnipsToRead = 500 [2019-07-17T11:50:21.059+08:00] [INFO] [Edit log
> tailer] : Fast-forwarding stream
> 'http://ip/getJournal?jid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?ipjid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?ipjid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH'
> to transaction ID 232056751662 [2019-07-17T11:50:21.059+08:00] [INFO] [Edit
> log tailer] ip: Fast-forwarding stream
> 'http://ip/getJournal?jid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH'
> to transaction ID 232056751662 [2019-07-17T11:50:21.061+08:00] [ERROR] [Edit
> log tailer] : Unknown error encountered while tailing edits. Shutting down
> standby NN. java.io.IOException: There appears to be a gap in the edit log.
> We expected txid 232056752162, but got txid 232077264498. at
> org.apache.hadoop.hdfs.server.namenode.MetaRecoveryContext.editLogLoaderPrompt(MetaRecoveryContext.java:94)
> at
> org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadEditRecords(FSEditLogLoader.java:239)
> at
> org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadFSEdits(FSEditLogLoader.java:161)
> at
> org.apache.hadoop.hdfs.server.namenode.FSImage.loadEdits(FSImage.java:895) at
> org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer.doTailEdits(EditLogTailer.java:321)
> at
> org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread.doWork(EditLogTailer.java:460)
> at
> org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread.access$400(EditLogTailer.java:410)
> at
> org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread$1.run(EditLogTailer.java:427)
> at
> org.apache.hadoop.security.SecurityUtil.doAsLoginUserOrFatal(SecurityUtil.java:414)
> at
> org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread.run(EditLogTailer.java:423)
> [2019-07-17T11:50:21.064+08:00] [INFO] [Edit log tailer] : Exiting with
> status 1 [2019-07-17T11:50:21.066+08:00] [INFO] [Thread-1] : SHUTDOWN_MSG:
> / SHUTDOWN_MSG:
> Shutting down NameNode at ip
> /
> {code}
>
> if dfs.ha.tail-edits.max-txns-per-lock value is 500,when the namenode load
> the editlog util 500,the current namenode will load the next editlog,but
> editlog more than 500.So,namenode got an unexpected txid when tail editlog.
>
>
> {code:java}
> //
> [2019-07-17T11:50:21.059+08:00] [INFO] [Edit log tailer] : Edits file
> http://ip/getJournal?jid=ns1003=232056426162=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
>
[jira] [Commented] (HDFS-14674) [SBN read] Got an unexpected txid when tail editlog
[
https://issues.apache.org/jira/browse/HDFS-14674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16907923#comment-16907923
]
wangzhaohui commented on HDFS-14674:
[~csun] [~shv] upload v009,fixed checkstyle and unit test,pls review.
> [SBN read] Got an unexpected txid when tail editlog
> ---
>
> Key: HDFS-14674
> URL: https://issues.apache.org/jira/browse/HDFS-14674
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: wangzhaohui
>Assignee: wangzhaohui
>Priority: Blocker
> Attachments: HDFS-14674-001.patch, HDFS-14674-003.patch,
> HDFS-14674-004.patch, HDFS-14674-005.patch, HDFS-14674-006.patch,
> HDFS-14674-007.patch, HDFS-14674-008.patch, HDFS-14674-009.patch, image.png
>
>
> Add the following configuration
> !image-2019-07-26-11-34-23-405.png!
> error:
> {code:java}
> //
> [2019-07-17T11:50:21.048+08:00] [INFO] [Edit log tailer] : replaying edit
> log: 1/20512836 transactions completed. (0%) [2019-07-17T11:50:21.059+08:00]
> [INFO] [Edit log tailer] : Edits file
> http://ip/getJournal?jid=ns1003=232056426162=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?ipjid=ns1003=232056426162=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?ipjid=ns1003=232056426162=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH
> of size 3126782311 edits # 500 loaded in 3 seconds
> [2019-07-17T11:50:21.059+08:00] [INFO] [Edit log tailer] : Reading
> org.apache.hadoop.hdfs.server.namenode.RedundantEditLogInputStream@51ceb7bc
> expecting start txid #232056752162 [2019-07-17T11:50:21.059+08:00] [INFO]
> [Edit log tailer] : Start loading edits file
> http://ip/getJournal?ipjid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?ipjid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?ipjid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH
> maxTxnipsToRead = 500 [2019-07-17T11:50:21.059+08:00] [INFO] [Edit log
> tailer] : Fast-forwarding stream
> 'http://ip/getJournal?jid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?ipjid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?ipjid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH'
> to transaction ID 232056751662 [2019-07-17T11:50:21.059+08:00] [INFO] [Edit
> log tailer] ip: Fast-forwarding stream
> 'http://ip/getJournal?jid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH'
> to transaction ID 232056751662 [2019-07-17T11:50:21.061+08:00] [ERROR] [Edit
> log tailer] : Unknown error encountered while tailing edits. Shutting down
> standby NN. java.io.IOException: There appears to be a gap in the edit log.
> We expected txid 232056752162, but got txid 232077264498. at
> org.apache.hadoop.hdfs.server.namenode.MetaRecoveryContext.editLogLoaderPrompt(MetaRecoveryContext.java:94)
> at
> org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadEditRecords(FSEditLogLoader.java:239)
> at
> org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadFSEdits(FSEditLogLoader.java:161)
> at
> org.apache.hadoop.hdfs.server.namenode.FSImage.loadEdits(FSImage.java:895) at
> org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer.doTailEdits(EditLogTailer.java:321)
> at
> org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread.doWork(EditLogTailer.java:460)
> at
> org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread.access$400(EditLogTailer.java:410)
> at
> org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread$1.run(EditLogTailer.java:427)
> at
> org.apache.hadoop.security.SecurityUtil.doAsLoginUserOrFatal(SecurityUtil.java:414)
> at
> org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread.run(EditLogTailer.java:423)
> [2019-07-17T11:50:21.064+08:00] [INFO] [Edit log tailer] : Exiting with
> status 1 [2019-07-17T11:50:21.066+08:00] [INFO] [Thread-1] : SHUTDOWN_MSG:
> / SHUTDOWN_MSG:
> Shutting down NameNode at ip
> /
> {code}
>
> if dfs.ha.tail-edits.max-txns-per-lock value is 500,when the namenode load
> the editlog util 500,the current namenode will load the next editlog,but
> editlog more than 500.So,namenode got an unexpected txid when tail editlog.
>
>
> {code:java}
> //
> [2019-07-17T11:50:21.059+08:00] [INFO] [Edit log tailer] : Edits file
> http://ip/getJournal?jid=ns1003=232056426162=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
>
[jira] [Updated] (HDFS-14674) [SBN read] Got an unexpected txid when tail editlog
[
https://issues.apache.org/jira/browse/HDFS-14674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
wangzhaohui updated HDFS-14674:
---
Attachment: HDFS-14674-009.patch
> [SBN read] Got an unexpected txid when tail editlog
> ---
>
> Key: HDFS-14674
> URL: https://issues.apache.org/jira/browse/HDFS-14674
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: wangzhaohui
>Assignee: wangzhaohui
>Priority: Blocker
> Attachments: HDFS-14674-001.patch, HDFS-14674-003.patch,
> HDFS-14674-004.patch, HDFS-14674-005.patch, HDFS-14674-006.patch,
> HDFS-14674-007.patch, HDFS-14674-008.patch, HDFS-14674-009.patch, image.png
>
>
> Add the following configuration
> !image-2019-07-26-11-34-23-405.png!
> error:
> {code:java}
> //
> [2019-07-17T11:50:21.048+08:00] [INFO] [Edit log tailer] : replaying edit
> log: 1/20512836 transactions completed. (0%) [2019-07-17T11:50:21.059+08:00]
> [INFO] [Edit log tailer] : Edits file
> http://ip/getJournal?jid=ns1003=232056426162=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?ipjid=ns1003=232056426162=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?ipjid=ns1003=232056426162=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH
> of size 3126782311 edits # 500 loaded in 3 seconds
> [2019-07-17T11:50:21.059+08:00] [INFO] [Edit log tailer] : Reading
> org.apache.hadoop.hdfs.server.namenode.RedundantEditLogInputStream@51ceb7bc
> expecting start txid #232056752162 [2019-07-17T11:50:21.059+08:00] [INFO]
> [Edit log tailer] : Start loading edits file
> http://ip/getJournal?ipjid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?ipjid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?ipjid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH
> maxTxnipsToRead = 500 [2019-07-17T11:50:21.059+08:00] [INFO] [Edit log
> tailer] : Fast-forwarding stream
> 'http://ip/getJournal?jid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?ipjid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?ipjid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH'
> to transaction ID 232056751662 [2019-07-17T11:50:21.059+08:00] [INFO] [Edit
> log tailer] ip: Fast-forwarding stream
> 'http://ip/getJournal?jid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH'
> to transaction ID 232056751662 [2019-07-17T11:50:21.061+08:00] [ERROR] [Edit
> log tailer] : Unknown error encountered while tailing edits. Shutting down
> standby NN. java.io.IOException: There appears to be a gap in the edit log.
> We expected txid 232056752162, but got txid 232077264498. at
> org.apache.hadoop.hdfs.server.namenode.MetaRecoveryContext.editLogLoaderPrompt(MetaRecoveryContext.java:94)
> at
> org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadEditRecords(FSEditLogLoader.java:239)
> at
> org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadFSEdits(FSEditLogLoader.java:161)
> at
> org.apache.hadoop.hdfs.server.namenode.FSImage.loadEdits(FSImage.java:895) at
> org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer.doTailEdits(EditLogTailer.java:321)
> at
> org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread.doWork(EditLogTailer.java:460)
> at
> org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread.access$400(EditLogTailer.java:410)
> at
> org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread$1.run(EditLogTailer.java:427)
> at
> org.apache.hadoop.security.SecurityUtil.doAsLoginUserOrFatal(SecurityUtil.java:414)
> at
> org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread.run(EditLogTailer.java:423)
> [2019-07-17T11:50:21.064+08:00] [INFO] [Edit log tailer] : Exiting with
> status 1 [2019-07-17T11:50:21.066+08:00] [INFO] [Thread-1] : SHUTDOWN_MSG:
> / SHUTDOWN_MSG:
> Shutting down NameNode at ip
> /
> {code}
>
> if dfs.ha.tail-edits.max-txns-per-lock value is 500,when the namenode load
> the editlog util 500,the current namenode will load the next editlog,but
> editlog more than 500.So,namenode got an unexpected txid when tail editlog.
>
>
> {code:java}
> //
> [2019-07-17T11:50:21.059+08:00] [INFO] [Edit log tailer] : Edits file
> http://ip/getJournal?jid=ns1003=232056426162=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
>
[jira] [Updated] (HDFS-14674) [SBN read] Got an unexpected txid when tail editlog
[
https://issues.apache.org/jira/browse/HDFS-14674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
wangzhaohui updated HDFS-14674:
---
Attachment: HDFS-14674-008.patch
> [SBN read] Got an unexpected txid when tail editlog
> ---
>
> Key: HDFS-14674
> URL: https://issues.apache.org/jira/browse/HDFS-14674
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: wangzhaohui
>Assignee: wangzhaohui
>Priority: Blocker
> Attachments: HDFS-14674-001.patch, HDFS-14674-003.patch,
> HDFS-14674-004.patch, HDFS-14674-005.patch, HDFS-14674-006.patch,
> HDFS-14674-007.patch, HDFS-14674-008.patch, image.png
>
>
> Add the following configuration
> !image-2019-07-26-11-34-23-405.png!
> error:
> {code:java}
> //
> [2019-07-17T11:50:21.048+08:00] [INFO] [Edit log tailer] : replaying edit
> log: 1/20512836 transactions completed. (0%) [2019-07-17T11:50:21.059+08:00]
> [INFO] [Edit log tailer] : Edits file
> http://ip/getJournal?jid=ns1003=232056426162=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?ipjid=ns1003=232056426162=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?ipjid=ns1003=232056426162=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH
> of size 3126782311 edits # 500 loaded in 3 seconds
> [2019-07-17T11:50:21.059+08:00] [INFO] [Edit log tailer] : Reading
> org.apache.hadoop.hdfs.server.namenode.RedundantEditLogInputStream@51ceb7bc
> expecting start txid #232056752162 [2019-07-17T11:50:21.059+08:00] [INFO]
> [Edit log tailer] : Start loading edits file
> http://ip/getJournal?ipjid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?ipjid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?ipjid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH
> maxTxnipsToRead = 500 [2019-07-17T11:50:21.059+08:00] [INFO] [Edit log
> tailer] : Fast-forwarding stream
> 'http://ip/getJournal?jid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?ipjid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?ipjid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH'
> to transaction ID 232056751662 [2019-07-17T11:50:21.059+08:00] [INFO] [Edit
> log tailer] ip: Fast-forwarding stream
> 'http://ip/getJournal?jid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH'
> to transaction ID 232056751662 [2019-07-17T11:50:21.061+08:00] [ERROR] [Edit
> log tailer] : Unknown error encountered while tailing edits. Shutting down
> standby NN. java.io.IOException: There appears to be a gap in the edit log.
> We expected txid 232056752162, but got txid 232077264498. at
> org.apache.hadoop.hdfs.server.namenode.MetaRecoveryContext.editLogLoaderPrompt(MetaRecoveryContext.java:94)
> at
> org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadEditRecords(FSEditLogLoader.java:239)
> at
> org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadFSEdits(FSEditLogLoader.java:161)
> at
> org.apache.hadoop.hdfs.server.namenode.FSImage.loadEdits(FSImage.java:895) at
> org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer.doTailEdits(EditLogTailer.java:321)
> at
> org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread.doWork(EditLogTailer.java:460)
> at
> org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread.access$400(EditLogTailer.java:410)
> at
> org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread$1.run(EditLogTailer.java:427)
> at
> org.apache.hadoop.security.SecurityUtil.doAsLoginUserOrFatal(SecurityUtil.java:414)
> at
> org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread.run(EditLogTailer.java:423)
> [2019-07-17T11:50:21.064+08:00] [INFO] [Edit log tailer] : Exiting with
> status 1 [2019-07-17T11:50:21.066+08:00] [INFO] [Thread-1] : SHUTDOWN_MSG:
> / SHUTDOWN_MSG:
> Shutting down NameNode at ip
> /
> {code}
>
> if dfs.ha.tail-edits.max-txns-per-lock value is 500,when the namenode load
> the editlog util 500,the current namenode will load the next editlog,but
> editlog more than 500.So,namenode got an unexpected txid when tail editlog.
>
>
> {code:java}
> //
> [2019-07-17T11:50:21.059+08:00] [INFO] [Edit log tailer] : Edits file
> http://ip/getJournal?jid=ns1003=232056426162=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?jid=ns1003=232056426162=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
>
[jira] [Commented] (HDFS-14674) [SBN read] Got an unexpected txid when tail editlog
[
https://issues.apache.org/jira/browse/HDFS-14674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16907105#comment-16907105
]
wangzhaohui commented on HDFS-14674:
[~Captainhzy] thanks for your comment, if i add the conf
{{DFS_HA_TAILEDITS_MAX_TXNS_PER_LOCK_KEY}}, then in this for{}
{code:java}
//
for (EditLogInputStream editIn : editStreams) {
...
}{code}
In the case of multiple editStreams，
the range read by the first editStream is [0-100],but never break, the second
editStream is set to read[200-300],so there will got an unexpected txid when
tail editlog.
> [SBN read] Got an unexpected txid when tail editlog
> ---
>
> Key: HDFS-14674
> URL: https://issues.apache.org/jira/browse/HDFS-14674
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: wangzhaohui
>Assignee: wangzhaohui
>Priority: Blocker
> Attachments: HDFS-14674-001.patch, HDFS-14674-003.patch,
> HDFS-14674-004.patch, HDFS-14674-005.patch, HDFS-14674-006.patch,
> HDFS-14674-007.patch, image.png
>
>
> Add the following configuration
> !image-2019-07-26-11-34-23-405.png!
> error:
> {code:java}
> //
> [2019-07-17T11:50:21.048+08:00] [INFO] [Edit log tailer] : replaying edit
> log: 1/20512836 transactions completed. (0%) [2019-07-17T11:50:21.059+08:00]
> [INFO] [Edit log tailer] : Edits file
> http://ip/getJournal?jid=ns1003=232056426162=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?ipjid=ns1003=232056426162=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?ipjid=ns1003=232056426162=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH
> of size 3126782311 edits # 500 loaded in 3 seconds
> [2019-07-17T11:50:21.059+08:00] [INFO] [Edit log tailer] : Reading
> org.apache.hadoop.hdfs.server.namenode.RedundantEditLogInputStream@51ceb7bc
> expecting start txid #232056752162 [2019-07-17T11:50:21.059+08:00] [INFO]
> [Edit log tailer] : Start loading edits file
> http://ip/getJournal?ipjid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?ipjid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?ipjid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH
> maxTxnipsToRead = 500 [2019-07-17T11:50:21.059+08:00] [INFO] [Edit log
> tailer] : Fast-forwarding stream
> 'http://ip/getJournal?jid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?ipjid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?ipjid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH'
> to transaction ID 232056751662 [2019-07-17T11:50:21.059+08:00] [INFO] [Edit
> log tailer] ip: Fast-forwarding stream
> 'http://ip/getJournal?jid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH'
> to transaction ID 232056751662 [2019-07-17T11:50:21.061+08:00] [ERROR] [Edit
> log tailer] : Unknown error encountered while tailing edits. Shutting down
> standby NN. java.io.IOException: There appears to be a gap in the edit log.
> We expected txid 232056752162, but got txid 232077264498. at
> org.apache.hadoop.hdfs.server.namenode.MetaRecoveryContext.editLogLoaderPrompt(MetaRecoveryContext.java:94)
> at
> org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadEditRecords(FSEditLogLoader.java:239)
> at
> org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadFSEdits(FSEditLogLoader.java:161)
> at
> org.apache.hadoop.hdfs.server.namenode.FSImage.loadEdits(FSImage.java:895) at
> org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer.doTailEdits(EditLogTailer.java:321)
> at
> org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread.doWork(EditLogTailer.java:460)
> at
> org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread.access$400(EditLogTailer.java:410)
> at
> org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread$1.run(EditLogTailer.java:427)
> at
> org.apache.hadoop.security.SecurityUtil.doAsLoginUserOrFatal(SecurityUtil.java:414)
> at
> org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread.run(EditLogTailer.java:423)
> [2019-07-17T11:50:21.064+08:00] [INFO] [Edit log tailer] : Exiting with
> status 1 [2019-07-17T11:50:21.066+08:00] [INFO] [Thread-1] : SHUTDOWN_MSG:
> / SHUTDOWN_MSG:
> Shutting down NameNode at ip
> /
> {code}
>
> if dfs.ha.tail-edits.max-txns-per-lock value is 500,when the namenode load
> the editlog util 500,the current namenode will load the next editlog,but
> editlog
[jira] [Updated] (HDFS-14674) [SBN read] Got an unexpected txid when tail editlog
[
https://issues.apache.org/jira/browse/HDFS-14674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
wangzhaohui updated HDFS-14674:
---
Attachment: HDFS-14674-007.patch
> [SBN read] Got an unexpected txid when tail editlog
> ---
>
> Key: HDFS-14674
> URL: https://issues.apache.org/jira/browse/HDFS-14674
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: wangzhaohui
>Assignee: wangzhaohui
>Priority: Blocker
> Attachments: HDFS-14674-001.patch, HDFS-14674-003.patch,
> HDFS-14674-004.patch, HDFS-14674-005.patch, HDFS-14674-006.patch,
> HDFS-14674-007.patch, image.png
>
>
> Add the following configuration
> !image-2019-07-26-11-34-23-405.png!
> error:
> {code:java}
> //
> [2019-07-17T11:50:21.048+08:00] [INFO] [Edit log tailer] : replaying edit
> log: 1/20512836 transactions completed. (0%) [2019-07-17T11:50:21.059+08:00]
> [INFO] [Edit log tailer] : Edits file
> http://ip/getJournal?jid=ns1003=232056426162=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?ipjid=ns1003=232056426162=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?ipjid=ns1003=232056426162=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH
> of size 3126782311 edits # 500 loaded in 3 seconds
> [2019-07-17T11:50:21.059+08:00] [INFO] [Edit log tailer] : Reading
> org.apache.hadoop.hdfs.server.namenode.RedundantEditLogInputStream@51ceb7bc
> expecting start txid #232056752162 [2019-07-17T11:50:21.059+08:00] [INFO]
> [Edit log tailer] : Start loading edits file
> http://ip/getJournal?ipjid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?ipjid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?ipjid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH
> maxTxnipsToRead = 500 [2019-07-17T11:50:21.059+08:00] [INFO] [Edit log
> tailer] : Fast-forwarding stream
> 'http://ip/getJournal?jid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?ipjid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?ipjid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH'
> to transaction ID 232056751662 [2019-07-17T11:50:21.059+08:00] [INFO] [Edit
> log tailer] ip: Fast-forwarding stream
> 'http://ip/getJournal?jid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH'
> to transaction ID 232056751662 [2019-07-17T11:50:21.061+08:00] [ERROR] [Edit
> log tailer] : Unknown error encountered while tailing edits. Shutting down
> standby NN. java.io.IOException: There appears to be a gap in the edit log.
> We expected txid 232056752162, but got txid 232077264498. at
> org.apache.hadoop.hdfs.server.namenode.MetaRecoveryContext.editLogLoaderPrompt(MetaRecoveryContext.java:94)
> at
> org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadEditRecords(FSEditLogLoader.java:239)
> at
> org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadFSEdits(FSEditLogLoader.java:161)
> at
> org.apache.hadoop.hdfs.server.namenode.FSImage.loadEdits(FSImage.java:895) at
> org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer.doTailEdits(EditLogTailer.java:321)
> at
> org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread.doWork(EditLogTailer.java:460)
> at
> org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread.access$400(EditLogTailer.java:410)
> at
> org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread$1.run(EditLogTailer.java:427)
> at
> org.apache.hadoop.security.SecurityUtil.doAsLoginUserOrFatal(SecurityUtil.java:414)
> at
> org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread.run(EditLogTailer.java:423)
> [2019-07-17T11:50:21.064+08:00] [INFO] [Edit log tailer] : Exiting with
> status 1 [2019-07-17T11:50:21.066+08:00] [INFO] [Thread-1] : SHUTDOWN_MSG:
> / SHUTDOWN_MSG:
> Shutting down NameNode at ip
> /
> {code}
>
> if dfs.ha.tail-edits.max-txns-per-lock value is 500,when the namenode load
> the editlog util 500,the current namenode will load the next editlog,but
> editlog more than 500.So,namenode got an unexpected txid when tail editlog.
>
>
> {code:java}
> //
> [2019-07-17T11:50:21.059+08:00] [INFO] [Edit log tailer] : Edits file
> http://ip/getJournal?jid=ns1003=232056426162=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?jid=ns1003=232056426162=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
>
[jira] [Commented] (HDFS-14713) RBF: RouterAdmin supports refreshRouterArgs command but not on display
[
https://issues.apache.org/jira/browse/HDFS-14713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16906851#comment-16906851
]
wangzhaohui commented on HDFS-14713:
sorry,my bad,upload v004 pls review,thanks.[~ayushtkn] [~elgoiri]
> RBF: RouterAdmin supports refreshRouterArgs command but not on display
> --
>
> Key: HDFS-14713
> URL: https://issues.apache.org/jira/browse/HDFS-14713
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: wangzhaohui
>Assignee: wangzhaohui
>Priority: Major
> Attachments: HDFS-14713-000.patch, HDFS-14713-001.patch,
> HDFS-14713-002.patch, HDFS-14713-003.patch, HDFS-14713-004.patch, after.png,
> before.png
>
>
> When the cmd commond is null,the refreshRouterArgs command is not display
> ,because there is one missing value in the String[] commands
> {code:java}
> //
> if (cmd == null) {
> String[] commands =
> {"-add", "-update", "-rm", "-ls", "-getDestination",
> "-setQuota", "-clrQuota",
> "-safemode", "-nameservice", "-getDisabledNameservices",
> "-refresh"};
> StringBuilder usage = new StringBuilder();
> usage.append("Usage: hdfs dfsrouteradmin :\n");
> for (int i = 0; i < commands.length; i++) {
> usage.append(getUsage(commands[i]));
> if (i + 1 < commands.length) {
> usage.append("\n");
> }
> }
>
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-14713) RBF: RouterAdmin supports refreshRouterArgs command but not on display
[
https://issues.apache.org/jira/browse/HDFS-14713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
wangzhaohui updated HDFS-14713:
---
Attachment: HDFS-14713-004.patch
> RBF: RouterAdmin supports refreshRouterArgs command but not on display
> --
>
> Key: HDFS-14713
> URL: https://issues.apache.org/jira/browse/HDFS-14713
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: wangzhaohui
>Assignee: wangzhaohui
>Priority: Major
> Attachments: HDFS-14713-000.patch, HDFS-14713-001.patch,
> HDFS-14713-002.patch, HDFS-14713-003.patch, HDFS-14713-004.patch, after.png,
> before.png
>
>
> When the cmd commond is null,the refreshRouterArgs command is not display
> ,because there is one missing value in the String[] commands
> {code:java}
> //
> if (cmd == null) {
> String[] commands =
> {"-add", "-update", "-rm", "-ls", "-getDestination",
> "-setQuota", "-clrQuota",
> "-safemode", "-nameservice", "-getDisabledNameservices",
> "-refresh"};
> StringBuilder usage = new StringBuilder();
> usage.append("Usage: hdfs dfsrouteradmin :\n");
> for (int i = 0; i < commands.length; i++) {
> usage.append(getUsage(commands[i]));
> if (i + 1 < commands.length) {
> usage.append("\n");
> }
> }
>
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14713) RBF: routeradmin support refreshRouterArgs command but it not on display
[
https://issues.apache.org/jira/browse/HDFS-14713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16905884#comment-16905884
]
wangzhaohui commented on HDFS-14713:
Hi,[~ayushtkn] [~elgoiri] upload v003,please review,thanks.
> RBF: routeradmin support refreshRouterArgs command but it not on display
>
>
> Key: HDFS-14713
> URL: https://issues.apache.org/jira/browse/HDFS-14713
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: wangzhaohui
>Assignee: wangzhaohui
>Priority: Major
> Attachments: HDFS-14713-000.patch, HDFS-14713-001.patch,
> HDFS-14713-002.patch, HDFS-14713-003.patch, after.png, before.png
>
>
> When the cmd commond is null,the refreshRouterArgs command is not display
> ,because there is one missing value in the String[] commands
> {code:java}
> //
> if (cmd == null) {
> String[] commands =
> {"-add", "-update", "-rm", "-ls", "-getDestination",
> "-setQuota", "-clrQuota",
> "-safemode", "-nameservice", "-getDisabledNameservices",
> "-refresh"};
> StringBuilder usage = new StringBuilder();
> usage.append("Usage: hdfs dfsrouteradmin :\n");
> for (int i = 0; i < commands.length; i++) {
> usage.append(getUsage(commands[i]));
> if (i + 1 < commands.length) {
> usage.append("\n");
> }
> }
>
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-14713) RBF: routeradmin support refreshRouterArgs command but it not on display
[
https://issues.apache.org/jira/browse/HDFS-14713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
wangzhaohui updated HDFS-14713:
---
Attachment: HDFS-14713-003.patch
> RBF: routeradmin support refreshRouterArgs command but it not on display
>
>
> Key: HDFS-14713
> URL: https://issues.apache.org/jira/browse/HDFS-14713
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: wangzhaohui
>Assignee: wangzhaohui
>Priority: Major
> Attachments: HDFS-14713-000.patch, HDFS-14713-001.patch,
> HDFS-14713-002.patch, HDFS-14713-003.patch, after.png, before.png
>
>
> When the cmd commond is null,the refreshRouterArgs command is not display
> ,because there is one missing value in the String[] commands
> {code:java}
> //
> if (cmd == null) {
> String[] commands =
> {"-add", "-update", "-rm", "-ls", "-getDestination",
> "-setQuota", "-clrQuota",
> "-safemode", "-nameservice", "-getDisabledNameservices",
> "-refresh"};
> StringBuilder usage = new StringBuilder();
> usage.append("Usage: hdfs dfsrouteradmin :\n");
> for (int i = 0; i < commands.length; i++) {
> usage.append(getUsage(commands[i]));
> if (i + 1 < commands.length) {
> usage.append("\n");
> }
> }
>
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-14713) RBF: routeradmin support refreshRouterArgs command but it not on display
[
https://issues.apache.org/jira/browse/HDFS-14713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
wangzhaohui updated HDFS-14713:
---
Attachment: HDFS-14713-002.patch
> RBF: routeradmin support refreshRouterArgs command but it not on display
>
>
> Key: HDFS-14713
> URL: https://issues.apache.org/jira/browse/HDFS-14713
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: wangzhaohui
>Assignee: wangzhaohui
>Priority: Major
> Attachments: HDFS-14713-000.patch, HDFS-14713-001.patch,
> HDFS-14713-002.patch, after.png, before.png
>
>
> When the cmd commond is null,the refreshRouterArgs command is not display
> ,because there is one missing value in the String[] commands
> {code:java}
> //
> if (cmd == null) {
> String[] commands =
> {"-add", "-update", "-rm", "-ls", "-getDestination",
> "-setQuota", "-clrQuota",
> "-safemode", "-nameservice", "-getDisabledNameservices",
> "-refresh"};
> StringBuilder usage = new StringBuilder();
> usage.append("Usage: hdfs dfsrouteradmin :\n");
> for (int i = 0; i < commands.length; i++) {
> usage.append(getUsage(commands[i]));
> if (i + 1 < commands.length) {
> usage.append("\n");
> }
> }
>
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-14713) RBF: routeradmin support refreshRouterArgs command but it not on display
[
https://issues.apache.org/jira/browse/HDFS-14713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
wangzhaohui updated HDFS-14713:
---
Attachment: HDFS-14713-001.patch
> RBF: routeradmin support refreshRouterArgs command but it not on display
>
>
> Key: HDFS-14713
> URL: https://issues.apache.org/jira/browse/HDFS-14713
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: wangzhaohui
>Assignee: wangzhaohui
>Priority: Major
> Attachments: HDFS-14713-000.patch, HDFS-14713-001.patch, after.png,
> before.png
>
>
> When the cmd commond is null,the refreshRouterArgs command is not display
> ,because there is one missing value in the String[] commands
> {code:java}
> //
> if (cmd == null) {
> String[] commands =
> {"-add", "-update", "-rm", "-ls", "-getDestination",
> "-setQuota", "-clrQuota",
> "-safemode", "-nameservice", "-getDisabledNameservices",
> "-refresh"};
> StringBuilder usage = new StringBuilder();
> usage.append("Usage: hdfs dfsrouteradmin :\n");
> for (int i = 0; i < commands.length; i++) {
> usage.append(getUsage(commands[i]));
> if (i + 1 < commands.length) {
> usage.append("\n");
> }
> }
>
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-14713) RBF: routeradmin support refreshRouterArgs command but it not on display
[
https://issues.apache.org/jira/browse/HDFS-14713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
wangzhaohui updated HDFS-14713:
---
Attachment: HDFS-14713-000.patch
> RBF: routeradmin support refreshRouterArgs command but it not on display
>
>
> Key: HDFS-14713
> URL: https://issues.apache.org/jira/browse/HDFS-14713
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: wangzhaohui
>Assignee: wangzhaohui
>Priority: Major
> Attachments: HDFS-14713-000.patch, after.png, before.png
>
>
> When the cmd commond is null,the refreshRouterArgs command is not display
> ,because there is one missing value in the String[] commands
> {code:java}
> //
> if (cmd == null) {
> String[] commands =
> {"-add", "-update", "-rm", "-ls", "-getDestination",
> "-setQuota", "-clrQuota",
> "-safemode", "-nameservice", "-getDisabledNameservices",
> "-refresh"};
> StringBuilder usage = new StringBuilder();
> usage.append("Usage: hdfs dfsrouteradmin :\n");
> for (int i = 0; i < commands.length; i++) {
> usage.append(getUsage(commands[i]));
> if (i + 1 < commands.length) {
> usage.append("\n");
> }
> }
>
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-14713) RBF: routeradmin support refreshRouterArgs command but it not on display
[
https://issues.apache.org/jira/browse/HDFS-14713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
wangzhaohui updated HDFS-14713:
---
Attachment: after.png
> RBF: routeradmin support refreshRouterArgs command but it not on display
>
>
> Key: HDFS-14713
> URL: https://issues.apache.org/jira/browse/HDFS-14713
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: wangzhaohui
>Assignee: wangzhaohui
>Priority: Major
> Attachments: after.png, before.png
>
>
> When the cmd commond is null,the refreshRouterArgs command is not display
> ,because there is one missing value in the String[] commands
> {code:java}
> //
> if (cmd == null) {
> String[] commands =
> {"-add", "-update", "-rm", "-ls", "-getDestination",
> "-setQuota", "-clrQuota",
> "-safemode", "-nameservice", "-getDisabledNameservices",
> "-refresh"};
> StringBuilder usage = new StringBuilder();
> usage.append("Usage: hdfs dfsrouteradmin :\n");
> for (int i = 0; i < commands.length; i++) {
> usage.append(getUsage(commands[i]));
> if (i + 1 < commands.length) {
> usage.append("\n");
> }
> }
>
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-14713) RBF: routeradmin support refreshRouterArgs command but it not on display
[
https://issues.apache.org/jira/browse/HDFS-14713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
wangzhaohui updated HDFS-14713:
---
Attachment: before.png
> RBF: routeradmin support refreshRouterArgs command but it not on display
>
>
> Key: HDFS-14713
> URL: https://issues.apache.org/jira/browse/HDFS-14713
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: wangzhaohui
>Assignee: wangzhaohui
>Priority: Major
> Attachments: before.png
>
>
> When the cmd commond is null,the refreshRouterArgs command is not display
> ,because there is one missing value in the String[] commands
> {code:java}
> //
> if (cmd == null) {
> String[] commands =
> {"-add", "-update", "-rm", "-ls", "-getDestination",
> "-setQuota", "-clrQuota",
> "-safemode", "-nameservice", "-getDisabledNameservices",
> "-refresh"};
> StringBuilder usage = new StringBuilder();
> usage.append("Usage: hdfs dfsrouteradmin :\n");
> for (int i = 0; i < commands.length; i++) {
> usage.append(getUsage(commands[i]));
> if (i + 1 < commands.length) {
> usage.append("\n");
> }
> }
>
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Resolved] (HDFS-14712) RBF: routeradmin support refreshRouterArgs command but it not on display
[
https://issues.apache.org/jira/browse/HDFS-14712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
wangzhaohui resolved HDFS-14712.
Resolution: Duplicate
> RBF: routeradmin support refreshRouterArgs command but it not on display
>
>
> Key: HDFS-14712
> URL: https://issues.apache.org/jira/browse/HDFS-14712
> Project: Hadoop HDFS
> Issue Type: Task
>Reporter: wangzhaohui
>Assignee: wangzhaohui
>Priority: Major
>
> {color:#cc7832}When the CMD command is null, the refreshRouterArgs command is
> not displayed，because there is one missing value in the String[] commands
> {color}
> {color:#cc7832}private {color}String {color:#ffc66d}getUsage{color}(String
> cmd) {
> {color:#cc7832}if {color}(cmd == {color:#cc7832}null{color}) {
> String[] commands =
> {{color:#6a8759}"-add"{color}{color:#cc7832},
> {color}{color:#6a8759}"-update"{color}{color:#cc7832},
> {color}{color:#6a8759}"-rm"{color}{color:#cc7832},
> {color}{color:#6a8759}"-ls"{color}{color:#cc7832},
> {color}{color:#6a8759}"-getDestination"{color}{color:#cc7832},
> {color} {color:#6a8759}"-setQuota"{color}{color:#cc7832},
> {color}{color:#6a8759}"-clrQuota"{color}{color:#cc7832},
> {color} {color:#6a8759}"-safemode"{color}{color:#cc7832},
> {color}{color:#6a8759}"-nameservice"{color}{color:#cc7832},
> {color}{color:#6a8759}"-getDisabledNameservices"{color}{color:#cc7832},
> {color} {color:#6a8759}"-refresh"{color}}{color:#cc7832};
> {color} StringBuilder usage = {color:#cc7832}new
> {color}StringBuilder(){color:#cc7832};
> {color} usage.append({color:#6a8759}"Usage: hdfs dfsrouteradmin
> :{color}{color:#cc7832}\n{color}{color:#6a8759}"{color}){color:#cc7832};
> {color}{color:#cc7832} for {color}({color:#cc7832}int {color}i =
> {color:#6897bb}0{color}{color:#cc7832}; {color}i <
> commands.{color:#9876aa}length{color}{color:#cc7832}; {color}i++) {
> usage.append(getUsage(commands[i])){color:#cc7832};
> {color}{color:#cc7832} if {color}(i + {color:#6897bb}1 {color}<
> commands.{color:#9876aa}length{color}) {
>
> usage.append({color:#6a8759}"{color}{color:#cc7832}\n{color}{color:#6a8759}"{color}){color:#cc7832};
> {color} }
> {color:#cc7832}...
> {color} }
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Created] (HDFS-14713) RBF: routeradmin support refreshRouterArgs command but it not on display
wangzhaohui created HDFS-14713:
--
Summary: RBF: routeradmin support refreshRouterArgs command but it
not on display
Key: HDFS-14713
URL: https://issues.apache.org/jira/browse/HDFS-14713
Project: Hadoop HDFS
Issue Type: Sub-task
Reporter: wangzhaohui
Assignee: wangzhaohui
When the cmd commond is null,the refreshRouterArgs command is not display
,because there is one missing value in the String[] commands
{code:java}
//
if (cmd == null) {
String[] commands =
{"-add", "-update", "-rm", "-ls", "-getDestination",
"-setQuota", "-clrQuota",
"-safemode", "-nameservice", "-getDisabledNameservices",
"-refresh"};
StringBuilder usage = new StringBuilder();
usage.append("Usage: hdfs dfsrouteradmin :\n");
for (int i = 0; i < commands.length; i++) {
usage.append(getUsage(commands[i]));
if (i + 1 < commands.length) {
usage.append("\n");
}
}
}
{code}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-14712) RBF: routeradmin support refreshRouterArgs command but it not on display
[
https://issues.apache.org/jira/browse/HDFS-14712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
wangzhaohui updated HDFS-14712:
---
Issue Type: Task (was: Bug)
> RBF: routeradmin support refreshRouterArgs command but it not on display
>
>
> Key: HDFS-14712
> URL: https://issues.apache.org/jira/browse/HDFS-14712
> Project: Hadoop HDFS
> Issue Type: Task
>Reporter: wangzhaohui
>Assignee: wangzhaohui
>Priority: Major
>
> {color:#cc7832}When the CMD command is null, the refreshRouterArgs command is
> not displayed，because there is one missing value in the String[] commands
> {color}
> {color:#cc7832}private {color}String {color:#ffc66d}getUsage{color}(String
> cmd) {
> {color:#cc7832}if {color}(cmd == {color:#cc7832}null{color}) {
> String[] commands =
> {{color:#6a8759}"-add"{color}{color:#cc7832},
> {color}{color:#6a8759}"-update"{color}{color:#cc7832},
> {color}{color:#6a8759}"-rm"{color}{color:#cc7832},
> {color}{color:#6a8759}"-ls"{color}{color:#cc7832},
> {color}{color:#6a8759}"-getDestination"{color}{color:#cc7832},
> {color} {color:#6a8759}"-setQuota"{color}{color:#cc7832},
> {color}{color:#6a8759}"-clrQuota"{color}{color:#cc7832},
> {color} {color:#6a8759}"-safemode"{color}{color:#cc7832},
> {color}{color:#6a8759}"-nameservice"{color}{color:#cc7832},
> {color}{color:#6a8759}"-getDisabledNameservices"{color}{color:#cc7832},
> {color} {color:#6a8759}"-refresh"{color}}{color:#cc7832};
> {color} StringBuilder usage = {color:#cc7832}new
> {color}StringBuilder(){color:#cc7832};
> {color} usage.append({color:#6a8759}"Usage: hdfs dfsrouteradmin
> :{color}{color:#cc7832}\n{color}{color:#6a8759}"{color}){color:#cc7832};
> {color}{color:#cc7832} for {color}({color:#cc7832}int {color}i =
> {color:#6897bb}0{color}{color:#cc7832}; {color}i <
> commands.{color:#9876aa}length{color}{color:#cc7832}; {color}i++) {
> usage.append(getUsage(commands[i])){color:#cc7832};
> {color}{color:#cc7832} if {color}(i + {color:#6897bb}1 {color}<
> commands.{color:#9876aa}length{color}) {
>
> usage.append({color:#6a8759}"{color}{color:#cc7832}\n{color}{color:#6a8759}"{color}){color:#cc7832};
> {color} }
> {color:#cc7832}...
> {color} }
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Assigned] (HDFS-14565) RBF: updateMount should validate the newEntry in RouterAdmin
[ https://issues.apache.org/jira/browse/HDFS-14565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangzhaohui reassigned HDFS-14565: -- Assignee: xuzq (was: wangzhaohui) > RBF: updateMount should validate the newEntry in RouterAdmin > > > Key: HDFS-14565 > URL: https://issues.apache.org/jira/browse/HDFS-14565 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: rbf >Reporter: xuzq >Assignee: xuzq >Priority: Major > Labels: RBF > Attachments: HDFS-14565-HDFS-13891-001.patch > > > RouterAdmin should validate the new mountEntry in updateMount. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Created] (HDFS-14712) RBF: routeradmin support refreshRouterArgs command but it not on display
wangzhaohui created HDFS-14712:
--
Summary: RBF: routeradmin support refreshRouterArgs command but it
not on display
Key: HDFS-14712
URL: https://issues.apache.org/jira/browse/HDFS-14712
Project: Hadoop HDFS
Issue Type: Bug
Reporter: wangzhaohui
Assignee: wangzhaohui
{color:#cc7832}When the CMD command is null, the refreshRouterArgs command is
not displayed，because there is one missing value in the String[] commands
{color}
{color:#cc7832}private {color}String {color:#ffc66d}getUsage{color}(String cmd)
{
{color:#cc7832}if {color}(cmd == {color:#cc7832}null{color}) {
String[] commands =
{{color:#6a8759}"-add"{color}{color:#cc7832},
{color}{color:#6a8759}"-update"{color}{color:#cc7832},
{color}{color:#6a8759}"-rm"{color}{color:#cc7832},
{color}{color:#6a8759}"-ls"{color}{color:#cc7832},
{color}{color:#6a8759}"-getDestination"{color}{color:#cc7832},
{color} {color:#6a8759}"-setQuota"{color}{color:#cc7832},
{color}{color:#6a8759}"-clrQuota"{color}{color:#cc7832},
{color} {color:#6a8759}"-safemode"{color}{color:#cc7832},
{color}{color:#6a8759}"-nameservice"{color}{color:#cc7832},
{color}{color:#6a8759}"-getDisabledNameservices"{color}{color:#cc7832},
{color} {color:#6a8759}"-refresh"{color}}{color:#cc7832};
{color} StringBuilder usage = {color:#cc7832}new
{color}StringBuilder(){color:#cc7832};
{color} usage.append({color:#6a8759}"Usage: hdfs dfsrouteradmin
:{color}{color:#cc7832}\n{color}{color:#6a8759}"{color}){color:#cc7832};
{color}{color:#cc7832} for {color}({color:#cc7832}int {color}i =
{color:#6897bb}0{color}{color:#cc7832}; {color}i <
commands.{color:#9876aa}length{color}{color:#cc7832}; {color}i++) {
usage.append(getUsage(commands[i])){color:#cc7832};
{color}{color:#cc7832} if {color}(i + {color:#6897bb}1 {color}<
commands.{color:#9876aa}length{color}) {
usage.append({color:#6a8759}"{color}{color:#cc7832}\n{color}{color:#6a8759}"{color}){color:#cc7832};
{color} }
{color:#cc7832}...
{color} }
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Assigned] (HDFS-14565) RBF: updateMount should validate the newEntry in RouterAdmin
[ https://issues.apache.org/jira/browse/HDFS-14565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wangzhaohui reassigned HDFS-14565: -- Assignee: wangzhaohui > RBF: updateMount should validate the newEntry in RouterAdmin > > > Key: HDFS-14565 > URL: https://issues.apache.org/jira/browse/HDFS-14565 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: rbf >Reporter: xuzq >Assignee: wangzhaohui >Priority: Major > Labels: RBF > Attachments: HDFS-14565-HDFS-13891-001.patch > > > RouterAdmin should validate the new mountEntry in updateMount. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HDFS-14674) [SBN read] Got an unexpected txid when tail editlog
[
https://issues.apache.org/jira/browse/HDFS-14674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16903495#comment-16903495
]
wangzhaohui edited comment on HDFS-14674 at 8/9/19 6:47 AM:
Hi,[~csun] [~shv],this patch v006 added unit test and fixed checkstyle,pls
take a look,thanks!
was (Author: wangzhaohui):
Hi,[~csun] [~shv],this patch v006 add unit test and fix checkstyle,pls take a
look,thanks!
> [SBN read] Got an unexpected txid when tail editlog
> ---
>
> Key: HDFS-14674
> URL: https://issues.apache.org/jira/browse/HDFS-14674
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: wangzhaohui
>Assignee: wangzhaohui
>Priority: Blocker
> Attachments: HDFS-14674-001.patch, HDFS-14674-003.patch,
> HDFS-14674-004.patch, HDFS-14674-005.patch, HDFS-14674-006.patch, image.png
>
>
> Add the following configuration
> !image-2019-07-26-11-34-23-405.png!
> error:
> {code:java}
> //
> [2019-07-17T11:50:21.048+08:00] [INFO] [Edit log tailer] : replaying edit
> log: 1/20512836 transactions completed. (0%) [2019-07-17T11:50:21.059+08:00]
> [INFO] [Edit log tailer] : Edits file
> http://ip/getJournal?jid=ns1003=232056426162=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?ipjid=ns1003=232056426162=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?ipjid=ns1003=232056426162=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH
> of size 3126782311 edits # 500 loaded in 3 seconds
> [2019-07-17T11:50:21.059+08:00] [INFO] [Edit log tailer] : Reading
> org.apache.hadoop.hdfs.server.namenode.RedundantEditLogInputStream@51ceb7bc
> expecting start txid #232056752162 [2019-07-17T11:50:21.059+08:00] [INFO]
> [Edit log tailer] : Start loading edits file
> http://ip/getJournal?ipjid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?ipjid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?ipjid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH
> maxTxnipsToRead = 500 [2019-07-17T11:50:21.059+08:00] [INFO] [Edit log
> tailer] : Fast-forwarding stream
> 'http://ip/getJournal?jid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?ipjid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?ipjid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH'
> to transaction ID 232056751662 [2019-07-17T11:50:21.059+08:00] [INFO] [Edit
> log tailer] ip: Fast-forwarding stream
> 'http://ip/getJournal?jid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH'
> to transaction ID 232056751662 [2019-07-17T11:50:21.061+08:00] [ERROR] [Edit
> log tailer] : Unknown error encountered while tailing edits. Shutting down
> standby NN. java.io.IOException: There appears to be a gap in the edit log.
> We expected txid 232056752162, but got txid 232077264498. at
> org.apache.hadoop.hdfs.server.namenode.MetaRecoveryContext.editLogLoaderPrompt(MetaRecoveryContext.java:94)
> at
> org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadEditRecords(FSEditLogLoader.java:239)
> at
> org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadFSEdits(FSEditLogLoader.java:161)
> at
> org.apache.hadoop.hdfs.server.namenode.FSImage.loadEdits(FSImage.java:895) at
> org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer.doTailEdits(EditLogTailer.java:321)
> at
> org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread.doWork(EditLogTailer.java:460)
> at
> org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread.access$400(EditLogTailer.java:410)
> at
> org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread$1.run(EditLogTailer.java:427)
> at
> org.apache.hadoop.security.SecurityUtil.doAsLoginUserOrFatal(SecurityUtil.java:414)
> at
> org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread.run(EditLogTailer.java:423)
> [2019-07-17T11:50:21.064+08:00] [INFO] [Edit log tailer] : Exiting with
> status 1 [2019-07-17T11:50:21.066+08:00] [INFO] [Thread-1] : SHUTDOWN_MSG:
> / SHUTDOWN_MSG:
> Shutting down NameNode at ip
> /
> {code}
>
> if dfs.ha.tail-edits.max-txns-per-lock value is 500,when the namenode load
> the editlog util 500,the current namenode will load the next editlog,but
> editlog more than 500.So,namenode got an unexpected txid when tail editlog.
>
>
> {code:java}
> //
> [2019-07-17T11:50:21.059+08:00] [INFO] [Edit log tailer] : Edits file
[jira] [Commented] (HDFS-14674) [SBN read] Got an unexpected txid when tail editlog
[
https://issues.apache.org/jira/browse/HDFS-14674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16903495#comment-16903495
]
wangzhaohui commented on HDFS-14674:
Hi,[~csun] [~shv],this patch v006 add unit test and fix checkstyle,pls take a
look,thanks!
> [SBN read] Got an unexpected txid when tail editlog
> ---
>
> Key: HDFS-14674
> URL: https://issues.apache.org/jira/browse/HDFS-14674
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: wangzhaohui
>Assignee: wangzhaohui
>Priority: Blocker
> Attachments: HDFS-14674-001.patch, HDFS-14674-003.patch,
> HDFS-14674-004.patch, HDFS-14674-005.patch, HDFS-14674-006.patch, image.png
>
>
> Add the following configuration
> !image-2019-07-26-11-34-23-405.png!
> error:
> {code:java}
> //
> [2019-07-17T11:50:21.048+08:00] [INFO] [Edit log tailer] : replaying edit
> log: 1/20512836 transactions completed. (0%) [2019-07-17T11:50:21.059+08:00]
> [INFO] [Edit log tailer] : Edits file
> http://ip/getJournal?jid=ns1003=232056426162=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?ipjid=ns1003=232056426162=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?ipjid=ns1003=232056426162=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH
> of size 3126782311 edits # 500 loaded in 3 seconds
> [2019-07-17T11:50:21.059+08:00] [INFO] [Edit log tailer] : Reading
> org.apache.hadoop.hdfs.server.namenode.RedundantEditLogInputStream@51ceb7bc
> expecting start txid #232056752162 [2019-07-17T11:50:21.059+08:00] [INFO]
> [Edit log tailer] : Start loading edits file
> http://ip/getJournal?ipjid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?ipjid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?ipjid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH
> maxTxnipsToRead = 500 [2019-07-17T11:50:21.059+08:00] [INFO] [Edit log
> tailer] : Fast-forwarding stream
> 'http://ip/getJournal?jid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?ipjid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?ipjid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH'
> to transaction ID 232056751662 [2019-07-17T11:50:21.059+08:00] [INFO] [Edit
> log tailer] ip: Fast-forwarding stream
> 'http://ip/getJournal?jid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH'
> to transaction ID 232056751662 [2019-07-17T11:50:21.061+08:00] [ERROR] [Edit
> log tailer] : Unknown error encountered while tailing edits. Shutting down
> standby NN. java.io.IOException: There appears to be a gap in the edit log.
> We expected txid 232056752162, but got txid 232077264498. at
> org.apache.hadoop.hdfs.server.namenode.MetaRecoveryContext.editLogLoaderPrompt(MetaRecoveryContext.java:94)
> at
> org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadEditRecords(FSEditLogLoader.java:239)
> at
> org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadFSEdits(FSEditLogLoader.java:161)
> at
> org.apache.hadoop.hdfs.server.namenode.FSImage.loadEdits(FSImage.java:895) at
> org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer.doTailEdits(EditLogTailer.java:321)
> at
> org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread.doWork(EditLogTailer.java:460)
> at
> org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread.access$400(EditLogTailer.java:410)
> at
> org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread$1.run(EditLogTailer.java:427)
> at
> org.apache.hadoop.security.SecurityUtil.doAsLoginUserOrFatal(SecurityUtil.java:414)
> at
> org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread.run(EditLogTailer.java:423)
> [2019-07-17T11:50:21.064+08:00] [INFO] [Edit log tailer] : Exiting with
> status 1 [2019-07-17T11:50:21.066+08:00] [INFO] [Thread-1] : SHUTDOWN_MSG:
> / SHUTDOWN_MSG:
> Shutting down NameNode at ip
> /
> {code}
>
> if dfs.ha.tail-edits.max-txns-per-lock value is 500,when the namenode load
> the editlog util 500,the current namenode will load the next editlog,but
> editlog more than 500.So,namenode got an unexpected txid when tail editlog.
>
>
> {code:java}
> //
> [2019-07-17T11:50:21.059+08:00] [INFO] [Edit log tailer] : Edits file
> http://ip/getJournal?jid=ns1003=232056426162=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
>
[jira] [Updated] (HDFS-14674) [SBN read] Got an unexpected txid when tail editlog
[
https://issues.apache.org/jira/browse/HDFS-14674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
wangzhaohui updated HDFS-14674:
---
Attachment: (was: image-2019-07-26-11-34-23-405.png)
> [SBN read] Got an unexpected txid when tail editlog
> ---
>
> Key: HDFS-14674
> URL: https://issues.apache.org/jira/browse/HDFS-14674
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: wangzhaohui
>Assignee: wangzhaohui
>Priority: Blocker
> Attachments: HDFS-14674-001.patch, HDFS-14674-003.patch,
> HDFS-14674-004.patch, HDFS-14674-005.patch, HDFS-14674-006.patch, image.png
>
>
> Add the following configuration
> !image-2019-07-26-11-34-23-405.png!
> error:
> {code:java}
> //
> [2019-07-17T11:50:21.048+08:00] [INFO] [Edit log tailer] : replaying edit
> log: 1/20512836 transactions completed. (0%) [2019-07-17T11:50:21.059+08:00]
> [INFO] [Edit log tailer] : Edits file
> http://ip/getJournal?jid=ns1003=232056426162=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?ipjid=ns1003=232056426162=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?ipjid=ns1003=232056426162=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH
> of size 3126782311 edits # 500 loaded in 3 seconds
> [2019-07-17T11:50:21.059+08:00] [INFO] [Edit log tailer] : Reading
> org.apache.hadoop.hdfs.server.namenode.RedundantEditLogInputStream@51ceb7bc
> expecting start txid #232056752162 [2019-07-17T11:50:21.059+08:00] [INFO]
> [Edit log tailer] : Start loading edits file
> http://ip/getJournal?ipjid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?ipjid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?ipjid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH
> maxTxnipsToRead = 500 [2019-07-17T11:50:21.059+08:00] [INFO] [Edit log
> tailer] : Fast-forwarding stream
> 'http://ip/getJournal?jid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?ipjid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?ipjid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH'
> to transaction ID 232056751662 [2019-07-17T11:50:21.059+08:00] [INFO] [Edit
> log tailer] ip: Fast-forwarding stream
> 'http://ip/getJournal?jid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH'
> to transaction ID 232056751662 [2019-07-17T11:50:21.061+08:00] [ERROR] [Edit
> log tailer] : Unknown error encountered while tailing edits. Shutting down
> standby NN. java.io.IOException: There appears to be a gap in the edit log.
> We expected txid 232056752162, but got txid 232077264498. at
> org.apache.hadoop.hdfs.server.namenode.MetaRecoveryContext.editLogLoaderPrompt(MetaRecoveryContext.java:94)
> at
> org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadEditRecords(FSEditLogLoader.java:239)
> at
> org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadFSEdits(FSEditLogLoader.java:161)
> at
> org.apache.hadoop.hdfs.server.namenode.FSImage.loadEdits(FSImage.java:895) at
> org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer.doTailEdits(EditLogTailer.java:321)
> at
> org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread.doWork(EditLogTailer.java:460)
> at
> org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread.access$400(EditLogTailer.java:410)
> at
> org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread$1.run(EditLogTailer.java:427)
> at
> org.apache.hadoop.security.SecurityUtil.doAsLoginUserOrFatal(SecurityUtil.java:414)
> at
> org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread.run(EditLogTailer.java:423)
> [2019-07-17T11:50:21.064+08:00] [INFO] [Edit log tailer] : Exiting with
> status 1 [2019-07-17T11:50:21.066+08:00] [INFO] [Thread-1] : SHUTDOWN_MSG:
> / SHUTDOWN_MSG:
> Shutting down NameNode at ip
> /
> {code}
>
> if dfs.ha.tail-edits.max-txns-per-lock value is 500,when the namenode load
> the editlog util 500,the current namenode will load the next editlog,but
> editlog more than 500.So,namenode got an unexpected txid when tail editlog.
>
>
> {code:java}
> //
> [2019-07-17T11:50:21.059+08:00] [INFO] [Edit log tailer] : Edits file
> http://ip/getJournal?jid=ns1003=232056426162=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?jid=ns1003=232056426162=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
>
[jira] [Updated] (HDFS-14674) [SBN read] Got an unexpected txid when tail editlog
[
https://issues.apache.org/jira/browse/HDFS-14674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
wangzhaohui updated HDFS-14674:
---
Attachment: HDFS-14674-006.patch
> [SBN read] Got an unexpected txid when tail editlog
> ---
>
> Key: HDFS-14674
> URL: https://issues.apache.org/jira/browse/HDFS-14674
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: wangzhaohui
>Assignee: wangzhaohui
>Priority: Blocker
> Attachments: HDFS-14674-001.patch, HDFS-14674-003.patch,
> HDFS-14674-004.patch, HDFS-14674-005.patch, HDFS-14674-006.patch,
> image-2019-07-26-11-34-23-405.png, image.png
>
>
> Add the following configuration
> !image-2019-07-26-11-34-23-405.png!
> error:
> {code:java}
> //
> [2019-07-17T11:50:21.048+08:00] [INFO] [Edit log tailer] : replaying edit
> log: 1/20512836 transactions completed. (0%) [2019-07-17T11:50:21.059+08:00]
> [INFO] [Edit log tailer] : Edits file
> http://ip/getJournal?jid=ns1003=232056426162=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?ipjid=ns1003=232056426162=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?ipjid=ns1003=232056426162=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH
> of size 3126782311 edits # 500 loaded in 3 seconds
> [2019-07-17T11:50:21.059+08:00] [INFO] [Edit log tailer] : Reading
> org.apache.hadoop.hdfs.server.namenode.RedundantEditLogInputStream@51ceb7bc
> expecting start txid #232056752162 [2019-07-17T11:50:21.059+08:00] [INFO]
> [Edit log tailer] : Start loading edits file
> http://ip/getJournal?ipjid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?ipjid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?ipjid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH
> maxTxnipsToRead = 500 [2019-07-17T11:50:21.059+08:00] [INFO] [Edit log
> tailer] : Fast-forwarding stream
> 'http://ip/getJournal?jid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?ipjid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?ipjid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH'
> to transaction ID 232056751662 [2019-07-17T11:50:21.059+08:00] [INFO] [Edit
> log tailer] ip: Fast-forwarding stream
> 'http://ip/getJournal?jid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH'
> to transaction ID 232056751662 [2019-07-17T11:50:21.061+08:00] [ERROR] [Edit
> log tailer] : Unknown error encountered while tailing edits. Shutting down
> standby NN. java.io.IOException: There appears to be a gap in the edit log.
> We expected txid 232056752162, but got txid 232077264498. at
> org.apache.hadoop.hdfs.server.namenode.MetaRecoveryContext.editLogLoaderPrompt(MetaRecoveryContext.java:94)
> at
> org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadEditRecords(FSEditLogLoader.java:239)
> at
> org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadFSEdits(FSEditLogLoader.java:161)
> at
> org.apache.hadoop.hdfs.server.namenode.FSImage.loadEdits(FSImage.java:895) at
> org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer.doTailEdits(EditLogTailer.java:321)
> at
> org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread.doWork(EditLogTailer.java:460)
> at
> org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread.access$400(EditLogTailer.java:410)
> at
> org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread$1.run(EditLogTailer.java:427)
> at
> org.apache.hadoop.security.SecurityUtil.doAsLoginUserOrFatal(SecurityUtil.java:414)
> at
> org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread.run(EditLogTailer.java:423)
> [2019-07-17T11:50:21.064+08:00] [INFO] [Edit log tailer] : Exiting with
> status 1 [2019-07-17T11:50:21.066+08:00] [INFO] [Thread-1] : SHUTDOWN_MSG:
> / SHUTDOWN_MSG:
> Shutting down NameNode at ip
> /
> {code}
>
> if dfs.ha.tail-edits.max-txns-per-lock value is 500,when the namenode load
> the editlog util 500,the current namenode will load the next editlog,but
> editlog more than 500.So,namenode got an unexpected txid when tail editlog.
>
>
> {code:java}
> //
> [2019-07-17T11:50:21.059+08:00] [INFO] [Edit log tailer] : Edits file
> http://ip/getJournal?jid=ns1003=232056426162=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?jid=ns1003=232056426162=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
>
[jira] [Updated] (HDFS-14674) [SBN read] Got an unexpected txid when tail editlog
[
https://issues.apache.org/jira/browse/HDFS-14674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
wangzhaohui updated HDFS-14674:
---
Attachment: HDFS-14674-005.patch
> [SBN read] Got an unexpected txid when tail editlog
> ---
>
> Key: HDFS-14674
> URL: https://issues.apache.org/jira/browse/HDFS-14674
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: wangzhaohui
>Assignee: wangzhaohui
>Priority: Blocker
> Attachments: HDFS-14674-001.patch, HDFS-14674-003.patch,
> HDFS-14674-004.patch, HDFS-14674-005.patch, image-2019-07-26-11-34-23-405.png
>
>
> Add the following configuration
> !image-2019-07-26-11-34-23-405.png!
> error:
> {code:java}
> //
> [2019-07-17T11:50:21.048+08:00] [INFO] [Edit log tailer] : replaying edit
> log: 1/20512836 transactions completed. (0%) [2019-07-17T11:50:21.059+08:00]
> [INFO] [Edit log tailer] : Edits file
> http://ip/getJournal?jid=ns1003=232056426162=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?ipjid=ns1003=232056426162=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?ipjid=ns1003=232056426162=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH
> of size 3126782311 edits # 500 loaded in 3 seconds
> [2019-07-17T11:50:21.059+08:00] [INFO] [Edit log tailer] : Reading
> org.apache.hadoop.hdfs.server.namenode.RedundantEditLogInputStream@51ceb7bc
> expecting start txid #232056752162 [2019-07-17T11:50:21.059+08:00] [INFO]
> [Edit log tailer] : Start loading edits file
> http://ip/getJournal?ipjid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?ipjid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?ipjid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH
> maxTxnipsToRead = 500 [2019-07-17T11:50:21.059+08:00] [INFO] [Edit log
> tailer] : Fast-forwarding stream
> 'http://ip/getJournal?jid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?ipjid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?ipjid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH'
> to transaction ID 232056751662 [2019-07-17T11:50:21.059+08:00] [INFO] [Edit
> log tailer] ip: Fast-forwarding stream
> 'http://ip/getJournal?jid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH'
> to transaction ID 232056751662 [2019-07-17T11:50:21.061+08:00] [ERROR] [Edit
> log tailer] : Unknown error encountered while tailing edits. Shutting down
> standby NN. java.io.IOException: There appears to be a gap in the edit log.
> We expected txid 232056752162, but got txid 232077264498. at
> org.apache.hadoop.hdfs.server.namenode.MetaRecoveryContext.editLogLoaderPrompt(MetaRecoveryContext.java:94)
> at
> org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadEditRecords(FSEditLogLoader.java:239)
> at
> org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadFSEdits(FSEditLogLoader.java:161)
> at
> org.apache.hadoop.hdfs.server.namenode.FSImage.loadEdits(FSImage.java:895) at
> org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer.doTailEdits(EditLogTailer.java:321)
> at
> org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread.doWork(EditLogTailer.java:460)
> at
> org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread.access$400(EditLogTailer.java:410)
> at
> org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread$1.run(EditLogTailer.java:427)
> at
> org.apache.hadoop.security.SecurityUtil.doAsLoginUserOrFatal(SecurityUtil.java:414)
> at
> org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread.run(EditLogTailer.java:423)
> [2019-07-17T11:50:21.064+08:00] [INFO] [Edit log tailer] : Exiting with
> status 1 [2019-07-17T11:50:21.066+08:00] [INFO] [Thread-1] : SHUTDOWN_MSG:
> / SHUTDOWN_MSG:
> Shutting down NameNode at ip
> /
> {code}
>
> if dfs.ha.tail-edits.max-txns-per-lock value is 500,when the namenode load
> the editlog util 500,the current namenode will load the next editlog,but
> editlog more than 500.So,namenode got an unexpected txid when tail editlog.
>
>
> {code:java}
> //
> [2019-07-17T11:50:21.059+08:00] [INFO] [Edit log tailer] : Edits file
> http://ip/getJournal?jid=ns1003=232056426162=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?jid=ns1003=232056426162=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
>
[jira] [Updated] (HDFS-14674) [SBN read] Got an unexpected txid when tail editlog
[
https://issues.apache.org/jira/browse/HDFS-14674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
wangzhaohui updated HDFS-14674:
---
Attachment: (was: HDFS-14674-004.patch)
> [SBN read] Got an unexpected txid when tail editlog
> ---
>
> Key: HDFS-14674
> URL: https://issues.apache.org/jira/browse/HDFS-14674
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: wangzhaohui
>Assignee: wangzhaohui
>Priority: Blocker
> Attachments: HDFS-14674-001.patch, HDFS-14674-003.patch,
> HDFS-14674-004.patch, image-2019-07-26-11-34-23-405.png
>
>
> Add the following configuration
> !image-2019-07-26-11-34-23-405.png!
> error:
> {code:java}
> //
> [2019-07-17T11:50:21.048+08:00] [INFO] [Edit log tailer] : replaying edit
> log: 1/20512836 transactions completed. (0%) [2019-07-17T11:50:21.059+08:00]
> [INFO] [Edit log tailer] : Edits file
> http://ip/getJournal?jid=ns1003=232056426162=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?ipjid=ns1003=232056426162=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?ipjid=ns1003=232056426162=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH
> of size 3126782311 edits # 500 loaded in 3 seconds
> [2019-07-17T11:50:21.059+08:00] [INFO] [Edit log tailer] : Reading
> org.apache.hadoop.hdfs.server.namenode.RedundantEditLogInputStream@51ceb7bc
> expecting start txid #232056752162 [2019-07-17T11:50:21.059+08:00] [INFO]
> [Edit log tailer] : Start loading edits file
> http://ip/getJournal?ipjid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?ipjid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?ipjid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH
> maxTxnipsToRead = 500 [2019-07-17T11:50:21.059+08:00] [INFO] [Edit log
> tailer] : Fast-forwarding stream
> 'http://ip/getJournal?jid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?ipjid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?ipjid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH'
> to transaction ID 232056751662 [2019-07-17T11:50:21.059+08:00] [INFO] [Edit
> log tailer] ip: Fast-forwarding stream
> 'http://ip/getJournal?jid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH'
> to transaction ID 232056751662 [2019-07-17T11:50:21.061+08:00] [ERROR] [Edit
> log tailer] : Unknown error encountered while tailing edits. Shutting down
> standby NN. java.io.IOException: There appears to be a gap in the edit log.
> We expected txid 232056752162, but got txid 232077264498. at
> org.apache.hadoop.hdfs.server.namenode.MetaRecoveryContext.editLogLoaderPrompt(MetaRecoveryContext.java:94)
> at
> org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadEditRecords(FSEditLogLoader.java:239)
> at
> org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadFSEdits(FSEditLogLoader.java:161)
> at
> org.apache.hadoop.hdfs.server.namenode.FSImage.loadEdits(FSImage.java:895) at
> org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer.doTailEdits(EditLogTailer.java:321)
> at
> org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread.doWork(EditLogTailer.java:460)
> at
> org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread.access$400(EditLogTailer.java:410)
> at
> org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread$1.run(EditLogTailer.java:427)
> at
> org.apache.hadoop.security.SecurityUtil.doAsLoginUserOrFatal(SecurityUtil.java:414)
> at
> org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread.run(EditLogTailer.java:423)
> [2019-07-17T11:50:21.064+08:00] [INFO] [Edit log tailer] : Exiting with
> status 1 [2019-07-17T11:50:21.066+08:00] [INFO] [Thread-1] : SHUTDOWN_MSG:
> / SHUTDOWN_MSG:
> Shutting down NameNode at ip
> /
> {code}
>
> if dfs.ha.tail-edits.max-txns-per-lock value is 500,when the namenode load
> the editlog util 500,the current namenode will load the next editlog,but
> editlog more than 500.So,namenode got an unexpected txid when tail editlog.
>
>
> {code:java}
> //
> [2019-07-17T11:50:21.059+08:00] [INFO] [Edit log tailer] : Edits file
> http://ip/getJournal?jid=ns1003=232056426162=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?jid=ns1003=232056426162=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
>
[jira] [Updated] (HDFS-14674) [SBN read] Got an unexpected txid when tail editlog
[
https://issues.apache.org/jira/browse/HDFS-14674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
wangzhaohui updated HDFS-14674:
---
Attachment: HDFS-14674-004.patch
> [SBN read] Got an unexpected txid when tail editlog
> ---
>
> Key: HDFS-14674
> URL: https://issues.apache.org/jira/browse/HDFS-14674
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: wangzhaohui
>Assignee: wangzhaohui
>Priority: Blocker
> Attachments: HDFS-14674-001.patch, HDFS-14674-003.patch,
> HDFS-14674-004.patch, image-2019-07-26-11-34-23-405.png
>
>
> Add the following configuration
> !image-2019-07-26-11-34-23-405.png!
> error:
> {code:java}
> //
> [2019-07-17T11:50:21.048+08:00] [INFO] [Edit log tailer] : replaying edit
> log: 1/20512836 transactions completed. (0%) [2019-07-17T11:50:21.059+08:00]
> [INFO] [Edit log tailer] : Edits file
> http://ip/getJournal?jid=ns1003=232056426162=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?ipjid=ns1003=232056426162=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?ipjid=ns1003=232056426162=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH
> of size 3126782311 edits # 500 loaded in 3 seconds
> [2019-07-17T11:50:21.059+08:00] [INFO] [Edit log tailer] : Reading
> org.apache.hadoop.hdfs.server.namenode.RedundantEditLogInputStream@51ceb7bc
> expecting start txid #232056752162 [2019-07-17T11:50:21.059+08:00] [INFO]
> [Edit log tailer] : Start loading edits file
> http://ip/getJournal?ipjid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?ipjid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?ipjid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH
> maxTxnipsToRead = 500 [2019-07-17T11:50:21.059+08:00] [INFO] [Edit log
> tailer] : Fast-forwarding stream
> 'http://ip/getJournal?jid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?ipjid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?ipjid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH'
> to transaction ID 232056751662 [2019-07-17T11:50:21.059+08:00] [INFO] [Edit
> log tailer] ip: Fast-forwarding stream
> 'http://ip/getJournal?jid=ns1003=232077264498=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH'
> to transaction ID 232056751662 [2019-07-17T11:50:21.061+08:00] [ERROR] [Edit
> log tailer] : Unknown error encountered while tailing edits. Shutting down
> standby NN. java.io.IOException: There appears to be a gap in the edit log.
> We expected txid 232056752162, but got txid 232077264498. at
> org.apache.hadoop.hdfs.server.namenode.MetaRecoveryContext.editLogLoaderPrompt(MetaRecoveryContext.java:94)
> at
> org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadEditRecords(FSEditLogLoader.java:239)
> at
> org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadFSEdits(FSEditLogLoader.java:161)
> at
> org.apache.hadoop.hdfs.server.namenode.FSImage.loadEdits(FSImage.java:895) at
> org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer.doTailEdits(EditLogTailer.java:321)
> at
> org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread.doWork(EditLogTailer.java:460)
> at
> org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread.access$400(EditLogTailer.java:410)
> at
> org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread$1.run(EditLogTailer.java:427)
> at
> org.apache.hadoop.security.SecurityUtil.doAsLoginUserOrFatal(SecurityUtil.java:414)
> at
> org.apache.hadoop.hdfs.server.namenode.ha.EditLogTailer$EditLogTailerThread.run(EditLogTailer.java:423)
> [2019-07-17T11:50:21.064+08:00] [INFO] [Edit log tailer] : Exiting with
> status 1 [2019-07-17T11:50:21.066+08:00] [INFO] [Thread-1] : SHUTDOWN_MSG:
> / SHUTDOWN_MSG:
> Shutting down NameNode at ip
> /
> {code}
>
> if dfs.ha.tail-edits.max-txns-per-lock value is 500,when the namenode load
> the editlog util 500,the current namenode will load the next editlog,but
> editlog more than 500.So,namenode got an unexpected txid when tail editlog.
>
>
> {code:java}
> //
> [2019-07-17T11:50:21.059+08:00] [INFO] [Edit log tailer] : Edits file
> http://ip/getJournal?jid=ns1003=232056426162=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
> http://ip/getJournal?jid=ns1003=232056426162=-63%3A1902204348%3A0%3ACID-hope-20180214-20161018-SQYH,
>
>
