[
https://issues.apache.org/jira/browse/RATIS-502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16796103#comment-16796103
]
Hadoop QA commented on RATIS-502:
---------------------------------
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 1m
32s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
44s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
45s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
18s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
41s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
34s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 14m 12s{color}
| {color:red} root in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
19s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 23m 7s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | ratis.examples.filestore.TestFileStoreWithNetty |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/ratis:date2019-03-19
|
| JIRA Issue | RATIS-502 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12962981/r502_20190319b.patch |
| Optional Tests | dupname asflicense javac javadoc unit findbugs
checkstyle compile |
| uname | Linux 47d9bee02ed8 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2
17:16:02 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-RATIS-Build/yetus-personality.sh
|
| git revision | master / 74b491c |
| maven | version: Apache Maven 3.6.0
(97c98ec64a1fdfee7767ce5ffb20918da4f719f3; 2018-10-24T18:41:47Z) |
| Default Java | 1.8.0_191 |
| unit |
https://builds.apache.org/job/PreCommit-RATIS-Build/693/artifact/out/patch-unit-root.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-RATIS-Build/693/testReport/ |
| Max. process+thread count | 4313 (vs. ulimit of 5000) |
| modules | C: ratis-server U: ratis-server |
| Console output |
https://builds.apache.org/job/PreCommit-RATIS-Build/693/console |
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was automatically generated.
> Commit Index less than the snapshot's commit indexes need to be ignored on
> restart
> ----------------------------------------------------------------------------------
>
> Key: RATIS-502
> URL: https://issues.apache.org/jira/browse/RATIS-502
> Project: Ratis
> Issue Type: Bug
> Components: server
> Reporter: Mukul Kumar Singh
> Assignee: Tsz Wo Nicholas Sze
> Priority: Major
> Attachments: r502_20190319.patch, r502_20190319b.patch
>
>
> This problem was seen with Ozone in datanodes state machine.
> Before restart of the datanode, a snapshot was taken at log index 6. Please
> note that the commit entry for this will be after this log index in the Raft
> Log.
> {code}
> 2019-03-19 08:56:30,225 INFO impl.StateMachineUpdater
> (StateMachineUpdater.java:stopAndJoin(109)) -
> StateMachineUpdater-4c165953-147b-48fb-89e1-951579e828eb: set stopIndex = 6
> 2019-03-19 08:56:30,225 INFO ratis.ContainerStateMachine
> (ContainerStateMachine.java:takeSnapshot(245)) - Taking snapshot at
> termIndex:(t:1, i:6)
> 2019-03-19 08:56:30,226 INFO ratis.ContainerStateMachine
> (ContainerStateMachine.java:takeSnapshot(249)) - Taking a snapshot to file
> /Users/msingh/code/apache/ozone/oz_new1/hadoop-ozone/integration-test/t
> arget/test-dir/MiniOzoneClusterImpl-0fa66624-f533-44bc-8f6f-99cf251fe4c3/datanode-0/data/ratis/34393916-e10d-4b3c-b212-5c910eea4935/sm/snapshot.1_6
> 2019-03-19 08:56:30,231 INFO impl.RaftServerImpl
> (ServerState.java:close(386)) - 4c165953-147b-48fb-89e1-951579e828eb closes.
> The last applied log index is 6
> {code}
> After restart, the state machine register 6 as the log index in the snapshot.
> {code}
> 2019-03-19 08:56:33,351 INFO ratis.ContainerStateMachine
> (ContainerStateMachine.java:loadSnapshot(209)) - Setting the last applied
> index to (t:1, i:6)
> {code}
> Now, after applying all the transactions after the snapshot, the state
> machine will encounter a commit entry for this log index (5). this hits the
> assert in the state machine.
> {code}
> java.io.IOException: java.lang.IllegalStateException: Failed to
> updateIncreasingly for commitIndex: 6 -> 5
> at org.apache.ratis.util.IOUtils.asIOException(IOUtils.java:54)
> at org.apache.ratis.util.IOUtils.toIOException(IOUtils.java:61)
> at org.apache.ratis.util.IOUtils.getFromFuture(IOUtils.java:70)
> at
> org.apache.ratis.server.impl.RaftServerProxy.getImpls(RaftServerProxy.java:283)
> at
> org.apache.ratis.server.impl.RaftServerProxy.start(RaftServerProxy.java:295)
> at
> org.apache.hadoop.ozone.container.common.transport.server.ratis.XceiverServerRatis.start(XceiverServerRatis.java:417)
> at
> org.apache.hadoop.ozone.container.ozoneimpl.OzoneContainer.start(OzoneContainer.java:182)
> at
> org.apache.hadoop.ozone.container.common.statemachine.DatanodeStateMachine.start(DatanodeStateMachine.java:165)
> at
> org.apache.hadoop.ozone.container.common.statemachine.DatanodeStateMachine.lambda$startDaemon$0(DatanodeStateMachine.java:334)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.lang.IllegalStateException: Failed to updateIncreasingly for
> commitIndex: 6 -> 5
> at
> org.apache.ratis.util.Preconditions.assertTrue(Preconditions.java:72)
> at
> org.apache.ratis.server.storage.RaftLogIndex.updateIncreasingly(RaftLogIndex.java:60)
> at
> org.apache.ratis.server.storage.RaftLog.lambda$open$7(RaftLog.java:245)
> at java.util.Optional.ifPresent(Optional.java:159)
> at org.apache.ratis.server.storage.RaftLog.open(RaftLog.java:244)
> at
> org.apache.ratis.server.impl.ServerState.initLog(ServerState.java:191)
> at
> org.apache.ratis.server.impl.ServerState.<init>(ServerState.java:114)
> at
> org.apache.ratis.server.impl.RaftServerImpl.<init>(RaftServerImpl.java:103)
> at
> org.apache.ratis.server.impl.RaftServerProxy.lambda$newRaftServerImpl$2(RaftServerProxy.java:207)
> at
> java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1590)
> at
> java.util.concurrent.CompletableFuture$AsyncSupply.exec(CompletableFuture.java:1582)
> at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
> at
> java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
> at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
> at
> java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
