[jira] [Commented] (RATIS-406) In RaftServerImpl, the RaftLog.append(entries) call should not hold the RaftServerImpl lock

2018-11-13 Thread Jitendra Nath Pandey (JIRA)


[ 
https://issues.apache.org/jira/browse/RATIS-406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16686171#comment-16686171
 ] 

Jitendra Nath Pandey commented on RATIS-406:


# Although {{runSequentially}} can be called by different threads at 
non-overlapping times, it will be really error-prone for a thread to attempt 
it, as it might throw a runtime exception. The caller must be sure that either 
it is the same thread or another thread is certainly not executing. Would it be 
simpler to either force only one thread can call it, or just use a dedicated 
lock instead of using RaftServerImpl lock?
 # {{RaftLog#appendImpl}} could be a private or package private like other 
"Impl" methods.

> In RaftServerImpl, the RaftLog.append(entries) call should not hold the 
> RaftServerImpl lock
> ---
>
> Key: RATIS-406
> URL: https://issues.apache.org/jira/browse/RATIS-406
> Project: Ratis
>  Issue Type: Improvement
>  Components: server
>Reporter: Tsz Wo Nicholas Sze
>Assignee: Tsz Wo Nicholas Sze
>Priority: Major
> Attachments: r406_20181109.patch, r406_20181113.patch
>
>
> In RaftServerImpl, the appendEntriesAsync(..) calls must be sequential 
> (although the actual log I/O is async).  Otherwise, the entries appended may 
> be out of order.
> As a result, RaftLog.append(entries) needs not to hold the RaftServerImpl 
> lock.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (RATIS-131) Add checkstyle configuration

2018-11-13 Thread Hadoop QA (JIRA)


[ 
https://issues.apache.org/jira/browse/RATIS-131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16686020#comment-16686020
 ] 

Hadoop QA commented on RATIS-131:
-

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
12s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red}  0m  
0s{color} | {color:red} The patch doesn't appear to include any new or modified 
tests. Please justify why no new tests are needed for this patch. Also please 
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  2m 
51s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
48s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
42s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  0m 
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green}  0m  
3s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
32s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 10m 27s{color} 
| {color:red} root in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
20s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 17m 39s{color} | 
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | ratis.grpc.TestWatchRequestWithGrpc |
|   | ratis.netty.TestServerRestartWithNetty |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/ratis:date2018-11-14 
|
| JIRA Issue | RATIS-131 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12948058/r131_20181113b.patch |
| Optional Tests |  dupname  asflicense  xml  javac  javadoc  unit  compile  |
| uname | Linux d7f96f2ff253 3.13.0-143-generic #192-Ubuntu SMP Tue Feb 27 
10:45:36 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/home/jenkins/jenkins-slave/workspace/PreCommit-RATIS-Build/yetus-personality.sh
 |
| git revision | master / 5c37675 |
| maven | version: Apache Maven 3.6.0 
(97c98ec64a1fdfee7767ce5ffb20918da4f719f3; 2018-10-24T18:41:47Z) |
| Default Java | 1.8.0_181 |
| unit | 
https://builds.apache.org/job/PreCommit-RATIS-Build/520/artifact/out/patch-unit-root.txt
 |
|  Test Results | 
https://builds.apache.org/job/PreCommit-RATIS-Build/520/testReport/ |
| Max. process+thread count | 2878 (vs. ulimit of 5000) |
| modules | C: . U: . |
| Console output | 
https://builds.apache.org/job/PreCommit-RATIS-Build/520/console |
| Powered by | Apache Yetus 0.8.0   http://yetus.apache.org |


This message was automatically generated.



> Add checkstyle configuration
> 
>
> Key: RATIS-131
> URL: https://issues.apache.org/jira/browse/RATIS-131
> Project: Ratis
>  Issue Type: Bug
>  Components: build
>Reporter: Tsz Wo Nicholas Sze
>Assignee: Tsz Wo Nicholas Sze
>Priority: Major
> Attachments: r131_20181113.patch, r131_20181113b.patch
>
>
> This JIRA is to setup our checkstyle coding conventions.  We probably should 
> start with [Sun coding 
> conventions|https://github.com/checkstyle/checkstyle/blob/master/src/main/resources/sun_checks.xml]
>  or [Google coding 
> conventions|https://github.com/checkstyle/checkstyle/blob/master/src/main/resources/google_checks.xml].



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (RATIS-409) Watch requests may not work if there is a conf change

2018-11-13 Thread Hadoop QA (JIRA)


[ 
https://issues.apache.org/jira/browse/RATIS-409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16686008#comment-16686008
 ] 

Hadoop QA commented on RATIS-409:
-

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  1m 
33s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m  
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 2 new or modified test 
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  2m 
50s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
45s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
32s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
43s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  0m 
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
45s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}  
0m 25s{color} | {color:orange} root: The patch generated 27 new + 716 unchanged 
- 11 fixed = 743 total (was 727) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
31s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 21m 28s{color} 
| {color:red} root in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
16s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 30m 48s{color} | 
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | ratis.examples.filestore.TestFileStoreWithNetty |
|   | ratis.server.simulation.TestRaftStateMachineExceptionWithSimulatedRpc |
|   | ratis.grpc.TestWatchRequestWithGrpc |
|   | ratis.server.simulation.TestRaftReconfigurationWithSimulatedRpc |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/ratis:date2018-11-14 
|
| JIRA Issue | RATIS-409 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12948038/r409_20181113b.patch |
| Optional Tests |  dupname  asflicense  javac  javadoc  unit  findbugs  
checkstyle  compile  |
| uname | Linux 85f8493335c7 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 
17:16:02 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/home/jenkins/jenkins-slave/workspace/PreCommit-RATIS-Build/yetus-personality.sh
 |
| git revision | master / 5c37675 |
| maven | version: Apache Maven 3.6.0 
(97c98ec64a1fdfee7767ce5ffb20918da4f719f3; 2018-10-24T18:41:47Z) |
| Default Java | 1.8.0_181 |
| checkstyle | 
https://builds.apache.org/job/PreCommit-RATIS-Build/518/artifact/out/diff-checkstyle-root.txt
 |
| unit | 
https://builds.apache.org/job/PreCommit-RATIS-Build/518/artifact/out/patch-unit-root.txt
 |
|  Test Results | 
https://builds.apache.org/job/PreCommit-RATIS-Build/518/testReport/ |
| Max. process+thread count | 4837 (vs. ulimit of 5000) |
| modules | C: ratis-server U: ratis-server |
| Console output | 
https://builds.apache.org/job/PreCommit-RATIS-Build/518/console |
| Powered by | Apache Yetus 0.8.0   http://yetus.apache.org |


This message was automatically generated.



> Watch requests may not work if there is a conf change
> -
>
> Key: RATIS-409
> URL: https://issues.apache.org/jira/browse/RATIS-409
> Project: Ratis
>  Issue Type: Bug
>  Components: server
>Reporter: Tsz Wo Nicholas Sze
>Assignee: 

[jira] [Updated] (RATIS-131) Add checkstyle configuration

2018-11-13 Thread Tsz Wo Nicholas Sze (JIRA)


 [ 
https://issues.apache.org/jira/browse/RATIS-131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tsz Wo Nicholas Sze updated RATIS-131:
--
Attachment: r131_20181113b.patch

> Add checkstyle configuration
> 
>
> Key: RATIS-131
> URL: https://issues.apache.org/jira/browse/RATIS-131
> Project: Ratis
>  Issue Type: Bug
>  Components: build
>Reporter: Tsz Wo Nicholas Sze
>Assignee: Tsz Wo Nicholas Sze
>Priority: Major
> Attachments: r131_20181113.patch, r131_20181113b.patch
>
>
> This JIRA is to setup our checkstyle coding conventions.  We probably should 
> start with [Sun coding 
> conventions|https://github.com/checkstyle/checkstyle/blob/master/src/main/resources/sun_checks.xml]
>  or [Google coding 
> conventions|https://github.com/checkstyle/checkstyle/blob/master/src/main/resources/google_checks.xml].



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Assigned] (RATIS-131) Add checkstyle configuration

2018-11-13 Thread Tsz Wo Nicholas Sze (JIRA)


 [ 
https://issues.apache.org/jira/browse/RATIS-131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tsz Wo Nicholas Sze reassigned RATIS-131:
-

Assignee: Tsz Wo Nicholas Sze

> Add checkstyle configuration
> 
>
> Key: RATIS-131
> URL: https://issues.apache.org/jira/browse/RATIS-131
> Project: Ratis
>  Issue Type: Bug
>  Components: build
>Reporter: Tsz Wo Nicholas Sze
>Assignee: Tsz Wo Nicholas Sze
>Priority: Major
> Attachments: r131_20181113.patch
>
>
> This JIRA is to setup our checkstyle coding conventions.  We probably should 
> start with [Sun coding 
> conventions|https://github.com/checkstyle/checkstyle/blob/master/src/main/resources/sun_checks.xml]
>  or [Google coding 
> conventions|https://github.com/checkstyle/checkstyle/blob/master/src/main/resources/google_checks.xml].



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (RATIS-131) Add checkstyle configuration

2018-11-13 Thread Tsz Wo Nicholas Sze (JIRA)


 [ 
https://issues.apache.org/jira/browse/RATIS-131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tsz Wo Nicholas Sze updated RATIS-131:
--
Attachment: r131_20181113.patch

> Add checkstyle configuration
> 
>
> Key: RATIS-131
> URL: https://issues.apache.org/jira/browse/RATIS-131
> Project: Ratis
>  Issue Type: Bug
>  Components: build
>Reporter: Tsz Wo Nicholas Sze
>Priority: Major
> Attachments: r131_20181113.patch
>
>
> This JIRA is to setup our checkstyle coding conventions.  We probably should 
> start with [Sun coding 
> conventions|https://github.com/checkstyle/checkstyle/blob/master/src/main/resources/sun_checks.xml]
>  or [Google coding 
> conventions|https://github.com/checkstyle/checkstyle/blob/master/src/main/resources/google_checks.xml].



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (RATIS-409) Watch requests may not work if there is a conf change

2018-11-13 Thread Hadoop QA (JIRA)


[ 
https://issues.apache.org/jira/browse/RATIS-409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16685928#comment-16685928
 ] 

Hadoop QA commented on RATIS-409:
-

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
17s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m  
1s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 2 new or modified test 
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  2m 
54s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
48s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
35s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
47s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  0m 
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
51s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}  
0m 29s{color} | {color:orange} root: The patch generated 27 new + 716 unchanged 
- 11 fixed = 743 total (was 727) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
33s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 19m 27s{color} 
| {color:red} root in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
18s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 28m  5s{color} | 
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | ratis.grpc.TestRaftStateMachineExceptionWithGrpc |
|   | ratis.netty.TestRaftStateMachineExceptionWithNetty |
|   | ratis.netty.TestRaftReconfigurationWithNetty |
|   | ratis.server.simulation.TestRaftReconfigurationWithSimulatedRpc |
|   | ratis.server.simulation.TestRaftStateMachineExceptionWithSimulatedRpc |
|   | ratis.server.simulation.TestRaftWithSimulatedRpc |
|   | ratis.grpc.TestRaftAsyncWithGrpc |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/ratis:date2018-11-13 
|
| JIRA Issue | RATIS-409 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12948038/r409_20181113b.patch |
| Optional Tests |  dupname  asflicense  javac  javadoc  unit  findbugs  
checkstyle  compile  |
| uname | Linux 6a51bf26b959 3.13.0-153-generic #203-Ubuntu SMP Thu Jun 14 
08:52:28 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/home/jenkins/jenkins-slave/workspace/PreCommit-RATIS-Build/yetus-personality.sh
 |
| git revision | master / 5c37675 |
| maven | version: Apache Maven 3.6.0 
(97c98ec64a1fdfee7767ce5ffb20918da4f719f3; 2018-10-24T18:41:47Z) |
| Default Java | 1.8.0_181 |
| checkstyle | 
https://builds.apache.org/job/PreCommit-RATIS-Build/515/artifact/out/diff-checkstyle-root.txt
 |
| unit | 
https://builds.apache.org/job/PreCommit-RATIS-Build/515/artifact/out/patch-unit-root.txt
 |
|  Test Results | 
https://builds.apache.org/job/PreCommit-RATIS-Build/515/testReport/ |
| Max. process+thread count | 2693 (vs. ulimit of 5000) |
| modules | C: ratis-server U: ratis-server |
| Console output | 
https://builds.apache.org/job/PreCommit-RATIS-Build/515/console |
| Powered by | Apache Yetus 0.8.0   http://yetus.apache.org |


This message was automatically generated.



> Watch requests may not work if there is a conf change
> -
>
> Key: RATIS-409
> URL: 

[jira] [Commented] (RATIS-352) Persist commit index in Raft log.

2018-11-13 Thread Hadoop QA (JIRA)


[ 
https://issues.apache.org/jira/browse/RATIS-352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16685927#comment-16685927
 ] 

Hadoop QA commented on RATIS-352:
-

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  1m 
18s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m  
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 6 new or modified test 
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  1m 
22s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  1m 
59s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
56s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
38s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
42s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m  
6s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  1m 
 3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green}  0m 
57s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red}  0m 57s{color} 
| {color:red} root generated 3 new + 23 unchanged - 0 fixed = 26 total (was 23) 
{color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}  
0m 35s{color} | {color:orange} root: The patch generated 65 new + 1152 
unchanged - 49 fixed = 1217 total (was 1201) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
40s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 15m  8s{color} 
| {color:red} root in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
18s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 25m 55s{color} | 
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | ratis.logservice.server.TestMetaServer |
|   | ratis.grpc.TestRaftWithGrpc |
|   | ratis.server.simulation.TestServerRestartWithSimulatedRpc |
|   | ratis.grpc.TestServerRestartWithGrpc |
|   | ratis.netty.TestServerRestartWithNetty |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/ratis:date2018-11-13 
|
| JIRA Issue | RATIS-352 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12947292/r352_20181107.patch |
| Optional Tests |  dupname  asflicense  javac  javadoc  unit  findbugs  
checkstyle  compile  cc  |
| uname | Linux 05deefdb4798 4.4.0-134-generic #160~14.04.1-Ubuntu SMP Fri Aug 
17 11:07:07 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/home/jenkins/jenkins-slave/workspace/PreCommit-RATIS-Build/yetus-personality.sh
 |
| git revision | master / 5c37675 |
| maven | version: Apache Maven 3.6.0 
(97c98ec64a1fdfee7767ce5ffb20918da4f719f3; 2018-10-24T18:41:47Z) |
| Default Java | 1.8.0_181 |
| javac | 
https://builds.apache.org/job/PreCommit-RATIS-Build/516/artifact/out/diff-compile-javac-root.txt
 |
| checkstyle | 
https://builds.apache.org/job/PreCommit-RATIS-Build/516/artifact/out/diff-checkstyle-root.txt
 |
| unit | 
https://builds.apache.org/job/PreCommit-RATIS-Build/516/artifact/out/patch-unit-root.txt
 |
|  Test Results | 
https://builds.apache.org/job/PreCommit-RATIS-Build/516/testReport/ |
| Max. process+thread count | 2115 (vs. ulimit of 5000) |
| modules | C: ratis-proto 

[jira] [Commented] (RATIS-406) In RaftServerImpl, the RaftLog.append(entries) call should not hold the RaftServerImpl lock

2018-11-13 Thread Hadoop QA (JIRA)


[ 
https://issues.apache.org/jira/browse/RATIS-406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16685926#comment-16685926
 ] 

Hadoop QA commented on RATIS-406:
-

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
11s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m  
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red}  0m  
0s{color} | {color:red} The patch doesn't appear to include any new or modified 
tests. Please justify why no new tests are needed for this patch. Also please 
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  2m 
58s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
56s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
33s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
46s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  1m 
 2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
56s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}  
0m 28s{color} | {color:orange} root: The patch generated 75 new + 584 unchanged 
- 40 fixed = 659 total (was 624) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
38s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 11m 12s{color} 
| {color:red} root in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
17s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 20m  8s{color} | 
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | ratis.grpc.TestWatchRequestWithGrpc |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/ratis:date2018-11-13 
|
| JIRA Issue | RATIS-406 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12948042/r406_20181113.patch |
| Optional Tests |  dupname  asflicense  javac  javadoc  unit  findbugs  
checkstyle  compile  |
| uname | Linux e5ad3f5a9db8 4.4.0-134-generic #160~14.04.1-Ubuntu SMP Fri Aug 
17 11:07:07 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/home/jenkins/jenkins-slave/workspace/PreCommit-RATIS-Build@2/yetus-personality.sh
 |
| git revision | master / 5c37675 |
| maven | version: Apache Maven 3.6.0 
(97c98ec64a1fdfee7767ce5ffb20918da4f719f3; 2018-10-24T18:41:47Z) |
| Default Java | 1.8.0_181 |
| checkstyle | 
https://builds.apache.org/job/PreCommit-RATIS-Build/517/artifact/out/diff-checkstyle-root.txt
 |
| unit | 
https://builds.apache.org/job/PreCommit-RATIS-Build/517/artifact/out/patch-unit-root.txt
 |
|  Test Results | 
https://builds.apache.org/job/PreCommit-RATIS-Build/517/testReport/ |
| Max. process+thread count | 1826 (vs. ulimit of 5000) |
| modules | C: ratis-server U: ratis-server |
| Console output | 
https://builds.apache.org/job/PreCommit-RATIS-Build/517/console |
| Powered by | Apache Yetus 0.8.0   http://yetus.apache.org |


This message was automatically generated.



> In RaftServerImpl, the RaftLog.append(entries) call should not hold the 
> RaftServerImpl lock
> ---
>
> Key: RATIS-406
> URL: https://issues.apache.org/jira/browse/RATIS-406
> Project: Ratis
>  Issue Type: Improvement
>  Components: server
>Reporter: Tsz Wo Nicholas Sze
>

[jira] [Commented] (RATIS-406) In RaftServerImpl, the RaftLog.append(entries) call should not hold the RaftServerImpl lock

2018-11-13 Thread Tsz Wo Nicholas Sze (JIRA)


[ 
https://issues.apache.org/jira/browse/RATIS-406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16685905#comment-16685905
 ] 

Tsz Wo Nicholas Sze commented on RATIS-406:
---

r406_20181113.patch: move SequentalRunner class to RaftLogSequentialOps.

> In RaftServerImpl, the RaftLog.append(entries) call should not hold the 
> RaftServerImpl lock
> ---
>
> Key: RATIS-406
> URL: https://issues.apache.org/jira/browse/RATIS-406
> Project: Ratis
>  Issue Type: Improvement
>  Components: server
>Reporter: Tsz Wo Nicholas Sze
>Assignee: Tsz Wo Nicholas Sze
>Priority: Major
> Attachments: r406_20181109.patch, r406_20181113.patch
>
>
> In RaftServerImpl, the appendEntriesAsync(..) calls must be sequential 
> (although the actual log I/O is async).  Otherwise, the entries appended may 
> be out of order.
> As a result, RaftLog.append(entries) needs not to hold the RaftServerImpl 
> lock.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (RATIS-406) In RaftServerImpl, the RaftLog.append(entries) call should not hold the RaftServerImpl lock

2018-11-13 Thread Tsz Wo Nicholas Sze (JIRA)


 [ 
https://issues.apache.org/jira/browse/RATIS-406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tsz Wo Nicholas Sze updated RATIS-406:
--
Attachment: r406_20181113.patch

> In RaftServerImpl, the RaftLog.append(entries) call should not hold the 
> RaftServerImpl lock
> ---
>
> Key: RATIS-406
> URL: https://issues.apache.org/jira/browse/RATIS-406
> Project: Ratis
>  Issue Type: Improvement
>  Components: server
>Reporter: Tsz Wo Nicholas Sze
>Assignee: Tsz Wo Nicholas Sze
>Priority: Major
> Attachments: r406_20181109.patch, r406_20181113.patch
>
>
> In RaftServerImpl, the appendEntriesAsync(..) calls must be sequential 
> (although the actual log I/O is async).  Otherwise, the entries appended may 
> be out of order.
> As a result, RaftLog.append(entries) needs not to hold the RaftServerImpl 
> lock.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (RATIS-409) Watch requests may not work if there is a conf change

2018-11-13 Thread Hadoop QA (JIRA)


[ 
https://issues.apache.org/jira/browse/RATIS-409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16685892#comment-16685892
 ] 

Hadoop QA commented on RATIS-409:
-

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
13s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m  
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 2 new or modified test 
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  2m 
59s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
46s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
33s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
45s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  0m 
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
45s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}  
0m 30s{color} | {color:orange} root: The patch generated 27 new + 716 unchanged 
- 11 fixed = 743 total (was 727) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 1s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
39s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}  9m 13s{color} 
| {color:red} root in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
17s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 17m 46s{color} | 
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | ratis.netty.TestRaftReconfigurationWithNetty |
|   | ratis.grpc.TestWatchRequestWithGrpc |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/ratis:date2018-11-13 
|
| JIRA Issue | RATIS-409 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12948038/r409_20181113b.patch |
| Optional Tests |  dupname  asflicense  javac  javadoc  unit  findbugs  
checkstyle  compile  |
| uname | Linux 19c7af3ad7c6 3.13.0-143-generic #192-Ubuntu SMP Tue Feb 27 
10:45:36 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/home/jenkins/jenkins-slave/workspace/PreCommit-RATIS-Build/yetus-personality.sh
 |
| git revision | master / d28b649 |
| maven | version: Apache Maven 3.6.0 
(97c98ec64a1fdfee7767ce5ffb20918da4f719f3; 2018-10-24T18:41:47Z) |
| Default Java | 1.8.0_181 |
| checkstyle | 
https://builds.apache.org/job/PreCommit-RATIS-Build/514/artifact/out/diff-checkstyle-root.txt
 |
| unit | 
https://builds.apache.org/job/PreCommit-RATIS-Build/514/artifact/out/patch-unit-root.txt
 |
|  Test Results | 
https://builds.apache.org/job/PreCommit-RATIS-Build/514/testReport/ |
| Max. process+thread count | 2711 (vs. ulimit of 5000) |
| modules | C: ratis-server U: ratis-server |
| Console output | 
https://builds.apache.org/job/PreCommit-RATIS-Build/514/console |
| Powered by | Apache Yetus 0.8.0   http://yetus.apache.org |


This message was automatically generated.



> Watch requests may not work if there is a conf change
> -
>
> Key: RATIS-409
> URL: https://issues.apache.org/jira/browse/RATIS-409
> Project: Ratis
>  Issue Type: Bug
>  Components: server
>Reporter: Tsz Wo Nicholas Sze
>Assignee: Tsz Wo Nicholas Sze
>Priority: Major
> Attachments: r409_20181112.patch, r409_20181113.patch, 
> r409_20181113b.patch
>
>
> When 

[jira] [Created] (RATIS-412) In RaftLog, change lastCommitted to private

2018-11-13 Thread Tsz Wo Nicholas Sze (JIRA)
Tsz Wo Nicholas Sze created RATIS-412:
-

 Summary: In RaftLog, change lastCommitted to private
 Key: RATIS-412
 URL: https://issues.apache.org/jira/browse/RATIS-412
 Project: Ratis
  Issue Type: Improvement
  Components: server
Reporter: Tsz Wo Nicholas Sze


lastCommitted is only used once outside RaftLog.  The usage is in a 
SegmentedRaftLog constructor.  We should change lastCommitted to private.

Then, SegmentedRaftLog could initialize lastCommitted using a RaftLog 
constructor (needs to add a parameter.)

BTW, let's also rename lastCommitted to commitIndex so that it is consistent 
with other code.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (RATIS-399) Move all tests to a new module

2018-11-13 Thread Josh Elser (JIRA)


[ 
https://issues.apache.org/jira/browse/RATIS-399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16685886#comment-16685886
 ] 

Josh Elser commented on RATIS-399:
--

Fine by me, [~szetszwo]. +1

> Move all tests to a new module
> --
>
> Key: RATIS-399
> URL: https://issues.apache.org/jira/browse/RATIS-399
> Project: Ratis
>  Issue Type: Improvement
>  Components: test
>Reporter: Tsz Wo Nicholas Sze
>Assignee: Tsz Wo Nicholas Sze
>Priority: Major
> Attachments: r399_20181107.patch
>
>
> When there is a test failed in an earlier module, all the tests in the 
> depended modules won't be run (even with mvn -fae).  The JIRA propose moving 
> all the tests to a new module as a workaround.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (RATIS-409) Watch requests may not work if there is a conf change

2018-11-13 Thread Tsz Wo Nicholas Sze (JIRA)


[ 
https://issues.apache.org/jira/browse/RATIS-409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16685876#comment-16685876
 ] 

Tsz Wo Nicholas Sze commented on RATIS-409:
---

r409_20181113b.patch reverts also other client side changes.

> Watch requests may not work if there is a conf change
> -
>
> Key: RATIS-409
> URL: https://issues.apache.org/jira/browse/RATIS-409
> Project: Ratis
>  Issue Type: Bug
>  Components: server
>Reporter: Tsz Wo Nicholas Sze
>Assignee: Tsz Wo Nicholas Sze
>Priority: Major
> Attachments: r409_20181112.patch, r409_20181113.patch, 
> r409_20181113b.patch
>
>
> When there is commitIndex changed, the watchRequests is updated with the 
> indices in SenderList.  However, when there is a conf change, SenderList 
> includes all servers in both old and new confs.  As a result, the computed 
> indices may be incorrect.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (RATIS-409) Watch requests may not work if there is a conf change

2018-11-13 Thread Tsz Wo Nicholas Sze (JIRA)


 [ 
https://issues.apache.org/jira/browse/RATIS-409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tsz Wo Nicholas Sze updated RATIS-409:
--
Attachment: r409_20181113.patch

> Watch requests may not work if there is a conf change
> -
>
> Key: RATIS-409
> URL: https://issues.apache.org/jira/browse/RATIS-409
> Project: Ratis
>  Issue Type: Bug
>  Components: server
>Reporter: Tsz Wo Nicholas Sze
>Assignee: Tsz Wo Nicholas Sze
>Priority: Major
> Attachments: r409_20181112.patch, r409_20181113.patch
>
>
> When there is commitIndex changed, the watchRequests is updated with the 
> indices in SenderList.  However, when there is a conf change, SenderList 
> includes all servers in both old and new confs.  As a result, the computed 
> indices may be incorrect.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (RATIS-409) Watch requests may not work if there is a conf change

2018-11-13 Thread Tsz Wo Nicholas Sze (JIRA)


[ 
https://issues.apache.org/jira/browse/RATIS-409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16685850#comment-16685850
 ] 

Tsz Wo Nicholas Sze commented on RATIS-409:
---

The RaftClientImpl causes some test failures.  Will work on it in a separated 
JIRA.

 r409_20181113.patch: reverts the RaftClientImpl changes.

> Watch requests may not work if there is a conf change
> -
>
> Key: RATIS-409
> URL: https://issues.apache.org/jira/browse/RATIS-409
> Project: Ratis
>  Issue Type: Bug
>  Components: server
>Reporter: Tsz Wo Nicholas Sze
>Assignee: Tsz Wo Nicholas Sze
>Priority: Major
> Attachments: r409_20181112.patch, r409_20181113.patch
>
>
> When there is commitIndex changed, the watchRequests is updated with the 
> indices in SenderList.  However, when there is a conf change, SenderList 
> includes all servers in both old and new confs.  As a result, the computed 
> indices may be incorrect.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (RATIS-386) Raft Client Async API's should honor Retry Policy

2018-11-13 Thread Tsz Wo Nicholas Sze (JIRA)


[ 
https://issues.apache.org/jira/browse/RATIS-386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16685836#comment-16685836
 ] 

Tsz Wo Nicholas Sze commented on RATIS-386:
---

Thanks [~shashikant].  Some comments on the patch:
- remove retryInterval from RaftClientImpl.  Just use the sleep time in 
retryPolicy.
-* Then, we can remove RETRY_INTERVAL_KEY and the related fields from 
RaftClientConfigKeys
- retryCount should be volatile.
-* BTW, retryCount seems misleading.  It is not clear whether the first attempt 
is included.  How about rename it to attemptCount?  If you agree, let's also 
rename the use of "retryCount" in RetryPolicy.
- Let's move the retryPolicy check to sendRequestAsync(..); see below.
-* The change of SlidingWindow is not required.
-* Use RaftException (or create a new subclass) instead of TimeoutExceptoin.

{code}
@@ -268,10 +271,10 @@ final class RaftClientImpl implements RaftClient {
   PendingAsyncRequest pending) {
 final RaftClientRequest request = pending.newRequest();
 final CompletableFuture f = pending.getReplyFuture();
-return sendRequestAsync(request).thenCompose(reply -> {
+return sendRequestAsync(request, 
pending.getRetryCount()).thenCompose(reply -> {
   if (reply == null) {
-LOG.debug("schedule a retry in {} for {}", retryInterval, request);
-scheduler.onTimeout(retryInterval,
+LOG.debug("schedule attempt #{} with policy {} for {}", 
pending.getRetryCount(), retryPolicy, request);
+scheduler.onTimeout(retryPolicy.getSleepTime(),
 () -> getSlidingWindow(request).retry(pending, 
this::sendRequestWithRetryAsync),
 LOG, () -> "Failed to retry " + request);
   } else {
@@ -301,12 +304,17 @@ final class RaftClientImpl implements RaftClient {
 }
   }
 
-  private CompletableFuture sendRequestAsync(
-  RaftClientRequest request) {
+  private CompletableFuture 
sendRequestAsync(RaftClientRequest request, int attemptCount) {
 LOG.debug("{}: send* {}", clientId, request);
 return clientRpc.sendRequestAsync(request).thenApply(reply -> {
   LOG.debug("{}: receive* {}", clientId, reply);
   reply = handleNotLeaderException(request, reply);
+  if (reply == null) {
+if (!retryPolicy.shouldRetry(attemptCount)) {
+  reply = new RaftClientReply(request, new RaftException(
+  "Failed " + request + " for " + attemptCount + " attempts with " 
+ retryPolicy), null);
+}
+  }
   if (reply != null) {
 getSlidingWindow(request).receiveReply(
{code}


> Raft Client Async API's should honor Retry Policy 
> --
>
> Key: RATIS-386
> URL: https://issues.apache.org/jira/browse/RATIS-386
> Project: Ratis
>  Issue Type: Improvement
>  Components: client
>Affects Versions: 0.3.0
>Reporter: Shashikant Banerjee
>Assignee: Shashikant Banerjee
>Priority: Major
> Fix For: 0.3.0
>
> Attachments: RATIS-386.000.patch
>
>
> Raft client sync Api has support for retry policies. Similarly, for Async 
> API's including watch Api, support for Retry Policy is required.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (RATIS-410) entry with stateMachineData may cause OOM

2018-11-13 Thread Tsz Wo Nicholas Sze (JIRA)


[ 
https://issues.apache.org/jira/browse/RATIS-410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16685785#comment-16685785
 ] 

Tsz Wo Nicholas Sze commented on RATIS-410:
---

[~msingh], do you mean that the cached LogEntryProto(s) in Ratis have included 
state machine data so that it goes OOM?  It is the case, we should remove the 
state machine data from cached LogEntryProto(s).

Would it work if we reduce 1GB raft log segment size in Ozone?


> entry with stateMachineData may cause OOM
> -
>
> Key: RATIS-410
> URL: https://issues.apache.org/jira/browse/RATIS-410
> Project: Ratis
>  Issue Type: Bug
>  Components: server
>Affects Versions: 0.3.0
>Reporter: Mukul Kumar Singh
>Assignee: Mukul Kumar Singh
>Priority: Major
> Attachments: RATIS-410.001.patch
>
>
> With Ozone, each of raft log segments are of 1GB in size. Also each of the 
> writeChunk request will have 16MB worth of payload. So this can result in 
> multiple writeChunk requests to be part of one log segment.
> Because the caching policy in ratis works on eviction of log segments, these 
> results in segments holding onto memory which is part of stateMachine.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (RATIS-386) Raft Client Async API's should honor Retry Policy

2018-11-13 Thread Hadoop QA (JIRA)


[ 
https://issues.apache.org/jira/browse/RATIS-386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16685267#comment-16685267
 ] 

Hadoop QA commented on RATIS-386:
-

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
21s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m  
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 2 new or modified test 
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  1m 
23s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  2m 
 3s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
49s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
27s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
41s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m  
6s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  0m 
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
48s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}  
0m 22s{color} | {color:orange} root: The patch generated 9 new + 295 unchanged 
- 1 fixed = 304 total (was 296) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
32s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}  5m 53s{color} 
| {color:red} root in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
15s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 14m 42s{color} | 
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | ratis.server.simulation.TestRaftWithSimulatedRpc |
|   | ratis.server.simulation.TestServerRestartWithSimulatedRpc |
|   | ratis.server.simulation.TestRaftExceptionWithSimulation |
|   | ratis.server.simulation.TestRaftStateMachineExceptionWithSimulatedRpc |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/ratis:date2018-11-13 
|
| JIRA Issue | RATIS-386 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12947988/RATIS-386.000.patch |
| Optional Tests |  dupname  asflicense  javac  javadoc  unit  findbugs  
checkstyle  compile  |
| uname | Linux fca4174d755c 3.13.0-153-generic #203-Ubuntu SMP Thu Jun 14 
08:52:28 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/home/jenkins/jenkins-slave/workspace/PreCommit-RATIS-Build@2/yetus-personality.sh
 |
| git revision | master / 6f3419a |
| maven | version: Apache Maven 3.6.0 
(97c98ec64a1fdfee7767ce5ffb20918da4f719f3; 2018-10-24T18:41:47Z) |
| Default Java | 1.8.0_181 |
| checkstyle | 
https://builds.apache.org/job/PreCommit-RATIS-Build/512/artifact/out/diff-checkstyle-root.txt
 |
| unit | 
https://builds.apache.org/job/PreCommit-RATIS-Build/512/artifact/out/patch-unit-root.txt
 |
|  Test Results | 
https://builds.apache.org/job/PreCommit-RATIS-Build/512/testReport/ |
| Max. process+thread count | 399 (vs. ulimit of 5000) |
| modules | C: ratis-common ratis-client ratis-server U: . |
| Console output | 
https://builds.apache.org/job/PreCommit-RATIS-Build/512/console |
| Powered by | Apache Yetus 0.8.0   http://yetus.apache.org |


This message was automatically generated.



> Raft Client Async API's should honor Retry 

[jira] [Commented] (RATIS-410) entry with stateMachineData may cause OOM

2018-11-13 Thread Hadoop QA (JIRA)


[ 
https://issues.apache.org/jira/browse/RATIS-410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16685256#comment-16685256
 ] 

Hadoop QA commented on RATIS-410:
-

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  3m 
53s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m  
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red}  0m  
0s{color} | {color:red} The patch doesn't appear to include any new or modified 
tests. Please justify why no new tests are needed for this patch. Also please 
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  2m 
48s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
45s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
25s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
42s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  0m 
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
44s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}  
0m 19s{color} | {color:orange} root: The patch generated 3 new + 124 unchanged 
- 3 fixed = 127 total (was 127) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
33s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}  5m 51s{color} 
| {color:red} root in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
15s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 17m 12s{color} | 
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | ratis.server.simulation.TestRaftWithSimulatedRpc |
|   | ratis.server.simulation.TestServerRestartWithSimulatedRpc |
|   | ratis.server.storage.TestRaftLogSegment |
|   | ratis.server.storage.TestSegmentedRaftLog |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/ratis:date2018-11-13 
|
| JIRA Issue | RATIS-410 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12947984/RATIS-410.001.patch |
| Optional Tests |  dupname  asflicense  javac  javadoc  unit  findbugs  
checkstyle  compile  |
| uname | Linux 3606aa4f3f82 3.13.0-153-generic #203-Ubuntu SMP Thu Jun 14 
08:52:28 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/home/jenkins/jenkins-slave/workspace/PreCommit-RATIS-Build/yetus-personality.sh
 |
| git revision | master / 6f3419a |
| maven | version: Apache Maven 3.6.0 
(97c98ec64a1fdfee7767ce5ffb20918da4f719f3; 2018-10-24T18:41:47Z) |
| Default Java | 1.8.0_181 |
| checkstyle | 
https://builds.apache.org/job/PreCommit-RATIS-Build/511/artifact/out/diff-checkstyle-root.txt
 |
| unit | 
https://builds.apache.org/job/PreCommit-RATIS-Build/511/artifact/out/patch-unit-root.txt
 |
|  Test Results | 
https://builds.apache.org/job/PreCommit-RATIS-Build/511/testReport/ |
| Max. process+thread count | 389 (vs. ulimit of 5000) |
| modules | C: ratis-server U: ratis-server |
| Console output | 
https://builds.apache.org/job/PreCommit-RATIS-Build/511/console |
| Powered by | Apache Yetus 0.8.0   http://yetus.apache.org |


This message was automatically generated.



> entry with stateMachineData may cause OOM
> -
>
> Key: RATIS-410
> URL: https://issues.apache.org/jira/browse/RATIS-410
> Project: Ratis
>  Issue Type: Bug
>  Components: 

[jira] [Updated] (RATIS-386) Raft Client Async API's should honor Retry Policy

2018-11-13 Thread Shashikant Banerjee (JIRA)


 [ 
https://issues.apache.org/jira/browse/RATIS-386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shashikant Banerjee updated RATIS-386:
--
Attachment: RATIS-386.000.patch

> Raft Client Async API's should honor Retry Policy 
> --
>
> Key: RATIS-386
> URL: https://issues.apache.org/jira/browse/RATIS-386
> Project: Ratis
>  Issue Type: Improvement
>  Components: client
>Affects Versions: 0.3.0
>Reporter: Shashikant Banerjee
>Assignee: Shashikant Banerjee
>Priority: Major
> Fix For: 0.3.0
>
> Attachments: RATIS-386.000.patch
>
>
> Raft client sync Api has support for retry policies. Similarly, for Async 
> API's including watch Api, support for Retry Policy is required.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (RATIS-410) entry with stateMachineData may cause OOM

2018-11-13 Thread Mukul Kumar Singh (JIRA)


[ 
https://issues.apache.org/jira/browse/RATIS-410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16685213#comment-16685213
 ] 

Mukul Kumar Singh commented on RATIS-410:
-

Patch v1 fixes the issue, leaving the current code, as this can be revisited 
later and fixed correctly.

> entry with stateMachineData may cause OOM
> -
>
> Key: RATIS-410
> URL: https://issues.apache.org/jira/browse/RATIS-410
> Project: Ratis
>  Issue Type: Bug
>  Components: server
>Affects Versions: 0.3.0
>Reporter: Mukul Kumar Singh
>Assignee: Mukul Kumar Singh
>Priority: Major
> Attachments: RATIS-410.001.patch
>
>
> With Ozone, each of raft log segments are of 1GB in size. Also each of the 
> writeChunk request will have 16MB worth of payload. So this can result in 
> multiple writeChunk requests to be part of one log segment.
> Because the caching policy in ratis works on eviction of log segments, these 
> results in segments holding onto memory which is part of stateMachine.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (RATIS-410) entry with stateMachineData may cause OOM

2018-11-13 Thread Mukul Kumar Singh (JIRA)


 [ 
https://issues.apache.org/jira/browse/RATIS-410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mukul Kumar Singh updated RATIS-410:

Attachment: RATIS-410.001.patch

> entry with stateMachineData may cause OOM
> -
>
> Key: RATIS-410
> URL: https://issues.apache.org/jira/browse/RATIS-410
> Project: Ratis
>  Issue Type: Bug
>  Components: server
>Affects Versions: 0.3.0
>Reporter: Mukul Kumar Singh
>Assignee: Mukul Kumar Singh
>Priority: Major
> Attachments: RATIS-410.001.patch
>
>
> With Ozone, each of raft log segments are of 1GB in size. Also each of the 
> writeChunk request will have 16MB worth of payload. So this can result in 
> multiple writeChunk requests to be part of one log segment.
> Because the caching policy in ratis works on eviction of log segments, these 
> results in segments holding onto memory which is part of stateMachine.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (RATIS-410) entry with stateMachineData may cause OOM

2018-11-13 Thread Mukul Kumar Singh (JIRA)
Mukul Kumar Singh created RATIS-410:
---

 Summary: entry with stateMachineData may cause OOM
 Key: RATIS-410
 URL: https://issues.apache.org/jira/browse/RATIS-410
 Project: Ratis
  Issue Type: Bug
  Components: server
Affects Versions: 0.3.0
Reporter: Mukul Kumar Singh
Assignee: Mukul Kumar Singh


With Ozone, each of raft log segments are of 1GB in size. Also each of the 
writeChunk request will have 16MB worth of payload. So this can result in 
multiple writeChunk requests to be part of one log segment.

Because the caching policy in ratis works on eviction of log segments, these 
results in segments holding onto memory which is part of stateMachine.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)