[jira] [Commented] (RATIS-284) FollowerInfo#toString should pring the elapsed time from last rpc

2018-07-26 Thread Hadoop QA (JIRA)


[ 
https://issues.apache.org/jira/browse/RATIS-284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16559273#comment-16559273
 ] 

Hadoop QA commented on RATIS-284:
-

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  1m 
21s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m  
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red}  0m  
0s{color} | {color:red} The patch doesn't appear to include any new or modified 
tests. Please justify why no new tests are needed for this patch. Also please 
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  1m 
47s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 
43s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
18s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
32s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  1m 
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  1m 
32s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}  
0m 15s{color} | {color:orange} root: The patch generated 2 new + 49 unchanged - 
1 fixed = 51 total (was 50) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
33s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 23m 
55s{color} | {color:green} root in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
 9s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 33m 56s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/ratis:date2018-07-27 
|
| JIRA Issue | RATIS-284 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12933306/RATIS-284.001.patch |
| Optional Tests |  asflicense  javac  javadoc  unit  findbugs  checkstyle  
compile  |
| uname | Linux a601fe920142 3.13.0-153-generic #203-Ubuntu SMP Thu Jun 14 
08:52:28 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/home/jenkins/jenkins-slave/workspace/PreCommit-RATIS-Build/yetus-personality.sh
 |
| git revision | master / a3a8332 |
| Default Java | 1.8.0_171 |
| checkstyle | 
https://builds.apache.org/job/PreCommit-RATIS-Build/254/artifact/out/diff-checkstyle-root.txt
 |
|  Test Results | 
https://builds.apache.org/job/PreCommit-RATIS-Build/254/testReport/ |
| modules | C: ratis-server U: ratis-server |
| Console output | 
https://builds.apache.org/job/PreCommit-RATIS-Build/254/console |
| Powered by | Apache Yetus 0.5.0   http://yetus.apache.org |


This message was automatically generated.



> FollowerInfo#toString should pring the elapsed time from last rpc
> -
>
> Key: RATIS-284
> URL: https://issues.apache.org/jira/browse/RATIS-284
> Project: Ratis
>  Issue Type: Bug
>Reporter: Mukul Kumar Singh
>Assignee: Mukul Kumar Singh
>Priority: Major
> Attachments: RATIS-284.001.patch
>
>
> FollowerInfo#toString currently prints the absolute time of last rpc, however 
> while debugging ratis issues it will be useful to have last elapsed time from 
> last rpc.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (RATIS-284) FollowerInfo#toString should pring the elapsed time from last rpc

2018-07-26 Thread Mukul Kumar Singh (JIRA)


 [ 
https://issues.apache.org/jira/browse/RATIS-284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mukul Kumar Singh updated RATIS-284:

Attachment: RATIS-284.001.patch

> FollowerInfo#toString should pring the elapsed time from last rpc
> -
>
> Key: RATIS-284
> URL: https://issues.apache.org/jira/browse/RATIS-284
> Project: Ratis
>  Issue Type: Bug
>Reporter: Mukul Kumar Singh
>Assignee: Mukul Kumar Singh
>Priority: Major
> Attachments: RATIS-284.001.patch
>
>
> FollowerInfo#toString currently prints the absolute time of last rpc, however 
> while debugging ratis issues it will be useful to have last elapsed time from 
> last rpc.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (RATIS-284) FollowerInfo#toString should pring the elapsed time from last rpc

2018-07-26 Thread Mukul Kumar Singh (JIRA)
Mukul Kumar Singh created RATIS-284:
---

 Summary: FollowerInfo#toString should pring the elapsed time from 
last rpc
 Key: RATIS-284
 URL: https://issues.apache.org/jira/browse/RATIS-284
 Project: Ratis
  Issue Type: Bug
Reporter: Mukul Kumar Singh
Assignee: Mukul Kumar Singh


FollowerInfo#toString currently prints the absolute time of last rpc, however 
while debugging ratis issues it will be useful to have last elapsed time from 
last rpc.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (RATIS-270) Replication_ALL requests should not be replied from retry cache if they are present in delayedReplies queue.

2018-07-26 Thread Tsz Wo Nicholas Sze (JIRA)


[ 
https://issues.apache.org/jira/browse/RATIS-270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16558957#comment-16558957
 ] 

Tsz Wo Nicholas Sze commented on RATIS-270:
---

Let's fix the retry for the same leader in this JIRA.  For changing leader, we 
need RATIS-234.

> Replication_ALL requests should not be replied from retry cache if they are 
> present in delayedReplies queue.
> 
>
> Key: RATIS-270
> URL: https://issues.apache.org/jira/browse/RATIS-270
> Project: Ratis
>  Issue Type: Bug
>  Components: server
>Reporter: Mukul Kumar Singh
>Assignee: Tsz Wo Nicholas Sze
>Priority: Major
>  Labels: ozone
>
> Retry requests are answered from the retry cache when requests have 
> Replication_ALL semantics. This leads to a case, where the client retries for 
> a response which is stuck in the delayed replies queue. This new retry is now 
> answered from the retry cache even though the request has not been completed 
> on all the nodes.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (RATIS-270) Replication_ALL requests should not be replied from retry cache if they are present in delayedReplies queue.

2018-07-26 Thread Tsz Wo Nicholas Sze (JIRA)


 [ 
https://issues.apache.org/jira/browse/RATIS-270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tsz Wo Nicholas Sze updated RATIS-270:
--
 Labels: ozone  (was: )
Component/s: server

> Replication_ALL requests should not be replied from retry cache if they are 
> present in delayedReplies queue.
> 
>
> Key: RATIS-270
> URL: https://issues.apache.org/jira/browse/RATIS-270
> Project: Ratis
>  Issue Type: Bug
>  Components: server
>Reporter: Mukul Kumar Singh
>Assignee: Tsz Wo Nicholas Sze
>Priority: Major
>  Labels: ozone
>
> Retry requests are answered from the retry cache when requests have 
> Replication_ALL semantics. This leads to a case, where the client retries for 
> a response which is stuck in the delayed replies queue. This new retry is now 
> answered from the retry cache even though the request has not been completed 
> on all the nodes.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (RATIS-11) Better retry policy support for Raft client

2018-07-26 Thread Jitendra Nath Pandey (JIRA)


[ 
https://issues.apache.org/jira/browse/RATIS-11?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16558904#comment-16558904
 ] 

Jitendra Nath Pandey commented on RATIS-11:
---

bq. [~jnp] could you help to re-trigger the pre-commit builds, I don't see 
''submit patch" button on the page

No need to do it again, once is enough and any subsequent patch is 
automatically picked up. Pre-commit is already done! Thanks for posting updated 
patch.

> Better retry policy support for Raft client
> ---
>
> Key: RATIS-11
> URL: https://issues.apache.org/jira/browse/RATIS-11
> Project: Ratis
>  Issue Type: Improvement
>  Components: client
>Reporter: Jing Zhao
>Assignee: Jing Chen
>Priority: Major
> Attachments: 
> 0001-RATIS-11-Better-retry-policy-support-for-Raft-client.patch
>
>
> We should have better retry policy support on the raft client side, such as 
> different retry policies, more flexible retry parameter settings, etc.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (RATIS-283) Integrate an RPC system that support KerberosV5

2018-07-26 Thread Josh Elser (JIRA)
Josh Elser created RATIS-283:


 Summary: Integrate an RPC system that support KerberosV5
 Key: RATIS-283
 URL: https://issues.apache.org/jira/browse/RATIS-283
 Project: Ratis
  Issue Type: Improvement
  Components: security
Reporter: Josh Elser


A big impediment towards using RATIS-271 within HBase is the lack of krb5 
authentication.

My understanding that this limitation exists in Ratis due to gRPC's lack of 
such authentication.

The two obvious paths forward would be adding krb5 authn support to gRPC to 
pick up in Ratis as-is, or to use a different RPC system that does support it.

Maybe there are other solutions/avenues forward. Suggestions very welcome.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (RATIS-281) Ratis should provide an api to readStateMachinedata when the cached segment is not available

2018-07-26 Thread Hadoop QA (JIRA)


[ 
https://issues.apache.org/jira/browse/RATIS-281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16558806#comment-16558806
 ] 

Hadoop QA commented on RATIS-281:
-

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
11s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m  
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 1 new or modified test 
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m  
6s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  1m 
47s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 
31s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
21s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
32s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m  
5s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  1m 
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green}  1m 
39s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red}  1m 39s{color} 
| {color:red} root generated 1 new + 88 unchanged - 3 fixed = 89 total (was 91) 
{color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}  
0m 22s{color} | {color:orange} root: The patch generated 15 new + 552 unchanged 
- 0 fixed = 567 total (was 552) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
34s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 23m 
54s{color} | {color:green} root in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
 9s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 33m  2s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/ratis:date2018-07-26 
|
| JIRA Issue | RATIS-281 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12933251/RATIS-281.001.patch |
| Optional Tests |  asflicense  javac  javadoc  unit  findbugs  checkstyle  
compile  cc  |
| uname | Linux b68c65946090 3.13.0-153-generic #203-Ubuntu SMP Thu Jun 14 
08:52:28 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/home/jenkins/jenkins-slave/workspace/PreCommit-RATIS-Build/yetus-personality.sh
 |
| git revision | master / a3a8332 |
| Default Java | 1.8.0_171 |
| javac | 
https://builds.apache.org/job/PreCommit-RATIS-Build/253/artifact/out/diff-compile-javac-root.txt
 |
| checkstyle | 
https://builds.apache.org/job/PreCommit-RATIS-Build/253/artifact/out/diff-checkstyle-root.txt
 |
|  Test Results | 
https://builds.apache.org/job/PreCommit-RATIS-Build/253/testReport/ |
| modules | C: ratis-proto-shaded ratis-common ratis-server U: . |
| Console output | 
https://builds.apache.org/job/PreCommit-RATIS-Build/253/console |
| Powered by | Apache Yetus 0.5.0   http://yetus.apache.org |


This message was automatically generated.



> Ratis should provide an api to readStateMachinedata when the cached segment 
> is not available
> 
>
> Key: RATIS-281
> URL: https://issues.apache.org/jira/browse/RATIS-281
> Project: Ratis
>

[jira] [Created] (RATIS-282) Create fault-injected tests

2018-07-26 Thread Josh Elser (JIRA)
Josh Elser created RATIS-282:


 Summary: Create fault-injected tests
 Key: RATIS-282
 URL: https://issues.apache.org/jira/browse/RATIS-282
 Project: Ratis
  Issue Type: Sub-task
Reporter: Josh Elser


Need to create some mechanism that will inject failures into a LogStream to see 
how the software fares.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (RATIS-271) Ratis-backed distributed log: "LogService"

2018-07-26 Thread Josh Elser (JIRA)


[ 
https://issues.apache.org/jira/browse/RATIS-271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16558797#comment-16558797
 ] 

Josh Elser commented on RATIS-271:
--

[~vrodionov] pointed out [https://github.com/OpenHFT/Chronicle-Queue] to me 
today which looks like a nice project to draw inspiration from.

> Ratis-backed distributed log: "LogService" 
> ---
>
> Key: RATIS-271
> URL: https://issues.apache.org/jira/browse/RATIS-271
> Project: Ratis
>  Issue Type: New Feature
>  Components: LogService
>Reporter: Josh Elser
>Priority: Major
>
> Umbrella issue for building a distributed log using Ratis:
> Doc: 
> [https://docs.google.com/document/d/1Su5py_T5Ytfh9RoTTX2s20KbSJwBHVxbO7ge5ORqbCk/edit#|https://docs.google.com/document/d/1Su5py_T5Ytfh9RoTTX2s20KbSJwBHVxbO7ge5ORqbCk/edit]
> Discuss: 
> https://lists.apache.org/thread.html/f80dc3900f6d9f4ee4d9f9e0898cee9a232e3b1ca9a4d9a53fea1d71@%3Cdev.ratis.apache.org%3E



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (RATIS-11) Better retry policy support for Raft client

2018-07-26 Thread Hadoop QA (JIRA)


[ 
https://issues.apache.org/jira/browse/RATIS-11?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16558779#comment-16558779
 ] 

Hadoop QA commented on RATIS-11:


| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
11s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m  
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 1 new or modified test 
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  1m 
53s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 
40s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
18s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
35s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  1m 
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  1m 
35s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}  
0m 17s{color} | {color:orange} root: The patch generated 81 new + 141 unchanged 
- 6 fixed = 222 total (was 147) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green}  0m  
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
32s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 23m 
27s{color} | {color:green} root in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
 9s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 32m 31s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/ratis:date2018-07-26 
|
| JIRA Issue | RATIS-11 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12933242/0001-RATIS-11-Better-retry-policy-support-for-Raft-client.patch
 |
| Optional Tests |  asflicense  javac  javadoc  unit  xml  compile  findbugs  
checkstyle  |
| uname | Linux 6cd2a81b3f28 3.13.0-153-generic #203-Ubuntu SMP Thu Jun 14 
08:52:28 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/home/jenkins/jenkins-slave/workspace/PreCommit-RATIS-Build/yetus-personality.sh
 |
| git revision | master / a3a8332 |
| Default Java | 1.8.0_171 |
| checkstyle | 
https://builds.apache.org/job/PreCommit-RATIS-Build/252/artifact/out/diff-checkstyle-root.txt
 |
|  Test Results | 
https://builds.apache.org/job/PreCommit-RATIS-Build/252/testReport/ |
| modules | C: ratis-client U: ratis-client |
| Console output | 
https://builds.apache.org/job/PreCommit-RATIS-Build/252/console |
| Powered by | Apache Yetus 0.5.0   http://yetus.apache.org |


This message was automatically generated.



> Better retry policy support for Raft client
> ---
>
> Key: RATIS-11
> URL: https://issues.apache.org/jira/browse/RATIS-11
> Project: Ratis
>  Issue Type: Improvement
>  Components: client
>Reporter: Jing Zhao
>Assignee: Jing Chen
>Priority: Major
> Attachments: 
> 0001-RATIS-11-Better-retry-policy-support-for-Raft-client.patch
>
>
> We should have better retry policy support on the raft client side, such as 
> different retry policies, more flexible retry parameter settings, etc.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (RATIS-281) Ratis should provide an api to readStateMachinedata when the cached segment is not available

2018-07-26 Thread Mukul Kumar Singh (JIRA)
Mukul Kumar Singh created RATIS-281:
---

 Summary: Ratis should provide an api to readStateMachinedata when 
the cached segment is not available
 Key: RATIS-281
 URL: https://issues.apache.org/jira/browse/RATIS-281
 Project: Ratis
  Issue Type: Bug
  Components: server
Reporter: Mukul Kumar Singh
Assignee: Mukul Kumar Singh


For requests with state Machine data, leader may loose the state Machine data 
when the entries are evicted from the cache. This jira proposes to read the 
entries from the statemachine and reconstruct log entries.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (RATIS-11) Better retry policy support for Raft client

2018-07-26 Thread Jing Chen (JIRA)


[ 
https://issues.apache.org/jira/browse/RATIS-11?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16558726#comment-16558726
 ] 

Jing Chen commented on RATIS-11:


resolved JUnitRunListener exception by adding ratis-common as test dependency 
of ratis-client, new patch attached.

 

[~jnp] could you help to re-trigger the pre-commit builds, I don't see ''submit 
patch" button on the page

 

> Better retry policy support for Raft client
> ---
>
> Key: RATIS-11
> URL: https://issues.apache.org/jira/browse/RATIS-11
> Project: Ratis
>  Issue Type: Improvement
>  Components: client
>Reporter: Jing Zhao
>Assignee: Jing Chen
>Priority: Major
> Attachments: 
> 0001-RATIS-11-Better-retry-policy-support-for-Raft-client.patch
>
>
> We should have better retry policy support on the raft client side, such as 
> different retry policies, more flexible retry parameter settings, etc.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (RATIS-11) Better retry policy support for Raft client

2018-07-26 Thread Jing Chen (JIRA)


 [ 
https://issues.apache.org/jira/browse/RATIS-11?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jing Chen updated RATIS-11:
---
Attachment: (was: 
0001-RATIS-11-Better-retry-policy-support-for-Raft-client.patch)

> Better retry policy support for Raft client
> ---
>
> Key: RATIS-11
> URL: https://issues.apache.org/jira/browse/RATIS-11
> Project: Ratis
>  Issue Type: Improvement
>  Components: client
>Reporter: Jing Zhao
>Assignee: Jing Chen
>Priority: Major
> Attachments: 
> 0001-RATIS-11-Better-retry-policy-support-for-Raft-client.patch
>
>
> We should have better retry policy support on the raft client side, such as 
> different retry policies, more flexible retry parameter settings, etc.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (RATIS-11) Better retry policy support for Raft client

2018-07-26 Thread Hadoop QA (JIRA)


[ 
https://issues.apache.org/jira/browse/RATIS-11?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16558611#comment-16558611
 ] 

Hadoop QA commented on RATIS-11:


| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
30s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m  
1s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 1 new or modified test 
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  3m 
32s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  3m  
8s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
40s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m 
29s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  3m 
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  3m 
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  3m 
12s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}  
0m 38s{color} | {color:orange} root: The patch generated 81 new + 141 unchanged 
- 6 fixed = 222 total (was 147) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green}  0m  
3s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m 
28s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}  2m 31s{color} 
| {color:red} root in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
17s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 21m 17s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/ratis:date2018-07-26 
|
| JIRA Issue | RATIS-11 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12933168/0001-RATIS-11-Better-retry-policy-support-for-Raft-client.patch
 |
| Optional Tests |  asflicense  javac  javadoc  unit  xml  compile  findbugs  
checkstyle  |
| uname | Linux 8a5ddcb7922d 3.13.0-153-generic #203-Ubuntu SMP Thu Jun 14 
08:52:28 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/home/jenkins/jenkins-slave/workspace/PreCommit-RATIS-Build/yetus-personality.sh
 |
| git revision | master / a3a8332 |
| Default Java | 1.8.0_171 |
| checkstyle | 
https://builds.apache.org/job/PreCommit-RATIS-Build/251/artifact/out/diff-checkstyle-root.txt
 |
| unit | 
https://builds.apache.org/job/PreCommit-RATIS-Build/251/artifact/out/patch-unit-root.txt
 |
|  Test Results | 
https://builds.apache.org/job/PreCommit-RATIS-Build/251/testReport/ |
| modules | C: ratis-client U: ratis-client |
| Console output | 
https://builds.apache.org/job/PreCommit-RATIS-Build/251/console |
| Powered by | Apache Yetus 0.5.0   http://yetus.apache.org |


This message was automatically generated.



> Better retry policy support for Raft client
> ---
>
> Key: RATIS-11
> URL: https://issues.apache.org/jira/browse/RATIS-11
> Project: Ratis
>  Issue Type: Improvement
>  Components: client
>Reporter: Jing Zhao
>Assignee: Jing Chen
>Priority: Major
> Attachments: 
> 0001-RATIS-11-Better-retry-policy-support-for-Raft-client.patch
>
>
> We should have better retry policy support on the raft client side, such as 
> different retry policies, more flexible retry parameter settings, etc.



--
This 

[jira] [Commented] (RATIS-11) Better retry policy support for Raft client

2018-07-26 Thread Jitendra Nath Pandey (JIRA)


[ 
https://issues.apache.org/jira/browse/RATIS-11?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16558536#comment-16558536
 ] 

Jitendra Nath Pandey commented on RATIS-11:
---

clicked 'submit patch' to trigger pre-commit builds.

> Better retry policy support for Raft client
> ---
>
> Key: RATIS-11
> URL: https://issues.apache.org/jira/browse/RATIS-11
> Project: Ratis
>  Issue Type: Improvement
>  Components: client
>Reporter: Jing Zhao
>Assignee: Jing Chen
>Priority: Major
> Attachments: 
> 0001-RATIS-11-Better-retry-policy-support-for-Raft-client.patch
>
>
> We should have better retry policy support on the raft client side, such as 
> different retry policies, more flexible retry parameter settings, etc.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (RATIS-277) Create an interface to support quorum creation

2018-07-26 Thread Josh Elser (JIRA)


 [ 
https://issues.apache.org/jira/browse/RATIS-277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Josh Elser updated RATIS-277:
-
Component/s: LogService

> Create an interface to support quorum creation
> --
>
> Key: RATIS-277
> URL: https://issues.apache.org/jira/browse/RATIS-277
> Project: Ratis
>  Issue Type: Sub-task
>  Components: LogService
>Reporter: Josh Elser
>Priority: Major
>
> Create an interface that allows users of the LogService to define how quorums 
> are formed among nodes.
> The use case is that HBase will have many RegionServers available to use. We 
> want to allow HBase to control which RegionServers are used to participate in 
> one LogStream.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (RATIS-272) Design ideal API

2018-07-26 Thread Josh Elser (JIRA)


 [ 
https://issues.apache.org/jira/browse/RATIS-272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Josh Elser updated RATIS-272:
-
Component/s: LogService

> Design ideal API
> 
>
> Key: RATIS-272
> URL: https://issues.apache.org/jira/browse/RATIS-272
> Project: Ratis
>  Issue Type: Sub-task
>  Components: LogService
>Reporter: Josh Elser
>Priority: Major
>
> With influence from Apache DistributedLog, Kafka, and BookKeeper, design an 
> API that balances the ideal notion of what a distribute log system should 
> look like, but also considers the needs of HBase to replace a WAL.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (RATIS-280) Support exporting of a LogStream to a remote filesystem

2018-07-26 Thread Josh Elser (JIRA)


 [ 
https://issues.apache.org/jira/browse/RATIS-280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Josh Elser updated RATIS-280:
-
Component/s: LogService

> Support exporting of a LogStream to a remote filesystem
> ---
>
> Key: RATIS-280
> URL: https://issues.apache.org/jira/browse/RATIS-280
> Project: Ratis
>  Issue Type: Sub-task
>  Components: LogService
>Reporter: Josh Elser
>Priority: Major
>
> LogStreams are kept on local filesystem as they are being written to; 
> however, this is not a good solution for long-term storage.
> We should provide the mechanism to invoke API that copies the log stream to a 
> distributed filesystem (e.g. HDFS, S3, WASB), to reclaim the local FS space.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (RATIS-273) Create new maven module

2018-07-26 Thread Josh Elser (JIRA)


 [ 
https://issues.apache.org/jira/browse/RATIS-273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Josh Elser updated RATIS-273:
-
Component/s: LogService

> Create new maven module
> ---
>
> Key: RATIS-273
> URL: https://issues.apache.org/jira/browse/RATIS-273
> Project: Ratis
>  Issue Type: Sub-task
>  Components: LogService
>Reporter: Josh Elser
>Priority: Major
>
> Create a log-service maven module and wire up to the parent project.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (RATIS-275) Read-path of log stream statemachine

2018-07-26 Thread Josh Elser (JIRA)


 [ 
https://issues.apache.org/jira/browse/RATIS-275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Josh Elser updated RATIS-275:
-
Component/s: LogService

> Read-path of log stream statemachine
> 
>
> Key: RATIS-275
> URL: https://issues.apache.org/jira/browse/RATIS-275
> Project: Ratis
>  Issue Type: Sub-task
>  Components: LogService
>Reporter: Josh Elser
>Priority: Major
>
> Implement the ability to read from a log stream.
> We want the ability to read linearly from a point in the log. Sub-linear seek 
> into a given offset in the log.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (RATIS-278) Export metrics about a LogStream

2018-07-26 Thread Josh Elser (JIRA)


 [ 
https://issues.apache.org/jira/browse/RATIS-278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Josh Elser updated RATIS-278:
-
Component/s: LogService

> Export metrics about a LogStream
> 
>
> Key: RATIS-278
> URL: https://issues.apache.org/jira/browse/RATIS-278
> Project: Ratis
>  Issue Type: Sub-task
>  Components: LogService
>Reporter: Josh Elser
>Priority: Major
>
> We need to have metrics exported to tell us information about a LogStream's 
> performance.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (RATIS-274) Write-path of log stream statemachine

2018-07-26 Thread Josh Elser (JIRA)


 [ 
https://issues.apache.org/jira/browse/RATIS-274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Josh Elser updated RATIS-274:
-
Component/s: LogService

> Write-path of log stream statemachine
> -
>
> Key: RATIS-274
> URL: https://issues.apache.org/jira/browse/RATIS-274
> Project: Ratis
>  Issue Type: Sub-task
>  Components: LogService
>Reporter: Josh Elser
>Priority: Major
>
> Implement the ability to write data to a log stream.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (RATIS-276) "Tail" a log stream

2018-07-26 Thread Josh Elser (JIRA)


 [ 
https://issues.apache.org/jira/browse/RATIS-276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Josh Elser updated RATIS-276:
-
Component/s: LogService

> "Tail" a log stream
> ---
>
> Key: RATIS-276
> URL: https://issues.apache.org/jira/browse/RATIS-276
> Project: Ratis
>  Issue Type: Sub-task
>  Components: LogService
>Reporter: Josh Elser
>Priority: Major
>
> We want the ability to "subscribe" to a log stream and automatically get 
> pushed updates when they are appended to the log stream.
> The analogy is calling the Unix "tail" command on a file.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (RATIS-271) Ratis-backed distributed log: "LogService"

2018-07-26 Thread Josh Elser (JIRA)


 [ 
https://issues.apache.org/jira/browse/RATIS-271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Josh Elser updated RATIS-271:
-
Component/s: LogService

> Ratis-backed distributed log: "LogService" 
> ---
>
> Key: RATIS-271
> URL: https://issues.apache.org/jira/browse/RATIS-271
> Project: Ratis
>  Issue Type: New Feature
>  Components: LogService
>Reporter: Josh Elser
>Priority: Major
>
> Umbrella issue for building a distributed log using Ratis:
> Doc: 
> [https://docs.google.com/document/d/1Su5py_T5Ytfh9RoTTX2s20KbSJwBHVxbO7ge5ORqbCk/edit#|https://docs.google.com/document/d/1Su5py_T5Ytfh9RoTTX2s20KbSJwBHVxbO7ge5ORqbCk/edit]
> Discuss: 
> https://lists.apache.org/thread.html/f80dc3900f6d9f4ee4d9f9e0898cee9a232e3b1ca9a4d9a53fea1d71@%3Cdev.ratis.apache.org%3E



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (RATIS-280) Support exporting of a LogStream to a remote filesystem

2018-07-26 Thread Josh Elser (JIRA)
Josh Elser created RATIS-280:


 Summary: Support exporting of a LogStream to a remote filesystem
 Key: RATIS-280
 URL: https://issues.apache.org/jira/browse/RATIS-280
 Project: Ratis
  Issue Type: Sub-task
Reporter: Josh Elser


LogStreams are kept on local filesystem as they are being written to; however, 
this is not a good solution for long-term storage.

We should provide the mechanism to invoke API that copies the log stream to a 
distributed filesystem (e.g. HDFS, S3, WASB), to reclaim the local FS space.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (RATIS-279) Create administrative API for a log stream

2018-07-26 Thread Josh Elser (JIRA)
Josh Elser created RATIS-279:


 Summary: Create administrative API for a log stream
 Key: RATIS-279
 URL: https://issues.apache.org/jira/browse/RATIS-279
 Project: Ratis
  Issue Type: Sub-task
Reporter: Josh Elser


We need to do basic things like:
 * List all log streams
 * Delete a log stream
 * Truncate a log stream

This may overlap with functionality that actually should live in HBase. Making 
that distinction is part of the tasks of this issue.

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (RATIS-271) Ratis-backed distributed log: "LogService"

2018-07-26 Thread Tsz Wo Nicholas Sze (JIRA)


[ 
https://issues.apache.org/jira/browse/RATIS-271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16558513#comment-16558513
 ] 

Tsz Wo Nicholas Sze commented on RATIS-271:
---

[~elserj], thanks for starting the LogService work!

I just have created a new "LogService" component for these works.

> Ratis-backed distributed log: "LogService" 
> ---
>
> Key: RATIS-271
> URL: https://issues.apache.org/jira/browse/RATIS-271
> Project: Ratis
>  Issue Type: New Feature
>Reporter: Josh Elser
>Priority: Major
>
> Umbrella issue for building a distributed log using Ratis:
> Doc: 
> [https://docs.google.com/document/d/1Su5py_T5Ytfh9RoTTX2s20KbSJwBHVxbO7ge5ORqbCk/edit#|https://docs.google.com/document/d/1Su5py_T5Ytfh9RoTTX2s20KbSJwBHVxbO7ge5ORqbCk/edit]
> Discuss: 
> https://lists.apache.org/thread.html/f80dc3900f6d9f4ee4d9f9e0898cee9a232e3b1ca9a4d9a53fea1d71@%3Cdev.ratis.apache.org%3E



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (RATIS-278) Export metrics about a LogStream

2018-07-26 Thread Josh Elser (JIRA)
Josh Elser created RATIS-278:


 Summary: Export metrics about a LogStream
 Key: RATIS-278
 URL: https://issues.apache.org/jira/browse/RATIS-278
 Project: Ratis
  Issue Type: Sub-task
Reporter: Josh Elser


We need to have metrics exported to tell us information about a LogStream's 
performance.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (RATIS-277) Create an interface to support quorum creation

2018-07-26 Thread Josh Elser (JIRA)
Josh Elser created RATIS-277:


 Summary: Create an interface to support quorum creation
 Key: RATIS-277
 URL: https://issues.apache.org/jira/browse/RATIS-277
 Project: Ratis
  Issue Type: Sub-task
Reporter: Josh Elser


Create an interface that allows users of the LogService to define how quorums 
are formed among nodes.

The use case is that HBase will have many RegionServers available to use. We 
want to allow HBase to control which RegionServers are used to participate in 
one LogStream.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (RATIS-276) "Tail" a log stream

2018-07-26 Thread Josh Elser (JIRA)
Josh Elser created RATIS-276:


 Summary: "Tail" a log stream
 Key: RATIS-276
 URL: https://issues.apache.org/jira/browse/RATIS-276
 Project: Ratis
  Issue Type: Sub-task
Reporter: Josh Elser


We want the ability to "subscribe" to a log stream and automatically get pushed 
updates when they are appended to the log stream.

The analogy is calling the Unix "tail" command on a file.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (RATIS-275) Read-path of log stream statemachine

2018-07-26 Thread Josh Elser (JIRA)
Josh Elser created RATIS-275:


 Summary: Read-path of log stream statemachine
 Key: RATIS-275
 URL: https://issues.apache.org/jira/browse/RATIS-275
 Project: Ratis
  Issue Type: Sub-task
Reporter: Josh Elser


Implement the ability to read from a log stream.

We want the ability to read linearly from a point in the log. Sub-linear seek 
into a given offset in the log.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (RATIS-274) Write-path of log stream statemachine

2018-07-26 Thread Josh Elser (JIRA)
Josh Elser created RATIS-274:


 Summary: Write-path of log stream statemachine
 Key: RATIS-274
 URL: https://issues.apache.org/jira/browse/RATIS-274
 Project: Ratis
  Issue Type: Sub-task
Reporter: Josh Elser


Implement the ability to write data to a log stream.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (RATIS-272) Design ideal API

2018-07-26 Thread Josh Elser (JIRA)
Josh Elser created RATIS-272:


 Summary: Design ideal API
 Key: RATIS-272
 URL: https://issues.apache.org/jira/browse/RATIS-272
 Project: Ratis
  Issue Type: Sub-task
Reporter: Josh Elser


With influence from Apache DistributedLog, Kafka, and BookKeeper, design an API 
that balances the ideal notion of what a distribute log system should look 
like, but also considers the needs of HBase to replace a WAL.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (RATIS-271) Ratis-backed distributed log: "LogService"

2018-07-26 Thread Josh Elser (JIRA)
Josh Elser created RATIS-271:


 Summary: Ratis-backed distributed log: "LogService" 
 Key: RATIS-271
 URL: https://issues.apache.org/jira/browse/RATIS-271
 Project: Ratis
  Issue Type: New Feature
Reporter: Josh Elser


Umbrella issue for building a distributed log using Ratis:

Doc: 
[https://docs.google.com/document/d/1Su5py_T5Ytfh9RoTTX2s20KbSJwBHVxbO7ge5ORqbCk/edit#|https://docs.google.com/document/d/1Su5py_T5Ytfh9RoTTX2s20KbSJwBHVxbO7ge5ORqbCk/edit]

Discuss: 
https://lists.apache.org/thread.html/f80dc3900f6d9f4ee4d9f9e0898cee9a232e3b1ca9a4d9a53fea1d71@%3Cdev.ratis.apache.org%3E



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (RATIS-259) Ratis waits infinitely with no timout when not able to connect to ozone datanode

2018-07-26 Thread Jing Chen (JIRA)


[ 
https://issues.apache.org/jira/browse/RATIS-259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16558081#comment-16558081
 ] 

Jing Chen commented on RATIS-259:
-

I believe we can merge RATIS-11 first, then we can simply apply proper retry 
policy here to solve this ticket.

> Ratis waits infinitely with no timout when not able to connect to ozone 
> datanode
> 
>
> Key: RATIS-259
> URL: https://issues.apache.org/jira/browse/RATIS-259
> Project: Ratis
>  Issue Type: Improvement
>  Components: client
>Reporter: Nilotpal Nandi
>Assignee: Nanda kumar
>Priority: Major
>  Labels: ozone
>
> In ozone, the pipeline type is Ratis with replication factor as 1.
> Ran the following command ozone command , it hung and did not complete
> 
> {noformat}
> [root@ozone-vm bin]# ./ozone oz -getKey /nnvolume1/buckettest1/passwd ./hello1
> Command Failed : {"httpCode":0,"shortMessage":"get key needs a file path to 
> download to","resource":null,"message":"get key needs a file path to download 
> to","requestID":null,"hostName":null}
> [root@ozone-vm bin]# ./ozone oz -getKey /nnvolume1/buckettest1/passwd -file .
> 2018-06-29 05:09:46,865 WARN util.NativeCodeLoader: Unable to load 
> native-hadoop library for your platform... using builtin-java classes where 
> applicable
> Command Failed : {"httpCode":0,"shortMessage":".exists. Download will 
> overwrite an existing file. Aborting.","resource":null,"message":".exists. 
> Download will overwrite an existing file. 
> Aborting.","requestID":null,"hostName":null}
> [root@ozone-vm bin]# ./ozone oz -getKey /nnvolume1/buckettest1/passwd -file 
> ./hello11
> 2018-06-29 05:10:27,661 WARN util.NativeCodeLoader: Unable to load 
> native-hadoop library for your platform... using builtin-java classes where 
> applicable
> 2018-06-29 05:10:28,373 INFO conf.ConfUtils: raft.rpc.type = GRPC (default)
> 2018-06-29 05:10:28,406 INFO conf.ConfUtils: raft.grpc.message.size.max = 
> 33554432 (custom)
> 2018-06-29 05:10:28,424 INFO conf.ConfUtils: raft.client.rpc.retryInterval = 
> 300 ms (default)
> 2018-06-29 05:10:28,428 INFO conf.ConfUtils: 
> raft.client.async.outstanding-requests.max = 100 (default)
> 2018-06-29 05:10:28,428 INFO conf.ConfUtils: 
> raft.client.async.scheduler-threads = 3 (default)
> 2018-06-29 05:10:28,595 INFO conf.ConfUtils: raft.grpc.flow.control.window = 
> 1MB (=1048576) (default)
> 2018-06-29 05:10:28,595 INFO conf.ConfUtils: raft.grpc.message.size.max = 
> 33554432 (custom)
> 2018-06-29 05:10:28,884 INFO conf.ConfUtils: raft.client.rpc.request.timeout 
> = 3000 ms (default){noformat}
>  
> jstack :
> 
>  
> {noformat}
> [root@ozone-vm logs]# jstack 4983
> 2018-06-29 05:31:00
> Full thread dump Java HotSpot(TM) 64-Bit Server VM (25.171-b11 mixed mode):
> "Attach Listener" #19 daemon prio=9 os_prio=0 tid=0x7fe9fc016000 
> nid=0x1804 waiting on condition [0x]
>  java.lang.Thread.State: RUNNABLE
> "threadDeathWatcher-3-1" #17 daemon prio=1 os_prio=0 tid=0x7fea191e5800 
> nid=0x13af waiting on condition [0x7fea02905000]
>  java.lang.Thread.State: TIMED_WAITING (sleeping)
>  at java.lang.Thread.sleep(Native Method)
>  at 
> org.apache.ratis.shaded.io.netty.util.ThreadDeathWatcher$Watcher.run(ThreadDeathWatcher.java:152)
>  at 
> org.apache.ratis.shaded.io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:138)
>  at java.lang.Thread.run(Thread.java:748)
> "grpc-default-worker-ELG-1-2" #16 daemon prio=5 os_prio=0 
> tid=0x7fea191d4000 nid=0x13ac runnable [0x7fea085fc000]
>  java.lang.Thread.State: RUNNABLE
>  at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
>  at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
>  at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
>  at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
>  - locked <0xe0c372f0> (a 
> org.apache.ratis.shaded.io.netty.channel.nio.SelectedSelectionKeySet)
>  - locked <0xe0c419c8> (a java.util.Collections$UnmodifiableSet)
>  - locked <0xe0c372a8> (a sun.nio.ch.EPollSelectorImpl)
>  at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
>  at 
> org.apache.ratis.shaded.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62)
>  at 
> org.apache.ratis.shaded.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:753)
>  at 
> org.apache.ratis.shaded.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:409)
>  at 
> org.apache.ratis.shaded.io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858)
>  at 
> 

[jira] [Updated] (RATIS-11) Better retry policy support for Raft client

2018-07-26 Thread Jing Chen (JIRA)


 [ 
https://issues.apache.org/jira/browse/RATIS-11?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jing Chen updated RATIS-11:
---
Attachment: 0001-RATIS-11-Better-retry-policy-support-for-Raft-client.patch

> Better retry policy support for Raft client
> ---
>
> Key: RATIS-11
> URL: https://issues.apache.org/jira/browse/RATIS-11
> Project: Ratis
>  Issue Type: Improvement
>  Components: client
>Reporter: Jing Zhao
>Assignee: Jing Chen
>Priority: Major
> Attachments: 
> 0001-RATIS-11-Better-retry-policy-support-for-Raft-client.patch
>
>
> We should have better retry policy support on the raft client side, such as 
> different retry policies, more flexible retry parameter settings, etc.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (RATIS-11) Better retry policy support for Raft client

2018-07-26 Thread Jing Chen (JIRA)


 [ 
https://issues.apache.org/jira/browse/RATIS-11?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jing Chen updated RATIS-11:
---
Attachment: (was: 
0001-RATIS-11-Better-retry-policy-support-for-Raft-client.patch)

> Better retry policy support for Raft client
> ---
>
> Key: RATIS-11
> URL: https://issues.apache.org/jira/browse/RATIS-11
> Project: Ratis
>  Issue Type: Improvement
>  Components: client
>Reporter: Jing Zhao
>Assignee: Jing Chen
>Priority: Major
> Attachments: 
> 0001-RATIS-11-Better-retry-policy-support-for-Raft-client.patch
>
>
> We should have better retry policy support on the raft client side, such as 
> different retry policies, more flexible retry parameter settings, etc.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (RATIS-11) Better retry policy support for Raft client

2018-07-26 Thread Jing Chen (JIRA)


[ 
https://issues.apache.org/jira/browse/RATIS-11?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16558055#comment-16558055
 ] 

Jing Chen commented on RATIS-11:


Hi [~szetszwo] , the patch is attached, could you have review those changes if 
you have any chance or forward the review request to the best reviewer. Thanks 
a lot.

> Better retry policy support for Raft client
> ---
>
> Key: RATIS-11
> URL: https://issues.apache.org/jira/browse/RATIS-11
> Project: Ratis
>  Issue Type: Improvement
>  Components: client
>Reporter: Jing Zhao
>Assignee: Jing Chen
>Priority: Major
> Attachments: 
> 0001-RATIS-11-Better-retry-policy-support-for-Raft-client.patch
>
>
> We should have better retry policy support on the raft client side, such as 
> different retry policies, more flexible retry parameter settings, etc.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (RATIS-11) Better retry policy support for Raft client

2018-07-26 Thread Jing Chen (JIRA)


 [ 
https://issues.apache.org/jira/browse/RATIS-11?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jing Chen updated RATIS-11:
---
Attachment: 0001-RATIS-11-Better-retry-policy-support-for-Raft-client.patch

> Better retry policy support for Raft client
> ---
>
> Key: RATIS-11
> URL: https://issues.apache.org/jira/browse/RATIS-11
> Project: Ratis
>  Issue Type: Improvement
>  Components: client
>Reporter: Jing Zhao
>Assignee: Jing Chen
>Priority: Major
> Attachments: 
> 0001-RATIS-11-Better-retry-policy-support-for-Raft-client.patch
>
>
> We should have better retry policy support on the raft client side, such as 
> different retry policies, more flexible retry parameter settings, etc.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)