[jira] [Assigned] (RATIS-657) GrpcClientProtocolClient logging is too verbose
[
https://issues.apache.org/jira/browse/RATIS-657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Siddharth Wagle reassigned RATIS-657:
-
Assignee: Siddharth Wagle
> GrpcClientProtocolClient logging is too verbose
> ---
>
> Key: RATIS-657
> URL: https://issues.apache.org/jira/browse/RATIS-657
> Project: Ratis
> Issue Type: Bug
> Components: client
>Affects Versions: 0.3.0
>Reporter: Mukul Kumar Singh
>Assignee: Siddharth Wagle
>Priority: Major
> Labels: ozone
>
> GrpcClientProtocolClient logging is too verbose, there are multiple instances
> of logging as following.
> {code}
> 19/08/14 12:08:47 INFO client.GrpcClientProtocolClient:
> client-9D2DF1304552->e89f9c37-e0e1-4026-b0cc-c6393c8e4777: receive
> RaftClientReply:client-9D2DF1304552->e89f9c37-e0e1-4026-b0cc-c6393c8e4777@group-533C90750B00,
> cid=1783, SUCCESS, logIndex=350,
> commits[e89f9c37-e0e1-4026-b0cc-c6393c8e4777:c350,
> 0fcf00d5-f7bd-4c92-841d-05627de071c9:c348,
> 23ee46ae-bf29-4654-895c-3481eb1249d6:c347]
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (RATIS-655) Change LeaderState and FollowerState and to use RaftGroupMemberId
[ https://issues.apache.org/jira/browse/RATIS-655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16921933#comment-16921933 ] Mukul Kumar Singh commented on RATIS-655: - Thanks for working on this [~szetszwo]. The patch looks good to me. Can we also remove references of getId() as well? > Change LeaderState and FollowerState and to use RaftGroupMemberId > -- > > Key: RATIS-655 > URL: https://issues.apache.org/jira/browse/RATIS-655 > Project: Ratis > Issue Type: Improvement > Components: server >Reporter: Tsz Wo Nicholas Sze >Assignee: Tsz Wo Nicholas Sze >Priority: Major > Attachments: r655_20190807.patch, r655_20190807b.patch, > r655_20190830.patch > > > This is the last JIRA split from the huge patch in RATIS-605. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (RATIS-672) Vagrant test's Maven 404's
[
https://issues.apache.org/jira/browse/RATIS-672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Clay B. updated RATIS-672:
--
Summary: Vagrant test's Maven 404's (was: Vagrant tests Maven 404's)
> Vagrant test's Maven 404's
> --
>
> Key: RATIS-672
> URL: https://issues.apache.org/jira/browse/RATIS-672
> Project: Ratis
> Issue Type: Improvement
> Components: test
>Reporter: Clay B.
>Assignee: Clay B.
>Priority: Trivial
>
> The Vagrant tests are
> [pointing|https://github.com/apache/incubator-ratis/blob/35838f032a4096d78843130fa1435bcddf5ce961/dev-support/vagrant/Vagrantfile#L92]
> to a specific mirror and a URL which broke with the new Maven 3.6.x release.
> We should point to an {{archive.apache.org}} URL if I understand correctly.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Created] (RATIS-672) Vagrant tests Maven 404's
Clay B. created RATIS-672:
-
Summary: Vagrant tests Maven 404's
Key: RATIS-672
URL: https://issues.apache.org/jira/browse/RATIS-672
Project: Ratis
Issue Type: Improvement
Components: test
Reporter: Clay B.
Assignee: Clay B.
The Vagrant tests are
[pointing|https://github.com/apache/incubator-ratis/blob/35838f032a4096d78843130fa1435bcddf5ce961/dev-support/vagrant/Vagrantfile#L92]
to a specific mirror and a URL which broke with the new Maven 3.6.x release.
We should point to an {{archive.apache.org}} URL if I understand correctly.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (RATIS-619) Avoid loading cache with pre-snapshot entries for the group
[
https://issues.apache.org/jira/browse/RATIS-619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16921683#comment-16921683
]
Tsz Wo Nicholas Sze commented on RATIS-619:
---
Why moving away from the original idea of skipping loading the entries before
the snapshot? It sounds even better.
Also, why this is a blocker?
> Avoid loading cache with pre-snapshot entries for the group
> ---
>
> Key: RATIS-619
> URL: https://issues.apache.org/jira/browse/RATIS-619
> Project: Ratis
> Issue Type: Bug
> Components: server
>Affects Versions: 0.3.0
>Reporter: Mukul Kumar Singh
>Assignee: Siddharth Wagle
>Priority: Blocker
> Labels: ozone
> Attachments: RATIS-619.01.patch, RATIS-619.02.patch
>
>
> Even after taking a snapshot, the raft log loads all the segment in the log
> {code}
> 2019-07-01 23:22:47,481 [pool-18-thread-1] INFO - Setting the last
> applied index to (t:2, i:15237039)
> {code}
> {code}
> 2019-07-01 23:22:47,516 INFO org.apache.ratis.server.RaftServerConfigKeys:
> raft.server.log.statemachine.data.caching.enabled = true (custom)
> 2019-07-01 23:22:47,531 INFO org.apache.ratis.server.impl.RaftServerImpl:
> 62941ca3-f244-4298-8497-f4c0bd57430a:group-4D230AB58084 set configuration 0:
> [1f3d7936-cb4e-4b68-86ed-578070472dea:1
> 0.17.213.36:9858, 62941ca3-f244-4298-8497-f4c0bd57430a:10.17.213.35:9858,
> f07c1f87-b377-40d9-8c56-4f1440c4fa77:10.17.213.37:9858], old=null at 0
> 2019-07-01 23:22:47,578 INFO org.apache.hadoop.http.HttpServer2: Jetty bound
> to port 9882
> 2019-07-01 23:22:47,579 INFO org.eclipse.jetty.server.Server:
> jetty-9.3.24.v20180605, build timestamp: 2018-06-05T10:11:56-07:00, git hash:
> 84205aa28f11a4f31f2a3b86d1bba2cc8ab69827
> 2019-07-01 23:22:47,601 INFO
> org.apache.ratis.server.raftlog.segmented.LogSegment: Successfully read 7461
> entries from segment file
> /data/1/ozone-0701/ratis/log/f7ddda32-45e0-4bec-a3e7-4d230
> ab58084/current/log_0-7460
> 2019-07-01 23:22:47,608 INFO org.eclipse.jetty.server.handler.ContextHandler:
> Started
> o.e.j.s.ServletContextHandler@6ce90bc5{/logs,file:///var/log/ozone/,AVAILABLE}
> 2019-07-01 23:22:47,608 INFO org.eclipse.jetty.server.handler.ContextHandler:
> Started
> o.e.j.s.ServletContextHandler@4b1c0397{/static,jar:file:/var/lib/hadoop-ozone/ozone-0.5.0-SNAPSHOT/share
> /ozone/lib/hadoop-hdds-container-service-0.5.0-SNAPSHOT.jar!/webapps/static,AVAILABLE}
> 2019-07-01 23:22:47,635 INFO
> org.apache.ratis.server.raftlog.segmented.LogSegment: Successfully read 7386
> entries from segment file
> /data/1/ozone-0701/ratis/log/f7ddda32-45e0-4bec-a3e7-4d230
> ab58084/current/log_7461-14846
> 2019-07-01 23:22:47,663 INFO
> org.apache.ratis.server.raftlog.segmented.LogSegment: Successfully read 7440
> entries from segment file
> /data/1/ozone-0701/ratis/log/f7ddda32-45e0-4bec-a3e7-4d230
> ab58084/current/log_14847-22286
> 2019-07-01 23:22:47,664 INFO org.eclipse.jetty.server.handler.ContextHandler:
> Started
> o.e.j.w.WebAppContext@8a62297{/,file:///tmp/jetty-0.0.0.0-9882-hddsDatanode-_-any-7539213566265642568.di
> r/webapp/,AVAILABLE}{/hddsDatanode}
> 2019-07-01 23:22:47,681 INFO
> org.apache.ratis.server.raftlog.segmented.LogSegment: Successfully read 7353
> entries from segment file
> /data/1/ozone-0701/ratis/log/f7ddda32-45e0-4bec-a3e7-4d230
> ab58084/current/log_22287-29639
> 2019-07-01 23:22:47,695 INFO org.eclipse.jetty.server.AbstractConnector:
> Started ServerConnector@5116ac09{HTTP/1.1,[http/1.1]}{0.0.0.0:9882}
> 2019-07-01 23:22:47,695 INFO
> org.apache.ratis.server.raftlog.segmented.LogSegment: Successfully read 7291
> entries from segment file
> /data/1/ozone-0701/ratis/log/f7ddda32-45e0-4bec-a3e7-4d230
> ab58084/current/log_29640-36930
> 2019-07-01 23:22:47,695 INFO org.eclipse.jetty.server.Server: Started @56648ms
> 2019-07-01 23:22:47,695 INFO org.apache.hadoop.hdds.server.BaseHttpServer:
> HTTP server of HDDSDATANODE is listening at http://0.0.0.0:9882
> 2019-07-01 23:22:47,709 INFO
> org.apache.ratis.server.raftlog.segmented.LogSegment: Successfully read 7049
> entries from segment file
> /data/1/ozone-0701/ratis/log/f7ddda32-45e0-4bec-a3e7-4d230
> ab58084/current/log_36931-43979
> 2019-07-01 23:22:47,732 INFO
> org.apache.ratis.server.raftlog.segmented.LogSegment: Successfully read 7141
> entries from segment file
> /data/1/ozone-0701/ratis/log/f7ddda32-45e0-4bec-a3e7-4d230
> ab58084/current/log_43980-51120
> 2019-07-01 23:22:47,747 INFO
> org.apache.ratis.server.raftlog.segmented.LogSegment: Successfully read 7321
> entries from segment file
> /data/1/ozone-0701/ratis/log/f7ddda32-45e0-4bec-a3e7-4d230ab58084/current/log_51121-58441
> 2019-07-01 23:22:47,768 INFO
> org.apache.ratis.server.raftlog.segmented.LogSegment: Successfully read
[jira] [Commented] (RATIS-668) Fix NOTICE file
[ https://issues.apache.org/jira/browse/RATIS-668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16921655#comment-16921655 ] Arpit Agarwal commented on RATIS-668: - v2 patch adds missing notices for DropWizard and JUnit. > Fix NOTICE file > --- > > Key: RATIS-668 > URL: https://issues.apache.org/jira/browse/RATIS-668 > Project: Ratis > Issue Type: Bug >Affects Versions: 0.4.0 >Reporter: Arpit Agarwal >Assignee: Arpit Agarwal >Priority: Blocker > Attachments: RATIS-668.01.patch, RATIS-668.02.patch > > > NOTICE file needs to be updated based on Justin's comments here: > > [https://mail-archives.apache.org/mod_mbox/incubator-general/201908.mbox/%3C8EA21F57-A972-4CBE-AC2F-D3830FE6BDB4%40classsoftware.com%3E] > > -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (RATIS-668) Fix NOTICE file
[ https://issues.apache.org/jira/browse/RATIS-668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Agarwal updated RATIS-668: Attachment: RATIS-668.02.patch > Fix NOTICE file > --- > > Key: RATIS-668 > URL: https://issues.apache.org/jira/browse/RATIS-668 > Project: Ratis > Issue Type: Bug >Affects Versions: 0.4.0 >Reporter: Arpit Agarwal >Assignee: Arpit Agarwal >Priority: Blocker > Attachments: RATIS-668.01.patch, RATIS-668.02.patch > > > NOTICE file needs to be updated based on Justin's comments here: > > [https://mail-archives.apache.org/mod_mbox/incubator-general/201908.mbox/%3C8EA21F57-A972-4CBE-AC2F-D3830FE6BDB4%40classsoftware.com%3E] > > -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (RATIS-668) Fix NOTICE file
[ https://issues.apache.org/jira/browse/RATIS-668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Agarwal updated RATIS-668: Attachment: RATIS-668.01.patch > Fix NOTICE file > --- > > Key: RATIS-668 > URL: https://issues.apache.org/jira/browse/RATIS-668 > Project: Ratis > Issue Type: Bug >Affects Versions: 0.4.0 >Reporter: Arpit Agarwal >Assignee: Arpit Agarwal >Priority: Blocker > Attachments: RATIS-668.01.patch > > > NOTICE file needs to be updated based on Justin's comments here: > > [https://mail-archives.apache.org/mod_mbox/incubator-general/201908.mbox/%3C8EA21F57-A972-4CBE-AC2F-D3830FE6BDB4%40classsoftware.com%3E] > > -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (RATIS-619) Avoid loading cache with pre-snapshot entries for the group
[
https://issues.apache.org/jira/browse/RATIS-619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Siddharth Wagle updated RATIS-619:
--
Summary: Avoid loading cache with pre-snapshot entries for the group (was:
Ratis server on restart loads all the entries for the group)
> Avoid loading cache with pre-snapshot entries for the group
> ---
>
> Key: RATIS-619
> URL: https://issues.apache.org/jira/browse/RATIS-619
> Project: Ratis
> Issue Type: Bug
> Components: server
>Affects Versions: 0.3.0
>Reporter: Mukul Kumar Singh
>Assignee: Siddharth Wagle
>Priority: Blocker
> Labels: ozone
> Attachments: RATIS-619.01.patch, RATIS-619.02.patch
>
>
> Even after taking a snapshot, the raft log loads all the segment in the log
> {code}
> 2019-07-01 23:22:47,481 [pool-18-thread-1] INFO - Setting the last
> applied index to (t:2, i:15237039)
> {code}
> {code}
> 2019-07-01 23:22:47,516 INFO org.apache.ratis.server.RaftServerConfigKeys:
> raft.server.log.statemachine.data.caching.enabled = true (custom)
> 2019-07-01 23:22:47,531 INFO org.apache.ratis.server.impl.RaftServerImpl:
> 62941ca3-f244-4298-8497-f4c0bd57430a:group-4D230AB58084 set configuration 0:
> [1f3d7936-cb4e-4b68-86ed-578070472dea:1
> 0.17.213.36:9858, 62941ca3-f244-4298-8497-f4c0bd57430a:10.17.213.35:9858,
> f07c1f87-b377-40d9-8c56-4f1440c4fa77:10.17.213.37:9858], old=null at 0
> 2019-07-01 23:22:47,578 INFO org.apache.hadoop.http.HttpServer2: Jetty bound
> to port 9882
> 2019-07-01 23:22:47,579 INFO org.eclipse.jetty.server.Server:
> jetty-9.3.24.v20180605, build timestamp: 2018-06-05T10:11:56-07:00, git hash:
> 84205aa28f11a4f31f2a3b86d1bba2cc8ab69827
> 2019-07-01 23:22:47,601 INFO
> org.apache.ratis.server.raftlog.segmented.LogSegment: Successfully read 7461
> entries from segment file
> /data/1/ozone-0701/ratis/log/f7ddda32-45e0-4bec-a3e7-4d230
> ab58084/current/log_0-7460
> 2019-07-01 23:22:47,608 INFO org.eclipse.jetty.server.handler.ContextHandler:
> Started
> o.e.j.s.ServletContextHandler@6ce90bc5{/logs,file:///var/log/ozone/,AVAILABLE}
> 2019-07-01 23:22:47,608 INFO org.eclipse.jetty.server.handler.ContextHandler:
> Started
> o.e.j.s.ServletContextHandler@4b1c0397{/static,jar:file:/var/lib/hadoop-ozone/ozone-0.5.0-SNAPSHOT/share
> /ozone/lib/hadoop-hdds-container-service-0.5.0-SNAPSHOT.jar!/webapps/static,AVAILABLE}
> 2019-07-01 23:22:47,635 INFO
> org.apache.ratis.server.raftlog.segmented.LogSegment: Successfully read 7386
> entries from segment file
> /data/1/ozone-0701/ratis/log/f7ddda32-45e0-4bec-a3e7-4d230
> ab58084/current/log_7461-14846
> 2019-07-01 23:22:47,663 INFO
> org.apache.ratis.server.raftlog.segmented.LogSegment: Successfully read 7440
> entries from segment file
> /data/1/ozone-0701/ratis/log/f7ddda32-45e0-4bec-a3e7-4d230
> ab58084/current/log_14847-22286
> 2019-07-01 23:22:47,664 INFO org.eclipse.jetty.server.handler.ContextHandler:
> Started
> o.e.j.w.WebAppContext@8a62297{/,file:///tmp/jetty-0.0.0.0-9882-hddsDatanode-_-any-7539213566265642568.di
> r/webapp/,AVAILABLE}{/hddsDatanode}
> 2019-07-01 23:22:47,681 INFO
> org.apache.ratis.server.raftlog.segmented.LogSegment: Successfully read 7353
> entries from segment file
> /data/1/ozone-0701/ratis/log/f7ddda32-45e0-4bec-a3e7-4d230
> ab58084/current/log_22287-29639
> 2019-07-01 23:22:47,695 INFO org.eclipse.jetty.server.AbstractConnector:
> Started ServerConnector@5116ac09{HTTP/1.1,[http/1.1]}{0.0.0.0:9882}
> 2019-07-01 23:22:47,695 INFO
> org.apache.ratis.server.raftlog.segmented.LogSegment: Successfully read 7291
> entries from segment file
> /data/1/ozone-0701/ratis/log/f7ddda32-45e0-4bec-a3e7-4d230
> ab58084/current/log_29640-36930
> 2019-07-01 23:22:47,695 INFO org.eclipse.jetty.server.Server: Started @56648ms
> 2019-07-01 23:22:47,695 INFO org.apache.hadoop.hdds.server.BaseHttpServer:
> HTTP server of HDDSDATANODE is listening at http://0.0.0.0:9882
> 2019-07-01 23:22:47,709 INFO
> org.apache.ratis.server.raftlog.segmented.LogSegment: Successfully read 7049
> entries from segment file
> /data/1/ozone-0701/ratis/log/f7ddda32-45e0-4bec-a3e7-4d230
> ab58084/current/log_36931-43979
> 2019-07-01 23:22:47,732 INFO
> org.apache.ratis.server.raftlog.segmented.LogSegment: Successfully read 7141
> entries from segment file
> /data/1/ozone-0701/ratis/log/f7ddda32-45e0-4bec-a3e7-4d230
> ab58084/current/log_43980-51120
> 2019-07-01 23:22:47,747 INFO
> org.apache.ratis.server.raftlog.segmented.LogSegment: Successfully read 7321
> entries from segment file
> /data/1/ozone-0701/ratis/log/f7ddda32-45e0-4bec-a3e7-4d230ab58084/current/log_51121-58441
> 2019-07-01 23:22:47,768 INFO
> org.apache.ratis.server.raftlog.segmented.LogSegment: Successfully read 7081
> entries from segment file
>
[jira] [Commented] (RATIS-619) Ratis server on restart loads all the entries for the group
[
https://issues.apache.org/jira/browse/RATIS-619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16921633#comment-16921633
]
Mukul Kumar Singh commented on RATIS-619:
-
[~szetszwo], the current patch will help in avoiding memory/ cache being
populated with entries which will be evicted pretty soon. So this patch will
help with better memory utilization during restart.
> Ratis server on restart loads all the entries for the group
> ---
>
> Key: RATIS-619
> URL: https://issues.apache.org/jira/browse/RATIS-619
> Project: Ratis
> Issue Type: Bug
> Components: server
>Affects Versions: 0.3.0
>Reporter: Mukul Kumar Singh
>Assignee: Siddharth Wagle
>Priority: Blocker
> Labels: ozone
> Attachments: RATIS-619.01.patch, RATIS-619.02.patch
>
>
> Even after taking a snapshot, the raft log loads all the segment in the log
> {code}
> 2019-07-01 23:22:47,481 [pool-18-thread-1] INFO - Setting the last
> applied index to (t:2, i:15237039)
> {code}
> {code}
> 2019-07-01 23:22:47,516 INFO org.apache.ratis.server.RaftServerConfigKeys:
> raft.server.log.statemachine.data.caching.enabled = true (custom)
> 2019-07-01 23:22:47,531 INFO org.apache.ratis.server.impl.RaftServerImpl:
> 62941ca3-f244-4298-8497-f4c0bd57430a:group-4D230AB58084 set configuration 0:
> [1f3d7936-cb4e-4b68-86ed-578070472dea:1
> 0.17.213.36:9858, 62941ca3-f244-4298-8497-f4c0bd57430a:10.17.213.35:9858,
> f07c1f87-b377-40d9-8c56-4f1440c4fa77:10.17.213.37:9858], old=null at 0
> 2019-07-01 23:22:47,578 INFO org.apache.hadoop.http.HttpServer2: Jetty bound
> to port 9882
> 2019-07-01 23:22:47,579 INFO org.eclipse.jetty.server.Server:
> jetty-9.3.24.v20180605, build timestamp: 2018-06-05T10:11:56-07:00, git hash:
> 84205aa28f11a4f31f2a3b86d1bba2cc8ab69827
> 2019-07-01 23:22:47,601 INFO
> org.apache.ratis.server.raftlog.segmented.LogSegment: Successfully read 7461
> entries from segment file
> /data/1/ozone-0701/ratis/log/f7ddda32-45e0-4bec-a3e7-4d230
> ab58084/current/log_0-7460
> 2019-07-01 23:22:47,608 INFO org.eclipse.jetty.server.handler.ContextHandler:
> Started
> o.e.j.s.ServletContextHandler@6ce90bc5{/logs,file:///var/log/ozone/,AVAILABLE}
> 2019-07-01 23:22:47,608 INFO org.eclipse.jetty.server.handler.ContextHandler:
> Started
> o.e.j.s.ServletContextHandler@4b1c0397{/static,jar:file:/var/lib/hadoop-ozone/ozone-0.5.0-SNAPSHOT/share
> /ozone/lib/hadoop-hdds-container-service-0.5.0-SNAPSHOT.jar!/webapps/static,AVAILABLE}
> 2019-07-01 23:22:47,635 INFO
> org.apache.ratis.server.raftlog.segmented.LogSegment: Successfully read 7386
> entries from segment file
> /data/1/ozone-0701/ratis/log/f7ddda32-45e0-4bec-a3e7-4d230
> ab58084/current/log_7461-14846
> 2019-07-01 23:22:47,663 INFO
> org.apache.ratis.server.raftlog.segmented.LogSegment: Successfully read 7440
> entries from segment file
> /data/1/ozone-0701/ratis/log/f7ddda32-45e0-4bec-a3e7-4d230
> ab58084/current/log_14847-22286
> 2019-07-01 23:22:47,664 INFO org.eclipse.jetty.server.handler.ContextHandler:
> Started
> o.e.j.w.WebAppContext@8a62297{/,file:///tmp/jetty-0.0.0.0-9882-hddsDatanode-_-any-7539213566265642568.di
> r/webapp/,AVAILABLE}{/hddsDatanode}
> 2019-07-01 23:22:47,681 INFO
> org.apache.ratis.server.raftlog.segmented.LogSegment: Successfully read 7353
> entries from segment file
> /data/1/ozone-0701/ratis/log/f7ddda32-45e0-4bec-a3e7-4d230
> ab58084/current/log_22287-29639
> 2019-07-01 23:22:47,695 INFO org.eclipse.jetty.server.AbstractConnector:
> Started ServerConnector@5116ac09{HTTP/1.1,[http/1.1]}{0.0.0.0:9882}
> 2019-07-01 23:22:47,695 INFO
> org.apache.ratis.server.raftlog.segmented.LogSegment: Successfully read 7291
> entries from segment file
> /data/1/ozone-0701/ratis/log/f7ddda32-45e0-4bec-a3e7-4d230
> ab58084/current/log_29640-36930
> 2019-07-01 23:22:47,695 INFO org.eclipse.jetty.server.Server: Started @56648ms
> 2019-07-01 23:22:47,695 INFO org.apache.hadoop.hdds.server.BaseHttpServer:
> HTTP server of HDDSDATANODE is listening at http://0.0.0.0:9882
> 2019-07-01 23:22:47,709 INFO
> org.apache.ratis.server.raftlog.segmented.LogSegment: Successfully read 7049
> entries from segment file
> /data/1/ozone-0701/ratis/log/f7ddda32-45e0-4bec-a3e7-4d230
> ab58084/current/log_36931-43979
> 2019-07-01 23:22:47,732 INFO
> org.apache.ratis.server.raftlog.segmented.LogSegment: Successfully read 7141
> entries from segment file
> /data/1/ozone-0701/ratis/log/f7ddda32-45e0-4bec-a3e7-4d230
> ab58084/current/log_43980-51120
> 2019-07-01 23:22:47,747 INFO
> org.apache.ratis.server.raftlog.segmented.LogSegment: Successfully read 7321
> entries from segment file
> /data/1/ozone-0701/ratis/log/f7ddda32-45e0-4bec-a3e7-4d230ab58084/current/log_51121-58441
> 2019-07-01 23:22:47,768 INFO
>
[jira] [Commented] (RATIS-661) Add call in state machine to handle group removal
[ https://issues.apache.org/jira/browse/RATIS-661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16921628#comment-16921628 ] Tsz Wo Nicholas Sze commented on RATIS-661: --- > I mean an api which can tell that the group is in Closing or Closed state. ... We should add such information to getGroupInfos. We may as well change it to return a reply instead of GroupMismatchException when the group does not exist. > Add call in state machine to handle group removal > - > > Key: RATIS-661 > URL: https://issues.apache.org/jira/browse/RATIS-661 > Project: Ratis > Issue Type: New Feature > Components: API >Reporter: Lokesh Jain >Assignee: Lokesh Jain >Priority: Major > Fix For: 0.4.0 > > Attachments: RATIS-661.001.patch, RATIS-661.002.patch, > RATIS-661.003.patch, RATIS-661.004.patch, RATIS-661.005.patch > > > Currently during RaftServerProxy#groupRemoveAsync there is no way for > stateMachine to know that the RaftGroup will be removed. This Jira aims to > add a call in the stateMachine to handle group removal. > It also changes the logic of groupRemoval api to remove the RaftServerImpl > from the RaftServerProxy#impls map after the shutdown is complete. This is > required to synchronize the removal with the corresponding api of > RaftServer#getGroupIds. RaftServer#getGroupIds uses the RaftServerProxy#impls > map to get the groupIds. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (RATIS-671) Too many "Request not found, ignoring reply" log produced
[ https://issues.apache.org/jira/browse/RATIS-671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16921521#comment-16921521 ] Mukul Kumar Singh commented on RATIS-671: - Thanks for creating this issue [~Sammi]. Just to add onto this, lets add a metrics for this case. > Too many "Request not found, ignoring reply" log produced > - > > Key: RATIS-671 > URL: https://issues.apache.org/jira/browse/RATIS-671 > Project: Ratis > Issue Type: Improvement >Reporter: Sammi Chen >Priority: Major > > If this log means there is no concern, I would suggest to lower its log level > to debug. > 2019-09-03 19:37:31,931 WARN org.apache.ratis.grpc.server.GrpcLogAppender: > org.apache.ratis.grpc.server.GrpcLogAppender$AppendLogResponseHandler@7dfaa82c: > Request not found, ignoring reply: > ba17ad5e-714e-4d82-85d8-ff2e0737fcf9<-1c08cf06-4612-4e97-b159-babbd3747bc8#3199:FAIL,NOT_LEADER,nextIndex:1486,term:6,followerCommit:1480 -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Created] (RATIS-671) Too many "Request not found, ignoring reply" log produced
Sammi Chen created RATIS-671: Summary: Too many "Request not found, ignoring reply" log produced Key: RATIS-671 URL: https://issues.apache.org/jira/browse/RATIS-671 Project: Ratis Issue Type: Improvement Reporter: Sammi Chen If this log means there is no concern, I would suggest to lower its log level to debug. 2019-09-03 19:37:31,931 WARN org.apache.ratis.grpc.server.GrpcLogAppender: org.apache.ratis.grpc.server.GrpcLogAppender$AppendLogResponseHandler@7dfaa82c: Request not found, ignoring reply: ba17ad5e-714e-4d82-85d8-ff2e0737fcf9<-1c08cf06-4612-4e97-b159-babbd3747bc8#3199:FAIL,NOT_LEADER,nextIndex:1486,term:6,followerCommit:1480 -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (RATIS-556) Detect node failures and close the log to prevent additional writes
[ https://issues.apache.org/jira/browse/RATIS-556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16921350#comment-16921350 ] Rajeshbabu Chintaguntla commented on RATIS-556: --- bq. We should get a big logger WARN message saying that we're closing the log. Done. bq.Should we be checking anything in this Reply? Checking for any failures now and retry on exception. bq.This isn't quite right. (new PeerGroups[1]).length is always greater than 0, but peerGroupsToRemove[0] may be null. Make this a List and just append (potentially) multiple PeerGroups to it? Correct [~elserj] fixed it. bq.Would it be possible to modify that test or add a new test which makes sure that the contents of each data structure we maintain are kept in sync? I am talking about map, peers, peerLogs,heartbeatInfo and avail? However you think easiest to test it would be good. We wouldn't want these data structures to drift and become out of sync (as they would just leak memory). Checking for peers are in sync in this data structures before closing log and after closing the log. > Detect node failures and close the log to prevent additional writes > --- > > Key: RATIS-556 > URL: https://issues.apache.org/jira/browse/RATIS-556 > Project: Ratis > Issue Type: Improvement >Reporter: Rajeshbabu Chintaguntla >Assignee: Rajeshbabu Chintaguntla >Priority: Major > Attachments: RATIS-556-wip.patch, RATIS-556_v1.patch, > RATIS-556_v2.patch, RATIS-556_v3.patch, RATIS-556_v4.patch > > > Currently there is no way to detect the node failures at master log servers > and add new nodes to the group serving the log. We need to analyze how Ozone > is working in this case. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (RATIS-556) Detect node failures and close the log to prevent additional writes
[ https://issues.apache.org/jira/browse/RATIS-556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajeshbabu Chintaguntla updated RATIS-556: -- Attachment: RATIS-556_v4.patch > Detect node failures and close the log to prevent additional writes > --- > > Key: RATIS-556 > URL: https://issues.apache.org/jira/browse/RATIS-556 > Project: Ratis > Issue Type: Improvement >Reporter: Rajeshbabu Chintaguntla >Assignee: Rajeshbabu Chintaguntla >Priority: Major > Attachments: RATIS-556-wip.patch, RATIS-556_v1.patch, > RATIS-556_v2.patch, RATIS-556_v3.patch, RATIS-556_v4.patch > > > Currently there is no way to detect the node failures at master log servers > and add new nodes to the group serving the log. We need to analyze how Ozone > is working in this case. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (RATIS-661) Add call in state machine to handle group removal
[ https://issues.apache.org/jira/browse/RATIS-661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16921316#comment-16921316 ] Lokesh Jain commented on RATIS-661: --- [~szetszwo] Thanks for reviewing and committing the patch! | getGroupInfos covers groupExists, although it will fail with GroupMismatchException when a group does not exist. I mean an api which can tell that the group is in Closing or Closed state. We can do it using the StateMachine#notifyGroupRemove api but I was thinking if we could add this functionality in the RaftServer class? If the api returns null then group does not exist in any state in the raft server. > Add call in state machine to handle group removal > - > > Key: RATIS-661 > URL: https://issues.apache.org/jira/browse/RATIS-661 > Project: Ratis > Issue Type: New Feature > Components: API >Reporter: Lokesh Jain >Assignee: Lokesh Jain >Priority: Major > Fix For: 0.4.0 > > Attachments: RATIS-661.001.patch, RATIS-661.002.patch, > RATIS-661.003.patch, RATIS-661.004.patch, RATIS-661.005.patch > > > Currently during RaftServerProxy#groupRemoveAsync there is no way for > stateMachine to know that the RaftGroup will be removed. This Jira aims to > add a call in the stateMachine to handle group removal. > It also changes the logic of groupRemoval api to remove the RaftServerImpl > from the RaftServerProxy#impls map after the shutdown is complete. This is > required to synchronize the removal with the corresponding api of > RaftServer#getGroupIds. RaftServer#getGroupIds uses the RaftServerProxy#impls > map to get the groupIds. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (RATIS-659) StateMachineUpdater#stopAndJoin might not take snapshot due to race condition
[ https://issues.apache.org/jira/browse/RATIS-659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16921231#comment-16921231 ] Lokesh Jain commented on RATIS-659: --- [~szetszwo] Thanks for reviewing and committing the patch! > StateMachineUpdater#stopAndJoin might not take snapshot due to race condition > - > > Key: RATIS-659 > URL: https://issues.apache.org/jira/browse/RATIS-659 > Project: Ratis > Issue Type: Bug > Components: server >Reporter: Lokesh Jain >Assignee: Lokesh Jain >Priority: Major > Fix For: 0.4.0 > > Attachments: RATIS-659.001.patch > > > StateMachineUpdater might not take snapshot during close. This might happen > if the StateMachineUpdater#stopAndJoin is called right after the snapshot > check in StateMachineUpdater:156-162. -- This message was sent by Atlassian Jira (v8.3.2#803003)
