Attaching the runner log snippet, where we can see that "Rebuilding token map" took most of the time. getAllroles is using quorum, don't if it is used during login https://github.com/apache/cassandra/blob/cc12665bb7645d17ba70edcf952ee6a1ea63127b/src/java/org/apache/cassandra/auth/CassandraRoleManager.java#L260
Vitali Djatsuk, On Fri, Nov 23, 2018 at 8:32 PM Jeff Jirsa <jji...@gmail.com> wrote: > I suspect some of the intermediate queries (determining role, etc) happen > at quorum in 2.2+, but I don’t have time to go read the code and prove it. > > In any case, RF > 10 per DC is probably excessive > > Also want to crank up the validity times so it uses cached info longer > > > -- > Jeff Jirsa > > > On Nov 23, 2018, at 10:18 AM, Vitali Dyachuk <vdjat...@gmail.com> wrote: > > no its not a cassandra user and as i understood all other users login > local_one. > > On Fri, 23 Nov 2018, 19:30 Jonathan Haddad <j...@jonhaddad.com wrote: > >> Any chance you’re logging in with the Cassandra user? It uses quorum >> reads. >> >> >> On Fri, Nov 23, 2018 at 11:38 AM Vitali Dyachuk <vdjat...@gmail.com> >> wrote: >> >>> Hi, >>> We have recently met a problem when we added 60 nodes in 1 region to the >>> cluster >>> and set an RF=60 for the system_auth ks, following this documentation >>> https://docs.datastax.com/en/cql/3.3/cql/cql_using/useUpdateKeyspaceRF.html >>> However we've started to see increased login latencies in the cluster 5x >>> bigger than before changing RF of system_auth ks. >>> We have casandra runner written is csharp, running against the cluster, >>> when analyzing the logs we notices that Rebuilding token map is >>> taking most of the time ~20s. >>> When we changed RF to 3 the issue has resolved. >>> We are using C* 3.0.17 , 4 DC, system_auth RF=3, "CassandraCSharpDriver" >>> version="3.2.1" >>> I've found somehow related to my problem ticket >>> https://datastax-oss.atlassian.net/browse/CSHARP-436 but it says in the >>> related tickets, that the issue with the token map rebuild time has been >>> fixed in the previous versions of the driver. >>> So my question is what is the best recommendation of the setting >>> system_auth ks RF? >>> >>> Regards, >>> Vitali Djatsuk. >>> >>> >>> -- >> Jon Haddad >> http://www.rustyrazorblade.com >> twitter: rustyrazorblade >> >
ControlConnection: 11/22/2018 10:30:32.170 +00:00 : Trying to connect the ControlConnection TcpSocket: 11/22/2018 10:30:32.170 +00:00 Socket connected, starting SSL client authentication TcpSocket: 11/22/2018 10:30:32.170 +00:00 Starting SSL authentication TcpSocket: 11/22/2018 10:30:32.217 +00:00 SSL authentication successful Connection: 11/22/2018 10:30:32.217 +00:00 Sending #0 for StartupRequest to node1:9042 Connection: 11/22/2018 10:30:32.217 +00:00 Received #0 from node1:9042 Connection: 11/22/2018 10:30:32.217 +00:00 Sending #0 for AuthResponseRequest to node1:9042 Connection: 11/22/2018 10:30:32.329 +00:00 Received #0 from node1:9042 ControlConnection: 11/22/2018 10:30:32.329 +00:00 : Connection established to node1:9042 Connection: 11/22/2018 10:30:32.329 +00:00 Sending #0 for RegisterForEventRequest to node1:9042 Connection: 11/22/2018 10:30:32.329 +00:00 Received #0 from node1:9042 ControlConnection: 11/22/2018 10:30:32.329 +00:00 : Refreshing node list Connection: 11/22/2018 10:30:32.329 +00:00 Sending #0 for QueryRequest to node1:9042 Connection: 11/22/2018 10:30:32.342 +00:00 Received #0 from node1:9042 Connection: 11/22/2018 10:30:32.342 +00:00 Sending #0 for QueryRequest to node1:9042 Connection: 11/22/2018 10:30:32.373 +00:00 Received #0 from node1:9042 ControlConnection: 11/22/2018 10:30:32.389 +00:00 : Node list retrieved successfully ControlConnection: 11/22/2018 10:30:32.389 +00:00 : Retrieving keyspaces metadata Connection: 11/22/2018 10:30:32.389 +00:00 Sending #0 for QueryRequest to node1:9042 Connection: 11/22/2018 10:30:32.389 +00:00 Received #0 from node1:9042 ControlConnection: 11/22/2018 10:30:32.389 +00:00 : Rebuilding token map Cluster: 11/22/2018 10:30:55.233 +00:00 : Cluster Connected using binary protocol version: [4]
--------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org For additional commands, e-mail: user-h...@cassandra.apache.org
