[
https://issues.apache.org/jira/browse/CASSANDRA-10439?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14941871#comment-14941871
]
Paulo Motta commented on CASSANDRA-10439:
-----------------------------------------
Hello [~gvenka008c], this is not a support forum. You should only open tickets
on this public JIRA when you think you're hitting a bug or want to request a
new feature on Cassandra (this is typically the last resort after you try to
find help on other locations).
You may find troubleshooting help on the following locations:
- irc.freenode.net - #cassandra channel
- mailing lists: http://www.planetcassandra.org/apache-cassandra-mailing-lists/
- DSE support: https://support.datastax.com/ (if you're a DSE subscriber)
Will close the ticket for now, feel free to reopen if you think you're hitting
an bug on Cassandra or want to request a new feature. Thanks for your
understanding.
> Cassandra Read Request Latency
> ------------------------------
>
> Key: CASSANDRA-10439
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10439
> Project: Cassandra
> Issue Type: Task
> Environment: PROD
> Reporter: Govindaraj
> Attachments: Screen Shot 2015-10-02 at 1.40.31 PM.png, Screen Shot
> 2015-10-02 at 12.08.06 PM.png, Screen Shot 2015-10-02 at 2.28.31 PM.png,
> Screen Shot 2015-10-02 at 2.28.39 PM.png, Screen Shot 2015-10-02 at 2.28.48
> PM.png, Screen Shot 2015-10-02 at 2.28.54 PM.png, Screen Shot 2015-10-02 at
> 2.39.57 PM.png
>
>
> Hi Team,
> Our PROD environment has two Data Centers configured as below.
> DC1 - Has 3 Cassandra nodes (dse 4.7.0)
> DC2 - Has 3 Cassandra nodes (dse 4.7.0)
> We are seeing the below issues repeatedly
> 1. Repeated alerts for Cassandra Read Request Latency. ReadStage has a lot of
> Pending queue.
> #nodetool tpstats
> Pool Name Active Pending Completed Blocked All
> time blocked
> MutationStage 0 0 8493347 0
> 0
> ReadStage 32 3699 5835 0
> 0
> Also during the same time we see a high CPU load on the cassandra nodes
> top
> top - 14:41:00 up 6 days, 21:58, 2 users, load average: 33.75, 27.80, 21.41
> Tasks: 226 total, 1 running, 225 sleeping, 0 stopped, 0 zombie
> Cpu(s): 96.8%us, 1.8%sy, 1.1%ni, 0.2%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
> Mem: 16331716k total, 16016864k used, 314852k free, 189380k buffers
> Swap: 0k total, 0k used, 0k free, 5786164k cached
> 2. We stopped the COMPACTION on the nodes were CPU load is high. Immediately
> the load came down and the read stage queue was cleared. All the pending
> tasks were processed in the Read Stage queue.
> Can you please suggest what might be causing this issue? Also how can we
> troubleshoot and fix it?
> Thanks,
> Venky
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
