The issue was persisting on a few nodes despite no changes to the topology. Even node restarting did not help. Only after we evacuated those nodes, the issue got resolved.
Do you think of a possible situation under which this could happen? Jaydeep On Sat, Oct 21, 2023 at 10:25 AM Jaydeep Chovatia < chovatia.jayd...@gmail.com> wrote: > Thanks, Jeff! > I will keep this thread updated on our findings. > > Jaydeep > > On Sat, Oct 21, 2023 at 9:37 AM Jeff Jirsa <jji...@gmail.com> wrote: > >> That code path was added to protect against invalid gossip states >> >> For this logger to be issued, the coordinator receiving the query must >> identify a set of replicas holding the data to serve the read, and one of >> the selected replicas must disagree that it’s a replica based on its view >> of the token ring >> >> This probably means that at least one node in your cluster has an invalid >> view of the ring - if you issue a “nodetool ring” from every host and >> compare them, you’ll probably notice one or more is wrong >> >> It’s also possible this happens for a few seconds during adding / moving >> / removing hosts >> >> If you weren’t changing the topology of the cluster, it’s likely the >> case that bouncing the cluster fixes it >> >> (Im unsure of the defaults and not able to look it up, but cassandra can >> log or log and drop the read - you probably want to drop the read log, >> which is the right solution so it doesn’t accidentally return a missing / >> empty result set as a valid query result, instead it’ll force it to read >> from other replicas or time out) >> >> >> >> >> >> On Oct 20, 2023, at 10:57 PM, Jaydeep Chovatia < >> chovatia.jayd...@gmail.com> wrote: >> >> >> >> Hi, >> >> I am using Cassandra 4.0.6 in production, and receiving the following error. >> This indicates that Cassandra nodes have mismatch in token-owership. >> >> Has anyone seen this issue before? >> >> Received a read request from /XX.XX.XXX.XXX:YYYYY for a range that is not >> owned by the current replica Read(keyspace.table columns=*/[c1] rowFilter= >> limits=LIMIT 100 key=7BE78B90-AD66-406B-AA05-6A062F72F542:0 >> filter=slice(slices=ALL, reversed=false), nowInSec=1697751757). >> >> Jaydeep >> >>
