Re: Inconsistent results after restore with Cassandra 3.11.1

Singh wrote: > Can you define "inconsistent" results.. ? What's the topology of the > cluster? What were you expecting and what did you get? > > On Thu, Mar 14, 2019 at 7:09 AM sandeep nethi > wrote: > >> Hello, >> >> Does anyone experience

Re: Inconsistent results after restore with Cassandra 3.11.1

Can you define "inconsistent" results.. ? What's the topology of the cluster? What were you expecting and what did you get? On Thu, Mar 14, 2019 at 7:09 AM sandeep nethi wrote: > Hello, > > Does anyone experience inconsistent results after restoring Cassandra > 3.11.

Inconsistent results after restore with Cassandra 3.11.1

Hello, Does anyone experience inconsistent results after restoring Cassandra 3.11.1 with refresh command? Was there any bug in this version of cassandra?? Thanks in advance. Regards, Sandeep

Re: Very odd & inconsistent results from SASI query

Apologies for the stream-of-consciousness replies, but are the dropped message stats output by tpstats an accumulation since the node came up, or are there processes which clear and/or time-out the info? On Mon, Mar 20, 2017 at 3:18 PM, Voytek Jarnot wrote: > No dropped

Re: Very odd & inconsistent results from SASI query

No dropped messages in tpstats on any of the nodes. On Mon, Mar 20, 2017 at 3:11 PM, Voytek Jarnot wrote: > Appreciate the reply, Kurt. > > I sanitized it out of the traces, but all trace outputs listed the same > node for all three queries (1 working, 2 not working).

Re: Very odd & inconsistent results from SASI query

Appreciate the reply, Kurt. I sanitized it out of the traces, but all trace outputs listed the same node for all three queries (1 working, 2 not working). Read repair chance set to 0.0 as recommended when using TWCS. I'll check tpstats - in this environment, load is not an issue, but network

Re: Very odd & inconsistent results from SASI query

As secondary indexes are stored individually on each node what you're suggesting sounds exactly like a consistency issue. the fact that you read 0 cells on one query implies the node that got the query did not have any data for the row. The reason you would sometimes see different behaviours is

Re: Very odd & inconsistent results from SASI query

A wrinkle further confounds the issue: running a repair on the node which was servicing the queries has cleared things up and all the queries now work. That doesn't make a whole lot of sense to me - my assumption was that a repair shouldn't have fixed it. On Fri, Mar 17, 2017 at 12:03 PM, Voytek

Very odd & inconsistent results from SASI query

Cassandra 3.9, 4 nodes, rf=3 Hi folks, we're see 0 results returned from queries that (a) should return results, and (b) will return results with minor tweaks. I've attached the sanitized trace outputs for the following 3 queries (pk1 and pk2 are partition keys, ck1 is clustering key, val1 is

Re: inconsistent results

Change your consistency levels in the cqlsh shell while you query, from ONE to QUORUM to ALL. If you see your results change that's a consistency issue. (Assuming these are simple inserts, if there's deletes, potentially update collections, etc. in the mix then things get a bit more complex.) To

Re: inconsistent results

I suspect this is true, but it has proven to be significantly harder to track down. Either cassandra is tickling some bug that nothing else does or something strange is going on internally. On an otherwise quiet system, I'd see instant results most of the time intermixed with queries (reads)

Re: inconsistent results

I'm sorry, yes. The primary key is (foo_prefix, foo), with foo_prefix being the partition key. The query is: select * from table WHERE foo_prefix='blah'; -JE

Re: inconsistent results

Well, if it's the primary key there should only ever be one result. Is this the partition key and you also have a clustering key? On Tue, Feb 14, 2017 at 1:43 PM Josh England wrote: > Super simple: > select * from table WHERE primary_key='foo'; > > -JE > > > On Tue, Feb 14, 2017

Re: inconsistent results

If you're getting a lot of timeouts you will almost certainly end up with consistency issues. You're going to need to fix the root cause, your cluster instability, or this sort of issue will be commonplace. On Tue, Feb 14, 2017 at 1:43 PM Josh England wrote: > I'll try it the

Re: inconsistent results

I'll try it the repair. Using quorum tends to lead to too many timeout problems though. :( -JE On Tue, Feb 14, 2017 at 1:39 PM, Oskar Kjellin wrote: > Repair might help. But you will end up in this situation again unless you > read/write using quorum (may be local)

Re: inconsistent results

Super simple: select * from table WHERE primary_key='foo'; -JE On Tue, Feb 14, 2017 at 1:38 PM, sfesc...@gmail.com wrote: > What is your query? I've seen this once when using secondary indices as it > has to reach out to all nodes for the answer. If a node doesn't respond

Re: inconsistent results

Repair might help. But you will end up in this situation again unless you read/write using quorum (may be local) Sent from my iPhone > On 14 Feb 2017, at 22:37, Josh England wrote: > > All client interactions are from python (python-driver 3.7.1) using default > consistency

Re: inconsistent results

What is your query? I've seen this once when using secondary indices as it has to reach out to all nodes for the answer. If a node doesn't respond in time those records seem to get dropped. On Tue, Feb 14, 2017 at 1:37 PM Josh England wrote: > All client interactions are from

Re: inconsistent results

All client interactions are from python (python-driver 3.7.1) using default consistency (LOCAL_ONE I think). Should I try repairing all nodes to make sure all data is consistent? -JE On Tue, Feb 14, 2017 at 1:32 PM, Oskar Kjellin wrote: > What consistency levels are

Re: inconsistent results

What consistency levels are you using for reads/writes? Sent from my iPhone > On 14 Feb 2017, at 22:27, Josh England wrote: > > I'm running Cassandra 3.9 on CentOS 6.7 in a 6-node cluster. I've got a > situation where the same query sometimes returns 2 records (correct), and

Re: Inconsistent results with Quorum at different times

Hi Jaydeep. > Now when I read using quorum then sometimes it returns data D1 and > sometimes it returns empty results. After tracing I found that when N1 and > N2 are chosen then we get empty data, when (N1/N2) and N3 are chosen then > D1 data is returned. This is an acceptable situation (ie.

Re: Inconsistent results with Quorum at different times

Hi Jaydeep, Yes, dealing with tombstones in Cassandra is very tricky. Cassandra keeps tombstones to mark deleted columns and distribute (hinted handoff, full repair, read repair ...) to the other nodes that missed the initial remove request. But Cassandra can't afford to keep those tombstones

Inconsistent results with Quorum at different times

Hi, We have three node (N1, N2, N3) cluster (RF=3) and data in SSTable as following: N1: SSTable: Partition key K1 is marked as tombstone at time T2 N2: SSTable: Partition key K1 is marked as tombstone at time T2 N3: SSTable: Partition key K1 is valid and has data D1 with lower time-stamp T1

Re: Cassandra CLI showing inconsistent results during gets

All inserts are at LOCAL_QUORUM DC1 I am confused because attempt-1 shows up the column, attempt-2 not found, attempt-3 again shows it up. These attempts were successive with no time delay from the same CLI!!! The data also is not tinkered with CUD operations from somewhere else during these

Cassandra CLI showing inconsistent results during gets

I ran the following set of commands via CLI in our servers. There is a data-discrepancy that I encountered as below during gets... We are running 1.2.4 version with replication-factor=3 (DC1) 2 (DC2). Reads and writes are at LOCAL_QUORUM create column family TestCF with

Re: Inconsistent results using secondary indexes between two DC

2011/5/23 Jonathan Ellis jbel...@gmail.com: It was installed as 0.7.2 and upgraded with each new official release. I bet that's the problem, then. https://issues.apache.org/jira/browse/CASSANDRA-2244 could cause indexes to not be updated for releases 0.7.4.  You'll want to rebuild the

Re: Inconsistent results using secondary indexes between two DC

It was installed as 0.7.2 and upgraded with each new official release. As I wrote in another message in this thread, now nodes are upgraded to 0.7.6 but it still seems that one of the problematic nodes returns inconsistent data. By the way - is it possible to force the rebuild of the secondary

Re: Inconsistent results using secondary indexes between two DC

On Mon, May 23, 2011 at 5:47 AM, Wojciech Pietrzok kosci...@gmail.com wrote: It was installed as 0.7.2 and upgraded with each new official release. I bet that's the problem, then. https://issues.apache.org/jira/browse/CASSANDRA-2244 could cause indexes to not be updated for releases 0.7.4.

Re: Inconsistent results using secondary indexes between two DC

I've already tried running nodetool repair severail times before but it didn't seem to help. Now I've upgraded Cassandra to 0.7.6, run nodetool scrub, and nodetool repair (twice). One of the problematic nodes seems to return correct results now. But the second one still returns inconsistent data.

Re: Inconsistent results using secondary indexes between two DC

Has this cluster always been on 0.7.5 or was it upgraded from an earlier version? On Thu, May 19, 2011 at 3:26 AM, Wojciech Pietrzok kosci...@gmail.com wrote: Just checked. Seems to be present in CF on all nodes (in both datacenters), but are not indexed correctly On each node I've used

Re: Inconsistent results using secondary indexes between two DC

Just checked. Seems to be present in CF on all nodes (in both datacenters), but are not indexed correctly On each node I've used sstablekeys for all CF_NAME-f-XX-Data.db files. In cassandra-cli I've (using node that behaves correctly) made query get CF_NAME where foo = bar, got correct number of

Re: Inconsistent results using secondary indexes between two DC

I am wondering if running nodetool repair will help in anyway -- View this message in context: http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/Inconsistent-results-using-secondary-indexes-between-two-DC-tp632p6382819.html Sent from the cassandra-u...@incubator.apache.org

Re: Inconsistent results using secondary indexes between two DC

Nothing comes to mind. I'd start by using sstable2json to see if the missing rows are in the main data CF -- i.e., are they just unindexed, or are they missing completely? On Sun, May 15, 2011 at 4:33 PM, Wojciech Pietrzok kosci...@gmail.com wrote: Hello, I've noticed strange behaviour of

Inconsistent results using secondary indexes between two DC

Hello, I've noticed strange behaviour of Cassandra when using secondary indexes. There are 2 Data Centers, each with 2 nodes, RF=4, on all nodes Cassandra 0.7.5 is installed. When I connect to one of the nodes in DC1 and perform query using secondary indexes (get ColumnFamily where column = 'foo'