[
https://issues.apache.org/jira/browse/CASSANDRA-11314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15188869#comment-15188869
]
Mircea Lemnaru commented on CASSANDRA-11314:
--------------------------------------------
Hi [~blerer], I did some more tests over the last couple of days and found that
there is not really a cassandra issue , it's rather something like a usage
issue. Let me describe what I found ...
So as I have mentioned before the cluster had 3 nodes node1 (seed node)
,node2,node3 and my keyspace had replication factor = 2. Then I did more tests
using the following scenario:
1. node1 UP , node2 DOWN, node3 DOWN , I tried to do the following query:
select count(*) from collected_data_day where collection_day=16462 --> this
query did not execute properly because node1 did not contain a replica of the
row = 16462 this suggested to me that the data must be ONLY on nodes node2 and
node3
2. node1 UP, node2 UP, node3 DOWN - executing the same query ( select count(*)
from collected_data_day where collection_day=16462 ) 20 times consecutively
resulted in exactly the same result ... COUNT = 55300 ... strange right ?
3. node1 UP, node2 DOWN, node3 UP - same as 2 and I got COUNT = 55677 - every
time , 20 times.
So this rang some bells , it seems that row = 16642 has 2 replicas = but the
replicas are not identical , it seems that when the data was written the
CONSISTENCY level was not properly set , in the sense that it was left default
, and that is ONE if I am not mistaking.
I then proceeded to drop the table and reimport the data. This time I imported
the data with CONSISTENCY = ALL and after this , all the counts were identical
, no matter how many times I ran the query.
So yeah ,this must have been the issue. I am guessing this can be closed as not
a real issue / bug :)
PS: One thing that confused me all along was the COUNT values I got while the
replicas were not identical ... if you remember , I got something like:
Count (done on cassandra cluster): 55300
Count (done on cassandra cluster): 55677
Count (done on cassandra cluster): 55677
Count (done on cassandra cluster): 55300
Count (done on cassandra cluster): 55677
Count (done on cassandra cluster): 55677
Count (done on cassandra cluster): 55634
Count (done on cassandra cluster): 55530
Count (done on cassandra cluster): 55677
Count (done on cassandra cluster): 55342
Count (done on cassandra cluster): 55300
Count (done on cassandra cluster): 55630
Count (done on cassandra cluster): 55677
Count (done on cassandra cluster): 55297
Count (done on cassandra cluster): 55677
Count (done on cassandra cluster): 55677
Count (done on cassandra cluster): 55352
Count (done on cassandra cluster): 55300
Count (done on cassandra cluster): 55677
Count (done on cassandra cluster): 55300
>From these results ... it's clear that 55300 came from counting the items from
>node1 and that 55677 came from node3 .... but what about 55634, 55342, 55352 ?
>Where were these results comming from ... ?
Anyway, I am glad this is not a bug but a configuration issue. Thank you.
> Inconsistent select count(*)
> ----------------------------
>
> Key: CASSANDRA-11314
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11314
> Project: Cassandra
> Issue Type: Bug
> Components: Local Write-Read Paths
> Environment: Ununtu 14.04 LTS
> Reporter: Mircea Lemnaru
> Assignee: Benjamin Lerer
> Attachments: testrun.log, vnodes_and_hosts
>
>
> Hello,
> I currently have this setup:
> Cassandra 3.3 (Community edition downloaded from Datastax) installed on 3
> nodes and I have created this table:
> CREATE TABLE billing.collected_data_day (
> collection_day int,
> timestamp timestamp,
> record_id uuid,
> dimensions map<text, text>,
> entity_id text,
> measurements map<text, text>,
> source_id text,
> PRIMARY KEY (collection_day, timestamp, record_id)
> ) WITH CLUSTERING ORDER BY (timestamp ASC, record_id ASC)
> AND bloom_filter_fp_chance = 0.01
> AND caching = {'keys': 'ALL', 'rows_per_partition': 'NONE'}
> AND comment = ''
> AND compaction = {'class':
> 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy',
> 'max_threshold': '32', 'min_threshold': '4'}
> AND compression = {'chunk_length_in_kb': '64', 'class':
> 'org.apache.cassandra.io.compress.LZ4Compressor'}
> AND crc_check_chance = 1.0
> AND dclocal_read_repair_chance = 0.1
> AND default_time_to_live = 0
> AND gc_grace_seconds = 864000
> AND max_index_interval = 2048
> AND memtable_flush_period_in_ms = 0
> AND min_index_interval = 128
> AND read_repair_chance = 0.0
> AND speculative_retry = '99PERCENTILE';
> This table as you notice is partitioned by collection_day. This is because at
> the end of the day we need to have fast access to all the data generated in a
> day. collection day will be the x day from 1970
> In this table we have inserted roughly 12milion rows for testing purposes and
> we did a simple count. As you can see the results vary ...
> cqlsh:billing> select count(*) from collected_data_day where
> collection_day=16462;
> count
> -------
> 55341
> (1 rows)
> cqlsh:billing> select count(*) from collected_data_day where
> collection_day=16462;
> count
> -------
> 55372
> (1 rows)
> cqlsh:billing> select count(*) from collected_data_day where
> collection_day=16462;
> count
> -------
> 55300
> (1 rows)
> cqlsh:billing> select count(*) from collected_data_day where
> collection_day=16462;
> count
> -------
> 55300
> (1 rows)
> cqlsh:billing> select count(*) from collected_data_day where
> collection_day=16462;
> count
> -------
> 55300
> (1 rows)
> cqlsh:billing> select count(*) from collected_data_day where
> collection_day=16462;
> count
> -------
> 55303
> (1 rows)
> cqlsh:billing> select count(*) from collected_data_day where
> collection_day=16462;
> count
> -------
> 55374
> (1 rows)
> I am running the query from the seed node of the cassandra cluster. As you
> can see most of the results are varying and I don't know the reason for this.
> We are not writing anything into the cluster at this time , we are only
> querying the cluster and only using this CQLSH.
> This is very similar to CASSANDRA-8940 but that is targeted for 2.1x
> Could it be that we are having the same issue in 3.3 ?
> Please let me know what extra info I can provide
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
