Oops, I was trimming out some irrelevant stuff, and trimmed out too much. The
second snippet should be this:
ResultSet rs = cassandraUtil.executeNamedQuery(Q_LIST_IMAGES, rollId);
int total = 0;
long lastImageId = -1;
for (Row row : rs)
{
long imageId = row.getLong(PARAM_IMAGE_ID);
if (imageId == lastImageId)
{
logger.warn("Cassandra duplicated " + imageId);
continue;
}
total++;
lastImageId = imageId;
}
On Oct 3, 2015, at 10:54 AM, Robert Wille
<[email protected]<mailto:[email protected]>> wrote:
I don’t think its an application problem. The following simple snippets produce
different totals:
ResultSet rs = cassandraUtil.executeNamedQuery(Q_LIST_IMAGES, rollId);
int total = 0;
for (Row row : rs)
{
total++;
}
---------------------
ResultSet rs = cassandraUtil.executeNamedQuery(Q_LIST_IMAGES, rollId);
int total = 0;
long lastImageId = -1;
for (Row row : rs)
{
long imageId = row.getLong(PARAM_IMAGE_ID);
total++;
lastImageId = imageId;
}
This doesn’t happen for all partitions. In fact most don’t have this problem
(maybe 20% do this). But the ones that do repeat records, do so
deterministically. I see this problem in multiple tables.
I’m only retrieving the clustering key, so it has nothing to do with the data
field.
I suspect this is a paging problem, and might be a driver issue.
Robert
On Oct 3, 2015, at 9:29 AM, Eric Stevens
<[email protected]<mailto:[email protected]>> wrote:
Can you give us an example of the duplicate records that comes back? How
reliable is it (i.e. is it every record, is it one record per read, etc)? By
any chance is it just the `data` field that duplicates while the other fields
change per row?
> I don’t see duplicates in cqlsh.
I've never seen this, and I can't think of a failure mode which would cause it
to happen. Not to say it's impossible, but Cassandra's standard read path
involves collapsing duplicate or otherwise overlapping answers from multiple
replicas; such a thing would be a pretty substantial deviation. Especially
since you don't see the duplicates in cqlsh, I have a hunch this is an
application bug.
On Fri, Oct 2, 2015 at 4:58 PM Robert Wille
<[email protected]<mailto:[email protected]>> wrote:
When I run the query "SELECT image FROM roll WHERE roll = :roll“ against this
table
CREATE TABLE roll (
roll INT,
image BIGINT,
data VARCHAR static,
mid VARCHAR,
imp_st VARCHAR,
PRIMARY KEY ((roll), image)
) WITH gc_grace_seconds = 3456000 AND compaction = { 'class' :
'LeveledCompactionStrategy', 'sstable_size_in_mb' : 160 };
I often get duplicate records back. Seems like a very simple query to botch.
I’m running 2.0.16 with RF=3 and CL=QUORUM and Java client 2.0.10.1. I don’t
see duplicates in cqlsh. Any thoughts?
Thanks
Robert
