[jira] [Commented] (CASSANDRA-12659) Query in reversed order brough back deleted data
[
https://issues.apache.org/jira/browse/CASSANDRA-12659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15514820#comment-15514820
]
Aleksey Yeschenko commented on CASSANDRA-12659:
---
I'm going ahead and closing the ticket as Duplicate of CASSANDRA-11733, as it
most likely is. Please feel free to reopen this JIRA ticket if you can
reproduce in 3.0.9. Thanks.
> Query in reversed order brough back deleted data
>
>
> Key: CASSANDRA-12659
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12659
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: Cassandra 3.0.5, 6 nodes cluster
>Reporter: Tai Khuu Tan
>
> We have and issues with our Cassandra 3.0.5. After we deleted a large amount
> of data in the multiple partition keys. Query those partition keys with
> reversed order on a clustering key return the deleted data. I have checked
> and there are no tombstones left. All of them are deleted. So I don't know
> where or how can those deleted data still exist. Is there any other place
> that Cassandra will read data when query in reverse order compare to normal
> order ?
> the schema is very simple
> {noformat}
> CREATE TABLE table ( uid varchar, version timestamp, data1 varchar, data2
> varchar, data3 varchar, data4 varchar, data5 varchar, PRIMARY KEY (uid,
> version, data1 , data2 , data3 , data4 ) ) with compact storage;
> {noformat}
> Query are doing reverse order on column timestamp
> Ex:
> {noformat}
> select * from data where uid="uid1" order by version DESC
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-12659) Query in reversed order brough back deleted data
[
https://issues.apache.org/jira/browse/CASSANDRA-12659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15514545#comment-15514545
]
Wei Deng commented on CASSANDRA-12659:
--
[~khuutan...@gmail.com] You can move SSTable from one cluster to another,
assuming the 2nd cluster can replicate the token ownership of the previous
cluster, and you also replicate the schema on the 2nd cluster. You probably
don't want to use sstableloader in this case as it might change the sstables
when they're landed on the destination.
> Query in reversed order brough back deleted data
>
>
> Key: CASSANDRA-12659
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12659
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: Cassandra 3.0.5, 6 nodes cluster
>Reporter: Tai Khuu Tan
>
> We have and issues with our Cassandra 3.0.5. After we deleted a large amount
> of data in the multiple partition keys. Query those partition keys with
> reversed order on a clustering key return the deleted data. I have checked
> and there are no tombstones left. All of them are deleted. So I don't know
> where or how can those deleted data still exist. Is there any other place
> that Cassandra will read data when query in reverse order compare to normal
> order ?
> the schema is very simple
> {noformat}
> CREATE TABLE table ( uid varchar, version timestamp, data1 varchar, data2
> varchar, data3 varchar, data4 varchar, data5 varchar, PRIMARY KEY (uid,
> version, data1 , data2 , data3 , data4 ) ) with compact storage;
> {noformat}
> Query are doing reverse order on column timestamp
> Ex:
> {noformat}
> select * from data where uid="uid1" order by version DESC
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-12659) Query in reversed order brough back deleted data
[
https://issues.apache.org/jira/browse/CASSANDRA-12659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15510190#comment-15510190
]
Tai Khuu Tan commented on CASSANDRA-12659:
--
[~iamaleksey] I will try to reproduce the issue. Is there anyway we can move
the sstable from 1 cluster to another, since the affected cluster is production
server it not easy to try different things on them. I know there is the
sstableloader but I don't know if using that will be able to replicate the
sstable?
> Query in reversed order brough back deleted data
>
>
> Key: CASSANDRA-12659
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12659
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: Cassandra 3.0.5, 6 nodes cluster
>Reporter: Tai Khuu Tan
>
> We have and issues with our Cassandra 3.0.5. After we deleted a large amount
> of data in the multiple partition keys. Query those partition keys with
> reversed order on a clustering key return the deleted data. I have checked
> and there are no tombstones left. All of them are deleted. So I don't know
> where or how can those deleted data still exist. Is there any other place
> that Cassandra will read data when query in reverse order compare to normal
> order ?
> the schema is very simple
> {noformat}
> CREATE TABLE table ( uid varchar, version timestamp, data1 varchar, data2
> varchar, data3 varchar, data4 varchar, data5 varchar, PRIMARY KEY (uid,
> version, data1 , data2 , data3 , data4 ) ) with compact storage;
> {noformat}
> Query are doing reverse order on column timestamp
> Ex:
> {noformat}
> select * from data where uid="uid1" order by version DESC
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-12659) Query in reversed order brough back deleted data
[
https://issues.apache.org/jira/browse/CASSANDRA-12659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15505312#comment-15505312
]
Aleksey Yeschenko commented on CASSANDRA-12659:
---
Could be a duplicate of CASSANDRA-11733.
> Query in reversed order brough back deleted data
>
>
> Key: CASSANDRA-12659
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12659
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: Cassandra 3.0.5, 6 nodes cluster
>Reporter: Tai Khuu Tan
>
> We have and issues with our Cassandra 3.0.5. After we deleted a large amount
> of data in the multiple partition keys. Query those partition keys with
> reversed order on a clustering key return the deleted data. I have checked
> and there are no tombstones left. All of them are deleted. So I don't know
> where or how can those deleted data still exist. Is there any other place
> that Cassandra will read data when query in reverse order compare to normal
> order ?
> the schema is very simple
> {noformat}
> CREATE TABLE table ( uid varchar, version timestamp, data1 varchar, data2
> varchar, data3 varchar, data4 varchar, data5 varchar, PRIMARY KEY (uid,
> version, data1 , data2 , data3 , data4 ) ) with compact storage;
> {noformat}
> Query are doing reverse order on column timestamp
> Ex:
> {noformat}
> select * from data where uid="uid1" order by version DESC
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-12659) Query in reversed order brough back deleted data
[
https://issues.apache.org/jira/browse/CASSANDRA-12659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15504043#comment-15504043
]
Tai Khuu Tan commented on CASSANDRA-12659:
--
I'm really sorry but I can't reproduce the issue. It's seem like the issue on
happen on server with a lot of data. I can't reproduce it on other cluster. I
can't put the data up here too since it contains sensitive information. Look
like we will have to deal with the zombie data by manual work arround.
> Query in reversed order brough back deleted data
>
>
> Key: CASSANDRA-12659
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12659
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: Cassandra 3.0.5, 6 nodes cluster
>Reporter: Tai Khuu Tan
>
> We have and issues with our Cassandra 3.0.5. After we deleted a large amount
> of data in the multiple partition keys. Query those partition keys with
> reversed order on a clustering key return the deleted data. I have checked
> and there are no tombstones left. All of them are deleted. So I don't know
> where or how can those deleted data still exist. Is there any other place
> that Cassandra will read data when query in reverse order compare to normal
> order ?
> the schema is very simple
> {noformat}
> CREATE TABLE table ( uid varchar, version timestamp, data1 varchar, data2
> varchar, data3 varchar, data4 varchar, data5 varchar, PRIMARY KEY (uid,
> version, data1 , data2 , data3 , data4 ) ) with compact storage;
> {noformat}
> Query are doing reverse order on column timestamp
> Ex:
> {noformat}
> select * from data where uid="uid1" order by version DESC
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-12659) Query in reversed order brough back deleted data
[
https://issues.apache.org/jira/browse/CASSANDRA-12659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15502203#comment-15502203
]
Wei Deng commented on CASSANDRA-12659:
--
Did you preserve the SSTables (snapshot) when you were able to reproduce the
problem? If yes, it will be useful to use "nodetool getendpoints" and "nodetool
getsstables" to extract a number of SSTables that contains the partition in
question and upload them here along with the schema (assuming it doesn't
contain sensitive information). If you're no longer able to reproduce the
problem, then there is no need to provide the SSTables.
Without a repro case, it will be hard for people to look into it. However,
filing this JIRA right now is still valuable because if other people run into
the same problem in a different occasion, they will have something to compare
notes to determine if this is a real problem. If after a while nobody else runs
into this issue and you are still not able to reproduce it, the JIRA could end
up getting closed eventually.
> Query in reversed order brough back deleted data
>
>
> Key: CASSANDRA-12659
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12659
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: Cassandra 3.0.5, 6 nodes cluster
>Reporter: Tai Khuu Tan
>
> We have and issues with our Cassandra 3.0.5. After we deleted a large amount
> of data in the multiple partition keys. Query those partition keys with
> reversed order on a clustering key return the deleted data. I have checked
> and there are no tombstones left. All of them are deleted. So I don't know
> where or how can those deleted data still exist. Is there any other place
> that Cassandra will read data when query in reverse order compare to normal
> order ?
> the schema is very simple
> {noformat}
> CREATE TABLE table ( uid varchar, version timestamp, data1 varchar, data2
> varchar, data3 varchar, data4 varchar, data5 varchar, PRIMARY KEY (uid,
> version, data1 , data2 , data3 , data4 ) ) with compact storage;
> {noformat}
> Query are doing reverse order on column timestamp
> Ex:
> {noformat}
> select * from data where uid="uid1" order by version DESC
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-12659) Query in reversed order brough back deleted data
[
https://issues.apache.org/jira/browse/CASSANDRA-12659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15501977#comment-15501977
]
Tai Khuu Tan commented on CASSANDRA-12659:
--
The weird thing is only reversed order query return deleted data, even with
Consistency level set to ALL, normal query won't return deleted data, and there
are no tombstone also, so I really don't know where the data come from. I tried
to reproduce it but i couldn't. I will keep trying to see if I can do it.
> Query in reversed order brough back deleted data
>
>
> Key: CASSANDRA-12659
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12659
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: Cassandra 3.0.5, 6 nodes cluster
>Reporter: Tai Khuu Tan
>
> We have and issues with our Cassandra 3.0.5. After we deleted a large amount
> of data in the multiple partition keys. Query those partition keys with
> reversed order on a clustering key return the deleted data. I have checked
> and there are no tombstones left. All of them are deleted. So I don't know
> where or how can those deleted data still exist. Is there any other place
> that Cassandra will read data when query in reverse order compare to normal
> order ?
> the schema is very simple
> {noformat}
> CREATE TABLE table ( uid varchar, version timestamp, data1 varchar, data2
> varchar, data3 varchar, data4 varchar, data5 varchar, PRIMARY KEY (uid,
> version, data1 , data2 , data3 , data4 ) ) with compact storage;
> {noformat}
> Query are doing reverse order on column timestamp
> Ex:
> {noformat}
> select * from data where uid="uid1" order by version DESC
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-12659) Query in reversed order brough back deleted data
[
https://issues.apache.org/jira/browse/CASSANDRA-12659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15499215#comment-15499215
]
Wei Deng commented on CASSANDRA-12659:
--
Can you reliably reproduce this problem? If yes, it would be very helpful if
you describe the repro steps so that somebody can take a closer look.
BTW, "there are no tombstones left" doesn't mean you're safe. You may have run
into a typical ["zombie
resurrection"|http://thelastpickle.com/blog/2016/07/27/about-deletes-and-tombstones.html]
situation when the system cleared up all of the tombstones.
> Query in reversed order brough back deleted data
>
>
> Key: CASSANDRA-12659
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12659
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: Cassandra 3.0.5, 6 nodes cluster
>Reporter: Tai Khuu Tan
>
> We have and issues with our Cassandra 3.0.5. After we deleted a large amount
> of data in the multiple partition keys. Query those partition keys with
> reversed order on a clustering key return the deleted data. I have checked
> and there are no tombstones left. All of them are deleted. So I don't know
> where or how can those deleted data still exist. Is there any other place
> that Cassandra will read data when query in reverse order compare to normal
> order ?
> the schema is very simple
> {noformat}
> CREATE TABLE table ( uid varchar, version timestamp, data1 varchar, data2
> varchar, data3 varchar, data4 varchar, data5 varchar, PRIMARY KEY (uid,
> version, data1 , data2 , data3 , data4 ) ) with compact storage;
> {noformat}
> Query are doing reverse order on column timestamp
> Ex:
> {noformat}
> select * from data where uid="uid1" order by version DESC
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
