[jira] [Commented] (CASSANDRA-2950) Data from truncated CF reappears after server restart

2011-08-11 Thread Hudson (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-2950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13083482#comment-13083482
 ] 

Hudson commented on CASSANDRA-2950:
---

Integrated in Cassandra-0.8 #272 (See 
[https://builds.apache.org/job/Cassandra-0.8/272/])
make sure truncate clears out the commitlog
patch by jbellis; reviewed by slebresne for CASSANDRA-2950

jbellis : 
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1156763
Files : 
* /cassandra/branches/cassandra-0.8/CHANGES.txt
* 
/cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/db/SystemTable.java
* 
/cassandra/branches/cassandra-0.8/test/unit/org/apache/cassandra/db/RecoveryManagerTruncateTest.java
* 
/cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/db/commitlog/CommitLog.java
* 
/cassandra/branches/cassandra-0.8/src/java/org/apache/cassandra/db/ColumnFamilyStore.java


 Data from truncated CF reappears after server restart
 -

 Key: CASSANDRA-2950
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2950
 Project: Cassandra
  Issue Type: Bug
Affects Versions: 0.8.0
Reporter: Cathy Daw
Assignee: Jonathan Ellis
 Fix For: 0.8.5

 Attachments: 2950-v2.txt, 2950-v3_0.8.patch, 2950.txt


 * Configure 3 node cluster
 * Ensure the java stress tool creates Keyspace1 with RF=3
 {code}
 // Run Stress Tool to generate 10 keys, 1 column
 stress --operation=INSERT -t 2 --num-keys=50 --columns=20 
 --consistency-level=QUORUM --average-size-values --replication-factor=3 
 --create-index=KEYS --nodes=cathy1,cathy2
 // Verify 50 keys in CLI
 use Keyspace1; 
 list Standard1; 
 // TRUNCATE CF in CLI
 use Keyspace1;
 truncate counter1;
 list counter1;
 // Run stress tool and verify creation of 1 key with 10 columns
 stress --operation=INSERT -t 2 --num-keys=1 --columns=10 
 --consistency-level=QUORUM --average-size-values --replication-factor=3 
 --create-index=KEYS --nodes=cathy1,cathy2
 // Verify 1 key in CLI
 use Keyspace1; 
 list Standard1; 
 // Restart all three nodes
 // You will see 51 keys in CLI
 use Keyspace1; 
 list Standard1; 
 {code}

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-2950) Data from truncated CF reappears after server restart

2011-08-09 Thread Brandon Williams (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-2950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13081941#comment-13081941
 ] 

Brandon Williams commented on CASSANDRA-2950:
-

Currently, truncate does:
* force a flush
* record the time
* delete any sstables older than the time

This isn't quite enough if the machine crashes shortly afterward, however, 
since there can be mutations present in the commitlog that were previously 
truncated and are now resurrected by CL replay.

One thing we could do is record the truncate time for the CF in the system ks 
and then ignore mutations older than that, however this would require time 
synchronization between the client and the server to be accurate.


 Data from truncated CF reappears after server restart
 -

 Key: CASSANDRA-2950
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2950
 Project: Cassandra
  Issue Type: Bug
Reporter: Cathy Daw
Assignee: Brandon Williams

 * Configure 3 node cluster
 * Ensure the java stress tool creates Keyspace1 with RF=3
 {code}
 // Run Stress Tool to generate 10 keys, 1 column
 stress --operation=INSERT -t 2 --num-keys=50 --columns=20 
 --consistency-level=QUORUM --average-size-values --replication-factor=3 
 --create-index=KEYS --nodes=cathy1,cathy2
 // Verify 50 keys in CLI
 use Keyspace1; 
 list Standard1; 
 // TRUNCATE CF in CLI
 use Keyspace1;
 truncate counter1;
 list counter1;
 // Run stress tool and verify creation of 1 key with 10 columns
 stress --operation=INSERT -t 2 --num-keys=1 --columns=10 
 --consistency-level=QUORUM --average-size-values --replication-factor=3 
 --create-index=KEYS --nodes=cathy1,cathy2
 // Verify 1 key in CLI
 use Keyspace1; 
 list Standard1; 
 // Restart all three nodes
 // You will see 51 keys in CLI
 use Keyspace1; 
 list Standard1; 
 {code}

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-2950) Data from truncated CF reappears after server restart

2011-08-09 Thread Jonathan Ellis (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-2950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13081966#comment-13081966
 ] 

Jonathan Ellis commented on CASSANDRA-2950:
---

but we record CL context at time of flush in the sstable it makes, and we on 
replay we ignore any mutations from before that position.

checked and we do wait for flush to complete in truncate.

 Data from truncated CF reappears after server restart
 -

 Key: CASSANDRA-2950
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2950
 Project: Cassandra
  Issue Type: Bug
Reporter: Cathy Daw
Assignee: Brandon Williams

 * Configure 3 node cluster
 * Ensure the java stress tool creates Keyspace1 with RF=3
 {code}
 // Run Stress Tool to generate 10 keys, 1 column
 stress --operation=INSERT -t 2 --num-keys=50 --columns=20 
 --consistency-level=QUORUM --average-size-values --replication-factor=3 
 --create-index=KEYS --nodes=cathy1,cathy2
 // Verify 50 keys in CLI
 use Keyspace1; 
 list Standard1; 
 // TRUNCATE CF in CLI
 use Keyspace1;
 truncate counter1;
 list counter1;
 // Run stress tool and verify creation of 1 key with 10 columns
 stress --operation=INSERT -t 2 --num-keys=1 --columns=10 
 --consistency-level=QUORUM --average-size-values --replication-factor=3 
 --create-index=KEYS --nodes=cathy1,cathy2
 // Verify 1 key in CLI
 use Keyspace1; 
 list Standard1; 
 // Restart all three nodes
 // You will see 51 keys in CLI
 use Keyspace1; 
 list Standard1; 
 {code}

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-2950) Data from truncated CF reappears after server restart

2011-08-09 Thread Brandon Williams (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-2950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13081969#comment-13081969
 ] 

Brandon Williams commented on CASSANDRA-2950:
-

bq. but we record CL context at time of flush in the sstable it makes, and we 
on replay we ignore any mutations from before that position.

I think there's something wrong with that, then:

{noformat}
 INFO 21:25:15,274 Replaying 
/var/lib/cassandra/commitlog/CommitLog-1312924388053.log
DEBUG 21:25:15,290 Replaying 
/var/lib/cassandra/commitlog/CommitLog-1312924388053.log starting at 0
DEBUG 21:25:15,291 Reading mutation at 0
DEBUG 21:25:15,295 replaying mutation for system.4c: {ColumnFamily(LocationInfo 
[47656e65726174696f6e:false:4@131292438814,])}
DEBUG 21:25:15,321 Reading mutation at 89
DEBUG 21:25:15,322 replaying mutation for system.426f6f747374726170: 
{ColumnFamily(LocationInfo [42:false:1@1312924388203,])}
DEBUG 21:25:15,322 Reading mutation at 174
DEBUG 21:25:15,322 replaying mutation for system.4c: {ColumnFamily(LocationInfo 
[546f6b656e:false:16@1312924388204,])}
DEBUG 21:25:15,322 Reading mutation at 270
DEBUG 21:25:15,324 replaying mutation for Keyspace1.3030: 
{ColumnFamily(Standard1 
[C0:false:34@1312924813259,C1:false:34@1312924813260,C2:false:34@1312924813260,C3:false:34@1312924813260,C4:false:34@1312924813260,])}
{noformat}

The last entry there is the first of many errant mutations.

 Data from truncated CF reappears after server restart
 -

 Key: CASSANDRA-2950
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2950
 Project: Cassandra
  Issue Type: Bug
Reporter: Cathy Daw
Assignee: Brandon Williams

 * Configure 3 node cluster
 * Ensure the java stress tool creates Keyspace1 with RF=3
 {code}
 // Run Stress Tool to generate 10 keys, 1 column
 stress --operation=INSERT -t 2 --num-keys=50 --columns=20 
 --consistency-level=QUORUM --average-size-values --replication-factor=3 
 --create-index=KEYS --nodes=cathy1,cathy2
 // Verify 50 keys in CLI
 use Keyspace1; 
 list Standard1; 
 // TRUNCATE CF in CLI
 use Keyspace1;
 truncate counter1;
 list counter1;
 // Run stress tool and verify creation of 1 key with 10 columns
 stress --operation=INSERT -t 2 --num-keys=1 --columns=10 
 --consistency-level=QUORUM --average-size-values --replication-factor=3 
 --create-index=KEYS --nodes=cathy1,cathy2
 // Verify 1 key in CLI
 use Keyspace1; 
 list Standard1; 
 // Restart all three nodes
 // You will see 51 keys in CLI
 use Keyspace1; 
 list Standard1; 
 {code}

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-2950) Data from truncated CF reappears after server restart

2011-07-26 Thread Cathy Daw (JIRA) 

[ 
https://issues.apache.org/jira/browse/CASSANDRA-2950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13071479#comment-13071479
 ] 

Cathy Daw commented on CASSANDRA-2950:
--

The other permutation of this bug looked like, assuming write with CL.Q:
* Insert 50 (3 nodes up)
* truncate CF (3 nodes up)
* Insert 1 (3 nodes up)
* Bring node3 down
* Delete 1  (2 nodes up)
* Bring up node3 and run repair
* Take down node1 and node2.
* Query node3 with CL.ONE: list Standard1;  --- 30 rows returned

Not sure, but this looked suspicious in my logs:
{code}
 INFO 01:19:45,616 Streaming to /50.57.114.45
 INFO 01:19:45,689 Finished streaming session 69860958341 from 
/50.57.107.176
 INFO 01:19:45,690 Finished streaming session 698609609994154 from /50.57.114.45
 INFO 01:19:46,501 Finished streaming repair with /50.57.114.45 for 
(0,56713727820156410577229101238628035242]: 0 oustanding to complete session
 INFO 01:19:46,531 Compacted to 
/var/lib/cassandra/data/Keyspace1/Standard1-tmp-g-106-Data.db.  16,646,523 to 
16,646,352 (~99% of original) bytes for 30 keys.  Time: 1,509ms.
 INFO 01:19:46,930 Finished streaming repair with /50.57.107.176 for 
(113427455640312821154458202477256070484,0]: 1 oustanding to complete session
 INFO 01:19:47,619 Finished streaming repair with /50.57.114.45 for 
(113427455640312821154458202477256070484,0]: 0 oustanding to complete session
 INFO 01:19:48,232 Finished streaming repair with /50.57.107.176 for 
(56713727820156410577229101238628035242,113427455640312821154458202477256070484]:
 1 oustanding to complete session
 INFO 01:19:48,856 Finished streaming repair with /50.57.114.45 for 
(56713727820156410577229101238628035242,113427455640312821154458202477256070484]:
 0 oustanding to complete session
{code}

 Data from truncated CF reappears after server restart
 -

 Key: CASSANDRA-2950
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2950
 Project: Cassandra
  Issue Type: Bug
Reporter: Cathy Daw
Assignee: Sylvain Lebresne

 * Configure 3 node cluster
 * Ensure the java stress tool creates Keyspace1 with RF=3
 {code}
 // Run Stress Tool to generate 10 keys, 1 column
 stress --operation=INSERT -t 2 --num-keys=50 --columns=20 
 --consistency-level=QUORUM --average-size-values --replication-factor=3 
 --create-index=KEYS --nodes=cathy1,cathy2
 // Verify 50 keys in CLI
 use Keyspace1; 
 list Standard1; 
 // TRUNCATE CF in CLI
 use Keyspace1;
 truncate counter1;
 list counter1;
 // Run stress tool and verify creation of 1 key with 10 columns
 stress --operation=INSERT -t 2 --num-keys=1 --columns=10 
 --consistency-level=QUORUM --average-size-values --replication-factor=3 
 --create-index=KEYS --nodes=cathy1,cathy2
 // Verify 1 key in CLI
 use Keyspace1; 
 list Standard1; 
 // Restart all three nodes
 // You will see 51 keys in CLI
 use Keyspace1; 
 list Standard1; 
 {code}

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira