[
https://issues.apache.org/jira/browse/CASSANDRA-2918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13067882#comment-13067882
]
Cathy Daw edited comment on CASSANDRA-2918 at 7/19/11 11:24 PM:
----------------------------------------------------------------
Interestingly, I did the same test case using the hector stress tool which
writes @ CL=QUORUM.
*Insert 1000 rows. 691 Rows Returned.*
* Bring up 3 nodes. Create a KS with RF=3.
{code}
[default@StressKeyspace] describe keyspace StressKeyspace;
Keyspace: StressKeyspace:
Replication Strategy: org.apache.cassandra.locator.SimpleStrategy
Durable Writes: true
Options: [replication_factor:3]
Column Families:
ColumnFamily: StressStandard
Key Validation Class: org.apache.cassandra.db.marshal.BytesType
Default column value validator: org.apache.cassandra.db.marshal.BytesType
Columns sorted by: org.apache.cassandra.db.marshal.UTF8Type
Row cache size / save period in seconds: 0.0/0
Key cache size / save period in seconds: 10000.0/14400
Memtable thresholds: 0.2859375/1440/32 (millions of ops/MB/minutes)
GC grace seconds: 864000
Compaction min/max thresholds: 4/32
Read repair chance: 1.0
Replicate on write: true
Built indexes: []
{code}
* Kill one node
*TEST #1: first two nodes don't agree*
* Insert 1000 rows with 10 columns each in batches of 100 into standard table.
{code}
Get code from: [https://github.com/zznate/cassandra-stress]
sh bin/stress -o insert -t 2 -b 100 -n 1000 -c 10 node1:9160
{code}
* Verify Data:
** node1: 691 rows
** node2: 1000 rows
** Nate verified the default is to write at QUORUM
* Bring up dead node. Run Repair.
* Kill two other nodes.
* Run in cli: consistencylevel as ONE;
* Run in cli: list StressStandard limit 2000;
** node3: 691 rows
*TEST #2: 1st two nodes agree*
* Insert 1000 rows with 10 columns each in batches of 100 into standard table.
Explicitly specify QUORUM.
{code}
sh bin/stress -o insert -t 2 -b 100 -n 1000 -c 10 -L QUORUM:QUORUM node1:9160
{code}
* Verify Data:
** node1: 1000 rows
** node2: 1000 rows
* Bring up dead node. Run Repair.
* Kill two other nodes.
* Run in cli: consistencylevel as ONE.
* Run in cli: list StressStandard limit 2000;
** node3: 332 rows
*TEST #3*
* flush node1 and node2
* Bring up dead node - node3. Run Repair.
* Kill node1 and node2
* Run in cli: consistencylevel as ONE.
* Run in cli: list StressStandard limit 2000;
** node3: 1000 rows
was (Author: cdaw):
Interestingly, I did the same test case using the hector stress tool which
writes @ CL=QUORUM.
*Insert 1000 rows. 691 Rows Returned.*
* Bring up 3 nodes. Create a KS with RF=3.
{code}
[default@StressKeyspace] describe keyspace StressKeyspace;
Keyspace: StressKeyspace:
Replication Strategy: org.apache.cassandra.locator.SimpleStrategy
Durable Writes: true
Options: [replication_factor:3]
Column Families:
ColumnFamily: StressStandard
Key Validation Class: org.apache.cassandra.db.marshal.BytesType
Default column value validator: org.apache.cassandra.db.marshal.BytesType
Columns sorted by: org.apache.cassandra.db.marshal.UTF8Type
Row cache size / save period in seconds: 0.0/0
Key cache size / save period in seconds: 10000.0/14400
Memtable thresholds: 0.2859375/1440/32 (millions of ops/MB/minutes)
GC grace seconds: 864000
Compaction min/max thresholds: 4/32
Read repair chance: 1.0
Replicate on write: true
Built indexes: []
{code}
* Kill one node
*TEST #1: first two nodes don't agree*
* Insert 1000 rows with 10 columns each in batches of 100 into standard table.
{code}
sh bin/stress -o insert -t 2 -b 100 -n 1000 -c 10 node1:9160
{code}
* Verify Data:
** node1: 691 rows
** node2: 1000 rows
** Nate verified the default is to write at QUORUM
* Bring up dead node. Run Repair.
* Kill two other nodes.
* Run in cli: consistencylevel as ONE;
* Run in cli: list StressStandard limit 2000;
** node3: 691 rows
*TEST #2: 1st two nodes agree*
* Insert 1000 rows with 10 columns each in batches of 100 into standard table.
Explicitly specify QUORUM.
{code}
sh bin/stress -o insert -t 2 -b 100 -n 1000 -c 10 -L QUORUM:QUORUM node1:9160
{code}
* Verify Data:
** node1: 1000 rows
** node2: 1000 rows
* Bring up dead node. Run Repair.
* Kill two other nodes.
* Run in cli: consistencylevel as ONE.
* Run in cli: list StressStandard limit 2000;
** node3: 332 rows
> After repair, one row missing from query when two rows expected
> ---------------------------------------------------------------
>
> Key: CASSANDRA-2918
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2918
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 0.8.2
> Environment: Cassandra-0.8 branch @ 07/18 around 1pm PST.
> Reporter: Cathy Daw
> Assignee: Sylvain Lebresne
>
> *Cluster Config*
> {code}
> cathy1 - 50.57.114.45 - Token: 0
> cathy2 - 50.57.107.176 - Token: 56713727820156410577229101238628035242
> cathy3 - 50.57.114.39 - Token: 113427455640312821154458202477256070484
> {code}
> *+2) Kill cathy3: 50.57.114.39+*
> {code}
> root@cathy2:~/cass-0.8/bin# ./nodetool -h localhost ring
> Address DC Rack Status State Load Owns
> Token
>
> 113427455640312821154458202477256070484
> 50.57.114.45 datacenter1 rack1 Up Normal 59.84 KB 33.33%
> 0
> 50.57.107.176 datacenter1 rack1 Up Normal 59.85 KB 33.33%
> 56713727820156410577229101238628035242
> 50.57.114.39 datacenter1 rack1 Down Normal 59.85 KB 33.33%
> 113427455640312821154458202477256070484
> {code}
> *+3) Run java stress tool+*
> {code}
> ./bin/stress -o insert -n 1000 -c 10 -l 3 -e QUORUM -d
> 50.57.114.45,50.57.107.176
> {code}
> *+4) Start Cassandra on cathy3+*
> *+5) Run repair on cathy3+*
> {code}
> nodetool -h cathy3 repair Keyspace1 Standard1
> {code}
> *+6) Kill cathy1 and cathy2+*
> {code}
> root@cathy3:~/cass-0.8/bin# ./nodetool -h cathy3 ring
> Address DC Rack Status State Load Owns
> Token
>
> 113427455640312821154458202477256070484
> 50.57.114.45 datacenter1 rack1 Down Normal 105.46 KB 33.33%
> 0
> 50.57.107.176 datacenter1 rack1 Down Normal 106 KB 33.33%
> 56713727820156410577229101238628035242
> 50.57.114.39 datacenter1 rack1 Up Normal 331.33 KB 33.33%
> 113427455640312821154458202477256070484
> {code}
> *+7) Log into cassandra-cli on cathy3 - expect 1000 rows returned+*
> {code}
> [default@Keyspace1] consistencylevel as ONE;
> Consistency level is set to 'ONE'.
> [default@Keyspace1] list Standard1 limit 2000;
> .....
> 323 Rows Returned.
> {code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
