[
https://issues.apache.org/jira/browse/CASSANDRA-1992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12982379#action_12982379
]
Mateusz Korniak commented on CASSANDRA-1992:
--------------------------------------------
Brandon, yes and no ;).
Unable with original contrib/py_stress as it uses only one CF to do all tests.
Most of my issues looks like missing row in 2nd CF or broken data in 2nd CF
(like contents of 1st CF injected into 2nd CF).
I slightly modified contrib/py_stress so it created 3 standard CFs and 3 super
CFs [1] and allows to select one wants to operate via --column_family_idx=
switch and I can reproduce:
Starting 1st node.
$ python stress.py --nodes 192.168.3.8 --operation insert --num-keys 100
--progress-interval 5 --keep-going --column_family_idx=1
Created keyspaces. Sleeping 1s for propagation.
total,interval_op_rate,interval_key_rate,avg_latency,elapsed_time
100,20,20,0.00823852300644,0
$ python stress.py --nodes 192.168.3.8 --operation insert --num-keys 100
--progress-interval 5 --keep-going --column_family_idx=2
Keyspace already exists.
total,interval_op_rate,interval_key_rate,avg_latency,elapsed_time
100,20,20,0.00132475852966,0
$ python stress.py --nodes 192.168.3.8 --operation insert --num-keys 100
--progress-interval 5 --keep-going --column_family_idx=3
Keyspace already exists.
total,interval_op_rate,interval_key_rate,avg_latency,elapsed_time
100,20,20,0.00138550519943,0
Verification of data in each CF:
$ python stress.py --nodes 192.168.3.8 --operation read --num-keys 100
--progress-interval 5 --keep-going --column_family_idx=3
total,interval_op_rate,interval_key_rate,avg_latency,elapsed_time
100,20,20,0.00282711744308,0
$ python stress.py --nodes 192.168.3.8 --operation read --num-keys 100
--progress-interval 5 --keep-going --column_family_idx=2
total,interval_op_rate,interval_key_rate,avg_latency,elapsed_time
100,20,20,0.00149053096771,0
$ python stress.py --nodes 192.168.3.8 --operation read --num-keys 100
--progress-interval 5 --keep-going --column_family_idx=1
total,interval_op_rate,interval_key_rate,avg_latency,elapsed_time
100,20,20,0.00125009775162,0
Bootstrap 2nd node and now failure:
$ python stress.py --nodes 192.168.3.8 --operation read --num-keys 100
--progress-interval 5 --keep-going --column_family_idx=1
total,interval_op_rate,interval_key_rate,avg_latency,elapsed_time
100,20,20,0.00376108169556,0
$ python stress.py --nodes 192.168.3.8 --operation read --num-keys 100
--progress-interval 5 --keep-going --column_family_idx=2
Key 074 not found
Key 061 not found
Key 047 not found
( cut 40 more Key 0xx not found)
Key 047 not found
Key 042 not found
Key 058 not found
total,interval_op_rate,interval_key_rate,avg_latency,elapsed_time
Key 033 not found
100,20,20,0.00241538286209,0
Similar failure for 3rd CF.
[1]: Modified stress.py from 0.7.0 with --column_family_idx= added.
http://beauty.ant.gliwice.pl/bugs/cassandra-bootstrap/stress.py
> Bootstrap breaks data stored (missing rows, extra rows, column values
> modified)
> -------------------------------------------------------------------------------
>
> Key: CASSANDRA-1992
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1992
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 0.7.0
> Environment: Linux 2.6.36-1 #1 SMP Tue Nov 9 09:56:02 CET 2010 x86_64
> Intel(R)_Core(TM)2_Quad_CPU____Q8300__@_2.50GHz PLD Linux
> glibc-2.12-4.i686
> java-sun-1.6.0.22-1.i686
> Reporter: Mateusz Korniak
> Assignee: Brandon Williams
> Fix For: 0.7.1
>
> Original Estimate: 8h
> Remaining Estimate: 8h
>
> Scenario:
> Two fresh (empty /data /commitog /saved_caches dirs) cassandra installs.
> Start first one.
> Run data inserting program [1], run again in verify mode - all data intact.
> Bootstrap 2nd node.
> Run verification again, now it fails.
> Issue is very strange to me as cassandra works perfectly for me when cluster
> nodes stay the same for days now but any bootstrap ( 1 -> 2 nodes, 2 -> 3
> nodes, 2->3 nodes RF=2) breaks data.
> I am running cassandra with 1GB heap size, 32bit userland on 64bit kernels,
> not sure what else could matter there.
> Any hints ?
> Thanks in advance, regards.
> [1] simple program generating data and later verifying data.
> http://beauty.ant.gliwice.pl/bugs/cassandra-bootstrap/test.py
> [2] Logs from 1st node:
> http://beauty.ant.gliwice.pl/bugs/cassandra-bootstrap/system-3.4.log
> [3] Logs from 2nd (bootstraping node)
> http://beauty.ant.gliwice.pl/bugs/cassandra-bootstrap/system-3.8.log
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
