Sean McCarthy created CASSANDRA-13202: -----------------------------------------
Summary: dtest failure in
repair_tests.incremental_repair_test.TestIncRepair.sstable_marking_test
Key: CASSANDRA-13202
URL: https://issues.apache.org/jira/browse/CASSANDRA-13202
Project: Cassandra
Issue Type: Bug
Components: Testing
Reporter: Sean McCarthy
Attachments: node1_debug.log, node1_gc.log, node1.log,
node2_debug.log, node2_gc.log, node2.log, node3_debug.log, node3_gc.log,
node3.log
example failure:
http://cassci.datastax.com/job/trunk_offheap_dtest/427/testReport/repair_tests.incremental_repair_test/TestIncRepair/sstable_marking_test
{code}
Error Message
'Repaired at: 0' unexpectedly found in 'SSTable:
/tmp/dtest-9PYhKy/test/node1/data0/keyspace1/standard1-17eaf440edbb11e68d99c3f653778b71/md-1-big\nPartitioner:
org.apache.cassandra.dht.Murmur3Partitioner\nBloom Filter FP chance:
0.010000\nMinimum timestamp: 1486529856104000\nMaximum timestamp:
1486529859637013\nSSTable min local deletion time: 2147483647\nSSTable max
local deletion time: 2147483647\nCompressor: -\nTTL min: 0\nTTL max: 0\nFirst
token: -9222701292667950301 (key=5032394c323239385030)\nLast token:
-3134717340917976237 (key=304b3338324b324b3430)\nEstimated droppable
tombstones: 0.0\nSSTable Level: 0\nRepaired at: 0\nPending repair:
26e751a0-edbb-11e6-accb-61d17d26194a\nReplay positions covered:
{CommitLogPosition(segmentId=1486529830270,
position=41626)=CommitLogPosition(segmentId=1486529830270,
position=2604016)}\ntotalColumnsSet: 16365\ntotalRows: 3273\nEstimated
tombstone drop times:\nCount Row Size Cell Count\n1
0 0\n2 0
0\n3 0 0\n4
0 0\n5 0 6546\n6
0 0\n7 0
0\n8 0 0\n10
0 0\n12 0 0\n14
0 0\n17 0
0\n20 0 0\n24
0 0\n29 0 0\n35
0 0\n42 0
0\n50 0 0\n60
0 0\n72 0 0\n86
0 0\n103 0
0\n124 0 0\n149
0 0\n179 0 0\n215
2 0\n258 3271
0\n310 0 0\n372
0 0\n446 0 0\n535
0 0\n642 0
0\n770 0 0\n924
0 0\n1109 0 0\n1331
0 0\n1597 0
0\n1916 0 0\n2299
0 0\n2759 0 0\n3311
0 0\n3973 0
0\n4768 0 0\n5722
0 0\n6866 0 0\n8239
0 0\n9887 0
0\n11864 0 0\n14237
0 0\n17084 0 0\n20501
0 0\n24601 0
0\n29521 0 0\n35425
0 0\n42510 0 0\n51012
0 0\n61214 0
0\n73457 0 0\n88148
0 0\n105778 0 0\n126934
0 0\n152321 0
0\n182785 0 0\n219342
0 0\n263210 0 0\n315852
0 0\n379022 0
0\n454826 0 0\n545791
0 0\n654949 0 0\n785939
0 0\n943127 0
0\n1131752 0 0\n1358102
0 0\n1629722 0 0\n1955666
0 0\n2346799 0
0\n2816159 0 0\n3379391
0 0\n4055269 0 0\n4866323
0 0\n5839588 0
0\n7007506 0 0\n8409007
0 0\n10090808 0 0\n12108970
0 0\n14530764 0
0\n17436917 0 0\n20924300
0 0\n25109160 0 0\n30130992
0 0\n36157190 0
0\n43388628 0 0\n52066354
0 0\n62479625 0 0\n74975550
0 0\n89970660 0
0\n107964792 0 0\n129557750
0 0\n155469300 0
0\n186563160 0 0\n223875792 0
0\n268650950 0 0\n322381140
0 0\n386857368 0
0\n464228842 0 0\n557074610 0
0\n668489532 0 0\n802187438
0 0\n962624926 0
0\n1155149911 0 0\n1386179893 0
0\n1663415872 0 0\n1996099046
0 0\n2395318855 0
0\n2874382626 0 \n3449259151 0
\n4139110981 0 \n4966933177
0 \n5960319812 0
\n7152383774 0 \n8582860529 0
\n10299432635 0 \n12359319162
0 \n14831182994 0
\n17797419593 0 \n21356903512
0 \n25628284214 0
\n30753941057 0 \n36904729268
0 \n44285675122 0
\n53142810146 0 \n63771372175
0 \n76525646610 0
\n91830775932 0 \n110196931118
0 \n132236317342 0
\n158683580810 0 \n190420296972
0 \n228504356366 0
\n274205227639 0 \n329046273167
0 \n394855527800 0
\n473826633360 0 \n568591960032
0 \n682310352038 0
\n818772422446 0 \n982526906935
0 \n1179032288322 0
\n1414838745986 0 \nEstimated cardinality:
3273\nEncodingStats minTTL: 0\nEncodingStats minLocalDeletionTime:
1442880000\nEncodingStats minTimestamp: 1486529856104000\nKeyType:
org.apache.cassandra.db.marshal.BytesType\nClusteringTypes:
[org.apache.cassandra.db.marshal.UTF8Type]\nStaticColumns:
{C3:org.apache.cassandra.db.marshal.BytesType,
C4:org.apache.cassandra.db.marshal.BytesType,
C0:org.apache.cassandra.db.marshal.BytesType,
C1:org.apache.cassandra.db.marshal.BytesType,
C2:org.apache.cassandra.db.marshal.BytesType}\nRegularColumns: {}\nSSTable:
/tmp/dtest-9PYhKy/test/node1/data1/keyspace1/standard1-17eaf440edbb11e68d99c3f653778b71/md-2-big\nPartitioner:
org.apache.cassandra.dht.Murmur3Partitioner\nBloom Filter FP chance:
0.010000\nMinimum timestamp: 1486529856104001\nMaximum timestamp:
1486529859637014\nSSTable min local deletion time: 2147483647\nSSTable max
local deletion time: 2147483647\nCompressor: -\nTTL min: 0\nTTL max: 0\nFirst
token: -3132058048208985688 (key=314f4d374b3631393730)\nLast token:
3156024197334322031 (key=3033384b323531303430)\nEstimated droppable tombstones:
0.0\nSSTable Level: 0\nRepaired at: 0\nPending repair:
26e751a0-edbb-11e6-accb-61d17d26194a\nReplay positions covered:
{CommitLogPosition(segmentId=1486529830270,
position=41626)=CommitLogPosition(segmentId=1486529830270,
position=2604016)}\ntotalColumnsSet: 17315\ntotalRows: 3463\nEstimated
tombstone drop times:\nCount Row Size Cell Count\n1
0 0\n2 0
0\n3 0 0\n4
0 0\n5 0 6926\n6
0 0\n7 0
0\n8 0 0\n10
0 0\n12 0 0\n14
0 0\n17 0
0\n20 0 0\n24
0 0\n29 0 0\n35
0 0\n42 0
0\n50 0 0\n60
0 0\n72 0 0\n86
0 0\n103 0
0\n124 0 0\n149
0 0\n179 0 0\n215
2 0\n258 3461
0\n310 0 0\n372
0 0\n446 0 0\n535
0 0\n642 0
0\n770 0 0\n924
0 0\n1109 0 0\n1331
0 0\n1597 0
0\n1916 0 0\n2299
0 0\n2759 0 0\n3311
0 0\n3973 0
0\n4768 0 0\n5722
0 0\n6866 0 0\n8239
0 0\n9887 0
0\n11864 0 0\n14237
0 0\n17084 0 0\n20501
0 0\n24601 0
0\n29521 0 0\n35425
0 0\n42510 0 0\n51012
0 0\n61214 0
0\n73457 0 0\n88148
0 0\n105778 0 0\n126934
0 0\n152321 0
0\n182785 0 0\n219342
0 0\n263210 0 0\n315852
0 0\n379022 0
0\n454826 0 0\n545791
0 0\n654949 0 0\n785939
0 0\n943127 0
0\n1131752 0 0\n1358102
0 0\n1629722 0 0\n1955666
0 0\n2346799 0
0\n2816159 0 0\n3379391
0 0\n4055269 0 0\n4866323
0 0\n5839588 0
0\n7007506 0 0\n8409007
0 0\n10090808 0 0\n12108970
0 0\n14530764 0
0\n17436917 0 0\n20924300
0 0\n25109160 0 0\n30130992
0 0\n36157190 0
0\n43388628 0 0\n52066354
0 0\n62479625 0 0\n74975550
0 0\n89970660 0
0\n107964792 0 0\n129557750
0 0\n155469300 0
0\n186563160 0 0\n223875792 0
0\n268650950 0 0\n322381140
0 0\n386857368 0
0\n464228842 0 0\n557074610 0
0\n668489532 0 0\n802187438
0 0\n962624926 0
0\n1155149911 0 0\n1386179893 0
0\n1663415872 0 0\n1996099046
0 0\n2395318855 0
0\n2874382626 0 \n3449259151 0
\n4139110981 0 \n4966933177
0 \n5960319812 0
\n7152383774 0 \n8582860529 0
\n10299432635 0 \n12359319162
0 \n14831182994 0
\n17797419593 0 \n21356903512
0 \n25628284214 0
\n30753941057 0 \n36904729268
0 \n44285675122 0
\n53142810146 0 \n63771372175
0 \n76525646610 0
\n91830775932 0 \n110196931118
0 \n132236317342 0
\n158683580810 0 \n190420296972
0 \n228504356366 0
\n274205227639 0 \n329046273167
0 \n394855527800 0
\n473826633360 0 \n568591960032
0 \n682310352038 0
\n818772422446 0 \n982526906935
0 \n1179032288322 0
\n1414838745986 0 \nEstimated cardinality:
3463\nEncodingStats minTTL: 0\nEncodingStats minLocalDeletionTime:
1442880000\nEncodingStats minTimestamp: 1486529856104000\nKeyType:
org.apache.cassandra.db.marshal.BytesType\nClusteringTypes:
[org.apache.cassandra.db.marshal.UTF8Type]\nStaticColumns:
{C3:org.apache.cassandra.db.marshal.BytesType,
C4:org.apache.cassandra.db.marshal.BytesType,
C0:org.apache.cassandra.db.marshal.BytesType,
C1:org.apache.cassandra.db.marshal.BytesType,
C2:org.apache.cassandra.db.marshal.BytesType}\nRegularColumns: {}\nSSTable:
/tmp/dtest-9PYhKy/test/node1/data2/keyspace1/standard1-17eaf440edbb11e68d99c3f653778b71/md-3-big\nPartitioner:
org.apache.cassandra.dht.Murmur3Partitioner\nBloom Filter FP chance:
0.010000\nMinimum timestamp: 1486529856105007\nMaximum timestamp:
1486529859637012\nSSTable min local deletion time: 2147483647\nSSTable max
local deletion time: 2147483647\nCompressor: -\nTTL min: 0\nTTL max: 0\nFirst
token: 3157556839665586758 (key=3136344d4c314d343631)\nLast token:
9222137691148971235 (key=4c30334f32394d4c3031)\nEstimated droppable tombstones:
0.0\nSSTable Level: 0\nRepaired at: 0\nPending repair:
26e751a0-edbb-11e6-accb-61d17d26194a\nReplay positions covered:
{CommitLogPosition(segmentId=1486529830270,
position=41626)=CommitLogPosition(segmentId=1486529830270,
position=2604016)}\ntotalColumnsSet: 16320\ntotalRows: 3264\nEstimated
tombstone drop times:\nCount Row Size Cell Count\n1
0 0\n2 0
0\n3 0 0\n4
0 0\n5 0 6528\n6
0 0\n7 0
0\n8 0 0\n10
0 0\n12 0 0\n14
0 0\n17 0
0\n20 0 0\n24
0 0\n29 0 0\n35
0 0\n42 0
0\n50 0 0\n60
0 0\n72 0 0\n86
0 0\n103 0
0\n124 0 0\n149
0 0\n179 0 0\n215
0 0\n258 3264
0\n310 0 0\n372
0 0\n446 0 0\n535
0 0\n642 0
0\n770 0 0\n924
0 0\n1109 0 0\n1331
0 0\n1597 0
0\n1916 0 0\n2299
0 0\n2759 0 0\n3311
0 0\n3973 0
0\n4768 0 0\n5722
0 0\n6866 0 0\n8239
0 0\n9887 0
0\n11864 0 0\n14237
0 0\n17084 0 0\n20501
0 0\n24601 0
0\n29521 0 0\n35425
0 0\n42510 0 0\n51012
0 0\n61214 0
0\n73457 0 0\n88148
0 0\n105778 0 0\n126934
0 0\n152321 0
0\n182785 0 0\n219342
0 0\n263210 0 0\n315852
0 0\n379022 0
0\n454826 0 0\n545791
0 0\n654949 0 0\n785939
0 0\n943127 0
0\n1131752 0 0\n1358102
0 0\n1629722 0 0\n1955666
0 0\n2346799 0
0\n2816159 0 0\n3379391
0 0\n4055269 0 0\n4866323
0 0\n5839588 0
0\n7007506 0 0\n8409007
0 0\n10090808 0 0\n12108970
0 0\n14530764 0
0\n17436917 0 0\n20924300
0 0\n25109160 0 0\n30130992
0 0\n36157190 0
0\n43388628 0 0\n52066354
0 0\n62479625 0 0\n74975550
0 0\n89970660 0
0\n107964792 0 0\n129557750
0 0\n155469300 0
0\n186563160 0 0\n223875792 0
0\n268650950 0 0\n322381140
0 0\n386857368 0
0\n464228842 0 0\n557074610 0
0\n668489532 0 0\n802187438
0 0\n962624926 0
0\n1155149911 0 0\n1386179893 0
0\n1663415872 0 0\n1996099046
0 0\n2395318855 0
0\n2874382626 0 \n3449259151 0
\n4139110981 0 \n4966933177
0 \n5960319812 0
\n7152383774 0 \n8582860529 0
\n10299432635 0 \n12359319162
0 \n14831182994 0
\n17797419593 0 \n21356903512
0 \n25628284214 0
\n30753941057 0 \n36904729268
0 \n44285675122 0
\n53142810146 0 \n63771372175
0 \n76525646610 0
\n91830775932 0 \n110196931118
0 \n132236317342 0
\n158683580810 0 \n190420296972
0 \n228504356366 0
\n274205227639 0 \n329046273167
0 \n394855527800 0
\n473826633360 0 \n568591960032
0 \n682310352038 0
\n818772422446 0 \n982526906935
0 \n1179032288322 0
\n1414838745986 0 \nEstimated cardinality:
3264\nEncodingStats minTTL: 0\nEncodingStats minLocalDeletionTime:
1442880000\nEncodingStats minTimestamp: 1486529856104000\nKeyType:
org.apache.cassandra.db.marshal.BytesType\nClusteringTypes:
[org.apache.cassandra.db.marshal.UTF8Type]\nStaticColumns:
{C3:org.apache.cassandra.db.marshal.BytesType,
C4:org.apache.cassandra.db.marshal.BytesType,
C0:org.apache.cassandra.db.marshal.BytesType,
C1:org.apache.cassandra.db.marshal.BytesType,
C2:org.apache.cassandra.db.marshal.BytesType}\nRegularColumns: {}\n'
{code}{code}
Stacktrace
File "/usr/lib/python2.7/unittest/case.py", line 329, in run
testMethod()
File
"/home/automaton/cassandra-dtest/repair_tests/incremental_repair_test.py", line
55, in sstable_marking_test
self.assertNotIn('Repaired at: 0', out)
File "/usr/lib/python2.7/unittest/case.py", line 810, in assertNotIn
self.fail(self._formatMessage(msg, standardMsg))
File "/usr/lib/python2.7/unittest/case.py", line 410, in fail
raise self.failureException(msg)
{code}
Related failures:
http://cassci.datastax.com/job/trunk_offheap_dtest/427/testReport/repair_tests.incremental_repair_test/TestIncRepair/sstable_repairedset_test/
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
