Michael Shuler created CASSANDRA-13210:
------------------------------------------

             Summary: test failure in 
repair_tests.incremental_repair_test.TestIncRepair.sstable_marking_test_not_intersecting_all_ranges
                 Key: CASSANDRA-13210
                 URL: https://issues.apache.org/jira/browse/CASSANDRA-13210
             Project: Cassandra
          Issue Type: Bug
            Reporter: Michael Shuler
         Attachments: node1_debug.log, node1_gc.log, node1.log, 
node2_debug.log, node2_gc.log, node2.log, node3_debug.log, node3_gc.log, 
node3.log, node4_debug.log, node4_gc.log, node4.log

example failure:

http://cassci.datastax.com/job/trunk_large_dtest/53/testReport/repair_tests.incremental_repair_test/TestIncRepair/sstable_marking_test_not_intersecting_all_ranges

{noformat}
Error Message

'Repaired at: 0' unexpectedly found in 'SSTable: 
/tmp/dtest-N7zjo6/test/node1/data0/keyspace1/standard1-a79a0c50efa211e6bf211330662f36ef/md-6-big\nPartitioner:
 org.apache.cassandra.dht.Murmur3Partitioner\nBloom Filter FP chance: 
0.010000\nMinimum timestamp: 1486739263230000\nMaximum timestamp: 
1486739263230000\nSSTable min local deletion time: 2147483647\nSSTable max 
local deletion time: 2147483647\nCompressor: -\nTTL min: 0\nTTL max: 0\nFirst 
token: 296988783704308703 (key=30503337373039503231)\nLast token: 
296988783704308703 (key=30503337373039503231)\nEstimated droppable tombstones: 
0.0\nSSTable Level: 0\nRepaired at: 0\nPending repair: 
b099d1f0-efa2-11e6-89ec-d14624f1e47e\nReplay positions covered: 
{CommitLogPosition(segmentId=1486739234777, 
position=46796)=CommitLogPosition(segmentId=1486739234777, 
position=50819)}\ntotalColumnsSet: 5\ntotalRows: 1\nEstimated tombstone drop 
times:\nCount               Row Size        Cell Count\n1                       
   0                 0\n2                          0                 0\n3       
                   0                 0\n4                          0            
     0\n5                          0                 2\n6                       
   0                 0\n7                          0                 0\n8       
                   0                 0\n10                         0            
     0\n12                         0                 0\n14                      
   0                 0\n17                         0                 0\n20      
                   0                 0\n24                         0            
     0\n29                         0                 0\n35                      
   0                 0\n42                         0                 0\n50      
                   0                 0\n60                         0            
     0\n72                         0                 0\n86                      
   0                 0\n103                        0                 0\n124     
                   0                 0\n149                        0            
     0\n179                        0                 0\n215                     
   1                 0\n258                        0                 0\n310     
                   0                 0\n372                        0            
     0\n446                        0                 0\n535                     
   0                 0\n642                        0                 0\n770     
                   0                 0\n924                        0            
     0\n1109                       0                 0\n1331                    
   0                 0\n1597                       0                 0\n1916    
                   0                 0\n2299                       0            
     0\n2759                       0                 0\n3311                    
   0                 0\n3973                       0                 0\n4768    
                   0                 0\n5722                       0            
     0\n6866                       0                 0\n8239                    
   0                 0\n9887                       0                 0\n11864   
                   0                 0\n14237                      0            
     0\n17084                      0                 0\n20501                   
   0                 0\n24601                      0                 0\n29521   
                   0                 0\n35425                      0            
     0\n42510                      0                 0\n51012                   
   0                 0\n61214                      0                 0\n73457   
                   0                 0\n88148                      0            
     0\n105778                     0                 0\n126934                  
   0                 0\n152321                     0                 0\n182785  
                   0                 0\n219342                     0            
     0\n263210                     0                 0\n315852                  
   0                 0\n379022                     0                 0\n454826  
                   0                 0\n545791                     0            
     0\n654949                     0                 0\n785939                  
   0                 0\n943127                     0                 0\n1131752 
                   0                 0\n1358102                    0            
     0\n1629722                    0                 0\n1955666                 
   0                 0\n2346799                    0                 0\n2816159 
                   0                 0\n3379391                    0            
     0\n4055269                    0                 0\n4866323                 
   0                 0\n5839588                    0                 0\n7007506 
                   0                 0\n8409007                    0            
     0\n10090808                   0                 0\n12108970                
   0                 0\n14530764                   0                 
0\n17436917                   0                 0\n20924300                   0 
                0\n25109160                   0                 0\n30130992     
              0                 0\n36157190                   0                 
0\n43388628                   0                 0\n52066354                   0 
                0\n62479625                   0                 0\n74975550     
              0                 0\n89970660                   0                 
0\n107964792                  0                 0\n129557750                  0 
                0\n155469300                  0                 0\n186563160    
              0                 0\n223875792                  0                 
0\n268650950                  0                 0\n322381140                  0 
                0\n386857368                  0                 0\n464228842    
              0                 0\n557074610                  0                 
0\n668489532                  0                 0\n802187438                  0 
                0\n962624926                  0                 0\n1155149911   
              0                 0\n1386179893                 0                 
0\n1663415872                 0                 0\n1996099046                 0 
                0\n2395318855                 0                 0\n2874382626   
              0                  \n3449259151                 0                 
 \n4139110981                 0                  \n4966933177                 0 
                 \n5960319812                 0                  \n7152383774   
              0                  \n8582860529                 0                 
 \n10299432635                 0                  \n12359319162                 
0                  \n14831182994                 0                  
\n17797419593                 0                  \n21356903512                 
0                  \n25628284214                 0                  
\n30753941057                 0                  \n36904729268                 
0                  \n44285675122                 0                  
\n53142810146                 0                  \n63771372175                 
0                  \n76525646610                 0                  
\n91830775932                 0                  \n110196931118                 
0                  \n132236317342                 0                  
\n158683580810                 0                  \n190420296972                
 0                  \n228504356366                 0                  
\n274205227639                 0                  \n329046273167                
 0                  \n394855527800                 0                  
\n473826633360                 0                  \n568591960032                
 0                  \n682310352038                 0                  
\n818772422446                 0                  \n982526906935                
 0                  \n1179032288322                 0                  
\n1414838745986                 0                  \nEstimated cardinality: 
1\nEncodingStats minTTL: 0\nEncodingStats minLocalDeletionTime: 
1442880000\nEncodingStats minTimestamp: 1486739263230000\nKeyType: 
org.apache.cassandra.db.marshal.BytesType\nClusteringTypes: 
[org.apache.cassandra.db.marshal.UTF8Type]\nStaticColumns: 
{C3:org.apache.cassandra.db.marshal.BytesType, 
C4:org.apache.cassandra.db.marshal.BytesType, 
C0:org.apache.cassandra.db.marshal.BytesType, 
C1:org.apache.cassandra.db.marshal.BytesType, 
C2:org.apache.cassandra.db.marshal.BytesType}\nRegularColumns: {}\nSSTable: 
/tmp/dtest-N7zjo6/test/node1/data0/keyspace1/standard1-a79a0c50efa211e6bf211330662f36ef/md-4-big\nPartitioner:
 org.apache.cassandra.dht.Murmur3Partitioner\nBloom Filter FP chance: 
0.010000\nMinimum timestamp: 1486739263253000\nMaximum timestamp: 
1486739263253000\nSSTable min local deletion time: 2147483647\nSSTable max 
local deletion time: 2147483647\nCompressor: -\nTTL min: 0\nTTL max: 0\nFirst 
token: -2998312467040114775 (key=4f503030314c35393330)\nLast token: 
-2998312467040114775 (key=4f503030314c35393330)\nEstimated droppable 
tombstones: 0.0\nSSTable Level: 0\nRepaired at: 0\nPending repair: 
b099d1f0-efa2-11e6-89ec-d14624f1e47e\nReplay positions covered: 
{CommitLogPosition(segmentId=1486739234777, 
position=46796)=CommitLogPosition(segmentId=1486739234777, 
position=50819)}\ntotalColumnsSet: 5\ntotalRows: 1\nEstimated tombstone drop 
times:\nCount               Row Size        Cell Count\n1                       
   0                 0\n2                          0                 0\n3       
                   0                 0\n4                          0            
     0\n5                          0                 2\n6                       
   0                 0\n7                          0                 0\n8       
                   0                 0\n10                         0            
     0\n12                         0                 0\n14                      
   0                 0\n17                         0                 0\n20      
                   0                 0\n24                         0            
     0\n29                         0                 0\n35                      
   0                 0\n42                         0                 0\n50      
                   0                 0\n60                         0            
     0\n72                         0                 0\n86                      
   0                 0\n103                        0                 0\n124     
                   0                 0\n149                        0            
     0\n179                        0                 0\n215                     
   1                 0\n258                        0                 0\n310     
                   0                 0\n372                        0            
     0\n446                        0                 0\n535                     
   0                 0\n642                        0                 0\n770     
                   0                 0\n924                        0            
     0\n1109                       0                 0\n1331                    
   0                 0\n1597                       0                 0\n1916    
                   0                 0\n2299                       0            
     0\n2759                       0                 0\n3311                    
   0                 0\n3973                       0                 0\n4768    
                   0                 0\n5722                       0            
     0\n6866                       0                 0\n8239                    
   0                 0\n9887                       0                 0\n11864   
                   0                 0\n14237                      0            
     0\n17084                      0                 0\n20501                   
   0                 0\n24601                      0                 0\n29521   
                   0                 0\n35425                      0            
     0\n42510                      0                 0\n51012                   
   0                 0\n61214                      0                 0\n73457   
                   0                 0\n88148                      0            
     0\n105778                     0                 0\n126934                  
   0                 0\n152321                     0                 0\n182785  
                   0                 0\n219342                     0            
     0\n263210                     0                 0\n315852                  
   0                 0\n379022                     0                 0\n454826  
                   0                 0\n545791                     0            
     0\n654949                     0                 0\n785939                  
   0                 0\n943127                     0                 0\n1131752 
                   0                 0\n1358102                    0            
     0\n1629722                    0                 0\n1955666                 
   0                 0\n2346799                    0                 0\n2816159 
                   0                 0\n3379391                    0            
     0\n4055269                    0                 0\n4866323                 
   0                 0\n5839588                    0                 0\n7007506 
                   0                 0\n8409007                    0            
     0\n10090808                   0                 0\n12108970                
   0                 0\n14530764                   0                 
0\n17436917                   0                 0\n20924300                   0 
                0\n25109160                   0                 0\n30130992     
              0                 0\n36157190                   0                 
0\n43388628                   0                 0\n52066354                   0 
                0\n62479625                   0                 0\n74975550     
              0                 0\n89970660                   0                 
0\n107964792                  0                 0\n129557750                  0 
                0\n155469300                  0                 0\n186563160    
              0                 0\n223875792                  0                 
0\n268650950                  0                 0\n322381140                  0 
                0\n386857368                  0                 0\n464228842    
              0                 0\n557074610                  0                 
0\n668489532                  0                 0\n802187438                  0 
                0\n962624926                  0                 0\n1155149911   
              0                 0\n1386179893                 0                 
0\n1663415872                 0                 0\n1996099046                 0 
                0\n2395318855                 0                 0\n2874382626   
              0                  \n3449259151                 0                 
 \n4139110981                 0                  \n4966933177                 0 
                 \n5960319812                 0                  \n7152383774   
              0                  \n8582860529                 0                 
 \n10299432635                 0                  \n12359319162                 
0                  \n14831182994                 0                  
\n17797419593                 0                  \n21356903512                 
0                  \n25628284214                 0                  
\n30753941057                 0                  \n36904729268                 
0                  \n44285675122                 0                  
\n53142810146                 0                  \n63771372175                 
0                  \n76525646610                 0                  
\n91830775932                 0                  \n110196931118                 
0                  \n132236317342                 0                  
\n158683580810                 0                  \n190420296972                
 0                  \n228504356366                 0                  
\n274205227639                 0                  \n329046273167                
 0                  \n394855527800                 0                  
\n473826633360                 0                  \n568591960032                
 0                  \n682310352038                 0                  
\n818772422446                 0                  \n982526906935                
 0                  \n1179032288322                 0                  
\n1414838745986                 0                  \nEstimated cardinality: 
1\nEncodingStats minTTL: 0\nEncodingStats minLocalDeletionTime: 
1442880000\nEncodingStats minTimestamp: 1486739263253000\nKeyType: 
org.apache.cassandra.db.marshal.BytesType\nClusteringTypes: 
[org.apache.cassandra.db.marshal.UTF8Type]\nStaticColumns: 
{C3:org.apache.cassandra.db.marshal.BytesType, 
C4:org.apache.cassandra.db.marshal.BytesType, 
C0:org.apache.cassandra.db.marshal.BytesType, 
C1:org.apache.cassandra.db.marshal.BytesType, 
C2:org.apache.cassandra.db.marshal.BytesType}\nRegularColumns: {}\nSSTable: 
/tmp/dtest-N7zjo6/test/node1/data0/keyspace1/standard1-a79a0c50efa211e6bf211330662f36ef/md-8-big\nPartitioner:
 org.apache.cassandra.dht.Murmur3Partitioner\nBloom Filter FP chance: 
0.010000\nMinimum timestamp: 1486739263242000\nMaximum timestamp: 
1486739263242000\nSSTable min local deletion time: 2147483647\nSSTable max 
local deletion time: 2147483647\nCompressor: -\nTTL min: 0\nTTL max: 0\nFirst 
token: 3151688024208981352 (key=4f384c4b37394c4f3631)\nLast token: 
3151688024208981352 (key=4f384c4b37394c4f3631)\nEstimated droppable tombstones: 
0.0\nSSTable Level: 0\nRepaired at: 0\nPending repair: 
b099d1f0-efa2-11e6-89ec-d14624f1e47e\nReplay positions covered: 
{CommitLogPosition(segmentId=1486739234777, 
position=46796)=CommitLogPosition(segmentId=1486739234777, 
position=50819)}\ntotalColumnsSet: 5\ntotalRows: 1\nEstimated tombstone drop 
times:\nCount               Row Size        Cell Count\n1                       
   0                 0\n2                          0                 0\n3       
                   0                 0\n4                          0            
     0\n5                          0                 2\n6                       
   0                 0\n7                          0                 0\n8       
                   0                 0\n10                         0            
     0\n12                         0                 0\n14                      
   0                 0\n17                         0                 0\n20      
                   0                 0\n24                         0            
     0\n29                         0                 0\n35                      
   0                 0\n42                         0                 0\n50      
                   0                 0\n60                         0            
     0\n72                         0                 0\n86                      
   0                 0\n103                        0                 0\n124     
                   0                 0\n149                        0            
     0\n179                        0                 0\n215                     
   1                 0\n258                        0                 0\n310     
                   0                 0\n372                        0            
     0\n446                        0                 0\n535                     
   0                 0\n642                        0                 0\n770     
                   0                 0\n924                        0            
     0\n1109                       0                 0\n1331                    
   0                 0\n1597                       0                 0\n1916    
                   0                 0\n2299                       0            
     0\n2759                       0                 0\n3311                    
   0                 0\n3973                       0                 0\n4768    
                   0                 0\n5722                       0            
     0\n6866                       0                 0\n8239                    
   0                 0\n9887                       0                 0\n11864   
                   0                 0\n14237                      0            
     0\n17084                      0                 0\n20501                   
   0                 0\n24601                      0                 0\n29521   
                   0                 0\n35425                      0            
     0\n42510                      0                 0\n51012                   
   0                 0\n61214                      0                 0\n73457   
                   0                 0\n88148                      0            
     0\n105778                     0                 0\n126934                  
   0                 0\n152321                     0                 0\n182785  
                   0                 0\n219342                     0            
     0\n263210                     0                 0\n315852                  
   0                 0\n379022                     0                 0\n454826  
                   0                 0\n545791                     0            
     0\n654949                     0                 0\n785939                  
   0                 0\n943127                     0                 0\n1131752 
                   0                 0\n1358102                    0            
     0\n1629722                    0                 0\n1955666                 
   0                 0\n2346799                    0                 0\n2816159 
                   0                 0\n3379391                    0            
     0\n4055269                    0                 0\n4866323                 
   0                 0\n5839588                    0                 0\n7007506 
                   0                 0\n8409007                    0            
     0\n10090808                   0                 0\n12108970                
   0                 0\n14530764                   0                 
0\n17436917                   0                 0\n20924300                   0 
                0\n25109160                   0                 0\n30130992     
              0                 0\n36157190                   0                 
0\n43388628                   0                 0\n52066354                   0 
                0\n62479625                   0                 0\n74975550     
              0                 0\n89970660                   0                 
0\n107964792                  0                 0\n129557750                  0 
                0\n155469300                  0                 0\n186563160    
              0                 0\n223875792                  0                 
0\n268650950                  0                 0\n322381140                  0 
                0\n386857368                  0                 0\n464228842    
              0                 0\n557074610                  0                 
0\n668489532                  0                 0\n802187438                  0 
                0\n962624926                  0                 0\n1155149911   
              0                 0\n1386179893                 0                 
0\n1663415872                 0                 0\n1996099046                 0 
                0\n2395318855                 0                 0\n2874382626   
              0                  \n3449259151                 0                 
 \n4139110981                 0                  \n4966933177                 0 
                 \n5960319812                 0                  \n7152383774   
              0                  \n8582860529                 0                 
 \n10299432635                 0                  \n12359319162                 
0                  \n14831182994                 0                  
\n17797419593                 0                  \n21356903512                 
0                  \n25628284214                 0                  
\n30753941057                 0                  \n36904729268                 
0                  \n44285675122                 0                  
\n53142810146                 0                  \n63771372175                 
0                  \n76525646610                 0                  
\n91830775932                 0                  \n110196931118                 
0                  \n132236317342                 0                  
\n158683580810                 0                  \n190420296972                
 0                  \n228504356366                 0                  
\n274205227639                 0                  \n329046273167                
 0                  \n394855527800                 0                  
\n473826633360                 0                  \n568591960032                
 0                  \n682310352038                 0                  
\n818772422446                 0                  \n982526906935                
 0                  \n1179032288322                 0                  
\n1414838745986                 0                  \nEstimated cardinality: 
1\nEncodingStats minTTL: 0\nEncodingStats minLocalDeletionTime: 
1442880000\nEncodingStats minTimestamp: 1486739263242000\nKeyType: 
org.apache.cassandra.db.marshal.BytesType\nClusteringTypes: 
[org.apache.cassandra.db.marshal.UTF8Type]\nStaticColumns: 
{C3:org.apache.cassandra.db.marshal.BytesType, 
C4:org.apache.cassandra.db.marshal.BytesType, 
C0:org.apache.cassandra.db.marshal.BytesType, 
C1:org.apache.cassandra.db.marshal.BytesType, 
C2:org.apache.cassandra.db.marshal.BytesType}\nRegularColumns: {}\n'
{noformat}

{noformat}
Stacktrace

  File "/usr/lib/python2.7/unittest/case.py", line 329, in run
    testMethod()
  File 
"/home/automaton/cassandra-dtest/repair_tests/incremental_repair_test.py", line 
644, in sstable_marking_test_not_intersecting_all_ranges
    self.assertNotIn('Repaired at: 0', out)
  File "/usr/lib/python2.7/unittest/case.py", line 810, in assertNotIn
    self.fail(self._formatMessage(msg, standardMsg))
  File "/usr/lib/python2.7/unittest/case.py", line 410, in fail
    raise self.failureException(msg)
{noformat}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Reply via email to