Joel Knighton created CASSANDRA-12337:
-----------------------------------------
Summary: dtest failure in
scrub_test.TestScrubIndexes.test_standalone_scrub
Key: CASSANDRA-12337
URL: https://issues.apache.org/jira/browse/CASSANDRA-12337
Project: Cassandra
Issue Type: Bug
Reporter: Joel Knighton
We have an existing open ticket for this test in [CASSANDRA-11236]. That ticket
is for a Windows failure with a different failure mode. Since the resolution
will likely be different, I've opened this ticket to track the most recent
failure.
example failure:
[http://cassci.datastax.com/job/cassandra-3.9_dtest/lastCompletedBuild/testReport/junit/scrub_test/TestScrubIndexes/test_standalone_scrub/]
{code}
sstablescrub failed
-------------------- >> begin captured logging << --------------------
dtest: DEBUG: cluster ccm directory: /tmp/dtest-Ilk7GU
dtest: DEBUG: Done setting configuration options:
{ 'initial_token': None,
'num_tokens': '32',
'phi_convict_threshold': 5,
'range_request_timeout_in_ms': 10000,
'read_request_timeout_in_ms': 10000,
'request_timeout_in_ms': 10000,
'truncate_request_timeout_in_ms': 10000,
'write_request_timeout_in_ms': 10000}
dtest: DEBUG: Checking sstables in
['/tmp/dtest-Ilk7GU/test/node1/data0/ks/users-b1956190547611e683f7630547833e83']
dtest: DEBUG: Found sstable file mb-1-big-Statistics.db
dtest: DEBUG: Found sstable file mb-1-big-CRC.db
dtest: DEBUG: Found sstable file mb-1-big-Filter.db
dtest: DEBUG: Found sstable file mb-1-big-Summary.db
dtest: DEBUG: Found sstable file mb-1-big-Data.db
dtest: DEBUG: Found sstable file mb-1-big-Index.db
dtest: DEBUG: Found sstable file mb-1-big-TOC.txt
dtest: DEBUG: Checking sstables in
['/tmp/dtest-Ilk7GU/test/node1/data0/ks/users-b1956190547611e683f7630547833e83']
dtest: DEBUG: Found sstable file mb-1-big-Statistics.db
dtest: DEBUG: Found sstable file mb-1-big-CRC.db
dtest: DEBUG: Found sstable file mb-1-big-Filter.db
dtest: DEBUG: Found sstable file mb-1-big-Summary.db
dtest: DEBUG: Found sstable file mb-1-big-Data.db
dtest: DEBUG: Found sstable file mb-1-big-Index.db
dtest: DEBUG: Found sstable file mb-1-big-TOC.txt
dtest: DEBUG: Checking sstables in
['/tmp/dtest-Ilk7GU/test/node1/data0/ks/users-b1956190547611e683f7630547833e83']
dtest: DEBUG: Found sstable file mb-1-big-Statistics.db
dtest: DEBUG: Found sstable file mb-1-big-CRC.db
dtest: DEBUG: Found sstable file mb-1-big-Filter.db
dtest: DEBUG: Found sstable file mb-1-big-Summary.db
dtest: DEBUG: Found sstable file mb-1-big-Data.db
dtest: DEBUG: Found sstable file mb-1-big-Index.db
dtest: DEBUG: Found sstable file mb-1-big-TOC.txt
dtest: DEBUG: Checking sstables in
['/tmp/dtest-Ilk7GU/test/node1/data0/ks/users-b1956190547611e683f7630547833e83']
dtest: DEBUG: Found sstable file mb-1-big-Statistics.db
dtest: DEBUG: Found sstable file mb-1-big-CRC.db
dtest: DEBUG: Found sstable file mb-1-big-Filter.db
dtest: DEBUG: Found sstable file mb-1-big-Summary.db
dtest: DEBUG: Found sstable file mb-1-big-Data.db
dtest: DEBUG: Found sstable file mb-1-big-Index.db
dtest: DEBUG: Found sstable file mb-1-big-TOC.txt
dtest: DEBUG: /home/automaton/cassandra/bin/sstablescrub
dtest: DEBUG: Pre-scrub sstables snapshotted into snapshot
pre-scrub-1469677957710
Scrubbing
BigTableReader(path='/tmp/dtest-Ilk7GU/test/node1/data0/ks/users-b1956190547611e683f7630547833e83/mb-1-big-Data.db')
(0.317KiB)
Scrub of
BigTableReader(path='/tmp/dtest-Ilk7GU/test/node1/data0/ks/users-b1956190547611e683f7630547833e83/mb-1-big-Data.db')
complete; looks like all 0 rows were tombstoned
dtest: DEBUG: /home/automaton/cassandra/bin/sstablescrub
dtest: DEBUG: Pre-scrub sstables snapshotted into snapshot
pre-scrub-1469677962057
Scrubbing
BigTableReader(path='/tmp/dtest-Ilk7GU/test/node1/data0/ks/users-b1956190547611e683f7630547833e83/.gender_idx/mb-1-big-Data.db')
(0.176KiB)
Scrub of
BigTableReader(path='/tmp/dtest-Ilk7GU/test/node1/data0/ks/users-b1956190547611e683f7630547833e83/.gender_idx/mb-1-big-Data.db')
complete; looks like all 0 rows were tombstoned
dtest: DEBUG: /home/automaton/cassandra/bin/sstablescrub
dtest: DEBUG: Pre-scrub sstables snapshotted into snapshot
pre-scrub-1469677966308
Scrubbing
BigTableReader(path='/tmp/dtest-Ilk7GU/test/node1/data0/ks/users-b1956190547611e683f7630547833e83/.state_idx/mb-1-big-Data.db')
(0.178KiB)
Scrub of
BigTableReader(path='/tmp/dtest-Ilk7GU/test/node1/data0/ks/users-b1956190547611e683f7630547833e83/.state_idx/mb-1-big-Data.db')
complete; looks like all 0 rows were tombstoned
dtest: DEBUG: /home/automaton/cassandra/bin/sstablescrub
dtest: DEBUG: Pre-scrub sstables snapshotted into snapshot
pre-scrub-1469677970549
Scrubbing
BigTableReader(path='/tmp/dtest-Ilk7GU/test/node1/data0/ks/users-b1956190547611e683f7630547833e83/.birth_year_idx/mb-1-big-Data.db')
(0.189KiB)
Scrub of
BigTableReader(path='/tmp/dtest-Ilk7GU/test/node1/data0/ks/users-b1956190547611e683f7630547833e83/.birth_year_idx/mb-1-big-Data.db')
complete; looks like all 0 rows were tombstoned
dtest: DEBUG: ERROR 03:52:50 Error in ThreadPoolExecutor
java.util.concurrent.RejectedExecutionException: ThreadPoolExecutor has shut
down
at
org.apache.cassandra.concurrent.DebuggableThreadPoolExecutor$1.rejectedExecution(DebuggableThreadPoolExecutor.java:61)
~[main/:na]
at
java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:823)
~[na:1.8.0_45]
at
java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1364)
~[na:1.8.0_45]
at
org.apache.cassandra.concurrent.DebuggableThreadPoolExecutor.execute(DebuggableThreadPoolExecutor.java:165)
~[main/:na]
at
java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112)
~[na:1.8.0_45]
at
org.apache.cassandra.db.compaction.CompactionManager.submitBackground(CompactionManager.java:184)
~[main/:na]
at
org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionCandidate.run(CompactionManager.java:270)
~[main/:na]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
~[na:1.8.0_45]
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
~[na:1.8.0_45]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
~[na:1.8.0_45]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[na:1.8.0_45]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
{code}
I haven't seen a similar failure on any branch.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
