Craig Kodman created CASSANDRA-12439:
----------------------------------------
Summary: dtest failure in
repair_tests.repair_test.TestRepair.test_multiple_concurrent_repairs
Key: CASSANDRA-12439
URL: https://issues.apache.org/jira/browse/CASSANDRA-12439
Project: Cassandra
Issue Type: Test
Reporter: Craig Kodman
Assignee: DS Test Eng
Attachments: node1.log, node1_debug.log, node1_gc.log, node2.log,
node2_debug.log, node2_gc.log, node3.log, node3_debug.log, node3_gc.log
example failure:
http://cassci.datastax.com/job/cassandra-2.2_novnode_dtest/318/testReport/repair_tests.repair_test/TestRepair/test_multiple_concurrent_repairs
{code}
Error Message
ERROR 15:43:07 Error creating pool to /127.0.0.3:9042
com.datastax.driver.core.TransportException: [/127.0.0.3:9042] Cannot connect
at
com.datastax.driver.core.Connection$1.operationComplete(Connection.java:156)
[cassandra-driver-core-2.2.0-rc2-SNAPSHOT-20150617-shaded.jar:na]
at
com.datastax.driver.core.Connection$1.operationComplete(Connection.java:139)
[cassandra-driver-core-2.2.0-rc2-SNAPSHOT-20150617-shaded.jar:na]
at
com.datastax.shaded.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:680)
[cassandra-driver-core-2.2.0-rc2-SNAPSHOT-20150617-shaded.jar:na]
at
com.datastax.shaded.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:603)
[cassandra-driver-core-2.2.0-rc2-SNAPSHOT-20150617-shaded.jar:na]
at
com.datastax.shaded.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:563)
[cassandra-driver-core-2.2.0-rc2-SNAPSHOT-20150617-shaded.jar:na]
at
com.datastax.shaded.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:424)
[cassandra-driver-core-2.2.0-rc2-SNAPSHOT-20150617-shaded.jar:na]
at
com.datastax.shaded.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.fulfillConnectPromise(AbstractNioChannel.java:268)
[cassandra-driver-core-2.2.0-rc2-SNAPSHOT-20150617-shaded.jar:na]
at
com.datastax.shaded.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:284)
[cassandra-driver-core-2.2.0-rc2-SNAPSHOT-20150617-shaded.jar:na]
at
com.datastax.shaded.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:528)
[cassandra-driver-core-2.2.0-rc2-SNAPSHOT-20150617-shaded.jar:na]
at
com.datastax.shaded.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468)
[cassandra-driver-core-2.2.0-rc2-SNAPSHOT-20150617-shaded.jar:na]
at
com.datastax.shaded.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382)
[cassandra-driver-core-2.2.0-rc2-SNAPSHOT-20150617-shaded.jar:na]
at
com.datastax.shaded.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354)
[cassandra-driver-core-2.2.0-rc2-SNAPSHOT-20150617-shaded.jar:na]
at
com.datastax.shaded.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:111)
[cassandra-driver-core-2.2.0-rc2-SNAPSHOT-20150617-shaded.jar:na]
at java.lang.Thread.run(Thread.java:745) [na:1.7.0_80]
Caused by: java.net.ConnectException: Connection refused: /127.0.0.3:9042
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
~[na:1.7.0_80]
at
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:744)
~[na:1.7.0_80]
at
com.datastax.shaded.netty.channel.socket.nio.NioSocketChannel.doFinishConnect(NioSocketChannel.java:224)
~[cassandra-driver-core-2.2.0-rc2-SNAPSHOT-20150617-shaded.jar:na]
at
com.datastax.shaded.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:281)
[cassandra-driver-core-2.2.0-rc2-SNAPSHOT-20150617-shaded.jar:na]
... 6 common frames omitted
ERROR 15:43:07 Error creating pool to /127.0.0.1:9042
com.datastax.driver.core.TransportException: [/127.0.0.1:9042] Cannot connect
at
com.datastax.driver.core.Connection$1.operationComplete(Connection.java:156)
[cassandra-driver-core-2.2.0-rc2-SNAPSHOT-20150617-shaded.jar:na]
at
com.datastax.driver.core.Connection$1.operationComplete(Connection.java:139)
[cassandra-driver-core-2.2.0-rc2-SNAPSHOT-20150617-shaded.jar:na]
at
com.datastax.shaded.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:680)
[cassandra-driver-core-2.2.0-rc2-SNAPSHOT-20150617-shaded.jar:na]
at
com.datastax.shaded.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:603)
[cassandra-driver-core-2.2.0-rc2-SNAPSHOT-20150617-shaded.jar:na]
at
com.datastax.shaded.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:563)
[cassandra-driver-core-2.2.0-rc2-SNAPSHOT-20150617-shaded.jar:na]
at
com.datastax.shaded.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:424)
[cassandra-driver-core-2.2.0-rc2-SNAPSHOT-20150617-shaded.jar:na]
at
com.datastax.shaded.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.fulfillConnectPromise(AbstractNioChannel.java:268)
[cassandra-driver-core-2.2.0-rc2-SNAPSHOT-20150617-shaded.jar:na]
at
com.datastax.shaded.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:284)
[cassandra-driver-core-2.2.0-rc2-SNAPSHOT-20150617-shaded.jar:na]
at
com.datastax.shaded.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:528)
[cassandra-driver-core-2.2.0-rc2-SNAPSHOT-20150617-shaded.jar:na]
at
com.datastax.shaded.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468)
[cassandra-driver-core-2.2.0-rc2-SNAPSHOT-20150617-shaded.jar:na]
at
com.datastax.shaded.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382)
[cassandra-driver-core-2.2.0-rc2-SNAPSHOT-20150617-shaded.jar:na]
at
com.datastax.shaded.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354)
[cassandra-driver-core-2.2.0-rc2-SNAPSHOT-20150617-shaded.jar:na]
at
com.datastax.shaded.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:111)
[cassandra-driver-core-2.2.0-rc2-SNAPSHOT-20150617-shaded.jar:na]
at java.lang.Thread.run(Thread.java:745) [na:1.7.0_80]
Caused by: java.net.ConnectException: Connection refused: /127.0.0.1:9042
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
~[na:1.7.0_80]
at
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:744)
~[na:1.7.0_80]
at
com.datastax.shaded.netty.channel.socket.nio.NioSocketChannel.doFinishConnect(NioSocketChannel.java:224)
~[cassandra-driver-core-2.2.0-rc2-SNAPSHOT-20150617-shaded.jar:na]
at
com.datastax.shaded.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:281)
[cassandra-driver-core-2.2.0-rc2-SNAPSHOT-20150617-shaded.jar:na]
... 6 common frames omitted
Failed to connect over JMX; not collecting these stats
-------------------- >> begin captured logging << --------------------
dtest: DEBUG: cluster ccm directory: /tmp/dtest-b0pWB9
dtest: DEBUG: Done setting configuration options:
{ 'num_tokens': None,
'phi_convict_threshold': 5,
'range_request_timeout_in_ms': 10000,
'read_request_timeout_in_ms': 10000,
'request_timeout_in_ms': 10000,
'truncate_request_timeout_in_ms': 10000,
'write_request_timeout_in_ms': 10000}
--------------------- >> end captured logging << ---------------------
{code}
{code}
Stacktrace
File "/usr/lib/python2.7/unittest/case.py", line 329, in run
testMethod()
File "/home/automaton/cassandra-dtest/repair_tests/repair_test.py", line 886,
in test_multiple_concurrent_repairs
self.assertTrue(len(stderr) == 0, stderr)
File "/usr/lib/python2.7/unittest/case.py", line 422, in assertTrue
raise self.failureException(msg)
"ERROR 15:43:07 Error creating pool to
/127.0.0.3:9042\ncom.datastax.driver.core.TransportException: [/127.0.0.3:9042]
Cannot connect\n\tat
com.datastax.driver.core.Connection$1.operationComplete(Connection.java:156)
[cassandra-driver-core-2.2.0-rc2-SNAPSHOT-20150617-shaded.jar:na]\n\tat
com.datastax.driver.core.Connection$1.operationComplete(Connection.java:139)
[cassandra-driver-core-2.2.0-rc2-SNAPSHOT-20150617-shaded.jar:na]\n\tat
com.datastax.shaded.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:680)
[cassandra-driver-core-2.2.0-rc2-SNAPSHOT-20150617-shaded.jar:na]\n\tat
com.datastax.shaded.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:603)
[cassandra-driver-core-2.2.0-rc2-SNAPSHOT-20150617-shaded.jar:na]\n\tat
com.datastax.shaded.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:563)
[cassandra-driver-core-2.2.0-rc2-SNAPSHOT-20150617-shaded.jar:na]\n\tat
com.datastax.shaded.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:424)
[cassandra-driver-core-2.2.0-rc2-SNAPSHOT-20150617-shaded.jar:na]\n\tat
com.datastax.shaded.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.fulfillConnectPromise(AbstractNioChannel.java:268)
[cassandra-driver-core-2.2.0-rc2-SNAPSHOT-20150617-shaded.jar:na]\n\tat
com.datastax.shaded.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:284)
[cassandra-driver-core-2.2.0-rc2-SNAPSHOT-20150617-shaded.jar:na]\n\tat
com.datastax.shaded.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:528)
[cassandra-driver-core-2.2.0-rc2-SNAPSHOT-20150617-shaded.jar:na]\n\tat
com.datastax.shaded.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468)
[cassandra-driver-core-2.2.0-rc2-SNAPSHOT-20150617-shaded.jar:na]\n\tat
com.datastax.shaded.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382)
[cassandra-driver-core-2.2.0-rc2-SNAPSHOT-20150617-shaded.jar:na]\n\tat
com.datastax.shaded.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354)
[cassandra-driver-core-2.2.0-rc2-SNAPSHOT-20150617-shaded.jar:na]\n\tat
com.datastax.shaded.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:111)
[cassandra-driver-core-2.2.0-rc2-SNAPSHOT-20150617-shaded.jar:na]\n\tat
java.lang.Thread.run(Thread.java:745) [na:1.7.0_80]\nCaused by:
java.net.ConnectException: Connection refused: /127.0.0.3:9042\n\tat
sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) ~[na:1.7.0_80]\n\tat
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:744)
~[na:1.7.0_80]\n\tat
com.datastax.shaded.netty.channel.socket.nio.NioSocketChannel.doFinishConnect(NioSocketChannel.java:224)
~[cassandra-driver-core-2.2.0-rc2-SNAPSHOT-20150617-shaded.jar:na]\n\tat
com.datastax.shaded.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:281)
[cassandra-driver-core-2.2.0-rc2-SNAPSHOT-20150617-shaded.jar:na]\n\t... 6
common frames omitted\nERROR 15:43:07 Error creating pool to
/127.0.0.1:9042\ncom.datastax.driver.core.TransportException: [/127.0.0.1:9042]
Cannot connect\n\tat
com.datastax.driver.core.Connection$1.operationComplete(Connection.java:156)
[cassandra-driver-core-2.2.0-rc2-SNAPSHOT-20150617-shaded.jar:na]\n\tat
com.datastax.driver.core.Connection$1.operationComplete(Connection.java:139)
[cassandra-driver-core-2.2.0-rc2-SNAPSHOT-20150617-shaded.jar:na]\n\tat
com.datastax.shaded.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:680)
[cassandra-driver-core-2.2.0-rc2-SNAPSHOT-20150617-shaded.jar:na]\n\tat
com.datastax.shaded.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:603)
[cassandra-driver-core-2.2.0-rc2-SNAPSHOT-20150617-shaded.jar:na]\n\tat
com.datastax.shaded.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:563)
[cassandra-driver-core-2.2.0-rc2-SNAPSHOT-20150617-shaded.jar:na]\n\tat
com.datastax.shaded.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:424)
[cassandra-driver-core-2.2.0-rc2-SNAPSHOT-20150617-shaded.jar:na]\n\tat
com.datastax.shaded.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.fulfillConnectPromise(AbstractNioChannel.java:268)
[cassandra-driver-core-2.2.0-rc2-SNAPSHOT-20150617-shaded.jar:na]\n\tat
com.datastax.shaded.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:284)
[cassandra-driver-core-2.2.0-rc2-SNAPSHOT-20150617-shaded.jar:na]\n\tat
com.datastax.shaded.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:528)
[cassandra-driver-core-2.2.0-rc2-SNAPSHOT-20150617-shaded.jar:na]\n\tat
com.datastax.shaded.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468)
[cassandra-driver-core-2.2.0-rc2-SNAPSHOT-20150617-shaded.jar:na]\n\tat
com.datastax.shaded.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382)
[cassandra-driver-core-2.2.0-rc2-SNAPSHOT-20150617-shaded.jar:na]\n\tat
com.datastax.shaded.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354)
[cassandra-driver-core-2.2.0-rc2-SNAPSHOT-20150617-shaded.jar:na]\n\tat
com.datastax.shaded.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:111)
[cassandra-driver-core-2.2.0-rc2-SNAPSHOT-20150617-shaded.jar:na]\n\tat
java.lang.Thread.run(Thread.java:745) [na:1.7.0_80]\nCaused by:
java.net.ConnectException: Connection refused: /127.0.0.1:9042\n\tat
sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) ~[na:1.7.0_80]\n\tat
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:744)
~[na:1.7.0_80]\n\tat
com.datastax.shaded.netty.channel.socket.nio.NioSocketChannel.doFinishConnect(NioSocketChannel.java:224)
~[cassandra-driver-core-2.2.0-rc2-SNAPSHOT-20150617-shaded.jar:na]\n\tat
com.datastax.shaded.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:281)
[cassandra-driver-core-2.2.0-rc2-SNAPSHOT-20150617-shaded.jar:na]\n\t... 6
common frames omitted\nFailed to connect over JMX; not collecting these
stats\n\n-------------------- >> begin captured logging <<
--------------------\ndtest: DEBUG: cluster ccm directory:
/tmp/dtest-b0pWB9\ndtest: DEBUG: Done setting configuration options:\n{
'num_tokens': None,\n 'phi_convict_threshold': 5,\n
'range_request_timeout_in_ms': 10000,\n 'read_request_timeout_in_ms':
10000,\n 'request_timeout_in_ms': 10000,\n
'truncate_request_timeout_in_ms': 10000,\n 'write_request_timeout_in_ms':
10000}\n--------------------- >> end captured logging << ---------------------"
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
