[
https://issues.apache.org/jira/browse/CASSANDRA-12886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bing Wu updated CASSANDRA-12886:
--------------------------------
Description:
While running "nodetool repair", I see many instances of
"javax.net.ssl.SSLException: java.net.SocketException: Connection reset" in
system.logs on some nodes in the cluster. Timestamp corresponds to streaming
source/initiator's error messages of "sync failed between ..."
Setup:
- Cassandra 3.7.01
- CentOS 6.7 in AWS (multi-region)
- JDK version: {noformat}
java version "1.8.0_102"
Java(TM) SE Runtime Environment (build 1.8.0_102-b14)
Java HotSpot(TM) 64-Bit Server VM (build 25.102-b14, mixed mode)
{noformat}
- cassandra.yaml:
{noformat}
server_encryption_options:
internode_encryption: all
keystore: [path]
keystore_password: [password]
truststore: [path]
truststore_password: [password]
# More advanced defaults below:
# protocol: TLS
# algorithm: SunX509
# store_type: JKS
# cipher_suites:
[TLS_RSA_WITH_AES_128_CBC_SHA,TLS_RSA_WITH_AES_256_CBC_SHA,TLS_DHE_RSA_WITH_AES_128_CBC_SHA,TLS_DHE_RSA_WITH_AES_256_CBC_SHA,TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA,TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA]
require_client_auth: false
{noformat}
Error messages in system.log on the target host:
{noformat}
ERROR [STREAM-OUT-/54.247.111.232:7001] 2016-11-07 07:30:56,475
StreamSession.java:529 - [Stream #e14abcb0-a4bb-11e6-9758-55b9ac38b78e]
Streaming error occurred on session with peer 54.247.111.232
javax.net.ssl.SSLException: Connection has been shutdown:
javax.net.ssl.SSLException: java.net.SocketException: Connection reset
at sun.security.ssl.SSLSocketImpl.checkEOF(SSLSocketImpl.java:1541)
~[na:1.8.0_102]
at sun.security.ssl.SSLSocketImpl.checkWrite(SSLSocketImpl.java:1553)
~[na:1.8.0_102]
at sun.security.ssl.AppOutputStream.write(AppOutputStream.java:71)
~[na:1.8.0_102]
at
java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
~[na:1.8.0_102]
at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
~[na:1.8.0_102]
at
org.apache.cassandra.io.util.WrappedDataOutputStreamPlus.flush(WrappedDataOutputStreamPlus.java:66)
~[apache-cassandra-3.7.0.jar:3.7.0]
at
org.apache.cassandra.streaming.ConnectionHandler$OutgoingMessageHandler.sendMessage(ConnectionHandler.java:371)
[apache-cassandra-3.7.0.jar:3.7.0]
at
org.apache.cassandra.streaming.ConnectionHandler$OutgoingMessageHandler.run(ConnectionHandler.java:342)
[apache-cassandra-3.7.0.jar:3.7.0]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_102]
Caused by: javax.net.ssl.SSLException: java.net.SocketException: Connection
reset
{noformat}
was:
While running "nodetool repair", I see many instances of
"javax.net.ssl.SSLException: java.net.SocketException: Connection reset" in
system.logs on some nodes in the cluster. Timestamp corresponds to streaming
source/initiator's error messages of "sync failed between ..."
Setup:
- Cassandra 3.7.01
- CentOS 6.7 in AWS (multi-region)
- JDK version: {noformat}
java version "1.8.0_102"
Java(TM) SE Runtime Environment (build 1.8.0_102-b14)
Java HotSpot(TM) 64-Bit Server VM (build 25.102-b14, mixed mode)
{noformat}
cassandra.yaml:
{noformat}
server_encryption_options:
internode_encryption: all
keystore: [path]
keystore_password: [password]
truststore: [path]
truststore_password: [password]
# More advanced defaults below:
# protocol: TLS
# algorithm: SunX509
# store_type: JKS
# cipher_suites:
[TLS_RSA_WITH_AES_128_CBC_SHA,TLS_RSA_WITH_AES_256_CBC_SHA,TLS_DHE_RSA_WITH_AES_128_CBC_SHA,TLS_DHE_RSA_WITH_AES_256_CBC_SHA,TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA,TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA]
require_client_auth: false
{noformat}
Error messages in system.log on the target host:
{noformat}
ERROR [STREAM-OUT-/54.247.111.232:7001] 2016-11-07 07:30:56,475
StreamSession.java:529 - [Stream #e14abcb0-a4bb-11e6-9758-55b9ac38b78e]
Streaming error occurred on session with peer 54.247.111.232
javax.net.ssl.SSLException: Connection has been shutdown:
javax.net.ssl.SSLException: java.net.SocketException: Connection reset
at sun.security.ssl.SSLSocketImpl.checkEOF(SSLSocketImpl.java:1541)
~[na:1.8.0_102]
at sun.security.ssl.SSLSocketImpl.checkWrite(SSLSocketImpl.java:1553)
~[na:1.8.0_102]
at sun.security.ssl.AppOutputStream.write(AppOutputStream.java:71)
~[na:1.8.0_102]
at
java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
~[na:1.8.0_102]
at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
~[na:1.8.0_102]
at
org.apache.cassandra.io.util.WrappedDataOutputStreamPlus.flush(WrappedDataOutputStreamPlus.java:66)
~[apache-cassandra-3.7.0.jar:3.7.0]
at
org.apache.cassandra.streaming.ConnectionHandler$OutgoingMessageHandler.sendMessage(ConnectionHandler.java:371)
[apache-cassandra-3.7.0.jar:3.7.0]
at
org.apache.cassandra.streaming.ConnectionHandler$OutgoingMessageHandler.run(ConnectionHandler.java:342)
[apache-cassandra-3.7.0.jar:3.7.0]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_102]
Caused by: javax.net.ssl.SSLException: java.net.SocketException: Connection
reset
{noformat}
> Streaming failed due to SSL Socket connection reset
> ---------------------------------------------------
>
> Key: CASSANDRA-12886
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12886
> Project: Cassandra
> Issue Type: Bug
> Reporter: Bing Wu
>
> While running "nodetool repair", I see many instances of
> "javax.net.ssl.SSLException: java.net.SocketException: Connection reset" in
> system.logs on some nodes in the cluster. Timestamp corresponds to streaming
> source/initiator's error messages of "sync failed between ..."
> Setup:
> - Cassandra 3.7.01
> - CentOS 6.7 in AWS (multi-region)
> - JDK version: {noformat}
> java version "1.8.0_102"
> Java(TM) SE Runtime Environment (build 1.8.0_102-b14)
> Java HotSpot(TM) 64-Bit Server VM (build 25.102-b14, mixed mode)
> {noformat}
> - cassandra.yaml:
> {noformat}
> server_encryption_options:
> internode_encryption: all
> keystore: [path]
> keystore_password: [password]
> truststore: [path]
> truststore_password: [password]
> # More advanced defaults below:
> # protocol: TLS
> # algorithm: SunX509
> # store_type: JKS
> # cipher_suites:
> [TLS_RSA_WITH_AES_128_CBC_SHA,TLS_RSA_WITH_AES_256_CBC_SHA,TLS_DHE_RSA_WITH_AES_128_CBC_SHA,TLS_DHE_RSA_WITH_AES_256_CBC_SHA,TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA,TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA]
> require_client_auth: false
> {noformat}
> Error messages in system.log on the target host:
> {noformat}
> ERROR [STREAM-OUT-/54.247.111.232:7001] 2016-11-07 07:30:56,475
> StreamSession.java:529 - [Stream #e14abcb0-a4bb-11e6-9758-55b9ac38b78e]
> Streaming error occurred on session with peer 54.247.111.232
> javax.net.ssl.SSLException: Connection has been shutdown:
> javax.net.ssl.SSLException: java.net.SocketException: Connection reset
> at sun.security.ssl.SSLSocketImpl.checkEOF(SSLSocketImpl.java:1541)
> ~[na:1.8.0_102]
> at sun.security.ssl.SSLSocketImpl.checkWrite(SSLSocketImpl.java:1553)
> ~[na:1.8.0_102]
> at sun.security.ssl.AppOutputStream.write(AppOutputStream.java:71)
> ~[na:1.8.0_102]
> at
> java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
> ~[na:1.8.0_102]
> at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
> ~[na:1.8.0_102]
> at
> org.apache.cassandra.io.util.WrappedDataOutputStreamPlus.flush(WrappedDataOutputStreamPlus.java:66)
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at
> org.apache.cassandra.streaming.ConnectionHandler$OutgoingMessageHandler.sendMessage(ConnectionHandler.java:371)
> [apache-cassandra-3.7.0.jar:3.7.0]
> at
> org.apache.cassandra.streaming.ConnectionHandler$OutgoingMessageHandler.run(ConnectionHandler.java:342)
> [apache-cassandra-3.7.0.jar:3.7.0]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_102]
> Caused by: javax.net.ssl.SSLException: java.net.SocketException: Connection
> reset
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
