[jira] [Commented] (CASSANDRA-14437) SSTableLoader does not work when "internode_encryption : all" is set
[
https://issues.apache.org/jira/browse/CASSANDRA-14437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16476371#comment-16476371
]
Paul Cheon commented on CASSANDRA-14437:
I tried sstableloader from a node which is not in the cluster, but it could not
connect to any cluster node
{noformat}
pcheon@tor-paul-cas002:~/test$ sstableloader -v -d 10.1.10.203
office_audit/log/ -f /etc/cassandra/cassandra.yaml -u pcheon -pw `cat .secret`
WARN 19:27:44,533 Small commitlog volume detected at
/var/lib/cassandra/commitlog; setting commitlog_total_space_in_mb to 2316. You
can override this in cassandra.yaml
WARN 19:27:44,542 Small cdc volume detected at /var/lib/cassandra/cdc_raw;
setting cdc_total_space_in_mb to 1158. You can override this in cassandra.yaml
WARN 19:27:44,652 Only 7.232GiB free across all data volumes. Consider adding
more capacity to your cluster or removing obsolete snapshots
WARN 19:27:46,784 Error while computing token map for keyspace
vdc_ca_cdval_prdrel with datacenter tor: could not achieve replication factor 3
(found 1 replicas only), check your keyspace replication settings.
Established connection to initial hosts
Opening sstables and calculating sections to stream
Streaming relevant part of
/home/pcheon/test/office_audit/log/mc-1083-big-Data.db
/home/pcheon/test/office_audit/log/mc-1100-big-Data.db
/home/pcheon/test/office_audit/log/mc-1101-big-Data.db
/home/pcheon/test/office_audit/log/mc-257-big-Data.db
/home/pcheon/test/office_audit/log/mc-984-big-Data.db to [/10.1.10.201,
/10.1.10.203, /10.1.10.202]
ERROR 19:27:51,281 [Stream #0e5a8a40-5876-11e8-90c3-61730bfdf719] Streaming
error occurred on session with peer 10.1.10.203
java.net.SocketException: Invalid argument or cannot assign requested address
at java.net.PlainSocketImpl.socketConnect(Native Method) ~[na:1.8.0_112]
at
java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
~[na:1.8.0_112]
at
java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
~[na:1.8.0_112]
at
java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
~[na:1.8.0_112]
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
~[na:1.8.0_112]
at java.net.Socket.connect(Socket.java:589) ~[na:1.8.0_112]
at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:668)
~[na:1.8.0_112]
at sun.security.ssl.SSLSocketImpl.(SSLSocketImpl.java:495)
~[na:1.8.0_112]
at
sun.security.ssl.SSLSocketFactoryImpl.createSocket(SSLSocketFactoryImpl.java:169)
~[na:1.8.0_112]
at
org.apache.cassandra.security.SSLFactory.getSocket(SSLFactory.java:81)
~[apache-cassandra-3.11.2.jar:3.11.2]
at
org.apache.cassandra.tools.BulkLoadConnectionFactory.createConnection(BulkLoadConnectionFactory.java:56)
~[apache-cassandra-3.11.2.jar:3.11.2]
at
org.apache.cassandra.streaming.StreamSession.createConnection(StreamSession.java:282)
~[apache-cassandra-3.11.2.jar:3.11.2]
at
org.apache.cassandra.streaming.ConnectionHandler.initiate(ConnectionHandler.java:86)
~[apache-cassandra-3.11.2.jar:3.11.2]
at
org.apache.cassandra.streaming.StreamSession.start(StreamSession.java:269)
~[apache-cassandra-3.11.2.jar:3.11.2]
at
org.apache.cassandra.streaming.StreamCoordinator$StreamSessionConnector.run(StreamCoordinator.java:263)
[apache-cassandra-3.11.2.jar:3.11.2]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[na:1.8.0_112]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[na:1.8.0_112]
at
org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:81)
[apache-cassandra-3.11.2.jar:3.11.2]
at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_112]
ERROR 19:27:51,286 [Stream #0e5a8a40-5876-11e8-90c3-61730bfdf719] Streaming
error occurred on session with peer 10.1.10.202
java.net.SocketException: Invalid argument or cannot assign requested address
at java.net.PlainSocketImpl.socketConnect(Native Method) ~[na:1.8.0_112]
at
java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
~[na:1.8.0_112]
at
java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
~[na:1.8.0_112]
at
java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
~[na:1.8.0_112]
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
~[na:1.8.0_112]
at java.net.Socket.connect(Socket.java:589) ~[na:1.8.0_112]
at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:668)
~[na:1.8.0_112]
at sun.security.ssl.SSLSocketImpl.(SSLSocketImpl.java:495)
~[na:1.8.0_112]
at
sun.security.ssl.SSLSocketFactoryImpl.createSocket(SSLSocketFactoryImpl.java:169)
[jira] [Comment Edited] (CASSANDRA-14437) SSTableLoader does not work when "internode_encryption : all" is set
[
https://issues.apache.org/jira/browse/CASSANDRA-14437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16476156#comment-16476156
]
Paul Cheon edited comment on CASSANDRA-14437 at 5/15/18 4:52 PM:
-
The content of yaml file is exact same except the node keystore file
{code}
keystore: "/etc/ssl/visier/10.1.119.203.jks"
{code}
The keystore password is same though. Just the certificate inside of the
keystore is generated with the IP address of each node
Trust keystore is same file in every node
was (Author: paul.ch...@visiercorp.com):
The content of yaml file is exact same except the node keystore file
{code}
keystore: "/etc/ssl/visier/10.1.119.203.jks"
{code}
The keystore password is same though. Just the certificate inside of the
keystore is generated with the IP address of each node
> SSTableLoader does not work when "internode_encryption : all" is set
>
>
> Key: CASSANDRA-14437
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14437
> Project: Cassandra
> Issue Type: Bug
> Components: Tools
>Reporter: Paul Cheon
>Priority: Major
> Fix For: 3.11.2
>
>
> I am trying to use sstableloader to restore snapshot.
> If "internode_encryption : all" is set, then it does not work and complain
> with below error messages. I initiated sstableloader from 10.1.10.203
> (yvr-paul-cas003), so 10.1.10.203 worked fine, but the the other two nodes
> (10.1.10.201 & 10.1.10.202 failed)
> {noformat}
> pcheon@yvr-paul-cas003:~/t$ sstableloader -v -d 10.1.10.203 office_audit/log/
> -f /etc/cassandra/cassandra.yaml -u pcheon -pw `cat .secret`
> WARN 17:23:45,166 Small commitlog volume detected at
> /var/lib/cassandra/commitlog; setting commitlog_total_space_in_mb to 2316.
> You can override this in cassandra.yaml
> WARN 17:23:45,170 Small cdc volume detected at /var/lib/cassandra/cdc_raw;
> setting cdc_total_space_in_mb to 1158. You can override this in
> cassandra.yaml
> WARN 17:23:45,285 Only 5.318GiB free across all data volumes. Consider
> adding more capacity to your cluster or removing obsolete snapshots
> Established connection to initial hosts
> Opening sstables and calculating sections to stream
> Streaming relevant part of
> /home/pcheon/t/office_audit/log/mc-1083-big-Data.db
> /home/pcheon/t/office_audit/log/mc-1100-big-Data.db
> /home/pcheon/t/office_audit/log/mc-1101-big-Data.db
> /home/pcheon/t/office_audit/log/mc-257-big-Data.db
> /home/pcheon/t/office_audit/log/mc-984-big-Data.db to [/10.1.10.201,
> /10.1.10.203, /10.1.10.202]
> ERROR 17:23:49,460 [Stream #938baee0-4e2d-11e8-9be0-ebc69ba4b87f] Streaming
> error occurred on session with peer 10.1.10.201
> java.net.SocketException: Invalid argument or cannot assign requested address
> at java.net.PlainSocketImpl.socketConnect(Native Method) ~[na:1.8.0_112]
> at
> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
> ~[na:1.8.0_112]
> at
> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
> ~[na:1.8.0_112]
> at
> java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
> ~[na:1.8.0_112]
> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
> ~[na:1.8.0_112]
> at java.net.Socket.connect(Socket.java:589) ~[na:1.8.0_112]
> at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:668)
> ~[na:1.8.0_112]
> at sun.security.ssl.SSLSocketImpl.(SSLSocketImpl.java:495)
> ~[na:1.8.0_112]
> at
> sun.security.ssl.SSLSocketFactoryImpl.createSocket(SSLSocketFactoryImpl.java:169)
> ~[na:1.8.0_112]
> at
> org.apache.cassandra.security.SSLFactory.getSocket(SSLFactory.java:81)
> ~[apache-cassandra-3.11.2.jar:3.11.2]
> at
> org.apache.cassandra.tools.BulkLoadConnectionFactory.createConnection(BulkLoadConnectionFactory.java:56)
> ~[apache-cassandra-3.11.2.jar:3.11.2]
> at
> org.apache.cassandra.streaming.StreamSession.createConnection(StreamSession.java:282)
> ~[apache-cassandra-3.11.2.jar:3.11.2]
> at
> org.apache.cassandra.streaming.ConnectionHandler.initiate(ConnectionHandler.java:86)
> ~[apache-cassandra-3.11.2.jar:3.11.2]
> at
> org.apache.cassandra.streaming.StreamSession.start(StreamSession.java:269)
> ~[apache-cassandra-3.11.2.jar:3.11.2]
> at
> org.apache.cassandra.streaming.StreamCoordinator$StreamSessionConnector.run(StreamCoordinator.java:263)
> [apache-cassandra-3.11.2.jar:3.11.2]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [na:1.8.0_112]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [na:1.8.0_112]
> at
>
[jira] [Commented] (CASSANDRA-14437) SSTableLoader does not work when "internode_encryption : all" is set
[
https://issues.apache.org/jira/browse/CASSANDRA-14437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16476156#comment-16476156
]
Paul Cheon commented on CASSANDRA-14437:
The content of yaml file is exact same except the node keystore file
{code}
keystore: "/etc/ssl/visier/10.1.119.203.jks"
{code}
The keystore password is same though. Just the certificate inside of the
keystore is generated with the IP address of each node
> SSTableLoader does not work when "internode_encryption : all" is set
>
>
> Key: CASSANDRA-14437
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14437
> Project: Cassandra
> Issue Type: Bug
> Components: Tools
>Reporter: Paul Cheon
>Priority: Major
> Fix For: 3.11.2
>
>
> I am trying to use sstableloader to restore snapshot.
> If "internode_encryption : all" is set, then it does not work and complain
> with below error messages. I initiated sstableloader from 10.1.10.203
> (yvr-paul-cas003), so 10.1.10.203 worked fine, but the the other two nodes
> (10.1.10.201 & 10.1.10.202 failed)
> {noformat}
> pcheon@yvr-paul-cas003:~/t$ sstableloader -v -d 10.1.10.203 office_audit/log/
> -f /etc/cassandra/cassandra.yaml -u pcheon -pw `cat .secret`
> WARN 17:23:45,166 Small commitlog volume detected at
> /var/lib/cassandra/commitlog; setting commitlog_total_space_in_mb to 2316.
> You can override this in cassandra.yaml
> WARN 17:23:45,170 Small cdc volume detected at /var/lib/cassandra/cdc_raw;
> setting cdc_total_space_in_mb to 1158. You can override this in
> cassandra.yaml
> WARN 17:23:45,285 Only 5.318GiB free across all data volumes. Consider
> adding more capacity to your cluster or removing obsolete snapshots
> Established connection to initial hosts
> Opening sstables and calculating sections to stream
> Streaming relevant part of
> /home/pcheon/t/office_audit/log/mc-1083-big-Data.db
> /home/pcheon/t/office_audit/log/mc-1100-big-Data.db
> /home/pcheon/t/office_audit/log/mc-1101-big-Data.db
> /home/pcheon/t/office_audit/log/mc-257-big-Data.db
> /home/pcheon/t/office_audit/log/mc-984-big-Data.db to [/10.1.10.201,
> /10.1.10.203, /10.1.10.202]
> ERROR 17:23:49,460 [Stream #938baee0-4e2d-11e8-9be0-ebc69ba4b87f] Streaming
> error occurred on session with peer 10.1.10.201
> java.net.SocketException: Invalid argument or cannot assign requested address
> at java.net.PlainSocketImpl.socketConnect(Native Method) ~[na:1.8.0_112]
> at
> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
> ~[na:1.8.0_112]
> at
> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
> ~[na:1.8.0_112]
> at
> java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
> ~[na:1.8.0_112]
> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
> ~[na:1.8.0_112]
> at java.net.Socket.connect(Socket.java:589) ~[na:1.8.0_112]
> at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:668)
> ~[na:1.8.0_112]
> at sun.security.ssl.SSLSocketImpl.(SSLSocketImpl.java:495)
> ~[na:1.8.0_112]
> at
> sun.security.ssl.SSLSocketFactoryImpl.createSocket(SSLSocketFactoryImpl.java:169)
> ~[na:1.8.0_112]
> at
> org.apache.cassandra.security.SSLFactory.getSocket(SSLFactory.java:81)
> ~[apache-cassandra-3.11.2.jar:3.11.2]
> at
> org.apache.cassandra.tools.BulkLoadConnectionFactory.createConnection(BulkLoadConnectionFactory.java:56)
> ~[apache-cassandra-3.11.2.jar:3.11.2]
> at
> org.apache.cassandra.streaming.StreamSession.createConnection(StreamSession.java:282)
> ~[apache-cassandra-3.11.2.jar:3.11.2]
> at
> org.apache.cassandra.streaming.ConnectionHandler.initiate(ConnectionHandler.java:86)
> ~[apache-cassandra-3.11.2.jar:3.11.2]
> at
> org.apache.cassandra.streaming.StreamSession.start(StreamSession.java:269)
> ~[apache-cassandra-3.11.2.jar:3.11.2]
> at
> org.apache.cassandra.streaming.StreamCoordinator$StreamSessionConnector.run(StreamCoordinator.java:263)
> [apache-cassandra-3.11.2.jar:3.11.2]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [na:1.8.0_112]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [na:1.8.0_112]
> at
> org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:81)
> [apache-cassandra-3.11.2.jar:3.11.2]
> at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_112]
> ERROR 17:23:49,458 [Stream #938baee0-4e2d-11e8-9be0-ebc69ba4b87f] Streaming
> error occurred on session with peer 10.1.10.202
> java.net.SocketException: Invalid argument or cannot assign requested address
> at
[jira] [Commented] (CASSANDRA-14437) SSTableLoader does not work when "internode_encryption : all" is set
[
https://issues.apache.org/jira/browse/CASSANDRA-14437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16474613#comment-16474613
]
Paul Cheon commented on CASSANDRA-14437:
If you look at the description in the above, I already defined the yaml file to
start sstableloader
{code}
sstableloader -v -d 10.1.10.203 office_audit/log/ -f
/etc/cassandra/cassandra.yaml -u pcheon -pw `cat .secret`
{code}
Am I doing something wrong?
> SSTableLoader does not work when "internode_encryption : all" is set
>
>
> Key: CASSANDRA-14437
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14437
> Project: Cassandra
> Issue Type: Bug
> Components: Tools
>Reporter: Paul Cheon
>Priority: Major
> Fix For: 3.11.2
>
>
> I am trying to use sstableloader to restore snapshot.
> If "internode_encryption : all" is set, then it does not work and complain
> with below error messages. I initiated sstableloader from 10.1.10.203
> (yvr-paul-cas003), so 10.1.10.203 worked fine, but the the other two nodes
> (10.1.10.201 & 10.1.10.202 failed)
> {noformat}
> pcheon@yvr-paul-cas003:~/t$ sstableloader -v -d 10.1.10.203 office_audit/log/
> -f /etc/cassandra/cassandra.yaml -u pcheon -pw `cat .secret`
> WARN 17:23:45,166 Small commitlog volume detected at
> /var/lib/cassandra/commitlog; setting commitlog_total_space_in_mb to 2316.
> You can override this in cassandra.yaml
> WARN 17:23:45,170 Small cdc volume detected at /var/lib/cassandra/cdc_raw;
> setting cdc_total_space_in_mb to 1158. You can override this in
> cassandra.yaml
> WARN 17:23:45,285 Only 5.318GiB free across all data volumes. Consider
> adding more capacity to your cluster or removing obsolete snapshots
> Established connection to initial hosts
> Opening sstables and calculating sections to stream
> Streaming relevant part of
> /home/pcheon/t/office_audit/log/mc-1083-big-Data.db
> /home/pcheon/t/office_audit/log/mc-1100-big-Data.db
> /home/pcheon/t/office_audit/log/mc-1101-big-Data.db
> /home/pcheon/t/office_audit/log/mc-257-big-Data.db
> /home/pcheon/t/office_audit/log/mc-984-big-Data.db to [/10.1.10.201,
> /10.1.10.203, /10.1.10.202]
> ERROR 17:23:49,460 [Stream #938baee0-4e2d-11e8-9be0-ebc69ba4b87f] Streaming
> error occurred on session with peer 10.1.10.201
> java.net.SocketException: Invalid argument or cannot assign requested address
> at java.net.PlainSocketImpl.socketConnect(Native Method) ~[na:1.8.0_112]
> at
> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
> ~[na:1.8.0_112]
> at
> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
> ~[na:1.8.0_112]
> at
> java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
> ~[na:1.8.0_112]
> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
> ~[na:1.8.0_112]
> at java.net.Socket.connect(Socket.java:589) ~[na:1.8.0_112]
> at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:668)
> ~[na:1.8.0_112]
> at sun.security.ssl.SSLSocketImpl.(SSLSocketImpl.java:495)
> ~[na:1.8.0_112]
> at
> sun.security.ssl.SSLSocketFactoryImpl.createSocket(SSLSocketFactoryImpl.java:169)
> ~[na:1.8.0_112]
> at
> org.apache.cassandra.security.SSLFactory.getSocket(SSLFactory.java:81)
> ~[apache-cassandra-3.11.2.jar:3.11.2]
> at
> org.apache.cassandra.tools.BulkLoadConnectionFactory.createConnection(BulkLoadConnectionFactory.java:56)
> ~[apache-cassandra-3.11.2.jar:3.11.2]
> at
> org.apache.cassandra.streaming.StreamSession.createConnection(StreamSession.java:282)
> ~[apache-cassandra-3.11.2.jar:3.11.2]
> at
> org.apache.cassandra.streaming.ConnectionHandler.initiate(ConnectionHandler.java:86)
> ~[apache-cassandra-3.11.2.jar:3.11.2]
> at
> org.apache.cassandra.streaming.StreamSession.start(StreamSession.java:269)
> ~[apache-cassandra-3.11.2.jar:3.11.2]
> at
> org.apache.cassandra.streaming.StreamCoordinator$StreamSessionConnector.run(StreamCoordinator.java:263)
> [apache-cassandra-3.11.2.jar:3.11.2]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [na:1.8.0_112]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [na:1.8.0_112]
> at
> org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:81)
> [apache-cassandra-3.11.2.jar:3.11.2]
> at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_112]
> ERROR 17:23:49,458 [Stream #938baee0-4e2d-11e8-9be0-ebc69ba4b87f] Streaming
> error occurred on session with peer 10.1.10.202
> java.net.SocketException: Invalid argument or cannot assign requested address
> at
[jira] [Commented] (CASSANDRA-14437) SSTableLoader does not work when "internode_encryption : all" is set
[
https://issues.apache.org/jira/browse/CASSANDRA-14437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16467717#comment-16467717
]
Paul Cheon commented on CASSANDRA-14437:
When I display the sstableloader help page, there is no option for
internode_encryption for sstableloader.
{noformat}
pcheon@yvr-paul-cas003:~$ sstableloader --help
usage: sstableloader [options]
Bulk load the sstables found in the directory to the configured
cluster.The parent directories of are used as the target
keyspace/table name. So for instance, to load an sstable named
Standard1-g-1-Data.db into Keyspace1/Standard1, you will need to have the
files Standard1-g-1-Data.db and Standard1-g-1-Index.db into a directory
/path/to/Keyspace1/Standard1/.
-alg,--ssl-alg Client SSL: algorithm
(default: SunX509)
-ap,--auth-provider custom AuthProvider
class name for cassandra
authentication
-ciphers,--ssl-ciphers Client SSL:
comma-separated list of
encryption suites to use
-cph,--connections-per-host number of concurrent
connections-per-host.
-d,--nodes Required. try to
connect to these hosts
(comma separated) initially for ring information
-f,--conf-path cassandra.yaml file
path for streaming
throughput and client/server SSL.
-h,--help display this help
message
-i,--ignore don't stream to this
(comma separated) list of
nodes
-idct,--inter-dc-throttleinter-datacenter
throttle speed in Mbits
(default unlimited)
-ks,--keystoreClient SSL: full path
to keystore
-kspw,--keystore-passwordClient SSL: password
of the keystore
--no-progressdon't display
progress
-p,--portport used for native
connection (default 9042)
-prtcl,--ssl-protocol Client SSL:
connections protocol to
use (default: TLS)
-pw,--passwordpassword for
cassandra authentication
-sp,--storage-portport used for
internode communication
(default 7000)
-ssp,--ssl-storage-port port used for TLS
internode communication
(default 7001)
-st,--store-typeClient SSL: type of
store
-t,--throttle throttle speed in
Mbits (default unlimited)
-ts,--truststoreClient SSL: full path
to truststore
-tspw,--truststore-passwordClient SSL: password
of the truststore
-u,--username username for
cassandra authentication
-v,--verboseverbose output
You can provide cassandra.yaml file with -f command line option to set up
streaming throughput, client and server encryption options. Only
stream_throughput_outbound_megabits_per_sec, server_encryption_options and
client_encryption_options are read from yaml. You can override options
read from cassandra.yaml with corresponding command line options.
pcheon@yvr-paul-cas003:~$
{noformat}
How can I enforce sstableloader to use internode_encryption settings on command
line?
> SSTableLoader does not work when "internode_encryption : all" is set
>
>
> Key: CASSANDRA-14437
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14437
> Project: Cassandra
> Issue Type: Bug
> Components: Tools
>Reporter: Paul Cheon
>Priority: Major
> Fix For: 3.11.2
>
>
> I am trying to use sstableloader to restore snapshot.
> If "internode_encryption : all" is set, then it does
[jira] [Comment Edited] (CASSANDRA-14437) SSTableLoader does not work when "internode_encryption : all" is set
[
https://issues.apache.org/jira/browse/CASSANDRA-14437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16461372#comment-16461372
]
Paul Cheon edited comment on CASSANDRA-14437 at 5/2/18 5:42 PM:
Here is the cassandra.yml file content I used with sstableloader
{code}
---
authenticator: PasswordAuthenticator
authorizer: CassandraAuthorizer
cluster_name: vdc_ca_cdval
commitlog_sync: periodic
commitlog_sync_period_in_ms: 1
endpoint_snitch: GossipingPropertyFileSnitch
listen_address: 10.1.10.203
rpc_address: 0.0.0.0
broadcast_rpc_address: 10.1.10.203
partitioner: org.apache.cassandra.dht.Murmur3Partitioner
seed_provider:
- class_name: org.apache.cassandra.locator.SimpleSeedProvider
parameters:
- seeds: 10.1.10.201
num_tokens: 64
start_native_transport: true
transparent_data_encryption_options:
enabled: true
chunk_length_kb: '64'
cipher: AES/CBC/PKCS5Padding
key_alias: atrestencryptionkey
key_provider:
- class_name: org.apache.cassandra.security.JKSKeyProvider
parameters:
- keystore: "/etc/ssl/visier/atrestencryptionkey.jceks"
keystore_password: somepassword
store_type: JCEKS
key_password: somepassword
server_encryption_options:
internode_encryption: all
keystore: "/etc/ssl/visier/10.1.119.203.jks"
keystore_password: somepassword
truststore: "/etc/ssl/visier/generic-server-truststore.jks"
truststore_password: somepassword
protocol:
- TLS
algorithm: SunX509
store_type: JKS
cipher_suites:
- TLS_DHE_RSA_WITH_AES_128_CBC_SHA
- TLS_DHE_RSA_WITH_AES_128_CBC_SHA256
- TLS_DHE_RSA_WITH_AES_128_GCM_SHA256
- TLS_DHE_RSA_WITH_AES_256_CBC_SHA
- TLS_DHE_RSA_WITH_AES_256_CBC_SHA256
- TLS_DHE_RSA_WITH_AES_256_GCM_SHA384
- TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA
- TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256
- TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
- TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA
- TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384
- TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
require_client_auth: true
client_encryption_options:
enabled: true
optional: false
require_client_auth: false
keystore: "/etc/ssl/visier/10.1.119.203.jks"
keystore_password: somepassword
truststore: "/etc/ssl/visier/generic-server-truststore.jks"
truststore_password: somepassword
protocol:
- TLS
cipher_suites:
- TLS_DHE_RSA_WITH_AES_128_CBC_SHA
- TLS_DHE_RSA_WITH_AES_128_CBC_SHA256
- TLS_DHE_RSA_WITH_AES_128_GCM_SHA256
- TLS_DHE_RSA_WITH_AES_256_CBC_SHA
- TLS_DHE_RSA_WITH_AES_256_CBC_SHA256
- TLS_DHE_RSA_WITH_AES_256_GCM_SHA384
- TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA
- TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256
- TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
- TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA
- TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384
- TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
batch_size_warn_threshold_in_kb: 10
slow_query_log_timeout_in_ms: 1000
commitlog_directory: "/var/lib/cassandra/commitlog"
data_file_directories:
- "/var/lib/cassandra/data"
hints_directory: "/var/lib/cassandra/hints"
saved_caches_directory: "/var/lib/cassandra/saved_caches"
{code}
was (Author: paul.ch...@visiercorp.com):
Here is the cassandra.yml file content I used with sstableloader
{code}
---
authenticator: PasswordAuthenticator
authorizer: CassandraAuthorizer
cluster_name: vdc_ca_cdval
commitlog_sync: periodic
commitlog_sync_period_in_ms: 1
endpoint_snitch: GossipingPropertyFileSnitch
listen_address: 10.1.10.203
rpc_address: 0.0.0.0
broadcast_rpc_address: 10.1.10.203
partitioner: org.apache.cassandra.dht.Murmur3Partitioner
seed_provider:
- class_name: org.apache.cassandra.locator.SimpleSeedProvider
parameters:
- seeds: 10.1.10.201,10.1.10.201
num_tokens: 64
start_native_transport: true
transparent_data_encryption_options:
enabled: true
chunk_length_kb: '64'
cipher: AES/CBC/PKCS5Padding
key_alias: atrestencryptionkey
key_provider:
- class_name: org.apache.cassandra.security.JKSKeyProvider
parameters:
- keystore: "/etc/ssl/visier/atrestencryptionkey.jceks"
keystore_password: somepassword
store_type: JCEKS
key_password: somepassword
server_encryption_options:
internode_encryption: all
keystore: "/etc/ssl/visier/10.1.119.203.jks"
keystore_password: somepassword
truststore: "/etc/ssl/visier/generic-server-truststore.jks"
truststore_password: somepassword
protocol:
- TLS
algorithm: SunX509
store_type: JKS
cipher_suites:
- TLS_DHE_RSA_WITH_AES_128_CBC_SHA
- TLS_DHE_RSA_WITH_AES_128_CBC_SHA256
- TLS_DHE_RSA_WITH_AES_128_GCM_SHA256
- TLS_DHE_RSA_WITH_AES_256_CBC_SHA
- TLS_DHE_RSA_WITH_AES_256_CBC_SHA256
- TLS_DHE_RSA_WITH_AES_256_GCM_SHA384
- TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA
- TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256
- TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
- TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA
- TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384
- TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
require_client_auth: true
[jira] [Commented] (CASSANDRA-14437) SSTableLoader does not work when "internode_encryption : all" is set
[
https://issues.apache.org/jira/browse/CASSANDRA-14437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16461372#comment-16461372
]
Paul Cheon commented on CASSANDRA-14437:
Here is the cassandra.yml file content I used with sstableloader
{code}
---
authenticator: PasswordAuthenticator
authorizer: CassandraAuthorizer
cluster_name: vdc_ca_cdval
commitlog_sync: periodic
commitlog_sync_period_in_ms: 1
endpoint_snitch: GossipingPropertyFileSnitch
listen_address: 10.1.10.203
rpc_address: 0.0.0.0
broadcast_rpc_address: 10.1.10.203
partitioner: org.apache.cassandra.dht.Murmur3Partitioner
seed_provider:
- class_name: org.apache.cassandra.locator.SimpleSeedProvider
parameters:
- seeds: 10.1.10.201,10.1.10.201
num_tokens: 64
start_native_transport: true
transparent_data_encryption_options:
enabled: true
chunk_length_kb: '64'
cipher: AES/CBC/PKCS5Padding
key_alias: atrestencryptionkey
key_provider:
- class_name: org.apache.cassandra.security.JKSKeyProvider
parameters:
- keystore: "/etc/ssl/visier/atrestencryptionkey.jceks"
keystore_password: somepassword
store_type: JCEKS
key_password: somepassword
server_encryption_options:
internode_encryption: all
keystore: "/etc/ssl/visier/10.1.119.203.jks"
keystore_password: somepassword
truststore: "/etc/ssl/visier/generic-server-truststore.jks"
truststore_password: somepassword
protocol:
- TLS
algorithm: SunX509
store_type: JKS
cipher_suites:
- TLS_DHE_RSA_WITH_AES_128_CBC_SHA
- TLS_DHE_RSA_WITH_AES_128_CBC_SHA256
- TLS_DHE_RSA_WITH_AES_128_GCM_SHA256
- TLS_DHE_RSA_WITH_AES_256_CBC_SHA
- TLS_DHE_RSA_WITH_AES_256_CBC_SHA256
- TLS_DHE_RSA_WITH_AES_256_GCM_SHA384
- TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA
- TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256
- TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
- TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA
- TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384
- TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
require_client_auth: true
client_encryption_options:
enabled: true
optional: false
require_client_auth: false
keystore: "/etc/ssl/visier/10.1.119.203.jks"
keystore_password: somepassword
truststore: "/etc/ssl/visier/generic-server-truststore.jks"
truststore_password: somepassword
protocol:
- TLS
cipher_suites:
- TLS_DHE_RSA_WITH_AES_128_CBC_SHA
- TLS_DHE_RSA_WITH_AES_128_CBC_SHA256
- TLS_DHE_RSA_WITH_AES_128_GCM_SHA256
- TLS_DHE_RSA_WITH_AES_256_CBC_SHA
- TLS_DHE_RSA_WITH_AES_256_CBC_SHA256
- TLS_DHE_RSA_WITH_AES_256_GCM_SHA384
- TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA
- TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256
- TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
- TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA
- TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384
- TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
batch_size_warn_threshold_in_kb: 10
slow_query_log_timeout_in_ms: 1000
commitlog_directory: "/var/lib/cassandra/commitlog"
data_file_directories:
- "/var/lib/cassandra/data"
hints_directory: "/var/lib/cassandra/hints"
saved_caches_directory: "/var/lib/cassandra/saved_caches"
{code}
> SSTableLoader does not work when "internode_encryption : all" is set
>
>
> Key: CASSANDRA-14437
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14437
> Project: Cassandra
> Issue Type: Bug
> Components: Tools
>Reporter: Paul Cheon
>Priority: Major
> Fix For: 3.11.2
>
>
> I am trying to use sstableloader to restore snapshot.
> If "internode_encryption : all" is set, then it does not work and complain
> with below error messages. I initiated sstableloader from 10.1.10.203
> (yvr-paul-cas003), so 10.1.10.203 worked fine, but the the other two nodes
> (10.1.10.201 & 10.1.10.202 failed)
> {noformat}
> pcheon@yvr-paul-cas003:~/t$ sstableloader -v -d 10.1.10.203 office_audit/log/
> -f /etc/cassandra/cassandra.yaml -u pcheon -pw `cat .secret`
> WARN 17:23:45,166 Small commitlog volume detected at
> /var/lib/cassandra/commitlog; setting commitlog_total_space_in_mb to 2316.
> You can override this in cassandra.yaml
> WARN 17:23:45,170 Small cdc volume detected at /var/lib/cassandra/cdc_raw;
> setting cdc_total_space_in_mb to 1158. You can override this in
> cassandra.yaml
> WARN 17:23:45,285 Only 5.318GiB free across all data volumes. Consider
> adding more capacity to your cluster or removing obsolete snapshots
> Established connection to initial hosts
> Opening sstables and calculating sections to stream
> Streaming relevant part of
> /home/pcheon/t/office_audit/log/mc-1083-big-Data.db
> /home/pcheon/t/office_audit/log/mc-1100-big-Data.db
> /home/pcheon/t/office_audit/log/mc-1101-big-Data.db
> /home/pcheon/t/office_audit/log/mc-257-big-Data.db
> /home/pcheon/t/office_audit/log/mc-984-big-Data.db to [/10.1.10.201,
> /10.1.10.203, /10.1.10.202]
> ERROR 17:23:49,460
[jira] [Created] (CASSANDRA-14437) SSTableLoader does not work when "internode_encryption : all" is set
Paul Cheon created CASSANDRA-14437:
--
Summary: SSTableLoader does not work when "internode_encryption :
all" is set
Key: CASSANDRA-14437
URL: https://issues.apache.org/jira/browse/CASSANDRA-14437
Project: Cassandra
Issue Type: Bug
Components: Tools
Reporter: Paul Cheon
Fix For: 3.11.2
I am trying to use sstableloader to restore snapshot.
If "internode_encryption : all" is set, then it does not work and complain
with below error messages. I initiated sstableloader from 10.1.10.203
(yvr-paul-cas003), so 10.1.10.203 worked fine, but the the other two nodes
(10.1.10.201 & 10.1.10.202 failed)
{noformat}
pcheon@yvr-paul-cas003:~/t$ sstableloader -v -d 10.1.10.203 office_audit/log/
-f /etc/cassandra/cassandra.yaml -u pcheon -pw `cat .secret`
WARN 17:23:45,166 Small commitlog volume detected at
/var/lib/cassandra/commitlog; setting commitlog_total_space_in_mb to 2316. You
can override this in cassandra.yaml
WARN 17:23:45,170 Small cdc volume detected at /var/lib/cassandra/cdc_raw;
setting cdc_total_space_in_mb to 1158. You can override this in cassandra.yaml
WARN 17:23:45,285 Only 5.318GiB free across all data volumes. Consider adding
more capacity to your cluster or removing obsolete snapshots
Established connection to initial hosts
Opening sstables and calculating sections to stream
Streaming relevant part of /home/pcheon/t/office_audit/log/mc-1083-big-Data.db
/home/pcheon/t/office_audit/log/mc-1100-big-Data.db
/home/pcheon/t/office_audit/log/mc-1101-big-Data.db
/home/pcheon/t/office_audit/log/mc-257-big-Data.db
/home/pcheon/t/office_audit/log/mc-984-big-Data.db to [/10.1.10.201,
/10.1.10.203, /10.1.10.202]
ERROR 17:23:49,460 [Stream #938baee0-4e2d-11e8-9be0-ebc69ba4b87f] Streaming
error occurred on session with peer 10.1.10.201
java.net.SocketException: Invalid argument or cannot assign requested address
at java.net.PlainSocketImpl.socketConnect(Native Method) ~[na:1.8.0_112]
at
java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
~[na:1.8.0_112]
at
java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
~[na:1.8.0_112]
at
java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
~[na:1.8.0_112]
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
~[na:1.8.0_112]
at java.net.Socket.connect(Socket.java:589) ~[na:1.8.0_112]
at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:668)
~[na:1.8.0_112]
at sun.security.ssl.SSLSocketImpl.(SSLSocketImpl.java:495)
~[na:1.8.0_112]
at
sun.security.ssl.SSLSocketFactoryImpl.createSocket(SSLSocketFactoryImpl.java:169)
~[na:1.8.0_112]
at
org.apache.cassandra.security.SSLFactory.getSocket(SSLFactory.java:81)
~[apache-cassandra-3.11.2.jar:3.11.2]
at
org.apache.cassandra.tools.BulkLoadConnectionFactory.createConnection(BulkLoadConnectionFactory.java:56)
~[apache-cassandra-3.11.2.jar:3.11.2]
at
org.apache.cassandra.streaming.StreamSession.createConnection(StreamSession.java:282)
~[apache-cassandra-3.11.2.jar:3.11.2]
at
org.apache.cassandra.streaming.ConnectionHandler.initiate(ConnectionHandler.java:86)
~[apache-cassandra-3.11.2.jar:3.11.2]
at
org.apache.cassandra.streaming.StreamSession.start(StreamSession.java:269)
~[apache-cassandra-3.11.2.jar:3.11.2]
at
org.apache.cassandra.streaming.StreamCoordinator$StreamSessionConnector.run(StreamCoordinator.java:263)
[apache-cassandra-3.11.2.jar:3.11.2]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[na:1.8.0_112]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[na:1.8.0_112]
at
org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:81)
[apache-cassandra-3.11.2.jar:3.11.2]
at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_112]
ERROR 17:23:49,458 [Stream #938baee0-4e2d-11e8-9be0-ebc69ba4b87f] Streaming
error occurred on session with peer 10.1.10.202
java.net.SocketException: Invalid argument or cannot assign requested address
at java.net.PlainSocketImpl.socketConnect(Native Method) ~[na:1.8.0_112]
at
java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
~[na:1.8.0_112]
at
java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
~[na:1.8.0_112]
at
java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
~[na:1.8.0_112]
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
~[na:1.8.0_112]
at java.net.Socket.connect(Socket.java:589) ~[na:1.8.0_112]
at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:668)
~[na:1.8.0_112]
at
