Re: Export/Importing keyspace from a different sized cluster
Also for testing purposes, you can send only one replica set to the Test DC. For instance with a RF=3 and 3 C* racks, you can just rsync/sstableload one rack. It will be faster and OK for tests. Best, Romain Le Mardi 20 septembre 2016 3h28, Michael Laws a écrit : I put together a shell wrapper around nodetool/sstableloader that I’ve been running for the past few years – https://github.com/AppliedInfrastructure/cassandra-snapshot-toolsAlways seemed to work well for these kinds of scenarios… Never really had to think about where SSTables were on the filesystem, etc. Mike From: Justin Sanciangco [mailto:jsancian...@blizzard.com] Sent: Monday, September 19, 2016 6:20 PM To: user@cassandra.apache.org Subject: RE: Export/Importing keyspace from a different sized cluster I am running cqlsh 5.0.1 | Cassandra 2.1.11.969 | DSE 4.8.3 | CQL spec 3.2.1 | Doing the below command seemed to worksstableloader -d Thanks for the help! From: Jeff Jirsa [mailto:jeff.ji...@crowdstrike.com] Sent: Monday, September 19, 2016 5:49 PM To: user@cassandra.apache.org Subject: Re: Export/Importing keyspace from a different sized cluster Something like that, depending on your version (which you didn’t specify). Note, though, that sstableloader is notoriously picky about the path to sstables. In particular, it really really really wants a directory structure that matches the directory structure on disk, and wants you to be at the equivalent of the parent/data_files_directory (so if you dump your sstables at /path/to/data/keyspace/table/, you’d want to run sstableloader from /path/to/data/ and provide keyspace/table/ as the location). From: Justin Sanciangco Reply-To: "user@cassandra.apache.org" Date: Monday, September 19, 2016 at 5:44 PM To: "user@cassandra.apache.org" Subject: RE: Export/Importing keyspace from a different sized cluster So if I rsync the the sstables say from source node 1 and source node 2 to target node 1. Would I just run the command like this? From target hostsstableloader -d From: Jeff Jirsa [mailto:jeff.ji...@crowdstrike.com] Sent: Monday, September 19, 2016 4:45 PM To: user@cassandra.apache.org Subject: Re: Export/Importing keyspace from a different sized cluster You can ship the sstables to the destination (or any other server with Cassandra binary tools installed) via ssh/rsync and run sstableloader on the destination cluster as well. From: Justin Sanciangco Reply-To: "user@cassandra.apache.org" Date: Monday, September 19, 2016 at 2:49 PM To: "user@cassandra.apache.org" Subject: Export/Importing keyspace from a different sized cluster Hello, Assuming I can’t get ports opened from source to target cluster to run sstableloader, what methods can I use to load a single keyspace from one cluster to another cluster of different size? Appreciate the help… Thanks,Justin
RE: Export/Importing keyspace from a different sized cluster
I put together a shell wrapper around nodetool/sstableloader that I’ve been running for the past few years – https://github.com/AppliedInfrastructure/cassandra-snapshot-tools Always seemed to work well for these kinds of scenarios… Never really had to think about where SSTables were on the filesystem, etc. Mike *From:* Justin Sanciangco [mailto:jsancian...@blizzard.com] *Sent:* Monday, September 19, 2016 6:20 PM *To:* user@cassandra.apache.org *Subject:* RE: Export/Importing keyspace from a different sized cluster I am running cqlsh 5.0.1 | Cassandra 2.1.11.969 | DSE 4.8.3 | CQL spec 3.2.1 | Doing the below command seemed to work sstableloader -d Thanks for the help! *From:* Jeff Jirsa [mailto:jeff.ji...@crowdstrike.com ] *Sent:* Monday, September 19, 2016 5:49 PM *To:* user@cassandra.apache.org *Subject:* Re: Export/Importing keyspace from a different sized cluster Something like that, depending on your version (which you didn’t specify). Note, though, that sstableloader is notoriously picky about the path to sstables. In particular, it really really really wants a directory structure that matches the directory structure on disk, and wants you to be at the equivalent of the parent/data_files_directory (so if you dump your sstables at /path/to/data/keyspace/table/, you’d want to run sstableloader from /path/to/data/ and provide keyspace/table/ as the location). *From: *Justin Sanciangco *Reply-To: *"user@cassandra.apache.org" *Date: *Monday, September 19, 2016 at 5:44 PM *To: *"user@cassandra.apache.org" *Subject: *RE: Export/Importing keyspace from a different sized cluster So if I rsync the the sstables say from source node 1 and source node 2 to target node 1. Would I just run the command like this? >From target host sstableloader -d *From:* Jeff Jirsa [mailto:jeff.ji...@crowdstrike.com ] *Sent:* Monday, September 19, 2016 4:45 PM *To:* user@cassandra.apache.org *Subject:* Re: Export/Importing keyspace from a different sized cluster You can ship the sstables to the destination (or any other server with Cassandra binary tools installed) via ssh/rsync and run sstableloader on the destination cluster as well. *From: *Justin Sanciangco *Reply-To: *"user@cassandra.apache.org" *Date: *Monday, September 19, 2016 at 2:49 PM *To: *"user@cassandra.apache.org" *Subject: *Export/Importing keyspace from a different sized cluster Hello, Assuming I can’t get ports opened from source to target cluster to run sstableloader, what methods can I use to load a single keyspace from one cluster to another cluster of different size? Appreciate the help… Thanks, Justin
RE: Export/Importing keyspace from a different sized cluster
I am running cqlsh 5.0.1 | Cassandra 2.1.11.969 | DSE 4.8.3 | CQL spec 3.2.1 | Doing the below command seemed to work sstableloader -d Thanks for the help! From: Jeff Jirsa [mailto:jeff.ji...@crowdstrike.com] Sent: Monday, September 19, 2016 5:49 PM To: user@cassandra.apache.org Subject: Re: Export/Importing keyspace from a different sized cluster Something like that, depending on your version (which you didn’t specify). Note, though, that sstableloader is notoriously picky about the path to sstables. In particular, it really really really wants a directory structure that matches the directory structure on disk, and wants you to be at the equivalent of the parent/data_files_directory (so if you dump your sstables at /path/to/data/keyspace/table/, you’d want to run sstableloader from /path/to/data/ and provide keyspace/table/ as the location). From: Justin Sanciangco mailto:jsancian...@blizzard.com>> Reply-To: "user@cassandra.apache.org<mailto:user@cassandra.apache.org>" mailto:user@cassandra.apache.org>> Date: Monday, September 19, 2016 at 5:44 PM To: "user@cassandra.apache.org<mailto:user@cassandra.apache.org>" mailto:user@cassandra.apache.org>> Subject: RE: Export/Importing keyspace from a different sized cluster So if I rsync the the sstables say from source node 1 and source node 2 to target node 1. Would I just run the command like this? From target host sstableloader -d From: Jeff Jirsa [mailto:jeff.ji...@crowdstrike.com] Sent: Monday, September 19, 2016 4:45 PM To: user@cassandra.apache.org<mailto:user@cassandra.apache.org> Subject: Re: Export/Importing keyspace from a different sized cluster You can ship the sstables to the destination (or any other server with Cassandra binary tools installed) via ssh/rsync and run sstableloader on the destination cluster as well. From: Justin Sanciangco mailto:jsancian...@blizzard.com>> Reply-To: "user@cassandra.apache.org<mailto:user@cassandra.apache.org>" mailto:user@cassandra.apache.org>> Date: Monday, September 19, 2016 at 2:49 PM To: "user@cassandra.apache.org<mailto:user@cassandra.apache.org>" mailto:user@cassandra.apache.org>> Subject: Export/Importing keyspace from a different sized cluster Hello, Assuming I can’t get ports opened from source to target cluster to run sstableloader, what methods can I use to load a single keyspace from one cluster to another cluster of different size? Appreciate the help… Thanks, Justin
Re: Export/Importing keyspace from a different sized cluster
Something like that, depending on your version (which you didn’t specify). Note, though, that sstableloader is notoriously picky about the path to sstables. In particular, it really really really wants a directory structure that matches the directory structure on disk, and wants you to be at the equivalent of the parent/data_files_directory (so if you dump your sstables at /path/to/data/keyspace/table/, you’d want to run sstableloader from /path/to/data/ and provide keyspace/table/ as the location). From: Justin Sanciangco Reply-To: "user@cassandra.apache.org" Date: Monday, September 19, 2016 at 5:44 PM To: "user@cassandra.apache.org" Subject: RE: Export/Importing keyspace from a different sized cluster So if I rsync the the sstables say from source node 1 and source node 2 to target node 1. Would I just run the command like this? >From target host sstableloader -d From: Jeff Jirsa [mailto:jeff.ji...@crowdstrike.com] Sent: Monday, September 19, 2016 4:45 PM To: user@cassandra.apache.org Subject: Re: Export/Importing keyspace from a different sized cluster You can ship the sstables to the destination (or any other server with Cassandra binary tools installed) via ssh/rsync and run sstableloader on the destination cluster as well. From: Justin Sanciangco Reply-To: "user@cassandra.apache.org" Date: Monday, September 19, 2016 at 2:49 PM To: "user@cassandra.apache.org" Subject: Export/Importing keyspace from a different sized cluster Hello, Assuming I can’t get ports opened from source to target cluster to run sstableloader, what methods can I use to load a single keyspace from one cluster to another cluster of different size? Appreciate the help… Thanks, Justin smime.p7s Description: S/MIME cryptographic signature
RE: Export/Importing keyspace from a different sized cluster
So if I rsync the the sstables say from source node 1 and source node 2 to target node 1. Would I just run the command like this? From target host sstableloader -d From: Jeff Jirsa [mailto:jeff.ji...@crowdstrike.com] Sent: Monday, September 19, 2016 4:45 PM To: user@cassandra.apache.org Subject: Re: Export/Importing keyspace from a different sized cluster You can ship the sstables to the destination (or any other server with Cassandra binary tools installed) via ssh/rsync and run sstableloader on the destination cluster as well. From: Justin Sanciangco mailto:jsancian...@blizzard.com>> Reply-To: "user@cassandra.apache.org<mailto:user@cassandra.apache.org>" mailto:user@cassandra.apache.org>> Date: Monday, September 19, 2016 at 2:49 PM To: "user@cassandra.apache.org<mailto:user@cassandra.apache.org>" mailto:user@cassandra.apache.org>> Subject: Export/Importing keyspace from a different sized cluster Hello, Assuming I can’t get ports opened from source to target cluster to run sstableloader, what methods can I use to load a single keyspace from one cluster to another cluster of different size? Appreciate the help… Thanks, Justin
Re: Export/Importing keyspace from a different sized cluster
You can ship the sstables to the destination (or any other server with Cassandra binary tools installed) via ssh/rsync and run sstableloader on the destination cluster as well. From: Justin Sanciangco Reply-To: "user@cassandra.apache.org" Date: Monday, September 19, 2016 at 2:49 PM To: "user@cassandra.apache.org" Subject: Export/Importing keyspace from a different sized cluster Hello, Assuming I can’t get ports opened from source to target cluster to run sstableloader, what methods can I use to load a single keyspace from one cluster to another cluster of different size? Appreciate the help… Thanks, Justin smime.p7s Description: S/MIME cryptographic signature
Re: Export/Importing keyspace from a different sized cluster
CQLSH COPY FROM / COPY TO? There are some significant performance improvements in recent versions: https://issues.apache.org/jira/browse/CASSANDRA-11053 On Tue, 20 Sep 2016 at 07:49 Justin Sanciangco wrote: > Hello, > > > > Assuming I can’t get ports opened from source to target cluster to run > sstableloader, what methods can I use to load a single keyspace from one > cluster to another cluster of different size? > > > > Appreciate the help… > > > > Thanks, > > Justin > > > -- Ben Slater Chief Product Officer Instaclustr: Cassandra + Spark - Managed | Consulting | Support +61 437 929 798
Export/Importing keyspace from a different sized cluster
Hello, Assuming I can't get ports opened from source to target cluster to run sstableloader, what methods can I use to load a single keyspace from one cluster to another cluster of different size? Appreciate the help... Thanks, Justin