Re: Cassandra loading data from another table

2018-10-01 Thread Christophe Schmitz
Have a look at using Spark on Cassandra. It's commonly used for data movement / data migration / reconciliation (on top of analytics). You will get much better performances. Christophe Schmitz - Instaclustr - Cassandra | Kafka | Spark Consulting On Tue, 2 Oct

Re: Cassandra loading data from another table

2018-10-01 Thread Richard Xin
Christophe, thanks for your insights,Sorry, I forgot to mention that currently both tableA and tableB are being updated by application (all newly inserted/updated records should be identical on A and B), exporting from tableB and COPY it back later on will result in older data overwrites

Re: Cassandra loading data from another table

2018-10-01 Thread Christophe Schmitz
Hi Richard, You could consider exporting your few thousands record of Table B in a file, with *COPY TO*. Then *TRUNCATE* Table B, copy the SSTable files of TableA to the data directory of Table A (make sure you *flush* the memtables first), then run nodetool *refresh*. Final step is to load the

Cassandra loading data from another table

2018-10-01 Thread Richard Xin
I have a tableA with about a few ten millions record, and I have tableB with a few thousands record,TableA and TableB have exact same schema (except that tableB doesnt have TTL) I want to load all data to tableB from tableA EXCEPT for those already on tableB (we don't want data on tableB to be