[
https://issues.apache.org/jira/browse/CASSANDRA-14927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
KALYAN CHAKRAVARTHY KANCHARLA updated CASSANDRA-14927:
------------------------------------------------------
Comment: was deleted
(was: I have migrated the data from 7 nodes to 21 nodes.
Along with the old data I see there is some new data being populated on the 21
nodes and also duplicate data on ids.sso_saml tables.)
> During data migration from 7 node to 21 node cluster using sstableloader, new
> data is being populated on the new tables & data is being duplicated on user
> type tables
> -----------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: CASSANDRA-14927
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14927
> Project: Cassandra
> Issue Type: Test
> Reporter: KALYAN CHAKRAVARTHY KANCHARLA
> Priority: Blocker
> Labels: test
> Fix For: 2.1.13
>
>
> I'm trying to migrate data from 7 node (single DC) cluster to a 21 node (3
> DC) cluster using sstableloader.
> We have same versions on both old and new clusters.
> *cqlsh 5.0.1*
> *Cassandra 2.1.13*
> *CQL spec 3.2.1*
> Old and New clusters are in different networks. So we opened the following
> ports between them.
> 7000- storage port
> 7001- ssl storage port
> 7199- JMX port
> 9042- client port
> 9160- Thrift client port
> We use vnodes in the clusters.
> We made sure cassandra.yaml file on the new cluster is set correct by
> changing following options,
>
> {{cluster_name: 'MyCassandraCluster' }}
> {{num_tokens: 256 }}
> {{seed_provider: - }}
> {{class_name: org.apache.cassandra.locator.SimpleSeedProvider }}
> {{parameters: - }}
> {{seeds: "10.168.66.41,10.176.170.59" }}
> {{listen_address: localhost}}
> {{endpoint_snitch: GossipingPropertyFileSnitch}}
> And also changes in cassaandra-rackdc-properties for each DC by specifying
> respective DC and rack.
> while creating keyspaces, changed Replication to NetworkTopologyStratagy.
>
> cluster looks healthy, all the node are UP and NORMAL.
>
> {color:#FF0000}*I was able to get the data from old cluster to new cluster.
> But, along with the data from old cluster, I see some new rows being
> populated in the tables on new cluster and data is being duplicated in the
> tables with user type*. {color}
> {color:#333333}We have used the following steps to migrate data:{color}
> # Took snapshorts for all the keyspaces that we want to migrate. (9
> keyspaces). Used the _nodetool snapshot_ command on source nodes to take
> snapshot of required keyspace/table by specifying _hostname, jmx port_ and
> _keyspace_
> __
> _/a/cassandra/bin/nodetool -u $(sudo su - company -c "cat
> /a/cassandra/jmxremote.password" | awk '\{print $1}') -pw $(sudo su - company
> -c "cat /a/cassandra/jmxremote.password" | awk '\{print $2}')_ *_-h
> localhost -p 7199 snapshot keyspace_name_*
> # After taking snapshots, move these snapshot directory from source nodes to
> target node.
>
> → Create a tar file on source node for the snapshot directory that we want to
> move on to target node.
> tar -cvf file.tar snapshot_name
> → Move this file.tar from source node to local machine.
> scp -S gwsh root@192.168.64.99:/a/cassandra/data/file.tar .
> → Now move this file.tar from local machine to a new directory(example: test)
> in the target node.
> scp -S gwsh file.tar root@192.168.58.41:/a/cassandra/data/test/.
> # Now untar this file.tar in test directory in target node.
> # The path of the sstables must be same in both source and target.
> # To bulk load these files using _sstableloader,run sstableloader on source
> node, indicate one or more nodes in the destination Cluster with -d flag,
> which can accept comma-separated list of IP addresses or hostnames, and
> specify the path to sstables in the source node._ __
> _/a/Cassandra/bin/_ *_./sstableloader -d host_IP path_to_sstables_*
> *_Example:_*
> [/a/cassandra/bin#|mailto:root@sqa-cassandra03.sqaextranet:/a/cassandra/bin]
> sstableloader -d 192.168.58.41 -u popps -pw ******* -tf
> org.apache.cassandra.thrift.SSLTransportFactory -ts
> /a/cassandra/ssl/truststore.jks -tspw test123 -ks
> /a/cassandra/ssl/keystore.jks -kspw test123 -f
> /a/cassandra/conf/cassandra.yaml
> /a/cassandra/data/app_properties/_admins-58524140431511e8bbb6357f562e11ca_/
> Summary statistics:
> Connections per host: : 1
> Total files transferred: : 9
> Total bytes transferred: : 1787893
> Total duration (ms): : 2936
> Average transfer rate (MB/s): : 0
> Peak transfer rate (MB/s): : 0
>
> Performed these steps on all the tables. And checked the row count in old and
> new tables using CQLSH
> cqlsh> SELECT count(*) FROM keyspace.table;
> example for a single table:
> count on new table: 341
> count on old table: 303
>
> And we are also able to identify the difference in tables by using 'sdiff'
> command. Followed the following steps:
> * created .txt/.csv files for tables in old and new clusters.
> * compared them using sdiff command
>
> *So I request someone can help me to know the cause behind the population of
> new data in the new tables.*
> Please let me know if you need more info.
> PS: After migrating the data for the first time and saw these issues, we have
> TRUNCATED all the tables and DROPPED tables with user 'type' and recreated
> the dropped tables. And did the same procedure for migrating data again.
> Still we see the same issues.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
