You do not need one spark session per cluster. Spark SQL with Datasource v1 http://www.russellspitzer.com/2016/02/16/Multiple-Clusters-SparkSql-Cassandra/
DatasourceV2 Would require making two catalog references then copying between them https://github.com/datastax/spark-cassandra-connector/blob/master/doc/1_connecting.md#example-of-reading-from-one-cluster-and-writing-to-another On Tue, Jul 28, 2020, 6:26 AM Amit Sharma <resolve...@gmail.com> wrote: > Hi, I have table A in the cassandra cluster cluster -1 in one data > center. I have table B in cluster -2 in another data center. I want to copy > the data from one cluster to another using spark. I faced the problem that > I can not create two spark sessions as we need spark sessions per cluster. > Please let me know if there is any way to use spark batch job to copy data > among two cassandra clusters. > > > Thanks > Amit >
