skoppu22 commented on code in PR #169:
URL:
https://github.com/apache/cassandra-analytics/pull/169#discussion_r3475610969
##########
cassandra-analytics-core/src/main/java/org/apache/cassandra/spark/bulkwriter/AbstractBulkWriterContext.java:
##########
@@ -98,10 +101,32 @@ protected AbstractBulkWriterContext(@NotNull BulkSparkConf
conf,
this.conf = conf;
this.sparkDefaultParallelism = sparkDefaultParallelism;
- // Build everything fresh on driver
- this.clusterInfo = buildClusterInfo();
+ // Retrieve lowest Cassandra version without building full ClusterInfo
+ String lowestCassandraVersion = getLowestCassandraVersion(conf);
+ Set<String> sstableVersionsOnCluster = null;
+
+ // Get SSTable versions from cluster only if SSTable version-based
selection is enabled
+ // If disabled, skip retrieval to allow job to proceed even when
SSTable version detection fails
+ if (!conf.isSSTableVersionBasedBridgeDisabled())
+ {
+ sstableVersionsOnCluster = getSSTableVersionsOnCluster(conf);
+ }
+
+ // Determine bridge version
+ this.bridgeVersion =
SSTableVersionAnalyzer.determineBridgeVersionForWrite(
+ sstableVersionsOnCluster,
Review Comment:
- determineBridgeVersionForWrite first checks for the
isSSTableVersionBasedBridgeDisabled, if true uses fallback logic without
looking at sstableVersionsOnCluster value.
- if isSSTableVersionBasedBridgeDisabled is false, then we must have
retrieved sstableVersionsOnCluster and there should be atleast one sstable on
the cluster, hence we shouldn't have it null or empty.
- We have feature flag available here, better to use it instead of hacking
on sstableVersionsOnCluster being null which can be error case as well by any
chance.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
