skoppu22 commented on code in PR #169:
URL:
https://github.com/apache/cassandra-analytics/pull/169#discussion_r3483450164
##########
cassandra-analytics-core/src/main/java/org/apache/cassandra/spark/bulkwriter/cloudstorage/coordinated/CassandraClusterInfoGroup.java:
##########
@@ -211,38 +214,6 @@ public TokenRangeMapping<RingInstance>
getTokenRangeMapping(boolean cached)
return consolidatedTokenRangeMapping;
}
- /**
- * @return the lowest cassandra version among all clusters
- */
- @Override
- public String getLowestCassandraVersion()
- {
- // Return cached value if available (executor-side reconstruction)
- if (cachedLowestCassandraVersion != null)
- {
- return cachedLowestCassandraVersion;
- }
-
- if (clusterInfos.size() == 1)
- {
- return clusterInfos.get(0).getLowestCassandraVersion();
- }
-
- Map<String, String> aggregated =
applyOnEach(ClusterInfo::getLowestCassandraVersion);
- List<CassandraVersionFeatures> versions = aggregated.values()
- .stream()
-
.map(CassandraVersionFeatures::cassandraVersionFeaturesFromCassandraVersion)
- .sorted()
-
.collect(Collectors.toList());
- CassandraVersionFeatures first = versions.get(0);
- CassandraVersionFeatures last = versions.get(versions.size() - 1);
- Preconditions.checkState(first.getMajorVersion() ==
last.getMajorVersion(),
- "Cluster versions are not compatible.
lowest=%s and highest=%s",
- first.getRawVersionString(),
last.getRawVersionString());
Review Comment:
- Create getBridgeVersion which first picks version override if available,
otherwise next picks bridge based on sstables version (if feature enabled),
otherwise picks bridge legacy style
- Modified determineBridgeVersionForWrite to pick lowest version as bridge,
this picks lowest compatible version for a cluster
- CassandraClusterInfoGroup's getBridgeVersion picks lowest compatible
version across clusters and verifies highest version can read sstables written
by selected bridge version
- Modified CassandraVersion enum to have lowestCompatibleVersionNumber
instead of calculating dynamically
- Modified bulk reader's validateSStableVersions (runs on executors) to
ensure bridge version can support sstables seen (previously was just checking
sstable versions seen are same as driver seen), this allows different sstable
versions to be written by the time executors run and works as long as bridge
can support them
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
