skoppu22 commented on code in PR #169:
URL:
https://github.com/apache/cassandra-analytics/pull/169#discussion_r3475560921
##########
cassandra-analytics-core/src/main/java/org/apache/cassandra/spark/bulkwriter/cloudstorage/coordinated/CassandraClusterInfoGroup.java:
##########
@@ -211,38 +214,6 @@ public TokenRangeMapping<RingInstance>
getTokenRangeMapping(boolean cached)
return consolidatedTokenRangeMapping;
}
- /**
- * @return the lowest cassandra version among all clusters
- */
- @Override
- public String getLowestCassandraVersion()
- {
- // Return cached value if available (executor-side reconstruction)
- if (cachedLowestCassandraVersion != null)
- {
- return cachedLowestCassandraVersion;
- }
-
- if (clusterInfos.size() == 1)
- {
- return clusterInfos.get(0).getLowestCassandraVersion();
- }
-
- Map<String, String> aggregated =
applyOnEach(ClusterInfo::getLowestCassandraVersion);
- List<CassandraVersionFeatures> versions = aggregated.values()
- .stream()
-
.map(CassandraVersionFeatures::cassandraVersionFeaturesFromCassandraVersion)
- .sorted()
-
.collect(Collectors.toList());
- CassandraVersionFeatures first = versions.get(0);
- CassandraVersionFeatures last = versions.get(versions.size() - 1);
- Preconditions.checkState(first.getMajorVersion() ==
last.getMajorVersion(),
- "Cluster versions are not compatible.
lowest=%s and highest=%s",
- first.getRawVersionString(),
last.getRawVersionString());
Review Comment:
On the cluster, if one node is running 5.0 and another node running 4.x, the
bulk writer selects highest support format 5.0 bridge and writes BTI or BIG-oa
sstables, 4.x node will fail to read them. This will impact live node
operations reading that table as well.
The assumption behind this new feature is
- either all nodes in a cluster are in 4.x version
- or all nodes in a cluster in 5.0 version and SCM is cassandra_4 or
upgrading or none or combination of these
In both cases, lowest and highest major version should match. Hence we need
to have this check in bridge selection logic for write as well, which I missed
adding.
Otherwise, we need to select lowest supported sstable format for write, so
all nodes understands the sstables being written. With this we need to allow
lowest and highest major versions to differ as long as compatible. i.e, we need
to change bridge determination logic to lowest sstable version for bulk write
and highest sstable version for bulk read.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
