skoppu22 commented on code in PR #169:
URL:
https://github.com/apache/cassandra-analytics/pull/169#discussion_r3475560921
##########
cassandra-analytics-core/src/main/java/org/apache/cassandra/spark/bulkwriter/cloudstorage/coordinated/CassandraClusterInfoGroup.java:
##########
@@ -211,38 +214,6 @@ public TokenRangeMapping<RingInstance>
getTokenRangeMapping(boolean cached)
return consolidatedTokenRangeMapping;
}
- /**
- * @return the lowest cassandra version among all clusters
- */
- @Override
- public String getLowestCassandraVersion()
- {
- // Return cached value if available (executor-side reconstruction)
- if (cachedLowestCassandraVersion != null)
- {
- return cachedLowestCassandraVersion;
- }
-
- if (clusterInfos.size() == 1)
- {
- return clusterInfos.get(0).getLowestCassandraVersion();
- }
-
- Map<String, String> aggregated =
applyOnEach(ClusterInfo::getLowestCassandraVersion);
- List<CassandraVersionFeatures> versions = aggregated.values()
- .stream()
-
.map(CassandraVersionFeatures::cassandraVersionFeaturesFromCassandraVersion)
- .sorted()
-
.collect(Collectors.toList());
- CassandraVersionFeatures first = versions.get(0);
- CassandraVersionFeatures last = versions.get(versions.size() - 1);
- Preconditions.checkState(first.getMajorVersion() ==
last.getMajorVersion(),
- "Cluster versions are not compatible.
lowest=%s and highest=%s",
- first.getRawVersionString(),
last.getRawVersionString());
Review Comment:
On the cluster, if one node is running 5.0 and another node running 4.x, the
bulk writer selects highest supported format 5.0 as bridge and writes BTI or
BIG-oa sstables. 4.x node will fail to read them. This will impact live node
operations reading that table as well.
The assumption behind this new feature is
- either all nodes in a cluster are in 4.x version
- or all nodes in a cluster in 5.0 version and SCM is cassandra_4 or
upgrading or none or combination of these.
In both cases, lowest and highest major version should match. Hence we need
to have this check in bridge selection logic for write as well, which I missed
adding.
Other option is, we need to select lowest supported sstable format for
write, so all nodes understands the sstables being written. With this we need
to allow lowest and highest major versions to differ as long as lowest sstable
version can be read on highest cassandra version node. i.e, we need to change
bridge determination logic to lowest sstable version for bulk write and highest
sstable version for bulk read.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
