skoppu22 commented on code in PR #211:
URL:
https://github.com/apache/cassandra-analytics/pull/211#discussion_r3335130246
##########
cassandra-analytics-core/src/main/java/org/apache/cassandra/spark/bulkwriter/SortedSSTableWriter.java:
##########
@@ -227,6 +229,15 @@ public synchronized Map<Path, Digest>
prepareSStablesToSend(@NotNull BulkWriterC
};
Set<Path> dataFilePaths = new HashSet<>();
Map<Path, Digest> fileDigests = new HashMap<>();
+ // FIXME: CQLSSTableWriter may produce incomplete Filter.db file,
rebuilding it manually (see CASSANALYTICS-167).
+ // rebuild Filter.db files before calculating their digest
+ try (DirectoryStream<Path> stream = getDataFileStream(sstableFilter))
+ {
+ for (Path path : stream)
Review Comment:
Can we just pass sstableFilter to rebuildFilterComponents as it is already
calling and looping through it?
##########
cassandra-analytics-core/src/main/java/org/apache/cassandra/spark/bulkwriter/SortedSSTableWriter.java:
##########
@@ -227,6 +229,15 @@ public synchronized Map<Path, Digest>
prepareSStablesToSend(@NotNull BulkWriterC
};
Set<Path> dataFilePaths = new HashSet<>();
Map<Path, Digest> fileDigests = new HashMap<>();
+ // FIXME: CQLSSTableWriter may produce incomplete Filter.db file,
rebuilding it manually (see CASSANALYTICS-167).
+ // rebuild Filter.db files before calculating their digest
+ try (DirectoryStream<Path> stream = getDataFileStream(sstableFilter))
+ {
+ for (Path path : stream)
Review Comment:
Can we just pass sstableFilter to rebuildFilterComponents as it is already
calling getDataFileStream and looping through it?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
