maedhroz commented on code in PR #254:
URL:
https://github.com/apache/cassandra-analytics/pull/254#discussion_r4406376576
##########
cassandra-five-zero-bridge/src/main/java/org/apache/cassandra/bridge/CassandraBridgeImplementation.java:
##########
@@ -227,6 +234,202 @@ public StreamScanner<RowData>
getCompactionScanner(@NotNull CqlTable table,
}));
}
+ @Override
+ public StreamScanner<RowData> getCompactionScanner(@NotNull CqlTable table,
+ @NotNull Partitioner
partitioner,
+ @NotNull
SSTablesSupplier ssTables,
+ @Nullable
SparkRangeFilter sparkRangeFilter,
+ @NotNull
Collection<PartitionKeyFilter> partitionKeyFilters,
+ @NotNull
SSTableTimeRangeFilter sstableTimeRangeFilter,
+ @Nullable
PruneColumnFilter columnFilter,
+ @NotNull TimeProvider
timeProvider,
+ boolean readIndexOffset,
+ boolean
useIncrementalRepair,
+ @NotNull Stats stats,
+ @NotNull
List<SaiFilter> saiFilters,
+ int
saiMaxCandidateTokens,
+ @NotNull
ExecutorService executorService)
+ {
+ // A complete partition-key lookup is already more selective and
cheaper than consulting SAI.
+ if (!partitionKeyFilters.isEmpty() || saiFilters.isEmpty())
+ {
+ return getCompactionScanner(table, partitioner, ssTables,
sparkRangeFilter, partitionKeyFilters,
+ sstableTimeRangeFilter, columnFilter,
timeProvider, readIndexOffset,
+ useIncrementalRepair, stats);
+ }
+
+ // NOTE: Need to use SchemaBuilder to init keyspace if not already set
in Cassandra Schema instance
+ SchemaBuilder schemaBuilder = new SchemaBuilder(table, partitioner);
+ TableMetadata metadata = schemaBuilder.tableMetaData();
+
+ // Resolve each sstable token span before opening any SAI resources so
data files outside this
+ // Spark worker's token range are ignored.
+ // Below block does not open SSTableReaders, but already selects
sstables to be processed later.
+ // Actual opening of the sstables has to be done for *exactly* same
sstables. Otherwise, SAI processing
Review Comment:
nit: Is that assumption something we can assert later?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
