skoppu22 commented on code in PR #220:
URL:
https://github.com/apache/cassandra-analytics/pull/220#discussion_r3596680932
##########
cassandra-four-zero-types/src/main/java/org/apache/cassandra/spark/reader/SchemaBuilder.java:
##########
@@ -222,6 +224,20 @@ private static Pair<KeyspaceMetadata, TableMetadata>
updateSchema(Schema schema,
}
tableMetadata.columns().forEach(columnValidator);
+
+ // This builder always produces an index-less table (indexes are
applied later by the 5.0 bridge), so a
+ // rebuild never carries indexes even if the caller passed index
statements. buildSchema runs repeatedly per
+ // table in a JVM; if an earlier call already registered indexes, copy
them onto this rebuild so it matches
+ // the registered table.
+ if (maybeExistingTableMetadata != null
+ && !maybeExistingTableMetadata.indexes.isEmpty()
+ && tableMetadata.indexes.isEmpty())
+ {
+ tableMetadata = tableMetadata.unbuild()
+
.indexes(maybeExistingTableMetadata.indexes)
+ .build();
+ }
+
Review Comment:
@jyothsnakonisa I have applied your recommended changes to pass index
statements as part of cqlTable. We still need beforeTableRegistered. Because
Reason 1, Some builders only receive a CREATE TABLE string, with no CqlTable
and no CREATE INDEX statements, so they can only build index‑less. These are
public CassandraBridge APIs:
```
encodePartitionKeys(...) / toTokens(...) / encodePartitionKey(...) → new
FiveZeroSchemaBuilder(createTableStmt, ks, rf, partitioner)
readPartitionKeys(...) → same createStmt‑only builder
```
We can't make these pass statements without changing the public bridge API,
and their callers frequently don't have the CREATE INDEX statements to pass. If
one of these is the first build of a table in a JVM, the table registers
without its SAI index.
Reason 2, The same table is built more than once per JVM in normal flows:
```
Write path, multiple partitions/executor: RecordWriter is constructed per
Spark partition (mapPartitions); the 2nd+ partition on an executor rebuilds the
already‑registered table.
Write path, single partition: after writing,
SortedSSTableWriter.validateSSTables(...) → buildLocalDataLayer → new
LocalDataLayer → buildSchema rebuilds the same table in the same JVM.
Read path: getCompactionScanner, getPartitionSizeIterator,
rebuildBloomFilter all build from a CqlTable, repeatedly within a JVM.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
