jiangzzhu opened a new pull request, #41511: URL: https://github.com/apache/beam/pull/41511
## Summary In `SpannerIO.readChangeStream()`, `PartitionMetadataTableNames` previously constructed secondary index names from `<databaseId>_<uuid>` (`WatermarkIdx_<databaseId>_<uuid>` and `CreatedAtIdx_<databaseId>_<uuid>`) in both `generateRandom(String databaseId)` and `fromExistingTable(String databaseId, String table)`: - Whenever a pipeline configured an explicit metadata table via `withMetadataTable(...)` and `PartitionMetadataAdminDao.createPartitionMetadataTable()` ran on a metadata table that had already been created (for example, overlapping initial launches, retries after a timed-out/cancelled `UpdateDatabaseDdl` LRO, or case-insensitive GoogleSQL table existence mismatches), `CREATE TABLE IF NOT EXISTS` succeeded as a no-op while `CREATE INDEX IF NOT EXISTS` created two new duplicate secondary indexes on the existing metadata table because the index names contained a newly generated UUID. This change unifies secondary index naming across both initial metadata table creation (`generateRandom`) and explicit metadata tables (`fromExistingTable`): - `generateRandom(databaseId)` continues to generate a unique metadata table name using a random UUID (`Metadata_<databaseId>_<uuid>`) and delegates to `fromExistingTable(databaseId, table)`. - Both `generateRandom` and `fromExistingTable` derive deterministic index names directly from the metadata table name (`WatermarkIdx_<tableName>` and `CreatedAtIdx_<tableName>`, replacing `-` with `_` and truncating to `MAX_NAME_LENGTH = 63`). - Makes `CREATE INDEX IF NOT EXISTS` idempotent across retries and restarts while directly associating each secondary index name with its metadata table. ## Testing - Updated `PartitionMetadataTableNamesTest` to verify deterministic `WatermarkIdx_<tableName>` and `CreatedAtIdx_<tableName>` naming across both `generateRandom` and `fromExistingTable`, hyphen replacement, and 63-character length bounds. - Verified `PartitionMetadataAdminDaoTest` and Checkstyle/Spotless checks pass. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
