jiangzzhu opened a new pull request, #41511:
URL: https://github.com/apache/beam/pull/41511

   ## Summary
   
   In `SpannerIO.readChangeStream()`, `PartitionMetadataTableNames` previously 
constructed secondary index names from `<databaseId>_<uuid>` 
(`WatermarkIdx_<databaseId>_<uuid>` and `CreatedAtIdx_<databaseId>_<uuid>`) in 
both `generateRandom(String databaseId)` and `fromExistingTable(String 
databaseId, String table)`:
   
   - Whenever a pipeline configured an explicit metadata table via 
`withMetadataTable(...)` and 
`PartitionMetadataAdminDao.createPartitionMetadataTable()` ran on a metadata 
table that had already been created (for example, overlapping initial launches, 
retries after a timed-out/cancelled `UpdateDatabaseDdl` LRO, or 
case-insensitive GoogleSQL table existence mismatches), `CREATE TABLE IF NOT 
EXISTS` succeeded as a no-op while `CREATE INDEX IF NOT EXISTS` created two new 
duplicate secondary indexes on the existing metadata table because the index 
names contained a newly generated UUID.
   
   This change unifies secondary index naming across both initial metadata 
table creation (`generateRandom`) and explicit metadata tables 
(`fromExistingTable`):
   
   - `generateRandom(databaseId)` continues to generate a unique metadata table 
name using a random UUID (`Metadata_<databaseId>_<uuid>`) and delegates to 
`fromExistingTable(databaseId, table)`.
   - Both `generateRandom` and `fromExistingTable` derive deterministic index 
names directly from the metadata table name (`WatermarkIdx_<tableName>` and 
`CreatedAtIdx_<tableName>`, replacing `-` with `_` and truncating to 
`MAX_NAME_LENGTH = 63`).
   - Makes `CREATE INDEX IF NOT EXISTS` idempotent across retries and restarts 
while directly associating each secondary index name with its metadata table.
   
   ## Testing
   
   - Updated `PartitionMetadataTableNamesTest` to verify deterministic 
`WatermarkIdx_<tableName>` and `CreatedAtIdx_<tableName>` naming across both 
`generateRandom` and `fromExistingTable`, hyphen replacement, and 63-character 
length bounds.
   - Verified `PartitionMetadataAdminDaoTest` and Checkstyle/Spotless checks 
pass.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to