rseetham opened a new pull request, #18830: URL: https://github.com/apache/pinot/pull/18830
Resolves #18739. Multi-topic realtime ingestion encodes partition group IDs as topicIndex * 10000 + streamPartitionId, where topicIndex is the positional index in the config list. Deleting any topic except the last one shifts subsequent indices, breaking existing segment decoding. This PR introduces stable, immutable config IDs (stream.config.id) that replace positional indices, and a 5-part segment name format that stores the config ID and stream partition ID as separate fields. Key changes: - stream.config.id auto-assigned on table create/update, with validation (no duplicates, no negatives, stale-nextId detection) - MultiTopicLLCSegmentName: tableName__configId__streamPartitionId__sequenceNumber__creationTime - LLCSegmentName parses both 4-part and 5-part formats transparently - New segments created in 5-part format for multi-topic tables - All getStreamConfigIndexFromPinotPartitionId() call sites migrated to segment-name-based config ID lookup -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
