chenboat commented on PR #16494: URL: https://github.com/apache/pinot/pull/16494#issuecomment-3220960979
> > One possible encoding scheme is to encode topic name, backfill id and partition id in the max allowed int 2,147,483,647 limit. The first 4 digits are for topic, next 2 digit for backfill id and the next 4 digits for partition id. I think it should accommodate for nearly possible practical needs. > > I can think about 2 encoding methods but neither could fulfill the requirement > > * Use the sequence number as backfill or topic id -> It is hard to remove the topic. The backfill topic id is encoded in the partitioned id today. E.g., 11010005 means the 5th partition of the 11th topic (in the topic map) and it is a backfill topic (so 01). This is the way we can detect a backfill topic. > * Use the hardcode to map string to int id -> It is hard to avoid and deal with the collision. Today in multi-topic, we use the index of the topic in the streamConfig map to map a topic to a int. The above proposal uses the same idea. Any issue with continuing today's practice. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
