Jackie-Jiang commented on PR #11486: URL: https://github.com/apache/pinot/pull/11486#issuecomment-1703380265
> Another approach to solve this is to explicitly enforce partition based replica group for upsert tables and make sure that each (replica group, partition) has 1 server. > > ``` > 6 servers, 6 partition, 2 replication > > (rg_id, partition_id) -> [list of servers] > 0, 0 -> server_0 > 0, 1 -> server_1 > 0, 2 -> server_2 > 0, 3 -> server_0 > 0, 4 -> server_1 > 0, 5 -> server_2 > 1, 0 -> server_3 > 1, 1 -> server_4 > 1, 2 -> server_5 > 1, 3 -> server_3 > 1, 4 -> server_4 > 1, 5- > server_5 > ``` > > This will provide the explicit guarantee for upsert property + protection from the rebalance. Also, we can keep the strict replica group logic as is. but, if we go with this route, we will need to go back to all the existing upsert tables and reconfigure the replica group setup. @snleee That is already here. The trigger for this issue is the instance partitions was updated, but rebalance was aborted without updating the IS. Essentially IS goes in-sync with the instance partitions. We want to gracefully handle it when adding a new segment to the IS -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
