ankitsultana opened a new issue, #10788: URL: https://github.com/apache/pinot/issues/10788
Taking an example of a realtime table, if the segment download url link for some segments are missing, issuing a Table Rebalance with Downtime can cause data loss with very high likelihood (even with peer download enabled). I think it is better to check if all segments have the download url link first before starting the Rebalance, and throw an error and skip Rebalance in that case. We could also improve the Rebalance algorithm to be more aware of which segments are safe for concurrent move operations (i.e. segments with valid deepstore links), and which are not, and perform a rebalance accordingly. In any state, I don't think the system should be allowed to cause a data loss. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
