Jackie-Jiang commented on issue #4741: Make the download url of a realtime LLC segment as a list of urls. URL: https://github.com/apache/incubator-pinot/pull/4741#issuecomment-546165081 @chenboat I didn't find the new section to discuss this downloading from peer proposal without touching segment ZK metadata but using external view. - Server call controller API has no difference with server directly modify the segment ZK metadata. The point here is that it is very hard to keep segment ZK metadata consistent with the actual segment locations. Who is in charge of updating the segment ZK metadata? When segment moves from one server to another server, how to update the segment ZK metadata? What if one server is down during the move? What if the segment goes into ERROR state? There are so many things to consider, and it is very hard to cover all scenarios. On contrast, external view already gathered all these information, and it is just way simpler than keep modifying the segment ZK metadata. As long as we provide the same REST API on each server, we can easily construct the download url from the instance id. - Segment download format is another independent design discussion. Compressing segment is a CPU intensive operation, which could hurt server's performance.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
