mayankshriv commented on issue #4372: Support for single phase and two-phase distributed hash aggregation URL: https://github.com/apache/incubator-pinot/issues/4372#issuecomment-512615638 @siddharthteotia Yes, the broker (1 reducer) will perform the final aggregation across nodes. If the cardinality is low, then the individual server responses will be small and single broker reducer should be able to handle easily. For scalability, we don't want too many nodes to serve the same query, so the number of nodes is also limited. If the cardinality is high, then we shouldn't shuffle anyways. So I have the same question as @kishoreg, if we can come up with a case where shuffle would be better than broker side reduce (for limited number of servers per query), then we can discuss in that context.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
