kishoreg commented on issue #4372: Support for single phase and two-phase distributed hash aggregation URL: https://github.com/apache/incubator-pinot/issues/4372#issuecomment-506155708 What we have in Pinot is very similar to two-phase hash aggregation but we merge groups across segments within the same node. We call this Combine Operator - analogous to the map-combine-reduce concept in M/R. We have S (number of segments) mappers, N (number of servers) Combiners, 1 Reducer. I think what you are suggesting S mappers, N reducers and 1 accumulator (union operator). I am trying to come up with a scenario where this would be better than the one we currently have in Pinot.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
