EnricoMi commented on issue #27377: [WIP][SPARK-30666][Core] Reliable single-stage accumulators URL: https://github.com/apache/spark/pull/27377#issuecomment-584673019 @databricks-david-lewis you are right, in that case `MAX` is not the notion that I want, it is more like "favour the fragment that is more comprehensive / has higher cardinality". In case of the LongAccumulator, that would be the one with the higher `fragmentCount`. Not sure how to name that mode, maybe `BIGGEST`. Mode name `RELIABLE` would be preferable, but this does not refer to the merge strategy but rather the outcome of that merge strategy. When an `AccumulatorV2` implementation cannot support such a mode, e.g. `SqlMetric`, it should simply throw an exception in `register`.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
