Souri created SPARK-12958:
-----------------------------
Summary: Map accumulator in spark
Key: SPARK-12958
URL: https://issues.apache.org/jira/browse/SPARK-12958
Project: Spark
Issue Type: Wish
Components: Spark Core
Reporter: Souri
Priority: Minor
Spark by default supports accumulators of Int,Long,Double,Float.
It would be good if we can have a Map accumulator where each executor can just
add key->value pairs and driver can have access to the aggregated value for
each key in the map.
In this way, it would also be easier to use accumulators for various metrics.
We can define metrics at runtime as the map can take any string key.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
