JavaSparkContext currentContext = ...;
AccumulatorInteger accumulator = currentContext.accumulator(0,
MyAccumulator);
will create an Accumulator of Integers. For many large Data problems
Integer is too small and Long is a better type.
I see a call like the following
It's the exact same API you've already found, and it's documented:
http://spark.apache.org/docs/latest/api/scala/index.html#org.apache.spark.AccumulatorParam
JavaSparkContext has helper methods for int and double but not long. You
can just make your own little implementation of
I see Javadoc Style documentation but nothing that looks like a code sample
I tried the following before asking
public static class LongAccumulableParam implements
AccumulableParamLong,Long,Serializable {
@Override
public Long addAccumulator(final Long r, final Long t) {
Look again, the type is AccumulatorParam, not AccumulableParam. But
yes that's what you do.
On Thu, Nov 13, 2014 at 4:32 AM, Steve Lewis lordjoe2...@gmail.com wrote:
I see Javadoc Style documentation but nothing that looks like a code sample
I tried the following before asking
public