How (in Java) do I create an Accumulator of type Long

2014-11-12 Thread Steve Lewis
JavaSparkContext currentContext = ...; AccumulatorInteger accumulator = currentContext.accumulator(0, MyAccumulator); will create an Accumulator of Integers. For many large Data problems Integer is too small and Long is a better type. I see a call like the following

Re: How (in Java) do I create an Accumulator of type Long

2014-11-12 Thread Sean Owen
It's the exact same API you've already found, and it's documented: http://spark.apache.org/docs/latest/api/scala/index.html#org.apache.spark.AccumulatorParam JavaSparkContext has helper methods for int and double but not long. You can just make your own little implementation of

Re: How (in Java) do I create an Accumulator of type Long

2014-11-12 Thread Steve Lewis
I see Javadoc Style documentation but nothing that looks like a code sample I tried the following before asking public static class LongAccumulableParam implements AccumulableParamLong,Long,Serializable { @Override public Long addAccumulator(final Long r, final Long t) {

Re: How (in Java) do I create an Accumulator of type Long

2014-11-12 Thread Sean Owen
Look again, the type is AccumulatorParam, not AccumulableParam. But yes that's what you do. On Thu, Nov 13, 2014 at 4:32 AM, Steve Lewis lordjoe2...@gmail.com wrote: I see Javadoc Style documentation but nothing that looks like a code sample I tried the following before asking public