Run it as part of integration testing, you can still use scala test but with a 
different sub folder (it or integrationtest) instead of test.
Within integrationtest you create a local Spark server that has also 
accumulators.

> On 10. Apr 2018, at 17:35, Guillermo Ortiz <konstt2...@gmail.com> wrote:
> 
> I have a unitTest in SparkStreaming which has an input parameters.
> -DStream[String]
> 
> Inside of the code I want to update an LongAccumulator. When I execute the 
> test I get an NullPointerException because the accumulator doesn't exist. Is 
> there any way to test this?
> 
> My accumulator is updated in different methods. 
> 
>     def execute(stream: DStream[String]): Unit = {
>         stream.foreachRDD { rdd =>
>           rdd.foreach { r =>
>             if (r == "A"){
>               acc.add(1)
>               sendKafka(...)
>             }`enter code here`
>         }      
>     }
> 
> It's possible to test this kind of method?
> 
>     runAction[String](input, service.execute)
> 
> When it try to update the accumulator it doesn't work because it doesn't have 
> inited. I could add a new parameter to the execute method, and it's okay, but 
> runAction doesn't admint more parameters either. 

---------------------------------------------------------------------
To unsubscribe e-mail: user-unsubscr...@spark.apache.org

Reply via email to