RE: Streaming with Java: Expected ReduceByWindow to Return JavaDStream

[Shao, Saisai]

Hi Jeff,

>From my understanding it seems more like a bug, since JavaDStreamLike is used 
>for Java code, return a Scala DStream is not reasonable. You can fix this by 
>submitting a PR, or I can help you to fix this.

Thanks
Jerry

From: Jeff Nadler [mailto:jnad...@srcginc.com]
Sent: Monday, January 19, 2015 2:04 PM
To: user@spark.apache.org
Subject: Streaming with Java: Expected ReduceByWindow to Return JavaDStream


Can anyone tell me if my expectations are sane?

I'm trying to do a reduceByWindow using the 3-arg signature (not providing an 
inverse reduce function):

JavaDStream<whatevs> reducedStream = messages.reduceByWindow((x, y) -> 
reduce(x, y), Durations.seconds(5), Durations.seconds(5));

This isn't building; looks like it's returning DStream not JavaDStream.

>From JavaDStreamLike.scala, looks like this sig returns DStream, the 4-arg sig 
>with the inverse reduce returns JavaDStream.

def reduceByWindow(
    reduceFunc: (T, T) => T,
    windowDuration: Duration,
    slideDuration: Duration
  ): DStream[T] = {
  dstream.reduceByWindow(reduceFunc, windowDuration, slideDuration)
}

So I'm just a noob.  Is this a bug or am I missing something?

Thanks!

Jeff Nadler