Hi, Folks.
We've just started looking at Spark Streaming, and I find myself a little
confused.
As I understood it, one of the main points of the system was that one could
use the same code when streaming, doing batch processing, or whatnot.
Yet when we try to apply a batch processor that analyzes RDDs to the
streaming case, we have to copy the code and replace RDDs with DStreams
everywhere, or dig into details of the component RDDs from which the
DStream is comprised.
Is there the intention of a common interface between RDD and DStream that
we could eventually use? Or is there a different paradigm of working with
both I'm just missing?
-Thanks,
Nathan
--
Nathan Kronenfeld
Senior Visualization Developer
Oculus Info Inc
2 Berkeley Street, Suite 600,
Toronto, Ontario M5A 4J5
Phone: +1-416-203-3003 x 238
Email: [email protected]
