Things like kafka and user-defined sources are not supported yet, just
because Structure Streaming is in alpha stage.
Things like sort are not supported because of implementation difficulty,
and I don't think DStream can support either
What I want to know is the difference between API (or abstrac
For the 2.0 release, look for "Unsupported Operations" here:
http://spark.apache.org/docs/latest/structured-streaming-programming-guide.html
Also, there are bigger gaps - like no Kafka support, no way to plug
user-defined sources or sinks etc
Ofir Manor
Co-Founder & CTO | Equalum
Mobile: +972-5
I don't understand what kind of low level control that DStream can do while
Structure Streaming can not
Thanks
Chang
On Wednesday, July 27, 2016, Matei Zaharia wrote:
> Yup, they will definitely coexist. Structured Streaming is currently alpha
> and will probably be complete in the next few rel
Yup, they will definitely coexist. Structured Streaming is currently alpha and
will probably be complete in the next few releases, but Spark Streaming will
continue to exist, because it gives the user more low-level control. It's
similar to DataFrames vs RDDs (RDDs are the lower-level API for wh
Structured Streaming in 2.0 is declared as alpha - plenty of bits still
missing:
http://spark.apache.org/docs/latest/structured-streaming-programming-guide.html
I assume that it will be declared stable / GA in a future 2.x release, and
then it will co-exist with DStream for quite a while before so
Hi guys
Structure Stream is coming with spark 2.0, but I noticed that DStream is
still here
What's the future of the DStream, will it be deprecated and removed
eventually? Or co-existed with Structure Stream forever?
Thanks
Chang
