As one of contributors in Structured Streaming, I would vote on having
migration guide doc for structured streaming as well, once we decide
standard format of migration guide.
In Spark 3.0.0 there're some breaking change on even SS area - one example
is SPARK-28199 which Sean took care of leaving
Thank you, Josh and Xiao. That sounds great.
Do you think we can have some parts of that improvement in `2.4.4` document
first since that is the very next release?
Bests,
Dongjoon.
On Sun, Jul 14, 2019 at 4:25 PM Xiao Li wrote:
> Yeah, Josh! All these ideas sound good to me. All the top
There’s another pr https://github.com/apache/spark/pull/21861 but which is
based the old V2 APIs.
We’d better link the JIRAs, SPARK-24907, SPARK-25547, and SPARK-28380 and
finalize a plan.
Xianyin
From: Shiv Prashant Sood
Date: Sunday, July 14, 2019 at 2:59 AM
To: Gabor Somogyi
Cc:
Yeah, Josh! All these ideas sound good to me. All the top commercial
database products have very detailed guide/document about the version
upgrading. You can easily find them.
Currently, only SQL and ML modules have the migration or upgrade guides.
Since Spark 2.3 release, we strictly require the
I'd like to discuss the Spark SQL migration / upgrade guides in the Spark
documentation: these are valuable resources and I think we could increase
that value by making these docs easier to discover and by adding a bit more
structure to the existing content.
For folks who aren't familiar with
