Github user dongjoon-hyun commented on a diff in the pull request:
https://github.com/apache/spark/pull/20208#discussion_r176933493
--- Diff: docs/sql-programming-guide.md ---
@@ -815,6 +815,54 @@ should start with, they can set `basePath` in the data
source options. For examp
when `path/to/table/gender=male` is the path of the data and
users set `basePath` to `path/to/table/`, `gender` will be a partitioning
column.
+### Schema Evolution
--- End diff --
Thank you so much for review, @gatorsmile . I waited for this moment. :)
I agree all of your comments. The main reason of those limitation is
because Spark file-based data sources doesn't have a capability to manage
multi-version schema and the column default values here. In fact, that is
beyond of Spark data sources' role. Thus, this PR is trying to add a test
coverage for AS-IS capability in order to prevent future regression and to make
a foundation to trust and to build on later. I don't think this is worthy of
documentation at the beginning. It's a start.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
