Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/20579#discussion_r175118143
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSource.scala
---
@@ -542,6 +542,11 @@ case class DataSource(
throw new AnalysisException("Cannot save interval data type into
external storage.")
}
+ if (data.schema.size == 0) {
--- End diff --
Currently, we are not blocking this. I do not think we should introduce
this behavior change. This is risky to block all the cases.
Previously, I tried to block CREATE TABLE with an empty schema. Later, I
hit a regression because some data sources are using options/table properties
to specify the schema...
A general guide here is to avoid behavior changes if possible. When we have
to introduce a behavior change, we should make it configurable. At least, users
can convert it back by using a flag.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
