Github user maropu commented on a diff in the pull request:
https://github.com/apache/spark/pull/20929#discussion_r186581349
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ---
@@ -887,6 +887,14 @@ object SQLConf {
.booleanConf
.createWithDefault(false)
+ val IGNORE_NULL_FIELDS_STREAMING_SCHEMA_INFERENCE =
+ buildConf("spark.sql.streaming.schemaInference.ignoreNullFields")
--- End diff --
ok, I used `dropFieldIfAllNull`.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
