dongjoon-hyun commented on code in PR #59943:
URL: https://github.com/apache/spark/pull/59943#discussion_r4487830870
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala:
##########
@@ -6255,6 +6255,21 @@ object SQLConf {
.stringConf
.createWithDefault("")
+ val STREAMING_V2_COLUMN_PRUNING_ENABLED =
+ buildConf("spark.sql.streaming.v2ColumnPruning.enabled")
+ .internal()
Review Comment:
**3. [Nit] The new error tells users to set an internal configuration**
`STREAMING_SCAN_MISSING_COLUMNS` tells users to set
`spark.sql.streaming.v2ColumnPruning.enabled` to `false`, and that is the way
out for a query that fails in the middle of a run. The configuration is
`.internal()`, though, so the generated configuration docs of Spark SQL do not
list it. Since the error message gives it to users, could it be a public
configuration?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
