cloud-fan commented on code in PR #44388:
URL: https://github.com/apache/spark/pull/44388#discussion_r1433674355
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala:
##########
@@ -4103,7 +4103,7 @@ object SQLConf {
.stringConf
.transform(_.toUpperCase(Locale.ROOT))
.checkValues(LegacyBehaviorPolicy.values.map(_.toString))
- .createWithDefault(LegacyBehaviorPolicy.EXCEPTION.toString)
+ .createWithDefault(LegacyBehaviorPolicy.CORRECTED.toString)
Review Comment:
The question is which one is more important. This is a behavior change for
existing Spark users, who might never read third-party parquet files. I don't
think there is much value of this change, as it only affects the very few
legacy parquet files.
On the other hand, I totally agree with flipping the default value for
write-side rebase configs. There shouldn't be people using very old Spark
versions to read parquet files.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
