dongjoon-hyun commented on code in PR #53120:
URL: https://github.com/apache/spark/pull/53120#discussion_r2546538477
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala:
##########
@@ -1593,6 +1593,14 @@ object SQLConf {
.booleanConf
.createWithDefault(false)
+ val PARQUET_IGNORE_VARIANT_ANNOTATION =
+ buildConf("spark.sql.parquet.ignoreVariantAnnotation")
+ .doc("When true, ignore the variant logical type annotation and treat
the Parquet " +
+ "column in the same way as the underlying struct type")
+ .version("4.1.0")
Review Comment:
> Not sure if the parquet version we use in Spark 4.0 has the variant
logical type. I'll leave it to @harshmotw-db
Thanks. We can continue our discussion if we are not sure. AFAIK, it means
there is no regression at Apache Spark 4.1.0 from Apache Spark 4.0.0.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
