pan3793 commented on PR #48851: URL: https://github.com/apache/spark/pull/48851#issuecomment-2628868027
@gene-db @chenhao-db This PR seems to wrongly point to an umbrella JIRA ticket, and the newly added conf `spark.sql.variant.allowReadingShredded` uses `true` as the default value. If I understand correctly, https://github.com/apache/parquet-format/pull/461 is going to make incompatible changes on the shredding spec, if Spark delivers the current shredding implementation as-is in Spark 4.0, additional migration/compatible efforts will be required in the future. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
