MaxGekk commented on pull request #31564: URL: https://github.com/apache/spark/pull/31564#issuecomment-780354284
> These options are for migration purposes (same as the legacy configs). I believe the rebase configs were placed to the `legacy` namespace mistakenly because they can be used not only for migration from previous Spark versions but also for reading (and writing) files written by other systems/frameworks/libs. So, the configs will stay with us forever. I would like to propose to "rename" existing configs via: `ConfigBuilder` has the method `withAlternative` , so, we can to introduce an alternative per each legacy rebase config: ``` spark.sql.legacy.parquet.int96RebaseModeInRead -> spark.sql.parquet.int96RebaseModeInRead ``` and deprecate ``` spark.sql.legacy.parquet.int96RebaseModeInRead ``` After that, document `spark.sql.parquet.int96RebaseModeInRead` in the Spark SQL guide. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
