peter-toth opened a new pull request, #59324: URL: https://github.com/apache/spark/pull/59324
### What changes were proposed in this pull request? This trims the comments and conf docs of #58895 (SPARK-59620) and #59269 (SPARK-60034), as @viirya suggested in https://github.com/apache/spark/pull/58895#issuecomment-6064730845. The comments keep only the invariants and the non-obvious trade-offs. The narrative goes: alternatives considered, past measurements, and the same point restated across `SQLConf`, `FileFormat`, `ParquetStorageFilter` and the reader. It also rewrites the doc of `spark.sql.parquet.storageFilterPushdown.enabled` to what changes a user's decision or answer, based on @viirya's text. It now also says that only `ParquetFileFormat` itself is supported, not its subclasses. The doc of the internal `maxSplicedRowGroupBytes` conf is shorter too. There are no code changes. ### Why are the changes needed? About 1,000 of the feature's ~2,400 non-test lines were comments, which made the reader hard to maintain. The public conf doc described internals rather than what a user needs to know. ### Does this PR introduce _any_ user-facing change? Only the doc of `spark.sql.parquet.storageFilterPushdown.enabled`. The feature is unreleased. ### How was this patch tested? Comments and doc strings only. `ParquetStorageFilterSuite`, `ParquetColumnIndexSuite` and `ParquetVectorizedSuite` pass, and so do scalastyle and checkstyle. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Claude Code with Claude Opus 5.5 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
