cshuo opened a new pull request, #19781: URL: https://github.com/apache/hudi/pull/19781
### Describe the issue this Pull Request addresses Closes #19780. The global Hadoop `parquet.compression.codec.zstd.level` setting currently applies to both Parquet base files and native Parquet log files. Native logs favor write latency and may not need the higher compression level selected for base files, but there is no independent native-log setting. ### Summary and Changelog - Add the advanced `hoodie.logfile.parquet.compression.codec.zstd.level` configuration with a default value of `1`. - Add a `ParquetUtils` helper that applies the native-log level only to native log paths, copying the storage configuration only when the global level is absent or different. - Apply the built-in native-log override before `HoodieParquetConfigInjector` in Spark, Flink, and Avro/Java Parquet writer factories, preserving the custom injector as the highest-priority extension point. - Add unit coverage for the default level, explicit override, equal-level configuration reuse, missing global configuration, and unaffected base-file paths. ### Impact Users can tune native Parquet log Zstd compression independently from base files. Native logs default to level `1`, while base files retain the global Hadoop setting. This adds one optional advanced configuration and does not change Hudi's storage format or public APIs. ### Risk Level Low. The override is limited to native log paths, base-file paths retain the original configuration instance, and custom Parquet config injectors still run last. The targeted `TestParquetUtils#testApplyNativeLogZstdCompressionLevel` and Flink Parquet writer factory test passed. ### Documentation Update The new configuration and its default behavior are documented in `HoodieStorageConfig`. No storage-format documentation changes are required. ### Contributor's checklist - [ ] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute) - [x] Enough context is provided in the sections above - [x] Adequate tests were added if applicable -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
