cshuo opened a new issue, #19780: URL: https://github.com/apache/hudi/issues/19780
### Feature Description **What the feature achieves:** Allow native Parquet log files to use a Zstandard compression level that is independent of the global Hadoop `parquet.compression.codec.zstd.level` setting used by Parquet base files. **Why this feature is needed:** Native log files favor write latency and do not generally need the same compression ratio as base files. Today, changing the Hadoop Zstd level affects both file types, so users cannot select a faster level for native logs while retaining a higher compression level for base files. ### User Experience **How users will use this feature:** - Configure `hoodie.logfile.parquet.compression.codec.zstd.level` for native Parquet logs. - The setting defaults to `1`. - Base files continue to use the global `parquet.compression.codec.zstd.level` setting. - The behavior is available through Spark, Flink, and Java writer paths. ### Hudi RFC Requirements **RFC PR link:** Not applicable. **Why an RFC isn't needed:** - Does this change public interfaces/APIs? Yes, it adds an optional advanced configuration. - Does this change storage format? No. - Justification: This is a narrowly scoped writer configuration that changes compression tuning only and does not alter the Parquet or Hudi on-disk format. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
