emkornfield commented on code in PR #588: URL: https://github.com/apache/parquet-format/pull/588#discussion_r3559959364
########## CONTRIBUTING.md: ########## @@ -110,10 +110,19 @@ demonstrate that the feature is mergeable to its implementation. The Parquet PMC aims to do releases of the format package only as needed when new features are introduced. If multiple new features are being proposed simultaneously some features might be consolidated into the same release. -Guidance is provided below on when implementations should enable features added -to the specification. Due to confusion in the past over Parquet versioning it -is not expected that there will be a 3.x release of the specification in the -foreseeable future. +Guidance is provided below on when implementations should enable features introduced +by new specification versions. If there is contributor bandwidth forwards-compatible +features may be backported to previous major versions of the specification and released +as a new minor version. + +Every forward incompatible change added to the specification requires Review Comment: > If a feature is forward compatible (will be ignored by old readers) why do we need to encode that in a minor version 🤔 This gets to the grouping features discussion. If we want to use versioning as a way of grouping features, there are forward compatible features that we also want to gate by version (not just have the user opt-in like bloom filters). Recent examples: 1. ieee754 sort order - As an end-user if the library just suddenly started emitting the new sort order and I was depending on the old one for pruning (in a different system) I'd be a little bit surprised. 2. The flat-buffers as thrift extension footer (I believe now mostly defunct). Also forward compatible, but as an end-user if I didn't upgrade the write version, why did my file sizes now increase by X% -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
