emkornfield commented on code in PR #588:
URL: https://github.com/apache/parquet-format/pull/588#discussion_r3456075078
##########
src/main/thrift/parquet.thrift:
##########
@@ -893,8 +880,13 @@ struct ColumnMetaData {
* whether we can decode those pages. **/
2: required list<Encoding> encodings
- /** Path in schema **/
- 3: required list<string> path_in_schema
+ /** Path in schema
+ *
+ * Made optional in parquet-format 3.0. If not written
+ * PARX magic number must be used (PATH_IN_SCHEMA_OMITTED, bit 1 in the
+ * feature flag bitmap, must be set).
+ **/
+ 3: optional list<string> path_in_schema
Review Comment:
> If we're changing this now, does that mean that we can't make other
changes without releasing 3 first?
Yes, once merged the next release off of main would be V3, and I expect
given community bandwidth this is the most likely outcome
> It feels like this is something we would change prior to a v3 release not
make the change, still continue to release v2 features (e.g. new types) and
later do a v3 release.
My proposal here is we can backport forward compatible changes. I added the
following the proposal:
> If there is contributor bandwidth forwards-compatible features may be
backported to previous major versions of the specification and released as a
new minor version.
I think this gets us out of the trap of trying curate releases, and still
allows writers that don't want to upgrade a path to back-porting if there is
enough demand.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
