Thanks for the reply, Gang. I don't know if deprecation is what I'm going for, I would just like a little clarity in Encodings.md around this issue. As it stands, it seems like using 33 bits is allowed by the specification, and given this<https://github.com/apache/arrow/issues/20374> issue pointed out by Fu Xuwei it seems like there isn't a clear path to supporting reading this in arrow-cpp (and perhaps arrow-rs as well?). I think adding words to the effect that writers should not produce such files, but readers should be able to read them (emphasis on "should"), would leave things as they currently stand, but at least acknowledge there's tension here. Does this seem reasonable?
Cheers, Ed ________________________________ From: Gang Wu <ust...@gmail.com> Sent: Wednesday, October 25, 2023 9:49 PM To: dev@parquet.apache.org <dev@parquet.apache.org> Subject: Re: Max bitwidth for delta encoding Hi Ed, My concern for changing specs is that existing writer implementations have already produced parquet files that the change intends to avoid. So it would be a long time to deprecate the old writers while any reader implementation should always be able to decode legacy files. Best, Gang
