Yes, that sounds reasonable to me. Best, Gang
On Fri, Oct 27, 2023 at 4:44 AM Edward Seidl <etse...@live.com> wrote: > Thanks for the reply, Gang. I don't know if deprecation is what I'm going > for, I would just like a little clarity in Encodings.md around this issue. > As it stands, it seems like using 33 bits is allowed by the specification, > and given this<https://github.com/apache/arrow/issues/20374> issue > pointed out by Fu Xuwei it seems like there isn't a clear path to > supporting reading this in arrow-cpp (and perhaps arrow-rs as well?). I > think adding words to the effect that writers should not produce such > files, but readers should be able to read them (emphasis on "should"), > would leave things as they currently stand, but at least acknowledge > there's tension here. Does this seem reasonable? > > Cheers, > Ed > > ________________________________ > From: Gang Wu <ust...@gmail.com> > Sent: Wednesday, October 25, 2023 9:49 PM > To: dev@parquet.apache.org <dev@parquet.apache.org> > Subject: Re: Max bitwidth for delta encoding > > Hi Ed, > > My concern for changing specs is that existing writer implementations > have already produced parquet files that the change intends to avoid. > So it would be a long time to deprecate the old writers while any reader > implementation should always be able to decode legacy files. > > Best, > Gang > >
