As I have said in another thread, Parquet V2 is a concept which contains a lot of features. FWIW, what are defined in the specs [1] are finalized and some of them have been implemented in various implementations. Any file that contains one or more of those features can be considered v2 but the community has never defined a formal approach to distinguish between v1 and v2. Parquet does have a field in the footer thrift definition to mark the file version [2]. However, not all implementations populate it correctly and some engines will even throw if the version is not 1. To avoid confusion, I strongly suggest not using any v2 feature in your case unless you are 100% sure that all your tools support the v2 feature set you have enabled.
[1] https://github.com/apache/parquet-format/blob/master/CHANGES.md [2] https://github.com/apache/parquet-format/blob/master/src/main/thrift/parquet.thrift#L1111 Best, Gang On Wed, Apr 24, 2024 at 10:29 AM Prem Sahoo <prem.re...@gmail.com> wrote: > Any one please shed some light on this ? > Sent from my iPhone > > > On Apr 23, 2024, at 4:30 PM, Prem Sahoo <prem.re...@gmail.com> wrote: > > > > Hello Team, > > How to find out if the Parquet file is V1 or V2 ? > > > > Do we have any tag/identifier which can say a Parquet file is created > thru V2 or V1 ? > > > > Is there any specific properties need to be there then only that parquet > can be written in Parquet V2? > > Sent from my iPhone >
