I think this is referring to changing the schema for List logical types [1] from deprecated schema layout to the "requirered" 3 level schema layout.
[1] https://github.com/apache/parquet-format/blob/master/LogicalTypes.md#lists On Sun, Jul 23, 2023 at 8:32 AM Xinli shang <[email protected]> wrote: > hI Rajesh, > > Thanks for posting! Can you give an example of the level 2 -> level 3 > converting? > > The current rewrite tool is to focus on speed by skipping a lot of > operations and the use case includes transforming the compression, > encryption, and deleting columns. @Gang Wu <[email protected]> > consolidated the tools to a universal rewriter and I think once you have > your requirement clarified, you can work with him for it. > > Xinli > > On Sat, Jul 22, 2023 at 12:07 PM Rajesh Mahindra <[email protected]> > wrote: > > > Hey folks, > > > > I have a bunch of parquets written with Level 2 list columns (among other > > columns). I was trying to extend the Parquet Rewrite tool to be able to > > read those parquet and only rewrite the list columns as Level 3. Any > > pointers on which classes or APIs i should leverage for this purpose? Any > > pointers would be appreciated. > > > > -- > > Take Care, > > Rajesh Mahindra > > > > > -- > Xinli Shang >
