OK. If I take the manual approach, do parquet / arrow care whether metadata is binary or not?
On 2020/11/04 14:16:37, Wes McKinney <[email protected]> wrote: > There is not to my knowledge. > > On Tue, Nov 3, 2020 at 5:55 PM Jason Sachs <[email protected]> wrote: > > > > Is there any built-in method to compress parquet metadata? From what I can > > tell, the main table columns are compressed, but not the metadata. > > > > I have metadata which includes 100-200KB of text (JSON format) that is > > easily compressible... is there any alternative to doing it myself? >
