You mean the key-value metadata at the schema/field-level? That can be binary (it gets base64-encoded when written to Parquet)
On Wed, Nov 4, 2020 at 10:22 AM Jason Sachs <[email protected]> wrote: > > OK. If I take the manual approach, do parquet / arrow care whether metadata > is binary or not? > > On 2020/11/04 14:16:37, Wes McKinney <[email protected]> wrote: > > There is not to my knowledge. > > > > On Tue, Nov 3, 2020 at 5:55 PM Jason Sachs <[email protected]> wrote: > > > > > > Is there any built-in method to compress parquet metadata? From what I > > > can tell, the main table columns are compressed, but not the metadata. > > > > > > I have metadata which includes 100-200KB of text (JSON format) that is > > > easily compressible... is there any alternative to doing it myself? > >
