You can use the parquet-cli or parquet-tools to inspect metadata, including compression codecs.
Dictionary pages are compressed with the same compression codec that the other pages use. On Tue, Jan 30, 2018 at 10:19 AM, Hao Luo <[email protected]> wrote: > Thanks Ryan. > > For the dictionary page, is it compressed using LZO? How do I find out > what codec is used when compressing the dictionary page? > > Hao > > On Tue, Jan 30, 2018 at 8:28 AM, Ryan Blue <[email protected]> > wrote: > >> Parquet doesn't use LZOP. The supported formats are the ones you see in >> CompressionCodecName, which is used to identify the compression codec. >> >> You can see the file metadata, including compression, using the >> parquet-cli >> module's meta command. >> >> rb >> >> On Mon, Jan 29, 2018 at 6:50 PM, Hao Luo <[email protected]> >> wrote: >> >> > Hi, >> > I have a question about LZO compression in Parquet file. Does >> Parquet >> > use LZOP compression? If it uses LZOP, how does it differentiate between >> > LZO and LZOP? In the CompressionCodecName I only see LZO there. How do I >> > find information about which codec to use when decompress a dictionary >> > page? >> > >> > Thanks. >> > Hao >> > >> >> >> >> -- >> Ryan Blue >> Software Engineer >> Netflix >> > > -- Ryan Blue Software Engineer Netflix
