Quanlong Huang created IMPALA-9074:
--------------------------------------
Summary: Add support for zstd in ORC
Key: IMPALA-9074
URL: https://issues.apache.org/jira/browse/IMPALA-9074
Project: IMPALA
Issue Type: New Feature
Reporter: Quanlong Huang
The ORC lib already supports reading/writing to zstd compressed ORC files.
However, I failed in a quick try in Impala:
{code:sql}
hive> create table orc_zstd (id int, name string) stored as orc;
$ hdfs dfs -put id_name_zstd.orc hdfs://localhost:20500/test-warehouse/orc_zstd
impala-shell> invalidate metadata orc_zstd;
impala-shell> select * from orc_zstd;
ERROR: Encountered parse error in tail of ORC file
hdfs://localhost:20500/test-warehouse/orc_zstd/id_name_zstd.orc: Unknown
compression codec 5
{code}
The ORC file is generated by the csv-import tool:
https://github.com/apache/orc/blob/rel/release-1.6.0/tools/src/CSVFileImport.cc
(Manually changing the compression from ZLIB to ZSTD in it)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
