[
https://issues.apache.org/jira/browse/DRILL-5674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Paul Rogers updated DRILL-5674:
-------------------------------
Description:
Zip is a very common compression format. Create a compressed CSV file with
column headers: data.csv.zip.
Define a storage plugin config for the file, call it "dfs.myws", set delimiter
= ",", extract header = true, skip header = false.
Run a simple query:
SELECT * FROM dfs.myws.`data.csv.zip`
The result is garbage as the CSV reader is trying to parse Zipped data as if it
were text.
DRILL-5506 asks how to do this; the responder said to add a library to the
path. Better would be to simply support zip out-of-the-box as a default format.
was:
Create a compressed CSV file with column headers: data.csv.zip.
Define a storage plugin config for the file, call it "dfs.myws", set delimiter
= ",", extract header = true, skip header = false.
Run a simple query:
SELECT * FROM dfs.myws.`data.csv.zip`
This works fine.
Now, do the same thing, but using a table function:
{code}
SELECT * FROM table(dfs.myws.`data.csv.zip` (type => 'text', fieldDelimiter =>
',' , extractHeader => true))
{code}
This does not work. Apparently, the reader tries to read the zip file as if it
were an uncompressed file:
{code}
DATA_READ ERROR: Drill failed to read your text file. Drill supports up to
65536 columns in a text file. Your file appears to have more than that.
Failure while reading file file:/Users/fred/data/data.csv.zip. Happened at or
shortly before byte position 121177.
{code}
> Drill should support .zip compression
> -------------------------------------
>
> Key: DRILL-5674
> URL: https://issues.apache.org/jira/browse/DRILL-5674
> Project: Apache Drill
> Issue Type: Bug
> Affects Versions: 1.10.0
> Reporter: Paul Rogers
>
> Zip is a very common compression format. Create a compressed CSV file with
> column headers: data.csv.zip.
> Define a storage plugin config for the file, call it "dfs.myws", set
> delimiter = ",", extract header = true, skip header = false.
> Run a simple query:
> SELECT * FROM dfs.myws.`data.csv.zip`
> The result is garbage as the CSV reader is trying to parse Zipped data as if
> it were text.
> DRILL-5506 asks how to do this; the responder said to add a library to the
> path. Better would be to simply support zip out-of-the-box as a default
> format.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
