benj created DRILL-7392:
---------------------------
Summary: Exclude some files when requesting directory
Key: DRILL-7392
URL: https://issues.apache.org/jira/browse/DRILL-7392
Project: Apache Drill
Issue Type: Wish
Reporter: benj
Fix For: 1.16.0
Currently Drill ignores files starting with dot ('.') or underscore ('_').
When requesting directory with file of different types or different schema and
present at multiple levels of the tree file, it will be useful/more flexible,
to have also option(s) to exclude some files by extension or maybe with a
regexp.
For Example:
{code:java}
myTable
|--D1
|--file1.csv
|-file2.csv
|--D2
| SubD2
|--file1.csv
|--file1.csv
|--file1.xml
|--file1.json
{code}
without enter in a debate of what is a good the organisation/disposition for
the data, currently to request all the csv files of this example, the way is:
{code:sql}
SELECT * FROM ....`myTable/*/*.csv`
UNION
SELECT * FROM ....`myTable/*/*/*.csv`
{code}
It will be useful to have the capacity to request directly _myTable_ like:
{code:sql}
/* ALTER SESSION SET exclude_files='xml,json' */
/* or */
/* ALTER SESSION SET only_files='csv' */
SELECT * FROM myTable
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
