benj created DRILL-7392: --------------------------- Summary: Exclude some files when requesting directory Key: DRILL-7392 URL: https://issues.apache.org/jira/browse/DRILL-7392 Project: Apache Drill Issue Type: Wish Reporter: benj Fix For: 1.16.0
Currently Drill ignores files starting with dot ('.') or underscore ('_'). When requesting directory with file of different types or different schema and present at multiple levels of the tree file, it will be useful/more flexible, to have also option(s) to exclude some files by extension or maybe with a regexp. For Example: {code:java} myTable |--D1 |--file1.csv |-file2.csv |--D2 | SubD2 |--file1.csv |--file1.csv |--file1.xml |--file1.json {code} without enter in a debate of what is a good the organisation/disposition for the data, currently to request all the csv files of this example, the way is: {code:sql} SELECT * FROM ....`myTable/*/*.csv` UNION SELECT * FROM ....`myTable/*/*/*.csv` {code} It will be useful to have the capacity to request directly _myTable_ like: {code:sql} /* ALTER SESSION SET exclude_files='xml,json' */ /* or */ /* ALTER SESSION SET only_files='csv' */ SELECT * FROM myTable {code} -- This message was sent by Atlassian Jira (v8.3.4#803005)