With regard to the last comment on directory based pruning, please watch DRILL-3759 (https://issues.apache.org/jira/browse/DRILL-3759). I don't have a timeline for it yet but hopefully in the next Drill release.
Aman On Mon, Oct 19, 2015 at 3:50 AM, Dhruv Gohil <[email protected]> wrote: > "What's needed in Drill to truly eliminate ETL" +1 but in another thread > ;-) > few 'hacks' we want to share there of our 'work rounds' related to > various drill limitations on multi directory queries (99% of our workload) > , e.g. avoiding empty directory failures, building queries with directory > pruning that 'works' etc.. > > > > On Monday 19 October 2015 01:47 PM, Stefán Baxter wrote: > >> Hi Ted, >> >> Your approach only works for a single directory, not a directory >> structure. >> >> I will create an improvement request later today. >> >> I would welcome a session on "What's needed in Drill to truly eliminate >> ETL" (Just an idea) >> >> Regards, >> -Stefan >> >> On Sun, Oct 18, 2015 at 10:30 PM, Stefán Baxter < >> [email protected]> >> wrote: >> >> than you Jacques, I will. >>> >>> On Sun, Oct 18, 2015 at 10:01 PM, Jacques Nadeau <[email protected]> >>> wrote: >>> >>> Stefan, can you open a JIRA for reading multiple files types in a single >>>> directory. It isn't the most common case we've run across but is >>>> definitely >>>> something that should be addressed. >>>> >>>> -- >>>> Jacques Nadeau >>>> CTO and Co-Founder, Dremio >>>> >>>> On Sat, Oct 17, 2015 at 10:33 AM, Stefán Baxter < >>>> [email protected]> >>>> wrote: >>>> >>>> Thanks Abhishek, >>>>> >>>>> I think Drill is still quite far from eliminating ETL and the list of >>>>> obstacles on the way to there seems growing. (yeah, disappointment got >>>>> >>>> me >>>> >>>>> for a bit) >>>>> >>>>> Regards, >>>>> -Stefan >>>>> >>>>> >>> >
