Hi John, I think patch for DRILL-2517 has been merged to the apache master branch. Have you tried your query on the latest master branch?
In DRILL-2517, I posted some performance number for 117k small parquet files. The patch did show improvement. Before DRILL-3996 is resolved, for now if your query relies on filter pushdown logic to push partitioning filter first, then the patch for DRILL-2517 will not help. On Wed, Mar 2, 2016 at 4:23 AM, John Omernik <[email protected]> wrote: > I'd like to request drill-2517 be added as a bandaid for the planning > issues when there are lots of directories of parquet files. This issue is > really hurting drill adoption for me in my org. > > Thanks, John > > > > On Tuesday, March 1, 2016, Edmon Begoli <[email protected]> wrote: > >> May I please ask to give this issue the attention for 1.6: >> https://issues.apache.org/jira/plugins/servlet/mobile#issue/DRILL-3149 >> >> I will try to suggest a patch. Given my time constraints I might not be >> able to submit a complete, unit tested code but at I least I will try to >> submit a snippet that will help with fixing it up (I think we just need to >> do a look ahead byte lookup to ensure that it is not \r\n. >> >> On Tuesday, March 1, 2016, Jacques Nadeau <[email protected] >> <javascript:;>> wrote: >> >> > It seems like a stretch to include DRILL-3623 right before the release. >> > This is a pretty fundamental change that seems like it should soak for a >> > bit of time before we release. If we want to include, I'd suggest that we >> > disable the functionality by default and consider it experimental. >> > >> > I'll propose a few other patches for inclusion shortly. >> > >> > -- >> > Jacques Nadeau >> > CTO and Co-Founder, Dremio >> > >> > On Tue, Mar 1, 2016 at 5:04 PM, Parth Chandra <[email protected] >> <javascript:;> >> > <javascript:;>> wrote: >> > >> > > Hello everyone, >> > > >> > > It's time to start looking into the 1.6 release. >> > > >> > > Can all the folks working on open issues let me know if there are any >> > > JIRAs you would like to get into the release? >> > > >> > > I know of the following - >> > > >> > > *DRILL-4281 *- Drill should support inbound impersonation. Pull request >> > > expected today. C++ client to be tested. >> > > >> > > *DRILL-4372* - Drill Operators and Functions should correctly expose >> > their >> > > types within Calcite. Waiting for review. >> > > >> > > *DRILL-3623* - Limit 0 should avoid execution when querying a known >> > schema. >> > > Pull request expected today. Need to add limitations of current impl to >> > the >> > > JIRA. Review needed. >> > > >> > > *DRILL-4313* - Improved client randomization. Update JIRA with warnings >> > > about using the feature. Waiting for review. >> > > >> > > >> > > Thanks >> > > >> > > Parth >> > > >> > >> > > > -- > Sent from my iThing
