I have added the content to the JIRA. Thanks Aman. -----Original Message----- From: Aman Sinha [mailto:[email protected]] Sent: 13 October 2015 17:15 To: [email protected] Subject: Re: Performance issue when setting dir0 & dir1 in where clause
This is related to partition pruning and is being addressed as part of DRILL-3759 (https://issues.apache.org/jira/browse/DRILL-3759). Unfortunately, this issue did not make into the 1.2 version but will likely be available with the next release. Could you please add your above use case in the JIRA ? Thanks. Aman On Tue, Oct 13, 2015 at 8:57 AM, <[email protected]> wrote: > Hi, > > I am doing a performance test on a Drill v1.1 query over directories. > I have the following two queries which are effectively doing the same thing: > > SELECT count(*) FROM `stg`.`ws`.`./2015/10/13` where columns[7] like > '%blah%' and columns[10] = ‘xx’ > > SELECT count(*) FROM `stg`.`ws`.`./2015` where dir0='10' and dir1='13' > and columns[7] like '%blah%' and columns[10] = ‘xx’ > I have noticed that the second query that includes ‘where dir0=’10’ > and dir1=’13’ ‘ in the where clause takes ~29 seconds to run, however > the first query takes ~8 seconds to run. The Fragment profiles show > the same explain plan and very similar operator profiles. The second > queries fragment profile starts from 22 seconds whereas the first > profile starts from 1 second. > > Can this performance issue be caused by the Calcite parser taking > longer to understand the where clause? > > Thanks, > Mike > > > This e-mail (including any attachments) is private and confidential, > may contain proprietary or privileged information and is intended for > the named > recipient(s) only. Unintended recipients are strictly prohibited from > taking action on the basis of information in this e-mail and must > contact the sender immediately, delete this e-mail (and all > attachments) and destroy any hard copies. Nomura will not accept > responsibility or liability for the accuracy or completeness of, or > the presence of any virus or disabling code in, this e-mail. If > verification is sought please request a hard copy. Any reference to > the terms of executed transactions should be treated as preliminary > only and subject to formal written confirmation by Nomura. Nomura > reserves the right to retain, monitor and intercept e-mail > communications through its networks (subject to and in accordance with > applicable laws). No confidentiality or privilege is waived or lost by > Nomura by any mistransmission of this e-mail. Any reference to > "Nomura" is a reference to any entity in the Nomura Holdings, Inc. group. > Please read our Electronic Communications Legal Notice which forms part of > this e-mail: > http://www.Nomura.com/email_disclaimer.htm > > This e-mail (including any attachments) is private and confidential, may contain proprietary or privileged information and is intended for the named recipient(s) only. Unintended recipients are strictly prohibited from taking action on the basis of information in this e-mail and must contact the sender immediately, delete this e-mail (and all attachments) and destroy any hard copies. Nomura will not accept responsibility or liability for the accuracy or completeness of, or the presence of any virus or disabling code in, this e-mail. If verification is sought please request a hard copy. Any reference to the terms of executed transactions should be treated as preliminary only and subject to formal written confirmation by Nomura. Nomura reserves the right to retain, monitor and intercept e-mail communications through its networks (subject to and in accordance with applicable laws). No confidentiality or privilege is waived or lost by Nomura by any mistransmission of this e-mail. Any reference to "Nomura" is a reference to any entity in the Nomura Holdings, Inc. group. Please read our Electronic Communications Legal Notice which forms part of this e-mail: http://www.Nomura.com/email_disclaimer.htm
