Hi Peter What does the profile for the query indicate?
Take a look at the operator overview. It will indicate which operator is using the most CPU cycles. If the average and max processing times vary wildly, it might be a problem of skew, where some fragments are doing a relatively excessive work. If that is the case, within that operator's profile segment, you can see the distribution of the fragments and see which ones are the long pole in your run. ~ KK On 8/6/2018 9:01:25 AM, Peter Edike <[email protected]> wrote: Hello all, The following query takes ages to complete in drill and more often that not, fails select * from dfs.`/iswdata/rj/201805` where unique_key in ( select unique_key from dfs.`/iswdata/rj/201805` group by unique_key having count(unique_key) > 1 ) limit 40 Please what can I do to improve the performance of this query Kind Regards Peter Edike Senior Software Engineer Research and Development, ENG Engineering Office NO: Mobile NO: Email: [email protected] [mailto:[email protected]] Interswitch Limited 1648C Oko-Awo Street, Victoria Island Lagos Customer Contact Centre 0700-9065000 ü http://www.interswitchgroup.com [http://www.interswitchgroup.com/] [https://www.quickteller.com/loan-request] This e-mail and all attachments transmitted with it remain the property of Interswitch Limited , the information contained herein are private confidential and intended solely for the use of the addressee. If you have received this e-mail in error, kindly notify the sender. If you are not the addressee, you should not disseminate, distribute or copy this e-mail. Kindly notify Interswitch immediately by email if you have received this email in error and delete this email and any attachment from your system Emails cannot be guaranteed to be secure or error free as the message and any attachments could be intercepted, corrupted, lost, delayed, incomplete or amended. the contents of this email or its attachments have been scanned for all viruses and all reasonable measures have been taken to ensure that no viruses are present. Interswitch Limited and its subsidiaries do not accept liability for damage caused by this email or any attachments.
