RE: Drill over Hive
Aman I opened by your request the jira bug https://issues.apache.org/jira/browse/DRILL-6033 FYI Regards, Dor -Original Message- From: Dor Ben Dov Sent: יום ה 14 דצמבר 2017 09:57 To: dev@drill.apache.org Subject: RE: Drill over Hive Aman Can you give me the link to jira so I will be able to file the bug. Regards, Dor -Original Message- From: Aman Sinha [mailto:amansi...@apache.org] Sent: יום ד 13 דצמבר 2017 19:46 To: dev@drill.apache.org Subject: Re: Drill over Hive I don't think we test Drill in this 3-tier configuration .. i.e Drill-->Hive-->Hbase. It is mainly Drill-->Hive or Drill-->Hbase. We would need to investigate why the filter pushdown did not get completely propagated down to Hbase.Can you pls file a JIRA ? -Aman On Wed, Dec 13, 2017 at 4:00 AM, Dor Ben Dov <dor.ben-...@amdocs.com> wrote: > Kunal > Correct this is the exact issue, it looks like drill is trying to push > while hive doesn't pass it. > When you work without hive, it works fine. > + > As result - you get very bad performance. > > Regards, > Dor > > -Original Message- > From: Kunal Khatua [mailto:kkha...@mapr.com] > Sent: יום ד 13 דצמבר 2017 09:43 > To: dev@drill.apache.org > Subject: RE: Drill over Hive > > It appears you are trying to use Drill to query an Hbase via Hive. It > is possible that Drill is not able to push down the filters to Hive, > though I'm not sure why. > > Aman/Chunhui > Any idea why is Drill triggering a full table scan? > > From: Dor Ben Dov [mailto:dor.ben-...@amdocs.com] > Sent: Tuesday, December 12, 2017 11:39 PM > To: dev@drill.apache.org > Subject: Drill over Hive > > Hi > > This is our query > select * from hive.mytable where key >= > '0001:10:2017:0410:3157781' > and key < '0001:10:2017:0410:3157782'; > > Same query in Drill over hbase takes less than a sec, in hive it takes > few seconds and with drill over hive it quits with a timeout. > When you look in the sql profile of drill (using the web), you see a > table full scan for millions of records, while actually it was > supposed to return > 9 rows. > > Does Drill on top of hive is using the key to access only the relevant > region of the table? > > Regards, > Dor Ben Dov > Research Manager, Digital - BSS - Research > +972-9-7764043 (office) > +972-52-3573492 (mobile) > [cid:image001.png@01D28AC9.B559B360] > > Follow us on Facebook<https://urldefense.proofpoint.com/v2/url?u=http- > 3A__www.facebook.com_amdocs_=DwMFAg=cskdkSMqhcnjZxdQVpwTXg=- > cT6otg6lpT_XkmYy7yg3A=DAE0fF7_P8TSmHMGxIfhw27jKB1kOw1D8qaGAn > 6q32M=bS0vtglEQKbSVkT6_3T5N1ckKa-48J2rFdz0A-GkFxg=>, Twitter< > https://urldefense.proofpoint.com/v2/url?u=http-3A__twitter.com_amdocs > = > DwMFAg=cskdkSMqhcnjZxdQVpwTXg=-cT6otg6lpT_XkmYy7yg3A=DAE0fF7_ > P8TSmHMGxIfhw27jKB1kOw1D8qaGAn6q32M=HwJx9gvHHvHV9u0nW_ > 0dLjEPhSGcfsWLH-iTUFNV6mc=>, LinkedIn<https://urldefense. > proofpoint.com/v2/url?u=http-3A__www.linkedin.com_company_ > amdocs=DwMFAg=cskdkSMqhcnjZxdQVpwTXg=-cT6otg6lpT_XkmYy7yg3A= > DAE0fF7_P8TSmHMGxIfhw27jKB1kOw1D8qaGAn6q32M= > cjcmCjbbkgov6zg8GJnuJOOAm6G9HTtY4ppZ170IDb4=>, YouTube< > https://urldefense.proofpoint.com/v2/url?u=http- > 3A__www.youtube.com_amdocsinc=DwMFAg=cskdkSMqhcnjZxdQVpwTXg=- > cT6otg6lpT_XkmYy7yg3A=DAE0fF7_P8TSmHMGxIfhw27jKB1kOw1D8qaGAn6q32M= > fp9qmbPyYRdiNYDwUDH44HUALaqsbKvfWBv_Dcy7UAo=>, Google+< > https://urldefense.proofpoint.com/v2/url?u=https-3A__plus.google.com_ > 105657940751678445194=DwMFAg=cskdkSMqhcnjZxdQVpwTXg=- > cT6otg6lpT_XkmYy7yg3A=DAE0fF7_P8TSmHMGxIfhw27jKB1kOw1D8qaGAn > 6q32M=n0coUEV3Sf7O_MbrqI7cAr5f6d7OG_vL1jK2n3z-AvM=> and the Amdocs > blog network<https://urldefense.proofpoint.com/v2/url?u=http- > 3A__blogs.amdocs.com_=DwMFAg=cskdkSMqhcnjZxdQVpwTXg=- > cT6otg6lpT_XkmYy7yg3A=DAE0fF7_P8TSmHMGxIfhw27jKB1kOw1D8qaGAn > 6q32M=RxKGDS-92_svSBQlnTdpn8NSr3-1A9Mcp7UbmZsv1bE=>| Digital BSS > Research@Yammer<https://urldefense.proofpoint.com/v2/ > url?u=https-3A__www.yammer.com_amdocs.com_-23_threads_ > inGroup-3Ftype-3Din-5Fgroup-26feedId-3D7043838=DwMFAg= > cskdkSMqhcnjZxdQVpwTXg=-cT6otg6lpT_XkmYy7yg3A=DAE0fF7_ > P8TSmHMGxIfhw27jKB1kOw1D8qaGAn6q32M=A20YnBgQgpBLzoXN- > rOK49KCStW9haqs7atynBoQz-s=> > > This message and the information contained herein is proprietary and > confidential and subject to the Amdocs policy statement, you may > review at https://www.amdocs.com/about/email-disclaimer<https:// > urldefense.proofpoint.com/v2/url?u=https-3A__www.amdocs. > com_about_email-2Ddisclaimer=DwMFAg=cskdkSMqhcnjZxdQVpwTXg=- > cT6otg6lpT_XkmYy7yg3A=DAE0fF7_P8TSmHMGxIfhw27jKB1kOw1D8qaGAn6q32M= > xqkKqSeAK9HGozYmTy7lO0eKTkY4CG5npyWKUM9XeAY=> &g
RE: Drill over Hive
Use https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=185=DRILL=1039 You need to click on the red Create button at the top to create the JIRA. Please provide as much detail as you can of the issue, including the version numbers of the components, (e.g. Drill, Hbase, Hadoop, etc). -Original Message- From: Dor Ben Dov [mailto:dor.ben-...@amdocs.com] Sent: Wednesday, December 13, 2017 11:57 PM To: dev@drill.apache.org Subject: RE: Drill over Hive Aman Can you give me the link to jira so I will be able to file the bug. Regards, Dor -Original Message- From: Aman Sinha [mailto:amansi...@apache.org] Sent: יום ד 13 דצמבר 2017 19:46 To: dev@drill.apache.org Subject: Re: Drill over Hive I don't think we test Drill in this 3-tier configuration .. i.e Drill-->Hive-->Hbase. It is mainly Drill-->Hive or Drill-->Hbase. We would need to investigate why the filter pushdown did not get completely propagated down to Hbase.Can you pls file a JIRA ? -Aman On Wed, Dec 13, 2017 at 4:00 AM, Dor Ben Dov <dor.ben-...@amdocs.com> wrote: > Kunal > Correct this is the exact issue, it looks like drill is trying to push > while hive doesn't pass it. > When you work without hive, it works fine. > + > As result - you get very bad performance. > > Regards, > Dor > > -Original Message- > From: Kunal Khatua [mailto:kkha...@mapr.com] > Sent: יום ד 13 דצמבר 2017 09:43 > To: dev@drill.apache.org > Subject: RE: Drill over Hive > > It appears you are trying to use Drill to query an Hbase via Hive. It > is possible that Drill is not able to push down the filters to Hive, > though I'm not sure why. > > Aman/Chunhui > Any idea why is Drill triggering a full table scan? > > From: Dor Ben Dov [mailto:dor.ben-...@amdocs.com] > Sent: Tuesday, December 12, 2017 11:39 PM > To: dev@drill.apache.org > Subject: Drill over Hive > > Hi > > This is our query > select * from hive.mytable where key >= > '0001:10:2017:0410:3157781' > and key < '0001:10:2017:0410:3157782'; > > Same query in Drill over hbase takes less than a sec, in hive it takes > few seconds and with drill over hive it quits with a timeout. > When you look in the sql profile of drill (using the web), you see a > table full scan for millions of records, while actually it was > supposed to return > 9 rows. > > Does Drill on top of hive is using the key to access only the relevant > region of the table? > > Regards, > Dor Ben Dov > Research Manager, Digital - BSS - Research > +972-9-7764043 (office) > +972-52-3573492 (mobile) > [cid:image001.png@01D28AC9.B559B360] > > Follow us on Facebook<https://urldefense.proofpoint.com/v2/url?u=http- > 3A__www.facebook.com_amdocs_=DwMFAg=cskdkSMqhcnjZxdQVpwTXg=- > cT6otg6lpT_XkmYy7yg3A=DAE0fF7_P8TSmHMGxIfhw27jKB1kOw1D8qaGAn > 6q32M=bS0vtglEQKbSVkT6_3T5N1ckKa-48J2rFdz0A-GkFxg=>, Twitter< > https://urldefense.proofpoint.com/v2/url?u=http-3A__twitter.com_amdocs > = > DwMFAg=cskdkSMqhcnjZxdQVpwTXg=-cT6otg6lpT_XkmYy7yg3A=DAE0fF7_ > P8TSmHMGxIfhw27jKB1kOw1D8qaGAn6q32M=HwJx9gvHHvHV9u0nW_ > 0dLjEPhSGcfsWLH-iTUFNV6mc=>, > LinkedIn<https://urldefense.proofpoint.com/v2/url?u=https-3A__urldefense=DwIGaQ=cskdkSMqhcnjZxdQVpwTXg=-cT6otg6lpT_XkmYy7yg3A=uWQC-v7AY6D84a58sK9tkykiGsF-OB22e_QbM4Aw1pw=NrK7-hvnHqwvLKK08mRHFoesNPQPoAoNh_zSG-1hZu8=. > proofpoint.com/v2/url?u=http-3A__www.linkedin.com_company_ > amdocs=DwMFAg=cskdkSMqhcnjZxdQVpwTXg=-cT6otg6lpT_XkmYy7yg3A= > DAE0fF7_P8TSmHMGxIfhw27jKB1kOw1D8qaGAn6q32M= > cjcmCjbbkgov6zg8GJnuJOOAm6G9HTtY4ppZ170IDb4=>, YouTube< > https://urldefense.proofpoint.com/v2/url?u=http- > 3A__www.youtube.com_amdocsinc=DwMFAg=cskdkSMqhcnjZxdQVpwTXg=- > cT6otg6lpT_XkmYy7yg3A=DAE0fF7_P8TSmHMGxIfhw27jKB1kOw1D8qaGAn6q32M= > fp9qmbPyYRdiNYDwUDH44HUALaqsbKvfWBv_Dcy7UAo=>, Google+< > https://urldefense.proofpoint.com/v2/url?u=https-3A__plus.google.com_ > 105657940751678445194=DwMFAg=cskdkSMqhcnjZxdQVpwTXg=- > cT6otg6lpT_XkmYy7yg3A=DAE0fF7_P8TSmHMGxIfhw27jKB1kOw1D8qaGAn > 6q32M=n0coUEV3Sf7O_MbrqI7cAr5f6d7OG_vL1jK2n3z-AvM=> and the Amdocs > blog network<https://urldefense.proofpoint.com/v2/url?u=http- > 3A__blogs.amdocs.com_=DwMFAg=cskdkSMqhcnjZxdQVpwTXg=- > cT6otg6lpT_XkmYy7yg3A=DAE0fF7_P8TSmHMGxIfhw27jKB1kOw1D8qaGAn > 6q32M=RxKGDS-92_svSBQlnTdpn8NSr3-1A9Mcp7UbmZsv1bE=>| Digital BSS > Research@Yammer<https://urldefense.proofpoint.com/v2/ > url?u=https-3A__www.yammer.com_amdocs.com_-23_threads_ > inGroup-3Ftype-3Din-5Fgroup-26feedId-3D7043838=DwMFAg= > cskdkSMqhcnjZxdQVpwTXg=-cT6otg6lpT_XkmYy7yg3A=DAE0fF7_ > P8TSmHMGxIfhw27jKB1kOw1D8qaGAn6q32
RE: Drill over Hive
Aman Can you give me the link to jira so I will be able to file the bug. Regards, Dor -Original Message- From: Aman Sinha [mailto:amansi...@apache.org] Sent: יום ד 13 דצמבר 2017 19:46 To: dev@drill.apache.org Subject: Re: Drill over Hive I don't think we test Drill in this 3-tier configuration .. i.e Drill-->Hive-->Hbase. It is mainly Drill-->Hive or Drill-->Hbase. We would need to investigate why the filter pushdown did not get completely propagated down to Hbase.Can you pls file a JIRA ? -Aman On Wed, Dec 13, 2017 at 4:00 AM, Dor Ben Dov <dor.ben-...@amdocs.com> wrote: > Kunal > Correct this is the exact issue, it looks like drill is trying to push > while hive doesn't pass it. > When you work without hive, it works fine. > + > As result - you get very bad performance. > > Regards, > Dor > > -Original Message- > From: Kunal Khatua [mailto:kkha...@mapr.com] > Sent: יום ד 13 דצמבר 2017 09:43 > To: dev@drill.apache.org > Subject: RE: Drill over Hive > > It appears you are trying to use Drill to query an Hbase via Hive. It > is possible that Drill is not able to push down the filters to Hive, > though I'm not sure why. > > Aman/Chunhui > Any idea why is Drill triggering a full table scan? > > From: Dor Ben Dov [mailto:dor.ben-...@amdocs.com] > Sent: Tuesday, December 12, 2017 11:39 PM > To: dev@drill.apache.org > Subject: Drill over Hive > > Hi > > This is our query > select * from hive.mytable where key >= > '0001:10:2017:0410:3157781' > and key < '0001:10:2017:0410:3157782'; > > Same query in Drill over hbase takes less than a sec, in hive it takes > few seconds and with drill over hive it quits with a timeout. > When you look in the sql profile of drill (using the web), you see a > table full scan for millions of records, while actually it was > supposed to return > 9 rows. > > Does Drill on top of hive is using the key to access only the relevant > region of the table? > > Regards, > Dor Ben Dov > Research Manager, Digital - BSS - Research > +972-9-7764043 (office) > +972-52-3573492 (mobile) > [cid:image001.png@01D28AC9.B559B360] > > Follow us on Facebook<https://urldefense.proofpoint.com/v2/url?u=http- > 3A__www.facebook.com_amdocs_=DwMFAg=cskdkSMqhcnjZxdQVpwTXg=- > cT6otg6lpT_XkmYy7yg3A=DAE0fF7_P8TSmHMGxIfhw27jKB1kOw1D8qaGAn > 6q32M=bS0vtglEQKbSVkT6_3T5N1ckKa-48J2rFdz0A-GkFxg=>, Twitter< > https://urldefense.proofpoint.com/v2/url?u=http-3A__twitter.com_amdocs > = > DwMFAg=cskdkSMqhcnjZxdQVpwTXg=-cT6otg6lpT_XkmYy7yg3A=DAE0fF7_ > P8TSmHMGxIfhw27jKB1kOw1D8qaGAn6q32M=HwJx9gvHHvHV9u0nW_ > 0dLjEPhSGcfsWLH-iTUFNV6mc=>, LinkedIn<https://urldefense. > proofpoint.com/v2/url?u=http-3A__www.linkedin.com_company_ > amdocs=DwMFAg=cskdkSMqhcnjZxdQVpwTXg=-cT6otg6lpT_XkmYy7yg3A= > DAE0fF7_P8TSmHMGxIfhw27jKB1kOw1D8qaGAn6q32M= > cjcmCjbbkgov6zg8GJnuJOOAm6G9HTtY4ppZ170IDb4=>, YouTube< > https://urldefense.proofpoint.com/v2/url?u=http- > 3A__www.youtube.com_amdocsinc=DwMFAg=cskdkSMqhcnjZxdQVpwTXg=- > cT6otg6lpT_XkmYy7yg3A=DAE0fF7_P8TSmHMGxIfhw27jKB1kOw1D8qaGAn6q32M= > fp9qmbPyYRdiNYDwUDH44HUALaqsbKvfWBv_Dcy7UAo=>, Google+< > https://urldefense.proofpoint.com/v2/url?u=https-3A__plus.google.com_ > 105657940751678445194=DwMFAg=cskdkSMqhcnjZxdQVpwTXg=- > cT6otg6lpT_XkmYy7yg3A=DAE0fF7_P8TSmHMGxIfhw27jKB1kOw1D8qaGAn > 6q32M=n0coUEV3Sf7O_MbrqI7cAr5f6d7OG_vL1jK2n3z-AvM=> and the Amdocs > blog network<https://urldefense.proofpoint.com/v2/url?u=http- > 3A__blogs.amdocs.com_=DwMFAg=cskdkSMqhcnjZxdQVpwTXg=- > cT6otg6lpT_XkmYy7yg3A=DAE0fF7_P8TSmHMGxIfhw27jKB1kOw1D8qaGAn > 6q32M=RxKGDS-92_svSBQlnTdpn8NSr3-1A9Mcp7UbmZsv1bE=>| Digital BSS > Research@Yammer<https://urldefense.proofpoint.com/v2/ > url?u=https-3A__www.yammer.com_amdocs.com_-23_threads_ > inGroup-3Ftype-3Din-5Fgroup-26feedId-3D7043838=DwMFAg= > cskdkSMqhcnjZxdQVpwTXg=-cT6otg6lpT_XkmYy7yg3A=DAE0fF7_ > P8TSmHMGxIfhw27jKB1kOw1D8qaGAn6q32M=A20YnBgQgpBLzoXN- > rOK49KCStW9haqs7atynBoQz-s=> > > This message and the information contained herein is proprietary and > confidential and subject to the Amdocs policy statement, you may > review at https://www.amdocs.com/about/email-disclaimer<https:// > urldefense.proofpoint.com/v2/url?u=https-3A__www.amdocs. > com_about_email-2Ddisclaimer=DwMFAg=cskdkSMqhcnjZxdQVpwTXg=- > cT6otg6lpT_XkmYy7yg3A=DAE0fF7_P8TSmHMGxIfhw27jKB1kOw1D8qaGAn6q32M= > xqkKqSeAK9HGozYmTy7lO0eKTkY4CG5npyWKUM9XeAY=> > This message and the information contained herein is proprietary and > confidential and subject to the Amdocs policy statement, > > you may review at https://www.amdocs.com/about/email-disclaimer < > https://www.amdocs.com/about/email-disclaimer> > This message and the information contained herein is proprietary and confidential and subject to the Amdocs policy statement, you may review at https://www.amdocs.com/about/email-disclaimer <https://www.amdocs.com/about/email-disclaimer>
Re: Drill over Hive
I don't think we test Drill in this 3-tier configuration .. i.e Drill-->Hive-->Hbase. It is mainly Drill-->Hive or Drill-->Hbase. We would need to investigate why the filter pushdown did not get completely propagated down to Hbase.Can you pls file a JIRA ? -Aman On Wed, Dec 13, 2017 at 4:00 AM, Dor Ben Dov <dor.ben-...@amdocs.com> wrote: > Kunal > Correct this is the exact issue, it looks like drill is trying to push > while hive doesn't pass it. > When you work without hive, it works fine. > + > As result - you get very bad performance. > > Regards, > Dor > > -Original Message- > From: Kunal Khatua [mailto:kkha...@mapr.com] > Sent: יום ד 13 דצמבר 2017 09:43 > To: dev@drill.apache.org > Subject: RE: Drill over Hive > > It appears you are trying to use Drill to query an Hbase via Hive. It is > possible that Drill is not able to push down the filters to Hive, though > I'm not sure why. > > Aman/Chunhui > Any idea why is Drill triggering a full table scan? > > From: Dor Ben Dov [mailto:dor.ben-...@amdocs.com] > Sent: Tuesday, December 12, 2017 11:39 PM > To: dev@drill.apache.org > Subject: Drill over Hive > > Hi > > This is our query > select * from hive.mytable where key >= > '0001:10:2017:0410:3157781' > and key < '0001:10:2017:0410:3157782'; > > Same query in Drill over hbase takes less than a sec, in hive it takes few > seconds and with drill over hive it quits with a timeout. > When you look in the sql profile of drill (using the web), you see a table > full scan for millions of records, while actually it was supposed to return > 9 rows. > > Does Drill on top of hive is using the key to access only the relevant > region of the table? > > Regards, > Dor Ben Dov > Research Manager, Digital - BSS - Research > +972-9-7764043 (office) > +972-52-3573492 (mobile) > [cid:image001.png@01D28AC9.B559B360] > > Follow us on Facebook<https://urldefense.proofpoint.com/v2/url?u=http- > 3A__www.facebook.com_amdocs_=DwMFAg=cskdkSMqhcnjZxdQVpwTXg=- > cT6otg6lpT_XkmYy7yg3A=DAE0fF7_P8TSmHMGxIfhw27jKB1kOw1D8qaGAn > 6q32M=bS0vtglEQKbSVkT6_3T5N1ckKa-48J2rFdz0A-GkFxg=>, Twitter< > https://urldefense.proofpoint.com/v2/url?u=http-3A__twitter.com_amdocs= > DwMFAg=cskdkSMqhcnjZxdQVpwTXg=-cT6otg6lpT_XkmYy7yg3A=DAE0fF7_ > P8TSmHMGxIfhw27jKB1kOw1D8qaGAn6q32M=HwJx9gvHHvHV9u0nW_ > 0dLjEPhSGcfsWLH-iTUFNV6mc=>, LinkedIn<https://urldefense. > proofpoint.com/v2/url?u=http-3A__www.linkedin.com_company_ > amdocs=DwMFAg=cskdkSMqhcnjZxdQVpwTXg=-cT6otg6lpT_XkmYy7yg3A= > DAE0fF7_P8TSmHMGxIfhw27jKB1kOw1D8qaGAn6q32M= > cjcmCjbbkgov6zg8GJnuJOOAm6G9HTtY4ppZ170IDb4=>, YouTube< > https://urldefense.proofpoint.com/v2/url?u=http- > 3A__www.youtube.com_amdocsinc=DwMFAg=cskdkSMqhcnjZxdQVpwTXg=- > cT6otg6lpT_XkmYy7yg3A=DAE0fF7_P8TSmHMGxIfhw27jKB1kOw1D8qaGAn6q32M= > fp9qmbPyYRdiNYDwUDH44HUALaqsbKvfWBv_Dcy7UAo=>, Google+< > https://urldefense.proofpoint.com/v2/url?u=https-3A__plus.google.com_ > 105657940751678445194=DwMFAg=cskdkSMqhcnjZxdQVpwTXg=- > cT6otg6lpT_XkmYy7yg3A=DAE0fF7_P8TSmHMGxIfhw27jKB1kOw1D8qaGAn > 6q32M=n0coUEV3Sf7O_MbrqI7cAr5f6d7OG_vL1jK2n3z-AvM=> and the Amdocs > blog network<https://urldefense.proofpoint.com/v2/url?u=http- > 3A__blogs.amdocs.com_=DwMFAg=cskdkSMqhcnjZxdQVpwTXg=- > cT6otg6lpT_XkmYy7yg3A=DAE0fF7_P8TSmHMGxIfhw27jKB1kOw1D8qaGAn > 6q32M=RxKGDS-92_svSBQlnTdpn8NSr3-1A9Mcp7UbmZsv1bE=>| Digital BSS > Research@Yammer<https://urldefense.proofpoint.com/v2/ > url?u=https-3A__www.yammer.com_amdocs.com_-23_threads_ > inGroup-3Ftype-3Din-5Fgroup-26feedId-3D7043838=DwMFAg= > cskdkSMqhcnjZxdQVpwTXg=-cT6otg6lpT_XkmYy7yg3A=DAE0fF7_ > P8TSmHMGxIfhw27jKB1kOw1D8qaGAn6q32M=A20YnBgQgpBLzoXN- > rOK49KCStW9haqs7atynBoQz-s=> > > This message and the information contained herein is proprietary and > confidential and subject to the Amdocs policy statement, you may review at > https://www.amdocs.com/about/email-disclaimer<https:// > urldefense.proofpoint.com/v2/url?u=https-3A__www.amdocs. > com_about_email-2Ddisclaimer=DwMFAg=cskdkSMqhcnjZxdQVpwTXg=- > cT6otg6lpT_XkmYy7yg3A=DAE0fF7_P8TSmHMGxIfhw27jKB1kOw1D8qaGAn6q32M= > xqkKqSeAK9HGozYmTy7lO0eKTkY4CG5npyWKUM9XeAY=> > This message and the information contained herein is proprietary and > confidential and subject to the Amdocs policy statement, > > you may review at https://www.amdocs.com/about/email-disclaimer < > https://www.amdocs.com/about/email-disclaimer> >
RE: Drill over Hive
Kunal Correct this is the exact issue, it looks like drill is trying to push while hive doesn't pass it. When you work without hive, it works fine. + As result - you get very bad performance. Regards, Dor -Original Message- From: Kunal Khatua [mailto:kkha...@mapr.com] Sent: יום ד 13 דצמבר 2017 09:43 To: dev@drill.apache.org Subject: RE: Drill over Hive It appears you are trying to use Drill to query an Hbase via Hive. It is possible that Drill is not able to push down the filters to Hive, though I'm not sure why. Aman/Chunhui Any idea why is Drill triggering a full table scan? From: Dor Ben Dov [mailto:dor.ben-...@amdocs.com] Sent: Tuesday, December 12, 2017 11:39 PM To: dev@drill.apache.org Subject: Drill over Hive Hi This is our query select * from hive.mytable where key >= '0001:10:2017:0410:3157781' and key < '0001:10:2017:0410:3157782'; Same query in Drill over hbase takes less than a sec, in hive it takes few seconds and with drill over hive it quits with a timeout. When you look in the sql profile of drill (using the web), you see a table full scan for millions of records, while actually it was supposed to return 9 rows. Does Drill on top of hive is using the key to access only the relevant region of the table? Regards, Dor Ben Dov Research Manager, Digital - BSS - Research +972-9-7764043 (office) +972-52-3573492 (mobile) [cid:image001.png@01D28AC9.B559B360] Follow us on Facebook<https://urldefense.proofpoint.com/v2/url?u=http-3A__www.facebook.com_amdocs_=DwMFAg=cskdkSMqhcnjZxdQVpwTXg=-cT6otg6lpT_XkmYy7yg3A=DAE0fF7_P8TSmHMGxIfhw27jKB1kOw1D8qaGAn6q32M=bS0vtglEQKbSVkT6_3T5N1ckKa-48J2rFdz0A-GkFxg=>, Twitter<https://urldefense.proofpoint.com/v2/url?u=http-3A__twitter.com_amdocs=DwMFAg=cskdkSMqhcnjZxdQVpwTXg=-cT6otg6lpT_XkmYy7yg3A=DAE0fF7_P8TSmHMGxIfhw27jKB1kOw1D8qaGAn6q32M=HwJx9gvHHvHV9u0nW_0dLjEPhSGcfsWLH-iTUFNV6mc=>, LinkedIn<https://urldefense.proofpoint.com/v2/url?u=http-3A__www.linkedin.com_company_amdocs=DwMFAg=cskdkSMqhcnjZxdQVpwTXg=-cT6otg6lpT_XkmYy7yg3A=DAE0fF7_P8TSmHMGxIfhw27jKB1kOw1D8qaGAn6q32M=cjcmCjbbkgov6zg8GJnuJOOAm6G9HTtY4ppZ170IDb4=>, YouTube<https://urldefense.proofpoint.com/v2/url?u=http-3A__www.youtube.com_amdocsinc=DwMFAg=cskdkSMqhcnjZxdQVpwTXg=-cT6otg6lpT_XkmYy7yg3A=DAE0fF7_P8TSmHMGxIfhw27jKB1kOw1D8qaGAn6q32M=fp9qmbPyYRdiNYDwUDH44HUALaqsbKvfWBv_Dcy7UAo=>, Google+<https://urldefense.proofpoint.com/v2/url?u=https-3A__plus.google.com_105657940751678445194=DwMFAg=cskdkSMqhcnjZxdQVpwTXg=-cT6otg6lpT_XkmYy7yg3A=DAE0fF7_P8TSmHMGxIfhw27jKB1kOw1D8qaGAn6q32M=n0coUEV3Sf7O_MbrqI7cAr5f6d7OG_vL1jK2n3z-AvM=> and the Amdocs blog network<https://urldefense.proofpoint.com/v2/url?u=http-3A__blogs.amdocs.com_=DwMFAg=cskdkSMqhcnjZxdQVpwTXg=-cT6otg6lpT_XkmYy7yg3A=DAE0fF7_P8TSmHMGxIfhw27jKB1kOw1D8qaGAn6q32M=RxKGDS-92_svSBQlnTdpn8NSr3-1A9Mcp7UbmZsv1bE=>| Digital BSS Research@Yammer<https://urldefense.proofpoint.com/v2/url?u=https-3A__www.yammer.com_amdocs.com_-23_threads_inGroup-3Ftype-3Din-5Fgroup-26feedId-3D7043838=DwMFAg=cskdkSMqhcnjZxdQVpwTXg=-cT6otg6lpT_XkmYy7yg3A=DAE0fF7_P8TSmHMGxIfhw27jKB1kOw1D8qaGAn6q32M=A20YnBgQgpBLzoXN-rOK49KCStW9haqs7atynBoQz-s=> This message and the information contained herein is proprietary and confidential and subject to the Amdocs policy statement, you may review at https://www.amdocs.com/about/email-disclaimer<https://urldefense.proofpoint.com/v2/url?u=https-3A__www.amdocs.com_about_email-2Ddisclaimer=DwMFAg=cskdkSMqhcnjZxdQVpwTXg=-cT6otg6lpT_XkmYy7yg3A=DAE0fF7_P8TSmHMGxIfhw27jKB1kOw1D8qaGAn6q32M=xqkKqSeAK9HGozYmTy7lO0eKTkY4CG5npyWKUM9XeAY=> This message and the information contained herein is proprietary and confidential and subject to the Amdocs policy statement, you may review at https://www.amdocs.com/about/email-disclaimer <https://www.amdocs.com/about/email-disclaimer>
