Re: Hive Join Running Out of Memory

2014-07-20 Thread Kristopher Kane
iolo [mailto:edlinuxg...@gmail.com] > Sent: Friday, July 18, 2014 10:57 AM > To: user@hive.apache.org > Subject: Re: Hive Join Running Out of Memory > > I believe that would be the one. > > On Fri, Jul 18, 2014 at 10:54 AM, Clay McDonald < > stuart.mcdon...@bateswhite.com> wr

RE: Hive Join Running Out of Memory

2014-07-18 Thread Clay McDonald
I changed the hive.auto.convert.join.noconditionaltask = false in the hive site and that seemed to do the trick. Thanks! From: Edward Capriolo [mailto:edlinuxg...@gmail.com] Sent: Friday, July 18, 2014 10:57 AM To: user@hive.apache.org Subject: Re: Hive Join Running Out of Memory I believe

Re: Hive Join Running Out of Memory

2014-07-18 Thread Edward Capriolo
mail.com] > Sent: Friday, July 18, 2014 10:45 AM > To: user@hive.apache.org > Subject: Re: Hive Join Running Out of Memory > > This is a failed optimization hive is trying to build the lookup table > locally and then put it in the distributed cache and then to a map join. > Look

RE: Hive Join Running Out of Memory

2014-07-18 Thread Clay McDonald
Thank you. Would it be acceptable to use the following? SET hive.exec.mode.local.auto=false; From: Edward Capriolo [mailto:edlinuxg...@gmail.com] Sent: Friday, July 18, 2014 10:45 AM To: user@hive.apache.org Subject: Re: Hive Join Running Out of Memory This is a failed optimization hive is

Re: Hive Join Running Out of Memory

2014-07-18 Thread Edward Capriolo
This is a failed optimization hive is trying to build the lookup table locally and then put it in the distributed cache and then to a map join. Look through your hive site for the configuration to turn these auto-map joins off. Based on your version the variables changed a names /deprecated etc so

Hive Join Running Out of Memory

2014-07-18 Thread Clay McDonald
Hello everyone. I need some assistance. I have a join that fails with return code 3. The query is; SELECT B.CARD_NBR AS CNT FROM TENDER_TABLE A JOIN LOYALTY_CARDS B ON A.CARD_NBR = B.CARD_NBR LIMIT 10; -- Row Counts -- LOYALTY_CARDS = 43,876,938 -- TENDER_TABLE = 1,412,228,333 The query exe