On Wed, Aug 4, 2010 at 12:50 PM, Ning Zhang <[email protected]> wrote: > Edward, did you have HIVE-543 patch merged in your Hive? That patch revolves > an issue of OOM in the hive client side. > > On Aug 4, 2010, at 9:22 AM, Edward Capriolo wrote: > >> On Wed, Aug 4, 2010 at 12:15 PM, lei liu <[email protected]> wrote: >>> Hello Edward Capriolo, >>> >>> Thank you for your reply. Are you sure that if you string enough 'or' >>> together (say 8000) the query parser which uses java beans serialization >>> will OOM? How many memory you assign to hive? >>> >>> 2010/8/4 Edward Capriolo <[email protected]> >>>> >>>> On Wed, Aug 4, 2010 at 6:10 AM, lei liu <[email protected]> wrote: >>>>> Because my company reuire we use 0.4.1 version, the version don't >>>>> support IN >>>>> clause. I want to use the OR clause(example:where id=1 or id=2 or id=3) >>>>> to >>>>> implement the IN clause(example: id in(1,2,3) ). I know it will be >>>>> slower >>>>> especially when the list after "in" is very long. Could anybody can >>>>> tell me >>>>> why is slow when use OR clause to implement In clause? >>>>> >>>>> >>>>> Thanks, >>>>> >>>>> >>>>> LiuLei >>>>> >>>> >>>> I can not imagine the performance difference between 'or' or 'in' >>>> would be that great but I never benchmarked it. The big looming >>>> problems is that if you string enough 'or' together (say 8000) the >>>> query parser which uses java beans serialization will OOM. >>>> >>>> Edward >>> >>> >> >> That is exactly what I am saying. I tested with 4GB and 8GB. I am not >> exactly sure how many OR's you can get away with for your memory size, >> but some upper limit exists currently. Most people never hit it. (I >> did because my middle name is "edge case" ) > >
No, I do not have HIVE-543 merged in yet. I am not having that problem at the moment but in the past I did. Might not be the case anymore.
