Edward, did you have HIVE-543 patch merged in your Hive? That patch revolves an issue of OOM in the hive client side.
On Aug 4, 2010, at 9:22 AM, Edward Capriolo wrote: > On Wed, Aug 4, 2010 at 12:15 PM, lei liu <[email protected]> wrote: >> Hello Edward Capriolo, >> >> Thank you for your reply. Are you sure that if you string enough 'or' >> together (say 8000) the query parser which uses java beans serialization >> will OOM? How many memory you assign to hive? >> >> 2010/8/4 Edward Capriolo <[email protected]> >>> >>> On Wed, Aug 4, 2010 at 6:10 AM, lei liu <[email protected]> wrote: >>>> Because my company reuire we use 0.4.1 version, the version don't >>>> support IN >>>> clause. I want to use the OR clause(example:where id=1 or id=2 or id=3) >>>> to >>>> implement the IN clause(example: id in(1,2,3) ). I know it will be >>>> slower >>>> especially when the list after "in" is very long. Could anybody can >>>> tell me >>>> why is slow when use OR clause to implement In clause? >>>> >>>> >>>> Thanks, >>>> >>>> >>>> LiuLei >>>> >>> >>> I can not imagine the performance difference between 'or' or 'in' >>> would be that great but I never benchmarked it. The big looming >>> problems is that if you string enough 'or' together (say 8000) the >>> query parser which uses java beans serialization will OOM. >>> >>> Edward >> >> > > That is exactly what I am saying. I tested with 4GB and 8GB. I am not > exactly sure how many OR's you can get away with for your memory size, > but some upper limit exists currently. Most people never hit it. (I > did because my middle name is "edge case" )
