Yup, I've turned it down to 1,000. Let's see if that helps! On Sun, Sep 26, 2010 at 5:09 PM, Chris K Wensel <[email protected]> wrote: > Try using a lower threshold value (the num of values in the LRU to cache). > this is the tradeoff of this approach. > > ckw > > On Sep 26, 2010, at 4:46 PM, Bradford Stephens wrote: > >> Sadly, making Chris's changes didn't help. >> >> Here's the Cascading code, it's pretty simple but uses the new >> "combiner"-like functionality: >> >> http://pastebin.com/ccvDmLSX >> >> >> >> On Sun, Sep 26, 2010 at 9:37 AM, Ted Dunning <[email protected]> wrote: >>> My feeling is that you have some kind of leak going on in your mappers or >>> reducers and that reducing the number of times the jvm is re-used would >>> improve matters. >>> >>> GC overhead limit indicates that your (tiny) heap is full and collection is >>> not reducing that. >>> >>> On Sun, Sep 26, 2010 at 12:55 AM, Bradford Stephens < >>> [email protected]> wrote: >>> >>>> mapred.job.reuse.jvm.num.tasks=50 >>>> >>> >> >> >> >> -- >> Bradford Stephens, >> Founder, Drawn to Scale >> drawntoscalehq.com >> 727.697.7528 >> >> http://www.drawntoscalehq.com -- The intuitive, cloud-scale data >> solution. Process, store, query, search, and serve all your data. >> >> http://www.roadtofailure.com -- The Fringes of Scalability, Social >> Media, and Computer Science >> >> -- >> You received this message because you are subscribed to the Google Groups >> "cascading-user" group. >> To post to this group, send email to [email protected]. >> To unsubscribe from this group, send email to >> [email protected]. >> For more options, visit this group at >> http://groups.google.com/group/cascading-user?hl=en. >> > > -- > Chris K Wensel > [email protected] > http://www.concurrentinc.com > > -- Concurrent, Inc. offers mentoring, support, and licensing for Cascading > >
-- Bradford Stephens, Founder, Drawn to Scale drawntoscalehq.com 727.697.7528 http://www.drawntoscalehq.com -- The intuitive, cloud-scale data solution. Process, store, query, search, and serve all your data. http://www.roadtofailure.com -- The Fringes of Scalability, Social Media, and Computer Science
