Try using a lower threshold value (the num of values in the LRU to cache). this is the tradeoff of this approach.
ckw On Sep 26, 2010, at 4:46 PM, Bradford Stephens wrote: > Sadly, making Chris's changes didn't help. > > Here's the Cascading code, it's pretty simple but uses the new > "combiner"-like functionality: > > http://pastebin.com/ccvDmLSX > > > > On Sun, Sep 26, 2010 at 9:37 AM, Ted Dunning <[email protected]> wrote: >> My feeling is that you have some kind of leak going on in your mappers or >> reducers and that reducing the number of times the jvm is re-used would >> improve matters. >> >> GC overhead limit indicates that your (tiny) heap is full and collection is >> not reducing that. >> >> On Sun, Sep 26, 2010 at 12:55 AM, Bradford Stephens < >> [email protected]> wrote: >> >>> mapred.job.reuse.jvm.num.tasks=50 >>> >> > > > > -- > Bradford Stephens, > Founder, Drawn to Scale > drawntoscalehq.com > 727.697.7528 > > http://www.drawntoscalehq.com -- The intuitive, cloud-scale data > solution. Process, store, query, search, and serve all your data. > > http://www.roadtofailure.com -- The Fringes of Scalability, Social > Media, and Computer Science > > -- > You received this message because you are subscribed to the Google Groups > "cascading-user" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]. > For more options, visit this group at > http://groups.google.com/group/cascading-user?hl=en. > -- Chris K Wensel [email protected] http://www.concurrentinc.com -- Concurrent, Inc. offers mentoring, support, and licensing for Cascading
