Dear Tanton, This means the MemoryManager is not successful at reclaiming memory. Did that happen in Map phase or Reduce phase? Did you use a custom chunker? How big is your split?
Pi On Fri, May 23, 2008 at 3:52 PM, Tanton Gibbs <[EMAIL PROTECTED]> wrote: > I upgraded to hadoop 17 and the latest Pig from svn. > > I'm now getting a ton of lines in my log files that say: > > 2008-05-23 00:49:27,832 INFO > org.apache.pig.impl.util.SpillableMemoryManager: low memory handler > called init = 1441792(1408K) used = 483176072(471851K) committed = > 641335296(626304K) max = 954466304(932096K) > > In addition, jobs on big files are running very slowly. > > Does anyone have any ideas as to what I could have screwed up? > > Thanks! > Tanton >
