Spill means Pig need to dump memory into disk. It happens when Pig deals with a large key, and Pig run short of memory. The high number indicates Pig need to write to disk frequently and performance may downgrade, and you may explore approach, such as using skewed join.
Daniel On Tue, Aug 2, 2011 at 11:43 AM, Sean Barry <sba...@cricketcommunications.com> wrote: > org.apache.pig.PigCounters > > PROACTIVE_SPILL_COUNT_RECS > > 0 > > 2,372,598 > > 2,372,598 > > SPILLABLE_MEMORY_MANAGER_SPILL_COUNT > > 0 > > 64 > > 64 > > PROACTIVE_SPILL_COUNT_BAGS > > > > I was checking my jobtracker and I have no idea what these three counters are > representative of... > Can anyone shed some light, please? > > -SB >
