Daniel, iirc spill requests are triggered by a gc, and spill_count is triggered by an actual spill, so the former number may be a bit misleading (if gc is effective, lots of gcs might be fine).
D On Wed, Aug 3, 2011 at 10:12 AM, Daniel Dai <da...@hortonworks.com> wrote: > Spill means Pig need to dump memory into disk. It happens when Pig > deals with a large key, and Pig run short of memory. The high number > indicates Pig need to write to disk frequently and performance may > downgrade, and you may explore approach, such as using skewed join. > > Daniel > > On Tue, Aug 2, 2011 at 11:43 AM, Sean Barry > <sba...@cricketcommunications.com> wrote: > > org.apache.pig.PigCounters > > > > PROACTIVE_SPILL_COUNT_RECS > > > > 0 > > > > 2,372,598 > > > > 2,372,598 > > > > SPILLABLE_MEMORY_MANAGER_SPILL_COUNT > > > > 0 > > > > 64 > > > > 64 > > > > PROACTIVE_SPILL_COUNT_BAGS > > > > > > > > I was checking my jobtracker and I have no idea what these three counters > are representative of... > > Can anyone shed some light, please? > > > > -SB > > >
