Collect number of spills per job
--------------------------------

                 Key: PIG-1102
                 URL: https://issues.apache.org/jira/browse/PIG-1102
             Project: Pig
          Issue Type: Improvement
            Reporter: Olga Natkovich
            Assignee: Sriranjan Manjunath
             Fix For: 0.7.0


Memory shortage is one of the main performance issues in Pig. Knowing when we 
spill do the disk is useful for understanding query performance and also to see 
how certain changes in Pig effect that.

Other interesting stats to collect would be average CPU usage and max mem usage 
but I am not sure if this information is easily retrievable.

Using Hadoop counters for this would make sense.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply via email to