[jira] Commented: (PIG-831) Records and bytes written reported by pig are wrong in a multi-store program

[Olga Natkovich (JIRA)]

[ 
https://issues.apache.org/jira/browse/PIG-831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12716392#action_12716392
 ] 

Olga Natkovich commented on PIG-831:


+1 on the patch. please, keep the bug open since we should at some point 
correctly report numbers for multiquery


 Records and bytes written reported by pig are wrong in a multi-store program
 

 Key: PIG-831
 URL: https://issues.apache.org/jira/browse/PIG-831
 Project: Pig
  Issue Type: Bug
  Components: impl
Affects Versions: 0.3.0
Reporter: Alan Gates
Assignee: Alan Gates
Priority: Minor
 Attachments: PIG-831.patch


 The stats features checked in as part of PIG-626 (reporting the number of 
 records and bytes written at the end of the query) print wrong values (often 
 but not always 0) when the pig script being run contains more than 1 store.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (PIG-831) Records and bytes written reported by pig are wrong in a multi-store program

[Alan Gates (JIRA)]

[ 
https://issues.apache.org/jira/browse/PIG-831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12715657#action_12715657
 ] 

Alan Gates commented on PIG-831:


There are a couple of issues going on here.

One, PigStats looks through the plan until it finds the first root and then 
stops.  So for multi-store scripts that have multiple roots in their plans, 
this does not work.

Two, Hadoop does not return accurate numbers for records written in many cases. 
 I do not know if this is a bug in hadoop or a bug in the output format pig 
uses when doing multiple stores in one job.

 Records and bytes written reported by pig are wrong in a multi-store program
 

 Key: PIG-831
 URL: https://issues.apache.org/jira/browse/PIG-831
 Project: Pig
  Issue Type: Bug
  Components: impl
Affects Versions: 0.3.0
Reporter: Alan Gates
Assignee: Alan Gates
Priority: Minor

 The stats features checked in as part of PIG-626 (reporting the number of 
 records and bytes written at the end of the query) print wrong values (often 
 but not always 0) when the pig script being run contains more than 1 store.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.