[
https://issues.apache.org/jira/browse/PIG-889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12733825#action_12733825
]
Santhosh Srinivasan commented on PIG-889:
-----------------------------------------
Comments:
The reporter inside the logger is setup correctly in PigInputFormat for Hadoop.
However the usage of the logger to retrieve the reporter and then increment
counters is flawed for the following reasons:
1. In the test case, the new loader uses PigHadoopLogger directly. When the
loader is used in local mode, the notion of Hadoop disappears and the reference
to PigHadoopLogger is not usable (i.e., will result in a NullPointerException).
{code}
+ @Override
+ public Tuple getNext() throws IOException {
+ PigHadoopLogger.getInstance().getReporter().incrCounter(
+ MyCounter.TupleCounter, 1);
+ return super.getNext();
+ }
{code}
2. The loggers were meant for warning aggregations. Here, there is a case being
made to expand the capabilities to allow user defined counter aggregations. If
thats the case, then new methods have to be added to the PigLogger interface.
> Pig can not access reporter of PigHadoopLog in Load Func
> --------------------------------------------------------
>
> Key: PIG-889
> URL: https://issues.apache.org/jira/browse/PIG-889
> Project: Pig
> Issue Type: Improvement
> Components: impl
> Affects Versions: 0.4.0
> Reporter: Jeff Zhang
> Assignee: Jeff Zhang
> Fix For: 0.4.0
>
> Attachments: Pig_889_Patch.txt
>
>
> I'd like to increment Counter in my own LoadFunc, but it will throw
> NullPointerException. It seems that the reporter is not initialized.
> I looked into this problem and find that it need to call
> PigHadoopLogger.getInstance().setReporter(reporter) in PigInputFormat.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
