[
https://issues.apache.org/jira/browse/PIG-3637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rohini Palaniswamy updated PIG-3637:
------------------------------------
Fix Version/s: 0.13.0
Affects Version/s: 0.11.1
Status: Patch Available (was: Open)
No new test added. But ran TestCombiner with -Dhadoopversion=23
find target | xargs grep -H "Aliases being processed" | grep Combiner | wc -l
Before the patch it was 34 and after the patch it was 5.
As for the usefulness of cutting down on this log statement - in the case
reported by one of our users, one map task alone had the same log line repeated
1001 times and the job had 32K maps.
> PigCombiner creating log spam
> -----------------------------
>
> Key: PIG-3637
> URL: https://issues.apache.org/jira/browse/PIG-3637
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.11.1
> Reporter: Rohini Palaniswamy
> Assignee: Rohini Palaniswamy
> Priority: Minor
> Fix For: 0.13.0
>
> Attachments: PIG-3637-1.patch
>
>
> "Aliases being processed per job phase" is logged a lot of times even though
> it is in setup because Combiners can be called multiple times in both map and
> reduce by MR framework (HADOOP-3226). With Hadoop 2.0 since logs go to hdfs
> this is inefficient.
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
