Martin,
When you look at the task logs for the particular reducer that's stuck,
what do you see? What kind of operations do you have going on in the
script, possibly a GROUP ALL?
--jacob
On Wed, 2011-01-26 at 12:44 -0300, Martin Z wrote:
> Hi all,
>
> I'm running a Pig script in local mode, and it finishes successfully. When I
> use the same dataset and script to run pig in its distributed mode, it hangs
> at 90% and the hadoop processes in the node machines takes almost all the
> memory. It always hangs at the reduce task of the last job.
>
> The conf/mapred-site.xml is:
>
> <property>
> <name>mapred.child.java.opts</name>
> <value>-Xmx1000m</value>
> </property>
> <property>
> <name>mapred.child.ulimit</name>
> <value>4000000</value>
> <final>true</final>
> </property>
>
> Do you know how I can debug the processes to find out where the problem is?
>
> Thanks!
>
