Thank you, that's very useful.
In addition, I changed the way the tasks work, so they store their data in
HBase now (since it's more suited for handling small files).
I'm not 100% sure yet if the problems have been resolved (still doing
extensive testing), but I think I might have gotten rid of
but I didn't find a config option
that allows ignoring tasks that fail.
If 0.18,
http://hadoop.apache.org/common/docs/r0.18.3/api/org/apache/hadoop/mapred/Jo
bConf.html#setMaxMapTaskFailuresPercent(int)
(mapred.max.map.failures.percent)
Hi,
I'm having some problems (Hadoop 0.20.0) where map tasks fail to report
status for 10 minutes and get killed eventually. All of the tasks output
around the same amount of data, some only take a few seconds before starting
the 'merge' on the segments, but some seem to fail by just stopping to
10 mins reminds me of parameter mapred.task.timeout . This is configurable. Or
alternatively you might just do a sysout to let tracker know of its existence (
not an ideal solution though )
Thanks,
Amogh
-Original Message-
From: Mathias De Maré [mailto:mathias.dem...@gmail.com]
Sent:
