On Sep 25, 2008, at 11:26 PM, 심탁길 wrote:
Hi all
Recently I tried 100,000 dummy map tasks job on the 100ea node(2GB,
Dual Core, 64Bit machine, Version: 0.16.4) cluster
Map task does nothing but sleeping one minute
I found that Jobtracker(1GB Heap) consumes about 650MB of heap
memory when the job is 50% done.
After all, the job failed at the 90% of progress because Jobtracker
hanged up(?) due to out of memory.
Funny that you bring this up today... I just now concluded that we
need to port
https://issues.apache.org/jira/browse/HADOOP-3670 to 0.17 branch to
fix a sluggish JobTracker who was spending all his time doing GC.
Please upgrade to 0.18 if you can, immdediately! *smile*
Arun
how do you handle this kind of issue?
another related issue:
while the above job was being processed, I clicked on the "Pending"
on jobdatails.jsp of web UI
then, Jobtracker consumed 100% of CPU. and 100% CPU status lasted a
couple of minutes
