It is not uncommon for the task tracker http servers to get overwhelmed with requests for map outputs, when there where many map tasks. Increasing the number of threads can help.
On Sat, Sep 19, 2009 at 6:32 PM, Kunsheng Chen <ke...@yahoo.com> wrote: > Hi everyone, > > > I am running two map-reduce program, they were working good but when the > data turns into around 900MB (50000+ files). things weird happen to remind > me as below: > > > 'Communication problem with server: java.net.SocketTimeoutException: timed > out waiting for rpc response' > > Also there is some other reminder like "fail to allocate memory". > > > Strange is that the program keeps running and shows mapping and reduce > percentage after those errors....seems it is still progressing in a slow > pace. > > > Does anyone have some idea ? > > > > Thanks, > > -Kun > > > > > > > -- Pro Hadoop, a book to guide you from beginner to hadoop mastery, http://www.amazon.com/dp/1430219424?tag=jewlerymall www.prohadoopbook.com a community for Hadoop Professionals
