Raghu, Great, thanks for the help. Regards, Sean
2009/2/13 Raghu Angadi <rang...@yahoo-inc.com> > Sean Knapp wrote: > >> Raghu, >> Apologies for the confusion. I was seeing the problem with any setting >> for dfs.datanode.max.xcievers... 1k, 2k and 8k. Likewise, I was also >> seeing >> the problem with different open file settings, all the way up to 32k. >> >> Since I installed the patch, HDFS has been performing much better. The >> current settings that work for me are 16k max open files with >> dfs.datanode.max.xcievers=8k, though under heavy balancer load I do start >> to >> hit the 16k max. >> > > Thanks. That clarifies many things. > > Even with the patch, if you have lot of simultaneous clients, DataNode does > need a lot of file descriptors (something like 6 times the number of active > 'xceivers'). In your case you seem to have lot of simultaneous clients. I > suggest increasing file limit to much higher (something like 64k). > > Raghu. > > > Regards, >> Sean >> >> 2009/2/13 Raghu Angadi <rang...@yahoo-inc.com> >> >> >
