I almost have the same setting. My Linux is Cent OS, I tried all sort of TCP tunings,but it still gives me the same error.
I found something else, it works fine with 2 threads, and if I run another instance of nutch on the same machine with 200 threads, the 200 threads one fails with No Buffer Space error after a while, but the 2 threads one still working fine!! It means the socket buffer is not global for all processes and VM assigns some certain portion of global buffer to each process. I could mange it to work on another machine with about 200 threads and same connection speed, I could get almost 28 pages/sec with that connection. Now, I am pretty sure this is OS related and nutch problem. Thanks for the helps and tips. Nima On 10/20/05, Fuad Efendi <[EMAIL PROTECTED]> wrote: > Check this please, > net.ipv4.ip_local_port_range = 1024 65000 > > > I don't know Linux in-depth... > > Funny, you have very good Internet connection 100Mbps, probably synchronous, > and such a problem may also happen if hardware is not fast enough... Some > kind of a handshake with TCP. After handshake, wnen both sites defined speed > of a hardware, Server sends to you probably 20 IP packets at a time, and > waits for 1 single IP packet with confirmation. "Buffer overload" at your > site... Smth at TCP layer... > > > At my Linux, etc/sysctl.conf (recommended by Oracle, should be higher than > that): > > kernel.shmall = 2097152 > kernel.shmmax = 2147483648 > kernel.shmmni = 4096 > kernel.sem = 250 32000 100 128 > fs.file-max = 65536 > net.ipv4.ip_local_port_range = 1024 65000 > rmem_default = 262144 > rmem_max = 262144 > wmem_default = 262144 > wmem_max = 262144 > > > > -----Original Message----- > Sent: Thursday, October 20, 2005 1:42 PM > To: [email protected] > Subject: Re: No buffer space available > > > I've been searching on Google for two days. I retuned all the Kernel > paramters. I did the following changes for the Kernel setting: > # increase TCP max buffer size > net.core.rmem_max = 16777216 > net.core.wmem_max = 16777216 > > # increase Linux autotuning TCP buffer limits > # min, default, and max number of bytes to use > net.ipv4.tcp_rmem = 4096 87380 16777216 > net.ipv4.tcp_wmem = 4096 65536 16777216 > > But I still have the same problem. It might be because of the maximum > number of TCP connection on Linux, do you have any idea how I can > figure out maximum possible number of TCP connection on Redhat? > > Thanks > > >
