Re: High ingest rate and FIN_WAIT1 problems

2010-07-20 Thread Thomas Downing
Yes, hadoop 0.20.2 and hbase 0.20.5. I will get the branch you suggest, and give it a whirl. I am leaving on vacation Thursday, so I may not have any results to report till I get back. When I do get back, I will catch up with versions/fixes and try some more. Meanwhile, thanks to all who have

Re: High ingest rate and FIN_WAIT1 problems

2010-07-20 Thread Stack
On Tue, Jul 20, 2010 at 10:15 AM, Thomas Downing tdown...@proteus-technologies.com wrote: Meanwhile, thanks to all who have responded to my posts. Thanks for persisting with this Thomas. You might also take a look at cloudera CDH3b2. It'll have the above fixes and then some. I've not looked

Re: High ingest rate and FIN_WAIT1 problems

2010-07-19 Thread Thomas Downing
Thanks for the response, but my problem is not with FIN_WAIT2, it is with FIN_WAIT1. If it was FIN_WAIT2, the only concern would be socket leakage, and if setting the time out solved the issue, that would be great. The problem with FIN_WAIT1 is twofold - first, it is incumbent on the

RE: High ingest rate and FIN_WAIT1 problems

2010-07-16 Thread Hegner, Travis
Hi Thomas, I ran into a very similar issue when running slony-I on postgresql to replicate 15-20 databases. Adjusting the TCP_FIN_TIMEOUT parameters for the kernel may help to slow (or hopefully stop), the leaking sockets. I found some notes about adjusting TCP parameters here:

Re: High ingest rate and FIN_WAIT1 problems

2010-07-16 Thread Thomas Downing
Thanks for the response. My understanding is that TCP_FIN_TIMEOUT affects only FIN_WAIT2, my problem is with FIN_WAIT1. While I do see some sockets in TIME_WAIT, they are only a few, and the number is not growing. On 7/16/2010 12:07 PM, Hegner, Travis wrote: Hi Thomas, I ran into a very

Re: High ingest rate and FIN_WAIT1 problems

2010-07-16 Thread Ryan Rawson
I've been running with this setting on both the HDFS side and the HBase side for over a year now, it's a bit of voodoo but you might be running into well known suckage of HDFS. Try this one and restart your hbase hdfs. The FIN_WAIT2/TIME_WAIT happens more on large concurrent gets, not so much