[SoldCloud] leaking file descriptors

2012-03-01 Thread Markus Jelsma
Hi, Yesterday we had an issue with too many open files, which was solved because a username was misspelled. But there is still a problem with open files. We cannot succesfully index a few millions documents from MapReduce to a 5-node Solr cloud cluster. One of the problems is that after a

Re: [SoldCloud] leaking file descriptors

2012-03-01 Thread Bernd Fehling
What is netstat telling you about the connections on the servers? Any connections in CLOSE_WAIT (passive close) hanging? Saw this on my servers last week. Used a little proggi to spoof a local connection on those servers ports and was able to fake the TCP-stack to close those connections. It

Re: [SoldCloud] leaking file descriptors

2012-03-01 Thread Sami Siren
Do you have autocommit enabled? I tested this with 1m docs indexed by using the default example config and saw used file descriptors go up to 2400 (did not come down even after the final commit at the end). Then I disabled autocommit, reindexed and the descriptor count stayed pretty much flat at

Re: [SoldCloud] leaking file descriptors

2012-03-01 Thread Markus Jelsma
On Thursday 01 March 2012 13:03:18 Bernd Fehling wrote: What is netstat telling you about the connections on the servers? Any connections in CLOSE_WAIT (passive close) hanging? I can't tell exact numbers right now but there were a lot between all the cores and the indexing clients. Saw