I should also point out that setting up and tearing down connections at high rates is what leads to a problem of lots of sockets in TIME_WAIT. The most elegant solution is to use a connection pool rather than create and tear down connections on demand. That way, sockets are only really closed infrequently.
Karl On Tue, Feb 22, 2011 at 6:09 AM, Karl Wright <[email protected]> wrote: > I've seen this before, when I was using HTTP and setting a connection > up and tearing it down for each communication. This is why HTTP > keep-alive was invented in fact. > > TIME_WAIT is the result of an oddity in the way sockets are managed. > A socket that has been closed but has not received an ACK from the > other end is in this state. Read here: > http://stackoverflow.com/questions/813790/too-many-time-wait-connections > > If you can figure out what socket is being left dangling, then the > right thing to do is create the socket using NO_WAIT, provided you > have access to that code. It's one of the socket parameters that you > can use only at socket create time. It tells the socket to not stick > around after close. > > If this is in the ManifoldCF code, or in JCifs, I'd love to know about it! ;-) > > Thanks, > Karl > > > On Tue, Feb 22, 2011 at 4:40 AM, <[email protected]> wrote: >> Hi Karl, >> >> your guess was right. There has been a problem with too many connections in >> status "TIME_WAIT". >> Even if I set max. connections per JVM to 1 for the Repository and Authority >> Connection I hat up to 3500 connections waiting. >> When investingating this further I found out that an unbind response from >> the LDAP server was missing. >> >> Sometimes this error occurs and sometimes it works fine. It's a bit weird, I >> wasn't able to figure out where it came from. >> >> Thanks for your hint. >> >> Carina >> >> >> -------- Original-Nachricht -------- >>> Datum: Fri, 18 Feb 2011 06:50:36 -0500 >>> Von: Karl Wright <[email protected]> >>> An: [email protected] >>> Betreff: Re: "Connection already in use" when crawling Windows Share >>> repositories >> >>> I don't think this is directly a ManifoldCF issue. My guess is that >>> you are running out of file handles. If you are on Windows, you might >>> want to track the number of handles you are using via netstat. If on >>> linux, then lsof is the right tool. >>> >>> If you decide that this is indeed what the issue is, then on linux you >>> need to either rebuild the kernel, or do stuff with ulimit to set the >>> number of handles. On Windows, I don't know how to do it. Limiting >>> the number of handles used by ManifoldCF is done by setting the number >>> of connections under the "Throttling" tab of the various connection >>> definitions you create, but that's not going to help with Tomcat. >>> >>> It may be worth checking if you aren't closing something. That could >>> easily cause this problem, eventually. Since it is happening at >>> crawling time, the Active Directory Authority is not actually >>> communicating with the LDAP server yet. JCIFS probably is, though. >>> >>> Let us know what you find. >>> >>> Thanks, >>> Karl >>> >>> >>> On Fri, Feb 18, 2011 at 6:31 AM, <[email protected]> wrote: >>> > I'm running MCF rev. 1071535 with Apache Tomcat 7. Solr and MCF are >>> using HTTPS. When crawling data from a Windows Share Connection with Active >>> Directory Authority after some time of crawling my job aborts with the >>> status >>> "Error: Couldn't set up SSL connection to ingestion API: Address already in >>> use: connect". >>> > >>> > In the mcf logfile I get: >>> > >>> > [2011-02-18 11:24:33,695]ERROR Exception tossed: Couldn't set up SSL >>> connection to ingestion API: Address already in use: connect >>> > org.apache.manifoldcf.core.interfaces.ManifoldCFException: Couldn't set >>> up SSL connection to ingestion API: Address already in use: connect >>> > at >>> org.apache.manifoldcf.agents.output.solr.HttpPoster.createSocket(HttpPoster.java:736) >>> > at >>> org.apache.manifoldcf.agents.output.solr.HttpPoster$IngestThread.run(HttpPoster.java:1006) >>> > Caused by: java.net.BindException: Address already in use: connect >>> > at java.net.PlainSocketImpl.socketConnect(Native Method) >>> > at >>> java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:333) >>> > at >>> java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:195) >>> > at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:182) >>> > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366) >>> > at java.net.Socket.connect(Socket.java:529) >>> > at >>> com.sun.net.ssl.internal.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:550) >>> > at >>> com.sun.net.ssl.internal.ssl.SSLSocketImpl.<init>(SSLSocketImpl.java:353) >>> > at >>> com.sun.net.ssl.internal.ssl.SSLSocketFactoryImpl.createSocket(SSLSocketFactoryImpl.java:71) >>> > at >>> org.apache.manifoldcf.agents.output.solr.HttpPoster.createSocket(HttpPoster.java:728) >>> > ... 1 more >>> > >>> > I use the same LDAP user >>> > - to authenticate at MCF >>> > - login credentials for the Solr Connection >>> > - credentials for authority connection >>> > - and for connecting to our LDAP server itself (in tomcat JNDI Realm >>> config). >>> > >>> > It seems to me that the connection to LDAP might be a problem. I tried >>> to configure connection pooling for tomcat. (I'm afraid I'm not so familiar >>> with that.) >>> > "-Dcom.sun.jndi.ldap.connect.pool=true >>> -Dcom.jndi.ldap.connect.pool.protocol=ssl" >>> > >>> > Did I miss sth in the config? Or how do I have to configure MCF? >>> > >>> > Thanks in advance. >>> > >>> > Carina >>> > -- >>> > GMX DSL Doppel-Flat ab 19,99 Euro/mtl.! Jetzt mit >>> > gratis Handy-Flat! http://portal.gmx.net/de/go/dsl >>> > >> >> -- >> GMX DSL Doppel-Flat ab 19,99 Euro/mtl.! Jetzt mit >> gratis Handy-Flat! http://portal.gmx.net/de/go/dsl >> >
