Hi Karl, your guess was right. There has been a problem with too many connections in status "TIME_WAIT". Even if I set max. connections per JVM to 1 for the Repository and Authority Connection I hat up to 3500 connections waiting. When investingating this further I found out that an unbind response from the LDAP server was missing.
Sometimes this error occurs and sometimes it works fine. It's a bit weird, I wasn't able to figure out where it came from. Thanks for your hint. Carina -------- Original-Nachricht -------- > Datum: Fri, 18 Feb 2011 06:50:36 -0500 > Von: Karl Wright <[email protected]> > An: [email protected] > Betreff: Re: "Connection already in use" when crawling Windows Share > repositories > I don't think this is directly a ManifoldCF issue. My guess is that > you are running out of file handles. If you are on Windows, you might > want to track the number of handles you are using via netstat. If on > linux, then lsof is the right tool. > > If you decide that this is indeed what the issue is, then on linux you > need to either rebuild the kernel, or do stuff with ulimit to set the > number of handles. On Windows, I don't know how to do it. Limiting > the number of handles used by ManifoldCF is done by setting the number > of connections under the "Throttling" tab of the various connection > definitions you create, but that's not going to help with Tomcat. > > It may be worth checking if you aren't closing something. That could > easily cause this problem, eventually. Since it is happening at > crawling time, the Active Directory Authority is not actually > communicating with the LDAP server yet. JCIFS probably is, though. > > Let us know what you find. > > Thanks, > Karl > > > On Fri, Feb 18, 2011 at 6:31 AM, <[email protected]> wrote: > > I'm running MCF rev. 1071535 with Apache Tomcat 7. Solr and MCF are > using HTTPS. When crawling data from a Windows Share Connection with Active > Directory Authority after some time of crawling my job aborts with the status > "Error: Couldn't set up SSL connection to ingestion API: Address already in > use: connect". > > > > In the mcf logfile I get: > > > > [2011-02-18 11:24:33,695]ERROR Exception tossed: Couldn't set up SSL > connection to ingestion API: Address already in use: connect > > org.apache.manifoldcf.core.interfaces.ManifoldCFException: Couldn't set > up SSL connection to ingestion API: Address already in use: connect > > at > org.apache.manifoldcf.agents.output.solr.HttpPoster.createSocket(HttpPoster.java:736) > > at > org.apache.manifoldcf.agents.output.solr.HttpPoster$IngestThread.run(HttpPoster.java:1006) > > Caused by: java.net.BindException: Address already in use: connect > > at java.net.PlainSocketImpl.socketConnect(Native Method) > > at > java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:333) > > at > java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:195) > > at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:182) > > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366) > > at java.net.Socket.connect(Socket.java:529) > > at > com.sun.net.ssl.internal.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:550) > > at > com.sun.net.ssl.internal.ssl.SSLSocketImpl.<init>(SSLSocketImpl.java:353) > > at > com.sun.net.ssl.internal.ssl.SSLSocketFactoryImpl.createSocket(SSLSocketFactoryImpl.java:71) > > at > org.apache.manifoldcf.agents.output.solr.HttpPoster.createSocket(HttpPoster.java:728) > > ... 1 more > > > > I use the same LDAP user > > - to authenticate at MCF > > - login credentials for the Solr Connection > > - credentials for authority connection > > - and for connecting to our LDAP server itself (in tomcat JNDI Realm > config). > > > > It seems to me that the connection to LDAP might be a problem. I tried > to configure connection pooling for tomcat. (I'm afraid I'm not so familiar > with that.) > > "-Dcom.sun.jndi.ldap.connect.pool=true > -Dcom.jndi.ldap.connect.pool.protocol=ssl" > > > > Did I miss sth in the config? Or how do I have to configure MCF? > > > > Thanks in advance. > > > > Carina > > -- > > GMX DSL Doppel-Flat ab 19,99 Euro/mtl.! Jetzt mit > > gratis Handy-Flat! http://portal.gmx.net/de/go/dsl > > -- GMX DSL Doppel-Flat ab 19,99 Euro/mtl.! Jetzt mit gratis Handy-Flat! http://portal.gmx.net/de/go/dsl
