Hi Oleg,
I am using Windows XP 32bit. I turned on context logging but I didn't
see anything interesting. How to check if the connections get really
reused? The only thing I see are the 'Invalid Cookies' (expired) warns.
This is the exception stack I get:
java.net.BindException: Address already in use: connect
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:333)
at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:195)
at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
at java.net.Socket.connect(Socket.java:519)
at
org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:106)
at
org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:129)
at
org.apache.http.impl.conn.AbstractPoolEntry.open(AbstractPoolEntry.java:164)
at
org.apache.http.impl.conn.AbstractPooledConnAdapter.open(AbstractPooledConnAdapter.java:119)
at
org.apache.http.impl.client.DefaultClientRequestDirector.execute(DefaultClientRequestDirector.java:309)
at
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:501)
at
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:456)
at
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:422)
at
ca.uottawa.met3.crawler.http.HttpEngine.getSiteAsStream(HttpEngine.java:96)
at
ca.uottawa.met3.crawler.engine.AbstractParser.readSite(AbstractParser.java:60)
at
ca.uottawa.met3.crawler.engine.AbstractParser.parseSite(AbstractParser.java:40)
at
ca.uottawa.met3.crawler.parsers.PaperParser.fillLink(PaperParser.java:36)
at
ca.uottawa.met3.crawler.MultiThreadedCrawlerController$PaperParsingTask.run(MultiThreadedCrawlerController.java:184)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:885)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:619)
Tomek
Oleg Kalnichevski wrote:
On Sat, 2008-08-30 at 20:50 -0400, Tomek Maciejewski wrote:
Hi,
I am trying to write a multithreaded crawler. I would like to execute
http requests simultaneously.
...
I use a one instance of this client. The instance is marked as static
and final. But after executing some number of requests I start to get
a java.net.BindException with message "Address already in use" almost
every time I want to execute request. The code responsible for
executing requests is below (HTTP_CLIENT is a instance of the class I
mentioned above):
...
Could you tell me what am I doing wrong? I thought that maybe I don't
release the connection and I don't unbind sockets, but in finally
section I use consumeContent() method which should do that.
Tomek,
(1) What platform (OS) are you using? Quite often BindException are
caused by platform specific TCP/IP settings.
(2) You can turn on context logging in order to see if connections get
really reused or not. If they do, this would suggest a TCP/IP stack
configuration issue.
Oleg
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
