Oleg Kalnichevski wrote:
Thorsten Scherler wrote:
On Tue, 2009-03-17 at 23:31 +0100, Thorsten Scherler wrote:
On Mon, 2009-03-16 at 14:25 -0400, Andy Gross wrote:
Droids seems like a very interesting project upon which I'd like to
base some of my custom web crawlers - but I'm running into this
error when I try to run the examples. The file
droids-core/src/test/java/
org/apache/droids/examples/SimpleRuntime.java, for example, throws
this exception. I've spent awhile trying to figure out what the
problem is, to no avail.
I've verified with TCPDump that running the example never initiates
any HTTP activity. I'm running droids-trunk, svn revision 754973,
on Mac OSX 10.5.6.
I just tried and I have a similar problem. Not sure ATM why but I can
reproduce the problem.
I do not get an exception like you but rather: <Finshed invocation,
waiting for workers to finish.>
2009-03-17 23:25:17,670 INFO [pool-1-thread-1] - <I/O exception
(org.apache.http.NoHttpResponseException) caught when processing
request: The target server failed to respond>
If I do Index:
droids-core/src/test/java/org/apache/droids/examples/SimpleRuntime.java
===================================================================
---
droids-core/src/test/java/org/apache/droids/examples/SimpleRuntime.java
(revision 755424)
+++
droids-core/src/test/java/org/apache/droids/examples/SimpleRuntime.java
(working copy)
@@ -98,6 +98,7 @@
DroidsHttpClient httpclient = new DroidsHttpClient(params);
HttpProtocol httpProtocol = new HttpProtocol(httpclient);
+ httpProtocol.setForceAllow(true);
protocolFactory.setMap(new HashMap<String, Object>());
I as well get. Seems that the http pool is not working.
Oleg do you have an idea?
Thorsten,
I'll look into it.
Oleg
The problem is not caused by the HTTP transport. The
MultiThreadedTaskMaster appears to have synchronization issues and
should also be reviewed for thread-safety.
Oleg
java.lang.IllegalStateException: Connection pool shut down.
at
org.apache.http.impl.conn.tsccm.ConnPoolByRoute.getEntryBlocking(ConnPoolByRoute.java:284)
at org.apache.http.impl.conn.tsccm.ConnPoolByRoute
$1.getPoolEntry(ConnPoolByRoute.java:238)
at org.apache.http.impl.conn.tsccm.ThreadSafeClientConnManager
$1.getConnection(ThreadSafeClientConnManager.java:175)
at
org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:324)
at
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:555)
at
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:487)
at
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:465)
at
org.apache.droids.protocol.http.HttpProtocol.load(HttpProtocol.java:63)
at
org.apache.droids.robot.crawler.CrawlingWorker.execute(CrawlingWorker.java:57)
at
org.apache.droids.robot.crawler.CrawlingWorker.execute(CrawlingWorker.java:1)
at org.apache.droids.impl.MultiThreadedTaskMaster
$WorkerRunner.run(MultiThreadedTaskMaster.java:280)
at java.util.concurrent.ThreadPoolExecutor
$Worker.runTask(ThreadPoolExecutor.java:885)
at java.util.concurrent.ThreadPoolExecutor
$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:619)
salu2
