Edward Quick wrote:
Ahh, I see this was already discussed in a recent thread:
http://www.mail-archive.com/[email protected]/msg11812.html
So in conclusion, is this saying it's not possible to fetch from the same site
at the same time on multiple nodes, or is there a way to override that?
Currently there is no way to override this behavior (unless you're
willing to modify the Generator class to use a different Partitioner).
The only thing you can do now to speed it up is to allow more threads
per host in the config. This is set to 1 by default, but since you
control the target server you can increase it to e.g. 10 and see how it
works.
--
Best regards,
Andrzej Bialecki <><
___. ___ ___ ___ _ _ __________________________________
[__ || __|__/|__||\/| Information Retrieval, Semantic Web
___|||__|| \| || | Embedded Unix, System Integration
http://www.sigram.com Contact: info at sigram dot com
