I don't think there is anything you can do about this on the Nutch end. I do
know that Java now has the ability to differentiate between different NICs, but
Nutch doesn't have support for that. There may be something you can do on the
OS level, though I don't have any concrete advice there, unfortunately.
Otis
--
Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch
- Original Message
From: Euan Clark [EMAIL PROTECTED]
To: nutch-dev@lucene.apache.org
Sent: Sunday, March 9, 2008 4:24:59 PM
Subject: Confine nutch to one NIC?
I'm running multiple nutch instances on one server and want to add
another couple of ethernet cards.
The nutch instances are concurrently performing different stages on
crawlset development.
User/Instance 1) searching
User/Instance 2) merging/linking/indexing
User/Instance 3) crawling
This results in multiple java processes, each running under their own
username. Isolates configuration, makes it easy identify the task you
want to kill etc.
Currently the server is co-located at an ISP and it's single NIC is
sharing the link back to the switch with other servers.
I want to add another network card for crawl-traffic that will have it's
own dedicated bandwidth.
Question? What's the best way to confine the crawling traffic to it's
own card?
(asked on nutch-user without success)
TIA, Euan Clark
