Oh i forgot, if you run Nutch locally you can also use trickle to shape
traffic, much easier on a per command basis. It may be difficult to get it to
work in distributed mode, because Hadoop itself spawn the child processes.
-----Original message-----
> From:Markus Jelsma <[email protected]>
> Sent: Wednesday 26th November 2014 20:44
> To: [email protected]
> Subject: RE: fetcher.throttle.bandwidth
>
> Maybe your network operator can limit the traffic in the switch or router
> your nodes are attached to? We have done this too at some point and it works
> fine. Also, if you are on linux, iptables and tc could help you to limit
> bandwidth on a per-user basis.
>
> -----Original message-----
> > From:Dan Kinder <[email protected]>
> > Sent: Tuesday 25th November 2014 0:10
> > To: [email protected]
> > Subject: fetcher.throttle.bandwidth
> >
> > Hi, I'm having trouble finding documentation about how bandwidth throttling
> > is actually implemented in Nutch. Is it implemented, and if so how? Or do
> > most people just use squid proxies, etc.?
> >
> > -dan
> >
