I agree its the simplest solution that is the best, drop the network card speed back to 10mbit half duplex, then you wont be using all of your isp's 10mbit. -J ----- Original Message ----- From: "Byron Miller" <[EMAIL PROTECTED]> To: <[email protected]> Sent: Tuesday, January 17, 2006 10:14 AM Subject: Re: throttling bandwidth
> Just to add my 2 cents, for the most part if you have > a decent nic card you could issue OS commands to drop > the port rate of your interface to 10mbit and not > waste cpu cycles on shaping/proxying. > > Although i do recommend squid for this since i too use > it to further filter/offload regex/hostname blocks as > well. > > -byron > > --- Jay Pound <[EMAIL PROTECTED]> wrote: > > > there are a number of linux packages for QOS/traffic > > shaping, my favorite is > > wondershaper, I havent set it up since the 2.4 > > kernel but it works well, > > also if your not inclined to do something that > > involved, your isp can give > > that machine's ip address a car statement in > > your/their cisco router > > preventing that particular machine from using a max > > of x bandwidth. or the > > do it yourself solution buy the cheapest POS router > > (linksys, generic) they > > wont be able to route 20mbit of data through the > > nat, at least older ones > > couldent get much more than 5mbit or so (newer ones > > can do 9mbit +) so there > > are some solutions to your problem. > > -J > > ----- Original Message ----- > > From: "Insurance Squared Inc." > > <[EMAIL PROTECTED]> > > To: <[email protected]> > > Sent: Monday, January 16, 2006 6:02 PM > > Subject: throttling bandwidth > > > > > > > My ISP called and said my nutch crawler is chewing > > up 20mbits on a line > > > he's only supposed to be using 10. Is there an > > easy way to tinker with > > > how much bandwidth we're using at once? I know we > > can change the number > > > of open threads the crawler has, but it seems to > > me this won't make a > > > huge difference. If I chop the number of open > > threads in half, it'll > > > just download half the pages, twice as fast? I > > stand to be corrected on > > > this. > > > > > > Any other thoughts? doesn't have to be correct or > > elegant as long as it > > > works. > > > > > > Failing a reasonable solution in nutch, is there > > some sort of linux > > > level tool that will easily allow me to throttle > > how much bandwidth the > > > crawl is using at once? > > > > > > Thanks. > > > > > > > > > > > > -- > > > This message has been scanned for viruses and > > > dangerous content by MailScanner, and is > > > believed to be clean. > > > > > > > > > > > > > > -- > > This message has been scanned for viruses and > > dangerous content by MailScanner, and is > > believed to be clean. > > > > > > > -- > This message has been scanned for viruses and > dangerous content by MailScanner, and is > believed to be clean. > > -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean.
