The hack almost works, but I've had issues with it myself and I'm still working on tracking down the bug that's causing the traffic to end up in the wrong queue.
--Bill On 11/17/05, Kieron Mohindra <[EMAIL PROTECTED]> wrote: > Ah, that's sharpened up my understanding of traffic shaping quite a bit. > > So, the bottom line is that to really make the most of squid, you should > be running it on another box? > > I could do the hack of upping the download limit, but this would remove > any benefits of traffic shaping in that direction. > > Thanks for the very comprehensive reply! > > Kieron > > > > -----Original Message----- > > From: Bill Marquette [mailto:[EMAIL PROTECTED] > > Sent: 17 November 2005 15:10 > > To: [email protected] > > Subject: Re: [pfSense-discussion] Squid and traffic shaper > > > > This couldn't have been a more timely question. Here's a post from > > the author of pf that explains all you'd ever want to know about > > shaping. > > > > --Bill > > > > From: Daniel Hartmeier <[EMAIL PROTECTED]> > > This question pops up frequently, if this reply is too wordy, that's > > just so I can reference it in the future and safe typing. My apologies > > to the poster if this is all obvious already. ;) > > > > Rate-limiting network packets means dropping packets. It's not like a > > water utility pipe where you can shut the faucet incrementally and > slow > > down the water running towards you from the water company, leaving > > unused water in their tanks. There are no reservoirs like that in a > > network (ignoring some very small buffers). If a sender is sending you > > packets at a rate higher than you can receive them, packets are > dropped > > whereever there are gaps of decreasing bandwidth. And these gaps are > on > > routers at your ISP and further upstream. Many of them will drop > random > > packets. Some can be configured to drop based on criteria, but you > don't > > control those criteria, because they're not your routers. > > > > Imagine you have a 1024 kbit/s downlink from your ISP to you. Assume > > your ISP himself has a much larger downlink himself. You're > downloading > > a file from a web server on the Internet. Then some evil person starts > > sending you a flood of pings. Let's say that person has an uplink of > > 2048 kbit/s. Now your ISP is receiving two kinds of packets destined > for > > you: a stream of TCP packets from the web server, and a stream of > pings > > at 2048 kbits/. He can't possibly forward all these packets to you, > > since only less than half of them fit onto the link to you. So he > simply > > forwards as many packets as he can, randomly dropping the rest. > > > > Obviously many of the TCP packets will get dropped randomly now. TCP > is > > clever and adjust to this, the sender recognizes that there is loss > > between him and you, and will start to send at a lower rate. Meanwhile > > the flooder continues to send you pings at a happy rate of 2048 > kbit/s. > > > > You'll notice how your download gets slower and slower, and you > consider > > using rate-limiting incoming packets. You identify that the http > packets > > are those that you prefer over the pings, and tell ALTQ to drop > incoming > > pings exceeding, say, a harmless rate of 64 kbit/s and reserve the > rest > > for the more important http packets. Fine, it could do that. > > > > But it wouldn't change anything, because the congestion is upstream of > > your ALTQ box. You can drop as many packets as you like after you > > received them, that doesn't free up any bandwidth on your downlink. > The > > downlink will continue to carry mostly ping packets, because you > > dropping packets has no influence on what happens at either sender, at > > your ISP's router dropping random packets, or on your downlink. Just > > like the rate of water you can draw from your water line isn't > influenced > > by what you do with the water that has already come out of it. Rate- > > limiting > > is not like a faucet. > > > > This is the reason you often get the answer "It just doesn't make > sense > > to rate-limit incoming packets", and I guess that's the reason why > ALTQ > > simply doesn't add queues for incoming packets, but only outgoing > ones. > > > > Now, if we forget about the DoS case, and assume you have only > > flow-controlled TCP connections with cooperative peers, things are a > > little > > different. If you receive two streams of TCP packets, and you start > > dropping packets of one stream (after you have received them and they > > have taken up bandwidth on your downlink), the corresponding peer will > > detect that loss and helpfully slow down sending, freeing up bandwidth > > on your downlink for the other peer. In fact, if you tell your ALTQ > box > > to limit one stream to, say, 128 kbit/s, and drop all excess packets, > > that peer will (eventually, by trial and error) come to the conclusion > > that sending you packets at a rate of about 128 kbit/s is the optimal > > thing to do. But it's important to realize that you're not really > > "enforcing" this limit, but it's the peer that kindly reacts in that > > way. > > > > If you want to do this with ALTQ, you can do so by limiting outgoing > > packets on the "other" interface, assuming the box is forwarding all > > packets between two interfaces. If a browser (on a separate local box) > > is downloading a file from an external web server _through_ the ALTQ > > box, you rate-limit packets going out through the internal interface. > > Every packet coming in on the external interface obviously goes out > > through the internal interface, hence rate-limiting outgoing packets > on > > the internal interface has the same effect as rate-limiting incoming > > packets on the external interface. > > > > This does not work if the client is on the ALTQ box itself, obviously > > (there is no "other" interface to rate-limit on). In this case you're > > facing a limitation of ALTQ itself. You might have to move ALTQ onto > an > > additional intermediate box, just so you do have a second interface. I > > don't think there are any plans to introduce incoming queues in ALTQ. > > > > > > > > On 11/17/05, Kieron Mohindra <[EMAIL PROTECTED]> wrote: > > > Hi there, > > > > > > When using squid and the traffic shaper is on, all downloads from > the > > > pfsense box to the LAN seem to be limited by the download bandwidth > > > limit as set on the traffic shaper, even for cache hits. > > > > > > Is there any way to speed up the serving of squid data to the LAN, > as it > > > kind of defeats the object otherwise. Unless I'm missing something > > > obvious? > > > > > > Thanks, > > > > > > Kieron > > > > > > >
