Re: [squid-users] How to use cbq
On Thu, 2010-12-23 at 19:05 +0100, lupuscramus wrote: Do you know someone who managed to use the squid marked packets to make a QoS based on ip source with classful queuing ? (cbq, htb) Yes, I do this. For an example you could have a look at my website. It is out of date and probably not exactly what you are looking for, but it would probably give you an idea: http://www.andybev.com/index.php/Fair_traffic_shaping_an_ADSL_line_for_a_lo cal_network_using_Linux On your website I don't see where you use Squid to mark packets. Sorry, it's just an example of using HTB, I've not updated it yet with my current Squid rules. Hum, I've noticed something : when i make tc class show dev eth0 I can see there are packets which pass by the class I want : they are packets marked by Squid : the source is the proxy and the destination is the web server. They represent a little proportion of packets between my user and my web server (there is mainly download with HTTP) However, I want to limit the download rate : packets from web server to proxy server. I'm a bit confused. Can you produce a diagram of your setup? Is your web server on a different server to Squid? If you want to limit packets *to* Squid, then you will need to set up HTB on the interface going to Squid (you might need to use IFB) *or* rate limit the packets going *from* Squid on the other interface to the one you are using now. Is there a way to do this ? Was the feature written for this ? Please provide some more info of your setup and I'll have a look. Andy
Re: [squid-users] How to use cbq
Do you know someone who managed to use the squid marked packets to make a QoS based on ip source with classful queuing ? (cbq, htb) Yes, I do this. For an example you could have a look at my website. It is out of date and probably not exactly what you are looking for, but it would probably give you an idea: http://www.andybev.com/index.php/Fair_traffic_shaping_an_ADSL_line_for_a_lo cal_network_using_Linux On your website I don't see where you use Squid to mark packets. Hum, I've noticed something : when i make tc class show dev eth0 I can see there are packets which pass by the class I want : they are packets marked by Squid : the source is the proxy and the destination is the web server. They represent a little proportion of packets between my user and my web server (there is mainly download with HTTP) However, I want to limit the download rate : packets from web server to proxy server. Is there a way to do this ? Was the feature written for this ? Thanks, Grégoire Leroy
Re: [squid-users] How to use cbq
That was 4 months ago. The netfilter MARK feature is now completed and working AFAIK in the 3.2 betas. Amos Hello, Is there a way to verify that packet are really marked ? In my squid.conf I've : acl user1 src /etc/squid/ip_user/user1 tcp_outgoing_mark 0x1 user1 acl user85 src /etc/squid/ip_user/user85 tcp_outgoing_mark 0x55 user85 I've made a script htb.sh : #!/bin/bash INT=eth0 BAND=12000kbit RATE=63kbit tc qdisc del dev $INT root tc qdisc add dev $INT root handle 1: htb default 30 tc class add dev $INT parent 1: classid 1:1 htb rate 100mbit burst 15k tc class add dev $INT parent 1:1 classid 1:10 htb rate $RATE ceil $RATE \ burst 15k tc class add dev $INT parent 1:1 classid 1:30 htb rate 88mbit burst 15k tc class add dev $INT parent 1:1 classid 1:20 htb rate $RATE ceil $BAND \ burst 15k tc filter add dev $INT protocol ip handle 1 fw flowid 1:10 tc filter add dev $INT protocol ip handle 0x55 fw flowid 1:20 However, when I try to download something with an ip in /etc/squid/ip_user/user85, I download at 88mbit/s... So, it seems my packets are not marked. tc filter show dev eth0 gives me : filter parent 1: protocol ip pref 49151 fw filter parent 1: protocol ip pref 49151 fw handle 0x55 classid 1:20 filter parent 1: protocol ip pref 49152 fw filter parent 1: protocol ip pref 49152 fw handle 0x1 classid 1:10 Have I done a misconfiguration in squid ? Thanks, Grégoire leroy
Re: [squid-users] How to use cbq
On 23/12/10 01:53, lupuscramus wrote: That was 4 months ago. The netfilter MARK feature is now completed and working AFAIK in the 3.2 betas. Amos Hello, Is there a way to verify that packet are really marked ? iptables should have something to track packet markings with. In my squid.conf I've : acl user1 src /etc/squid/ip_user/user1 tcp_outgoing_mark 0x1 user1 acl user85 src /etc/squid/ip_user/user85 tcp_outgoing_mark 0x55 user85 Looks right for the Squid part. Also check that Squid was built with the netfilter-conntrack library and the QoS feature enabled. --enable-zph-qos --with-netfilter-conntrack when not explicitly specified for ./configure the feature support is at the mercy of the automatics and may be disabled relatively silently if required system pieces are missing. I'm not familiar with the specific tc configuration to help further sorry. Amos -- Please be using Current Stable Squid 2.7.STABLE9 or 3.1.9 Beta testers wanted for 3.2.0.3
Re: [squid-users] How to use cbq
Also check that Squid was built with the netfilter-conntrack library and the QoS feature enabled. --enable-zph-qos --with-netfilter-conntrack Oh, I hadn't add these options. when not explicitly specified for ./configure the feature support is at the mercy of the automatics and may be disabled relatively silently if required system pieces are missing. Hum, should there be some errors when I use squid -k parse squid -k reconfigure if these options are disabled ? I have no warning or error. Thanks, Grégoire Leroy
Re: [squid-users] How to use cbq
On 23/12/10 02:21, lupuscramus wrote: Also check that Squid was built with the netfilter-conntrack library and the QoS feature enabled. --enable-zph-qos --with-netfilter-conntrack Oh, I hadn't add these options. when not explicitly specified for ./configure the feature support is at the mercy of the automatics and may be disabled relatively silently if required system pieces are missing. Hum, should there be some errors when I use squid -k parse squid -k reconfigure if these options are disabled ? They will warn about kernel support if the SO_MARK feature is not available there. Otherwise silent. Sorry I miss-remembered about the library it is not needed for this particular marking option. Amos -- Please be using Current Stable Squid 2.7.STABLE9 or 3.1.9 Beta testers wanted for 3.2.0.3
Re: [squid-users] How to use cbq
Looks right for the Squid part. Also check that Squid was built with the netfilter-conntrack library and the QoS feature enabled. --enable-zph-qos --with-netfilter-conntrack when not explicitly specified for ./configure the feature support is at the mercy of the automatics and may be disabled relatively silently if required system pieces are missing. iptables should have something to track packet markings with. I've recompiled squid with the missed options, and in the iptables logs I can see the packets are marked. For information, if someone want to know how verify if the packets are marked : iptables -A OUTPUT -j LOG iptables -A INPUT -j LOG iptables -A FORWARD -j LOG cat /var/log/kern.log So, the problem comes from tc, not squid. I'm not familiar with the specific tc configuration to help further sorry. Do you know someone who managed to use the squid marked packets to make a QoS based on ip source with classful queuing ? (cbq, htb) Or do you know where I can ask help ? It seems the LARTC mailing-list is dead. Thanks, Grégoire Leroy
Re: [squid-users] How to use cbq
On 23/12/10 11:57, lupuscramus wrote: Looks right for the Squid part. snip I'm not familiar with the specific tc configuration to help further sorry. Do you know someone who managed to use the squid marked packets to make a QoS based on ip source with classful queuing ? (cbq, htb) Or do you know where I can ask help ? It seems the LARTC mailing-list is dead. The original feature author, Andrew Beverley, is the only one who comes to mind. He is on this list and has agreed to support its use here. Amos -- Please be using Current Stable Squid 2.7.STABLE9 or 3.1.9 Beta testers wanted for 3.2.0.3
Re: [squid-users] How to use cbq
On Wed, 2010-12-22 at 23:57 +0100, lupuscramus wrote: Looks right for the Squid part. Also check that Squid was built with the netfilter-conntrack library and the QoS feature enabled. --enable-zph-qos --with-netfilter-conntrack when not explicitly specified for ./configure the feature support is at the mercy of the automatics and may be disabled relatively silently if required system pieces are missing. That reminds me, I was going to have a further look at this, because if LIBCAP isn't present but netfilter-conntrack is, then the features get built but just don't work. There should probably be a warning either when compiling or when running. iptables should have something to track packet markings with. I've recompiled squid with the missed options, and in the iptables logs I can see the packets are marked. For information, if someone want to know how verify if the packets are marked : iptables -A OUTPUT -j LOG iptables -A INPUT -j LOG iptables -A FORWARD -j LOG cat /var/log/kern.log So, the problem comes from tc, not squid. I'm not familiar with the specific tc configuration to help further sorry. Do you know someone who managed to use the squid marked packets to make a QoS based on ip source with classful queuing ? (cbq, htb) Yes, I do this. For an example you could have a look at my website. It is out of date and probably not exactly what you are looking for, but it would probably give you an idea: http://www.andybev.com/index.php/Fair_traffic_shaping_an_ADSL_line_for_a_local_network_using_Linux HTB is a tricky beast. There is information out there, but sometimes need to look hard. Or do you know where I can ask help ? It seems the LARTC mailing-list is dead. The netfilter mailing lists are the places to ask. The LARTC mailing list unfortunately seemed to become unused a couple of years back. Andy
Re: [squid-users] How to use cbq
I recall there was a discussion on this recently on this list. A a search of the archives should find this. Tris Maybe you talk about http://squid-web-proxy- cache.1019090.n4.nabble.com/Marking-outgoing-connections-with-a-mark-acc-to- client-IP-td2340794.html However the conclusion is that the feature is not yet implemented. Have I missed something ? Is there any solution ? Thanks, Gregoire
Re: [squid-users] How to use cbq
On Tue, 14 Dec 2010 14:54:09 +0100, lupuscramus lupuscra...@hyperthese.net wrote: I recall there was a discussion on this recently on this list. A a search of the archives should find this. Tris Maybe you talk about http://squid-web-proxy- cache.1019090.n4.nabble.com/Marking-outgoing-connections-with-a-mark-acc-to- client-IP-td2340794.html However the conclusion is that the feature is not yet implemented. Have I missed something ? Is there any solution ? That was 4 months ago. The netfilter MARK feature is now completed and working AFAIK in the 3.2 betas. Amos
[squid-users] How to use cbq
Hi, I would use the cbq (or htb, or any algo which is based on ip source) queuing in order to manage my bandwith (~ 12 Mbits fot 150 users). I can't make a rule in my firewall because my firewall doesn't know what is the ip source : he knows the proxy server ip. I have not seen the feature in squid - the only bandwith management I've seen is the delay pools -, is there any way to use a QOS based on ip source ? Or must I code the use of cbq ? Thanks, Grégoire Leroy
Re: [squid-users] How to use cbq
On 13/12/2010 21:03, lupuscramus wrote: Hi, I would use the cbq (or htb, or any algo which is based on ip source) queuing in order to manage my bandwith (~ 12 Mbits fot 150 users). I can't make a rule in my firewall because my firewall doesn't know what is the ip source : he knows the proxy server ip. I have not seen the feature in squid - the only bandwith management I've seen is the delay pools -, is there any way to use a QOS based on ip source ? Or must I code the use of cbq ? Thanks, Grégoire Leroy Hi Gregorie, CBQ and HTB are policy's that are applied to the network interface on the server, they are independent of Squid. I've used HTB in various scenarios where I've wanted to control a specific users bandwidth independent of the proxy software that has been running on the box. Using the info on lartc.org it is fairly easy to compile a simple shaping policy. If you want to do more advanced things though you will need to investigate marking the packets on the firewall and also look at using Squid to apply QOS markers to the traffic. I recall there was a discussion on this recently on this list. A a search of the archives should find this. Tris * This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please notify postmas...@bgfl.org The views expressed within this email are those of the individual, and not necessarily those of the organisation *
Re: [squid-users] How to use cbq
Hi Gregorie, CBQ and HTB are policy's that are applied to the network interface on the server, they are independent of Squid. I've used HTB in various scenarios where I've wanted to control a specific users bandwidth independent of the proxy software that has been running on the box. Using the info on lartc.org it is fairly easy to compile a simple shaping policy. Yes, I've seen lartc.org, but it doesn't seem it works for me. I can act on 2 servers : the gateway (firewall) or the proxy. I want act on the HTTP traffic only, other traffic don't pass on the same line (I've 2 ADSL for HTTP and 1 SDSL for other traffic). All the HTTP traffic pass by the proxy, so the firewall doesn't know the ip source. So, I can't act on the gateway. I can manage either the in or the out traffic, for the proxy. If I manage the out with some rules, I've the same problem : the ip source will be the proxy's. If I manage the in traffic, there is no differences between the cached requests and the non cache requests = I lose the interest of the cache. If you want to do more advanced things though you will need to investigate marking the packets on the firewall and also look at using Squid to apply QOS markers to the traffic. I recall there was a discussion on this recently on this list. A a search of the archives should find this. Yes, I'm going search this, it could be what I need. Tris Thanks for your help, Gregoire
