Re: [LARTC] Question about how TC enforces bandwidth limiting
Martin A. Brown wrote:
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
Greetings again,
: So you are saying I have to not only do traffic shaping, but also
: traffic policing on my internal device? Or do I have to do
: traffic shaping on both devices and no traffic policing? In other
: words, how much traffic shaping/policing do I need to put into
: effect, and on which interfaces.
Be careful. Shaping and policing are two very different things. I
probably could have chosen a word other than policy to make this
clear. I'll restate this.
If you wish to shape upload traffic, then you must put your traffic
control structures on the the device closest to the Internet. If
you wish to shape download traffic, then you must put your traffic
control structures on the device closest to the internal network(s).
( If you would like to see how you can break this basic rule, see
some advanced traffic control structures called imq and ifb [*]. )
No need for advanced traffic controls for me. All I want to do is
control how much band width
gets out to the internet. Internally it doesn't matter as its all CAT5,
so there is plenty of data pipe.
: I have no idea what that means. How do I vary the depth of the
: FIFO to help control latency?
OK, in order to vary the exact depth of the fifo to suit your fancy,
you'd need to use terminal FIFOs instead of using terminal SFQs:
tc qdisc add dev $INTERFACE parent 10:1 handle 100:0 pfifo limit 10
Assuming your HTB tree has a leaf class of 10:1, into which you have
classified some of your traffic, you now have a very short queue
there. If a burst of traffic involving more than ten packets
arrives, the traffic will tail drop. In heavily congested
situations, this can be useful, although this still allows for a
single dominating UDP flow, so I'd probably prefer the SFQ solution,
and maybe I shouldn't have brought up this little trick.
I think I'll stick to htb...
: I ran both watch -n 1 tc -s class show dev eth0 and watch -n 1 tc
: -s qdisc show eth0. (When I ran class show, i did not have enough
: room to see classes 80 and 90. When I ran qdisc show, I was able
: to see all the classes.) During my runs of tc in this manner, I
: saw zero traffic going to class 100 when running, starting, or
: stopping bit torrent.
If you expect that the torrential traffic to appear in class 100,
and it does not, then you have done something wrong in your
classifier. Look there.
By classifier I think you mean:
iptables -t mangle -A POSTROUTING -o ${IFext} -p tcp --dport 1:1
-j CLASSIFY --set-class 1:100
And having looked at that, I see part of my problem. --dport should be
--sport as I have no way to control what port
the other client wishes to use, but I do have control over what ports I
use to send my data stream.
: Almost all the traffic was going to class 10 and 90 (default)
: with the exception of my ICMP and UDP traffic which was going to
: class 70 and class 60 which I have set aside for IRC traffic.
: Class 100 saw absolutely zero traffic.
:
: Is this a case where the default class (90) is getting all the
: traffic because it can handle it as my LAN has very little
: other traffic most of the time to deal with, so there is no
: need to throttle it back? If so, how can I force a particular
: class to be used regardless of the default, so that I can control
: individual apps by them selves?
Try looking at your classifiers (tc filter statements) to
determine where the problem is. If you can't figure it out, then
send the list your classifiers and class structure (tc commands).
I ran tc filter on the command line, but received no output in return.
I read the man page and it leads me to believe
that it's not meant for viewing the filters. I also did some packet
sniffing on my router PC. I am not 100% sure why yet,
but, when I would filter out port 1-10004, wireshark would return
packets that had source ports other than the
range I was looking for. While I cannot be sure as of yet, I think this
is a result of the NAT process and the router box
is using whatever port it has available while allowing my bit torrent
client to send on the proper ports. (I am still learning
about NAT, so I'm still confused about some things. To be honest, I
would like to be able to NAT not only IPs, but
ports as well so that I do not have to reconfigure things like SSH to
run on non-standard ports. But thats for another
mailing list. :P) One thing I did notice was, while I did see traffic in
class 100 for the first time, my torrent client still showed
outgoing bandwidth of more than 20kbit. Is this simply a function of the
router actually limiting the traffic and the torrent
client simply not knowing? Or (and I assume this is incorrect thinking)
should the torrent client visibly indicate to me that
it can only send at X rate because its limited?
To make this much simpler, I will paste my tc rules
Re: [LARTC] Question about how TC enforces bandwidth limiting
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
Good morning,
: By classifier I think you mean:
: iptables -t mangle -A POSTROUTING -o ${IFext} -p tcp --dport 1:1 -j
: CLASSIFY --set-class 1:100
Exactly.
: And having looked at that, I see part of my problem. --dport should be
: --sport as I have no way to control what port
: the other client wishes to use, but I do have control over what ports I use
: to send my data stream.
Yes. We think about services (at TCP) as being on a standard port,
but iptables operates at L3 (even though it can examine higher
layers). So, if the packet in question is a return packet from an
HTTP server to a client, then as far as iptables is concerned, the
source port is tcp/80.
: I ran tc filter on the command line, but received no output in
: return. I read the man page and it leads me to believe that it's
: not meant for viewing the filters.
Depends, but yes, the tc filter output is not necessarily
attractive. You can classify with tc filter instead of iptables,
but if you reach your goal, it doesn't matter which mechanism you
use to classify your packets.
: One thing I did notice was, while I did see traffic in class 100
: for the first time, my torrent client still showed outgoing
: bandwidth of more than 20kbit.
Well, a typical torrent client will use a number of connections.
Perhaps some of the connections are classified correctly and some
are not?
: Is this simply a function of the router actually limiting the
: traffic and the torrent client simply not knowing? Or (and I
: assume this is incorrect thinking) should the torrent client
: visibly indicate to me that it can only send at X rate because
: its limited?
I would expect the torrent client to be reporting the actual
speed(s), so I would expect it to be reporting 20kbit rates.
: To make this much simpler, I will paste my tc rules and iptables
: rules (which classify my traffic) at the bottom of this e-mail. I
: hope you can find something (related specifically to bit torrent)
: that will allow me to limit torrent traffic without the need to
: limit each client by hand.
You are getting there.
: So I am at a total loss as to why (outgoing) SSH traffic would
: become so slow, because it has access to more bandwidth than
: torrent (at least in my thinking).
Are you sure that it's outgoing SSH traffic that is slow? Consider
the following scenario:
* your outgoing queues are configured completely correctly
* outgoing ssh IP packet gets into correct queue
* inbound return IP packet from ssh server is delayed inbound
because you are not shaping downstream traffic
So, before you conclude that your shaping isn't working, I think
you'll need to apply some sort of mechanisms also on your internal
interface.
Snipped iptables classification. Nothing looks fishy to me there
(though I haven't used the iptables CLASSIFY target).
: $IFext is of course eth0 (link to the modem), $QUANTUM is 1490
: (due to, if I assume correctly, my MTU being 1492, in the modem),
: $MAX_RATE is 360kbit (360kbps conventional talk)
I wouldn't set quantum unless you need to do so. HTB will calculate
this for you. If you do need to set the quantum, I'd recommend
setting it just a bit larger than the MTU...so 1500 or 1536 in your
pppoe situation.
Good luck,
- -Martin
- --
Martin A. Brown
http://linux-ip.net/
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.2 (GNU/Linux)
Comment: pgf-0.72 (http://linux-ip.net/sw/pine-gpg-filter/)
iD8DBQFG3Vd1HEoZD1iZ+YcRAv0pAKDZ0qtpxyOkGJJQ7H5rWtFi3HlM2gCgrugd
WyARMeCjVI9TD/1CcTTDAsw=
=RKrP
-END PGP SIGNATURE-
___
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
Re: [LARTC] Question about how TC enforces bandwidth limiting
Martin A. Brown wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Good morning, : I ran tc filter on the command line, but received no output in : return. I read the man page and it leads me to believe that it's : not meant for viewing the filters. Depends, but yes, the tc filter output is not necessarily attractive. You can classify with tc filter instead of iptables, but if you reach your goal, it doesn't matter which mechanism you use to classify your packets. Can you provide a simple example of how to filter with tc rather than iptables? Just enough of an idea for me to grasp it on my own. I think it would be better to use tc to do the filtering rather than iptables, as thats what tc is meant to do. : One thing I did notice was, while I did see traffic in class 100 : for the first time, my torrent client still showed outgoing : bandwidth of more than 20kbit. Well, a typical torrent client will use a number of connections. Perhaps some of the connections are classified correctly and some are not? One thing I had failed to take into account was the possibility that bit torrent *may* be using some UDP ports. In an attempt to test this theory, I added the udp filters to iptables and watched the data stream. No change. -_- Bit torrent is still showing outgoing speeds of above 20k. So, I then limitted the filter to the LAN IP for my router box and forced my torrent client to use that IP. Still above 20k. Long story short, I tried about 5 different things to get it to work properly. No luck. I think part of the problem is that my torrent client doesn't seem to honor the port ranges I put into effect. Which would definitely allow traffic to get past it. : Is this simply a function of the router actually limiting the : traffic and the torrent client simply not knowing? Or (and I : assume this is incorrect thinking) should the torrent client : visibly indicate to me that it can only send at X rate because : its limited? I would expect the torrent client to be reporting the actual speed(s), so I would expect it to be reporting 20kbit rates. As would I. : To make this much simpler, I will paste my tc rules and iptables : rules (which classify my traffic) at the bottom of this e-mail. I : hope you can find something (related specifically to bit torrent) : that will allow me to limit torrent traffic without the need to : limit each client by hand. You are getting there. : So I am at a total loss as to why (outgoing) SSH traffic would : become so slow, because it has access to more bandwidth than : torrent (at least in my thinking). Are you sure that it's outgoing SSH traffic that is slow? Consider the following scenario: * your outgoing queues are configured completely correctly * outgoing ssh IP packet gets into correct queue * inbound return IP packet from ssh server is delayed inbound because you are not shaping downstream traffic So, before you conclude that your shaping isn't working, I think you'll need to apply some sort of mechanisms also on your internal interface. Bah, I need to figure out outgoing before I mess with incoming. I'm liable to cut my self off from the internet. :P Snipped iptables classification. Nothing looks fishy to me there (though I haven't used the iptables CLASSIFY target). : $IFext is of course eth0 (link to the modem), $QUANTUM is 1490 : (due to, if I assume correctly, my MTU being 1492, in the modem), : $MAX_RATE is 360kbit (360kbps conventional talk) I wouldn't set quantum unless you need to do so. HTB will calculate this for you. If you do need to set the quantum, I'd recommend setting it just a bit larger than the MTU...so 1500 or 1536 in your pppoe situation. Ok, quantum has always been 1512 for me, so I assume tc is taking care of it. Good luck, - -Martin - -- Martin A. Brown http://linux-ip.net/ -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.2 (GNU/Linux) Comment: pgf-0.72 (http://linux-ip.net/sw/pine-gpg-filter/) iD8DBQFG3Vd1HEoZD1iZ+YcRAv0pAKDZ0qtpxyOkGJJQ7H5rWtFi3HlM2gCgrugd WyARMeCjVI9TD/1CcTTDAsw= =RKrP -END PGP SIGNATURE- So, I am at a total loss as to why this isn't working. Well, not a total loss, but enough of a total loss as to disable some of the shaping I had been using that was giving me fits. I think it's much better for me to let the default class handle everything and go from there for now. As I said above, can you provide a simple example of how to filter with tc? I think filtering in tc will be both more appropriate and less hassling. I really appreciate you taking the time to help me with this. I am by no means a noob to networking, but when it comes to traffic shaping, I might as well be. Though, on a positive note, at least I've been able to properly shape some traffic, like IRC. :) (I'm also going to contact the developers of my torrent client and ask them about port range limiting. And why it doesn't
[LARTC] Re: 2 ISP connection sharing problem
Hi all, I am now testing on a simplest scenario. I have an ip 192.168.3.5 on intranet. I want to route it through ISP1. All other traffic will go through ISP2 which is default gateway on machine so I dont need to add any rule for that. I have executed following commands echo 150 ISP1 /etc/iproute2/rt_tables ip rule add from 193.168.3.5/32 to 0.0.0.0/0 table ISP1 ip route add default via 192.168.1.1 table ISP1 ip route flush cache Following is the tables state [EMAIL PROTECTED] ~]# ip rule list 0: from all lookup local 32764: from 193.168.3.5 lookup ISP1 32765: from 193.168.3.5 lookup ISP1 32766: from all lookup main 32767: from all lookup default [EMAIL PROTECTED] ~]# route Kernel IP routing table Destination Gateway Genmask Flags Metric RefUse Iface 192.168.3.0 * 255.255.255.0 U 0 00 eth0 203.81.198.0* 255.255.255.0 U 0 00 eth2 192.168.1.0 * 255.255.255.0 U 0 00 eth1 169.254.0.0 * 255.255.0.0 U 0 00 eth1 default 203.81.194.24 0.0.0.0 UG0 00 eth2 [EMAIL PROTECTED] ~]# ip route list table ISP1 default via 192.168.1.1 dev eth1 system is natted. I have checked for ip 192.168.3.5 but this is still from default gateway. Not going my settings. Internet is working for 192.168.3.5 but through ISP2. Will I have to do something with main table? -- Regards, Arman ___ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
[LARTC] Re: 2 ISP connection sharing problem
Here is my natting script -- IPTABLES=/sbin/iptables echo 1 /proc/sys/net/ipv4/ip_forward $IPTABLES -F $IPTABLES -t nat -F /sbin/modprobe ip_nat_ftp INTERNAL_NETWORK=192.168.3.0/24 $IPTABLES -t nat -A POSTROUTING -s $INTERNAL_NETWORK -o eth2 -j MASQUERADE $IPTABLES -t nat -A POSTROUTING -s $INTERNAL_NETWORK -o eth1 -j MASQUERADE $IPTABLES -A INPUT -i eth0 -s $INTERNAL_NETWORK -m state --state ESTABLISHED,RELATED -j ACCEPT $IPTABLES -A OUTPUT -m state --state ESTABLISHED,RELATED -j ACCEPT $IPTABLES -A FORWARD -i eth0 -o eth2 -s $INTERNAL_NETWORK -m state --state ESTABLISHED,RELATED -j ACCEPT $IPTABLES -A FORWARD -i eth0 -o eth1 -s $INTERNAL_NETWORK -m state --state ESTABLISHED,RELATED -j ACCEPT --- On 9/5/07, Pan'ko Alexander [EMAIL PROTECTED] wrote: On Tue, 4 Sep 2007 22:55:17 +0500 Arman [EMAIL PROTECTED] wrote: Hi all, I am now testing on a simplest scenario. I have an ip 192.168.3.5on intranet. I want to route it through ISP1. All other traffic will go through ISP2 which is default gateway on machine so I dont need to add any rule for that. I have executed following commands echo 150 ISP1 /etc/iproute2/rt_tables ip rule add from 193.168.3.5/32 to 0.0.0.0/0 table ISP1 ip route add default via 192.168.1.1 table ISP1 ip route flush cache Following is the tables state [EMAIL PROTECTED] ~]# ip rule list 0: from all lookup local 32764: from 193.168.3.5 lookup ISP1 32765: from 193.168.3.5 lookup ISP1 32766: from all lookup main 32767: from all lookup default [EMAIL PROTECTED] ~]# route Kernel IP routing table Destination Gateway Genmask Flags Metric RefUse Iface 192.168.3.0 * 255.255.255.0 U 0 00 eth0 203.81.198.0* 255.255.255.0 U 0 00 eth2 192.168.1.0 * 255.255.255.0 U 0 00 eth1 169.254.0.0 * 255.255.0.0 U 0 00 eth1 default 203.81.194.24 0.0.0.0 UG0 00 eth2 [EMAIL PROTECTED] ~]# ip route list table ISP1 default via 192.168.1.1 dev eth1 I have very nearest configuration and it is working system is natted. What and how is natted? The right is: iptables -t nat -A POSTROUTING -s 192.168.3.5 -j MASQUERADE Or iptables -t nat -A POSTROUTING -s 192.168.3.5 -j SNAT --to-source (ip of eth1) May be you have iptables -t nat -A POSTROUTING -s 192.168.3.0/24 -j SNAT --to-source (ip of eth2) It is wrong. Try to determin routing by tcpdump -i ethX. I have checked for ip 192.168.3.5 but this is still from default gateway. Not going my settings. Internet is working for 192.168.3.5 but through ISP2. Will I have to do something with main table? -- Regards, Arman -- With best regards, Pan'ko Alexander. -- Regards, Arman Anwar ___ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
[LARTC] NAT-aware traffic analysis
I have tried using iptraf for my NAT firewall to analyse the IP traffic. Basically I am faced with this difficulty of related the source IP to the outgoing interface to the internet, so I am wondering if anyone has a suggestion for a different ways to do it, or a suggestion for a better tool. Details :- Supposed : eth0 - LAN eth1 - WAN1 eth2 - WAN2 And then all source IPs in the LAN are SNAT to the respective WAN interface when leave for internet. There are also DNAT traffic from internet to the LAN. I want to breakdown the statistic of LAN users using the internet. If I run iptraf on eth0, I will see the LAN stats, but I don't know for sure which one really go out to which WAN ( some traffic does not even go out to the WAN at all ! ). Then when I sniff at eth1 or eth2, I lost the information about the LAN IPs. How could I do a stateful or NAT-aware traffic analysis ? Anyone has a good suggestion ? Important Warning! *** This electronic communication (including any attached files) may contain confidential and/or legally privileged information and is only intended for the use of the person to whom it is addressed. If you are not the intended recipient, you do not have permission to read, use, disseminate, distribute, copy or retain any part of this communication or its attachments in any form. If this e-mail was sent to you by mistake, please take the time to notify the sender so that they can identify the problem and avoid any more mistakes in sending e-mail to you. The unauthorised use of information contained in this communication or its attachments may result in legal action against any person who uses it. ___ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
RE: [LARTC] NAT-aware traffic analysis
A different approach is to use iptables counters in FORWARD chain (-s $CLIENT_IP -i eth0 -o ! eth0). That would require a rule for each user. -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Ming-Ching Tiew Sent: Wednesday, September 05, 2007 11:09 AM To: lartc@mailman.ds9a.nl Subject: [LARTC] NAT-aware traffic analysis I have tried using iptraf for my NAT firewall to analyse the IP traffic. Basically I am faced with this difficulty of related the source IP to the outgoing interface to the internet, so I am wondering if anyone has a suggestion for a different ways to do it, or a suggestion for a better tool. Details :- Supposed : eth0 - LAN eth1 - WAN1 eth2 - WAN2 And then all source IPs in the LAN are SNAT to the respective WAN interface when leave for internet. There are also DNAT traffic from internet to the LAN. I want to breakdown the statistic of LAN users using the internet. If I run iptraf on eth0, I will see the LAN stats, but I don't know for sure which one really go out to which WAN ( some traffic does not even go out to the WAN at all ! ). Then when I sniff at eth1 or eth2, I lost the information about the LAN IPs. How could I do a stateful or NAT-aware traffic analysis ? Anyone has a good suggestion ? Important Warning! *** This electronic communication (including any attached files) may contain confidential and/or legally privileged information and is only intended for the use of the person to whom it is addressed. If you are not the intended recipient, you do not have permission to read, use, disseminate, distribute, copy or retain any part of this communication or its attachments in any form. If this e-mail was sent to you by mistake, please take the time to notify the sender so that they can identify the problem and avoid any more mistakes in sending e-mail to you. The unauthorised use of information contained in this communication or its attachments may result in legal action against any person who uses it. ___ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc ___ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
Re: [LARTC] NAT-aware traffic analysis
From: Salim S I [EMAIL PROTECTED] A different approach is to use iptables counters in FORWARD chain (-s $CLIENT_IP -i eth0 -o ! eth0). That would require a rule for each user. Well sort of theoretically possible but bad in pratice. If I have 300 internal users, I will have to create 300 iptable rules. Then if I want to analyse based on sport or dport, you can imagine the number of rules will be quite many. Anyone has other suggestions ? ___ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
Re: [LARTC] NAT-aware traffic analysis
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Greetings, : I have tried using iptraf for my NAT firewall to analyse the IP : traffic. Basically I am faced with this difficulty of related the : source IP to the outgoing interface to the internet, so I am : wondering if anyone has a suggestion for a different ways to do : it, or a suggestion for a better tool. I don't know of a flow analysis tool that records internal and external addresses at the NAT boundary. Without knowing how you separate your traffic outbound, it'd be hard for us to guess what the shortcomings of any of these solutions might be, but here are a few ideas: * Record the state of /proc/net/ip_conntrack and your flow information snapshots at exactly the same time. Use the ip_conntrack state information (programmatically) to yield the answers you want about usage information. * Use a flow analysis tool (e.g., argus) to record the flow information on your internal interface. Since you built the rules for distributing traffic and selecting the path for outbound flows, you should be able to map this same logic onto your recorded flows. In short, I think you may have better luck approaching the problem as a flow-analysis problem than a statistical summarization of traffic on any specific interface. Good luck, - -Martin - -- Martin A. Brown http://linux-ip.net/ -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.2 (GNU/Linux) Comment: pgf-0.72 (http://linux-ip.net/sw/pine-gpg-filter/) iD8DBQFG3i65HEoZD1iZ+YcRAkqiAJ4rp7p3Sg+b4i0PYvpXRlHZtrm/ogCfe52L 00fFE3OOeNHP8QIiTRuB9LM= =Egrt -END PGP SIGNATURE- ___ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
