Hi Stuart,
First off, thank you for the feedback you provided. Unfortunately, you
misunderstood the issue I'm encountering because you haven't read my
entire post. So, I'll try to make a brief post of the previous
message, to make it easier to read, and hope you will have the
patience to read it this time :).
First, some concepts of queueing/policing I learned from my previous
experience in other environments(correct me if I'm wrong) :
In order to apply downstream queueing/policing for a particular host one can :
- match host's destination ip address on the interface towards the
host and implement queueing on the interface towards the host.
- match host's destination ip address on interfaces towards the ISP
and implement queueing on the interface towards the host. (!!! only if
there is no NAT involved!!!, otherwise it is the outside interface IP
address that has to be matched as a destination, but this somehow
defeats the purpose, since the most common NAT scenario is that
multiple hosts get translated to a single outside address and it will
end up that the downstream of all hosts gets queued. It would only be
a succesfull queing policy on scenarios where 1-to-1 NAT mapping is
applied.)
In order to apply upstream queueing/policing for a particular host one can :
- match the host's source ip address on the interface towards the ISP
and implement queueing on the interface towards the ISP.
- match the host's source ip address on the interface towards the host
and implement rate limiting(ingress policy) on the interface towards
the host.
- when NAT is involved, mark/tag host's source ip address on the
interface towards the host and implement queueing on the interface
towards the ISP based on mark/tag.
Now, on openbsd, in order to succefully get downstream queue policy
for a particular host (192.168.110.2) I was forced to match the
destination address of a specific location (86.55.8.30) from where the
host was downloading on the interface towards the host.
This is really nonsense to me since "pass out quick on em0 to
86.55.8.30 queue ftp" or "pass in quick on em1 to 86.55.8.30 queue
ftp" it's clearly uptream queueing (em0 is ISP side and em1 host side
and ALTQ was only appplied on em1 interface).
If I apply the queue designation rules according to my understandings
nothing gets matched/queued :
~# grep -v \# /etc/pf.conf | grep -v ^$
altq on em1 cbq bandwidth 100Mb queue { ftp,other }
queue ftp on em1 bandwidth 1Mb priority 0 cbq(ecn)
queue other on em1 bandwidth 99Mb priority 1 cbq(ecn,default)
pass out log(all) quick on em1 from 86.55.8.30 to 192.168.110.2 queue ftp
pass in log(all) quick on em0 from 86.55.8.30 to 192.168.110.2 queue ftp
pass log(all) all
"tcpdump -nettipflog0 src host 86.55.8.30 and dst host 192.168.110.2"
shows that packets bypass the first two rules hitting the last one
directly :
1227709000.698775 rule 2/(match) pass out on em1: 86.55.8.30.27304 >
192.168.110.2.3202: [|tcp] (DF)
1227709000.698892 rule 2/(match) pass in on em0: 86.55.8.30.27304 >
192.168.110.2.3202: [|tcp] (DF)
1227709000.698895 rule 2/(match) pass out on em1: 86.55.8.30.27304 >
192.168.110.2.3202: [|tcp] (DF)
1227709000.699016 rule 2/(match) pass in on em0: 86.55.8.30.27304 >
192.168.110.2.3202: [|tcp] (DF)
1227709000.699019 rule 2/(match) pass out on em1: 86.55.8.30.27304 >
192.168.110.2.3202: [|tcp] (DF)
1227709000.699139 rule 2/(match) pass in on em0: 86.55.8.30.27304 >
192.168.110.2.3202: [|tcp] (DF)
1227709000.699141 rule 2/(match) pass out on em1: 86.55.8.30.27304 >
192.168.110.2.3202: [|tcp] (DF)
1227709000.699265 rule 2/(match) pass in on em0: 86.55.8.30.27304 >
192.168.110.2.3202: [|tcp] (DF)
1227709000.699268 rule 2/(match) pass out on em1: 86.55.8.30.27304 >
192.168.110.2.3202: [|tcp] (DF)
1227709000.699389 rule 2/(match) pass in on em0: 86.55.8.30.27304 >
192.168.110.2.3202: [|tcp] (DF)
1227709000.699393 rule 2/(match) pass out on em1: 86.55.8.30.27304 >
192.168.110.2.3202: [|tcp] (DF)
1227709000.699518 rule 2/(match) pass in on em0: 86.55.8.30.27304 >
192.168.110.2.3202: [|tcp] (DF)
1227709000.699521 rule 2/(match) pass out on em1: 86.55.8.30.27304 >
192.168.110.2.3202: [|tcp] (DF)
1227709000.699638 rule 2/(match) pass in on em0: 86.55.8.30.27304 >
192.168.110.2.3202: [|tcp] (DF)
1227709000.699641 rule 2/(match) pass out on em1: 86.55.8.30.27304 >
192.168.110.2.3202: [|tcp] (DF)
1227709000.699763 rule 2/(match) pass in on em0: 86.55.8.30.27304 >
192.168.110.2.3202: [|tcp] (DF)
So why do these packets bypass the first rules ?
Thank you and regards.
On Tue, Nov 25, 2008 at 11:47 PM, Stuart Henderson <[EMAIL PROTECTED]> wrote:
> On 2008-11-25, Damian Higgins <[EMAIL PROTECTED]> wrote:
>> According to what pf logged, it permits traffic incoming on em1 to
>> 86.55.8.30 and outgoing trafic on em0 to 86.55.8.30. So I modified
>> my designation rule to :
>>
>> "pass out quick on em0 to 86.55.8.30 queue ftp"
>>
>> And traffic started hitting the first rule :
>
> Good.
>
>> So it seems http://www.openbsd.org/faq/pf/queueing.html is outdated
>> since it says queue designation should take place on a different
>> interface than the one where queueing policy is applied.
>
> do you mean this, "Note that queue designation /can/ happen on
> an interface other than the one defined in the altq on directive"?
> (my emphasis)..
>
>> The result of the above queue designation rules was a constantly 1mbit
>> download rate for the initiator (192.168.110.2). So my understanding
>> is that ALTQ is per flow/session connection oriented rather than per
>> source/destination.
>
> the rules assign traffic to a queue, the bandwidth available for
> the queue is shared between all traffic assigned to it.
>
> if you're looking to limit each individual user to 1Mb/s, but not
> impose a total limit, altq can't really help.
>
> I didn't read everything of the rest of your post but hopefully the
> things I mention will help understanding:
>
>> "pass in quick on em0 from 86.55.8.30 queue ftp" and "pass out quick
>> on em1 from 86.55.8.30 queue ftp" matched nothing
>>
>> "pass out quick on em0 to 86.55.8.30 queue ftp" , "pass in quick on
>> em1 to 86.55.8.30 queue ftp" (which looks more like ingress rate
>> limiting to me) matched trafic and it is the trafic back to the
>> initiator that got queued.
>
> with altq, queueing happens when packets exit an interface, the only
> thing you can do at ingress is create state with a queue name attached
> so that any later traffic matching that state uses that queue.
>
> for traffic you are passing to other hosts, you can "fake up" ingress
> rate limiting by controlling the rate you feed those packets out,
> so for protocols which have rate control (like TCP) it will indirectly
> affect the sender's rate.
>
>> So for 192.168.6.216(initiator)'s downstream queueing I set up the
>> following pf rules :
>>
>> ~# grep -v \# /etc/pf.conf | grep -v ^$
>> altq on em0 cbq bandwidth 100Mb queue { ftp,other }
>> queue ftp on em0 bandwidth 2Mb priority 0 cbq(ecn)
>> queue other on em0 bandwidth 98Mb priority 1 cbq(ecn,default)
>> altq on em1 cbq bandwidth 100Mb queue { ftp,other }
>> queue ftp on em1 bandwidth 1Mb priority 0 cbq(ecn)
>> queue other on em1 bandwidth 99Mb priority 1 cbq(ecn,default)
>> nat on em0 from 192.168.110.2 to <test> -> 192.168.100.233
>> table <test> { any , !192.168.0.0/16 }
>> pass in quick on em0 to 192.168.110.2 queue ftp
>> pass log all
>>
>> Then started a ftp session from 192.168.6.216 with 192.168.110.2 on
>> port 2121 but nothing got matched, it was only a few packets at
>> beginning of the session :
>
> look into how the protocol works and you'll see you have to handle
> the FTP _data_ connections as well as the control connections, these
> are potentially made in either direction (for passive mode, the client
> makes a second connection to the server, for active mode the server
> connects back to the client).
>
> if you haven't seen it before, look at the actual packet contents
> on the interface (tcpdump -niem0 -s1500 -X host some.ftp.server)
> while connecting to an FTP server and listing a directory (which,
> like transferring a file, also opens an extra connection).
>
> note where new connections are opened (the letter S after the
> destination port) and in which direction.
>
> if you think you've got it, it can be educational to manually
> carry out an ftp transfer, you can do passive mode just with
> the telnet command in two windows, for active mode you also
> need something like netcat (/usr/bin/nc) which can listen for
> an incoming connection.
>
>> ~# tcpdump -nettipflog0 | grep -E 192.168.110.2\|192.168.6.216
>
> using grep on tcpdump output will catch you out. you could use
> line-buffered mode (-l) but better to use tcpdump expressions to
> select the traffic you're interested in. at least on OpenBSD,
> this works fine on pflog0. on other OS, YMMV.
>
> [also note, on pflog, by default only packets _creating state_
> are logged, you can log the others if you use "log (all)"]
