a better list for this sort of question is the cerowrt-devel list. Regrettably the load balancing part of the question requires more brain cells than I have available at the moment. The SQM scripts could be hacked up to do what you want, I think, but I have to think about it (and this week is ietf which I am trying to put to bed)
On Fri, Oct 31, 2014 at 4:23 AM, Carlos R. Pasqualini <[email protected]> wrote: > Hi all > > I opt to write this lengthy mail to document all the process, i hope these > not be a problem to the list's members. > > > In our campus we have a Debian box managing the bulk internet access of our > users. > > For the sake of simplicity, until everything works as expected, i have put > in separated machines the firewall rules, intervlan rules and every other > complexity (found in the FWLAN Box), from the load balance and QoS which is > the machine i will explain; this machine i called 'BALANCER'. > > > On the LAN side i have a real NIC, eth0, with IP 10.110.84.1/24 > > On the WAN side i have one NIC, eth1, with N ADSLs (3 at the moment but, i > need to be ready to increase that number so i prefer to build a script with > a N variable and not a fixed number) each one in his own VLAN: > > > |-- eth1.1001 -- 10.12.1.244/24 gw 10.12.1.2 > |Local VLANS 172.24/16| -- [FWLAN] 10.110.84.2 -- eth0 -- [BALANCER] > eth1--|-- eth1.1002 -- 10.12.2.244/24 gw 10.12.2.2 > > |-- eth1.1003 -- 10.12.3.244/24 gw 10.12.3.2 > > Everything works (almost) as expected without QoS (suffering bufferbloat, > but that is a queue problem, not a router problem) > > > So yesterday i was working on adding some QoS (HTB/fq_codel) to this ADSLs. > > As documented, to be effective, fq_codel needs to be in the bottleneck so i > need to shape the traffic of each ADSL line to numbers under those > advertised by our ISP (Thinking to using HTB for it) > > Reading somewhere, i found that i cannot apply a queue to a VLAN interface, > only to a real one, so the suggested solution was to put a bridge on each > VLAN and apply the queue to each bridge which can apply those queues > > well i have made that modification: > > |-- eth1.1001 -- br1 -- 10.12.1.244/24 gw 10.12.1.2 > eth0 -- BALANCER eth1--|-- eth1.1002 -- br2 -- 10.12.2.244/24 gw 10.12.2.2 > |-- eth1.1003 -- br3 -- 10.12.3.244/24 gw 10.12.3.2 > > and with some little tweaking i was able to get it to work (still without > queues or shaper) > > The script that seems to be working is (or the interesting part of it): > > firewall_balancer_bridges.sh > ------------------------ > $IP route del default > $IP rule show|grep ADSL|awk '{gsub($1,"ip rule delete",$0);print $0}'|bash > > $IPT -t mangle -A PREROUTING -j CONNMARK --restore-mark > $IPT -t mangle -A PREROUTING -m comment --comment "this stream is already > marked; escape early" -m mark ! --mark 0 -j ACCEPT > > ROUTERS="route add default scope global " > for LINK in `links`; do > TABLE=`link_table $LINK` > FWMARK=`link_fwmark $LINK` > LINKIP=`if_address $LINK` > LINKPEER=`if_peer $LINK` > > $IP route add $LINKPEER dev $LINK src $LINKIP table $TABLE > $IP route add default via $LINKPEER table $TABLE > $IP route add 172.23.0.0/16 via 10.110.84.2 dev eth0 table $TABLE > $IP route add 172.24.0.0/16 via 10.110.84.2 dev eth0 table $TABLE > $IP route add 172.25.0.0/16 via 10.110.84.2 dev eth0 table $TABLE > $IP route add 10.110.84.0/24 dev eth0 table $TABLE > > $IP route add $LINKPEER dev $LINK src $LINKIP > $IP rule add from $LINKIP table $TABLE > $IP rule add iif $LINK table $TABLE > $IP rule add oif $LINK table $TABLE > $IP rule add fwmark $FWMARK table $TABLE > > $IPT -t mangle -N $TABLE > $IPT -t mangle -A $TABLE -m comment --comment "send via $TABLE" -j > MARK --set-mark $FWMARK > $IPT -t mangle -A $TABLE -j CONNMARK --save-mark > $IPT -t mangle -A $TABLE -j RETURN > > $IPT -t mangle -A PREROUTING -m comment --comment "prevent > asynchronous routing $LINK" -i $LINK -m conntrack --ctstate NEW -j $TABLE > > $IPT -t mangle -A OUTPUT -o $LINK -m conntrack --ctstate NEW -j > $TABLE > $IPT -t mangle -A POSTROUTING -o $LINK -m conntrack --ctstate NEW -j > $TABLE > > $IPT -t nat -A POSTROUTING -o $LINK -j SNAT --to-source $LINKIP > if [[ `ping -c $TCANT -I $LINK 8.8.8.8|grep -c '64 bytes'` -eq > $TCANT ]]; then > ROUTERS="$ROUTERS nexthop via $LINKPEER dev $LINK weight 1" > else > echo "La ruta $TABLE/$LINK presenta pérdida de paquetes" > fi > done > $IP $ROUTERS > > > echo 1 > /proc/sys/net/ipv4/ip_forward > ---------------------------- > > (In Argentina we do not have IPv6 yet on the ISPs, so i want it working IPv4 > only right now and add IPv6 later) > > > I found a script which seems to be OK in: > > http://wiki.gentoo.org/wiki/Traffic_shaping > > > And started to adapt it to suit my needs, and here is where everything broke > up: > > > firewall_balancer_bridges_shaper.sh > ----------------------------------------- > ... > # Ensure to have enough IFBs > modprobe ifb numifbs=8 > ... > ROUTERS="route add default scope global " > for LINK in `links`; do > TABLE=`link_table $LINK` > FWMARK=`link_fwmark $LINK` > LINKIP=`if_address $LINK` > LINKPEER=`if_peer $LINK` > IFVDEV=`link_ifb $LINK` > > > ############################################################################ > # Traffic Shaper > > # Clear old queuing disciplines (qdisc) on the interfaces > $TC qdisc del dev $LINK root > $TC qdisc del dev $LINK ingress > $TC qdisc del dev $IFVDEV root > $TC qdisc del dev $IFVDEV ingress > > # Ingress > > # Create ingress on external interface > $TC qdisc add dev $LINK handle ffff: ingress > > $IFC $IFVDEV up # if the interace is not up bad things happen > > # Forward all ingress traffic to the IFB device > $TC filter add dev $LINK parent ffff: protocol all u32 match u32 0 0 > action mirred egress redirect dev $IFVDEV > > # Create an EGRESS filter on the IFB device > $TC qdisc add dev $IFVDEV root handle 1: htb default 11 > > # Add root class HTB with rate limiting > > $TC class add dev $IFVDEV parent 1: classid 1:1 htb rate $BWDOWN > $TC class add dev $IFVDEV parent 1:1 classid 1:11 htb rate $BWDOWN > prio 0 quantum $HTB_Q > > > # Add FQ_CODEL qdisc with ECN support (if you want ecn) > $TC qdisc add dev $IFVDEV parent 1:11 fq_codel quantum $CODEL_Q ecn > > # Egress > # Add FQ_CODEL to EGRESS on external interface > $TC qdisc add dev $LINK root handle 1: htb default 11 > > # Add root class HTB with rate limiting > $TC class add dev $LINK parent 1: classid 1:1 htb rate $BWUP > $TC class add dev $LINK parent 1:1 classid 1:11 htb rate $BWUP prio > 0 quantum $HTB_Q > > # Note: You can apply a packet limit here and on ingress if you are > memory constrained - e.g > # for low bandwidths and machines with < 64MB of ram, limit 1000 is > good, otherwise no point > > # Add FQ_CODEL qdisc without ECN support - on egress it's generally > better to just drop the packet > # but feel free to enable it if you want. > > $TC qdisc add dev $LINK parent 1:11 fq_codel quantum $CODEL_Q noecn > > > > ############################################################################ > > $IP route add $LINKPEER dev $LINK src $LINKIP table $TABLE > $IP route add default via $LINKPEER table $TABLE > ... > -------------------------- > > > but it stopped to work... i even was unable to ping each ADSL's router IP > (10.12.1.2) > I reverted to the previous script (firewall_balancer_bridges.sh) and it > worked but without fq_codel :( > > > I feel stuck on this, can anyone give a hint on what to change/test and/or a > different way of doing this? > > > I don't feel sane to build a bridge for every vlan for applying a shaper (on > the FWLAN box i have 34 VLANs), this shall be done using tc only but didn't > found an example of it, but this is another story. > > > > Best Regards > > -- > > > Carlos R. Pasqualini > Administración de Redes > Facultad de Ciencias de la Alimentación > Universidad Nacional de Entre Ríos > > --------------------------------------------------------- > Recuerde que ningun Administrador de red le solicitará > el envío de una contraseña por correo electrónico. > Si recibe un correo en el que se solicita una contraseña > seguramente se trata de un fraude; avise del mismo a sus > administradores de red a la brevedad. > --------------------------------------------------------- > Facultad de Ciencias de la Alimentación > Universidad Nacional de Entre Ríos > http://www.fcal.uner.edu.ar/ > > > > _______________________________________________ > Bloat mailing list > [email protected] > https://lists.bufferbloat.net/listinfo/bloat -- Dave Täht thttp://www.bufferbloat.net/projects/bloat/wiki/Upcoming_Talks _______________________________________________ Cerowrt-devel mailing list [email protected] https://lists.bufferbloat.net/listinfo/cerowrt-devel
