Re: [Cake] flow dissector idea/enhancement - help

[Kevin Darbyshire-Bryant]

On 01/07/16 09:11, Kevin Darbyshire-Bryant wrote:



I'm beginning to think there's a reason why enhanced sfq which is 
where I found that code never made it upstream :-/   It's been/being 
an interesting little diversion...and I've modified a kernel module 
that so far doesn't crash :-)


KDB
The brain cell awoke with the thought that 'we don't need all the 
conntrack info filled in, all we need is to find the conntrack entry 
matching the src/dst tuple depending on direction. nf_ct_get(skb, 
) is a function of conveniencewe need to do it the 
inconvenient way.  there's gotta be conntrack tuple lookups available 
and we must have enough info for this as the skb is right there.  KDB 
goes digging.


KDB


If I'm honest I'm getting a little despondent.  Coding right at the 
limit of your understand of C and the kernel and everything else is 
really, really tiring and hard.


I discovered net/ipv4/netfilter/nf_conntrack_l3proto_ipv4.c has static int
getorigdst(struct sock *sk, int optval, void __user *user, int *len)  
The comment above it is


/* Fast function for those who don't want to parse /proc (and I don't
   blame them). */
/* Reversing the socket's dst/src point of view gives us the reply
   mapping. */

Those interested to see how bad my C really is can 
https://github.com/kdarbyshirebryant/sch_cake/tree/demasq


Maybe those who know what they're doing can offer some assistance.

Cheers,

Kevin


___
Cake mailing list
Cake@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/cake


___
Cake mailing list
Cake@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/cake

Re: [Cake] flow dissector idea/enhancement - help

[Kevin Darbyshire-Bryant]

On 30/06/16 20:23, Kevin Darbyshire-Bryant wrote:



On 30/06/16 11:27, Jonathan Morton wrote:
On 30 Jun, 2016, at 12:33, Kevin Darbyshire-Bryant 
 wrote:


+#ifdef CONFIG_NET_SCH_ESFQ_NFCT
+   enum ip_conntrack_info ctinfo;
+   struct nf_conn *ct = nf_ct_get(skb, );
+#endif
Good find.  If this actually works the way we want it to, it’ll make 
all the host-dependent modes (including triple-isolation) much more 
useful on the outer side of a NAT.


My main concern is that the conntrack state might not be sorted out 
until it hits the firewall or routing logic.  I’ll be very pleased if 
it happens sooner, or is actually triggered by the query rather than 
passing to some specific stage of processing.


I have other work to do on the host and flow processing, but I think 
that’ll be independent of the hash function, which is where you want 
to be looking.
I knew I shouldn't have got so excited.  It sort of half works-ish. I 
hacked something together and then added a few printk statements to 
help with debugging.  For testing I setup 'dual-dst' on the ifb4wan 
(ingress) interface, and set 'dual-src' on the wan4 (egress) 
interface.  Testing showed the input side of things hadn't got as far 
as conntrack association by the time it hit the qdisc.  The output 
side of things definitely has conntrack associations. They're a bit 
strange and I need to do a bit more digging to see if there's anything 
useful here, it might be possible to do per host egress fairness.   
For reasons I don't understand it looks as though I'm occasionally 
seeing the 'reply' conntrack association and not just the 
'incoming'...in other words I'm seeing 'internet' source addresses, 
not just internal lan.


I'm beginning to think there's a reason why enhanced sfq which is 
where I found that code never made it upstream :-/   It's been/being 
an interesting little diversion...and I've modified a kernel module 
that so far doesn't crash :-)


KDB
The brain cell awoke with the thought that 'we don't need all the 
conntrack info filled in, all we need is to find the conntrack entry 
matching the src/dst tuple depending on direction.  nf_ct_get(skb, 
) is a function of conveniencewe need to do it the 
inconvenient way.  there's gotta be conntrack tuple lookups available 
and we must have enough info for this as the skb is right there.  KDB 
goes digging.


KDB
___
Cake mailing list
Cake@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/cake

Re: [Cake] flow dissector idea/enhancement - help

[Jonathan Morton]

> On 30 Jun, 2016, at 12:33, Kevin Darbyshire-Bryant 
>  wrote:
> 
> +#ifdef CONFIG_NET_SCH_ESFQ_NFCT
> +   enum ip_conntrack_info ctinfo;
> +   struct nf_conn *ct = nf_ct_get(skb, );
> +#endif

Good find.  If this actually works the way we want it to, it’ll make all the 
host-dependent modes (including triple-isolation) much more useful on the outer 
side of a NAT.

My main concern is that the conntrack state might not be sorted out until it 
hits the firewall or routing logic.  I’ll be very pleased if it happens sooner, 
or is actually triggered by the query rather than passing to some specific 
stage of processing.

I have other work to do on the host and flow processing, but I think that’ll be 
independent of the hash function, which is where you want to be looking.

 - Jonathan Morton

___
Cake mailing list
Cake@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/cake

Re: [Cake] flow dissector idea/enhancement - help

[Kevin Darbyshire-Bryant]

On 02/06/16 13:29, Jonathan Morton wrote:

On 2 Jun, 2016, at 14:09, Kevin Darbyshire-Bryant 
 wrote:

Cake uses the flow dissector API to do flow hashing...including per host flows 
for dual/triple isolation.  The unfortunate bit is that the qdisc inevitably 
gets placed after packets have been NATed on egress and before they've been 
de-NATed on ingress.

When mentioned before Johnathan said "flow dissector ideally needs to be tweaked to 
do this" or words to that effect.

I'd like to progress that idea...the thought of me kernel programming should 
horrify everyone but really I'm asking for help in being pointed in the right 
direction to ask for help...and go from there :-)

I believe Linux does NAT using a “connection tracker” subsystem.  That would 
contain the necessary data for resolving NAT equivalents.  I don’t know how 
easy it is to query in a qdisc context, though.
Imagine my joy of discovering http://fatooh.org/esfq-2.6/  - someone has 
already bl**dy done itand I found it lurking in LEDE as part of a patch.


So there relevant bits are something of the order:


+#ifdef CONFIG_NET_SCH_ESFQ_NFCT
+   enum ip_conntrack_info ctinfo;
+   struct nf_conn *ct = nf_ct_get(skb, );
+#endif

+#ifdef CONFIG_NET_SCH_ESFQ_NFCT
+   /* defaults if there is no conntrack info */
+   info.ctorigsrc = info.src;
+   info.ctorigdst = info.dst;
+   info.ctreplsrc = info.dst;
+   info.ctrepldst = info.src;
+   /* collect conntrack info */
+   if (ct && ct != _conntrack_untracked) {
+   if (skb->protocol == __constant_htons(ETH_P_IP)) {
+   info.ctorigsrc = 
ct->tuplehash[IP_CT_DIR_ORIGINAL].tuple.src.u3.ip;
+   info.ctorigdst = 
ct->tuplehash[IP_CT_DIR_ORIGINAL].tuple.dst.u3.ip;
+   info.ctreplsrc = 
ct->tuplehash[IP_CT_DIR_REPLY].tuple.src.u3.ip;
+   info.ctrepldst = 
ct->tuplehash[IP_CT_DIR_REPLY].tuple.dst.u3.ip;

+   }
+   else if (skb->protocol == __constant_htons(ETH_P_IPV6)) {
+   /* Again, hash ipv6 addresses into a single u32. */
+   info.ctorigsrc = 
jhash2(ct->tuplehash[IP_CT_DIR_ORIGINAL].tuple.src.u3.ip6, 4, 
q->perturbation);
+   info.ctorigdst = 
jhash2(ct->tuplehash[IP_CT_DIR_ORIGINAL].tuple.dst.u3.ip6, 4, 
q->perturbation);
+   info.ctreplsrc = 
jhash2(ct->tuplehash[IP_CT_DIR_REPLY].tuple.src.u3.ip6, 4, q->perturbation);
+   info.ctrepldst = 
jhash2(ct->tuplehash[IP_CT_DIR_REPLY].tuple.dst.u3.ip6, 4, q->perturbation);

+   }
+
+   }
+#endif

I'd rip out the IPv6 conntrack stuff as I'm much more concerned by 
handling IPv4 NAT.  And I'm not sure how to get it into cake's host 
handling yet but


I can feel an experiment and hackery coming on later today :-)

Am overjoyed!
___
Cake mailing list
Cake@lists.bufferbloat.net
https://lists.bufferbloat.net/listinfo/cake