Hi Glenn, On 11/04/2013 08:37 PM, Glenn Schmottlach wrote:
I have a proposal which I hope the Connman developers will consider.
Sure :)
As I understand it, Connman sessions are tracked, in part, by marking packets associated with the session's UID, GID, or SELinux context information. This is translated into iptable rules to which "mark" the connections as described in session-overview.txt. Per session iptables rules: iptables -t mangle -A OUTPUT -m owner [--uid-owner|--gid-owner] $OWNER \ -j MARK --set-mark $MARK iptables -t filter -A INPUT -m mark --mark $MARK \ -m nfacct --nfacct-name session-input-$MARK iptables -t filter -A OUTPUT -m mark --mark $MARK \ -m nfacct --nfacct-name session-output-$MARK
BTW, the nfacct is going away. We are going to use NFQUEUE in future. Though we still need the MARK unless we can convince the netdev guys that the lookup for a policy routing table could be something else e.g. cgroup id. For the time being I keep the assumption we need the marker.
These rules instruct iptables to collect basic usage metrics (bytes/packets sent/received) on behalf of Connman. I have a requirement to do additional filtering and statistics collection using my own set of rules and would like to filter on these same MARK'ed packets while adding additional "marks" (or fields) to the packets so that I can separate them into additional categories needed by my application. This is where I need some additional information from Connman and a slight modification to how packets are MARK'ed and compared. As I understand it, each netfilter packet has an associated 32-bit mark value which is being set to $MARK in the above rules. When setting a MARK value you can provide a mask which will only allow specific bits to be set. Looking at the session.c code (line 39) I see that new sessions are initialized starting at 256 and are incremented upwards from there. So my assumption is that Connman assumes it "owns" the upper 24-bits of the netfiler mark value. Likewise, it would seem from the iptables rules that it does not expect other rules in the iptable to use these same bits for marking packets.
ConnMan starts counting at 256 because that is the first 'unmanaged' ID for a policy routing table. To simplify I used the same value for the routing table as for the marker. Obviously the current implementation doesn't really play nicely by assuming it has complete control. I have no problem to change that.
What I am proposing is the following: 1) Modify Connman to provide a mechanism to retrieve the iptable "MARK" value associated with a session as well as the bitmask used to filter/extract those bits from the underlying U32 netfilter value. Perhaps both the session MARK value and bitmask could be considered read-only "settings" of the session object and would be updated when the iptable rule is written.
I don't really follow here. Could you please elaborate it a bit?
2) Provide a configuration option (either at compile-time or via configuration file item), that specifies the offset/mask for the session's MARK value. For instance, currently the inferred offset (by examining the code) is 8 bits with 0xFFFFFFFF as the default mask. This doesn't allow the Connman markings to co-exist with any other external markings. For my application I would like to reserve the upper 16 bits for the Connman session value and the lower 16 bits for my own use. The Connman code that generates the iptables rule might be modified to look like this: iptables -t mangle -A OUTPUT -m owner [--uid-owner|--gid-owner] $OWNER \ -j MARK --set-mark $MARK/0x0000FFFF iptables -t filter -A INPUT -m mark --mark $MARK/0xFFFF0000 \ -m nfacct --nfacct-name session-input-$MARK iptables -t filter -A OUTPUT -m mark --mark $MARK/0xFFFF0000 \ -m nfacct --nfacct-name session-output-$MARK
That sounds reasonable to me. I don't think we need 2^24 ids, we should be happy with 2^16 applications :)
As you can see, setting the Connman MARK is no longer destructive to a value that might already exist in the lower 16-bits of the 32-bit word. Likewise, when reading and comparing the value the lower 16-bits of the 32-bits value are effectively masked out so it won't impact the Connman session match test. Part of my problem is that I need to create additional iptable rules that place data-caps (e.g. a quota) on certain sessions (apps) which are then optionally partitioned into additional groups. I can use the power/flexibility of iptables to implement this logic but I need to cooperatively use the same netfilter 32-bit value to store additional information about the connection and their associated packets. This change would help Connman behave as a better netfilter citizen while allowing me to piggy-back on top of the existing marking done by Connman.
Well, the quota thing is something we want also to support natively with sessions. That is also why count the packets (currently with nfacct but we don't use that information yet).
I'm willing to make the necessary code changes to implement this solution. I'm just seeking feedback of whether a better solution exists (that I'm unaware of) or perhaps recommendations on my approach. Of course any patches I make would be available to the community and hopefully could eventually be integrated into the main baseline.
I am happy to see patches! cheers, daniel _______________________________________________ connman mailing list [email protected] https://lists.connman.net/mailman/listinfo/connman
