Hi Ed, this is my view on the subject. Your netfilter plugin is a good starting point but IMHO is not the ultimate solution for various reasons 1. I think that putting DPI code into the kernel is suboptimal, mainly for safety as the kernel is a core component and a bug in the DPI subsystem (very likely the more protocols you add) will crash the system. Better think of having a userspace app that speaks with the kernel (e.g. via NFQUEUE) so that in the worst case the app crashes but the kernel does not 2. analyzing a single flow like you do is not enough. For encrypted traffic with generic SSL certificates often sent by CDNs (content delivery networks) the DNS protocol should also be analysed (address lookup) to indetify the protocol. In essence this is is not a task for the kernel but more for a monitoring architecture such as the microcloud we are developing (http://www.ntop.org/nprobe/monitoring-on-the-microcloud/). Not using that means more load on DPI and inability to detect all protocols we could detect potentially 3. netfilter is pretty flexible yet heavy. It seems to me that your plugin is going through packets too often, and that it then should be optimised a bit, as well remove redundant code or support for old Linux versions that make the code a bit complex to read.
On Oct 27, 2012, at 8:37 PM, Ed W <[email protected]> wrote: > Hi > > Moving this discussion to the public list (note also a repost to give it a > separate thread, more people might read it...): > >> As of ndpi I am working at making http configurable so that we can >> configure urls easily for matching protocols without having to code >> them in C. This is the next step on our side, addition to better >> configurability of the code. > > Incidentally - this was something I was *just* about to email you with - you > just slipped in first!! When I look at the code it seems that the big > development is in matching HTTP(s) streams, so yes, it would be good to find > a way to avoid the situation that every website in the world gets a protocol > id in ndpi...! > > > On a related note I also observe: > > 1) With the netfilter module, the http identification of facebook/twitter > doesn't work for me. Not debugged why... Other identifications based on say > "user-agent" (eg Windows update) *do* work correctly in the netfilter module. > Some bug with "host" matching it seems > > 2) Identification via https is *far* more important and does not appear to be > synced with the http module? Gmail/Facebook/Twitter and others are now https > only, so we need to move to certificate CNAME identification and have that > synced with the HTTP module > Please try to add some debug message to track it. On CNDs as Facebook, my previous comment 2. applies > 3) What do you think about adjusting the classification of SSL into known > protocols based on port number? Eg at present IMAP without encryption is > identified as IMAP, but IMAP over SSL simply shows up as "SSL"... Why don't > we map "proto=SSL, port=585|993 == IMAPS"? Same also with the other common > protocols, encrypted smtp, pop, mysql, etc, if the port matches then map it > appropriately? This make sense to me > > I think 3) is certainly appropriate where there is technically a different > name for the encrypted protocol such as POP3 and POP3S. I can see an > argument both ways for the case where there is no special name for the > encrypted version, eg mysql..? > ok > 4) What about adding a "GROUP" or some kind of hierarchy structure to the > detected protocols. In many cases I just want to block/allow P2P, why don't > we group all these under a P2P group. This could also be done by making the > protocol name structured, eg "p2p/bittorrent". Follow-on question, one level > hierarchy or two level? eg is "POP3" part of a group "POP", which is part of > a group "MAIL", or do we just have a one level hierarchy and then a debate on > how granular that layer should be? > So you would like to have something like <protocol family>,<subprotocol>? Isn't this making things more complicated for protocols such as SVN that are both stand-alone or under http(s) Regards Luca > > Cheers > > Ed W > > > _______________________________________________ > Ntop-dev mailing list > [email protected] > http://listgateway.unipi.it/mailman/listinfo/ntop-dev > _______________________________________________ > Ntop-dev mailing list > [email protected] > http://listgateway.unipi.it/mailman/listinfo/ntop-dev _______________________________________________ Ntop-dev mailing list [email protected] http://listgateway.unipi.it/mailman/listinfo/ntop-dev
