Hi Robin, I thought more about your generalized idea and would like to follow up. To start, adding link-level features to the connection ID hash, while perhaps useful in some contexts, does not provide us the functionality we desire. I have an incoming feed of VLAN-tagged traffic (both VLAN and 802.1ah) with perhaps dozens of different VLANs, and I would like to handle the connections differently in scripts but also mainly in offline log analysis depending upon which VLANs the traffic is associated with.
Initially I had proposed simply adding the VLAN Ids to the conn.log file, but that is certainly too specific of a solution. What are your thoughts on exposing link-level features at the script layer for connections? For example, if all observed VLAN tags for a connection were in a set variable of the script-level Connection record, I could then label my data by matching VLAN Ids, then process them differently accordingly. Thoughts? -- Eric Thomas [email protected] On 4/17/15, 8:55 AM, "Robin Sommer" <[email protected]> wrote: >(Cc'ing bro-dev, I suggest we continue the thread there). > >This sounds generally reasonable, however I think we could take the >opportunity here to generalize this a bit more for generally including >link-layer information into connection handling. > >One thing that I didn't quite get form your description is if the >objective is really just to get the VLAN ID into conn.log, or whether >you also want to use it for defining what constitutes a connection in >the first place. The latter would aim at the situations where the same >IP addresses can appear on different VLANs for independent >connections. Right now, Bro can't keep them apart, but if we made the >VLAN part of the connection index into the session table, it would >treat them separately. Same applies to other link-level features that >could sometimes be useful to be a part of a connection's ID (like MAC >addresses). > >With that in mind, some thoughts on generalizing this (note, I not >sureif you're working from 2.3 or git master. The PktSrc API has >changed a bit recently, I'll take git as my starting point). > > - One challenge is passing the the VLAN ID through to the various > packet-related methods. You're suggesting additional parameters, > which would work. However, these methods are already taking a > bunch of parameters, and if in the future we wanted to pass > through further link-layer info, we'd have to add even more. A > more flexible alternative would be switching to simply passing a > Packet structure around that encapsulates all the information, > including what's already there (e.g., timestmap, pcap_hdr, > payload, etc.). The new PktSrc API already has such a class: > PktSrc::Packet; from a quick look I think we could elevate that > to be something passed around more generally, and then extend it > accordingly. > > - For the connections, I would store the VLAN inside the ConnID > struct, and then modify BuildConnIDHashKey() to take it into > account. That way, the session table will make it part of its > index. Same for the script-land conn_id record; that will then > make script-level tables work that index by conn_id. > > - Extending the ConnID like this could actually be made a run-time > option: I believe it shouldn't be too difficult to let users > chose the fields defining a ConnID, so that they can decide if, > say, they want to VLAN to be in there or not. We could predefine > a set of potential features to choose from, along with some > script-land API to pick the set to use, with the current 4-tuple > being the default. (This could be a 2nd step for later; if the > first two points above were in place, this extension should > become mainly a question of finding the right configuration > interface.) > >I haven't thought this thruogh too carefully, so it's conceivable that >I'm missing something. But I think it would be really helpful for many >folks to get more flexibility into the definition of what consitutes a >connection, with VLANs being a good initial target to support. > >Robin > >On Tue, Apr 14, 2015 at 16:59 +0000, you wrote: > >> Dear Bro developers, >> >> I've been tasked with trying to modify the Bro source code so that >> conn.log includes the VLAN IDs (including 802.1ah) that have been >>observed >> in packets associated with that connection. I've scoped out a solution, >> but I want to run it by you first before I start to go for it, in case >>I'm >> missing something really big. >> >> PktSrc::Process() does processing of VLAN and 802.1ah, but it just skips >> over them by advancing the data pointer. I will, in addition, store >>those >> VLAN IDs in a new member of the modified PktSrc class. This gets passed >>on >> through net_packet_dispatch() and NetSessions::DispatchPacket(). At this >> point NetSessions::NextPacket() gets called, but since the PktSrc >>doesn't >> get passed to it, I'd need another way to pass it the VLAN ID. I am >> considering two options: >> >> 1. duplicate NextPacket() adding a new parameter to pass it the VLAN >>IDs, >> and call that instead, or >> 2. store the VLAN IDs in the NetSessions class, in DispatchPacket() so >> it¹s available to NextPacket() and DoNextPacket() <- Is there a reason >> this wouldn¹t work, e.g. issues with multi-threading/multi-processing? >> >> Is there one option that seems better to you? >> >> NetSessions::DoNextPacket() is called next and I would also need a >> modification to pass it VLAN IDs, using one of the options above. In >>this >> method we finally get access to the appropriate Connection instance, so >>I >> would store the VLAN IDs in that instance in DoNextPacket(). >> >> I'd need to modify the Connection class in Conn.h to include a new >>member >> for tracking VLAN IDs. I'd modify Connection::BuildConnVal() and >> scripts/base/init-bare.bro's connection record to make the VLAN IDs >> available to scripts. Lastly, I'd write a script to redef the conn Info >> structure and handle one or more connection events (perhaps >> connection_state_remove) to copy the VLAN IDs from the connection record >> to the Info record. >> >> Is there anything I'm missing? Is there a better way to approach this? >> > > > >-- >Robin Sommer * ICSI/LBNL * [email protected] * www.icir.org/robin _______________________________________________ bro-dev mailing list [email protected] http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
