On Mon, Jul 30, 2018 at 09:01 -0700, I wrote:
> Is there a summary somewhere of what events & topics the cluster nodes
> are currently exchanging?
So I went through the exercise of collecting this information: what
connections do we have between nodes, who's subscribing to what, and
who's publishing what; see the attached PDF. This is based on all the
standard scripts, with some special cases ignored (like the control
framework).
I'm not fully sure yet what to conclude from this, but a few quick
observations:
- The main topics are bro/cluster/<node-type> and
bro/cluster/node/<name>. For these we wouldn't have a problem
with loops if we enabled automatic, topic-driven forwading as
far as I can see.
- bro/cluster/broadcast seems to be the main case with a looping
problem, because everybody subscribes to it. It's hardly used
though. (bro/config/change is used similarly though).
- Relaying is hardly used.
- There are a couple of script-specific topics where I'm wondering
if these could switch to using bro/cluster/<node-type> instead
(bro/intel/*, bro/irc/dcc_transfer_update). In other words: when
clusterizing scripts, prefer not to introduce new topics.
- There's a lot of checks in publishing code of the type "if I am
(not) of node type X".
- Pools are used for two different things: 1. the known-* scripts
pick a proxy to process and log the information; whereas 2. the
Intel scripts pick a proxy just as a relay to broadcast stuff
out, reducing load. That 1st application is a good, but the 2nd
feels like should be handled differently.
Need to mull over this more, thoughts welcome.
Overall I have to say I found it pretty hard to follow this all
because we don't have much consistency right now in how scripts
structure their communication. That's not surprising, given that we're
just starting to use all this, but it suggests that we have room for
improvement in our abstractions. :)
Robin
--
Robin Sommer * Corelight, Inc. * ro...@corelight.com * www.corelight.com
Broker Communication.pdf
Description: Adobe PDF document
_______________________________________________ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
