Sorry for the late response on this >> >> Is it possible to run SEC in a purely distributed way? Like use a cluster >> of nodes, each node process a partition of the incoming events, with the >> load balanced automatically and communication traffic not overwhelming? >> > > There are no such features built into the sec core. Since by design all > memory-based correlation data structures must be fully shareable (readable > and writable) between all rules, event correlation operations, and other > sec entities, that would require implementing many complex things like > common memory for several physical nodes. Also, due to data sharing events > have to be processed sequentially -- if two events are processed in > parallel and both of them modify the same data structure (say, they assign > different values to a variable), result is unpredictable. Because of these > complexities, sec is implemented as a single threaded UNIX tool. > Whatever parallelism you want to have, you first need to split your > rulebase into several independent parts, and set up a separate sec instance > for each of those parts. Naturally, it is possible to run several instances > of sec on one node or multiple nodes (as you can do with any UNIX tool), > provided that each of those instances receives relevant events. Most > popular choices for event forwarding have probably been various syslog > servers like rsyslog and syslog-ng.
take a look at this paper http://static.usenix.org/events/lisa10/tech/full_papers/Krizak.pdf it's not about SEC, but the ideas presented for splitting the work across multiple machines, but then combining the results is applicable. David Lang ------------------------------------------------------------------------------ Dive into the World of Parallel Programming! The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net _______________________________________________ Simple-evcorr-users mailing list Simple-evcorr-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users
