[
https://bro-tracker.atlassian.net/browse/BIT-1388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matthias Vallentin reassigned BIT-1388:
---------------------------------------
Assignee: Matthias Vallentin
> Broker's integration in Bro's main/run loop
> -------------------------------------------
>
> Key: BIT-1388
> URL: https://bro-tracker.atlassian.net/browse/BIT-1388
> Project: Bro Issue Tracker
> Issue Type: Problem
> Components: Bro, Broker
> Reporter: Jon Siwek
> Assignee: Matthias Vallentin
> Fix For: 2.5
>
>
> * There's a cost to Broker queues being idle. Whenever Broker gets a chance
> to process messages, it looks for updates to all
> connections/message-queues/data-stores. That involves sending synchronous
> messages between actors, and for empty queues, it just gets back an empty
> deque object it needs to destroy.
> * Broker queues integrate into Bro's run loop by exposing a file descriptor
> that's ready when the queue is non-empty. Users have the capability of
> adding arbitrary numbers of queues at run-time (e.g. they can freely add
> subscriptions to any amount of logs, events, etc.). Relying on select() may
> become a bottleneck if someone has hundreds of Broker queues, or could
> possibly break on some systems if FD_SETSIZE is limited to 1024.
> Ideas on how to fix:
> * Improve Bro's main run loop and dedicate an IOSource to each Broker queue
> (instead of sharing a single IOSource like they do now). There might be
> several things that could be tweaked in the main run loop, but at a minimum,
> epoll()/kqueue() could alternatively replace select(). Could also think
> about using something like libev
> (http://pod.tst.eu/http://cvs.schmorp.de/libev/ev.pod) to abstract what
> particular polling backend is used. Might even be able to use libev's timers
> to fix how Bro's timers are currently coupled w/ there being an active
> IOSource consistently driving time forward.
> * Move the draining of Broker queues completely off to their own threads.
> This maybe is adding a bit too much complexity (Broker/CAF uses threads for
> queues, then Bro would use more threads just to talk to those other
> threads...). Since CAF becomes a requirement, it may be simpler to start
> replacing/allowing some areas of Bro's threading to be done w/ CAF actors.
> And then if Broker exposed an optional API to talk directly w/ CAF actors,
> the integration w/ Bro may actually become more straightforward.
> And those ideas don't have to be mutually exclusive.
--
This message was sent by Atlassian JIRA
(v7.2.0-OD-05-030#72002)
_______________________________________________
bro-dev mailing list
[email protected]
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
