Re: [Bro-Dev] [Bro-Commits] [git/broker] topic/actor-system: Add broker::bro::RelayEvent message type. (ce86016)

2017-12-14 Thread Jon Siwek 
On Thu, Dec 14, 2017 at 2:07 PM, Robin Sommer  wrote:

> I'm curious what's the use case for this?

>From last month's thread regarding cluster API/topology changes, the
message pattern of sending an event from a worker to all other workers
or proxy/data to all other proxy/data nodes (within these classes,
nodes aren't connected with each other) was frequent enough to warrant
putting in a single, simple function call like this.

e.g. the next thing to come will be adding a function call to do this:

Cluster::relay_round_robin(Cluster::proxy_pool, key,
Cluster::worker_topic, event, event_args);

Which you could use to send an event from a worker to all other
workers via a rotating proxy/data node.  I think at least Justin has
been pushing for something like this.

> Generally I think it's best
> to use a combination of Broker-internal forwarding and topics to
> control where messages get propagated too, as that will better
> generalize to larger topologies later.

At least currently, Bro has the forwarding capability of Broker turned
off.  IIRC, it was very easy to unintentionally create routing loops
when it was turned on and when I asked about it, no one gave any
justification or use-case for it, thus it got turned off.

> The less of the routing we
> control manually at the Bro level, the better I'd say. But that's not
> an absolute rule of course, and I may just be missing what this is
> aiming at.

I'm probably definitely missing some info on long-term plans for
larger topologies and deep-clustering, so let me know if it's
something important to be considering, though it seems like in the
near-term goals and use-cases for the cluster topology, the implicit
routing actually may be more confusing/difficult/error-prone to use
than a simple function call like this which clearly convey's the
sender's intent and expectations.

- Jon
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

Re: [Bro-Dev] [Bro-Commits] [git/broker] topic/actor-system: Add broker::bro::RelayEvent message type. (ce86016)

2017-12-14 Thread Robin Sommer 
Hi Jon,

I'm curious what's the use case for this? Generally I think it's best
to use a combination of Broker-internal forwarding and topics to
control where messages get propagated too, as that will better
generalize to larger topologies later. The less of the routing we
control manually at the Bro level, the better I'd say. But that's not
an absolute rule of course, and I may just be missing what this is
aiming at.

Robin

On Thu, Dec 14, 2017 at 12:55 -0600, Jonathan Siwek wrote:

> Repository : ssh://g...@bro-ids.icir.org/broker
> On branch  : topic/actor-system
> 
> >---
> 
> commit ce860168961285c6d961973aa9e1ef6b7de87887
> Author: Jon Siwek 
> Date:   Thu Dec 14 12:55:52 2017 -0600
> 
> Add broker::bro::RelayEvent message type.
> 
> This is meant to be a more convenient/controlled/explicit way of doing
> simple one-hop message forwarding.
> 
> 
> >---
> 
> ce860168961285c6d961973aa9e1ef6b7de87887
>  broker/bro.hh | 36 
>  1 file changed, 32 insertions(+), 4 deletions(-)
> 
> diff --git a/broker/bro.hh b/broker/bro.hh
> index 0d7f0e7..0ec93f2 100644
> --- a/broker/bro.hh
> +++ b/broker/bro.hh
> @@ -18,6 +18,7 @@ public:
>  LogWrite = 3,
>  IdentifierUpdate = 4,
>  Batch = 5,
> +RelayEvent = 6,
>};
>  
>Type type() const {
> @@ -50,7 +51,7 @@ protected:
>  class Event : public Message {
>public:
>Event(std::string name, vector args)
> -: Message(Message::Type::Event, {name, std::move(args)}) {}
> +: Message(Message::Type::Event, {std::move(name), std::move(args)}) {}
>Event(data msg) : Message(std::move(msg)) {}
>  
>const std::string& name() const {
> @@ -62,6 +63,30 @@ class Event : public Message {
>}
>  };
>  
> +/// A Bro relayed event (automatically republished after a single hop).
> +class RelayEvent : public Message {
> +  public:
> +  RelayEvent(set relay_topics, std::string name, vector args)
> +: Message(Message::Type::RelayEvent, {std::move(relay_topics),
> +   std::move(name),
> +   std::move(args)})
> + {}
> +  RelayEvent(data msg) : Message(std::move(msg)) {}
> +
> +  const set& topics() const {
> +return get(get(msg_[2])[0]);
> +  }
> +
> +  const std::string& name() const {
> +return get(get(msg_[2])[1]);
> +  }
> +
> +  const vector& args() const {
> +return get(get(msg_[2])[2]);
> +  }
> +};
> +
> +
>  /// A batch of other messages.
>  class Batch : public Message {
>public:
> @@ -81,7 +106,8 @@ public:
>LogCreate(enum_value stream_id, enum_value writer_id, data writer_info,
>  data fields_data)
>  : Message(Message::Type::LogCreate,
> -  {stream_id, writer_id, writer_info, fields_data}) {
> +  {std::move(stream_id), std::move(writer_id),
> +   std::move(writer_info), std::move(fields_data)}) {
>}
>  
>LogCreate(data msg) : Message(std::move(msg)) {
> @@ -108,7 +134,8 @@ public:
>LogWrite(enum_value stream_id, enum_value writer_id, data path,
>  data vals_data)
>  : Message(Message::Type::LogWrite,
> -  {stream_id, writer_id, path, vals_data}) {
> +  {std::move(stream_id), std::move(writer_id),
> +   std::move(path), std::move(vals_data)}) {
>}
>  
>LogWrite(data msg) : Message(std::move(msg)) {
> @@ -131,7 +158,8 @@ public:
>  class IdentifierUpdate : public Message {
>  public:
>IdentifierUpdate(std::string id_name, data id_value)
> -: Message(Message::Type::IdentifierUpdate, {id_name, id_value}) {
> +: Message(Message::Type::IdentifierUpdate, {std::move(id_name),
> + std::move(id_value)}) {
>}
>  
>IdentifierUpdate(data msg) : Message(std::move(msg)) {
> 
> 
> 
> ___
> bro-commits mailing list
> bro-comm...@bro.org
> http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-commits
> 




-- 
Robin Sommer * ICSI/LBNL * ro...@icir.org * www.icir.org/robin
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev