Re: [Bro-Dev] broctl archive copy vs move

[Azoff, Justin S]

> On Dec 13, 2016, at 3:12 PM, Aashish Sharma  wrote:
> 
> So if we have compresscmd unset then archive-log script does a copy:
> 
> archive-log:nice cp $file_name "$dest"
> 
> Any reason why it doesn't do move instead ? 
> 
> I propose changing cp to mv 

No.. I don't think there's any good reason.  The way it was originally written 
must have assumed the logs were going to be archived to a different filesystem.

-- 
- Justin Azoff


___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

[Bro-Dev] broctl archive copy vs move

[Aashish Sharma]

So if we have compresscmd unset then archive-log script does a copy:

archive-log:nice cp $file_name "$dest"

Any reason why it doesn't do move instead ? 

I propose changing cp to mv 

Aashish 
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

Re: [Bro-Dev] [Proposal] Language extensions for better Broker support

[Matthias Vallentin]

> I don't really like using a record like that, as that would associate
> specific semantics with what's really a user-definable type. 

It was only meant to illustrate the idea of error handling and function
composition. These ideas still hold up when substituting the
user-defined result type with the proper language-level construct.

> We could generalize that to support other data types as well, although
> I don't really see a need for going beyond opaque right now

Going beyond opaque would have the advantage of applying a uniform error
handling strategy to both synchronous and asynchronous code. That's not
needed right now, because "when" doesn't make it easy to handle errors,
but it could be an opportunity to provide a unified mechanism for a
currently neglected aspect of the Bro language.

> Maybe the conversion to bool that I proposed originally should really
> be a conversion to a dedicated error type, so that one can
> differentiate what happened. 

I like that.

Matthias
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

Re: [Bro-Dev] [Proposal] Language extensions for better Broker support

[Robin Sommer]

> >type result = record {
> >  error: failure 
> >  value: any 
> >}

I don't really like using a record like that, as that would associate
specific semantics with what's really a user-definable type. I.e.,
we'd hardcode into the language that the record used here needs to
have exactly these fields. On top of that, it also feels rather clumsy
to me as well. Indeed it's one of the things I don't like about the
current Broker framework API (which currently has to do it that way
just because there's no better mechanism). My draft proposal extends
opaque types to support conversion to boolean to allow for more
elegant error checks. We could generalize that to support other data
types as well, although I don't really see a need for going beyond
opaque right now (assuming we also add the cast operator, per the
proposal, so that we aren't passing anys around).

> With the proposed monad, you could factor the implementation of the
> check function check into a single line: 
> 
>   local r = put(store, key, test(lookup(store, key)));

Honestly, I think this operates on a level that's quite beyond any
other concepts that the language currently offers, and therefore
doesn't really fit in. I'm pretty certain this wouldn't really enter
the "active working set" of pretty much any Bro user out there. For
the time being at least, explicit error checking seems good/convinient
enough to me.

> r$error == timeout

> Based on how flexible we design the type in r$error, it can represent
> all sorts of errors.

That's a piece that I like: Being able to flag different error types.
Maybe the conversion to bool that I proposed originally should really
be a conversion to a dedicated error type, so that one can
differentiate what happened. I need to think about it, but that could
eliminate the need for simply aborting the event handler on timeout
(which in turn would help with the problem that Bro isn't good at
aborting code)

Robin

-- 
Robin Sommer * ICSI/LBNL * ro...@icir.org * www.icir.org/robin
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

Re: [Bro-Dev] [Proposal] Language extensions for better Broker support

[Matthias Vallentin]

> >  local r = put(store, key, test(lookup(store, key)));
> 
> For prototyping purposes, I see the convenience in that, but wonder if
> the runtime will do something that’s useful and widely applicable
> enough for that to translate directly into production code.  What
> exactly does the runtime do if “lookup” fails here besides turn the
> outer function calls into no-ops?

The runtime doesn't do anything but error propagation. That's a plus in
my opinion, because it's simple to understand and efficient to
implement.

> Just guessing, but in many cases one would additionally want to log a
> warning and others where they even want to schedule the operation to
> retry at a later time.  i.e. the treatment of the failure varies with
> the context.  Is this type of composition flexible enough for that?

It's up to the user to check the result variable (here: r) and decide
what to do: abort, retry, continue, or report an error. Based on what
constitutes a self-contained unit in an algorithm, there are natural
points where one would try again. In the above example, perhaps at the
granularity of that put-test-lookup chain. To try again after a timeout,
I would imagine it as follows:

local algorithm = function() : result {
  return put(store, key, test(lookup(store, key)))  = 3 sec;
};

local r = result();
do {
  r = algorithm(); 
} while (r?$error && r$error == timeout);

Based on how flexible we design the type in r$error, it can represent
all sorts of errors. The runtime could abort for critical errors, but
give control back to the user when it's a recoverable one.

Matthias
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

Re: [Bro-Dev] [Proposal] Language extensions for better Broker support

[Siwek, Jon]

> On Dec 13, 2016, at 6:02 AM, Matthias Vallentin  wrote:
> 
>  local r = put(store, key, test(lookup(store, key)));

For prototyping purposes, I see the convenience in that, but wonder if the 
runtime will do something that’s useful and widely applicable enough for that 
to translate directly into production code.  What exactly does the runtime do 
if “lookup” fails here besides turn the outer function calls into no-ops?

Just guessing, but in many cases one would additionally want to log a warning 
and others where they even want to schedule the operation to retry at a later 
time.  i.e. the treatment of the failure varies with the context.  Is this type 
of composition flexible enough for that?

- Jon

___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

Re: [Bro-Dev] [Proposal] Language extensions for better Broker support

[Matthias Vallentin]

> I might prefer just doing the unpacking myself. Having
> separate/explicit checks for each individual return value would
> naturally occur over time anyway (e.g. when debugging, adding extra
> logging, improving error handling).

> How common do you expect async function chains to occur?  Any specific
> examples in mind where auto-chaining is very helpful?  Maybe it’s more
> of a nice-to-have feature than a requirement?

This is a bit of a chicken-and-egg problem: we don't have much use of
when at the moment, because it's difficult to nest. If we provide a
convenient means for composition from the get-go, I'd imagine we see a
quicker adoption of the new features.

The standard workflow I anticipate is a lookup-compute-update cycle.
Here's a half-baked example:

  function test(attempts: count): result {
if (attempts > threshold)
  NOTICE(...);
return [$value = attempts + 1]; // no error
  }

  function check(..) : result {
local key = fmt("/outbound/%s", host);
local r = async lookup(store, key);
if (r?$error)
  return local?$error;
return async put(store, key, failed_attempts + 1);
  }

With the proposed monad, you could factor the implementation of the
check function check into a single line: 

  local r = put(store, key, test(lookup(store, key)));
  
Conceptually, this is equivalent to "binding" your function calls:

  local r = lookup(store, key)) 
>>= test 
>>= function(x: count) { return put(store, key, x); }; // curry

...where >>= would be the bind operator performing error/value
unpacking. Less functional clutter with the "do" notation:

  do
x1 <- lookup store key
x2 <- test x1
x3 <- put store key x2

The last two snippets derail the conversation a bit---just to show where
these ideas come from. 

Note that the function "test" is synchronous in my example and that this
doesn't matter. The proposed error handling makes it straight-forward to
mix the two invocation styles.

Matthias
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev