Re: [Bro-Dev] CBAN design proposal

[Siwek, Jon]

> On May 26, 2016, at 11:14 PM, Robin Sommer  wrote:
> 
>> Is it an important use case for the client to be able to interact w/
>> other repos that are structured like the one the Bro Team will be
>> hosting?  Seems less likely that people will want to do that
>> especially if the CBAN repository is easy enough for people to submit
> 
> Yeah, agree it's less important with that liberal model. I would still
> like to support it though unless it's a major pain.

Ok, I’ll plan from the beginning to be able point to multiple CBAN-like 
repository structures and then complain if I find out it’s complicated to 
support that.

>> The client's “update” command will do a “git pull” in the parent repo
>> and then a “git fetch” in any installed submodules.
> 
> So does that mean the client ignores the version that the central CBAN
> parent repository points to? Say Seth finished version 1.0 of his
> cool-plugin and files a merge request. We add the submodule to CBAN;
> it will point to 1.0. Then Seth updates to 1.1 on his end. CBAN would
> still point to 1.0, but clients would just move their local clones to
> 1.1, ignoring the parent repository's state. Is that what you have in
> mind?

Yeah, the parent repo would always point to the first version that was 
submitted, but when a user chooses to install something, that submodule gets 
initialized/cloned and the user can choose to use whatever version of it they 
want.

And it won’t be the common case for a user to be doing any actual work inside 
the parent repo (unless they’re working on the cban client code), so it 
shouldn’t be a big deal if things like `git status` in the parent repo show a 
bunch of the “new commits” messages for each installed submodule.  But I’ll 
also probably structure it so all submodules get cloned in to a subdirectory 
and then list that subdirectory in .gitignore to limit the noise.

- Jon

___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

Re: [Bro-Dev] CBAN design proposal

[Siwek, Jon]

> On May 26, 2016, at 4:07 PM, Matthias Vallentin  wrote:
> 
> How do submodules scale? Or asked differently, what are the number
> (order of magnitude) of submodules we're expecting?

I imagine it will scale because when a user clones the main/parent CBAN repo, 
we’re telling them to just clone that and *not* recursively clone all the 
submodules (unlike the process for the ‘bro’ repo that you’re familiar with).  
Then the user will pick and choose the individual submodules they want.  I 
don’t know much the average user will end up  installing, but I’m guessing in 
the order of 10s and not 100s.

Does that make sense or is there something more to your concern I might be 
missing or misunderstood?

> Is the idea that all meta data will reside in a single file? What format
> would that file be, YAML? If each user repo has 20 lines of meta data,
> then the file would be 2KB to track 100 repositories. Seems fine to me,
> but I wonder how much will end up in there to get a tractable upper
> bound.

I didn’t have a standardized format in mind to begin with, I just planned to do 
“the simplest thing that works” first and then let it evolve from there 
(clients can be updated in a way that’s resilient to changes in the format), 
but if enough people want to start w/ a particular format to begin with that’s 
fine too.

> Binaries are a big project, I believe. Not only do we need to support
> differently platforms, but also different OS versions and distributions.
> It would probably mean we have to invest into a bigger Jenkins setup,
> and then push the binaries into a CDN after a successful build.

I imagined it would still be left up to individual authors to make precompiled 
binaries available and support/maintain them.  It would also be optional for 
them to make binaries available, not required.

- Jon

___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

Re: [Bro-Dev] CBAN design proposal

[Robin Sommer]

On Thu, May 26, 2016 at 15:57 +, you wrote:

> But we are wrapping everything in the Bro plugin architecture, right?

I think we'll need to play with that a bit still to see how exactly a
minimal script module would be distributed. The binary plugins require
a bit more structure, and are loaded a bit differently, than something
that has just scripts would need. Maybe we can simplify the plugin
layout a bit to support that case better, not sure. That said, even if
using all the same structure, in my head I still associate "plugin"
with one of the binary extensions (log writer, packet source) etc. and
that's also how the documentation uses that term currently.

Robin

-- 
Robin Sommer * ICSI/LBNL * ro...@icir.org * www.icir.org/robin
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

Re: [Bro-Dev] CBAN design proposal

[Robin Sommer]

On Thu, May 26, 2016 at 14:07 -0700, you wrote:

> Is it necessary to enforce this? Intuitively, I would just leave
> namespacing to the user.

No need to enforce, but it would be good to have some guidelines at
least. And part of the guidelines should be keeping the "Bro"
namespace reserved.

> "extension" comes to mind, or maybe "bundle."

I like "bundle".

Robin

-- 
Robin Sommer * ICSI/LBNL * ro...@icir.org * www.icir.org/robin
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

Re: [Bro-Dev] CBAN design proposal

[Jan Grashöfer]

> Here are some bro'ish suggestions (not all are serious ones):
> 
> - brow (part of the logo already)
> - broom (sweep new code into bro)
> - broil (Bake your enhancements into bro)
> - broad (extends Bro in a broader sense)
> - broem (pleasing in a literary manner)
> - bropane (hot stuff for your bro)
> - brorito (original scripts from Mexico!)
> - brotein (makes you grow like a bro)
> - hebro (how about catering to Isreal?)
> - bronx (or to NYC?)
> 
> And some random word forging:
> 
> - bkg (instead of FreeBSD's pkg)
> - lion (animal I like)
> - crank (crank up that Bro)
> - scrit (tweak on "script")
> - berk (a Bro perk...or short for Berkeley)
> - bip (pip for Bro)
> - bang (starts with a "b" and sounds hip)
> - beco (Bro ecosystem)

I like bkg, broil and brow. What about bob (helps you building up you
Bro ;)?
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

Re: [Bro-Dev] CBAN design proposal

[Matthias Vallentin]

> - having both "upgrade" vs "update" commands sounds confusing, I'm
>   sure I would constantly mix up the two. Suggest to rename one of
>   them.

I think this comes from Homebrew (and maybe other package managers as
well). I kinda got used to it in this context, but occasionally still
trip over it. So yeah, I also think we should use either "update" or
"upgrade," but not both.

> - What's the policy for namespaces (both script-land and plugin-land),
>   and what's the relationship to the CBAN path? Should people use
>   their GitHub name as namespace for their module? On the one hand
>   that would be an easy way to avoid conflicts. On the other hand that
>   makes forking difficult.

Is it necessary to enforce this? Intuitively, I would just leave
namespacing to the user. Clearly, to be interoperable, a convention
could be using your github handle, but I don't see it being a necessity.
If multiple devs want to collaborate and enhance the same namespace, we
should not make this a cumbersome due to namespace hassles.

> - Originally I wanted to suggest allowing more than one plugin per git
>   repository but I really like the idea to leverage submodules, so I'm
>   skipping that suggestion. :-)

How do submodules scale? Or asked differently, what are the number
(order of magnitude) of submodules we're expecting?

> - Using a gist for meta data sounds good. We should then also have a
>   nice permanent *.bro.org URL redirecting there so that we hide the
>   actual location a bit for easier relocation.

Is the idea that all meta data will reside in a single file? What format
would that file be, YAML? If each user repo has 20 lines of meta data,
then the file would be 2KB to track 100 repositories. Seems fine to me,
but I wonder how much will end up in there to get a tractable upper
bound.

> - Any thoughts on distributing precompiled plugins? It would be nice
>   if, say, I could just install the Mac version of plugin XYZ without
>   having to compile it locally. The authors could optionally offer
>   selected prebuilds for whatever platforms they want to support.
>   Probably more a a feature for CBAN v2, but would be wort keeping
>   mind how binaries would fit in.

Binaries are a big project, I believe. Not only do we need to support
differently platforms, but also different OS versions and distributions.
It would probably mean we have to invest into a bigger Jenkins setup,
and then push the binaries into a CDN after a successful build.

> - Terminology 1: agree that we should find a better name for CBAN.

Here are some bro'ish suggestions (not all are serious ones):

- brow (part of the logo already)
- broom (sweep new code into bro)
- broil (Bake your enhancements into bro)
- broad (extends Bro in a broader sense)
- broem (pleasing in a literary manner)
- bropane (hot stuff for your bro)
- brorito (original scripts from Mexico!)
- brotein (makes you grow like a bro)
- hebro (how about catering to Isreal?)
- bronx (or to NYC?)

And some random word forging:

- bkg (instead of FreeBSD's pkg)
- lion (animal I like)
- crank (crank up that Bro)
- scrit (tweak on "script")
- berk (a Bro perk...or short for Berkeley)
- bip (pip for Bro)
- bang (starts with a "b" and sounds hip)
- beco (Bro ecosystem)

> The best alternative names I can come up with are "module" or
> "package", which are also overloaded already, but at least more
> generic.

Of the two, I prefer "module" because "package" to me reminds me of a
compiled thing, which is not always the case. Other than that,
"extension" comes to mind, or maybe "bundle."

> - We could create a public mailing list for CBAN for all discussions
>   of individual plugins/modules, including in particular for decisions
>   to remove something. Would be good to keep this all open and
>   transparent.

...and a Gitter channel, once we have converged on a name :-).

Matthias
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

Re: [Bro-Dev] CBAN design proposal

[Siwek, Jon]

> On May 26, 2016, at 10:46 AM, Robin Sommer  wrote:
> 
> - Let's keep BroControl integration in mind at leat. I agree that a
>  standalone client makes most sense right now, but if there's
>  something we can do that will make it easier for BroControl later to
>  integrate, that's worth preparing for.
> 
> - One idea along those integration lines (BroControl and even
>  elsewhere): would be nice to structure the client so that the main
>  functionality resides in a Python module with a well-defined API.
>  The client itself would then become just a small command-line
>  frontend to that library.

Ack.

> - Where's that backend-side "out-of-band process" located? Is that
>  part of the client as well? Or maybe at least part of that Python
>  module? Would be nice to keep it easy to operate for CBAN-like
>  repositories that people maintain externally.

I imagined it being part of the nightly process that does quality/metric 
gathering.

The code to do it could be part of the client, but I expect the aggregation 
process could be time consuming enough that a user wouldn’t be happy w/ it.  
Especially if CBAN grows to contain many submodules, I don’t think it would 
scale well for each client to have to do the metadata aggregation themselves.

Is it an important use case for the client to be able to interact w/ other 
repos that are structured like the one the Bro Team will be hosting?  Seems 
less likely that people will want to do that especially if the CBAN repository 
is easy enough for people to submit to and the client can handle the case of 
adding individual external repos just fine.

The metadata from individual, externally added repos would be searchable along 
with the bulk/aggregated metadata.  e.g. all the submodules a user has locally 
installed have the latest metadata available, but all the submodules the user 
has not installed will fall back on the aggregated metadata that was fetched 
the last time they ran an “update” command.

> - having both "upgrade" vs "update" commands sounds confusing, I'm
>  sure I would constantly mix up the two. Suggest to rename one of
>  them.

I do confuse them too, but it’s also what some other package managers use (e.g. 
home-brew and apt-get).  I’ll look at others package managers for more examples 
to see if I find something.

> - What's the policy for namespaces (both script-land and plugin-land),
>  and what's the relationship to the CBAN path? Should people use
>  their GitHub name as namespace for their module? On the one hand
>  that would be an easy way to avoid conflicts. On the other hand that
>  makes forking difficult.

Didn’t really have plans/ideas for a namespace policy, but checking for 
conflicts could be added to the list of things the automated nightly 
quality/metrics thing checks for.

But maybe there’s a slicker thing you could add to the Bro language itself to 
let users specify a custom namespace mapping for certain file paths allowing 
them to resolve conflicts themselves.

> - How do updating a plugin works now? If the author just updates the
>  remote code, the submodules won't move, so is there an additional
>  step there?

The client's “update” command will do a “git pull” in the parent repo and then 
a “git fetch” in any installed submodules.  Any submodules that are not 
installed are a no-op because they are just pointers that got updated by the 
“git pull”.  This means the client should now be aware of updates to all the 
plugins they have installed, but are still using the old version until they 
choose to perform the “upgrade” command which then moves the installed 
submodules to the latest version.

Or at least it will look something like that.  I haven’t had a chance to test 
if there will need to be a separate “working copy” of submodules the user 
chooses to install.

> - Any thoughts on distributing precompiled plugins? It would be nice
>  if, say, I could just install the Mac version of plugin XYZ without
>  having to compile it locally. The authors could optionally offer
>  selected prebuilds for whatever platforms they want to support.
>  Probably more a a feature for CBAN v2, but would be wort keeping
>  mind how binaries would fit in.

Didn’t think much about it yet, but I expect a plugin to be able to define its 
own build steps.  For precompiled plugins they could likely just do a no-op for 
the build step.  Maybe there’s other install paths that need to get set up that 
I’m forgetting, but that shouldn't be too much extra to get working.  Let me 
know if you had other thoughts that would make this more complicated than I’m 
expecting.

> - Terminology 2: Using "plugin" as the entity name for everything is
>  confusing I think, as right now I think most people understand it as
>  something that gets compiled; nobody calls a script a plugin (unless
>  it's a plugin for a specific script-framework; but that's even more
>  confusing then :). The best alternative names I can come up with are
>  

Re: [Bro-Dev] CBAN design proposal

[Dopheide, Jeannette M]

CBAN is a good name. It’s short, easy to say, and what the acronym means, the 
“Comprehensive Bro Archive Network”, makes sense.  But I don’t really have a 
dog in this fight. 

--
Jeannette Dopheide
Training and Outreach Coordinator
National Center for Supercomputing Applications
University of Illinois at Urbana-Champaign
 

On 5/26/16, 10:57 AM, "bro-dev-boun...@bro.org on behalf of Slagell, Adam J" 
 wrote:

> On May 26, 2016, at 10:46 AM, Robin Sommer  wrote:
> 
> - Terminology 1: agree that we should find a better name for CBAN.

BroForge?



___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

Re: [Bro-Dev] CBAN design proposal

[Slagell, Adam J]

> On May 26, 2016, at 10:46 AM, Robin Sommer  wrote:
> 
> - Terminology 1: agree that we should find a better name for CBAN.

BroForge?

> 
> - Terminology 2: Using "plugin" as the entity name for everything is
>  confusing I think, as right now I think most people understand it as
>  something that gets compiled; nobody calls a script a plugin (unless
>  it's a plugin for a specific script-framework; but that's even more
>  confusing then :). The best alternative names I can come up with are
>  "module" or "package", which are also overloaded already, but at
>  least more generic. Other suggestions?
> 

But we are wrapping everything in the Bro plugin architecture, right? I guess I 
I don’t think of plugins as necessarily binary, but maybe we shift nomenclature 
completely and start calling plugins packages across our documentation. But if 
we refer to the same thing as a plugin sometimes and a package other times, I 
think it gets even more confusing.

> - We could create a public mailing list for CBAN for all discussions
>  of individual plugins/modules, including in particular for decisions
>  to remove something. Would be good to keep this all open and
>  transparent.


I like the idea of a CBAN list.

--

Adam J. Slagell
Chief Information Security Officer
Director, Cybersecurity Division
National Center for Supercomputing Applications
University of Illinois at Urbana-Champaign
www.slagell.info

"Under the Illinois Freedom of Information Act (FOIA), any written 
communication to or from University employees regarding University business is 
a public record and may be subject to public disclosure." 









___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

Re: [Bro-Dev] CBAN design proposal

[Siwek, Jon]

Here’s a revised/alternate design based on feedback so far:

https://www.bro.org/development/projects/cban3.html

Note that I put these at unique URLs and didn’t update in place since they’re 
different enough that, if someone tried to follow along from the start of the 
thread, I didn’t think it would make sense to them.

- Jon

___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

Re: [Bro-Dev] CBAN design proposal

[Slagell, Adam J]

> On May 25, 2016, at 11:25 AM, Siwek, Jon  wrote:
> 
>> 
>> On May 25, 2016, at 9:49 AM, Slagell, Adam J  wrote:
>> 
>> So this has become an involved thread. Do we need a call, or do you think 
>> you can pull out enough to get started Jon?
> 
> Yes I can reorganize all latest ideas into an alternate design document. 

That’s all I meant. Thanks.

--

Adam J. Slagell
Chief Information Security Officer
Director, Cybersecurity Division
National Center for Supercomputing Applications
University of Illinois at Urbana-Champaign
www.slagell.info

"Under the Illinois Freedom of Information Act (FOIA), any written 
communication to or from University employees regarding University business is 
a public record and may be subject to public disclosure." 









___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

Re: [Bro-Dev] CBAN design proposal

[Siwek, Jon]

> On May 25, 2016, at 9:49 AM, Slagell, Adam J  wrote:
> 
> So this has become an involved thread. Do we need a call, or do you think you 
> can pull out enough to get started Jon?

Yes I can reorganize all latest ideas into an alternate design document.  Or if 
you meant get started working on the client, the open question about what 
metadata will be required has the most impact on how the client works, but like 
I said, I think things might also work if I start w/ the assumption that 
plugins are not required to present any metadata.

- Jon
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

Re: [Bro-Dev] CBAN design proposal

[Siwek, Jon]

> On May 25, 2016, at 12:12 AM, Robin Sommer  wrote:
> 
>> - discoverability metadata is aggregated during the nightly quality
>> check processes and automatically commits that information to the
>> “bro/cban” repo.
> 
> Would it be better to maintain this information outside of git in a
> state file that clients download? Otherwise the repository will
> clutter up quite a bit over time with tons of automatic commits.

Yeah, I like that idea.  But where to host it?  Pulling it from bro.org means 
an additional point of failure, so maybe it just all gets shoved into a 
separate gist file that lives on github?

- Jon

___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

Re: [Bro-Dev] CBAN design proposal

[Siwek, Jon]

> On May 24, 2016, at 4:46 PM, Matthias Vallentin  wrote:
> 
> More generally, there will presumably some functionality to add
> "remotes" to one's configuration, allowing plugin writers to point to
> experimental code if they wish. Then they can still hack out code and
> mix it with existing CBAN plugins, at their own risk.

Yes, part of the plan is to allow one to add arbitrary git repos.  i.e. repos 
that haven’t formally been submitted to CBAN, private ones, etc.

> On a cosmetic note, will thing be called CBAN? I find it a very cryptic
> name, often confused it with BPAN (even though it doesn't make sense),
> and was wondering whether we should converge on some more pronounceable
> candidates.

I’m just using CBAN since that seems like the name that stuck around longest 
without anything better being proposed, so I think a rename isn’t out of the 
question if there’s something else everyone comes to like.

- Jon

___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

Re: [Bro-Dev] CBAN design proposal

[Siwek, Jon]

> On May 24, 2016, at 2:52 PM, Jan Grashöfer  wrote:
> 
> 
> Imagine there was someone who published an awesome script but a new
> version of Bro breaks it. Another one patched the script and sends the
> patch to the original author. What will happen, in case he does not
> respond? Personally I don't like repositories which end up with entries
> like: "awesome-script", "awesome-script v2", "awesome-script by Jan" ...
> To avoid this one might consider to support forking plugins or organize
> the plugins user-centered ("jan/awesome-script", "anna/awesome-script").

I also think going with a hierarchical organization like that would help the 
situation you describe — if an original author of a plugin becomes unresponsive 
then another person who wants to start maintaining it could “fork” and submit 
that to cban,  Then hopefully when a cban user is searching around it would be 
apparent that a submission got forked by other users in the community and the 
answer to why and which version they should use is a few “cban info ” 
commands away.

- Jon

___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

Re: [Bro-Dev] CBAN design proposal

[Slagell, Adam J]

> On May 25, 2016, at 12:12 AM, Robin Sommer  wrote:
> 
> Overall, I agree that we can always add more restrictions later if it
> turns out necessary. It's not that we'll have 1000s of Bro modules in
> there within the first two weeks (as long as we prevent somebody
> spamming us

So this has become an involved thread. Do we need a call, or do you think you 
can pull out enough to get started Jon?
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

Re: [Bro-Dev] CBAN design proposal

[Robin Sommer]

On Tue, May 24, 2016 at 16:21 +, you wrote:

> Yeah, I have ideas, but seems like there might be another day of some
> discussion before I try to formally reframe a design doc. Here’s the
> direction I'm thinking:

I like the process you sketch, that sounds like the right way to go to
me.

A few notes, also trying to address some of Adam's comments:

> - the initial submission process involves doing a pull request on the
> bro/cban repo where the only change made is the addition of a
> submodule.  These merges probably can be automated, but if a human
> were to do it, I’d expect it wouldn’t be a time-consuming task

Yeah, maybe that initial merge is one task we leave to a human, who
could then actually take a quick 30sec look at the module to see if
it's not totally off the mark. That would address Adam's point about
what if somebody submits something that's not even a Bro thing (but I
wouldn't go further; e.g., don't try to compile, etc.. Everything
looking roughly right gets in at this time.)

> - submodules that are found to have never been in a working state are
> auto-removed (or could initially be a task that’s not a big deal for a
> human to do every so often if metrics of brokenness are consistently
> available)

Auto-removal sounds dangerous to me; there may be different reasons
why something's not in a good state. I'd leave cleanup to humans too:
if there's a module that's consistently flagged as broken, that's when
we can send a mail to the author and remove it manually if no
improvement is in sight. I'd rather err on the side of having a broken
module than remove something that could actually still be useful.

> - metadata logically can be categorized in two types, one type is
> related to discoverability (tags, author, license, etc) and one type
> is related to interoperability (version number, dependencies)

I wouldn't object to making some meta-data mandatory, per Adam's
comments. For example enforcing having an author and a license would
be useful I think. Author gets us contact information and license is
always worth clarifying.

> - discoverability metadata is aggregated during the nightly quality
> check processes and automatically commits that information to the
> “bro/cban” repo.

Would it be better to maintain this information outside of git in a
state file that clients download? Otherwise the repository will
clutter up quite a bit over time with tons of automatic commits.


Overall, I agree that we can always add more restrictions later if it
turns out necessary. It's not that we'll have 1000s of Bro modules in
there within the first two weeks (as long as we prevent somebody
spamming us).

Robin

-- 
Robin Sommer * ICSI/LBNL * ro...@icir.org * www.icir.org/robin
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

Re: [Bro-Dev] CBAN design proposal

[Slagell, Adam J]

> On May 24, 2016, at 4:46 PM, Matthias Vallentin  wrote:
> 
> If I understold it correctly, I don't think that the central CBAN
> repository will accumulate clutter. The automatic checks will help
> simply age out repos that do not comply with the minimal standards. It's
> up to the devs to comply if the want to be integrated.

I think that’s fine, but that isn’t what I thought Robin was saying. I thought 
he did not want minimal standards.

___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

Re: [Bro-Dev] CBAN design proposal

[Matthias Vallentin]

(I will respond to the actual proposal in more depth later.)

> That is a good point. I am more concerned about accumulating clutter.

If I understold it correctly, I don't think that the central CBAN
repository will accumulate clutter. The automatic checks will help
simply age out repos that do not comply with the minimal standards. It's
up to the devs to comply if the want to be integrated.

More generally, there will presumably some functionality to add
"remotes" to one's configuration, allowing plugin writers to point to
experimental code if they wish. Then they can still hack out code and
mix it with existing CBAN plugins, at their own risk.

With a small linter in place, we would also lower the bar for devs to
comply. Homebrew has a nice checker, for example.

On a cosmetic note, will thing be called CBAN? I find it a very cryptic
name, often confused it with BPAN (even though it doesn't make sense),
and was wondering whether we should converge on some more pronounceable
candidates.

Matthias
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

Re: [Bro-Dev] CBAN design proposal

[Jan Grashöfer]

> - it’s not a big deal for a submodule to temporarily enter in to a broken 
> state — cban users can always roll back to a previous version or just 
> uninstall it.  It’s up to the community to communicate/collaborate directly 
> w/ the author here to get things fixed.

I really like the community-centric approach. Regarding the discussion
about checks and consistency I think that basically all conventions,
that could be enforced automatically, will make the archive easier to
work with (for authors and for end-users). But there is another thing
that came to my mind: How are situations handled in which the author
becomes the bottleneck?

Imagine there was someone who published an awesome script but a new
version of Bro breaks it. Another one patched the script and sends the
patch to the original author. What will happen, in case he does not
respond? Personally I don't like repositories which end up with entries
like: "awesome-script", "awesome-script v2", "awesome-script by Jan" ...
To avoid this one might consider to support forking plugins or organize
the plugins user-centered ("jan/awesome-script", "anna/awesome-script").

Best regards,
Jan
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

Re: [Bro-Dev] CBAN design proposal

[Siwek, Jon]

> On May 24, 2016, at 11:49 AM, Slagell, Adam J  wrote:
> 
> I propose that we keep mandatory checks minimal, but not non-existent, and 
> then we reevaluate when we have real data about how well this works. But I 
> would really like more feedback from the community. Maybe I am an outlier 
> here?

I think starting w/ either approach could end up evolving/devolving in to the 
other? 

If you had no checks in place, but then later instituted mandatory checks, you 
might be able to have the cban client not remove things a user has already 
checked out.  So you can delist plugins if they fail the new checks, but users 
would still have the local version they can use (if somehow they’ve got it in a 
configuration that’s usable to them, but that doesn’t pass the new mandatory 
quality checks).

I lean toward starting w/ the most streamlined and least complicated approach 
and seeing what quality control checks you need to layer on top of it because 
we might just expend a lot of effort planning for problems that don’t actual 
ever pop up in practice.  But as a person that has to do development work on 
cban I might be biased toward doing what seems easier for me, so I’m fine not 
having a vote.

- Jon

___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

Re: [Bro-Dev] CBAN design proposal

[Slagell, Adam J]

> On May 24, 2016, at 12:07 PM, Siwek, Jon  wrote:
> 
>> 
>> On May 23, 2016, at 6:30 PM, Slagell, Adam J  wrote:
>> 
>> I guess there is a balance here. If we do no mandatory checks and you could 
>> submit something that isn’t even a Bro plugin, the repository could become 
>> cluttered with junk. Do we really want things that don’t even “compile”?
> 
> The clutter could still be removed by an out-of-band process.  e.g. there’s 
> no initial check for whether a submission actually works, but after X days of 
> a nightly process finding it is broken, it gets auto-removed.

That is a good point. I am more concerned about accumulating clutter.

--

Adam J. Slagell
Chief Information Security Officer
Director, Cybersecurity Division
National Center for Supercomputing Applications
University of Illinois at Urbana-Champaign
www.slagell.info

"Under the Illinois Freedom of Information Act (FOIA), any written 
communication to or from University employees regarding University business is 
a public record and may be subject to public disclosure." 









___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

Re: [Bro-Dev] CBAN design proposal

[Siwek, Jon]

> On May 23, 2016, at 6:30 PM, Slagell, Adam J  wrote:
> 
> I guess there is a balance here. If we do no mandatory checks and you could 
> submit something that isn’t even a Bro plugin, the repository could become 
> cluttered with junk. Do we really want things that don’t even “compile”?

The clutter could still be removed by an out-of-band process.  e.g. there’s no 
initial check for whether a submission actually works, but after X days of a 
nightly process finding it is broken, it gets auto-removed.

> However, where we can’t do that is with the metadata we collect. If we don’t 
> require what we think is important metadata in the beginning, then we will 
> have a gap if we decide it was important all along. So there I would err 
> towards overcorrecting in the beginning, and make things optional in the 
> future if it turns out not to be important.

I think the most important metadata has to do w/ plugin interoperability 
(versioning and dependency info) and metadata that improves discoverability and 
search features is less important.  For one reason, the former has a more 
objective correctness to it and the later is more subjective.  Having wrong or 
missing discoverability metadata is also not going to cause things to break 
this missing interoperability data would.

But even though I think interoperability metadata is important, I’m also not 
sure it needs to be collected/aggregated before plugin submissions are accepted 
— it might be something the client can collect “just in time” directly from a 
clone of the plugin itself.  Even if a plugin doesn’t initially include this, 
the expected behavior could be for the cban client to use the plugin’s master 
branch and assume it will work w/ everything.  If the user finds that not to be 
true, then they just uninstall it and ask the author to add proper 
versioning/dependency info or they might even try to add it themselves and 
submit the fixes back to the author.

Metadata that helps improve discoverability and search features 
(topics/tags/keywords, author, license, etc) I don’t see becoming so important 
but underused to the point that you’d wish it were a requirement for 
submissions to be accepted.

I don't expect adding metadata to be so much a burden that people avoid it 
entirely.  Were there other reasons to think people won’t eventually add 
metadata info even if none is initially required?

- Jon

___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

Re: [Bro-Dev] CBAN design proposal

[Slagell, Adam J]

> On May 24, 2016, at 11:21 AM, Siwek, Jon  wrote:
> 
> I think all those points make things easy on contributors, minimize direct 
> involvement of the Bro Team in sorting out problems related to particular 
> plugins, and provide a useful way for users to discover and maintain Bro 
> plugins.  There’s more potential for users to encounter broken/bad plugins, 
> but maybe that also encourages stronger community involvement w/ users more 
> likely to try and help get problems resolved.

I don’t feel like we have converged on agreement regarding the balance of 
mandatory vs. optional checks.

I think we need to define some basic metadata as a requirement for 
interoperability and discovery. Otherwise, what do we really end up providing 
above and beyond GitHub. 

Other quality checks can be optional, as long as we can change that in the 
future. I still think we should do do some basic checks to avoid completely 
broken stuff. It might mean more work for us in making sure we have good 
feedback and documentation.

In general we all want to avoid human interaction becoming a bottleneck to 
submissions.

I propose that we keep mandatory checks minimal, but not non-existent, and then 
we reevaluate when we have real data about how well this works. But I would 
really like more feedback from the community. Maybe I am an outlier here?

--

Adam J. Slagell
Chief Information Security Officer
Director, Cybersecurity Division
National Center for Supercomputing Applications
University of Illinois at Urbana-Champaign
www.slagell.info

"Under the Illinois Freedom of Information Act (FOIA), any written 
communication to or from University employees regarding University business is 
a public record and may be subject to public disclosure." 









___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

Re: [Bro-Dev] CBAN design proposal

[Siwek, Jon]

> On May 23, 2016, at 4:59 PM, Robin Sommer  wrote:
> 
>> That would make life easier for authors, and that’s maybe even a
>> higher priority than maximizing the quality/consistency of user
>> experience because, without authors, there won’t be much for users to
>> experience in the first place.
> 
> Yeah, that's exactly what I'm advocating: making it easy should really
> be priority number one, with everything else coming second. If you see
> ways to adapt the design to target that specifically, I'm all for it.

Yeah, I have ideas, but seems like there might be another day of some 
discussion before I try to formally reframe a design doc. Here’s the direction 
I'm thinking:

- still have a "bro/cban" github repository, but it now contains git submodules 
that point directly to other github accounts/repos

- the initial submission process involves doing a pull request on the bro/cban 
repo where the only change made is the addition of a submodule.  These merges 
probably can be automated, but if a human were to do it, I’d expect it wouldn’t 
be a time-consuming task — just check if the change is adding a submodule and 
then click a button to merge (don’t even have to look at the contents of the 
submodule).

- a person that has submitted something to cban needs no further interaction 
with it and they resume their typical development workflow — cban client's 
“update” command will fetch/pull directly from their git repo.

- all submodules get scanned by an out-of-band nightly process which checks for 
brokenness and other quality metrics

- submodules that are found to have never been in a working state are 
auto-removed (or could initially be a task that’s not a big deal for a human to 
do every so often if metrics of brokenness are consistently available)

- it’s not a big deal for a submodule to temporarily enter in to a broken state 
— cban users can always roll back to a previous version or just uninstall it.  
It’s up to the community to communicate/collaborate directly w/ the author here 
to get things fixed.

- metadata associated w/ plugins is all optional, but its existence contributes 
to some arbitrary “quality” rating/metrics

- metadata logically can be categorized in two types, one type is related to 
discoverability (tags, author, license, etc) and one type is related to 
interoperability (version number, dependencies)

- discoverability metadata is aggregated during the nightly quality check 
processes and automatically commits that information to the “bro/cban” repo.  
Without doing this, I think cban clients would have an incredibly slow “search” 
command that goes out to each submodule individually and gathers metadata.  
(features related to discoverability might be lower priority in general)

- interoperability metadata can also be aggregated nightly along the 
discoverability metadata, but when the cban client is actually going to perform 
specific operations on a particular submodule, it gets this data directly from 
the cloned submodule(s) to make sure the info is up-to-date.  Version numbering 
can probably be done via git tags, but dependency info stored in a canonically 
named text file.

I think all those points make things easy on contributors, minimize direct 
involvement of the Bro Team in sorting out problems related to particular 
plugins, and provide a useful way for users to discover and maintain Bro 
plugins.  There’s more potential for users to encounter broken/bad plugins, but 
maybe that also encourages stronger community involvement w/ users more likely 
to try and help get problems resolved.

- Jon

___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

Re: [Bro-Dev] CBAN design proposal

[Slagell, Adam J]

> On May 23, 2016, at 5:40 PM, Robin Sommer  wrote:
> 
>> 
>> When a contributor submits a new script, there would be some mandatory
>> checks that would need to pass for the script to be included:
> 
> The "mandatory" is where I disagree. I believe there's just to much
> involved with any initial vetting, even if conceptually simply, that
> it will create a bottleneck.

I guess there is a balance here. If we do no mandatory checks and you could 
submit something that isn’t even a Bro plugin, the repository could become 
cluttered with junk. Do we really want things that don’t even “compile”? I 
guess we can wait and see for some of these decisions, meaning start with 
optional and decide to make them mandatory if it becomes a problem. 

However, where we can’t do that is with the metadata we collect. If we don’t 
require what we think is important metadata in the beginning, then we will have 
a gap if we decide it was important all along. So there I would err towards 
overcorrecting in the beginning, and make things optional in the future if it 
turns out not to be important.

___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

Re: [Bro-Dev] CBAN design proposal

[Robin Sommer]

On Mon, May 23, 2016 at 11:22 -0500, you wrote:

> When a contributor submits a new script, there would be some mandatory
> checks that would need to pass for the script to be included:

The "mandatory" is where I disagree. I believe there's just to much
involved with any initial vetting, even if conceptually simply, that
it will create a bottleneck. Take your first question as an example:
"Is the plugin structure valid?" We wouldn't get very far with a
simple yes/no answer. We'd have to explain what exactly is not valid,
with some of that being just convention, or maybe something that
matters in general for plugins but for some reason not the particular
one. Or what if the plugin works with some version of Bro, but not
another. Are we going say it needs to work with the current release?
What if a new release comes out and breaks all existing plugins? What
if then an update for a plugin comes in that doesn't move to the new
version yet?

> At this point, though, I think we could run some "quality tests" and
> give the plugin a quality score. This might be things like:

Yep, I'm fully on board with this part, that's good information we can
provide to users about the state of something. And that state could be
"totally broken". :-)

Robin

-- 
Robin Sommer * ICSI/LBNL * ro...@icir.org * www.icir.org/robin
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

Re: [Bro-Dev] CBAN design proposal

[Robin Sommer]

On Mon, May 23, 2016 at 17:26 +, you wrote:

> To clarify, is the concern w/ the amount of non-automatable tasks or
> with the model of requiring authors to “ping” some middle-man in order
> for updates to their plugin to become visible to all CBAN users?

Both actually. Putting us into the path introduces delay and requires
somebody making time available. This is already not working well for
the current plugin repository, where things stall because we would
like to provide feedback and help guide the author along, but then
don't have the cycles for actually doing that. The delay/effort will
be shorter/less the more tasks can be automated, but at the beginning
we won't have much automation in place I imagine and even long-term
certain stuff could never be automated. So I'm wondering if we really
need to be in the path at all, seems that can only cause friction that
would be better to avoid in particular as we kick things off.

> Because most what I had outlined could be automated

Yep, understood, my mail was not directly targetting your proposal;
that just triggered me thinking about this again. My comments were
meant more broadly, we've been discussing different notions of vetting
over time with various subsets of people.

> That would make life easier for authors, and that’s maybe even a
> higher priority than maximizing the quality/consistency of user
> experience because, without authors, there won’t be much for users to
> experience in the first place.

Yeah, that's exactly what I'm advocating: making it easy should really
be priority number one, with everything else coming second. If you see
ways to adapt the design to target that specifically, I'm all for it.

Robin

-- 
Robin Sommer * ICSI/LBNL * ro...@icir.org * www.icir.org/robin
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

Re: [Bro-Dev] CBAN design proposal

[Siwek, Jon]

> On May 21, 2016, at 5:44 PM, Robin Sommer  wrote:
> 
> I'm getting concerned that we're creating an unncessary bottleneck by
> imposing the Bro Team into the critical path for submissions and
> updates. Why not let people control their stuff themselves? They'd
> register things with CBAN to make it available to everybody, but we'd
> stay out of the loop for anything further. 

To clarify, is the concern w/ the amount of non-automatable tasks or with the 
model of requiring authors to “ping” some middle-man in order for updates to 
their plugin to become visible to all CBAN users?

Because most what I had outlined could be automated (apart from the suggestion 
that initial submissions have someone from the Bro Team quickly review whether 
it looks legit).

Though I am on board for trying to draw up a contrasting design where the focus 
is on directly connecting users w/ plugin authors without any blocking 
processes or bureaucracy in the middle-man.  That would make life easier for 
authors, and that’s maybe even a higher priority than maximizing the 
quality/consistency of user experience because, without authors, there won’t be 
much for users to experience in the first place.

- Jon

___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

Re: [Bro-Dev] CBAN design proposal

[Robin Sommer]

On Sat, May 21, 2016 at 23:16 +, you wrote:

> I think there is a broad spectrum from no interaction to vetting and
> pulling into the main repository. It is about finding the right
> balance.

Yep, and I'm arguing for going very far towards the "no interaction"
side, with just some automatic checks for the most crucial things.
Maybe the initial pull request for integration could be merged
manually to avoid any spamming, but any updates for example should not
require any interaction from us.

> I do think there is another level of non blocking checks and quality
> control we can provide.

Yes, indeed, I like this. With things already merged in, we can in
parallel, in the background, build up a toolbox of things to check for
and mark a module accordingly.

Robin

-- 
Robin Sommer * ICSI/LBNL * ro...@icir.org * www.icir.org/robin
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

Re: [Bro-Dev] CBAN design proposal

[Slagell, Adam J]

> On May 21, 2016, at 5:44 PM, Robin Sommer  wrote:
> 
> As I read through the design doc, I started questioning our plan of
> curating CBAN content. I know that's what we've been intending to do,
> but is that really the best approach? I don't know of script
> repositories for other languages that enforce quality control on
> submissions beyond checking technical conventions like certain meta
> data being there.

I think there is a broad spectrum from no interaction to vetting and pulling 
into the main repository. It is about finding the right balance. 

I agree with minimal restrictions that block submissions. There needs to be 
some basic quality control and standardization there. For example, do you have 
all the right pieces. 

I do think there is another level of non blocking checks and quality control we 
can provide. For example, we can do checks for exec calls and give warnings to 
users. I think Puppet Forge has a nice model here. We won't block a submission, 
but these checks encourage better development and help new users trust 
submissions. That said, I think these must be automated. They can't block on a 
human reviewing them. 

Finally, I think we need a way to let the whole community endorse scripts or 
script authors. 
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

Re: [Bro-Dev] CBAN design proposal

[Aashish Sharma]

> In other words, my proposal is to put authors into control of their
> code, and make them fully responsible for it too --- not us. We'd just
> connect authors with users, with as little friction as possible.
> 

I support this completely. 

> If we want some kind of quality measure, we could introduce stars for
> modules that user assign if they like something; or even some facility
> to leave comments or other feedback that's visible to everybody. We

I think community vetting and feedback (and experience stories) is the right 
way to go here. 

Bro team vetting something will be very hard. My personal experience says there 
are times when scripts bring surprises weeks and months down the line - so 
testing isn't merely run a few days and give an OK.

Aashish 


On Sat, May 21, 2016 at 03:44:01PM -0700, Robin Sommer wrote:
> 
> 
> On Fri, May 20, 2016 at 20:16 +, Jon wrote:
> 
> > Here’s an updated design doc for CBAN, a plugin manager for Bro, with
> > some new ideas on how it could work:
> 
> Cool, thanks. I'm going to send some feedback but first I wanted to
> bring something up that might be controversial. :-)
> 
> As I read through the design doc, I started questioning our plan of
> curating CBAN content. I know that's what we've been intending to do,
> but is that really the best approach? I don't know of script
> repositories for other languages that enforce quality control on
> submissions beyond checking technical conventions like certain meta
> data being there.
> 
> I'm getting concerned that we're creating an unncessary bottleneck by
> imposing the Bro Team into the critical path for submissions and
> updates. Why not let people control their stuff themselves? They'd
> register things with CBAN to make it available to everybody, but we'd
> stay out of the loop for anything further. We may still want to keep a
> local copy of the code to protect against stuff going offline, but
> that could be automated. Or we could even move that burden to the
> authors as well and just keep a reference where to find the code,
> without a local copy; if it disappears, so be it.
> 
> In other words, my proposal is to put authors into control of their
> code, and make them fully responsible for it too --- not us. We'd just
> connect authors with users, with as little friction as possible.
> 
> If we want some kind of quality measure, we could introduce stars for
> modules that user assign if they like something; or even some facility
> to leave comments or other feedback that's visible to everybody. We
> could also verify if a plugins builds and loads correctly, or if tests
> pass. But we wouldn't block it if something fails, just mark it (say,
> with a banner saying "tests pass", "tests fail", "no tests").
> 
> Thoughts?
> 
> Robin
> 
> -- 
> Robin Sommer * ICSI/LBNL * ro...@icir.org * www.icir.org/robin
> ___
> bro-dev mailing list
> bro-dev@bro.org
> http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

Re: [Bro-Dev] CBAN design proposal

[Robin Sommer]

On Fri, May 20, 2016 at 20:16 +, Jon wrote:

> Here’s an updated design doc for CBAN, a plugin manager for Bro, with
> some new ideas on how it could work:

Cool, thanks. I'm going to send some feedback but first I wanted to
bring something up that might be controversial. :-)

As I read through the design doc, I started questioning our plan of
curating CBAN content. I know that's what we've been intending to do,
but is that really the best approach? I don't know of script
repositories for other languages that enforce quality control on
submissions beyond checking technical conventions like certain meta
data being there.

I'm getting concerned that we're creating an unncessary bottleneck by
imposing the Bro Team into the critical path for submissions and
updates. Why not let people control their stuff themselves? They'd
register things with CBAN to make it available to everybody, but we'd
stay out of the loop for anything further. We may still want to keep a
local copy of the code to protect against stuff going offline, but
that could be automated. Or we could even move that burden to the
authors as well and just keep a reference where to find the code,
without a local copy; if it disappears, so be it.

In other words, my proposal is to put authors into control of their
code, and make them fully responsible for it too --- not us. We'd just
connect authors with users, with as little friction as possible.

If we want some kind of quality measure, we could introduce stars for
modules that user assign if they like something; or even some facility
to leave comments or other feedback that's visible to everybody. We
could also verify if a plugins builds and loads correctly, or if tests
pass. But we wouldn't block it if something fails, just mark it (say,
with a banner saying "tests pass", "tests fail", "no tests").

Thoughts?

Robin

-- 
Robin Sommer * ICSI/LBNL * ro...@icir.org * www.icir.org/robin
___
bro-dev mailing list
bro-dev@bro.org
http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev