[Zeek-Dev] Re: Moving policy scripts into packages
On Mon, Aug 24, 2020 at 6:43 AM Robin Sommer wrote: > 1. Move each into a a separate repository on the zeek/ GitHub >account. > > 2. Similar, but to avoid cluttering zeek/, create a new GitHub >organization "zeek-packages". I'm thinking (2). Technically, either one can likely be made to work, but (1) has a slight downside in terms of hurting a primitive browsability use-case: I imagine people want to simply scroll through a list of packages on GH, and the existing non-packages in zeek/ would distract from that goal. > 3. Put them all into a single mono-repository (e.g., >zeek/standard-packages), i.e., treat them a one package. The shortcoming of that structure is the lack of customizability. If a user only wants a specific set of functionality, or wants to avoid a set of scripts due to intrinsic overhead/conflict, it's better if they're able to start from blank slate and add only the pieces they want. > 4. Do (1) or (2), and additionally create "zeek-standard-packages" >that's full of submodules pointing to them (and also to >community packages). > > 5. Do (1) or (2), and teach zkg to understand "collections" of >packages that can be installed/managed as a group, defined >through some meta data somewhere. Between those, (5) may fit more naturally -- zkg may already be fit to handle "meta packages" via simple package metadata dependency configuration. Plus, that's a similar pattern to other package management environments AFAIK. On Mon, Aug 24, 2020 at 11:49 AM Johanna Amann via zeek-dev wrote: > This, from my point of view, it would be neat to have a way to still > easily install a rather large set of packages (potentially nearly > everything that is in policy at the moment) and run test on them. Agree that type of integration test is helpful -- may find conflicts / bad-interactions / versioning-issues that way. > Sometimes we perform changes to Zeek > that change a lot of the test baselines - especially when we touch > something that affects connection-ID hashing, or the order of elements > in hashmaps. These cases might now require an update to the test-cases > in a large number of packages. It would be neat to have an easy way to > perform this. Might be a chance to go another direction and revamp the test-writing guidelines/patterns (or provide internal config options that help produce "canonical" outputs) such that less test-cases are fragile to these kinds of widespread, low-level changes in the first place. E.g. if the order of elements in hashmaps is not defined, it's the fault of any given test-case that chose to rely on a specific order being produced. Or if an irrelevant change in UID breaks a test, the test either needs to canonify or exclude UID from its pass/fail criteria. > I think if we want to do this, we need to have a better story for > versioning than we, at the moment, have with zkg. To expand on this - at > the moment the policy scripts just work with the version of Zeek that we > distribute them with. I see the current story for versioning as this: package metadata advertises Zeek version compatibility, zkg knows how to enforce that dependency, and package authors are left in control of deciding their compatibility requirements and implementing them (likely via @if or #if directives). Once scripts that used to get distributed along with Zeek become independent packages, I agree they should start abiding by that story, but I'm not sure if there were any further ideas to help minimize the overall maintenance burden/friction associated with this new requirement. > It would be nice if, afterwards, it would still be possible to install a > working set of a script for the running version of Zeek. Meaning that - > if someone happens to run a version of Zeek that is 12 months out of > date - they should probably get the version of the policy script that is > known to work with this version of Zeek and where the tests pass with > this version of Zeek. Seems like a couple distinct questions: * What's the LTS policy for packages? It can be made different/longer than LTS policy for Zeek-proper if that's desirable. But if a 12-month LTS cycle is decided for packages, too, any extra effort spect to support older Zeeks sends mixed messages. * How to obtain an aggregate set of packages that were validated with Zeek X.Y.Z: seems like a job for the (meta)package dependency metadata to point directly to specific package versions (in the realm of git tags/branches/commits, etc.). - Jon ___ zeek-dev mailing list -- zeek-dev@lists.zeek.org To unsubscribe send an email to zeek-dev-le...@lists.zeek.org
[Zeek-Dev] Re: Moving policy scripts into packages
On 8/24/20 11:49 AM, Johanna Amann via zeek-dev wrote: * Testing: Currently, some of the policy scripts have tests that use Zeek functionality in rather unique ways / or are the only tests for some Zeek functionality. The SSL validation scripts are one example. This, from my point of view, it would be neat to have a way to still easily install a rather large set of packages (potentially nearly everything that is in policy at the moment) and run test on them. On a related note, I think it would be quite helpful if the Zeek install tree would include some of the standard btest helpers, such as random.seed and the various canonification scripts. This also comes with a fun problem. Sometimes we perform changes to Zeek that change a lot of the test baselines - especially when we touch something that affects connection-ID hashing, or the order of elements in hashmaps. These cases might now require an update to the test-cases in a large number of packages. It would be neat to have an easy way to perform this. I agree that this would be really handy. We could build some tooling that would still let you maintain the packages as individual repos but that enables such use use cases. It'd be pretty handy to have a zkg command that clones a given package plus all of its dependencies for development work, for example. Since zkg already speaks git rather well, it could even synthesize a submodule structure. Thanks, Christian ___ zeek-dev mailing list -- zeek-dev@lists.zeek.org To unsubscribe send an email to zeek-dev-le...@lists.zeek.org
[Zeek-Dev] Re: Moving policy scripts into packages
On 8/24/20 9:51 AM, Robin Sommer wrote: Also, one additional thought: Jon reminded me that zkg can manage dependencies already. So the "collection" I mentioned could be a meta-package that depends on all the ones we want. Yeah, agreed -- I prefer #2 for the same reason. Best, Christian ___ zeek-dev mailing list -- zeek-dev@lists.zeek.org To unsubscribe send an email to zeek-dev-le...@lists.zeek.org
[Zeek-Dev] Re: Moving policy scripts into packages
Hi, just a few thoughts about this. Generally - I like the idea of breaking this up. I would like to list a few thoughts about additional technical points that we should perhaps think about and that play into this decision. * Testing: Currently, some of the policy scripts have tests that use Zeek functionality in rather unique ways / or are the only tests for some Zeek functionality. The SSL validation scripts are one example. This, from my point of view, it would be neat to have a way to still easily install a rather large set of packages (potentially nearly everything that is in policy at the moment) and run test on them. This also comes with a fun problem. Sometimes we perform changes to Zeek that change a lot of the test baselines - especially when we touch something that affects connection-ID hashing, or the order of elements in hashmaps. These cases might now require an update to the test-cases in a large number of packages. It would be neat to have an easy way to perform this. * Versioning: I think if we want to do this, we need to have a better story for versioning than we, at the moment, have with zkg. To expand on this - at the moment the policy scripts just work with the version of Zeek that we distribute them with. It would be nice if, afterwards, it would still be possible to install a working set of a script for the running version of Zeek. Meaning that - if someone happens to run a version of Zeek that is 12 months out of date - they should probably get the version of the policy script that is known to work with this version of Zeek and where the tests pass with this version of Zeek. It would be super nice if this worked rather fine-granular - so even for development versions of Zeek. * Documentation: At the moment the documentation of the policy scripts just lives together with the Zeek documentation. This has a few advantages - it e.g. shows redefs that are performed in policy scripts. It would be neat to have a place that contains the combined documentation of these scripts. Currently, especially given the "update-tons-of-test-baselines-simultaneously" problem, I am kind of tempted by 3. 3 would also enable relatively easy versioning and mapping to Zeek versions that the packages are known to work with. This should also allow to keep the current testing infrastructure more or less working as it is. It does, however, not give really fine-grained access to individual packages. Johanna On Mon, Aug 24, 2020 at 01:43:21PM +, Robin Sommer wrote: > Looking for some thoughts here. One of the items on the roadmap for > 4.0 is moving scripts that currently live in policy/ over into Zeek > packages. The goals here are to (1) facilitate maintaining & testing > them independently of Zeek releases; and (2) come to a more flexible > notion of "default scripts" that can incorporate community-maintained > packages as well. This is tracked by issue > https://github.com/zeek/zeek/issues/414, including a 1st pass over the > existing policy scripts to understand what should/can be moved. > (Thanks, Vlad!) > > Before we can begin working on this, we need to figure out how to > organize this new world. One particular question is where the moved > packages will live. I see the following options so far: > > 1. Move each into a a separate repository on the zeek/ GitHub >account. > > 2. Similar, but to avoid cluttering zeek/, create a new GitHub >organization "zeek-packages". > > 3. Put them all into a single mono-repository (e.g., >zeek/standard-packages), i.e., treat them a one package. > > 4. Do (1) or (2), and additionally create "zeek-standard-packages" >that's full of submodules pointing to them (and also to >community packages). > > 5. Do (1) or (2), and teach zkg to understand "collections" of >packages that can be installed/managed as a group, defined >through some meta data somewhere. > > Along with all of this comes a question of how to make it easy for > people to install a set of default packages now that these won't come > with Zeek itself anymore. Some of the schemes above make that easier > than others. > > Thoughts/opinions/more ideas? > > Robin > > -- > Robin Sommer * Corelight, Inc. * ro...@corelight.com * www.corelight.com > ___ > zeek-dev mailing list -- zeek-dev@lists.zeek.org > To unsubscribe send an email to zeek-dev-le...@lists.zeek.org ___ zeek-dev mailing list -- zeek-dev@lists.zeek.org To unsubscribe send an email to zeek-dev-le...@lists.zeek.org
[Zeek-Dev] Re: Moving policy scripts into packages
On Mon, Aug 24, 2020 at 11:26 -0500, Michael Dopheide wrote: > I like (2) for cleanliness. Vote counted! > there should be an easy way to distinguish them from other packages > when doing a 'zkg list'. Good point. Also, one additional thought: Jon reminded me that zkg can manage dependencies already. So the "collection" I mentioned could be a meta-package that depends on all the ones we want. We might need to make that a bit more explicit for this use case (like you say for example in the output of "list"), but the basic functionality is there. Robin -- Robin Sommer * Corelight, Inc. * ro...@corelight.com * www.corelight.com ___ zeek-dev mailing list -- zeek-dev@lists.zeek.org To unsubscribe send an email to zeek-dev-le...@lists.zeek.org
[Zeek-Dev] Re: Moving policy scripts into packages
I like (2) for cleanliness. I think some people would be fine with one large package, but in other cases, you may want the ability to easily enable/disable various standard scripts. Probably not wanting to maintain the same script in multiple places, I think that eliminates (3). Towards, (4) & (5) and iven the number of standard scripts, there should be an easy way to distinguish them from other packages when doing a 'zkg list'. Maybe that's just done via a tag or in the naming scheme. -Dop On Mon, Aug 24, 2020 at 8:43 AM Robin Sommer wrote: > Looking for some thoughts here. One of the items on the roadmap for > 4.0 is moving scripts that currently live in policy/ over into Zeek > packages. The goals here are to (1) facilitate maintaining & testing > them independently of Zeek releases; and (2) come to a more flexible > notion of "default scripts" that can incorporate community-maintained > packages as well. This is tracked by issue > https://github.com/zeek/zeek/issues/414, including a 1st pass over the > existing policy scripts to understand what should/can be moved. > (Thanks, Vlad!) > > Before we can begin working on this, we need to figure out how to > organize this new world. One particular question is where the moved > packages will live. I see the following options so far: > > 1. Move each into a a separate repository on the zeek/ GitHub >account. > > 2. Similar, but to avoid cluttering zeek/, create a new GitHub >organization "zeek-packages". > > 3. Put them all into a single mono-repository (e.g., >zeek/standard-packages), i.e., treat them a one package. > > 4. Do (1) or (2), and additionally create "zeek-standard-packages" >that's full of submodules pointing to them (and also to >community packages). > > 5. Do (1) or (2), and teach zkg to understand "collections" of >packages that can be installed/managed as a group, defined >through some meta data somewhere. > > Along with all of this comes a question of how to make it easy for > people to install a set of default packages now that these won't come > with Zeek itself anymore. Some of the schemes above make that easier > than others. > > Thoughts/opinions/more ideas? > > Robin > > -- > Robin Sommer * Corelight, Inc. * ro...@corelight.com * www.corelight.com > ___ > zeek-dev mailing list -- zeek-dev@lists.zeek.org > To unsubscribe send an email to zeek-dev-le...@lists.zeek.org > ___ zeek-dev mailing list -- zeek-dev@lists.zeek.org To unsubscribe send an email to zeek-dev-le...@lists.zeek.org
[Zeek-Dev] Moving policy scripts into packages
Looking for some thoughts here. One of the items on the roadmap for 4.0 is moving scripts that currently live in policy/ over into Zeek packages. The goals here are to (1) facilitate maintaining & testing them independently of Zeek releases; and (2) come to a more flexible notion of "default scripts" that can incorporate community-maintained packages as well. This is tracked by issue https://github.com/zeek/zeek/issues/414, including a 1st pass over the existing policy scripts to understand what should/can be moved. (Thanks, Vlad!) Before we can begin working on this, we need to figure out how to organize this new world. One particular question is where the moved packages will live. I see the following options so far: 1. Move each into a a separate repository on the zeek/ GitHub account. 2. Similar, but to avoid cluttering zeek/, create a new GitHub organization "zeek-packages". 3. Put them all into a single mono-repository (e.g., zeek/standard-packages), i.e., treat them a one package. 4. Do (1) or (2), and additionally create "zeek-standard-packages" that's full of submodules pointing to them (and also to community packages). 5. Do (1) or (2), and teach zkg to understand "collections" of packages that can be installed/managed as a group, defined through some meta data somewhere. Along with all of this comes a question of how to make it easy for people to install a set of default packages now that these won't come with Zeek itself anymore. Some of the schemes above make that easier than others. Thoughts/opinions/more ideas? Robin -- Robin Sommer * Corelight, Inc. * ro...@corelight.com * www.corelight.com ___ zeek-dev mailing list -- zeek-dev@lists.zeek.org To unsubscribe send an email to zeek-dev-le...@lists.zeek.org
