At BroCon several people pointed out a need to install Bro packages on machines that do not have a direct external connection. One idea would be some kind of proxy scheme where an intermediary git repository mirrors packages locally; bro-pkg would then pull from there. However, I don't think any of us really liked that idea much. After a few rounds of discussions, Seth and I came up with a different idea that seems easier to manage: extending bro-pkg to bundle packages into deployment files that can be easily pushed to Bro systems simply by copying them over.
I’ve tried to flesh this out a bit more, and would be interested to hear what you all think about this approach. And @Jon: Do you think this would be doable that way? Here’s the idea: 1. Generally, one first uses bro-pkg as usual to install packages onto a local Bro system that does have external Internet connectivity (this could be just a dummy Bro installation). One installs new packages there, updates existing ones, etc., until reaching a state that one wants to push out to the actual Bro system. 2. We add a new “bundle” command to bro-pkg that serializes the current state of packages into a single file on disk, a “package bundle”. The bundle contains the complete content of all currently installed packages, using some kind of suitable container format (could be just a ZIP file, or whatever works; the internal representation doesn’t really matter). 3. Users create such a bundle on the local system and then simply copy that bundle file over to all target Bro machines that do not have a external connectivity themselves, using whatever mechanism they have available (e.g., just scp; or maybe through some configuration management system like Ansible etc.). 4. On the target machine, one runs a corresponding “bro-pkg unbundle” command on that bundle file. That command will completely replace the system’s current set of packages with the bundle’s content. As a result, that machine will now have exactly the same packages installed as the original system. This would be the general scheme. A couple of people I talked to at BroCon confirmed that this would offer a viable solution for them, and that they would indeed much prefer copying files around over maintaining local git mirrors. Some additional thoughts on variations/extensions of this basic scheme: - I’m not quite sure if the bundle should contain just the packages themselves or further bro-pkg state as well, such as which packages are currently loaded. Right now I’m learning towards saying “just the packages”; that would basically treat bundles just as a transport mechanism to get packages over to another box. The actual Bro machines would still keep control over which packages to actually load, etc. - As it is described above, Step 1 would require having a local Bro installation into which packages get installed before they can be bundled up. It would be nice to have a mode where bro-pkg can operate without having a Bro around at all, just downloading packages locally somewhere for bundling them up. I could also see offering an even simpler mode where one simply lists packages to bundle on the command-line: “bro-pkg bundle <pkg1> <pkg2> <pkg3>”. That would be particularly useful with configuration management systems I think. - It would be neat if bro-pkg's Python library exposed operations to inspect & retrieve the content of a bundle, such as iterating over the packages inside a bundle and iterating over the files inside a package. That way one could easily build target-side scripts that process and validate bundles before going ahead and installing them (e.g., imposing custom restrictions on what kind of packages one allows to put in place; or ensuring that a bundle always contains a set of packages the site deems mandatory, to avoid configuration mistakes; or even just logging what gets pushed out). What do you guys think about this? Robin -- Robin Sommer * ICSI/LBNL * [email protected] * www.icir.org/robin _______________________________________________ bro-dev mailing list [email protected] http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev
