Otto, Can you just clarify what you mean by parsers in this instance. To my mind parsers in metron are be classes, and do not have any configuration settings. Instances of parsers are referred to in the ui as sensors, and are essentially concrete instances of parsers and as such do have config.
The parser vs sensor distinction feels like a valuable one to make here. I'd really like a clearer understanding of the role of bundles in config, and how we maintain the ability to control config without the need for files in the bundle. Maybe some samples / a demo would really clear this up, but to be honest right now I'm a little confused about how this would work for an average SOC team who do not have maven available. Thanks, Simon Sent from my iPhone > On 20 Sep 2017, at 23:43, Otto Fowler <[email protected]> wrote: > > Note: Grok, CSV and JSONMap would be ‘always there’, as they are still > part of the system and not installed individually. > > > > On September 20, 2017 at 09:39:38, Otto Fowler ([email protected]) > wrote: > > The question has come up about the metron parsers installation vs. parser > extension installation differences, and I’d like to get some comments. > > Right now ( let’s pretend the UI PR get’s merged to the feature branch for > a minute ) in the original take on this the metron parsers ( bro, yaf, > snort etc ) get installed > into the system basically as before with regards to zookeeper ( although > they ALL get installed since they all have demo compatible default configs > now). The parser extensions, installed after the fact through ui however > have a new parser extension configuration > that is registered into a new zookeeper area, and are listed and managed in > the new UI. They can be installed or uninstalled basically. > > The question is if the parsers installed by metron should *also* be > registered in the same way, such that the parser extension ui lists all of > the parsers. > > This would allow removal and installation of metron parsers installed by > the system. Also, following on we can customize the install to not install > everything. It may also be more simple concept wise. > > That is the basic thing. So the question is if we want to go in this > direction. > > The not so basic thing is to still deploy the extensions into the system as > packages, but not install them, such that you can add an extension from a > file *and also* from a ‘repository’ of > extensions. Down the line we can support local and remote repositories etc. > > So the options are: > > 1. As it is now on the feature branch ( still pretending ;)), the system > installed parsers are not the same as the extension installed parsers. > While the project can uninstall and replace these parsers, the user/ui > cannot. > 2. All parsers are installed as extensions and can be removed, but are all > initially installed > 3. All parsers are extensions, but not installed during the original > deployment, rather they are put into a repository that the ui lets you > browse to select, in addition to allowing > install from file ( like intellij and other plugin systems do ). **This > would have implications for the demo system, since we would want to still > install the bro, snort and maybe yaf parsers. > > > In considering these questions, we need to keep in mind where we want to > go, and how much of this is required for first release of the extension > system. There is a lot of ‘while we are already doing xxx, we might as > well do yyyy since it will be harder later’ in this. > > Thanks for your time and your timely responses. > > ottO
