So, The distinction between ‘sensor’ and ‘parser’ is not clear to me either, if it is defined somewhere and I have missed it, please point me in the right direction. While I don’t want to fork the discussion, the question is where you find it so to speak, so about bundles and configurations.
With the extension system you have two options for ‘config’ only parser/sensor installs. 1. The previous mechanism of creating the configurations and manually pushing them zookeeper and possibly patterns still works, although they will not be managed as extensions. This I believe is a feature gap that already exists and I did not address it as part of this effort. 2. Create a new parser from the archetype, but remove all the src/main code and make it configuration only. This I think should be the recommended way to create configuration only parsers, since such parsers will still have the unit and integration tests. Flows where you start with the archetype and refine through tests or manually deploy, refine and then build from the archetype can be imagined. The main question for this thread however, is should metron parsers and 3rd party parsers be treated the same -> they are all extensions, manageable through the extensions ui. I can demo for you whenever you are free if you don’t want to wait for the community demo. On September 20, 2017 at 09:52:56, Simon Elliston Ball ( si...@simonellistonball.com) wrote: Otto, Can you just clarify what you mean by parsers in this instance. To my mind parsers in metron are be classes, and do not have any configuration settings. Instances of parsers are referred to in the ui as sensors, and are essentially concrete instances of parsers and as such do have config. The parser vs sensor distinction feels like a valuable one to make here. I'd really like a clearer understanding of the role of bundles in config, and how we maintain the ability to control config without the need for files in the bundle. Maybe some samples / a demo would really clear this up, but to be honest right now I'm a little confused about how this would work for an average SOC team who do not have maven available. Thanks, Simon Sent from my iPhone > On 20 Sep 2017, at 23:43, Otto Fowler <ottobackwa...@gmail.com> wrote: > > Note: Grok, CSV and JSONMap would be ‘always there’, as they are still > part of the system and not installed individually. > > > > On September 20, 2017 at 09:39:38, Otto Fowler (ottobackwa...@gmail.com) > wrote: > > The question has come up about the metron parsers installation vs. parser > extension installation differences, and I’d like to get some comments. > > Right now ( let’s pretend the UI PR get’s merged to the feature branch for > a minute ) in the original take on this the metron parsers ( bro, yaf, > snort etc ) get installed > into the system basically as before with regards to zookeeper ( although > they ALL get installed since they all have demo compatible default configs > now). The parser extensions, installed after the fact through ui however > have a new parser extension configuration > that is registered into a new zookeeper area, and are listed and managed in > the new UI. They can be installed or uninstalled basically. > > The question is if the parsers installed by metron should *also* be > registered in the same way, such that the parser extension ui lists all of > the parsers. > > This would allow removal and installation of metron parsers installed by > the system. Also, following on we can customize the install to not install > everything. It may also be more simple concept wise. > > That is the basic thing. So the question is if we want to go in this > direction. > > The not so basic thing is to still deploy the extensions into the system as > packages, but not install them, such that you can add an extension from a > file *and also* from a ‘repository’ of > extensions. Down the line we can support local and remote repositories etc. > > So the options are: > > 1. As it is now on the feature branch ( still pretending ;)), the system > installed parsers are not the same as the extension installed parsers. > While the project can uninstall and replace these parsers, the user/ui > cannot. > 2. All parsers are installed as extensions and can be removed, but are all > initially installed > 3. All parsers are extensions, but not installed during the original > deployment, rather they are put into a repository that the ui lets you > browse to select, in addition to allowing > install from file ( like intellij and other plugin systems do ). **This > would have implications for the demo system, since we would want to still > install the bro, snort and maybe yaf parsers. > > > In considering these questions, we need to keep in mind where we want to > go, and how much of this is required for first release of the extension > system. There is a lot of ‘while we are already doing xxx, we might as > well do yyyy since it will be harder later’ in this. > > Thanks for your time and your timely responses. > > ottO
