It may. We may get to the point where we can handle more complex objects. Until then, I made the approach pluggable and put up a quick JIRA/PR for people to tinker with here <https://github.com/apache/incubator-metron/pull/261>. I had this dude already done in a long languishing branch, so I figure I might as well see if it's useful.
On Mon, Sep 19, 2016 at 10:25 AM, David Lyle <dlyle65...@gmail.com> wrote: > Does Elasticsearch Nested Objects [1] help with that? > > [1] > https://www.elastic.co/guide/en/elasticsearch/guide/ > current/nested-objects.html > > On Mon, Sep 19, 2016 at 9:43 AM, Casey Stella <ceste...@gmail.com> wrote: > > > So, just curious, what kind of behavior would you expect if the JSON had > a > > complex map inside of it (e.g. { "foo" : { "bar" : 1 }, "numeric" : 7 } > )? > > As it is now, our indices in ES do not handle complex structures. Would > > you want those fields dropped, folded in to the larger structure (e.g. { > > "foo.bar" : 1, "numeric" : 7 }) or an error to occur? Or, would you want > > that to be pluggable? > > > > Casey > > > > On Mon, Sep 19, 2016 at 3:56 AM, Egon Kidmose <kidm...@gmail.com> wrote: > > > > > +1 on the pass through parser that just sends JSON onwards > > > > > > > > > > > > Mvh. / BR > > > Egon Kidmose > > > > > > On Thu, Sep 15, 2016 at 6:08 PM, Casey Stella <ceste...@gmail.com> > > wrote: > > > > > > > Just to tack onto the parser thread (love it, btw :). I'd love to > see > > a > > > > couple of general ones: > > > > > > > > - Arbitrary XML with the ability to map xpaths to columns in the > > JSON > > > > - Pass through parser (in the situation where your data is a JSON > > map > > > > already) > > > > > > > > > > > > On Thu, Sep 15, 2016 at 11:36 AM, zeo...@gmail.com <zeo...@gmail.com > > > > > > wrote: > > > > > > > > > I would love to tack onto this thread - we are also working on some > > > > parsers > > > > > for various technologies and plan to contribute them back. If > others > > > are > > > > > not working on it we will do it ourselves, but it would be great to > > > speed > > > > > things up with help from the community. > > > > > > > > > > - Shibboleth v2 (link > > > > > <https://wiki.shibboleth.net/confluence/display/SHIB2/IdPLogging>) > > > > > - 389 Directory Server (link > > > > > <https://wiki.shibboleth.net/confluence/display/SHIB2/IdPLogging>) > > > > > - OpenLDAP (link <http://www.openldap.org/>) > > > > > - Aruba ClearPass > > > > > - sshd > > > > > - FreeRADIUS > > > > > > > > > > Jon > > > > > > > > > > On Thu, Sep 15, 2016 at 9:57 AM Joe Gumke <joegu...@gmail.com> > > wrote: > > > > > > > > > > > Let me know if I can be of any assistance. Ill need documentation > > and > > > > > such > > > > > > to help build the parsers. > > > > > > > > > > > > On Sep 14, 2016 17:58, "Satish Abburi" <satish.abb...@sstech.us> > > > > wrote: > > > > > > > > > > > > > > > > > > > > Thanks, timelines are 2 weeks from now. Thanks. > > > > > > > > > > > > > > From: Poornima Ravindra Mulukutla <gprmuluku...@gmail.com< > > mailto: > > > > > > > gprmuluku...@gmail.com>> > > > > > > > Reply-To: "u...@metron.incubator.apache.org<mailto:user@metron > . > > > > > > > incubator.apache.org>" <u...@metron.incubator.apache.org > <mailto: > > > > > > > u...@metron.incubator.apache.org>> > > > > > > > Date: Wednesday, September 14, 2016 at 3:26 PM > > > > > > > To: "u...@metron.incubator.apache.org<mailto:user@metron. > > > > > > > incubator.apache.org>" <u...@metron.incubator.apache.org > <mailto: > > > > > > > u...@metron.incubator.apache.org>> > > > > > > > Cc: "dev@metron.incubator.apache.org<mailto:dev@metron. > > > > > > > incubator.apache.org>" <dev@metron.incubator.apache.org > <mailto: > > > > > > dev@metron. > > > > > > > incubator.apache.org>> > > > > > > > Subject: Re: log parsers- > > > > > > > > > > > > > > Thank you > > > > > > > > > > > > > > I am happy to take up ASA log file analyser, what is the > timeline > > > you > > > > > are > > > > > > > looking for so that I will plan accordingly? > > > > > > > > > > > > > > In the past I have done BlueCoat log analyser when I was doing > > > > research > > > > > > on > > > > > > > HTTP specification (published a patent has created big change > in > > > HTTP > > > > > > > designs), recently adopted for the Microsoft IE 11 > > > > > > > > > > > > > > On Wed, Sep 14, 2016 at 6:54 PM, Satish Abburi < > > > > > satish.abb...@sstech.us< > > > > > > > mailto:satish.abb...@sstech.us>> wrote: > > > > > > > > > > > > > > Hi, we are trying to build parsers for our Phase1 demo on > Metron > > > > > > platform. > > > > > > > Would like to find, if anyone already has these parsers > > developed. > > > > > > > We already started working on Windows parser, rest planning to > > > start > > > > > > this > > > > > > > week. We can leverage if some thing avaialble or collaborate > > > > > > appropriately. > > > > > > > > > > > > > > > > > > > > > * ASA (Firewall) Metron-363 > > > > > > > * Windows (Desktop) - METRON-165 > > > > > > > * Unix (OS) Metron-175 > > > > > > > * Email > > > > > > > * BlueCoat(Proxy) METRON-162 > > > > > > > > > > > > > > Thanks for your help! > > > > > > > Satish > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > Jon > > > > > > > > > > > > > > >
