Hi Victor, the new message pipeline for Graylog 2.0.0 hasn't been released yet. We plan to include it in the upcoming beta versions of Graylog 2.0.0.
It will be possible to run you messages against regular expressions and Grok patterns, just like with extractors today, while simplifying and unifying a lot of things in Graylog. So yes, you could probably get rid of logstash to split up those messages. […] Graylog 2 2.0 (surely I can't be the only one that finds that naming > confusing...haha) We've renamed Graylog2 to Graylog with the release of Graylog 1.0.0. So yes, I expect a lot of confusion, once Graylog 2.0.0 – *not Graylog2 2.0.0* – will be released. ;-) Cheers, Jochen On Thursday, 25 February 2016 00:34:02 UTC+1, Victor Hooi wrote: > > Hi, > > > I saw on the Graylog2 homepage about a new message pipeline in Graylog 2 > 2.0 (surely I can't be the only one that finds that naming > confusing...haha): > > INGEST AND ENRICH ALL YOUR DATA >> Message processing pipeline >> It's easy to parse and enrich logs from any data source using Graylog's >> flexible processing engine. Reuse code and simplify rule management by >> composing stages in named pipelines. Add your own enrichment and parsing >> functions for additional flexibility. (Available in v2.0) > > > Are there any details, docs etc. on this new pipeline? Super interested to > see what it can do, or how we might use it. > > Does it supplant the need for using Logstash with Graylog2? > > Some of the loglines I'm looking at parsing are like this - just need to > get some structure out of them: > > 362974:2016-01-29 20:23:09,885 +0200 INFO pid=963 4681629696:Worker-1 > snapshot_sqlite.pyo:143 Adding local entry inode=68244955, filename=None > 362977:2016-01-29 20:23:09,886 +0200 INFO pid=963 4681629696:Worker-1 > snapshot_sqlite.pyo:230 Updating local entry inode=68244955, > filename=Something something.pptx, modified=1454042625, > checksum=1dac0196bdeb5aed8b0a7d3fa990d3d4, size=1570159, is_folder=False > 362978:2016-01-29 20:23:09,886 +0200 INFO pid=963 4681629696:Worker-1 > snapshot_sqlite.pyo:542 Adding local relation child_inode=68244955, > parent_inode=68243733 > 362980:2016-01-29 20:23:09,887 +0200 INFO pid=963 4681629696:Worker-1 > snapshot_sqlite.pyo:562 Adding Mapping inode=68244955, > doc_id=0B1k0j0xY2d57SjNSanZaSjkxU0E > > (And yes, the filename in line 2 isn't quoted, which is a bit > annoying...not sure if this will be able to handle that.) > > Cheers, > Victor > -- You received this message because you are subscribed to the Google Groups "Graylog Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/graylog2/83888723-8978-4c26-b77a-023de984d9ae%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
