I am not saying it is faster, just giving some info. Also, that part of the documentation is not referring to regex v. grok, but grok verses a custom non-regex parser, at least as I read it.
If you have the ability to build, deploy, test and maintain a custom parser ( unless you will be submitting it to the project? ), then in most cases where performance is the top issue ( or rather throughput ) you are most likely going to get better results that way. Accepting that you are ok with the tradeoffs. If you have 10M mps parsing might night be your bottleneck. On July 11, 2018 at 11:01:19, Muhammed Irshad (irshadkt....@gmail.com) wrote: Otto Fowler, Thanks for the reply. I saw it uses same Java regex under the hood. I got bit sceptic by seeing this open issue <https://github.com/thekrakken/java-grok/issues/75> in java-grok which says grok is much slower when compared with pure regex. The fix is not available yet in metron as it need few changes in the API and issue to be closed. As data volume is so huge in my requirement I had to double check and confirm before I go with one. Also metron documentation < https://metron.apache.org/current-book/metron-platform/metron-parsers/index.html> itself says the below statement under Parser Adapter section. "Grok parser adapters are designed primarly for someone who is not a Java coder for quickly standing up a parser adapter for lower velocity topologies. Grok relies on Regex for message parsing, which is much slower than purpose-built Java parsers, but is more extensible. Grok parsers are defined via a config file and the topplogy does not need to be recombiled in order to make changes to them." On Wed, Jul 11, 2018 at 8:01 PM, Otto Fowler <ottobackwa...@gmail.com> wrote: > Java-Grok IS java regex. It is just a DSL over Java regex. It takes grok > expressions ( that can reference other expressions and be compound ) and > parses/resolves them and then builds one big regex out of them. > Also, Groks, once parsed / used are re-used, so at that point they are > like compiled regex’s. > > That is not to say that that takes 0 time, but it may help you to > understand. > > https://github.com/thekrakken/java-grok/blob/master/src/ > main/java/io/krakens/grok/api/Grok.java > https://github.com/thekrakken/java-grok/blob/master/src/ > main/java/io/krakens/grok/api/GrokCompiler.java > > On July 11, 2018 at 07:13:38, Muhammed Irshad (irshadkt....@gmail.com) > wrote: > > Thanks a lot Kevin for replying. Which thread are you mentioning ? The > stackoverflow link ? I could not see any such option. > > On Wed, Jul 11, 2018 at 3:04 PM, Kevin Waterson <kevin.water...@gmail.com> > > wrote: > > > Like the thread says, the two regex engines are wildly different, > however.. > > you can increase the threads using -w option in grok to increase the > > threads. > > > > Kevin > > > > On Wed, Jul 11, 2018 at 5:35 PM Muhammed Irshad <irshadkt....@gmail.com> > > > wrote: > > > > > Hi All, > > > > > > I am trying to write Java custom parser for parsing AD logs. I am > > expecting > > > log flow of 10 million AD events per second. Is using Java regex to > parse > > > benefit over using Grok parser in terms of performance ? Is there any > > > performance benchmark or insights regarding the same ? > > > > > > I found this stackoverflow > > > < > > > https://stackoverflow.com/questions/43222863/logstash- > > grok-filter-is-slower-than-java-regex-pattern-matching > > > > > > > question which inspired me for this post. > > > > > > -- > > > Muhammed Irshad K T > > > Senior Software Engineer > > > +919447946359 > > > irshadkt....@gmail.com > > > Skype : muhammed.irshad.k.t > > > > > > > > > -- > Muhammed Irshad K T > Senior Software Engineer > +919447946359 > irshadkt....@gmail.com > Skype : muhammed.irshad.k.t > > -- Muhammed Irshad K T Senior Software Engineer +919447946359 irshadkt....@gmail.com Skype : muhammed.irshad.k.t
