[
https://issues.apache.org/jira/browse/NIFI-9863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17516424#comment-17516424
]
Otto Fowler commented on NIFI-9863:
-----------------------------------
Sorry [~exceptionfactory]
Even our documentation is confusing on that matter:
Grok Pattern File Path to a file that contains Grok
Patterns to use for parsing logs. If not specified, a built-in default Pattern
file will be used. If specified, all patterns in the given pattern file will
override the default patterns. See the Controller Service's Additional Details
for a list of pre-defined patterns.
This property requires exactly one file to be provided..
->>>> pattern here too
Grok Expression Specifies the format of a log line in Grok
format. This allows the Record Reader to understand how to parse each log line.
If a line in the log file does not match this pattern, the line will be assumed
to belong to the previous log message.If other Grok expressions are referenced
by this expression, they need to be supplied in the Grok Pattern File
So what I had in mind was a controller that supplied patterns, or more
specifically a controller INTERFACE for a controller that supplied patterns,
some default implementation
But, there would be no reason for it to provide expressions as well as patterns.
Maybe some kind of lookup would be more generic actually.
Sorry for the confusion. I usually just use pattern for Grok * from back when
we used it in metron and a contributed to it a small bit. And as you see the
docs slightly confusing ( at least to me ).
The idea in my head, maybe invalid ( or probably ) is that expression like
this, that may be reused between different processors in the flow, should be
shared and updatable from a single source as opposed to have to type them in
properties over and over again as much as possible.
This is probably something that I wouldn't implement until there is an ask
though
> Controller Service for managing custom Grok patterns
> ----------------------------------------------------
>
> Key: NIFI-9863
> URL: https://issues.apache.org/jira/browse/NIFI-9863
> Project: Apache NiFi
> Issue Type: New Feature
> Reporter: Otto Fowler
> Priority: Major
>
> Managing custom Grok expressions in properties for the Grok processors or
> Record readers is cumbersome and not ideal.
> Having a service that managed these expressions in a centralized and reusable
> way would be a benefit to those using Grok patterns.
> This service would allow the configuration of some number custom Grok
> patterns as the service configuration. The MVP would be manual entry, but
> loading patterns from File ( upload to configuration? ) or from some external
> location could be allowed as well down the line.
> In use, it could be argued that the patterns should be loaded from something
> like the schema registry.
> consumers of the service should then be able select the specific service
> instance and then using dependent properties select which patterns provided
> by the service to consume.
> To this end, it may be nice to have the service support pattern 'groups',
> such that you can select all patterns for a group at once. This would be the
> easy button version of the linked multiple expressions to grok reader issue.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
