We could refer to logstash extensible patterns: https://github.com/logstash-plugins/logstash-patterns-core/tree/master/patterns
https://www.elastic.co/guide/en/logstash/current/plugins-filters-grok.html I think logstash is well designed as a general-purpose pipeline for stream processing though single process only now :-) Thanks, Hao On 12/7/15, 2:29 PM, "Zhang, Edward (GDI Hadoop)" <[email protected]> wrote: >I have not figured out what the policy template looks like, but like you >said, that should include variable. and this template should be populated >into UI. > >Eagle-68 was previously proposed by Hemanth by customizing HDFS policy UI >to simplify complex policy onboard, but I think we can do better. > >Edward > >On 12/6/15, 22:15, "Liangfei.Su" <[email protected]> wrote: > >>I would second this template way to keep the user from the error-prone >>command assembling define. >>What kind of json schema as you mentioned in EAGLE-68? Is the simple >>policy >>DSL definition enough here (with template variable)? >> >>Thanks, >>Ralph >> >>On Mon, Dec 7, 2015 at 1:12 PM, Edward Zhang <[email protected]> >>wrote: >> >>> I want to start some discussion on how to support complex policy >>>template >>> gracefully. >>> >>> Today if we want to support a policy like "alert when a user deletes >>>some >>> sensitivity file", then user has to compose very complex policy because >>>in >>> Hdfs file deletion will spawn multiple granular hdfs audit events. It is >>> hard for user to define such a simple policy in a straightforward way. >>> >>> I want to propose to solve the problem with the following approach >>> EAGLE-68 <https://issues.apache.org/jira/browse/EAGLE-68>, EAGLE-14 >>> <https://issues.apache.org/jira/browse/EAGLE-14> >>> >>> First in stream processing phase, Eagle will reassemble user level >>>command >>> from granular audit event which is defined by EAGLE-14 >>> <https://issues.apache.org/jira/browse/EAGLE-14> >>> Second, in UI we provide a general feature for user to import a >>>predefined >>> policy template and those policy templates can be hosted in eagle source >>> code externalPolices for example. this is defined in EAGLE-68 >>> <https://issues.apache.org/jira/browse/EAGLE-68> >>> >>> With this approach, we don't need customize HDFS policy UI and I hope we >>> can always avoid customizing a UI for a specified data source. >>> >>> Please suggest. >>> >>> Thanks >>> Edward Zhang >>> >
