[
https://issues.apache.org/jira/browse/METRON-534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Otto Fowler updated METRON-534:
-------------------------------
Description:
With the INotify library it is possible to receive notifications from HDFS on
file changes.
A service that utilized this and was configured with "rules" to process
notifications on files of interest ( configurations, rules etc ) in the system
would be able to send notifications to Zookeeper
This may allow for things that are better stored in HDFS to remain there, and
not force the system to put things in Zookeeper just for change notifications.
Issues:
* Inotify requires hdfs super user, what are the implications on a secure
cluster etc?
* performance / scalability and load
* Can it be deployed with yarn?
* Is Zookeeper actually the right place for all the configurations
---------------------------------------------------------------------
!inotify-ascii.png!
was:
With the INotify library it is possible to receive notifications from HDFS on
file changes.
A service that utilized this and was configured with "rules" to process
notifications on files of interest ( configurations, rules etc ) in the system
would be able to send notifications to Zookeeper
This may allow for things that are better stored in HDFS to remain there, and
not force the system to put things in Zookeeper just for change notifications.
Issues:
* Inotify requires hdfs super user, what are the implications on a secure
cluster etc?
* performance / scalability and load
* Can it be deployed with yarn?
* Is Zookeeper actually the right place for all the configurations
> Metron Should have service to monitor HDFS for changes to files
> ---------------------------------------------------------------
>
> Key: METRON-534
> URL: https://issues.apache.org/jira/browse/METRON-534
> Project: Metron
> Issue Type: New Feature
> Reporter: Otto Fowler
> Priority: Minor
> Attachments: inotify-ascii.png
>
>
> With the INotify library it is possible to receive notifications from HDFS on
> file changes.
> A service that utilized this and was configured with "rules" to process
> notifications on files of interest ( configurations, rules etc ) in the
> system would be able to send notifications to Zookeeper
> This may allow for things that are better stored in HDFS to remain there,
> and not force the system to put things in Zookeeper just for change
> notifications.
> Issues:
> * Inotify requires hdfs super user, what are the implications on a secure
> cluster etc?
> * performance / scalability and load
> * Can it be deployed with yarn?
> * Is Zookeeper actually the right place for all the configurations
> ---------------------------------------------------------------------
>
> !inotify-ascii.png!
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
