[
https://issues.apache.org/jira/browse/METRON-534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Otto Fowler updated METRON-534:
-------------------------------
Description:
With the INotify library it is possible to receive notifications from HDFS on
file changes.
A service that utilized this and was configured with "rules" to process
notifications on files of interest ( configurations, rules etc ) in the system
would be able to send notifications to Zookeeper
This may allow for things that are better stored in HDFS to remain there, and
not force the system to put things in Zookeeper just for change notifications.
Issues:
* Inotify requires hdfs super user, what are the implications on a secure
cluster etc?
* performance / scalability and load
* can it be deployed with yarn?
was:
With the INotify library it is possible to receive notifications from HDFS on
file changes.
A service that utilized this and was configured to receive notifications on
files of interest ( configurations, rules etc ) in the system would be able to
send notifications to Zookeeper
This may allow for things that are better stored in HDFS to remain there, and
not force the system to put things in Zookeeper just for change notifications.
Issues:
* Inotify requires hdfs super user, what are the implications on a secure
cluster etc?
* performance / scalability and load
* can it be deployed with yarn?
> Metron Should have service to monitor HDFS for changes to files
> ---------------------------------------------------------------
>
> Key: METRON-534
> URL: https://issues.apache.org/jira/browse/METRON-534
> Project: Metron
> Issue Type: New Feature
> Reporter: Otto Fowler
>
> With the INotify library it is possible to receive notifications from HDFS on
> file changes.
> A service that utilized this and was configured with "rules" to process
> notifications on files of interest ( configurations, rules etc ) in the
> system would be able to send notifications to Zookeeper
> This may allow for things that are better stored in HDFS to remain there, and
> not force the system to put things in Zookeeper just for change
> notifications.
> Issues:
> * Inotify requires hdfs super user, what are the implications on a secure
> cluster etc?
> * performance / scalability and load
> * can it be deployed with yarn?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
