[
https://issues.apache.org/jira/browse/NIFI-1037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15315561#comment-15315561
]
ASF GitHub Bot commented on NIFI-1037:
--------------------------------------
GitHub user jjmeyer0 opened a pull request:
https://github.com/apache/nifi/pull/493
NIFI-1037 Created processor that handles HDFS' inotify event stream.
NIFI-1037 provides a processor that allows a user to consume HDFS'
notification event stream. The processor allows for a specific directory to be
watched for specific events. Each event will be converted to a flow file where
the content is the event information in JSON format.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/jjmeyer0/nifi NIFI-1037
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/nifi/pull/493.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #493
----
commit 38352af4f08bbb923b5c1c659cc0d8c7ccf98a24
Author: JJ <[email protected]>
Date: 2016-05-23T15:10:55Z
NIFI-1037 Created a processor for HDFS' inotify events. This processor gets
notifications for a select path.
commit 0401beabfbec3312f8371e1e41be45a5ea373fc2
Author: JJ <[email protected]>
Date: 2016-05-24T13:48:35Z
Renamed things to have them make more sense.
commit 21a84dbce10588af4ba8b2dd7bd348a302602f76
Author: JJ <[email protected]>
Date: 2016-05-24T13:51:57Z
refactoring.
commit e0bf6e915cc1e17e6a0666a8fd03ec045fc1ad16
Author: JJ <[email protected]>
Date: 2016-06-04T13:56:40Z
NIFI-1037 updated nofication processor to write event information to flow
file's content instead of attributes.
commit 5664abf7f8136f5022d398d20c3ffd30dbfabf75
Author: JJ <[email protected]>
Date: 2016-06-04T14:00:17Z
Refactoring to conform to most prevalent naming standards.
commit 61fdf22cea2de5a3bfe78cf4feed329623e7975e
Author: JJ <[email protected]>
Date: 2016-06-04T14:04:27Z
Made all properties and relationships package scope.
commit 251d8996e3d7f0d42179c3969fd6aa59b5227225
Author: JJ Meyer <[email protected]>
Date: 2016-06-04T14:21:49Z
Rename GetHdfsEvents.java to GetHDFSEvents.java
commit 03f44eb91a11f24f7697ec76255a37e74768adce
Author: JJ Meyer <[email protected]>
Date: 2016-06-04T14:22:48Z
Rename TestGetHdfsEvents.java to TestGetHDFSEvents.java
----
> Hdfs Inotify processor
> ----------------------
>
> Key: NIFI-1037
> URL: https://issues.apache.org/jira/browse/NIFI-1037
> Project: Apache NiFi
> Issue Type: New Feature
> Reporter: nicolas maillard
> Assignee: Josh Meyer
> Priority: Minor
>
> HDFS has an Inotify interface that enables to access the HDFS edit stream.
> https://issues.apache.org/jira/browse/HDFS-6634
> Creating a processor to listen in and get notifications either for select
> directories or select actions would have many applications.
> - Stream to a search engine the activity on HDFS
> - Wait for specific actions or files to trigger workflows, like duplication
> to other clusters
> - Validate ingestion processes
> etc..
> probably more I don't think of.
> I have a first working beta version that needs to evolve
> it reuses the Hadoop-nar-bundle
> Needs a HDFS 2.7 dependency currently done through editing the Hadop-lib
> bundle
> let me know if this idea makes sense and would be of interest to the community
> would love to contribute the idea
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
