[
https://issues.apache.org/jira/browse/NIFI-856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15230087#comment-15230087
]
ASF GitHub Bot commented on NIFI-856:
-------------------------------------
GitHub user trixpan reopened a pull request:
https://github.com/apache/nifi/pull/290
NIFI-856 Implements experimental ListenLumberjack Processor
ListenLumberjack processor should allow NiFi to act as a drop in
replacement on pipelines
using logstash-forwarder and Logstash's Lumberjack output to move data
created by
distributed systems.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/trixpan/nifi NIFI-856v2
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/nifi/pull/290.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #290
----
commit 777852f1e994be737eeb3f8b832ffd3db25e02f7
Author: Andre F de Miranda <[email protected]>
Date: 2016-03-18T14:41:35Z
NIFI-856 Implements experimental ListenLumberjack Processor
commit e9d67d5ed04b63d71acaf7524045922054b8c251
Author: Andre F de Miranda <[email protected]>
Date: 2016-03-19T00:29:01Z
NIFI-856 Add license header to some of the test units
commit d4b5b10d5e14ff52612f8d36dfae206ddb6b6ffe
Author: Andre F de Miranda <[email protected]>
Date: 2016-03-19T01:00:15Z
NIFI-856 Fix checkstyle:check warnings
commit f33307b16b6690ee1b3c7f25c0f2423d65844c6b
Author: Andre F de Miranda <[email protected]>
Date: 2016-03-19T06:18:13Z
NIFI-856 Fix org.apache.rat:apache-rat-plugin:0.11:check validaiton errors
commit 5be4b91e5e3a279f469e11c7519fd6b8d1d93f95
Author: Andre F de Miranda <[email protected]>
Date: 2016-03-19T10:53:28Z
NIFI-856 use JSON to carry Lumberjack fields between Handler and onTrigger
method
commit 83b5781fb6fce22690d0b2360485bc1bfd8a4754
Author: Andre F de Miranda <[email protected]>
Date: 2016-03-19T15:32:06Z
NIFI-856 Remove unused imports (contrib-check now passes)
commit 1d7b75caf8bdeb5affc9ec6c383cd7541284a47a
Author: Andre F de Miranda <[email protected]>
Date: 2016-04-07T09:11:19Z
NIFI-856 - Bump version to trigger travis
commit 9857b5c74490cb57b1fae622a2b639d812335234
Author: Andre F de Miranda <[email protected]>
Date: 2016-04-07T09:20:17Z
Revert "NIFI-856 - Bump version to trigger travis again..."
This reverts commit 1d7b75caf8bdeb5affc9ec6c383cd7541284a47a.
commit bedff1e50cff8d6aa574f8a7158701b0c416388c
Author: Andre F de Miranda <[email protected]>
Date: 2016-04-07T10:53:24Z
Merge remote branch 'upstream/master' into NIFI-856v2
commit ff8d20d323dbaf43b978faef839eb49e12788d69
Author: Andre F de Miranda <[email protected]>
Date: 2016-04-07T10:56:25Z
NIFI-856 - Matches upstream version (0.6.0)
----
> Add Processor for Lumberjack protocol
> -------------------------------------
>
> Key: NIFI-856
> URL: https://issues.apache.org/jira/browse/NIFI-856
> Project: Apache NiFi
> Issue Type: New Feature
> Reporter: Mike de Rhino
> Labels: features
> Attachments: NIFI-856.patch
>
>
> It would be great if NIFI could support the [lumberjack
> protocol|https://github.com/elastic/logstash-forwarder/blob/master/PROTOCOL.md]
> so to enable the use of logstash forwarder as a source of data.
> A lot of non Java shops tend to avoid installing Java at data producing nodes
> and instead of Flume they end up using things like kafka, heka, fluentd or
> logstash-forwarded as data shipping mechanisms.
> Kafka is great but its architecture seem to be better focused on multi-DC
> environments instead of multi-branch scenarios (imagine having to manager 80
> Zookeeper quorum, one for each country where you operate?)
> [Heka|https://github.com/mozilla-services/heka] is fine, it has decent
> backpressure buffering but no concept of acknowledgement on the receiving
> side of a TCP stream. If the other end of a TCP stream is capable of
> listening but gets stuck with its messages it will keep spitting data through
> the pipe, oblivious to the woes at the other end.
> Logstash forwarder in the other hand, is a quite simple tool, with a
> reasonable implementation of acknowledgments on the receiving side but... it
> depends on Logstash(and logstash has its own issues).
> It would be great if NIFI could serve as a middle man, receiving lumberjack
> messages and offloading some of the hard work Logstash seems to struggle with
> (e.g. using NIFI to save to HDFS while a downstream Logstash writes into ES).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
