[
https://issues.apache.org/jira/browse/NIFI-856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15261646#comment-15261646
]
ASF GitHub Bot commented on NIFI-856:
-------------------------------------
Github user trixpan commented on a diff in the pull request:
https://github.com/apache/nifi/pull/290#discussion_r61381697
--- Diff:
nifi-nar-bundles/nifi-lumberjack-bundle/nifi-lumberjack-processors/src/main/java/org/apache/nifi/processors/lumberjack/handler/LumberjackSSLSocketChannelHandler.java
---
@@ -0,0 +1,95 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.lumberjack.handler;
+
+import org.apache.nifi.logging.ProcessorLog;
+import
org.apache.nifi.processor.util.listen.dispatcher.AsyncChannelDispatcher;
+import org.apache.nifi.processor.util.listen.event.Event;
+import org.apache.nifi.processor.util.listen.event.EventFactory;
+import
org.apache.nifi.processor.util.listen.handler.socket.SSLSocketChannelHandler;
+import
org.apache.nifi.processor.util.listen.response.socket.SSLSocketChannelResponder;
+import org.apache.nifi.processors.lumberjack.frame.LumberjackDecoder;
+import org.apache.nifi.processors.lumberjack.frame.LumberjackFrame;
+import
org.apache.nifi.processors.lumberjack.frame.LumberjackFrameException;
+import org.apache.nifi.remote.io.socket.ssl.SSLSocketChannel;
+
+import java.io.IOException;
+import java.net.InetAddress;
+import java.nio.channels.SelectionKey;
+import java.nio.channels.SocketChannel;
+import java.nio.charset.Charset;
+import java.util.List;
+import java.util.concurrent.BlockingQueue;
+
+/**
+ * A Lumberjack implementation of SSLSocketChannelHandler.
+ */
+public class LumberjackSSLSocketChannelHandler<E extends
Event<SocketChannel>> extends SSLSocketChannelHandler<E> {
+
+ private LumberjackDecoder decoder;
+ private LumberjackFrameHandler<E> frameHandler;
+
+ public LumberjackSSLSocketChannelHandler(final SelectionKey key,
+ final AsyncChannelDispatcher
dispatcher,
+ final Charset charset,
+ final EventFactory<E> eventFactory,
+ final BlockingQueue<E> events,
+ final ProcessorLog logger) {
+ super(key, dispatcher, charset, eventFactory, events, logger);
+ this.decoder = new LumberjackDecoder(charset);
+ this.frameHandler = new LumberjackFrameHandler<>(key, charset,
eventFactory, events, dispatcher, logger);
+ }
+
+ @Override
+ protected void processBuffer(final SSLSocketChannel sslSocketChannel,
final SocketChannel socketChannel,
+ final int bytesRead, final byte[] buffer)
throws InterruptedException, IOException {
+
+ final InetAddress sender = socketChannel.socket().getInetAddress();
+ try {
+
+ // go through the buffer parsing the Lumberjack command
+ for (int i = 0; i < bytesRead; i++) {
+ byte currByte = buffer[i];
+
+ // if we found the end of a frame, handle the frame and
mark the buffer
+ if (decoder.process(currByte)) {
+ final List<LumberjackFrame> frames =
decoder.getFrames();
+ // A list of events has been generated
+ for (LumberjackFrame frame : frames) {
+ //TODO: Clean this
--- End diff --
Thank. No. The TODO comment is residual. Oops.
> Add Processor for Lumberjack protocol
> -------------------------------------
>
> Key: NIFI-856
> URL: https://issues.apache.org/jira/browse/NIFI-856
> Project: Apache NiFi
> Issue Type: New Feature
> Reporter: Mike de Rhino
> Labels: features
> Attachments: NIFI-856.patch
>
>
> It would be great if NIFI could support the [lumberjack
> protocol|https://github.com/elastic/logstash-forwarder/blob/master/PROTOCOL.md]
> so to enable the use of logstash forwarder as a source of data.
> A lot of non Java shops tend to avoid installing Java at data producing nodes
> and instead of Flume they end up using things like kafka, heka, fluentd or
> logstash-forwarded as data shipping mechanisms.
> Kafka is great but its architecture seem to be better focused on multi-DC
> environments instead of multi-branch scenarios (imagine having to manager 80
> Zookeeper quorum, one for each country where you operate?)
> [Heka|https://github.com/mozilla-services/heka] is fine, it has decent
> backpressure buffering but no concept of acknowledgement on the receiving
> side of a TCP stream. If the other end of a TCP stream is capable of
> listening but gets stuck with its messages it will keep spitting data through
> the pipe, oblivious to the woes at the other end.
> Logstash forwarder in the other hand, is a quite simple tool, with a
> reasonable implementation of acknowledgments on the receiving side but... it
> depends on Logstash(and logstash has its own issues).
> It would be great if NIFI could serve as a middle man, receiving lumberjack
> messages and offloading some of the hard work Logstash seems to struggle with
> (e.g. using NIFI to save to HDFS while a downstream Logstash writes into ES).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
