[jira] [Commented] (NIFI-3238) ListenLumberjack should support the *beat protocol

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15889381#comment-15889381
 ] 

ASF GitHub Bot commented on NIFI-3238:
--

Github user trixpan commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1418#discussion_r103599675
  
--- Diff: 
nifi-nar-bundles/nifi-beats-bundle/nifi-beats-processors/src/main/java/org/apache/nifi/processors/beats/ListenBeats.java
 ---
@@ -0,0 +1,216 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.beats;
+
+import java.io.IOException;
+import java.nio.ByteBuffer;
+import java.nio.charset.Charset;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.BlockingQueue;
+
+import javax.net.ssl.SSLContext;
+
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.WritesAttribute;
+import org.apache.nifi.annotation.behavior.WritesAttributes;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.SeeAlso;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.flowfile.attributes.CoreAttributes;
+import org.apache.nifi.flowfile.attributes.FlowFileAttributeKey;
+import org.apache.nifi.processor.DataUnit;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import 
org.apache.nifi.processor.util.listen.AbstractListenEventBatchingProcessor;
+import 
org.apache.nifi.processor.util.listen.dispatcher.AsyncChannelDispatcher;
+import org.apache.nifi.processor.util.listen.dispatcher.ChannelDispatcher;
+import 
org.apache.nifi.processor.util.listen.dispatcher.SocketChannelDispatcher;
+import org.apache.nifi.processor.util.listen.event.EventFactory;
+import org.apache.nifi.processor.util.listen.handler.ChannelHandlerFactory;
+import org.apache.nifi.processor.util.listen.response.ChannelResponder;
+import org.apache.nifi.processor.util.listen.response.ChannelResponse;
+import org.apache.nifi.processors.beats.event.BeatsEvent;
+import org.apache.nifi.processors.beats.event.BeatsEventFactory;
+import org.apache.nifi.processors.beats.frame.BeatsEncoder;
+import 
org.apache.nifi.processors.beats.handler.BeatsSocketChannelHandlerFactory;
+import org.apache.nifi.processors.beats.response.BeatsChannelResponse;
+import org.apache.nifi.processors.beats.response.BeatsResponse;
+import org.apache.nifi.ssl.SSLContextService;
+
+@InputRequirement(InputRequirement.Requirement.INPUT_FORBIDDEN)
+@Tags({"listen", "beats", "tcp", "logs"})
+@CapabilityDescription("Listens for messages sent by libbeat compatible 
clients (e.g. filebeats, metricbeats, etc) being sent to a given port over TCP, 
writing its contents in" +
+"JSON format to the content of the message to a FlowFile." +
+"This processor replaces the now deprecated ListenLumberjack")
+@WritesAttributes({
+@WritesAttribute(attribute = "beats.sender", description = "The 
sending host of the messages."),
+@WritesAttribute(attribute = "beats.port", description = "The sending 
port the messages were received over."),
+@WritesAttribute(attribute = "beats.sequencenumber", description = 
"The sequence number of the message. Only included if  is 1."),
+@WritesAttribute(attribute = "mime.type", description = "The mime.type 
of the content which is application/json")
+})
+@SeeAlso(classNames = {"org.apache.nifi.processors.standard.ParseSyslog"})
+public class ListenBeats 

[GitHub] nifi pull request #1418: NIFI-3238 - Introduce ListenBeats processor and dep...

[trixpan]

Github user trixpan commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1418#discussion_r103599675
  
--- Diff: 
nifi-nar-bundles/nifi-beats-bundle/nifi-beats-processors/src/main/java/org/apache/nifi/processors/beats/ListenBeats.java
 ---
@@ -0,0 +1,216 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.beats;
+
+import java.io.IOException;
+import java.nio.ByteBuffer;
+import java.nio.charset.Charset;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.BlockingQueue;
+
+import javax.net.ssl.SSLContext;
+
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.WritesAttribute;
+import org.apache.nifi.annotation.behavior.WritesAttributes;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.SeeAlso;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.flowfile.attributes.CoreAttributes;
+import org.apache.nifi.flowfile.attributes.FlowFileAttributeKey;
+import org.apache.nifi.processor.DataUnit;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import 
org.apache.nifi.processor.util.listen.AbstractListenEventBatchingProcessor;
+import 
org.apache.nifi.processor.util.listen.dispatcher.AsyncChannelDispatcher;
+import org.apache.nifi.processor.util.listen.dispatcher.ChannelDispatcher;
+import 
org.apache.nifi.processor.util.listen.dispatcher.SocketChannelDispatcher;
+import org.apache.nifi.processor.util.listen.event.EventFactory;
+import org.apache.nifi.processor.util.listen.handler.ChannelHandlerFactory;
+import org.apache.nifi.processor.util.listen.response.ChannelResponder;
+import org.apache.nifi.processor.util.listen.response.ChannelResponse;
+import org.apache.nifi.processors.beats.event.BeatsEvent;
+import org.apache.nifi.processors.beats.event.BeatsEventFactory;
+import org.apache.nifi.processors.beats.frame.BeatsEncoder;
+import 
org.apache.nifi.processors.beats.handler.BeatsSocketChannelHandlerFactory;
+import org.apache.nifi.processors.beats.response.BeatsChannelResponse;
+import org.apache.nifi.processors.beats.response.BeatsResponse;
+import org.apache.nifi.ssl.SSLContextService;
+
+@InputRequirement(InputRequirement.Requirement.INPUT_FORBIDDEN)
+@Tags({"listen", "beats", "tcp", "logs"})
+@CapabilityDescription("Listens for messages sent by libbeat compatible 
clients (e.g. filebeats, metricbeats, etc) being sent to a given port over TCP, 
writing its contents in" +
+"JSON format to the content of the message to a FlowFile." +
+"This processor replaces the now deprecated ListenLumberjack")
+@WritesAttributes({
+@WritesAttribute(attribute = "beats.sender", description = "The 
sending host of the messages."),
+@WritesAttribute(attribute = "beats.port", description = "The sending 
port the messages were received over."),
+@WritesAttribute(attribute = "beats.sequencenumber", description = 
"The sequence number of the message. Only included if  is 1."),
+@WritesAttribute(attribute = "mime.type", description = "The mime.type 
of the content which is application/json")
+})
+@SeeAlso(classNames = {"org.apache.nifi.processors.standard.ParseSyslog"})
+public class ListenBeats extends 
AbstractListenEventBatchingProcessor {
+
+public static final PropertyDescriptor SSL_CONTEXT_SERVICE = new 
PropertyDescriptor.Builder()
+.name("SSL Context Service")
+.description("The Controller 

[jira] [Commented] (NIFI-3238) ListenLumberjack should support the *beat protocol

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15889380#comment-15889380
 ] 

ASF GitHub Bot commented on NIFI-3238:
--

Github user trixpan commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1418#discussion_r103599499
  
--- Diff: 
nifi-nar-bundles/nifi-beats-bundle/nifi-beats-nar/src/main/resources/META-INF/LICENSE
 ---
@@ -0,0 +1,233 @@
+
+ Apache License
+   Version 2.0, January 2004
+http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   1. Definitions.
+
+  "License" shall mean the terms and conditions for use, reproduction,
+  and distribution as defined by Sections 1 through 9 of this document.
+
+  "Licensor" shall mean the copyright owner or entity authorized by
+  the copyright owner that is granting the License.
+
+  "Legal Entity" shall mean the union of the acting entity and all
+  other entities that control, are controlled by, or are under common
+  control with that entity. For the purposes of this definition,
+  "control" means (i) the power, direct or indirect, to cause the
+  direction or management of such entity, whether by contract or
+  otherwise, or (ii) ownership of fifty percent (50%) or more of the
+  outstanding shares, or (iii) beneficial ownership of such entity.
+
+  "You" (or "Your") shall mean an individual or Legal Entity
+  exercising permissions granted by this License.
+
+  "Source" form shall mean the preferred form for making modifications,
+  including but not limited to software source code, documentation
+  source, and configuration files.
+
+  "Object" form shall mean any form resulting from mechanical
+  transformation or translation of a Source form, including but
+  not limited to compiled object code, generated documentation,
+  and conversions to other media types.
+
+  "Work" shall mean the work of authorship, whether in Source or
+  Object form, made available under the License, as indicated by a
+  copyright notice that is included in or attached to the work
+  (an example is provided in the Appendix below).
+
+  "Derivative Works" shall mean any work, whether in Source or Object
+  form, that is based on (or derived from) the Work and for which the
+  editorial revisions, annotations, elaborations, or other 
modifications
+  represent, as a whole, an original work of authorship. For the 
purposes
+  of this License, Derivative Works shall not include works that remain
+  separable from, or merely link (or bind by name) to the interfaces 
of,
+  the Work and Derivative Works thereof.
+
+  "Contribution" shall mean any work of authorship, including
+  the original version of the Work and any modifications or additions
+  to that Work or Derivative Works thereof, that is intentionally
+  submitted to Licensor for inclusion in the Work by the copyright 
owner
+  or by an individual or Legal Entity authorized to submit on behalf of
+  the copyright owner. For the purposes of this definition, "submitted"
+  means any form of electronic, verbal, or written communication sent
+  to the Licensor or its representatives, including but not limited to
+  communication on electronic mailing lists, source code control 
systems,
+  and issue tracking systems that are managed by, or on behalf of, the
+  Licensor for the purpose of discussing and improving the Work, but
+  excluding communication that is conspicuously marked or otherwise
+  designated in writing by the copyright owner as "Not a Contribution."
+
+  "Contributor" shall mean Licensor and any individual or Legal Entity
+  on behalf of whom a Contribution has been received by Licensor and
+  subsequently incorporated within the Work.
+
+   2. Grant of Copyright License. Subject to the terms and conditions of
+  this License, each Contributor hereby grants to You a perpetual,
+  worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+  copyright license to reproduce, prepare Derivative Works of,
+  publicly display, publicly perform, sublicense, and distribute the
+  Work and such Derivative Works in Source or Object form.
+
+   3. Grant of Patent License. Subject to the terms and conditions of
+  this License, each Contributor hereby grants to You a perpetual,
+  worldwide, non-exclusive, no-charge, royalty-free, 

[GitHub] nifi pull request #1418: NIFI-3238 - Introduce ListenBeats processor and dep...

[trixpan]

Github user trixpan commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1418#discussion_r103599499
  
--- Diff: 
nifi-nar-bundles/nifi-beats-bundle/nifi-beats-nar/src/main/resources/META-INF/LICENSE
 ---
@@ -0,0 +1,233 @@
+
+ Apache License
+   Version 2.0, January 2004
+http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   1. Definitions.
+
+  "License" shall mean the terms and conditions for use, reproduction,
+  and distribution as defined by Sections 1 through 9 of this document.
+
+  "Licensor" shall mean the copyright owner or entity authorized by
+  the copyright owner that is granting the License.
+
+  "Legal Entity" shall mean the union of the acting entity and all
+  other entities that control, are controlled by, or are under common
+  control with that entity. For the purposes of this definition,
+  "control" means (i) the power, direct or indirect, to cause the
+  direction or management of such entity, whether by contract or
+  otherwise, or (ii) ownership of fifty percent (50%) or more of the
+  outstanding shares, or (iii) beneficial ownership of such entity.
+
+  "You" (or "Your") shall mean an individual or Legal Entity
+  exercising permissions granted by this License.
+
+  "Source" form shall mean the preferred form for making modifications,
+  including but not limited to software source code, documentation
+  source, and configuration files.
+
+  "Object" form shall mean any form resulting from mechanical
+  transformation or translation of a Source form, including but
+  not limited to compiled object code, generated documentation,
+  and conversions to other media types.
+
+  "Work" shall mean the work of authorship, whether in Source or
+  Object form, made available under the License, as indicated by a
+  copyright notice that is included in or attached to the work
+  (an example is provided in the Appendix below).
+
+  "Derivative Works" shall mean any work, whether in Source or Object
+  form, that is based on (or derived from) the Work and for which the
+  editorial revisions, annotations, elaborations, or other 
modifications
+  represent, as a whole, an original work of authorship. For the 
purposes
+  of this License, Derivative Works shall not include works that remain
+  separable from, or merely link (or bind by name) to the interfaces 
of,
+  the Work and Derivative Works thereof.
+
+  "Contribution" shall mean any work of authorship, including
+  the original version of the Work and any modifications or additions
+  to that Work or Derivative Works thereof, that is intentionally
+  submitted to Licensor for inclusion in the Work by the copyright 
owner
+  or by an individual or Legal Entity authorized to submit on behalf of
+  the copyright owner. For the purposes of this definition, "submitted"
+  means any form of electronic, verbal, or written communication sent
+  to the Licensor or its representatives, including but not limited to
+  communication on electronic mailing lists, source code control 
systems,
+  and issue tracking systems that are managed by, or on behalf of, the
+  Licensor for the purpose of discussing and improving the Work, but
+  excluding communication that is conspicuously marked or otherwise
+  designated in writing by the copyright owner as "Not a Contribution."
+
+  "Contributor" shall mean Licensor and any individual or Legal Entity
+  on behalf of whom a Contribution has been received by Licensor and
+  subsequently incorporated within the Work.
+
+   2. Grant of Copyright License. Subject to the terms and conditions of
+  this License, each Contributor hereby grants to You a perpetual,
+  worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+  copyright license to reproduce, prepare Derivative Works of,
+  publicly display, publicly perform, sublicense, and distribute the
+  Work and such Derivative Works in Source or Object form.
+
+   3. Grant of Patent License. Subject to the terms and conditions of
+  this License, each Contributor hereby grants to You a perpetual,
+  worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+  (except as stated in this section) patent license to make, have made,
+  use, offer to sell, sell, import, and otherwise transfer the Work,
+  where such license applies only to those patent 

[GitHub] nifi pull request #1418: NIFI-3238 - Introduce ListenBeats processor and dep...

[trixpan]

Github user trixpan commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1418#discussion_r103599319
  
--- Diff: 
nifi-nar-bundles/nifi-lumberjack-bundle/nifi-lumberjack-processors/src/main/java/org/apache/nifi/processors/lumberjack/ListenLumberjack.java
 ---
@@ -62,12 +62,13 @@
 
 import com.google.gson.Gson;
 
+@Deprecated
 @InputRequirement(InputRequirement.Requirement.INPUT_FORBIDDEN)
 @Tags({"listen", "lumberjack", "tcp", "logs"})
-@CapabilityDescription("Listens for Lumberjack messages being sent to a 
given port over TCP. Each message will be " +
+@CapabilityDescription("This processor is deprecated and will be removed 
in the near future. Listens for Lumberjack messages being sent to a given port 
over TCP. Each message will be " +
--- End diff --

will rephrase 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-3238) ListenLumberjack should support the *beat protocol

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15889374#comment-15889374
 ] 

ASF GitHub Bot commented on NIFI-3238:
--

Github user trixpan commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1418#discussion_r103599241
  
--- Diff: 
nifi-nar-bundles/nifi-beats-bundle/nifi-beats-processors/src/main/java/org/apache/nifi/processors/beats/frame/BeatsDecoder.java
 ---
@@ -0,0 +1,330 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.beats.frame;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.nio.ByteBuffer;
+import java.nio.charset.Charset;
+import java.util.LinkedList;
+import java.util.List;
+import java.util.zip.InflaterInputStream;
+
+import org.apache.nifi.stream.io.ByteArrayInputStream;
+import org.apache.nifi.stream.io.ByteArrayOutputStream;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * Decodes a Beats frame by maintaining a state based on each byte that 
has been processed. This class
+ * should not be shared by multiple threads.
+ */
+public class BeatsDecoder {
+
+
+static final Logger logger = 
LoggerFactory.getLogger(BeatsDecoder.class);
--- End diff --

sure. Will adjust.


> ListenLumberjack should support the *beat protocol
> --
>
> Key: NIFI-3238
> URL: https://issues.apache.org/jira/browse/NIFI-3238
> Project: Apache NiFi
>  Issue Type: Improvement
>Reporter: Andre F de Miranda
>Assignee: Andre F de Miranda
>
> ListenLumberjack currently only supports v1 of the Lumberjack protocol. This 
> version has been deprecated in favor of v2, which is used on *beat (e.g. 
> filebeat, packetbeat, etc) edge components of the ELK  stack.
> We should consider deprecating ListenLumberjack or to extend it to handle 
> both v1 and v2 of the protocol.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[GitHub] nifi pull request #1418: NIFI-3238 - Introduce ListenBeats processor and dep...

[trixpan]

Github user trixpan commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1418#discussion_r103599241
  
--- Diff: 
nifi-nar-bundles/nifi-beats-bundle/nifi-beats-processors/src/main/java/org/apache/nifi/processors/beats/frame/BeatsDecoder.java
 ---
@@ -0,0 +1,330 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.beats.frame;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.nio.ByteBuffer;
+import java.nio.charset.Charset;
+import java.util.LinkedList;
+import java.util.List;
+import java.util.zip.InflaterInputStream;
+
+import org.apache.nifi.stream.io.ByteArrayInputStream;
+import org.apache.nifi.stream.io.ByteArrayOutputStream;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * Decodes a Beats frame by maintaining a state based on each byte that 
has been processed. This class
+ * should not be shared by multiple threads.
+ */
+public class BeatsDecoder {
+
+
+static final Logger logger = 
LoggerFactory.getLogger(BeatsDecoder.class);
--- End diff --

sure. Will adjust.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-3238) ListenLumberjack should support the *beat protocol

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15889356#comment-15889356
 ] 

ASF GitHub Bot commented on NIFI-3238:
--

Github user mattyb149 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1418#discussion_r103597989
  
--- Diff: 
nifi-nar-bundles/nifi-beats-bundle/nifi-beats-nar/src/main/resources/META-INF/LICENSE
 ---
@@ -0,0 +1,233 @@
+
+ Apache License
+   Version 2.0, January 2004
+http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   1. Definitions.
+
+  "License" shall mean the terms and conditions for use, reproduction,
+  and distribution as defined by Sections 1 through 9 of this document.
+
+  "Licensor" shall mean the copyright owner or entity authorized by
+  the copyright owner that is granting the License.
+
+  "Legal Entity" shall mean the union of the acting entity and all
+  other entities that control, are controlled by, or are under common
+  control with that entity. For the purposes of this definition,
+  "control" means (i) the power, direct or indirect, to cause the
+  direction or management of such entity, whether by contract or
+  otherwise, or (ii) ownership of fifty percent (50%) or more of the
+  outstanding shares, or (iii) beneficial ownership of such entity.
+
+  "You" (or "Your") shall mean an individual or Legal Entity
+  exercising permissions granted by this License.
+
+  "Source" form shall mean the preferred form for making modifications,
+  including but not limited to software source code, documentation
+  source, and configuration files.
+
+  "Object" form shall mean any form resulting from mechanical
+  transformation or translation of a Source form, including but
+  not limited to compiled object code, generated documentation,
+  and conversions to other media types.
+
+  "Work" shall mean the work of authorship, whether in Source or
+  Object form, made available under the License, as indicated by a
+  copyright notice that is included in or attached to the work
+  (an example is provided in the Appendix below).
+
+  "Derivative Works" shall mean any work, whether in Source or Object
+  form, that is based on (or derived from) the Work and for which the
+  editorial revisions, annotations, elaborations, or other 
modifications
+  represent, as a whole, an original work of authorship. For the 
purposes
+  of this License, Derivative Works shall not include works that remain
+  separable from, or merely link (or bind by name) to the interfaces 
of,
+  the Work and Derivative Works thereof.
+
+  "Contribution" shall mean any work of authorship, including
+  the original version of the Work and any modifications or additions
+  to that Work or Derivative Works thereof, that is intentionally
+  submitted to Licensor for inclusion in the Work by the copyright 
owner
+  or by an individual or Legal Entity authorized to submit on behalf of
+  the copyright owner. For the purposes of this definition, "submitted"
+  means any form of electronic, verbal, or written communication sent
+  to the Licensor or its representatives, including but not limited to
+  communication on electronic mailing lists, source code control 
systems,
+  and issue tracking systems that are managed by, or on behalf of, the
+  Licensor for the purpose of discussing and improving the Work, but
+  excluding communication that is conspicuously marked or otherwise
+  designated in writing by the copyright owner as "Not a Contribution."
+
+  "Contributor" shall mean Licensor and any individual or Legal Entity
+  on behalf of whom a Contribution has been received by Licensor and
+  subsequently incorporated within the Work.
+
+   2. Grant of Copyright License. Subject to the terms and conditions of
+  this License, each Contributor hereby grants to You a perpetual,
+  worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+  copyright license to reproduce, prepare Derivative Works of,
+  publicly display, publicly perform, sublicense, and distribute the
+  Work and such Derivative Works in Source or Object form.
+
+   3. Grant of Patent License. Subject to the terms and conditions of
+  this License, each Contributor hereby grants to You a perpetual,
+  worldwide, non-exclusive, no-charge, 

[jira] [Commented] (NIFI-3238) ListenLumberjack should support the *beat protocol

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15889352#comment-15889352
 ] 

ASF GitHub Bot commented on NIFI-3238:
--

Github user mattyb149 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1418#discussion_r103595838
  
--- Diff: 
nifi-nar-bundles/nifi-beats-bundle/nifi-beats-processors/src/main/java/org/apache/nifi/processors/beats/frame/BeatsDecoder.java
 ---
@@ -0,0 +1,330 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.beats.frame;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.nio.ByteBuffer;
+import java.nio.charset.Charset;
+import java.util.LinkedList;
+import java.util.List;
+import java.util.zip.InflaterInputStream;
+
+import org.apache.nifi.stream.io.ByteArrayInputStream;
+import org.apache.nifi.stream.io.ByteArrayOutputStream;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * Decodes a Beats frame by maintaining a state based on each byte that 
has been processed. This class
+ * should not be shared by multiple threads.
+ */
+public class BeatsDecoder {
+
+
+static final Logger logger = 
LoggerFactory.getLogger(BeatsDecoder.class);
+
+private BeatsFrame.Builder frameBuilder;
+private BeatsState currState = BeatsState.VERSION;
+private byte decodedFrameType;
+
+private byte[] unprocessedData;
+
+private final Charset charset;
+private final ByteArrayOutputStream currBytes;
+
+private long windowSize;
+
+static final int MIN_FRAME_HEADER_LENGTH = 2; // Version + Type
+static final int WINDOWSIZE_LENGTH = MIN_FRAME_HEADER_LENGTH + 4; // 
32bit unsigned window size
+static final int COMPRESSED_MIN_LENGTH = MIN_FRAME_HEADER_LENGTH + 4; 
// 32 bit unsigned + payload
+static final int JSON_MIN_LENGTH = MIN_FRAME_HEADER_LENGTH + 8; // 32 
bit unsigned sequence number + 32 bit unsigned payload length
+
+public static final byte FRAME_WINDOWSIZE = 0x57, FRAME_DATA = 0x44, 
FRAME_COMPRESSED = 0x43, FRAME_ACK = 0x41, FRAME_JSON = 0x4a;
+
+/**
+ * @param charset the charset to decode bytes from the frame
+ */
+public BeatsDecoder(final Charset charset) {
+this(charset, new ByteArrayOutputStream(4096));
--- End diff --

Is 4K ok as a default size? Should it be passed in here and perhaps default 
somewhere else in case it needs to be configurable?


> ListenLumberjack should support the *beat protocol
> --
>
> Key: NIFI-3238
> URL: https://issues.apache.org/jira/browse/NIFI-3238
> Project: Apache NiFi
>  Issue Type: Improvement
>Reporter: Andre F de Miranda
>Assignee: Andre F de Miranda
>
> ListenLumberjack currently only supports v1 of the Lumberjack protocol. This 
> version has been deprecated in favor of v2, which is used on *beat (e.g. 
> filebeat, packetbeat, etc) edge components of the ELK  stack.
> We should consider deprecating ListenLumberjack or to extend it to handle 
> both v1 and v2 of the protocol.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (NIFI-3238) ListenLumberjack should support the *beat protocol

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15889350#comment-15889350
 ] 

ASF GitHub Bot commented on NIFI-3238:
--

Github user mattyb149 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1418#discussion_r103597193
  
--- Diff: 
nifi-nar-bundles/nifi-lumberjack-bundle/nifi-lumberjack-processors/src/main/java/org/apache/nifi/processors/lumberjack/ListenLumberjack.java
 ---
@@ -62,12 +62,13 @@
 
 import com.google.gson.Gson;
 
+@Deprecated
 @InputRequirement(InputRequirement.Requirement.INPUT_FORBIDDEN)
 @Tags({"listen", "lumberjack", "tcp", "logs"})
-@CapabilityDescription("Listens for Lumberjack messages being sent to a 
given port over TCP. Each message will be " +
+@CapabilityDescription("This processor is deprecated and will be removed 
in the near future. Listens for Lumberjack messages being sent to a given port 
over TCP. Each message will be " +
--- End diff --

maybe rephrase to "may be removed from the distribution". Deprecated things 
have a tendency to linger ;)


> ListenLumberjack should support the *beat protocol
> --
>
> Key: NIFI-3238
> URL: https://issues.apache.org/jira/browse/NIFI-3238
> Project: Apache NiFi
>  Issue Type: Improvement
>Reporter: Andre F de Miranda
>Assignee: Andre F de Miranda
>
> ListenLumberjack currently only supports v1 of the Lumberjack protocol. This 
> version has been deprecated in favor of v2, which is used on *beat (e.g. 
> filebeat, packetbeat, etc) edge components of the ELK  stack.
> We should consider deprecating ListenLumberjack or to extend it to handle 
> both v1 and v2 of the protocol.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (NIFI-3238) ListenLumberjack should support the *beat protocol

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15889351#comment-15889351
 ] 

ASF GitHub Bot commented on NIFI-3238:
--

Github user mattyb149 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1418#discussion_r103597308
  
--- Diff: 
nifi-nar-bundles/nifi-lumberjack-bundle/nifi-lumberjack-processors/src/main/java/org/apache/nifi/processors/lumberjack/event/LumberjackEvent.java
 ---
@@ -24,6 +24,7 @@
 /**
  * A Lumberjack event which adds the transaction number and command to the 
StandardEvent.
  */
+@Deprecated
--- End diff --

For all @Deprecated tags, please provide @deprecated Javadoc for why it is 
being deprecated and what they should switch to


> ListenLumberjack should support the *beat protocol
> --
>
> Key: NIFI-3238
> URL: https://issues.apache.org/jira/browse/NIFI-3238
> Project: Apache NiFi
>  Issue Type: Improvement
>Reporter: Andre F de Miranda
>Assignee: Andre F de Miranda
>
> ListenLumberjack currently only supports v1 of the Lumberjack protocol. This 
> version has been deprecated in favor of v2, which is used on *beat (e.g. 
> filebeat, packetbeat, etc) edge components of the ELK  stack.
> We should consider deprecating ListenLumberjack or to extend it to handle 
> both v1 and v2 of the protocol.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (NIFI-3238) ListenLumberjack should support the *beat protocol

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15889353#comment-15889353
 ] 

ASF GitHub Bot commented on NIFI-3238:
--

Github user mattyb149 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1418#discussion_r103596699
  
--- Diff: 
nifi-nar-bundles/nifi-beats-bundle/nifi-beats-processors/src/main/java/org/apache/nifi/processors/beats/ListenBeats.java
 ---
@@ -0,0 +1,216 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.beats;
+
+import java.io.IOException;
+import java.nio.ByteBuffer;
+import java.nio.charset.Charset;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.BlockingQueue;
+
+import javax.net.ssl.SSLContext;
+
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.WritesAttribute;
+import org.apache.nifi.annotation.behavior.WritesAttributes;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.SeeAlso;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.flowfile.attributes.CoreAttributes;
+import org.apache.nifi.flowfile.attributes.FlowFileAttributeKey;
+import org.apache.nifi.processor.DataUnit;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import 
org.apache.nifi.processor.util.listen.AbstractListenEventBatchingProcessor;
+import 
org.apache.nifi.processor.util.listen.dispatcher.AsyncChannelDispatcher;
+import org.apache.nifi.processor.util.listen.dispatcher.ChannelDispatcher;
+import 
org.apache.nifi.processor.util.listen.dispatcher.SocketChannelDispatcher;
+import org.apache.nifi.processor.util.listen.event.EventFactory;
+import org.apache.nifi.processor.util.listen.handler.ChannelHandlerFactory;
+import org.apache.nifi.processor.util.listen.response.ChannelResponder;
+import org.apache.nifi.processor.util.listen.response.ChannelResponse;
+import org.apache.nifi.processors.beats.event.BeatsEvent;
+import org.apache.nifi.processors.beats.event.BeatsEventFactory;
+import org.apache.nifi.processors.beats.frame.BeatsEncoder;
+import 
org.apache.nifi.processors.beats.handler.BeatsSocketChannelHandlerFactory;
+import org.apache.nifi.processors.beats.response.BeatsChannelResponse;
+import org.apache.nifi.processors.beats.response.BeatsResponse;
+import org.apache.nifi.ssl.SSLContextService;
+
+@InputRequirement(InputRequirement.Requirement.INPUT_FORBIDDEN)
+@Tags({"listen", "beats", "tcp", "logs"})
+@CapabilityDescription("Listens for messages sent by libbeat compatible 
clients (e.g. filebeats, metricbeats, etc) being sent to a given port over TCP, 
writing its contents in" +
+"JSON format to the content of the message to a FlowFile." +
+"This processor replaces the now deprecated ListenLumberjack")
+@WritesAttributes({
+@WritesAttribute(attribute = "beats.sender", description = "The 
sending host of the messages."),
+@WritesAttribute(attribute = "beats.port", description = "The sending 
port the messages were received over."),
+@WritesAttribute(attribute = "beats.sequencenumber", description = 
"The sequence number of the message. Only included if  is 1."),
+@WritesAttribute(attribute = "mime.type", description = "The mime.type 
of the content which is application/json")
+})
+@SeeAlso(classNames = {"org.apache.nifi.processors.standard.ParseSyslog"})
+public class ListenBeats 

[jira] [Commented] (NIFI-3238) ListenLumberjack should support the *beat protocol

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15889357#comment-15889357
 ] 

ASF GitHub Bot commented on NIFI-3238:
--

Github user mattyb149 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1418#discussion_r103595420
  
--- Diff: 
nifi-nar-bundles/nifi-beats-bundle/nifi-beats-processors/src/main/java/org/apache/nifi/processors/beats/ListenBeats.java
 ---
@@ -0,0 +1,216 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.beats;
+
+import java.io.IOException;
+import java.nio.ByteBuffer;
+import java.nio.charset.Charset;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.BlockingQueue;
+
+import javax.net.ssl.SSLContext;
+
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.WritesAttribute;
+import org.apache.nifi.annotation.behavior.WritesAttributes;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.SeeAlso;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.flowfile.attributes.CoreAttributes;
+import org.apache.nifi.flowfile.attributes.FlowFileAttributeKey;
+import org.apache.nifi.processor.DataUnit;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import 
org.apache.nifi.processor.util.listen.AbstractListenEventBatchingProcessor;
+import 
org.apache.nifi.processor.util.listen.dispatcher.AsyncChannelDispatcher;
+import org.apache.nifi.processor.util.listen.dispatcher.ChannelDispatcher;
+import 
org.apache.nifi.processor.util.listen.dispatcher.SocketChannelDispatcher;
+import org.apache.nifi.processor.util.listen.event.EventFactory;
+import org.apache.nifi.processor.util.listen.handler.ChannelHandlerFactory;
+import org.apache.nifi.processor.util.listen.response.ChannelResponder;
+import org.apache.nifi.processor.util.listen.response.ChannelResponse;
+import org.apache.nifi.processors.beats.event.BeatsEvent;
+import org.apache.nifi.processors.beats.event.BeatsEventFactory;
+import org.apache.nifi.processors.beats.frame.BeatsEncoder;
+import 
org.apache.nifi.processors.beats.handler.BeatsSocketChannelHandlerFactory;
+import org.apache.nifi.processors.beats.response.BeatsChannelResponse;
+import org.apache.nifi.processors.beats.response.BeatsResponse;
+import org.apache.nifi.ssl.SSLContextService;
+
+@InputRequirement(InputRequirement.Requirement.INPUT_FORBIDDEN)
+@Tags({"listen", "beats", "tcp", "logs"})
+@CapabilityDescription("Listens for messages sent by libbeat compatible 
clients (e.g. filebeats, metricbeats, etc) being sent to a given port over TCP, 
writing its contents in" +
+"JSON format to the content of the message to a FlowFile." +
+"This processor replaces the now deprecated ListenLumberjack")
+@WritesAttributes({
+@WritesAttribute(attribute = "beats.sender", description = "The 
sending host of the messages."),
+@WritesAttribute(attribute = "beats.port", description = "The sending 
port the messages were received over."),
+@WritesAttribute(attribute = "beats.sequencenumber", description = 
"The sequence number of the message. Only included if  is 1."),
+@WritesAttribute(attribute = "mime.type", description = "The mime.type 
of the content which is application/json")
+})
+@SeeAlso(classNames = {"org.apache.nifi.processors.standard.ParseSyslog"})
+public class ListenBeats 

[jira] [Commented] (NIFI-3238) ListenLumberjack should support the *beat protocol

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15889354#comment-15889354
 ] 

ASF GitHub Bot commented on NIFI-3238:
--

Github user mattyb149 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1418#discussion_r103597456
  
--- Diff: nifi-nar-bundles/nifi-beats-bundle/nifi-beats-processors/pom.xml 
---
@@ -0,0 +1,76 @@
+
+
+http://maven.apache.org/POM/4.0.0; 
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance; 
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 
http://maven.apache.org/xsd/maven-4.0.0.xsd;>
+4.0.0
+
+
+org.apache.nifi
+nifi-beats-bundle
+1.2.0-SNAPSHOT
+
+
+nifi-beats-processors
+jar
+
+
+
+org.apache.nifi
+nifi-api
+
+
+org.apache.nifi
+nifi-processor-utils
+
+
+org.apache.nifi
+nifi-mock
+test
+
+
+org.apache.nifi
+nifi-socket-utils
+
+
+org.apache.nifi
+nifi-utils
+
+
+org.apache.nifi
+nifi-flowfile-packager
+
+
+org.apache.nifi
+nifi-ssl-context-service-api
+provided
+
+
+org.slf4j
+slf4j-simple
+test
+
+
+junit
+junit
+4.11
+test
+
+
+com.google.code.gson
--- End diff --

if this is a compile dependency and "you're already in there", please move 
this up above the "test" and "provided" dependencies. Not a requirement but a 
suggestion :)


> ListenLumberjack should support the *beat protocol
> --
>
> Key: NIFI-3238
> URL: https://issues.apache.org/jira/browse/NIFI-3238
> Project: Apache NiFi
>  Issue Type: Improvement
>Reporter: Andre F de Miranda
>Assignee: Andre F de Miranda
>
> ListenLumberjack currently only supports v1 of the Lumberjack protocol. This 
> version has been deprecated in favor of v2, which is used on *beat (e.g. 
> filebeat, packetbeat, etc) edge components of the ELK  stack.
> We should consider deprecating ListenLumberjack or to extend it to handle 
> both v1 and v2 of the protocol.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (NIFI-3238) ListenLumberjack should support the *beat protocol

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15889355#comment-15889355
 ] 

ASF GitHub Bot commented on NIFI-3238:
--

Github user mattyb149 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1418#discussion_r103596545
  
--- Diff: 
nifi-nar-bundles/nifi-beats-bundle/nifi-beats-processors/src/main/java/org/apache/nifi/processors/beats/ListenBeats.java
 ---
@@ -0,0 +1,216 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.beats;
+
+import java.io.IOException;
+import java.nio.ByteBuffer;
+import java.nio.charset.Charset;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.BlockingQueue;
+
+import javax.net.ssl.SSLContext;
+
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.WritesAttribute;
+import org.apache.nifi.annotation.behavior.WritesAttributes;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.SeeAlso;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.flowfile.attributes.CoreAttributes;
+import org.apache.nifi.flowfile.attributes.FlowFileAttributeKey;
+import org.apache.nifi.processor.DataUnit;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import 
org.apache.nifi.processor.util.listen.AbstractListenEventBatchingProcessor;
+import 
org.apache.nifi.processor.util.listen.dispatcher.AsyncChannelDispatcher;
+import org.apache.nifi.processor.util.listen.dispatcher.ChannelDispatcher;
+import 
org.apache.nifi.processor.util.listen.dispatcher.SocketChannelDispatcher;
+import org.apache.nifi.processor.util.listen.event.EventFactory;
+import org.apache.nifi.processor.util.listen.handler.ChannelHandlerFactory;
+import org.apache.nifi.processor.util.listen.response.ChannelResponder;
+import org.apache.nifi.processor.util.listen.response.ChannelResponse;
+import org.apache.nifi.processors.beats.event.BeatsEvent;
+import org.apache.nifi.processors.beats.event.BeatsEventFactory;
+import org.apache.nifi.processors.beats.frame.BeatsEncoder;
+import 
org.apache.nifi.processors.beats.handler.BeatsSocketChannelHandlerFactory;
+import org.apache.nifi.processors.beats.response.BeatsChannelResponse;
+import org.apache.nifi.processors.beats.response.BeatsResponse;
+import org.apache.nifi.ssl.SSLContextService;
+
+@InputRequirement(InputRequirement.Requirement.INPUT_FORBIDDEN)
+@Tags({"listen", "beats", "tcp", "logs"})
+@CapabilityDescription("Listens for messages sent by libbeat compatible 
clients (e.g. filebeats, metricbeats, etc) being sent to a given port over TCP, 
writing its contents in" +
+"JSON format to the content of the message to a FlowFile." +
+"This processor replaces the now deprecated ListenLumberjack")
+@WritesAttributes({
+@WritesAttribute(attribute = "beats.sender", description = "The 
sending host of the messages."),
+@WritesAttribute(attribute = "beats.port", description = "The sending 
port the messages were received over."),
+@WritesAttribute(attribute = "beats.sequencenumber", description = 
"The sequence number of the message. Only included if  is 1."),
+@WritesAttribute(attribute = "mime.type", description = "The mime.type 
of the content which is application/json")
+})
+@SeeAlso(classNames = {"org.apache.nifi.processors.standard.ParseSyslog"})
+public class ListenBeats 

[jira] [Commented] (NIFI-3238) ListenLumberjack should support the *beat protocol

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15889349#comment-15889349
 ] 

ASF GitHub Bot commented on NIFI-3238:
--

Github user mattyb149 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1418#discussion_r103595712
  
--- Diff: 
nifi-nar-bundles/nifi-beats-bundle/nifi-beats-processors/src/main/java/org/apache/nifi/processors/beats/frame/BeatsDecoder.java
 ---
@@ -0,0 +1,330 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.beats.frame;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.nio.ByteBuffer;
+import java.nio.charset.Charset;
+import java.util.LinkedList;
+import java.util.List;
+import java.util.zip.InflaterInputStream;
+
+import org.apache.nifi.stream.io.ByteArrayInputStream;
+import org.apache.nifi.stream.io.ByteArrayOutputStream;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * Decodes a Beats frame by maintaining a state based on each byte that 
has been processed. This class
+ * should not be shared by multiple threads.
+ */
+public class BeatsDecoder {
+
+
+static final Logger logger = 
LoggerFactory.getLogger(BeatsDecoder.class);
--- End diff --

Should we instead pass in a reference to the ComponentLog rather than 
depend on SLF4J as the implementation? I realize SLF4J is itself a facade, but 
this couples the logging directly to it rather than using the NiFi logging 
"framework"


> ListenLumberjack should support the *beat protocol
> --
>
> Key: NIFI-3238
> URL: https://issues.apache.org/jira/browse/NIFI-3238
> Project: Apache NiFi
>  Issue Type: Improvement
>Reporter: Andre F de Miranda
>Assignee: Andre F de Miranda
>
> ListenLumberjack currently only supports v1 of the Lumberjack protocol. This 
> version has been deprecated in favor of v2, which is used on *beat (e.g. 
> filebeat, packetbeat, etc) edge components of the ELK  stack.
> We should consider deprecating ListenLumberjack or to extend it to handle 
> both v1 and v2 of the protocol.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[GitHub] nifi pull request #1418: NIFI-3238 - Introduce ListenBeats processor and dep...

[mattyb149]

Github user mattyb149 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1418#discussion_r103595420
  
--- Diff: 
nifi-nar-bundles/nifi-beats-bundle/nifi-beats-processors/src/main/java/org/apache/nifi/processors/beats/ListenBeats.java
 ---
@@ -0,0 +1,216 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.beats;
+
+import java.io.IOException;
+import java.nio.ByteBuffer;
+import java.nio.charset.Charset;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.BlockingQueue;
+
+import javax.net.ssl.SSLContext;
+
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.WritesAttribute;
+import org.apache.nifi.annotation.behavior.WritesAttributes;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.SeeAlso;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.flowfile.attributes.CoreAttributes;
+import org.apache.nifi.flowfile.attributes.FlowFileAttributeKey;
+import org.apache.nifi.processor.DataUnit;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import 
org.apache.nifi.processor.util.listen.AbstractListenEventBatchingProcessor;
+import 
org.apache.nifi.processor.util.listen.dispatcher.AsyncChannelDispatcher;
+import org.apache.nifi.processor.util.listen.dispatcher.ChannelDispatcher;
+import 
org.apache.nifi.processor.util.listen.dispatcher.SocketChannelDispatcher;
+import org.apache.nifi.processor.util.listen.event.EventFactory;
+import org.apache.nifi.processor.util.listen.handler.ChannelHandlerFactory;
+import org.apache.nifi.processor.util.listen.response.ChannelResponder;
+import org.apache.nifi.processor.util.listen.response.ChannelResponse;
+import org.apache.nifi.processors.beats.event.BeatsEvent;
+import org.apache.nifi.processors.beats.event.BeatsEventFactory;
+import org.apache.nifi.processors.beats.frame.BeatsEncoder;
+import 
org.apache.nifi.processors.beats.handler.BeatsSocketChannelHandlerFactory;
+import org.apache.nifi.processors.beats.response.BeatsChannelResponse;
+import org.apache.nifi.processors.beats.response.BeatsResponse;
+import org.apache.nifi.ssl.SSLContextService;
+
+@InputRequirement(InputRequirement.Requirement.INPUT_FORBIDDEN)
+@Tags({"listen", "beats", "tcp", "logs"})
+@CapabilityDescription("Listens for messages sent by libbeat compatible 
clients (e.g. filebeats, metricbeats, etc) being sent to a given port over TCP, 
writing its contents in" +
+"JSON format to the content of the message to a FlowFile." +
+"This processor replaces the now deprecated ListenLumberjack")
+@WritesAttributes({
+@WritesAttribute(attribute = "beats.sender", description = "The 
sending host of the messages."),
+@WritesAttribute(attribute = "beats.port", description = "The sending 
port the messages were received over."),
+@WritesAttribute(attribute = "beats.sequencenumber", description = 
"The sequence number of the message. Only included if  is 1."),
+@WritesAttribute(attribute = "mime.type", description = "The mime.type 
of the content which is application/json")
+})
+@SeeAlso(classNames = {"org.apache.nifi.processors.standard.ParseSyslog"})
+public class ListenBeats extends 
AbstractListenEventBatchingProcessor {
+
+public static final PropertyDescriptor SSL_CONTEXT_SERVICE = new 
PropertyDescriptor.Builder()
+.name("SSL Context Service")
+.description("The 

[GitHub] nifi pull request #1418: NIFI-3238 - Introduce ListenBeats processor and dep...

[mattyb149]

Github user mattyb149 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1418#discussion_r103597308
  
--- Diff: 
nifi-nar-bundles/nifi-lumberjack-bundle/nifi-lumberjack-processors/src/main/java/org/apache/nifi/processors/lumberjack/event/LumberjackEvent.java
 ---
@@ -24,6 +24,7 @@
 /**
  * A Lumberjack event which adds the transaction number and command to the 
StandardEvent.
  */
+@Deprecated
--- End diff --

For all @Deprecated tags, please provide @deprecated Javadoc for why it is 
being deprecated and what they should switch to


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request #1418: NIFI-3238 - Introduce ListenBeats processor and dep...

[mattyb149]

Github user mattyb149 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1418#discussion_r103595838
  
--- Diff: 
nifi-nar-bundles/nifi-beats-bundle/nifi-beats-processors/src/main/java/org/apache/nifi/processors/beats/frame/BeatsDecoder.java
 ---
@@ -0,0 +1,330 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.beats.frame;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.nio.ByteBuffer;
+import java.nio.charset.Charset;
+import java.util.LinkedList;
+import java.util.List;
+import java.util.zip.InflaterInputStream;
+
+import org.apache.nifi.stream.io.ByteArrayInputStream;
+import org.apache.nifi.stream.io.ByteArrayOutputStream;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * Decodes a Beats frame by maintaining a state based on each byte that 
has been processed. This class
+ * should not be shared by multiple threads.
+ */
+public class BeatsDecoder {
+
+
+static final Logger logger = 
LoggerFactory.getLogger(BeatsDecoder.class);
+
+private BeatsFrame.Builder frameBuilder;
+private BeatsState currState = BeatsState.VERSION;
+private byte decodedFrameType;
+
+private byte[] unprocessedData;
+
+private final Charset charset;
+private final ByteArrayOutputStream currBytes;
+
+private long windowSize;
+
+static final int MIN_FRAME_HEADER_LENGTH = 2; // Version + Type
+static final int WINDOWSIZE_LENGTH = MIN_FRAME_HEADER_LENGTH + 4; // 
32bit unsigned window size
+static final int COMPRESSED_MIN_LENGTH = MIN_FRAME_HEADER_LENGTH + 4; 
// 32 bit unsigned + payload
+static final int JSON_MIN_LENGTH = MIN_FRAME_HEADER_LENGTH + 8; // 32 
bit unsigned sequence number + 32 bit unsigned payload length
+
+public static final byte FRAME_WINDOWSIZE = 0x57, FRAME_DATA = 0x44, 
FRAME_COMPRESSED = 0x43, FRAME_ACK = 0x41, FRAME_JSON = 0x4a;
+
+/**
+ * @param charset the charset to decode bytes from the frame
+ */
+public BeatsDecoder(final Charset charset) {
+this(charset, new ByteArrayOutputStream(4096));
--- End diff --

Is 4K ok as a default size? Should it be passed in here and perhaps default 
somewhere else in case it needs to be configurable?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request #1418: NIFI-3238 - Introduce ListenBeats processor and dep...

[mattyb149]

Github user mattyb149 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1418#discussion_r103597193
  
--- Diff: 
nifi-nar-bundles/nifi-lumberjack-bundle/nifi-lumberjack-processors/src/main/java/org/apache/nifi/processors/lumberjack/ListenLumberjack.java
 ---
@@ -62,12 +62,13 @@
 
 import com.google.gson.Gson;
 
+@Deprecated
 @InputRequirement(InputRequirement.Requirement.INPUT_FORBIDDEN)
 @Tags({"listen", "lumberjack", "tcp", "logs"})
-@CapabilityDescription("Listens for Lumberjack messages being sent to a 
given port over TCP. Each message will be " +
+@CapabilityDescription("This processor is deprecated and will be removed 
in the near future. Listens for Lumberjack messages being sent to a given port 
over TCP. Each message will be " +
--- End diff --

maybe rephrase to "may be removed from the distribution". Deprecated things 
have a tendency to linger ;)


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request #1418: NIFI-3238 - Introduce ListenBeats processor and dep...

[mattyb149]

Github user mattyb149 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1418#discussion_r103597989
  
--- Diff: 
nifi-nar-bundles/nifi-beats-bundle/nifi-beats-nar/src/main/resources/META-INF/LICENSE
 ---
@@ -0,0 +1,233 @@
+
+ Apache License
+   Version 2.0, January 2004
+http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   1. Definitions.
+
+  "License" shall mean the terms and conditions for use, reproduction,
+  and distribution as defined by Sections 1 through 9 of this document.
+
+  "Licensor" shall mean the copyright owner or entity authorized by
+  the copyright owner that is granting the License.
+
+  "Legal Entity" shall mean the union of the acting entity and all
+  other entities that control, are controlled by, or are under common
+  control with that entity. For the purposes of this definition,
+  "control" means (i) the power, direct or indirect, to cause the
+  direction or management of such entity, whether by contract or
+  otherwise, or (ii) ownership of fifty percent (50%) or more of the
+  outstanding shares, or (iii) beneficial ownership of such entity.
+
+  "You" (or "Your") shall mean an individual or Legal Entity
+  exercising permissions granted by this License.
+
+  "Source" form shall mean the preferred form for making modifications,
+  including but not limited to software source code, documentation
+  source, and configuration files.
+
+  "Object" form shall mean any form resulting from mechanical
+  transformation or translation of a Source form, including but
+  not limited to compiled object code, generated documentation,
+  and conversions to other media types.
+
+  "Work" shall mean the work of authorship, whether in Source or
+  Object form, made available under the License, as indicated by a
+  copyright notice that is included in or attached to the work
+  (an example is provided in the Appendix below).
+
+  "Derivative Works" shall mean any work, whether in Source or Object
+  form, that is based on (or derived from) the Work and for which the
+  editorial revisions, annotations, elaborations, or other 
modifications
+  represent, as a whole, an original work of authorship. For the 
purposes
+  of this License, Derivative Works shall not include works that remain
+  separable from, or merely link (or bind by name) to the interfaces 
of,
+  the Work and Derivative Works thereof.
+
+  "Contribution" shall mean any work of authorship, including
+  the original version of the Work and any modifications or additions
+  to that Work or Derivative Works thereof, that is intentionally
+  submitted to Licensor for inclusion in the Work by the copyright 
owner
+  or by an individual or Legal Entity authorized to submit on behalf of
+  the copyright owner. For the purposes of this definition, "submitted"
+  means any form of electronic, verbal, or written communication sent
+  to the Licensor or its representatives, including but not limited to
+  communication on electronic mailing lists, source code control 
systems,
+  and issue tracking systems that are managed by, or on behalf of, the
+  Licensor for the purpose of discussing and improving the Work, but
+  excluding communication that is conspicuously marked or otherwise
+  designated in writing by the copyright owner as "Not a Contribution."
+
+  "Contributor" shall mean Licensor and any individual or Legal Entity
+  on behalf of whom a Contribution has been received by Licensor and
+  subsequently incorporated within the Work.
+
+   2. Grant of Copyright License. Subject to the terms and conditions of
+  this License, each Contributor hereby grants to You a perpetual,
+  worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+  copyright license to reproduce, prepare Derivative Works of,
+  publicly display, publicly perform, sublicense, and distribute the
+  Work and such Derivative Works in Source or Object form.
+
+   3. Grant of Patent License. Subject to the terms and conditions of
+  this License, each Contributor hereby grants to You a perpetual,
+  worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+  (except as stated in this section) patent license to make, have made,
+  use, offer to sell, sell, import, and otherwise transfer the Work,
+  where such license applies only to those patent 

[GitHub] nifi pull request #1418: NIFI-3238 - Introduce ListenBeats processor and dep...

[mattyb149]

Github user mattyb149 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1418#discussion_r103595712
  
--- Diff: 
nifi-nar-bundles/nifi-beats-bundle/nifi-beats-processors/src/main/java/org/apache/nifi/processors/beats/frame/BeatsDecoder.java
 ---
@@ -0,0 +1,330 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.beats.frame;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.nio.ByteBuffer;
+import java.nio.charset.Charset;
+import java.util.LinkedList;
+import java.util.List;
+import java.util.zip.InflaterInputStream;
+
+import org.apache.nifi.stream.io.ByteArrayInputStream;
+import org.apache.nifi.stream.io.ByteArrayOutputStream;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * Decodes a Beats frame by maintaining a state based on each byte that 
has been processed. This class
+ * should not be shared by multiple threads.
+ */
+public class BeatsDecoder {
+
+
+static final Logger logger = 
LoggerFactory.getLogger(BeatsDecoder.class);
--- End diff --

Should we instead pass in a reference to the ComponentLog rather than 
depend on SLF4J as the implementation? I realize SLF4J is itself a facade, but 
this couples the logging directly to it rather than using the NiFi logging 
"framework"


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request #1418: NIFI-3238 - Introduce ListenBeats processor and dep...

[mattyb149]

Github user mattyb149 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1418#discussion_r103597456
  
--- Diff: nifi-nar-bundles/nifi-beats-bundle/nifi-beats-processors/pom.xml 
---
@@ -0,0 +1,76 @@
+
+
+http://maven.apache.org/POM/4.0.0; 
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance; 
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 
http://maven.apache.org/xsd/maven-4.0.0.xsd;>
+4.0.0
+
+
+org.apache.nifi
+nifi-beats-bundle
+1.2.0-SNAPSHOT
+
+
+nifi-beats-processors
+jar
+
+
+
+org.apache.nifi
+nifi-api
+
+
+org.apache.nifi
+nifi-processor-utils
+
+
+org.apache.nifi
+nifi-mock
+test
+
+
+org.apache.nifi
+nifi-socket-utils
+
+
+org.apache.nifi
+nifi-utils
+
+
+org.apache.nifi
+nifi-flowfile-packager
+
+
+org.apache.nifi
+nifi-ssl-context-service-api
+provided
+
+
+org.slf4j
+slf4j-simple
+test
+
+
+junit
+junit
+4.11
+test
+
+
+com.google.code.gson
--- End diff --

if this is a compile dependency and "you're already in there", please move 
this up above the "test" and "provided" dependencies. Not a requirement but a 
suggestion :)


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request #1418: NIFI-3238 - Introduce ListenBeats processor and dep...

[mattyb149]

Github user mattyb149 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1418#discussion_r103596545
  
--- Diff: 
nifi-nar-bundles/nifi-beats-bundle/nifi-beats-processors/src/main/java/org/apache/nifi/processors/beats/ListenBeats.java
 ---
@@ -0,0 +1,216 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.beats;
+
+import java.io.IOException;
+import java.nio.ByteBuffer;
+import java.nio.charset.Charset;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.BlockingQueue;
+
+import javax.net.ssl.SSLContext;
+
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.WritesAttribute;
+import org.apache.nifi.annotation.behavior.WritesAttributes;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.SeeAlso;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.flowfile.attributes.CoreAttributes;
+import org.apache.nifi.flowfile.attributes.FlowFileAttributeKey;
+import org.apache.nifi.processor.DataUnit;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import 
org.apache.nifi.processor.util.listen.AbstractListenEventBatchingProcessor;
+import 
org.apache.nifi.processor.util.listen.dispatcher.AsyncChannelDispatcher;
+import org.apache.nifi.processor.util.listen.dispatcher.ChannelDispatcher;
+import 
org.apache.nifi.processor.util.listen.dispatcher.SocketChannelDispatcher;
+import org.apache.nifi.processor.util.listen.event.EventFactory;
+import org.apache.nifi.processor.util.listen.handler.ChannelHandlerFactory;
+import org.apache.nifi.processor.util.listen.response.ChannelResponder;
+import org.apache.nifi.processor.util.listen.response.ChannelResponse;
+import org.apache.nifi.processors.beats.event.BeatsEvent;
+import org.apache.nifi.processors.beats.event.BeatsEventFactory;
+import org.apache.nifi.processors.beats.frame.BeatsEncoder;
+import 
org.apache.nifi.processors.beats.handler.BeatsSocketChannelHandlerFactory;
+import org.apache.nifi.processors.beats.response.BeatsChannelResponse;
+import org.apache.nifi.processors.beats.response.BeatsResponse;
+import org.apache.nifi.ssl.SSLContextService;
+
+@InputRequirement(InputRequirement.Requirement.INPUT_FORBIDDEN)
+@Tags({"listen", "beats", "tcp", "logs"})
+@CapabilityDescription("Listens for messages sent by libbeat compatible 
clients (e.g. filebeats, metricbeats, etc) being sent to a given port over TCP, 
writing its contents in" +
+"JSON format to the content of the message to a FlowFile." +
+"This processor replaces the now deprecated ListenLumberjack")
+@WritesAttributes({
+@WritesAttribute(attribute = "beats.sender", description = "The 
sending host of the messages."),
+@WritesAttribute(attribute = "beats.port", description = "The sending 
port the messages were received over."),
+@WritesAttribute(attribute = "beats.sequencenumber", description = 
"The sequence number of the message. Only included if  is 1."),
+@WritesAttribute(attribute = "mime.type", description = "The mime.type 
of the content which is application/json")
+})
+@SeeAlso(classNames = {"org.apache.nifi.processors.standard.ParseSyslog"})
+public class ListenBeats extends 
AbstractListenEventBatchingProcessor {
--- End diff --

Inheriting the RECV_BUFFER_SIZE property from the other 
"AbstractListenEventProcessor" properties might be a little confusing here as 
it is not a "round" number 

[GitHub] nifi pull request #1418: NIFI-3238 - Introduce ListenBeats processor and dep...

[mattyb149]

Github user mattyb149 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1418#discussion_r103596699
  
--- Diff: 
nifi-nar-bundles/nifi-beats-bundle/nifi-beats-processors/src/main/java/org/apache/nifi/processors/beats/ListenBeats.java
 ---
@@ -0,0 +1,216 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.beats;
+
+import java.io.IOException;
+import java.nio.ByteBuffer;
+import java.nio.charset.Charset;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.BlockingQueue;
+
+import javax.net.ssl.SSLContext;
+
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.WritesAttribute;
+import org.apache.nifi.annotation.behavior.WritesAttributes;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.SeeAlso;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.flowfile.attributes.CoreAttributes;
+import org.apache.nifi.flowfile.attributes.FlowFileAttributeKey;
+import org.apache.nifi.processor.DataUnit;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import 
org.apache.nifi.processor.util.listen.AbstractListenEventBatchingProcessor;
+import 
org.apache.nifi.processor.util.listen.dispatcher.AsyncChannelDispatcher;
+import org.apache.nifi.processor.util.listen.dispatcher.ChannelDispatcher;
+import 
org.apache.nifi.processor.util.listen.dispatcher.SocketChannelDispatcher;
+import org.apache.nifi.processor.util.listen.event.EventFactory;
+import org.apache.nifi.processor.util.listen.handler.ChannelHandlerFactory;
+import org.apache.nifi.processor.util.listen.response.ChannelResponder;
+import org.apache.nifi.processor.util.listen.response.ChannelResponse;
+import org.apache.nifi.processors.beats.event.BeatsEvent;
+import org.apache.nifi.processors.beats.event.BeatsEventFactory;
+import org.apache.nifi.processors.beats.frame.BeatsEncoder;
+import 
org.apache.nifi.processors.beats.handler.BeatsSocketChannelHandlerFactory;
+import org.apache.nifi.processors.beats.response.BeatsChannelResponse;
+import org.apache.nifi.processors.beats.response.BeatsResponse;
+import org.apache.nifi.ssl.SSLContextService;
+
+@InputRequirement(InputRequirement.Requirement.INPUT_FORBIDDEN)
+@Tags({"listen", "beats", "tcp", "logs"})
+@CapabilityDescription("Listens for messages sent by libbeat compatible 
clients (e.g. filebeats, metricbeats, etc) being sent to a given port over TCP, 
writing its contents in" +
+"JSON format to the content of the message to a FlowFile." +
+"This processor replaces the now deprecated ListenLumberjack")
+@WritesAttributes({
+@WritesAttribute(attribute = "beats.sender", description = "The 
sending host of the messages."),
+@WritesAttribute(attribute = "beats.port", description = "The sending 
port the messages were received over."),
+@WritesAttribute(attribute = "beats.sequencenumber", description = 
"The sequence number of the message. Only included if  is 1."),
+@WritesAttribute(attribute = "mime.type", description = "The mime.type 
of the content which is application/json")
+})
+@SeeAlso(classNames = {"org.apache.nifi.processors.standard.ParseSyslog"})
+public class ListenBeats extends 
AbstractListenEventBatchingProcessor {
+
+public static final PropertyDescriptor SSL_CONTEXT_SERVICE = new 
PropertyDescriptor.Builder()
+.name("SSL Context Service")
+.description("The 

[jira] [Updated] (NIFI-3048) Update every NAR with appropriate L entries for Bouncy Castle

[Matt Burgess (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-3048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matt Burgess updated NIFI-3048:
---
Fix Version/s: 1.1.0

> Update every NAR with appropriate L entries for Bouncy Castle 
> 
>
> Key: NIFI-3048
> URL: https://issues.apache.org/jira/browse/NIFI-3048
> Project: Apache NiFi
>  Issue Type: Bug
>Reporter: Oleg Zhurakousky
>Assignee: Oleg Zhurakousky
>Priority: Blocker
> Fix For: 1.1.0
>
>
> Due to nifi-utils dependency on Bouncy Castle libs every NAR is now 
> distributed with Bouncy Castle packages and therefore needs appropriate L 
> entries



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (NIFI-3238) ListenLumberjack should support the *beat protocol

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15889292#comment-15889292
 ] 

ASF GitHub Bot commented on NIFI-3238:
--

Github user mattyb149 commented on the issue:

https://github.com/apache/nifi/pull/1418
  
Reviewing...


> ListenLumberjack should support the *beat protocol
> --
>
> Key: NIFI-3238
> URL: https://issues.apache.org/jira/browse/NIFI-3238
> Project: Apache NiFi
>  Issue Type: Improvement
>Reporter: Andre F de Miranda
>Assignee: Andre F de Miranda
>
> ListenLumberjack currently only supports v1 of the Lumberjack protocol. This 
> version has been deprecated in favor of v2, which is used on *beat (e.g. 
> filebeat, packetbeat, etc) edge components of the ELK  stack.
> We should consider deprecating ListenLumberjack or to extend it to handle 
> both v1 and v2 of the protocol.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[GitHub] nifi issue #1418: NIFI-3238 - Introduce ListenBeats processor and deprecates...

[mattyb149]

Github user mattyb149 commented on the issue:

https://github.com/apache/nifi/pull/1418
  
Reviewing...


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-3439) Making password property in HiveConnectionPool a hidden show.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3439?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15888920#comment-15888920
 ] 

ASF GitHub Bot commented on NIFI-3439:
--

Github user mcgilman commented on the issue:

https://github.com/apache/nifi/pull/1546
  
Hey @scottyaslan,

This is a great start, but I think there are a few fundamental issues with 
how `PropertyDescriptors` relay what sort of editor should be used. There does 
not appear to be a cross-browser support for masking `textareas` (multi-line 
input). There exists sensitive properties that currently expect the field to 
support multi-line input. The `input[type=password]` only supports a single 
line.

I think we would need to introduce a new attribute of `PropetyDescriptor` 
which would drive whether or not to use a masked single line input or the 
existing multi-line `textarea`. The existing multi-line `textareas` support 
sensitive by not returning/showing the value once set. We would not be able to 
support a `PropertyDescriptor` that requires masking and multi-line input.

Additional complications exist for sensitive properties that support 
expression language. We'd have to spend some time to determine if `CodeMirror` 
supports masking or if it's possible to introduce that sort of capability. If 
not, we would not be able to support a `PropertyDescriptor` that requires 
masking and supports expression language.


> Making password property in HiveConnectionPool a hidden show.
> -
>
> Key: NIFI-3439
> URL: https://issues.apache.org/jira/browse/NIFI-3439
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core UI
>Affects Versions: 1.1.1
> Environment: Google Chrome browser
>Reporter: Ramakrishnan V
>Assignee: Scott Aslan
>Priority: Minor
>  Labels: ControllerService, Hive, security
>   Original Estimate: 168h
>  Remaining Estimate: 168h
>
> When user types the password in the password attribute in hive connection 
> pool controller service, the characters are visible. Once completed the 
> password is encrypted and no longer visible.
> shouldn't be me made as normal hidden show password box ?
> The user always has the option to delete the controller service and 
> re-configure if he has entered a wrong password.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[GitHub] nifi issue #1546: [NIFI-3439] when users type into sensitive property fields...

[mcgilman]

Github user mcgilman commented on the issue:

https://github.com/apache/nifi/pull/1546
  
Hey @scottyaslan,

This is a great start, but I think there are a few fundamental issues with 
how `PropertyDescriptors` relay what sort of editor should be used. There does 
not appear to be a cross-browser support for masking `textareas` (multi-line 
input). There exists sensitive properties that currently expect the field to 
support multi-line input. The `input[type=password]` only supports a single 
line.

I think we would need to introduce a new attribute of `PropetyDescriptor` 
which would drive whether or not to use a masked single line input or the 
existing multi-line `textarea`. The existing multi-line `textareas` support 
sensitive by not returning/showing the value once set. We would not be able to 
support a `PropertyDescriptor` that requires masking and multi-line input.

Additional complications exist for sensitive properties that support 
expression language. We'd have to spend some time to determine if `CodeMirror` 
supports masking or if it's possible to introduce that sort of capability. If 
not, we would not be able to support a `PropertyDescriptor` that requires 
masking and supports expression language.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Updated] (NIFI-3268) Add AUTO_INCREMENT column in GenerateTableFetch to benefit index

[qfdk (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-3268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

qfdk updated NIFI-3268:
---
Status: Patch Available  (was: Open)

submit patch for testing

> Add AUTO_INCREMENT column in GenerateTableFetch to benefit index
> 
>
> Key: NIFI-3268
> URL: https://issues.apache.org/jira/browse/NIFI-3268
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Core Framework
>Affects Versions: 1.1.1
> Environment: - ubuntu 16.04
> - java version "1.8.0_111"
> - Java(TM) SE Runtime Environment (build 1.8.0_111-b14)
> - Java HotSpot(TM) 64-Bit Server VM (build 25.111-b14, mixed mode)
>Reporter: qfdk
>  Labels: easyfix
> Fix For: 1.2.0
>
>
> I added AUTO_INCREMENT column in  GenerateTableFetch to benefit index column
> By default this processor uses OFFSET, i have  problems with large data. 
> somme column has index so we could use index to speed up query time.
> I posted question here :
> https://community.hortonworks.com/questions/72586/how-can-i-use-an-array-with-putelasticsearch.html
> If you indexed un column (id), you could use this sql
> ```
> select xxx
> From x
> where 20=>id
> order by id
> limit 20
> ```
> “OFFSET is bad for skipping previous rows.” [Online]. Available: 
> http://Use-The-Index-Luke.com/sql/partial-results/fetch-next-page. [Accessed: 
> 27-Dec-2016].
> Thank you in advance



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (NIFI-3540) QueryDatabaseTable Failing to Track MS SQL Max Values

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1542#comment-1542
 ] 

ASF GitHub Bot commented on NIFI-3540:
--

Github user patricker commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1547#discussion_r103548585
  
--- Diff: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/QueryDatabaseTable.java
 ---
@@ -437,7 +439,7 @@ public void processRow(ResultSet resultSet) throws 
IOException {
 if (nrOfColumns > 0) {
 for (int i = 1; i <= nrOfColumns; i++) {
 String colName = 
meta.getColumnName(i).toLowerCase();
-String fullyQualifiedMaxValueKey = 
getStateKey(meta.getTableName(i), colName);
+String fullyQualifiedMaxValueKey = 
getStateKey(tableName, colName);
--- End diff --

I found a discussion on this from way back in 2007 where it sounded like it 
depended on which cursor type was used to bring back the results, and which 
column types were included.

https://social.msdn.microsoft.com/Forums/sqlserver/en-US/55e8cbb2-b11c-446e-93ab-dc30658caf99/resultsetmetadatagettablename-returns-instead-of-table-name?forum=sqldataaccess



> QueryDatabaseTable Failing to Track MS SQL Max Values
> -
>
> Key: NIFI-3540
> URL: https://issues.apache.org/jira/browse/NIFI-3540
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Affects Versions: 1.2.0
>Reporter: Peter Wicks
>Assignee: Peter Wicks
>Priority: Blocker
>
> A recent change was made to allow for EL to be used in QDB and GTF processors 
> for dynamic table name and max value column tracking.
> Part of this change lead to table names being retrieved dynamically from the 
> ResultSetMetadata rather then using the value returned by resolving the EL.  
> In the MS SQL JDBC driver there is a known bug (for the last 10 years...) 
> where sometimes getTableName returns an empty string rather then the 
> table/view name.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[GitHub] nifi pull request #1547: NIFI-3540 - QueryDatabaseTable Failing to Track MS ...

[patricker]

Github user patricker commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1547#discussion_r103548585
  
--- Diff: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/QueryDatabaseTable.java
 ---
@@ -437,7 +439,7 @@ public void processRow(ResultSet resultSet) throws 
IOException {
 if (nrOfColumns > 0) {
 for (int i = 1; i <= nrOfColumns; i++) {
 String colName = 
meta.getColumnName(i).toLowerCase();
-String fullyQualifiedMaxValueKey = 
getStateKey(meta.getTableName(i), colName);
+String fullyQualifiedMaxValueKey = 
getStateKey(tableName, colName);
--- End diff --

I found a discussion on this from way back in 2007 where it sounded like it 
depended on which cursor type was used to bring back the results, and which 
column types were included.

https://social.msdn.microsoft.com/Forums/sqlserver/en-US/55e8cbb2-b11c-446e-93ab-dc30658caf99/resultsetmetadatagettablename-returns-instead-of-table-name?forum=sqldataaccess



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-3540) QueryDatabaseTable Failing to Track MS SQL Max Values

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1526#comment-1526
 ] 

ASF GitHub Bot commented on NIFI-3540:
--

Github user mattyb149 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1547#discussion_r103546463
  
--- Diff: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/QueryDatabaseTable.java
 ---
@@ -437,7 +439,7 @@ public void processRow(ResultSet resultSet) throws 
IOException {
 if (nrOfColumns > 0) {
 for (int i = 1; i <= nrOfColumns; i++) {
 String colName = 
meta.getColumnName(i).toLowerCase();
-String fullyQualifiedMaxValueKey = 
getStateKey(meta.getTableName(i), colName);
+String fullyQualifiedMaxValueKey = 
getStateKey(tableName, colName);
--- End diff --

Ugh, their driver doesn't include table name info in the result set?! I'll 
give this a try on MySQL, Oracle, and Postgres as well, to make sure there are 
no issues with case sensitivity, etc.  Good catch thanks!


> QueryDatabaseTable Failing to Track MS SQL Max Values
> -
>
> Key: NIFI-3540
> URL: https://issues.apache.org/jira/browse/NIFI-3540
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Affects Versions: 1.2.0
>Reporter: Peter Wicks
>Assignee: Peter Wicks
>Priority: Blocker
>
> A recent change was made to allow for EL to be used in QDB and GTF processors 
> for dynamic table name and max value column tracking.
> Part of this change lead to table names being retrieved dynamically from the 
> ResultSetMetadata rather then using the value returned by resolving the EL.  
> In the MS SQL JDBC driver there is a known bug (for the last 10 years...) 
> where sometimes getTableName returns an empty string rather then the 
> table/view name.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[GitHub] nifi pull request #1547: NIFI-3540 - QueryDatabaseTable Failing to Track MS ...

[mattyb149]

Github user mattyb149 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/1547#discussion_r103546463
  
--- Diff: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/QueryDatabaseTable.java
 ---
@@ -437,7 +439,7 @@ public void processRow(ResultSet resultSet) throws 
IOException {
 if (nrOfColumns > 0) {
 for (int i = 1; i <= nrOfColumns; i++) {
 String colName = 
meta.getColumnName(i).toLowerCase();
-String fullyQualifiedMaxValueKey = 
getStateKey(meta.getTableName(i), colName);
+String fullyQualifiedMaxValueKey = 
getStateKey(tableName, colName);
--- End diff --

Ugh, their driver doesn't include table name info in the result set?! I'll 
give this a try on MySQL, Oracle, and Postgres as well, to make sure there are 
no issues with case sensitivity, etc.  Good catch thanks!


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-3540) QueryDatabaseTable Failing to Track MS SQL Max Values

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1506#comment-1506
 ] 

ASF GitHub Bot commented on NIFI-3540:
--

GitHub user patricker opened a pull request:

https://github.com/apache/nifi/pull/1547

NIFI-3540

@mattyb149, ran into a bug with the MS SQL JDBC driver where the change you 
made for NIFI-2881/#1407 causes problems.  Small fix. 

### For all changes:
- [x] Is there a JIRA ticket associated with this PR? Is it referenced 
 in the commit message?

- [x] Does your PR title start with NIFI- where  is the JIRA number 
you are trying to resolve? Pay particular attention to the hyphen "-" character.

- [x] Has your PR been rebased against the latest commit within the target 
branch (typically master)?

- [x] Is your initial contribution a single, squashed commit?

### For code changes:
- [ ] Have you ensured that the full suite of tests is executed via mvn 
-Pcontrib-check clean install at the root nifi folder?
- [ ] Have you written or updated unit tests to verify your changes?
- [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)? 
- [ ] If applicable, have you updated the LICENSE file, including the main 
LICENSE file under nifi-assembly?
- [ ] If applicable, have you updated the NOTICE file, including the main 
NOTICE file found under nifi-assembly?
- [ ] If adding new Properties, have you added .displayName in addition to 
.name (programmatic access) for each of the new properties?

### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in 
which it is rendered?

### Note:
Please ensure that once the PR is submitted, you check travis-ci for build 
issues and submit an update to your PR as soon as possible.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/patricker/nifi NIFI-3540

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/1547.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1547


commit 626deeb460825309efb23c7160245c5af58fef9d
Author: patricker 
Date:   2017-02-28T20:18:36Z

NIFI-3540




> QueryDatabaseTable Failing to Track MS SQL Max Values
> -
>
> Key: NIFI-3540
> URL: https://issues.apache.org/jira/browse/NIFI-3540
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Affects Versions: 1.2.0
>Reporter: Peter Wicks
>Assignee: Peter Wicks
>Priority: Blocker
>
> A recent change was made to allow for EL to be used in QDB and GTF processors 
> for dynamic table name and max value column tracking.
> Part of this change lead to table names being retrieved dynamically from the 
> ResultSetMetadata rather then using the value returned by resolving the EL.  
> In the MS SQL JDBC driver there is a known bug (for the last 10 years...) 
> where sometimes getTableName returns an empty string rather then the 
> table/view name.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[GitHub] nifi pull request #1547: NIFI-3540

[patricker]

GitHub user patricker opened a pull request:

https://github.com/apache/nifi/pull/1547

NIFI-3540

@mattyb149, ran into a bug with the MS SQL JDBC driver where the change you 
made for NIFI-2881/#1407 causes problems.  Small fix. 

### For all changes:
- [x] Is there a JIRA ticket associated with this PR? Is it referenced 
 in the commit message?

- [x] Does your PR title start with NIFI- where  is the JIRA number 
you are trying to resolve? Pay particular attention to the hyphen "-" character.

- [x] Has your PR been rebased against the latest commit within the target 
branch (typically master)?

- [x] Is your initial contribution a single, squashed commit?

### For code changes:
- [ ] Have you ensured that the full suite of tests is executed via mvn 
-Pcontrib-check clean install at the root nifi folder?
- [ ] Have you written or updated unit tests to verify your changes?
- [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)? 
- [ ] If applicable, have you updated the LICENSE file, including the main 
LICENSE file under nifi-assembly?
- [ ] If applicable, have you updated the NOTICE file, including the main 
NOTICE file found under nifi-assembly?
- [ ] If adding new Properties, have you added .displayName in addition to 
.name (programmatic access) for each of the new properties?

### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in 
which it is rendered?

### Note:
Please ensure that once the PR is submitted, you check travis-ci for build 
issues and submit an update to your PR as soon as possible.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/patricker/nifi NIFI-3540

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/nifi/pull/1547.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1547


commit 626deeb460825309efb23c7160245c5af58fef9d
Author: patricker 
Date:   2017-02-28T20:18:36Z

NIFI-3540




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (NIFI-3540) QueryDatabaseTable Failing to Track MS SQL Max Values

[Peter Wicks (JIRA)]

Peter Wicks created NIFI-3540:
-

 Summary: QueryDatabaseTable Failing to Track MS SQL Max Values
 Key: NIFI-3540
 URL: https://issues.apache.org/jira/browse/NIFI-3540
 Project: Apache NiFi
  Issue Type: Bug
  Components: Core Framework
Affects Versions: 1.2.0
Reporter: Peter Wicks
Assignee: Peter Wicks
Priority: Blocker


A recent change was made to allow for EL to be used in QDB and GTF processors 
for dynamic table name and max value column tracking.

Part of this change lead to table names being retrieved dynamically from the 
ResultSetMetadata rather then using the value returned by resolving the EL.  In 
the MS SQL JDBC driver there is a known bug (for the last 10 years...) where 
sometimes getTableName returns an empty string rather then the table/view name.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (NIFI-3366) MoveHDFS Processor

[Peter Wicks (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15888790#comment-15888790
 ] 

Peter Wicks commented on NIFI-3366:
---

[~ggwizdz1], thanks for the contribution. Couple of thoughts, let me know what 
you think/if you'd be willing to do some additional work:

 - Make Input Folder optional, and if not present use an incoming FlowFile, 
either it's value or an attribute on it.
 - Allow for Copying along with Moving. This is a common use case I run into 
where right now I end up Putting to HDFS twice (once for HIVE ingest and once 
for Archive) where a copy would be a great second step.

Copy could be a separate ticket, but it might be nice to have a Copy/Move 
processor all in one.

Thoughts?

> MoveHDFS Processor
> --
>
> Key: NIFI-3366
> URL: https://issues.apache.org/jira/browse/NIFI-3366
> Project: Apache NiFi
>  Issue Type: New Feature
>Reporter: Joseph Niemiec
>Priority: Minor
> Attachments: 0001-MoveHDFS-Processor.patch
>
>
> Today the PutHDFS Processor merely places a file into HDFS from NiFi. There 
> are times when we may want to move Files/Directories around on HDFS as part 
> of a workflow.  This could be after the PutHDFS processor has placed the 
> file, or from some other trigger. 
> Initially we are targeting to take a flow file attribute of an absolute HDFS 
> path, and be able to move it to a target HDFS Path using the Filesystem 
> RENAME API. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (NIFI-3366) MoveHDFS Processor

[Joseph Niemiec (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-3366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joseph Niemiec updated NIFI-3366:
-
Assignee: (was: Joseph Niemiec)
  Status: Patch Available  (was: Open)

> MoveHDFS Processor
> --
>
> Key: NIFI-3366
> URL: https://issues.apache.org/jira/browse/NIFI-3366
> Project: Apache NiFi
>  Issue Type: New Feature
>Reporter: Joseph Niemiec
>Priority: Minor
> Attachments: 0001-MoveHDFS-Processor.patch
>
>
> Today the PutHDFS Processor merely places a file into HDFS from NiFi. There 
> are times when we may want to move Files/Directories around on HDFS as part 
> of a workflow.  This could be after the PutHDFS processor has placed the 
> file, or from some other trigger. 
> Initially we are targeting to take a flow file attribute of an absolute HDFS 
> path, and be able to move it to a target HDFS Path using the Filesystem 
> RENAME API. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (NIFI-3035) URL to display a particular process group in UI

[Andrew Lim (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15888662#comment-15888662
 ] 

Andrew Lim commented on NIFI-3035:
--

Thanks [~scottyaslan].  I filed NIFI-3539 and put it under the Epic you 
suggested.   NIFI-3539 captures adding "deep linking" functionality as a whole. 
 We can break it up as needed when there is bandwidth to work on this new 
feature.

> URL to display a particular process group in UI
> ---
>
> Key: NIFI-3035
> URL: https://issues.apache.org/jira/browse/NIFI-3035
> Project: Apache NiFi
>  Issue Type: New Feature
>  Components: Core UI
>Reporter: Christine Draper
>Assignee: Scott Aslan
>
> Our use case has multiple teams of users working on specific process groups. 
> We would like to be able to give them a URL that will launch the UI on the 
> specific group they are working on, rather than them having to navigate to it 
> from the root group.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (NIFI-3539) Add "deep linking" capability to NiFi

[Andrew Lim (JIRA)]

Andrew Lim created NIFI-3539:


 Summary: Add "deep linking" capability to NiFi
 Key: NIFI-3539
 URL: https://issues.apache.org/jira/browse/NIFI-3539
 Project: Apache NiFi
  Issue Type: New Feature
  Components: Core UI
Reporter: Andrew Lim


Provide exact URLs to specific areas/views of the application.  Examples:

-In the Data provenance window, a URL could show results for a specific search 
with sorting applied
-A URL which highlights a processor, centers it in view and resets the zoom.
-Similarly, a URL which opens the configuration window for a specific processor 
and takes the user to a specific configuration tab



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (NIFI-3538) Add DeleteHBase processor(s)

[Matt Burgess (JIRA)]

Matt Burgess created NIFI-3538:
--

 Summary: Add DeleteHBase processor(s)
 Key: NIFI-3538
 URL: https://issues.apache.org/jira/browse/NIFI-3538
 Project: Apache NiFi
  Issue Type: New Feature
  Components: Extensions
Reporter: Matt Burgess


NiFi currently has processors for storing and retrieving cells/rows in HBase, 
but there is no mechanism for deleting records and/or tables.

I'm not sure if a single DeleteHBase processor could accomplish both, that can 
be discussed under this Jira (and can be split out if necessary).



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (NIFI-3204) delete hdfs processor throws an error stating transfer relationship not specified even when all relationships are present

[JIRA]

[ 
https://issues.apache.org/jira/browse/NIFI-3204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15888519#comment-15888519
 ] 

François Prunier commented on NIFI-3204:


This only happens if the processor handles an incoming flowfile, ie  
!context.hasIncomingConnection() is true.

I have a patch for it but it breaks the semantics of the processor: what do we 
do with the original flowfile ?

> delete hdfs processor throws an error stating transfer relationship not 
> specified even when all relationships are present
> -
>
> Key: NIFI-3204
> URL: https://issues.apache.org/jira/browse/NIFI-3204
> Project: Apache NiFi
>  Issue Type: Bug
>Affects Versions: 1.0.0, 1.1.0
>Reporter: Arpit Gupta
>
> Following flow was setup
> get file -> extract text -> delete hdfs
> A bunch of files were written each having one line which was the path to 
> delete. Some of these path's were files, some were directories and some were 
> patterns. Extract text would extract the line and assign to an attribute 
> which delete hdfs would use to populate the path to delete.
> However the processor would run into an error when ever it tried to process 
> the path which was a pattern matching multiple paths.
> {code}
> 2016-12-14 11:32:43,335 ERROR [Timer-Driven Process Thread-7] 
> o.a.nifi.processors.hadoop.DeleteHDFS 
> DeleteHDFS[id=fed0acf6-0158-1000-b7ab-8cc724e4142d] 
> DeleteHDFS[id=fed0acf6-0158-1000-b7ab-8cc724e4142d] failed to process session 
> due to org.apache.nifi.processor.exception
> .FlowFileHandlingException: 
> StandardFlowFileRecord[uuid=af8be94a-e527-4203-bb87-a0115f84e582,claim=StandardContentClaim
>  [resourceClaim=StandardResourceClaim[id=1481656798897-1, container=default, 
> section=1], offset=6518, length=75],offset=0,name=noyg3p7km8.txt,size=75] tr
> ansfer relationship not specified: 
> org.apache.nifi.processor.exception.FlowFileHandlingException: 
> StandardFlowFileRecord[uuid=af8be94a-e527-4203-bb87-a0115f84e582,claim=StandardContentClaim
>  [resourceClaim=StandardResourceClaim[id=1481656798897-1, container=default, 
> sectio
> n=1], offset=6518, length=75],offset=0,name=noyg3p7km8.txt,size=75] transfer 
> relationship not specified
> 2016-12-14 11:32:43,335 ERROR [Timer-Driven Process Thread-7] 
> o.a.nifi.processors.hadoop.DeleteHDFS
> org.apache.nifi.processor.exception.FlowFileHandlingException: 
> StandardFlowFileRecord[uuid=af8be94a-e527-4203-bb87-a0115f84e582,claim=StandardContentClaim
>  [resourceClaim=StandardResourceClaim[id=1481656798897-1, container=default, 
> section=1], offset=6518, length=75],offse
> t=0,name=noyg3p7km8.txt,size=75] transfer relationship not specified
> at 
> org.apache.nifi.controller.repository.StandardProcessSession.checkpoint(StandardProcessSession.java:234)
>  ~[nifi-framework-core-1.1.0.jar:1.1.0]
> at 
> org.apache.nifi.controller.repository.StandardProcessSession.commit(StandardProcessSession.java:304)
>  ~[nifi-framework-core-1.1.0.jar:1.1.0]
> at 
> org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:28)
>  ~[nifi-api-1.1.0.jar:1.1.0]
> at 
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1099)
>  ~[nifi-framework-core-1.1.0.jar:1.1.0]
> at 
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:136)
>  [nifi-framework-core-1.1.0.jar:1.1.0]
> at 
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)
>  [nifi-framework-core-1.1.0.jar:1.1.0]
> at 
> org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:132)
>  [nifi-framework-core-1.1.0.jar:1.1.0]
> at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
> [na:1.8.0_92]
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) 
> [na:1.8.0_92]
> at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
>  [na:1.8.0_92]
> at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
>  [na:1.8.0_92]
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>  [na:1.8.0_92]
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>  [na:1.8.0_92]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_92]
> 2016-12-14 11:32:43,335 WARN [Timer-Driven Process Thread-7] 
> o.a.nifi.processors.hadoop.DeleteHDFS 
> DeleteHDFS[id=fed0acf6-0158-1000-b7ab-8cc724e4142d] Processor 
> Administratively Yielded for 1 sec due to processing failure
> 

[jira] [Commented] (NIFI-3035) URL to display a particular process group in UI

[Scott Aslan (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15888201#comment-15888201
 ] 

Scott Aslan commented on NIFI-3035:
---

Andrew,

I like this idea but I don't think it will be completed as part of this ticket. 
Please create a new Jira and add it under this epic: 
https://issues.apache.org/jira/browse/NIFI-3396?jql=project%20%3D%20NIFI%20AND%20issuetype%20%3D%20Epic%20AND%20status%20in%20(Open%2C%20Reopened)%20AND%20component%20%3D%20%22Core%20UI%22

> URL to display a particular process group in UI
> ---
>
> Key: NIFI-3035
> URL: https://issues.apache.org/jira/browse/NIFI-3035
> Project: Apache NiFi
>  Issue Type: New Feature
>  Components: Core UI
>Reporter: Christine Draper
>Assignee: Scott Aslan
>
> Our use case has multiple teams of users working on specific process groups. 
> We would like to be able to give them a URL that will launch the UI on the 
> specific group they are working on, rather than them having to navigate to it 
> from the root group.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Assigned] (NIFI-3035) URL to display a particular process group in UI

[Scott Aslan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-3035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Scott Aslan reassigned NIFI-3035:
-

Assignee: Scott Aslan  (was: Matt Gilman)

> URL to display a particular process group in UI
> ---
>
> Key: NIFI-3035
> URL: https://issues.apache.org/jira/browse/NIFI-3035
> Project: Apache NiFi
>  Issue Type: New Feature
>  Components: Core UI
>Reporter: Christine Draper
>Assignee: Scott Aslan
>
> Our use case has multiple teams of users working on specific process groups. 
> We would like to be able to give them a URL that will launch the UI on the 
> specific group they are working on, rather than them having to navigate to it 
> from the root group.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (NIFI-3439) Making password property in HiveConnectionPool a hidden show.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3439?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15888139#comment-15888139
 ] 

ASF GitHub Bot commented on NIFI-3439:
--

Github user mcgilman commented on the issue:

https://github.com/apache/nifi/pull/1546
  
Will review...


> Making password property in HiveConnectionPool a hidden show.
> -
>
> Key: NIFI-3439
> URL: https://issues.apache.org/jira/browse/NIFI-3439
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core UI
>Affects Versions: 1.1.1
> Environment: Google Chrome browser
>Reporter: Ramakrishnan V
>Assignee: Scott Aslan
>Priority: Minor
>  Labels: ControllerService, Hive, security
>   Original Estimate: 168h
>  Remaining Estimate: 168h
>
> When user types the password in the password attribute in hive connection 
> pool controller service, the characters are visible. Once completed the 
> password is encrypted and no longer visible.
> shouldn't be me made as normal hidden show password box ?
> The user always has the option to delete the controller service and 
> re-configure if he has entered a wrong password.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (NIFI-3038) Update layout of Cluster Summary window

[Matt Gilman (JIRA)]

 [ 
https://issues.apache.org/jira/browse/NIFI-3038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matt Gilman updated NIFI-3038:
--
   Resolution: Fixed
Fix Version/s: 1.2.0
   Status: Resolved  (was: Patch Available)

> Update layout of Cluster  Summary window
> ---
>
> Key: NIFI-3038
> URL: https://issues.apache.org/jira/browse/NIFI-3038
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Core UI
>Affects Versions: 1.0.0
>Reporter: Rob Moran
>Assignee: Scott Aslan
>Priority: Minor
> Fix For: 1.2.0
>
>
> To make the Cluster  Summary window more consistent with other 
> areas of the application, move the selected component details to align left 
> and move the refresh action and timestamp to the bottom left (below the 
> table).



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (NIFI-3038) Update layout of Cluster Summary window

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15888134#comment-15888134
 ] 

ASF subversion and git services commented on NIFI-3038:
---

Commit c3dbbb845fd0cb9079a3a1652930976e098fa2b6 in nifi's branch 
refs/heads/master from [~scottyaslan]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=c3dbbb8 ]

[NIFI-3038] Update cluster summary dialog layout. This closes #1534


> Update layout of Cluster  Summary window
> ---
>
> Key: NIFI-3038
> URL: https://issues.apache.org/jira/browse/NIFI-3038
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Core UI
>Affects Versions: 1.0.0
>Reporter: Rob Moran
>Assignee: Scott Aslan
>Priority: Minor
>
> To make the Cluster  Summary window more consistent with other 
> areas of the application, move the selected component details to align left 
> and move the refresh action and timestamp to the bottom left (below the 
> table).



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (NIFI-3038) Update layout of Cluster Summary window

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15888135#comment-15888135
 ] 

ASF GitHub Bot commented on NIFI-3038:
--

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/1534


> Update layout of Cluster  Summary window
> ---
>
> Key: NIFI-3038
> URL: https://issues.apache.org/jira/browse/NIFI-3038
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Core UI
>Affects Versions: 1.0.0
>Reporter: Rob Moran
>Assignee: Scott Aslan
>Priority: Minor
>
> To make the Cluster  Summary window more consistent with other 
> areas of the application, move the selected component details to align left 
> and move the refresh action and timestamp to the bottom left (below the 
> table).



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[GitHub] nifi issue #1546: [NIFI-3439] when users type into sensitive property fields...

[mcgilman]

Github user mcgilman commented on the issue:

https://github.com/apache/nifi/pull/1546
  
Will review...


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-3038) Update layout of Cluster Summary window

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-3038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15888136#comment-15888136
 ] 

ASF GitHub Bot commented on NIFI-3038:
--

Github user mcgilman commented on the issue:

https://github.com/apache/nifi/pull/1534
  
Looks good @scottyaslan. This has been merged to master.


> Update layout of Cluster  Summary window
> ---
>
> Key: NIFI-3038
> URL: https://issues.apache.org/jira/browse/NIFI-3038
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Core UI
>Affects Versions: 1.0.0
>Reporter: Rob Moran
>Assignee: Scott Aslan
>Priority: Minor
>
> To make the Cluster  Summary window more consistent with other 
> areas of the application, move the selected component details to align left 
> and move the refresh action and timestamp to the bottom left (below the 
> table).



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[GitHub] nifi issue #1534: [NIFI-3038] Update cluster summary dialog layout

[mcgilman]

Github user mcgilman commented on the issue:

https://github.com/apache/nifi/pull/1534
  
Looks good @scottyaslan. This has been merged to master.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] nifi pull request #1534: [NIFI-3038] Update cluster summary dialog layout

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/1534


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (NIFI-2923) Add expression language support to Kerberos parameters used by processors

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-2923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15887661#comment-15887661
 ] 

ASF GitHub Bot commented on NIFI-2923:
--

Github user mauriziocolleluori commented on the issue:

https://github.com/apache/nifi/pull/1148
  
@trixpan @pvillard31 thank you for you comments. I will look into the Kafka 
processors and revert back once it's done.


> Add expression language support to Kerberos parameters used by processors
> -
>
> Key: NIFI-2923
> URL: https://issues.apache.org/jira/browse/NIFI-2923
> Project: Apache NiFi
>  Issue Type: Improvement
>Reporter: Maurizio Colleluori
>Priority: Minor
>
> Kerberos properties (e.g. principal, keytab) available as attributes in 
> certain processors (e.g. HDFS processors) only accept a constant value. 
> Support for expression language could be enabled and allow for 
> parameterisation.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[GitHub] nifi issue #1148: NIFI-2923 Add expression language support to Kerberos para...

[mauriziocolleluori]

Github user mauriziocolleluori commented on the issue:

https://github.com/apache/nifi/pull/1148
  
@trixpan @pvillard31 thank you for you comments. I will look into the Kafka 
processors and revert back once it's done.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---