[GitHub] nifi issue #2016: NIFI-4196 - Expose AWS proxy authentication settings
Github user trixpan commented on the issue: https://github.com/apache/nifi/pull/2016 @pvillard31 will rebase and resubmit... ---
[GitHub] nifi issue #2017: NIFI-4197 - Expose some proxy settings to GCS Processors
Github user trixpan commented on the issue: https://github.com/apache/nifi/pull/2017 @jvwing Apologies for the delay I believe the feedback has been addressed. ---
[GitHub] nifi issue #2348: NIFI-4706 - Bump ParCEFone version
Github user trixpan commented on the issue: https://github.com/apache/nifi/pull/2348 @joewitt my bad. I forked from an old repo. 1.2.6 should be online. ---
[GitHub] nifi pull request #2348: NIFI-4706 - Bump ParCEFone version
GitHub user trixpan opened a pull request: https://github.com/apache/nifi/pull/2348 NIFI-4706 - Bump ParCEFone version Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [X] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [X] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [X] Has your PR been rebased against the latest commit within the target branch (typically master)? - [X] Is your initial contribution a single, squashed commit? ### For code changes: - [X] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [X] Have you written or updated unit tests to verify your changes? - [X] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - ~~~[] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly~~~? - ~~~[ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly~~~? - ~~~[ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties~~~? ### For documentation related changes: - ~~~[ ] Have you ensured that format looks appropriate for the output in which it is rendered~~~? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/trixpan/nifi NIFI-4706 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2348.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2348 commit 3c86375dc26c419955b6b9cab1360440531f75f9 Author: trixpan <trix...@users.noreply.github.com> Date: 2017-12-18T13:08:03Z NIFI-4706 - Bump ParCEFone version ---
[GitHub] nifi issue #1968: NIFI-3931 - Added EL to properties in SFTP transfer
Github user trixpan commented on the issue: https://github.com/apache/nifi/pull/1968 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #2022: NIFI-4200 - Initial commit for a ControlNiFi processor
Github user trixpan commented on the issue: https://github.com/apache/nifi/pull/2022 @pvillard31 - I assume the processor should be able to control nifi components only. Should we rename it to make this explicit? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #2016: NIFI-4196 - Expose AWS proxy authentication settings
Github user trixpan commented on the issue: https://github.com/apache/nifi/pull/2016 hopefully fixed --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #2018: NIFI-4175 - Add HTTP proxy support to *SFTP process...
Github user trixpan commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2018#discussion_r128152991
--- Diff:
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/util/SFTPTransfer.java
---
@@ -92,6 +93,39 @@
.defaultValue("true")
.required(true)
.build();
+public static final PropertyDescriptor PROXY_HOST = new
PropertyDescriptor.Builder()
--- End diff --
@pvillard31 I saw that and considered it but ended up adopting the current
approach due to my view that I prefer not supporting SOCKS ... Don't get me
wrong, I appreciate the reason why SOCKS exists but most processors are only
able to support HTTP so I rather provide... consistency (I know I have been
sounding repetitive lately... ð )
Since FTP already supports SOCKS I reckon we can leave it there, but for
SFTP I rather not add it at all.
Let me know if you agree and I will adjust accordingly.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi issue #1696: NIFI-1655 - Add .gitattributes to specifically define
Github user trixpan commented on the issue: https://github.com/apache/nifi/pull/1696 @jfrazee I resolved conflict with master (unrelated to the change). I would agree that the "text" qualifier was indeed missing. Fixed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1644: NIFI-11 - Capture StringIndexOutOfBoundsException to preve...
Github user trixpan commented on the issue: https://github.com/apache/nifi/pull/1644 @mcgilman - Sorry for the delay. rebased and hopefully addressed the jUnit issue --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #2018: NIFI-4175 - Add HTTP proxy support to *SFTP process...
GitHub user trixpan opened a pull request: https://github.com/apache/nifi/pull/2018 NIFI-4175 - Add HTTP proxy support to *SFTP processors Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [X] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [X] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [X] Has your PR been rebased against the latest commit within the target branch (typically master)? - [X] Is your initial contribution a single, squashed commit? ### For code changes: - [X] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [] Have you written or updated unit tests to verify your changes? - [X] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [X] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [X] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [X] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [X] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/trixpan/nifi NIFI-4175 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2018.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2018 commit fc93aefd3351c939b06e83db77c89ce0957d025f Author: Andre F de Miranda <trix...@users.noreply.github.com> Date: 2017-07-18T11:04:48Z NIFI-4175 - Add HTTP proxy support to *SFTP processors --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #2017: NIFI-4197 - Expose some proxy settings to GCS Proce...
GitHub user trixpan opened a pull request: https://github.com/apache/nifi/pull/2017 NIFI-4197 - Expose some proxy settings to GCS Processors Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [X] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [X] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [X] Has your PR been rebased against the latest commit within the target branch (typically master)? - [X] Is your initial contribution a single, squashed commit? ### For code changes: - [X] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [] Have you written or updated unit tests to verify your changes? - [X] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [X] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [X] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [X] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [X] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/trixpan/nifi NIFI-4197 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2017.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2017 commit 368e9feee54948ca6c9676b4bd87ea5c97cc3d36 Author: Andre F de Miranda <trix...@users.noreply.github.com> Date: 2017-07-18T10:07:13Z NIFI-4197 - Expose some proxy settings to GCS Processors --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #2016: NIFI-4196 - Expose AWS proxy authentication settings
Github user trixpan commented on the issue: https://github.com/apache/nifi/pull/2016 Please note the displayName settings were not added as most of the AWS related processors lack these --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #2016: NIFI-4196 - Expose AWS proxy authentication setting...
GitHub user trixpan opened a pull request: https://github.com/apache/nifi/pull/2016 NIFI-4196 - Expose AWS proxy authentication settings Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [X] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [X] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [X] Has your PR been rebased against the latest commit within the target branch (typically master)? - [X] Is your initial contribution a single, squashed commit? ### For code changes: - [X] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [X] Have you written or updated unit tests to verify your changes? - [X] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [X] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [X] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [X] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [X] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/trixpan/nifi NIFI-4196 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2016.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2016 commit 15707bc2a912b385bd584998c83f9d5250746134 Author: Andre F de Miranda <trix...@users.noreply.github.com> Date: 2017-07-18T08:41:22Z NIFI-4196 - Expose AWS proxy authentication settings --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #2013: NIFI-4098 - Adjust ParseCEF to re-use bean validato...
GitHub user trixpan opened a pull request: https://github.com/apache/nifi/pull/2013 NIFI-4098 - Adjust ParseCEF to re-use bean validator in order to impr⦠â¦ove performance Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [X] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [X] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [X] Has your PR been rebased against the latest commit within the target branch (typically master)? - [X] Is your initial contribution a single, squashed commit? ### For code changes: - [X] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [X] Have you written or updated unit tests to verify your changes? - [X] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [X] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [X] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [X] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [X] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/trixpan/nifi NIFI-4098 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2013.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2013 commit 39236a9cd73b3164d73cfa2ce25c897a756dc966 Author: Andre F de Miranda <trix...@users.noreply.github.com> Date: 2017-06-21T05:08:08Z NIFI-4098 - Adjust ParseCEF to re-use bean validator in order to improve performance --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1921: NIFI-4081 - Added raw message option in GrokReader
Github user trixpan commented on the issue: https://github.com/apache/nifi/pull/1921 @pvillard31 I accept it is a bit of bit picking but is there any chance we call it _raw? IIRC this is the name we used on ParseCEF. Naming conventions make user experience smoother ð --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1696: NIFI-1655 - Add .gitattributes to specifically define
Github user trixpan commented on the issue: https://github.com/apache/nifi/pull/1696 @jfrazee? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #1950: NIFI-4126 - Add SupportBatching, SideEffectFree and...
GitHub user trixpan opened a pull request: https://github.com/apache/nifi/pull/1950 NIFI-4126 - Add SupportBatching, SideEffectFree and EventDriven annot⦠â¦ations to ExtractGrok Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [X] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [X] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [X] Has your PR been rebased against the latest commit within the target branch (typically master)? - [X] Is your initial contribution a single, squashed commit? ### For code changes: - [X] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [X] Have you written or updated unit tests to verify your changes? - [X] ~~~If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)?~~~ - [X] ~~~If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly?~~~ - [X] ~~~If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly?~~~ - [X] ~~~If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties?~~~ ### For documentation related changes: - [X] ~~~Have you ensured that format looks appropriate for the output in which it is rendered?~~~ ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/trixpan/nifi NIFI-4126 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/1950.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1950 commit ed53d88096a48c3e633ebcc8b43b3c5c389850b9 Author: Andre F de Miranda <trix...@users.noreply.github.com> Date: 2017-06-27T01:49:29Z NIFI-4126 - Add SupportBatching, SideEffectFree and EventDriven annotations to ExtractGrok --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1644: NIFI-11 - Capture StringIndexOutOfBoundsException to preve...
Github user trixpan commented on the issue: https://github.com/apache/nifi/pull/1644 @mcgilman indeed. Hopefully the last commit should address the remaining ones. Cheers --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1644: NIFI-11 - Capture StringIndexOutOfBoundsException to preve...
Github user trixpan commented on the issue: https://github.com/apache/nifi/pull/1644 @mcgilman Hopefully I managed to address your feedback in this last iteration --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1794: NIFI-1709 - Introduce logic to probe Linux version using /...
Github user trixpan commented on the issue: https://github.com/apache/nifi/pull/1794 @apiri @jfrazee - should I go ahead and merge this? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1623: NIFI-55 - Log IP of clients generating expired holds and o...
Github user trixpan commented on the issue: https://github.com/apache/nifi/pull/1623 @joewitt any chance of having a look on this PR? Cheers --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1541: NIFI-329 - Introduce IRC Client Services and ConsumeIRC pr...
Github user trixpan commented on the issue: https://github.com/apache/nifi/pull/1541 @joewitt feedback should be addressed. @trkurc - special thanks for the review and PR. I have been having been able to commit enough resources to the project lately and your PR certainly helped. Truly appreciated ! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #1541: NIFI-329 - Introduce IRC Client Services and Consum...
Github user trixpan commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1541#discussion_r119568598
--- Diff:
nifi-nar-bundles/nifi-irc-bundle/nifi-irc-client-service/src/main/java/org/apache/nifi/irc/StandardIRCClientService.java
---
@@ -0,0 +1,233 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.irc;
+
+import java.io.File;
+import java.io.FileInputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.security.KeyStore;
+import java.security.KeyStoreException;
+import java.security.NoSuchAlgorithmException;
+import java.security.Provider;
+import java.security.cert.CertificateException;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.concurrent.atomic.AtomicInteger;
+
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnDisabled;
+import org.apache.nifi.annotation.lifecycle.OnEnabled;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.controller.AbstractControllerService;
+import org.apache.nifi.controller.ConfigurationContext;
+import org.apache.nifi.irc.handlers.ServiceEventHandler;
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.reporting.InitializationException;
+import org.apache.nifi.ssl.SSLContextService;
+import org.apache.nifi.security.util.KeyStoreUtils;
+import org.apache.nifi.util.StringUtils;
+import org.kitteh.irc.client.library.Client;
+import org.kitteh.irc.client.library.feature.sending.SingleDelaySender;
+
+
+import javax.net.ssl.SSLContext;
+import javax.net.ssl.TrustManagerFactory;
+
+@Tags({ "irc"})
+@CapabilityDescription("IRC client controller.")
+public class StandardIRCClientService extends AbstractControllerService
implements IRCClientService {
+
+public static final PropertyDescriptor IRC_SERVER = new
PropertyDescriptor
+.Builder().name("IRC_SERVER")
+.displayName("IRC Server")
+.description("The IRC server you want to connect to")
+.required(true)
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.build();
+public static final PropertyDescriptor IRC_SERVER_PORT = new
PropertyDescriptor
+.Builder().name("IRC_SERVER_PORT")
+.displayName("IRC Server Port")
+.description("The IRC server port you want to connect to")
+.required(true)
+.defaultValue("6667")
+.addValidator(StandardValidators.PORT_VALIDATOR)
+.build();
+public static final PropertyDescriptor SSL_CONTEXT_SERVICE = new
PropertyDescriptor.Builder()
+.name("SSL_CONTEXT_SERVICE")
+.displayName("SSL Context Service")
+.description("The Controller Service to use in order to obtain
an SSL Context. If this property is set, "
++ "IRC connection will be established over a secure
connection.")
+.required(false)
+.identifiesControllerService(SSLContextService.class)
+.build();
+public static final PropertyDescriptor IRC_NICK = new
PropertyDescriptor
+.Builder().name("IRC_NICK")
+.displayName("Nickname")
+.description("The Nickname to use when connecting to the IRC
server")
+.required(true)
[GitHub] nifi pull request #1541: NIFI-329 - Introduce IRC Client Services and Consum...
Github user trixpan commented on a diff in the pull request: https://github.com/apache/nifi/pull/1541#discussion_r119568494 --- Diff: nifi-nar-bundles/nifi-irc-bundle/nifi-irc-processors-nar/pom.xml --- @@ -0,0 +1,41 @@ + + +http://maven.apache.org/POM/4.0.0; xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance; xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd;> +4.0.0 + + +org.apache.nifi +nifi-irc-bundle +1.3.0-SNAPSHOT + + +nifi-irc-processors-nar +1.3.0-SNAPSHOT +nar + +true +true + + + + +org.apache.nifi +nifi-irc-processors +1.3.0-SNAPSHOT --- End diff -- seen. makes sense --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #1541: NIFI-329 - Introduce IRC Client Services and Consum...
Github user trixpan commented on a diff in the pull request: https://github.com/apache/nifi/pull/1541#discussion_r119568348 --- Diff: nifi-assembly/NOTICE --- @@ -1364,6 +1364,8 @@ The following binary components are provided under the MIT License. See project (MIT License) EWS Java API (com.microsoft.ews-java-api:ews-java-api:2.0 - https://github.com/OfficeDev/ews-java-api) (MIT License) libffi (libffi-3.2.1 - http://sourceware.org/libffi/) + (MIT License) Kitteh IRC Client Library (kitteh client-lib 3.0.3 - http://kicl.kitteh.org/) + (MIT License) 'mbassador' (mbassador 1.2.4 - https://github.com/bennidi/mbassador) --- End diff -- addressed --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #1875: NIFI-4001 - Bump ParCEFone version
GitHub user trixpan opened a pull request: https://github.com/apache/nifi/pull/1875 NIFI-4001 - Bump ParCEFone version Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [X] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [X] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [X] Has your PR been rebased against the latest commit within the target branch (typically master)? - [X] Is your initial contribution a single, squashed commit? ### For code changes: - [X] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [X] ~~~Have you written or updated unit tests to verify your changes?~~~ - [X] ~~~If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)?~~~ - [X] ~~~If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly?~~~ - [X] ~~~If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly?~~~ - [X] ~~~If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties?~~~ ### For documentation related changes: - [X] ~~~Have you ensured that format looks appropriate for the output in which it is rendered?~~~ ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/trixpan/nifi NIFI-4001 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/1875.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1875 commit 286e226b477e00febda0dceacdaa03a0cf3abebf Author: Andre F de Miranda <trix...@users.noreply.github.com> Date: 2017-06-01T08:46:18Z NIFI-4001 - Bump ParCEFone version --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1696: NIFI-1655 - Add .gitattributes to specifically define
Github user trixpan commented on the issue: https://github.com/apache/nifi/pull/1696 @jfrazee this seems to be passing tests now. Let me know what you think --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #1541: NIFI-329 - Introduce IRC Client Services and Consum...
Github user trixpan commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1541#discussion_r119273679
--- Diff:
nifi-nar-bundles/nifi-irc-bundle/nifi-irc-processors/src/main/java/org/apache/nifi/processors/irc/PublishIRC.java
---
@@ -0,0 +1,139 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.irc;
+
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.TriggerSerially;
+import org.apache.nifi.annotation.behavior.WritesAttribute;
+import org.apache.nifi.annotation.behavior.WritesAttributes;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnStopped;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessSessionFactory;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.io.InputStreamCallback;
+import org.apache.nifi.processors.irc.handlers.PublisherEventHandler;
+import org.apache.nifi.stream.io.StreamUtils;
+import org.apache.nifi.util.StopWatch;
+
+import java.io.ByteArrayOutputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.util.concurrent.TimeUnit;
+
+@Tags({"publish", "irc"})
+@TriggerSerially
+@InputRequirement(InputRequirement.Requirement.INPUT_FORBIDDEN)
--- End diff --
ouch. yes
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi issue #1696: NIFI-1655 - Add .gitattributes to specifically define
Github user trixpan commented on the issue: https://github.com/apache/nifi/pull/1696 @jfrazee now that HL7 is out of the way, giving it another try... Will let you know when I am happy with code. Cheers --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1845: NIFI-3960 - ExtractGrok - Include exception message when c...
Github user trixpan commented on the issue: https://github.com/apache/nifi/pull/1845 thanks for the contribution LGTM. merging --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1794: NIFI-1709 - Introduce logic to probe Linux version using /...
Github user trixpan commented on the issue: https://github.com/apache/nifi/pull/1794 @jfrazee I intentionally used ID as I noticed ID_LIKE seemed like a mix bag... https://gist.github.com/djcp/6132025 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1794: NIFI-1709 - Introduce logic to probe Linux version using /...
Github user trixpan commented on the issue: https://github.com/apache/nifi/pull/1794 @jfrazee sorry. I misread your comment. It works on leap?! Great. :-) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1794: NIFI-1709 - Introduce logic to probe Linux version using /...
Github user trixpan commented on the issue: https://github.com/apache/nifi/pull/1794 @jfrazee thanks. I haven't tested on leap. as not working you mean the file is not being linked or the service isn't being started? Cheers --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1799: NIFI-3896 - Makes DeprecationNotice more intuitive
Github user trixpan commented on the issue: https://github.com/apache/nifi/pull/1799 @mcgilman here it goes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #1799: NIFI-3896 - Makes DeprecationNotice more intuitive
GitHub user trixpan opened a pull request: https://github.com/apache/nifi/pull/1799 NIFI-3896 - Makes DeprecationNotice more intuitive Update developers guide on how to deprecate a component Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [X] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [X] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [X] Has your PR been rebased against the latest commit within the target branch (typically master)? - [X] Is your initial contribution a single, squashed commit? ### For code changes: - [X] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [X] Have you written or updated unit tests to verify your changes? - [X] ~~~If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)?~~~ - [X] ~~~If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly?~~~ - [X] ~~~If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly?~~~ - [X] ~~~If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties?~~~ ### For documentation related changes: - [X] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/trixpan/nifi NIFI-3896 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/1799.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1799 commit b628f8c1d24e437bd1e4c597969e2bfa766ecb61 Author: Andre F de Miranda <trix...@users.noreply.github.com> Date: 2017-05-15T14:30:02Z NIFI-3896 - Makes DeprecationNotice more intuitive Update developers guide on how to deprecate a component --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1541: NIFI-329 - Introduce IRC Client Services and ConsumeIRC pr...
Github user trixpan commented on the issue: https://github.com/apache/nifi/pull/1541 Good news! work has been "completed" and PR squashed and rebased to master. Things to notice: 1. At this stage, test units are marked as ignore as they expect the availability of a live IRC server on localhost for testing. Keen to hear feedback on preferred approach for test units. 2. TLS related code is not as clean as the traditional NiFi processor code but seems to work. Given I am not used to use TLS related APIs directly, would appreciate if a more experienced coder could do a sanity check. Looking forward for your feedback --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1741: NIFI-3536 Adding GetSOAP processor which supports generica...
Github user trixpan commented on the issue: https://github.com/apache/nifi/pull/1741 @apsaltis Is it my impression or the new dependencies weren't added to the nifi-assembly `NOTICE` ? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1583: NIFI-1131 renamed relationships
Github user trixpan commented on the issue: https://github.com/apache/nifi/pull/1583 @josephxsxn in light of Joe's comments, would you like to close this PR? Cheers! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1794: NIFI-1709 - Introduce logic to probe Linux version using /...
Github user trixpan commented on the issue: https://github.com/apache/nifi/pull/1794 @apiri may I ask you to have a look on this? Idea here is to use `/etc/os-release` to identify the Linux distribution and use that to flag some particular distro specific issues such as use of `systemd` vs `upstart` vs `openrc` vs whatever and adjust whenever needed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #1794: NIFI-1709 - Introduce logic to probe Linux version ...
GitHub user trixpan opened a pull request: https://github.com/apache/nifi/pull/1794 NIFI-1709 - Introduce logic to probe Linux version using /etc/os-rele⦠â¦ase to nifi.sh Add explicit paths to support SLES 11 SP4 / OpenSUSE init.d layout Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [X] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [X] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [X] Has your PR been rebased against the latest commit within the target branch (typically master)? - [X] Is your initial contribution a single, squashed commit? ### For code changes: - [X] ~~~Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder?~~~ - [X] ~~~Have you written or updated unit tests to verify your changes?~~~ - [X] ~~~If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)?~~~ - [X] ~~~If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly?~~~ - [X] ~~~If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly?~~~ - [X] ~~~If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties?~~~ ### For documentation related changes: - [X] ~~~Have you ensured that format looks appropriate for the output in which it is rendered?~~~ ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/trixpan/nifi NIFI-1709 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/1794.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1794 commit 4be08841dcd1e315c8fbec9ac81377155366f75f Author: Andre F de Miranda <trix...@users.noreply.github.com> Date: 2017-05-14T12:14:33Z NIFI-1709 - Introduce logic to probe Linux version using /etc/os-release to nifi.sh Add explicit paths to support SLES 11 SP4 / OpenSUSE init.d layout --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #1718: NIFI-391 - Introduce the ability to deprecate a com...
Github user trixpan commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1718#discussion_r116357827
--- Diff:
nifi-api/src/main/java/org/apache/nifi/annotation/documentation/DeprecationNotice.java
---
@@ -0,0 +1,51 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.annotation.documentation;
+
+import org.apache.nifi.components.ConfigurableComponent;
+
+import java.lang.annotation.Documented;
+import java.lang.annotation.ElementType;
+import java.lang.annotation.Inherited;
+import java.lang.annotation.Retention;
+import java.lang.annotation.RetentionPolicy;
+import java.lang.annotation.Target;
+
+/**
+ * Annotation that can be applied to a {@link
org.apache.nifi.processor.Processor Processor},
+ * {@link org.apache.nifi.controller.ControllerService ControllerService},
or
+ * {@link org.apache.nifi.reporting.ReportingTask ReportingTask} in order
to
+ * warn about the deprecation of the component. The deprecation warning is
informational only
+ * and doesn't affect the processor run time behavior in any way
+ */
+@Documented
+@Target({ElementType.TYPE})
+@Retention(RetentionPolicy.RUNTIME)
+@Inherited
+public @interface DeprecationNotice {
+Class[] value() default {};
+
+String[] classNames() default {};
+
+String reason() default "";
+
+
+// The following are still not utilised but implemented
+// for future use.
+String deprecationTargetVersion() default "";
--- End diff --
@mcgilman yeah. happy for you to remove
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request #1718: NIFI-391 - Introduce the ability to deprecate a com...
Github user trixpan commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1718#discussion_r116357266
--- Diff:
nifi-api/src/main/java/org/apache/nifi/annotation/documentation/DeprecationNotice.java
---
@@ -0,0 +1,51 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.annotation.documentation;
+
+import org.apache.nifi.components.ConfigurableComponent;
+
+import java.lang.annotation.Documented;
+import java.lang.annotation.ElementType;
+import java.lang.annotation.Inherited;
+import java.lang.annotation.Retention;
+import java.lang.annotation.RetentionPolicy;
+import java.lang.annotation.Target;
+
+/**
+ * Annotation that can be applied to a {@link
org.apache.nifi.processor.Processor Processor},
+ * {@link org.apache.nifi.controller.ControllerService ControllerService},
or
+ * {@link org.apache.nifi.reporting.ReportingTask ReportingTask} in order
to
+ * warn about the deprecation of the component. The deprecation warning is
informational only
+ * and doesn't affect the processor run time behavior in any way
+ */
+@Documented
+@Target({ElementType.TYPE})
+@Retention(RetentionPolicy.RUNTIME)
+@Inherited
+public @interface DeprecationNotice {
+Class[] value() default {};
+
+String[] classNames() default {};
+
+String reason() default "";
+
+
+// The following are still not utilised but implemented
+// for future use.
+String deprecationTargetVersion() default "";
--- End diff --
@mcgilman - I placed it in there to address a commend from the list (I
think it was Andy's suggestion to have it) but given I think it will take a
while before we can truly agree on deprecating something by version X, it may
be reasonable to remove it for now.
From the other hand, unless you foresee this going into the UI we may well
adjust the documentation generation logic to take this field in consideration.
Happy with whatever you prefer.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi issue #1767: NIFI-3802 Adding ASF HipChat information to README
Github user trixpan commented on the issue: https://github.com/apache/nifi/pull/1767 merging --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1718: NIFI-391 - Introduce the ability to deprecate a component
Github user trixpan commented on the issue: https://github.com/apache/nifi/pull/1718 @mcgilman I suspect when you wrote `DocumentTypeDTO` you meant `DocumentedTypeDTO`? If that is the case, hopefully this address the comment --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1718: NIFI-391 - Introduce the ability to deprecate a component
Github user trixpan commented on the issue: https://github.com/apache/nifi/pull/1718 @mcgilman - will adjust --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1718: NIFI-391 - Introduce the ability to deprecate a component
Github user trixpan commented on the issue: https://github.com/apache/nifi/pull/1718 @mcgilman Thank you for looking at this. Yes, I would say as long this part is ok we could merge. I also would prefer having the UI work done separately. Cheers --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1718: NIFI-391 - Introduce the ability to deprecate a component
Github user trixpan commented on the issue: https://github.com/apache/nifi/pull/1718 @mcgilman based on the following screenshot( this is a snipped of data related to a locally deprecated `ListenLumberjack`) I suspect this should be a good starting point for the UX work (I've left anything but documentation untouched).  Please let me know if I missed something. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1746: NIFI-3794 - Expose the control of ListenRELP's CLIENT_AUTH...
Github user trixpan commented on the issue: https://github.com/apache/nifi/pull/1746 adjusted the commit to refer to to the correct JIRA ticket (NIFI-3794) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #1746: NIFI-3726 - Expose the control of ListenRELP's CLIE...
GitHub user trixpan opened a pull request: https://github.com/apache/nifi/pull/1746 NIFI-3726 - Expose the control of ListenRELP's CLIENT_AUTH property t⦠â¦o DFM Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [X] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [X] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [X] Has your PR been rebased against the latest commit within the target branch (typically master)? - [X] Is your initial contribution a single, squashed commit? ### For code changes: - [X] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [X] Have you written or updated unit tests to verify your changes? - [X] ~~~If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)?~~~ - [X] ~~~If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly?~~~ - [X] ~~~If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly?~~~ - [X] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [X] ~~~Have you ensured that format looks appropriate for the output in which it is rendered?~~~ ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/trixpan/nifi NIFI-3794 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/1746.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1746 commit a82f426b7827c4a2620a87543337ae6761b0602d Author: Andre F de Miranda <trix...@users.noreply.github.com> Date: 2017-05-03T22:00:14Z NIFI-3726 - Expose the control of ListenRELP's CLIENT_AUTH property to DFM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1738: NIFI-3718: Fixed timezone issue in TestAvroReaderWithEmbed...
Github user trixpan commented on the issue: https://github.com/apache/nifi/pull/1738 Oops. GMT+10. Forgot my daylight savings is over. :smile: --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1738: NIFI-3718: Fixed timezone issue in TestAvroReaderWithEmbed...
Github user trixpan commented on the issue: https://github.com/apache/nifi/pull/1738 The people from the future salute you, oh fellow space traveler! LGTM. And works on GMT+11. Suggest waiting for travis to complete to ensure it runs on other Time zones (I assume you are behind GMT) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1692: NIFI-3726 - Introduces CompareFuzzyHash processor
Github user trixpan commented on the issue: https://github.com/apache/nifi/pull/1692 @mattyb149 just to register: TLSH requires more than 512 characters to work so your hello message triggers a safety mechanism that routes the flow to error. I should have used error but I think I changed this to info during the peer review Should be fixed now  --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1692: NIFI-3726 - Introduces CompareFuzzyHash processor
Github user trixpan commented on the issue: https://github.com/apache/nifi/pull/1692 pom has also been fixed --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1692: NIFI-3726 - Introduces CompareFuzzyHash processor
Github user trixpan commented on the issue: https://github.com/apache/nifi/pull/1692 @alopresto @mattyb149 last two commits should address the two residual issues you identified. One was a simple reference to HashContent.class instead of the fully qualified class path as string. The other one was the use of .info in message too short when it should be error. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1692: NIFI-3726 - Introduces CompareFuzzyHash processor
Github user trixpan commented on the issue: https://github.com/apache/nifi/pull/1692 Andy, easy to solve --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1696: NIFI-1655 - Add .gitattributes to specifically define
Github user trixpan commented on the issue: https://github.com/apache/nifi/pull/1696 @jfrazee while we are doing HL7 is there any chance you can look at the EncryptContent tests as well? Due to the nature of PGP ASCII Armor files they use a mix of DOS and Unix line format as well. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #1729: NIFI-3769 - Document detail value of 'nifi.provenan...
Github user trixpan closed the pull request at: https://github.com/apache/nifi/pull/1729 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1729: NIFI-3769 - Document detail value of 'nifi.provenance.repo...
Github user trixpan commented on the issue: https://github.com/apache/nifi/pull/1729 closing PR as it seems @markap14 has merged it to master but forgot to refer to PR when siging off the commit --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1339: NIFI-1231: cloned files not getting new entryDate
Github user trixpan commented on the issue: https://github.com/apache/nifi/pull/1339 @lexvanderstoep Given @joewitt 's JIRA comments, would you mind closing this PR and update the JIRA as Not a bug? Kind regards --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #399: NIFI-1833 - Adding Azure Storage processors
Github user trixpan commented on the issue: https://github.com/apache/nifi/pull/399 @simonellistonball - Seems like your PR has been adopted by @jtstorck and @aperepel so closing this PR as we currently have 3 PRs for the same JIRA. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #1729: NIFI-3769 - Document detail value of 'nifi.provenan...
GitHub user trixpan opened a pull request: https://github.com/apache/nifi/pull/1729 NIFI-3769 - Document detail value of 'nifi.provenance.repository.warm⦠â¦.cache.frequency' Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [X] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [X] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [X] Has your PR been rebased against the latest commit within the target branch (typically master)? - [X] Is your initial contribution a single, squashed commit? ### For code changes: - [X] ~~~Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder?~~~ - [X] ~~~Have you written or updated unit tests to verify your changes?~~~ - [X] ~~~If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)?~~~ - [X] ~~~If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly?~~~ - [X] ~~~If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly?~~~ - [X] ~~~If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties?~~~ ### For documentation related changes: - [X] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/trixpan/nifi NIFI-3769 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/1729.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1729 commit 023dd0bf08df28c4850feac325a4300de002c338 Author: Andre F de Miranda <trix...@users.noreply.github.com> Date: 2017-05-02T08:43:58Z NIFI-3769 - Document detail value of 'nifi.provenance.repository.warm.cache.frequency' --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1721: NIFI-3651 - Add logo to README.md
Github user trixpan commented on the issue: https://github.com/apache/nifi/pull/1721 @apiri - I used the embedded HTML markdown notation as this seems to be a more universal approach to image resizing. There's are other options http://stackoverflow.com/questions/14675913/how-to-change-image-size-markdown --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #1721: NIFI-3651 - Add logo to README.md
GitHub user trixpan opened a pull request: https://github.com/apache/nifi/pull/1721 NIFI-3651 - Add logo to README.md Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [X] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [X] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [X] Has your PR been rebased against the latest commit within the target branch (typically master)? - [X] Is your initial contribution a single, squashed commit? ### For code changes: - [X] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [X] ~~~Have you written or updated unit tests to verify your changes?~~~ - [X] ~~~If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)?~~~ - [X] ~~~If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly?~~~ - [X] ~~~If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly?~~~ - [X] ~~~If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties?~~~ ### For documentation related changes: - [X] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/trixpan/nifi NIFI-3651 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/1721.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1721 commit 9c0a99f1e6fbe13c8eb9c442f16d34eb8ca4b8b1 Author: Andre F de Miranda <trix...@users.noreply.github.com> Date: 2017-05-01T12:56:52Z NIFI-3651 - Add logo to README.md --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1720: NIFI-3670 - Expose the control of ListenSyslog's CLIENT_AU...
Github user trixpan commented on the issue: https://github.com/apache/nifi/pull/1720 @bbende - hopefully this will address the intent behind the JIRA ticket. Please note that I also: - added displayName to all properties - re-indented some of the code (as it had double tabs) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #1720: NIFI-3670 - Expose the control of ListenSyslog's CL...
GitHub user trixpan opened a pull request: https://github.com/apache/nifi/pull/1720 NIFI-3670 - Expose the control of ListenSyslog's CLIENT_AUTH property⦠⦠to DFM Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [X] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [X] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [X] Has your PR been rebased against the latest commit within the target branch (typically master)? - [X] Is your initial contribution a single, squashed commit? ### For code changes: - [X] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [X] ~~~Have you written or updated unit tests to verify your changes?~~~ - [X] ~~~If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)?~~~ - [X] ~~~If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly?~~~ - [X] ~~~If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly?~~~ - [X] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [X] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/trixpan/nifi NIFI-3670 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/1720.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1720 commit 6d7411d305bbaf3ecc87f8a795f4b114f4841f02 Author: Andre F de Miranda <trix...@users.noreply.github.com> Date: 2017-05-01T12:32:51Z NIFI-3670 - Expose the control of ListenSyslog's CLIENT_AUTH property to DFM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1142: NIFI-2624 JdbcCommon treats BigDecimals now as Avro Logica...
Github user trixpan commented on the issue: https://github.com/apache/nifi/pull/1142 Adding @ToivoAdams comment here as his last comment on JIRA did use github's @ mentions instead of JIRAs [~ ] mentions: > @ijokarumawak > > Sorry I didn't have time to look your code. > Your approach looks much better. > My only concern is â do we generate record which conforms to Avro standard? > Can any other consumer (outside NiFi) read our Avro record and get back same BigDecimal? > > @joewitt > Right, letâs try to finish this. > Thanks > Toivo --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1102: NIFI-2867 The ColumnSize of datetime type is 19 length in ...
Github user trixpan commented on the issue: https://github.com/apache/nifi/pull/1102 @combineads - thanks for the PR. Apologies for lacking the background but I was wondering what the issue effectively is? From what you say the issue is that PutSQL requires -MM-dd HH:mm:ss.SSS to convert into timestamp, yet, the change is on ConvertJSONToSQL? Would you mind helping me to understand the issue a bit better? Hopefully once I understand it a bit better I should be able to help you reviewing this PR. Kind regards --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1696: NIFI-1655 - Add .gitattributes to specifically define
Github user trixpan commented on the issue: https://github.com/apache/nifi/pull/1696 @joewitt as suspected HL7 does inded use CR as a segment separator so I renamed the files to hl7 and treated hl7 as binary. Tests passed locally and I am now checking what travis has to say --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1015: NIFI-2767 Periodically reload properties from file-based v...
Github user trixpan commented on the issue: https://github.com/apache/nifi/pull/1015 @jfrazee any chance of rebasing this? Keen to revisit this PR and get the review going, otherwise happy for you to close it as well. Cheers --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #584: closing InputStream after execution completes
Github user trixpan commented on the issue: https://github.com/apache/nifi/pull/584 @vcharmcaster - there has been a while since you opened this PR and @olegz question still an answered. Is this PR still needed? Otherwise, would you mind closing it? Kind regards --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1608: NIFI-3607 Allow multi files mode with fixed names
Github user trixpan commented on the issue: https://github.com/apache/nifi/pull/1608 @pvillard31 thanks for the rebase. LGTM. Merged --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #1718: NIFI-391 - Introduce the ability to deprecate a com...
GitHub user trixpan opened a pull request: https://github.com/apache/nifi/pull/1718 NIFI-391 - Introduce the ability to deprecate a component Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [X] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [X] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [X] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [X] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [X] Have you written or updated unit tests to verify your changes? - [X] ~~~If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)?~~~ - [X] ~~~If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly?~~~ - [X] ~~~If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly?~~~ - [X] ~~~If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties?~~~ ### For documentation related changes: - [X] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/trixpan/nifi NIFI-391 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/1718.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1718 commit f4caef7b396701cda601749bb0e54cf5f5ba84d3 Author: Andre F de Miranda <trix...@users.noreply.github.com> Date: 2017-04-29T03:09:22Z NIFI-3761 - adjust testFullyDocumentedProcessor to correctly eval to false when needed commit eca1e2a465bedab060f26b2804e7e6d8c55b7e22 Author: Andre F de Miranda <trix...@users.noreply.github.com> Date: 2017-04-29T06:06:15Z NIFI-391 - Introduce the ability to deprecate a component NIFI-439 - Allow documentation to inform user about deprecation of a component --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #1639: NIFI-1939 - Correct issue where ParseSyslog was una...
Github user trixpan commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1639#discussion_r114039631
--- Diff:
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ParseSyslog.java
---
@@ -57,13 +57,17 @@
@SupportsBatching
@InputRequirement(Requirement.INPUT_REQUIRED)
@Tags({"logs", "syslog", "attributes", "system", "event", "message"})
-@CapabilityDescription("Parses the contents of a Syslog message and adds
attributes to the FlowFile for each of the parts of the Syslog message")
+@CapabilityDescription("Attempts to parses the contents of a Syslog
message in accordance to RFC5424 and RFC3164 " +
+"formats and adds attributes to the FlowFile for each of the parts
of the Syslog message." +
+"Note: Be mindfull that RFC3164 is informational and a wide range
of different implementations are present in" +
+" the wild. If messages fail parsing, considering using RFC5424 or
using a generic parsin processors such as " +
+"ExtractGrok.")
@WritesAttributes({@WritesAttribute(attribute = "syslog.priority",
description = "The priority of the Syslog message."),
@WritesAttribute(attribute = "syslog.severity", description = "The
severity of the Syslog message derived from the priority."),
@WritesAttribute(attribute = "syslog.facility", description = "The
facility of the Syslog message derived from the priority."),
@WritesAttribute(attribute = "syslog.version", description = "The
optional version from the Syslog message."),
@WritesAttribute(attribute = "syslog.timestamp", description = "The
timestamp of the Syslog message."),
-@WritesAttribute(attribute = "syslog.hostname", description = "The
hostname of the Syslog message."),
+@WritesAttribute(attribute = "syslog.hostname", description = "The
hostname or IP address of the Syslog message."),
--- End diff --
good call. fixed
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request #1692: NIFI-3726 - Introduces CompareFuzzyHash processor
Github user trixpan commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1692#discussion_r113932077
--- Diff:
nifi-nar-bundles/nifi-cybersecurity-bundle/nifi-cybersecurity-processors/src/main/java/org/apache/nifi/processors/cybersecurity/FuzzyHashContent.java
---
@@ -178,11 +161,11 @@ public void process(final InputStream in) throws
IOException {
try (ByteArrayOutputStream holder = new
ByteArrayOutputStream()) {
StreamUtils.copy(in,holder);
-if
(context.getProperty(HASH_ALGORITHM).getValue().equals(allowableValueSSDEEP.getValue()))
{
+if
(algorithm.equals(allowableValueSSDEEP.getValue())) {
--- End diff --
added to the same class as the checkMinimumRequirements. wonder if we
should create a separate class just to hold those help functions?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request #1692: NIFI-3726 - Introduces CompareFuzzyHash processor
Github user trixpan commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1692#discussion_r113917927
--- Diff:
nifi-nar-bundles/nifi-cybersecurity-bundle/nifi-cybersecurity-processors/src/main/java/org/apache/nifi/processors/cybersecurity/CompareFuzzyHash.java
---
@@ -0,0 +1,289 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.processors.cybersecurity;
+
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.behavior.WritesAttribute;
+import org.apache.nifi.annotation.behavior.WritesAttributes;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.SeeAlso;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.AllowableValue;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.processors.cybersecurity.matchers.FuzzyHashMatcher;
+import org.apache.nifi.processors.cybersecurity.matchers.SSDeepHashMatcher;
+import org.apache.nifi.processors.cybersecurity.matchers.TLSHHashMatcher;
+
+import java.io.BufferedReader;
+import java.io.File;
+import java.io.FileInputStream;
+import java.io.FileNotFoundException;
+import java.io.IOException;
+import java.io.InputStreamReader;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.concurrent.ConcurrentHashMap;
+
+
+@EventDriven
+@SideEffectFree
+@SupportsBatching
+@InputRequirement(InputRequirement.Requirement.INPUT_REQUIRED)
+@SeeAlso({FuzzyHashContent.class})
+@Tags({"hashing", "fuzzy-hashing", "cyber-security"})
+@CapabilityDescription("Compares an attribute containing a Fuzzy Hash
against a file containing a list of fuzzy hashes, " +
+"appending an attribute to the FlowFile in case of a successful
match.")
+
+@WritesAttributes({
+@WritesAttribute(attribute = ".N.match", description = "The
match that resembles the attribute specified " +
+"by the property. Note that: 'XXX'
gets replaced with the "),
+@WritesAttribute(attribute = ".N.similarity", description =
"The similarity score between this flowfile" +
+"and its match of the same number N. Note that: 'XXX' gets
replaced with the ")})
+
+public class CompareFuzzyHash extends AbstractFuzzyHashProcessor {
+public static final AllowableValue singleMatch = new AllowableValue(
+"single",
+"single",
+"Send FlowFile to matched after the first match above
threshold");
+public static final AllowableValue multiMatch = new AllowableValue(
+"multi-match",
+"multi-match",
+"Iterate full list of hashes before deciding to send FlowFile
to matched or unmatched");
+
+public static final PropertyDescriptor HASH_LIST_FILE = new
PropertyDescriptor.Builder()
+.name("H
[GitHub] nifi pull request #1692: NIFI-3726 - Introduces CompareFuzzyHash processor
Github user trixpan commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1692#discussion_r113918024
--- Diff:
nifi-nar-bundles/nifi-cybersecurity-bundle/nifi-cybersecurity-processors/src/main/java/org/apache/nifi/processors/cybersecurity/CompareFuzzyHash.java
---
@@ -183,38 +181,24 @@ public void onTrigger(ProcessContext context,
ProcessSession session) throws Pro
return;
}
-Digest inputDigest = null;
-SpamSum spamSum = null;
+FuzzyHashMatcher fuzzyHashMatcher = null;
switch (algorithm) {
case tlsh:
-// In case we are using TLSH, makes sense to create the
source Digest just once
-inputDigest = compareStringToTLSHDigest(inputHash);
-// we test the validation for null (failed)
-if (inputDigest == null) {
-// and if that is the case we log
-logger.error("Invalid hash provided. Sending to
failure");
-// and send to failure
-session.transfer(flowFile, REL_FAILURE);
-session.commit();
-return;
-}
+fuzzyHashMatcher = new TLSHHashMatcher(getLogger());
break;
case ssdeep:
-// However, in SSDEEP, the compare function uses the two
desired strings.
-// So we try a poor man validation (the SpamSum comparison
function seems to
-// be resilient enough but we still want to route to
failure in case it
-// clearly bogus data
-if (looksLikeSpamSum(inputHash) == true) {
-spamSum = new SpamSum();
-} else {
-// and if that is the case we log
-logger.error("Invalid hash provided. Sending to
failure");
-// and send to failure
-session.transfer(flowFile, REL_FAILURE);
-session.commit();
-return;
-}
+fuzzyHashMatcher = new SSDeepHashMatcher(getLogger());
+break;
+}
+
+if (fuzzyHashMatcher.isValidHash(inputHash) == false) {
--- End diff --
introduced a `default` to `switch(algorithm)` so that we don't get here
with null anymore.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi issue #1716: NIFI-3759 - avro append for PutHDFS processor
Github user trixpan commented on the issue: https://github.com/apache/nifi/pull/1716 Speed and overall project roadmap. The approach introduced there brings significant advantages to the use of PutHDFS. https://github.com/apache/nifi/pull/1712 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1716: NIFI-3759 - avro append for PutHDFS processor
Github user trixpan commented on the issue: https://github.com/apache/nifi/pull/1716 @jonashartwig thank you for your contribution. I was looking at your code and was wondering if have you had a chance to see the put / fetch parquet @bbende submitted a few hours ago? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #1692: NIFI-3726 - Introduces CompareFuzzyHash processor
Github user trixpan commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1692#discussion_r113587423
--- Diff:
nifi-nar-bundles/nifi-cybersecurity-bundle/nifi-cybersecurity-processors/src/main/java/org/apache/nifi/processors/cybersecurity/FuzzyHashContent.java
---
@@ -157,9 +138,11 @@ public void onTrigger(final ProcessContext context,
final ProcessSession session
}
final ComponentLog logger = getLogger();
+String algorithm = context.getProperty(HASH_ALGORITHM).getValue();
// Check if content matches minimum length requirement
-if (context.getProperty(HASH_ALGORITHM).equals(allowableValueTLSH)
&& flowFile.getSize() < 512 ) {
+
+if (checkMinimumAlgorithmRequirements(algorithm, flowFile) ==
false) {
logger.info("The content of {} is smaller than the minimum
required by TLSH, routing to failure", new Object[]{flowFile});
--- End diff --
fixed
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request #1692: NIFI-3726 - Introduces CompareFuzzyHash processor
Github user trixpan commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1692#discussion_r113586627
--- Diff:
nifi-nar-bundles/nifi-cybersecurity-bundle/nifi-cybersecurity-processors/src/main/java/org/apache/nifi/processors/cybersecurity/CompareFuzzyHash.java
---
@@ -0,0 +1,289 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.processors.cybersecurity;
+
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.behavior.WritesAttribute;
+import org.apache.nifi.annotation.behavior.WritesAttributes;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.SeeAlso;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.AllowableValue;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.processors.cybersecurity.matchers.FuzzyHashMatcher;
+import org.apache.nifi.processors.cybersecurity.matchers.SSDeepHashMatcher;
+import org.apache.nifi.processors.cybersecurity.matchers.TLSHHashMatcher;
+
+import java.io.BufferedReader;
+import java.io.File;
+import java.io.FileInputStream;
+import java.io.FileNotFoundException;
+import java.io.IOException;
+import java.io.InputStreamReader;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.concurrent.ConcurrentHashMap;
+
+
+@EventDriven
+@SideEffectFree
+@SupportsBatching
+@InputRequirement(InputRequirement.Requirement.INPUT_REQUIRED)
+@SeeAlso({FuzzyHashContent.class})
+@Tags({"hashing", "fuzzy-hashing", "cyber-security"})
+@CapabilityDescription("Compares an attribute containing a Fuzzy Hash
against a file containing a list of fuzzy hashes, " +
+"appending an attribute to the FlowFile in case of a successful
match.")
+
+@WritesAttributes({
+@WritesAttribute(attribute = ".N.match", description = "The
match that resembles the attribute specified " +
+"by the property. Note that: 'XXX'
gets replaced with the "),
+@WritesAttribute(attribute = ".N.similarity", description =
"The similarity score between this flowfile" +
+"and its match of the same number N. Note that: 'XXX' gets
replaced with the ")})
+
+public class CompareFuzzyHash extends AbstractFuzzyHashProcessor {
+public static final AllowableValue singleMatch = new AllowableValue(
+"single",
+"single",
+"Send FlowFile to matched after the first match above
threshold");
+public static final AllowableValue multiMatch = new AllowableValue(
+"multi-match",
+"multi-match",
+"Iterate full list of hashes before deciding to send FlowFile
to matched or unmatched");
+
+public static final PropertyDescriptor HASH_LIST_FILE = new
PropertyDescriptor.Builder()
+.name("H
[GitHub] nifi pull request #1692: NIFI-3726 - Introduces CompareFuzzyHash processor
Github user trixpan commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1692#discussion_r113586323
--- Diff:
nifi-nar-bundles/nifi-cybersecurity-bundle/nifi-cybersecurity-processors/src/main/java/org/apache/nifi/processors/cybersecurity/CompareFuzzyHash.java
---
@@ -183,38 +181,24 @@ public void onTrigger(ProcessContext context,
ProcessSession session) throws Pro
return;
}
-Digest inputDigest = null;
-SpamSum spamSum = null;
+FuzzyHashMatcher fuzzyHashMatcher = null;
switch (algorithm) {
case tlsh:
-// In case we are using TLSH, makes sense to create the
source Digest just once
-inputDigest = compareStringToTLSHDigest(inputHash);
-// we test the validation for null (failed)
-if (inputDigest == null) {
-// and if that is the case we log
-logger.error("Invalid hash provided. Sending to
failure");
-// and send to failure
-session.transfer(flowFile, REL_FAILURE);
-session.commit();
-return;
-}
+fuzzyHashMatcher = new TLSHHashMatcher(getLogger());
break;
case ssdeep:
-// However, in SSDEEP, the compare function uses the two
desired strings.
-// So we try a poor man validation (the SpamSum comparison
function seems to
-// be resilient enough but we still want to route to
failure in case it
-// clearly bogus data
-if (looksLikeSpamSum(inputHash) == true) {
-spamSum = new SpamSum();
-} else {
-// and if that is the case we log
-logger.error("Invalid hash provided. Sending to
failure");
-// and send to failure
-session.transfer(flowFile, REL_FAILURE);
-session.commit();
-return;
-}
+fuzzyHashMatcher = new SSDeepHashMatcher(getLogger());
+break;
+}
--- End diff --
fixed
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request #1692: NIFI-3726 - Introduces CompareFuzzyHash processor
Github user trixpan commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1692#discussion_r113585955
--- Diff:
nifi-nar-bundles/nifi-cybersecurity-bundle/nifi-cybersecurity-processors/src/main/java/org/apache/nifi/processors/cybersecurity/CompareFuzzyHash.java
---
@@ -110,24 +107,25 @@
public static final PropertyDescriptor MATCHING_MODE = new
PropertyDescriptor.Builder()
.name("MATCHING_MODE")
.displayName("Matching mode")
-.description("The ")
+.description("Defines if the Processor should try to match as
many entries as possible (" + multiMatch.getDisplayName() +
+") or if it should stio after the first match (" +
singleMatch.getDisplayName() + ")")
.required(true)
.allowableValues(singleMatch,multiMatch)
.defaultValue(singleMatch.getValue())
.build();
-public static final Relationship REL_MATCH = new Relationship.Builder()
-.name("Matched")
+public static final Relationship REL_FOUND = new Relationship.Builder()
+.name("found")
.description("Any FlowFile that is successfully matched to an
existing hash will be sent to this Relationship.")
.build();
-public static final Relationship REL_NON_MATCH = new
Relationship.Builder()
-.name("non-match")
+public static final Relationship REL_NOT_FOUND = new
Relationship.Builder()
+.name("not found")
--- End diff --
sounds like good practice to me. Fixed
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request #1692: NIFI-3726 - Introduces CompareFuzzyHash processor
Github user trixpan commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1692#discussion_r113585822
--- Diff:
nifi-nar-bundles/nifi-cybersecurity-bundle/nifi-cybersecurity-processors/src/main/java/org/apache/nifi/processors/cybersecurity/CompareFuzzyHash.java
---
@@ -110,24 +107,25 @@
public static final PropertyDescriptor MATCHING_MODE = new
PropertyDescriptor.Builder()
.name("MATCHING_MODE")
.displayName("Matching mode")
-.description("The ")
+.description("Defines if the Processor should try to match as
many entries as possible (" + multiMatch.getDisplayName() +
+") or if it should stio after the first match (" +
singleMatch.getDisplayName() + ")")
--- End diff --
fixed
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi issue #1692: NIFI-3726 - Introduces CompareFuzzyHash processor
Github user trixpan commented on the issue: https://github.com/apache/nifi/pull/1692 @alopresto - I put together the interface based approach and indeed it seems a much cleaner code so thank you for pointing me into this direction. The code has not been optimised and still has a few sharp edges around String split that must be solved before merging but I would appreciate if you could confirm I am moving in the right direction. Once again, thank you for the suggestion. It is certainly a great chance to try to do things "properly". Cheers --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #1692: NIFI-3726 - Introduces CompareFuzzyHash processor
Github user trixpan commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1692#discussion_r113463146
--- Diff:
nifi-nar-bundles/nifi-cybersecurity-bundle/nifi-cybersecurity-processors/src/main/java/org/apache/nifi/processors/cybersecurity/CompareFuzzyHash.java
---
@@ -0,0 +1,358 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.cybersecurity;
+
+import com.idealista.tlsh.digests.Digest;
+import com.idealista.tlsh.digests.DigestBuilder;
+import info.debatty.java.spamsum.SpamSum;
+
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.behavior.WritesAttribute;
+import org.apache.nifi.annotation.behavior.WritesAttributes;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.SeeAlso;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.AllowableValue;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.util.StandardValidators;
+
+import java.io.BufferedReader;
+import java.io.File;
+import java.io.FileInputStream;
+import java.io.FileNotFoundException;
+import java.io.IOException;
+import java.io.InputStreamReader;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Scanner;
+import java.util.Set;
+import java.util.concurrent.ConcurrentHashMap;
+
+
+@EventDriven
+@SideEffectFree
+@SupportsBatching
+@InputRequirement(InputRequirement.Requirement.INPUT_REQUIRED)
+@SeeAlso({FuzzyHashContent.class})
+@Tags({"hashing", "fuzzy-hashing", "cyber-security"})
+@CapabilityDescription("Compares an attribute containing a Fuzzy Hash
against a file containing a list of fuzzy hashes, " +
+"appending an attribute to the FlowFile in case of a successful
match.")
+
+@WritesAttributes({
+@WritesAttribute(attribute = ".N.match", description = "The
match that ressambles the attribute specified " +
+"by the property. Note that: 'XXX'
gets replaced with the "),
+@WritesAttribute(attribute = ".N.similarity", description =
"The similarity score between this flowfile" +
+"and the its match of the same number N. Note that: 'XXX'
gets replaced with the ")})
+
+public class CompareFuzzyHash extends AbstractFuzzyHashProcessor {
+public static final AllowableValue singleMatch = new AllowableValue(
+"single",
+"single",
+"Send FlowFile to matched after the first match above
threshold");
+public static final AllowableValue multiMatch = new AllowableValue(
+"multi-match",
+"multi-match",
+"Iterate full list of hashes before deciding to send FlowFile
to matched or unmatched");
+
+public static final PropertyDescriptor HASH_LIST_FILE = new
PropertyDescriptor.Builder()
+.name("HASH_LIST_FILE")
+.displayName("Hash List source
[GitHub] nifi pull request #1692: NIFI-3726 - Introduces CompareFuzzyHash processor
Github user trixpan commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1692#discussion_r113462407
--- Diff:
nifi-nar-bundles/nifi-cybersecurity-bundle/nifi-cybersecurity-processors/src/main/java/org/apache/nifi/processors/cybersecurity/CompareFuzzyHash.java
---
@@ -0,0 +1,358 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.cybersecurity;
+
+import com.idealista.tlsh.digests.Digest;
+import com.idealista.tlsh.digests.DigestBuilder;
+import info.debatty.java.spamsum.SpamSum;
+
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.behavior.WritesAttribute;
+import org.apache.nifi.annotation.behavior.WritesAttributes;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.SeeAlso;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.AllowableValue;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.util.StandardValidators;
+
+import java.io.BufferedReader;
+import java.io.File;
+import java.io.FileInputStream;
+import java.io.FileNotFoundException;
+import java.io.IOException;
+import java.io.InputStreamReader;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Scanner;
+import java.util.Set;
+import java.util.concurrent.ConcurrentHashMap;
+
+
+@EventDriven
+@SideEffectFree
+@SupportsBatching
+@InputRequirement(InputRequirement.Requirement.INPUT_REQUIRED)
+@SeeAlso({FuzzyHashContent.class})
+@Tags({"hashing", "fuzzy-hashing", "cyber-security"})
+@CapabilityDescription("Compares an attribute containing a Fuzzy Hash
against a file containing a list of fuzzy hashes, " +
+"appending an attribute to the FlowFile in case of a successful
match.")
+
+@WritesAttributes({
+@WritesAttribute(attribute = ".N.match", description = "The
match that ressambles the attribute specified " +
--- End diff --
fixed
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request #1692: NIFI-3726 - Introduces CompareFuzzyHash processor
Github user trixpan commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1692#discussion_r113462423
--- Diff:
nifi-nar-bundles/nifi-cybersecurity-bundle/nifi-cybersecurity-processors/src/main/java/org/apache/nifi/processors/cybersecurity/CompareFuzzyHash.java
---
@@ -0,0 +1,358 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.cybersecurity;
+
+import com.idealista.tlsh.digests.Digest;
+import com.idealista.tlsh.digests.DigestBuilder;
+import info.debatty.java.spamsum.SpamSum;
+
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.behavior.WritesAttribute;
+import org.apache.nifi.annotation.behavior.WritesAttributes;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.SeeAlso;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.AllowableValue;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.util.StandardValidators;
+
+import java.io.BufferedReader;
+import java.io.File;
+import java.io.FileInputStream;
+import java.io.FileNotFoundException;
+import java.io.IOException;
+import java.io.InputStreamReader;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Scanner;
+import java.util.Set;
+import java.util.concurrent.ConcurrentHashMap;
+
+
+@EventDriven
+@SideEffectFree
+@SupportsBatching
+@InputRequirement(InputRequirement.Requirement.INPUT_REQUIRED)
+@SeeAlso({FuzzyHashContent.class})
+@Tags({"hashing", "fuzzy-hashing", "cyber-security"})
+@CapabilityDescription("Compares an attribute containing a Fuzzy Hash
against a file containing a list of fuzzy hashes, " +
+"appending an attribute to the FlowFile in case of a successful
match.")
+
+@WritesAttributes({
+@WritesAttribute(attribute = ".N.match", description = "The
match that ressambles the attribute specified " +
+"by the property. Note that: 'XXX'
gets replaced with the "),
+@WritesAttribute(attribute = ".N.similarity", description =
"The similarity score between this flowfile" +
+"and the its match of the same number N. Note that: 'XXX'
gets replaced with the ")})
--- End diff --
fixed
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request #1692: NIFI-3726 - Introduces CompareFuzzyHash processor
Github user trixpan commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1692#discussion_r113462459
--- Diff:
nifi-nar-bundles/nifi-cybersecurity-bundle/nifi-cybersecurity-processors/src/main/java/org/apache/nifi/processors/cybersecurity/CompareFuzzyHash.java
---
@@ -0,0 +1,358 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.cybersecurity;
+
+import com.idealista.tlsh.digests.Digest;
+import com.idealista.tlsh.digests.DigestBuilder;
+import info.debatty.java.spamsum.SpamSum;
+
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.behavior.WritesAttribute;
+import org.apache.nifi.annotation.behavior.WritesAttributes;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.SeeAlso;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.AllowableValue;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.util.StandardValidators;
+
+import java.io.BufferedReader;
+import java.io.File;
+import java.io.FileInputStream;
+import java.io.FileNotFoundException;
+import java.io.IOException;
+import java.io.InputStreamReader;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Scanner;
+import java.util.Set;
+import java.util.concurrent.ConcurrentHashMap;
+
+
+@EventDriven
+@SideEffectFree
+@SupportsBatching
+@InputRequirement(InputRequirement.Requirement.INPUT_REQUIRED)
+@SeeAlso({FuzzyHashContent.class})
+@Tags({"hashing", "fuzzy-hashing", "cyber-security"})
+@CapabilityDescription("Compares an attribute containing a Fuzzy Hash
against a file containing a list of fuzzy hashes, " +
+"appending an attribute to the FlowFile in case of a successful
match.")
+
+@WritesAttributes({
+@WritesAttribute(attribute = ".N.match", description = "The
match that ressambles the attribute specified " +
+"by the property. Note that: 'XXX'
gets replaced with the "),
+@WritesAttribute(attribute = ".N.similarity", description =
"The similarity score between this flowfile" +
+"and the its match of the same number N. Note that: 'XXX'
gets replaced with the ")})
+
+public class CompareFuzzyHash extends AbstractFuzzyHashProcessor {
+public static final AllowableValue singleMatch = new AllowableValue(
+"single",
+"single",
+"Send FlowFile to matched after the first match above
threshold");
+public static final AllowableValue multiMatch = new AllowableValue(
+"multi-match",
+"multi-match",
+"Iterate full list of hashes before deciding to send FlowFile
to matched or unmatched");
+
+public static final PropertyDescriptor HASH_LIST_FILE = new
PropertyDescriptor.Builder()
+.name("HASH_LIST_FILE")
+.displayName("Hash List source
[GitHub] nifi issue #1696: NIFI-1655 - Add .gitattributes to specifically define
Github user trixpan commented on the issue: https://github.com/apache/nifi/pull/1696 @joewitt - When doing this change I noticed some changes to HL7 files and looking online it seems like HL7 uses CR as a "segment separator". Would you mind if I renamed the relevant files with extension .hl7 (which seems to be accepted as a de-facto HL7 file extension) and exclude them accordingly? Cheers --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #1696: NIFI-1655 - Add .gitattributes to specifically defi...
GitHub user trixpan opened a pull request: https://github.com/apache/nifi/pull/1696 NIFI-1655 - Add .gitattributes to specifically define the line ending format of particular file type (i.e. .java = LF, .bat=CRLF) Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [X] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [X] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [X] Has your PR been rebased against the latest commit within the target branch (typically master)? - [X] Is your initial contribution a single, squashed commit? ### For code changes: - [X] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [X] ~~~Have you written or updated unit tests to verify your changes?~~~ - [X] ~~~If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)?~~~ - [X] ~~~If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly?~~~ - [X] ~~~If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly?~~~ - [X] ~~~If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties?~~~ ### For documentation related changes: - [X] ~~~Have you ensured that format looks appropriate for the output in which it is rendered?~~~ ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/trixpan/nifi NIFI-1655 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/1696.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1696 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #1650: NIFI-2661 - Abstract the GeoEnrichIP processor
Github user trixpan commented on a diff in the pull request: https://github.com/apache/nifi/pull/1650#discussion_r113431000 --- Diff: nifi-nar-bundles/nifi-enrich-bundle/nifi-enrich-processors/src/test/java/org/apache/nifi/processors/TestISPEnrichIP.java --- @@ -0,0 +1,310 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.nifi.processors; + +import com.fasterxml.jackson.databind.DeserializationFeature; +import com.fasterxml.jackson.databind.InjectableValues; +import com.fasterxml.jackson.databind.ObjectMapper; +import com.maxmind.geoip2.exception.GeoIp2Exception; +import com.maxmind.geoip2.model.IspResponse; +import org.apache.nifi.annotation.lifecycle.OnScheduled; +import org.apache.nifi.flowfile.FlowFile; +import org.apache.nifi.processor.ProcessContext; +import org.apache.nifi.processors.maxmind.DatabaseReader; +import org.apache.nifi.util.MockFlowFile; +import org.apache.nifi.util.TestRunner; +import org.apache.nifi.util.TestRunners; +import org.junit.Before; +import org.junit.Test; +import org.junit.runner.RunWith; +import org.powermock.core.classloader.annotations.PrepareForTest; +import org.powermock.modules.junit4.PowerMockRunner; + +import java.io.IOException; +import java.net.InetAddress; +import java.net.UnknownHostException; +import java.util.Collections; +import java.util.HashMap; +import java.util.List; +import java.util.Map; + +import static org.junit.Assert.*; --- End diff -- just realised the intelliJ settings for import and import static are independent... :disappointed: fixed --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #1650: NIFI-2661 - Abstract the GeoEnrichIP processor
Github user trixpan commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1650#discussion_r113428044
--- Diff:
nifi-nar-bundles/nifi-enrich-bundle/nifi-enrich-processors/src/main/java/org/apache/nifi/processors/ISPEnrichIP.java
---
@@ -0,0 +1,132 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors;
+
+import com.maxmind.geoip2.exception.GeoIp2Exception;
+import com.maxmind.geoip2.model.IspResponse;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.InputRequirement.Requirement;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.behavior.WritesAttribute;
+import org.apache.nifi.annotation.behavior.WritesAttributes;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processors.maxmind.DatabaseReader;
+import org.apache.nifi.util.StopWatch;
+
+import java.io.IOException;
+import java.net.InetAddress;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.concurrent.TimeUnit;
+
+@EventDriven
+@SideEffectFree
+@SupportsBatching
+@Tags({"ISP", "enrich", "ip", "maxmind"})
+@InputRequirement(Requirement.INPUT_REQUIRED)
+@CapabilityDescription("Looks up ISP information for an IP address and
adds the information to FlowFile attributes. The "
++ "ISP data is provided as a MaxMind database. The attribute that
contains the IP address to lookup is provided by the "
++ "'IP Address Attribute' property. If the name of the attribute
provided is 'X', then the the attributes added by enrichment "
++ "will take the form X.isp.")
+@WritesAttributes({
+@WritesAttribute(attribute = "X.isp.lookup.micros", description = "The
number of microseconds that the geo lookup took"),
+@WritesAttribute(attribute = "X.isp.asn", description = "The
Autonomous System Number (ASN) identified for the IP address"),
+@WritesAttribute(attribute = "X.isp.asn.organization", description =
"The Organization Associated with the ASN identified"),
+@WritesAttribute(attribute = "X.isp.name", description = "The name of
the ISP associated with the IP address provided"),
+@WritesAttribute(attribute = "X.isp.organization", description = "The
Organization associated with the IP address provided"),})
+public class ISPEnrichIP extends AbstractEnrichIP {
+
+@Override
+public void onTrigger(final ProcessContext context, final
ProcessSession session) throws ProcessException {
+FlowFile flowFile = session.get();
+if (flowFile == null) {
+return;
+}
+
+final DatabaseReader dbReader = databaseReaderRef.get();
+final String ipAttributeName =
context.getProperty(IP_ADDRESS_ATTRIBUTE).evaluateAttributeExpressions(flowFile).getValue();
+final String ipAttributeValue =
flowFile.getAttribute(ipAttributeName);
+if (StringUtils.isEmpty(ipAttributeName)) { //TODO need to add
additional validation - should look like an IPv4 or IPv6 addr for instance
--- End diff --
I was revisiting the code and to be honest that TODO should be simply
removed. The validation is done just following that section when we call:
```
try {
[GitHub] nifi pull request #1650: NIFI-2661 - Abstract the GeoEnrichIP processor
Github user trixpan commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1650#discussion_r113411154
--- Diff:
nifi-nar-bundles/nifi-enrich-bundle/nifi-enrich-processors/src/main/java/org/apache/nifi/processors/GeoEnrichIP.java
---
@@ -70,6 +55,7 @@
@WritesAttributes({
@WritesAttribute(attribute = "X.geo.lookup.micros", description = "The
number of microseconds that the geo lookup took"),
@WritesAttribute(attribute = "X.geo.city", description = "The city
identified for the IP address"),
+@WritesAttribute(attribute = "X.geo.accuracy", description = "The
accuracy radius if provided by the database"),
--- End diff --
fixed
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request #1650: NIFI-2661 - Abstract the GeoEnrichIP processor
Github user trixpan commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1650#discussion_r113340946
--- Diff:
nifi-nar-bundles/nifi-enrich-bundle/nifi-enrich-processors/src/main/java/org/apache/nifi/processors/GeoEnrichIP.java
---
@@ -208,6 +120,8 @@ public void onTrigger(final ProcessContext context,
final ProcessSession session
attrs.put(new
StringBuilder(ipAttributeName).append(".geo.longitude").toString(),
longitude.toString());
}
+attrs.put(new
StringBuilder(ipAttributeName).append(".accuracy").toString(),
response.getLocation().getAccuracyRadius().toString() );
--- End diff --
Makes sense
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request #1692: NIFI-3726 - Introduces CompareFuzzyHash processor
Github user trixpan commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1692#discussion_r113340505
--- Diff:
nifi-nar-bundles/nifi-cybersecurity-bundle/nifi-cybersecurity-processors/src/main/java/org/apache/nifi/processors/cybersecurity/CompareFuzzyHash.java
---
@@ -0,0 +1,358 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.cybersecurity;
+
+import com.idealista.tlsh.digests.Digest;
+import com.idealista.tlsh.digests.DigestBuilder;
+import info.debatty.java.spamsum.SpamSum;
+
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.behavior.WritesAttribute;
+import org.apache.nifi.annotation.behavior.WritesAttributes;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.SeeAlso;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.AllowableValue;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.logging.ComponentLog;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.ProcessorInitializationContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.util.StandardValidators;
+
+import java.io.BufferedReader;
+import java.io.File;
+import java.io.FileInputStream;
+import java.io.FileNotFoundException;
+import java.io.IOException;
+import java.io.InputStreamReader;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Scanner;
+import java.util.Set;
+import java.util.concurrent.ConcurrentHashMap;
+
+
+@EventDriven
+@SideEffectFree
+@SupportsBatching
+@InputRequirement(InputRequirement.Requirement.INPUT_REQUIRED)
+@SeeAlso({FuzzyHashContent.class})
+@Tags({"hashing", "fuzzy-hashing", "cyber-security"})
+@CapabilityDescription("Compares an attribute containing a Fuzzy Hash
against a file containing a list of fuzzy hashes, " +
+"appending an attribute to the FlowFile in case of a successful
match.")
+
+@WritesAttributes({
+@WritesAttribute(attribute = ".N.match", description = "The
match that ressambles the attribute specified " +
+"by the property. Note that: 'XXX'
gets replaced with the "),
+@WritesAttribute(attribute = ".N.similarity", description =
"The similarity score between this flowfile" +
+"and the its match of the same number N. Note that: 'XXX'
gets replaced with the ")})
+
+public class CompareFuzzyHash extends AbstractFuzzyHashProcessor {
+public static final AllowableValue singleMatch = new AllowableValue(
+"single",
+"single",
+"Send FlowFile to matched after the first match above
threshold");
+public static final AllowableValue multiMatch = new AllowableValue(
+"multi-match",
+"multi-match",
+"Iterate full list of hashes before deciding to send FlowFile
to matched or unmatched");
+
+public static final PropertyDescriptor HASH_LIST_FILE = new
PropertyDescriptor.Builder()
+.name("HASH_LIST_FILE")
+.displayName("Hash List source
[GitHub] nifi issue #1693: NIFI-3737 - Ran optipng on all png files
Github user trixpan commented on the issue: https://github.com/apache/nifi/pull/1693 @andrewmlim :+1: :+1: :+1: :+1: for catching this... --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1644: NIFI-11 - Capture StringIndexOutOfBoundsException to preve...
Github user trixpan commented on the issue: https://github.com/apache/nifi/pull/1644 @mcgilman makes total sense. Will adjust --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1642: NIFI-1336 - Ensure that flowfiles are penalized upon excep...
Github user trixpan commented on the issue: https://github.com/apache/nifi/pull/1642 @markap14 no worries. Thanks for the clarification I will try to give it another go. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1395: NIFI-3280 PostHTTP Option to write response to attribute o...
Github user trixpan commented on the issue: https://github.com/apache/nifi/pull/1395 @nickcarenza thanks for the PR, much appreciated. I was going through it and got myself wondering: Is there any reason why we are doing this when InvokeHTTP processor cater for the very same need (i.e. POST followed by content to attributes) From its documentation: > **Put Response Body In Attribute** > > _If set, the response body received back will be put into an attribute of the original FlowFile instead of a separate FlowFile. The attribute key to put to is determined by evaluating value of this property. > Supports Expression Language: true_ Keen to understand the reason as I suspect I am missing something --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
