[GitHub] nifi issue #3165: NIFI-5318 Implement NiFi test harness
Github user MikeThomsen commented on the issue: https://github.com/apache/nifi/pull/3165 @peter-gergely-horvath Tried your latest change and still get these results: ``` [INFO] --- [INFO] Running org.apache.nifi.testharness.samples.NiFiFlowTest [INFO] Running org.apache.nifi.testharness.samples.NiFiMockFlowTest [INFO] [INFO] Results: [INFO] [INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0 [INFO] [INFO] [INFO] BUILD FAILURE [INFO] [INFO] Total time: 01:46 min [INFO] Finished at: 2018-12-12T11:20:20-05:00 [INFO] Final Memory: 36M/602M [INFO] [ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.20.1:test (default-test) on project nifi-testharness: There are test failures. [ERROR] [ERROR] Please refer to /Users/michaelthomsen/workspace/nifi/nifi-testharness/target/surefire-reports for the individual test results. [ERROR] Please refer to dump files (if any exist) [date]-jvmRun[N].dump, [date].dumpstream and [date]-jvmRun[N].dumpstream. [ERROR] ExecutionException The forked VM terminated without properly saying goodbye. VM crash or System.exit called? [ERROR] Command was /bin/sh -c cd /Users/michaelthomsen/workspace/nifi/nifi-testharness/nifi_testharness_nifi_home && /Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/jre/bin/java -Xmx1G -Djava.net.preferIPv4Stack=true -jar /Users/michaelthomsen/workspace/nifi/nifi-testharness/target/surefire/surefirebooter5354893279908884469.jar /Users/michaelthomsen/workspace/nifi/nifi-testharness/target/surefire 2018-12-12T11-18-44_222-jvmRun1 surefire377069077642032tmp surefire_03864170411723641611tmp [ERROR] Error occurred in starting fork, check output in log [ERROR] Process Exit Code: 1 [ERROR] Crashed tests: [ERROR] org.apache.nifi.testharness.samples.NiFiFlowTest [ERROR] ExecutionException The forked VM terminated without properly saying goodbye. VM crash or System.exit called? [ERROR] Command was /bin/sh -c cd /Users/michaelthomsen/workspace/nifi/nifi-testharness/nifi_testharness_nifi_home && /Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/jre/bin/java -Xmx1G -Djava.net.preferIPv4Stack=true -jar /Users/michaelthomsen/workspace/nifi/nifi-testharness/target/surefire/surefirebooter8415059538712188018.jar /Users/michaelthomsen/workspace/nifi/nifi-testharness/target/surefire 2018-12-12T11-18-44_222-jvmRun1 surefire7043843761330522886tmp surefire_12826311853280960897tmp [ERROR] Error occurred in starting fork, check output in log [ERROR] Process Exit Code: 1 [ERROR] Crashed tests: [ERROR] org.apache.nifi.testharness.samples.NiFiMockFlowTest [ERROR] org.apache.maven.surefire.booter.SurefireBooterForkException: ExecutionException The forked VM terminated without properly saying goodbye. VM crash or System.exit called? [ERROR] Command was /bin/sh -c cd /Users/michaelthomsen/workspace/nifi/nifi-testharness/nifi_testharness_nifi_home && /Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/jre/bin/java -Xmx1G -Djava.net.preferIPv4Stack=true -jar /Users/michaelthomsen/workspace/nifi/nifi-testharness/target/surefire/surefirebooter5354893279908884469.jar /Users/michaelthomsen/workspace/nifi/nifi-testharness/target/surefire 2018-12-12T11-18-44_222-jvmRun1 surefire377069077642032tmp surefire_03864170411723641611tmp [ERROR] Error occurred in starting fork, check output in log [ERROR] Process Exit Code: 1 [ERROR] Crashed tests: [ERROR] org.apache.nifi.testharness.samples.NiFiFlowTest [ERROR] ExecutionException The forked VM terminated without properly saying goodbye. VM crash or System.exit called? [ERROR] Command was /bin/sh -c cd /Users/michaelthomsen/workspace/nifi/nifi-testharness/nifi_testharness_nifi_home && /Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/jre/bin/java -Xmx1G -Djava.net.preferIPv4Stack=true -jar /Users/michaelthomsen/workspace/nifi/nifi-testharness/target/surefire/surefirebooter8415059538712188018.jar /Users/michaelthomsen/workspace/nifi/nifi-testharness/target/surefire 2018-12-12T11-18-44_222-jvmRun1 surefire7043843761330522886tmp surefire_12826311853280960897tmp [ERROR] Error occurred in starting fork, check output in log [ERROR] Process Exit Code: 1 [ERROR] Crashed tests: [ERROR] org.apache.nifi.testharness.samples.NiFiMockFlowTest [ERROR] at org.apache.maven.plugin.surefire.booterclient.ForkStarter.awaitResultsDone(ForkStarter.java:496) [ERROR] at org.apache.maven.plugin.surefire.booterclient.For
[GitHub] nifi issue #3105: NIFI-5621: Added Cassandra connection provider service
Github user MikeThomsen commented on the issue: https://github.com/apache/nifi/pull/3105 Merged, thanks @zenfenan ---
[GitHub] nifi pull request #3105: NIFI-5621: Added Cassandra connection provider serv...
Github user MikeThomsen commented on a diff in the pull request: https://github.com/apache/nifi/pull/3105#discussion_r241047229 --- Diff: nifi-nar-bundles/nifi-cassandra-bundle/nifi-cassandra-services-nar/src/main/resources/META-INF/LICENSE --- @@ -0,0 +1,369 @@ + + Apache License + Version 2.0, January 2004 +http://www.apache.org/licenses/ + + TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + + 1. Definitions. + + "License" shall mean the terms and conditions for use, reproduction, + and distribution as defined by Sections 1 through 9 of this document. + + "Licensor" shall mean the copyright owner or entity authorized by + the copyright owner that is granting the License. + + "Legal Entity" shall mean the union of the acting entity and all + other entities that control, are controlled by, or are under common + control with that entity. For the purposes of this definition, + "control" means (i) the power, direct or indirect, to cause the + direction or management of such entity, whether by contract or + otherwise, or (ii) ownership of fifty percent (50%) or more of the + outstanding shares, or (iii) beneficial ownership of such entity. + + "You" (or "Your") shall mean an individual or Legal Entity + exercising permissions granted by this License. + + "Source" form shall mean the preferred form for making modifications, + including but not limited to software source code, documentation + source, and configuration files. + + "Object" form shall mean any form resulting from mechanical + transformation or translation of a Source form, including but + not limited to compiled object code, generated documentation, + and conversions to other media types. + + "Work" shall mean the work of authorship, whether in Source or + Object form, made available under the License, as indicated by a + copyright notice that is included in or attached to the work + (an example is provided in the Appendix below). + + "Derivative Works" shall mean any work, whether in Source or Object + form, that is based on (or derived from) the Work and for which the + editorial revisions, annotations, elaborations, or other modifications + represent, as a whole, an original work of authorship. For the purposes + of this License, Derivative Works shall not include works that remain + separable from, or merely link (or bind by name) to the interfaces of, + the Work and Derivative Works thereof. + + "Contribution" shall mean any work of authorship, including + the original version of the Work and any modifications or additions + to that Work or Derivative Works thereof, that is intentionally + submitted to Licensor for inclusion in the Work by the copyright owner + or by an individual or Legal Entity authorized to submit on behalf of + the copyright owner. For the purposes of this definition, "submitted" + means any form of electronic, verbal, or written communication sent + to the Licensor or its representatives, including but not limited to + communication on electronic mailing lists, source code control systems, + and issue tracking systems that are managed by, or on behalf of, the + Licensor for the purpose of discussing and improving the Work, but + excluding communication that is conspicuously marked or otherwise + designated in writing by the copyright owner as "Not a Contribution." + + "Contributor" shall mean Licensor and any individual or Legal Entity + on behalf of whom a Contribution has been received by Licensor and + subsequently incorporated within the Work. + + 2. Grant of Copyright License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + copyright license to reproduce, prepare Derivative Works of, + publicly display, publicly perform, sublicense, and distribute the + Work and such Derivative Works in Source or Object form. + + 3. Grant of Patent License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + (except as stated in this section) patent license
[GitHub] nifi pull request #3214: NIFI-5890 Added a unit test that proves that 1.9 fi...
Github user MikeThomsen commented on a diff in the pull request:
https://github.com/apache/nifi/pull/3214#discussion_r240980560
--- Diff:
nifi-nar-bundles/nifi-elasticsearch-bundle/nifi-elasticsearch-processors/src/main/java/org/apache/nifi/processors/elasticsearch/PutElasticsearchHttpRecord.java
---
@@ -404,7 +404,7 @@ public void onTrigger(final ProcessContext context,
final ProcessSession session
reason =
itemNode.findPath("reason").asText();
}
errorReason = reason;
-logger.error("Failed to process {}
due to {}, transferring to failure",
+talogger.error("Failed to process
{} due to {}, transferring to failure",
--- End diff --
Yeah, definitely not. Thanks!
---
[GitHub] nifi pull request #3214: NIFI-5890 Added a unit test that proves that 1.9 fi...
GitHub user MikeThomsen opened a pull request: https://github.com/apache/nifi/pull/3214 NIFI-5890 Added a unit test that proves that 1.9 fixes a bug in the d⦠â¦ate conversion behavior in 1.8. Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/MikeThomsen/nifi NIFI-5890 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/3214.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3214 commit ef8bac12ba38d6eaeae6ccb45238c2046126a8d4 Author: Mike Thomsen Date: 2018-12-11T16:18:52Z NIFI-5890 Added a unit test that proves that 1.9 fixes a bug in the date conversion behavior in 1.8. ---
[GitHub] nifi pull request #3105: NIFI-5621: Added Cassandra connection provider serv...
Github user MikeThomsen commented on a diff in the pull request: https://github.com/apache/nifi/pull/3105#discussion_r240660225 --- Diff: nifi-nar-bundles/nifi-cassandra-bundle/nifi-cassandra-services-nar/src/main/resources/META-INF/NOTICE --- @@ -0,0 +1,291 @@ +nifi-cassandra-services-nar +Copyright 2016 The Apache Software Foundation + +This product includes software developed at +The Apache Software Foundation (http://www.apache.org/). + +** +Apache Software License v2 +** + +The following binary components are provided under the Apache Software License v2 + + (ASLv2) DataStax Java Driver for Apache Cassandra - Core + The following NOTICE information applies: +DataStax Java Driver for Apache Cassandra - Core +Copyright (C) 2012-2017 DataStax Inc. + + (ASLv2) Jackson JSON processor +The following NOTICE information applies: + # Jackson JSON processor + + Jackson is a high-performance, Free/Open Source JSON processing library. + It was originally written by Tatu Saloranta (tatu.salora...@iki.fi), and has + been in development since 2007. + It is currently developed by a community of developers, as well as supported + commercially by FasterXML.com. + + ## Licensing + + Jackson core and extension components may licensed under different licenses. + To find the details that apply to this artifact see the accompanying LICENSE file. + For more information, including possible other licensing options, contact + FasterXML.com (http://fasterxml.com). + + ## Credits + + A list of contributors may be found from CREDITS file, which is included + in some artifacts (usually source distributions); but is always available + from the source code management (SCM) system project uses. + + (ASLv2) Apache Commons Codec +The following NOTICE information applies: + Apache Commons Codec + Copyright 2002-2014 The Apache Software Foundation + + src/test/org/apache/commons/codec/language/DoubleMetaphoneTest.java + contains test data from http://aspell.net/test/orig/batch0.tab. + Copyright (C) 2002 Kevin Atkinson (kev...@gnu.org) + + === + + The content of package org.apache.commons.codec.language.bm has been translated + from the original php source code available at http://stevemorse.org/phoneticinfo.htm + with permission from the original authors. + Original source copyright: + Copyright (c) 2008 Alexander Beider & Stephen P. Morse. + + (ASLv2) Apache Commons Lang +The following NOTICE information applies: + Apache Commons Lang + Copyright 2001-2017 The Apache Software Foundation + + This product includes software from the Spring Framework, + under the Apache License 2.0 (see: StringUtils.containsWhitespace()) + + (ASLv2) Guava +The following NOTICE information applies: + Guava + Copyright 2015 The Guava Authors + + (ASLv2) JSON-SMART +The following NOTICE information applies: +Copyright 2011 JSON-SMART authors + + (ASLv2) Dropwizard Metrics +The following NOTICE information applies: + Copyright (c) 2010-2013 Coda Hale, Yammer.com + + (ASLv2) The Netty Project + The following NOTICE information applies: + Copyright 2014 The Netty Project + --- + This product contains the extensions to Java Collections Framework which has + been derived from the works by JSR-166 EG, Doug Lea, and Jason T. Greene: + + * LICENSE: + * license/LICENSE.jsr166y.txt (Public Domain) + * HOMEPAGE: + * http://gee.cs.oswego.edu/cgi-bin/viewcvs.cgi/jsr166/ + * http://viewvc.jboss.org/cgi-bin/viewvc.cgi/jbosscache/experimental/jsr166/ + + This product contains a modified version of Robert Harder's Public Domain + Base64 Encoder and Decoder, which can be obtained at: + + * LICENSE: + * license/LICENSE.base64.txt (Public Domain) + * HOMEPAGE: + * http://iharder.sourceforge.net/current/java/base64/ + + This product contains a modified portion of 'Webbit', an event based + WebSocket and HTTP server, which can be obtained at: + + * LICENSE: + * license/LICENSE.webbit.txt (BSD License) + * HOMEPAGE: + * https://github.com/joewalnes/webbit + + This product contains a modified portion of 'SLF4J', a simple logging + fa
[GitHub] nifi pull request #3105: NIFI-5621: Added Cassandra connection provider serv...
Github user MikeThomsen commented on a diff in the pull request: https://github.com/apache/nifi/pull/3105#discussion_r240659974 --- Diff: nifi-nar-bundles/nifi-cassandra-bundle/nifi-cassandra-services-nar/src/main/resources/META-INF/LICENSE --- @@ -0,0 +1,369 @@ + + Apache License + Version 2.0, January 2004 +http://www.apache.org/licenses/ + + TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + + 1. Definitions. + + "License" shall mean the terms and conditions for use, reproduction, + and distribution as defined by Sections 1 through 9 of this document. + + "Licensor" shall mean the copyright owner or entity authorized by + the copyright owner that is granting the License. + + "Legal Entity" shall mean the union of the acting entity and all + other entities that control, are controlled by, or are under common + control with that entity. For the purposes of this definition, + "control" means (i) the power, direct or indirect, to cause the + direction or management of such entity, whether by contract or + otherwise, or (ii) ownership of fifty percent (50%) or more of the + outstanding shares, or (iii) beneficial ownership of such entity. + + "You" (or "Your") shall mean an individual or Legal Entity + exercising permissions granted by this License. + + "Source" form shall mean the preferred form for making modifications, + including but not limited to software source code, documentation + source, and configuration files. + + "Object" form shall mean any form resulting from mechanical + transformation or translation of a Source form, including but + not limited to compiled object code, generated documentation, + and conversions to other media types. + + "Work" shall mean the work of authorship, whether in Source or + Object form, made available under the License, as indicated by a + copyright notice that is included in or attached to the work + (an example is provided in the Appendix below). + + "Derivative Works" shall mean any work, whether in Source or Object + form, that is based on (or derived from) the Work and for which the + editorial revisions, annotations, elaborations, or other modifications + represent, as a whole, an original work of authorship. For the purposes + of this License, Derivative Works shall not include works that remain + separable from, or merely link (or bind by name) to the interfaces of, + the Work and Derivative Works thereof. + + "Contribution" shall mean any work of authorship, including + the original version of the Work and any modifications or additions + to that Work or Derivative Works thereof, that is intentionally + submitted to Licensor for inclusion in the Work by the copyright owner + or by an individual or Legal Entity authorized to submit on behalf of + the copyright owner. For the purposes of this definition, "submitted" + means any form of electronic, verbal, or written communication sent + to the Licensor or its representatives, including but not limited to + communication on electronic mailing lists, source code control systems, + and issue tracking systems that are managed by, or on behalf of, the + Licensor for the purpose of discussing and improving the Work, but + excluding communication that is conspicuously marked or otherwise + designated in writing by the copyright owner as "Not a Contribution." + + "Contributor" shall mean Licensor and any individual or Legal Entity + on behalf of whom a Contribution has been received by Licensor and + subsequently incorporated within the Work. + + 2. Grant of Copyright License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + copyright license to reproduce, prepare Derivative Works of, + publicly display, publicly perform, sublicense, and distribute the + Work and such Derivative Works in Source or Object form. + + 3. Grant of Patent License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + (except as stated in this section) patent license
[GitHub] nifi pull request #3105: NIFI-5621: Added Cassandra connection provider serv...
Github user MikeThomsen commented on a diff in the pull request: https://github.com/apache/nifi/pull/3105#discussion_r240587144 --- Diff: nifi-nar-bundles/nifi-cassandra-bundle/nifi-cassandra-services-api-nar/src/main/resources/META-INF/NOTICE --- @@ -0,0 +1,226 @@ +nifi-cassandra-services-api-nar +Copyright 2016 The Apache Software Foundation + +This product includes software developed at +The Apache Software Foundation (http://www.apache.org/). + +** +Apache Software License v2 +** + + (ASLv2) The Netty Project +The following NOTICE information applies: + Copyright 2014 The Netty Project + --- + This product contains the extensions to Java Collections Framework which has + been derived from the works by JSR-166 EG, Doug Lea, and Jason T. Greene: + + * LICENSE: + * license/LICENSE.jsr166y.txt (Public Domain) + * HOMEPAGE: + * http://gee.cs.oswego.edu/cgi-bin/viewcvs.cgi/jsr166/ + * http://viewvc.jboss.org/cgi-bin/viewvc.cgi/jbosscache/experimental/jsr166/ + + This product contains a modified version of Robert Harder's Public Domain + Base64 Encoder and Decoder, which can be obtained at: + + * LICENSE: + * license/LICENSE.base64.txt (Public Domain) + * HOMEPAGE: + * http://iharder.sourceforge.net/current/java/base64/ + + This product contains a modified portion of 'Webbit', an event based + WebSocket and HTTP server, which can be obtained at: + + * LICENSE: + * license/LICENSE.webbit.txt (BSD License) + * HOMEPAGE: + * https://github.com/joewalnes/webbit + + This product contains a modified portion of 'SLF4J', a simple logging + facade for Java, which can be obtained at: + + * LICENSE: + * license/LICENSE.slf4j.txt (MIT License) + * HOMEPAGE: + * http://www.slf4j.org/ + + This product contains a modified portion of 'Apache Harmony', an open source + Java SE, which can be obtained at: + + * LICENSE: + * license/LICENSE.harmony.txt (Apache License 2.0) + * HOMEPAGE: + * http://archive.apache.org/dist/harmony/ + + This product contains a modified portion of 'jbzip2', a Java bzip2 compression + and decompression library written by Matthew J. Francis. It can be obtained at: + + * LICENSE: + * license/LICENSE.jbzip2.txt (MIT License) + * HOMEPAGE: + * https://code.google.com/p/jbzip2/ + + This product contains a modified portion of 'libdivsufsort', a C API library to construct + the suffix array and the Burrows-Wheeler transformed string for any input string of + a constant-size alphabet written by Yuta Mori. It can be obtained at: + + * LICENSE: + * license/LICENSE.libdivsufsort.txt (MIT License) + * HOMEPAGE: + * https://github.com/y-256/libdivsufsort + + This product contains a modified portion of Nitsan Wakart's 'JCTools', Java Concurrency Tools for the JVM, +which can be obtained at: + + * LICENSE: + * license/LICENSE.jctools.txt (ASL2 License) + * HOMEPAGE: + * https://github.com/JCTools/JCTools + + This product optionally depends on 'JZlib', a re-implementation of zlib in + pure Java, which can be obtained at: + + * LICENSE: + * license/LICENSE.jzlib.txt (BSD style License) + * HOMEPAGE: + * http://www.jcraft.com/jzlib/ + + This product optionally depends on 'Compress-LZF', a Java library for encoding and + decoding data in LZF format, written by Tatu Saloranta. It can be obtained at: + + * LICENSE: + * license/LICENSE.compress-lzf.txt (Apache License 2.0) + * HOMEPAGE: + * https://github.com/ning/compress + + This product optionally depends on 'lz4', a LZ4 Java compression + and decompression library written by Adrien Grand. It can be obtained at: + + * LICENSE: + * license/LICENSE.lz4.txt (Apache License 2.0) + * HOMEPAGE
[GitHub] nifi pull request #3105: NIFI-5621: Added Cassandra connection provider serv...
Github user MikeThomsen commented on a diff in the pull request: https://github.com/apache/nifi/pull/3105#discussion_r240587170 --- Diff: nifi-nar-bundles/nifi-cassandra-bundle/nifi-cassandra-services-api-nar/src/main/resources/META-INF/NOTICE --- @@ -0,0 +1,226 @@ +nifi-cassandra-services-api-nar +Copyright 2016 The Apache Software Foundation + +This product includes software developed at +The Apache Software Foundation (http://www.apache.org/). + +** +Apache Software License v2 +** + + (ASLv2) The Netty Project +The following NOTICE information applies: + Copyright 2014 The Netty Project + --- + This product contains the extensions to Java Collections Framework which has + been derived from the works by JSR-166 EG, Doug Lea, and Jason T. Greene: + + * LICENSE: + * license/LICENSE.jsr166y.txt (Public Domain) + * HOMEPAGE: + * http://gee.cs.oswego.edu/cgi-bin/viewcvs.cgi/jsr166/ + * http://viewvc.jboss.org/cgi-bin/viewvc.cgi/jbosscache/experimental/jsr166/ + + This product contains a modified version of Robert Harder's Public Domain + Base64 Encoder and Decoder, which can be obtained at: + + * LICENSE: + * license/LICENSE.base64.txt (Public Domain) + * HOMEPAGE: + * http://iharder.sourceforge.net/current/java/base64/ + + This product contains a modified portion of 'Webbit', an event based + WebSocket and HTTP server, which can be obtained at: + + * LICENSE: + * license/LICENSE.webbit.txt (BSD License) + * HOMEPAGE: + * https://github.com/joewalnes/webbit + + This product contains a modified portion of 'SLF4J', a simple logging + facade for Java, which can be obtained at: + + * LICENSE: + * license/LICENSE.slf4j.txt (MIT License) + * HOMEPAGE: + * http://www.slf4j.org/ + + This product contains a modified portion of 'Apache Harmony', an open source + Java SE, which can be obtained at: + + * LICENSE: + * license/LICENSE.harmony.txt (Apache License 2.0) + * HOMEPAGE: + * http://archive.apache.org/dist/harmony/ + + This product contains a modified portion of 'jbzip2', a Java bzip2 compression + and decompression library written by Matthew J. Francis. It can be obtained at: + + * LICENSE: + * license/LICENSE.jbzip2.txt (MIT License) + * HOMEPAGE: + * https://code.google.com/p/jbzip2/ + + This product contains a modified portion of 'libdivsufsort', a C API library to construct + the suffix array and the Burrows-Wheeler transformed string for any input string of + a constant-size alphabet written by Yuta Mori. It can be obtained at: + + * LICENSE: + * license/LICENSE.libdivsufsort.txt (MIT License) + * HOMEPAGE: + * https://github.com/y-256/libdivsufsort + + This product contains a modified portion of Nitsan Wakart's 'JCTools', Java Concurrency Tools for the JVM, +which can be obtained at: + + * LICENSE: + * license/LICENSE.jctools.txt (ASL2 License) + * HOMEPAGE: + * https://github.com/JCTools/JCTools + + This product optionally depends on 'JZlib', a re-implementation of zlib in + pure Java, which can be obtained at: + + * LICENSE: + * license/LICENSE.jzlib.txt (BSD style License) + * HOMEPAGE: + * http://www.jcraft.com/jzlib/ + + This product optionally depends on 'Compress-LZF', a Java library for encoding and + decoding data in LZF format, written by Tatu Saloranta. It can be obtained at: + + * LICENSE: + * license/LICENSE.compress-lzf.txt (Apache License 2.0) + * HOMEPAGE: + * https://github.com/ning/compress + + This product optionally depends on 'lz4', a LZ4 Java compression + and decompression library written by Adrien Grand. It can be obtained at: + + * LICENSE: + * license/LICENSE.lz4.txt (Apache License 2.0) + * HOMEPAGE
[GitHub] nifi pull request #3105: NIFI-5621: Added Cassandra connection provider serv...
Github user MikeThomsen commented on a diff in the pull request: https://github.com/apache/nifi/pull/3105#discussion_r240584516 --- Diff: nifi-nar-bundles/nifi-cassandra-bundle/nifi-cassandra-services-api-nar/src/main/resources/META-INF/LICENSE --- @@ -0,0 +1,266 @@ + --- End diff -- +1 LGTM ---
[GitHub] nifi pull request #3105: NIFI-5621: Added Cassandra connection provider serv...
Github user MikeThomsen commented on a diff in the pull request: https://github.com/apache/nifi/pull/3105#discussion_r240587016 --- Diff: nifi-nar-bundles/nifi-cassandra-bundle/nifi-cassandra-services-api-nar/src/main/resources/META-INF/NOTICE --- @@ -0,0 +1,226 @@ +nifi-cassandra-services-api-nar +Copyright 2016 The Apache Software Foundation + +This product includes software developed at +The Apache Software Foundation (http://www.apache.org/). + +** +Apache Software License v2 +** + + (ASLv2) The Netty Project +The following NOTICE information applies: + Copyright 2014 The Netty Project + --- + This product contains the extensions to Java Collections Framework which has + been derived from the works by JSR-166 EG, Doug Lea, and Jason T. Greene: + + * LICENSE: + * license/LICENSE.jsr166y.txt (Public Domain) + * HOMEPAGE: + * http://gee.cs.oswego.edu/cgi-bin/viewcvs.cgi/jsr166/ + * http://viewvc.jboss.org/cgi-bin/viewvc.cgi/jbosscache/experimental/jsr166/ + + This product contains a modified version of Robert Harder's Public Domain + Base64 Encoder and Decoder, which can be obtained at: + + * LICENSE: + * license/LICENSE.base64.txt (Public Domain) + * HOMEPAGE: + * http://iharder.sourceforge.net/current/java/base64/ + + This product contains a modified portion of 'Webbit', an event based + WebSocket and HTTP server, which can be obtained at: + + * LICENSE: + * license/LICENSE.webbit.txt (BSD License) + * HOMEPAGE: + * https://github.com/joewalnes/webbit + + This product contains a modified portion of 'SLF4J', a simple logging + facade for Java, which can be obtained at: + + * LICENSE: + * license/LICENSE.slf4j.txt (MIT License) + * HOMEPAGE: + * http://www.slf4j.org/ + + This product contains a modified portion of 'Apache Harmony', an open source + Java SE, which can be obtained at: + + * LICENSE: + * license/LICENSE.harmony.txt (Apache License 2.0) + * HOMEPAGE: + * http://archive.apache.org/dist/harmony/ + + This product contains a modified portion of 'jbzip2', a Java bzip2 compression + and decompression library written by Matthew J. Francis. It can be obtained at: + + * LICENSE: + * license/LICENSE.jbzip2.txt (MIT License) + * HOMEPAGE: + * https://code.google.com/p/jbzip2/ + + This product contains a modified portion of 'libdivsufsort', a C API library to construct + the suffix array and the Burrows-Wheeler transformed string for any input string of + a constant-size alphabet written by Yuta Mori. It can be obtained at: + + * LICENSE: + * license/LICENSE.libdivsufsort.txt (MIT License) + * HOMEPAGE: + * https://github.com/y-256/libdivsufsort + + This product contains a modified portion of Nitsan Wakart's 'JCTools', Java Concurrency Tools for the JVM, +which can be obtained at: + + * LICENSE: + * license/LICENSE.jctools.txt (ASL2 License) + * HOMEPAGE: + * https://github.com/JCTools/JCTools + + This product optionally depends on 'JZlib', a re-implementation of zlib in + pure Java, which can be obtained at: + + * LICENSE: + * license/LICENSE.jzlib.txt (BSD style License) + * HOMEPAGE: + * http://www.jcraft.com/jzlib/ + + This product optionally depends on 'Compress-LZF', a Java library for encoding and + decoding data in LZF format, written by Tatu Saloranta. It can be obtained at: + + * LICENSE: + * license/LICENSE.compress-lzf.txt (Apache License 2.0) + * HOMEPAGE: + * https://github.com/ning/compress + + This product optionally depends on 'lz4', a LZ4 Java compression + and decompression library written by Adrien Grand. It can be obtained at: + + * LICENSE: + * license/LICENSE.lz4.txt (Apache License 2.0) + * HOMEPAGE
[GitHub] nifi pull request #3105: NIFI-5621: Added Cassandra connection provider serv...
Github user MikeThomsen commented on a diff in the pull request: https://github.com/apache/nifi/pull/3105#discussion_r240586483 --- Diff: nifi-nar-bundles/nifi-cassandra-bundle/nifi-cassandra-services-api-nar/src/main/resources/META-INF/NOTICE --- @@ -0,0 +1,226 @@ +nifi-cassandra-services-api-nar +Copyright 2016 The Apache Software Foundation + +This product includes software developed at +The Apache Software Foundation (http://www.apache.org/). + +** +Apache Software License v2 +** + + (ASLv2) The Netty Project --- End diff -- Netty looks good. ---
[GitHub] nifi issue #3203: NIFI-5871 ignore UUID attribute when copying flow file att...
Github user MikeThomsen commented on the issue: https://github.com/apache/nifi/pull/3203 AFAIK, there is no reason why uuid should be inherited by a flowfile even in a parent-child scenario since the child is a new flowfile branched off. ---
[GitHub] nifi issue #3203: NIFI-5871 ignore UUID attribute when copying flow file att...
Github user MikeThomsen commented on the issue: https://github.com/apache/nifi/pull/3203 Approval still stands. Will wait to see what Travis does. ---
[GitHub] nifi pull request #3203: NIFI-5871 ignore UUID attribute when copying flow f...
Github user MikeThomsen commented on a diff in the pull request: https://github.com/apache/nifi/pull/3203#discussion_r239160879 --- Diff: nifi-mock/src/main/java/org/apache/nifi/util/MockProcessSession.java --- @@ -496,7 +496,10 @@ public MockFlowFile putAllAttributes(FlowFile flowFile, final Map attrCopy = new HashMap<>(); +attrCopy.putAll(attrs); --- End diff -- Only change I'd make here is putting that into the constructor. Other than that LGTM. ---
[GitHub] nifi issue #3105: NIFI-5621: Added Cassandra connection provider service
Github user MikeThomsen commented on the issue: https://github.com/apache/nifi/pull/3105 @zenfenan you still working this? ---
[GitHub] nifi pull request #3165: NIFI-5318 Implement NiFi test harness
Github user MikeThomsen commented on a diff in the pull request: https://github.com/apache/nifi/pull/3165#discussion_r236045789 --- Diff: nifi-testharness/src/test/resources/sample_technology_rss.xml --- @@ -0,0 +1,28 @@ + + --- End diff -- Yes, it absolutely must be changed because the BBC is asserting its IP rights here: ``` ``` ---
[GitHub] nifi issue #3165: NIFI-5318 Implement NiFi test harness
Github user MikeThomsen commented on the issue: https://github.com/apache/nifi/pull/3165 Will try to get back into it soon. BTW, thanks for putting it all back onto a branch. ---
[GitHub] nifi issue #3038: [NIFI-5646] Dockerfile.alpine: Added.
Github user MikeThomsen commented on the issue: https://github.com/apache/nifi/pull/3038 @apiri What do you think? > An alternative would be to put the Dockerfiles into subdirectories if the above is a limitation in the plugin. I think an alpine build would be a nice thing to have. ---
[GitHub] nifi pull request #3105: NIFI-5621: Added Cassandra connection provider serv...
Github user MikeThomsen commented on a diff in the pull request: https://github.com/apache/nifi/pull/3105#discussion_r236041500 --- Diff: nifi-nar-bundles/nifi-cassandra-bundle/nifi-cassandra-services-nar/src/main/resources/META-INF/NOTICE --- @@ -0,0 +1,327 @@ +nifi-cassandra-services-nar --- End diff -- Missing logback dependency reference ---
[GitHub] nifi pull request #3105: NIFI-5621: Added Cassandra connection provider serv...
Github user MikeThomsen commented on a diff in the pull request: https://github.com/apache/nifi/pull/3105#discussion_r236041495 --- Diff: nifi-nar-bundles/nifi-cassandra-bundle/nifi-cassandra-services-nar/src/main/resources/META-INF/NOTICE --- @@ -0,0 +1,327 @@ +nifi-cassandra-services-nar +Copyright 2016 The Apache Software Foundation + +This product includes software developed at +The Apache Software Foundation (http://www.apache.org/). + +** +Apache Software License v2 +** + +The following binary components are provided under the Apache Software License v2 + + (ASLv2) DataStax Java Driver for Apache Cassandra - Core + The following NOTICE information applies: +DataStax Java Driver for Apache Cassandra - Core +Copyright (C) 2012-2017 DataStax Inc. + + (ASLv2) Apache Avro --- End diff -- It can be removed. ---
[GitHub] nifi pull request #3105: NIFI-5621: Added Cassandra connection provider serv...
Github user MikeThomsen commented on a diff in the pull request: https://github.com/apache/nifi/pull/3105#discussion_r236041589 --- Diff: nifi-nar-bundles/nifi-cassandra-bundle/nifi-cassandra-services-nar/src/main/resources/META-INF/NOTICE --- @@ -0,0 +1,327 @@ +nifi-cassandra-services-nar --- End diff -- Missing notices for this entire branch: ``` [INFO] | | +- org.antlr:antlr-runtime:jar:3.5.2:compile [INFO] | | +- com.jayway.jsonpath:json-path:jar:2.4.0:compile [INFO] | | | \- net.minidev:json-smart:jar:2.3:compile [INFO] | | | \- net.minidev:accessors-smart:jar:1.2:compile ``` ---
[GitHub] nifi pull request #3105: NIFI-5621: Added Cassandra connection provider serv...
Github user MikeThomsen commented on a diff in the pull request: https://github.com/apache/nifi/pull/3105#discussion_r236041556 --- Diff: nifi-nar-bundles/nifi-cassandra-bundle/nifi-cassandra-services-nar/src/main/resources/META-INF/NOTICE --- @@ -0,0 +1,327 @@ +nifi-cassandra-services-nar --- End diff -- Missing notice for Netty ---
[GitHub] nifi pull request #3105: NIFI-5621: Added Cassandra connection provider serv...
Github user MikeThomsen commented on a diff in the pull request: https://github.com/apache/nifi/pull/3105#discussion_r236041536 --- Diff: nifi-nar-bundles/nifi-cassandra-bundle/nifi-cassandra-services-nar/src/main/resources/META-INF/NOTICE --- @@ -0,0 +1,327 @@ +nifi-cassandra-services-nar --- End diff -- Missing notice for groupId `org.ow2.asm` ---
[GitHub] nifi pull request #3105: NIFI-5621: Added Cassandra connection provider serv...
Github user MikeThomsen commented on a diff in the pull request: https://github.com/apache/nifi/pull/3105#discussion_r236041568 --- Diff: nifi-nar-bundles/nifi-cassandra-bundle/nifi-cassandra-services-nar/src/main/resources/META-INF/NOTICE --- @@ -0,0 +1,327 @@ +nifi-cassandra-services-nar --- End diff -- Missing notice for `de.svenkubiak:jBCrypt` ---
[GitHub] nifi pull request #3105: NIFI-5621: Added Cassandra connection provider serv...
Github user MikeThomsen commented on a diff in the pull request: https://github.com/apache/nifi/pull/3105#discussion_r236041602 --- Diff: nifi-nar-bundles/nifi-cassandra-bundle/nifi-cassandra-services-nar/src/main/resources/META-INF/NOTICE --- @@ -0,0 +1,327 @@ +nifi-cassandra-services-nar +Copyright 2016 The Apache Software Foundation + +This product includes software developed at +The Apache Software Foundation (http://www.apache.org/). + +** +Apache Software License v2 +** + +The following binary components are provided under the Apache Software License v2 + + (ASLv2) DataStax Java Driver for Apache Cassandra - Core + The following NOTICE information applies: +DataStax Java Driver for Apache Cassandra - Core +Copyright (C) 2012-2017 DataStax Inc. + + (ASLv2) Apache Avro + The following NOTICE information applies: +Apache Avro +Copyright 2009-2017 The Apache Software Foundation + + (ASLv2) Jackson JSON processor --- End diff -- It is referenced in `mvn dependency:tree` on your services NAR. ---
[GitHub] nifi issue #3105: NIFI-5621: Added Cassandra connection provider service
Github user MikeThomsen commented on the issue: https://github.com/apache/nifi/pull/3105 You also need to verify the LICENSE as well. ---
[GitHub] nifi pull request #3105: NIFI-5621: Added Cassandra connection provider serv...
Github user MikeThomsen commented on a diff in the pull request: https://github.com/apache/nifi/pull/3105#discussion_r236041517 --- Diff: nifi-nar-bundles/nifi-cassandra-bundle/nifi-cassandra-services-nar/src/main/resources/META-INF/NOTICE --- @@ -0,0 +1,327 @@ +nifi-cassandra-services-nar --- End diff -- Missing notice for `io.dropwizard.metrics:metrics-core:jar:3.2.2:compile` ---
[GitHub] nifi issue #3105: NIFI-5621: Added Cassandra connection provider service
Github user MikeThomsen commented on the issue: https://github.com/apache/nifi/pull/3105 @zenfenan I have a table with uuid primary key. How do I set a value for an insert in our processors for that field type (if you don't know can you recommend a schema for testing)? ---
[GitHub] nifi issue #3105: NIFI-5621: Added Cassandra connection provider service
Github user MikeThomsen commented on the issue: https://github.com/apache/nifi/pull/3105 Reviewing... ---
[GitHub] nifi issue #3167: NIFI-5812: Marked Database processors as 'PrimaryNodeOnly'
Github user MikeThomsen commented on the issue: https://github.com/apache/nifi/pull/3167 @zenfenan If you're too busy, I can make that change and merge. ---
[GitHub] nifi pull request #3167: NIFI-5812: Marked Database processors as 'PrimaryNo...
Github user MikeThomsen commented on a diff in the pull request: https://github.com/apache/nifi/pull/3167#discussion_r234999192 --- Diff: nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/GenerateTableFetch.java --- @@ -114,6 +114,7 @@ + "max value for max value columns. Properties should be added in the format `initial.maxvalue.`. This value is only used the first time " + "the table is accessed (when a Maximum Value Column is specified). In the case of incoming connections, the value is only used the first time for each table " + "specified in the flow files.") +@PrimaryNodeOnly --- End diff -- Agreed ---
[GitHub] nifi pull request #3011: NIFI-5333 Added GetMongoRecord.
Github user MikeThomsen commented on a diff in the pull request:
https://github.com/apache/nifi/pull/3011#discussion_r234461028
--- Diff:
nifi-nar-bundles/nifi-mongodb-bundle/nifi-mongodb-processors/src/main/java/org/apache/nifi/processors/mongodb/GetMongoRecord.java
---
@@ -0,0 +1,213 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.nifi.processors.mongodb;
+
+import com.mongodb.client.FindIterable;
+import com.mongodb.client.MongoCollection;
+import com.mongodb.client.MongoCursor;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.WritesAttribute;
+import org.apache.nifi.annotation.behavior.WritesAttributes;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.expression.ExpressionLanguageScope;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.mongodb.MongoDBClientService;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.schema.access.SchemaNotFoundException;
+import org.apache.nifi.serialization.RecordSetWriter;
+import org.apache.nifi.serialization.RecordSetWriterFactory;
+import org.apache.nifi.serialization.record.MapRecord;
+import org.apache.nifi.serialization.record.Record;
+import org.apache.nifi.serialization.record.RecordSchema;
+import org.bson.Document;
+import org.bson.types.ObjectId;
+
+import java.io.OutputStream;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+
+@CapabilityDescription("A record-based version of GetMongo that uses the
Record writers to write the MongoDB result set.")
+@Tags({"mongo", "mongodb", "get", "fetch", "record", "json"})
+@InputRequirement(InputRequirement.Requirement.INPUT_ALLOWED)
+@WritesAttributes({
+@WritesAttribute(attribute = GetMongo.DB_NAME, description = "The
database where the results came from."),
+@WritesAttribute(attribute = GetMongo.COL_NAME, description = "The
collection where the results came from.")
+})
+public class GetMongoRecord extends AbstractMongoQueryProcessor {
+public static final PropertyDescriptor CLIENT = new
PropertyDescriptor.Builder()
--- End diff --
Just pushed an update that fixed some bad behavior. Ran the whole IT suite
against dockerized Mongo and no issues.
---
[GitHub] nifi pull request #3011: NIFI-5333 Added GetMongoRecord.
Github user MikeThomsen commented on a diff in the pull request:
https://github.com/apache/nifi/pull/3011#discussion_r234457340
--- Diff:
nifi-nar-bundles/nifi-mongodb-bundle/nifi-mongodb-processors/src/main/java/org/apache/nifi/processors/mongodb/GetMongoRecord.java
---
@@ -0,0 +1,213 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.nifi.processors.mongodb;
+
+import com.mongodb.client.FindIterable;
+import com.mongodb.client.MongoCollection;
+import com.mongodb.client.MongoCursor;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.WritesAttribute;
+import org.apache.nifi.annotation.behavior.WritesAttributes;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.expression.ExpressionLanguageScope;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.mongodb.MongoDBClientService;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.apache.nifi.schema.access.SchemaNotFoundException;
+import org.apache.nifi.serialization.RecordSetWriter;
+import org.apache.nifi.serialization.RecordSetWriterFactory;
+import org.apache.nifi.serialization.record.MapRecord;
+import org.apache.nifi.serialization.record.Record;
+import org.apache.nifi.serialization.record.RecordSchema;
+import org.bson.Document;
+import org.bson.types.ObjectId;
+
+import java.io.OutputStream;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+
+@CapabilityDescription("A record-based version of GetMongo that uses the
Record writers to write the MongoDB result set.")
+@Tags({"mongo", "mongodb", "get", "fetch", "record", "json"})
+@InputRequirement(InputRequirement.Requirement.INPUT_ALLOWED)
+@WritesAttributes({
+@WritesAttribute(attribute = GetMongo.DB_NAME, description = "The
database where the results came from."),
+@WritesAttribute(attribute = GetMongo.COL_NAME, description = "The
collection where the results came from.")
+})
+public class GetMongoRecord extends AbstractMongoQueryProcessor {
+public static final PropertyDescriptor CLIENT = new
PropertyDescriptor.Builder()
--- End diff --
Fixed. Couldn't test locally because I'm having connectivity issues for one
of the maven dependencies in nifi-hdfs-processors.
---
[GitHub] nifi issue #3011: NIFI-5333 Added GetMongoRecord.
Github user MikeThomsen commented on the issue: https://github.com/apache/nifi/pull/3011 Thanks @zenfenan. Could you and/or @mattyb149 take a look at 4975 as well sometime? That's the GridFS support. Would really like to give NiFi the ability to work natively with that. ---
[GitHub] nifi pull request #3011: NIFI-5333 Added GetMongoRecord.
Github user MikeThomsen commented on a diff in the pull request: https://github.com/apache/nifi/pull/3011#discussion_r232453403 --- Diff: nifi-nar-bundles/nifi-mongodb-bundle/nifi-mongodb-processors/pom.xml --- @@ -101,6 +101,7 @@ org.apache.nifi nifi-mongodb-client-service-api +1.8.0-SNAPSHOT --- End diff -- Done. ---
[GitHub] nifi pull request #2861: NIFI-5248 Added new Elasticsearch json and record p...
Github user MikeThomsen commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2861#discussion_r229396964
--- Diff:
nifi-nar-bundles/nifi-elasticsearch-bundle/nifi-elasticsearch-restapi-processors/src/main/java/org/apache/nifi/processors/elasticsearch/PutElasticsearchRecord.java
---
@@ -0,0 +1,264 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.processors.elasticsearch;
+
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.elasticsearch.ElasticSearchClientService;
+import org.apache.nifi.elasticsearch.ElasticSearchError;
+import org.apache.nifi.elasticsearch.IndexOperationRequest;
+import org.apache.nifi.expression.ExpressionLanguageScope;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.record.path.FieldValue;
+import org.apache.nifi.record.path.RecordPath;
+import org.apache.nifi.record.path.RecordPathResult;
+import org.apache.nifi.record.path.util.RecordPathCache;
+import org.apache.nifi.record.path.validation.RecordPathValidator;
+import org.apache.nifi.serialization.RecordReader;
+import org.apache.nifi.serialization.RecordReaderFactory;
+import org.apache.nifi.serialization.record.Record;
+import org.apache.nifi.serialization.record.RecordFieldType;
+import org.apache.nifi.serialization.record.util.DataTypeUtils;
+
+import java.io.InputStream;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Optional;
+import java.util.Set;
+
+@InputRequirement(InputRequirement.Requirement.INPUT_REQUIRED)
+@Tags({"json", "elasticsearch", "elasticsearch5", "elasticsearch6", "put",
"index", "record"})
+@CapabilityDescription("A record-aware ElasticSearch put processor that
uses the official Elastic REST client libraries.")
+public class PutElasticsearchRecord extends AbstractProcessor implements
ElasticSearchRestProcessor {
+static final PropertyDescriptor RECORD_READER = new
PropertyDescriptor.Builder()
+.name("put-es-record-reader")
+.displayName("Record Reader")
+.description("The record reader to use for reading incoming
records from flowfiles.")
+.identifiesControllerService(RecordReaderFactory.class)
+.required(true)
+.build();
+
+static final PropertyDescriptor OPERATION_RECORD_PATH = new
PropertyDescriptor.Builder()
--- End diff --
Sorry, early morning and coffee didn't kick in.
Yeah, those values should be removed and they are now. I moved to just
purge nulls and empty strings because they're semantically meaningless AFAICT
in Lucene and ES since they are truthy-false whether they don't exist, are null
or empty. So all gone now if you specify the record path.
---
[GitHub] nifi issue #2861: NIFI-5248 Added new Elasticsearch json and record processo...
Github user MikeThomsen commented on the issue: https://github.com/apache/nifi/pull/2861 @mattyb149 review changes are in. ---
[GitHub] nifi issue #3041: NIFI-5224 Added SolrClientService.
Github user MikeThomsen commented on the issue: https://github.com/apache/nifi/pull/3041 @mattyb149 changes made. ---
[GitHub] nifi pull request #2872: NIFI-5318 Implement NiFi test harness: initial comm...
Github user MikeThomsen commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2872#discussion_r228689270
--- Diff:
nifi-testharness/src/test/java/org/apache/nifi/testharness/samples/Constants.java
---
@@ -0,0 +1,32 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+
+package org.apache.nifi.testharness.samples;
+
+import java.io.File;
+
+public final class Constants {
+
+static final File OUTPUT_DIR = new File("./NiFiTest/NiFiReadTest");
+
+// NOTE: you will have to have the NiFi distribution ZIP placed into
this directory.
+// Its version must be the same as the one referenced in the flow.xml,
otherwise it will not work!
+static final File NIFI_ZIP_DIR = new
File(System.getProperty("user.home"), "Downloads");
--- End diff --
That folder evaluated to `$HOME/Downloads` for my user account when I ran
it. Are you expecting the user to sudo or something?
---
[GitHub] nifi pull request #3041: NIFI-5224 Added SolrClientService.
Github user MikeThomsen commented on a diff in the pull request: https://github.com/apache/nifi/pull/3041#discussion_r228537197 --- Diff: nifi-nar-bundles/nifi-solr-bundle/nifi-solr-processors/pom.xml --- @@ -147,6 +147,12 @@ 2.2.1 test + +org.apache.nifi +nifi-solr-client-api +1.8.0-SNAPSHOT +provided --- End diff -- > The solr-bundle NAR pom also needs to be updated to have a parent of the client API NAR. Could you explain? I added the client api nar to the assemble, rebuilt and was able to bring in a new client service just fine. ---
[GitHub] nifi pull request #2872: NIFI-5318 Implement NiFi test harness: initial comm...
Github user MikeThomsen commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2872#discussion_r228511342
--- Diff:
nifi-testharness/src/test/java/org/apache/nifi/testharness/samples/Constants.java
---
@@ -0,0 +1,32 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+
+package org.apache.nifi.testharness.samples;
+
+import java.io.File;
+
+public final class Constants {
+
+static final File OUTPUT_DIR = new File("./NiFiTest/NiFiReadTest");
+
+// NOTE: you will have to have the NiFi distribution ZIP placed into
this directory.
+// Its version must be the same as the one referenced in the flow.xml,
otherwise it will not work!
+static final File NIFI_ZIP_DIR = new
File(System.getProperty("user.home"), "Downloads");
--- End diff --
Since this is an internal unit test, it should be pointing to the build
output of nifi-assembly.
---
[GitHub] nifi pull request #3041: NIFI-5224 Added SolrClientService.
Github user MikeThomsen commented on a diff in the pull request:
https://github.com/apache/nifi/pull/3041#discussion_r228507330
--- Diff:
nifi-nar-bundles/nifi-solr-bundle/nifi-solr-processors/src/main/java/org/apache/nifi/processors/solr/SolrProcessor.java
---
@@ -176,70 +172,11 @@ protected final KeytabUser getKerberosKeytabUser() {
final protected Collection
customValidate(ValidationContext context) {
final List problems = new ArrayList<>();
-if
(SOLR_TYPE_CLOUD.equals(context.getProperty(SOLR_TYPE).getValue())) {
-final String collection =
context.getProperty(COLLECTION).getValue();
-if (collection == null || collection.trim().isEmpty()) {
-problems.add(new ValidationResult.Builder()
-.subject(COLLECTION.getName())
-.input(collection).valid(false)
-.explanation("A collection must specified for Solr
Type of Cloud")
-.build());
-}
-}
-
-// For solr cloud the location will be the ZooKeeper host:port so
we can't validate the SSLContext, but for standard solr
-// we can validate if the url starts with https we need an
SSLContextService, if it starts with http we can't have an SSLContextService
-if
(SOLR_TYPE_STANDARD.equals(context.getProperty(SOLR_TYPE).getValue())) {
-final String solrLocation =
context.getProperty(SOLR_LOCATION).evaluateAttributeExpressions().getValue();
-if (solrLocation != null) {
-final SSLContextService sslContextService =
context.getProperty(SSL_CONTEXT_SERVICE).asControllerService(SSLContextService.class);
-if (solrLocation.startsWith("https:") && sslContextService
== null) {
-problems.add(new ValidationResult.Builder()
-.subject(SSL_CONTEXT_SERVICE.getDisplayName())
-.valid(false)
-.explanation("an SSLContextService must be
provided when using https")
-.build());
-} else if (solrLocation.startsWith("http:") &&
sslContextService != null) {
-problems.add(new ValidationResult.Builder()
-.subject(SSL_CONTEXT_SERVICE.getDisplayName())
-.valid(false)
-.explanation("an SSLContextService can not be
provided when using http")
-.build());
-}
-}
-}
-
-// Validate that we username and password are provided together,
or that neither are provided
-final String username =
context.getProperty(BASIC_USERNAME).evaluateAttributeExpressions().getValue();
-final String password =
context.getProperty(BASIC_PASSWORD).evaluateAttributeExpressions().getValue();
-
-final boolean basicUsernameProvided =
!StringUtils.isBlank(username);
-final boolean basicPasswordProvided =
!StringUtils.isBlank(password);
-
-if (basicUsernameProvided && !basicPasswordProvided) {
-problems.add(new ValidationResult.Builder()
-.subject(BASIC_PASSWORD.getDisplayName())
-.valid(false)
-.explanation("a password must be provided for the
given username")
-.build());
-}
-
-if (basicPasswordProvided && !basicUsernameProvided) {
-problems.add(new ValidationResult.Builder()
-.subject(BASIC_USERNAME.getDisplayName())
-.valid(false)
-.explanation("a username must be provided for the
given password")
-.build());
-}
-
-// Validate that only kerberos or basic auth can be set, but not
both
-final KerberosCredentialsService kerberosCredentialsService =
context.getProperty(KERBEROS_CREDENTIALS_SERVICE).asControllerService(KerberosCredentialsService.class);
-if (kerberosCredentialsService != null && basicUsernameProvided &&
basicPasswordProvided) {
-problems.add(new ValidationResult.Builder()
-.subject(KERBEROS_CREDENTIALS_SERVICE.getDisplayName())
-.valid(false)
-.explanation("basic auth and kerberos cannot be
configured at the same time")
-.build());
+List _temp = new
ArrayList<>(validateConnectionDetails(context));
+if (_temp.size() == 0 &&
context.getProperty(CLIENT_SERVICE).isSet()) {
--- End diff --
Fixed that too.
---
[GitHub] nifi issue #3105: NIFI-5621: Added Cassandra connection provider service
Github user MikeThomsen commented on the issue: https://github.com/apache/nifi/pull/3105 Easiest way to do the L for your NAR here is this: `mvn dependency:tree | grep -i compile` Provided and test dependencies aren't part of the NAR, so only focus on compile. ---
[GitHub] nifi issue #2872: NIFI-5318 Implement NiFi test harness: initial commit of n...
Github user MikeThomsen commented on the issue: https://github.com/apache/nifi/pull/2872 > Hi @MikeThomsen sorry for the delay, I have been just very busy. Thanks. I've got a lot on my plate as well, but will try to find some time early this week to review. ---
[GitHub] nifi issue #3079: NIFI-5706 : Added ConvertAvroToParquet processor
Github user MikeThomsen commented on the issue: https://github.com/apache/nifi/pull/3079 From Jira: > PutParquet support is limited to HDFS. It's actually not. You can write it to the local file system by supplying a Hadoop configuration that tells it to use that. It's not clear to me that this differs fundamentally from PutParquet other than the output target. Am I missing something? ---
[GitHub] nifi issue #2861: NIFI-5248 Added new Elasticsearch json and record processo...
Github user MikeThomsen commented on the issue: https://github.com/apache/nifi/pull/2861 @mattyb149 anything? ---
[GitHub] nifi pull request #3025: NIFI-5605 Added UpdateCassandra.
Github user MikeThomsen closed the pull request at: https://github.com/apache/nifi/pull/3025 ---
[GitHub] nifi issue #3025: NIFI-5605 Added UpdateCassandra.
Github user MikeThomsen commented on the issue: https://github.com/apache/nifi/pull/3025 Going to close this because I think the consensus is that it should be folded into an existing processor. ---
[GitHub] nifi issue #2856: NIFI-4811 Added two missing entries to the nifi-redis-serv...
Github user MikeThomsen commented on the issue: https://github.com/apache/nifi/pull/2856 @joewitt we should be good to go now. ---
[GitHub] nifi issue #3049: NIFI-5664 Support List in DataTypeUtils#toArray
Github user MikeThomsen commented on the issue: https://github.com/apache/nifi/pull/3049 Cool. FYI, the long term goal is to make controller services the exclusive source of configurations for configurations. That's because a single `MongoClient` is really a connection pool and is--according to Mongo Inc.--thread-safe and intended to be used across an entire app. That should help Mongo users tighten up on how many connections they have open from NiFi, especially a cluster (since each node has N number of MongoClients where N = # of mongo processors). ---
[GitHub] nifi issue #3049: NIFI-5664 Support List in DataTypeUtils#toArray
Github user MikeThomsen commented on the issue: https://github.com/apache/nifi/pull/3049 @snagacarl since this was for the Mongo lookup service, wanted to make sure that you're aware that in 1.8 there will be a slight breaking change. The "configuration" controller service and the lookup service are now cleanly separated so that the lookup service can be cheerfully ignorant of connection details like the URL, SSL and authentication details. That way plugging in a new cluster or something will be faster and easier for Mongo users. ---
[GitHub] nifi issue #3049: NIFI-5664 Support List in DataTypeUtils#toArray
Github user MikeThomsen commented on the issue: https://github.com/apache/nifi/pull/3049 I'll try to review tonight. Been pretty sick. Thanks for the updates @snagacarl ---
[GitHub] nifi pull request #3049: NIFI-5664 Support ArrayList in DataTypeUtils#toArra...
Github user MikeThomsen commented on a diff in the pull request:
https://github.com/apache/nifi/pull/3049#discussion_r223181352
--- Diff:
nifi-commons/nifi-record/src/test/java/org/apache/nifi/serialization/record/TestDataTypeUtils.java
---
@@ -168,6 +168,21 @@ public void testConvertRecordFieldToObject() {
}
+@Test
+public void testToArray() {
+final ArrayList arrayList = new ArrayList<>();
+arrayList.add("Seven");
--- End diff --
This could also be reduced to `final list arrayList =
Arrays.asList("Seven", "Eleven", "Thirteen");`
---
[GitHub] nifi pull request #3049: NIFI-5664 Support ArrayList in DataTypeUtils#toArra...
Github user MikeThomsen commented on a diff in the pull request:
https://github.com/apache/nifi/pull/3049#discussion_r223181320
--- Diff:
nifi-commons/nifi-record/src/main/java/org/apache/nifi/serialization/record/util/DataTypeUtils.java
---
@@ -339,6 +339,11 @@ public static boolean isRecordTypeCompatible(final
Object value) {
return dest;
}
+if (value instanceof ArrayList) {
--- End diff --
I agree with @ottobackwards. It really should not be a specific
implementation.
---
[GitHub] nifi pull request #3049: NIFI-5664 Support ArrayList in DataTypeUtils#toArra...
Github user MikeThomsen commented on a diff in the pull request:
https://github.com/apache/nifi/pull/3049#discussion_r223181332
--- Diff:
nifi-commons/nifi-record/src/test/java/org/apache/nifi/serialization/record/TestDataTypeUtils.java
---
@@ -168,6 +168,21 @@ public void testConvertRecordFieldToObject() {
}
+@Test
+public void testToArray() {
+final ArrayList arrayList = new ArrayList<>();
--- End diff --
Should be `final List`
---
[GitHub] nifi issue #2956: NIFI-5537 Create Neo4J cypher execution processor
Github user MikeThomsen commented on the issue: https://github.com/apache/nifi/pull/2956 @mans2singh before we continue, do you mind joining in [this discussion](http://apache-nifi-developer-list.39713.n7.nabble.com/Graph-database-support-w-NiFi-tp18680p19706.html) that I reopened on the mailing list? ---
[GitHub] nifi issue #3025: NIFI-5605 Added UpdateCassandra.
Github user MikeThomsen commented on the issue: https://github.com/apache/nifi/pull/3025 @zenfenan do you have any opinion on where to put this functionality? ---
[GitHub] nifi issue #3045: Support/nifi 1.7.x
Github user MikeThomsen commented on the issue: https://github.com/apache/nifi/pull/3045 @swatinarvekar if you need support, please subscribe to [one of the mailing lists](http://nifi.apache.org/mailing_lists.html). If this is an attempted code submission, let us know what you're trying to do and we can help you through the process. If not, please close this PR. ---
[GitHub] nifi issue #2420: NIFI-4731
Github user MikeThomsen commented on the issue: https://github.com/apache/nifi/pull/2420 > May I ask what is currently blocking it? @nologic appears to have not addressed all of the review feedback. @joewitt I can rebase and address the author tags. If the tests run, I can merge myself. You want to do that? ---
[GitHub] nifi issue #2546: NIFI-4975 Add GridFS processors
Github user MikeThomsen commented on the issue: https://github.com/apache/nifi/pull/2546 @zenfenan @mattyb149 any chance we could get a review going for this before 1.8? ---
[GitHub] nifi issue #3041: NIFI-5224 Added SolrClientService.
Github user MikeThomsen commented on the issue: https://github.com/apache/nifi/pull/3041 Thanks, @bbende I'll look into those. ---
[GitHub] nifi issue #3015: NIFI-5254 [WIP] Updated to Groovy 2.5.2
Github user MikeThomsen commented on the issue: https://github.com/apache/nifi/pull/3015 @mattyb149 @joewitt Removed the following as compile dependencies because I can't see any valid use for them in our scripting scenarios: * groovy-cli* * groovy-groovydoc * groovy-docgenerator * groovy-swing * groovy-test * groovy-test-junit5 * groovy-testng Think that's it. We might want to release an unofficial test build of NiFi w/ 2.5.X because I'm not 100% sure there's a full guarantee of compatibility between 2.4 and 2.5 ---
[GitHub] nifi pull request #2956: NIFI-5537 Create Neo4J cypher execution processor
Github user MikeThomsen commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2956#discussion_r66870
--- Diff:
nifi-nar-bundles/nifi-neo4j-bundle/nifi-neo4j-processors/src/main/java/org/apache/nifi/processors/neo4j/AbstractNeo4JCypherExecutor.java
---
@@ -0,0 +1,281 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.neo4j;
+
+import java.io.File;
+import java.util.concurrent.TimeUnit;
+
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.annotation.lifecycle.OnStopped;
+import org.apache.nifi.components.AllowableValue;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.PropertyValue;
+import org.apache.nifi.expression.ExpressionLanguageScope;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.neo4j.driver.v1.AuthTokens;
+import org.neo4j.driver.v1.Config;
+import org.neo4j.driver.v1.Config.ConfigBuilder;
+import org.neo4j.driver.v1.Config.LoadBalancingStrategy;
+import org.neo4j.driver.v1.Config.TrustStrategy;
+import org.neo4j.driver.v1.Driver;
+import org.neo4j.driver.v1.GraphDatabase;
+
+/**
+ * Abstract base class for Neo4JCypherExecutor processors
+ */
+abstract class AbstractNeo4JCypherExecutor extends AbstractProcessor {
+
+protected static final PropertyDescriptor QUERY = new
PropertyDescriptor.Builder()
+.name("neo4J-query")
+.displayName("Neo4J Query")
+.description("Specifies the Neo4j Query.")
+.required(true)
+
.expressionLanguageSupported(ExpressionLanguageScope.FLOWFILE_ATTRIBUTES)
+.addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor CONNECTION_URL = new
PropertyDescriptor.Builder()
+.name("neo4j-connection-url")
+.displayName("Neo4j Connection URL")
+.description("Neo4J endpoing to connect to.")
+.required(true)
+.defaultValue("bolt://localhost:7687")
+
.expressionLanguageSupported(ExpressionLanguageScope.VARIABLE_REGISTRY)
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor USERNAME = new
PropertyDescriptor.Builder()
+.name("neo4j-username")
+.displayName("Username")
+.description("Username for accessing Neo4J")
+.required(true)
+
.expressionLanguageSupported(ExpressionLanguageScope.VARIABLE_REGISTRY)
+.addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor PASSWORD = new
PropertyDescriptor.Builder()
+.name("neo4j-password")
+.displayName("Password")
+.description("Password for Neo4J user")
+.required(true)
+.sensitive(true)
+
.expressionLanguageSupported(ExpressionLanguageScope.VARIABLE_REGISTRY)
+.addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+.build();
+
+public static AllowableValue LOAD_BALANCING_STRATEGY_ROUND_ROBIN = new
AllowableValue(LoadBalancingStrategy.ROUND_ROBIN.name(), "Round Robin", "Round
Robin Strategy");
+
+public static AllowableValue LOAD_BALANCING_STRATEGY_LEAST_CONNECTED =
new AllowableValue(LoadBalancingStrategy.LEAST_CONNECTED.n
[GitHub] nifi pull request #2956: NIFI-5537 Create Neo4J cypher execution processor
Github user MikeThomsen commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2956#discussion_r65949
--- Diff:
nifi-nar-bundles/nifi-neo4j-bundle/nifi-neo4j-processors/src/main/java/org/apache/nifi/processors/neo4j/AbstractNeo4JCypherExecutor.java
---
@@ -0,0 +1,281 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.neo4j;
+
+import java.io.File;
+import java.util.concurrent.TimeUnit;
+
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.annotation.lifecycle.OnStopped;
+import org.apache.nifi.components.AllowableValue;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.PropertyValue;
+import org.apache.nifi.expression.ExpressionLanguageScope;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.neo4j.driver.v1.AuthTokens;
+import org.neo4j.driver.v1.Config;
+import org.neo4j.driver.v1.Config.ConfigBuilder;
+import org.neo4j.driver.v1.Config.LoadBalancingStrategy;
+import org.neo4j.driver.v1.Config.TrustStrategy;
+import org.neo4j.driver.v1.Driver;
+import org.neo4j.driver.v1.GraphDatabase;
+
+/**
+ * Abstract base class for Neo4JCypherExecutor processors
+ */
+abstract class AbstractNeo4JCypherExecutor extends AbstractProcessor {
+
+protected static final PropertyDescriptor QUERY = new
PropertyDescriptor.Builder()
+.name("neo4J-query")
+.displayName("Neo4J Query")
+.description("Specifies the Neo4j Query.")
+.required(true)
+
.expressionLanguageSupported(ExpressionLanguageScope.FLOWFILE_ATTRIBUTES)
+.addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor CONNECTION_URL = new
PropertyDescriptor.Builder()
+.name("neo4j-connection-url")
+.displayName("Neo4j Connection URL")
+.description("Neo4J endpoing to connect to.")
+.required(true)
+.defaultValue("bolt://localhost:7687")
+
.expressionLanguageSupported(ExpressionLanguageScope.VARIABLE_REGISTRY)
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor USERNAME = new
PropertyDescriptor.Builder()
+.name("neo4j-username")
+.displayName("Username")
+.description("Username for accessing Neo4J")
+.required(true)
+
.expressionLanguageSupported(ExpressionLanguageScope.VARIABLE_REGISTRY)
+.addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor PASSWORD = new
PropertyDescriptor.Builder()
+.name("neo4j-password")
+.displayName("Password")
+.description("Password for Neo4J user")
+.required(true)
+.sensitive(true)
+
.expressionLanguageSupported(ExpressionLanguageScope.VARIABLE_REGISTRY)
+.addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+.build();
+
+public static AllowableValue LOAD_BALANCING_STRATEGY_ROUND_ROBIN = new
AllowableValue(LoadBalancingStrategy.ROUND_ROBIN.name(), "Round Robin", "Round
Robin Strategy");
+
+public static AllowableValue LOAD_BALANCING_STRATEGY_LEAST_CONNECTED =
new AllowableValue(LoadBalancingStrategy.LEAST_CONNECTED.n
[GitHub] nifi issue #3011: NIFI-5333 Added GetMongoRecord.
Github user MikeThomsen commented on the issue: https://github.com/apache/nifi/pull/3011 @zenfenan any chance we can get a review underway? ---
[GitHub] nifi issue #3015: NIFI-5254 [WIP] Updated to Groovy 2.5.2
Github user MikeThomsen commented on the issue: https://github.com/apache/nifi/pull/3015 @mattyb149 @joewitt while I'm doing L validation on their dependencies, I can also cut out some of the fat by dropping Groovy modules like the CLI one that have no bearing on NiFi scripting. Thoughts? ---
[GitHub] nifi issue #3015: NIFI-5254 [WIP] Updated to Groovy 2.5.2
Github user MikeThomsen commented on the issue: https://github.com/apache/nifi/pull/3015 Thanks, that's what I thought, but need a second opinion because we gotta cross our `T`s and dot our `I`s on this stuff. Will fix. ---
[GitHub] nifi issue #2861: NIFI-5248 Added new Elasticsearch json and record processo...
Github user MikeThomsen commented on the issue: https://github.com/apache/nifi/pull/2861 @mattyb149 should be good to go now. Found the bug and pushed a fix that worked locally. ---
[GitHub] nifi issue #2956: NIFI-5537 Create Neo4J cypher execution processor
Github user MikeThomsen commented on the issue: https://github.com/apache/nifi/pull/2956 > @MikeThomsen - Regarding your graph output questions - It looks correct but can you please let me know what are the commands/files that were ingested for above output ? I'll try to find some time to replay and get back to you on that. ---
[GitHub] nifi pull request #2956: NIFI-5537 Create Neo4J cypher execution processor
Github user MikeThomsen commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2956#discussion_r221723913
--- Diff:
nifi-nar-bundles/nifi-neo4j-bundle/nifi-neo4j-processors/src/main/java/org/apache/nifi/processors/neo4j/AbstractNeo4JCypherExecutor.java
---
@@ -0,0 +1,281 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.neo4j;
+
+import java.io.File;
+import java.util.concurrent.TimeUnit;
+
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.annotation.lifecycle.OnStopped;
+import org.apache.nifi.components.AllowableValue;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.PropertyValue;
+import org.apache.nifi.expression.ExpressionLanguageScope;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.neo4j.driver.v1.AuthTokens;
+import org.neo4j.driver.v1.Config;
+import org.neo4j.driver.v1.Config.ConfigBuilder;
+import org.neo4j.driver.v1.Config.LoadBalancingStrategy;
+import org.neo4j.driver.v1.Config.TrustStrategy;
+import org.neo4j.driver.v1.Driver;
+import org.neo4j.driver.v1.GraphDatabase;
+
+/**
+ * Abstract base class for Neo4JCypherExecutor processors
+ */
+abstract class AbstractNeo4JCypherExecutor extends AbstractProcessor {
+
+protected static final PropertyDescriptor QUERY = new
PropertyDescriptor.Builder()
+.name("neo4J-query")
+.displayName("Neo4J Query")
+.description("Specifies the Neo4j Query.")
+.required(true)
+
.expressionLanguageSupported(ExpressionLanguageScope.FLOWFILE_ATTRIBUTES)
+.addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor CONNECTION_URL = new
PropertyDescriptor.Builder()
+.name("neo4j-connection-url")
+.displayName("Neo4j Connection URL")
+.description("Neo4J endpoing to connect to.")
+.required(true)
+.defaultValue("bolt://localhost:7687")
+
.expressionLanguageSupported(ExpressionLanguageScope.VARIABLE_REGISTRY)
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor USERNAME = new
PropertyDescriptor.Builder()
+.name("neo4j-username")
+.displayName("Username")
+.description("Username for accessing Neo4J")
+.required(true)
+
.expressionLanguageSupported(ExpressionLanguageScope.VARIABLE_REGISTRY)
+.addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor PASSWORD = new
PropertyDescriptor.Builder()
+.name("neo4j-password")
+.displayName("Password")
+.description("Password for Neo4J user")
+.required(true)
+.sensitive(true)
+
.expressionLanguageSupported(ExpressionLanguageScope.VARIABLE_REGISTRY)
+.addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+.build();
+
+public static AllowableValue LOAD_BALANCING_STRATEGY_ROUND_ROBIN = new
AllowableValue(LoadBalancingStrategy.ROUND_ROBIN.name(), "Round Robin", "Round
Robin Strategy");
+
+public static AllowableValue LOAD_BALANCING_STRATEGY_LEAST_CONNECTED =
new AllowableValue(LoadBalancingStrategy.LEAST_CONNECTED.n
[GitHub] nifi issue #2861: NIFI-5248 Added new Elasticsearch json and record processo...
Github user MikeThomsen commented on the issue: https://github.com/apache/nifi/pull/2861 One of the lookup integration tests is failing. I need to figure out why. ---
[GitHub] nifi issue #3042: NIFI-5650: Added Xerces to scripting bundle for Jython 2.7...
Github user MikeThomsen commented on the issue: https://github.com/apache/nifi/pull/3042 Merged. ---
[GitHub] nifi issue #3042: NIFI-5650: Added Xerces to scripting bundle for Jython 2.7...
Github user MikeThomsen commented on the issue: https://github.com/apache/nifi/pull/3042 @mattyb149 I'll merge when Travis is done and at least one build passes. ---
[GitHub] nifi pull request #3042: NIFI-5650: Added Xerces to scripting bundle for Jyt...
Github user MikeThomsen commented on a diff in the pull request: https://github.com/apache/nifi/pull/3042#discussion_r221630928 --- Diff: nifi-nar-bundles/nifi-scripting-bundle/nifi-scripting-nar/src/main/resources/META-INF/NOTICE --- @@ -73,6 +73,20 @@ The following binary components are provided under the Apache Software License v The original software and related information is available at http://www.jcraft.com/jsch/. + (ASLv2) Apache Xerces Java --- End diff -- L looks good. ---
[GitHub] nifi pull request #3041: NIFI-5224 Added SolrClientService.
GitHub user MikeThomsen opened a pull request: https://github.com/apache/nifi/pull/3041 NIFI-5224 Added SolrClientService. Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/MikeThomsen/nifi NIFI-5224 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/3041.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3041 commit ca5c881b1d2ceee82bb7d6700191eb209612adbc Author: Mike Thomsen Date: 2018-10-01T13:53:16Z NIFI-5224 Added SolrClientService. ---
[GitHub] nifi pull request #2956: NIFI-5537 Create Neo4J cypher execution processor
Github user MikeThomsen commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2956#discussion_r221426131
--- Diff:
nifi-nar-bundles/nifi-neo4j-bundle/nifi-neo4j-processors/src/main/java/org/apache/nifi/processors/neo4j/AbstractNeo4JCypherExecutor.java
---
@@ -0,0 +1,281 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.neo4j;
+
+import java.io.File;
+import java.util.concurrent.TimeUnit;
+
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.annotation.lifecycle.OnStopped;
+import org.apache.nifi.components.AllowableValue;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.PropertyValue;
+import org.apache.nifi.expression.ExpressionLanguageScope;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.neo4j.driver.v1.AuthTokens;
+import org.neo4j.driver.v1.Config;
+import org.neo4j.driver.v1.Config.ConfigBuilder;
+import org.neo4j.driver.v1.Config.LoadBalancingStrategy;
+import org.neo4j.driver.v1.Config.TrustStrategy;
+import org.neo4j.driver.v1.Driver;
+import org.neo4j.driver.v1.GraphDatabase;
+
+/**
+ * Abstract base class for Neo4JCypherExecutor processors
+ */
+abstract class AbstractNeo4JCypherExecutor extends AbstractProcessor {
+
+protected static final PropertyDescriptor QUERY = new
PropertyDescriptor.Builder()
+.name("neo4J-query")
+.displayName("Neo4J Query")
+.description("Specifies the Neo4j Query.")
+.required(true)
+
.expressionLanguageSupported(ExpressionLanguageScope.FLOWFILE_ATTRIBUTES)
+.addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor CONNECTION_URL = new
PropertyDescriptor.Builder()
+.name("neo4j-connection-url")
+.displayName("Neo4j Connection URL")
+.description("Neo4J endpoing to connect to.")
+.required(true)
+.defaultValue("bolt://localhost:7687")
+
.expressionLanguageSupported(ExpressionLanguageScope.VARIABLE_REGISTRY)
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor USERNAME = new
PropertyDescriptor.Builder()
+.name("neo4j-username")
+.displayName("Username")
+.description("Username for accessing Neo4J")
+.required(true)
+
.expressionLanguageSupported(ExpressionLanguageScope.VARIABLE_REGISTRY)
+.addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor PASSWORD = new
PropertyDescriptor.Builder()
+.name("neo4j-password")
+.displayName("Password")
+.description("Password for Neo4J user")
+.required(true)
+.sensitive(true)
+
.expressionLanguageSupported(ExpressionLanguageScope.VARIABLE_REGISTRY)
+.addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+.build();
+
+public static AllowableValue LOAD_BALANCING_STRATEGY_ROUND_ROBIN = new
AllowableValue(LoadBalancingStrategy.ROUND_ROBIN.name(), "Round Robin", "Round
Robin Strategy");
+
+public static AllowableValue LOAD_BALANCING_STRATEGY_LEAST_CONNECTED =
new AllowableValue(LoadBalancingStrategy.LEAST_CONNECTED.n
[GitHub] nifi pull request #2956: NIFI-5537 Create Neo4J cypher execution processor
Github user MikeThomsen commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2956#discussion_r221425925
--- Diff:
nifi-nar-bundles/nifi-neo4j-bundle/nifi-neo4j-processors/src/main/java/org/apache/nifi/processors/neo4j/AbstractNeo4JCypherExecutor.java
---
@@ -0,0 +1,281 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.neo4j;
+
+import java.io.File;
+import java.util.concurrent.TimeUnit;
+
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.annotation.lifecycle.OnStopped;
+import org.apache.nifi.components.AllowableValue;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.PropertyValue;
+import org.apache.nifi.expression.ExpressionLanguageScope;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.neo4j.driver.v1.AuthTokens;
+import org.neo4j.driver.v1.Config;
+import org.neo4j.driver.v1.Config.ConfigBuilder;
+import org.neo4j.driver.v1.Config.LoadBalancingStrategy;
+import org.neo4j.driver.v1.Config.TrustStrategy;
+import org.neo4j.driver.v1.Driver;
+import org.neo4j.driver.v1.GraphDatabase;
+
+/**
+ * Abstract base class for Neo4JCypherExecutor processors
+ */
+abstract class AbstractNeo4JCypherExecutor extends AbstractProcessor {
+
+protected static final PropertyDescriptor QUERY = new
PropertyDescriptor.Builder()
+.name("neo4J-query")
+.displayName("Neo4J Query")
+.description("Specifies the Neo4j Query.")
+.required(true)
+
.expressionLanguageSupported(ExpressionLanguageScope.FLOWFILE_ATTRIBUTES)
+.addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor CONNECTION_URL = new
PropertyDescriptor.Builder()
+.name("neo4j-connection-url")
+.displayName("Neo4j Connection URL")
+.description("Neo4J endpoing to connect to.")
+.required(true)
+.defaultValue("bolt://localhost:7687")
+
.expressionLanguageSupported(ExpressionLanguageScope.VARIABLE_REGISTRY)
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor USERNAME = new
PropertyDescriptor.Builder()
+.name("neo4j-username")
+.displayName("Username")
+.description("Username for accessing Neo4J")
+.required(true)
+
.expressionLanguageSupported(ExpressionLanguageScope.VARIABLE_REGISTRY)
+.addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor PASSWORD = new
PropertyDescriptor.Builder()
--- End diff --
You should put something in there saying that if authentication is
disabled, you still need to supply a dummy password.
---
[GitHub] nifi issue #2872: NIFI-5318 Implement NiFi test harness: initial commit of n...
Github user MikeThomsen commented on the issue: https://github.com/apache/nifi/pull/2872 @peter-gergely-horvath ok. I'm pretty sure that the surefire plugin can be disabled in the POM, but manually activated, so we'll need to look at that because those tests should be runnable if someone wants to modify the test harness and not roll their own test case. ---
[GitHub] nifi pull request #2956: NIFI-5537 Create Neo4J cypher execution processor
Github user MikeThomsen commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2956#discussion_r219939777
--- Diff:
nifi-nar-bundles/nifi-neo4j-bundle/nifi-neo4j-processors/src/main/java/org/apache/nifi/processors/neo4j/AbstractNeo4JCypherExecutor.java
---
@@ -0,0 +1,281 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.neo4j;
+
+import java.io.File;
+import java.util.concurrent.TimeUnit;
+
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.annotation.lifecycle.OnStopped;
+import org.apache.nifi.components.AllowableValue;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.PropertyValue;
+import org.apache.nifi.expression.ExpressionLanguageScope;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.util.StandardValidators;
+import org.neo4j.driver.v1.AuthTokens;
+import org.neo4j.driver.v1.Config;
+import org.neo4j.driver.v1.Config.ConfigBuilder;
+import org.neo4j.driver.v1.Config.LoadBalancingStrategy;
+import org.neo4j.driver.v1.Config.TrustStrategy;
+import org.neo4j.driver.v1.Driver;
+import org.neo4j.driver.v1.GraphDatabase;
+
+/**
+ * Abstract base class for Neo4JCypherExecutor processors
+ */
+abstract class AbstractNeo4JCypherExecutor extends AbstractProcessor {
+
+protected static final PropertyDescriptor QUERY = new
PropertyDescriptor.Builder()
+.name("neo4J-query")
+.displayName("Neo4J Query")
+.description("Specifies the Neo4j Query.")
+.required(true)
+
.expressionLanguageSupported(ExpressionLanguageScope.FLOWFILE_ATTRIBUTES)
+.addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor CONNECTION_URL = new
PropertyDescriptor.Builder()
+.name("neo4j-connection-url")
+.displayName("Neo4j Connection URL")
+.description("Neo4J endpoing to connect to.")
+.required(true)
+.defaultValue("bolt://localhost:7687")
+
.expressionLanguageSupported(ExpressionLanguageScope.VARIABLE_REGISTRY)
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor USERNAME = new
PropertyDescriptor.Builder()
+.name("neo4j-username")
+.displayName("Username")
+.description("Username for accessing Neo4J")
+.required(true)
+
.expressionLanguageSupported(ExpressionLanguageScope.VARIABLE_REGISTRY)
+.addValidator(StandardValidators.NON_BLANK_VALIDATOR)
+.build();
+
+public static final PropertyDescriptor PASSWORD = new
PropertyDescriptor.Builder()
--- End diff --
Needs a notice on how to use it if authentication is disabled.
---
[GitHub] nifi issue #3025: NIFI-5605 Added UpdateCassandra.
Github user MikeThomsen commented on the issue: https://github.com/apache/nifi/pull/3025 @zenfenan this is geared toward delete/update operations. ---
[GitHub] nifi pull request #3025: NIFI-5605 Added UpdateCassandra.
GitHub user MikeThomsen opened a pull request: https://github.com/apache/nifi/pull/3025 NIFI-5605 Added UpdateCassandra. Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/MikeThomsen/nifi NIFI-5605 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/3025.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3025 commit d9cb1ba54e672952e172241324f895c8dbf0f7f1 Author: Mike Thomsen Date: 2018-09-19T01:14:29Z NIFI-5605 Added UpdateCassandra. ---
[GitHub] nifi issue #3011: NIFI-5333 Added GetMongoRecord.
Github user MikeThomsen commented on the issue: https://github.com/apache/nifi/pull/3011 @zenfenan @mattyb149 it's back. ---
[GitHub] nifi issue #3014: NIFI-5614: Update commons-dbcp to commons-dbcp2 for DBCPCo...
Github user MikeThomsen commented on the issue: https://github.com/apache/nifi/pull/3014 Checked against postgres 10.5 and worked fine. Merging. Thanks @mattyb149 ---
[GitHub] nifi issue #3014: NIFI-5614: Update commons-dbcp to commons-dbcp2 for DBCPCo...
Github user MikeThomsen commented on the issue: https://github.com/apache/nifi/pull/3014 Reviewing... ---
[GitHub] nifi issue #2956: NIFI-5537 Create Neo4J cypher execution processor
Github user MikeThomsen commented on the issue: https://github.com/apache/nifi/pull/2956 @mans2singh you're going to need to do a rebase off of master. ---
[GitHub] nifi issue #3015: NIFI-5254 [WIP] Updated to Groovy 2.5.2
Github user MikeThomsen commented on the issue: https://github.com/apache/nifi/pull/3015 @joewitt In 2.5.0 they removed the groovy-all fat jar in favor of doing a pom import that brings in all of the individual groovy components. Should I change the `NOTICE` to say "groovy-all pom and associated Groovy modules" (where it currently reads something like "groovy-all 2.4.X jar") or should I remove that completely in favor of enumerating each Groovy component? ---
[GitHub] nifi pull request #3015: NIFI-5254 [WIP] Updated to Groovy 2.5.2
GitHub user MikeThomsen opened a pull request: https://github.com/apache/nifi/pull/3015 NIFI-5254 [WIP] Updated to Groovy 2.5.2 Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/MikeThomsen/nifi NIFI-5254 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/3015.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3015 commit 248e2656f2216156fe59fcde0a93f6157140dab8 Author: Mike Thomsen Date: 2018-09-20T00:06:29Z NIFI-5254 [WIP] Updated to Groovy 2.5.2 ---
[GitHub] nifi issue #2820: NIFI-5327 Adding Netflowv5 protocol parser
Github user MikeThomsen commented on the issue: https://github.com/apache/nifi/pull/2820 @PrashanthVenkatesan if any of the builds passed, don't worry about it. It's the same test, just localized to different locale settings. ---
[GitHub] nifi pull request #3011: NIFI-5333 Added GetMongoRecord.
GitHub user MikeThomsen opened a pull request: https://github.com/apache/nifi/pull/3011 NIFI-5333 Added GetMongoRecord. Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/MikeThomsen/nifi NIFI-5333 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/3011.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3011 commit d9fbe281b2a01c66cea9c4a6523c84c84c303be7 Author: Mike Thomsen Date: 2018-09-02T19:58:33Z NIFI-5333 Added GetMongoRecord. ---
[GitHub] nifi issue #2813: NIFI-5084 Added GenerateRecord processor.
Github user MikeThomsen commented on the issue: https://github.com/apache/nifi/pull/2813 NP. ---
[GitHub] nifi issue #2813: NIFI-5084 Added GenerateRecord processor.
Github user MikeThomsen commented on the issue: https://github.com/apache/nifi/pull/2813 I don't think it's compatible with your use case. What I built, based on how I read the ticket, was a processor that uses a Confluent library to generate random data based on an Avro schema. So maybe I'm missing something, but I think that's different from what you need. ---
[GitHub] nifi issue #2813: NIFI-5084 Added GenerateRecord processor.
Github user MikeThomsen commented on the issue: https://github.com/apache/nifi/pull/2813 Also, the space issue is becoming a serious concern for our binary distributions. So now things that aren't really important for core have to be weighed against that consideration if you want normal users to have an easy time getting to them. ---
[GitHub] nifi issue #2813: NIFI-5084 Added GenerateRecord processor.
Github user MikeThomsen commented on the issue: https://github.com/apache/nifi/pull/2813 @JPercivall Here you go: http://apache-nifi-developer-list.39713.n7.nabble.com/Should-I-withdraw-this-PR-td19475.html I'm fine with reopening this if you want to take over the code review. Folks have been pretty busy and it was starting to get to stale, so I pulled the trigger to include it in a larger bundle of testing tools for users. ---
[GitHub] nifi pull request #2820: NIFI-5327 Adding Netflowv5 protocol parser
Github user MikeThomsen commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2820#discussion_r218569285
--- Diff:
nifi-nar-bundles/nifi-network-bundle/nifi-network-utils/src/main/java/org/apache/nifi/processors/network/parser/Netflowv5Parser.java
---
@@ -0,0 +1,141 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.network.parser;
+
+import java.util.OptionalInt;
+import static
org.apache.nifi.processors.network.parser.util.ConversionUtil.toShort;
+import static
org.apache.nifi.processors.network.parser.util.ConversionUtil.toInt;
+import static
org.apache.nifi.processors.network.parser.util.ConversionUtil.toLong;
+import static
org.apache.nifi.processors.network.parser.util.ConversionUtil.toIPV4;
+
+/**
+ * Networkv5 is Cisco data export format which contains one header and one
or more flow records. This Parser parses the netflowv5 format. More
information: @see
+ * https://www.cisco.com/c/en/us/td/docs/net_mgmt/netflow_collection_engine/3-6/user/guide/format.html;>Netflowv5
+ */
+public final class Netflowv5Parser {
+private static final int HEADER_SIZE = 24;
+private static final int RECORD_SIZE = 48;
+
+private static final int SHORT_TYPE = 0;
+private static final int INTEGER_TYPE = 1;
+private static final int LONG_TYPE = 2;
+private static final int IPV4_TYPE = 3;
+
+private static final String headerField[] = { "version", "count",
"sys_uptime", "unix_secs", "unix_nsecs", "flow_sequence", "engine_type",
"engine_id", "sampling_interval" };
+private static final String recordField[] = { "srcaddr", "dstaddr",
"nexthop", "input", "output", "dPkts", "dOctets", "first", "last", "srcport",
"dstport", "pad1", "tcp_flags", "prot", "tos",
+"src_as", "dst_as", "src_mask", "dst_mask", "pad2" };
+
+private final int portNumber;
+
+private Object headerData[];
+private Object recordData[][];
+
+public Netflowv5Parser(final OptionalInt portNumber) {
+this.portNumber = (portNumber.isPresent()) ? portNumber.getAsInt()
: 0;
+}
+
+public final int parse(final byte[] buffer) throws Throwable {
+if( !isValid(buffer.length) )
--- End diff --
Curly brackets are missing here. and below with the version check.
---
[GitHub] nifi pull request #2820: NIFI-5327 Adding Netflowv5 protocol parser
Github user MikeThomsen commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2820#discussion_r218567833
--- Diff:
nifi-nar-bundles/nifi-network-bundle/nifi-network-processors/src/main/java/org/apache/nifi/processors/network/ParseNetflowv5.java
---
@@ -178,7 +178,7 @@ private void generateJSON(final List
multipleRecords, final ProcessSes
results.set("port",
mapper.valueToTree(parser.getPortNumber()));
results.set("format", mapper.valueToTree("netflowv5"));
-recordFlowFile = session.clone(flowFile);
+recordFlowFile = session.create(flowFile);
--- End diff --
Not related to this PR, but for future reference if you choose
`INPUT_ALLOWED` you'll need to do something like this: `flowFile != null ?
session.create(flowFile) : session.create()` because `ProcessSession`'s impl
doesn't handle a null input.
---
[GitHub] nifi pull request #2820: NIFI-5327 Adding Netflowv5 protocol parser
Github user MikeThomsen commented on a diff in the pull request: https://github.com/apache/nifi/pull/2820#discussion_r218566840 --- Diff: nifi-nar-bundles/nifi-network-bundle/nifi-network-processors/pom.xml --- @@ -0,0 +1,67 @@ + + +http://maven.apache.org/POM/4.0.0; xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance; + xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd;> + 4.0.0 + + + org.apache.nifi + nifi-network-bundle + 1.8.0-SNAPSHOT + + + nifi-network-processors + jar + + + + org.apache.nifi + nifi-api + + + org.apache.nifi + nifi-utils + 1.8.0-SNAPSHOT --- End diff -- No, you should be able to safely remove the version number. ---
[GitHub] nifi issue #2992: NIFI-5510: Introducing PutCassandraRecord processor
Github user MikeThomsen commented on the issue: https://github.com/apache/nifi/pull/2992 Merged. Thanks for the contribution. ---
[GitHub] nifi pull request #2820: NIFI-5327 Adding Netflowv5 protocol parser
Github user MikeThomsen commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2820#discussion_r218401785
--- Diff:
nifi-nar-bundles/nifi-network-bundle/nifi-network-processors/src/main/java/org/apache/nifi/processors/network/ParseNetflowv5.java
---
@@ -0,0 +1,258 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.network;
+
+import static
org.apache.nifi.processors.network.parser.Netflowv5Parser.getHeaderFields;
+import static
org.apache.nifi.processors.network.parser.Netflowv5Parser.getRecordFields;
+
+import java.io.BufferedOutputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.OutputStream;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.OptionalInt;
+import java.util.Set;
+
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.InputRequirement.Requirement;
+import org.apache.nifi.annotation.behavior.ReadsAttribute;
+import org.apache.nifi.annotation.behavior.ReadsAttributes;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.behavior.WritesAttribute;
+import org.apache.nifi.annotation.behavior.WritesAttributes;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.flowfile.attributes.CoreAttributes;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.io.InputStreamCallback;
+import org.apache.nifi.processor.io.OutputStreamCallback;
+import org.apache.nifi.processors.network.parser.Netflowv5Parser;
+import org.apache.nifi.stream.io.StreamUtils;
+
+import com.fasterxml.jackson.core.JsonProcessingException;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import com.fasterxml.jackson.databind.node.ObjectNode;
+
+@EventDriven
+@SideEffectFree
+@SupportsBatching
+@InputRequirement(Requirement.INPUT_REQUIRED)
+@Tags({ "network", "netflow", "attributes", "datagram", "v5", "packet",
"byte" })
+@CapabilityDescription("Parses netflowv5 byte ingest and add to NiFi
flowfile as attributes or JSON content.")
+@ReadsAttributes({ @ReadsAttribute(attribute = "udp.port", description =
"Optionally read if packets are received from UDP datagrams.") })
+@WritesAttributes({ @WritesAttribute(attribute = "netflowv5.header.*",
description = "The key and value generated by the parsing of the header
fields."),
+@WritesAttribute(attribute = "netflowv5.record.*", description =
"The key and value generated by the parsing of the record fields.") })
+
+public class ParseNetflowv5 extends AbstractProcessor {
+private String destination;
+// Add mapper
+private static final ObjectMapper mapper = new ObjectMapper();
+
+public static final String DESTINATION_CONTENT = "flowfile-content";
+public static final String DESTINATION_ATTRIBUTES =
"flowfile-attribute";
+public static final PropertyDescriptor FIELDS_DESTINATION = new
PropertyDescriptor.Builder().name("FIELDS_DESTINATION").di
[GitHub] nifi pull request #2820: NIFI-5327 Adding Netflowv5 protocol parser
Github user MikeThomsen commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2820#discussion_r218401389
--- Diff:
nifi-nar-bundles/nifi-network-bundle/nifi-network-processors/src/main/java/org/apache/nifi/processors/network/ParseNetflowv5.java
---
@@ -0,0 +1,258 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.network;
+
+import static
org.apache.nifi.processors.network.parser.Netflowv5Parser.getHeaderFields;
+import static
org.apache.nifi.processors.network.parser.Netflowv5Parser.getRecordFields;
+
+import java.io.BufferedOutputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.OutputStream;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.OptionalInt;
+import java.util.Set;
+
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.InputRequirement.Requirement;
+import org.apache.nifi.annotation.behavior.ReadsAttribute;
+import org.apache.nifi.annotation.behavior.ReadsAttributes;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.behavior.WritesAttribute;
+import org.apache.nifi.annotation.behavior.WritesAttributes;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.flowfile.attributes.CoreAttributes;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.io.InputStreamCallback;
+import org.apache.nifi.processor.io.OutputStreamCallback;
+import org.apache.nifi.processors.network.parser.Netflowv5Parser;
+import org.apache.nifi.stream.io.StreamUtils;
+
+import com.fasterxml.jackson.core.JsonProcessingException;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import com.fasterxml.jackson.databind.node.ObjectNode;
+
+@EventDriven
+@SideEffectFree
+@SupportsBatching
+@InputRequirement(Requirement.INPUT_REQUIRED)
+@Tags({ "network", "netflow", "attributes", "datagram", "v5", "packet",
"byte" })
+@CapabilityDescription("Parses netflowv5 byte ingest and add to NiFi
flowfile as attributes or JSON content.")
+@ReadsAttributes({ @ReadsAttribute(attribute = "udp.port", description =
"Optionally read if packets are received from UDP datagrams.") })
+@WritesAttributes({ @WritesAttribute(attribute = "netflowv5.header.*",
description = "The key and value generated by the parsing of the header
fields."),
+@WritesAttribute(attribute = "netflowv5.record.*", description =
"The key and value generated by the parsing of the record fields.") })
+
+public class ParseNetflowv5 extends AbstractProcessor {
+private String destination;
+// Add mapper
+private static final ObjectMapper mapper = new ObjectMapper();
+
+public static final String DESTINATION_CONTENT = "flowfile-content";
+public static final String DESTINATION_ATTRIBUTES =
"flowfile-attribute";
+public static final PropertyDescriptor FIELDS_DESTINATION = new
PropertyDescriptor.Builder().name("FIELDS_DESTINATION").di
