[jira] [Commented] (NIFI-3332) Bug in ListXXX causes matching timestamps to be ignored on later runs
[ https://issues.apache.org/jira/browse/NIFI-3332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16535959#comment-16535959 ] Kon Soulianidis commented on NIFI-3332: --- Echoing [~doaks80] 's concerns. We've been using the ListFiles processor to listen to a smb mounted share. Every 1-2 weeks, the processor just misses files. I have checked the security logs in our company and the service account that nifi runs under has not even attempted to open those files. However, it has pulled files received in that directory seconds earlier. Our volumes aren't very high or frequent (usually 500 files (<100k) sent in small bursts over the course of 6 hours) To workaround this issue, I created a processor that at the end of the day, issues a `touch *` on the directory that the ListFiles processor is listening on. This updates the timestamp of any files left behind and triggers nifi to reprocess them Obviously, the big selling point of using any solution that handles Enterprise Integration Pattern flows is the ability to handle the basics of synchronization. This has left a bad impression of nifi within our organisation which is a shame given the otherwise clean UI to build flows and audibility that provenance brings Our nifi is a standalone docker instance using 1.6.0 > Bug in ListXXX causes matching timestamps to be ignored on later runs > - > > Key: NIFI-3332 > URL: https://issues.apache.org/jira/browse/NIFI-3332 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 0.7.1, 1.1.1 >Reporter: Joe Skora >Assignee: Koji Kawamura >Priority: Critical > Fix For: 1.4.0 > > Attachments: Test-showing-ListFile-timestamp-bug.log, > Test-showing-ListFile-timestamp-bug.patch, listfiles.png > > > The new state implementation for the ListXXX processors based on > AbstractListProcessor creates a race conditions when processor runs occur > while a batch of files is being written with the same timestamp. > The changes to state management dropped tracking of the files processed for a > given timestamp. Without the record of files processed, the remainder of the > batch is ignored on the next processor run since their timestamp is not > greater than the one timestamp stored in processor state. With the file > tracking it was possible to process files that matched the timestamp exactly > and exclude the previously processed files. > A basic time goes as follows. > T0 - system creates or receives batch of files with Tx timestamp where Tx > is more than the current timestamp in processor state. > T1 - system writes 1st half of Tx batch to the ListFile source directory. > T2 - ListFile runs picking up 1st half of Tx batch and stores Tx timestamp > in processor state. > T3 - system writes 2nd half of Tx batch to ListFile source directory. > T4 - ListFile runs ignoring any files with T <= Tx, eliminating 2nd half Tx > timestamp batch. > I've attached a patch[1] for TestListFile.java that adds an instrumented unit > test demonstrates the problem and a log[2] of the output from one such run. > The test writes 3 files each in two batches with processor runs after each > batch. Batch 2 writes files with timestamps older than, equal to, and newer > than the timestamp stored when batch 1 was processed, but only the newer file > is picked up. The older file is correctly ignored but file with the matchin > timestamp file should have been processed. > [1] Test-showing-ListFile-timestamp-bug.patch > [2] Test-showing-ListFile-timestamp-bug.log -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4802) PutSQS not UTF-8 encoding output
[
https://issues.apache.org/jira/browse/NIFI-4802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16535903#comment-16535903
]
ASF GitHub Bot commented on NIFI-4802:
--
Github user HorizonNet commented on the issue:
https://github.com/apache/nifi/pull/2859
Build failure seems to be unrelated to the actual change.
> PutSQS not UTF-8 encoding output
>
>
> Key: NIFI-4802
> URL: https://issues.apache.org/jira/browse/NIFI-4802
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.4.0
> Environment: OSX, Linux
>Reporter: Richard St. John
>Priority: Major
>
> PutSQS processor does not properly encode content. Line 117 reads,
> {code:java}
> final string flowFileContent = baos.toString();
> {code}
> Instead, the output stream should default to UTF-8, or have a configuration
> similar to GetSQS.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] nifi issue #2859: NIFI-4802 Added character configuration to PutSQS
Github user HorizonNet commented on the issue: https://github.com/apache/nifi/pull/2859 Build failure seems to be unrelated to the actual change. ---
[jira] [Commented] (NIFI-1451) Activated JDK versions for disable-doclint profile
[ https://issues.apache.org/jira/browse/NIFI-1451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16535895#comment-16535895 ] ASF GitHub Bot commented on NIFI-1451: -- Github user HorizonNet commented on the issue: https://github.com/apache/nifi/pull/2858 Build failure seems to be unrelated to the actual change. > Activated JDK versions for disable-doclint profile > -- > > Key: NIFI-1451 > URL: https://issues.apache.org/jira/browse/NIFI-1451 > Project: Apache NiFi > Issue Type: Bug > Components: Tools and Build >Reporter: Matt Gilman >Priority: Trivial > > The disable-doclint profile is activated when using JDK 1.8. This needs to be > greater or equal to JDK 1.8. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi issue #2858: NIFI-1451 disable-doclint profile now gets activated for J...
Github user HorizonNet commented on the issue: https://github.com/apache/nifi/pull/2858 Build failure seems to be unrelated to the actual change. ---
[jira] [Commented] (NIFI-4802) PutSQS not UTF-8 encoding output
[
https://issues.apache.org/jira/browse/NIFI-4802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16535893#comment-16535893
]
ASF GitHub Bot commented on NIFI-4802:
--
GitHub user HorizonNet opened a pull request:
https://github.com/apache/nifi/pull/2859
NIFI-4802 Added character configuration to PutSQS
Thank you for submitting a contribution to Apache NiFi.
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
### For all changes:
- [x] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?
- [x] Does your PR title start with NIFI- where is the JIRA number
you are trying to resolve? Pay particular attention to the hyphen "-" character.
- [x] Has your PR been rebased against the latest commit within the target
branch (typically master)?
- [x] Is your initial contribution a single, squashed commit?
### For code changes:
- [x] Have you ensured that the full suite of tests is executed via mvn
-Pcontrib-check clean install at the root nifi folder?
- [ ] Have you written or updated unit tests to verify your changes?
- [ ] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the LICENSE file, including the main
LICENSE file under nifi-assembly?
- [ ] If applicable, have you updated the NOTICE file, including the main
NOTICE file found under nifi-assembly?
- [ ] If adding new Properties, have you added .displayName in addition to
.name (programmatic access) for each of the new properties?
### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in
which it is rendered?
### Note:
Please ensure that once the PR is submitted, you check travis-ci for build
issues and submit an update to your PR as soon as possible.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/ultratendency/nifi NIFI-4802
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/nifi/pull/2859.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2859
commit 6a1bcb39d80d40518d766a24fee65865b5710d3d
Author: Jan Hentschel
Date: 2018-07-07T20:27:20Z
NIFI-4802 Added character configuration to PutSQS
> PutSQS not UTF-8 encoding output
>
>
> Key: NIFI-4802
> URL: https://issues.apache.org/jira/browse/NIFI-4802
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.4.0
> Environment: OSX, Linux
>Reporter: Richard St. John
>Priority: Major
>
> PutSQS processor does not properly encode content. Line 117 reads,
> {code:java}
> final string flowFileContent = baos.toString();
> {code}
> Instead, the output stream should default to UTF-8, or have a configuration
> similar to GetSQS.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] nifi pull request #2859: NIFI-4802 Added character configuration to PutSQS
GitHub user HorizonNet opened a pull request: https://github.com/apache/nifi/pull/2859 NIFI-4802 Added character configuration to PutSQS Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [x] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/ultratendency/nifi NIFI-4802 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2859.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2859 commit 6a1bcb39d80d40518d766a24fee65865b5710d3d Author: Jan Hentschel Date: 2018-07-07T20:27:20Z NIFI-4802 Added character configuration to PutSQS ---
[jira] [Commented] (NIFI-1451) Activated JDK versions for disable-doclint profile
[ https://issues.apache.org/jira/browse/NIFI-1451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16535875#comment-16535875 ] ASF GitHub Bot commented on NIFI-1451: -- GitHub user HorizonNet opened a pull request: https://github.com/apache/nifi/pull/2858 NIFI-1451 disable-doclint profile now gets activated for JDK 1.8 and newer Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [x] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/ultratendency/nifi NIFI-1451 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2858.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2858 commit b9a8174f421d7782df4135c38a206e9eb6bbeded Author: Jan Hentschel Date: 2018-07-07T19:24:42Z NIFI-1451 disable-doclint profile now gets activated for JDK 1.8 and newer > Activated JDK versions for disable-doclint profile > -- > > Key: NIFI-1451 > URL: https://issues.apache.org/jira/browse/NIFI-1451 > Project: Apache NiFi > Issue Type: Bug > Components: Tools and Build >Reporter: Matt Gilman >Priority: Trivial > > The disable-doclint profile is activated when using JDK 1.8. This needs to be > greater or equal to JDK 1.8. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi pull request #2858: NIFI-1451 disable-doclint profile now gets activate...
GitHub user HorizonNet opened a pull request: https://github.com/apache/nifi/pull/2858 NIFI-1451 disable-doclint profile now gets activated for JDK 1.8 and newer Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [x] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/ultratendency/nifi NIFI-1451 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2858.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2858 commit b9a8174f421d7782df4135c38a206e9eb6bbeded Author: Jan Hentschel Date: 2018-07-07T19:24:42Z NIFI-1451 disable-doclint profile now gets activated for JDK 1.8 and newer ---
[jira] [Commented] (NIFI-1305) Clarify Javadoc on DistributedMapCacheClient.getAndPutIfAbsent
[ https://issues.apache.org/jira/browse/NIFI-1305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16535871#comment-16535871 ] ASF GitHub Bot commented on NIFI-1305: -- GitHub user HorizonNet opened a pull request: https://github.com/apache/nifi/pull/2857 NIFI-1305 Extended JavaDoc on DistributedMapCacheClient.getAndPutIfAbsent Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [x] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/ultratendency/nifi NIFI-1305 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2857.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2857 commit ba5bd534e04b00c8a84414ac26fc748266b2940d Author: Jan Hentschel Date: 2018-07-07T18:58:37Z NIFI-1305 Extended JavaDoc on DistributedMapCacheClient.getAndPutIfAbsent > Clarify Javadoc on DistributedMapCacheClient.getAndPutIfAbsent > -- > > Key: NIFI-1305 > URL: https://issues.apache.org/jira/browse/NIFI-1305 > Project: Apache NiFi > Issue Type: Improvement > Components: Documentation Website >Affects Versions: 0.4.0 >Reporter: Michael Kobit >Priority: Trivial > > The Javadoc for this method on DistributedMapCacheClient > (https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-standard-services/nifi-distributed-cache-client-service-api/src/main/java/org/apache/nifi/distributed/cache/client/DistributedMapCacheClient.java#L53-L71) > is unclear about what happens if the key does not exist in the cache. > It looks like from the implementation of DistributedMapCacheClientService > that it returns the value if it present, otherwise returns the computed > value. The Javadoc doesn't state what is returned happens if an item does not > exist. > I spent some time trying to figure out what makes sense from a client's > perspective, and I had some trouble trying to understand what should happen. > It seems obvious to me now that the cache should return either the existing > value or the newly computed value, but it took some time to get there. > I think the documentation should be clearer about what is returned in the > case where the item does not exist -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi pull request #2857: NIFI-1305 Extended JavaDoc on DistributedMapCacheCl...
GitHub user HorizonNet opened a pull request: https://github.com/apache/nifi/pull/2857 NIFI-1305 Extended JavaDoc on DistributedMapCacheClient.getAndPutIfAbsent Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [x] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/ultratendency/nifi NIFI-1305 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2857.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2857 commit ba5bd534e04b00c8a84414ac26fc748266b2940d Author: Jan Hentschel Date: 2018-07-07T18:58:37Z NIFI-1305 Extended JavaDoc on DistributedMapCacheClient.getAndPutIfAbsent ---
[jira] [Commented] (NIFI-5355) ResizeImage Fails to read PNG type on some OS's
[ https://issues.apache.org/jira/browse/NIFI-5355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16535819#comment-16535819 ] ASF GitHub Bot commented on NIFI-5355: -- Github user patricker commented on the issue: https://github.com/apache/nifi/pull/2854 Some further research. Original source code location: https://web.archive.org/web/20080516181120/http://today.java.net/pub/a/today/2007/04/03/perils-of-image-getscaledinstance.html The code has no listed license. So according to that website: https://web.archive.org/web/20080512032235/http://java.net/terms.csp Source Code Submissions. You agree that any source code You contribute to a Project will be submitted under, and subject to, the license posted for that Project. If no license is posted, You agree that Your Submission will be governed by the Apache License, Version 2.0, which terms can be found at http://opensource.org/licenses/apache2.0.php. You acknowledge that You are responsible for including all applicable copyright notices and licenses with Your Submissions, and that You assume the risks of failing to do so, including the potential loss of Your rights to Your Submissions. > ResizeImage Fails to read PNG type on some OS's > --- > > Key: NIFI-5355 > URL: https://issues.apache.org/jira/browse/NIFI-5355 > Project: Apache NiFi > Issue Type: Bug >Reporter: Peter Wicks >Assignee: Peter Wicks >Priority: Minor > > On some operating systems ResizeImage will fail to properly read the PNG file > type when calling getType. It will return '0' instead of '5'. > > [https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-media-bundle/nifi-media-processors/src/main/java/org/apache/nifi/processors/image/ResizeImage.java#L177] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi issue #2854: NIFI-5355 ResizeImage Fails to read PNG type on some OS's
Github user patricker commented on the issue: https://github.com/apache/nifi/pull/2854 Some further research. Original source code location: https://web.archive.org/web/20080516181120/http://today.java.net/pub/a/today/2007/04/03/perils-of-image-getscaledinstance.html The code has no listed license. So according to that website: https://web.archive.org/web/20080512032235/http://java.net/terms.csp Source Code Submissions. You agree that any source code You contribute to a Project will be submitted under, and subject to, the license posted for that Project. If no license is posted, You agree that Your Submission will be governed by the Apache License, Version 2.0, which terms can be found at http://opensource.org/licenses/apache2.0.php. You acknowledge that You are responsible for including all applicable copyright notices and licenses with Your Submissions, and that You assume the risks of failing to do so, including the potential loss of Your rights to Your Submissions. ---
[jira] [Commented] (NIFI-5355) ResizeImage Fails to read PNG type on some OS's
[ https://issues.apache.org/jira/browse/NIFI-5355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16535816#comment-16535816 ] ASF GitHub Bot commented on NIFI-5355: -- Github user MikeThomsen commented on a diff in the pull request: https://github.com/apache/nifi/pull/2854#discussion_r200815742 --- Diff: nifi-nar-bundles/nifi-media-bundle/nifi-media-processors/src/main/java/org/apache/nifi/processors/image/ResizeImage.java --- @@ -196,4 +187,53 @@ public void process(final InputStream rawIn, final OutputStream out) throws IOEx } } +// https://stackoverflow.com/questions/7951290/re-sizing-an-image-without-losing-quality --- End diff -- So yeah, it's possible that the code was stolen from somewhere else, but the prima facie licensing of that content is CC share alike. From what I read on the licensing page, it's only CC non commercial that is problematic. > ResizeImage Fails to read PNG type on some OS's > --- > > Key: NIFI-5355 > URL: https://issues.apache.org/jira/browse/NIFI-5355 > Project: Apache NiFi > Issue Type: Bug >Reporter: Peter Wicks >Assignee: Peter Wicks >Priority: Minor > > On some operating systems ResizeImage will fail to properly read the PNG file > type when calling getType. It will return '0' instead of '5'. > > [https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-media-bundle/nifi-media-processors/src/main/java/org/apache/nifi/processors/image/ResizeImage.java#L177] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi pull request #2854: NIFI-5355 ResizeImage Fails to read PNG type on som...
Github user MikeThomsen commented on a diff in the pull request: https://github.com/apache/nifi/pull/2854#discussion_r200815742 --- Diff: nifi-nar-bundles/nifi-media-bundle/nifi-media-processors/src/main/java/org/apache/nifi/processors/image/ResizeImage.java --- @@ -196,4 +187,53 @@ public void process(final InputStream rawIn, final OutputStream out) throws IOEx } } +// https://stackoverflow.com/questions/7951290/re-sizing-an-image-without-losing-quality --- End diff -- So yeah, it's possible that the code was stolen from somewhere else, but the prima facie licensing of that content is CC share alike. From what I read on the licensing page, it's only CC non commercial that is problematic. ---
[jira] [Commented] (NIFI-5355) ResizeImage Fails to read PNG type on some OS's
[ https://issues.apache.org/jira/browse/NIFI-5355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16535815#comment-16535815 ] ASF GitHub Bot commented on NIFI-5355: -- Github user MikeThomsen commented on a diff in the pull request: https://github.com/apache/nifi/pull/2854#discussion_r200815662 --- Diff: nifi-nar-bundles/nifi-media-bundle/nifi-media-processors/src/main/java/org/apache/nifi/processors/image/ResizeImage.java --- @@ -196,4 +187,53 @@ public void process(final InputStream rawIn, final OutputStream out) throws IOEx } } +// https://stackoverflow.com/questions/7951290/re-sizing-an-image-without-losing-quality --- End diff -- @joewitt StackOverflow has the following disclaimer: > site design / logo © 2018 Stack Exchange Inc; user contributions licensed under cc by-sa 3.0 with attribution required. rev 2018.7.5.30950 > ResizeImage Fails to read PNG type on some OS's > --- > > Key: NIFI-5355 > URL: https://issues.apache.org/jira/browse/NIFI-5355 > Project: Apache NiFi > Issue Type: Bug >Reporter: Peter Wicks >Assignee: Peter Wicks >Priority: Minor > > On some operating systems ResizeImage will fail to properly read the PNG file > type when calling getType. It will return '0' instead of '5'. > > [https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-media-bundle/nifi-media-processors/src/main/java/org/apache/nifi/processors/image/ResizeImage.java#L177] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi pull request #2854: NIFI-5355 ResizeImage Fails to read PNG type on som...
Github user MikeThomsen commented on a diff in the pull request: https://github.com/apache/nifi/pull/2854#discussion_r200815662 --- Diff: nifi-nar-bundles/nifi-media-bundle/nifi-media-processors/src/main/java/org/apache/nifi/processors/image/ResizeImage.java --- @@ -196,4 +187,53 @@ public void process(final InputStream rawIn, final OutputStream out) throws IOEx } } +// https://stackoverflow.com/questions/7951290/re-sizing-an-image-without-losing-quality --- End diff -- @joewitt StackOverflow has the following disclaimer: > site design / logo © 2018 Stack Exchange Inc; user contributions licensed under cc by-sa 3.0 with attribution required. rev 2018.7.5.30950 ---
[jira] [Resolved] (NIFI-5384) FlowFile's queued in batches should all have the same Queue time
[ https://issues.apache.org/jira/browse/NIFI-5384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Thomsen resolved NIFI-5384. Resolution: Fixed Fix Version/s: 1.8.0 > FlowFile's queued in batches should all have the same Queue time > > > Key: NIFI-5384 > URL: https://issues.apache.org/jira/browse/NIFI-5384 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Reporter: Peter Wicks >Assignee: Peter Wicks >Priority: Minor > Fix For: 1.8.0 > > > When performing a Session.Transfer on a list of FlowFile's, all FlowFIle's > should have the same queue time. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-5384) FlowFile's queued in batches should all have the same Queue time
[ https://issues.apache.org/jira/browse/NIFI-5384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16535741#comment-16535741 ] ASF GitHub Bot commented on NIFI-5384: -- Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/2849 > FlowFile's queued in batches should all have the same Queue time > > > Key: NIFI-5384 > URL: https://issues.apache.org/jira/browse/NIFI-5384 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Reporter: Peter Wicks >Assignee: Peter Wicks >Priority: Minor > > When performing a Session.Transfer on a list of FlowFile's, all FlowFIle's > should have the same queue time. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi pull request #2849: NIFI-5384 FlowFile's queued in batches should all h...
Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/2849 ---
[jira] [Commented] (NIFI-5384) FlowFile's queued in batches should all have the same Queue time
[ https://issues.apache.org/jira/browse/NIFI-5384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16535740#comment-16535740 ] ASF subversion and git services commented on NIFI-5384: --- Commit d50e3f17471e4ade3373b9e95e5c6a8e364248a6 in nifi's branch refs/heads/master from patricker [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=d50e3f1 ] NIFI-5384 FlowFile's queued in batches should all have the same Queue time This closes #2849 Signed-off-by: Mike Thomsen > FlowFile's queued in batches should all have the same Queue time > > > Key: NIFI-5384 > URL: https://issues.apache.org/jira/browse/NIFI-5384 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Reporter: Peter Wicks >Assignee: Peter Wicks >Priority: Minor > > When performing a Session.Transfer on a list of FlowFile's, all FlowFIle's > should have the same queue time. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-5384) FlowFile's queued in batches should all have the same Queue time
[ https://issues.apache.org/jira/browse/NIFI-5384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16535739#comment-16535739 ] ASF GitHub Bot commented on NIFI-5384: -- Github user MikeThomsen commented on the issue: https://github.com/apache/nifi/pull/2849 [Session_Test.xml.txt](https://github.com/apache/nifi/files/2172744/Session_Test.xml.txt) Test flow I used. > FlowFile's queued in batches should all have the same Queue time > > > Key: NIFI-5384 > URL: https://issues.apache.org/jira/browse/NIFI-5384 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Reporter: Peter Wicks >Assignee: Peter Wicks >Priority: Minor > > When performing a Session.Transfer on a list of FlowFile's, all FlowFIle's > should have the same queue time. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi issue #2849: NIFI-5384 FlowFile's queued in batches should all have the...
Github user MikeThomsen commented on the issue: https://github.com/apache/nifi/pull/2849 [Session_Test.xml.txt](https://github.com/apache/nifi/files/2172744/Session_Test.xml.txt) Test flow I used. ---
[jira] [Resolved] (NIFI-5385) assertAttributeNotExists Is hard to understand
[ https://issues.apache.org/jira/browse/NIFI-5385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Thomsen resolved NIFI-5385. Resolution: Fixed Fix Version/s: 1.8.0 > assertAttributeNotExists Is hard to understand > -- > > Key: NIFI-5385 > URL: https://issues.apache.org/jira/browse/NIFI-5385 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Reporter: Peter Wicks >Assignee: Peter Wicks >Priority: Minor > Fix For: 1.8.0 > > > In the mocking/testing framework, `assertAttributeNotExists` has an unclear > error message. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-5385) assertAttributeNotExists Is hard to understand
[ https://issues.apache.org/jira/browse/NIFI-5385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16535736#comment-16535736 ] ASF GitHub Bot commented on NIFI-5385: -- Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/2850 > assertAttributeNotExists Is hard to understand > -- > > Key: NIFI-5385 > URL: https://issues.apache.org/jira/browse/NIFI-5385 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Reporter: Peter Wicks >Assignee: Peter Wicks >Priority: Minor > Fix For: 1.8.0 > > > In the mocking/testing framework, `assertAttributeNotExists` has an unclear > error message. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi pull request #2850: NIFI-5385 assertAttributeNotExists Is hard to under...
Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/2850 ---
[jira] [Commented] (NIFI-5385) assertAttributeNotExists Is hard to understand
[ https://issues.apache.org/jira/browse/NIFI-5385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16535735#comment-16535735 ] ASF subversion and git services commented on NIFI-5385: --- Commit 0c5e159ebc0d49cedfc209302a514e9f423298fd in nifi's branch refs/heads/master from patricker [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=0c5e159 ] NIFI-5385 assertAttributeNotExists Is hard to understand This closes #2850 Signed-off-by: Mike Thomsen > assertAttributeNotExists Is hard to understand > -- > > Key: NIFI-5385 > URL: https://issues.apache.org/jira/browse/NIFI-5385 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Reporter: Peter Wicks >Assignee: Peter Wicks >Priority: Minor > > In the mocking/testing framework, `assertAttributeNotExists` has an unclear > error message. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4811) Use a newer version of spring-data-redis
[ https://issues.apache.org/jira/browse/NIFI-4811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16535734#comment-16535734 ] ASF GitHub Bot commented on NIFI-4811: -- Github user MikeThomsen commented on the issue: https://github.com/apache/nifi/pull/2856 @joewitt nifi-redis-nar was right AFAICT. The way I'm looking for implicated jars is `mvn dependency:tree | grep compile`. I am skipping ones that aren't scoped to compile. > Use a newer version of spring-data-redis > > > Key: NIFI-4811 > URL: https://issues.apache.org/jira/browse/NIFI-4811 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Johan Oceanstorm >Assignee: Mike Thomsen >Priority: Major > Fix For: 1.8.0 > > > NiFi uses version 1.8.3 of spring-data-redis (as you can see > [here|[https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-redis-bundle/pom.xml#L31].] > This version is an old version. The newer stable version is 2.0.2. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi issue #2856: NIFI-4811 Added two missing entries to the nifi-redis-serv...
Github user MikeThomsen commented on the issue: https://github.com/apache/nifi/pull/2856 @joewitt nifi-redis-nar was right AFAICT. The way I'm looking for implicated jars is `mvn dependency:tree | grep compile`. I am skipping ones that aren't scoped to compile. ---
[jira] [Commented] (NIFI-4811) Use a newer version of spring-data-redis
[ https://issues.apache.org/jira/browse/NIFI-4811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16535733#comment-16535733 ] ASF GitHub Bot commented on NIFI-4811: -- GitHub user MikeThomsen opened a pull request: https://github.com/apache/nifi/pull/2856 NIFI-4811 Added two missing entries to the nifi-redis-service-api-nar… … NOTICE file. Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/MikeThomsen/nifi NIFI-4811 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2856.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2856 commit d5d6ed1ce7c0df4c45b6be5b46499cd2cae12982 Author: Mike Thomsen Date: 2018-07-07T11:21:15Z NIFI-4811 Added two missing entries to the nifi-redis-service-api-nar NOTICE file. > Use a newer version of spring-data-redis > > > Key: NIFI-4811 > URL: https://issues.apache.org/jira/browse/NIFI-4811 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Johan Oceanstorm >Assignee: Mike Thomsen >Priority: Major > Fix For: 1.8.0 > > > NiFi uses version 1.8.3 of spring-data-redis (as you can see > [here|[https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-redis-bundle/pom.xml#L31].] > This version is an old version. The newer stable version is 2.0.2. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi pull request #2856: NIFI-4811 Added two missing entries to the nifi-red...
GitHub user MikeThomsen opened a pull request: https://github.com/apache/nifi/pull/2856 NIFI-4811 Added two missing entries to the nifi-redis-service-api-nar⦠⦠NOTICE file. Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/MikeThomsen/nifi NIFI-4811 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2856.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2856 commit d5d6ed1ce7c0df4c45b6be5b46499cd2cae12982 Author: Mike Thomsen Date: 2018-07-07T11:21:15Z NIFI-4811 Added two missing entries to the nifi-redis-service-api-nar NOTICE file. ---
[jira] [Commented] (NIFI-4811) Use a newer version of spring-data-redis
[ https://issues.apache.org/jira/browse/NIFI-4811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16535732#comment-16535732 ] Mike Thomsen commented on NIFI-4811: [~joewitt] having had a night to sleep on it, I started going back and found that the original NOTICE appears to have been bad as well. It was missing at least two references it should have had (commons logging and spring framework). I have the api-nar one almost wrapped up. Will have them ready for review shortly. > Use a newer version of spring-data-redis > > > Key: NIFI-4811 > URL: https://issues.apache.org/jira/browse/NIFI-4811 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Johan Oceanstorm >Assignee: Mike Thomsen >Priority: Major > Fix For: 1.8.0 > > > NiFi uses version 1.8.3 of spring-data-redis (as you can see > [here|[https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-redis-bundle/pom.xml#L31].] > This version is an old version. The newer stable version is 2.0.2. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-5383) Use ${project.version} instead of hard-coded version in nifi-redis-bundle
[
https://issues.apache.org/jira/browse/NIFI-5383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16535722#comment-16535722
]
Mike Thomsen commented on NIFI-5383:
[~joewitt] shouldn't this be resolved as "won't fix" with an explanation
instead of left open?
> Use ${project.version} instead of hard-coded version in nifi-redis-bundle
> -
>
> Key: NIFI-5383
> URL: https://issues.apache.org/jira/browse/NIFI-5383
> Project: Apache NiFi
> Issue Type: Improvement
>Reporter: Jan Hentschel
>Assignee: Mike Thomsen
>Priority: Trivial
>
> Currently the {{nifi-redis-bundle}} module and its sub-modules use a
> hard-coded NiFi version. To make version updates easier this should be
> replaced by {{${project.version}}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFI-4811) Use a newer version of spring-data-redis
[ https://issues.apache.org/jira/browse/NIFI-4811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16535721#comment-16535721 ] Mike Thomsen commented on NIFI-4811: [~joewitt] yeah, I did a L scan. I probably didn't take into consideration removals. I should have time this weekend to do a full comparison. > Use a newer version of spring-data-redis > > > Key: NIFI-4811 > URL: https://issues.apache.org/jira/browse/NIFI-4811 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Johan Oceanstorm >Assignee: Mike Thomsen >Priority: Major > Fix For: 1.8.0 > > > NiFi uses version 1.8.3 of spring-data-redis (as you can see > [here|[https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-redis-bundle/pom.xml#L31].] > This version is an old version. The newer stable version is 2.0.2. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-5318) Implement NiFi test harness
[ https://issues.apache.org/jira/browse/NIFI-5318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16535712#comment-16535712 ] ASF GitHub Bot commented on NIFI-5318: -- Github user peter-gergely-horvath commented on the issue: https://github.com/apache/nifi/pull/2843 @joewitt The idea is that this component would allow you to write Java test cases against your NiFi flow. Something like [NiFiFlowTest](https://github.com/peter-gergely-horvath/nifi/blob/master/nifi-testharness/src/test/java/org/apache/nifi/testharness/samples/NiFiFlowTest.java#L101). The JavaDoc on [TestNiFiInstance](https://github.com/peter-gergely-horvath/nifi/blob/master/nifi-testharness/src/main/java/org/apache/nifi/testharness/TestNiFiInstance.java#L45) should explain the concept quite well. > Implement NiFi test harness > --- > > Key: NIFI-5318 > URL: https://issues.apache.org/jira/browse/NIFI-5318 > Project: Apache NiFi > Issue Type: New Feature >Reporter: Peter Horvath >Priority: Major > > Currently, it is not really possible to automatically test the behaviour of a > specific NiFi flow and make unit test type asserts if it works as expected. > For example, if the expected behaviour of a NiFi flow is that a file placed > to a specific directory will trigger some operation after which some output > file will appear at another directory, once currently can only do one thing: > test the NiFi flow manually. > Manual testing is especially hard to manage if a NiFi flow is being actively > developed: any change to a complex, existing NiFi flow might require a lot of > manual testing just to ensure there are no regressions introduced. > Some kind of Java API that allows managing a NiFi instance and manipulating > flow deployments like for example, [Codehaus > Cargo|]https://codehaus-cargo.github.io/] would be of great help. > > > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi issue #2843: NIFI-5318 Implement NiFi test harness: initial commit of n...
Github user peter-gergely-horvath commented on the issue: https://github.com/apache/nifi/pull/2843 @joewitt The idea is that this component would allow you to write Java test cases against your NiFi flow. Something like [NiFiFlowTest](https://github.com/peter-gergely-horvath/nifi/blob/master/nifi-testharness/src/test/java/org/apache/nifi/testharness/samples/NiFiFlowTest.java#L101). The JavaDoc on [TestNiFiInstance](https://github.com/peter-gergely-horvath/nifi/blob/master/nifi-testharness/src/main/java/org/apache/nifi/testharness/TestNiFiInstance.java#L45) should explain the concept quite well. ---
[jira] [Commented] (NIFI-5318) Implement NiFi test harness
[
https://issues.apache.org/jira/browse/NIFI-5318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16535709#comment-16535709
]
ASF GitHub Bot commented on NIFI-5318:
--
Github user peter-gergely-horvath commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2843#discussion_r200807442
--- Diff:
nifi-toolkit/nifi-toolkit-tls/src/main/java/org/apache/nifi/test/util/FileUtils.java
---
@@ -0,0 +1,84 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+
+
+package org.apache.nifi.test.util;
+
+import java.io.File;
+import java.io.IOException;
+import java.nio.file.*;
+import java.nio.file.attribute.BasicFileAttributes;
+import java.util.Arrays;
+
+public final class FileUtils {
+
+
+private static final String MAC_DS_STORE_NAME = ".DS_Store";
--- End diff --
I think this is probably fine: if you look at the code usages, it is used
to avoid symlinking a Mac system directory; it should not have any side effect
on other systems.
> Implement NiFi test harness
> ---
>
> Key: NIFI-5318
> URL: https://issues.apache.org/jira/browse/NIFI-5318
> Project: Apache NiFi
> Issue Type: New Feature
>Reporter: Peter Horvath
>Priority: Major
>
> Currently, it is not really possible to automatically test the behaviour of a
> specific NiFi flow and make unit test type asserts if it works as expected.
> For example, if the expected behaviour of a NiFi flow is that a file placed
> to a specific directory will trigger some operation after which some output
> file will appear at another directory, once currently can only do one thing:
> test the NiFi flow manually.
> Manual testing is especially hard to manage if a NiFi flow is being actively
> developed: any change to a complex, existing NiFi flow might require a lot of
> manual testing just to ensure there are no regressions introduced.
> Some kind of Java API that allows managing a NiFi instance and manipulating
> flow deployments like for example, [Codehaus
> Cargo|]https://codehaus-cargo.github.io/] would be of great help.
>
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] nifi pull request #2843: NIFI-5318 Implement NiFi test harness: initial comm...
Github user peter-gergely-horvath commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2843#discussion_r200807442
--- Diff:
nifi-toolkit/nifi-toolkit-tls/src/main/java/org/apache/nifi/test/util/FileUtils.java
---
@@ -0,0 +1,84 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+
+
+package org.apache.nifi.test.util;
+
+import java.io.File;
+import java.io.IOException;
+import java.nio.file.*;
+import java.nio.file.attribute.BasicFileAttributes;
+import java.util.Arrays;
+
+public final class FileUtils {
+
+
+private static final String MAC_DS_STORE_NAME = ".DS_Store";
--- End diff --
I think this is probably fine: if you look at the code usages, it is used
to avoid symlinking a Mac system directory; it should not have any side effect
on other systems.
---
[jira] [Commented] (NIFI-5318) Implement NiFi test harness
[
https://issues.apache.org/jira/browse/NIFI-5318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16535707#comment-16535707
]
ASF GitHub Bot commented on NIFI-5318:
--
Github user peter-gergely-horvath commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2843#discussion_r200807322
--- Diff:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-documentation/src/main/java/org/apache/nifi/documentation/DocGenerator.java
---
@@ -57,6 +57,9 @@
* @param extensionMapping extension mapping
*/
public static void generate(final NiFiProperties properties, final
ExtensionMapping extensionMapping) {
+
--- End diff --
yes, indeed, this somehow slipped in: I'll fix in my fork and create a new
pull request later on.
> Implement NiFi test harness
> ---
>
> Key: NIFI-5318
> URL: https://issues.apache.org/jira/browse/NIFI-5318
> Project: Apache NiFi
> Issue Type: New Feature
>Reporter: Peter Horvath
>Priority: Major
>
> Currently, it is not really possible to automatically test the behaviour of a
> specific NiFi flow and make unit test type asserts if it works as expected.
> For example, if the expected behaviour of a NiFi flow is that a file placed
> to a specific directory will trigger some operation after which some output
> file will appear at another directory, once currently can only do one thing:
> test the NiFi flow manually.
> Manual testing is especially hard to manage if a NiFi flow is being actively
> developed: any change to a complex, existing NiFi flow might require a lot of
> manual testing just to ensure there are no regressions introduced.
> Some kind of Java API that allows managing a NiFi instance and manipulating
> flow deployments like for example, [Codehaus
> Cargo|]https://codehaus-cargo.github.io/] would be of great help.
>
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFI-5318) Implement NiFi test harness
[
https://issues.apache.org/jira/browse/NIFI-5318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16535708#comment-16535708
]
ASF GitHub Bot commented on NIFI-5318:
--
Github user peter-gergely-horvath commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2843#discussion_r200807326
--- Diff:
nifi-toolkit/nifi-toolkit-tls/src/main/java/org/apache/nifi/test/SimpleNiFiFlowDefinitionEditor.java
---
@@ -0,0 +1,144 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+
+
+package org.apache.nifi.test;
+
+import org.apache.nifi.test.api.FlowFileEditorCallback;
+import org.w3c.dom.Document;
+import org.w3c.dom.Node;
+
+import javax.xml.xpath.XPath;
+import javax.xml.xpath.XPathConstants;
+import javax.xml.xpath.XPathFactory;
+import java.util.LinkedList;
+
+
+/**
+ *
+ * A facility to describe simple, common changes to a NiFi flow before it
is installed to the test
+ * NiFi instance. Intended to be used by
+ * {@link
TestNiFiInstance.Builder#modifyFlowXmlBeforeInstalling(FlowFileEditorCallback)}
+ *
+ *
+ *
+ * The desired edits can be configured via the {@link Builder} object
returned by the {@link #builder()}
+ * method. Once fully configured, the {@link Builder#build()} emits a
{@code FlowFileEditorCallback}
+ * object that can be passed to
+ * {@link
TestNiFiInstance.Builder#modifyFlowXmlBeforeInstalling(FlowFileEditorCallback)}.
+ *
+ *
+ *
+ * CAUTION: THIS IS AN EXPERIMENTAL API: EXPECT CHANGES!
+ * Efforts will be made to retain backwards API compatibility, but
+ * no guarantee is given.
+ *
+ *
+ * @see
TestNiFiInstance.Builder#modifyFlowXmlBeforeInstalling(FlowFileEditorCallback)
+ *
+ * @author Peter G. Horvath
--- End diff --
I'll fix in my fork and create a new pull request later on.
> Implement NiFi test harness
> ---
>
> Key: NIFI-5318
> URL: https://issues.apache.org/jira/browse/NIFI-5318
> Project: Apache NiFi
> Issue Type: New Feature
>Reporter: Peter Horvath
>Priority: Major
>
> Currently, it is not really possible to automatically test the behaviour of a
> specific NiFi flow and make unit test type asserts if it works as expected.
> For example, if the expected behaviour of a NiFi flow is that a file placed
> to a specific directory will trigger some operation after which some output
> file will appear at another directory, once currently can only do one thing:
> test the NiFi flow manually.
> Manual testing is especially hard to manage if a NiFi flow is being actively
> developed: any change to a complex, existing NiFi flow might require a lot of
> manual testing just to ensure there are no regressions introduced.
> Some kind of Java API that allows managing a NiFi instance and manipulating
> flow deployments like for example, [Codehaus
> Cargo|]https://codehaus-cargo.github.io/] would be of great help.
>
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFI-5318) Implement NiFi test harness
[
https://issues.apache.org/jira/browse/NIFI-5318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16535706#comment-16535706
]
ASF GitHub Bot commented on NIFI-5318:
--
Github user peter-gergely-horvath commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2843#discussion_r200807310
--- Diff: nifi-docs/src/main/asciidoc/developer-guide.adoc ---
@@ -2296,6 +2296,32 @@ threads that should be used to run the Processor can
be set via the `setThreadCount(int)` method.
+=== Experimental NiFi Flow test harness
+
+NiFi now has an experimental feature for full end-to-end testing of flows.
This allows us
+to take a NiFi flow, install it to a test NiFi instance, run it and make
Java unit test
+like asserts regarding its behaviour.
+
+The class `org.apache.nifi.test.TestNiFiInstance` is a thin wrapper that
allows us
+to manipulate a NiFi installation and deploy a flow with some adjustments
+to its configuration, including changing processor properties and
replacing processor
+classes with mocks.
+
+In order to add the necessary classes to your project,
+you can use the Maven dependency:
+
+[source]
+
+
+ org.apache.nifi
+ nifi-test
+ ${nifi version}
+
+
+
+For further documentation, please consult the JavaDoc of
+`org.apache.nifi.test.TestNiFiInstance`. For samples, please take a look at
+link:https://github.com/apache/nifi/tree/master/nifi-test/src/test/java/org/apache/test/samples[samples
on GitHub^].
--- End diff --
The idea is that users should be able to write test cases in Java for NiFi
flows: these code samples linked there would demonstrate creating a test case
like that.
> Implement NiFi test harness
> ---
>
> Key: NIFI-5318
> URL: https://issues.apache.org/jira/browse/NIFI-5318
> Project: Apache NiFi
> Issue Type: New Feature
>Reporter: Peter Horvath
>Priority: Major
>
> Currently, it is not really possible to automatically test the behaviour of a
> specific NiFi flow and make unit test type asserts if it works as expected.
> For example, if the expected behaviour of a NiFi flow is that a file placed
> to a specific directory will trigger some operation after which some output
> file will appear at another directory, once currently can only do one thing:
> test the NiFi flow manually.
> Manual testing is especially hard to manage if a NiFi flow is being actively
> developed: any change to a complex, existing NiFi flow might require a lot of
> manual testing just to ensure there are no regressions introduced.
> Some kind of Java API that allows managing a NiFi instance and manipulating
> flow deployments like for example, [Codehaus
> Cargo|]https://codehaus-cargo.github.io/] would be of great help.
>
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] nifi pull request #2843: NIFI-5318 Implement NiFi test harness: initial comm...
Github user peter-gergely-horvath commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2843#discussion_r200807310
--- Diff: nifi-docs/src/main/asciidoc/developer-guide.adoc ---
@@ -2296,6 +2296,32 @@ threads that should be used to run the Processor can
be set via the `setThreadCount(int)` method.
+=== Experimental NiFi Flow test harness
+
+NiFi now has an experimental feature for full end-to-end testing of flows.
This allows us
+to take a NiFi flow, install it to a test NiFi instance, run it and make
Java unit test
+like asserts regarding its behaviour.
+
+The class `org.apache.nifi.test.TestNiFiInstance` is a thin wrapper that
allows us
+to manipulate a NiFi installation and deploy a flow with some adjustments
+to its configuration, including changing processor properties and
replacing processor
+classes with mocks.
+
+In order to add the necessary classes to your project,
+you can use the Maven dependency:
+
+[source]
+
+
+ org.apache.nifi
+ nifi-test
+ ${nifi version}
+
+
+
+For further documentation, please consult the JavaDoc of
+`org.apache.nifi.test.TestNiFiInstance`. For samples, please take a look at
+link:https://github.com/apache/nifi/tree/master/nifi-test/src/test/java/org/apache/test/samples[samples
on GitHub^].
--- End diff --
The idea is that users should be able to write test cases in Java for NiFi
flows: these code samples linked there would demonstrate creating a test case
like that.
---
[GitHub] nifi pull request #2843: NIFI-5318 Implement NiFi test harness: initial comm...
Github user peter-gergely-horvath commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2843#discussion_r200807326
--- Diff:
nifi-toolkit/nifi-toolkit-tls/src/main/java/org/apache/nifi/test/SimpleNiFiFlowDefinitionEditor.java
---
@@ -0,0 +1,144 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+
+
+package org.apache.nifi.test;
+
+import org.apache.nifi.test.api.FlowFileEditorCallback;
+import org.w3c.dom.Document;
+import org.w3c.dom.Node;
+
+import javax.xml.xpath.XPath;
+import javax.xml.xpath.XPathConstants;
+import javax.xml.xpath.XPathFactory;
+import java.util.LinkedList;
+
+
+/**
+ *
+ * A facility to describe simple, common changes to a NiFi flow before it
is installed to the test
+ * NiFi instance. Intended to be used by
+ * {@link
TestNiFiInstance.Builder#modifyFlowXmlBeforeInstalling(FlowFileEditorCallback)}
+ *
+ *
+ *
+ * The desired edits can be configured via the {@link Builder} object
returned by the {@link #builder()}
+ * method. Once fully configured, the {@link Builder#build()} emits a
{@code FlowFileEditorCallback}
+ * object that can be passed to
+ * {@link
TestNiFiInstance.Builder#modifyFlowXmlBeforeInstalling(FlowFileEditorCallback)}.
+ *
+ *
+ *
+ * CAUTION: THIS IS AN EXPERIMENTAL API: EXPECT CHANGES!
+ * Efforts will be made to retain backwards API compatibility, but
+ * no guarantee is given.
+ *
+ *
+ * @see
TestNiFiInstance.Builder#modifyFlowXmlBeforeInstalling(FlowFileEditorCallback)
+ *
+ * @author Peter G. Horvath
--- End diff --
I'll fix in my fork and create a new pull request later on.
---
[GitHub] nifi pull request #2843: NIFI-5318 Implement NiFi test harness: initial comm...
Github user peter-gergely-horvath commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2843#discussion_r200807322
--- Diff:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-documentation/src/main/java/org/apache/nifi/documentation/DocGenerator.java
---
@@ -57,6 +57,9 @@
* @param extensionMapping extension mapping
*/
public static void generate(final NiFiProperties properties, final
ExtensionMapping extensionMapping) {
+
--- End diff --
yes, indeed, this somehow slipped in: I'll fix in my fork and create a new
pull request later on.
---
[jira] [Commented] (NIFI-5318) Implement NiFi test harness
[ https://issues.apache.org/jira/browse/NIFI-5318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16535702#comment-16535702 ] ASF GitHub Bot commented on NIFI-5318: -- Github user peter-gergely-horvath commented on the issue: https://github.com/apache/nifi/pull/2843 Hi @joewitt and @MikeThomsen I messed up this pull request with some incorrectly merged code changes including the non-NiFi groupId and the whitespace change in nifi-toolkit-tls module etc. Sorry about that! For now, I will close any pull requests. Since you had some really valuable ideas, could you please review the code in https://github.com/peter-gergely-horvath/nifi/tree/master/nifi-testharness and create issues with the inputs you have? I've implemented changes you mentioned in this pull request there already. > Implement NiFi test harness > --- > > Key: NIFI-5318 > URL: https://issues.apache.org/jira/browse/NIFI-5318 > Project: Apache NiFi > Issue Type: New Feature >Reporter: Peter Horvath >Priority: Major > > Currently, it is not really possible to automatically test the behaviour of a > specific NiFi flow and make unit test type asserts if it works as expected. > For example, if the expected behaviour of a NiFi flow is that a file placed > to a specific directory will trigger some operation after which some output > file will appear at another directory, once currently can only do one thing: > test the NiFi flow manually. > Manual testing is especially hard to manage if a NiFi flow is being actively > developed: any change to a complex, existing NiFi flow might require a lot of > manual testing just to ensure there are no regressions introduced. > Some kind of Java API that allows managing a NiFi instance and manipulating > flow deployments like for example, [Codehaus > Cargo|]https://codehaus-cargo.github.io/] would be of great help. > > > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-5318) Implement NiFi test harness
[ https://issues.apache.org/jira/browse/NIFI-5318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16535704#comment-16535704 ] ASF GitHub Bot commented on NIFI-5318: -- Github user peter-gergely-horvath commented on a diff in the pull request: https://github.com/apache/nifi/pull/2843#discussion_r200807283 --- Diff: nifi-test/pom.xml --- @@ -0,0 +1,145 @@ +http://maven.apache.org/POM/4.0.0; xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance; + xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd;> + +nifi +org.apache.nifi +1.8.0-SNAPSHOT + + +4.0.0 + --- End diff -- Yes, this was incorrect: I'll fix in my fork and create a new pull request later on. > Implement NiFi test harness > --- > > Key: NIFI-5318 > URL: https://issues.apache.org/jira/browse/NIFI-5318 > Project: Apache NiFi > Issue Type: New Feature >Reporter: Peter Horvath >Priority: Major > > Currently, it is not really possible to automatically test the behaviour of a > specific NiFi flow and make unit test type asserts if it works as expected. > For example, if the expected behaviour of a NiFi flow is that a file placed > to a specific directory will trigger some operation after which some output > file will appear at another directory, once currently can only do one thing: > test the NiFi flow manually. > Manual testing is especially hard to manage if a NiFi flow is being actively > developed: any change to a complex, existing NiFi flow might require a lot of > manual testing just to ensure there are no regressions introduced. > Some kind of Java API that allows managing a NiFi instance and manipulating > flow deployments like for example, [Codehaus > Cargo|]https://codehaus-cargo.github.io/] would be of great help. > > > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-5318) Implement NiFi test harness
[ https://issues.apache.org/jira/browse/NIFI-5318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16535705#comment-16535705 ] ASF GitHub Bot commented on NIFI-5318: -- Github user peter-gergely-horvath commented on a diff in the pull request: https://github.com/apache/nifi/pull/2843#discussion_r200807286 --- Diff: nifi-test/pom.xml --- @@ -0,0 +1,145 @@ +http://maven.apache.org/POM/4.0.0; xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance; + xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd;> + +nifi +org.apache.nifi +1.8.0-SNAPSHOT + + +4.0.0 + +com.thinkbiganalytics +nifi-test --- End diff -- It makes sense. I'll fix in my fork and create a new pull request later on. > Implement NiFi test harness > --- > > Key: NIFI-5318 > URL: https://issues.apache.org/jira/browse/NIFI-5318 > Project: Apache NiFi > Issue Type: New Feature >Reporter: Peter Horvath >Priority: Major > > Currently, it is not really possible to automatically test the behaviour of a > specific NiFi flow and make unit test type asserts if it works as expected. > For example, if the expected behaviour of a NiFi flow is that a file placed > to a specific directory will trigger some operation after which some output > file will appear at another directory, once currently can only do one thing: > test the NiFi flow manually. > Manual testing is especially hard to manage if a NiFi flow is being actively > developed: any change to a complex, existing NiFi flow might require a lot of > manual testing just to ensure there are no regressions introduced. > Some kind of Java API that allows managing a NiFi instance and manipulating > flow deployments like for example, [Codehaus > Cargo|]https://codehaus-cargo.github.io/] would be of great help. > > > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi pull request #2843: NIFI-5318 Implement NiFi test harness: initial comm...
Github user peter-gergely-horvath commented on a diff in the pull request: https://github.com/apache/nifi/pull/2843#discussion_r200807286 --- Diff: nifi-test/pom.xml --- @@ -0,0 +1,145 @@ +http://maven.apache.org/POM/4.0.0; xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance; + xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd;> + +nifi +org.apache.nifi +1.8.0-SNAPSHOT + + +4.0.0 + +com.thinkbiganalytics +nifi-test --- End diff -- It makes sense. I'll fix in my fork and create a new pull request later on. ---
[GitHub] nifi issue #2843: NIFI-5318 Implement NiFi test harness: initial commit of n...
Github user peter-gergely-horvath commented on the issue: https://github.com/apache/nifi/pull/2843 Hi @joewitt and @MikeThomsen I messed up this pull request with some incorrectly merged code changes including the non-NiFi groupId and the whitespace change in nifi-toolkit-tls module etc. Sorry about that! For now, I will close any pull requests. Since you had some really valuable ideas, could you please review the code in https://github.com/peter-gergely-horvath/nifi/tree/master/nifi-testharness and create issues with the inputs you have? I've implemented changes you mentioned in this pull request there already. ---
[GitHub] nifi pull request #2843: NIFI-5318 Implement NiFi test harness: initial comm...
Github user peter-gergely-horvath commented on a diff in the pull request: https://github.com/apache/nifi/pull/2843#discussion_r200807283 --- Diff: nifi-test/pom.xml --- @@ -0,0 +1,145 @@ +http://maven.apache.org/POM/4.0.0; xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance; + xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd;> + +nifi +org.apache.nifi +1.8.0-SNAPSHOT + + +4.0.0 + --- End diff -- Yes, this was incorrect: I'll fix in my fork and create a new pull request later on. ---
[jira] [Commented] (NIFI-5318) Implement NiFi test harness
[ https://issues.apache.org/jira/browse/NIFI-5318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16535691#comment-16535691 ] ASF GitHub Bot commented on NIFI-5318: -- Github user peter-gergely-horvath commented on the issue: https://github.com/apache/nifi/pull/2855 Closed for now to avoid confusion > Implement NiFi test harness > --- > > Key: NIFI-5318 > URL: https://issues.apache.org/jira/browse/NIFI-5318 > Project: Apache NiFi > Issue Type: New Feature >Reporter: Peter Horvath >Priority: Major > > Currently, it is not really possible to automatically test the behaviour of a > specific NiFi flow and make unit test type asserts if it works as expected. > For example, if the expected behaviour of a NiFi flow is that a file placed > to a specific directory will trigger some operation after which some output > file will appear at another directory, once currently can only do one thing: > test the NiFi flow manually. > Manual testing is especially hard to manage if a NiFi flow is being actively > developed: any change to a complex, existing NiFi flow might require a lot of > manual testing just to ensure there are no regressions introduced. > Some kind of Java API that allows managing a NiFi instance and manipulating > flow deployments like for example, [Codehaus > Cargo|]https://codehaus-cargo.github.io/] would be of great help. > > > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-5318) Implement NiFi test harness
[ https://issues.apache.org/jira/browse/NIFI-5318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16535692#comment-16535692 ] ASF GitHub Bot commented on NIFI-5318: -- Github user peter-gergely-horvath closed the pull request at: https://github.com/apache/nifi/pull/2855 > Implement NiFi test harness > --- > > Key: NIFI-5318 > URL: https://issues.apache.org/jira/browse/NIFI-5318 > Project: Apache NiFi > Issue Type: New Feature >Reporter: Peter Horvath >Priority: Major > > Currently, it is not really possible to automatically test the behaviour of a > specific NiFi flow and make unit test type asserts if it works as expected. > For example, if the expected behaviour of a NiFi flow is that a file placed > to a specific directory will trigger some operation after which some output > file will appear at another directory, once currently can only do one thing: > test the NiFi flow manually. > Manual testing is especially hard to manage if a NiFi flow is being actively > developed: any change to a complex, existing NiFi flow might require a lot of > manual testing just to ensure there are no regressions introduced. > Some kind of Java API that allows managing a NiFi instance and manipulating > flow deployments like for example, [Codehaus > Cargo|]https://codehaus-cargo.github.io/] would be of great help. > > > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi pull request #2855: NIFI-5318 Implement NiFi test harness: initial comm...
Github user peter-gergely-horvath closed the pull request at: https://github.com/apache/nifi/pull/2855 ---
[GitHub] nifi issue #2855: NIFI-5318 Implement NiFi test harness: initial commit of n...
Github user peter-gergely-horvath commented on the issue: https://github.com/apache/nifi/pull/2855 Closed for now to avoid confusion ---
[jira] [Commented] (NIFI-5318) Implement NiFi test harness
[ https://issues.apache.org/jira/browse/NIFI-5318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16535690#comment-16535690 ] ASF GitHub Bot commented on NIFI-5318: -- GitHub user peter-gergely-horvath opened a pull request: https://github.com/apache/nifi/pull/2855 NIFI-5318 Implement NiFi test harness: initial commit of nifi-testharness Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [x] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/peter-gergely-horvath/nifi master Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2855.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2855 commit 7916daf9e3fcdc5a396ae7815069c3d8bfa168d6 Author: Peter G. Horvath Date: 2018-07-07T08:48:30Z NIFI-5318 Implement NiFi test harness: initial commit of nifi-testharness > Implement NiFi test harness > --- > > Key: NIFI-5318 > URL: https://issues.apache.org/jira/browse/NIFI-5318 > Project: Apache NiFi > Issue Type: New Feature >Reporter: Peter Horvath >Priority: Major > > Currently, it is not really possible to automatically test the behaviour of a > specific NiFi flow and make unit test type asserts if it works as expected. > For example, if the expected behaviour of a NiFi flow is that a file placed > to a specific directory will trigger some operation after which some output > file will appear at another directory, once currently can only do one thing: > test the NiFi flow manually. > Manual testing is especially hard to manage if a NiFi flow is being actively > developed: any change to a complex, existing NiFi flow might require a lot of > manual testing just to ensure there are no regressions introduced. > Some kind of Java API that allows managing a NiFi instance and manipulating > flow deployments like for example, [Codehaus > Cargo|]https://codehaus-cargo.github.io/] would be of great help. > > > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi pull request #2855: NIFI-5318 Implement NiFi test harness: initial comm...
GitHub user peter-gergely-horvath opened a pull request: https://github.com/apache/nifi/pull/2855 NIFI-5318 Implement NiFi test harness: initial commit of nifi-testharness Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [x] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/peter-gergely-horvath/nifi master Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2855.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2855 commit 7916daf9e3fcdc5a396ae7815069c3d8bfa168d6 Author: Peter G. Horvath Date: 2018-07-07T08:48:30Z NIFI-5318 Implement NiFi test harness: initial commit of nifi-testharness ---
