[jira] [Commented] (NIFI-6681) NIFI PutHbaseRecord
[ https://issues.apache.org/jira/browse/NIFI-6681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16933059#comment-16933059 ] Amogh commented on NIFI-6681: - Yes whatever value comes from my flow file it should be made has rowid and it will never be null in my case > NIFI PutHbaseRecord > --- > > Key: NIFI-6681 > URL: https://issues.apache.org/jira/browse/NIFI-6681 > Project: Apache NiFi > Issue Type: Bug > Components: Flow Versioning >Affects Versions: 1.8.0 > Environment: Linux >Reporter: Amogh >Priority: Minor > Attachments: CSVReader.PNG, Conf.PNG, Error.png, Hbase.PNG > > > Hi All, > > I am getting below error while loading a data hbase from nifi, Even though > the column is not null data is not loading and i'm getting the error -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] patricker edited a comment on issue #3726: NIFI-5816 Switch SFTP processors to use SSHJ
patricker edited a comment on issue #3726: NIFI-5816 Switch SFTP processors to use SSHJ URL: https://github.com/apache/nifi/pull/3726#issuecomment-532953793 I'm excited by these changes, I was working on some changes ~ a year ago to add basic SSH processors like `ExecuteSSH`, but wasn't all that thrilled by the existing library. To prepare, I added some unit tests to make sure I didn't break SFTP: [#2846]. Have you looked to see if any of these features we are trying to resolve, like larger key sizes, can be tested by extending the existing test SSH Server I added: nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/test/java/org/apache/nifi/processors/standard/util/SSHTestServer.java? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [nifi] patricker commented on issue #3726: NIFI-5816 Switch SFTP processors to use SSHJ
patricker commented on issue #3726: NIFI-5816 Switch SFTP processors to use SSHJ URL: https://github.com/apache/nifi/pull/3726#issuecomment-532953793 I'm excited by these changes, I was working on some changes ~ a year ago to add basic SSH processors like `ExecuteSSH`, but wasn't all that thrilled by the existing library. To prepare, I added some unit tests to make sure I didn't break SFTP: NIFI-5381. Have you looked to see if any of these features we are trying to resolve, like larger key sizes, can be tested by extending the existing test SSH Server I added: nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/test/java/org/apache/nifi/processors/standard/util/SSHTestServer.java? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [nifi] granthenke commented on issue #3732: NIFI-6662: Adding Kudu Lookup Service
granthenke commented on issue #3732: NIFI-6662: Adding Kudu Lookup Service URL: https://github.com/apache/nifi/pull/3732#issuecomment-532936797 LGTM This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (NIFI-6583) need a way to report nifi metrics to Azure Log Analytics
[ https://issues.apache.org/jira/browse/NIFI-6583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Seokwon Yang updated NIFI-6583: --- Summary: need a way to report nifi metrics to Azure Log Analytics (was: need a way to report nifi metrics to Azure Log Analystics) > need a way to report nifi metrics to Azure Log Analytics > > > Key: NIFI-6583 > URL: https://issues.apache.org/jira/browse/NIFI-6583 > Project: Apache NiFi > Issue Type: Improvement > Components: Examples >Affects Versions: 1.9.2 >Reporter: Seokwon Yang >Priority: Minor > Labels: patch > Fix For: 1.10.0 > > Original Estimate: 168h > Time Spent: 20m > Remaining Estimate: 167h 40m > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (NIFI-6583) need a way to report nifi metrics to Azure Log Analystics
[ https://issues.apache.org/jira/browse/NIFI-6583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Seokwon Yang updated NIFI-6583: --- Description: (was: I see multiple options to report metrics to azure log analystics. 1) Implement a new ReporterService like GraphiteMetricReporterService and reuse the existing MetricsReportingTask: nifi-metrics-reporting-task only implements one ControllerService, called GraphiteMetricReporterService as a reporting service, that is used by MetricsReportingTask. 2) Implement a new ReportingTask: This is the pattern I often find multiple reporting tasks nar-bundle in github. 3) Implement a new ReportingTask based on MetricService utilities under nifi-standard-bundle. 4) create a reporting script and run the script in the scription execution engine task. ) > need a way to report nifi metrics to Azure Log Analystics > - > > Key: NIFI-6583 > URL: https://issues.apache.org/jira/browse/NIFI-6583 > Project: Apache NiFi > Issue Type: Improvement > Components: Examples >Affects Versions: 1.9.2 >Reporter: Seokwon Yang >Priority: Minor > Labels: patch > Fix For: 1.10.0 > > Original Estimate: 168h > Time Spent: 20m > Remaining Estimate: 167h 40m > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (NIFI-6583) need a way to report nifi metrics to Azure Log Analystics
[ https://issues.apache.org/jira/browse/NIFI-6583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Seokwon Yang updated NIFI-6583: --- Issue Type: Improvement (was: New Feature) > need a way to report nifi metrics to Azure Log Analystics > - > > Key: NIFI-6583 > URL: https://issues.apache.org/jira/browse/NIFI-6583 > Project: Apache NiFi > Issue Type: Improvement > Components: Examples >Affects Versions: 1.9.2 >Reporter: Seokwon Yang >Priority: Minor > Labels: patch > Fix For: 1.10.0 > > Original Estimate: 168h > Time Spent: 20m > Remaining Estimate: 167h 40m > > I see multiple options to report metrics to azure log analystics. > 1) Implement a new ReporterService like GraphiteMetricReporterService and > reuse the existing MetricsReportingTask: nifi-metrics-reporting-task only > implements one ControllerService, called GraphiteMetricReporterService as a > reporting service, that is used by MetricsReportingTask. > 2) Implement a new ReportingTask: This is the pattern I often find multiple > reporting tasks nar-bundle in github. > 3) Implement a new ReportingTask based on MetricService utilities under > nifi-standard-bundle. > 4) create a reporting script and run the script in the scription execution > engine task. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (NIFI-6653) Only allow bootstrap port to be set during initial startup
[ https://issues.apache.org/jira/browse/NIFI-6653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andy LoPresto resolved NIFI-6653. - Resolution: Fixed > Only allow bootstrap port to be set during initial startup > -- > > Key: NIFI-6653 > URL: https://issues.apache.org/jira/browse/NIFI-6653 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Reporter: Kevin Doran >Assignee: Kevin Doran >Priority: Minor > Fix For: 1.10.0 > > Time Spent: 0.5h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-6653) Only allow bootstrap port to be set during initial startup
[ https://issues.apache.org/jira/browse/NIFI-6653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16932932#comment-16932932 ] ASF subversion and git services commented on NIFI-6653: --- Commit 3686ee2a164266543daea7395c211a1bdb31afb9 in nifi's branch refs/heads/master from Kevin Doran [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=3686ee2 ] NIFI-6653 Change bootstrap port command handling This closes #3722. Signed-off-by: Andy LoPresto > Only allow bootstrap port to be set during initial startup > -- > > Key: NIFI-6653 > URL: https://issues.apache.org/jira/browse/NIFI-6653 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Reporter: Kevin Doran >Assignee: Kevin Doran >Priority: Minor > Fix For: 1.10.0 > > Time Spent: 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] asfgit closed pull request #3722: NIFI-6653 Change bootstrap port command handling
asfgit closed pull request #3722: NIFI-6653 Change bootstrap port command handling URL: https://github.com/apache/nifi/pull/3722 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (NIFI-6125) NiFi Docker image does not start when PATH variable does not match default
[ https://issues.apache.org/jira/browse/NIFI-6125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16932908#comment-16932908 ] Qiong Wei commented on NIFI-6125: - start looking into this. > NiFi Docker image does not start when PATH variable does not match default > -- > > Key: NIFI-6125 > URL: https://issues.apache.org/jira/browse/NIFI-6125 > Project: Apache NiFi > Issue Type: Bug >Reporter: Eric Yang >Priority: Major > > NiFI Docker image entrypoint startup.sh script does not have -l flag to > enforce login with /etc/profile or setup PATH variable. Some command such as > "hostname" and "sed" depends on the version on the PATH variable. If someone > start docker image with --env-file flag to override PATH variable to run > hostname and sed from alternate mounted location. This allows user to hijack > the startup script unintentionally. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] MuazmaZ opened a new pull request #3751: NIFI-6665 updated NiFi toolkit bind issue with cli.bat file
MuazmaZ opened a new pull request #3751: NIFI-6665 updated NiFi toolkit bind issue with cli.bat file URL: https://github.com/apache/nifi/pull/3751 Thank you for submitting a contribution to Apache NiFi. ### Description of PR NIFI-6665 Changed cli.bat from SET JAVA_PARAMS=-cp %LIB_DIR%* %JAVA_OPTS% org.apache.nifi.toolkit.cli.CLIMain to SET JAVA_PARAMS=-cp %LIB_DIR%* %JAVA_OPTS% --add-modules=java.xml.bind org.apache.nifi.toolkit.cli.CLIMain It fixes the bug when any cli command executed with cli.bat file the exception comes with like "class not found" ### For all changes: - [X ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ X] Does your PR title start with **NIFI-** where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ X] Has your PR been rebased against the latest commit within the target branch (typically `master`)? - [ ] Is your initial contribution a single, squashed commit? _Additional commits in response to PR reviewer feedback should be made on this branch and pushed to allow change tracking. Do not `squash` or use `--force` when pushing to allow for clean monitoring of changes._ ### For code changes: - [ X] Have you ensured that the full suite of tests is executed via `mvn -Pcontrib-check clean install` at the root `nifi` folder? - [ ] Have you written or updated unit tests to verify your changes? - [X ] Have you verified that the full build is successful on both JDK 8 and JDK 11? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the `LICENSE` file, including the main `LICENSE` file under `nifi-assembly`? - [ ] If applicable, have you updated the `NOTICE` file, including the main `NOTICE` file found under `nifi-assembly`? - [ ] If adding new Properties, have you added `.displayName` in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [X ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [nifi] sjyang18 closed pull request #3737: NIFI-6583 metrics reporting to azure log ws
sjyang18 closed pull request #3737: NIFI-6583 metrics reporting to azure log ws URL: https://github.com/apache/nifi/pull/3737 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [nifi] SamHjelmfelt commented on issue #3732: NIFI-6662: Adding Kudu Lookup Service
SamHjelmfelt commented on issue #3732: NIFI-6662: Adding Kudu Lookup Service URL: https://github.com/apache/nifi/pull/3732#issuecomment-532887385 The Travis build for Java 8 EN succeeded, but the others failed due to a check-style issue in nifi-shell-authorizer (src/test/java/org/apache/nifi/authorization/ShellUserGroupProviderIT.java). This check-style issue was fixed in NIFI-6400. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [nifi] sushilkm opened a new pull request #3750: NIFI-4913 Added "-E" with sudo to read environment variables
sushilkm opened a new pull request #3750: NIFI-4913 Added "-E" with sudo to
read environment variables
URL: https://github.com/apache/nifi/pull/3750
Description of PR
fixes bug NIFI-4913
Specifying "run.as" in bootstrap.properties causes environment variables to
not be passed to NiFi
Added "-E" with sudo to read environment variables set in current shell
environment while running nifi with bootstrap property "run.as" set as to some
user
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
### For all changes:
- [x] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?
- [x] Does your PR title start with **NIFI-** where is the JIRA
number you are trying to resolve? Pay particular attention to the hyphen "-"
character.
- [x] Has your PR been rebased against the latest commit within the target
branch (typically `master`)?
- [x] Is your initial contribution a single, squashed commit? _Additional
commits in response to PR reviewer feedback should be made on this branch and
pushed to allow change tracking. Do not `squash` or use `--force` when pushing
to allow for clean monitoring of changes._
### For code changes:
- [x] Have you ensured that the full suite of tests is executed via `mvn
-Pcontrib-check clean install` at the root `nifi` folder?
- [ ] Have you written or updated unit tests to verify your changes?
- [ ] Have you verified that the full build is successful on both JDK 8 and
JDK 11?
- [ ] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the `LICENSE` file, including the main
`LICENSE` file under `nifi-assembly`?
- [ ] If applicable, have you updated the `NOTICE` file, including the main
`NOTICE` file found under `nifi-assembly`?
- [ ] If adding new Properties, have you added `.displayName` in addition to
.name (programmatic access) for each of the new properties?
### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in which
it is rendered?
### Note:
Please ensure that once the PR is submitted, you check travis-ci for build
issues and submit an update to your PR as soon as possible.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [nifi] sburges opened a new pull request #3749: NIFI-4012 Azure Event Hub UI typos and cleanup
sburges opened a new pull request #3749: NIFI-4012 Azure Event Hub UI typos and
cleanup
URL: https://github.com/apache/nifi/pull/3749
Description of PR
Cleans up property names and descriptions in the Azure Event Hubs processors.
Namely:
1. Correct spelling
2. Consistent and correct product naming (Azure Event Hubs and an event hub)
3. Do not repeat Event Hubs in every property as it is already an event hub
processor so the phrasing is redundant
4. Leave existing property names as is but update display values where
appropriate
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
### For all changes:
- [X] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?
- [X] Does your PR title start with **NIFI-** where is the JIRA
number you are trying to resolve? Pay particular attention to the hyphen "-"
character.
- [X] Has your PR been rebased against the latest commit within the target
branch (typically `master`)?
- [X] Is your initial contribution a single, squashed commit? _Additional
commits in response to PR reviewer feedback should be made on this branch and
pushed to allow change tracking. Do not `squash` or use `--force` when pushing
to allow for clean monitoring of changes._
### For code changes:
- [X] Have you ensured that the full suite of tests is executed via `mvn
-Pcontrib-check clean install` at the root `nifi` folder?
- [X] Have you written or updated unit tests to verify your changes?
- [ ] Have you verified that the full build is successful on both JDK 8 and
JDK 11?
- [] If adding new dependencies to the code, are these dependencies licensed
in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [] If applicable, have you updated the `LICENSE` file, including the main
`LICENSE` file under `nifi-assembly`?
- [] If applicable, have you updated the `NOTICE` file, including the main
`NOTICE` file found under `nifi-assembly`?
- [X] If adding new Properties, have you added `.displayName` in addition to
.name (programmatic access) for each of the new properties?
### For documentation related changes:
- [X] Have you ensured that format looks appropriate for the output in which
it is rendered?
### Note:
Please ensure that once the PR is submitted, you check travis-ci for build
issues and submit an update to your PR as soon as possible.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Updated] (NIFI-4012) Azure Event Hub UI typos and cleanup
[ https://issues.apache.org/jira/browse/NIFI-4012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shayne Burgess updated NIFI-4012: - Attachment: (was: nifi-4012.patch.zip) > Azure Event Hub UI typos and cleanup > > > Key: NIFI-4012 > URL: https://issues.apache.org/jira/browse/NIFI-4012 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Andrew Grande >Priority: Trivial > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (NIFI-4012) Azure Event Hub UI typos and cleanup
[ https://issues.apache.org/jira/browse/NIFI-4012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shayne Burgess updated NIFI-4012: - Attachment: nifi-4012.patch.zip > Azure Event Hub UI typos and cleanup > > > Key: NIFI-4012 > URL: https://issues.apache.org/jira/browse/NIFI-4012 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Andrew Grande >Priority: Trivial > Attachments: nifi-4012.patch.zip > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (NIFI-3662) Components which allow users to alter the classpath at runtime should be 'restricted'
[ https://issues.apache.org/jira/browse/NIFI-3662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qiong Wei updated NIFI-3662: Attachment: NIFI-6189Offloading a shut down node will never end until a full reset.url > Components which allow users to alter the classpath at runtime should be > 'restricted' > - > > Key: NIFI-3662 > URL: https://issues.apache.org/jira/browse/NIFI-3662 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Reporter: Joseph Witt >Priority: Critical > Attachments: NIFI-6189Offloading a shut down node will never end > until a full reset.url > > > A lot of great work has gone into making it easy to run multiple versions of > the same component in parallel with isolated classloaders and to restructure > classloaders on the fly as needed to deal with certain libraries that do evil > things with statics and to enable users to add jars/code to the classpath of > their components at runtime as needed. For example, the HDFS processors can > now be extended to support protocols like WASB. > These things are all really valuable. > We also recently introduced the concept of restricted components to ensure > that special authorization is required for users to be able to use certain > processors which could easily be used to access portions of the system or > execute arbitrary code. > We should update the framework to detect components/extensions that allow > runtime classpath manipulation and automatically mark them as restricted > regardless of whether the developer did so. And we should update the > restricted components docs to articulate this. > By being able to alter the classpath at runtime this is just as justified as > the other reasons we had previously noted for marking a component > 'restricted'. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (NIFIREG-319) Remove code coverage instrumentation from nifi-fds js modules
[ https://issues.apache.org/jira/browse/NIFIREG-319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Aslan updated NIFIREG-319: Labels: UI (was: ) > Remove code coverage instrumentation from nifi-fds js modules > - > > Key: NIFIREG-319 > URL: https://issues.apache.org/jira/browse/NIFIREG-319 > Project: NiFi Registry > Issue Type: Improvement >Reporter: Scott Aslan >Assignee: Scott Aslan >Priority: Major > Labels: UI > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi-registry] scottyaslan opened a new pull request #229: [NIFIREG-319] remove code coverage instrumentation from nifi-fds js m…
scottyaslan opened a new pull request #229: [NIFIREG-319] remove code coverage instrumentation from nifi-fds js m… URL: https://github.com/apache/nifi-registry/pull/229 …odules This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Created] (NIFIREG-319) Remove code coverage instrumentation from nifi-fds js modules
Scott Aslan created NIFIREG-319: --- Summary: Remove code coverage instrumentation from nifi-fds js modules Key: NIFIREG-319 URL: https://issues.apache.org/jira/browse/NIFIREG-319 Project: NiFi Registry Issue Type: Improvement Reporter: Scott Aslan Assignee: Scott Aslan -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] tpalfy commented on a change in pull request #3746: NIFI-6679 - Patch MoveHDFS
tpalfy commented on a change in pull request #3746: NIFI-6679 - Patch MoveHDFS
URL: https://github.com/apache/nifi/pull/3746#discussion_r325823361
##
File path:
nifi-nar-bundles/nifi-hadoop-bundle/nifi-hdfs-processors/src/main/java/org/apache/nifi/processors/hadoop/MoveHDFS.java
##
@@ -357,9 +357,10 @@ public Object run() {
if (destinationExists) {
switch (processorConfig.getConflictResolution()) {
case REPLACE_RESOLUTION:
-if (hdfs.delete(file, false)) {
+// Remove destination file (newFile) to
replace
Review comment:
Maybe we could add some unit tests, something like this:
```java
@Test
public void testPutWhenAlreadyExistingShouldFailWhenFAIL_RESOLUTION()
throws IOException {
testPutWhenAlreadyExisting(MoveHDFS.FAIL_RESOLUTION,
MoveHDFS.REL_FAILURE, "randombytes-1");
}
@Test
public void
testPutWhenAlreadyExistingShouldIgnoreWhenIGNORE_RESOLUTION() throws
IOException {
testPutWhenAlreadyExisting(MoveHDFS.IGNORE_RESOLUTION,
MoveHDFS.REL_SUCCESS, "randombytes-1");
}
@Test
public void
testPutWhenAlreadyExistingShouldReplaceWhenREPLACE_RESOLUTION() throws
IOException {
testPutWhenAlreadyExisting(MoveHDFS.REPLACE_RESOLUTION,
MoveHDFS.REL_SUCCESS, "randombytes-2");
}
private void testPutWhenAlreadyExisting(String conflictResolution,
Relationship expectedDestination, String expectedContent) throws IOException {
// GIVEN
Files.createDirectories(Path.of(INPUT_DIRECTORY));
Files.createDirectories(Path.of(OUTPUT_DIRECTORY));
Files.copy(Path.of(TEST_DATA_DIRECTORY, "randombytes-2"),
Path.of(INPUT_DIRECTORY, "randombytes-1"));
Files.copy(Path.of(TEST_DATA_DIRECTORY, "randombytes-1"),
Path.of(OUTPUT_DIRECTORY, "randombytes-1"));
MoveHDFS processor = new MoveHDFS();
TestRunner runner = TestRunners.newTestRunner(processor);
runner.setProperty(MoveHDFS.INPUT_DIRECTORY_OR_FILE,
INPUT_DIRECTORY);
runner.setProperty(MoveHDFS.OUTPUT_DIRECTORY, OUTPUT_DIRECTORY);
runner.setProperty(MoveHDFS.CONFLICT_RESOLUTION, conflictResolution);
byte[] expected = Files.readAllBytes(Path.of(TEST_DATA_DIRECTORY,
expectedContent));
// WHEN
runner.enqueue(new byte[0]);
runner.run();
// THEN
runner.assertAllFlowFilesTransferred(expectedDestination);
List flowFiles =
runner.getFlowFilesForRelationship(expectedDestination);
Assert.assertEquals(1, flowFiles.size());
byte[] actual = Files.readAllBytes(Path.of(OUTPUT_DIRECTORY,
"randombytes-1"));
assertArrayEquals(expected, actual);
}
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Commented] (NIFI-6400) Add Shell User Group Provider Configuration Options
[ https://issues.apache.org/jira/browse/NIFI-6400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16932724#comment-16932724 ] ASF subversion and git services commented on NIFI-6400: --- Commit 171e37ddda501d218c1d18fa4bebc9d729d69459 in nifi's branch refs/heads/master from Matt Gilman [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=171e37d ] NIFI-6400 Fixing checkstyle issue. > Add Shell User Group Provider Configuration Options > --- > > Key: NIFI-6400 > URL: https://issues.apache.org/jira/browse/NIFI-6400 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Troy Melhase >Assignee: Troy Melhase >Priority: Major > Fix For: 1.10.0 > > Time Spent: 2h 20m > Remaining Estimate: 0h > > * combine both exisinting interval parameters > * add group filter parameter -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (NIFI-6688) UpdateRecord Processor not handling the parent ("../") operator correctly
Jeff created NIFI-6688:
--
Summary: UpdateRecord Processor not handling the parent ("../")
operator correctly
Key: NIFI-6688
URL: https://issues.apache.org/jira/browse/NIFI-6688
Project: Apache NiFi
Issue Type: Bug
Components: Tools and Build
Affects Versions: 1.9.2
Reporter: Jeff
Attachments: UpdateRecordScreenshot.png
I have the following JSON:
_{_
_"name": "testRecord",_
_"data": [_
_\{"name":"A"},_
_\{"name":"B"},_
_\{"name":"C"} ]_
_}_
When I apply the UpdateRecord transform in the attached screenshot (using the
parent "../" operator to reference the parent element of the wildcarded array
values). I end up with the following JSON:
_{_
_"name": "testRecord",_
_"data": [_
_\{"key":"C", "name":"A"},_
_\{"key":"C", "name":"B"},_
_\{"key":"C", "name":"C"} ]_
_}_
The "../name" operator seems to return only the "name" field of the last
element in the array for all cases rather than the "key" field of the parent
element being updated. This seems to work contrary to the way things should
as described in
[https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.9.2/org.apache.nifi.processors.standard.UpdateRecord/additionalDetails.html]
I think output should be
_{_
_"name": "testRecord",_
_"data": [_
_\{"key":"A", "name":"A"},_
_\{"key":"B", "name":"B"},_
_\{"key":"C", "name":"C"} ]_
_}_
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (NIFI-6687) Update Ranger dependencies
[ https://issues.apache.org/jira/browse/NIFI-6687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Gilman reassigned NIFI-6687: - Assignee: Matt Gilman > Update Ranger dependencies > -- > > Key: NIFI-6687 > URL: https://issues.apache.org/jira/browse/NIFI-6687 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Matt Gilman >Assignee: Matt Gilman >Priority: Major > > Upgrade the Ranger authorizer to leverage version 2.x. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (NIFI-6687) Update Ranger dependencies
Matt Gilman created NIFI-6687: - Summary: Update Ranger dependencies Key: NIFI-6687 URL: https://issues.apache.org/jira/browse/NIFI-6687 Project: Apache NiFi Issue Type: Improvement Components: Extensions Reporter: Matt Gilman Upgrade the Ranger authorizer to leverage version 2.x. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] MikeThomsen commented on issue #3723: NIFI-6656 Added a default visibility expression configuration item to…
MikeThomsen commented on issue #3723: NIFI-6656 Added a default visibility expression configuration item to… URL: https://github.com/apache/nifi/pull/3723#issuecomment-532798795 @bbende should be good to go. Looked at the error logs and the style check failure was in nifi-shell-authorizer. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (NIFI-6686) ResizeImage Processor Throws Uncaught IOException
[ https://issues.apache.org/jira/browse/NIFI-6686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16932706#comment-16932706 ] Matthew Knight commented on NIFI-6686: -- I am working on digging up an image to replicate this. > ResizeImage Processor Throws Uncaught IOException > - > > Key: NIFI-6686 > URL: https://issues.apache.org/jira/browse/NIFI-6686 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 1.9.2 > Environment: Ubuntu OS, OpenJDK 1.8 >Reporter: Matthew Knight >Priority: Major > Labels: beginner, imageio > Original Estimate: 48h > Remaining Estimate: 48h > > The ResizeImage Processor's process method which performs the actual image > resize throws an IOException which is not directly caught. The offending > flowfile gets penalized, but whenever it is reprocessed this error will just > happen again because the image itself is the source of the problem and that > has not changed. Processing time becomes wasted as the same file is > processed repeatedly at every penalty duration with the same result. > ImageResize IOExceptions should be caught and handled explicitly as > REL_FAILURE. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] kjmccarthy opened a new pull request #3748: NIFI-6680: Nifi PutKudu processor - Convert record field names to low…
kjmccarthy opened a new pull request #3748: NIFI-6680: Nifi PutKudu processor -
Convert record field names to low…
URL: https://github.com/apache/nifi/pull/3748
…ercase
Thank you for submitting a contribution to Apache NiFi.
Please provide a short description of the PR here:
Description of PR
_Enables X functionality; fixes bug NIFI-._
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
### For all changes:
- [ ] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?
- [ ] Does your PR title start with **NIFI-** where is the JIRA
number you are trying to resolve? Pay particular attention to the hyphen "-"
character.
- [ ] Has your PR been rebased against the latest commit within the target
branch (typically `master`)?
- [ ] Is your initial contribution a single, squashed commit? _Additional
commits in response to PR reviewer feedback should be made on this branch and
pushed to allow change tracking. Do not `squash` or use `--force` when pushing
to allow for clean monitoring of changes._
### For code changes:
- [ ] Have you ensured that the full suite of tests is executed via `mvn
-Pcontrib-check clean install` at the root `nifi` folder?
- [ ] Have you written or updated unit tests to verify your changes?
- [ ] Have you verified that the full build is successful on both JDK 8 and
JDK 11?
- [ ] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the `LICENSE` file, including the main
`LICENSE` file under `nifi-assembly`?
- [ ] If applicable, have you updated the `NOTICE` file, including the main
`NOTICE` file found under `nifi-assembly`?
- [ ] If adding new Properties, have you added `.displayName` in addition to
.name (programmatic access) for each of the new properties?
### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in which
it is rendered?
### Note:
Please ensure that once the PR is submitted, you check travis-ci for build
issues and submit an update to your PR as soon as possible.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Created] (NIFI-6686) ResizeImage Processor Throws Uncaught IOException
Matthew Knight created NIFI-6686: Summary: ResizeImage Processor Throws Uncaught IOException Key: NIFI-6686 URL: https://issues.apache.org/jira/browse/NIFI-6686 Project: Apache NiFi Issue Type: Improvement Components: Extensions Affects Versions: 1.9.2 Environment: Ubuntu OS, OpenJDK 1.8 Reporter: Matthew Knight The ResizeImage Processor's process method which performs the actual image resize throws an IOException which is not directly caught. The offending flowfile gets penalized, but whenever it is reprocessed this error will just happen again because the image itself is the source of the problem and that has not changed. Processing time becomes wasted as the same file is processed repeatedly at every penalty duration with the same result. ImageResize IOExceptions should be caught and handled explicitly as REL_FAILURE. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-4913) Specifying "run.as" in bootstrap.properties causes environment variables to not be passed to NiFi
[ https://issues.apache.org/jira/browse/NIFI-4913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16932669#comment-16932669 ] Sushil Kumar commented on NIFI-4913: Looking at it. > Specifying "run.as" in bootstrap.properties causes environment variables to > not be passed to NiFi > - > > Key: NIFI-4913 > URL: https://issues.apache.org/jira/browse/NIFI-4913 > Project: Apache NiFi > Issue Type: Bug > Components: Configuration >Affects Versions: 1.5.0 >Reporter: Jonathan Peterson >Priority: Minor > Labels: beginner > > I solved this by including "-E" after "sudo" in nifi.sh [1]. > [1] > [https://github.com/apache/nifi/blob/rel/nifi-1.5.0/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-resources/src/main/resources/bin/nifi.sh#L312] > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi-minifi-cpp] phrocker opened a new pull request #648: WIP: Minificpp 1025 -- add metadata
phrocker opened a new pull request #648: WIP: Minificpp 1025 -- add metadata URL: https://github.com/apache/nifi-minifi-cpp/pull/648 Thank you for submitting a contribution to Apache NiFi - MiNiFi C++. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with MINIFICPP- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file? - [ ] If applicable, have you updated the NOTICE file? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [nifi] andrewmlim opened a new pull request #3747: NIFI-6558 Add Parameters to User Guide & Admin Guide
andrewmlim opened a new pull request #3747: NIFI-6558 Add Parameters to User Guide & Admin Guide URL: https://github.com/apache/nifi/pull/3747 - New "Parameters" section - Added "Parameters in Versioned Flows" to Versioned Flows - Updated "Managing Local Changes" with parameter content This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (NIFI-6558) Add documentation for Parameters
[ https://issues.apache.org/jira/browse/NIFI-6558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16932556#comment-16932556 ] Andrew Lim commented on NIFI-6558: -- Also need to update the "Managing Local Changes" section with content related to parameters. > Add documentation for Parameters > > > Key: NIFI-6558 > URL: https://issues.apache.org/jira/browse/NIFI-6558 > Project: Apache NiFi > Issue Type: Sub-task > Components: Documentation Website >Reporter: Andrew Lim >Assignee: Andrew Lim >Priority: Major > Fix For: 1.10.0 > > Time Spent: 5h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] kjmccarthy closed pull request #3744: NIFI-6680: Nifi PutKudu processor - Convert record field names to low…
kjmccarthy closed pull request #3744: NIFI-6680: Nifi PutKudu processor - Convert record field names to low… URL: https://github.com/apache/nifi/pull/3744 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [nifi] eduardofontes opened a new pull request #3746: NIFI-6679 - Patch MoveHDFS
eduardofontes opened a new pull request #3746: NIFI-6679 - Patch MoveHDFS
URL: https://github.com/apache/nifi/pull/3746
MoveHDFS removes original file when destination exists.
Thank you for submitting a contribution to Apache NiFi.
Please provide a short description of the PR here:
Description of PR
_Fixes bug NIFI-6679._
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
### For all changes:
- [ ] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?
- [ ] Does your PR title start with **NIFI-** where is the JIRA
number you are trying to resolve? Pay particular attention to the hyphen "-"
character.
- [ ] Has your PR been rebased against the latest commit within the target
branch (typically `master`)?
- [ ] Is your initial contribution a single, squashed commit? _Additional
commits in response to PR reviewer feedback should be made on this branch and
pushed to allow change tracking. Do not `squash` or use `--force` when pushing
to allow for clean monitoring of changes._
### For code changes:
- [ ] Have you ensured that the full suite of tests is executed via `mvn
-Pcontrib-check clean install` at the root `nifi` folder?
- [ ] Have you written or updated unit tests to verify your changes?
- [ ] Have you verified that the full build is successful on both JDK 8 and
JDK 11?
- [ ] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the `LICENSE` file, including the main
`LICENSE` file under `nifi-assembly`?
- [ ] If applicable, have you updated the `NOTICE` file, including the main
`NOTICE` file found under `nifi-assembly`?
- [ ] If adding new Properties, have you added `.displayName` in addition to
.name (programmatic access) for each of the new properties?
### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in which
it is rendered?
### Note:
Please ensure that once the PR is submitted, you check travis-ci for build
issues and submit an update to your PR as soon as possible.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Updated] (MINIFICPP-1033) PublishKafka sends success flow even though kafka server is not running
[
https://issues.apache.org/jira/browse/MINIFICPP-1033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arpad Boda updated MINIFICPP-1033:
--
Description:
Log text:
{code:java}
[2019-09-18 13:33:18.458]
[org::apache::nifi::minifi::processors::PublishKafka] [debug] Sent flow with
length 8 to kafka topic test
[2019-09-18 13:33:18.458] [org::apache::nifi::minifi::core::ProcessSession]
[info] Transferring 1c74f7f4-da08-11e9-8b37-3d7ac48bc574 from PublishKafka to
relationship success
[2019-09-18 13:33:18.557]
[org::apache::nifi::minifi::processors::KafkaConnection] [error]
[thrd:IP:9092/bootstrap]: IP:9092/bootstrap: Connect to ipv4#IP:9092 failed:
Connection refused (after 99ms in state CONNECT)
[2019-09-18 13:33:18.557]
[org::apache::nifi::minifi::processors::KafkaConnection] [error]
[thrd:IP:9092/bootstrap]: IP:9092/bootstrap: Connect to ipv4#IP:9092 failed:
Connection refused (after 99ms in state CONNECT)
[2019-09-18 13:33:18.557]
[org::apache::nifi::minifi::processors::KafkaConnection] [error]
[thrd:IP:9092/bootstrap]: 1/1 brokers are down{code}
was:
Log text:
[2019-09-18 13:33:18.458] [org::apache::nifi::minifi::processors::PublishKafka]
[debug] Sent flow with length 8 to kafka topic test
[2019-09-18 13:33:18.458] [org::apache::nifi::minifi::core::ProcessSession]
[info] Transferring 1c74f7f4-da08-11e9-8b37-3d7ac48bc574 from PublishKafka to
relationship success
[2019-09-18 13:33:18.557]
[org::apache::nifi::minifi::processors::KafkaConnection] [error]
[thrd:IP:9092/bootstrap]: IP:9092/bootstrap: Connect to ipv4#IP:9092 failed:
Connection refused (after 99ms in state CONNECT)
[2019-09-18 13:33:18.557]
[org::apache::nifi::minifi::processors::KafkaConnection] [error]
[thrd:IP:9092/bootstrap]: IP:9092/bootstrap: Connect to ipv4#IP:9092 failed:
Connection refused (after 99ms in state CONNECT)
[2019-09-18 13:33:18.557]
[org::apache::nifi::minifi::processors::KafkaConnection] [error]
[thrd:IP:9092/bootstrap]: 1/1 brokers are down
> PublishKafka sends success flow even though kafka server is not running
> ---
>
> Key: MINIFICPP-1033
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1033
> Project: Apache NiFi MiNiFi C++
> Issue Type: Bug
>Affects Versions: 0.6.0
>Reporter: Nghia Le
>Priority: Major
> Fix For: 0.7.0
>
>
> Log text:
> {code:java}
> [2019-09-18 13:33:18.458]
> [org::apache::nifi::minifi::processors::PublishKafka] [debug] Sent flow with
> length 8 to kafka topic test
> [2019-09-18 13:33:18.458] [org::apache::nifi::minifi::core::ProcessSession]
> [info] Transferring 1c74f7f4-da08-11e9-8b37-3d7ac48bc574 from PublishKafka to
> relationship success
> [2019-09-18 13:33:18.557]
> [org::apache::nifi::minifi::processors::KafkaConnection] [error]
> [thrd:IP:9092/bootstrap]: IP:9092/bootstrap: Connect to ipv4#IP:9092 failed:
> Connection refused (after 99ms in state CONNECT)
> [2019-09-18 13:33:18.557]
> [org::apache::nifi::minifi::processors::KafkaConnection] [error]
> [thrd:IP:9092/bootstrap]: IP:9092/bootstrap: Connect to ipv4#IP:9092 failed:
> Connection refused (after 99ms in state CONNECT)
> [2019-09-18 13:33:18.557]
> [org::apache::nifi::minifi::processors::KafkaConnection] [error]
> [thrd:IP:9092/bootstrap]: 1/1 brokers are down{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (MINIFICPP-1033) PublishKafka sends success flow even though kafka server is not running
[ https://issues.apache.org/jira/browse/MINIFICPP-1033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpad Boda updated MINIFICPP-1033: -- Fix Version/s: 0.7.0 Affects Version/s: 0.6.0 > PublishKafka sends success flow even though kafka server is not running > --- > > Key: MINIFICPP-1033 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1033 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Affects Versions: 0.6.0 >Reporter: Nghia Le >Priority: Major > Fix For: 0.7.0 > > > Log text: > [2019-09-18 13:33:18.458] > [org::apache::nifi::minifi::processors::PublishKafka] [debug] Sent flow with > length 8 to kafka topic test > [2019-09-18 13:33:18.458] [org::apache::nifi::minifi::core::ProcessSession] > [info] Transferring 1c74f7f4-da08-11e9-8b37-3d7ac48bc574 from PublishKafka to > relationship success > [2019-09-18 13:33:18.557] > [org::apache::nifi::minifi::processors::KafkaConnection] [error] > [thrd:IP:9092/bootstrap]: IP:9092/bootstrap: Connect to ipv4#IP:9092 failed: > Connection refused (after 99ms in state CONNECT) > [2019-09-18 13:33:18.557] > [org::apache::nifi::minifi::processors::KafkaConnection] [error] > [thrd:IP:9092/bootstrap]: IP:9092/bootstrap: Connect to ipv4#IP:9092 failed: > Connection refused (after 99ms in state CONNECT) > [2019-09-18 13:33:18.557] > [org::apache::nifi::minifi::processors::KafkaConnection] [error] > [thrd:IP:9092/bootstrap]: 1/1 brokers are down -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] tpalfy commented on a change in pull request #3742: NIFI-6550: Create controller service for Azure Storage Credentials
tpalfy commented on a change in pull request #3742: NIFI-6550: Create
controller service for Azure Storage Credentials
URL: https://github.com/apache/nifi/pull/3742#discussion_r325272421
##
File path:
nifi-nar-bundles/nifi-azure-bundle/nifi-azure-processors/src/test/java/org/apache/nifi/services/azure/storage/MockAzureStorageProcessor.java
##
@@ -0,0 +1,45 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.services.azure.storage;
+
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.exception.ProcessException;
+
+import java.util.ArrayList;
+import java.util.List;
+
+public class MockAzureStorageProcessor extends AbstractProcessor {
+
+@Override
+public void onTrigger(ProcessContext context, ProcessSession session)
throws ProcessException {
+}
+
+@Override
Review comment:
Do we need to override this?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [nifi] tpalfy commented on a change in pull request #3742: NIFI-6550: Create controller service for Azure Storage Credentials
tpalfy commented on a change in pull request #3742: NIFI-6550: Create
controller service for Azure Storage Credentials
URL: https://github.com/apache/nifi/pull/3742#discussion_r325274601
##
File path:
nifi-nar-bundles/nifi-azure-bundle/nifi-azure-processors/src/test/java/org/apache/nifi/processors/azure/storage/utils/TestAzureStorageUtilsGetStorageCredentialsDetails.java
##
@@ -0,0 +1,164 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.azure.storage.utils;
+
+import com.microsoft.azure.storage.StorageCredentialsAccountAndKey;
+import com.microsoft.azure.storage.StorageCredentialsSharedAccessSignature;
+import com.microsoft.azure.storage.core.Base64;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.processor.Processor;
+import org.apache.nifi.processors.azure.storage.ListAzureBlobStorage;
+import
org.apache.nifi.services.azure.storage.AzureStorageCredentialsControllerService;
+import org.apache.nifi.services.azure.storage.AzureStorageCredentialsDetails;
+import org.apache.nifi.util.MockConfigurationContext;
+import org.apache.nifi.util.MockProcessContext;
+import org.junit.Before;
+import org.junit.Test;
+
+import java.util.HashMap;
+import java.util.Map;
+
+import static org.junit.Assert.assertEquals;
+import static org.junit.Assert.assertTrue;
+
+public class TestAzureStorageUtilsGetStorageCredentialsDetails {
+
+private static final String CREDENTIALS_SERVICE_VALUE =
"CredentialsService";
+private static final String ACCOUNT_NAME_VALUE = "AccountName";
+private static final String ACCOUNT_KEY_VALUE =
Base64.encode("AccountKey".getBytes());
+private static final String SAS_TOKEN_VALUE = "SasToken";
+
+private MockProcessContext processContext;
+
+@Before
+public void setUp() {
+Processor processor = new ListAzureBlobStorage();
Review comment:
Maybe it would be more consistent to use `MockAzureStorageProcessor` here
(or use `ListAzureBlobStorage` elsewhere as well and get rid of the mock
processor class).
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [nifi] tpalfy commented on a change in pull request #3742: NIFI-6550: Create controller service for Azure Storage Credentials
tpalfy commented on a change in pull request #3742: NIFI-6550: Create
controller service for Azure Storage Credentials
URL: https://github.com/apache/nifi/pull/3742#discussion_r325141379
##
File path:
nifi-nar-bundles/nifi-azure-bundle/nifi-azure-processors/src/main/java/org/apache/nifi/processors/azure/storage/utils/AzureStorageUtils.java
##
@@ -112,55 +125,76 @@ private AzureStorageUtils() {
* Account Name, Account Key or SAS Token. This can be
null if not available.
*/
public static CloudBlobClient createCloudBlobClient(ProcessContext
context, ComponentLog logger, FlowFile flowFile) {
-final String accountName;
-final String accountKey;
-final String sasToken;
-
-if (flowFile == null) {
-accountName =
context.getProperty(AzureStorageUtils.ACCOUNT_NAME).evaluateAttributeExpressions().getValue();
-accountKey =
context.getProperty(AzureStorageUtils.ACCOUNT_KEY).evaluateAttributeExpressions().getValue();
-sasToken =
context.getProperty(AzureStorageUtils.PROP_SAS_TOKEN).evaluateAttributeExpressions().getValue();
+final AzureStorageCredentialsDetails storageCredentialsDetails =
getStorageCredentialsDetails(context, flowFile);
+
+final URI baseUri = getBaseUri(FORMAT_BASE_URI,
storageCredentialsDetails.getStorageAccountName(), context, logger);
+
+final CloudBlobClient cloudBlobClient = new CloudBlobClient(baseUri,
storageCredentialsDetails.getStorageCredentials());
+
+return cloudBlobClient;
+}
+
+public static AzureStorageCredentialsDetails
getStorageCredentialsDetails(PropertyContext context, FlowFile flowFile) {
+final Map attributes = flowFile != null ?
flowFile.getAttributes() : Collections.emptyMap();
+
+final AzureStorageCredentialsService storageCredentialsService =
context.getProperty(STORAGE_CREDENTIALS_SERVICE).asControllerService(AzureStorageCredentialsService.class);
+
+if (storageCredentialsService != null) {
+return
storageCredentialsService.getStorageCredentialsDetails(attributes);
} else {
-accountName =
context.getProperty(AzureStorageUtils.ACCOUNT_NAME).evaluateAttributeExpressions(flowFile).getValue();
-accountKey =
context.getProperty(AzureStorageUtils.ACCOUNT_KEY).evaluateAttributeExpressions(flowFile).getValue();
-sasToken =
context.getProperty(AzureStorageUtils.PROP_SAS_TOKEN).evaluateAttributeExpressions(flowFile).getValue();
+return createStorageCredentialsDetails(context, attributes);
+}
+}
+
+public static AzureStorageCredentialsDetails
createStorageCredentialsDetails(PropertyContext context, Map
attributes) {
+final String accountName =
context.getProperty(ACCOUNT_NAME).evaluateAttributeExpressions(attributes).getValue();
+final String accountKey =
context.getProperty(ACCOUNT_KEY).evaluateAttributeExpressions(attributes).getValue();
+final String sasToken =
context.getProperty(PROP_SAS_TOKEN).evaluateAttributeExpressions(attributes).getValue();
+
+if (StringUtils.isBlank(accountName)) {
+throw new IllegalArgumentException(String.format("'%s' must not be
empty.", ACCOUNT_NAME.getDisplayName()));
}
-CloudBlobClient cloudBlobClient;
+StorageCredentials storageCredentials;
+
+if (StringUtils.isNotBlank(accountKey)) {
+storageCredentials = new
StorageCredentialsAccountAndKey(accountName, accountKey);
Review comment:
Sure this will work? In the original code the `accountName` + `accountKey`
case was handled intentionally differently, if we can believe the comment that
was added:
`// sas token and acct name/key have different ways of creating a secure
connection (e.g. new StorageCredentialsAccountAndKey didn't work)`
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [nifi] tpalfy commented on a change in pull request #3742: NIFI-6550: Create controller service for Azure Storage Credentials
tpalfy commented on a change in pull request #3742: NIFI-6550: Create
controller service for Azure Storage Credentials
URL: https://github.com/apache/nifi/pull/3742#discussion_r325144059
##
File path:
nifi-nar-bundles/nifi-azure-bundle/nifi-azure-processors/src/main/java/org/apache/nifi/processors/azure/storage/queue/AbstractAzureQueueStorage.java
##
@@ -1,110 +1,83 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-package org.apache.nifi.processors.azure.storage.queue;
-
-import com.microsoft.azure.storage.CloudStorageAccount;
-import com.microsoft.azure.storage.StorageCredentials;
-import com.microsoft.azure.storage.StorageCredentialsSharedAccessSignature;
-import com.microsoft.azure.storage.queue.CloudQueueClient;
-
-import org.apache.commons.lang3.StringUtils;
-import org.apache.nifi.components.PropertyDescriptor;
-import org.apache.nifi.expression.ExpressionLanguageScope;
-import org.apache.nifi.flowfile.FlowFile;
-import org.apache.nifi.processor.AbstractProcessor;
-import org.apache.nifi.processor.ProcessContext;
-import org.apache.nifi.processor.Relationship;
-import org.apache.nifi.processor.util.StandardValidators;
-import org.apache.nifi.processors.azure.storage.utils.AzureStorageUtils;
-
-import java.net.URI;
-import java.net.URISyntaxException;
-import java.security.InvalidKeyException;
-import java.util.Arrays;
-import java.util.Collections;
-import java.util.HashSet;
-import java.util.Set;
-
-public abstract class AbstractAzureQueueStorage extends AbstractProcessor {
-
-public static final PropertyDescriptor QUEUE = new
PropertyDescriptor.Builder()
-.name("storage-queue-name")
-.displayName("Queue Name")
-.description("Name of the Azure Storage Queue")
-.required(true)
-.addValidator(StandardValidators.NON_EMPTY_EL_VALIDATOR)
-
.expressionLanguageSupported(ExpressionLanguageScope.FLOWFILE_ATTRIBUTES)
-.build();
-
-public static final Relationship REL_SUCCESS = new Relationship.Builder()
-.name("success")
-.description("All successfully processed FlowFiles are routed to
this relationship")
-.build();
-
-public static final Relationship REL_FAILURE = new Relationship.Builder()
-.name("failure")
-.description("Unsuccessful operations will be transferred to the
failure relationship.")
-.build();
-
-private static final String FORMAT_QUEUE_CONNECTION_STRING =
"DefaultEndpointsProtocol=https;AccountName=%s;AccountKey=%s";
-private static final String FORMAT_QUEUE_BASE_URI =
"https://%s.queue.core.windows.net;;
-
-private static final Set relationships =
Collections.unmodifiableSet(new HashSet<>(Arrays.asList(REL_SUCCESS,
REL_FAILURE)));
-
-@Override
-public Set getRelationships() {
-return relationships;
-}
-
-protected final CloudQueueClient createCloudQueueClient(final
ProcessContext context, final FlowFile flowFile) {
-final String storageAccountName;
-final String storageAccountKey;
-final String sasToken;
-final String connectionString;
-
-if (flowFile == null) {
-storageAccountName =
context.getProperty(AzureStorageUtils.ACCOUNT_NAME).evaluateAttributeExpressions().getValue();
-storageAccountKey =
context.getProperty(AzureStorageUtils.ACCOUNT_KEY).evaluateAttributeExpressions().getValue();
-sasToken =
context.getProperty(AzureStorageUtils.PROP_SAS_TOKEN).evaluateAttributeExpressions().getValue();
-} else {
-storageAccountName =
context.getProperty(AzureStorageUtils.ACCOUNT_NAME).evaluateAttributeExpressions(flowFile).getValue();
-storageAccountKey =
context.getProperty(AzureStorageUtils.ACCOUNT_KEY).evaluateAttributeExpressions(flowFile).getValue();
-sasToken =
context.getProperty(AzureStorageUtils.PROP_SAS_TOKEN).evaluateAttributeExpressions(flowFile).getValue();
-}
-
-CloudQueueClient cloudQueueClient;
-try {
-if (StringUtils.isNoneBlank(sasToken)) {
-connectionString = String.format(FORMAT_QUEUE_BASE_URI,
[GitHub] [nifi] tpalfy commented on a change in pull request #3742: NIFI-6550: Create controller service for Azure Storage Credentials
tpalfy commented on a change in pull request #3742: NIFI-6550: Create
controller service for Azure Storage Credentials
URL: https://github.com/apache/nifi/pull/3742#discussion_r325178117
##
File path:
nifi-nar-bundles/nifi-azure-bundle/nifi-azure-processors/src/test/java/org/apache/nifi/processors/azure/storage/AbstractAzureStorageIT.java
##
@@ -64,42 +53,48 @@
} catch (FileNotFoundException e) {
fail("Could not open credentials file " + CREDENTIALS_FILE + ": "
+ e.getLocalizedMessage());
}
-
}
-public static String getAccountName() {
+protected static String getAccountName() {
return CONFIG.getProperty("accountName");
}
-public static String getAccountKey() {
+protected static String getAccountKey() {
return CONFIG.getProperty("accountKey");
}
-public static CloudBlobContainer getContainer(String containerName) throws
InvalidKeyException, URISyntaxException, StorageException {
-CloudBlobClient blobClient =
getStorageAccount().createCloudBlobClient();
-return blobClient.getContainerReference(containerName);
-}
+protected TestRunner runner;
+
+@Before
+public void setUpAzureStorageIT() {
+runner = TestRunners.newTestRunner(getProcessorClass());
-public static CloudQueue getQueue(String queueName) throws
URISyntaxException, InvalidKeyException, StorageException {
-CloudQueueClient cloudQueueClient =
getStorageAccount().createCloudQueueClient();
-cloudQueue = cloudQueueClient.getQueueReference(queueName);
-return cloudQueue;
+runner.setProperty(AzureStorageUtils.ACCOUNT_NAME, getAccountName());
+runner.setProperty(AzureStorageUtils.ACCOUNT_KEY, getAccountKey());
}
-private static CloudStorageAccount getStorageAccount() throws
URISyntaxException, InvalidKeyException {
+protected abstract Class getProcessorClass();
+
+protected CloudStorageAccount getStorageAccount() throws Exception {
String storageConnectionString =
String.format(FORMAT_CONNECTION_STRING, getAccountName(), getAccountKey());
Review comment:
As this is an integration test, shouldn't we create the
`CloudBlob/QueueClient` the same way as it is done in the refactored/changed
production code?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [nifi] belugabehr edited a comment on issue #3643: NIFI-6542: Upgrade nifi-hadoop-bundles Hadoop version dependency to 3…
belugabehr edited a comment on issue #3643: NIFI-6542: Upgrade nifi-hadoop-bundles Hadoop version dependency to 3… URL: https://github.com/apache/nifi/pull/3643#issuecomment-532690704 I ran `mvn dependency:list` against both version. The only meaningful change in dependency is the addition of: `org.apache.yetus:audience-annotations:jar:0.5.0` `dnsjava:dnsjava:jar:2.1.7` EDIT: https://github.com/dnsjava/dnsjava/blob/master/LICENSE This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [nifi] belugabehr commented on issue #3643: NIFI-6542: Upgrade nifi-hadoop-bundles Hadoop version dependency to 3…
belugabehr commented on issue #3643: NIFI-6542: Upgrade nifi-hadoop-bundles Hadoop version dependency to 3… URL: https://github.com/apache/nifi/pull/3643#issuecomment-532690704 I ran `mvn dependency:list` against both version. The only meaningful change in dependency is the addition of: `org.apache.yetus:audience-annotations:jar:0.5.0` `dnsjava:dnsjava:jar:2.1.7` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [nifi-minifi-cpp] nghiaxlee opened a new pull request #647: WIP: MINIFICPP-976 - Add Kafka broker to docker verify
nghiaxlee opened a new pull request #647: WIP: MINIFICPP-976 - Add Kafka broker to docker verify URL: https://github.com/apache/nifi-minifi-cpp/pull/647 Thank you for submitting a contribution to Apache NiFi - MiNiFi C++. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with MINIFICPP- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file? - [ ] If applicable, have you updated the NOTICE file? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (NIFI-6681) NIFI PutHbaseRecord
[
https://issues.apache.org/jira/browse/NIFI-6681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16932450#comment-16932450
]
Bryan Bende commented on NIFI-6681:
---
In the PutHBaseRecord processor, I think you want Row Identifier Field Name to
be "TRANSACTION_START_TIME" and not "${TRANSACTION_START_TIME}".
Using "${...}" would mean you want to get the value of a flow file attribute
name TRANSACTION_START_TIME, and then use the value of that attribute as the
ROW ID.
> NIFI PutHbaseRecord
> ---
>
> Key: NIFI-6681
> URL: https://issues.apache.org/jira/browse/NIFI-6681
> Project: Apache NiFi
> Issue Type: Bug
> Components: Flow Versioning
>Affects Versions: 1.8.0
> Environment: Linux
>Reporter: Amogh
>Priority: Minor
> Attachments: CSVReader.PNG, Conf.PNG, Error.png, Hbase.PNG
>
>
> Hi All,
>
> I am getting below error while loading a data hbase from nifi, Even though
> the column is not null data is not loading and i'm getting the error
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (MINIFICPP-1033) PublishKafka sends success flow even though kafka server is not running
Nghia Le created MINIFICPP-1033: --- Summary: PublishKafka sends success flow even though kafka server is not running Key: MINIFICPP-1033 URL: https://issues.apache.org/jira/browse/MINIFICPP-1033 Project: Apache NiFi MiNiFi C++ Issue Type: Bug Reporter: Nghia Le Log text: [2019-09-18 13:33:18.458] [org::apache::nifi::minifi::processors::PublishKafka] [debug] Sent flow with length 8 to kafka topic test [2019-09-18 13:33:18.458] [org::apache::nifi::minifi::core::ProcessSession] [info] Transferring 1c74f7f4-da08-11e9-8b37-3d7ac48bc574 from PublishKafka to relationship success [2019-09-18 13:33:18.557] [org::apache::nifi::minifi::processors::KafkaConnection] [error] [thrd:IP:9092/bootstrap]: IP:9092/bootstrap: Connect to ipv4#IP:9092 failed: Connection refused (after 99ms in state CONNECT) [2019-09-18 13:33:18.557] [org::apache::nifi::minifi::processors::KafkaConnection] [error] [thrd:IP:9092/bootstrap]: IP:9092/bootstrap: Connect to ipv4#IP:9092 failed: Connection refused (after 99ms in state CONNECT) [2019-09-18 13:33:18.557] [org::apache::nifi::minifi::processors::KafkaConnection] [error] [thrd:IP:9092/bootstrap]: 1/1 brokers are down -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-6400) Add Shell User Group Provider Configuration Options
[ https://issues.apache.org/jira/browse/NIFI-6400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16932431#comment-16932431 ] ASF subversion and git services commented on NIFI-6400: --- Commit 4596ef7c8a1b4a002dd81d210ceab4f99bc32fd6 in nifi's branch refs/heads/master from Troy Melhase [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=4596ef7 ] NIFI-6400 Better options, consistent ids for ShellUserGroupProvider. NIFI-6400 Fixes to address PR feedback. NIFI-6400 Accepts proposed changes. This closes #3637 > Add Shell User Group Provider Configuration Options > --- > > Key: NIFI-6400 > URL: https://issues.apache.org/jira/browse/NIFI-6400 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Troy Melhase >Assignee: Troy Melhase >Priority: Major > Time Spent: 2h > Remaining Estimate: 0h > > * combine both exisinting interval parameters > * add group filter parameter -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (NIFI-6400) Add Shell User Group Provider Configuration Options
[ https://issues.apache.org/jira/browse/NIFI-6400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Gilman updated NIFI-6400: -- Fix Version/s: 1.10.0 Resolution: Fixed Status: Resolved (was: Patch Available) > Add Shell User Group Provider Configuration Options > --- > > Key: NIFI-6400 > URL: https://issues.apache.org/jira/browse/NIFI-6400 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Troy Melhase >Assignee: Troy Melhase >Priority: Major > Fix For: 1.10.0 > > Time Spent: 2h 20m > Remaining Estimate: 0h > > * combine both exisinting interval parameters > * add group filter parameter -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-6400) Add Shell User Group Provider Configuration Options
[ https://issues.apache.org/jira/browse/NIFI-6400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16932430#comment-16932430 ] ASF subversion and git services commented on NIFI-6400: --- Commit 4596ef7c8a1b4a002dd81d210ceab4f99bc32fd6 in nifi's branch refs/heads/master from Troy Melhase [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=4596ef7 ] NIFI-6400 Better options, consistent ids for ShellUserGroupProvider. NIFI-6400 Fixes to address PR feedback. NIFI-6400 Accepts proposed changes. This closes #3637 > Add Shell User Group Provider Configuration Options > --- > > Key: NIFI-6400 > URL: https://issues.apache.org/jira/browse/NIFI-6400 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Troy Melhase >Assignee: Troy Melhase >Priority: Major > Time Spent: 2h > Remaining Estimate: 0h > > * combine both exisinting interval parameters > * add group filter parameter -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-6400) Add Shell User Group Provider Configuration Options
[ https://issues.apache.org/jira/browse/NIFI-6400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16932429#comment-16932429 ] ASF subversion and git services commented on NIFI-6400: --- Commit 4596ef7c8a1b4a002dd81d210ceab4f99bc32fd6 in nifi's branch refs/heads/master from Troy Melhase [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=4596ef7 ] NIFI-6400 Better options, consistent ids for ShellUserGroupProvider. NIFI-6400 Fixes to address PR feedback. NIFI-6400 Accepts proposed changes. This closes #3637 > Add Shell User Group Provider Configuration Options > --- > > Key: NIFI-6400 > URL: https://issues.apache.org/jira/browse/NIFI-6400 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Troy Melhase >Assignee: Troy Melhase >Priority: Major > Time Spent: 2h > Remaining Estimate: 0h > > * combine both exisinting interval parameters > * add group filter parameter -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] mcgilman commented on issue #3637: NIFI-6400 Better options, consistent ids for ShellUserGroupProvider.
mcgilman commented on issue #3637: NIFI-6400 Better options, consistent ids for ShellUserGroupProvider. URL: https://github.com/apache/nifi/pull/3637#issuecomment-532676561 Thanks @natural! This has been merged to master. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [nifi] asfgit closed pull request #3637: NIFI-6400 Better options, consistent ids for ShellUserGroupProvider.
asfgit closed pull request #3637: NIFI-6400 Better options, consistent ids for ShellUserGroupProvider. URL: https://github.com/apache/nifi/pull/3637 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [nifi] joewitt commented on issue #3643: NIFI-6542: Upgrade nifi-hadoop-bundles Hadoop version dependency to 3…
joewitt commented on issue #3643: NIFI-6542: Upgrade nifi-hadoop-bundles Hadoop version dependency to 3… URL: https://github.com/apache/nifi/pull/3643#issuecomment-532675053 has a review of all the dependencies in 3.0 vs 3.2 been done and if so was there no changes - no new libs and no removed libs such that we have no need to update Licensing and Notice information? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [nifi] tpalfy commented on issue #3643: NIFI-6542: Upgrade nifi-hadoop-bundles Hadoop version dependency to 3…
tpalfy commented on issue #3643: NIFI-6542: Upgrade nifi-hadoop-bundles Hadoop version dependency to 3… URL: https://github.com/apache/nifi/pull/3643#issuecomment-532672648 @belugabehr Unfortunately I can't do much about the failed CI jobs. I can't rerun them. They are partially successful, which is not really a rare case. So I think they can be considered ok overall. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Comment Edited] (NIFI-6681) NIFI PutHbaseRecord
[ https://issues.apache.org/jira/browse/NIFI-6681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16932398#comment-16932398 ] Amogh edited comment on NIFI-6681 at 9/18/19 12:48 PM: --- Hi Shawn, Thanks for the reply Yes i have added the screen shots of conf.png is HBase_1_1_2_ClientService CSVReader.png is CSVReader Hbase.png is PutHBaseRecord Myflow is simple file to hbase was (Author: amogh.balachandra): Yes i have added the screen shots of conf.png is HBase_1_1_2_ClientService CSVReader.png is CSVReader Hbase.png is PutHBaseRecord Myflow is simple file to hbase > NIFI PutHbaseRecord > --- > > Key: NIFI-6681 > URL: https://issues.apache.org/jira/browse/NIFI-6681 > Project: Apache NiFi > Issue Type: Bug > Components: Flow Versioning >Affects Versions: 1.8.0 > Environment: Linux >Reporter: Amogh >Priority: Minor > Attachments: CSVReader.PNG, Conf.PNG, Error.png, Hbase.PNG > > > Hi All, > > I am getting below error while loading a data hbase from nifi, Even though > the column is not null data is not loading and i'm getting the error -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-6681) NIFI PutHbaseRecord
[ https://issues.apache.org/jira/browse/NIFI-6681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16932398#comment-16932398 ] Amogh commented on NIFI-6681: - Yes i have added the screen shots of conf.png is HBase_1_1_2_ClientService CSVReader.png is CSVReader Hbase.png is PutHBaseRecord Myflow is simple file to hbase > NIFI PutHbaseRecord > --- > > Key: NIFI-6681 > URL: https://issues.apache.org/jira/browse/NIFI-6681 > Project: Apache NiFi > Issue Type: Bug > Components: Flow Versioning >Affects Versions: 1.8.0 > Environment: Linux >Reporter: Amogh >Priority: Minor > Attachments: CSVReader.PNG, Conf.PNG, Error.png, Hbase.PNG > > > Hi All, > > I am getting below error while loading a data hbase from nifi, Even though > the column is not null data is not loading and i'm getting the error -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-6681) NIFI PutHbaseRecord
[ https://issues.apache.org/jira/browse/NIFI-6681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16932386#comment-16932386 ] Shawn Weeks commented on NIFI-6681: --- Was that screen shot from the put hbase record processor? > NIFI PutHbaseRecord > --- > > Key: NIFI-6681 > URL: https://issues.apache.org/jira/browse/NIFI-6681 > Project: Apache NiFi > Issue Type: Bug > Components: Flow Versioning >Affects Versions: 1.8.0 > Environment: Linux >Reporter: Amogh >Priority: Minor > Attachments: CSVReader.PNG, Conf.PNG, Error.png, Hbase.PNG > > > Hi All, > > I am getting below error while loading a data hbase from nifi, Even though > the column is not null data is not loading and i'm getting the error -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-6681) NIFI PutHbaseRecord
[ https://issues.apache.org/jira/browse/NIFI-6681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16932387#comment-16932387 ] Shawn Weeks commented on NIFI-6681: --- Also your example link appears to go nowhere. > NIFI PutHbaseRecord > --- > > Key: NIFI-6681 > URL: https://issues.apache.org/jira/browse/NIFI-6681 > Project: Apache NiFi > Issue Type: Bug > Components: Flow Versioning >Affects Versions: 1.8.0 > Environment: Linux >Reporter: Amogh >Priority: Minor > Attachments: CSVReader.PNG, Conf.PNG, Error.png, Hbase.PNG > > > Hi All, > > I am getting below error while loading a data hbase from nifi, Even though > the column is not null data is not loading and i'm getting the error -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] adarmiento commented on issue #3700: NIFI-6638: Empty multiple queues at once at different flow levels
adarmiento commented on issue #3700: NIFI-6638: Empty multiple queues at once at different flow levels URL: https://github.com/apache/nifi/pull/3700#issuecomment-532613745 I'd say that to empty a subset of the flow I am currently viewing, I'd go for the "selected queues: empty the selected queues", however, I understand your urge to add more convenience methods. I'd say I would add all the non-core feature in 1+ separate PRs, keeping this "core" implementation as simple as possible. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [nifi] VinceCastro commented on issue #3700: NIFI-6638: Empty multiple queues at once at different flow levels
VinceCastro commented on issue #3700: NIFI-6638: Empty multiple queues at once at different flow levels URL: https://github.com/apache/nifi/pull/3700#issuecomment-532601047 Got it! So you suggest to move the empty multiple queues logic from the front-end to the back-end side by adding one or more rest api to the exsisting endpoints. We may have cases where the user wants to empty different selected process groups at once, or different selected queues at once, so I would suggest to add two rest api to the "FlowFile Queues" endpoint: the first one should accept a list of process group IDs in order to empty the queues inside them(recursively or not); the second one should accept a list of connections IDs to empty. We may collect all IDs into a json body. This way we can efficiently cover all cases where the user wants to: - empty the selected queue - empty the selected queues - empty the selected process group (recursively or not) - empty the selected process groups (recursively or not) - empty the current process group (recursively or not) What do you think about? @adarmiento you stated: > I think that the most useful features are: > >> selected queue: empty the selected queue >> selected queues: empty the selected queues >> current process group (recursive): empty all queues inside the current process group recursively What if I want to empty only a subset of the flow I'm currently viewing? In that case features like "empty the selected process group" or "empty the selected process groups" are necessary, do you agree? However, as mentioned by @adarmiento > Are there use cases in which you are interested into emptying all queue of a processor group which is not the current one AND at the same time we do not want to delete the queue recursively? I have doubts about the usefulness of the non recursive actions. Are there use cases where we do not really need to empty a process group recursively? Do you guys @mcgilman @adarmiento suggest to add the non recursive actions or postpone them to another pr? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Created] (NIFI-6685) Sort and aling
Nitin Ullal created NIFI-6685: - Summary: Sort and aling Key: NIFI-6685 URL: https://issues.apache.org/jira/browse/NIFI-6685 Project: Apache NiFi Issue Type: Wish Components: Core UI Reporter: Nitin Ullal Attachments: nifi.png It would be nice to have a feature that sorts and aligns Process Groups by their name and aligns them to a grid (for example 3x3). Check out attached screenshot. Every time I add something I have to manually sort and align. -- This message was sent by Atlassian Jira (v8.3.4#803005)
