[jira] [Commented] (NIFI-4866) HBase_1_1_2_ClientService performance dropped due to accessing HBase admin upon every FlowFile transfer
[
https://issues.apache.org/jira/browse/NIFI-4866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16359267#comment-16359267
]
ASF GitHub Bot commented on NIFI-4866:
--
Github user zenfenan commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2464#discussion_r167390069
--- Diff:
nifi-nar-bundles/nifi-standard-services/nifi-hbase_1_1_2-client-service-bundle/nifi-hbase_1_1_2-client-service/src/main/java/org/apache/nifi/hbase/HBase_1_1_2_ClientService.java
---
@@ -549,11 +551,7 @@ public String toTransitUri(String tableName, String
rowKey) {
logger.warn("Connection has not been established, could not
create a transit URI. Returning null.");
return null;
}
-try {
-final String masterAddress =
connection.getAdmin().getClusterStatus().getMaster().getHostAndPort();
-return "hbase://" + masterAddress + "/" + tableName + (rowKey
!= null && !rowKey.isEmpty() ? "/" + rowKey : "");
-} catch (IOException e) {
-throw new RuntimeException("Failed to get HBase Admin
interface, due to " + e, e);
-}
+final String transitUriMasterAddress =
StringUtils.isEmpty(masterAddress) ? "unknown" : masterAddress;
--- End diff --
As far as I'm aware, such a scenario has no possibilities of happening. I
just wanted to start the thinking in that direction so that people who know in
and out of HBase can pitch in their thoughts.
It looks good to me. Thanks :)
> HBase_1_1_2_ClientService performance dropped due to accessing HBase admin
> upon every FlowFile transfer
> ---
>
> Key: NIFI-4866
> URL: https://issues.apache.org/jira/browse/NIFI-4866
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Reporter: Koji Kawamura
>Assignee: Koji Kawamura
>Priority: Critical
>
> NIFI-4543 added
> "connection.getAdmin().getClusterStatus().getMaster().getHostAndPort()” call
> from HBase_1_1_2_ClientService.toTransitUri method in order to get HBase
> master address, which is called upon every FlowFile transfer and essentially
> doubles the traffic through the HBase connector.
> The performance of our PutHBaseJSON processor dropped to 1/3 after deploying
> NiFi 1.5.0.
> Those lines of code should be moved from toTransitUri to createConnection.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] nifi pull request #2464: NIFI-4866: Fixed HBase performance issue
Github user zenfenan commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2464#discussion_r167390069
--- Diff:
nifi-nar-bundles/nifi-standard-services/nifi-hbase_1_1_2-client-service-bundle/nifi-hbase_1_1_2-client-service/src/main/java/org/apache/nifi/hbase/HBase_1_1_2_ClientService.java
---
@@ -549,11 +551,7 @@ public String toTransitUri(String tableName, String
rowKey) {
logger.warn("Connection has not been established, could not
create a transit URI. Returning null.");
return null;
}
-try {
-final String masterAddress =
connection.getAdmin().getClusterStatus().getMaster().getHostAndPort();
-return "hbase://" + masterAddress + "/" + tableName + (rowKey
!= null && !rowKey.isEmpty() ? "/" + rowKey : "");
-} catch (IOException e) {
-throw new RuntimeException("Failed to get HBase Admin
interface, due to " + e, e);
-}
+final String transitUriMasterAddress =
StringUtils.isEmpty(masterAddress) ? "unknown" : masterAddress;
--- End diff --
As far as I'm aware, such a scenario has no possibilities of happening. I
just wanted to start the thinking in that direction so that people who know in
and out of HBase can pitch in their thoughts.
It looks good to me. Thanks :)
---
[jira] [Commented] (NIFI-4866) HBase_1_1_2_ClientService performance dropped due to accessing HBase admin upon every FlowFile transfer
[
https://issues.apache.org/jira/browse/NIFI-4866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16359264#comment-16359264
]
ASF GitHub Bot commented on NIFI-4866:
--
Github user ijokarumawak commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2464#discussion_r167389731
--- Diff:
nifi-nar-bundles/nifi-standard-services/nifi-hbase_1_1_2-client-service-bundle/nifi-hbase_1_1_2-client-service/src/main/java/org/apache/nifi/hbase/HBase_1_1_2_ClientService.java
---
@@ -549,11 +551,7 @@ public String toTransitUri(String tableName, String
rowKey) {
logger.warn("Connection has not been established, could not
create a transit URI. Returning null.");
return null;
}
-try {
-final String masterAddress =
connection.getAdmin().getClusterStatus().getMaster().getHostAndPort();
-return "hbase://" + masterAddress + "/" + tableName + (rowKey
!= null && !rowKey.isEmpty() ? "/" + rowKey : "");
-} catch (IOException e) {
-throw new RuntimeException("Failed to get HBase Admin
interface, due to " + e, e);
-}
+final String transitUriMasterAddress =
StringUtils.isEmpty(masterAddress) ? "unknown" : masterAddress;
--- End diff --
@zenfenan Thanks for your comment. Yes, it would be a problem if this
actually happens. But it only happens when the ControllerService was not able
to get the master address when it was enabled. I think HBase processors will
not be able to access HBase, too.
It depends on the programs how critical this can be, for those use the
returned transit URI later. For example, NiFi and Atlas integration case,
ReportLineageToAtlas reporting task will not be able to determine the right
cluster name in Atlas.
Please let me know any situation that the ControllerServer can connect to a
HBase, but can not get its master address. I write the code that way just to be
more protective, but not sure if that actually happens.
> HBase_1_1_2_ClientService performance dropped due to accessing HBase admin
> upon every FlowFile transfer
> ---
>
> Key: NIFI-4866
> URL: https://issues.apache.org/jira/browse/NIFI-4866
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Reporter: Koji Kawamura
>Assignee: Koji Kawamura
>Priority: Critical
>
> NIFI-4543 added
> "connection.getAdmin().getClusterStatus().getMaster().getHostAndPort()” call
> from HBase_1_1_2_ClientService.toTransitUri method in order to get HBase
> master address, which is called upon every FlowFile transfer and essentially
> doubles the traffic through the HBase connector.
> The performance of our PutHBaseJSON processor dropped to 1/3 after deploying
> NiFi 1.5.0.
> Those lines of code should be moved from toTransitUri to createConnection.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] nifi pull request #2464: NIFI-4866: Fixed HBase performance issue
Github user ijokarumawak commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2464#discussion_r167389731
--- Diff:
nifi-nar-bundles/nifi-standard-services/nifi-hbase_1_1_2-client-service-bundle/nifi-hbase_1_1_2-client-service/src/main/java/org/apache/nifi/hbase/HBase_1_1_2_ClientService.java
---
@@ -549,11 +551,7 @@ public String toTransitUri(String tableName, String
rowKey) {
logger.warn("Connection has not been established, could not
create a transit URI. Returning null.");
return null;
}
-try {
-final String masterAddress =
connection.getAdmin().getClusterStatus().getMaster().getHostAndPort();
-return "hbase://" + masterAddress + "/" + tableName + (rowKey
!= null && !rowKey.isEmpty() ? "/" + rowKey : "");
-} catch (IOException e) {
-throw new RuntimeException("Failed to get HBase Admin
interface, due to " + e, e);
-}
+final String transitUriMasterAddress =
StringUtils.isEmpty(masterAddress) ? "unknown" : masterAddress;
--- End diff --
@zenfenan Thanks for your comment. Yes, it would be a problem if this
actually happens. But it only happens when the ControllerService was not able
to get the master address when it was enabled. I think HBase processors will
not be able to access HBase, too.
It depends on the programs how critical this can be, for those use the
returned transit URI later. For example, NiFi and Atlas integration case,
ReportLineageToAtlas reporting task will not be able to determine the right
cluster name in Atlas.
Please let me know any situation that the ControllerServer can connect to a
HBase, but can not get its master address. I write the code that way just to be
more protective, but not sure if that actually happens.
---
[jira] [Commented] (NIFI-4866) HBase_1_1_2_ClientService performance dropped due to accessing HBase admin upon every FlowFile transfer
[
https://issues.apache.org/jira/browse/NIFI-4866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16359240#comment-16359240
]
ASF GitHub Bot commented on NIFI-4866:
--
Github user zenfenan commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2464#discussion_r167388849
--- Diff:
nifi-nar-bundles/nifi-standard-services/nifi-hbase_1_1_2-client-service-bundle/nifi-hbase_1_1_2-client-service/src/main/java/org/apache/nifi/hbase/HBase_1_1_2_ClientService.java
---
@@ -549,11 +551,7 @@ public String toTransitUri(String tableName, String
rowKey) {
logger.warn("Connection has not been established, could not
create a transit URI. Returning null.");
return null;
}
-try {
-final String masterAddress =
connection.getAdmin().getClusterStatus().getMaster().getHostAndPort();
-return "hbase://" + masterAddress + "/" + tableName + (rowKey
!= null && !rowKey.isEmpty() ? "/" + rowKey : "");
-} catch (IOException e) {
-throw new RuntimeException("Failed to get HBase Admin
interface, due to " + e, e);
-}
+final String transitUriMasterAddress =
StringUtils.isEmpty(masterAddress) ? "unknown" : masterAddress;
--- End diff --
We are sending back "unknown" if the master address is empty. Won't that be
a problem, if it happens to be empty? I mean when
`session.getProvenanceReporter().fetch(handlerFlowFile, transitUri);` gets
executed on the HBase processors?
> HBase_1_1_2_ClientService performance dropped due to accessing HBase admin
> upon every FlowFile transfer
> ---
>
> Key: NIFI-4866
> URL: https://issues.apache.org/jira/browse/NIFI-4866
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Reporter: Koji Kawamura
>Assignee: Koji Kawamura
>Priority: Critical
>
> NIFI-4543 added
> "connection.getAdmin().getClusterStatus().getMaster().getHostAndPort()” call
> from HBase_1_1_2_ClientService.toTransitUri method in order to get HBase
> master address, which is called upon every FlowFile transfer and essentially
> doubles the traffic through the HBase connector.
> The performance of our PutHBaseJSON processor dropped to 1/3 after deploying
> NiFi 1.5.0.
> Those lines of code should be moved from toTransitUri to createConnection.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] nifi pull request #2464: NIFI-4866: Fixed HBase performance issue
Github user zenfenan commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2464#discussion_r167388849
--- Diff:
nifi-nar-bundles/nifi-standard-services/nifi-hbase_1_1_2-client-service-bundle/nifi-hbase_1_1_2-client-service/src/main/java/org/apache/nifi/hbase/HBase_1_1_2_ClientService.java
---
@@ -549,11 +551,7 @@ public String toTransitUri(String tableName, String
rowKey) {
logger.warn("Connection has not been established, could not
create a transit URI. Returning null.");
return null;
}
-try {
-final String masterAddress =
connection.getAdmin().getClusterStatus().getMaster().getHostAndPort();
-return "hbase://" + masterAddress + "/" + tableName + (rowKey
!= null && !rowKey.isEmpty() ? "/" + rowKey : "");
-} catch (IOException e) {
-throw new RuntimeException("Failed to get HBase Admin
interface, due to " + e, e);
-}
+final String transitUriMasterAddress =
StringUtils.isEmpty(masterAddress) ? "unknown" : masterAddress;
--- End diff --
We are sending back "unknown" if the master address is empty. Won't that be
a problem, if it happens to be empty? I mean when
`session.getProvenanceReporter().fetch(handlerFlowFile, transitUri);` gets
executed on the HBase processors?
---
[jira] [Updated] (NIFI-4866) HBase_1_1_2_ClientService performance dropped due to accessing HBase admin upon every FlowFile transfer
[ https://issues.apache.org/jira/browse/NIFI-4866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Kawamura updated NIFI-4866: Status: Patch Available (was: In Progress) > HBase_1_1_2_ClientService performance dropped due to accessing HBase admin > upon every FlowFile transfer > --- > > Key: NIFI-4866 > URL: https://issues.apache.org/jira/browse/NIFI-4866 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Reporter: Koji Kawamura >Assignee: Koji Kawamura >Priority: Critical > > NIFI-4543 added > "connection.getAdmin().getClusterStatus().getMaster().getHostAndPort()” call > from HBase_1_1_2_ClientService.toTransitUri method in order to get HBase > master address, which is called upon every FlowFile transfer and essentially > doubles the traffic through the HBase connector. > The performance of our PutHBaseJSON processor dropped to 1/3 after deploying > NiFi 1.5.0. > Those lines of code should be moved from toTransitUri to createConnection. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4866) HBase_1_1_2_ClientService performance dropped due to accessing HBase admin upon every FlowFile transfer
[ https://issues.apache.org/jira/browse/NIFI-4866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16359211#comment-16359211 ] ASF GitHub Bot commented on NIFI-4866: -- Github user ijokarumawak commented on the issue: https://github.com/apache/nifi/pull/2464 Confirmed that transit URI is created correctly: - 1.5 hbase://0.hdp.aws.mine:16000/nifi-4866/a20f31cb-f503-4e0c-9806-73a2ed7fd43e - 1.6 hbase://0.hdp.aws.mine:16000/nifi-4866/93376ca2-ec3c-421f-8e4e-93cf28677b31 Confirmed performance issue has been addressed: - Used a simple GenerateFlowFile and PutHBaseCell flow, template is available [here](https://gist.github.com/ijokarumawak/85db60ca71f1825f543c18c62bf7c3fd) - Putting only 1 cell with 4bytes data, batched size is 25 - 1.5 saturated about at the rate of 14k FlowFIles/5min - 1.6 saturated about at the rate of 500k FlowFiles/5min NiFi 1.5.0  NiFi 1.6.0-SNAPSHOT with this fix  > HBase_1_1_2_ClientService performance dropped due to accessing HBase admin > upon every FlowFile transfer > --- > > Key: NIFI-4866 > URL: https://issues.apache.org/jira/browse/NIFI-4866 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Reporter: Koji Kawamura >Assignee: Koji Kawamura >Priority: Critical > > NIFI-4543 added > "connection.getAdmin().getClusterStatus().getMaster().getHostAndPort()” call > from HBase_1_1_2_ClientService.toTransitUri method in order to get HBase > master address, which is called upon every FlowFile transfer and essentially > doubles the traffic through the HBase connector. > The performance of our PutHBaseJSON processor dropped to 1/3 after deploying > NiFi 1.5.0. > Those lines of code should be moved from toTransitUri to createConnection. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi issue #2464: NIFI-4866: Fixed HBase performance issue
Github user ijokarumawak commented on the issue: https://github.com/apache/nifi/pull/2464 Confirmed that transit URI is created correctly: - 1.5 hbase://0.hdp.aws.mine:16000/nifi-4866/a20f31cb-f503-4e0c-9806-73a2ed7fd43e - 1.6 hbase://0.hdp.aws.mine:16000/nifi-4866/93376ca2-ec3c-421f-8e4e-93cf28677b31 Confirmed performance issue has been addressed: - Used a simple GenerateFlowFile and PutHBaseCell flow, template is available [here](https://gist.github.com/ijokarumawak/85db60ca71f1825f543c18c62bf7c3fd) - Putting only 1 cell with 4bytes data, batched size is 25 - 1.5 saturated about at the rate of 14k FlowFIles/5min - 1.6 saturated about at the rate of 500k FlowFiles/5min NiFi 1.5.0  NiFi 1.6.0-SNAPSHOT with this fix  ---
[jira] [Commented] (NIFI-4866) HBase_1_1_2_ClientService performance dropped due to accessing HBase admin upon every FlowFile transfer
[ https://issues.apache.org/jira/browse/NIFI-4866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16359207#comment-16359207 ] ASF GitHub Bot commented on NIFI-4866: -- GitHub user ijokarumawak opened a pull request: https://github.com/apache/nifi/pull/2464 NIFI-4866: Fixed HBase performance issue HBase_1_1_2_ClientService performance dropped due to accessing HBase admin upon every FlowFile transfer. Fixed that by getting master address only once when the service is enabled and a connection is established. Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/ijokarumawak/nifi nifi-4866 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2464.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2464 commit 0c8bf33d6d7165e089a43adbecb6539d6f64a5f5 Author: Koji Kawamura Date: 2018-02-10T02:27:17Z NIFI-4866: Fixed HBase performance issue HBase_1_1_2_ClientService performance dropped due to accessing HBase admin upon every FlowFile transfer. Fixed that by getting master address only once when the service is enabled and a connection is established. > HBase_1_1_2_ClientService performance dropped due to accessing HBase admin > upon every FlowFile transfer > --- > > Key: NIFI-4866 > URL: https://issues.apache.org/jira/browse/NIFI-4866 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Reporter: Koji Kawamura >Assignee: Koji Kawamura >Priority: Critical > > NIFI-4543 added > "connection.getAdmin().getClusterStatus().getMaster().getHostAndPort()” call > from HBase_1_1_2_ClientService.toTransitUri method in order to get HBase > master address, which is called upon every FlowFile transfer and essentially > doubles the traffic through the HBase connector. > The performance of our PutHBaseJSON processor dropped to 1/3 after deploying > NiFi 1.5.0. > Those lines of code should be moved from toTransitUri to createConnection. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi pull request #2464: NIFI-4866: Fixed HBase performance issue
GitHub user ijokarumawak opened a pull request: https://github.com/apache/nifi/pull/2464 NIFI-4866: Fixed HBase performance issue HBase_1_1_2_ClientService performance dropped due to accessing HBase admin upon every FlowFile transfer. Fixed that by getting master address only once when the service is enabled and a connection is established. Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/ijokarumawak/nifi nifi-4866 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2464.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2464 commit 0c8bf33d6d7165e089a43adbecb6539d6f64a5f5 Author: Koji Kawamura Date: 2018-02-10T02:27:17Z NIFI-4866: Fixed HBase performance issue HBase_1_1_2_ClientService performance dropped due to accessing HBase admin upon every FlowFile transfer. Fixed that by getting master address only once when the service is enabled and a connection is established. ---
[jira] [Updated] (NIFI-3367) TLS Toolkit should enforce minimum length restriction on CA token
[
https://issues.apache.org/jira/browse/NIFI-3367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andy LoPresto updated NIFI-3367:
Resolution: Fixed
Fix Version/s: 1.6.0
Status: Resolved (was: Patch Available)
A minimum length check of 16 bytes is enforced on the token. The rate limiting
was not implemented.
> TLS Toolkit should enforce minimum length restriction on CA token
> -
>
> Key: NIFI-3367
> URL: https://issues.apache.org/jira/browse/NIFI-3367
> Project: Apache NiFi
> Issue Type: Bug
> Components: Tools and Build
>Affects Versions: 1.1.1
>Reporter: Andy LoPresto
>Assignee: Andy LoPresto
>Priority: Major
> Labels: security, tls-toolkit
> Fix For: 1.6.0
>
>
> The TLS Toolkit uses a shared secret "token" when running in client/server
> mode in order to perform pre-authentication when requesting a signed
> certificate from the CA. There is a validation that this token is *required*,
> but not that it is of a certain length. Because the HMAC construction is
> available in the source code, the process could easily be brute-forced if the
> token value is short. We should enforce a minimum length of 16 bytes
> (regardless if read from {{config.json}} or provided via command line).
> We may also want to add exponential rate-limiting on failed HMAC values for
> the same requested public key DN in order to mitigate malicious requests.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFI-3367) TLS Toolkit should enforce minimum length restriction on CA token
[
https://issues.apache.org/jira/browse/NIFI-3367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16359190#comment-16359190
]
ASF GitHub Bot commented on NIFI-3367:
--
Github user asfgit closed the pull request at:
https://github.com/apache/nifi/pull/2463
> TLS Toolkit should enforce minimum length restriction on CA token
> -
>
> Key: NIFI-3367
> URL: https://issues.apache.org/jira/browse/NIFI-3367
> Project: Apache NiFi
> Issue Type: Bug
> Components: Tools and Build
>Affects Versions: 1.1.1
>Reporter: Andy LoPresto
>Assignee: Andy LoPresto
>Priority: Major
> Labels: security, tls-toolkit
>
> The TLS Toolkit uses a shared secret "token" when running in client/server
> mode in order to perform pre-authentication when requesting a signed
> certificate from the CA. There is a validation that this token is *required*,
> but not that it is of a certain length. Because the HMAC construction is
> available in the source code, the process could easily be brute-forced if the
> token value is short. We should enforce a minimum length of 16 bytes
> (regardless if read from {{config.json}} or provided via command line).
> We may also want to add exponential rate-limiting on failed HMAC values for
> the same requested public key DN in order to mitigate malicious requests.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] nifi pull request #2463: NIFI-3367 Enforced minimum length of shared secret ...
Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/2463 ---
[jira] [Commented] (NIFI-3367) TLS Toolkit should enforce minimum length restriction on CA token
[
https://issues.apache.org/jira/browse/NIFI-3367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16359189#comment-16359189
]
ASF subversion and git services commented on NIFI-3367:
---
Commit b7fdb235ee1055e24fdb3ac000cc8039751199ad in nifi's branch
refs/heads/master from Lori Buettner
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=b7fdb23 ]
NIFI-3367 Added token length check and unit test.
This closes #2463.
Signed-off-by: Andy LoPresto
> TLS Toolkit should enforce minimum length restriction on CA token
> -
>
> Key: NIFI-3367
> URL: https://issues.apache.org/jira/browse/NIFI-3367
> Project: Apache NiFi
> Issue Type: Bug
> Components: Tools and Build
>Affects Versions: 1.1.1
>Reporter: Andy LoPresto
>Assignee: Andy LoPresto
>Priority: Major
> Labels: security, tls-toolkit
>
> The TLS Toolkit uses a shared secret "token" when running in client/server
> mode in order to perform pre-authentication when requesting a signed
> certificate from the CA. There is a validation that this token is *required*,
> but not that it is of a certain length. Because the HMAC construction is
> available in the source code, the process could easily be brute-forced if the
> token value is short. We should enforce a minimum length of 16 bytes
> (regardless if read from {{config.json}} or provided via command line).
> We may also want to add exponential rate-limiting on failed HMAC values for
> the same requested public key DN in order to mitigate malicious requests.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] nifi issue #2463: NIFI-3367 Enforced minimum length of shared secret in TLS ...
Github user alopresto commented on the issue: https://github.com/apache/nifi/pull/2463 Ran the toolkit in client/server mode. Both rejected a token < 16 bytes with an informative error message. With a long token, the process succeeded. Ran `contrib-check` and all tests pass. +1, merging. ---
[jira] [Commented] (NIFI-3367) TLS Toolkit should enforce minimum length restriction on CA token
[
https://issues.apache.org/jira/browse/NIFI-3367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16359187#comment-16359187
]
ASF GitHub Bot commented on NIFI-3367:
--
Github user alopresto commented on the issue:
https://github.com/apache/nifi/pull/2463
Ran the toolkit in client/server mode. Both rejected a token < 16 bytes
with an informative error message. With a long token, the process succeeded.
Ran `contrib-check` and all tests pass. +1, merging.
> TLS Toolkit should enforce minimum length restriction on CA token
> -
>
> Key: NIFI-3367
> URL: https://issues.apache.org/jira/browse/NIFI-3367
> Project: Apache NiFi
> Issue Type: Bug
> Components: Tools and Build
>Affects Versions: 1.1.1
>Reporter: Andy LoPresto
>Assignee: Andy LoPresto
>Priority: Major
> Labels: security, tls-toolkit
>
> The TLS Toolkit uses a shared secret "token" when running in client/server
> mode in order to perform pre-authentication when requesting a signed
> certificate from the CA. There is a validation that this token is *required*,
> but not that it is of a certain length. Because the HMAC construction is
> available in the source code, the process could easily be brute-forced if the
> token value is short. We should enforce a minimum length of 16 bytes
> (regardless if read from {{config.json}} or provided via command line).
> We may also want to add exponential rate-limiting on failed HMAC values for
> the same requested public key DN in order to mitigate malicious requests.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (NIFI-4866) HBase_1_1_2_ClientService performance dropped due to accessing HBase admin upon every FlowFile transfer
Koji Kawamura created NIFI-4866: --- Summary: HBase_1_1_2_ClientService performance dropped due to accessing HBase admin upon every FlowFile transfer Key: NIFI-4866 URL: https://issues.apache.org/jira/browse/NIFI-4866 Project: Apache NiFi Issue Type: Bug Components: Extensions Reporter: Koji Kawamura Assignee: Koji Kawamura NIFI-4543 added "connection.getAdmin().getClusterStatus().getMaster().getHostAndPort()” call from HBase_1_1_2_ClientService.toTransitUri method in order to get HBase master address, which is called upon every FlowFile transfer and essentially doubles the traffic through the HBase connector. The performance of our PutHBaseJSON processor dropped to 1/3 after deploying NiFi 1.5.0. Those lines of code should be moved from toTransitUri to createConnection. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-3367) TLS Toolkit should enforce minimum length restriction on CA token
[
https://issues.apache.org/jira/browse/NIFI-3367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16359174#comment-16359174
]
ASF GitHub Bot commented on NIFI-3367:
--
Github user alopresto commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2463#discussion_r167382434
--- Diff:
nifi-toolkit/nifi-toolkit-tls/src/test/java/org/apache/nifi/toolkit/tls/util/TlsHelperTest.java
---
@@ -171,6 +172,43 @@ private Date inFuture(int days) {
return new Date(System.currentTimeMillis() +
TimeUnit.DAYS.toMillis(days));
}
+@Test
+public void testTokenLengthInCalculateHmac() throws
CertificateException, NoSuchAlgorithmException {
+List badTokens = new ArrayList<>();
+List goodTokens = new ArrayList<>();
+badTokens.add(null);
+badTokens.add("");
+badTokens.add("123");
+goodTokens.add("0123456789abcdefghijklm");
+goodTokens.add("0123456789abcdef");
+
+String dn = "CN=testDN,O=testOrg";
+X509Certificate x509Certificate =
CertificateUtils.generateSelfSignedX509Certificate(TlsHelper.generateKeyPair(keyPairAlgorithm,
keySize), dn, signingAlgorithm, days);
+PublicKey pubKey = x509Certificate.getPublicKey();
+
+for (String token : badTokens) {
+try {
+TlsHelper.calculateHMac(token, pubKey);
+fail("HMAC was calculated with a token that was too
short.");
+} catch (GeneralSecurityException e) {
+assertEquals("Token does not meet minimum size of 16
bytes.", e.getMessage());
+} catch (IllegalArgumentException e) {
+assertEquals("Token cannot be null", e.getMessage());
+}
+}
+
+for (String token : goodTokens) {
+try {
+byte[] hmac = TlsHelper.calculateHMac(token, pubKey);
+assertTrue("HMAC length ok", hmac.length > 0);
+} catch (GeneralSecurityException e) {
+fail(e.getMessage());
+}
+}
+
--- End diff --
Please remove unnecessary whitespace.
> TLS Toolkit should enforce minimum length restriction on CA token
> -
>
> Key: NIFI-3367
> URL: https://issues.apache.org/jira/browse/NIFI-3367
> Project: Apache NiFi
> Issue Type: Bug
> Components: Tools and Build
>Affects Versions: 1.1.1
>Reporter: Andy LoPresto
>Assignee: Andy LoPresto
>Priority: Major
> Labels: security, tls-toolkit
>
> The TLS Toolkit uses a shared secret "token" when running in client/server
> mode in order to perform pre-authentication when requesting a signed
> certificate from the CA. There is a validation that this token is *required*,
> but not that it is of a certain length. Because the HMAC construction is
> available in the source code, the process could easily be brute-forced if the
> token value is short. We should enforce a minimum length of 16 bytes
> (regardless if read from {{config.json}} or provided via command line).
> We may also want to add exponential rate-limiting on failed HMAC values for
> the same requested public key DN in order to mitigate malicious requests.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] nifi pull request #2463: NIFI-3367 Enforced minimum length of shared secret ...
Github user alopresto commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2463#discussion_r167382434
--- Diff:
nifi-toolkit/nifi-toolkit-tls/src/test/java/org/apache/nifi/toolkit/tls/util/TlsHelperTest.java
---
@@ -171,6 +172,43 @@ private Date inFuture(int days) {
return new Date(System.currentTimeMillis() +
TimeUnit.DAYS.toMillis(days));
}
+@Test
+public void testTokenLengthInCalculateHmac() throws
CertificateException, NoSuchAlgorithmException {
+List badTokens = new ArrayList<>();
+List goodTokens = new ArrayList<>();
+badTokens.add(null);
+badTokens.add("");
+badTokens.add("123");
+goodTokens.add("0123456789abcdefghijklm");
+goodTokens.add("0123456789abcdef");
+
+String dn = "CN=testDN,O=testOrg";
+X509Certificate x509Certificate =
CertificateUtils.generateSelfSignedX509Certificate(TlsHelper.generateKeyPair(keyPairAlgorithm,
keySize), dn, signingAlgorithm, days);
+PublicKey pubKey = x509Certificate.getPublicKey();
+
+for (String token : badTokens) {
+try {
+TlsHelper.calculateHMac(token, pubKey);
+fail("HMAC was calculated with a token that was too
short.");
+} catch (GeneralSecurityException e) {
+assertEquals("Token does not meet minimum size of 16
bytes.", e.getMessage());
+} catch (IllegalArgumentException e) {
+assertEquals("Token cannot be null", e.getMessage());
+}
+}
+
+for (String token : goodTokens) {
+try {
+byte[] hmac = TlsHelper.calculateHMac(token, pubKey);
+assertTrue("HMAC length ok", hmac.length > 0);
+} catch (GeneralSecurityException e) {
+fail(e.getMessage());
+}
+}
+
--- End diff --
Please remove unnecessary whitespace.
---
[jira] [Commented] (NIFI-3367) TLS Toolkit should enforce minimum length restriction on CA token
[
https://issues.apache.org/jira/browse/NIFI-3367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16359173#comment-16359173
]
ASF GitHub Bot commented on NIFI-3367:
--
Github user alopresto commented on the issue:
https://github.com/apache/nifi/pull/2463
Reviewing...
> TLS Toolkit should enforce minimum length restriction on CA token
> -
>
> Key: NIFI-3367
> URL: https://issues.apache.org/jira/browse/NIFI-3367
> Project: Apache NiFi
> Issue Type: Bug
> Components: Tools and Build
>Affects Versions: 1.1.1
>Reporter: Andy LoPresto
>Assignee: Andy LoPresto
>Priority: Major
> Labels: security, tls-toolkit
>
> The TLS Toolkit uses a shared secret "token" when running in client/server
> mode in order to perform pre-authentication when requesting a signed
> certificate from the CA. There is a validation that this token is *required*,
> but not that it is of a certain length. Because the HMAC construction is
> available in the source code, the process could easily be brute-forced if the
> token value is short. We should enforce a minimum length of 16 bytes
> (regardless if read from {{config.json}} or provided via command line).
> We may also want to add exponential rate-limiting on failed HMAC values for
> the same requested public key DN in order to mitigate malicious requests.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] nifi issue #2463: NIFI-3367 Enforced minimum length of shared secret in TLS ...
Github user alopresto commented on the issue: https://github.com/apache/nifi/pull/2463 Reviewing... ---
[jira] [Updated] (NIFI-3367) TLS Toolkit should enforce minimum length restriction on CA token
[
https://issues.apache.org/jira/browse/NIFI-3367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andy LoPresto updated NIFI-3367:
Status: Patch Available (was: In Progress)
Lori Buettner contributed two commits which enforce a minimum length on the
token and added/fixed existing unit tests to demonstrate this.
> TLS Toolkit should enforce minimum length restriction on CA token
> -
>
> Key: NIFI-3367
> URL: https://issues.apache.org/jira/browse/NIFI-3367
> Project: Apache NiFi
> Issue Type: Bug
> Components: Tools and Build
>Affects Versions: 1.1.1
>Reporter: Andy LoPresto
>Assignee: Andy LoPresto
>Priority: Major
> Labels: security, tls-toolkit
>
> The TLS Toolkit uses a shared secret "token" when running in client/server
> mode in order to perform pre-authentication when requesting a signed
> certificate from the CA. There is a validation that this token is *required*,
> but not that it is of a certain length. Because the HMAC construction is
> available in the source code, the process could easily be brute-forced if the
> token value is short. We should enforce a minimum length of 16 bytes
> (regardless if read from {{config.json}} or provided via command line).
> We may also want to add exponential rate-limiting on failed HMAC values for
> the same requested public key DN in order to mitigate malicious requests.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (NIFI-3367) TLS Toolkit should enforce minimum length restriction on CA token
[
https://issues.apache.org/jira/browse/NIFI-3367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andy LoPresto reassigned NIFI-3367:
---
Assignee: Andy LoPresto
> TLS Toolkit should enforce minimum length restriction on CA token
> -
>
> Key: NIFI-3367
> URL: https://issues.apache.org/jira/browse/NIFI-3367
> Project: Apache NiFi
> Issue Type: Bug
> Components: Tools and Build
>Affects Versions: 1.1.1
>Reporter: Andy LoPresto
>Assignee: Andy LoPresto
>Priority: Major
> Labels: security, tls-toolkit
>
> The TLS Toolkit uses a shared secret "token" when running in client/server
> mode in order to perform pre-authentication when requesting a signed
> certificate from the CA. There is a validation that this token is *required*,
> but not that it is of a certain length. Because the HMAC construction is
> available in the source code, the process could easily be brute-forced if the
> token value is short. We should enforce a minimum length of 16 bytes
> (regardless if read from {{config.json}} or provided via command line).
> We may also want to add exponential rate-limiting on failed HMAC values for
> the same requested public key DN in order to mitigate malicious requests.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFI-3367) TLS Toolkit should enforce minimum length restriction on CA token
[
https://issues.apache.org/jira/browse/NIFI-3367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16359171#comment-16359171
]
ASF GitHub Bot commented on NIFI-3367:
--
GitHub user alopresto opened a pull request:
https://github.com/apache/nifi/pull/2463
NIFI-3367 Enforced minimum length of shared secret in TLS toolkit
client/server mode
Thank you for submitting a contribution to Apache NiFi.
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
### For all changes:
- [x] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?
- [x] Does your PR title start with NIFI- where is the JIRA number
you are trying to resolve? Pay particular attention to the hyphen "-" character.
- [x] Has your PR been rebased against the latest commit within the target
branch (typically master)?
- [ ] Is your initial contribution a single, squashed commit?
### For code changes:
- [x] Have you ensured that the full suite of tests is executed via mvn
-Pcontrib-check clean install at the root nifi folder?
- [x] Have you written or updated unit tests to verify your changes?
- [ ] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the LICENSE file, including the main
LICENSE file under nifi-assembly?
- [ ] If applicable, have you updated the NOTICE file, including the main
NOTICE file found under nifi-assembly?
- [ ] If adding new Properties, have you added .displayName in addition to
.name (programmatic access) for each of the new properties?
### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in
which it is rendered?
### Note:
Please ensure that once the PR is submitted, you check travis-ci for build
issues and submit an update to your PR as soon as possible.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/alopresto/nifi NIFI-3367
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/nifi/pull/2463.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2463
commit 79a0aafe51a6d806ef3952650139ebbcd007e274
Author: Lori Buettner
Date: 2018-02-10T00:46:33Z
NIFI-3367 Added token length check and unit test.
commit 5d9680a513bc9e28067eed45b07823f250c7
Author: Lori Buettner
Date: 2018-02-10T01:04:58Z
NIFI-3367 Resolved failing unit tests.
> TLS Toolkit should enforce minimum length restriction on CA token
> -
>
> Key: NIFI-3367
> URL: https://issues.apache.org/jira/browse/NIFI-3367
> Project: Apache NiFi
> Issue Type: Bug
> Components: Tools and Build
>Affects Versions: 1.1.1
>Reporter: Andy LoPresto
>Priority: Major
> Labels: security, tls-toolkit
>
> The TLS Toolkit uses a shared secret "token" when running in client/server
> mode in order to perform pre-authentication when requesting a signed
> certificate from the CA. There is a validation that this token is *required*,
> but not that it is of a certain length. Because the HMAC construction is
> available in the source code, the process could easily be brute-forced if the
> token value is short. We should enforce a minimum length of 16 bytes
> (regardless if read from {{config.json}} or provided via command line).
> We may also want to add exponential rate-limiting on failed HMAC values for
> the same requested public key DN in order to mitigate malicious requests.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] nifi pull request #2463: NIFI-3367 Enforced minimum length of shared secret ...
GitHub user alopresto opened a pull request: https://github.com/apache/nifi/pull/2463 NIFI-3367 Enforced minimum length of shared secret in TLS toolkit client/server mode Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [x] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [x] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/alopresto/nifi NIFI-3367 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2463.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2463 commit 79a0aafe51a6d806ef3952650139ebbcd007e274 Author: Lori Buettner Date: 2018-02-10T00:46:33Z NIFI-3367 Added token length check and unit test. commit 5d9680a513bc9e28067eed45b07823f250c7 Author: Lori Buettner Date: 2018-02-10T01:04:58Z NIFI-3367 Resolved failing unit tests. ---
[GitHub] nifi issue #2180: Added GetMongoAggregation to support running Mongo aggrega...
Github user MikeThomsen commented on the issue: https://github.com/apache/nifi/pull/2180 @mattyb149 This should be ready for merge now. ---
[GitHub] nifi issue #2294: NIFI-3538 Added DeleteHBaseRow
Github user MikeThomsen commented on the issue: https://github.com/apache/nifi/pull/2294 @ijokarumawak I think all of your concerns have been addressed now. ---
[jira] [Commented] (NIFI-3538) Add DeleteHBase processor(s)
[ https://issues.apache.org/jira/browse/NIFI-3538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16359093#comment-16359093 ] ASF GitHub Bot commented on NIFI-3538: -- Github user MikeThomsen commented on the issue: https://github.com/apache/nifi/pull/2294 @ijokarumawak I think all of your concerns have been addressed now. > Add DeleteHBase processor(s) > > > Key: NIFI-3538 > URL: https://issues.apache.org/jira/browse/NIFI-3538 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Reporter: Matt Burgess >Assignee: Mike Thomsen >Priority: Major > > NiFi currently has processors for storing and retrieving cells/rows in HBase, > but there is no mechanism for deleting records and/or tables. > I'm not sure if a single DeleteHBase processor could accomplish both, that > can be discussed under this Jira (and can be split out if necessary). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (NIFI-4865) Registry API command Create Version can produce a VersionedProcessGroup that causes NiFi to NPE on Version change
Daniel Chaffelson created NIFI-4865:
---
Summary: Registry API command Create Version can produce a
VersionedProcessGroup that causes NiFi to NPE on Version change
Key: NIFI-4865
URL: https://issues.apache.org/jira/browse/NIFI-4865
Project: Apache NiFi
Issue Type: Bug
Components: Flow Versioning
Affects Versions: 1.5.0
Environment: NiFi Docker 1.5.0 / NiFi-Registry Docker 0.1.0
Reporter: Daniel Chaffelson
I imported a complex (300kb) xml template into NiFi and placed it under version
control using the typical GUI commands.
Then using the Registry API command 'create version', I instructed it to make a
zero change new version, resulting in ver1 and ver2 being in the same bucket
and flow object.
Returning to the GUI, I instructed it to update to the new version, which
causes the below NPE.
I have retested this with a fresh environment and am able to reproduce it at
will, though as the template belongs to a client and is very detailed I cannot
in good faith share it directly on a public service - I am happy to supply it
directly in the interests of locking down this bug.
{noformat}
2018-02-09 21:45:40,900 INFO [Version Control Update Thread-1]
o.a.nifi.groups.StandardProcessGroup Updating
StandardProcessGroup[identifier=420d499d-2b1d-3af8-6cc3-2722817f4d49] to
VersionedFlowSnapshot[flowId=e799c617-0dd9-4685-b471-2987b19ad121,
flowName=test1, version=2, comments=null]; there are 0 differences to take into
account:
2018-02-09 21:45:40,903 INFO [Version Control Update Thread-1]
org.apache.nifi.web.api.VersionsResource Successfully updated flow; re-enabling
0 Controller Services
2018-02-09 21:45:40,903 INFO [Version Control Update Thread-1]
org.apache.nifi.web.api.VersionsResource Restarting 0 Processors
2018-02-09 21:45:40,903 ERROR [Version Control Update Thread-1]
org.apache.nifi.web.api.VersionsResource Failed to update flow to new version
java.lang.NullPointerException: null
at
org.apache.nifi.groups.StandardProcessGroup.updateConnection(StandardProcessGroup.java:3784)
at
org.apache.nifi.groups.StandardProcessGroup.updateProcessGroup(StandardProcessGroup.java:3656)
at
org.apache.nifi.groups.StandardProcessGroup.updateProcessGroup(StandardProcessGroup.java:3488)
at
org.apache.nifi.groups.StandardProcessGroup.updateFlow(StandardProcessGroup.java:3295)
at
org.apache.nifi.web.dao.impl.StandardProcessGroupDAO.updateProcessGroupFlow(StandardProcessGroupDAO.java:279)
at
org.apache.nifi.web.dao.impl.StandardProcessGroupDAO$$FastClassBySpringCGLIB$$10a99b47.invoke()
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
at
org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:738)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
at
org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:85)
at
org.apache.nifi.audit.ProcessGroupAuditor.updateProcessGroupFlowAdvice(ProcessGroupAuditor.java:258)
at sun.reflect.GeneratedMethodAccessor644.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:629)
at
org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:618)
at
org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:70)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at
org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at
org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:673)
at
org.apache.nifi.web.dao.impl.StandardProcessGroupDAO$$EnhancerBySpringCGLIB$$3bf4fdb8.updateProcessGroupFlow()
at
org.apache.nifi.web.StandardNiFiServiceFacade$13.update(StandardNiFiServiceFacade.java:4304)
at
org.apache.nifi.web.revision.NaiveRevisionManager.updateRevision(NaiveRevisionManager.java:120)
at
org.apache.nifi.web.StandardNiFiServiceFacade.updateProcessGroupContents(StandardNiFiServiceFacade.java:4300)
at
org.apache.nifi.web.StandardNiFiServiceFacade$$FastClassBySpringCGLIB$$358780e0.invoke()
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
at
org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:738)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
at
o
[jira] [Updated] (MINIFICPP-396) RPG does not pass SSLContextService to site to site client in a consistent way
[ https://issues.apache.org/jira/browse/MINIFICPP-396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] marco polo updated MINIFICPP-396: - Description: The RPG doesn't always pass the SSLContextService in a a consistent way. The documentation is confusing, so we should clear this up. Further, there are two paths to configure TLS comms, so we should make this more cohesive, but also backwards compatible. was:The RPG doesn't always pass the SSLContextService in a a consistent way. The documentation is confusing, so we should clear this up. > RPG does not pass SSLContextService to site to site client in a consistent way > -- > > Key: MINIFICPP-396 > URL: https://issues.apache.org/jira/browse/MINIFICPP-396 > Project: NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: marco polo >Assignee: marco polo >Priority: Blocker > > The RPG doesn't always pass the SSLContextService in a a consistent way. The > documentation is confusing, so we should clear this up. > Further, there are two paths to configure TLS comms, so we should make this > more cohesive, but also backwards compatible. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFIREG-124) As a user I want the sidenav table sorting to persist when I open dialogs.
[ https://issues.apache.org/jira/browse/NIFIREG-124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16358959#comment-16358959 ] ASF GitHub Bot commented on NIFIREG-124: GitHub user scottyaslan opened a pull request: https://github.com/apache/nifi-registry/pull/100 [NIFIREG-124] persist sidenav table sorting You can merge this pull request into a Git repository by running: $ git pull https://github.com/scottyaslan/nifi-registry NIFIREG-124 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi-registry/pull/100.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #100 commit edb1330b7a068cd1f60a29e9dd74c73e8306f2db Author: Scott Aslan Date: 2018-02-09T19:31:52Z [NIFIREG-124] persist sidenav table sorting > As a user I want the sidenav table sorting to persist when I open dialogs. > -- > > Key: NIFIREG-124 > URL: https://issues.apache.org/jira/browse/NIFIREG-124 > Project: NiFi Registry > Issue Type: Bug >Reporter: Scott Aslan >Assignee: Scott Aslan >Priority: Major > > When editing a group, if the user sorts the listed users in the group table > to be descending, but then click Add User button, the sort order in the users > in group table switches back to ascending, even if I make no change in the > Add User dialog. This bug also exists in when editing a user as well as when > editing a bucket. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi-registry pull request #100: [NIFIREG-124] persist sidenav table sorting
GitHub user scottyaslan opened a pull request: https://github.com/apache/nifi-registry/pull/100 [NIFIREG-124] persist sidenav table sorting You can merge this pull request into a Git repository by running: $ git pull https://github.com/scottyaslan/nifi-registry NIFIREG-124 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi-registry/pull/100.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #100 commit edb1330b7a068cd1f60a29e9dd74c73e8306f2db Author: Scott Aslan Date: 2018-02-09T19:31:52Z [NIFIREG-124] persist sidenav table sorting ---
[jira] [Resolved] (NIFI-4863) If changing version of flow and new flow has no connection to RPG Port, can get error saying unable to start components
[ https://issues.apache.org/jira/browse/NIFI-4863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Bende resolved NIFI-4863. --- Resolution: Fixed Fix Version/s: 1.6.0 > If changing version of flow and new flow has no connection to RPG Port, can > get error saying unable to start components > --- > > Key: NIFI-4863 > URL: https://issues.apache.org/jira/browse/NIFI-4863 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.5.0 >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Major > Fix For: 1.6.0 > > > When we have a Remote Process Group with multiple ports, we push to the flow > registry the ports that we have connections for. I have a connection to Port > A and save that to the flow registry, then create a connection to Port B. I > have RPG transmitting data. Then I revert local changes. This properly > reverts the flow but fails to restart components. This is because it is > attempting to start Port B, and there are no connections for that Port. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4863) If changing version of flow and new flow has no connection to RPG Port, can get error saying unable to start components
[ https://issues.apache.org/jira/browse/NIFI-4863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16358936#comment-16358936 ] ASF GitHub Bot commented on NIFI-4863: -- Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/2462 > If changing version of flow and new flow has no connection to RPG Port, can > get error saying unable to start components > --- > > Key: NIFI-4863 > URL: https://issues.apache.org/jira/browse/NIFI-4863 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.5.0 >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Major > Fix For: 1.6.0 > > > When we have a Remote Process Group with multiple ports, we push to the flow > registry the ports that we have connections for. I have a connection to Port > A and save that to the flow registry, then create a connection to Port B. I > have RPG transmitting data. Then I revert local changes. This properly > reverts the flow but fails to restart components. This is because it is > attempting to start Port B, and there are no connections for that Port. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4863) If changing version of flow and new flow has no connection to RPG Port, can get error saying unable to start components
[ https://issues.apache.org/jira/browse/NIFI-4863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16358935#comment-16358935 ] ASF subversion and git services commented on NIFI-4863: --- Commit 61c6f0305bc7b5c68456fc2d40e0536698821c47 in nifi's branch refs/heads/master from [~markap14] [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=61c6f03 ] NIFI-4863: Bug fixes to the way that we handled Remote Group Ports when changing flow version / reverting local changes - Everywhere that we ignore adding remote ports we should ignore removing remote ports as well in flow diffs This closes #2462. Signed-off-by: Bryan Bende > If changing version of flow and new flow has no connection to RPG Port, can > get error saying unable to start components > --- > > Key: NIFI-4863 > URL: https://issues.apache.org/jira/browse/NIFI-4863 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.5.0 >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Major > Fix For: 1.6.0 > > > When we have a Remote Process Group with multiple ports, we push to the flow > registry the ports that we have connections for. I have a connection to Port > A and save that to the flow registry, then create a connection to Port B. I > have RPG transmitting data. Then I revert local changes. This properly > reverts the flow but fails to restart components. This is because it is > attempting to start Port B, and there are no connections for that Port. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi pull request #2462: NIFI-4863: Bug fixes to the way that we handled Rem...
Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/2462 ---
[GitHub] nifi issue #2462: NIFI-4863: Bug fixes to the way that we handled Remote Gro...
Github user bbende commented on the issue: https://github.com/apache/nifi/pull/2462 +1 Ran through test scenarios with input and output ports on RPGs in versioned flows and this appears to clear up all the issues, merging to master ---
[jira] [Commented] (NIFI-4863) If changing version of flow and new flow has no connection to RPG Port, can get error saying unable to start components
[ https://issues.apache.org/jira/browse/NIFI-4863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16358934#comment-16358934 ] ASF GitHub Bot commented on NIFI-4863: -- Github user bbende commented on the issue: https://github.com/apache/nifi/pull/2462 +1 Ran through test scenarios with input and output ports on RPGs in versioned flows and this appears to clear up all the issues, merging to master > If changing version of flow and new flow has no connection to RPG Port, can > get error saying unable to start components > --- > > Key: NIFI-4863 > URL: https://issues.apache.org/jira/browse/NIFI-4863 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.5.0 >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Major > > When we have a Remote Process Group with multiple ports, we push to the flow > registry the ports that we have connections for. I have a connection to Port > A and save that to the flow registry, then create a connection to Port B. I > have RPG transmitting data. Then I revert local changes. This properly > reverts the flow but fails to restart components. This is because it is > attempting to start Port B, and there are no connections for that Port. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFIREG-139) Nifi registry should pick up the controller service definitions while deploying a flow
[
https://issues.apache.org/jira/browse/NIFIREG-139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16358918#comment-16358918
]
Kevin Doran commented on NIFIREG-139:
-
{quote}If it is at a higher level then it is an external reference and will not
be copied but rather an attempt to bind to a similar one in the other
environment will be made.{quote}
I'm not sure this is true for controller services. [~markap14] could verify.
> Nifi registry should pick up the controller service definitions while
> deploying a flow
> --
>
> Key: NIFIREG-139
> URL: https://issues.apache.org/jira/browse/NIFIREG-139
> Project: NiFi Registry
> Issue Type: Bug
>Affects Versions: 0.1.0
>Reporter: ohad israeli
>Priority: Critical
>
> I am trying to use the registry to copy a simple flow for example of
> httprequest->joltconversion->httpresponse.
> This flow is using the internal http context service which is installed and
> configured on the source (dev env). While deploying the flow to the target
> (prod env) the flow cant find the service on the target (prod end)
> to summarize, on deployment of a flow if the flow is using a service
> controller its setting should also be deployed on the target machine
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (MINIFICPP-31) Support UpdateAttribute for nifi-minifi-cpp
[ https://issues.apache.org/jira/browse/MINIFICPP-31?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16358881#comment-16358881 ] ASF GitHub Bot commented on MINIFICPP-31: - Github user phrocker commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/262 UpdateAttributeTests fails when expression language is not built. May want to put the expression language specific portion of the test under Expression language tests and have the no-op tests where it currently exists. Just an idea. > Support UpdateAttribute for nifi-minifi-cpp > --- > > Key: MINIFICPP-31 > URL: https://issues.apache.org/jira/browse/MINIFICPP-31 > Project: NiFi MiNiFi C++ > Issue Type: New Feature >Reporter: Randy Gelhausen >Assignee: Andrew Christianson >Priority: Major > > nifi-minifi-cpp agents can generate multiple "streams" of flowfiles. > For instance, to monitor a host, nifi-minifi-cpp runs nmon, ps, netstat, and > gathers logfiles from applications. > But, for a given flowfile, any downstream NiFi collectors wont have > visibility into the originating hostname, nor metadata about which "stream" > (ExecuteProcess(nmon), ExecuteProcess(ps), TailFile(app1), TailFile(app2)) > generated it. > One solution is to use a separate InputPort for each stream. This works, but > burdens both the team working on agent flows, and the team managing the > collector- they have to be in concert. > A simpler (better?) approach is to allow agent teams to tag flowfiles with > differentiating metadata via use of UpdateAttribute. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi-minifi-cpp issue #262: MINIFICPP-31 Added UpdateAttribute processor
Github user phrocker commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/262 UpdateAttributeTests fails when expression language is not built. May want to put the expression language specific portion of the test under Expression language tests and have the no-op tests where it currently exists. Just an idea. ---
[jira] [Commented] (NIFI-4863) If changing version of flow and new flow has no connection to RPG Port, can get error saying unable to start components
[ https://issues.apache.org/jira/browse/NIFI-4863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16358849#comment-16358849 ] ASF GitHub Bot commented on NIFI-4863: -- Github user bbende commented on the issue: https://github.com/apache/nifi/pull/2462 Reviewing... > If changing version of flow and new flow has no connection to RPG Port, can > get error saying unable to start components > --- > > Key: NIFI-4863 > URL: https://issues.apache.org/jira/browse/NIFI-4863 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.5.0 >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Major > > When we have a Remote Process Group with multiple ports, we push to the flow > registry the ports that we have connections for. I have a connection to Port > A and save that to the flow registry, then create a connection to Port B. I > have RPG transmitting data. Then I revert local changes. This properly > reverts the flow but fails to restart components. This is because it is > attempting to start Port B, and there are no connections for that Port. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4360) Add support for Azure Data Lake Store (ADLS)
[ https://issues.apache.org/jira/browse/NIFI-4360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16358848#comment-16358848 ] Atul Sikaria commented on NIFI-4360: [~joewitt], [~pvillard], [~jzemerick]: This has been hanging in the air for a while. Looks like all the comments are addressed here - is there anything else we can do to push this along? Thanks for your help! > Add support for Azure Data Lake Store (ADLS) > > > Key: NIFI-4360 > URL: https://issues.apache.org/jira/browse/NIFI-4360 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Reporter: Milan Chandna >Assignee: Milan Chandna >Priority: Major > Labels: adls, azure, hdfs > Original Estimate: 336h > Remaining Estimate: 336h > > Currently ingress and egress in ADLS account is possible only using HDFS > processors. > Opening this feature to support separate processors for interaction with ADLS > accounts directly. > Benefits are many like > - simple configuration. > - Helping users not familiar with HDFS > - Helping users who currently are accessing ADLS accounts directly. > - using the ADLS SDK rather than HDFS client, one lesser layer to go through. > Can be achieved by adding separate ADLS processors. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi issue #2462: NIFI-4863: Bug fixes to the way that we handled Remote Gro...
Github user bbende commented on the issue: https://github.com/apache/nifi/pull/2462 Reviewing... ---
[jira] [Resolved] (NIFIREG-126) Entering an invalid bucket id in a deep link causes JS error
[ https://issues.apache.org/jira/browse/NIFIREG-126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Doran resolved NIFIREG-126. - Resolution: Fixed Fix Version/s: 0.2.0 > Entering an invalid bucket id in a deep link causes JS error > > > Key: NIFIREG-126 > URL: https://issues.apache.org/jira/browse/NIFIREG-126 > Project: NiFi Registry > Issue Type: Bug >Affects Versions: 0.1.0 >Reporter: Scott Aslan >Assignee: Scott Aslan >Priority: Major > Fix For: 0.2.0 > > > As a user when I enter an invalid deep link I want to be routed to view all > the items in all the buckets that I am authorized to view and to be notified > that the requested bucket/item id is invalid. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi-registry pull request #99: [NIFIREG-126] adding some polish and testing...
Github user asfgit closed the pull request at: https://github.com/apache/nifi-registry/pull/99 ---
[jira] [Commented] (NIFIREG-126) Entering an invalid bucket id in a deep link causes JS error
[ https://issues.apache.org/jira/browse/NIFIREG-126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16358847#comment-16358847 ] ASF GitHub Bot commented on NIFIREG-126: Github user asfgit closed the pull request at: https://github.com/apache/nifi-registry/pull/99 > Entering an invalid bucket id in a deep link causes JS error > > > Key: NIFIREG-126 > URL: https://issues.apache.org/jira/browse/NIFIREG-126 > Project: NiFi Registry > Issue Type: Bug >Affects Versions: 0.1.0 >Reporter: Scott Aslan >Assignee: Scott Aslan >Priority: Major > > As a user when I enter an invalid deep link I want to be routed to view all > the items in all the buckets that I am authorized to view and to be notified > that the requested bucket/item id is invalid. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi-minifi-cpp pull request #263: Minificpp 396
GitHub user phrocker opened a pull request: https://github.com/apache/nifi-minifi-cpp/pull/263 Minificpp 396 Thank you for submitting a contribution to Apache NiFi - MiNiFi C++. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with MINIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file? - [ ] If applicable, have you updated the NOTICE file? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/phrocker/nifi-minifi-cpp MINIFICPP-396 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi-minifi-cpp/pull/263.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #263 commit b84a02c3e400e175c5863460f7ebe60b85872bb1 Author: Marc Parisi Date: 2018-02-09T18:02:45Z MINIFICPP-396: Make ability to configure SSL in HTTP and Socket SiteToSite backwards compatible commit 756c9a327a2b57849b5a7038d3039d9947abfc33 Author: Marc Parisi Date: 2018-02-09T19:16:26Z MINIFICPP-396: Fix MQTT travis failures ---
[jira] [Commented] (NIFI-3538) Add DeleteHBase processor(s)
[ https://issues.apache.org/jira/browse/NIFI-3538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16358824#comment-16358824 ] ASF GitHub Bot commented on NIFI-3538: -- Github user MikeThomsen commented on a diff in the pull request: https://github.com/apache/nifi/pull/2294#discussion_r167319106 --- Diff: nifi-nar-bundles/nifi-standard-services/nifi-hbase-client-service-api/src/main/java/org/apache/nifi/hbase/HBaseClientService.java --- @@ -116,6 +117,13 @@ */ void delete(String tableName, byte[] rowId) throws IOException; +/** + * Deletes a list of rows in HBase. All cells are deleted. + * + */ + --- End diff -- Done > Add DeleteHBase processor(s) > > > Key: NIFI-3538 > URL: https://issues.apache.org/jira/browse/NIFI-3538 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Reporter: Matt Burgess >Assignee: Mike Thomsen >Priority: Major > > NiFi currently has processors for storing and retrieving cells/rows in HBase, > but there is no mechanism for deleting records and/or tables. > I'm not sure if a single DeleteHBase processor could accomplish both, that > can be discussed under this Jira (and can be split out if necessary). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-3538) Add DeleteHBase processor(s)
[
https://issues.apache.org/jira/browse/NIFI-3538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16358823#comment-16358823
]
ASF GitHub Bot commented on NIFI-3538:
--
Github user MikeThomsen commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2294#discussion_r167319088
--- Diff:
nifi-nar-bundles/nifi-hbase-bundle/nifi-hbase-processors/src/test/java/org/apache/nifi/hbase/MockHBaseClientService.java
---
@@ -71,6 +73,43 @@ public void delete(String tableName, byte[] rowId)
throws IOException {
throw new UnsupportedOperationException();
}
+@Override
+public void delete(String tableName, List rowIds) throws
IOException {
+if (throwException) {
+throw new RuntimeException("Simulated connectivity error");
+}
+
+Random random = new Random();
+int location = 0;
+if (rowIds.size() > 1) {
+while (location == 0) {
+location = random.nextInt(rowIds.size());
--- End diff --
Done
> Add DeleteHBase processor(s)
>
>
> Key: NIFI-3538
> URL: https://issues.apache.org/jira/browse/NIFI-3538
> Project: Apache NiFi
> Issue Type: New Feature
> Components: Extensions
>Reporter: Matt Burgess
>Assignee: Mike Thomsen
>Priority: Major
>
> NiFi currently has processors for storing and retrieving cells/rows in HBase,
> but there is no mechanism for deleting records and/or tables.
> I'm not sure if a single DeleteHBase processor could accomplish both, that
> can be discussed under this Jira (and can be split out if necessary).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] nifi pull request #2294: NIFI-3538 Added DeleteHBaseRow
Github user MikeThomsen commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2294#discussion_r167319088
--- Diff:
nifi-nar-bundles/nifi-hbase-bundle/nifi-hbase-processors/src/test/java/org/apache/nifi/hbase/MockHBaseClientService.java
---
@@ -71,6 +73,43 @@ public void delete(String tableName, byte[] rowId)
throws IOException {
throw new UnsupportedOperationException();
}
+@Override
+public void delete(String tableName, List rowIds) throws
IOException {
+if (throwException) {
+throw new RuntimeException("Simulated connectivity error");
+}
+
+Random random = new Random();
+int location = 0;
+if (rowIds.size() > 1) {
+while (location == 0) {
+location = random.nextInt(rowIds.size());
--- End diff --
Done
---
[GitHub] nifi pull request #2294: NIFI-3538 Added DeleteHBaseRow
Github user MikeThomsen commented on a diff in the pull request: https://github.com/apache/nifi/pull/2294#discussion_r167319106 --- Diff: nifi-nar-bundles/nifi-standard-services/nifi-hbase-client-service-api/src/main/java/org/apache/nifi/hbase/HBaseClientService.java --- @@ -116,6 +117,13 @@ */ void delete(String tableName, byte[] rowId) throws IOException; +/** + * Deletes a list of rows in HBase. All cells are deleted. + * + */ + --- End diff -- Done ---
[jira] [Commented] (NIFI-4863) If changing version of flow and new flow has no connection to RPG Port, can get error saying unable to start components
[ https://issues.apache.org/jira/browse/NIFI-4863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16358822#comment-16358822 ] ASF GitHub Bot commented on NIFI-4863: -- GitHub user markap14 opened a pull request: https://github.com/apache/nifi/pull/2462 NIFI-4863: Bug fixes to the way that we handled Remote Group Ports wh… …en changing flow version / reverting local changes Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/markap14/nifi NIFI-4863 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2462.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2462 commit ef64b56f215cac51a9935500fbf553816e0ffa92 Author: Mark Payne Date: 2018-02-09T19:09:23Z NIFI-4863: Bug fixes to the way that we handled Remote Group Ports when changing flow version / reverting local changes > If changing version of flow and new flow has no connection to RPG Port, can > get error saying unable to start components > --- > > Key: NIFI-4863 > URL: https://issues.apache.org/jira/browse/NIFI-4863 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.5.0 >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Major > > When we have a Remote Process Group with multiple ports, we push to the flow > registry the ports that we have connections for. I have a connection to Port > A and save that to the flow registry, then create a connection to Port B. I > have RPG transmitting data. Then I revert local changes. This properly > reverts the flow but fails to restart components. This is because it is > attempting to start Port B, and there are no connections for that Port. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi pull request #2462: NIFI-4863: Bug fixes to the way that we handled Rem...
GitHub user markap14 opened a pull request: https://github.com/apache/nifi/pull/2462 NIFI-4863: Bug fixes to the way that we handled Remote Group Ports wh⦠â¦en changing flow version / reverting local changes Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/markap14/nifi NIFI-4863 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2462.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2462 commit ef64b56f215cac51a9935500fbf553816e0ffa92 Author: Mark Payne Date: 2018-02-09T19:09:23Z NIFI-4863: Bug fixes to the way that we handled Remote Group Ports when changing flow version / reverting local changes ---
[jira] [Created] (MINIFICPP-398) Support schema config version 4
Aldrin Piri created MINIFICPP-398: - Summary: Support schema config version 4 Key: MINIFICPP-398 URL: https://issues.apache.org/jira/browse/MINIFICPP-398 Project: NiFi MiNiFi C++ Issue Type: Improvement Reporter: Aldrin Piri As part of MINIFI-66, and to help us carry out some initial C2 functionalities, a closer mapping of config schema to templates will allow us to more easily make use of registry functionality. This will require that instance based configuration (things like repositories, security and the like) are extracted to other files such as the conf and/or other .properties files. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (MINIFICPP-31) Support UpdateAttribute for nifi-minifi-cpp
[ https://issues.apache.org/jira/browse/MINIFICPP-31?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16358809#comment-16358809 ] ASF GitHub Bot commented on MINIFICPP-31: - GitHub user achristianson opened a pull request: https://github.com/apache/nifi-minifi-cpp/pull/262 MINIFICPP-31 Added UpdateAttribute processor Thank you for submitting a contribution to Apache NiFi - MiNiFi C++. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with MINIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [x] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [x] If applicable, have you updated the LICENSE file? - [x] If applicable, have you updated the NOTICE file? ### For documentation related changes: - [x] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/achristianson/nifi-minifi-cpp MINIFICPP-31 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi-minifi-cpp/pull/262.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #262 commit fb4e92d20c578299780fcf764fc9d0636af3259d Author: Andrew I. Christianson Date: 2018-02-08T18:51:08Z MINIFICPP-31 Added UpdateAttribute processor > Support UpdateAttribute for nifi-minifi-cpp > --- > > Key: MINIFICPP-31 > URL: https://issues.apache.org/jira/browse/MINIFICPP-31 > Project: NiFi MiNiFi C++ > Issue Type: New Feature >Reporter: Randy Gelhausen >Assignee: Andrew Christianson >Priority: Major > > nifi-minifi-cpp agents can generate multiple "streams" of flowfiles. > For instance, to monitor a host, nifi-minifi-cpp runs nmon, ps, netstat, and > gathers logfiles from applications. > But, for a given flowfile, any downstream NiFi collectors wont have > visibility into the originating hostname, nor metadata about which "stream" > (ExecuteProcess(nmon), ExecuteProcess(ps), TailFile(app1), TailFile(app2)) > generated it. > One solution is to use a separate InputPort for each stream. This works, but > burdens both the team working on agent flows, and the team managing the > collector- they have to be in concert. > A simpler (better?) approach is to allow agent teams to tag flowfiles with > differentiating metadata via use of UpdateAttribute. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi-minifi-cpp pull request #262: MINIFICPP-31 Added UpdateAttribute proces...
GitHub user achristianson opened a pull request: https://github.com/apache/nifi-minifi-cpp/pull/262 MINIFICPP-31 Added UpdateAttribute processor Thank you for submitting a contribution to Apache NiFi - MiNiFi C++. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with MINIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [x] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [x] If applicable, have you updated the LICENSE file? - [x] If applicable, have you updated the NOTICE file? ### For documentation related changes: - [x] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/achristianson/nifi-minifi-cpp MINIFICPP-31 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi-minifi-cpp/pull/262.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #262 commit fb4e92d20c578299780fcf764fc9d0636af3259d Author: Andrew I. Christianson Date: 2018-02-08T18:51:08Z MINIFICPP-31 Added UpdateAttribute processor ---
[jira] [Created] (MINIFICPP-397) Implement RouteOnAttribute
Andrew Christianson created MINIFICPP-397: - Summary: Implement RouteOnAttribute Key: MINIFICPP-397 URL: https://issues.apache.org/jira/browse/MINIFICPP-397 Project: NiFi MiNiFi C++ Issue Type: Improvement Reporter: Andrew Christianson Assignee: Andrew Christianson RouteOnAttribute is notably missing from MiNiFi - C++ and should be implemented. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-3538) Add DeleteHBase processor(s)
[
https://issues.apache.org/jira/browse/NIFI-3538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16358790#comment-16358790
]
ASF GitHub Bot commented on NIFI-3538:
--
Github user MikeThomsen commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2294#discussion_r167307609
--- Diff:
nifi-nar-bundles/nifi-hbase-bundle/nifi-hbase-processors/src/main/java/org/apache/nifi/hbase/DeleteHBaseRow.java
---
@@ -0,0 +1,183 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.hbase;
+
+import org.apache.nifi.annotation.behavior.WritesAttribute;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.components.AllowableValue;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.Validator;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.util.StandardValidators;
+
+import java.io.ByteArrayOutputStream;
+import java.util.ArrayList;
+import java.util.List;
+
+@WritesAttribute( attribute = "restart.index", description = "If a delete
batch fails, it will restart from restart.index" )
+@Tags({ "delete", "hbase" })
+@CapabilityDescription(
+"Delete HBase records individually or in batches. The input can be
a single row ID in the body, one ID per line, " +
+"row IDs separated by commas or a combination of the two. ")
+public class DeleteHBaseRow extends AbstractDeleteHBase {
+static final AllowableValue ROW_ID_BODY = new AllowableValue("body",
"FlowFile content", "Get the row key(s) from the flowfile content.");
+static final AllowableValue ROW_ID_ATTR = new AllowableValue("attr",
"FlowFile attributes", "Get the row key from an expression language
statement.");
+
+static final PropertyDescriptor ROW_ID_LOCATION = new
PropertyDescriptor.Builder()
+.name("delete-hb-row-id-location")
+.displayName("Row ID Location")
+.description("The location of the row ID to use for building
the delete. Can be from the content or an expression language statement.")
+.required(true)
+.defaultValue(ROW_ID_BODY.getValue())
+.allowableValues(ROW_ID_BODY, ROW_ID_ATTR)
+.addValidator(Validator.VALID)
+.build();
+
+static final PropertyDescriptor FLOWFILE_FETCH_COUNT = new
PropertyDescriptor.Builder()
+.name("delete-hb-flowfile-fetch-count")
+.displayName("Flowfile Fetch Count")
+.description("The number of flowfiles to fetch per run.")
+.required(true)
+.addValidator(StandardValidators.POSITIVE_INTEGER_VALIDATOR)
+.defaultValue("5")
+.expressionLanguageSupported(false)
+.build();
+
+static final PropertyDescriptor BATCH_SIZE = new
PropertyDescriptor.Builder()
+.name("delete-hb-row-ff-count")
+.displayName("Batch Size")
+.description("The number of deletes to send per batch.")
+.required(true)
+.defaultValue("50")
+.addValidator(StandardValidators.POSITIVE_INTEGER_VALIDATOR)
+.expressionLanguageSupported(false)
+.build();
+
+static final PropertyDescriptor KEY_SEPARATOR = new
PropertyDescriptor.Builder()
+.name("delete-hb-separator")
+.displayName("Delete Row Key Separator")
+.description("The separator character(s) that separate
multiple row keys " +
+"when multiple row keys are provided in the flowfile
body")
+.required(true)
[GitHub] nifi pull request #2294: NIFI-3538 Added DeleteHBaseRow
Github user MikeThomsen commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2294#discussion_r167307609
--- Diff:
nifi-nar-bundles/nifi-hbase-bundle/nifi-hbase-processors/src/main/java/org/apache/nifi/hbase/DeleteHBaseRow.java
---
@@ -0,0 +1,183 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.hbase;
+
+import org.apache.nifi.annotation.behavior.WritesAttribute;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.components.AllowableValue;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.Validator;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.util.StandardValidators;
+
+import java.io.ByteArrayOutputStream;
+import java.util.ArrayList;
+import java.util.List;
+
+@WritesAttribute( attribute = "restart.index", description = "If a delete
batch fails, it will restart from restart.index" )
+@Tags({ "delete", "hbase" })
+@CapabilityDescription(
+"Delete HBase records individually or in batches. The input can be
a single row ID in the body, one ID per line, " +
+"row IDs separated by commas or a combination of the two. ")
+public class DeleteHBaseRow extends AbstractDeleteHBase {
+static final AllowableValue ROW_ID_BODY = new AllowableValue("body",
"FlowFile content", "Get the row key(s) from the flowfile content.");
+static final AllowableValue ROW_ID_ATTR = new AllowableValue("attr",
"FlowFile attributes", "Get the row key from an expression language
statement.");
+
+static final PropertyDescriptor ROW_ID_LOCATION = new
PropertyDescriptor.Builder()
+.name("delete-hb-row-id-location")
+.displayName("Row ID Location")
+.description("The location of the row ID to use for building
the delete. Can be from the content or an expression language statement.")
+.required(true)
+.defaultValue(ROW_ID_BODY.getValue())
+.allowableValues(ROW_ID_BODY, ROW_ID_ATTR)
+.addValidator(Validator.VALID)
+.build();
+
+static final PropertyDescriptor FLOWFILE_FETCH_COUNT = new
PropertyDescriptor.Builder()
+.name("delete-hb-flowfile-fetch-count")
+.displayName("Flowfile Fetch Count")
+.description("The number of flowfiles to fetch per run.")
+.required(true)
+.addValidator(StandardValidators.POSITIVE_INTEGER_VALIDATOR)
+.defaultValue("5")
+.expressionLanguageSupported(false)
+.build();
+
+static final PropertyDescriptor BATCH_SIZE = new
PropertyDescriptor.Builder()
+.name("delete-hb-row-ff-count")
+.displayName("Batch Size")
+.description("The number of deletes to send per batch.")
+.required(true)
+.defaultValue("50")
+.addValidator(StandardValidators.POSITIVE_INTEGER_VALIDATOR)
+.expressionLanguageSupported(false)
+.build();
+
+static final PropertyDescriptor KEY_SEPARATOR = new
PropertyDescriptor.Builder()
+.name("delete-hb-separator")
+.displayName("Delete Row Key Separator")
+.description("The separator character(s) that separate
multiple row keys " +
+"when multiple row keys are provided in the flowfile
body")
+.required(true)
+.defaultValue(",")
+.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
+.expressionLanguageSupported(true)
+.build();
+static final PropertyDescriptor CHARSET = new
PropertyDescrip
[jira] [Created] (NIFI-4864) Additional Resources property pointing at a directory won't find new JARs
Bryan Bende created NIFI-4864: - Summary: Additional Resources property pointing at a directory won't find new JARs Key: NIFI-4864 URL: https://issues.apache.org/jira/browse/NIFI-4864 Project: Apache NiFi Issue Type: Bug Affects Versions: 1.5.0, 1.4.0, 1.3.0, 1.2.0 Reporter: Bryan Bende If you have a processor that has a property with dynamicallyModifiesClasspath(true) and you set the value to a directory, the resources in that directory will only be calculated when that property changes. This means if you added JARs to the directory later, and stopped and started your processor, those new JARs still won't be available. You would have to change the property to a new directory, or back and forth to some other directory, to force a recalculation. The setProperties method in AbstractConfiguredComponent is where it looks at incoming property changes and determines if any were for classpath related properties and then calls reload accordingly. We would need to consider the case where setProperties is never even being called, someone just stops and starts the processor and would want to pick up any new JARs added. A possible solution might be to computer some kind of hash/fingerprint of the URLs each time reload is called, and then when starting the processor we could recompute the fingerprint and compare it to the previous one. If they are different then we call reload before starting the component. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi pull request #2461: NIFI-4856 Removed deprecated ByteArrayInputStream r...
Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/2461 ---
[jira] [Commented] (NIFI-4856) ByteCountingInputStream should override available()
[
https://issues.apache.org/jira/browse/NIFI-4856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16358607#comment-16358607
]
ASF GitHub Bot commented on NIFI-4856:
--
Github user asfgit closed the pull request at:
https://github.com/apache/nifi/pull/2461
> ByteCountingInputStream should override available()
> ---
>
> Key: NIFI-4856
> URL: https://issues.apache.org/jira/browse/NIFI-4856
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.5.0
>Reporter: Brandon DeVries
>Assignee: Andy LoPresto
>Priority: Major
> Fix For: 1.6.0
>
>
> ByteCountingInputStream [1] does not override the InputStream.available()
> method as indicated in the InputStream javadocs [2]. In 0.x[3] there was
> another implementation of ByteCountingInputStream that correctly overrode
> available(). However, this was replaced with the incorrect version in 1.x.
>
> This causes any processor that attempts to use the available() method of the
> InputStream returned from ProcessSession.read() to erroneously believe it has
> reached the end of the stream, as this method is supposed to return "an
> estimate of the number of bytes that can be read (or skipped over) from this
> input stream without blocking *or {{0}} when it reaches the end* of the input
> stream."
>
> The following method needs to be added:
> {code:java}
> @Override
> public int available() throws IOException {
> return in.available();
> }{code}
>
> [1]
> [https://github.com/apache/nifi/blob/rel/nifi-1.5.0/nifi-commons/nifi-utils/src/main/java/org/apache/nifi/stream/io/ByteCountingInputStream.java]
> [2] [https://docs.oracle.com/javase/8/docs/api/index.html]
> [3]
> [https://github.com/apache/nifi/blob/0.x/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/controller/repository/io/ByteCountingInputStream.java]
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (NIFI-4856) ByteCountingInputStream should override available()
[
https://issues.apache.org/jira/browse/NIFI-4856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kevin Doran updated NIFI-4856:
--
Resolution: Fixed
Fix Version/s: 1.6.0
Status: Resolved (was: Patch Available)
> ByteCountingInputStream should override available()
> ---
>
> Key: NIFI-4856
> URL: https://issues.apache.org/jira/browse/NIFI-4856
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.5.0
>Reporter: Brandon DeVries
>Assignee: Andy LoPresto
>Priority: Major
> Fix For: 1.6.0
>
>
> ByteCountingInputStream [1] does not override the InputStream.available()
> method as indicated in the InputStream javadocs [2]. In 0.x[3] there was
> another implementation of ByteCountingInputStream that correctly overrode
> available(). However, this was replaced with the incorrect version in 1.x.
>
> This causes any processor that attempts to use the available() method of the
> InputStream returned from ProcessSession.read() to erroneously believe it has
> reached the end of the stream, as this method is supposed to return "an
> estimate of the number of bytes that can be read (or skipped over) from this
> input stream without blocking *or {{0}} when it reaches the end* of the input
> stream."
>
> The following method needs to be added:
> {code:java}
> @Override
> public int available() throws IOException {
> return in.available();
> }{code}
>
> [1]
> [https://github.com/apache/nifi/blob/rel/nifi-1.5.0/nifi-commons/nifi-utils/src/main/java/org/apache/nifi/stream/io/ByteCountingInputStream.java]
> [2] [https://docs.oracle.com/javase/8/docs/api/index.html]
> [3]
> [https://github.com/apache/nifi/blob/0.x/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/controller/repository/io/ByteCountingInputStream.java]
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFI-4856) ByteCountingInputStream should override available()
[
https://issues.apache.org/jira/browse/NIFI-4856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16358606#comment-16358606
]
ASF subversion and git services commented on NIFI-4856:
---
Commit 336d3cf1f2641bdf72e8f1847f16f01d4780b186 in nifi's branch
refs/heads/master from [~alopresto]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=336d3cf ]
NIFI-4856 Removed deprecated ByteArrayInputStream references in
ByteCountingInputStreamTest.
Added failing unit test for #available() at various states (initial, during
read, after read).
Implemented #available() delegation.
All tests pass.
This closes #2461.
Signed-off-by: Kevin Doran
> ByteCountingInputStream should override available()
> ---
>
> Key: NIFI-4856
> URL: https://issues.apache.org/jira/browse/NIFI-4856
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.5.0
>Reporter: Brandon DeVries
>Assignee: Andy LoPresto
>Priority: Major
>
> ByteCountingInputStream [1] does not override the InputStream.available()
> method as indicated in the InputStream javadocs [2]. In 0.x[3] there was
> another implementation of ByteCountingInputStream that correctly overrode
> available(). However, this was replaced with the incorrect version in 1.x.
>
> This causes any processor that attempts to use the available() method of the
> InputStream returned from ProcessSession.read() to erroneously believe it has
> reached the end of the stream, as this method is supposed to return "an
> estimate of the number of bytes that can be read (or skipped over) from this
> input stream without blocking *or {{0}} when it reaches the end* of the input
> stream."
>
> The following method needs to be added:
> {code:java}
> @Override
> public int available() throws IOException {
> return in.available();
> }{code}
>
> [1]
> [https://github.com/apache/nifi/blob/rel/nifi-1.5.0/nifi-commons/nifi-utils/src/main/java/org/apache/nifi/stream/io/ByteCountingInputStream.java]
> [2] [https://docs.oracle.com/javase/8/docs/api/index.html]
> [3]
> [https://github.com/apache/nifi/blob/0.x/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/controller/repository/io/ByteCountingInputStream.java]
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (MINIFICPP-118) Dynamic Properties support for processors
[ https://issues.apache.org/jira/browse/MINIFICPP-118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Christianson updated MINIFICPP-118: -- Resolution: Fixed Status: Resolved (was: Patch Available) > Dynamic Properties support for processors > - > > Key: MINIFICPP-118 > URL: https://issues.apache.org/jira/browse/MINIFICPP-118 > Project: NiFi MiNiFi C++ > Issue Type: Improvement >Affects Versions: 0.1.0 >Reporter: Jeremy Dyer >Assignee: Andrew Christianson >Priority: Major > > Currently any Property read from the config.yml file that is not explicitly > defined in the processor's implementation will be ignored by Processor.cpp > when reading the configurations. This prevents any dynamic property from > being defined in the config.yml and passed to the processor at runtime. > Certain processors rely heavily on the concept of dynamic properties that are > passed to them at runtime to handle things like setting dynamic properties > based on properties that are declared. All of these possibilities cannot be > handled upfront so there should be a mechanism, most likely in Processor.cpp, > that allows for a list of dynamicProperties that are parsed form the > config.yml file to be stored and accessed by the underlying processor > implementation at runtime and use them as the processor desires. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFIREG-126) Entering an invalid bucket id in a deep link causes JS error
[ https://issues.apache.org/jira/browse/NIFIREG-126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16358584#comment-16358584 ] ASF GitHub Bot commented on NIFIREG-126: Github user scottyaslan commented on the issue: https://github.com/apache/nifi-registry/pull/99 @kevdoran I have updated this PR with the new message as well as logging info. > Entering an invalid bucket id in a deep link causes JS error > > > Key: NIFIREG-126 > URL: https://issues.apache.org/jira/browse/NIFIREG-126 > Project: NiFi Registry > Issue Type: Bug >Affects Versions: 0.1.0 >Reporter: Scott Aslan >Assignee: Scott Aslan >Priority: Major > > As a user when I enter an invalid deep link I want to be routed to view all > the items in all the buckets that I am authorized to view and to be notified > that the requested bucket/item id is invalid. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFIREG-126) Entering an invalid bucket id in a deep link causes JS error
[ https://issues.apache.org/jira/browse/NIFIREG-126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16358585#comment-16358585 ] ASF GitHub Bot commented on NIFIREG-126: Github user kevdoran commented on the issue: https://github.com/apache/nifi-registry/pull/99 Thanks! Will review > Entering an invalid bucket id in a deep link causes JS error > > > Key: NIFIREG-126 > URL: https://issues.apache.org/jira/browse/NIFIREG-126 > Project: NiFi Registry > Issue Type: Bug >Affects Versions: 0.1.0 >Reporter: Scott Aslan >Assignee: Scott Aslan >Priority: Major > > As a user when I enter an invalid deep link I want to be routed to view all > the items in all the buckets that I am authorized to view and to be notified > that the requested bucket/item id is invalid. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi-registry issue #99: [NIFIREG-126] adding some polish and testing around...
Github user kevdoran commented on the issue: https://github.com/apache/nifi-registry/pull/99 Thanks! Will review ---
[GitHub] nifi-registry issue #99: [NIFIREG-126] adding some polish and testing around...
Github user scottyaslan commented on the issue: https://github.com/apache/nifi-registry/pull/99 @kevdoran I have updated this PR with the new message as well as logging info. ---
[jira] [Commented] (NIFIREG-133) [Documentation] Explain how variables are captured when versioning a PG
[ https://issues.apache.org/jira/browse/NIFIREG-133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16358583#comment-16358583 ] Andrew Lim commented on NIFIREG-133: Will likely document this in the NiFi User Guide. If so, will open a separate Jira in NiFi project. Am in the process of gathering use cases for a Flow Deployment Guide. Variables in versioned process groups would be a good section for that new document. > [Documentation] Explain how variables are captured when versioning a PG > --- > > Key: NIFIREG-133 > URL: https://issues.apache.org/jira/browse/NIFIREG-133 > Project: NiFi Registry > Issue Type: Improvement >Reporter: Pierre Villard >Assignee: Andrew Lim >Priority: Minor > > Context: > I have a process group PG that contains an embedded process group PG1. I have > a variable defined at PG level that is referenced in a PG1's processor. When > versioning both PG and PG1 in NiFi Registry, a copy of the variable defined > at PG level will be created at PG1 level. > Consequently, when importing PG in another environment, the variable needs to > be modified at PG1 level because the one created at PG1 level overwrites the > one initially defined at PG level. > > Would be nice to add this behavior in the documentation. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (NIFI-4862) Add Copy original attributtes to SelectHiveQL processer
[ https://issues.apache.org/jira/browse/NIFI-4862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jakub Leś updated NIFI-4862: Attachment: 0001-NIFI-4862-Add-Copy-original-attributtes-to-SelectHiv.patch > Add Copy original attributtes to SelectHiveQL processer > --- > > Key: NIFI-4862 > URL: https://issues.apache.org/jira/browse/NIFI-4862 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 1.5.0 >Reporter: Jakub Leś >Priority: Minor > Attachments: > 0001-NIFI-4862-Add-Copy-original-attributtes-to-SelectHiv.patch > > > Hi, > Please add "Copy original attributes" to processor SelectHiveQL. Thanks to > that we can use HttpRequest and HttpResponse to synchronize fetching query > result in webservice. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (NIFI-4862) Add Copy original attributtes to SelectHiveQL processer
[ https://issues.apache.org/jira/browse/NIFI-4862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jakub Leś updated NIFI-4862: Status: Patch Available (was: Open) This patch adds additional parameter "copy original attributtes" and gives ability to add to each flowfile original attributes. > Add Copy original attributtes to SelectHiveQL processer > --- > > Key: NIFI-4862 > URL: https://issues.apache.org/jira/browse/NIFI-4862 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 1.5.0 >Reporter: Jakub Leś >Priority: Minor > Attachments: > 0001-NIFI-4862-Add-Copy-original-attributtes-to-SelectHiv.patch > > > Hi, > Please add "Copy original attributes" to processor SelectHiveQL. Thanks to > that we can use HttpRequest and HttpResponse to synchronize fetching query > result in webservice. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (NIFI-4863) If changing version of flow and new flow has no connection to RPG Port, can get error saying unable to start components
Mark Payne created NIFI-4863: Summary: If changing version of flow and new flow has no connection to RPG Port, can get error saying unable to start components Key: NIFI-4863 URL: https://issues.apache.org/jira/browse/NIFI-4863 Project: Apache NiFi Issue Type: Bug Components: Core Framework Affects Versions: 1.5.0 Reporter: Mark Payne Assignee: Mark Payne When we have a Remote Process Group with multiple ports, we push to the flow registry the ports that we have connections for. I have a connection to Port A and save that to the flow registry, then create a connection to Port B. I have RPG transmitting data. Then I revert local changes. This properly reverts the flow but fails to restart components. This is because it is attempting to start Port B, and there are no connections for that Port. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (NIFIREG-142) First start problem: Error creating bean with name 'flywayInitializer'
[
https://issues.apache.org/jira/browse/NIFIREG-142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16358548#comment-16358548
]
Bryan Bende edited comment on NIFIREG-142 at 2/9/18 3:32 PM:
-
[~doaks80] sorry for the troubles here, can you try deleting the 'database'
directory and then starting again?
It should get created relative to the base directory for NiFi registry:
{code:java}
ls -1
LICENSE
NOTICE
README
bin
conf
database
docs
flow_storage
lib
logs
run
work{code}
I'm hoping that the H2 database is just in some bad state from the first failed
start, and by wiping it out it will start from scratch and hopefully work since
you have the JDK now.
was (Author: bende):
[~doaks80] sorry for the troubles here, can you try deleting the 'database'
directory in the directory and then starting again?
It should get created relative to the base directory for NiFi registry:
{code:java}
ls -1
LICENSE
NOTICE
README
bin
conf
database
docs
flow_storage
lib
logs
run
work{code}
I'm hoping that the H2 database is just in some bad state from the first failed
start, and by wiping it out it will start from scratch and hopefully work since
you have the JDK now.
> First start problem: Error creating bean with name 'flywayInitializer'
> --
>
> Key: NIFIREG-142
> URL: https://issues.apache.org/jira/browse/NIFIREG-142
> Project: NiFi Registry
> Issue Type: Bug
>Affects Versions: 0.1.0
> Environment: RHEL7
> openjdk version "1.8.0_161"
>Reporter: Daniel Oakley
>Priority: Major
> Attachments: nifi-registry-app.log, nifi-registry-bootstrap.log
>
>
> Downloaded 0.1.0 tarball and and tried to run on RHEL7. No changes to any
> config files.
> Error in log file was:
> java.lang.RuntimeException:
> org.springframework.beans.factory.BeanCreationException: Error creating bean
> with name 'flywayInitializer' defined in class path resource
> [org/springframework/boot/autoconfigure/flyway/FlywayAutoConfiguration$FlywayConfiguration.class]:
> Invocation of init method failed; nested exception is
> org.flywaydb.core.api.FlywayException: Validate failed: Detected failed
> migration to version 1.3 (DropBucketItemNameUniqueness)
> I could not find anything about "flyway" in the docs or config files...any
> hints on how to get around this problem?
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFI-2630) Allow PublishJMS processor to create TextMessages
[
https://issues.apache.org/jira/browse/NIFI-2630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16358556#comment-16358556
]
ASF GitHub Bot commented on NIFI-2630:
--
Github user mosermw commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2458#discussion_r167258538
--- Diff:
nifi-nar-bundles/nifi-jms-bundle/nifi-jms-processors/src/main/java/org/apache/nifi/jms/processors/PublishJMS.java
---
@@ -131,4 +143,10 @@ protected JMSPublisher
finishBuildingJmsWorker(CachingConnectionFactory connecti
session.read(flowFile, in -> StreamUtils.fillBuffer(in,
messageContent, true));
return messageContent;
}
+
+private String extractTextMessageBody(FlowFile flowFile,
ProcessSession session) {
+final StringWriter writer = new StringWriter();
+session.read(flowFile, in -> IOUtils.copy(in, writer,
Charset.defaultCharset()));
--- End diff --
And now that I look closer, I see there is already a precedent for this in
the code, such as in LogAttribute.
> Allow PublishJMS processor to create TextMessages
> -
>
> Key: NIFI-2630
> URL: https://issues.apache.org/jira/browse/NIFI-2630
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
>Affects Versions: 0.6.0
>Reporter: James Anderson
>Assignee: Michael Moser
>Priority: Minor
> Labels: patch
> Attachments:
> 0001-NIFI-2630-Allow-PublishJMS-processor-to-create-TextM.patch
>
>
> Create a new configuration option for PublishJMS that allows the processor to
> be configured to emit instances of TextMessages as well as BytesMessage.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] nifi pull request #2458: NIFI-2630 Allow PublishJMS to send TextMessages
Github user mosermw commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2458#discussion_r167258538
--- Diff:
nifi-nar-bundles/nifi-jms-bundle/nifi-jms-processors/src/main/java/org/apache/nifi/jms/processors/PublishJMS.java
---
@@ -131,4 +143,10 @@ protected JMSPublisher
finishBuildingJmsWorker(CachingConnectionFactory connecti
session.read(flowFile, in -> StreamUtils.fillBuffer(in,
messageContent, true));
return messageContent;
}
+
+private String extractTextMessageBody(FlowFile flowFile,
ProcessSession session) {
+final StringWriter writer = new StringWriter();
+session.read(flowFile, in -> IOUtils.copy(in, writer,
Charset.defaultCharset()));
--- End diff --
And now that I look closer, I see there is already a precedent for this in
the code, such as in LogAttribute.
---
[jira] [Updated] (NIFI-4841) NPE when reverting local modifications to a versioned process group
[
https://issues.apache.org/jira/browse/NIFI-4841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mark Payne updated NIFI-4841:
-
Resolution: Fixed
Fix Version/s: 1.6.0
Status: Resolved (was: Patch Available)
> NPE when reverting local modifications to a versioned process group
> ---
>
> Key: NIFI-4841
> URL: https://issues.apache.org/jira/browse/NIFI-4841
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.5.0
>Reporter: Charlie Meyer
>Assignee: Bryan Bende
>Priority: Major
> Fix For: 1.6.0
>
> Attachments: NIFI-4841.xml
>
>
> I created a process group via importing from the registry. I then made a few
> modifications including settings properties and connecting some components. I
> then attempted to revert my local changes so I could update the flow to a
> newer version. When reverting the local changes, NiFi threw a NPE with the
> following stack trace:
> {noformat}
> 2018-02-05 17:18:52,356 INFO [Version Control Update Thread-1]
> org.apache.nifi.web.api.VersionsResource Stopping 1 Processors
> 2018-02-05 17:18:52,477 ERROR [Version Control Update Thread-1]
> org.apache.nifi.web.api.VersionsResource Failed to update flow to new version
> java.lang.NullPointerException: null
> at
> org.apache.nifi.web.dao.impl.StandardProcessGroupDAO.scheduleComponents(StandardProcessGroupDAO.java:179)
> at
> org.apache.nifi.web.dao.impl.StandardProcessGroupDAO$$FastClassBySpringCGLIB$$10a99b47.invoke()
> at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
> at
> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:738)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
> at
> org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
> at
> org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:673)
> at
> org.apache.nifi.web.dao.impl.StandardProcessGroupDAO$$EnhancerBySpringCGLIB$$bc287b8b.scheduleComponents()
> at
> org.apache.nifi.web.StandardNiFiServiceFacade$3.update(StandardNiFiServiceFacade.java:981)
> at
> org.apache.nifi.web.revision.NaiveRevisionManager.updateRevision(NaiveRevisionManager.java:120)
> at
> org.apache.nifi.web.StandardNiFiServiceFacade.scheduleComponents(StandardNiFiServiceFacade.java:976)
> at
> org.apache.nifi.web.StandardNiFiServiceFacade$$FastClassBySpringCGLIB$$358780e0.invoke()
> at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
> at
> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:738)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
> at
> org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:85)
> at
> org.apache.nifi.web.NiFiServiceFacadeLock.proceedWithWriteLock(NiFiServiceFacadeLock.java:173)
> at
> org.apache.nifi.web.NiFiServiceFacadeLock.scheduleLock(NiFiServiceFacadeLock.java:102)
> at sun.reflect.GeneratedMethodAccessor557.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:629)
> at
> org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:618)
> at
> org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:70)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
> at
> org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
> at
> org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:673)
> at
> org.apache.nifi.web.StandardNiFiServiceFacade$$EnhancerBySpringCGLIB$$8a758fa4.scheduleComponents()
> at
> org.apache.nifi.web.util.LocalComponentLifecycle.stopComponents(LocalComponentLifecycle.java:125)
> at
> org.apache.nifi.web.util.LocalComponentLifecycle.scheduleComponents(LocalComponentLifecycle.java:66)
> at
> org.apache.nifi.web.api.VersionsResource.updateFlowVersion(VersionsResource.java:1365)
[jira] [Commented] (NIFI-4841) NPE when reverting local modifications to a versioned process group
[
https://issues.apache.org/jira/browse/NIFI-4841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16358550#comment-16358550
]
ASF subversion and git services commented on NIFI-4841:
---
Commit 34b678d30d25c930678b008a5db8a26a6a2938b3 in nifi's branch
refs/heads/master from [~bbende]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=34b678d ]
NIFI-4841 Fixing NPE when reverting local changes involving remote group ports.
This closes #2454.
Signed-off-by: Mark Payne
> NPE when reverting local modifications to a versioned process group
> ---
>
> Key: NIFI-4841
> URL: https://issues.apache.org/jira/browse/NIFI-4841
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.5.0
>Reporter: Charlie Meyer
>Assignee: Bryan Bende
>Priority: Major
> Attachments: NIFI-4841.xml
>
>
> I created a process group via importing from the registry. I then made a few
> modifications including settings properties and connecting some components. I
> then attempted to revert my local changes so I could update the flow to a
> newer version. When reverting the local changes, NiFi threw a NPE with the
> following stack trace:
> {noformat}
> 2018-02-05 17:18:52,356 INFO [Version Control Update Thread-1]
> org.apache.nifi.web.api.VersionsResource Stopping 1 Processors
> 2018-02-05 17:18:52,477 ERROR [Version Control Update Thread-1]
> org.apache.nifi.web.api.VersionsResource Failed to update flow to new version
> java.lang.NullPointerException: null
> at
> org.apache.nifi.web.dao.impl.StandardProcessGroupDAO.scheduleComponents(StandardProcessGroupDAO.java:179)
> at
> org.apache.nifi.web.dao.impl.StandardProcessGroupDAO$$FastClassBySpringCGLIB$$10a99b47.invoke()
> at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
> at
> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:738)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
> at
> org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
> at
> org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:673)
> at
> org.apache.nifi.web.dao.impl.StandardProcessGroupDAO$$EnhancerBySpringCGLIB$$bc287b8b.scheduleComponents()
> at
> org.apache.nifi.web.StandardNiFiServiceFacade$3.update(StandardNiFiServiceFacade.java:981)
> at
> org.apache.nifi.web.revision.NaiveRevisionManager.updateRevision(NaiveRevisionManager.java:120)
> at
> org.apache.nifi.web.StandardNiFiServiceFacade.scheduleComponents(StandardNiFiServiceFacade.java:976)
> at
> org.apache.nifi.web.StandardNiFiServiceFacade$$FastClassBySpringCGLIB$$358780e0.invoke()
> at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
> at
> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:738)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
> at
> org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:85)
> at
> org.apache.nifi.web.NiFiServiceFacadeLock.proceedWithWriteLock(NiFiServiceFacadeLock.java:173)
> at
> org.apache.nifi.web.NiFiServiceFacadeLock.scheduleLock(NiFiServiceFacadeLock.java:102)
> at sun.reflect.GeneratedMethodAccessor557.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:629)
> at
> org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:618)
> at
> org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:70)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
> at
> org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
> at
> org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:673)
> at
> org.apache.nifi.web.StandardNiFiServiceFacade$$EnhancerBySpringCGLIB$$8a758fa4.scheduleComponents()
> at
> org.apache.nifi.web.util.LocalComponentLifec
[jira] [Created] (NIFI-4862) Add Copy original attributtes to SelectHiveQL processer
Jakub Leś created NIFI-4862: --- Summary: Add Copy original attributtes to SelectHiveQL processer Key: NIFI-4862 URL: https://issues.apache.org/jira/browse/NIFI-4862 Project: Apache NiFi Issue Type: Improvement Components: Extensions Affects Versions: 1.5.0 Reporter: Jakub Leś Hi, Please add "Copy original attributes" to processor SelectHiveQL. Thanks to that we can use HttpRequest and HttpResponse to synchronize fetching query result in webservice. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4841) NPE when reverting local modifications to a versioned process group
[
https://issues.apache.org/jira/browse/NIFI-4841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16358551#comment-16358551
]
ASF GitHub Bot commented on NIFI-4841:
--
Github user asfgit closed the pull request at:
https://github.com/apache/nifi/pull/2454
> NPE when reverting local modifications to a versioned process group
> ---
>
> Key: NIFI-4841
> URL: https://issues.apache.org/jira/browse/NIFI-4841
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.5.0
>Reporter: Charlie Meyer
>Assignee: Bryan Bende
>Priority: Major
> Attachments: NIFI-4841.xml
>
>
> I created a process group via importing from the registry. I then made a few
> modifications including settings properties and connecting some components. I
> then attempted to revert my local changes so I could update the flow to a
> newer version. When reverting the local changes, NiFi threw a NPE with the
> following stack trace:
> {noformat}
> 2018-02-05 17:18:52,356 INFO [Version Control Update Thread-1]
> org.apache.nifi.web.api.VersionsResource Stopping 1 Processors
> 2018-02-05 17:18:52,477 ERROR [Version Control Update Thread-1]
> org.apache.nifi.web.api.VersionsResource Failed to update flow to new version
> java.lang.NullPointerException: null
> at
> org.apache.nifi.web.dao.impl.StandardProcessGroupDAO.scheduleComponents(StandardProcessGroupDAO.java:179)
> at
> org.apache.nifi.web.dao.impl.StandardProcessGroupDAO$$FastClassBySpringCGLIB$$10a99b47.invoke()
> at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
> at
> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:738)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
> at
> org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
> at
> org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:673)
> at
> org.apache.nifi.web.dao.impl.StandardProcessGroupDAO$$EnhancerBySpringCGLIB$$bc287b8b.scheduleComponents()
> at
> org.apache.nifi.web.StandardNiFiServiceFacade$3.update(StandardNiFiServiceFacade.java:981)
> at
> org.apache.nifi.web.revision.NaiveRevisionManager.updateRevision(NaiveRevisionManager.java:120)
> at
> org.apache.nifi.web.StandardNiFiServiceFacade.scheduleComponents(StandardNiFiServiceFacade.java:976)
> at
> org.apache.nifi.web.StandardNiFiServiceFacade$$FastClassBySpringCGLIB$$358780e0.invoke()
> at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
> at
> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:738)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
> at
> org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:85)
> at
> org.apache.nifi.web.NiFiServiceFacadeLock.proceedWithWriteLock(NiFiServiceFacadeLock.java:173)
> at
> org.apache.nifi.web.NiFiServiceFacadeLock.scheduleLock(NiFiServiceFacadeLock.java:102)
> at sun.reflect.GeneratedMethodAccessor557.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:629)
> at
> org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:618)
> at
> org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:70)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
> at
> org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
> at
> org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:673)
> at
> org.apache.nifi.web.StandardNiFiServiceFacade$$EnhancerBySpringCGLIB$$8a758fa4.scheduleComponents()
> at
> org.apache.nifi.web.util.LocalComponentLifecycle.stopComponents(LocalComponentLifecycle.java:125)
> at
> org.apache.nifi.web.util.LocalComponentLifecycle.scheduleComponents(LocalComponentLifecycle.java:66)
> at
> org.apache.nifi.web.api.VersionsResource.updateFlowVersion(
[GitHub] nifi pull request #2454: NIFI-4841 Fixing NPE when reverting local changes i...
Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/2454 ---
[jira] [Commented] (NIFIREG-142) First start problem: Error creating bean with name 'flywayInitializer'
[
https://issues.apache.org/jira/browse/NIFIREG-142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16358548#comment-16358548
]
Bryan Bende commented on NIFIREG-142:
-
[~doaks80] sorry for the troubles here, can you try deleting the 'database'
directory in the directory and then starting again?
It should get created relative to the base directory for NiFi registry:
{code:java}
ls -1
LICENSE
NOTICE
README
bin
conf
database
docs
flow_storage
lib
logs
run
work{code}
I'm hoping that the H2 database is just in some bad state from the first failed
start, and by wiping it out it will start from scratch and hopefully work since
you have the JDK now.
> First start problem: Error creating bean with name 'flywayInitializer'
> --
>
> Key: NIFIREG-142
> URL: https://issues.apache.org/jira/browse/NIFIREG-142
> Project: NiFi Registry
> Issue Type: Bug
>Affects Versions: 0.1.0
> Environment: RHEL7
> openjdk version "1.8.0_161"
>Reporter: Daniel Oakley
>Priority: Major
> Attachments: nifi-registry-app.log, nifi-registry-bootstrap.log
>
>
> Downloaded 0.1.0 tarball and and tried to run on RHEL7. No changes to any
> config files.
> Error in log file was:
> java.lang.RuntimeException:
> org.springframework.beans.factory.BeanCreationException: Error creating bean
> with name 'flywayInitializer' defined in class path resource
> [org/springframework/boot/autoconfigure/flyway/FlywayAutoConfiguration$FlywayConfiguration.class]:
> Invocation of init method failed; nested exception is
> org.flywaydb.core.api.FlywayException: Validate failed: Detected failed
> migration to version 1.3 (DropBucketItemNameUniqueness)
> I could not find anything about "flyway" in the docs or config files...any
> hints on how to get around this problem?
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFI-2630) Allow PublishJMS processor to create TextMessages
[
https://issues.apache.org/jira/browse/NIFI-2630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16358541#comment-16358541
]
ASF GitHub Bot commented on NIFI-2630:
--
Github user mosermw commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2458#discussion_r167255119
--- Diff:
nifi-nar-bundles/nifi-jms-bundle/nifi-jms-processors/src/main/java/org/apache/nifi/jms/processors/PublishJMS.java
---
@@ -131,4 +143,10 @@ protected JMSPublisher
finishBuildingJmsWorker(CachingConnectionFactory connecti
session.read(flowFile, in -> StreamUtils.fillBuffer(in,
messageContent, true));
return messageContent;
}
+
+private String extractTextMessageBody(FlowFile flowFile,
ProcessSession session) {
+final StringWriter writer = new StringWriter();
+session.read(flowFile, in -> IOUtils.copy(in, writer,
Charset.defaultCharset()));
--- End diff --
Oh, I didn't think about that approach! I like it. I'll make the changes.
> Allow PublishJMS processor to create TextMessages
> -
>
> Key: NIFI-2630
> URL: https://issues.apache.org/jira/browse/NIFI-2630
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
>Affects Versions: 0.6.0
>Reporter: James Anderson
>Assignee: Michael Moser
>Priority: Minor
> Labels: patch
> Attachments:
> 0001-NIFI-2630-Allow-PublishJMS-processor-to-create-TextM.patch
>
>
> Create a new configuration option for PublishJMS that allows the processor to
> be configured to emit instances of TextMessages as well as BytesMessage.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] nifi pull request #2458: NIFI-2630 Allow PublishJMS to send TextMessages
Github user mosermw commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2458#discussion_r167255119
--- Diff:
nifi-nar-bundles/nifi-jms-bundle/nifi-jms-processors/src/main/java/org/apache/nifi/jms/processors/PublishJMS.java
---
@@ -131,4 +143,10 @@ protected JMSPublisher
finishBuildingJmsWorker(CachingConnectionFactory connecti
session.read(flowFile, in -> StreamUtils.fillBuffer(in,
messageContent, true));
return messageContent;
}
+
+private String extractTextMessageBody(FlowFile flowFile,
ProcessSession session) {
+final StringWriter writer = new StringWriter();
+session.read(flowFile, in -> IOUtils.copy(in, writer,
Charset.defaultCharset()));
--- End diff --
Oh, I didn't think about that approach! I like it. I'll make the changes.
---
[jira] [Commented] (NIFI-2630) Allow PublishJMS processor to create TextMessages
[
https://issues.apache.org/jira/browse/NIFI-2630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16358513#comment-16358513
]
ASF GitHub Bot commented on NIFI-2630:
--
Github user devriesb commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2458#discussion_r167252827
--- Diff:
nifi-nar-bundles/nifi-jms-bundle/nifi-jms-processors/src/main/java/org/apache/nifi/jms/processors/PublishJMS.java
---
@@ -131,4 +143,10 @@ protected JMSPublisher
finishBuildingJmsWorker(CachingConnectionFactory connecti
session.read(flowFile, in -> StreamUtils.fillBuffer(in,
messageContent, true));
return messageContent;
}
+
+private String extractTextMessageBody(FlowFile flowFile,
ProcessSession session) {
+final StringWriter writer = new StringWriter();
+session.read(flowFile, in -> IOUtils.copy(in, writer,
Charset.defaultCharset()));
--- End diff --
i'm not sure if this is a helpful "middle ground" suggestion or not, but
you could add a charset property that itself defaults to
Charset.defaultCharset(). That way the out of the box behavior uses the
charset specified for the jvm, but can be overridden on a per processor basis
if needed.
> Allow PublishJMS processor to create TextMessages
> -
>
> Key: NIFI-2630
> URL: https://issues.apache.org/jira/browse/NIFI-2630
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
>Affects Versions: 0.6.0
>Reporter: James Anderson
>Assignee: Michael Moser
>Priority: Minor
> Labels: patch
> Attachments:
> 0001-NIFI-2630-Allow-PublishJMS-processor-to-create-TextM.patch
>
>
> Create a new configuration option for PublishJMS that allows the processor to
> be configured to emit instances of TextMessages as well as BytesMessage.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] nifi pull request #2458: NIFI-2630 Allow PublishJMS to send TextMessages
Github user devriesb commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2458#discussion_r167252827
--- Diff:
nifi-nar-bundles/nifi-jms-bundle/nifi-jms-processors/src/main/java/org/apache/nifi/jms/processors/PublishJMS.java
---
@@ -131,4 +143,10 @@ protected JMSPublisher
finishBuildingJmsWorker(CachingConnectionFactory connecti
session.read(flowFile, in -> StreamUtils.fillBuffer(in,
messageContent, true));
return messageContent;
}
+
+private String extractTextMessageBody(FlowFile flowFile,
ProcessSession session) {
+final StringWriter writer = new StringWriter();
+session.read(flowFile, in -> IOUtils.copy(in, writer,
Charset.defaultCharset()));
--- End diff --
i'm not sure if this is a helpful "middle ground" suggestion or not, but
you could add a charset property that itself defaults to
Charset.defaultCharset(). That way the out of the box behavior uses the
charset specified for the jvm, but can be overridden on a per processor basis
if needed.
---
[jira] [Commented] (NIFIREG-126) Entering an invalid bucket id in a deep link causes JS error
[
https://issues.apache.org/jira/browse/NIFIREG-126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16358502#comment-16358502
]
ASF GitHub Bot commented on NIFIREG-126:
Github user moranr commented on a diff in the pull request:
https://github.com/apache/nifi-registry/pull/99#discussion_r167249634
--- Diff:
nifi-registry-web-api/src/main/java/org/apache/nifi/registry/web/api/AccessPolicyResource.java
---
@@ -174,7 +174,7 @@ public Response getAccessPolicy(
final AccessPolicy accessPolicy =
authorizationService.getAccessPolicy(identifier);
if (accessPolicy == null) {
-throw new ResourceNotFoundException("No access policy found
with ID + " + identifier);
+throw new ResourceNotFoundException("The specified access
policy does not exist for this bucket.");
--- End diff --
I think that works, we could even add _"...in this registry."_ to end of
that. It's consistent with the other messages.
> Entering an invalid bucket id in a deep link causes JS error
>
>
> Key: NIFIREG-126
> URL: https://issues.apache.org/jira/browse/NIFIREG-126
> Project: NiFi Registry
> Issue Type: Bug
>Affects Versions: 0.1.0
>Reporter: Scott Aslan
>Assignee: Scott Aslan
>Priority: Major
>
> As a user when I enter an invalid deep link I want to be routed to view all
> the items in all the buckets that I am authorized to view and to be notified
> that the requested bucket/item id is invalid.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] nifi-registry pull request #99: [NIFIREG-126] adding some polish and testing...
Github user moranr commented on a diff in the pull request:
https://github.com/apache/nifi-registry/pull/99#discussion_r167249634
--- Diff:
nifi-registry-web-api/src/main/java/org/apache/nifi/registry/web/api/AccessPolicyResource.java
---
@@ -174,7 +174,7 @@ public Response getAccessPolicy(
final AccessPolicy accessPolicy =
authorizationService.getAccessPolicy(identifier);
if (accessPolicy == null) {
-throw new ResourceNotFoundException("No access policy found
with ID + " + identifier);
+throw new ResourceNotFoundException("The specified access
policy does not exist for this bucket.");
--- End diff --
I think that works, we could even add _"...in this registry."_ to end of
that. It's consistent with the other messages.
---
[jira] [Commented] (NIFIREG-126) Entering an invalid bucket id in a deep link causes JS error
[
https://issues.apache.org/jira/browse/NIFIREG-126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16358490#comment-16358490
]
ASF GitHub Bot commented on NIFIREG-126:
Github user scottyaslan commented on a diff in the pull request:
https://github.com/apache/nifi-registry/pull/99#discussion_r167246736
--- Diff:
nifi-registry-web-api/src/main/java/org/apache/nifi/registry/web/api/AccessPolicyResource.java
---
@@ -174,7 +174,7 @@ public Response getAccessPolicy(
final AccessPolicy accessPolicy =
authorizationService.getAccessPolicy(identifier);
if (accessPolicy == null) {
-throw new ResourceNotFoundException("No access policy found
with ID + " + identifier);
+throw new ResourceNotFoundException("The specified access
policy does not exist for this bucket.");
--- End diff --
Good catchI think we can simply change it to say "The specified policy
does not exist."?
> Entering an invalid bucket id in a deep link causes JS error
>
>
> Key: NIFIREG-126
> URL: https://issues.apache.org/jira/browse/NIFIREG-126
> Project: NiFi Registry
> Issue Type: Bug
>Affects Versions: 0.1.0
>Reporter: Scott Aslan
>Assignee: Scott Aslan
>Priority: Major
>
> As a user when I enter an invalid deep link I want to be routed to view all
> the items in all the buckets that I am authorized to view and to be notified
> that the requested bucket/item id is invalid.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] nifi-registry pull request #99: [NIFIREG-126] adding some polish and testing...
Github user scottyaslan commented on a diff in the pull request:
https://github.com/apache/nifi-registry/pull/99#discussion_r167246736
--- Diff:
nifi-registry-web-api/src/main/java/org/apache/nifi/registry/web/api/AccessPolicyResource.java
---
@@ -174,7 +174,7 @@ public Response getAccessPolicy(
final AccessPolicy accessPolicy =
authorizationService.getAccessPolicy(identifier);
if (accessPolicy == null) {
-throw new ResourceNotFoundException("No access policy found
with ID + " + identifier);
+throw new ResourceNotFoundException("The specified access
policy does not exist for this bucket.");
--- End diff --
Good catchI think we can simply change it to say "The specified policy
does not exist."?
---
[jira] [Commented] (NIFI-2630) Allow PublishJMS processor to create TextMessages
[
https://issues.apache.org/jira/browse/NIFI-2630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16358487#comment-16358487
]
ASF GitHub Bot commented on NIFI-2630:
--
Github user anderson900 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2458#discussion_r167245934
--- Diff:
nifi-nar-bundles/nifi-jms-bundle/nifi-jms-processors/src/main/java/org/apache/nifi/jms/processors/PublishJMS.java
---
@@ -131,4 +143,10 @@ protected JMSPublisher
finishBuildingJmsWorker(CachingConnectionFactory connecti
session.read(flowFile, in -> StreamUtils.fillBuffer(in,
messageContent, true));
return messageContent;
}
+
+private String extractTextMessageBody(FlowFile flowFile,
ProcessSession session) {
+final StringWriter writer = new StringWriter();
+session.read(flowFile, in -> IOUtils.copy(in, writer,
Charset.defaultCharset()));
--- End diff --
Shouldn't we rely on the charset specified for the JVM in boostrap.conf,
which would become the default charset?
> Allow PublishJMS processor to create TextMessages
> -
>
> Key: NIFI-2630
> URL: https://issues.apache.org/jira/browse/NIFI-2630
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
>Affects Versions: 0.6.0
>Reporter: James Anderson
>Assignee: Michael Moser
>Priority: Minor
> Labels: patch
> Attachments:
> 0001-NIFI-2630-Allow-PublishJMS-processor-to-create-TextM.patch
>
>
> Create a new configuration option for PublishJMS that allows the processor to
> be configured to emit instances of TextMessages as well as BytesMessage.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] nifi pull request #2458: NIFI-2630 Allow PublishJMS to send TextMessages
Github user anderson900 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2458#discussion_r167245934
--- Diff:
nifi-nar-bundles/nifi-jms-bundle/nifi-jms-processors/src/main/java/org/apache/nifi/jms/processors/PublishJMS.java
---
@@ -131,4 +143,10 @@ protected JMSPublisher
finishBuildingJmsWorker(CachingConnectionFactory connecti
session.read(flowFile, in -> StreamUtils.fillBuffer(in,
messageContent, true));
return messageContent;
}
+
+private String extractTextMessageBody(FlowFile flowFile,
ProcessSession session) {
+final StringWriter writer = new StringWriter();
+session.read(flowFile, in -> IOUtils.copy(in, writer,
Charset.defaultCharset()));
--- End diff --
Shouldn't we rely on the charset specified for the JVM in boostrap.conf,
which would become the default charset?
---
[jira] [Commented] (NIFI-2630) Allow PublishJMS processor to create TextMessages
[
https://issues.apache.org/jira/browse/NIFI-2630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16358476#comment-16358476
]
ASF GitHub Bot commented on NIFI-2630:
--
Github user mosermw commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2458#discussion_r167244022
--- Diff:
nifi-nar-bundles/nifi-jms-bundle/nifi-jms-processors/src/main/java/org/apache/nifi/jms/processors/PublishJMS.java
---
@@ -131,4 +143,10 @@ protected JMSPublisher
finishBuildingJmsWorker(CachingConnectionFactory connecti
session.read(flowFile, in -> StreamUtils.fillBuffer(in,
messageContent, true));
return messageContent;
}
+
+private String extractTextMessageBody(FlowFile flowFile,
ProcessSession session) {
+final StringWriter writer = new StringWriter();
+session.read(flowFile, in -> IOUtils.copy(in, writer,
Charset.defaultCharset()));
--- End diff --
I did think about this, and I chose this approach on purpose. It's kind of
the whole point of setting a default Charset in your JVM, to produce results
that are specific to your desired environment. Is this not right?
> Allow PublishJMS processor to create TextMessages
> -
>
> Key: NIFI-2630
> URL: https://issues.apache.org/jira/browse/NIFI-2630
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
>Affects Versions: 0.6.0
>Reporter: James Anderson
>Assignee: Michael Moser
>Priority: Minor
> Labels: patch
> Attachments:
> 0001-NIFI-2630-Allow-PublishJMS-processor-to-create-TextM.patch
>
>
> Create a new configuration option for PublishJMS that allows the processor to
> be configured to emit instances of TextMessages as well as BytesMessage.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] nifi pull request #2458: NIFI-2630 Allow PublishJMS to send TextMessages
Github user mosermw commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2458#discussion_r167244022
--- Diff:
nifi-nar-bundles/nifi-jms-bundle/nifi-jms-processors/src/main/java/org/apache/nifi/jms/processors/PublishJMS.java
---
@@ -131,4 +143,10 @@ protected JMSPublisher
finishBuildingJmsWorker(CachingConnectionFactory connecti
session.read(flowFile, in -> StreamUtils.fillBuffer(in,
messageContent, true));
return messageContent;
}
+
+private String extractTextMessageBody(FlowFile flowFile,
ProcessSession session) {
+final StringWriter writer = new StringWriter();
+session.read(flowFile, in -> IOUtils.copy(in, writer,
Charset.defaultCharset()));
--- End diff --
I did think about this, and I chose this approach on purpose. It's kind of
the whole point of setting a default Charset in your JVM, to produce results
that are specific to your desired environment. Is this not right?
---
[jira] [Updated] (NIFI-4853) PutMongoRecord doesn't handle nested records
[ https://issues.apache.org/jira/browse/NIFI-4853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-4853: - Resolution: Fixed Fix Version/s: 1.6.0 Status: Resolved (was: Patch Available) > PutMongoRecord doesn't handle nested records > > > Key: NIFI-4853 > URL: https://issues.apache.org/jira/browse/NIFI-4853 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Reporter: Matt Burgess >Assignee: Matt Burgess >Priority: Major > Fix For: 1.6.0 > > > PutMongoRecord works well with flat records, but when attempting to put in > nested records (records whose fields are arrays or records themselves), the > Mongo serializer doesn't know how to handle the complex NiFi record field > types (such as MapRecord). > The fix is to traverse all fields in the record, transforming the fields (if > necessary) to Java objects for use by the Mongo serializer(s). Something very > similar was done for PutDruidRecord, and in fact there is a utility method > DataTypeUtils.convertRecordFieldtoObject() that can perform this task. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-4853) PutMongoRecord doesn't handle nested records
[ https://issues.apache.org/jira/browse/NIFI-4853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16358464#comment-16358464 ] ASF GitHub Bot commented on NIFI-4853: -- Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/2457 > PutMongoRecord doesn't handle nested records > > > Key: NIFI-4853 > URL: https://issues.apache.org/jira/browse/NIFI-4853 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Reporter: Matt Burgess >Assignee: Matt Burgess >Priority: Major > > PutMongoRecord works well with flat records, but when attempting to put in > nested records (records whose fields are arrays or records themselves), the > Mongo serializer doesn't know how to handle the complex NiFi record field > types (such as MapRecord). > The fix is to traverse all fields in the record, transforming the fields (if > necessary) to Java objects for use by the Mongo serializer(s). Something very > similar was done for PutDruidRecord, and in fact there is a utility method > DataTypeUtils.convertRecordFieldtoObject() that can perform this task. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi pull request #2457: NIFI-4853: Fixed PutMongoRecord handling of nested ...
Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/2457 ---
[jira] [Commented] (NIFI-4853) PutMongoRecord doesn't handle nested records
[ https://issues.apache.org/jira/browse/NIFI-4853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16358463#comment-16358463 ] ASF subversion and git services commented on NIFI-4853: --- Commit 25e0bbb68dfb26ee49216c53a481280c776cce45 in nifi's branch refs/heads/master from [~ca9mbu] [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=25e0bbb ] NIFI-4853 - Fixed PutMongoRecord handling of nested records Signed-off-by: Pierre Villard This closes #2457. > PutMongoRecord doesn't handle nested records > > > Key: NIFI-4853 > URL: https://issues.apache.org/jira/browse/NIFI-4853 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Reporter: Matt Burgess >Assignee: Matt Burgess >Priority: Major > > PutMongoRecord works well with flat records, but when attempting to put in > nested records (records whose fields are arrays or records themselves), the > Mongo serializer doesn't know how to handle the complex NiFi record field > types (such as MapRecord). > The fix is to traverse all fields in the record, transforming the fields (if > necessary) to Java objects for use by the Mongo serializer(s). Something very > similar was done for PutDruidRecord, and in fact there is a utility method > DataTypeUtils.convertRecordFieldtoObject() that can perform this task. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
