[jira] [Commented] (NIFI-5777) Update the tag and the property of LogMessage
[
https://issues.apache.org/jira/browse/NIFI-5777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16673770#comment-16673770
]
ASF GitHub Bot commented on NIFI-5777:
--
Github user ijokarumawak commented on a diff in the pull request:
https://github.com/apache/nifi/pull/3122#discussion_r230526767
--- Diff:
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/test/java/org/apache/nifi/processors/standard/TestLogMessage.java
---
@@ -57,7 +57,7 @@ MockComponentLog getMockComponentLog() {
public void before() throws InitializationException {
testableLogMessage = new TestableLogMessage();
runner = TestRunners.newTestRunner(testableLogMessage);
-
+runner.setValidateExpressionUsage(false);
--- End diff --
Sorry for the confusion. I was just looking at the source code. When I run
the processor, I realized that the processor doesn't allow EL. The logLevel is
shown as a free text input, and I can input EL, but validation failed because
of the configured allowable values.
I still think changing log level by EL would be nice:
My suggestion is, changing:
```
.description("The Log Level to use when logging the message")
.allowableValues(MessageLogLevel.values())
```
to
```
.description("The Log Level to use when logging the message: "
+ Arrays.toString(MessageLogLevel.values()))
.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
```
How do you guys think?
> Update the tag and the property of LogMessage
> -
>
> Key: NIFI-5777
> URL: https://issues.apache.org/jira/browse/NIFI-5777
> Project: Apache NiFi
> Issue Type: Improvement
>Affects Versions: 1.8.0
>Reporter: Kotaro Terada
>Assignee: Kotaro Terada
>Priority: Major
>
> There are a few points to update in {{LogMessage}}:
> * The processor tags are a little bit strange. The current tags are
> "attributes" and "logging". A tag "attributes" is not suitable for this
> processor. I suggest just "logging" is enough.
> * The property "Log Level" should be selected using a drop-down list (as it
> is done in {{LogAttribute}}). Currently, the field is just a text box, and
> users need to type a log level manually. If we set "expression language
> supported" on the property, does it force to make the property become a text
> field in the Web UI?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] nifi pull request #3122: NIFI-5777: Update the tag and the property of LogMe...
Github user ijokarumawak commented on a diff in the pull request:
https://github.com/apache/nifi/pull/3122#discussion_r230526767
--- Diff:
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/test/java/org/apache/nifi/processors/standard/TestLogMessage.java
---
@@ -57,7 +57,7 @@ MockComponentLog getMockComponentLog() {
public void before() throws InitializationException {
testableLogMessage = new TestableLogMessage();
runner = TestRunners.newTestRunner(testableLogMessage);
-
+runner.setValidateExpressionUsage(false);
--- End diff --
Sorry for the confusion. I was just looking at the source code. When I run
the processor, I realized that the processor doesn't allow EL. The logLevel is
shown as a free text input, and I can input EL, but validation failed because
of the configured allowable values.
I still think changing log level by EL would be nice:
My suggestion is, changing:
```
.description("The Log Level to use when logging the message")
.allowableValues(MessageLogLevel.values())
```
to
```
.description("The Log Level to use when logging the message: "
+ Arrays.toString(MessageLogLevel.values()))
.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
```
How do you guys think?
---
[jira] [Commented] (NIFI-5677) Add/clarify why modifying/creating variables are not considered local changes in versioned flows
[ https://issues.apache.org/jira/browse/NIFI-5677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16673438#comment-16673438 ] ASF GitHub Bot commented on NIFI-5677: -- GitHub user andrewmlim opened a pull request: https://github.com/apache/nifi/pull/3125 NIFI-5677 Added note to clarify why modifying/creating variables not … …local changes in versioned flows You can merge this pull request into a Git repository by running: $ git pull https://github.com/andrewmlim/nifi NIFI-5677 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/3125.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3125 > Add/clarify why modifying/creating variables are not considered local changes > in versioned flows > > > Key: NIFI-5677 > URL: https://issues.apache.org/jira/browse/NIFI-5677 > Project: Apache NiFi > Issue Type: Improvement > Components: Documentation Website >Reporter: Andrew Lim >Assignee: Andrew Lim >Priority: Minor > > There has been some confusion over why creating or modifying variables in a > versioned flow do not trigger local changes in the flow. > Will improve the relevant section in the User Guide > (https://nifi.apache.org/docs/nifi-docs/html/user-guide.html#managing_local_changes) > with the following clarifications: > Modifying doesn’t trigger local changes because variable values are intended > to be different in each environment. When a flow is imported to an > environment, it is assumed there is a one-time operation required to set > those variables specific for the given environment. > Creating a variable doesn’t trigger a local change because just creating a > variable on its own has not changed anything about what the flow processes. > A component will have to be created/modified that uses the new variable, > which will trigger a local change. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (MINIFICPP-652) Add monitored threads to stacks protocol
[ https://issues.apache.org/jira/browse/MINIFICPP-652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aldrin Piri updated MINIFICPP-652: -- Fix Version/s: 0.6.0 > Add monitored threads to stacks protocol > > > Key: MINIFICPP-652 > URL: https://issues.apache.org/jira/browse/MINIFICPP-652 > Project: NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Mr TheSegfault >Assignee: Mr TheSegfault >Priority: Major > Fix For: 0.6.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (MINIFICPP-652) Add monitored threads to stacks protocol
[ https://issues.apache.org/jira/browse/MINIFICPP-652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aldrin Piri resolved MINIFICPP-652. --- Resolution: Fixed > Add monitored threads to stacks protocol > > > Key: MINIFICPP-652 > URL: https://issues.apache.org/jira/browse/MINIFICPP-652 > Project: NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Mr TheSegfault >Assignee: Mr TheSegfault >Priority: Major > Fix For: 0.6.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (MINIFICPP-652) Add monitored threads to stacks protocol
[ https://issues.apache.org/jira/browse/MINIFICPP-652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16673375#comment-16673375 ] ASF GitHub Bot commented on MINIFICPP-652: -- Github user asfgit closed the pull request at: https://github.com/apache/nifi-minifi-cpp/pull/428 > Add monitored threads to stacks protocol > > > Key: MINIFICPP-652 > URL: https://issues.apache.org/jira/browse/MINIFICPP-652 > Project: NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Mr TheSegfault >Assignee: Mr TheSegfault >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi-minifi-cpp pull request #428: MINIFICPP-652: Add traceability to reposi...
Github user asfgit closed the pull request at: https://github.com/apache/nifi-minifi-cpp/pull/428 ---
[jira] [Assigned] (NIFI-5677) Add/clarify why modifying/creating variables are not considered local changes in versioned flows
[ https://issues.apache.org/jira/browse/NIFI-5677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Lim reassigned NIFI-5677: Assignee: Andrew Lim > Add/clarify why modifying/creating variables are not considered local changes > in versioned flows > > > Key: NIFI-5677 > URL: https://issues.apache.org/jira/browse/NIFI-5677 > Project: Apache NiFi > Issue Type: Improvement > Components: Documentation Website >Reporter: Andrew Lim >Assignee: Andrew Lim >Priority: Minor > > There has been some confusion over why creating or modifying variables in a > versioned flow do not trigger local changes in the flow. > Will improve the relevant section in the User Guide > (https://nifi.apache.org/docs/nifi-docs/html/user-guide.html#managing_local_changes) > with the following clarifications: > Modifying doesn’t trigger local changes because variable values are intended > to be different in each environment. When a flow is imported to an > environment, it is assumed there is a one-time operation required to set > those variables specific for the given environment. > Creating a variable doesn’t trigger a local change because just creating a > variable on its own has not changed anything about what the flow processes. > A component will have to be created/modified that uses the new variable, > which will trigger a local change. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (NIFIREG-210) Document-links are incorrect in documentation.jsp
[ https://issues.apache.org/jira/browse/NIFIREG-210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Aslan resolved NIFIREG-210. - Resolution: Fixed Fix Version/s: 0.4.0 > Document-links are incorrect in documentation.jsp > - > > Key: NIFIREG-210 > URL: https://issues.apache.org/jira/browse/NIFIREG-210 > Project: NiFi Registry > Issue Type: Improvement >Reporter: Andrew Lim >Assignee: Andrew Lim >Priority: Trivial > Fix For: 0.4.0 > > > All of the document-link references are to "admin-guide", even for the > Getting Started and User Guides. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFIREG-210) Document-links are incorrect in documentation.jsp
[ https://issues.apache.org/jira/browse/NIFIREG-210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16673259#comment-16673259 ] ASF GitHub Bot commented on NIFIREG-210: Github user asfgit closed the pull request at: https://github.com/apache/nifi-registry/pull/145 > Document-links are incorrect in documentation.jsp > - > > Key: NIFIREG-210 > URL: https://issues.apache.org/jira/browse/NIFIREG-210 > Project: NiFi Registry > Issue Type: Improvement >Reporter: Andrew Lim >Assignee: Andrew Lim >Priority: Trivial > > All of the document-link references are to "admin-guide", even for the > Getting Started and User Guides. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi-registry pull request #145: NIFIREG-210 Corrected incorrect document-li...
Github user asfgit closed the pull request at: https://github.com/apache/nifi-registry/pull/145 ---
[jira] [Commented] (NIFIREG-210) Document-links are incorrect in documentation.jsp
[ https://issues.apache.org/jira/browse/NIFIREG-210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16673146#comment-16673146 ] ASF GitHub Bot commented on NIFIREG-210: Github user scottyaslan commented on the issue: https://github.com/apache/nifi-registry/pull/145 Reviewing... > Document-links are incorrect in documentation.jsp > - > > Key: NIFIREG-210 > URL: https://issues.apache.org/jira/browse/NIFIREG-210 > Project: NiFi Registry > Issue Type: Improvement >Reporter: Andrew Lim >Assignee: Andrew Lim >Priority: Trivial > > All of the document-link references are to "admin-guide", even for the > Getting Started and User Guides. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi-registry issue #145: NIFIREG-210 Corrected incorrect document-links
Github user scottyaslan commented on the issue: https://github.com/apache/nifi-registry/pull/145 Reviewing... ---
[jira] [Assigned] (NIFI-5784) With the addition of the Toolkit Guide, edit other docs that contain duplicate content
[ https://issues.apache.org/jira/browse/NIFI-5784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Lim reassigned NIFI-5784: Assignee: Andrew Lim > With the addition of the Toolkit Guide, edit other docs that contain > duplicate content > -- > > Key: NIFI-5784 > URL: https://issues.apache.org/jira/browse/NIFI-5784 > Project: Apache NiFi > Issue Type: Improvement > Components: Documentation Website >Reporter: Andrew Lim >Assignee: Andrew Lim >Priority: Minor > > The Admin Guide has duplicate content for the following: > * Configuration encryption - > [https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#encrypt-config_tool] > * File manager - > [https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#file-manager] > * Node manager - > [https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#node-manager] > * TLS Toolkit - > [https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#tls_generation_toolkit] > * ZooKeeper migrator - > [https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#zookeeper_migrator] > Will remove and add links to Toolkit Guide as needed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (NIFI-5784) With the addition of the Toolkit Guide, edit other docs that contain duplicate content
Andrew Lim created NIFI-5784: Summary: With the addition of the Toolkit Guide, edit other docs that contain duplicate content Key: NIFI-5784 URL: https://issues.apache.org/jira/browse/NIFI-5784 Project: Apache NiFi Issue Type: Improvement Components: Documentation Website Reporter: Andrew Lim The Admin Guide has duplicate content for the following: * Configuration encryption - [https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#encrypt-config_tool] * File manager - [https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#file-manager] * Node manager - [https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#node-manager] * TLS Toolkit - [https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#tls_generation_toolkit] * ZooKeeper migrator - [https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#zookeeper_migrator] Will remove and add links to Toolkit Guide as needed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] nifi issue #3124: NIFI-5767 Added NiFi Toolkit Guide to docs
Github user andrewmlim commented on the issue: https://github.com/apache/nifi/pull/3124 I will file a Jira to edit the other docs that have duplicate content. The other docs can refer to the Toolkit Guide as needed. ---
[GitHub] nifi pull request #3124: NIFI-5767 Added NiFi Toolkit Guide to docs
Github user andrewmlim commented on a diff in the pull request: https://github.com/apache/nifi/pull/3124#discussion_r230381765 --- Diff: nifi-docs/src/main/asciidoc/toolkit-guide.adoc --- @@ -0,0 +1,1257 @@ +// +// Licensed to the Apache Software Foundation (ASF) under one or more +// contributor license agreements. See the NOTICE file distributed with +// this work for additional information regarding copyright ownership. +// The ASF licenses this file to You under the Apache License, Version 2.0 +// (the "License"); you may not use this file except in compliance with +// the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. +// += Apache NiFi Toolkit Guide +Apache NiFi Team +:homepage: http://nifi.apache.org +:linkattrs: + +== Overview +The NiFi Toolkit contains several command line utilities to setup and support NiFi in standalone and clustered environments. The utilities include: + +* CLI -- The `cli` tool enables administrators to interact with NiFi and NiFi Registry instances to automate tasks such as deploying versioned flows and managing process groups and cluster nodes. +* Encrypt Config -- The `encrypt-config` tool encrypts the sensitive keys in the _nifi.properties_ file to facilitate the setup of a secure NiFi instance. +* File Manager -- The `file-manager` tool enables administrators to backup, install or restore a NiFi installation from backup. +* Flow Analyzer -- The `flow-analyzer` tool produces a report that helps administrators understand the max amount of data which can be stored in backpressure for a given flow. +* Node Manager -- The `node-manager` tool enables administrators to perform status checks on nodes as well as the ability to connect, disconnect, or remove nodes from the cluster. +* Notify -- The `notify` tool enables administrators to send bulletins to the NiFi UI. +* S2S -- The `s2s` tool enables administrators to send data into or out of NiFi flows over site-to-site. +* TLS Toolkit -- The `tls-toolkit` utility generates the required keystores, truststore, and relevant configuration files to facilitate the setup of a secure NiFi instance. +* ZooKeeper Migrator -- The `zk-migrator` tool enables administrators to: +** move ZooKeeper information from one ZooKeeper cluster to another +** migrate ZooKeeper node ownership + +The utilities are executed with scripts found in the `bin` folder of your NiFi Toolkit installation. + +NOTE: The NiFi Toolkit is downloaded separately from NiFi (see the link:https://nifi.apache.org/download.html[Apache NiFi downloads page^]). + +=== Prerequisites for Running in a Secure Environment +For secured nodes and clusters, two policies should be configured in advance: + +* Access the controller â A user that will have access to these utilities should be authorized in NiFi by creating an âaccess the controllerâ policy (`/controller`) with both view and modify rights +* Proxy user request â If not previously set, nodeâs identity (the DN value of the nodeâs certificate) should be authorized to proxy requests on behalf of a user + +When executing either the Notify or Node Manager tools in a secured environment the `proxyDN` flag option should be used in order to properly identify the user that was authorized to execute these commands. In non-secure environments, or if running the status operation on the Node Manager tool, the flag is ignored. + +== NiFi CLI +This tool offers a CLI focused on interacting with NiFi and NiFi Registry in order to automate tasks, such as deploying flows from a NIFi Registy to a NiFi instance or managing process groups and cluster nodes. + +=== Usage +The CLI toolkit can be executed in standalone mode to execute a single command, or interactive mode to enter an interactive shell. + +To execute a single command: + + ./bin/cli.sh + +To launch the interactive shell: + + ./bin/cli.sh + +To show help: + + cli.sh -h + +The following are available options: + + demo quick-import + nifi current-user + nifi cluster-summary + nifi connect-node + nifi delete-node + nifi disconnect-node + nifi get-root-id + nifi get-node + nifi get-nodes + nifi offload-node + nifi list-reg-clients + nifi create-reg-client + nifi update-reg-client + nifi get-reg-client-id + nifi pg-import +
[jira] [Commented] (NIFI-5767) Documentation of the NiFi Toolkit
[ https://issues.apache.org/jira/browse/NIFI-5767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16673133#comment-16673133 ] ASF GitHub Bot commented on NIFI-5767: -- Github user andrewmlim commented on a diff in the pull request: https://github.com/apache/nifi/pull/3124#discussion_r230381765 --- Diff: nifi-docs/src/main/asciidoc/toolkit-guide.adoc --- @@ -0,0 +1,1257 @@ +// +// Licensed to the Apache Software Foundation (ASF) under one or more +// contributor license agreements. See the NOTICE file distributed with +// this work for additional information regarding copyright ownership. +// The ASF licenses this file to You under the Apache License, Version 2.0 +// (the "License"); you may not use this file except in compliance with +// the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. +// += Apache NiFi Toolkit Guide +Apache NiFi Team +:homepage: http://nifi.apache.org +:linkattrs: + +== Overview +The NiFi Toolkit contains several command line utilities to setup and support NiFi in standalone and clustered environments. The utilities include: + +* CLI -- The `cli` tool enables administrators to interact with NiFi and NiFi Registry instances to automate tasks such as deploying versioned flows and managing process groups and cluster nodes. +* Encrypt Config -- The `encrypt-config` tool encrypts the sensitive keys in the _nifi.properties_ file to facilitate the setup of a secure NiFi instance. +* File Manager -- The `file-manager` tool enables administrators to backup, install or restore a NiFi installation from backup. +* Flow Analyzer -- The `flow-analyzer` tool produces a report that helps administrators understand the max amount of data which can be stored in backpressure for a given flow. +* Node Manager -- The `node-manager` tool enables administrators to perform status checks on nodes as well as the ability to connect, disconnect, or remove nodes from the cluster. +* Notify -- The `notify` tool enables administrators to send bulletins to the NiFi UI. +* S2S -- The `s2s` tool enables administrators to send data into or out of NiFi flows over site-to-site. +* TLS Toolkit -- The `tls-toolkit` utility generates the required keystores, truststore, and relevant configuration files to facilitate the setup of a secure NiFi instance. +* ZooKeeper Migrator -- The `zk-migrator` tool enables administrators to: +** move ZooKeeper information from one ZooKeeper cluster to another +** migrate ZooKeeper node ownership + +The utilities are executed with scripts found in the `bin` folder of your NiFi Toolkit installation. + +NOTE: The NiFi Toolkit is downloaded separately from NiFi (see the link:https://nifi.apache.org/download.html[Apache NiFi downloads page^]). + +=== Prerequisites for Running in a Secure Environment +For secured nodes and clusters, two policies should be configured in advance: + +* Access the controller – A user that will have access to these utilities should be authorized in NiFi by creating an “access the controller” policy (`/controller`) with both view and modify rights +* Proxy user request – If not previously set, node’s identity (the DN value of the node’s certificate) should be authorized to proxy requests on behalf of a user + +When executing either the Notify or Node Manager tools in a secured environment the `proxyDN` flag option should be used in order to properly identify the user that was authorized to execute these commands. In non-secure environments, or if running the status operation on the Node Manager tool, the flag is ignored. + +== NiFi CLI +This tool offers a CLI focused on interacting with NiFi and NiFi Registry in order to automate tasks, such as deploying flows from a NIFi Registy to a NiFi instance or managing process groups and cluster nodes. + +=== Usage +The CLI toolkit can be executed in standalone mode to execute a single command, or interactive mode to enter an interactive shell. + +To execute a single command: + + ./bin/cli.sh + +To launch the interactive shell: + + ./bin/cli.sh + +To show help: + + cli.sh -h + +The following are available options: + + demo quick-import + nifi current-user + nifi cluster-summary + nifi connect-node + nifi delete-node + nifi disconnect-node
[jira] [Commented] (NIFI-5767) Documentation of the NiFi Toolkit
[ https://issues.apache.org/jira/browse/NIFI-5767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16673132#comment-16673132 ] ASF GitHub Bot commented on NIFI-5767: -- Github user andrewmlim commented on a diff in the pull request: https://github.com/apache/nifi/pull/3124#discussion_r230381451 --- Diff: nifi-docs/src/main/asciidoc/toolkit-guide.adoc --- @@ -0,0 +1,1257 @@ +// +// Licensed to the Apache Software Foundation (ASF) under one or more +// contributor license agreements. See the NOTICE file distributed with +// this work for additional information regarding copyright ownership. +// The ASF licenses this file to You under the Apache License, Version 2.0 +// (the "License"); you may not use this file except in compliance with +// the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. +// += Apache NiFi Toolkit Guide +Apache NiFi Team +:homepage: http://nifi.apache.org +:linkattrs: + +== Overview +The NiFi Toolkit contains several command line utilities to setup and support NiFi in standalone and clustered environments. The utilities include: + +* CLI -- The `cli` tool enables administrators to interact with NiFi and NiFi Registry instances to automate tasks such as deploying versioned flows and managing process groups and cluster nodes. +* Encrypt Config -- The `encrypt-config` tool encrypts the sensitive keys in the _nifi.properties_ file to facilitate the setup of a secure NiFi instance. +* File Manager -- The `file-manager` tool enables administrators to backup, install or restore a NiFi installation from backup. +* Flow Analyzer -- The `flow-analyzer` tool produces a report that helps administrators understand the max amount of data which can be stored in backpressure for a given flow. +* Node Manager -- The `node-manager` tool enables administrators to perform status checks on nodes as well as the ability to connect, disconnect, or remove nodes from the cluster. +* Notify -- The `notify` tool enables administrators to send bulletins to the NiFi UI. +* S2S -- The `s2s` tool enables administrators to send data into or out of NiFi flows over site-to-site. +* TLS Toolkit -- The `tls-toolkit` utility generates the required keystores, truststore, and relevant configuration files to facilitate the setup of a secure NiFi instance. +* ZooKeeper Migrator -- The `zk-migrator` tool enables administrators to: +** move ZooKeeper information from one ZooKeeper cluster to another +** migrate ZooKeeper node ownership + +The utilities are executed with scripts found in the `bin` folder of your NiFi Toolkit installation. + +NOTE: The NiFi Toolkit is downloaded separately from NiFi (see the link:https://nifi.apache.org/download.html[Apache NiFi downloads page^]). + +=== Prerequisites for Running in a Secure Environment +For secured nodes and clusters, two policies should be configured in advance: + +* Access the controller – A user that will have access to these utilities should be authorized in NiFi by creating an “access the controller” policy (`/controller`) with both view and modify rights +* Proxy user request – If not previously set, node’s identity (the DN value of the node’s certificate) should be authorized to proxy requests on behalf of a user + +When executing either the Notify or Node Manager tools in a secured environment the `proxyDN` flag option should be used in order to properly identify the user that was authorized to execute these commands. In non-secure environments, or if running the status operation on the Node Manager tool, the flag is ignored. + +== NiFi CLI +This tool offers a CLI focused on interacting with NiFi and NiFi Registry in order to automate tasks, such as deploying flows from a NIFi Registy to a NiFi instance or managing process groups and cluster nodes. + +=== Usage +The CLI toolkit can be executed in standalone mode to execute a single command, or interactive mode to enter an interactive shell. + +To execute a single command: + + ./bin/cli.sh + +To launch the interactive shell: + + ./bin/cli.sh + +To show help: + + cli.sh -h --- End diff -- Do you think we should preface every help command example in the doc with "./bin/"? I can make those changes. > Documentation of the NiFi Toolkit >
[GitHub] nifi pull request #3124: NIFI-5767 Added NiFi Toolkit Guide to docs
Github user andrewmlim commented on a diff in the pull request: https://github.com/apache/nifi/pull/3124#discussion_r230381451 --- Diff: nifi-docs/src/main/asciidoc/toolkit-guide.adoc --- @@ -0,0 +1,1257 @@ +// +// Licensed to the Apache Software Foundation (ASF) under one or more +// contributor license agreements. See the NOTICE file distributed with +// this work for additional information regarding copyright ownership. +// The ASF licenses this file to You under the Apache License, Version 2.0 +// (the "License"); you may not use this file except in compliance with +// the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. +// += Apache NiFi Toolkit Guide +Apache NiFi Team +:homepage: http://nifi.apache.org +:linkattrs: + +== Overview +The NiFi Toolkit contains several command line utilities to setup and support NiFi in standalone and clustered environments. The utilities include: + +* CLI -- The `cli` tool enables administrators to interact with NiFi and NiFi Registry instances to automate tasks such as deploying versioned flows and managing process groups and cluster nodes. +* Encrypt Config -- The `encrypt-config` tool encrypts the sensitive keys in the _nifi.properties_ file to facilitate the setup of a secure NiFi instance. +* File Manager -- The `file-manager` tool enables administrators to backup, install or restore a NiFi installation from backup. +* Flow Analyzer -- The `flow-analyzer` tool produces a report that helps administrators understand the max amount of data which can be stored in backpressure for a given flow. +* Node Manager -- The `node-manager` tool enables administrators to perform status checks on nodes as well as the ability to connect, disconnect, or remove nodes from the cluster. +* Notify -- The `notify` tool enables administrators to send bulletins to the NiFi UI. +* S2S -- The `s2s` tool enables administrators to send data into or out of NiFi flows over site-to-site. +* TLS Toolkit -- The `tls-toolkit` utility generates the required keystores, truststore, and relevant configuration files to facilitate the setup of a secure NiFi instance. +* ZooKeeper Migrator -- The `zk-migrator` tool enables administrators to: +** move ZooKeeper information from one ZooKeeper cluster to another +** migrate ZooKeeper node ownership + +The utilities are executed with scripts found in the `bin` folder of your NiFi Toolkit installation. + +NOTE: The NiFi Toolkit is downloaded separately from NiFi (see the link:https://nifi.apache.org/download.html[Apache NiFi downloads page^]). + +=== Prerequisites for Running in a Secure Environment +For secured nodes and clusters, two policies should be configured in advance: + +* Access the controller â A user that will have access to these utilities should be authorized in NiFi by creating an âaccess the controllerâ policy (`/controller`) with both view and modify rights +* Proxy user request â If not previously set, nodeâs identity (the DN value of the nodeâs certificate) should be authorized to proxy requests on behalf of a user + +When executing either the Notify or Node Manager tools in a secured environment the `proxyDN` flag option should be used in order to properly identify the user that was authorized to execute these commands. In non-secure environments, or if running the status operation on the Node Manager tool, the flag is ignored. + +== NiFi CLI +This tool offers a CLI focused on interacting with NiFi and NiFi Registry in order to automate tasks, such as deploying flows from a NIFi Registy to a NiFi instance or managing process groups and cluster nodes. + +=== Usage +The CLI toolkit can be executed in standalone mode to execute a single command, or interactive mode to enter an interactive shell. + +To execute a single command: + + ./bin/cli.sh + +To launch the interactive shell: + + ./bin/cli.sh + +To show help: + + cli.sh -h --- End diff -- Do you think we should preface every help command example in the doc with "./bin/"? I can make those changes. ---
[jira] [Created] (NIFI-5783) Deploying a process group with inner process group from registry fails in recreating the connections
sujeeth created NIFI-5783: - Summary: Deploying a process group with inner process group from registry fails in recreating the connections Key: NIFI-5783 URL: https://issues.apache.org/jira/browse/NIFI-5783 Project: Apache NiFi Issue Type: Bug Components: Configuration Management, SDLC Affects Versions: 1.6.0 Environment: software platform - registry and nifi integration Reporter: sujeeth Attachments: p1.jpg, p2.jpg i deploy a nifi flow dev environment. Check in the flow in registry from dev. I import the flow in SIT environment. There will be multiple self connections in the processor in SIT environment which were not there in dev env. I,e The flow though it was checked in to registry properly [ image p1 ]the checked out version to a new environment does not match [ image p2 ] the checked in environment. this happens only with the subprocess group connections. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (NIFI-5767) Documentation of the NiFi Toolkit
[ https://issues.apache.org/jira/browse/NIFI-5767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16672868#comment-16672868 ] ASF GitHub Bot commented on NIFI-5767: -- Github user pvillard31 commented on a diff in the pull request: https://github.com/apache/nifi/pull/3124#discussion_r230324180 --- Diff: nifi-docs/src/main/asciidoc/toolkit-guide.adoc --- @@ -0,0 +1,1257 @@ +// +// Licensed to the Apache Software Foundation (ASF) under one or more +// contributor license agreements. See the NOTICE file distributed with +// this work for additional information regarding copyright ownership. +// The ASF licenses this file to You under the Apache License, Version 2.0 +// (the "License"); you may not use this file except in compliance with +// the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. +// += Apache NiFi Toolkit Guide +Apache NiFi Team +:homepage: http://nifi.apache.org +:linkattrs: + +== Overview +The NiFi Toolkit contains several command line utilities to setup and support NiFi in standalone and clustered environments. The utilities include: + +* CLI -- The `cli` tool enables administrators to interact with NiFi and NiFi Registry instances to automate tasks such as deploying versioned flows and managing process groups and cluster nodes. +* Encrypt Config -- The `encrypt-config` tool encrypts the sensitive keys in the _nifi.properties_ file to facilitate the setup of a secure NiFi instance. +* File Manager -- The `file-manager` tool enables administrators to backup, install or restore a NiFi installation from backup. +* Flow Analyzer -- The `flow-analyzer` tool produces a report that helps administrators understand the max amount of data which can be stored in backpressure for a given flow. +* Node Manager -- The `node-manager` tool enables administrators to perform status checks on nodes as well as the ability to connect, disconnect, or remove nodes from the cluster. +* Notify -- The `notify` tool enables administrators to send bulletins to the NiFi UI. +* S2S -- The `s2s` tool enables administrators to send data into or out of NiFi flows over site-to-site. +* TLS Toolkit -- The `tls-toolkit` utility generates the required keystores, truststore, and relevant configuration files to facilitate the setup of a secure NiFi instance. +* ZooKeeper Migrator -- The `zk-migrator` tool enables administrators to: +** move ZooKeeper information from one ZooKeeper cluster to another +** migrate ZooKeeper node ownership + +The utilities are executed with scripts found in the `bin` folder of your NiFi Toolkit installation. + +NOTE: The NiFi Toolkit is downloaded separately from NiFi (see the link:https://nifi.apache.org/download.html[Apache NiFi downloads page^]). + +=== Prerequisites for Running in a Secure Environment +For secured nodes and clusters, two policies should be configured in advance: + +* Access the controller – A user that will have access to these utilities should be authorized in NiFi by creating an “access the controller” policy (`/controller`) with both view and modify rights +* Proxy user request – If not previously set, node’s identity (the DN value of the node’s certificate) should be authorized to proxy requests on behalf of a user + +When executing either the Notify or Node Manager tools in a secured environment the `proxyDN` flag option should be used in order to properly identify the user that was authorized to execute these commands. In non-secure environments, or if running the status operation on the Node Manager tool, the flag is ignored. + +== NiFi CLI +This tool offers a CLI focused on interacting with NiFi and NiFi Registry in order to automate tasks, such as deploying flows from a NIFi Registy to a NiFi instance or managing process groups and cluster nodes. + +=== Usage +The CLI toolkit can be executed in standalone mode to execute a single command, or interactive mode to enter an interactive shell. + +To execute a single command: + + ./bin/cli.sh + +To launch the interactive shell: + + ./bin/cli.sh + +To show help: + + cli.sh -h + +The following are available options: + + demo quick-import + nifi current-user + nifi cluster-summary + nifi connect-node + nifi delete-node + nifi disconnect-node
[GitHub] nifi pull request #3124: NIFI-5767 Added NiFi Toolkit Guide to docs
Github user pvillard31 commented on a diff in the pull request: https://github.com/apache/nifi/pull/3124#discussion_r230325510 --- Diff: nifi-docs/src/main/asciidoc/toolkit-guide.adoc --- @@ -0,0 +1,1257 @@ +// +// Licensed to the Apache Software Foundation (ASF) under one or more +// contributor license agreements. See the NOTICE file distributed with +// this work for additional information regarding copyright ownership. +// The ASF licenses this file to You under the Apache License, Version 2.0 +// (the "License"); you may not use this file except in compliance with +// the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. +// += Apache NiFi Toolkit Guide +Apache NiFi Team +:homepage: http://nifi.apache.org +:linkattrs: + +== Overview +The NiFi Toolkit contains several command line utilities to setup and support NiFi in standalone and clustered environments. The utilities include: + +* CLI -- The `cli` tool enables administrators to interact with NiFi and NiFi Registry instances to automate tasks such as deploying versioned flows and managing process groups and cluster nodes. +* Encrypt Config -- The `encrypt-config` tool encrypts the sensitive keys in the _nifi.properties_ file to facilitate the setup of a secure NiFi instance. +* File Manager -- The `file-manager` tool enables administrators to backup, install or restore a NiFi installation from backup. +* Flow Analyzer -- The `flow-analyzer` tool produces a report that helps administrators understand the max amount of data which can be stored in backpressure for a given flow. +* Node Manager -- The `node-manager` tool enables administrators to perform status checks on nodes as well as the ability to connect, disconnect, or remove nodes from the cluster. +* Notify -- The `notify` tool enables administrators to send bulletins to the NiFi UI. +* S2S -- The `s2s` tool enables administrators to send data into or out of NiFi flows over site-to-site. +* TLS Toolkit -- The `tls-toolkit` utility generates the required keystores, truststore, and relevant configuration files to facilitate the setup of a secure NiFi instance. +* ZooKeeper Migrator -- The `zk-migrator` tool enables administrators to: +** move ZooKeeper information from one ZooKeeper cluster to another +** migrate ZooKeeper node ownership + +The utilities are executed with scripts found in the `bin` folder of your NiFi Toolkit installation. + +NOTE: The NiFi Toolkit is downloaded separately from NiFi (see the link:https://nifi.apache.org/download.html[Apache NiFi downloads page^]). + +=== Prerequisites for Running in a Secure Environment +For secured nodes and clusters, two policies should be configured in advance: + +* Access the controller â A user that will have access to these utilities should be authorized in NiFi by creating an âaccess the controllerâ policy (`/controller`) with both view and modify rights +* Proxy user request â If not previously set, nodeâs identity (the DN value of the nodeâs certificate) should be authorized to proxy requests on behalf of a user + +When executing either the Notify or Node Manager tools in a secured environment the `proxyDN` flag option should be used in order to properly identify the user that was authorized to execute these commands. In non-secure environments, or if running the status operation on the Node Manager tool, the flag is ignored. + +== NiFi CLI +This tool offers a CLI focused on interacting with NiFi and NiFi Registry in order to automate tasks, such as deploying flows from a NIFi Registy to a NiFi instance or managing process groups and cluster nodes. + +=== Usage +The CLI toolkit can be executed in standalone mode to execute a single command, or interactive mode to enter an interactive shell. + +To execute a single command: + + ./bin/cli.sh + +To launch the interactive shell: + + ./bin/cli.sh + +To show help: + + cli.sh -h + +The following are available options: + + demo quick-import + nifi current-user + nifi cluster-summary + nifi connect-node + nifi delete-node + nifi disconnect-node + nifi get-root-id + nifi get-node + nifi get-nodes + nifi offload-node + nifi list-reg-clients + nifi create-reg-client + nifi update-reg-client + nifi get-reg-client-id + nifi pg-import +
[jira] [Commented] (NIFI-5767) Documentation of the NiFi Toolkit
[ https://issues.apache.org/jira/browse/NIFI-5767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16672869#comment-16672869 ] ASF GitHub Bot commented on NIFI-5767: -- Github user pvillard31 commented on a diff in the pull request: https://github.com/apache/nifi/pull/3124#discussion_r230322139 --- Diff: nifi-docs/src/main/asciidoc/toolkit-guide.adoc --- @@ -0,0 +1,1257 @@ +// +// Licensed to the Apache Software Foundation (ASF) under one or more +// contributor license agreements. See the NOTICE file distributed with +// this work for additional information regarding copyright ownership. +// The ASF licenses this file to You under the Apache License, Version 2.0 +// (the "License"); you may not use this file except in compliance with +// the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. +// += Apache NiFi Toolkit Guide +Apache NiFi Team +:homepage: http://nifi.apache.org +:linkattrs: + +== Overview +The NiFi Toolkit contains several command line utilities to setup and support NiFi in standalone and clustered environments. The utilities include: + +* CLI -- The `cli` tool enables administrators to interact with NiFi and NiFi Registry instances to automate tasks such as deploying versioned flows and managing process groups and cluster nodes. +* Encrypt Config -- The `encrypt-config` tool encrypts the sensitive keys in the _nifi.properties_ file to facilitate the setup of a secure NiFi instance. +* File Manager -- The `file-manager` tool enables administrators to backup, install or restore a NiFi installation from backup. +* Flow Analyzer -- The `flow-analyzer` tool produces a report that helps administrators understand the max amount of data which can be stored in backpressure for a given flow. +* Node Manager -- The `node-manager` tool enables administrators to perform status checks on nodes as well as the ability to connect, disconnect, or remove nodes from the cluster. +* Notify -- The `notify` tool enables administrators to send bulletins to the NiFi UI. +* S2S -- The `s2s` tool enables administrators to send data into or out of NiFi flows over site-to-site. +* TLS Toolkit -- The `tls-toolkit` utility generates the required keystores, truststore, and relevant configuration files to facilitate the setup of a secure NiFi instance. +* ZooKeeper Migrator -- The `zk-migrator` tool enables administrators to: +** move ZooKeeper information from one ZooKeeper cluster to another +** migrate ZooKeeper node ownership + +The utilities are executed with scripts found in the `bin` folder of your NiFi Toolkit installation. + +NOTE: The NiFi Toolkit is downloaded separately from NiFi (see the link:https://nifi.apache.org/download.html[Apache NiFi downloads page^]). + +=== Prerequisites for Running in a Secure Environment +For secured nodes and clusters, two policies should be configured in advance: + +* Access the controller – A user that will have access to these utilities should be authorized in NiFi by creating an “access the controller” policy (`/controller`) with both view and modify rights +* Proxy user request – If not previously set, node’s identity (the DN value of the node’s certificate) should be authorized to proxy requests on behalf of a user + +When executing either the Notify or Node Manager tools in a secured environment the `proxyDN` flag option should be used in order to properly identify the user that was authorized to execute these commands. In non-secure environments, or if running the status operation on the Node Manager tool, the flag is ignored. + +== NiFi CLI +This tool offers a CLI focused on interacting with NiFi and NiFi Registry in order to automate tasks, such as deploying flows from a NIFi Registy to a NiFi instance or managing process groups and cluster nodes. + +=== Usage +The CLI toolkit can be executed in standalone mode to execute a single command, or interactive mode to enter an interactive shell. + +To execute a single command: + + ./bin/cli.sh + +To launch the interactive shell: + + ./bin/cli.sh + +To show help: + + cli.sh -h --- End diff -- ```suggestion ./bin/cli.sh -h ``` > Documentation of the NiFi Toolkit > - > > Key: NIFI-5767 > URL:
[jira] [Commented] (NIFI-5767) Documentation of the NiFi Toolkit
[ https://issues.apache.org/jira/browse/NIFI-5767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16672870#comment-16672870 ] ASF GitHub Bot commented on NIFI-5767: -- Github user pvillard31 commented on a diff in the pull request: https://github.com/apache/nifi/pull/3124#discussion_r230325510 --- Diff: nifi-docs/src/main/asciidoc/toolkit-guide.adoc --- @@ -0,0 +1,1257 @@ +// +// Licensed to the Apache Software Foundation (ASF) under one or more +// contributor license agreements. See the NOTICE file distributed with +// this work for additional information regarding copyright ownership. +// The ASF licenses this file to You under the Apache License, Version 2.0 +// (the "License"); you may not use this file except in compliance with +// the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. +// += Apache NiFi Toolkit Guide +Apache NiFi Team +:homepage: http://nifi.apache.org +:linkattrs: + +== Overview +The NiFi Toolkit contains several command line utilities to setup and support NiFi in standalone and clustered environments. The utilities include: + +* CLI -- The `cli` tool enables administrators to interact with NiFi and NiFi Registry instances to automate tasks such as deploying versioned flows and managing process groups and cluster nodes. +* Encrypt Config -- The `encrypt-config` tool encrypts the sensitive keys in the _nifi.properties_ file to facilitate the setup of a secure NiFi instance. +* File Manager -- The `file-manager` tool enables administrators to backup, install or restore a NiFi installation from backup. +* Flow Analyzer -- The `flow-analyzer` tool produces a report that helps administrators understand the max amount of data which can be stored in backpressure for a given flow. +* Node Manager -- The `node-manager` tool enables administrators to perform status checks on nodes as well as the ability to connect, disconnect, or remove nodes from the cluster. +* Notify -- The `notify` tool enables administrators to send bulletins to the NiFi UI. +* S2S -- The `s2s` tool enables administrators to send data into or out of NiFi flows over site-to-site. +* TLS Toolkit -- The `tls-toolkit` utility generates the required keystores, truststore, and relevant configuration files to facilitate the setup of a secure NiFi instance. +* ZooKeeper Migrator -- The `zk-migrator` tool enables administrators to: +** move ZooKeeper information from one ZooKeeper cluster to another +** migrate ZooKeeper node ownership + +The utilities are executed with scripts found in the `bin` folder of your NiFi Toolkit installation. + +NOTE: The NiFi Toolkit is downloaded separately from NiFi (see the link:https://nifi.apache.org/download.html[Apache NiFi downloads page^]). + +=== Prerequisites for Running in a Secure Environment +For secured nodes and clusters, two policies should be configured in advance: + +* Access the controller – A user that will have access to these utilities should be authorized in NiFi by creating an “access the controller” policy (`/controller`) with both view and modify rights +* Proxy user request – If not previously set, node’s identity (the DN value of the node’s certificate) should be authorized to proxy requests on behalf of a user + +When executing either the Notify or Node Manager tools in a secured environment the `proxyDN` flag option should be used in order to properly identify the user that was authorized to execute these commands. In non-secure environments, or if running the status operation on the Node Manager tool, the flag is ignored. + +== NiFi CLI +This tool offers a CLI focused on interacting with NiFi and NiFi Registry in order to automate tasks, such as deploying flows from a NIFi Registy to a NiFi instance or managing process groups and cluster nodes. + +=== Usage +The CLI toolkit can be executed in standalone mode to execute a single command, or interactive mode to enter an interactive shell. + +To execute a single command: + + ./bin/cli.sh + +To launch the interactive shell: + + ./bin/cli.sh + +To show help: + + cli.sh -h + +The following are available options: + + demo quick-import + nifi current-user + nifi cluster-summary + nifi connect-node + nifi delete-node + nifi disconnect-node
[GitHub] nifi pull request #3124: NIFI-5767 Added NiFi Toolkit Guide to docs
Github user pvillard31 commented on a diff in the pull request: https://github.com/apache/nifi/pull/3124#discussion_r230322139 --- Diff: nifi-docs/src/main/asciidoc/toolkit-guide.adoc --- @@ -0,0 +1,1257 @@ +// +// Licensed to the Apache Software Foundation (ASF) under one or more +// contributor license agreements. See the NOTICE file distributed with +// this work for additional information regarding copyright ownership. +// The ASF licenses this file to You under the Apache License, Version 2.0 +// (the "License"); you may not use this file except in compliance with +// the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. +// += Apache NiFi Toolkit Guide +Apache NiFi Team +:homepage: http://nifi.apache.org +:linkattrs: + +== Overview +The NiFi Toolkit contains several command line utilities to setup and support NiFi in standalone and clustered environments. The utilities include: + +* CLI -- The `cli` tool enables administrators to interact with NiFi and NiFi Registry instances to automate tasks such as deploying versioned flows and managing process groups and cluster nodes. +* Encrypt Config -- The `encrypt-config` tool encrypts the sensitive keys in the _nifi.properties_ file to facilitate the setup of a secure NiFi instance. +* File Manager -- The `file-manager` tool enables administrators to backup, install or restore a NiFi installation from backup. +* Flow Analyzer -- The `flow-analyzer` tool produces a report that helps administrators understand the max amount of data which can be stored in backpressure for a given flow. +* Node Manager -- The `node-manager` tool enables administrators to perform status checks on nodes as well as the ability to connect, disconnect, or remove nodes from the cluster. +* Notify -- The `notify` tool enables administrators to send bulletins to the NiFi UI. +* S2S -- The `s2s` tool enables administrators to send data into or out of NiFi flows over site-to-site. +* TLS Toolkit -- The `tls-toolkit` utility generates the required keystores, truststore, and relevant configuration files to facilitate the setup of a secure NiFi instance. +* ZooKeeper Migrator -- The `zk-migrator` tool enables administrators to: +** move ZooKeeper information from one ZooKeeper cluster to another +** migrate ZooKeeper node ownership + +The utilities are executed with scripts found in the `bin` folder of your NiFi Toolkit installation. + +NOTE: The NiFi Toolkit is downloaded separately from NiFi (see the link:https://nifi.apache.org/download.html[Apache NiFi downloads page^]). + +=== Prerequisites for Running in a Secure Environment +For secured nodes and clusters, two policies should be configured in advance: + +* Access the controller â A user that will have access to these utilities should be authorized in NiFi by creating an âaccess the controllerâ policy (`/controller`) with both view and modify rights +* Proxy user request â If not previously set, nodeâs identity (the DN value of the nodeâs certificate) should be authorized to proxy requests on behalf of a user + +When executing either the Notify or Node Manager tools in a secured environment the `proxyDN` flag option should be used in order to properly identify the user that was authorized to execute these commands. In non-secure environments, or if running the status operation on the Node Manager tool, the flag is ignored. + +== NiFi CLI +This tool offers a CLI focused on interacting with NiFi and NiFi Registry in order to automate tasks, such as deploying flows from a NIFi Registy to a NiFi instance or managing process groups and cluster nodes. + +=== Usage +The CLI toolkit can be executed in standalone mode to execute a single command, or interactive mode to enter an interactive shell. + +To execute a single command: + + ./bin/cli.sh + +To launch the interactive shell: + + ./bin/cli.sh + +To show help: + + cli.sh -h --- End diff -- ```suggestion ./bin/cli.sh -h ``` ---
[GitHub] nifi pull request #3124: NIFI-5767 Added NiFi Toolkit Guide to docs
Github user pvillard31 commented on a diff in the pull request: https://github.com/apache/nifi/pull/3124#discussion_r230324180 --- Diff: nifi-docs/src/main/asciidoc/toolkit-guide.adoc --- @@ -0,0 +1,1257 @@ +// +// Licensed to the Apache Software Foundation (ASF) under one or more +// contributor license agreements. See the NOTICE file distributed with +// this work for additional information regarding copyright ownership. +// The ASF licenses this file to You under the Apache License, Version 2.0 +// (the "License"); you may not use this file except in compliance with +// the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. +// += Apache NiFi Toolkit Guide +Apache NiFi Team +:homepage: http://nifi.apache.org +:linkattrs: + +== Overview +The NiFi Toolkit contains several command line utilities to setup and support NiFi in standalone and clustered environments. The utilities include: + +* CLI -- The `cli` tool enables administrators to interact with NiFi and NiFi Registry instances to automate tasks such as deploying versioned flows and managing process groups and cluster nodes. +* Encrypt Config -- The `encrypt-config` tool encrypts the sensitive keys in the _nifi.properties_ file to facilitate the setup of a secure NiFi instance. +* File Manager -- The `file-manager` tool enables administrators to backup, install or restore a NiFi installation from backup. +* Flow Analyzer -- The `flow-analyzer` tool produces a report that helps administrators understand the max amount of data which can be stored in backpressure for a given flow. +* Node Manager -- The `node-manager` tool enables administrators to perform status checks on nodes as well as the ability to connect, disconnect, or remove nodes from the cluster. +* Notify -- The `notify` tool enables administrators to send bulletins to the NiFi UI. +* S2S -- The `s2s` tool enables administrators to send data into or out of NiFi flows over site-to-site. +* TLS Toolkit -- The `tls-toolkit` utility generates the required keystores, truststore, and relevant configuration files to facilitate the setup of a secure NiFi instance. +* ZooKeeper Migrator -- The `zk-migrator` tool enables administrators to: +** move ZooKeeper information from one ZooKeeper cluster to another +** migrate ZooKeeper node ownership + +The utilities are executed with scripts found in the `bin` folder of your NiFi Toolkit installation. + +NOTE: The NiFi Toolkit is downloaded separately from NiFi (see the link:https://nifi.apache.org/download.html[Apache NiFi downloads page^]). + +=== Prerequisites for Running in a Secure Environment +For secured nodes and clusters, two policies should be configured in advance: + +* Access the controller â A user that will have access to these utilities should be authorized in NiFi by creating an âaccess the controllerâ policy (`/controller`) with both view and modify rights +* Proxy user request â If not previously set, nodeâs identity (the DN value of the nodeâs certificate) should be authorized to proxy requests on behalf of a user + +When executing either the Notify or Node Manager tools in a secured environment the `proxyDN` flag option should be used in order to properly identify the user that was authorized to execute these commands. In non-secure environments, or if running the status operation on the Node Manager tool, the flag is ignored. + +== NiFi CLI +This tool offers a CLI focused on interacting with NiFi and NiFi Registry in order to automate tasks, such as deploying flows from a NIFi Registy to a NiFi instance or managing process groups and cluster nodes. + +=== Usage +The CLI toolkit can be executed in standalone mode to execute a single command, or interactive mode to enter an interactive shell. + +To execute a single command: + + ./bin/cli.sh + +To launch the interactive shell: + + ./bin/cli.sh + +To show help: + + cli.sh -h + +The following are available options: + + demo quick-import + nifi current-user + nifi cluster-summary + nifi connect-node + nifi delete-node + nifi disconnect-node + nifi get-root-id + nifi get-node + nifi get-nodes + nifi offload-node + nifi list-reg-clients + nifi create-reg-client + nifi update-reg-client + nifi get-reg-client-id + nifi pg-import +
[jira] [Commented] (NIFI-5728) Inconsistent behavior in XMLRecordSetWriter for Root Record Tag
[
https://issues.apache.org/jira/browse/NIFI-5728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16672781#comment-16672781
]
ASF GitHub Bot commented on NIFI-5728:
--
Github user asfgit closed the pull request at:
https://github.com/apache/nifi/pull/3098
> Inconsistent behavior in XMLRecordSetWriter for Root Record Tag
> ---
>
> Key: NIFI-5728
> URL: https://issues.apache.org/jira/browse/NIFI-5728
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.7.1
>Reporter: Ed Berezitsky
>Assignee: Ed Berezitsky
>Priority: Major
> Fix For: 1.9.0
>
>
> In XMLRecordSetWriter:
> When used with "Use Schema Text" and "Name of Record Tag" is empty (so record
> name should be used to wrapping XML tag) - works correctly.
> When used with Schema Registry and "Name of Record Tag" is empty - it doesn't
> write record name, but uses schema name instead. I believe this inconsistency
> is coming from the fact that when schema in a writer defined by "Use Schema
> Text" - it doesn't have a name and then name is taken from a record. But when
> it comes from registry - it simply uses schema idetifier:
> {code:java}
> recordSchema.getIdentifier().getName();
> {code}
> IMO root record name should be used in this case instead of schema name.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (NIFI-5728) Inconsistent behavior in XMLRecordSetWriter for Root Record Tag
[
https://issues.apache.org/jira/browse/NIFI-5728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Villard resolved NIFI-5728.
--
Resolution: Fixed
Fix Version/s: 1.9.0
> Inconsistent behavior in XMLRecordSetWriter for Root Record Tag
> ---
>
> Key: NIFI-5728
> URL: https://issues.apache.org/jira/browse/NIFI-5728
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.7.1
>Reporter: Ed Berezitsky
>Assignee: Ed Berezitsky
>Priority: Major
> Fix For: 1.9.0
>
>
> In XMLRecordSetWriter:
> When used with "Use Schema Text" and "Name of Record Tag" is empty (so record
> name should be used to wrapping XML tag) - works correctly.
> When used with Schema Registry and "Name of Record Tag" is empty - it doesn't
> write record name, but uses schema name instead. I believe this inconsistency
> is coming from the fact that when schema in a writer defined by "Use Schema
> Text" - it doesn't have a name and then name is taken from a record. But when
> it comes from registry - it simply uses schema idetifier:
> {code:java}
> recordSchema.getIdentifier().getName();
> {code}
> IMO root record name should be used in this case instead of schema name.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFI-5728) Inconsistent behavior in XMLRecordSetWriter for Root Record Tag
[
https://issues.apache.org/jira/browse/NIFI-5728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16672779#comment-16672779
]
ASF subversion and git services commented on NIFI-5728:
---
Commit 2812fe60a2ed486b1a6cb80916e764aa9af10f9b in nifi's branch
refs/heads/master from Ed B
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=2812fe6 ]
NIFI-5728 - XML Writer to populate record tag name properly
Signed-off-by: Pierre Villard
This closes #3098.
> Inconsistent behavior in XMLRecordSetWriter for Root Record Tag
> ---
>
> Key: NIFI-5728
> URL: https://issues.apache.org/jira/browse/NIFI-5728
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.7.1
>Reporter: Ed Berezitsky
>Assignee: Ed Berezitsky
>Priority: Major
>
> In XMLRecordSetWriter:
> When used with "Use Schema Text" and "Name of Record Tag" is empty (so record
> name should be used to wrapping XML tag) - works correctly.
> When used with Schema Registry and "Name of Record Tag" is empty - it doesn't
> write record name, but uses schema name instead. I believe this inconsistency
> is coming from the fact that when schema in a writer defined by "Use Schema
> Text" - it doesn't have a name and then name is taken from a record. But when
> it comes from registry - it simply uses schema idetifier:
> {code:java}
> recordSchema.getIdentifier().getName();
> {code}
> IMO root record name should be used in this case instead of schema name.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] nifi pull request #3098: NIFI-5728 XML Writer to populate record tag name pr...
Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/3098 ---
[jira] [Commented] (NIFI-5728) Inconsistent behavior in XMLRecordSetWriter for Root Record Tag
[
https://issues.apache.org/jira/browse/NIFI-5728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16672776#comment-16672776
]
ASF GitHub Bot commented on NIFI-5728:
--
Github user pvillard31 commented on the issue:
https://github.com/apache/nifi/pull/3098
+1, merging to master
> Inconsistent behavior in XMLRecordSetWriter for Root Record Tag
> ---
>
> Key: NIFI-5728
> URL: https://issues.apache.org/jira/browse/NIFI-5728
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.7.1
>Reporter: Ed Berezitsky
>Assignee: Ed Berezitsky
>Priority: Major
>
> In XMLRecordSetWriter:
> When used with "Use Schema Text" and "Name of Record Tag" is empty (so record
> name should be used to wrapping XML tag) - works correctly.
> When used with Schema Registry and "Name of Record Tag" is empty - it doesn't
> write record name, but uses schema name instead. I believe this inconsistency
> is coming from the fact that when schema in a writer defined by "Use Schema
> Text" - it doesn't have a name and then name is taken from a record. But when
> it comes from registry - it simply uses schema idetifier:
> {code:java}
> recordSchema.getIdentifier().getName();
> {code}
> IMO root record name should be used in this case instead of schema name.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] nifi issue #3098: NIFI-5728 XML Writer to populate record tag name properly
Github user pvillard31 commented on the issue: https://github.com/apache/nifi/pull/3098 +1, merging to master ---
[jira] [Commented] (NIFI-5777) Update the tag and the property of LogMessage
[
https://issues.apache.org/jira/browse/NIFI-5777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16672765#comment-16672765
]
ASF GitHub Bot commented on NIFI-5777:
--
Github user pvillard31 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/3122#discussion_r230300635
--- Diff:
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/test/java/org/apache/nifi/processors/standard/TestLogMessage.java
---
@@ -57,7 +57,7 @@ MockComponentLog getMockComponentLog() {
public void before() throws InitializationException {
testableLogMessage = new TestableLogMessage();
runner = TestRunners.newTestRunner(testableLogMessage);
-
+runner.setValidateExpressionUsage(false);
--- End diff --
In that case I'm not sure to understand the intent of this PR. Initially I
thought it was about removing
``.expressionLanguageSupported(ExpressionLanguageScope.FLOWFILE_ATTRIBUTES)``
on ``LOG_LEVEL`` since the property only accepts a list of allowed values. If
we do that, then we should remove the
``.evaluateAttributeExpressions(flowFile)``, no?
On the other hand, if we do want to allow EL on this property, then the
property and code should be changed to allow free text value that is validated
against the list of allowed values only if EL is not used. And then if EL is
used, there should be a default behavior in case the value, after EL
evaluation, is not an allowed value.
Am I missing something?
> Update the tag and the property of LogMessage
> -
>
> Key: NIFI-5777
> URL: https://issues.apache.org/jira/browse/NIFI-5777
> Project: Apache NiFi
> Issue Type: Improvement
>Affects Versions: 1.8.0
>Reporter: Kotaro Terada
>Assignee: Kotaro Terada
>Priority: Major
>
> There are a few points to update in {{LogMessage}}:
> * The processor tags are a little bit strange. The current tags are
> "attributes" and "logging". A tag "attributes" is not suitable for this
> processor. I suggest just "logging" is enough.
> * The property "Log Level" should be selected using a drop-down list (as it
> is done in {{LogAttribute}}). Currently, the field is just a text box, and
> users need to type a log level manually. If we set "expression language
> supported" on the property, does it force to make the property become a text
> field in the Web UI?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] nifi pull request #3122: NIFI-5777: Update the tag and the property of LogMe...
Github user pvillard31 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/3122#discussion_r230300635
--- Diff:
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/test/java/org/apache/nifi/processors/standard/TestLogMessage.java
---
@@ -57,7 +57,7 @@ MockComponentLog getMockComponentLog() {
public void before() throws InitializationException {
testableLogMessage = new TestableLogMessage();
runner = TestRunners.newTestRunner(testableLogMessage);
-
+runner.setValidateExpressionUsage(false);
--- End diff --
In that case I'm not sure to understand the intent of this PR. Initially I
thought it was about removing
``.expressionLanguageSupported(ExpressionLanguageScope.FLOWFILE_ATTRIBUTES)``
on ``LOG_LEVEL`` since the property only accepts a list of allowed values. If
we do that, then we should remove the
``.evaluateAttributeExpressions(flowFile)``, no?
On the other hand, if we do want to allow EL on this property, then the
property and code should be changed to allow free text value that is validated
against the list of allowed values only if EL is not used. And then if EL is
used, there should be a default behavior in case the value, after EL
evaluation, is not an allowed value.
Am I missing something?
---
[jira] [Commented] (NIFI-5752) Load balancing fails with wildcard certs
[
https://issues.apache.org/jira/browse/NIFI-5752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16672703#comment-16672703
]
ASF GitHub Bot commented on NIFI-5752:
--
Github user ijokarumawak commented on a diff in the pull request:
https://github.com/apache/nifi/pull/3110#discussion_r230287682
--- Diff:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/controller/queue/clustered/server/ClusterLoadBalanceAuthorizer.java
---
@@ -33,14 +42,27 @@
private final ClusterCoordinator clusterCoordinator;
private final EventReporter eventReporter;
+private final HostnameVerifier hostnameVerifier;
public ClusterLoadBalanceAuthorizer(final ClusterCoordinator
clusterCoordinator, final EventReporter eventReporter) {
this.clusterCoordinator = clusterCoordinator;
this.eventReporter = eventReporter;
+this.hostnameVerifier = new DefaultHostnameVerifier();
}
@Override
-public String authorize(final Collection clientIdentities)
throws NotAuthorizedException {
+public String authorize(SSLSocket sslSocket) throws
NotAuthorizedException, IOException {
+final SSLSession sslSession = sslSocket.getSession();
+
+final Set clientIdentities;
+try {
+clientIdentities = getCertificateIdentities(sslSession);
+} catch (final CertificateException e) {
+throw new IOException("Failed to extract Client Certificate",
e);
+}
+
+logger.debug("Will perform authorization against Client Identities
'{}'", clientIdentities);
+
if (clientIdentities == null) {
--- End diff --
Now we only call this `authorize()` method if socket is a SSLSocket. We can
remove this block.
> Load balancing fails with wildcard certs
>
>
> Key: NIFI-5752
> URL: https://issues.apache.org/jira/browse/NIFI-5752
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.8.0
>Reporter: Kotaro Terada
>Assignee: Kotaro Terada
>Priority: Major
>
> Load balancing fails when we construct a secure cluster with wildcard certs.
> For example, assume that we have a valid wildcard cert for {{*.example.com}}
> and a cluster consists of {{nf1.example.com}}, {{nf2.example.com}}, and
> {{nf3.example.com}} . We cannot transfer a FlowFile between nodes for load
> balancing because of the following authorization error:
> {noformat}
> 2018-10-25 19:05:13,520 WARN [Load Balance Server Thread-2]
> o.a.n.c.q.c.s.ClusterLoadBalanceAuthorizer Authorization failed for Client
> ID's [*.example.com] to Load Balance data because none of the ID's are known
> Cluster Node Identifiers
> 2018-10-25 19:05:13,521 ERROR [Load Balance Server Thread-2]
> o.a.n.c.q.c.s.ConnectionLoadBalanceServer Failed to communicate with Peer
> /xxx.xxx.xxx.xxx:x
> org.apache.nifi.controller.queue.clustered.server.NotAuthorizedException:
> Client ID's [*.example.com] are not authorized to Load Balance data
> at
> org.apache.nifi.controller.queue.clustered.server.ClusterLoadBalanceAuthorizer.authorize(ClusterLoadBalanceAuthorizer.java:65)
> at
> org.apache.nifi.controller.queue.clustered.server.StandardLoadBalanceProtocol.receiveFlowFiles(StandardLoadBalanceProtocol.java:142)
> at
> org.apache.nifi.controller.queue.clustered.server.ConnectionLoadBalanceServer$CommunicateAction.run(ConnectionLoadBalanceServer.java:176)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> {noformat}
> This problem occurs because in {{authorize}} method in
> {{ClusterLoadBalanceAuthorizer}} class, authorization is tried by just
> matching strings.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (NIFI-5752) Load balancing fails with wildcard certs
[
https://issues.apache.org/jira/browse/NIFI-5752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16672702#comment-16672702
]
ASF GitHub Bot commented on NIFI-5752:
--
Github user ijokarumawak commented on a diff in the pull request:
https://github.com/apache/nifi/pull/3110#discussion_r230287180
--- Diff:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/controller/queue/clustered/server/ClusterLoadBalanceAuthorizer.java
---
@@ -57,11 +79,35 @@ public String authorize(final Collection
clientIdentities) throws NotAut
}
}
-final String message = String.format("Authorization failed for
Client ID's %s to Load Balance data because none of the ID's are known Cluster
Node Identifiers",
-clientIdentities);
+// If there are no matches of Client IDs, try to verify it by
HostnameVerifier. In this way, we can support wildcard certificates.
+for (final String nodeId : nodeIds) {
+if (hostnameVerifier.verify(nodeId, sslSession)) {
+final String clientId =
sslSocket.getInetAddress().getHostName();
+logger.debug("The request was verified with node '{}'. The
hostname derived from the socket is '{}'. Authorizing Client to Load Balance
data", nodeId, clientId);
+return clientId;
+}
+}
+
+final String message = String.format("Authorization failed for
Client ID's to Load Balance data because none of the ID's are known Cluster
Node Identifiers");
--- End diff --
We don't have to use `String.format()` here, please the String to
`logger.warn()` directly.
> Load balancing fails with wildcard certs
>
>
> Key: NIFI-5752
> URL: https://issues.apache.org/jira/browse/NIFI-5752
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.8.0
>Reporter: Kotaro Terada
>Assignee: Kotaro Terada
>Priority: Major
>
> Load balancing fails when we construct a secure cluster with wildcard certs.
> For example, assume that we have a valid wildcard cert for {{*.example.com}}
> and a cluster consists of {{nf1.example.com}}, {{nf2.example.com}}, and
> {{nf3.example.com}} . We cannot transfer a FlowFile between nodes for load
> balancing because of the following authorization error:
> {noformat}
> 2018-10-25 19:05:13,520 WARN [Load Balance Server Thread-2]
> o.a.n.c.q.c.s.ClusterLoadBalanceAuthorizer Authorization failed for Client
> ID's [*.example.com] to Load Balance data because none of the ID's are known
> Cluster Node Identifiers
> 2018-10-25 19:05:13,521 ERROR [Load Balance Server Thread-2]
> o.a.n.c.q.c.s.ConnectionLoadBalanceServer Failed to communicate with Peer
> /xxx.xxx.xxx.xxx:x
> org.apache.nifi.controller.queue.clustered.server.NotAuthorizedException:
> Client ID's [*.example.com] are not authorized to Load Balance data
> at
> org.apache.nifi.controller.queue.clustered.server.ClusterLoadBalanceAuthorizer.authorize(ClusterLoadBalanceAuthorizer.java:65)
> at
> org.apache.nifi.controller.queue.clustered.server.StandardLoadBalanceProtocol.receiveFlowFiles(StandardLoadBalanceProtocol.java:142)
> at
> org.apache.nifi.controller.queue.clustered.server.ConnectionLoadBalanceServer$CommunicateAction.run(ConnectionLoadBalanceServer.java:176)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> {noformat}
> This problem occurs because in {{authorize}} method in
> {{ClusterLoadBalanceAuthorizer}} class, authorization is tried by just
> matching strings.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] nifi pull request #3110: NIFI-5752: Load balancing fails with wildcard certs
Github user ijokarumawak commented on a diff in the pull request:
https://github.com/apache/nifi/pull/3110#discussion_r230287180
--- Diff:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/controller/queue/clustered/server/ClusterLoadBalanceAuthorizer.java
---
@@ -57,11 +79,35 @@ public String authorize(final Collection
clientIdentities) throws NotAut
}
}
-final String message = String.format("Authorization failed for
Client ID's %s to Load Balance data because none of the ID's are known Cluster
Node Identifiers",
-clientIdentities);
+// If there are no matches of Client IDs, try to verify it by
HostnameVerifier. In this way, we can support wildcard certificates.
+for (final String nodeId : nodeIds) {
+if (hostnameVerifier.verify(nodeId, sslSession)) {
+final String clientId =
sslSocket.getInetAddress().getHostName();
+logger.debug("The request was verified with node '{}'. The
hostname derived from the socket is '{}'. Authorizing Client to Load Balance
data", nodeId, clientId);
+return clientId;
+}
+}
+
+final String message = String.format("Authorization failed for
Client ID's to Load Balance data because none of the ID's are known Cluster
Node Identifiers");
--- End diff --
We don't have to use `String.format()` here, please the String to
`logger.warn()` directly.
---
[GitHub] nifi pull request #3110: NIFI-5752: Load balancing fails with wildcard certs
Github user ijokarumawak commented on a diff in the pull request:
https://github.com/apache/nifi/pull/3110#discussion_r230287682
--- Diff:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/controller/queue/clustered/server/ClusterLoadBalanceAuthorizer.java
---
@@ -33,14 +42,27 @@
private final ClusterCoordinator clusterCoordinator;
private final EventReporter eventReporter;
+private final HostnameVerifier hostnameVerifier;
public ClusterLoadBalanceAuthorizer(final ClusterCoordinator
clusterCoordinator, final EventReporter eventReporter) {
this.clusterCoordinator = clusterCoordinator;
this.eventReporter = eventReporter;
+this.hostnameVerifier = new DefaultHostnameVerifier();
}
@Override
-public String authorize(final Collection clientIdentities)
throws NotAuthorizedException {
+public String authorize(SSLSocket sslSocket) throws
NotAuthorizedException, IOException {
+final SSLSession sslSession = sslSocket.getSession();
+
+final Set clientIdentities;
+try {
+clientIdentities = getCertificateIdentities(sslSession);
+} catch (final CertificateException e) {
+throw new IOException("Failed to extract Client Certificate",
e);
+}
+
+logger.debug("Will perform authorization against Client Identities
'{}'", clientIdentities);
+
if (clientIdentities == null) {
--- End diff --
Now we only call this `authorize()` method if socket is a SSLSocket. We can
remove this block.
---
