[jira] [Commented] (NIFI-4187) If NiFi process is killed, bootstrap auto-restart hangs on missing sensitive key file
[ https://issues.apache.org/jira/browse/NIFI-4187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16123189#comment-16123189 ] Pierre Villard commented on NIFI-4187: -- Went through the verification process suggested in the comment before and after applying the change. Confirmed it's working as expected. Run contrib-check. All good, merging to master. > If NiFi process is killed, bootstrap auto-restart hangs on missing sensitive > key file > - > > Key: NIFI-4187 > URL: https://issues.apache.org/jira/browse/NIFI-4187 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.3.0 >Reporter: Andy LoPresto >Assignee: Andy LoPresto >Priority: Critical > Labels: bootstrap, encryption > Fix For: 1.4.0 > > > If the NiFi process is killed and {{autoRestart}} is enabled in the bootstrap > process ({{RunNiFi.java}}) (enabled by default except when in {{run}} mode) > and the instance is using encrypted configuration files, the > {{sensitive.key}} file which contains the master encryption key is no longer > available when restart occurs. This results in the application exiting. > To resolve this, the logic to prepare and generate the sensitive key file > should be extracted to a method and invoked during initial start *and* during > the auto-restart loop at > https://github.com/apache/nifi/blob/master/nifi-bootstrap/src/main/java/org/apache/nifi/bootstrap/RunNiFi.java#L1159 > . -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (NIFI-4187) If NiFi process is killed, bootstrap auto-restart hangs on missing sensitive key file
[ https://issues.apache.org/jira/browse/NIFI-4187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16123188#comment-16123188 ] ASF GitHub Bot commented on NIFI-4187: -- Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/2073 > If NiFi process is killed, bootstrap auto-restart hangs on missing sensitive > key file > - > > Key: NIFI-4187 > URL: https://issues.apache.org/jira/browse/NIFI-4187 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.3.0 >Reporter: Andy LoPresto >Assignee: Andy LoPresto >Priority: Critical > Labels: bootstrap, encryption > Fix For: 1.4.0 > > > If the NiFi process is killed and {{autoRestart}} is enabled in the bootstrap > process ({{RunNiFi.java}}) (enabled by default except when in {{run}} mode) > and the instance is using encrypted configuration files, the > {{sensitive.key}} file which contains the master encryption key is no longer > available when restart occurs. This results in the application exiting. > To resolve this, the logic to prepare and generate the sensitive key file > should be extracted to a method and invoked during initial start *and* during > the auto-restart loop at > https://github.com/apache/nifi/blob/master/nifi-bootstrap/src/main/java/org/apache/nifi/bootstrap/RunNiFi.java#L1159 > . -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (NIFI-4187) If NiFi process is killed, bootstrap auto-restart hangs on missing sensitive key file
[ https://issues.apache.org/jira/browse/NIFI-4187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16122529#comment-16122529 ] ASF GitHub Bot commented on NIFI-4187: -- Github user alopresto commented on the issue: https://github.com/apache/nifi/pull/2073 To verify this fix: 1. modify the `conf/bootstrap.conf` file to provide a *master encryption key* (it doesn't matter if any `conf/nifi.properties` values are encrypted with it, the mere presence will trigger the relevant code) 1. Start NiFi as normal (i.e. `$ ./bin/nifi.sh start`) 1. Verify that NiFi starts (UI, `nifi-bootstrap.log`, `nifi-app.log`, etc.) 1. Identify the NiFi process (*not* the bootstrap process) and kill it (imitating an `OOME` or external process killing NiFi) * `ps -ef | grep nifi` to get the PID * `kill -9 ` 1. Verify that NiFi successfully restarts The `logs/nifi-bootstrap.log` will look like this: ``` 2017-08-10 16:08:49,675 INFO [main] o.a.n.b.NotificationServiceManager Successfully loaded the following 0 services: [] 2017-08-10 16:08:49,679 INFO [main] org.apache.nifi.bootstrap.RunNiFi Registered no Notification Services for Notification Type NIFI_STARTED 2017-08-10 16:08:49,679 INFO [main] org.apache.nifi.bootstrap.RunNiFi Registered no Notification Services for Notification Type NIFI_STOPPED 2017-08-10 16:08:49,679 INFO [main] org.apache.nifi.bootstrap.RunNiFi Registered no Notification Services for Notification Type NIFI_DIED 2017-08-10 16:08:49,736 INFO [main] org.apache.nifi.bootstrap.Command Starting Apache NiFi... 2017-08-10 16:08:49,736 INFO [main] org.apache.nifi.bootstrap.Command Working Directory: /Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.4.0-SNAPSHOT-bin/nifi-1.4.0-SNAPSHOT 2017-08-10 16:08:49,736 INFO [main] org.apache.nifi.bootstrap.Command Command: /Users/alopresto/.jenv/versions/1.8/bin/java -classpath /Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.4.0-SNAPSHOT-bin/nifi-1.4.0-SNAPSHOT/./conf:/Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.4.0-SNAPSHOT-bin/nifi-1.4.0-SNAPSHOT/./lib/javax.servlet-api-3.1.0.jar:/Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.4.0-SNAPSHOT-bin/nifi-1.4.0-SNAPSHOT/./lib/jcl-over-slf4j-1.7.25.jar:/Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.4.0-SNAPSHOT-bin/nifi-1.4.0-SNAPSHOT/./lib/jetty-schemas-3.1.jar:/Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.4.0-SNAPSHOT-bin/nifi-1.4.0-SNAPSHOT/./lib/jul-to-slf4j-1.7.25.jar:/Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.4.0-SNAPSHOT-bin/nifi-1.4.0-SNAPSHOT/./lib/log4j-over-slf4j-1.7.25.jar:/Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.4.0-SNAPSHOT-bin/nifi-1.4.0-SNAPSHOT/./lib/logback-classic-1.2.3.jar:/Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.4.0-SNAPSHOT-bin/nifi-1.4.0-SNAPSHOT/./lib/logback-core-1.2.3.jar:/Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.4.0-SNAPSHOT-bin/nifi-1.4.0-SNAPSHOT/./lib/nifi-api-1.4.0-SNAPSHOT.jar:/Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.4.0-SNAPSHOT-bin/nifi-1.4.0-SNAPSHOT/./lib/nifi-framework-api-1.4.0-SNAPSHOT.jar:/Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.4.0-SNAPSHOT-bin/nifi-1.4.0-SNAPSHOT/./lib/nifi-nar-utils-1.4.0-SNAPSHOT.jar:/Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.4.0-SNAPSHOT-bin/nifi-1.4.0-SNAPSHOT/./lib/nifi-properties-1.4.0-SNAPSHOT.jar:/Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.4.0-SNAPSHOT-bin/nifi-1.4.0-SNAPSHOT/./lib/nifi-runtime-1.4.0-SNAPSHOT.jar:/Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.4.0-SNAPSHOT-bin/nifi-1.4.0-SNAPSHOT/./lib/slf4j-api-1.7.25.jar -Dorg.apache.jasper.compiler.disablejsr199=true -Xmx512m -Xms512m -Djava.security.egd=file:/dev/urandom -Dsun.net.http.allowRestrictedHeaders=true -Djava.net.preferIPv4Stack=true -Djava.awt.headless=true -XX:+UseG1GC -Djava.protocol.handler.pkgs=sun.net.www.protocol -Dnifi.properties.file.path=/Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.4.0-SNAPSHOT-bin/nifi-1.4.0-SNAPSHOT/./conf/nifi.properties -Dnifi.bootstrap.listen.port=52192 -Dapp=NiFi -Dorg.apache.nifi.bootstrap.config.log.dir=/Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.4.0-SNAPSHOT-bin/nifi-1.4.0-SNAPSHOT/logs org.apache.nifi.NiFi -K /Users/alopresto/Workspace/nifi/nifi-assembly/target/nifi-1.4.0-SNAPSHOT-bin/nifi-1.4.0-SNAPSHOT/./conf/sensitive.key 2017-08-10 16:08:49,749 INFO [main] org.apache.nifi.bootstrap.Command Launched Apache NiFi with Process ID 90937 2017-08-10 16:08:50,294 INFO [NiFi Bootstrap Command Listener] org.apache.nifi.bootstrap.RunNiFi Apache NiFi now running and listening for Bootstrap requests on port 52193 -External command kills NiFi process- 2017-08-10 16:11:13,185 WARN [m
[jira] [Commented] (NIFI-4187) If NiFi process is killed, bootstrap auto-restart hangs on missing sensitive key file
[ https://issues.apache.org/jira/browse/NIFI-4187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16122517#comment-16122517 ] ASF GitHub Bot commented on NIFI-4187: -- GitHub user alopresto opened a pull request: https://github.com/apache/nifi/pull/2073 NIFI-4187 Resolved issue where restart failed due to missing sensitiv… …e key file. Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [x] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/alopresto/nifi NIFI-4187 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2073.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2073 commit 7dc0bd2084820a9f924a0593f2e938efb1a0ceed Author: Andy LoPresto Date: 2017-08-10T23:16:29Z NIFI-4187 Resolved issue where restart failed due to missing sensitive key file. > If NiFi process is killed, bootstrap auto-restart hangs on missing sensitive > key file > - > > Key: NIFI-4187 > URL: https://issues.apache.org/jira/browse/NIFI-4187 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.3.0 >Reporter: Andy LoPresto >Priority: Critical > Labels: bootstrap, encryption > > If the NiFi process is killed and {{autoRestart}} is enabled in the bootstrap > process ({{RunNiFi.java}}) (enabled by default except when in {{run}} mode) > and the instance is using encrypted configuration files, the > {{sensitive.key}} file which contains the master encryption key is no longer > available when restart occurs. This results in the application exiting. > To resolve this, the logic to prepare and generate the sensitive key file > should be extracted to a method and invoked during initial start *and* during > the auto-restart loop at > https://github.com/apache/nifi/blob/master/nifi-bootstrap/src/main/java/org/apache/nifi/bootstrap/RunNiFi.java#L1159 > . -- This message was sent by Atlassian JIRA (v6.4.14#64029)