Andre and all, So you’ve mis-read this basically, but I should also provide an update. We cannot use sudo rights when deploying in bitbucket pipelines. That is one of the restrictions to consider. Let me go through a few more things again and add to them this time. However, please remember that we are now w/ this newer version of nifi having difficulty killing nifi (will explain details), which is the big reason for a desire for help.
1) We are configuring bootstrap.conf to start up nifi as a non-root user. (The user does happen to have sudo privileges, but we’re trying not to use them.) This nifi.sh will start up in this way w/ the user configured there. 2) If nifi was installed as an init.d service and bootstrap.conf had a non-root user filled in the “run.as” area, then on reboot nifi would not start up – BAD. This is the kicker of why I went the systemd service route as we can have run-as configured, have the service start-up on reboot, and have things work. W/ the init.d problem we however could start up the service as that user, but it was a dealbreaker to not have the service automatically come up when the VM was starting back up. 3) We use a nifi.service file to configure nifi as a systemd service. This is an example below of how we are doing that via an example nifi.service snippet. [Unit] Description=nifi After=network.target [Service] User=nonroot-user Group=nonroot-user WorkingDirectory=/tools/nifi-1.2.0/lib/bootstrap ExecStart=/usr/bin/java -cp /tools/nifi-1.2.0/lib/bootstrap/activation-1.1.jar:/tools/nifi-1.2.0/lib/bootstrap/json-smart-2.1.1.jar:/tools/nifi-1.2.0/lib/bootstrap/antlr-runtime-3.5.2.jar:/tools/nifi-1.2.0/lib/bootstrap/logback-classic-1.2.3.jar:/tools/nifi-1.2.0/lib/bootstrap/asm-1.0.2.jar:/tools/nifi-1.2.0/lib/bootstrap/logback-core-1.2.3.jar:/tools/nifi-1.2.0/lib/bootstrap/asm-3.3.1.jar:/tools/nifi-1.2.0/lib/bootstrap/mail-1.4.7.jar:/tools/nifi-1.2.0/lib/bootstrap/bcpkix-jdk15on-1.55.jar:/tools/nifi-1.2.0/lib/bootstrap/nifi-api-1.2.0.jar:/tools/nifi-1.2.0/lib/bootstrap/bcprov-jdk15on-1.55.jar:/tools/nifi-1.2.0/lib/bootstrap/nifi-bootstrap-1.2.0.jar:/tools/nifi-1.2.0/lib/bootstrap/commons-codec-1.10.jar:/tools/nifi-1.2.0/lib/bootstrap/nifi-expression-language-1.2.0.jar:/tools/nifi-1.2.0/lib/bootstrap/commons-lang3-3.5.jar:/tools/nifi-1.2.0/lib/bootstrap/nifi-properties-loader-1.2.0.jar:/tools/nifi-1.2.0/lib/bootstrap/jackson-annotations-2.6.0.jar:/tools/nifi-1.2.0/lib/bootstrap/nifi-security-utils-1.2.0.jar:/tools/nifi-1.2.0/lib/bootstrap/jackson-core-2.6.1.jar:/tools/nifi-1.2.0/lib/bootstrap/nifi-utils-1.2.0.jar:/tools/nifi-1.2.0/lib/bootstrap/jackson-databind-2.6.1.jar:/tools/nifi-1.2.0/lib/bootstrap/okhttp-3.6.0.jar:/tools/nifi-1.2.0/lib/bootstrap/jna-4.4.0.jar:/tools/nifi-1.2.0/lib/bootstrap/okio-1.11.0.jar:/tools/nifi-1.2.0/lib/bootstrap/jna-platform-4.4.0.jar:/tools/nifi-1.2.0/lib/bootstrap/slf4j-api-1.7.25.jar:/tools/nifi-1.2.0/lib/bootstrap/json-path-2.0.0.jar -Xms12m -Xmx24m -Dorg.apache.nifi.bootstrap.config.file=/tools/nifi-1.2.0/conf/bootstrap.conf org.apache.nifi.bootstrap.RunNiFi start Type=simple SuccessExitStatus=143 [Install] WantedBy=multi-user.target 4) In nifi versions prior to 1.2.0, we’ve been able to kill nifi over bitbucket piplelines w/ the simple use of “nifi.sh stop” when nifi has previously started either as a systemd service or a deployment has occurred and it has started up via “nifi.sh start” on bitbucket pipelines. If an auto-deployment has not occurred yet on a particular day, then nifi is still running as a systemd service until the first deployment. a. As of 1.2.0, if nifi is still running as a systemd service, we seem to be able to only kill nifi via explicit execution of “org.apache.nifi.bootstrap.RunNiFi stop”. We’ll then observe a “PING” occurring, which will help cause nifi to shutdown. b. As of 1.2.0, if nifi is still running as a systemd service, we no longer seem to be able to kill its execution any longer with simple use of “nifi.sh stop” as we have for all previous versions. We were surprised about this and wanted to reach out for help for especially this reason and concern. c. Once nifi is running, but not as a service, the use of “nifi.sh stop” works, but using “org.apache.nifi.bootstrap.RunNiFi stop” does not. This is not horrible, but this means that we have to use both ways of killing nifi, the “org.apache.nifi.bootstrap.RunNiFi stop” technique and the “nifi.sh stop” technique. And, this seems silly as you’d think both ways would work the same or at least “nifi.sh stop” would always work as it always did before, but they do not work equivalently in our case. And, thus, this sloppy need for both is yet another reason for us to reach out along w/ 4b. Further thoughts? Further thoughts on #4? Regards, Jeremy From: Andre <[email protected]> Reply-To: "[email protected]" <[email protected]> Date: Friday, May 26, 2017 at 8:48 PM To: "[email protected]" <[email protected]> Subject: Re: difficulty killing NiFi w/ nifi.sh stop under 1.2.0 now vs. 1.1.2 when run as systemd service Jeremy, I am not familiar with Bitbucket pipelines but would you mind explaining why are you wrapping the deployment with sudo? Reason I ask is because nifi.sh already has logic to drop the process privileges from root to an uid of choice. Cheers On Sat, May 27, 2017 at 1:33 AM, Jeremy Taylor <[email protected]<mailto:[email protected]>> wrote: Greetings, I’m hoping to upgrade the my software team’s baseline to use NiFi 1.2.0. I’m having trouble w/ a particular situation of trying to take nifi down. Prior to this version, I’ve been able to use the nifi.sh script w/o sudo rights to kill nifi when it has been brought up by a service (systemd, not init.d). Main points: 1) We have had auto-deploy working via bitbucket pipelines to a staging system that has auto deployed all the nifi required files from our baseline (not nifi internal configs). 2) The AWS VM that runs nifi autmatically starts up nifi in a service state via systemd services. We masquerade running it as a non-root user w/ sudo rights, but do not use those sudo rights as scripts in bitbucket pipelines won’t support sudo rights. 3) The deploy script for our staging server that runs nifi attempts to take down nifi with a `nifi.sh stop`, which has worked prior to this version. The newer nifi flow being deployed is manipulated via XSL and then copied in where it needs to go before restarting nifi w/ `nifi.sh start` ; (we stop using systemd upon the first deploy to the staging system) 4) I’ve done a diff between the latest nifi.sh and the previous nifi.sh from 1.1.2 and only see a tiny difference. 5) Our staging server starts up every morning and brings up nifi and related services via systemd services. 6) I realize using nifi as an init.d service is more supported than using it as a systemd service. However, we’ve not been able to masquerade as a different user properly very well when using init.d. Having to be root for bitbucket pipelines would also be a dealbreaker for us. Thus, having the service being run as a user other than root is important to us. 7) For reference, our systemd service file only deals with calling the RunNiFi class and does not bother with the nifi shell script. Questions: 1) Would anyone know why a simple “nifi.sh stop” would no longer kill a pre-existing nifi process being run as a systemd service in nifi 1.2.0? 2) We are thinking of attempting a brute force kill that would kill the “RunNiFi start” Java process. We are concerned that not exiting gracefully would be really bad for nifi and related nifi repositories. Would this route be recommended anyway in our circumstance? 3) Any further recommendations? --Jeremy
