----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64637/ -----------------------------------------------------------
Review request for Ambari, Dmytro Grinenko, Dmitro Lisnichenko, and Nate Cole. Bugs: AMBARI-22655 https://issues.apache.org/jira/browse/AMBARI-22655 Repository: ambari Description ------- While restarting Livy and Livy2 on a non-root cluster, the following is seen: ``` 17/12/14 14:36:23 WARN LivyConf: The configuration key livy.repl.enableHiveContext has been deprecated as of Livy 0.4 and may be removed in the future. Please use the new key livy.repl.enable-hive-context instead. 17/12/14 14:36:23 WARN LivyConf: The configuration key livy.server.csrf_protection.enabled has been deprecated as of Livy 0.4 and may be removed in the future. Please use the new key livy.server.csrf-protection.enabled instead. 17/12/14 14:36:23 INFO AccessManager: AccessControlManager acls disabled;users with view permission: ;users with modify permission: ;users with super permission: cstm-zeppelin;other allowed users: * 17/12/14 14:36:28 INFO LineBufferedStream: stdout: Welcome to 17/12/14 14:36:28 INFO LineBufferedStream: stdout: ____ __ 17/12/14 14:36:28 INFO LineBufferedStream: stdout: / __/__ ___ _____/ /__ 17/12/14 14:36:28 INFO LineBufferedStream: stdout: _\ \/ _ \/ _ `/ __/ '_/ 17/12/14 14:36:28 INFO LineBufferedStream: stdout: /___/ .__/_,_/_/ /_/_\ version 2.2.0.2.6.4.0-73 17/12/14 14:36:28 INFO LineBufferedStream: stdout: /_/ 17/12/14 14:36:28 INFO LineBufferedStream: stdout: 17/12/14 14:36:28 INFO LineBufferedStream: stdout: Using Scala version 2.11.8, OpenJDK 64-Bit Server VM, 1.8.0_131 17/12/14 14:36:28 INFO LineBufferedStream: stdout: Branch HEAD 17/12/14 14:36:28 INFO LineBufferedStream: stdout: Compiled by user jenkins on 2017-12-13T19:08:32Z 17/12/14 14:36:28 INFO LineBufferedStream: stdout: Revision a24017869f5450397136ee8b11be818e7cd3facb 17/12/14 14:36:28 INFO LineBufferedStream: stdout: Url g...@github.com:hortonworks/spark2.git 17/12/14 14:36:28 INFO LineBufferedStream: stdout: Type --help for more information. 17/12/14 14:36:29 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 17/12/14 14:36:31 INFO AHSProxy: Connecting to Application History server at nat-yc-r7-ovvs-ambari-autostart-4-re-2.openstacklocal/172.22.121.144:10200 17/12/14 14:36:32 WARN DomainSocketFactory: The short-circuit local reads feature cannot be used because libhadoop cannot be loaded. 17/12/14 14:36:33 INFO StateStore$: Using FileSystemStateStore for recovery. 17/12/14 14:36:33 INFO BatchSessionManager: Recovered 0 batch sessions. Next session id: 0 17/12/14 14:36:33 INFO InteractiveSessionManager: Recovered 0 interactive sessions. Next session id: 0 17/12/14 14:36:33 INFO InteractiveSessionManager: Heartbeat watchdog thread started. 17/12/14 14:36:33 INFO LivyServer: SPNEGO auth enabled (principal = HTTP/nat-yc-r7-ovvs-ambari-autostart-4-re-3.openstacklo...@example.com) 17/12/14 14:36:33 INFO LivyServer: CSRF protection is enabled. 17/12/14 14:36:34 INFO KerberosAuthenticationHandler: Login using keytab /etc/security/keytabs/spnego.service.keytab, for principal HTTP/nat-yc-r7-ovvs-ambari-autostart-4-re-3.openstacklo...@example.com 17/12/14 14:36:34 INFO KerberosAuthenticationHandler: Map server: nat-yc-r7-ovvs-ambari-autostart-4-re-3.openstacklocal to principal: [HTTP/nat-yc-r7-ovvs-ambari-autostart-4-re-3.openstacklo...@example.com], added = true 17/12/14 14:36:34 WARN AbstractLifeCycle: FAILED ServerConnector@df1cff6{SSL-http/1.1}{0.0.0.0:8999}: java.net.BindException: Address already in use java.net.BindException: Address already in use at sun.nio.ch.Net.bind0(Native Method) ``` This occurs because the PID file cannot be accessed by the non-root agent and returns an exit code of 1: {code} call returned (1, 'cat: /var/run/livy/livy-cstm-livy-server.pid: Permission denied') {code} This tricks out PID detection into thinking that there is no process running (or a missing PID file) and therefore, we do not need to stop Livy/Livy2. Diffs ----- ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/livy_service.py cce2148ab6 ambari-server/src/main/resources/common-services/SPARK2/2.0.0/package/scripts/livy2_service.py dfadd84613 ambari-server/src/main/resources/common-services/SPARK2/2.0.0/package/scripts/setup_livy2.py c4284889a1 ambari-server/src/main/resources/stacks/HDP/2.0.6/properties/stack_packages.json 62a46b91bd Diff: https://reviews.apache.org/r/64637/diff/1/ Testing ------- Manually verified PID detection on non-root cluster. Thanks, Jonathan Hurley