----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64637/#review193918 -----------------------------------------------------------
Ship it! Ship It! - Nate Cole On Dec. 14, 2017, 10:38 p.m., Jonathan Hurley wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64637/ > ----------------------------------------------------------- > > (Updated Dec. 14, 2017, 10:38 p.m.) > > > Review request for Ambari, Dmytro Grinenko, Dmitro Lisnichenko, and Nate Cole. > > > Bugs: AMBARI-22655 > https://issues.apache.org/jira/browse/AMBARI-22655 > > > Repository: ambari > > > Description > ------- > > While restarting Livy and Livy2 on a non-root cluster, the following is seen: > > ``` > 17/12/14 14:36:23 WARN LivyConf: The configuration key > livy.repl.enableHiveContext has been deprecated as of Livy 0.4 and may be > removed in the future. Please use the new key livy.repl.enable-hive-context > instead. > 17/12/14 14:36:23 WARN LivyConf: The configuration key > livy.server.csrf_protection.enabled has been deprecated as of Livy 0.4 and > may be removed in the future. Please use the new key > livy.server.csrf-protection.enabled instead. > 17/12/14 14:36:23 INFO AccessManager: AccessControlManager acls > disabled;users with view permission: ;users with modify permission: ;users > with super permission: cstm-zeppelin;other allowed users: * > 17/12/14 14:36:28 INFO LineBufferedStream: stdout: Welcome to > 17/12/14 14:36:28 INFO LineBufferedStream: stdout: ____ __ > 17/12/14 14:36:28 INFO LineBufferedStream: stdout: / __/__ ___ _____/ > /__ > 17/12/14 14:36:28 INFO LineBufferedStream: stdout: _\ \/ _ \/ _ `/ __/ > '_/ > 17/12/14 14:36:28 INFO LineBufferedStream: stdout: /___/ .__/_,_/_/ /_/_\ > version 2.2.0.2.6.4.0-73 > 17/12/14 14:36:28 INFO LineBufferedStream: stdout: /_/ > 17/12/14 14:36:28 INFO LineBufferedStream: stdout: > 17/12/14 14:36:28 INFO LineBufferedStream: stdout: Using Scala version > 2.11.8, OpenJDK 64-Bit Server VM, 1.8.0_131 > 17/12/14 14:36:28 INFO LineBufferedStream: stdout: Branch HEAD > 17/12/14 14:36:28 INFO LineBufferedStream: stdout: Compiled by user jenkins > on 2017-12-13T19:08:32Z > 17/12/14 14:36:28 INFO LineBufferedStream: stdout: Revision > a24017869f5450397136ee8b11be818e7cd3facb > 17/12/14 14:36:28 INFO LineBufferedStream: stdout: Url > g...@github.com:hortonworks/spark2.git > 17/12/14 14:36:28 INFO LineBufferedStream: stdout: Type --help for more > information. > 17/12/14 14:36:29 WARN NativeCodeLoader: Unable to load native-hadoop library > for your platform... using builtin-java classes where applicable > 17/12/14 14:36:31 INFO AHSProxy: Connecting to Application History server at > nat-yc-r7-ovvs-ambari-autostart-4-re-2.openstacklocal/172.22.121.144:10200 > 17/12/14 14:36:32 WARN DomainSocketFactory: The short-circuit local reads > feature cannot be used because libhadoop cannot be loaded. > 17/12/14 14:36:33 INFO StateStore$: Using FileSystemStateStore for recovery. > 17/12/14 14:36:33 INFO BatchSessionManager: Recovered 0 batch sessions. Next > session id: 0 > 17/12/14 14:36:33 INFO InteractiveSessionManager: Recovered 0 interactive > sessions. Next session id: 0 > 17/12/14 14:36:33 INFO InteractiveSessionManager: Heartbeat watchdog thread > started. > 17/12/14 14:36:33 INFO LivyServer: SPNEGO auth enabled (principal = > HTTP/nat-yc-r7-ovvs-ambari-autostart-4-re-3.openstacklo...@example.com) > 17/12/14 14:36:33 INFO LivyServer: CSRF protection is enabled. > 17/12/14 14:36:34 INFO KerberosAuthenticationHandler: Login using keytab > /etc/security/keytabs/spnego.service.keytab, for principal > HTTP/nat-yc-r7-ovvs-ambari-autostart-4-re-3.openstacklo...@example.com > 17/12/14 14:36:34 INFO KerberosAuthenticationHandler: Map server: > nat-yc-r7-ovvs-ambari-autostart-4-re-3.openstacklocal to principal: > [HTTP/nat-yc-r7-ovvs-ambari-autostart-4-re-3.openstacklo...@example.com], > added = true > 17/12/14 14:36:34 WARN AbstractLifeCycle: FAILED > ServerConnector@df1cff6{SSL-http/1.1}{0.0.0.0:8999}: java.net.BindException: > Address already in use > java.net.BindException: Address already in use > at sun.nio.ch.Net.bind0(Native Method) > ``` > > This occurs because the PID file cannot be accessed by the non-root agent and > returns an exit code of 1: > {code} > call returned (1, 'cat: /var/run/livy/livy-cstm-livy-server.pid: Permission > denied') > {code} > > This tricks out PID detection into thinking that there is no process running > (or a missing PID file) and therefore, we do not need to stop Livy/Livy2. > > > Diffs > ----- > > > ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/livy_service.py > cce2148ab6 > > ambari-server/src/main/resources/common-services/SPARK2/2.0.0/package/scripts/livy2_service.py > dfadd84613 > > ambari-server/src/main/resources/common-services/SPARK2/2.0.0/package/scripts/setup_livy2.py > c4284889a1 > > ambari-server/src/main/resources/stacks/HDP/2.0.6/properties/stack_packages.json > 62a46b91bd > ambari-server/src/test/python/stacks/2.5/SPARK/test_spark_livy.py > 4f1fb17b64 > ambari-server/src/test/python/stacks/2.6/SPARK2/test_spark_livy2.py > 60e7fd2a42 > > > Diff: https://reviews.apache.org/r/64637/diff/2/ > > > Testing > ------- > > ---------------------------------------------------------------------- > Total run:1200 > Total errors:0 > Total failures:0 > OK > [INFO] > ------------------------------------------------------------------------ > [INFO] BUILD SUCCESS > [INFO] > ------------------------------------------------------------------------ > [INFO] Total time: 02:37 min > [INFO] Finished at: 2017-12-14T22:37:14-05:00 > [INFO] Final Memory: 21M/619M > [INFO] > ------------------------------------------------------------------------ > > > Thanks, > > Jonathan Hurley > >
