-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51766/
-----------------------------------------------------------
Review request for Ambari, Alejandro Fernandez, Andrew Onischuk, and Sumit
Mohanty.
Bugs: AMBARI-18351
https://issues.apache.org/jira/browse/AMBARI-18351
Repository: ambari
Description
-------
**Issue:**
Ambari currently runs run.sh for LLAP as such:
======================================================================================================================
2016-08-29 20:34:47,770 -
Execute['/var/lib/ambari-agent/tmp/llap-slider2016-08-29_20-34-01/run.sh']
{'user': 'hive'}
2016-08-29 20:35:51,856 - Submitted LLAP app name : llap0
======================================================================================================================
Above is from Hive Interactive start operation output. Good to have the output
(stdout and stderr) of 'run.sh', for debugging purposes.
**Fix:**
- Included the output (stdout and stderr) of 'run.sh' command in the operation
logs, for debugging purposes.
Diffs
-----
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_server_interactive.py
2df001c
ambari-server/src/test/python/stacks/2.5/HIVE/test_hive_server_int.py 389e45d
Diff: https://reviews.apache.org/r/51766/diff/
Testing
-------
- Python UT passes.
- Jenkins Pending.
- Tested on live cluster.
Logs: (o/p of run.sh)
======================================================================================================================
2016-09-09 18:30:11,250 -
checked_call['/var/lib/ambari-agent/tmp/llap-slider2016-09-09_18-24-36/run.sh']
{'logoutput': True, 'user': 'hive', 'stderr': -1}
2016-09-09 18:30:15,750 [main] INFO impl.TimelineClientImpl - Timeline service
address: http://swap-8thsept-hive-2.openstacklocal:8188/ws/v1/timeline/
2016-09-09 18:30:17,324 [main] WARN shortcircuit.DomainSocketFactory - The
short-circuit local reads feature cannot be used because libhadoop cannot be
loaded.
2016-09-09 18:30:17,343 [main] INFO client.RMProxy - Connecting to
ResourceManager at swap-8thsept-hive-2.openstacklocal/172.22.114.139:8050
2016-09-09 18:30:17,581 [main] INFO client.AHSProxy - Connecting to
Application History server at
swap-8thsept-hive-2.openstacklocal/172.22.114.139:10200
2016-09-09 18:30:17,795 [main] ERROR main.ServiceLauncher - Unknown application
instance : llap0
(definition not found at
hdfs://swap-8thsept-hive-1.openstacklocal:8020/user/hive/.slider/cluster/llap0/app_config.json
2016-09-09 18:30:17,799 [main] INFO util.ExitUtil - Exiting with status 69
2016-09-09 18:30:23,168 [main] INFO impl.TimelineClientImpl - Timeline service
address: http://swap-8thsept-hive-2.openstacklocal:8188/ws/v1/timeline/
2016-09-09 18:30:24,965 [main] WARN shortcircuit.DomainSocketFactory - The
short-circuit local reads feature cannot be used because libhadoop cannot be
loaded.
2016-09-09 18:30:24,989 [main] INFO client.RMProxy - Connecting to
ResourceManager at swap-8thsept-hive-2.openstacklocal/172.22.114.139:8050
2016-09-09 18:30:25,166 [main] INFO client.AHSProxy - Connecting to
Application History server at
swap-8thsept-hive-2.openstacklocal/172.22.114.139:10200
2016-09-09 18:30:25,793 [main] INFO imps.CuratorFrameworkImpl - Starting
2016-09-09 18:30:25,921 [main-EventThread] INFO state.ConnectionStateManager -
State change: CONNECTED
2016-09-09 18:30:25,978 [main] INFO client.SliderClient - Destroyed cluster
llap0
2016-09-09 18:30:25,981 [main] INFO util.ExitUtil - Exiting with status 0
2016-09-09 18:30:30,649 [main] INFO impl.TimelineClientImpl - Timeline service
address: http://swap-8thsept-hive-2.openstacklocal:8188/ws/v1/timeline/
2016-09-09 18:30:32,060 [main] WARN shortcircuit.DomainSocketFactory - The
short-circuit local reads feature cannot be used because libhadoop cannot be
loaded.
2016-09-09 18:30:32,081 [main] INFO client.RMProxy - Connecting to
ResourceManager at swap-8thsept-hive-2.openstacklocal/172.22.114.139:8050
2016-09-09 18:30:32,237 [main] INFO client.AHSProxy - Connecting to
Application History server at
swap-8thsept-hive-2.openstacklocal/172.22.114.139:10200
2016-09-09 18:30:32,475 [main] WARN client.SliderClient - The install-package
option has been deprecated. Please use 'package --install'.
2016-09-09 18:30:32,505 [main] INFO client.SliderClient - Installing package
file:/var/lib/ambari-agent/tmp/llap-slider2016-09-09_18-24-36/llap-09Sep2016.zip
at
hdfs://swap-8thsept-hive-1.openstacklocal:8020/user/hive/.slider/package/LLAP/llap-09Sep2016.zip
and overwrite is true.
2016-09-09 18:30:35,399 [main] INFO util.ExitUtil - Exiting with status 0
2016-09-09 18:30:39,883 [main] INFO impl.TimelineClientImpl - Timeline service
address: http://swap-8thsept-hive-2.openstacklocal:8188/ws/v1/timeline/
2016-09-09 18:30:41,393 [main] WARN shortcircuit.DomainSocketFactory - The
short-circuit local reads feature cannot be used because libhadoop cannot be
loaded.
2016-09-09 18:30:41,411 [main] INFO client.RMProxy - Connecting to
ResourceManager at swap-8thsept-hive-2.openstacklocal/172.22.114.139:8050
2016-09-09 18:30:41,659 [main] INFO client.AHSProxy - Connecting to
Application History server at
swap-8thsept-hive-2.openstacklocal/172.22.114.139:10200
2016-09-09 18:30:43,551 [main] INFO agent.AgentClientProvider - Validating app
definition .slider/package/LLAP/llap-09Sep2016.zip
2016-09-09 18:30:43,559 [main] INFO agent.AgentUtils - Reading metainfo at
.slider/package/LLAP/llap-09Sep2016.zip
2016-09-09 18:30:44,902 [main] INFO tools.SliderUtils - Reading metainfo.xml
of size 1998
2016-09-09 18:30:45,440 [main] INFO client.SliderClient - No credentials
requested
2016-09-09 18:30:45,639 [main] INFO agent.AgentUtils - Reading metainfo at
.slider/package/LLAP/llap-09Sep2016.zip
2016-09-09 18:30:46,422 [main] INFO tools.SliderUtils - Reading metainfo.xml
of size 1998
2016-09-09 18:30:46,529 [main] INFO launch.AbstractLauncher - Setting
yarn.resourcemanager.am.retry-count-window-ms to 300000
2016-09-09 18:30:46,538 [main] INFO launch.AbstractLauncher - Log include
patterns: .*.done
2016-09-09 18:30:46,538 [main] INFO launch.AbstractLauncher - Log exclude
patterns:
2016-09-09 18:30:46,539 [main] INFO launch.AbstractLauncher - Modified log
include patterns: .*.done
2016-09-09 18:30:46,539 [main] INFO launch.AbstractLauncher - Modified log
exclude patterns:
2016-09-09 18:30:47,042 [main] INFO slideram.SliderAMClientProvider - Loading
all dependencies for AM.
2016-09-09 18:30:47,046 [main] INFO tools.SliderUtils - Loading all
dependencies from slider.tar.gz
2016-09-09 18:30:47,050 [main] INFO agent.AgentClientProvider - Automatically
uploading the agent tarball at
hdfs://swap-8thsept-hive-1.openstacklocal:8020/user/hive/.slider/cluster/llap0/tmp/application_1473378798908_0017/agent
2016-09-09 18:30:47,113 [main] INFO agent.AgentClientProvider - Validating app
definition .slider/package/LLAP/llap-09Sep2016.zip
2016-09-09 18:30:47,133 [main] INFO client.SliderClient - Using queue llap for
the application instance.
2016-09-09 18:30:47,134 [main] INFO client.SliderClient - Submitting
application application_1473378798908_0017
2016-09-09 18:30:47,139 [main] INFO launch.AppMasterLauncher - Submitting
application to Resource Manager
2016-09-09 18:30:47,378 [main] INFO impl.YarnClientImpl - Submitted
application application_1473378798908_0017
2016-09-09 18:30:47,383 [main] INFO util.ExitUtil - Exiting with status 0
2016-09-09 18:30:47,838 - checked_call returned (0, "2016-09-09 18:30:15,750
[main] INFO impl.TimelineClientImpl - Timeline service address:
http://swap-8thsept-hive-2.openstacklocal:8188/ws/v1/timeline/\n2016-09-09
18:30:17,324 [main] WARN shortcircuit.DomainSocketFactory - The short-circuit
local reads feature cannot be used because libhadoop cannot be
loaded.\n2016-09-09 18:30:17,343 [main] INFO client.RMProxy - Connecting to
ResourceManager at
swap-8thsept-hive-2.openstacklocal/172.22.114.139:8050\n2016-09-09 18:30:17,581
[main] INFO client.AHSProxy - Connecting to Application History server at
swap-8thsept-hive-2.openstacklocal/172.22.114.139:10200\n2016-09-09
18:30:17,795 [main] ERROR main.ServiceLauncher - Unknown application instance :
llap0\n (definition not found at
hdfs://swap-8thsept-hive-1.openstacklocal:8020/user/hive/.slider/cluster/llap0/app_config.json\n2016-09-09
18:30:17,799 [main] INFO util.ExitUtil - Exiting with status 69\n2016-09-09
18:30:23,168 [main] INF
O impl.TimelineClientImpl - Timeline service address:
http://swap-8thsept-hive-2.openstacklocal:8188/ws/v1/timeline/\n2016-09-09
18:30:24,965 [main] WARN shortcircuit.DomainSocketFactory - The short-circuit
local reads feature cannot be used because libhadoop cannot be
loaded.\n2016-09-09 18:30:24,989 [main] INFO client.RMProxy - Connecting to
ResourceManager at
swap-8thsept-hive-2.openstacklocal/172.22.114.139:8050\n2016-09-09 18:30:25,166
[main] INFO client.AHSProxy - Connecting to Application History server at
swap-8thsept-hive-2.openstacklocal/172.22.114.139:10200\n2016-09-09
18:30:25,793 [main] INFO imps.CuratorFrameworkImpl - Starting\n2016-09-09
18:30:25,921 [main-EventThread] INFO state.ConnectionStateManager - State
change: CONNECTED\n2016-09-09 18:30:25,978 [main] INFO client.SliderClient -
Destroyed cluster llap0\n2016-09-09 18:30:25,981 [main] INFO util.ExitUtil -
Exiting with status 0\n2016-09-09 18:30:30,649 [main] INFO
impl.TimelineClientImpl - Timeline servi
ce address:
http://swap-8thsept-hive-2.openstacklocal:8188/ws/v1/timeline/\n2016-09-09
18:30:32,060 [main] WARN shortcircuit.DomainSocketFactory - The short-circuit
local reads feature cannot be used because libhadoop cannot be
loaded.\n2016-09-09 18:30:32,081 [main] INFO client.RMProxy - Connecting to
ResourceManager at
swap-8thsept-hive-2.openstacklocal/172.22.114.139:8050\n2016-09-09 18:30:32,237
[main] INFO client.AHSProxy - Connecting to Application History server at
swap-8thsept-hive-2.openstacklocal/172.22.114.139:10200\n2016-09-09
18:30:32,475 [main] WARN client.SliderClient - The install-package option has
been deprecated. Please use 'package --install'.\n2016-09-09 18:30:32,505
[main] INFO client.SliderClient - Installing package
file:/var/lib/ambari-agent/tmp/llap-slider2016-09-09_18-24-36/llap-09Sep2016.zip
at
hdfs://swap-8thsept-hive-1.openstacklocal:8020/user/hive/.slider/package/LLAP/llap-09Sep2016.zip
and overwrite is true.\n2016-09-09 18:30:35,399 [main] INFO
util.ExitUtil - Exiting with status 0\n2016-09-09 18:30:39,883 [main] INFO
impl.TimelineClientImpl - Timeline service address:
http://swap-8thsept-hive-2.openstacklocal:8188/ws/v1/timeline/\n2016-09-09
18:30:41,393 [main] WARN shortcircuit.DomainSocketFactory - The short-circuit
local reads feature cannot be used because libhadoop cannot be
loaded.\n2016-09-09 18:30:41,411 [main] INFO client.RMProxy - Connecting to
ResourceManager at
swap-8thsept-hive-2.openstacklocal/172.22.114.139:8050\n2016-09-09 18:30:41,659
[main] INFO client.AHSProxy - Connecting to Application History server at
swap-8thsept-hive-2.openstacklocal/172.22.114.139:10200\n2016-09-09
18:30:43,551 [main] INFO agent.AgentClientProvider - Validating app definition
.slider/package/LLAP/llap-09Sep2016.zip\n2016-09-09 18:30:43,559 [main] INFO
agent.AgentUtils - Reading metainfo at
.slider/package/LLAP/llap-09Sep2016.zip\n2016-09-09 18:30:44,902 [main] INFO
tools.SliderUtils - Reading metainfo.xml of size 1998\n201
6-09-09 18:30:45,440 [main] INFO client.SliderClient - No credentials
requested\n2016-09-09 18:30:45,639 [main] INFO agent.AgentUtils - Reading
metainfo at .slider/package/LLAP/llap-09Sep2016.zip\n2016-09-09 18:30:46,422
[main] INFO tools.SliderUtils - Reading metainfo.xml of size 1998\n2016-09-09
18:30:46,529 [main] INFO launch.AbstractLauncher - Setting
yarn.resourcemanager.am.retry-count-window-ms to 300000\n2016-09-09
18:30:46,538 [main] INFO launch.AbstractLauncher - Log include patterns:
.*\.done\n2016-09-09 18:30:46,538 [main] INFO launch.AbstractLauncher - Log
exclude patterns: \n2016-09-09 18:30:46,539 [main] INFO
launch.AbstractLauncher - Modified log include patterns: .*\.done\n2016-09-09
18:30:46,539 [main] INFO launch.AbstractLauncher - Modified log exclude
patterns: \n2016-09-09 18:30:47,042 [main] INFO
slideram.SliderAMClientProvider - Loading all dependencies for AM.\n2016-09-09
18:30:47,046 [main] INFO tools.SliderUtils - Loading all dependencies from slid
er.tar.gz\n2016-09-09 18:30:47,050 [main] INFO agent.AgentClientProvider -
Automatically uploading the agent tarball at
hdfs://swap-8thsept-hive-1.openstacklocal:8020/user/hive/.slider/cluster/llap0/tmp/application_1473378798908_0017/agent\n2016-09-09
18:30:47,113 [main] INFO agent.AgentClientProvider - Validating app
definition .slider/package/LLAP/llap-09Sep2016.zip\n2016-09-09 18:30:47,133
[main] INFO client.SliderClient - Using queue llap for the application
instance.\n2016-09-09 18:30:47,134 [main] INFO client.SliderClient -
Submitting application application_1473378798908_0017\n2016-09-09 18:30:47,139
[main] INFO launch.AppMasterLauncher - Submitting application to Resource
Manager\n2016-09-09 18:30:47,378 [main] INFO impl.YarnClientImpl - Submitted
application application_1473378798908_0017\n2016-09-09 18:30:47,383 [main] INFO
util.ExitUtil - Exiting with status 0", '')
======================================================================================================================
Thanks,
Swapan Shridhar
