[
https://issues.apache.org/jira/browse/AMBARI-10160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14608319#comment-14608319
]
Hudson commented on AMBARI-10160:
---------------------------------
ABORTED: Integrated in Ambari-branch-2.0.0 #122 (See
[https://builds.apache.org/job/Ambari-branch-2.0.0/122/])
AMBARI-10160 - Hive Metastore alert check "show databases" results in a Tez
session causing RM to be unresponsive (jonathanhurley) (jhurley:
http://git-wip-us.apache.org/repos/asf?p=ambari.git&a=commit&h=5397621db6eb0c43aeea60493c02fa78e88e8989)
*
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/alerts/alert_hive_metastore.py
> Hive Metastore alert check "show databases" results in a Tez session causing
> RM to be unresponsive
> --------------------------------------------------------------------------------------------------
>
> Key: AMBARI-10160
> URL: https://issues.apache.org/jira/browse/AMBARI-10160
> Project: Ambari
> Issue Type: Bug
> Components: ambari-server
> Affects Versions: 2.0.0
> Reporter: Jonathan Hurley
> Assignee: Jonathan Hurley
> Priority: Critical
> Fix For: 2.0.0
>
> Attachments: AMBARI-10160.patch
>
>
> The HS2 check does this:
> {code}
> export HIVE_CONF_DIR='/etc/hive/conf.server/' ; hive --hiveconf
> hive.metastore.uris=thrift://revo3.hortonworks.local:9083 -e 'show databases;'
> {code}
> Looks like this does this to hive:
> {code}
> [3/19/15, 7:47:37 PM] Paul Codding: 2015-03-19 18:44:22,959 INFO [main]:
> parse.ParseDriver (ParseDriver.java:parse(185)) - Parsing command: show
> databases
> 2015-03-19 18:44:23,801 INFO [main]: parse.ParseDriver
> (ParseDriver.java:parse(206)) - Parse Completed
> 2015-03-19 18:44:23,809 INFO [main]: log.PerfLogger
> (PerfLogger.java:PerfLogEnd(135)) - </PERFLOG method=parse
> start=1426808662932 end=1426808663809 duration=877
> from=org.apache.hadoop.hive.ql.Driver>
> 2015-03-19 18:44:23,818 INFO [main]: log.PerfLogger
> (PerfLogger.java:PerfLogBegin(108)) - <PERFLOG method=semanticAnalyze
> from=org.apache.hadoop.hive.ql.Driver>
> 2015-03-19 18:44:24,002 INFO [main]: ql.Driver (Driver.java:compile(427)) -
> Semantic Analysis Completed
> 2015-03-19 18:44:24,003 INFO [main]: log.PerfLogger
> (PerfLogger.java:PerfLogEnd(135)) - </PERFLOG method=semanticAnalyze
> start=1426808663818 end=1426808664003 duration=185
> from=org.apache.hadoop.hive.ql.Driver>
> 2015-03-19 18:44:24,386 INFO [main]: exec.ListSinkOperator
> (Operator.java:initialize(346)) - Initializing Self OP[0]
> 2015-03-19 18:44:24,394 INFO [main]: exec.ListSinkOperator
> (Operator.java:initializeChildren(419)) - Operator 0 OP initialized
> 2015-03-19 18:44:24,395 INFO [main]: exec.ListSinkOperator
> (Operator.java:initialize(394)) - Initialization Done 0 OP
> 2015-03-19 18:44:24,419 INFO [main]: ql.Driver (Driver.java:getSchema(235))
> - Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:database_name,
> type:string, comment:from deserializer)], properties:null)
> 2015-03-19 18:44:24,420 INFO [main]: log.PerfLogger
> (PerfLogger.java:PerfLogEnd(135)) - </PERFLOG method=compile
> start=1426808662818 end=1426808664420 duration=1602
> from=org.apache.hadoop.hive.ql.Driver>
> 2015-03-19 18:44:24,420 INFO [main]: log.PerfLogger
> (PerfLogger.java:PerfLogBegin(108)) - <PERFLOG method=Driver.execute
> from=org.apache.hadoop.hive.ql.Driver>
> 2015-03-19 18:44:24,420 INFO [main]: ql.Driver (Driver.java:execute(1285)) -
> Starting command: show databases
> 2015-03-19 18:44:25,090 INFO [main]: impl.TimelineClientImpl
> (TimelineClientImpl.java:serviceInit(285)) - Timeline service address:
> http://revo2.hortonworks.local:8188/ws/v1/timeline/
> 2015-03-19 18:44:25,093 INFO [main]: hooks.ATSHook (ATSHook.java:<init>(87))
> - Created ATS Hook
> 2015-03-19 18:44:25,093 INFO [main]: log.PerfLogger
> (PerfLogger.java:PerfLogBegin(108)) - <PERFLOG
> method=PreHook.org.apache.hadoop.hive.ql.hooks.ATSHook
> from=org.apache.hadoop.hive.ql.Driver>
> 2015-03-19 18:44:25,098 INFO [main]: log.PerfLogger
> (PerfLogger.java:PerfLogEnd(135)) - </PERFLOG
> method=PreHook.org.apache.hadoop.hive.ql.hooks.ATSHook start=1426808665093
> end=1426808665098 duration=5 from=org.apache.hadoop.hive.ql.Driver>
> 2015-03-19 18:44:25,103 INFO [main]: log.PerfLogger
> (PerfLogger.java:PerfLogEnd(135)) - </PERFLOG method=TimeToSubmit
> start=1426808662818 end=1426808665103 duration=2285
> from=org.apache.hadoop.hive.ql.Driver>
> 2015-03-19 18:44:25,103 INFO [main]: log.PerfLogger
> (PerfLogger.java:PerfLogBegin(108)) - <PERFLOG method=runTasks
> from=org.apache.hadoop.hive.ql.Driver>
> 2015-03-19 18:44:25,103 INFO [main]: log.PerfLogger
> (PerfLogger.java:PerfLogBegin(108)) - <PERFLOG method=task.DDL.Stage-0
> from=org.apache.hadoop.hive.ql.Driver>
> 2015-03-19 18:44:25,114 INFO [main]: ql.Driver
> (Driver.java:launchTask(1602)) - Starting task [Stage-0:DDL] in serial mode
> 2015-03-19 18:44:25,309 INFO [main]: exec.DDLTask
> (DDLTask.java:showDatabases(2173)) - results : 1
> 2015-03-19 18:44:25,317 INFO [main]: log.PerfLogger
> (PerfLogger.java:PerfLogEnd(135)) - </PERFLOG method=runTasks
> start=1426808665103 end=1426808665317 duration=214
> from=org.apache.hadoop.hive.ql.Driver>
> 2015-03-19 18:44:25,319 INFO [main]: hooks.ATSHook (ATSHook.java:<init>(87))
> - Created ATS Hook
> 2015-03-19 18:44:25,319 INFO [main]: log.PerfLogger
> (PerfLogger.java:PerfLogBegin(108)) - <PERFLOG
> method=PostHook.org.apache.hadoop.hive.ql.hooks.ATSHook
> from=org.apache.hadoop.hive.ql.Driver>
> 2015-03-19 18:44:25,320 INFO [main]: log.PerfLogger
> (PerfLogger.java:PerfLogEnd(135)) - </PERFLOG
> method=PostHook.org.apache.hadoop.hive.ql.hooks.ATSHook start=1426808665319
> end=1426808665320 duration=1 from=org.apache.hadoop.hive.ql.Driver>
> 2015-03-19 18:44:25,321 INFO [main]: log.PerfLogger
> (PerfLogger.java:PerfLogEnd(135)) - </PERFLOG method=Driver.execute
> start=1426808664420 end=1426808665321 duration=901
> from=org.apache.hadoop.hive.ql.Driver>
> 2015-03-19 18:44:25,330 INFO [main]: ql.Driver
> (SessionState.java:printInfo(824)) - OK
> 2015-03-19 18:44:25,331 INFO [main]: log.PerfLogger
> (PerfLogger.java:PerfLogBegin(108)) - <PERFLOG method=releaseLocks
> from=org.apache.hadoop.hive.ql.Driver>
> 2015-03-19 18:44:25,331 INFO [main]: log.PerfLogger
> (PerfLogger.java:PerfLogEnd(135)) - </PERFLOG method=releaseLocks
> start=1426808665331 end=1426808665331 duration=0
> from=org.apache.hadoop.hive.ql.Driver>
> 2015-03-19 18:44:25,332 INFO [main]: log.PerfLogger
> (PerfLogger.java:PerfLogEnd(135)) - </PERFLOG method=Driver.run
> start=1426808662817 end=1426808665332 duration=2515
> from=org.apache.hadoop.hive.ql.Driver>
> 2015-03-19 18:44:25,353 INFO [main]: Configuration.deprecation
> (Configuration.java:warnOnceIfDeprecated(1051)) - mapred.input.dir is
> deprecated. Instead, use mapreduce.input.fileinputformat.inputdir
> 2015-03-19 18:44:25,536 INFO [main]: mapred.FileInputFormat
> (FileInputFormat.java:listStatus(247)) - Total input paths to process : 1
> 2015-03-19 18:44:25,635 INFO [main]: exec.ListSinkOperator
> (Operator.java:close(595)) - 0 finished. closing...
> 2015-03-19 18:44:25,635 INFO [main]: exec.ListSinkOperator
> (Operator.java:close(613)) - 0 Close done
> 2015-03-19 18:44:25,660 INFO [main]: CliDriver
> (SessionState.java:printInfo(824)) - Time taken: 2.529 seconds, Fetched: 1
> row(s)
> 2015-03-19 18:44:25,660 INFO [main]: log.PerfLogger
> (PerfLogger.java:PerfLogBegin(108)) - <PERFLOG method=releaseLocks
> from=org.apache.hadoop.hive.ql.Driver>
> 2015-03-19 18:44:25,662 INFO [main]: log.PerfLogger
> (PerfLogger.java:PerfLogEnd(135)) - </PERFLOG method=releaseLocks
> start=1426808665660 end=1426808665662 duration=2
> from=org.apache.hadoop.hive.ql.Driver>
> 2015-03-19 18:44:25,708 INFO [main]: tez.TezSessionPoolManager
> (TezSessionPoolManager.java:close(174)) - Closing tez session default? false
> 2015-03-19 18:44:25,709 INFO [main]: tez.TezSessionState
> (TezSessionState.java:close(263)) - Closing Tez Session
> 2015-03-19 18:44:25,710 INFO [main]: client.TezClient
> (TezClient.java:stop(434)) - Shutting down Tez Session,
> sessionName=HIVE-f96da6a9-ad75-4949-b8ba-a9e3b65278d3,
> applicationId=application_1426808089446_0009
> [3/19/15, 7:47:50 PM] Paul Codding: salient
> [3/19/15, 7:47:50 PM] Paul Codding: 2015-03-19 18:44:22,959 INFO [main]:
> parse.ParseDriver (ParseDriver.java:parse(185)) - Parsing command: show
> databases
> 2015-03-19 18:44:23,801 INFO [main]: parse.ParseDriver
> (ParseDriver.java:parse(206)) - Parse Completed
> {code}
> Changing this check to use the MR enginer instead of TEZ corrects the problem:
> {code}
> export HIVE_CONF_DIR='/etc/hive/conf.server/' ; hive --hiveconf
> hive.metastore.uris=thrift://revo3.hortonworks.local:9083 --hiveconf
> hive.execution.engine=mr -e 'show databases;'
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
