[
https://issues.apache.org/jira/browse/HIVE-22391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16957270#comment-16957270
]
Jason Dere commented on HIVE-22391:
-----------------------------------
What happened was HS2 and the QueryResultsCache had previously hit the
following error during HS2 initialization:
{noformat}
2019-10-21T14:50:34,602 WARN [main]: server.HiveServer2
(HiveServer2.java:startHiveServer2(1100)) - Error starting HiveServer2 on
attempt 1, will retry in 60000ms
java.lang.RuntimeException: Error initializing the query results cache
at
org.apache.hive.service.server.HiveServer2.init(HiveServer2.java:266)
~[hive-service-3.1.0.3.1.0.86-1.jar:3.1.0.3.1.0.86-1]
at
org.apache.hive.service.server.HiveServer2.startHiveServer2(HiveServer2.java:1072)
[hive-service-3.1.0.3.1.0.86-1.jar:3.1.0.3.1.0.86-1]
at
org.apache.hive.service.server.HiveServer2.access$1700(HiveServer2.java:135)
[hive-service-3.1.0.3.1.0.86-1.jar:3.1.0.3.1.0.86-1]
at
org.apache.hive.service.server.HiveServer2$StartOptionExecutor.execute(HiveServer2.java:1341)
[hive-service-3.1.0.3.1.0.86-1.jar:3.1.0.3.1.0.86-1]
at
org.apache.hive.service.server.HiveServer2.main(HiveServer2.java:1185)
[hive-service-3.1.0.3.1.0.86-1.jar:3.1.0.3.1.0.86-1]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
~[?:1.8.0_191]
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
~[?:1.8.0_191]
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
~[?:1.8.0_191]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_191]
at org.apache.hadoop.util.RunJar.run(RunJar.java:318)
[hadoop-common-3.1.1.3.1.0.86-1.jar:?]
at org.apache.hadoop.util.RunJar.main(RunJar.java:232)
[hadoop-common-3.1.1.3.1.0.86-1.jar:?]
Caused by: java.lang.RuntimeException: The dir: /tmp/hive/_resultscache_ on
HDFS should be writable. Current permissions are: rwxr-xr-x
at
org.apache.hadoop.hive.ql.exec.Utilities.ensurePathIsWritable(Utilities.java:4512)
~[hive-exec-3.1.0.3.1.0.86-1.jar:3.1.0.3.1.0.86-1]
at
org.apache.hadoop.hive.ql.cache.results.QueryResultsCache.<init>(QueryResultsCache.java:365)
~[hive-exec-3.1.0.3.1.0.86-1.jar:3.1.0.3.1.0.86-1]
at
org.apache.hadoop.hive.ql.cache.results.QueryResultsCache.initialize(QueryResultsCache.java:395)
~[hive-exec-3.1.0.3.1.0.86-1.jar:3.1.0.3.1.0.86-1]
at
org.apache.hive.service.server.HiveServer2.init(HiveServer2.java:264)
~[hive-service-3.1.0.3.1.0.86-1.jar:3.1.0.3.1.0.86-1]
... 10 more
{noformat}
HS2 should have failed startup with this error, but it looks like HS2
re-attempts initialization more than once (without restarting).
QueryResultsCache.inited should have been resetted back to false (so that
QueryResultsCache.initialize() would hit this same error again), but it looks
like the exception handling was only catching IOException, and so
RuntimeException was not handled within QueryResultsCache.initialize(). The
error handling here should be fixed.
> NPE while checking Hive query results cache
> -------------------------------------------
>
> Key: HIVE-22391
> URL: https://issues.apache.org/jira/browse/HIVE-22391
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
> Reporter: Jason Dere
> Assignee: Jason Dere
> Priority: Major
>
> NPE when results cache was enabled:
> {noformat}
> 2019-10-21T14:51:55,718 ERROR [b7d7bea8-eef0-4ea4-ae12-951cb5dc96e3
> HiveServer2-Handler-Pool: Thread-210]: ql.Driver (:()) - FAILED:
> NullPointerException null
> java.lang.NullPointerException
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.checkResultsCache(SemanticAnalyzer.java:15061)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:12320)
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:360)
> at
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:289)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:664)
> at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1869)
> at
> org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1816)
> at
> org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1811)
> at
> org.apache.hadoop.hive.ql.reexec.ReExecDriver.compileAndRespond(ReExecDriver.java:126)
> at
> org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:197)
> at
> org.apache.hive.service.cli.operation.SQLOperation.runInternal(SQLOperation.java:262)
> at
> org.apache.hive.service.cli.operation.Operation.run(Operation.java:247)
> at
> org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(HiveSessionImpl.java:575)
> at
> org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementAsync(HiveSessionImpl.java:561)
> at
> org.apache.hive.service.cli.CLIService.executeStatementAsync(CLIService.java:315)
> at
> org.apache.hive.service.cli.thrift.ThriftCLIService.ExecuteStatement(ThriftCLIService.java:566)
> at
> org.apache.hive.service.rpc.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1557)
> at
> org.apache.hive.service.rpc.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1542)
> at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
> at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
> at
> org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor.process(HadoopThriftAuthBridge.java:647)
> at
> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
