[jira] [Commented] (HIVE-22391) NPE while checking Hive query results cache
[
https://issues.apache.org/jira/browse/HIVE-22391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16957643#comment-16957643
]
Hive QA commented on HIVE-22391:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12983774/HIVE-22391.1.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17545 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19121/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19121/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19121/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12983774 - PreCommit-HIVE-Build
> NPE while checking Hive query results cache
> ---
>
> Key: HIVE-22391
> URL: https://issues.apache.org/jira/browse/HIVE-22391
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Reporter: Jason Dere
>Assignee: Jason Dere
>Priority: Major
> Attachments: HIVE-22391.1.patch
>
>
> NPE when results cache was enabled:
> {noformat}
> 2019-10-21T14:51:55,718 ERROR [b7d7bea8-eef0-4ea4-ae12-951cb5dc96e3
> HiveServer2-Handler-Pool: Thread-210]: ql.Driver (:()) - FAILED:
> NullPointerException null
> java.lang.NullPointerException
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.checkResultsCache(SemanticAnalyzer.java:15061)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:12320)
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:360)
> at
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:289)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:664)
> at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1869)
> at
> org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1816)
> at
> org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1811)
> at
> org.apache.hadoop.hive.ql.reexec.ReExecDriver.compileAndRespond(ReExecDriver.java:126)
> at
> org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:197)
> at
> org.apache.hive.service.cli.operation.SQLOperation.runInternal(SQLOperation.java:262)
> at
> org.apache.hive.service.cli.operation.Operation.run(Operation.java:247)
> at
> org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(HiveSessionImpl.java:575)
> at
> org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementAsync(HiveSessionImpl.java:561)
> at
> org.apache.hive.service.cli.CLIService.executeStatementAsync(CLIService.java:315)
> at
> org.apache.hive.service.cli.thrift.ThriftCLIService.ExecuteStatement(ThriftCLIService.java:566)
> at
> org.apache.hive.service.rpc.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1557)
> at
> org.apache.hive.service.rpc.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1542)
> at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
> at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
> at
> org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor.process(HadoopThriftAuthBridge.java:647)
> at
> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22391) NPE while checking Hive query results cache
[
https://issues.apache.org/jira/browse/HIVE-22391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16957618#comment-16957618
]
Hive QA commented on HIVE-22391:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
28s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
6s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
37s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
59s{color} | {color:blue} ql in master has 1545 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
0s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 24m 15s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-19121/dev-support/hive-personality.sh
|
| git revision | master / f1e59e9 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19121/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> NPE while checking Hive query results cache
> ---
>
> Key: HIVE-22391
> URL: https://issues.apache.org/jira/browse/HIVE-22391
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Reporter: Jason Dere
>Assignee: Jason Dere
>Priority: Major
> Attachments: HIVE-22391.1.patch
>
>
> NPE when results cache was enabled:
> {noformat}
> 2019-10-21T14:51:55,718 ERROR [b7d7bea8-eef0-4ea4-ae12-951cb5dc96e3
> HiveServer2-Handler-Pool: Thread-210]: ql.Driver (:()) - FAILED:
> NullPointerException null
> java.lang.NullPointerException
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.checkResultsCache(SemanticAnalyzer.java:15061)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:12320)
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:360)
> at
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:289)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:664)
> at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1869)
> at
> org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1816)
> at
> org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1811)
> at
> org.apache.hadoop.hive.ql.reexec.ReExecDriver.compileAndRespond(ReExecDriver.java:126)
> at
> org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:197)
> at
> org.apache.hive.service.cli.operation.SQLOperatio
[jira] [Commented] (HIVE-22391) NPE while checking Hive query results cache
[
https://issues.apache.org/jira/browse/HIVE-22391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16957367#comment-16957367
]
Gopal Vijayaraghavan commented on HIVE-22391:
-
LGTM - +1
> NPE while checking Hive query results cache
> ---
>
> Key: HIVE-22391
> URL: https://issues.apache.org/jira/browse/HIVE-22391
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Reporter: Jason Dere
>Assignee: Jason Dere
>Priority: Major
> Attachments: HIVE-22391.1.patch
>
>
> NPE when results cache was enabled:
> {noformat}
> 2019-10-21T14:51:55,718 ERROR [b7d7bea8-eef0-4ea4-ae12-951cb5dc96e3
> HiveServer2-Handler-Pool: Thread-210]: ql.Driver (:()) - FAILED:
> NullPointerException null
> java.lang.NullPointerException
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.checkResultsCache(SemanticAnalyzer.java:15061)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:12320)
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:360)
> at
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:289)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:664)
> at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1869)
> at
> org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1816)
> at
> org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1811)
> at
> org.apache.hadoop.hive.ql.reexec.ReExecDriver.compileAndRespond(ReExecDriver.java:126)
> at
> org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:197)
> at
> org.apache.hive.service.cli.operation.SQLOperation.runInternal(SQLOperation.java:262)
> at
> org.apache.hive.service.cli.operation.Operation.run(Operation.java:247)
> at
> org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(HiveSessionImpl.java:575)
> at
> org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementAsync(HiveSessionImpl.java:561)
> at
> org.apache.hive.service.cli.CLIService.executeStatementAsync(CLIService.java:315)
> at
> org.apache.hive.service.cli.thrift.ThriftCLIService.ExecuteStatement(ThriftCLIService.java:566)
> at
> org.apache.hive.service.rpc.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1557)
> at
> org.apache.hive.service.rpc.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1542)
> at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
> at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
> at
> org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor.process(HadoopThriftAuthBridge.java:647)
> at
> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22391) NPE while checking Hive query results cache
[
https://issues.apache.org/jira/browse/HIVE-22391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16957270#comment-16957270
]
Jason Dere commented on HIVE-22391:
---
What happened was HS2 and the QueryResultsCache had previously hit the
following error during HS2 initialization:
{noformat}
2019-10-21T14:50:34,602 WARN [main]: server.HiveServer2
(HiveServer2.java:startHiveServer2(1100)) - Error starting HiveServer2 on
attempt 1, will retry in 6ms
java.lang.RuntimeException: Error initializing the query results cache
at
org.apache.hive.service.server.HiveServer2.init(HiveServer2.java:266)
~[hive-service-3.1.0.3.1.0.86-1.jar:3.1.0.3.1.0.86-1]
at
org.apache.hive.service.server.HiveServer2.startHiveServer2(HiveServer2.java:1072)
[hive-service-3.1.0.3.1.0.86-1.jar:3.1.0.3.1.0.86-1]
at
org.apache.hive.service.server.HiveServer2.access$1700(HiveServer2.java:135)
[hive-service-3.1.0.3.1.0.86-1.jar:3.1.0.3.1.0.86-1]
at
org.apache.hive.service.server.HiveServer2$StartOptionExecutor.execute(HiveServer2.java:1341)
[hive-service-3.1.0.3.1.0.86-1.jar:3.1.0.3.1.0.86-1]
at
org.apache.hive.service.server.HiveServer2.main(HiveServer2.java:1185)
[hive-service-3.1.0.3.1.0.86-1.jar:3.1.0.3.1.0.86-1]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
~[?:1.8.0_191]
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
~[?:1.8.0_191]
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
~[?:1.8.0_191]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_191]
at org.apache.hadoop.util.RunJar.run(RunJar.java:318)
[hadoop-common-3.1.1.3.1.0.86-1.jar:?]
at org.apache.hadoop.util.RunJar.main(RunJar.java:232)
[hadoop-common-3.1.1.3.1.0.86-1.jar:?]
Caused by: java.lang.RuntimeException: The dir: /tmp/hive/_resultscache_ on
HDFS should be writable. Current permissions are: rwxr-xr-x
at
org.apache.hadoop.hive.ql.exec.Utilities.ensurePathIsWritable(Utilities.java:4512)
~[hive-exec-3.1.0.3.1.0.86-1.jar:3.1.0.3.1.0.86-1]
at
org.apache.hadoop.hive.ql.cache.results.QueryResultsCache.(QueryResultsCache.java:365)
~[hive-exec-3.1.0.3.1.0.86-1.jar:3.1.0.3.1.0.86-1]
at
org.apache.hadoop.hive.ql.cache.results.QueryResultsCache.initialize(QueryResultsCache.java:395)
~[hive-exec-3.1.0.3.1.0.86-1.jar:3.1.0.3.1.0.86-1]
at
org.apache.hive.service.server.HiveServer2.init(HiveServer2.java:264)
~[hive-service-3.1.0.3.1.0.86-1.jar:3.1.0.3.1.0.86-1]
... 10 more
{noformat}
HS2 should have failed startup with this error, but it looks like HS2
re-attempts initialization more than once (without restarting).
QueryResultsCache.inited should have been resetted back to false (so that
QueryResultsCache.initialize() would hit this same error again), but it looks
like the exception handling was only catching IOException, and so
RuntimeException was not handled within QueryResultsCache.initialize(). The
error handling here should be fixed.
> NPE while checking Hive query results cache
> ---
>
> Key: HIVE-22391
> URL: https://issues.apache.org/jira/browse/HIVE-22391
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Reporter: Jason Dere
>Assignee: Jason Dere
>Priority: Major
>
> NPE when results cache was enabled:
> {noformat}
> 2019-10-21T14:51:55,718 ERROR [b7d7bea8-eef0-4ea4-ae12-951cb5dc96e3
> HiveServer2-Handler-Pool: Thread-210]: ql.Driver (:()) - FAILED:
> NullPointerException null
> java.lang.NullPointerException
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.checkResultsCache(SemanticAnalyzer.java:15061)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:12320)
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:360)
> at
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:289)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:664)
> at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1869)
> at
> org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1816)
> at
> org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1811)
> at
> org.apache.hadoop.hive.ql.reexec.ReExecDriver.compileAndRespond(ReExecDriver.java:126)
> at
> org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:197)
> at
> org.apache.hive.service.cli.operation.SQLOperation.runInternal(SQLOperation.java:262)
> at
> org.apache.hive.service.cli.operation.Operation.run(Operation.java:247)
> at
> org.apache.hive.service.cli.sess
