[jira] [Updated] (HIVE-20441) NPE in GenericUDF when hive.allow.udf.load.on.demand is set to true
[ https://issues.apache.org/jira/browse/HIVE-20441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Vary updated HIVE-20441: -- Fix Version/s: 4.0.0 Resolution: Fixed Status: Resolved (was: Patch Available) Pushed to master. Thanks for the patch [~dengzh] and sorry for the delay! > NPE in GenericUDF when hive.allow.udf.load.on.demand is set to true > > > Key: HIVE-20441 > URL: https://issues.apache.org/jira/browse/HIVE-20441 > Project: Hive > Issue Type: Bug > Components: CLI, HiveServer2 >Affects Versions: 1.2.1, 2.3.3 >Reporter: Hui Huang >Assignee: Zhihua Deng >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Attachments: HIVE-20441.1.patch, HIVE-20441.2.patch, > HIVE-20441.3.patch, HIVE-20441.4.patch, HIVE-20441.patch > > Time Spent: 2h 20m > Remaining Estimate: 0h > > When hive.allow.udf.load.on.demand is set to true and hiveserver2 has been > started, the new created function from other clients or hiveserver2 will be > loaded from the metastore at the first time. > When the udf is used in where clause, we got a NPE like: > {code:java} > Error executing statement: > org.apache.hive.service.cli.HiveSQLException: Error while compiling > statement: FAILED: NullPointerException null > at > org.apache.hive.service.cli.operation.Operation.toSQLException(Operation.java:380) > ~[hive-service-2.3.4-SNAPSHOT.jar:2.3.4-SNAPSHOT] > at > org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:206) > ~[hive-service-2.3.4-SNAPSHOT.jar:2.3.4-SNAPSHOT] > at > org.apache.hive.service.cli.operation.SQLOperation.runInternal(SQLOperation.java:290) > ~[hive-service-2.3.4-SNAPSHOT.jar:2.3.4-SNAPSHOT] > at > org.apache.hive.service.cli.operation.Operation.run(Operation.java:320) > ~[hive-service-2.3.4-SNAPSHOT.jar:2.3.4-SNAPSHOT] > at > org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(HiveSessionImpl.java:530) > ~[hive-service-2.3.4-SNAPSHOT.jar:2.3.4-SNAP > SHOT] > at > org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementAsync(HiveSessionImpl.java:517) > ~[hive-service-2.3.4-SNAPSHOT.jar:2.3.4-SNAPSHO > T] > at > org.apache.hive.service.cli.CLIService.executeStatementAsync(CLIService.java:310) > ~[hive-service-2.3.4-SNAPSHOT.jar:2.3.4-SNAPSHOT] > at > org.apache.hive.service.cli.thrift.ThriftCLIService.ExecuteStatement(ThriftCLIService.java:542) > ~[hive-service-2.3.4-SNAPSHOT.jar:2.3.4-SNAPSHOT] > at > org.apache.hive.service.rpc.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1437) > ~[hive-exec-2.3.4-SNAPSHOT.jar:2.3.4-SNA > PSHOT] > at > org.apache.hive.service.rpc.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1422) > ~[hive-exec-2.3.4-SNAPSHOT.jar:2.3.4-SNA > PSHOT] > at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) > ~[hive-exec-2.3.4-SNAPSHOT.jar:2.3.4-SNAPSHOT] > at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39) > ~[hive-exec-2.3.4-SNAPSHOT.jar:2.3.4-SNAPSHOT] > at > org.apache.hive.service.auth.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:57) > ~[hive-service-2.3.4-SNAPSHOT.jar:2.3.4-SNAPSHOT] > at > org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286) > ~[hive-exec-2.3.4-SNAPSHOT.jar:2.3.4-SNAPSHOT] > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > [?:1.8.0_77] > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > [?:1.8.0_77] > at java.lang.Thread.run(Thread.java:745) [?:1.8.0_77] > Caused by: java.lang.NullPointerException > at > org.apache.hadoop.hive.ql.plan.ExprNodeGenericFuncDesc.newInstance(ExprNodeGenericFuncDesc.java:236) > ~[hive-exec-2.3.4-SNAPSHOT.jar:2.3.4-SNAPSHOT] > at > org.apache.hadoop.hive.ql.parse.TypeCheckProcFactory$DefaultExprProcessor.getXpathOrFuncExprNodeDesc(TypeCheckProcFactory.java:1104) > ~[hive-exec-2. > 3.4-SNAPSHOT.jar:2.3.4-SNAPSHOT] > at > org.apache.hadoop.hive.ql.parse.TypeCheckProcFactory$DefaultExprProcessor.process(TypeCheckProcFactory.java:1359) > ~[hive-exec-2.3.4-SNAPSHOT.jar:2. > 3.4-SNAPSHOT] > at > org.apache.hadoop.hive.ql.lib.DefaultRuleDispatcher.dispatch(DefaultRuleDispatcher.java:90) > ~[hive-exec-2.3.4-SNAPSHOT.jar:2.3.4-SNAPSHOT] > at > org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatchAndReturn(DefaultGraphWalker.java:105) > ~[hive-exec-2.3.4-SNAPSHOT.jar:2.3.4-SNAPSHOT] > at >
[jira] [Updated] (HIVE-20441) NPE in GenericUDF when hive.allow.udf.load.on.demand is set to true
[ https://issues.apache.org/jira/browse/HIVE-20441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-20441: -- Labels: pull-request-available (was: ) > NPE in GenericUDF when hive.allow.udf.load.on.demand is set to true > > > Key: HIVE-20441 > URL: https://issues.apache.org/jira/browse/HIVE-20441 > Project: Hive > Issue Type: Bug > Components: CLI, HiveServer2 >Affects Versions: 1.2.1, 2.3.3 >Reporter: Hui Huang >Assignee: Zhihua Deng >Priority: Major > Labels: pull-request-available > Attachments: HIVE-20441.1.patch, HIVE-20441.2.patch, > HIVE-20441.3.patch, HIVE-20441.4.patch, HIVE-20441.patch > > Time Spent: 10m > Remaining Estimate: 0h > > When hive.allow.udf.load.on.demand is set to true and hiveserver2 has been > started, the new created function from other clients or hiveserver2 will be > loaded from the metastore at the first time. > When the udf is used in where clause, we got a NPE like: > {code:java} > Error executing statement: > org.apache.hive.service.cli.HiveSQLException: Error while compiling > statement: FAILED: NullPointerException null > at > org.apache.hive.service.cli.operation.Operation.toSQLException(Operation.java:380) > ~[hive-service-2.3.4-SNAPSHOT.jar:2.3.4-SNAPSHOT] > at > org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:206) > ~[hive-service-2.3.4-SNAPSHOT.jar:2.3.4-SNAPSHOT] > at > org.apache.hive.service.cli.operation.SQLOperation.runInternal(SQLOperation.java:290) > ~[hive-service-2.3.4-SNAPSHOT.jar:2.3.4-SNAPSHOT] > at > org.apache.hive.service.cli.operation.Operation.run(Operation.java:320) > ~[hive-service-2.3.4-SNAPSHOT.jar:2.3.4-SNAPSHOT] > at > org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(HiveSessionImpl.java:530) > ~[hive-service-2.3.4-SNAPSHOT.jar:2.3.4-SNAP > SHOT] > at > org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementAsync(HiveSessionImpl.java:517) > ~[hive-service-2.3.4-SNAPSHOT.jar:2.3.4-SNAPSHO > T] > at > org.apache.hive.service.cli.CLIService.executeStatementAsync(CLIService.java:310) > ~[hive-service-2.3.4-SNAPSHOT.jar:2.3.4-SNAPSHOT] > at > org.apache.hive.service.cli.thrift.ThriftCLIService.ExecuteStatement(ThriftCLIService.java:542) > ~[hive-service-2.3.4-SNAPSHOT.jar:2.3.4-SNAPSHOT] > at > org.apache.hive.service.rpc.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1437) > ~[hive-exec-2.3.4-SNAPSHOT.jar:2.3.4-SNA > PSHOT] > at > org.apache.hive.service.rpc.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1422) > ~[hive-exec-2.3.4-SNAPSHOT.jar:2.3.4-SNA > PSHOT] > at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) > ~[hive-exec-2.3.4-SNAPSHOT.jar:2.3.4-SNAPSHOT] > at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39) > ~[hive-exec-2.3.4-SNAPSHOT.jar:2.3.4-SNAPSHOT] > at > org.apache.hive.service.auth.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:57) > ~[hive-service-2.3.4-SNAPSHOT.jar:2.3.4-SNAPSHOT] > at > org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286) > ~[hive-exec-2.3.4-SNAPSHOT.jar:2.3.4-SNAPSHOT] > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > [?:1.8.0_77] > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > [?:1.8.0_77] > at java.lang.Thread.run(Thread.java:745) [?:1.8.0_77] > Caused by: java.lang.NullPointerException > at > org.apache.hadoop.hive.ql.plan.ExprNodeGenericFuncDesc.newInstance(ExprNodeGenericFuncDesc.java:236) > ~[hive-exec-2.3.4-SNAPSHOT.jar:2.3.4-SNAPSHOT] > at > org.apache.hadoop.hive.ql.parse.TypeCheckProcFactory$DefaultExprProcessor.getXpathOrFuncExprNodeDesc(TypeCheckProcFactory.java:1104) > ~[hive-exec-2. > 3.4-SNAPSHOT.jar:2.3.4-SNAPSHOT] > at > org.apache.hadoop.hive.ql.parse.TypeCheckProcFactory$DefaultExprProcessor.process(TypeCheckProcFactory.java:1359) > ~[hive-exec-2.3.4-SNAPSHOT.jar:2. > 3.4-SNAPSHOT] > at > org.apache.hadoop.hive.ql.lib.DefaultRuleDispatcher.dispatch(DefaultRuleDispatcher.java:90) > ~[hive-exec-2.3.4-SNAPSHOT.jar:2.3.4-SNAPSHOT] > at > org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatchAndReturn(DefaultGraphWalker.java:105) > ~[hive-exec-2.3.4-SNAPSHOT.jar:2.3.4-SNAPSHOT] > at > org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatch(DefaultGraphWalker.java:89) > ~[hive-exec-2.3.4-SNAPSHOT.jar:2.3.4-SNAPSHOT] > at >
[jira] [Updated] (HIVE-20441) NPE in GenericUDF when hive.allow.udf.load.on.demand is set to true
[ https://issues.apache.org/jira/browse/HIVE-20441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihua Deng updated HIVE-20441: --- Summary: NPE in GenericUDF when hive.allow.udf.load.on.demand is set to true (was: NPE in ExprNodeGenericFuncDesc when hive.allow.udf.load.on.demand is set to true) > NPE in GenericUDF when hive.allow.udf.load.on.demand is set to true > > > Key: HIVE-20441 > URL: https://issues.apache.org/jira/browse/HIVE-20441 > Project: Hive > Issue Type: Bug > Components: CLI, HiveServer2 >Affects Versions: 1.2.1, 2.3.3 >Reporter: Hui Huang >Assignee: Hui Huang >Priority: Major > Attachments: HIVE-20441.1.patch, HIVE-20441.2.patch, > HIVE-20441.3.patch, HIVE-20441.4.patch, HIVE-20441.patch > > > When hive.allow.udf.load.on.demand is set to true and hiveserver2 has been > started, the new created function from other clients or hiveserver2 will be > loaded from the metastore at the first time. > When the udf is used in where clause, we got a NPE like: > {code:java} > Error executing statement: > org.apache.hive.service.cli.HiveSQLException: Error while compiling > statement: FAILED: NullPointerException null > at > org.apache.hive.service.cli.operation.Operation.toSQLException(Operation.java:380) > ~[hive-service-2.3.4-SNAPSHOT.jar:2.3.4-SNAPSHOT] > at > org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:206) > ~[hive-service-2.3.4-SNAPSHOT.jar:2.3.4-SNAPSHOT] > at > org.apache.hive.service.cli.operation.SQLOperation.runInternal(SQLOperation.java:290) > ~[hive-service-2.3.4-SNAPSHOT.jar:2.3.4-SNAPSHOT] > at > org.apache.hive.service.cli.operation.Operation.run(Operation.java:320) > ~[hive-service-2.3.4-SNAPSHOT.jar:2.3.4-SNAPSHOT] > at > org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(HiveSessionImpl.java:530) > ~[hive-service-2.3.4-SNAPSHOT.jar:2.3.4-SNAP > SHOT] > at > org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementAsync(HiveSessionImpl.java:517) > ~[hive-service-2.3.4-SNAPSHOT.jar:2.3.4-SNAPSHO > T] > at > org.apache.hive.service.cli.CLIService.executeStatementAsync(CLIService.java:310) > ~[hive-service-2.3.4-SNAPSHOT.jar:2.3.4-SNAPSHOT] > at > org.apache.hive.service.cli.thrift.ThriftCLIService.ExecuteStatement(ThriftCLIService.java:542) > ~[hive-service-2.3.4-SNAPSHOT.jar:2.3.4-SNAPSHOT] > at > org.apache.hive.service.rpc.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1437) > ~[hive-exec-2.3.4-SNAPSHOT.jar:2.3.4-SNA > PSHOT] > at > org.apache.hive.service.rpc.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1422) > ~[hive-exec-2.3.4-SNAPSHOT.jar:2.3.4-SNA > PSHOT] > at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) > ~[hive-exec-2.3.4-SNAPSHOT.jar:2.3.4-SNAPSHOT] > at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39) > ~[hive-exec-2.3.4-SNAPSHOT.jar:2.3.4-SNAPSHOT] > at > org.apache.hive.service.auth.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:57) > ~[hive-service-2.3.4-SNAPSHOT.jar:2.3.4-SNAPSHOT] > at > org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286) > ~[hive-exec-2.3.4-SNAPSHOT.jar:2.3.4-SNAPSHOT] > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > [?:1.8.0_77] > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > [?:1.8.0_77] > at java.lang.Thread.run(Thread.java:745) [?:1.8.0_77] > Caused by: java.lang.NullPointerException > at > org.apache.hadoop.hive.ql.plan.ExprNodeGenericFuncDesc.newInstance(ExprNodeGenericFuncDesc.java:236) > ~[hive-exec-2.3.4-SNAPSHOT.jar:2.3.4-SNAPSHOT] > at > org.apache.hadoop.hive.ql.parse.TypeCheckProcFactory$DefaultExprProcessor.getXpathOrFuncExprNodeDesc(TypeCheckProcFactory.java:1104) > ~[hive-exec-2. > 3.4-SNAPSHOT.jar:2.3.4-SNAPSHOT] > at > org.apache.hadoop.hive.ql.parse.TypeCheckProcFactory$DefaultExprProcessor.process(TypeCheckProcFactory.java:1359) > ~[hive-exec-2.3.4-SNAPSHOT.jar:2. > 3.4-SNAPSHOT] > at > org.apache.hadoop.hive.ql.lib.DefaultRuleDispatcher.dispatch(DefaultRuleDispatcher.java:90) > ~[hive-exec-2.3.4-SNAPSHOT.jar:2.3.4-SNAPSHOT] > at > org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatchAndReturn(DefaultGraphWalker.java:105) > ~[hive-exec-2.3.4-SNAPSHOT.jar:2.3.4-SNAPSHOT] > at > org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatch(DefaultGraphWalker.java:89) > ~[hive-exec-2.3.4-SNAPSHOT.jar:2.3.4-SNAPSHOT] > at >