EthanLiu1987 opened a new issue #2888:
URL: https://github.com/apache/incubator-dolphinscheduler/issues/2888
*For better global communication, please give priority to using English
description, thx! *
**Describe the bug**
1、upload a hive udf in resource center successfully
2、create a work flow , use this udf to query , but return error。
3、dolphin version is 1.2.0 release,hive version is 1.2.1 and 2.7.3 both test
failed
4、we test the udf in the hive cli, query successfully,so the jar is good
**To Reproduce**
Steps to reproduce the behavior, for example:
1. upload a hive udf in resource center
2. create a work flow , use this udf to query
3. See error，error info :
[INFO] 2020-06-03 15:03:54.560 - [taskAppId=TASK-8-133-287]:[109] -
SqlParameters{type='HIVE', datasource=2, sql='select name,id,idcardcheck(id) as
result from ods_people', sqlType=0, udfs='', showType='TABLE', connParams='',
title='test', receivers='[email protected]', receiversCc='', preStatements=[],
postStatements=[]}
[INFO] 2020-06-03 15:03:54.567 - [taskAppId=TASK-8-133-287]:[110] - sql
type : HIVE, datasource : 2, sql : select name,id,idcardcheck(id) as result
from ods_people , localParams : [],udfs : ,showType : TABLE,connParams :
[INFO] 2020-06-03 15:03:54.570 - [taskAppId=TASK-8-133-287]:[127] -
datasource name : hivetest2 , type : HIVE , desc : , user_id : 2 , parameter
:
{"address":"jdbc:hive2://10.19.83.39:10000","database":"default","jdbcUrl":"jdbc:hive2://10.19.83.39:10000/default","user":"hadoop","password":"hadoop"}
[INFO] 2020-06-03 15:03:54.571 - [taskAppId=TASK-8-133-287]:[214] - SQL
tile : test
[INFO] 2020-06-03 15:03:54.572 - [taskAppId=TASK-8-133-287]:[448] - after
replace sql , preparing : select name,id,idcardcheck(id) as result from
ods_people
[INFO] 2020-06-03 15:03:54.572 - [taskAppId=TASK-8-133-287]:[453] -
replaced sql , parameters:
[INFO] 2020-06-03 15:03:54.904 - [taskAppId=TASK-8-133-287]:[364] - prepare
statement replace sql : org.apache.hive.jdbc.HivePreparedStatement@67d16b19
[ERROR] 2020-06-03 15:03:54.977 - [taskAppId=TASK-8-133-287]:[336] - Error
while compiling statement: FAILED: SemanticException [Error 10011]: Line 1:15
Invalid function 'idcardcheck'
org.apache.hive.service.cli.HiveSQLException: Error while compiling
statement: FAILED: SemanticException [Error 10011]: Line 1:15 Invalid function
'idcardcheck'
at org.apache.hive.jdbc.Utils.verifySuccess(Utils.java:264)
at org.apache.hive.jdbc.Utils.verifySuccessWithInfo(Utils.java:250)
at
org.apache.hive.jdbc.HiveStatement.runAsyncOnServer(HiveStatement.java:309)
at org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:250)
at
org.apache.hive.jdbc.HiveStatement.executeQuery(HiveStatement.java:434)
at
org.apache.hive.jdbc.HivePreparedStatement.executeQuery(HivePreparedStatement.java:109)
at
org.apache.dolphinscheduler.server.worker.task.sql.SqlTask.executeFuncAndSql(SqlTask.java:295)
at
org.apache.dolphinscheduler.server.worker.task.sql.SqlTask.handle(SqlTask.java:176)
at
org.apache.dolphinscheduler.server.worker.runner.TaskScheduleThread.run(TaskScheduleThread.java:142)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.hive.service.cli.HiveSQLException: Error while
compiling statement: FAILED: SemanticException [Error 10011]: Line 1:15 Invalid
function 'idcardcheck'
at
org.apache.hive.service.cli.operation.Operation.toSQLException(Operation.java:315)
at
org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:112)
at
org.apache.hive.service.cli.operation.SQLOperation.runInternal(SQLOperation.java:181)
at
org.apache.hive.service.cli.operation.Operation.run(Operation.java:257)
at
org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(HiveSessionImpl.java:388)
at
org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementAsync(HiveSessionImpl.java:375)
at sun.reflect.GeneratedMethodAccessor51.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:78)
at
org.apache.hive.service.cli.session.HiveSessionProxy.access$000(HiveSessionProxy.java:36)
at
org.apache.hive.service.cli.session.HiveSessionProxy$1.run(HiveSessionProxy.java:63)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1762)
at
org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:59)
at com.sun.proxy.$Proxy20.executeStatementAsync(Unknown Source)
at
org.apache.hive.service.cli.CLIService.executeStatementAsync(CLIService.java:274)
at
org.apache.hive.service.cli.thrift.ThriftCLIService.ExecuteStatement(ThriftCLIService.java:486)
at
org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1313)
at
org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1298)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
at
org.apache.hive.service.auth.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:56)
at
org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:285)
... 3 common frames omitted
Caused by: java.lang.RuntimeException:
org.apache.hadoop.hive.ql.parse.SemanticException:Line 1:15 Invalid function
'idcardcheck'
at
org.apache.hadoop.hive.ql.parse.TypeCheckProcFactory$DefaultExprProcessor.getXpathOrFuncExprNodeDesc(TypeCheckProcFactory.java:925)
at
org.apache.hadoop.hive.ql.parse.TypeCheckProcFactory$DefaultExprProcessor.process(TypeCheckProcFactory.java:1265)
at
org.apache.hadoop.hive.ql.lib.DefaultRuleDispatcher.dispatch(DefaultRuleDispatcher.java:90)
at
org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatchAndReturn(DefaultGraphWalker.java:95)
at
org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatch(DefaultGraphWalker.java:79)
at
org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.walk(DefaultGraphWalker.java:133)
at
org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.startWalking(DefaultGraphWalker.java:110)
at
org.apache.hadoop.hive.ql.parse.TypeCheckProcFactory.genExprNode(TypeCheckProcFactory.java:205)
at
org.apache.hadoop.hive.ql.parse.TypeCheckProcFactory.genExprNode(TypeCheckProcFactory.java:149)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genAllExprNodeDesc(SemanticAnalyzer.java:10512)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genExprNodeDesc(SemanticAnalyzer.java:10468)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genSelectPlan(SemanticAnalyzer.java:3840)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genSelectPlan(SemanticAnalyzer.java:3619)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPostGroupByBodyPlan(SemanticAnalyzer.java:8956)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genBodyPlan(SemanticAnalyzer.java:8911)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:9756)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:9649)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genOPTree(SemanticAnalyzer.java:10122)
at
org.apache.hadoop.hive.ql.parse.CalcitePlanner.genOPTree(CalcitePlanner.java:325)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:10133)
at
org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:209)
at
org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:227)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:424)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:308)
at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1122)
at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1116)
at
org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:110)
... 26 common frames omitted
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
