[
https://issues.apache.org/jira/browse/SPARK-28672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16911937#comment-16911937
]
ABHISHEK KUMAR GUPTA commented on SPARK-28672:
----------------------------------------------
Thanks for update.
This JIRA is for not allowing duplicate function name.
Hive works as below:
It does not list the permanent function
: jdbc:hive2://10.18.98.147:21066/> create function mul3 AS
'com.huawei.bigdata.hive.example.udf.multiply' using jar
'hdfs://hacluster/user/Multiply.jar';
INFO : Compiling
command(queryId=omm_20190821115530_03819dba-4c28-46c9-92a6-461cc2762f94):
create function mul3 AS 'com.huawei.bigdata.hive.example.udf.multiply' using
jar 'hdfs://hacluster/user/Multiply.jar'--0; Current
sessionId=8d2e1845-5254-4021-935e-4e1beb484a72
INFO : Concurrency mode is disabled, not creating a lock manager
INFO : Semantic Analysis Completed (retrial = false)
INFO : Returning Hive schema: Schema(fieldSchemas:null, properties:null)
INFO : Completed compiling
command(queryId=omm_20190821115530_03819dba-4c28-46c9-92a6-461cc2762f94); Time
taken: 0.699 seconds
INFO : Concurrency mode is disabled, not creating a lock manager
INFO : Executing
command(queryId=omm_20190821115530_03819dba-4c28-46c9-92a6-461cc2762f94):
create function mul3 AS 'com.huawei.bigdata.hive.example.udf.multiply' using
jar 'hdfs://hacluster/user/Multiply.jar'--0; Current
sessionId=8d2e1845-5254-4021-935e-4e1beb484a72
INFO : Starting task [Stage-0:FUNC] in serial mode
INFO : Added
[/opt/huawei/Bigdata/tmp/hivelocaltmp/session_resources/8d2e1845-5254-4021-935e-4e1beb484a72_resources/Multiply.jar]
to class path
INFO : Added resources: [hdfs://hacluster/user/Multiply.jar]
INFO : Completed executing
command(queryId=omm_20190821115530_03819dba-4c28-46c9-92a6-461cc2762f94); Time
taken: 0.043 seconds
INFO : OK
INFO : Concurrency mode is disabled, not creating a lock manager
No rows affected (0.785 seconds)
0: jdbc:hive2://10.18.98.147:21066/> create temporary function mul3 AS
'com.huawei.bigdata.hive.example.udf.multiply' using jar
'hdfs://hacluster/user/Multiply.jar';
INFO : Compiling
command(queryId=omm_20190821115600_26c3076d-857b-45d8-aef2-00118edbb14e):
create temporary function mul3 AS
'com.huawei.bigdata.hive.example.udf.multiply' using jar
'hdfs://hacluster/user/Multiply.jar'--0; Current
sessionId=8d2e1845-5254-4021-935e-4e1beb484a72
INFO : Concurrency mode is disabled, not creating a lock manager
INFO : Semantic Analysis Completed (retrial = false)
INFO : Returning Hive schema: Schema(fieldSchemas:null, properties:null)
INFO : Completed compiling
command(queryId=omm_20190821115600_26c3076d-857b-45d8-aef2-00118edbb14e); Time
taken: 0.754 seconds
INFO : Concurrency mode is disabled, not creating a lock manager
INFO : Executing
command(queryId=omm_20190821115600_26c3076d-857b-45d8-aef2-00118edbb14e):
create temporary function mul3 AS
'com.huawei.bigdata.hive.example.udf.multiply' using jar
'hdfs://hacluster/user/Multiply.jar'--0; Current
sessionId=8d2e1845-5254-4021-935e-4e1beb484a72
INFO : Starting task [Stage-0:FUNC] in serial mode
INFO : Added
[/opt/huawei/Bigdata/tmp/hivelocaltmp/session_resources/8d2e1845-5254-4021-935e-4e1beb484a72_resources/Multiply.jar]
to class path
INFO : Added resources: [hdfs://hacluster/user/Multiply.jar]
INFO : Completed executing
command(queryId=omm_20190821115600_26c3076d-857b-45d8-aef2-00118edbb14e); Time
taken: 0.004 seconds
INFO : OK
INFO : Concurrency mode is disabled, not creating a lock manager
No rows affected (0.834 seconds)
0: jdbc:hive2://10.18.98.147:21066/> show functions like mul3;
INFO : Compiling
command(queryId=omm_20190821115614_3ae7fdd3-04da-4d7b-8fdd-4a50e22491ca): show
functions like mul3--0; Current sessionId=8d2e1845-5254-4021-935e-4e1beb484a72
INFO : Concurrency mode is disabled, not creating a lock manager
INFO : Semantic Analysis Completed (retrial = false)
INFO : Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:tab_name,
type:string, comment:from deserializer)], properties:null)
INFO : Completed compiling
command(queryId=omm_20190821115614_3ae7fdd3-04da-4d7b-8fdd-4a50e22491ca); Time
taken: 0.136 seconds
INFO : Concurrency mode is disabled, not creating a lock manager
INFO : Executing
command(queryId=omm_20190821115614_3ae7fdd3-04da-4d7b-8fdd-4a50e22491ca): show
functions like mul3--0; Current sessionId=8d2e1845-5254-4021-935e-4e1beb484a72
INFO : Starting task [Stage-0:DDL] in serial mode
INFO : Completed executing
command(queryId=omm_20190821115614_3ae7fdd3-04da-4d7b-8fdd-4a50e22491ca); Time
taken: 0.003 seconds
INFO : OK
INFO : Concurrency mode is disabled, not creating a lock manager
+-----------+
| tab_name |
+-----------+
| mul3 |
+-----------+
Now my question is if user creates both permanent and temporary function with
same name then in current session permanent function get overriden by temporary
function so end user should be aware with the creation of function with the
same name.
So solution should be ( My suggestion )
1. Not allowed duplicate function name whether it is temporary or permanent.
Or
2. Give end user msg saying " Permanent function already exist with the same
name if user creates then it will be Overriden with temporary function"
> [UDF] Duplicate function creation should not allow
> ---------------------------------------------------
>
> Key: SPARK-28672
> URL: https://issues.apache.org/jira/browse/SPARK-28672
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 2.4.0
> Reporter: ABHISHEK KUMAR GUPTA
> Priority: Minor
>
> {code}
> 0: jdbc:hive2://10.18.18.214:23040/default> create function addm_3 AS
> 'com.huawei.bigdata.hive.example.udf.multiply' using jar
> 'hdfs://hacluster/user/Multiply.jar';
> +---------+--+
> | Result |
> +---------+--+
> +---------+--+
> No rows selected (0.084 seconds)
> {code}
> {code}
> 0: jdbc:hive2://10.18.18.214:23040/default> create temporary function addm_3
> AS 'com.huawei.bigdata.hive.example.udf.multiply' using jar
> 'hdfs://hacluster/user/Multiply.jar';
> INFO : converting to local hdfs://hacluster/user/Multiply.jar
> INFO : Added
> [/tmp/8a396308-41f8-4335-9de4-8268ce5c70fe_resources/Multiply.jar] to class
> path
> INFO : Added resources: [hdfs://hacluster/user/Multiply.jar]
> +---------+--+
> | Result |
> +---------+--+
> +---------+--+
> No rows selected (0.134 seconds)
> {code}
> {code}
> 0: jdbc:hive2://10.18.18.214:23040/default> show functions like addm_3;
> +-----------------+--+
> | function |
> +-----------------+--+
> | addm_3 |
> | default.addm_3 |
> +-----------------+--+
> 2 rows selected (0.047 seconds)
> {code}
> When show function executed it is listing both the function but what about
> the db for permanent function when user has not specified.
> Duplicate should not be allowed if user creating temporary one with the same
> name.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
