[
https://issues.apache.org/jira/browse/SPARK-26602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Haripriya updated SPARK-26602:
------------------------------
Description:
In sql,
1.Query the existing udf(say myFunc1)
2. create and select the udf registered with incorrect path (say myFunc2)
3.Now again query the existing udf in the same session - Wil throw exception
stating that couldn't read resource of myFunc2's path
4.Even the basic operations like insert and select will fail giving the same
error
Result:
java.lang.RuntimeException: Failed to read external resource
hdfs:///tmp/hari_notexists1/two_udfs.jar
at
org.apache.hadoop.hive.ql.session.SessionState.downloadResource(SessionState.java:1288)
at
org.apache.hadoop.hive.ql.session.SessionState.resolveAndDownload(SessionState.java:1242)
at
org.apache.hadoop.hive.ql.session.SessionState.add_resources(SessionState.java:1163)
at
org.apache.hadoop.hive.ql.session.SessionState.add_resources(SessionState.java:1149)
at
org.apache.hadoop.hive.ql.processors.AddResourceProcessor.run(AddResourceProcessor.java:67)
at
org.apache.spark.sql.hive.client.HiveClientImpl$$anonfun$runHive$1.apply(HiveClientImpl.scala:737)
at
org.apache.spark.sql.hive.client.HiveClientImpl$$anonfun$runHive$1.apply(HiveClientImpl.scala:706)
at
org.apache.spark.sql.hive.client.HiveClientImpl$$anonfun$withHiveState$1.apply(HiveClientImpl.scala:275)
at
org.apache.spark.sql.hive.client.HiveClientImpl.liftedTree1$1(HiveClientImpl.scala:213)
at
org.apache.spark.sql.hive.client.HiveClientImpl.retryLocked(HiveClientImpl.scala:212)
at
org.apache.spark.sql.hive.client.HiveClientImpl.withHiveState(HiveClientImpl.scala:258)
at
org.apache.spark.sql.hive.client.HiveClientImpl.runHive(HiveClientImpl.scala:706)
at
org.apache.spark.sql.hive.client.HiveClientImpl.runSqlHive(HiveClientImpl.scala:696)
at
org.apache.spark.sql.hive.client.HiveClientImpl.addJar(HiveClientImpl.scala:841)
at
org.apache.spark.sql.hive.HiveSessionResourceLoader.addJar(HiveSessionStateBuilder.scala:112)
was:
In sql,
1.Query the existing udf(say myFunc1)
2. create and select the udf registered with incorrect path (say myFunc2)
3.Now again query the existing udf in the same session - Wil throw exception
stating that couldn't read resource of myFunc2's path
Result:
java.lang.RuntimeException: Failed to read external resource
hdfs:///tmp/hari_notexists1/two_udfs.jar
at
org.apache.hadoop.hive.ql.session.SessionState.downloadResource(SessionState.java:1288)
at
org.apache.hadoop.hive.ql.session.SessionState.resolveAndDownload(SessionState.java:1242)
at
org.apache.hadoop.hive.ql.session.SessionState.add_resources(SessionState.java:1163)
at
org.apache.hadoop.hive.ql.session.SessionState.add_resources(SessionState.java:1149)
at
org.apache.hadoop.hive.ql.processors.AddResourceProcessor.run(AddResourceProcessor.java:67)
at
org.apache.spark.sql.hive.client.HiveClientImpl$$anonfun$runHive$1.apply(HiveClientImpl.scala:737)
at
org.apache.spark.sql.hive.client.HiveClientImpl$$anonfun$runHive$1.apply(HiveClientImpl.scala:706)
at
org.apache.spark.sql.hive.client.HiveClientImpl$$anonfun$withHiveState$1.apply(HiveClientImpl.scala:275)
at
org.apache.spark.sql.hive.client.HiveClientImpl.liftedTree1$1(HiveClientImpl.scala:213)
at
org.apache.spark.sql.hive.client.HiveClientImpl.retryLocked(HiveClientImpl.scala:212)
at
org.apache.spark.sql.hive.client.HiveClientImpl.withHiveState(HiveClientImpl.scala:258)
at
org.apache.spark.sql.hive.client.HiveClientImpl.runHive(HiveClientImpl.scala:706)
at
org.apache.spark.sql.hive.client.HiveClientImpl.runSqlHive(HiveClientImpl.scala:696)
at
org.apache.spark.sql.hive.client.HiveClientImpl.addJar(HiveClientImpl.scala:841)
at
org.apache.spark.sql.hive.HiveSessionResourceLoader.addJar(HiveSessionStateBuilder.scala:112)
Summary: Once creating and quering udf with incorrect path,followed by
querying tables or functions registered with correct path gives the runtime
exception within the same session (was: Once creating and quering udf with
incorrect path,even the functions registered with correct path follows the same
incorrect path in that session)
> Once creating and quering udf with incorrect path,followed by querying tables
> or functions registered with correct path gives the runtime exception within
> the same session
> ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: SPARK-26602
> URL: https://issues.apache.org/jira/browse/SPARK-26602
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 2.4.0
> Reporter: Haripriya
> Priority: Major
>
> In sql,
> 1.Query the existing udf(say myFunc1)
> 2. create and select the udf registered with incorrect path (say myFunc2)
> 3.Now again query the existing udf in the same session - Wil throw exception
> stating that couldn't read resource of myFunc2's path
> 4.Even the basic operations like insert and select will fail giving the same
> error
> Result:
> java.lang.RuntimeException: Failed to read external resource
> hdfs:///tmp/hari_notexists1/two_udfs.jar
> at
> org.apache.hadoop.hive.ql.session.SessionState.downloadResource(SessionState.java:1288)
> at
> org.apache.hadoop.hive.ql.session.SessionState.resolveAndDownload(SessionState.java:1242)
> at
> org.apache.hadoop.hive.ql.session.SessionState.add_resources(SessionState.java:1163)
> at
> org.apache.hadoop.hive.ql.session.SessionState.add_resources(SessionState.java:1149)
> at
> org.apache.hadoop.hive.ql.processors.AddResourceProcessor.run(AddResourceProcessor.java:67)
> at
> org.apache.spark.sql.hive.client.HiveClientImpl$$anonfun$runHive$1.apply(HiveClientImpl.scala:737)
> at
> org.apache.spark.sql.hive.client.HiveClientImpl$$anonfun$runHive$1.apply(HiveClientImpl.scala:706)
> at
> org.apache.spark.sql.hive.client.HiveClientImpl$$anonfun$withHiveState$1.apply(HiveClientImpl.scala:275)
> at
> org.apache.spark.sql.hive.client.HiveClientImpl.liftedTree1$1(HiveClientImpl.scala:213)
> at
> org.apache.spark.sql.hive.client.HiveClientImpl.retryLocked(HiveClientImpl.scala:212)
> at
> org.apache.spark.sql.hive.client.HiveClientImpl.withHiveState(HiveClientImpl.scala:258)
> at
> org.apache.spark.sql.hive.client.HiveClientImpl.runHive(HiveClientImpl.scala:706)
> at
> org.apache.spark.sql.hive.client.HiveClientImpl.runSqlHive(HiveClientImpl.scala:696)
> at
> org.apache.spark.sql.hive.client.HiveClientImpl.addJar(HiveClientImpl.scala:841)
> at
> org.apache.spark.sql.hive.HiveSessionResourceLoader.addJar(HiveSessionStateBuilder.scala:112)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
