Cheng Hao created SPARK-3904:
--------------------------------
Summary: HQL doesn't support the ConstantObjectInspector to pass
into UDFs
Key: SPARK-3904
URL: https://issues.apache.org/jira/browse/SPARK-3904
Project: Spark
Issue Type: Bug
Components: SQL
Reporter: Cheng Hao
In HQL, we convert all of the data type into normal ObjectInspectors for UDFs,
most of cases it work, however, some of the UDF actually requires the input
ObjectInspector to be the ConstantObjectInspector, which will cause exception.
e.g.
{panel}
select named_struct("x", "str") from src limit 1
{panel}
It will throws exception like
{panel}
14/10/10 16:25:17 INFO parse.ParseDriver: Parsing command: select
named_struct("x", "str") from src
14/10/10 16:25:17 INFO parse.ParseDriver: Parse Completed
14/10/10 16:25:17 INFO metastore.HiveMetaStore: 0: get_table : db=default
tbl=src
14/10/10 16:25:17 INFO HiveMetaStore.audit: ugi=hcheng ip=unknown-ip-addr
cmd=get_table : db=default tbl=tmp2
14/10/10 16:25:17 ERROR thriftserver.SparkSQLDriver: Failed in [select
named_struct("x", "str") from src]
org.apache.hadoop.hive.ql.exec.UDFArgumentTypeException: Even arguments to
NAMED_STRUCT must be a constant
STRING.org.apache.hadoop.hive.serde2.objectinspector.primitive.JavaStringObjectInspector@2f2dbcfc
at
org.apache.hadoop.hive.ql.udf.generic.GenericUDFNamedStruct.initialize(GenericUDFNamedStruct.java:55)
at
org.apache.spark.sql.hive.HiveGenericUdf.returnInspector$lzycompute(hiveUdfs.scala:129)
at
org.apache.spark.sql.hive.HiveGenericUdf.returnInspector(hiveUdfs.scala:129)
at org.apache.spark.sql.hive.HiveGenericUdf.eval(hiveUdfs.scala:158)
at
org.apache.spark.sql.catalyst.optimizer.ConstantFolding$$anonfun$apply$6$$anonfun$applyOrElse$2.applyOrElse(Optimizer.scala:267)
at
org.apache.spark.sql.catalyst.optimizer.ConstantFolding$$anonfun$apply$6$$anonfun$applyOrElse$2.applyOrElse(Optimizer.scala:260)
{panel}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
