[
https://issues.apache.org/jira/browse/SPARK-33704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yuming Wang resolved SPARK-33704.
---------------------------------
Resolution: Duplicate
> Support latest version of initialize() in HiveGenericUDTF
> ---------------------------------------------------------
>
> Key: SPARK-33704
> URL: https://issues.apache.org/jira/browse/SPARK-33704
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 2.2.0, 2.3.4, 2.4.3
> Reporter: chenliang
> Priority: Major
>
> For HiveGenericUDTF , there are two initialization methods:
> {code:java}
> public StructObjectInspector initialize(StructObjectInspector argOIs)
> throws UDFArgumentException {
> List<? extends StructField> inputFields = argOIs.getAllStructFieldRefs();
> ObjectInspector[] udtfInputOIs = new ObjectInspector[inputFields.size()];
> for (int i = 0; i < inputFields.size(); i++) {
> udtfInputOIs[i] = inputFields.get(i).getFieldObjectInspector();
> }
> return initialize(udtfInputOIs);
> }
> @Deprecated
> public StructObjectInspector initialize(ObjectInspector[] argOIs)
> throws UDFArgumentException {
> throw new IllegalStateException("Should not be called directly");
> }
> {code}
> As https://issues.apache.org/jira/browse/HIVE-5737 mentioned, hive provided
> StructObjectInspector for UDTFs rather than ObjectInspect[], but Spark SQL
> still only support deprecated function.
> An exception will be reported before fix:
> Error in query: No handler for UDF/UDAF/UDTF 'FeatureParseUDTF1':
> java.lang.IllegalStateException: Should not be called directly
> Please make sure your function overrides public StructObjectInspector
> initialize(ObjectInspector[] args).; line 1 pos 7
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
