[
https://issues.apache.org/jira/browse/HIVE-30211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18139803#comment-18139803
]
Raghav Aggarwal commented on HIVE-30211:
----------------------------------------
[~krisztiankasa] , Thanks a lot for taking a look and for great observations.
Even I missed few details initially like HIVE-12377 which is same issue as I
reported and after your comments I checked and found :)
Let me answer to best of my understanding:
1. Are generated names like col_0, col_1 ... col_n the only names passed here?
> Yes, after more checking I found that all places in Hive i.e.
> {_}SemanticAnalyzer{_}, _HiveTableFunctionScanVisitor_ and _UDTFOperator_
> already call {{genericUDTF.initialize(structOI)}} and passes the generated
> internal names (_col0, _{_}col1). *HiveFunctionHelper*{_} *was the only place
> calling the deprecated array method call.*
The Standard internal column names are centrally defined in
[HiveConf|https://github.com/apache/hive/blob/f6e3e77f2b70edb1adacbb9122b1e95e009cbb26/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java#L7007-L7009]
used in aforementioned 3 classes
{code:java}
public static String getColumnInternalName(int pos) {
return "_col" + pos;
}
{code}
*I'll update PR to use HiveConf.getColumnInternalName(pos) instead of explicit
"col_" + pos to be consistent.*
2. to un-deprecate the old initialize method and deprecate the new one instead.
> I think it will be problematic. UDTF like explode were written before
> HIVE-5737 that's why they use old API and it will be a big breaking change
> for exising user 3-rd party UDTF's. Please check the comment under HIVE-12377
> hoping it helps.
{*}In summary{*}, {{initialize(StructObjectInspector)}} automatically falls
back to the old one for legacy UDTFs whereas
calling the old method breaks newer UDTFs that only implements the new API.
> Fix IllegalStateException in Calcite HiveFunctionHelper for GenericUDTF
> using StructObjectInspector
> ----------------------------------------------------------------------------------------------------
>
> Key: HIVE-30211
> URL: https://issues.apache.org/jira/browse/HIVE-30211
> Project: Hive
> Issue Type: Improvement
> Reporter: Raghav Aggarwal
> Assignee: Raghav Aggarwal
> Priority: Major
> Labels: pull-request-available
>
> GenericUDTF explicitly deprecates the array-based initialization method in
> favor of the struct-based overload:
> {code:java}
> public StructObjectInspector initialize(StructObjectInspector argOIs) throws
> UDFArgumentException {
> ...
> }
> @Deprecated
> public StructObjectInspector initialize(ObjectInspector[] argOIs) throws
> UDFArgumentException {
> throw new IllegalStateException("Should not be called directly");
> }
> {code}
> However, during Calcite query compilation,
> {{HiveFunctionHelper.getReturnType()}} directly calls the deprecated array
> overload, resulting in {{java.lang.IllegalStateException: Should not be
> called directly}}
>
> This bug forces all UDTF implementations to implement the deprecated method,
> which violates the API contract and creates issues for:
> 1. New native UDTFs (e.g., Iceberg snapshot lineage functions like
> iceberg_ancestors_of).
> 2. 3rd-party / custom user UDTFs (e.g., external jars loaded dynamically
> via ADD JAR in Beeline).
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
