[
https://issues.apache.org/jira/browse/HIVE-24645?focusedWorklogId=542981&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-542981
]
ASF GitHub Bot logged work on HIVE-24645:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 27/Jan/21 18:21
Start Date: 27/Jan/21 18:21
Worklog Time Spent: 10m
Work Description: jfsii commented on a change in pull request #1876:
URL: https://github.com/apache/hive/pull/1876#discussion_r565532925
##########
File path:
ql/src/java/org/apache/hadoop/hive/ql/exec/ExprNodeGenericFuncEvaluator.java
##########
@@ -140,8 +141,23 @@ public ObjectInspector initialize(ObjectInspector
rowInspector) throws HiveExcep
childrenOIs[i] = children[i].initialize(rowInspector);
}
MapredContext context = MapredContext.get();
+ // It is possible that there is no context at this point. For example a
context is not created
+ // when "hive.fetch.task.conversion" occurs.
if (context != null) {
context.setup(genericUDF);
+ } else {
+ // It is a bit unfortunate that currently the UDF configuration
signature expects a
+ // MapredContext (even if execution is tez or another engine) - this
causes an
+ // impedence mismatch. For example: MapredContext has Reporter objects
that may or
+ // may not make sense for the current engine.
+ //
+ // We attempt to create a dummyContext that has at least access to the
currently set
+ // configuration. The other unfortunate issue is that some paths set the
configuration
+ // when creating ExprNodeGenericFuncEvaluator while others do not, so we
fallback to a
+ // "default" new HiveConf object which might be missing configuration
that changed during
+ // runtime.
+ MapredContext dummyContext = MapredContext.createDummy(getConf() != null
? getConf() : new HiveConf());
Review comment:
Thanks, I'll give that a try since a fully formed one is better (also I
think new HiveConf is slightly expensive looking at all the stuff the
construction takes).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 542981)
Time Spent: 1h 10m (was: 1h)
> UDF configure not called when fetch task conversion occurs
> ----------------------------------------------------------
>
> Key: HIVE-24645
> URL: https://issues.apache.org/jira/browse/HIVE-24645
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
> Reporter: John Sherman
> Assignee: John Sherman
> Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> When hive.fetch.task.conversion kicks in - UDF configure is not called.
> This is likely due to MapredContext not being available when this conversion
> occurs.
> The approach I suggest is to create a dummy MapredContext and provide it with
> the current configuration from ExprNodeGenericFuncEvaluator.
> It is slightly unfortunate that the UDF API relies on MapredContext since
> some aspects of the context do not apply to the variety of engines and
> invocation paths for UDFs which makes it difficult to make a fully formed
> dummy object such as the Reporter objects and the boolean around if it is a
> Map context.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
