[
https://issues.apache.org/jira/browse/HIVE-25685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HIVE-25685:
----------------------------------
Labels: pull-request-available (was: )
> HBaseStorageHandler: ensure that hbase properties are present in final
> JobConf for Tez
> --------------------------------------------------------------------------------------
>
> Key: HIVE-25685
> URL: https://issues.apache.org/jira/browse/HIVE-25685
> Project: Hive
> Issue Type: Bug
> Reporter: László Bodor
> Assignee: László Bodor
> Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> After some debugging I found that in the following codepath configureJobConf
> is called to prepare the jobConf object for special storage handlers,
> however, if the jobConf hasn't contained the needed hbase props, then this
> code path doesn't make sure of adding that, even if the method name suggests.
>
> {code:java}
> HBaseConfiguration.addHbaseResources(Configuration) line: 82
> HBaseConfiguration.create() line: 98
> HBaseConfiguration.create(Configuration) line: 107
> HBaseStorageHandler.setConf(Configuration) line: 134
> ReflectionUtils.setConf(Object, Configuration) line: 77
> ReflectionUtils.newInstance(Class<T>, Configuration) line: 137
> HiveUtils.getStorageHandler(Configuration, String) line: 299
> PlanUtils.configureJobConf(TableDesc, JobConf) line: 995
> MapWork.configureJobConf(JobConf) line: 661
> TezWork.configureJobConfAndExtractJars(JobConf) line: 346
> TezTask.execute() line: 186
> TezTask(Task<T>).executeTask(HiveHistory) line: 213
> {code}
> Here, addHBaseResources adds hbase xml configs to
> HBaseStorageHandler.hbaseConf during the reflection flow, but then they're
> not added to the passed jobConf in configureJobConf.
> This can cause problems in secure clusters under some circumstances, where
> hbase props are not present in the conf which is used in TezTask.execute.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
