[
https://issues.apache.org/jira/browse/HIVE-28831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17937076#comment-17937076
]
Anmol Sundaram edited comment on HIVE-28831 at 3/20/25 9:59 AM:
----------------------------------------------------------------
Hi guys, Could some one please help review this simple PR.
The philosophy of the change is similar to what is being done with
[OrcEncodedDataReader|https://github.com/apache/hive/blob/master/llap-server/src/java/org/apache/hadoop/hive/llap/io/encoded/OrcEncodedDataReader.java#L241]
The change is to simply use jobconf for initializing the FileSystem in
_SerDeEncodedDataReader_ and _VectorDeserializeOrcWriter_ as well.
was (Author: JIRAUSER288438):
Hi guys, Could some one please help review this simple PR.
The philosophy of the change is similar to what is being done with
[OrcEncodedDataReader|https://github.com/apache/hive/blob/master/llap-server/src/java/org/apache/hadoop/hive/llap/io/encoded/OrcEncodedDataReader.java#L241]
The change is to simply use jobconf for initializing the FileSystem in
_SerDeEncodedDataReader_ and _VectorDeserializeOrcWriter_ as well.
> FS Initialisation for SerDeEncodedDataReader & VectorDeserializeOrcWriter
> should be based on jobConf
> ----------------------------------------------------------------------------------------------------
>
> Key: HIVE-28831
> URL: https://issues.apache.org/jira/browse/HIVE-28831
> Project: Hive
> Issue Type: Bug
> Components: llap
> Affects Versions: 4.1.0
> Environment: Passing configs by setting them in Hive code, when using
> LLAP IO, these configs are not being honored by the FS being initialized by
> {_}SerDeEncodedDataReader{_}. This is because _SerDeEncodedDataReader_ uses
> LLAP IO's daemon config, rather than the jobConf (which typically contains
> custom set of configs as well).
> Reporter: Anmol Sundaram
> Priority: Major
> Labels: PatchAvailable
>
> Similar to what is currently being done in {_}OrcEncodedDataReader{_}, FS
> Initialisation for _SerDeEncodedDataReader_ & _VectorDeserializeOrcWriter_
> should be based on jobConf rather than {_}daemonConf{_}.
> This will ensure that the FileSystems are being initiated with the
> configurations being passed as part of _jobConf_ which could be different on
> a Job basis instead of LLAP IO's {_}daemonConf{_}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
