[
https://issues.apache.org/jira/browse/HIVE-22527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
zhangbutao updated HIVE-22527:
------------------------------
Description:
Hive on Tez. We enable small file merge configuration with set
*hive.merge.tezfiles=true*. So , There will be another job launched for merging
files after sql job. However, the merge file job is submitted into another yarn
queue, not the queue of current beeline client session. It seems that the
merging files job start a new tez session with new conf which is different the
current session conf, leading to the merging file job goes into default queue.
Attachment *hive logs.png* shows that current session queue is
*root.bdoc.production* String queueName = session.getQueueName();
was:
Hive on Tez. We enable small file merge configuration with set
*hive.merge.tezfiles=true*. So , There will be another job launched for merging
files after sql job. However, the merge file job is submitted into another yarn
queue, not the queue of current beeline client session. It seems that the
merging files job start a new tez session with new conf which is different the
current session conf, leading to the merging file job goes into default queue.
> Hive on Tez : Job of merging samll files will be submitted into another queue
> (default queue)
> ---------------------------------------------------------------------------------------------
>
> Key: HIVE-22527
> URL: https://issues.apache.org/jira/browse/HIVE-22527
> Project: Hive
> Issue Type: Bug
> Affects Versions: 3.1.0, 3.1.1
> Reporter: zhangbutao
> Priority: Blocker
> Attachments: explain with merge files.png, hive logs.png
>
>
> Hive on Tez. We enable small file merge configuration with set
> *hive.merge.tezfiles=true*. So , There will be another job launched for
> merging files after sql job. However, the merge file job is submitted into
> another yarn queue, not the queue of current beeline client session. It seems
> that the merging files job start a new tez session with new conf which is
> different the current session conf, leading to the merging file job goes into
> default queue.
>
> Attachment *hive logs.png* shows that current session queue is
> *root.bdoc.production* String queueName = session.getQueueName();
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
