[
https://issues.apache.org/jira/browse/HIVE-21100?focusedWorklogId=717576&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-717576
]
ASF GitHub Bot logged work on HIVE-21100:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 29/Jan/22 19:00
Start Date: 29/Jan/22 19:00
Worklog Time Spent: 10m
Work Description: hsnusonic commented on pull request #2921:
URL: https://github.com/apache/hive/pull/2921#issuecomment-1024967920
Thanks @pvary for reviewing and letting me understand the performance
impact. I did much more testing and found a workaround, which is `set
hive.exec.dynamic.partition.mode = nonstrict;`. We can get rid of
HIVE_UNION_SUBDIR with this config because the query plan has one more reducer
that doesn't use HIVE_UNION_SUBDIR. In sum, now that we have a workaround, I
will close this PR and also resolve the ticket.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 717576)
Time Spent: 2h 20m (was: 2h 10m)
> Allow flattening of table subdirectories resulted when using TEZ engine and
> UNION clause
> ----------------------------------------------------------------------------------------
>
> Key: HIVE-21100
> URL: https://issues.apache.org/jira/browse/HIVE-21100
> Project: Hive
> Issue Type: Improvement
> Reporter: George Pachitariu
> Assignee: George Pachitariu
> Priority: Minor
> Labels: pull-request-available
> Attachments: HIVE-21100.1.patch, HIVE-21100.2.patch,
> HIVE-21100.3.patch, HIVE-21100.patch
>
> Time Spent: 2h 20m
> Remaining Estimate: 0h
>
> Right now, when writing data into a table with Tez engine and the clause
> UNION ALL is the last step of the query, Hive on Tez will create a
> subdirectory for each branch of the UNION ALL.
> With this patch the subdirectories are removed, and the files are renamed and
> moved to the parent directory.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
