[
https://issues.apache.org/jira/browse/HIVE-23410?focusedWorklogId=521750&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-521750
]
ASF GitHub Bot logged work on HIVE-23410:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 08/Dec/20 15:48
Start Date: 08/Dec/20 15:48
Worklog Time Spent: 10m
Work Description: kuczoram commented on a change in pull request #1660:
URL: https://github.com/apache/hive/pull/1660#discussion_r538520768
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMapRedUtils.java
##########
@@ -1895,7 +1901,20 @@ public static boolean isSkewedStoredAsDirs(FileSinkDesc
fsInputDesc) {
}
if ((srcDir != null) && srcDir.equals(fsopFinalDir)) {
- return mvTsk;
+ if (isDirectInsert || isMmFsop) {
+ if (moveTaskId != null && fsoMoveTaskId != null &&
moveTaskId.equals(fsoMoveTaskId)) {
+ // If the ACID direct insert is on, the MoveTasks cannot be
identified by the srcDir as
+ // in this case the srcDir is always the root directory of the
table.
+ // We need to consider the ACID write type to identify the
MoveTasks.
+ return mvTsk;
+ }
+ if ((moveTaskId == null || fsoMoveTaskId == null) &&
moveTaskWriteType != null
Review comment:
There was a test which was failing if this was not there, but since then
I think I fixed the moveTaskId generation, so cannot be null. It think this is
not needed. I will remove it and let's see what the tests say.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 521750)
Time Spent: 3h 20m (was: 3h 10m)
> ACID: Improve the delete and update operations to avoid the move step
> ---------------------------------------------------------------------
>
> Key: HIVE-23410
> URL: https://issues.apache.org/jira/browse/HIVE-23410
> Project: Hive
> Issue Type: Improvement
> Affects Versions: 4.0.0
> Reporter: Marta Kuczora
> Assignee: Marta Kuczora
> Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-23410.1.patch
>
> Time Spent: 3h 20m
> Remaining Estimate: 0h
>
> This is a follow-up task for
> [HIVE-21164|https://issues.apache.org/jira/browse/HIVE-21164], where the
> insert operation has been modified to write directly to the table locations
> instead of the staging directory. The same improvement should be done for the
> ACID update and delete operations as well.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
