[
https://issues.apache.org/jira/browse/HIVE-20517?focusedWorklogId=442744&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-442744
]
ASF GitHub Bot logged work on HIVE-20517:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 09/Jun/20 15:53
Start Date: 09/Jun/20 15:53
Worklog Time Spent: 10m
Work Description: github-actions[bot] commented on pull request #430:
URL: https://github.com/apache/hive/pull/430#issuecomment-641144475
This pull request has been automatically marked as stale because it has not
had recent activity. It will be closed if no further activity occurs.
Feel free to reach out on the [email protected] list if the patch is in
need of reviews.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 442744)
Remaining Estimate: 0h
Time Spent: 10m
> Creation of staging directory and Move operation is taking time in S3
> ---------------------------------------------------------------------
>
> Key: HIVE-20517
> URL: https://issues.apache.org/jira/browse/HIVE-20517
> Project: Hive
> Issue Type: Sub-task
> Components: repl
> Affects Versions: 4.0.0
> Reporter: mahesh kumar behera
> Assignee: mahesh kumar behera
> Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-20517.01.patch, HIVE-20517.02.patch,
> HIVE-20517.03.patch, HIVE-20517.04.patch, HIVE-20517.05.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Operations like insert and add partition creates a staging directory to
> generate the files and then move the files created to actual location. In
> replication flow, the files are first copied to the staging directory and
> then moved (rename) to the actual table location. In case of S3, move is not
> an atomic operation. It internally does a copy and delete. So it can not
> guarantee the consistency required. So it is better to copy the files
> directly to the actual location. This will help in avoiding the staging
> directory creation (which takes 1-2 seconds in s3) and move (which takes time
> proportional to file size).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
