[
https://issues.apache.org/jira/browse/IMPALA-3474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tim Armstrong reassigned IMPALA-3474:
-------------------------------------
Assignee: (was: Sailesh Mukil)
> Speed up DROP TABLE for S3 tables
> ---------------------------------
>
> Key: IMPALA-3474
> URL: https://issues.apache.org/jira/browse/IMPALA-3474
> Project: IMPALA
> Issue Type: Improvement
> Components: Backend
> Affects Versions: Impala 2.6.0
> Reporter: Sailesh Mukil
> Priority: Minor
> Labels: performance, s3
>
> Currently if we have S3 tables with a lot of partitions or a lot of files,
> dropping that table takes a very long time.
> S3 only promises eventual consistency for DELETEs, so while deleting a small
> number of files is quick, deleting a large number of files(objects), gets the
> files queued up for deletion.
> The problem is that the Hadoop S3A connector code chooses to wait until all
> these files are deleted. Which means that the query also ends up waiting for
> however long S3 takes to delete these files.
> Consider alternatives like asynchronous delete by Impala.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
