[
https://issues.apache.org/jira/browse/TAJO-2063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15130526#comment-15130526
]
Jaehwa Jung commented on TAJO-2063:
-----------------------------------
This issue will eliminate {{FileSystem::getContentsSummary}} for partition
table.
> Refactor FileTablespace::commitOutputData.
> ------------------------------------------
>
> Key: TAJO-2063
> URL: https://issues.apache.org/jira/browse/TAJO-2063
> Project: Tajo
> Issue Type: Improvement
> Components: Storage
> Reporter: Jaehwa Jung
> Assignee: Jaehwa Jung
> Fix For: 0.12.0
>
>
> {{FileTablespace::commitOutputData}} has some problems as following:
> First, it is too long and complexed because it handles various cases in a
> single method. We need to refactor this method into several small and
> well-defined methods.
> Second, {{FileSystem::listStatus}} is widely used while committing output
> data. It especially causes a lot of overhead to list partitioned directories
> in S3. Also it occurs in HDFS too with large partitioned tables. We need to
> minimize its usage.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
