[
https://issues.apache.org/jira/browse/CARBONDATA-3641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Cai resolved CARBONDATA-3641.
-----------------------------------
Resolution: Fixed
> Should improve data loading performance for partition table
> -----------------------------------------------------------
>
> Key: CARBONDATA-3641
> URL: https://issues.apache.org/jira/browse/CARBONDATA-3641
> Project: CarbonData
> Issue Type: Improvement
> Components: data-load
> Reporter: David Cai
> Priority: Major
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> [Background]
> # only implemented commit algorithm version 1
> # generated too many segment files during loading
> # generated too many small data files and index files
> [Modification]
> 1. implemented carbon commit algorithm, avoid to move data file and
> index files
> 2. generate the final segment file directly
> 3. optimize global_sort to avoid small files issue
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
