[
https://issues.apache.org/jira/browse/CARBONDATA-2428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ravindra Pesala updated CARBONDATA-2428:
----------------------------------------
Description:
1. Flat folder makes all carbondata files store flat under table path.
2. It is controlled through table property `flat_folder`. By default it is
false.
3. It cannot be hybrid, so user cannot change the property once table created.
4. Segment file is created for each loading.And segment file is created under
MetaData folder under table path.
5. Segment number is added as part of carbondata and index files.
6. All datamap files now create directly under table path with
<tablepath>/<dmname>/<segment_number>/<task_name>/dm
IUD : It supports but list files during IUD may hit performance.
Compaction: Supports
Delete Segment : No impact
Clean files : No impact
Alter table : No impact
Pre Agg : Property need to inherited to child, so it also supports flat folder
structure.
Partition : No Impact on this feature as it already has flat folder structure.
Streaming : Only during handoff it supports flat folder structure. Streaming
segment location is no change.
was:Currently carbondata writing happens in fixed path
tablepath/Fact/Part0/Segment_NUM folder and it is not same as hive/parquet
folder structure. This PR makes all files written will be inside tablepath, it
does not maintain any segment folder structure. Only for partition it adds the
folder.
> Support Flat folder structure in carbon.
> ----------------------------------------
>
> Key: CARBONDATA-2428
> URL: https://issues.apache.org/jira/browse/CARBONDATA-2428
> Project: CarbonData
> Issue Type: Improvement
> Reporter: Ravindra Pesala
> Priority: Major
> Time Spent: 6h 50m
> Remaining Estimate: 0h
>
> 1. Flat folder makes all carbondata files store flat under table path.
> 2. It is controlled through table property `flat_folder`. By default it is
> false.
> 3. It cannot be hybrid, so user cannot change the property once table created.
> 4. Segment file is created for each loading.And segment file is created under
> MetaData folder under table path.
> 5. Segment number is added as part of carbondata and index files.
> 6. All datamap files now create directly under table path with
> <tablepath>/<dmname>/<segment_number>/<task_name>/dm
>
> IUD : It supports but list files during IUD may hit performance.
> Compaction: Supports
> Delete Segment : No impact
> Clean files : No impact
> Alter table : No impact
> Pre Agg : Property need to inherited to child, so it also supports flat
> folder structure.
> Partition : No Impact on this feature as it already has flat folder structure.
> Streaming : Only during handoff it supports flat folder structure. Streaming
> segment location is no change.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
