[
https://issues.apache.org/jira/browse/CARBONDATA-1230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ravindra Pesala updated CARBONDATA-1230:
----------------------------------------
Attachment: DataMap Design.pdf
> Datamap framework for Carbondata to leverage indexing
> -----------------------------------------------------
>
> Key: CARBONDATA-1230
> URL: https://issues.apache.org/jira/browse/CARBONDATA-1230
> Project: CarbonData
> Issue Type: New Feature
> Reporter: Ravindra Pesala
> Assignee: Ravindra Pesala
> Attachments: DataMap Design.pdf
>
>
> Datamap should be single point interface for indexing and pruning.
> It could be two types
> # 1. Coarse grained datamap.
> # 2 Fine grained datamap.
> h3. Coarse grained datamap
> These datamaps contains the information of blocklets. so it can prune till
> blocklet level. It could be loaded on driver side or executor side depends on
> size of datamap.
> Default implementation for this type is BlockletDataMap. It contains all
> necessary information of blocklet with stats like startkey, endkey and max
> and min value. Using this information all filter queries would be pruned by
> datamap.
> h3. Fine grained datamap
> These datamap contains information up to page and row level. It is stored
> executor side and used as part of filtering to speed up the queries.
>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
