GitHub user ravipesala opened a pull request:
https://github.com/apache/carbondata/pull/1510
[WIP] Supported DataMap chooser and expression for supporting multiple
datamaps in single query
This PR supports 3 features.
1. Load datamaps from the DataMapSchema which are created through DDL.
2. DataMap Chooser: It chooses the datamap out of available datamaps based
on simple logic. Like if there is filter condition on column1 then for
supposing 2 datamaps(1. column1 2. column1+column2) are supporting this column
then we choose the datamap which has fewer columns that is the first datamap.
3. Expression support: Based on the filter expressions we convert them to
the possible DataMap expressions and do apply expression on it.
For example, there are 2 datamaps available on table1
Datamap1 : column1
Datamap2 : column2
Query: `select * from table1 where column1 ='a' and column2 = `b` `
For the above query, we create datamap expression as
`AndDataMapExpression(Datamap1, DataMap2)`. So for the above query both the
datamaps are included and the output of them will be applied AND condition to
improve the performance
- [X] Any interfaces changed?
Yes, DataMap interfaces are changed in order to support datamap
expression.
- [X] Any backward compatibility impacted?
NO
- [X] Document update required?
Yes, dev guide need to be updated
- [X] Testing done
Tests are added
- [ ] For large changes, please consider breaking it into sub-tasks under
an umbrella JIRA.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/ravipesala/incubator-carbondata
datamap-fg-chooser1
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/carbondata/pull/1510.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1510
----
commit c2b898468acdfb3d1738c34ce0557100700f6f80
Author: ravipesala <[email protected]>
Date: 2017-11-15T14:18:40Z
Added FG implementation and testcases
commit 3a405355874c051fd640cc3ad00049a1a73fd8c6
Author: ravipesala <[email protected]>
Date: 2017-11-08T16:00:32Z
Added DataMapChooser
----
---
