GitHub user kumarvishal09 opened a pull request:
https://github.com/apache/carbondata/pull/2213
[CARBONDATA-2386]Fix Query performance issue with PreAggregate table
**Problem:** Query on pre aggregate table is consuming too much time.
**Root cause:** Time consumption to calculate size of selecting the
smallest Pre-Aggregate table is approximately 76 seconds. This is index file is
being read when segment file is present to compute the size of Pre-Aggregate
table
**Solution:** Read table status and get the size of data file and index
file for valid segments. For older segments were datasize and indexsize is not
present calculate the size of store folder
- [ ] Any interfaces changed?
- [ ] Any backward compatibility impacted?
- [ ] Document update required?
- [ ] Testing done
Tested in Cluster with 2 billion rows and 16 aggregate tables
Older Time:76 seconds
New Time: 6 seconds
- [ ] For large changes, please consider breaking it into sub-tasks under
an umbrella JIRA.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/kumarvishal09/incubator-carbondata
preagg_perfix
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/carbondata/pull/2213.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2213
----
commit b8857c9e4064f30ec69fd55a6dfb47306e95616e
Author: kumarvishal09 <kumarvishal1802@...>
Date: 2018-04-23T11:33:00Z
Fix performance issue
----
---
