GitHub user kumarvishal09 opened a pull request:
https://github.com/apache/carbondata/pull/2149
[WIP]Page level uncompress and Improve Unsafe Variable length store
performance
**Changes Done:**
**Page Level Decoder for query**
Added page level on demand decoding, in current code, all pages of blocklet
is getting uncompressed, because of this memory footprint is too high and cause
OOM, Now added code to support page level decoding, one page will be decoding
when all the records are processed next page data will be decoded. It will
improve query performance for example limit query.
**Unsafe No Dictionary(Unsafe variable length)**
Optimized getRow(for Vector processing) And putArray method
- [ ] Any interfaces changed?
- [ ] Any backward compatibility impacted?
- [ ] Document update required?
- [ ] Testing done
Existing all the testcase are applicable.
- [ ] For large changes, please consider breaking it into sub-tasks under
an umbrella JIRA.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/kumarvishal09/incubator-carbondata
PageLevelUnCompress
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/carbondata/pull/2149.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2149
----
commit 5a3ca1a3b6bc3d799c20ef9dda901367f3ac2d2c
Author: kumarvishal <kumarvishal.1802@...>
Date: 2018-04-09T11:40:13Z
Page level uncompress
----
---
