[
https://issues.apache.org/jira/browse/CARBONDATA-3006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
kumar vishal updated CARBONDATA-3006:
-------------------------------------
Summary: Carbon Store Size Optimization and Query Performance Improvement
(was: Carbon Store Size Optimization and Scan Query Performance Improvement)
> Carbon Store Size Optimization and Query Performance Improvement
> ----------------------------------------------------------------
>
> Key: CARBONDATA-3006
> URL: https://issues.apache.org/jira/browse/CARBONDATA-3006
> Project: CarbonData
> Issue Type: Improvement
> Reporter: kumar vishal
> Priority: Major
>
> *String/Varchar Datatype Store Size Optimization:*
> Currently length is stored as Short/Int for String/varchar datatype because
> of this store size is more. To reduce the store size Adaptive encoding is
> applied for length part irrespective of String/Varchar type so during
> processing there will not be separate handling for String/Varchar datatype.
> *String/Varchar datatype query processing optimization:*
> Currently for processing the String/Varchar datatype during query
> offset(positions of data) is calculated and based on position data is
> fetched. Because of this many cacheline misses is happening and its degrading
> query performance.
> To handle this for full scan query with no inverted index, data is fetched is
> in linear way to avoid cache line misses.
> *Adaptive encoding for Global/Direct/Local dictionary columns*
> Currently Global/Direct/Local dictionary are stored in binary format and only
> snappy is applied for compression. As Global/Direct/Local dictionary values
> are of Integer data type it can adaptability stored with the data type
> smaller than Integer.
> Added adaptive for global/direct dictionary column to reduce the store size.
> *Method In-lining Optimization*
> JIT will inline any method if method size is less than 325 byte code size and
> if it is called more than 10K times(default value). If method is private or
> static it will be easier for JIT to inline as type safe check is not
> required, for protected/public method it will add a overhead of type check
> and because of this it will not behave as inline.
> Because of above case some refactoring is done for primitive no dictionary
> data type columns. Earlier ColumnPageWrapper.java was handling query
> processing for all primitive no dictionary data type column now in This PR
> separate classes are created for each data type handling and all the HOT
> method is kept as private and protected methods are overridden and other
> methods are added in Super classes
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
