[
https://issues.apache.org/jira/browse/CARBONDATA-80?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15401914#comment-15401914
]
ASF GitHub Bot commented on CARBONDATA-80:
------------------------------------------
Github user kumarvishal09 commented on a diff in the pull request:
https://github.com/apache/incubator-carbondata/pull/44#discussion_r72965345
--- Diff:
core/src/main/java/org/carbondata/core/cache/dictionary/AbstractColumnDictionaryInfo.java
---
@@ -241,22 +257,21 @@ private void decrementAccessCount() {
*/
protected byte[] getDictionaryBytesFromSurrogate(int surrogateKey) {
byte[] dictionaryValueInBytes = null;
- int totalSizeOfDictionaryChunksTraversed = 0;
- for (List<byte[]> oneDictionaryChunk : dictionaryChunks) {
- totalSizeOfDictionaryChunksTraversed =
- totalSizeOfDictionaryChunksTraversed + oneDictionaryChunk.size();
- // skip the dictionary chunk till surrogate key is lesser than size
of
- // dictionary chunks traversed
- if (totalSizeOfDictionaryChunksTraversed < surrogateKey) {
- continue;
+ if (surrogateKey > 0) {
--- End diff --
why we need this check ? As data loading surrogate key starts from 1 , so
in any case if surrogate key is less than 1 then there is some bug in
unpacking the mdkey
> Dictionary values should be equally distributed in buckets while loading in
> memory
> ----------------------------------------------------------------------------------
>
> Key: CARBONDATA-80
> URL: https://issues.apache.org/jira/browse/CARBONDATA-80
> Project: CarbonData
> Issue Type: Improvement
> Reporter: Manish Gupta
> Assignee: Manish Gupta
> Priority: Minor
>
> Whenever a query is executed, dictionary for columns queried is loaded in
> memory. For incremental loads dictionary values are loaded incrementally and
> thus one list contains several sub lists with dictionary values.
> The dictionary values on incremental load may not be equally distributed in
> the sub buckets and this might increase the search time of a value if there
> are too many incremental loads.
> Therefore the dictionary values should be divided equally in the sub buckets.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
