GitHub user manishgupta88 opened a pull request:
https://github.com/apache/incubator-carbondata/pull/292
[CARBONDATA-375] Dictionary cache not getting cleared after task completion
in dictionary decoder
Problem: Dictionary cache not getting cleared after task completion in
dictionary decoder
Analysis: Currently LRU cache eviction policy is based on dictionary access
count. For cache to remove a entry its access count must be 0. In dictionary
decoder after conversion of surrogate key to actual value the access count for
dictionary columns in query is not getting decremented due to which it will
never be cleared from memory when LRU cache size is configured.
Fix: Add a task completion listener which will take care of clearing the
dictionary in case of both success and failure
Impact area: LRU cache eviction policy which can lead to query and data
load failure
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/manishgupta88/incubator-carbondata
dictionary_decoder_clear_dictionary
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-carbondata/pull/292.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #292
----
commit b305f34e1014267b3706c287cef7070189fc3c28
Author: manishgupta88 <[email protected]>
Date: 2016-11-03T15:48:03Z
Problem: Dictionary cache not getting cleared after task completion in
dictionary decoder
Analysis: Currently LRU cache eviction policy is based on dictionary access
count. For cache to remove a entry its access count must be 0. In dictionary
decoder after conversion of surrogate key to actual value the access count for
dictionary columns in query is not getting decremented due to which it will
never be cleared from memory when LRU cache size is configured.
Fix: Add a task completion listener which will take care of clearing the
dictionary in case of both success and failure
Impact area: LRU cache eviction policy which can lead to query and data
load failure
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
