[
https://issues.apache.org/jira/browse/CARBONDATA-484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15795669#comment-15795669
]
Venkata Ramana G commented on CARBONDATA-484:
---------------------------------------------
This issue extends current LRU cache already supported in CarbonData for B-Tree.
> Implement LRU cache for B-Tree
> -------------------------------
>
> Key: CARBONDATA-484
> URL: https://issues.apache.org/jira/browse/CARBONDATA-484
> Project: CarbonData
> Issue Type: New Feature
> Reporter: Mohammad Shahid Khan
> Assignee: Mohammad Shahid Khan
> Attachments: B-Tree LRU Cache.pdf
>
> Time Spent: 5h
> Remaining Estimate: 0h
>
> LRU Cache for B-Tree is proposed to ensure to avoid out memory, when too
> many number of tables exits and all are not frequently used.
> Problem:
> CarbonData is maintaining two level of B-Tree cache, one at the driver level
> and another at executor level. Currently CarbonData has the mechanism to
> invalidate the segments and blocks cache for the invalid table segments, but
> there is no eviction policy for the unused cached object. So the instance at
> which complete memory is utilized then the system will not be able to process
> any new requests.
> Solution:
> In the cache maintained at the driver level and at the executor there must be
> objects in cache currently not in use. Therefore system should have the
> mechanism to below mechanism.
> 1. Set the max memory limit till which objects could be hold in the
> memory.
> 2. When configured memory limit reached then identify the cached
> objects currently not in use so that the required memory could be freed
> without impacting the existing process.
> 3. Eviction should be done only till the required memory is not meet.
> For details please refer to attachments.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
