[
https://issues.apache.org/jira/browse/CARBONDATA-429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Liang Chen resolved CARBONDATA-429.
-----------------------------------
Resolution: Fixed
Assignee: Jacky Li (was: Ashok Kumar)
> Eliminate unnecessary file name check in dictionary cache
> ---------------------------------------------------------
>
> Key: CARBONDATA-429
> URL: https://issues.apache.org/jira/browse/CARBONDATA-429
> Project: CarbonData
> Issue Type: Sub-task
> Components: core
> Affects Versions: 0.1.1-incubating
> Reporter: Jacky Li
> Assignee: Jacky Li
> Fix For: 1.0.0-incubating
>
> Time Spent: 5h 50m
> Remaining Estimate: 0h
>
> 1.there are currently many file name check for each column's dictionary
> cache, which cause unnecessary calls to HDFS getFileStatus.
> 2. in checkAndLoadDictionaryData, we get meta file's mtime from hdfs each
> time we invoke cache.get to check if the local cache is valid or not. The
> local dictionary cache may be invalid after parallel data load. This will in
> turn increase number of calls to getFileStatus as well.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
