GitHub user ajantha-bhat opened a pull request:
https://github.com/apache/carbondata/pull/2876
[CARBONDATA-3054] Fix Dictionary file cannot be read in S3a with
CarbonDictionaryDecoder.doConsume() codeGen
[CARBONDATA-3054] Fix Dictionary file cannot be read in S3a with
CarbonDictionaryDecoder.doConsume() codeGen
**problem:** In S3a environment, when queried the data which has dictionary
files,
Dictionary file cannot be read in S3a with
CarbonDictionaryDecoder.doConsume() codeGen even though file is present.
**cause:** CarbonDictionaryDecoder.doConsume() codeGen doesn't set hadoop
conf in thread local variable, only doExecute() sets it.
Hence, when getDictionaryWrapper() called from doConsume() codeGen,
AbstractDictionaryCache.getDictionaryMetaCarbonFile() returns false for
fileExists() operation.
**solution:**
In CarbonDictionaryDecoder.doConsume() codeGen, set hadoop conf in thread
local variable
Be sure to do all of the following checklist to help us incorporate
your contribution quickly and easily:
- [ ] Any interfaces changed? NA
- [ ] Any backward compatibility impacted? NA
- [ ] Document update required? NA
- [ ] Testing done
done
- [ ] For large changes, please consider breaking it into sub-tasks under
an umbrella JIRA. NA
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/ajantha-bhat/carbondata master_new
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/carbondata/pull/2876.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2876
----
commit 492823671d0aead8393a55f147d5e11c21e0bdfa
Author: ajantha-bhat <ajanthabhat@...>
Date: 2018-10-29T12:26:29Z
[CARBONDATA-3054] Fix Dictionary file cannot be read in S3a with
CarbonDictionaryDecoder.doConsume() codeGen
----
---
