GitHub user ajantha-bhat opened a pull request:
https://github.com/apache/carbondata/pull/2895
[HOTFIX] Fix NPE in spark, when same vector reads files with local
dictionary and without local dictionary
problem: NPE in spark, when same vector reads files with local dictionary
and without local dictionary
cause: when two carbondata files are present, one with local dictionary and
one without local dictionary. If same vector is used to read this files [can
happen if task is launched to group of files]. If local dictionary files are
found first, dictionary is set for that vector. But it was never reset for
another file reading.
solution: reset dictionary once batch is processed,set only for local
dictionary batch processing.
Be sure to do all of the following checklist to help us incorporate
your contribution quickly and easily:
- [ ] Any interfaces changed? NA
- [ ] Any backward compatibility impacted? NA
- [ ] Document update required? NA
- [ ] Testing done
yes, cluster testing done.
- [ ] For large changes, please consider breaking it into sub-tasks under
an umbrella JIRA. NA
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/ajantha-bhat/carbondata master_new
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/carbondata/pull/2895.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2895
----
commit 99c7621336e3cf180bfa0c3a326a2f1fafe51631
Author: ajantha-bhat <ajanthabhat@...>
Date: 2018-11-05T10:00:27Z
Fix vectcor reading with local dictionary and without local dictionary
----
---
