GitHub user manishgupta88 opened a pull request:
https://github.com/apache/carbondata/pull/1935
[CARBONDATA-2134] Prevent implicit column filter list from getting
serialized while submitting task to executor
**Problem**
In the current store blocklet pruning in driver and no further pruning
takes place in the executor side. But still the implicit column filter list
being sent to executor. As the size of list grows the cost of serializing and
deserializing the list is increasing which can impact the query performance.
**Solution**
Remove the list from the filter expression before submitting the task to
executor.
Be sure to do all of the following checklist to help us incorporate
your contribution quickly and easily:
- [ ] Any interfaces changed?
No
- [ ] Any backward compatibility impacted?
No
- [ ] Document update required?
No
- [ ] Testing done
UT added
- [ ] For large changes, please consider breaking it into sub-tasks under
an umbrella JIRA.
NA
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/manishgupta88/carbondata
executor_filter_list_serialization
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/carbondata/pull/1935.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1935
----
commit 4630dbf30f26adff8b59ae57af12a2b183e8195f
Author: m00258959 <manish.gupta@...>
Date: 2018-02-05T11:40:18Z
Modified code to prevent implicit column array list from serializing and
deserializing to executor to improve query performance
----
---
