Github user ravipesala commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2269#discussion_r189423352
--- Diff:
core/src/main/java/org/apache/carbondata/core/datamap/DistributableDataMapFormat.java
---
@@ -158,7 +166,11 @@ public float getProgress() throws IOException,
InterruptedException {
@Override
public void close() throws IOException {
-
+ if (null != dataMaps) {
+ for (DataMap dataMap : dataMaps) {
+ dataMap.clear();
--- End diff --
This impacts the datamaps which they want to cache on executor side. It
always clears cache for each call. Better add another method in datamap
interface close or finish to release the resources which are need to released
for every call
---
