Github user Xaprice commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2136#discussion_r179941484
--- Diff:
integration/spark-common/src/main/scala/org/apache/carbondata/spark/rdd/CarbonScanRDD.scala
---
@@ -402,7 +402,7 @@ class CarbonScanRDD(
// one query id per table
model.setQueryId(queryId)
// get RecordReader by FileFormat
- val reader: RecordReader[Void, Object] = inputSplit.getFileFormat
match {
+ var reader: RecordReader[Void, Object] = inputSplit.getFileFormat
match {
--- End diff --
reader will be set null in closeReader() method to reduce memory occupation
when using coalesce, otherwise there will be lots of reader instances in memory.
---