Github user ravipesala commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2623#discussion_r208856247
--- Diff:
integration/spark2/src/main/scala/org/apache/carbondata/spark/rdd/CarbonDataRDDFactory.scala
---
@@ -720,7 +723,9 @@ object CarbonDataRDDFactory {
// because partitionId=segmentIdIndex*parallelism+RandomPart and
RandomPart<parallelism,
// so segmentIdIndex=partitionId/parallelism, this has been verified.
+ val conf = new SerializableConfiguration(hadoopConf)
--- End diff --
Please copy only necessary configurations instead of serializing whole
object. Better use new Serializable class and copy the useful properties to it.
---
