Github user ajantha-bhat commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2949#discussion_r240900313
--- Diff:
core/src/main/java/org/apache/carbondata/core/datamap/TableDataMap.java ---
@@ -205,26 +195,53 @@ public BlockletDetailsFetcher
getBlockletDetailsFetcher() {
final FilterResolverIntf filterExp, final List<PartitionSpec>
partitions,
List<ExtendedBlocklet> blocklets, final Map<Segment, List<DataMap>>
dataMaps,
int totalFiles) {
+ /*
+
*********************************************************************************
+ * Below is the example of how this part of code works.
+ * consider a scenario of having 5 segments, 10 datamaps in each
segment,
--- End diff --
BlockDatamap and blockletDatamap can store multiple files information. Each
file is one row in that datamap. But non-default datamaps are not like that, so
default datamaps distribution in multithread happens based on number of entries
in datamaps, for non-default datamps distribution is based on number of
datamaps (one datamap is considered as one record for non-default datamaps)
ALso 10 datamap in a segment means, one merge index file has info of 10
index files
---
