[ https://issues.apache.org/jira/browse/CARBONDATA-3481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ajantha Bhat updated CARBONDATA-3481: ------------------------------------- Fix Version/s: (was: 1.6.0) 1.6.1 > Multi-thread pruning fails when datamaps count is just near > numOfThreadsForPruning > ---------------------------------------------------------------------------------- > > Key: CARBONDATA-3481 > URL: https://issues.apache.org/jira/browse/CARBONDATA-3481 > Project: CarbonData > Issue Type: Bug > Reporter: Ajantha Bhat > Priority: Minor > Fix For: 1.6.1 > > Time Spent: 8h 10m > Remaining Estimate: 0h > > Problem : Multi-thread pruning fails when datamaps count is just near > numOfThreadsForPruning. > Cause : When the datamaps count is just near numOfThreadsForPruning, > As code is checking '>= ', last thread may not get the datamaps for prune. > Hence array out of index exception is thrown in this scenario. > There is no issues with higher number of datamaps. > solution: In this scenario launch threads based on the distribution value, > not on the hardcoded value > > steps to reproduce: > 5 index files with total 0.32 million data file (each has 50k, 120k, 50k, > 60k, 50k each) > with default thread count as 4, array of bound index is observed for below > line. > final List<SegmentDataMapGroup> segmentDataMapGroups = > datamapListForEachThread.get(i); -- This message was sent by Atlassian Jira (v8.3.4#803005)