[
https://issues.apache.org/jira/browse/CARBONDATA-3481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ajantha Bhat updated CARBONDATA-3481:
-------------------------------------
Fix Version/s: (was: 1.6.0)
1.6.1
> Multi-thread pruning fails when datamaps count is just near
> numOfThreadsForPruning
> ----------------------------------------------------------------------------------
>
> Key: CARBONDATA-3481
> URL: https://issues.apache.org/jira/browse/CARBONDATA-3481
> Project: CarbonData
> Issue Type: Bug
> Reporter: Ajantha Bhat
> Priority: Minor
> Fix For: 1.6.1
>
> Time Spent: 8h 10m
> Remaining Estimate: 0h
>
> Problem : Multi-thread pruning fails when datamaps count is just near
> numOfThreadsForPruning.
> Cause : When the datamaps count is just near numOfThreadsForPruning,
> As code is checking '>= ', last thread may not get the datamaps for prune.
> Hence array out of index exception is thrown in this scenario.
> There is no issues with higher number of datamaps.
> solution: In this scenario launch threads based on the distribution value,
> not on the hardcoded value
>
> steps to reproduce:
> 5 index files with total 0.32 million data file (each has 50k, 120k, 50k,
> 60k, 50k each)
> with default thread count as 4, array of bound index is observed for below
> line.
> final List<SegmentDataMapGroup> segmentDataMapGroups =
> datamapListForEachThread.get(i);
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
