[ https://issues.apache.org/jira/browse/CARBONDATA-3056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Naman Rastogi updated CARBONDATA-3056: -------------------------------------- Summary: Implement concurrent reading through CarbonReader (was: Implement Concurrent SDK Reader) > Implement concurrent reading through CarbonReader > ------------------------------------------------- > > Key: CARBONDATA-3056 > URL: https://issues.apache.org/jira/browse/CARBONDATA-3056 > Project: CarbonData > Issue Type: Sub-task > Reporter: Naman Rastogi > Priority: Minor > > The current reading through SDK is slow as in CarbonReader, we are reading > the carbondata files sequentially, even though we have individual > CarbonRecordReader for each file. We can parallelize this by adding an API in > CarbonReader class > *List<CarbonReader> readers = CarbonReader.split(numSplits)* > which returns a list of CarbonReaders, which can be used to read parallelly, > as reading each file is independent of other files. > > This enables the SDK user to read the files as it is, or in a multithreaded > environment. -- This message was sent by Atlassian JIRA (v7.6.3#76005)