Jacky Li created CARBONDATA-307:
-----------------------------------
Summary: Support full functionality in CarbonInputFormat
Key: CARBONDATA-307
URL: https://issues.apache.org/jira/browse/CARBONDATA-307
Project: CarbonData
Issue Type: Improvement
Components: spark-integration
Affects Versions: 0.1.0-incubating
Reporter: Jacky Li
Fix For: 0.2.0-incubating
Currently, there are two read path in carbon-spark module:
1. CarbonContext => CarbonDatasourceRelation => CarbonScanRDD => QueryExecutor
In this case, CarbonScanRDD uses CarbonInputFormat to get the split, and use
QueryExecutor for scan.
2. SqlContext => CarbonDatasourceHadoopRelation => CarbonHadoopFSRDD =>
CarbonRecordReader
In this case, CarbonHadoopFSRDD uses CarbonInputFormat to do both get split and
scan
It create unnecessary duplicate code, they need to be unified.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
