Fei Wang created CARBONDATA-322:

             Summary: integrate spark 2.x 
                 Key: CARBONDATA-322
                 URL: https://issues.apache.org/jira/browse/CARBONDATA-322
             Project: CarbonData
          Issue Type: Bug
          Components: spark-integration
    Affects Versions: 0.2.0-incubating
            Reporter: Fei Wang
             Fix For: 0.3.0-incubating

As spark 2.0 released. there are many nice features such as more efficient 
parser, vectorized execution, adaptive execution. It is good to integrate with 
spark 2.x

Another side now spark integration is heavy coupling with spark, we should 
redesign the spark integration, it should satisfy flowing requirement:

1. decoupled with spark, integrate according to spark datasource API(V2)
2. This integration should support vectorized carbon reader
3. Supoort write to carbondata from dadatrame

This message was sent by Atlassian JIRA

Reply via email to