WilliamZhu created CARBONDATA-465:
-------------------------------------
Summary: Spark streaming dataframe support
Key: CARBONDATA-465
URL: https://issues.apache.org/jira/browse/CARBONDATA-465
Project: CarbonData
Issue Type: Improvement
Components: data-load
Affects Versions: 0.3.0-incubating
Reporter: WilliamZhu
Assignee: WilliamZhu
Priority: Minor
Fix For: 0.3.0-incubating
Carbondata-0.3.0 support load data with spark data frame api. There is a limit
that kettle is still required since DataFrameLoaderRDD still depends on kettle.
We provide NewDataFrameLoaderRDD to load data with new flow .
Also,we discovered some bugs:
1. CarbonMetastoreCatalog.createTableFromThrift
```
/**
* schemaFilePath starts with file:// will not create meta files
successfully
* while thriftWriter will have no complains.
* This will cause some weired error eg. No table found.
*/
val thriftWriter = new ThriftWriter(schemaFilePath, false)
thriftWriter.open()
thriftWriter.write(thriftTableInfo)
thriftWriter.close()
```
2. There are some exceptions raised even when you have set useKettle to false.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
