Here some encodings can be done on each field level and some can be done on
blocklet(batch of column data) level. So DICTIONARY encoding is done on
each field level and this FieldConverter is only encoding data on field
RLE is applied on blocklet level so it is applied while writing
Do you build the package from source? If so, please check spark version.
Make sure your spark assembly version is 1.6 and build carbondata with
profile specified eg.
mvn package -Pspark-1.6
I have integrated CarbonData to project StreamingPro (
http://www.jianshu.com/p/7733da82a9ce). StreamingPro ships with
carbondata-0.3.0 without kettle dependency and support spark streaming.
Maybe you can have a try,hope this can save your time.
Dear Dev Team,
I have a question about run-length encoding (RLE) support in CarbonData.
In Encoding enum type I can see RLE, DELTA and some other encodings.
However while I look at the code (FieldEncoderFactory.createFieldEncoder), I
notice there is an if / else like the following:
i clone from git ,branch master , compile mvn hadoop 2.6.3 ,spark 1.6.1
follow the quick start , then run spark-shell
$SPARK_HOME/bin/spark-shell --verbose --master local --jars
View this message in context:
Sent from the Apache CarbonData Mailing List archive mailing list archive at
WilliamZhu created CARBONDATA-476:
Summary: storeLocation start with file:/// cause table not found
Anurag Srivastava created CARBONDATA-475:
Summary: Implement unit test cases for core.carbon.querystatics
Anurag Srivastava created CARBONDATA-474:
Summary: Implement unit test cases for
Fei Wang created CARBONDATA-473:
Summary: spark 2 stable datasource api integration
Mail list logo