[jira] [Created] (CARBONDATA-472) Improve code coverage for core.cache package.

2016-11-29 Thread SWATI RAO (JIRA)
SWATI RAO created CARBONDATA-472: Summary: Improve code coverage for core.cache package. Key: CARBONDATA-472 URL: https://issues.apache.org/jira/browse/CARBONDATA-472 Project: CarbonData Issu

Re: read parquet file --No such file or directory

2016-11-29 Thread william
Can you try to create CarbonContext as follows: val cc = new CarbonContext(sc, storeLocation, metaLocation") cc.setConf("carbon.kettle.home", kettleHome)

read parquet file --No such file or directory

2016-11-29 Thread Lu Cao
Hi, I got a strange error that carbonContext can not read parquet file on hdfs in my program. I test on spark shell and it returns the same error, anything goes wrong? >>Test for sqlContext, Success ---

Re: compile error from spark project: scala.reflect.internal.MissingRequirmentError: object scala.runtime in compiler mirror not found

2016-11-29 Thread william
Specify the Profile eg. -Pspark-1.6 On Wed, Nov 30, 2016 at 9:53 AM, bill.zhou wrote: > hi all > > today fetch the latest code from the master branch, then compile the > project. > when compile the project spark it gives following issue. who knows this > issue ? > >

compile error from spark project: scala.reflect.internal.MissingRequirmentError: object scala.runtime in compiler mirror not found

2016-11-29 Thread bill.zhou
hi all today fetch the latest code from the master branch, then compile the project. when compile the project spark it gives following issue. who knows this issue ? -- View this message in context:

[jira] [Created] (CARBONDATA-471) Optimize no kettle flow and fix issues in cluster

2016-11-29 Thread Ravindra Pesala (JIRA)
Ravindra Pesala created CARBONDATA-471: -- Summary: Optimize no kettle flow and fix issues in cluster Key: CARBONDATA-471 URL: https://issues.apache.org/jira/browse/CARBONDATA-471 Project: CarbonDat

[jira] [Created] (CARBONDATA-470) Add unsafe offheap and on-heap sort in carbodata loading

2016-11-29 Thread Ravindra Pesala (JIRA)
Ravindra Pesala created CARBONDATA-470: -- Summary: Add unsafe offheap and on-heap sort in carbodata loading Key: CARBONDATA-470 URL: https://issues.apache.org/jira/browse/CARBONDATA-470 Project: Ca

Re: [Discussion] Please vote and comment for carbon data file format change

2016-11-29 Thread Kumar Vishal
Hi All, Please find the JIRA issue which I have raised for above discussion. https://issues.apache.org/jira/browse/CARBONDATA-458 -Regards Kumar Vishal On Tue, Nov 29, 2016 at 7:14 PM, Kumar Vishal wrote: > Hi Jihong Ma, > Please find the attachment. > > -Regards > Kumar Vishal > > On Fri, Nov

[jira] [Created] (CARBONDATA-469) Optimize join in spark using bucketing information

2016-11-29 Thread Ravindra Pesala (JIRA)
Ravindra Pesala created CARBONDATA-469: -- Summary: Optimize join in spark using bucketing information Key: CARBONDATA-469 URL: https://issues.apache.org/jira/browse/CARBONDATA-469 Project: CarbonD

[jira] [Created] (CARBONDATA-468) Add pruning in driver side to improve query performance

2016-11-29 Thread Ravindra Pesala (JIRA)
Ravindra Pesala created CARBONDATA-468: -- Summary: Add pruning in driver side to improve query performance Key: CARBONDATA-468 URL: https://issues.apache.org/jira/browse/CARBONDATA-468 Project: Car

[jira] [Created] (CARBONDATA-467) Add bucketing information while creating table and update in thrift format.

2016-11-29 Thread Ravindra Pesala (JIRA)
Ravindra Pesala created CARBONDATA-467: -- Summary: Add bucketing information while creating table and update in thrift format. Key: CARBONDATA-467 URL: https://issues.apache.org/jira/browse/CARBONDATA-467

[jira] [Created] (CARBONDATA-466) Implement bucketing table in carbondata

2016-11-29 Thread Ravindra Pesala (JIRA)
Ravindra Pesala created CARBONDATA-466: -- Summary: Implement bucketing table in carbondata Key: CARBONDATA-466 URL: https://issues.apache.org/jira/browse/CARBONDATA-466 Project: CarbonData

Re: [Discussion] Please vote and comment for carbon data file format change

2016-11-29 Thread Kumar Vishal
Hi Jihong Ma, Please find the attachment. -Regards Kumar Vishal On Fri, Nov 4, 2016 at 12:16 AM, Jihong Ma wrote: > Hi Kumar, > > Please place the proposed format changes in attachment or attach to the > associated JIRA, I would like to take a look. > > Thanks! > > Jihong > > -Original Mess

Re: carbon data

2016-11-29 Thread william
Try Code like this: ``` if (cc.tableNames().filter(f => f == _cfg.get("tableName").get).size == 0) { df.sqlContext.sql(s"DROP TABLE IF EXISTS ${_cfg.get("tableName").get}") writer.options(_cfg).mode(SaveMode.Overwrite).format(_format).save() } else { writer.options(_cfg).mode(Sa

Re: carbon data

2016-11-29 Thread Lu Cao
Thank you for the response Liang. I think I have followed the example but it still returns error: Data loading failed. table not found: default.carbontest attached my code below: I read data from a hive table with HiveContext and convert it to CarbonContext then generate the df and save to h