Re: Some questions about compiling carbondata

Hi, You do not need to specify spark.version variable, you can try these: mvn clean package -DskipTests -Pspark-2.0 (to build carbon with spark-2.0.2) mvn clean package -DskipTests (to build carbon with spark-1.5.2, which is default profile) Regards, Jacky -- View this message in context:

Re: [DISCUSSION] CarbonData loading solution discussion

+1We should flexibility choose loading solution according to Scenario 1 and 2, and will get performance benefits. -- View this message in context: http://apache-carbondata-mailing-list-archive.1130556.n5.nabble.com/DISCUSSION-CarbonData-loading-solution-discussion-tp4490p4520.html Sent from

[jira] [Created] (CARBONDATA-537) Bug fix for DICTIONARY_EXCLUDE option in spark2 integration

Jacky Li created CARBONDATA-537: --- Summary: Bug fix for DICTIONARY_EXCLUDE option in spark2 integration Key: CARBONDATA-537 URL: https://issues.apache.org/jira/browse/CARBONDATA-537 Project: CarbonData

[jira] [Created] (CARBONDATA-538) Add test case to spark2 integration

Jacky Li created CARBONDATA-538: --- Summary: Add test case to spark2 integration Key: CARBONDATA-538 URL: https://issues.apache.org/jira/browse/CARBONDATA-538 Project: CarbonData Issue Type:

Re: [DISCUSSION] CarbonData loading solution discussion

+1 to have separate output formats, now user can have flexibility to choose as per scenario. On Fri, Dec 16, 2016, 2:47 AM Jihong Ma wrote: > > It is great idea to have separate OutputFormat for regular Carbon data > files, index files as well as meta data files, For

RE: [DISCUSSION] CarbonData loading solution discussion

It is great idea to have separate OutputFormat for regular Carbon data files, index files as well as meta data files, For instance: dictionary file, schema file, global index file etc.. for writing Carbon generated files laid out HDFS, and it is orthogonal to the actual data load process.

Support of Float Data Type in Carbon Data

Hi, Carbon Data is not supporting Float Data type. Do we need to fix this Jira issue [CARBONDATA-390] ? I think float data type should have its own range. So do we need to support range for Float data type? Proposed Solution : We have to

Re: [DISCUSSION] CarbonData loading solution discussion

Hi Jacky Thanks you started a good discussion. see if i understand your points: Scenario1 likes the current load data solution(0.2.0). 1.0.0 Will provide a new solution option of "single-pass data loading" to meet this kind of scenario: For subsequent data loads if the most dictionary code has

[DISCUSSION] CarbonData loading solution discussion

Hi community, Since CarbonData has global dictionary feature, currently when loading data to CarbonData, it requires two times of scan of the input data. First scan is to generate dictionary, second scan to do actual data encoding and write to carbon files. Obviously, this approach is simple,

Re: [DISCUSSION] CarbonData loading solution discussion

Hi community, Sorry for the incorrect formatting of previous post. I corrected it in this post. Since CarbonData has global dictionary feature, currently when loading data to CarbonData, it requires two times of scan of the input data. First scan is to generate dictionary, second scan to do

[jira] [Created] (CARBONDATA-535) carbondata should support datatype: Date and Char

QiangCai created CARBONDATA-535: --- Summary: carbondata should support datatype: Date and Char Key: CARBONDATA-535 URL: https://issues.apache.org/jira/browse/CARBONDATA-535 Project: CarbonData

Some questions about compiling carbondata

Hi all, I've tried the following two ways to compile carbondata, but they finally all failed, if anyone could help me: 1. with the latest verson carbondata at github and spark 2.0.0, use the command mvn -DskipTests -Pspark-2.0 -Dspark.version=2.0.0 clean package and it

Re: Some questions about compiling carbondata

Carbon don't support spark 2.x -- Original -- From: "??";<251469...@qq.com>; Date: Thu, Dec 15, 2016 07:45 PM To: "dev"; Subject: Some questions about compiling carbondata Hi all, I've tried the following two