date:20170917

[GitHub] carbondata issue #1322: [CARBONDATA-1450] Support timestamp more than 68 yea...

2017-09-17 Thread CarbonDataQA

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/1322
  
Build Success with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/210/



---

[GitHub] carbondata issue #1364: [CARBONDATA-1483]open auto merge when loading and in...

2017-09-17 Thread ravipesala

Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/1364
  
SDV Build Fail , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/839/



---

[GitHub] carbondata issue #1322: [CARBONDATA-1450] Support timestamp more than 68 yea...

2017-09-17 Thread CarbonDataQA

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/1322
  
Build Success with Spark 1.6, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/86/



---

[GitHub] carbondata issue #1322: [CARBONDATA-1450] Support timestamp more than 68 yea...

2017-09-17 Thread dhatchayani

Github user dhatchayani commented on the issue:

https://github.com/apache/carbondata/pull/1322
  
Retest this please


---

[GitHub] carbondata issue #1364: [CARBONDATA-1483]open auto merge when loading and in...

2017-09-17 Thread CarbonDataQA

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/1364
  
Build Success with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/208/



---

[GitHub] carbondata issue #1364: [CARBONDATA-1483]open auto merge when loading and in...

2017-09-17 Thread CarbonDataQA

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/1364
  
Build Success with Spark 1.6, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/85/



---

[jira] [Updated] (CARBONDATA-1483) open auto merge

2017-09-17 Thread xubo245 (JIRA)


 [ 
https://issues.apache.org/jira/browse/CARBONDATA-1483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

xubo245 updated CARBONDATA-1483:

Description: 
auto compaction when loading and inserting into
* set  DEFAULT_ENABLE_AUTO_LOAD_MERGE as true
* add test case and verify the change is ok


  was:
auto compaction when loading data 
* set  DEFAULT_ENABLE_AUTO_LOAD_MERGE as true
* add test case and verify the change is ok


Summary: open auto merge  (was: open auto merge when loading data)

> open auto merge
> ---
>
> Key: CARBONDATA-1483
> URL: https://issues.apache.org/jira/browse/CARBONDATA-1483
> Project: CarbonData
>  Issue Type: Improvement
>Reporter: xubo245
>Assignee: xubo245
>Priority: Minor
>   Original Estimate: 120h
>  Time Spent: 0.5h
>  Remaining Estimate: 119.5h
>
> auto compaction when loading and inserting into
> * set  DEFAULT_ENABLE_AUTO_LOAD_MERGE as true
> * add test case and verify the change is ok



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] carbondata issue #1364: [CARBONDATA-1483]open auto merge when loading data

2017-09-17 Thread CarbonDataQA

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/1364
  
Build Failed  with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/207/



---

[GitHub] carbondata issue #1364: [CARBONDATA-1483]open auto merge when loading data

2017-09-17 Thread CarbonDataQA

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/1364
  
Build Failed with Spark 1.6, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/84/



---

[jira] [Updated] (CARBONDATA-1483) open auto merge when loading data

2017-09-17 Thread xubo245 (JIRA)


 [ 
https://issues.apache.org/jira/browse/CARBONDATA-1483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

xubo245 updated CARBONDATA-1483:

Description: 
auto compaction when loading data 
* set  DEFAULT_ENABLE_AUTO_LOAD_MERGE as true
* add test case and verify the change is ok


  was:
auto compaction when loading data 
set  DEFAULT_ENABLE_AUTO_LOAD_MERGE as true
add test case and verify the change is ok
 


> open auto merge when loading data
> -
>
> Key: CARBONDATA-1483
> URL: https://issues.apache.org/jira/browse/CARBONDATA-1483
> Project: CarbonData
>  Issue Type: Improvement
>Reporter: xubo245
>Assignee: xubo245
>Priority: Minor
>   Original Estimate: 120h
>  Remaining Estimate: 120h
>
> auto compaction when loading data 
> * set  DEFAULT_ENABLE_AUTO_LOAD_MERGE as true
> * add test case and verify the change is ok



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] carbondata pull request #1364: [CARBONDATA-1483]open auto merge when loading...

2017-09-17 Thread xubo245

GitHub user xubo245 opened a pull request:

https://github.com/apache/carbondata/pull/1364

[CARBONDATA-1483]open auto merge when loading data

auto compaction when loading data 
*  set DEFAULT_ENABLE_AUTO_LOAD_MERGE as true
*  add test case and verify the change is ok

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/xubo245/carbondata compaction

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/carbondata/pull/1364.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1364


commit 1b2be64cebe55baf5661262a008bc1bbbcb16b67
Author: xubo245 <601450...@qq.com>
Date:   2017-09-18T03:41:03Z

open auto merge when loading data




---

[jira] [Created] (CARBONDATA-1483) open auto merge when loading data

2017-09-17 Thread xubo245 (JIRA)

xubo245 created CARBONDATA-1483:
---

 Summary: open auto merge when loading data
 Key: CARBONDATA-1483
 URL: https://issues.apache.org/jira/browse/CARBONDATA-1483
 Project: CarbonData
  Issue Type: Improvement
Reporter: xubo245
Assignee: xubo245
Priority: Minor


auto compaction when loading data 
set  DEFAULT_ENABLE_AUTO_LOAD_MERGE as true
add test case and verify the change is ok
 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] carbondata issue #1317: [CARBONDATA-1316] drop partition

2017-09-17 Thread ravipesala

Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/1317
  
SDV Build Success , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/838/



---

[GitHub] carbondata issue #1361: [CARBONDATA-1481]Compaction support global sort

2017-09-17 Thread xubo245

Github user xubo245 commented on the issue:

https://github.com/apache/carbondata/pull/1361
  
Have SDV accident error?  I haven't changed the SDV code after SDV Build 
fail, but now it shows Success.


---

[GitHub] carbondata issue #1317: [CARBONDATA-1316] drop partition

2017-09-17 Thread CarbonDataQA

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/1317
  
Build Success with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/206/



---

[GitHub] carbondata issue #1361: [CARBONDATA-1481]Compaction support global sort

2017-09-17 Thread ravipesala

Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/1361
  
SDV Build Success , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/837/



---

[GitHub] carbondata issue #1317: [CARBONDATA-1316] drop partition

2017-09-17 Thread CarbonDataQA

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/1317
  
Build Success with Spark 1.6, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/83/



---

[GitHub] carbondata pull request #1259: [Review][CARBONDATA-1381] Add test cases for ...

2017-09-17 Thread lionelcao

Github user lionelcao commented on a diff in the pull request:

https://github.com/apache/carbondata/pull/1259#discussion_r139330679
  
--- Diff: 
core/src/main/java/org/apache/carbondata/core/metadata/schema/PartitionInfo.java
 ---
@@ -17,17 +17,21 @@
 
 package org.apache.carbondata.core.metadata.schema;
 
+import java.io.DataInput;
+import java.io.DataOutput;
+import java.io.IOException;
 import java.io.Serializable;
 import java.util.ArrayList;
 import java.util.List;
 
 import org.apache.carbondata.core.metadata.schema.partition.PartitionType;
+import org.apache.carbondata.core.metadata.schema.table.Writable;
 import 
org.apache.carbondata.core.metadata.schema.table.column.ColumnSchema;
 
 /**
  * Partition information of carbon partition table
  */
-public class PartitionInfo implements Serializable {
+public class PartitionInfo implements Serializable, Writable {
--- End diff --

This modification will fix a bug : PartitionInfo is null in new CarbonTable 
in getSplits function in CarbonTableInputFormat. Could you split it into a 
independent PR? Looks inappropriate with the description of current PR.


---

[GitHub] carbondata issue #1361: [CARBONDATA-1481]Compaction support global sort

2017-09-17 Thread CarbonDataQA

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/1361
  
Build Success with Spark 1.6, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/82/



---

[GitHub] carbondata issue #1361: [CARBONDATA-1481]Compaction support global sort

2017-09-17 Thread CarbonDataQA

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/1361
  
Build Success with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/205/



---

[GitHub] carbondata issue #1361: [CARBONDATA-1481]Compaction support global sort

2017-09-17 Thread ravipesala

Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/1361
  
SDV Build Fail , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/836/



---

[GitHub] carbondata issue #1361: [CARBONDATA-1481]Compaction support global sort

2017-09-17 Thread CarbonDataQA

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/1361
  
Build Success with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/204/



---

[GitHub] carbondata issue #1361: [CARBONDATA-1481]Compaction support global sort

2017-09-17 Thread CarbonDataQA

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/1361
  
Build Success with Spark 1.6, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/81/



---

[GitHub] carbondata issue #1317: [CARBONDATA-1316] drop partition

2017-09-17 Thread ravipesala

Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/1317
  
SDV Build Success , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/835/



---

[GitHub] carbondata pull request #1291: [CARBONDATA-1343] Hive can't query data when ...

2017-09-17 Thread anubhav100

Github user anubhav100 commented on a diff in the pull request:

https://github.com/apache/carbondata/pull/1291#discussion_r139315604
  
--- Diff: 
hadoop/src/main/java/org/apache/carbondata/hadoop/CarbonInputFormat.java ---
@@ -133,7 +133,7 @@ public static void setTableInfo(Configuration 
configuration, TableInfo tableInfo
   /**
* Get TableInfo object from `configuration`
*/
-  private TableInfo getTableInfo(Configuration configuration) throws 
IOException {
+  public TableInfo getTableInfo(Configuration configuration) throws 
IOException {
--- End diff --

@cenyuhai change in this file is not required now  because 
MapredCarbonInputFormat now extends CarbonTableInputFormat 


---

[GitHub] carbondata issue #1317: [CARBONDATA-1316] drop partition

2017-09-17 Thread ravipesala

Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/1317
  
SDV Build Fail , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/834/



---

[GitHub] carbondata issue #1317: [CARBONDATA-1316] drop partition

2017-09-17 Thread CarbonDataQA

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/1317
  
Build Failed  with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/203/



---

[GitHub] carbondata issue #1317: [CARBONDATA-1316] drop partition

2017-09-17 Thread CarbonDataQA

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/1317
  
Build Success with Spark 1.6, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/80/



---

[GitHub] carbondata issue #1317: [CARBONDATA-1316] drop partition

2017-09-17 Thread CarbonDataQA

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/1317
  
Build Failed  with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/202/



---

[GitHub] carbondata issue #1317: [CARBONDATA-1316] drop partition

2017-09-17 Thread CarbonDataQA

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/1317
  
Build Failed with Spark 1.6, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/79/



---

[GitHub] carbondata issue #1336: [CARBONDATA-1425] Inappropriate Exception displays w...

2017-09-17 Thread lionelcao

Github user lionelcao commented on the issue:

https://github.com/apache/carbondata/pull/1336
  
LGTM


---

[GitHub] carbondata issue #1317: [CARBONDATA-1316] drop partition

2017-09-17 Thread ravipesala

Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/1317
  
SDV Build Fail , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/833/



---

[GitHub] carbondata issue #1363: [CARBONDATA-1378] support creating table in hive

2017-09-17 Thread ravipesala

Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/1363
  
SDV Build Success , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/832/



---

[GitHub] carbondata issue #1317: [CARBONDATA-1316] drop partition

2017-09-17 Thread CarbonDataQA

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/1317
  
Build Failed with Spark 1.6, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/78/



---

[GitHub] carbondata issue #1317: [CARBONDATA-1316] drop partition

2017-09-17 Thread CarbonDataQA

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/1317
  
Build Failed  with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/201/



---

[GitHub] carbondata issue #1317: [CARBONDATA-1316] drop partition

2017-09-17 Thread chenliang613

Github user chenliang613 commented on the issue:

https://github.com/apache/carbondata/pull/1317
  
retest this please


---

[GitHub] carbondata issue #1363: [CARBONDATA-1378] support creating table in hive

2017-09-17 Thread cenyuhai

Github user cenyuhai commented on the issue:

https://github.com/apache/carbondata/pull/1363
  
retest please


---

[GitHub] carbondata issue #1363: [CARBONDATA-1378] support creating table in hive

2017-09-17 Thread CarbonDataQA

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/1363
  
Build Success with Spark 1.6, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/77/



---

[GitHub] carbondata issue #1363: [CARBONDATA-1378] support creating table in hive

2017-09-17 Thread CarbonDataQA

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/1363
  
Build Failed  with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/200/



---

[jira] [Assigned] (CARBONDATA-1378) Support create carbon table in Hive

2017-09-17 Thread cen yuhai (JIRA)


 [ 
https://issues.apache.org/jira/browse/CARBONDATA-1378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

cen yuhai reassigned CARBONDATA-1378:
-

Assignee: cen yuhai

> Support create carbon table in Hive
> ---
>
> Key: CARBONDATA-1378
> URL: https://issues.apache.org/jira/browse/CARBONDATA-1378
> Project: CarbonData
>  Issue Type: Sub-task
>  Components: hive-integration
>Reporter: cen yuhai
>Assignee: cen yuhai
>  Time Spent: 40m
>  Remaining Estimate: 0h
>
> Support create carbon table in Hive



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] carbondata issue #1363: [CARBONDATA-1378] support creating table in hive

2017-09-17 Thread ravipesala

Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/1363
  
SDV Build Success , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/831/



---

[GitHub] carbondata issue #1363: [CARBONDATA-1378] support creating table in hive

2017-09-17 Thread CarbonDataQA

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/1363
  
Build Failed  with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/199/



---

[GitHub] carbondata issue #1363: [CARBONDATA-1378] support creating table in hive

2017-09-17 Thread CarbonDataQA

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/1363
  
Build Success with Spark 1.6, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/76/



---

[GitHub] carbondata pull request #1363: [CARBONDATA-1378] support creating table in h...

2017-09-17 Thread cenyuhai

GitHub user cenyuhai opened a pull request:

https://github.com/apache/carbondata/pull/1363

[CARBONDATA-1378] support creating table in hive

support creating table in hive

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/cenyuhai/incubator-carbondata CARBONDATA-1378

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/carbondata/pull/1363.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1363


commit d4196f823edc4a732a1d0081d2e71f8b87041d41
Author: CenYuhai 
Date:   2017-09-17T10:28:30Z

support creating table in hive




---

[GitHub] carbondata issue #1361: [CARBONDATA-1481]Compaction support global sort

2017-09-17 Thread jackylk

Github user jackylk commented on the issue:

https://github.com/apache/carbondata/pull/1361
  
please rebase to master


---

[GitHub] carbondata pull request #1361: [CARBONDATA-1481]Compaction support global so...

2017-09-17 Thread jackylk

Github user jackylk commented on a diff in the pull request:

https://github.com/apache/carbondata/pull/1361#discussion_r139307748
  
--- Diff: 
integration/spark-common-cluster-test/src/test/scala/org/apache/carbondata/cluster/sdv/generated/BatchSortLoad3TestCase.scala
 ---
@@ -112,33 +112,15 @@ class BatchSortLoad3TestCase extends QueryTest with 
BeforeAndAfterAll {
 sql(s"""drop table if exists t3""").collect
   }
 
-
-  //Batch_sort_Loading_001-01-01-01_001-TC_056
-  test("Batch_sort_Loading_001-01-01-01_001-TC_056", Include) {
--- End diff --

why removing it?


---

[jira] [Commented] (CARBONDATA-1444) CarbonData unsupport Boolean data type,

2017-09-17 Thread Jacky Li (JIRA)


[ 
https://issues.apache.org/jira/browse/CARBONDATA-1444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16169264#comment-16169264
 ] 

Jacky Li commented on CARBONDATA-1444:
--

Can you explain what kind of encoding will be added to support boolean data 
type?

> CarbonData unsupport Boolean data type,
> ---
>
> Key: CARBONDATA-1444
> URL: https://issues.apache.org/jira/browse/CARBONDATA-1444
> Project: CarbonData
>  Issue Type: Improvement
>  Components: sql
>Affects Versions: 1.2.0
>Reporter: xubo245
>Assignee: xubo245
>Priority: Minor
> Fix For: 1.2.0
>
>   Original Estimate: 240h
>  Time Spent: 2h
>  Remaining Estimate: 238h
>
>   Spark/Hive table support Boolean data type, the internal table also should 
> support Boolean data type.
>Boolean data type Range: TRUE or FALSE. Do not use quotation marks around 
> the TRUE and FALSE literal values. You can write the literal values in 
> uppercase, lowercase, or mixed case. The values queried from a table are 
> always returned in lowercase, true or false.
> Boolean data type column can be added to dimension and use single-byte to 
> store in carbon file.
> CarbonData should support boolean data type in following aspects:
>   1. create table 
>2.insert into table values and select
>3.load data
>4.filter



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] carbondata pull request #1362: [CARBONDATA-1444]Support boolean

2017-09-17 Thread jackylk

Github user jackylk commented on a diff in the pull request:

https://github.com/apache/carbondata/pull/1362#discussion_r139307667
  
--- Diff: 
core/src/main/java/org/apache/carbondata/core/datastore/page/encoding/bool/BooleanEncoderMeta.java
 ---
@@ -0,0 +1,40 @@
+package org.apache.carbondata.core.datastore.page.encoding.bool;
+
+import java.io.DataInput;
+import java.io.DataOutput;
+import java.io.IOException;
+
+import org.apache.carbondata.core.datastore.TableSpec;
+import 
org.apache.carbondata.core.datastore.page.encoding.ColumnPageEncoderMeta;
+import 
org.apache.carbondata.core.datastore.page.statistics.SimpleStatsResult;
+import org.apache.carbondata.core.metadata.datatype.DataType;
+import org.apache.carbondata.core.metadata.schema.table.Writable;
+
+public class BooleanEncoderMeta extends ColumnPageEncoderMeta implements 
Writable {
+  private String compressorName;
+
+  public BooleanEncoderMeta() {
+  }
+
+  public BooleanEncoderMeta(TableSpec.ColumnSpec columnSpec, DataType 
storeDataType,
+SimpleStatsResult stats, String 
compressorName) {
+super(columnSpec,storeDataType,stats,compressorName);
+this.compressorName = compressorName;
--- End diff --

compressorName is already stored in ColumnPageEncoderMeta, this 
BooleanEncoderMeta is not required.


---

[GitHub] carbondata pull request #1362: [CARBONDATA-1444]Support boolean

2017-09-17 Thread jackylk

Github user jackylk commented on a diff in the pull request:

https://github.com/apache/carbondata/pull/1362#discussion_r139307650
  
--- Diff: 
core/src/main/java/org/apache/carbondata/core/datastore/page/encoding/bool/BooleanConvert.java
 ---
@@ -0,0 +1,61 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.carbondata.core.datastore.page.encoding.bool;
+
+/**
+ * convert tools for boolean data type
+ */
+public class BooleanConvert {
--- End diff --

I think this is not required, it is simple conversion only.


---

[GitHub] carbondata pull request #1362: [CARBONDATA-1444]Support boolean

2017-09-17 Thread jackylk

Github user jackylk commented on a diff in the pull request:

https://github.com/apache/carbondata/pull/1362#discussion_r139307631
  
--- Diff: 
core/src/main/java/org/apache/carbondata/core/datastore/page/encoding/EncodingFactory.java
 ---
@@ -92,6 +90,10 @@ public ColumnPageDecoder createDecoder(List 
encodings, List

[GitHub] carbondata pull request #1362: [CARBONDATA-1444]Support boolean

2017-09-17 Thread jackylk

Github user jackylk commented on a diff in the pull request:

https://github.com/apache/carbondata/pull/1362#discussion_r139307244
  
--- Diff: 
core/src/main/java/org/apache/carbondata/core/datastore/page/ColumnPage.java ---
@@ -187,6 +179,7 @@ public static ColumnPage newPage(TableSpec.ColumnSpec 
columnSpec, DataType dataT
 case BYTE:
 case SHORT:
 case SHORT_INT:
+case BOOLEAN:
--- End diff --

Move it before BYTE. Please modify the same in all switch case


---

[GitHub] carbondata pull request #1362: [CARBONDATA-1444]Support boolean

2017-09-17 Thread jackylk

Github user jackylk commented on a diff in the pull request:

https://github.com/apache/carbondata/pull/1362#discussion_r139307234
  
--- Diff: 
core/src/main/java/org/apache/carbondata/core/constants/CarbonCommonConstants.java
 ---
@@ -23,6 +23,10 @@
 
 public final class CarbonCommonConstants {
   /**
+   * Boolean size in bytes
+   */
+  public static final int BOOLEAN_SIZE_IN_BYTE = 1;
--- End diff --

This is not required, it should be added in DataType enum


---

[GitHub] carbondata pull request #1362: [CARBONDATA-1444]Support boolean

2017-09-17 Thread jackylk

Github user jackylk commented on a diff in the pull request:

https://github.com/apache/carbondata/pull/1362#discussion_r139307225
  
--- Diff: 
core/src/main/java/org/apache/carbondata/core/constants/CarbonCommonConstants.java
 ---
@@ -707,6 +711,10 @@
*/
   public static final char BIG_INT_MEASURE = 'd';
   /**
+   * BOOLEAN_VALUE_MEASURE
+   */
+  public static final char BOOLEAN_MEASURE = 'a';
--- End diff --

This is not required. Others are for backward compatible only.


---

[GitHub] carbondata issue #1362: [CARBONDATA-1444]Support boolean

2017-09-17 Thread CarbonDataQA

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/1362
  
Build Failed  with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/198/



---

[GitHub] carbondata issue #1362: [CARBONDATA-1444]Support boolean

2017-09-17 Thread ravipesala

Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/1362
  
SDV Build Fail , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/830/



---

[GitHub] carbondata issue #1362: [CARBONDATA-1444]Support boolean

2017-09-17 Thread CarbonDataQA

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/1362
  
Build Failed with Spark 1.6, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/75/



---

[GitHub] carbondata pull request #1362: [CARBONDATA-1444]Support boolean

2017-09-17 Thread xubo245

GitHub user xubo245 opened a pull request:

https://github.com/apache/carbondata/pull/1362

[CARBONDATA-1444]Support boolean

CarbonData should support boolean data type in following aspects:
1. create table 
2.insert into table values and select
3.load data
4.filter

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/xubo245/carbondata supportBoolean

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/carbondata/pull/1362.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1362


commit 581288d82c53247e4f68e37d638bfe7cc7fb7ded
Author: xubo245 <601450...@qq.com>
Date:   2017-09-15T15:42:44Z

support Boolean data type

commit e7d279a41f6e58b6640f828d9fcb1c9962097cb8
Author: xubo245 <601450...@qq.com>
Date:   2017-09-17T07:57:40Z

fix findbugs

commit 6e8f5da599038649b22a1866425941f706650135
Author: xubo245 <601450...@qq.com>
Date:   2017-09-17T10:01:06Z

add test case




---

[jira] [Resolved] (CARBONDATA-1438) Unify the sort column and sort scope in create table command

2017-09-17 Thread Jacky Li (JIRA)


 [ 
https://issues.apache.org/jira/browse/CARBONDATA-1438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jacky Li resolved CARBONDATA-1438.
--
   Resolution: Fixed
Fix Version/s: 1.2.0

> Unify the sort column and sort scope in create table command
> 
>
> Key: CARBONDATA-1438
> URL: https://issues.apache.org/jira/browse/CARBONDATA-1438
> Project: CarbonData
>  Issue Type: Improvement
>Reporter: chenerlu
> Fix For: 1.2.0
>
>  Time Spent: 14h 40m
>  Remaining Estimate: 0h
>
> 1 Requirement
> Currently, Users can specify sort column in table properties when create 
> table. And when load data, users can also specify sort scope in load options.
> In order to improve the ease of use for users, it will be better to specify 
> the sort related parameters all in create table command.
> Once sort scope is specified in create table command, it will be used in load 
> data even users have specified in load options.
> 2 Detailed design
> 2.1   Task-01
> Requirement： Create table can support specify sort scope
> Implement: Take use of table properties (Map), will specify 
> sort scope in table properties by key/value pair, then existing interface 
> will be called to write this key/value pair into metastore.
> Will support Global Sort，Local Sort and No Sort，it can be specified in sql 
> command:
> CREATE TABLE tableWithGlobalSort (
> shortField SHORT,
> intField INT,
> bigintField LONG,
> doubleField DOUBLE,
> stringField STRING,
> timestampField TIMESTAMP,
> decimalField DECIMAL(18,2),
> dateField DATE,
> charField CHAR(5)
> )
> STORED BY 'carbondata'
> TBLPROPERTIES('SORT_COLUMNS'='stringField', 'SORT_SCOPE'='GLOBAL_SORT')
>  
> Tips：If the sort scope is global Sort, users should specify 
> GLOBAL_SORT_PARTITIONS. If users do not specify it, it will use the number of 
> map task. GLOBAL_SORT_PARTITIONS should be Integer type, the range is 
> [1,Integer.MaxValue]，it is only used when the sort scope is global sort. 
> Global Sort   Use orderby operator in spark, data is ordered in segment level.
> Local SortNode ordered, carbondata file is ordered if it is written by 
> one task. 
> No Sort   No sort
> Tips：key and value is case-insensitive.
> 2.2   Task-02
> Requirement:
> Load data in will support local sort, no sort, global sort 
> Ignore the sort scope specified in load data and use the parameter which 
> specified in create table.
> Currently, user can specify the sort scope and global sort partitions in load 
> options, After modification, it will ignore the sort scope which specified in 
> load options and will get sort scope from table properties.
> Current logic: sort scope is from load options
> NumberPrerequisiteSort scope
> 1 isSortTable is true && Sort Scope is Global SortGlobal 
> Sort(first check)
> 2 isSortTable is falseNo Sort
> 3 isSortTable is true Local Sort
> Tips: isSortTable is true means this table contains sort column or it 
> contains dimensions (except complex type), like string type.
> For example:
> Create table xxx1 (col1 string col2 int) stored by ‘carbondata’ --- sort table
> Create table xx1 (col1 int, col2 int) stored by ‘carbondata’ --- not sort 
> table
> Create table xx (col1 int, col2 string) stored by ‘carbondata’ tblproperties 
> (‘sort_column’=’col1’)  –- sort table
> New logic：sort scope is from create table
> NumberPrerequisiteCode branch
> 1 isSortTable = true && Sort Scope is Global Sort Global Sort(first check)
> 2 isSortTable= false || Sort Scope is No Sort No Sort
> 3 isSortTable is true && Sort Scope is Local Sort Local Sort
> 4 isSortTable is true，without specify Sort Scope  Local Sort, (Keep 
> current logic) 
> 3 Acceptance standard
> NumberAcceptance standard
> 1 Use can specify sort scope(global, local, no sort) when create carbon 
> table in sql type
> 2 Load data will ignore the sort scope specified in load options and will 
> use the parameter which specify in create table command. If user still 
> specify the sort scope in load options, will give warning and inform user 
> that he will use the sort scope which specified in create table.
> 4 Feature restrictions
> NA
> 5 Dependencies
> NA
> 6 Technical risk
> NA



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] carbondata pull request #1321: [CARBONDATA-1438] Unify the sort column and s...

2017-09-17 Thread asfgit

Github user asfgit closed the pull request at:

https://github.com/apache/carbondata/pull/1321


---

[GitHub] carbondata issue #1321: [CARBONDATA-1438] Unify the sort column and sort sco...

2017-09-17 Thread jackylk

Github user jackylk commented on the issue:

https://github.com/apache/carbondata/pull/1321
  
LGTM


---

[GitHub] carbondata issue #1321: [CARBONDATA-1438] Unify the sort column and sort sco...

2017-09-17 Thread ravipesala

Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/1321
  
SDV Build Success , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/829/



---

[GitHub] carbondata issue #1321: [CARBONDATA-1438] Unify the sort column and sort sco...

2017-09-17 Thread CarbonDataQA

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/1321
  
Build Success with Spark 1.6, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/74/



---

[GitHub] carbondata issue #1321: [CARBONDATA-1438] Unify the sort column and sort sco...

2017-09-17 Thread CarbonDataQA

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/1321
  
Build Success with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/197/



---

[GitHub] carbondata issue #1321: [CARBONDATA-1438] Unify the sort column and sort sco...

2017-09-17 Thread chenerlu

Github user chenerlu commented on the issue:

https://github.com/apache/carbondata/pull/1321
  
retest this please


---

[GitHub] carbondata issue #1321: [CARBONDATA-1438] Unify the sort column and sort sco...

2017-09-17 Thread ravipesala

Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/1321
  
SDV Build Success , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/828/



---

[GitHub] carbondata issue #1321: [CARBONDATA-1438] Unify the sort column and sort sco...

2017-09-17 Thread CarbonDataQA

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/1321
  
Build Success with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/196/



---

[GitHub] carbondata issue #1321: [CARBONDATA-1438] Unify the sort column and sort sco...

2017-09-17 Thread CarbonDataQA

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/1321
  
Build Success with Spark 1.6, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/73/



---

[GitHub] carbondata issue #1321: [CARBONDATA-1438] Unify the sort column and sort sco...

2017-09-17 Thread ravipesala

Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/1321
  
SDV Build Success , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/827/



---

68 matches

Mail list logo