[GitHub] carbondata issue #2056: [CARBONDATA-2238][DataLoad] Merge and spill in-memor...

2018-04-01 Thread ravipesala
Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/2056
  
SDV Build Fail , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/4239/



---


[GitHub] carbondata issue #2128: [CARBONDATA-2303] [WIP] If dataload is failed for pa...

2018-04-01 Thread CarbonDataQA
Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2128
  
Build Success with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/4733/



---


[GitHub] carbondata issue #2128: [CARBONDATA-2303] [WIP] If dataload is failed for pa...

2018-04-01 Thread CarbonDataQA
Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2128
  
Build Success with Spark 2.2.1, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/3506/



---


[jira] [Created] (CARBONDATA-2303) If dataload is failed for parition table then cleanup is not working.

2018-04-01 Thread Rahul Kumar (JIRA)
Rahul Kumar created CARBONDATA-2303:
---

 Summary: If dataload is failed for parition table then cleanup is 
not working.
 Key: CARBONDATA-2303
 URL: https://issues.apache.org/jira/browse/CARBONDATA-2303
 Project: CarbonData
  Issue Type: Bug
Reporter: Rahul Kumar


Test Step : 

1. create table 

2. load data (make sure data load is failed either manually or other)

3. clean files for table

*Expected Output*: after clean files data from HDFS should be delete for 
segments which is Marked for delete.
*Actual Output:*Alter cleanup ,data are not deleted from HDFS

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[GitHub] carbondata issue #2129: [CARBONDATA-2298][BACKPORT-1.3]Delete segment lock f...

2018-04-01 Thread ravipesala
Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/2129
  
SDV Build Fail , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/4238/



---


[GitHub] carbondata issue #2056: [CARBONDATA-2238][DataLoad] Merge and spill in-memor...

2018-04-01 Thread ravipesala
Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/2056
  
SDV Build Fail , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/4237/



---


[GitHub] carbondata issue #2130: [CARBONDATA-2302]Fix some bugs when separate visible...

2018-04-01 Thread zzcclp
Github user zzcclp commented on the issue:

https://github.com/apache/carbondata/pull/2130
  
@ravipesala @jackylk  please review, thanks


---


[GitHub] carbondata issue #2128: [WIP] partition table clean files fixed

2018-04-01 Thread rahulforallp
Github user rahulforallp commented on the issue:

https://github.com/apache/carbondata/pull/2128
  
retest this please


---


[GitHub] carbondata issue #2126: [CARBONDATA-2300] Add ENABLE_UNSAFE_IN_QUERY_EXECUTI...

2018-04-01 Thread ravipesala
Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/2126
  
SDV Build Success , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/4236/



---


[GitHub] carbondata issue #2056: [CARBONDATA-2238][DataLoad] Merge and spill in-memor...

2018-04-01 Thread CarbonDataQA
Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2056
  
Build Success with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/4732/



---


[GitHub] carbondata issue #2056: [CARBONDATA-2238][DataLoad] Merge and spill in-memor...

2018-04-01 Thread CarbonDataQA
Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2056
  
Build Failed with Spark 2.2.1, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/3505/



---


[GitHub] carbondata issue #2083: [CARBONDATA-2269]Support Query On PreAggregate table...

2018-04-01 Thread kumarvishal09
Github user kumarvishal09 commented on the issue:

https://github.com/apache/carbondata/pull/2083
  
retest sdv please


---


[GitHub] carbondata issue #2099: [CARBONDATA-2276][SDK] Support API to read schema in...

2018-04-01 Thread CarbonDataQA
Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2099
  
Build Success with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/4731/



---


[GitHub] carbondata issue #2099: [CARBONDATA-2276][SDK] Support API to read schema in...

2018-04-01 Thread CarbonDataQA
Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2099
  
Build Success with Spark 2.2.1, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/3503/



---


[GitHub] carbondata issue #2126: [CARBONDATA-2300] Add ENABLE_UNSAFE_IN_QUERY_EXECUTI...

2018-04-01 Thread CarbonDataQA
Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2126
  
Build Success with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/4727/



---


[GitHub] carbondata issue #2126: [CARBONDATA-2300] Add ENABLE_UNSAFE_IN_QUERY_EXECUTI...

2018-04-01 Thread CarbonDataQA
Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2126
  
Build Success with Spark 2.2.1, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/3500/



---


[GitHub] carbondata issue #2130: [CARBONDATA-2302]Fix some bugs when separate visible...

2018-04-01 Thread ravipesala
Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/2130
  
SDV Build Success , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/4235/



---


[GitHub] carbondata issue #2056: [CARBONDATA-2238][DataLoad] Merge and spill in-memor...

2018-04-01 Thread CarbonDataQA
Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2056
  
Build Failed  with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/4730/



---


[GitHub] carbondata issue #2127: [CARBONDATA-2301][SDK] CarbonStore interface and que...

2018-04-01 Thread CarbonDataQA
Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2127
  
Build Failed with Spark 2.2.1, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/3502/



---


[GitHub] carbondata pull request #2126: [CARBONDATA-2300] Add ENABLE_UNSAFE_IN_QUERY_...

2018-04-01 Thread jackylk
Github user jackylk commented on a diff in the pull request:

https://github.com/apache/carbondata/pull/2126#discussion_r178482124
  
--- Diff: 
integration/presto/src/main/java/org/apache/carbondata/presto/impl/CarbonTableReader.java
 ---
@@ -372,6 +372,11 @@ public TBase create() {
   
CarbonProperties.getInstance().addProperty(CarbonCommonConstants.UNSAFE_WORKING_MEMORY_IN_MB,
   config.getUnsafeMemoryInMb());
 }
+if(config.getEnableUnsafeInQueryExecution() != null) {
+  CarbonProperties.getInstance()
--- End diff --

change the style to like
```
 CarbonProperties.getInstance().addProperty(
 CarbonCommonConstants.ENABLE_UNSAFE_IN_QUERY_EXECUTION,
 config.getEnableUnsafeInQueryExecution());
```


---


[GitHub] carbondata issue #2127: [CARBONDATA-2301][SDK] CarbonStore interface and que...

2018-04-01 Thread CarbonDataQA
Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2127
  
Build Failed  with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/4729/



---


[GitHub] carbondata issue #2099: [CARBONDATA-2276][SDK] Support API to read schema in...

2018-04-01 Thread jackylk
Github user jackylk commented on the issue:

https://github.com/apache/carbondata/pull/2099
  
retest this please


---


[GitHub] carbondata issue #2129: [CARBONDATA-2298][BACKPORT-1.3]Delete segment lock f...

2018-04-01 Thread zzcclp
Github user zzcclp commented on the issue:

https://github.com/apache/carbondata/pull/2129
  
retest sdv please


---


[GitHub] carbondata issue #2056: [CARBONDATA-2238][DataLoad] Merge and spill in-memor...

2018-04-01 Thread CarbonDataQA
Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2056
  
Build Failed  with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/4728/



---


[GitHub] carbondata issue #2083: [CARBONDATA-2269]Support Query On PreAggregate table...

2018-04-01 Thread CarbonDataQA
Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2083
  
Build Success with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/4726/



---


[GitHub] carbondata issue #2056: [CARBONDATA-2238][DataLoad] Merge and spill in-memor...

2018-04-01 Thread CarbonDataQA
Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2056
  
Build Failed with Spark 2.2.1, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/3501/



---


[GitHub] carbondata issue #2083: [CARBONDATA-2269]Support Query On PreAggregate table...

2018-04-01 Thread CarbonDataQA
Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2083
  
Build Success with Spark 2.2.1, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/3499/



---


[GitHub] carbondata issue #2130: [CARBONDATA-2302]Fix some bugs when separate visible...

2018-04-01 Thread ravipesala
Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/2130
  
retest sdv please


---


[GitHub] carbondata issue #1853: [CARBONDATA-2072][TEST] Add dropTables method for op...

2018-04-01 Thread xubo245
Github user xubo245 commented on the issue:

https://github.com/apache/carbondata/pull/1853
  
@sraghunandan @ravipesala @QiangCai @kumarvishal09 @jackylk please review it


---


[GitHub] carbondata issue #1990: [CARBONDATA-2195] Add new test case for partition fe...

2018-04-01 Thread xubo245
Github user xubo245 commented on the issue:

https://github.com/apache/carbondata/pull/1990
  
@anubhav100 How to handle this PR?


---


[GitHub] carbondata issue #2073: [CARBONDATA-2260] CarbonThriftServer should support ...

2018-04-01 Thread xubo245
Github user xubo245 commented on the issue:

https://github.com/apache/carbondata/pull/2073
  
@jackylk please review it.


---


[GitHub] carbondata issue #2083: [CARBONDATA-2269]Support Query On PreAggregate table...

2018-04-01 Thread ravipesala
Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/2083
  
SDV Build Fail , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/4234/



---


[GitHub] carbondata issue #2099: [CARBONDATA-2276][SDK] Support API to read schema in...

2018-04-01 Thread CarbonDataQA
Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2099
  
Build Failed with Spark 2.2.1, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/3498/



---


[GitHub] carbondata issue #2099: [CARBONDATA-2276][SDK] Support API to read schema in...

2018-04-01 Thread CarbonDataQA
Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2099
  
Build Success with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/4725/



---


[GitHub] carbondata issue #2130: [CARBONDATA-2302]Fix some bugs when separate visible...

2018-04-01 Thread ravipesala
Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/2130
  
SDV Build Fail , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/4233/



---


[GitHub] carbondata issue #2130: [CARBONDATA-2302]Fix some bugs when separate visible...

2018-04-01 Thread ravipesala
Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/2130
  
retest sdv please


---


[GitHub] carbondata issue #2099: [CARBONDATA-2276][SDK] Support API to read schema in...

2018-04-01 Thread ravipesala
Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/2099
  
SDV Build Fail , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/4232/



---


[jira] [Updated] (CARBONDATA-2302) Fix some bugs when separate visible and invisible segments info into two files

2018-04-01 Thread Zhichao Zhang (JIRA)

 [ 
https://issues.apache.org/jira/browse/CARBONDATA-2302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhichao  Zhang updated CARBONDATA-2302:
---
Description: 
There are some bugs when separate visible and invisible segments info into two 
files:
 # It will not delete physical data of history segments after separating
 # Generate duplicated segment id.

  was:
There are some bugs where separate visible and invisible segments info into two 
files:
 # It will not delete physical data of history segments after separating
 # Generate duplicated segment id.


> Fix some bugs when separate visible and invisible segments info into two files
> --
>
> Key: CARBONDATA-2302
> URL: https://issues.apache.org/jira/browse/CARBONDATA-2302
> Project: CarbonData
>  Issue Type: Bug
>  Components: core, data-load
>Affects Versions: 1.4.0
>Reporter: Zhichao  Zhang
>Assignee: Zhichao  Zhang
>Priority: Major
> Fix For: 1.4.0
>
>  Time Spent: 1h 40m
>  Remaining Estimate: 0h
>
> There are some bugs when separate visible and invisible segments info into 
> two files:
>  # It will not delete physical data of history segments after separating
>  # Generate duplicated segment id.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[GitHub] carbondata issue #2130: [CARBONDATA-2302]Fix some bugs when separate visible...

2018-04-01 Thread ravipesala
Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/2130
  
SDV Build Fail , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/4231/



---


[GitHub] carbondata issue #2127: [CARBONDATA-2301] CarbonStore interface and query AP...

2018-04-01 Thread CarbonDataQA
Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2127
  
Build Failed  with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/4724/



---


[GitHub] carbondata issue #2127: [CARBONDATA-2301] CarbonStore interface and query AP...

2018-04-01 Thread CarbonDataQA
Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2127
  
Build Failed with Spark 2.2.1, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/3497/



---


[GitHub] carbondata issue #2127: [CARBONDATA-2301] CarbonStore interface and query AP...

2018-04-01 Thread ravipesala
Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/2127
  
SDV Build Fail , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/4230/



---


[GitHub] carbondata issue #2130: [CARBONDATA-2302]Fix some bugs when separate visible...

2018-04-01 Thread zzcclp
Github user zzcclp commented on the issue:

https://github.com/apache/carbondata/pull/2130
  
retest sdv please


---


[GitHub] carbondata issue #2130: [CARBONDATA-2302]Fix some bugs when separate visible...

2018-04-01 Thread ravipesala
Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/2130
  
SDV Build Fail , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/4229/



---


[GitHub] carbondata issue #2130: [CARBONDATA-2302]Fix some bugs when separate visible...

2018-04-01 Thread zzcclp
Github user zzcclp commented on the issue:

https://github.com/apache/carbondata/pull/2130
  
retest sdv please


---


[GitHub] carbondata issue #2099: [CARBONDATA-2276][SDK] Support API to read schema in...

2018-04-01 Thread CarbonDataQA
Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2099
  
Build Success with Spark 2.2.1, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/3496/



---


[GitHub] carbondata issue #2099: [CARBONDATA-2276][SDK] Support API to read schema in...

2018-04-01 Thread CarbonDataQA
Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2099
  
Build Success with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/4723/



---


[GitHub] carbondata issue #2129: [CARBONDATA-2298][BACKPORT-1.3]Delete segment lock f...

2018-04-01 Thread CarbonDataQA
Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2129
  
Build Success with Spark 2.2.1, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/3495/



---


[GitHub] carbondata issue #2129: [CARBONDATA-2298][BACKPORT-1.3]Delete segment lock f...

2018-04-01 Thread CarbonDataQA
Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2129
  
Build Success with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/4722/



---


[GitHub] carbondata issue #2099: [CARBONDATA-2276][SDK] Support API to read schema in...

2018-04-01 Thread ravipesala
Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/2099
  
retest this please


---


[GitHub] carbondata pull request #2099: [CARBONDATA-2276][SDK] Support API to read sc...

2018-04-01 Thread ravipesala
Github user ravipesala commented on a diff in the pull request:

https://github.com/apache/carbondata/pull/2099#discussion_r178471774
  
--- Diff: 
core/src/main/java/org/apache/carbondata/core/util/CarbonUtil.java ---
@@ -2392,6 +2376,12 @@ static DataType thriftDataTyopeToWrapperDataType(
   tableInfo.setDataMapSchemas(null);
   return tableInfo;
 } else {
+  TBaseCreator createTBase = new ThriftReader.TBaseCreator() {
--- End diff --

I think you can use `readSchemaFile` method here.


---


[GitHub] carbondata issue #2130: [CARBONDATA-2302]Fix some bugs when separate visible...

2018-04-01 Thread ravipesala
Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/2130
  
SDV Build Fail , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/4228/



---


[GitHub] carbondata issue #2129: [CARBONDATA-2298][BACKPORT-1.3]Delete segment lock f...

2018-04-01 Thread zzcclp
Github user zzcclp commented on the issue:

https://github.com/apache/carbondata/pull/2129
  
retest this please


---


[GitHub] carbondata issue #2130: [CARBONDATA-2302]Fix some bugs when separate visible...

2018-04-01 Thread zzcclp
Github user zzcclp commented on the issue:

https://github.com/apache/carbondata/pull/2130
  
retest sdv please


---


[GitHub] carbondata issue #2130: [CARBONDATA-2302]Fix some bugs when separate visible...

2018-04-01 Thread CarbonDataQA
Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2130
  
Build Success with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/4720/



---


[GitHub] carbondata issue #2130: [CARBONDATA-2302]Fix some bugs when separate visible...

2018-04-01 Thread CarbonDataQA
Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2130
  
Build Success with Spark 2.2.1, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/3493/



---


[GitHub] carbondata issue #2130: [CARBONDATA-2302]Fix some bugs when separate visible...

2018-04-01 Thread ravipesala
Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/2130
  
SDV Build Fail , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/4227/



---


[GitHub] carbondata issue #2129: [CARBONDATA-2298][BACKPORT-1.3]Delete segment lock f...

2018-04-01 Thread CarbonDataQA
Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2129
  
Build Failed  with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/4721/



---


[GitHub] carbondata issue #2128: [WIP] partition table clean files fixed

2018-04-01 Thread ravipesala
Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/2128
  
SDV Build Fail , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/4226/



---


[GitHub] carbondata issue #2129: [CARBONDATA-2298][BACKPORT-1.3]Delete segment lock f...

2018-04-01 Thread CarbonDataQA
Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2129
  
Build Success with Spark 2.2.1, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/3494/



---


[GitHub] carbondata issue #2128: [WIP] partition table clean files fixed

2018-04-01 Thread ravipesala
Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/2128
  
SDV Build Fail , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/4225/



---


[GitHub] carbondata pull request #2130: [CARBONDATA-2302]Fix some bugs when separate ...

2018-04-01 Thread zzcclp
GitHub user zzcclp opened a pull request:

https://github.com/apache/carbondata/pull/2130

[CARBONDATA-2302]Fix some bugs when separate visible and invisible segments 
info into two files

There are some bugs where separate visible and invisible segments info into 
two files:
1. It will not delete physical data of history segments after separating
2. Generate duplicated segment id

Be sure to do all of the following checklist to help us incorporate 
your contribution quickly and easily:

 - [ ] Any interfaces changed?  No
 
 - [ ] Any backward compatibility impacted?  No
 
 - [ ] Document update required?  No

 - [ ] Testing done
Please provide details on 
- Whether new unit test cases have been added or why no new tests 
are required?
- How it is tested? Please attach test report.
- Is it a performance related change? Please attach the performance 
test report.
- Any additional information to help reviewers in testing this 
change.
   
 - [ ] For large changes, please consider breaking it into sub-tasks under 
an umbrella JIRA. 



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/zzcclp/carbondata CARBONDATA-2302

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/carbondata/pull/2130.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2130


commit 5aae4310110fb69f0def02654bb49a96fe606bf3
Author: Zhang Zhichao <441586683@...>
Date:   2018-04-01T17:09:52Z

[CARBONDATA-2302]Fix some bugs when separate visible and invisible segments 
info into two files

There are some bugs where separate visible and invisible segments info into 
two files:
1. It will not delete physical data of history segments after separating
2. Generate duplicated segment id




---


[GitHub] carbondata issue #2129: [CARBONDATA-2298][BACKPORT-1.3]Delete segment lock f...

2018-04-01 Thread zzcclp
Github user zzcclp commented on the issue:

https://github.com/apache/carbondata/pull/2129
  
retest this please


---


[GitHub] carbondata issue #2129: [CARBONDATA-2298][BACKPORT-1.3]Delete segment lock f...

2018-04-01 Thread ravipesala
Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/2129
  
SDV Build Fail , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/4224/



---


[GitHub] carbondata issue #2128: [WIP] partition table clean files fixed

2018-04-01 Thread ravipesala
Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/2128
  
SDV Build Success , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/4223/



---


[GitHub] carbondata issue #2128: [WIP] partition table clean files fixed

2018-04-01 Thread CarbonDataQA
Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2128
  
Build Failed  with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/4719/



---


[GitHub] carbondata issue #2129: [CARBONDATA-2298][BACKPORT-1.3]Delete segment lock f...

2018-04-01 Thread CarbonDataQA
Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2129
  
Build Success with Spark 2.2.1, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/3490/



---


[GitHub] carbondata issue #2128: [WIP] partition table clean files fixed

2018-04-01 Thread CarbonDataQA
Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2128
  
Build Failed with Spark 2.2.1, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/3492/



---


[GitHub] carbondata issue #2128: [WIP] partition table clean files fixed

2018-04-01 Thread ravipesala
Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/2128
  
SDV Build Success , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/4222/



---


[GitHub] carbondata pull request #2129: [CARBONDATA-2298][BACKPORT-1.3]Delete segment...

2018-04-01 Thread zzcclp
GitHub user zzcclp opened a pull request:

https://github.com/apache/carbondata/pull/2129

[CARBONDATA-2298][BACKPORT-1.3]Delete segment lock files before update 
metadata

If there are some COMPACTED segments and their last modified time is within 
one hour, the segment lock files deletion operation will not be executed.

Be sure to do all of the following checklist to help us incorporate 
your contribution quickly and easily:

 - [ ] Any interfaces changed? No
 
 - [ ] Any backward compatibility impacted? No
 
 - [ ] Document update required? No

 - [ ] Testing done
Please provide details on 
- Whether new unit test cases have been added or why no new tests 
are required?
- How it is tested? Please attach test report.
- Is it a performance related change? Please attach the performance 
test report.
- Any additional information to help reviewers in testing this 
change.
   
 - [ ] For large changes, please consider breaking it into sub-tasks under 
an umbrella JIRA. 



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/zzcclp/carbondata CARBONDATA-2298_backport_1.3

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/carbondata/pull/2129.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2129


commit 74dc33e7bba6c34c368b7df3a9aee17e891a5635
Author: Zhang Zhichao <441586683@...>
Date:   2018-04-01T16:18:06Z

[CARBONDATA-2298][BACKPORT-1.3]Delete segment lock files before update 
metadata

If there are some COMPACTED segments and their last modified time is within 
one hour, the segment lock files deletion operation will not be executed.




---


[jira] [Created] (CARBONDATA-2302) Fix some bugs when separate visible and invisible segments info into two files

2018-04-01 Thread Zhichao Zhang (JIRA)
Zhichao  Zhang created CARBONDATA-2302:
--

 Summary: Fix some bugs when separate visible and invisible 
segments info into two files
 Key: CARBONDATA-2302
 URL: https://issues.apache.org/jira/browse/CARBONDATA-2302
 Project: CarbonData
  Issue Type: Bug
  Components: core, data-load
Affects Versions: 1.4.0
Reporter: Zhichao  Zhang
Assignee: Zhichao  Zhang
 Fix For: 1.4.0


There are some bugs where separate visible and invisible segments info into two 
files:
 # It will not delete physical data of history segments after separating
 # Generate duplicated segment id.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[GitHub] carbondata issue #2128: [WIP] partition table clean files fixed

2018-04-01 Thread CarbonDataQA
Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2128
  
Build Failed with Spark 2.2.1, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/3489/



---


[GitHub] carbondata issue #1713: [CARBONDATA-1899] Optimize CarbonData concurrency te...

2018-04-01 Thread xubo245
Github user xubo245 commented on the issue:

https://github.com/apache/carbondata/pull/1713
  
@jackylk CI pass, please check it.


---


[GitHub] carbondata issue #2127: [CARBONDATA-2301] CarbonStore interface and query AP...

2018-04-01 Thread ravipesala
Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/2127
  
SDV Build Fail , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/4221/



---


[GitHub] carbondata issue #2128: [WIP] partition table clean files fixed

2018-04-01 Thread ravipesala
Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/2128
  
SDV Build Fail , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/4220/



---


[GitHub] carbondata issue #2127: [CARBONDATA-2301] CarbonStore interface and query AP...

2018-04-01 Thread ravipesala
Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/2127
  
SDV Build Success , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/4219/



---


[GitHub] carbondata issue #2128: [WIP] partition table clean files fixed

2018-04-01 Thread CarbonDataQA
Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2128
  
Build Failed  with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/4713/



---


[GitHub] carbondata issue #2128: [WIP] partition table clean files fixed

2018-04-01 Thread CarbonDataQA
Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2128
  
Build Failed with Spark 2.2.1, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/3486/



---


[GitHub] carbondata issue #2099: [CARBONDATA-2276][SDK] Support API to read schema in...

2018-04-01 Thread ravipesala
Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/2099
  
SDV Build Success , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/4218/



---


[GitHub] carbondata issue #2127: [CARBONDATA-2301] CarbonStore interface and query AP...

2018-04-01 Thread CarbonDataQA
Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2127
  
Build Failed with Spark 2.2.1, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/3487/



---


[GitHub] carbondata issue #2099: [CARBONDATA-2276][SDK] Support API to read schema in...

2018-04-01 Thread CarbonDataQA
Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2099
  
Build Success with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/4711/



---


[GitHub] carbondata issue #2099: [CARBONDATA-2276][SDK] Support API to read schema in...

2018-04-01 Thread CarbonDataQA
Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2099
  
Build Failed with Spark 2.2.1, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/3484/



---


[GitHub] carbondata pull request #2125: [CARBONDATA-2299]Support showing all segment ...

2018-04-01 Thread jackylk
Github user jackylk commented on a diff in the pull request:

https://github.com/apache/carbondata/pull/2125#discussion_r178457078
  
--- Diff: 
integration/spark2/src/test/scala/org/apache/spark/util/CarbonCommandSuite.scala
 ---
@@ -190,4 +190,21 @@ class CarbonCommandSuite extends Spark2QueryTest with 
BeforeAndAfterAll {
 dropTable(tableName)
   }
 
+  test("show history segments") {
+val tableName = "test_tablestatus_history"
+sql(s"drop table if exists ${tableName}")
+sql(s"create table ${tableName} (name String, age int) stored by 
'carbondata' "
+  + 
"TBLPROPERTIES('AUTO_LOAD_MERGE'='true','COMPACTION_LEVEL_THRESHOLD'='2,2')")
+val carbonTable = 
CarbonMetadata.getInstance().getCarbonTable("default", tableName)
+sql(s"insert into ${tableName} select 'abc1',1")
+sql(s"insert into ${tableName} select 'abc2',2")
+sql(s"insert into ${tableName} select 'abc3',3")
+assert(sql(s"show segments for table ${tableName}").collect().length 
== 4)
+assert(sql(s"show history segments for table 
${tableName}").collect().length == 4)
+sql(s"clean files for table ${tableName}")
+assert(sql(s"show segments for table ${tableName}").collect().length 
== 2)
+assert(sql(s"show history segments for table 
${tableName}").collect().length == 4)
--- End diff --

can you assert the segment name in the collect result


---


[GitHub] carbondata issue #2127: [CARBONDATA-2301] CarbonStore interface and query AP...

2018-04-01 Thread CarbonDataQA
Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2127
  
Build Failed  with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/4712/



---


[GitHub] carbondata issue #2127: [CARBONDATA-2301] CarbonStore interface and query AP...

2018-04-01 Thread CarbonDataQA
Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2127
  
Build Failed with Spark 2.2.1, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/3485/



---


[GitHub] carbondata issue #2127: [CARBONDATA-2301] CarbonStore interface and query AP...

2018-04-01 Thread ravipesala
Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/2127
  
SDV Build Fail , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/4217/



---


[GitHub] carbondata issue #1713: [CARBONDATA-1899] Optimize CarbonData concurrency te...

2018-04-01 Thread ravipesala
Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/1713
  
SDV Build Success , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/4216/



---


[GitHub] carbondata issue #1713: [CARBONDATA-1899] Optimize CarbonData concurrency te...

2018-04-01 Thread CarbonDataQA
Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/1713
  
Build Success with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/4709/



---


[GitHub] carbondata issue #1713: [CARBONDATA-1899] Optimize CarbonData concurrency te...

2018-04-01 Thread CarbonDataQA
Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/1713
  
Build Success with Spark 2.2.1, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/3482/



---


[GitHub] carbondata issue #2127: [CARBONDATA-2301] CarbonStore interface and query AP...

2018-04-01 Thread CarbonDataQA
Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2127
  
Build Failed  with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/4710/



---


[GitHub] carbondata pull request #2128: [WIP] partition table clean files fixed

2018-04-01 Thread rahulforallp
GitHub user rahulforallp opened a pull request:

https://github.com/apache/carbondata/pull/2128

[WIP] partition table clean files fixed

Be sure to do all of the following checklist to help us incorporate 
your contribution quickly and easily:

 - [ ] Any interfaces changed?
 
 - [ ] Any backward compatibility impacted?
 
 - [ ] Document update required?

 - [ ] Testing done
Please provide details on 
- Whether new unit test cases have been added or why no new tests 
are required?
- How it is tested? Please attach test report.
- Is it a performance related change? Please attach the performance 
test report.
- Any additional information to help reviewers in testing this 
change.
   
 - [ ] For large changes, please consider breaking it into sub-tasks under 
an umbrella JIRA. 



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/rahulforallp/incubator-carbondata 
part_tab_cleanFile

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/carbondata/pull/2128.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2128


commit 8044edb5afa858fa72ae7b2d0d1cf0685cf92597
Author: rahulforallp 
Date:   2018-04-01T12:08:51Z

partition table clean files fixed




---


[GitHub] carbondata issue #2127: [CARBONDATA-2301] CarbonStore interface and query AP...

2018-04-01 Thread CarbonDataQA
Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/2127
  
Build Failed with Spark 2.2.1, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/3483/



---


[GitHub] carbondata pull request #2127: [CARBONDATA-2301] CarbonStore interface and q...

2018-04-01 Thread jackylk
GitHub user jackylk opened a pull request:

https://github.com/apache/carbondata/pull/2127

[CARBONDATA-2301] CarbonStore interface and query API

User should be able to query carbondata using CarbonStore interface.
1. Get API: It can be used for filter query. It accepts projection column 
names and filter expression, and returns matched rows.
2. SQL API: it accepts SQL statement and return query result set.

This PR is on top of #2099 

 - [ ] Any interfaces changed?
 
 - [ ] Any backward compatibility impacted?
 
 - [ ] Document update required?

 - [ ] Testing done
Please provide details on 
- Whether new unit test cases have been added or why no new tests 
are required?
- How it is tested? Please attach test report.
- Is it a performance related change? Please attach the performance 
test report.
- Any additional information to help reviewers in testing this 
change.
   
 - [ ] For large changes, please consider breaking it into sub-tasks under 
an umbrella JIRA. 


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/jackylk/incubator-carbondata carbonstore-query

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/carbondata/pull/2127.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2127


commit 0909d4b205962c6fd4a859eb7b55c7806ae5c370
Author: Jacky Li 
Date:   2018-03-24T14:38:20Z

support read schema

commit 67e150902e4c0ec75eb07225306eeecd14afbef1
Author: Jacky Li 
Date:   2018-03-24T16:05:37Z

fix style:

commit 062616e2ba2431d33cf0374306020e01b3f863d3
Author: Jacky Li 
Date:   2018-03-24T16:06:20Z

fix style

commit 25be162f51070791da4a35c7171fbb1df5cb1739
Author: Jacky Li 
Date:   2018-04-01T08:30:22Z

support CarbonStore API




---


[jira] [Created] (CARBONDATA-2301) Support query interface in CarbonStore

2018-04-01 Thread Jacky Li (JIRA)
Jacky Li created CARBONDATA-2301:


 Summary: Support query interface in CarbonStore
 Key: CARBONDATA-2301
 URL: https://issues.apache.org/jira/browse/CARBONDATA-2301
 Project: CarbonData
  Issue Type: Sub-task
Reporter: Jacky Li
Assignee: Jacky Li
 Fix For: 1.4.0


User should be able to query carbondata using CarbonStore API.
1. Get API: It can be used for filter query. It accepts projection column names 
and filter expression, and returns matched rows.
2. SQL API: it accepts SQL statement and return query result set.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Resolved] (CARBONDATA-1601) Add carbon store module

2018-04-01 Thread Jacky Li (JIRA)

 [ 
https://issues.apache.org/jira/browse/CARBONDATA-1601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jacky Li resolved CARBONDATA-1601.
--
Resolution: Fixed
  Assignee: Jacky Li

> Add carbon store module
> ---
>
> Key: CARBONDATA-1601
> URL: https://issues.apache.org/jira/browse/CARBONDATA-1601
> Project: CarbonData
>  Issue Type: Sub-task
>Reporter: Jacky Li
>Assignee: Jacky Li
>Priority: Major
> Fix For: 1.4.0
>
>  Time Spent: 3h
>  Remaining Estimate: 0h
>
> Add carbondata-store module and move corresponding code from integration 
> module



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[GitHub] carbondata pull request #1713: [CARBONDATA-1899] Optimize CarbonData concurr...

2018-04-01 Thread xubo245
Github user xubo245 commented on a diff in the pull request:

https://github.com/apache/carbondata/pull/1713#discussion_r178455504
  
--- Diff: 
examples/spark2/src/main/scala/org/apache/carbondata/benchmark/ConcurrentQueryBenchmark.scala
 ---
@@ -0,0 +1,575 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.carbondata.benchmark
+
+import java.io.File
+import java.text.SimpleDateFormat
+import java.util
+import java.util.Date
+import java.util.concurrent.{Callable, Executors, Future, TimeUnit}
+
+import scala.util.Random
+
+import org.apache.spark.sql.{DataFrame, Row, SaveMode, SparkSession}
+import org.apache.spark.sql.types._
+
+import org.apache.carbondata.core.constants.{CarbonCommonConstants, 
CarbonVersionConstants}
+import org.apache.carbondata.core.util.{CarbonProperties, CarbonUtil}
+
+// scalastyle:off println
+/**
+ * Test concurrent query performance of CarbonData
+ *
+ * This benchmark will print out some information:
+ * 1.Environment information
+ * 2.Parameters information
+ * 3.concurrent query performance result using parquet format
+ * 4.concurrent query performance result using CarbonData format
+ *
+ * This benchmark default run in local model,
+ * user can change 'runInLocal' to false if want to run in cluster,
+ * user can change variables like:
+ *
+ * spark-submit \
+--class org.apache.carbondata.benchmark.ConcurrentQueryBenchmark \
--- End diff --

ok


---


[GitHub] carbondata pull request #1713: [CARBONDATA-1899] Optimize CarbonData concurr...

2018-04-01 Thread xubo245
Github user xubo245 commented on a diff in the pull request:

https://github.com/apache/carbondata/pull/1713#discussion_r178455501
  
--- Diff: 
examples/spark2/src/main/scala/org/apache/carbondata/benchmark/ConcurrentQueryBenchmark.scala
 ---
@@ -0,0 +1,575 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.carbondata.benchmark
+
+import java.io.File
+import java.text.SimpleDateFormat
+import java.util
+import java.util.Date
+import java.util.concurrent.{Callable, Executors, Future, TimeUnit}
+
+import scala.util.Random
+
+import org.apache.spark.sql.{DataFrame, Row, SaveMode, SparkSession}
+import org.apache.spark.sql.types._
+
+import org.apache.carbondata.core.constants.{CarbonCommonConstants, 
CarbonVersionConstants}
+import org.apache.carbondata.core.util.{CarbonProperties, CarbonUtil}
+
+// scalastyle:off println
+/**
+ * Test concurrent query performance of CarbonData
+ *
+ * This benchmark will print out some information:
+ * 1.Environment information
+ * 2.Parameters information
+ * 3.concurrent query performance result using parquet format
+ * 4.concurrent query performance result using CarbonData format
+ *
+ * This benchmark default run in local model,
+ * user can change 'runInLocal' to false if want to run in cluster,
+ * user can change variables like:
+ *
+ * spark-submit \
+--class org.apache.carbondata.benchmark.ConcurrentQueryBenchmark \
+--master  yarn \
+--deploy-mode client \
+--driver-memory 16g \
+--executor-cores 4g \
+--executor-memory 24g \
+--num-executors 3  \
+concurrencyTest.jar \
+totalNum threadNum taskNum resultIsEmpty runInLocal generateFile 
deleteFile
+ * details in initParameters method of this benchmark
+ */
+object ConcurrentQueryBenchmark {
+
+  // generate number of data
+  var totalNum = 1 * 10 * 1000
+  // the number of thread pool
+  var threadNum = 16
+  // task number of spark sql query
+  var taskNum = 100
+  // whether is result empty, if true then result is empty
+  var resultIsEmpty = true
+  // the store path of task details
+  var path: String = "/tmp/carbondata"
+  // whether run in local or cluster
+  var runInLocal = true
+  // whether generate new file
+  var generateFile = true
+  // whether delete file
+  var deleteFile = true
+
+  val cardinalityId = 100 * 1000 * 1000
+  val cardinalityCity = 6
+
+  def parquetTableName: String = "Num" + totalNum + "_" + 
"comparetest_parquet"
+
+  def orcTableName: String = "Num" + totalNum + "_" + "comparetest_orc"
+
+  def carbonTableName(version: String): String =
+"Num" + totalNum + "_" + s"comparetest_carbonV$version"
+
+
--- End diff --

ok


---


[GitHub] carbondata pull request #1713: [CARBONDATA-1899] Optimize CarbonData concurr...

2018-04-01 Thread xubo245
Github user xubo245 commented on a diff in the pull request:

https://github.com/apache/carbondata/pull/1713#discussion_r178455498
  
--- Diff: 
examples/spark2/src/main/scala/org/apache/carbondata/benchmark/GenerateData.scala
 ---
@@ -0,0 +1,81 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.carbondata.benchmark
+
+import org.apache.spark.sql.{DataFrame, Row, SparkSession}
+import org.apache.spark.sql.types._
+
+object GenerateData {
--- End diff --

ok


---


[GitHub] carbondata issue #1853: [CARBONDATA-2072][TEST] Add dropTables method for op...

2018-04-01 Thread CarbonDataQA
Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/1853
  
Build Success with Spark 2.2.1, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/3481/



---


[GitHub] carbondata issue #1853: [CARBONDATA-2072][TEST] Add dropTables method for op...

2018-04-01 Thread CarbonDataQA
Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/1853
  
Build Success with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/4708/



---


  1   2   >