[GitHub] carbondata issue #1867: [CARBONDATA-2055][Streaming][WIP]Support integrating...

[ravipesala]

Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/1867
  
SDV Build Fail , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/3155/



---

[GitHub] carbondata issue #1359: [CARBONDATA-1480]Min Max Index Example for DataMap

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/1359
  
Build Success with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/3189/



---

[GitHub] carbondata pull request #1863: [CI][Streaming] Reduce the execution time of ...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/carbondata/pull/1863


---

[GitHub] carbondata issue #1764: [CARBONDATA-1987] Make package name and directory pa...

[chenliang613]

Github user chenliang613 commented on the issue:

https://github.com/apache/carbondata/pull/1764
  
retest this please


---

[jira] [Resolved] (CARBONDATA-2069) Data is not loaded into preaggregate table when table is created when data load is in progress for main table

[Ravindra Pesala (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CARBONDATA-2069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ravindra Pesala resolved CARBONDATA-2069.
-
   Resolution: Fixed
Fix Version/s: 1.3.0

> Data is not loaded into preaggregate table when table is created when data 
> load is in progress for main table
> -
>
> Key: CARBONDATA-2069
> URL: https://issues.apache.org/jira/browse/CARBONDATA-2069
> Project: CarbonData
>  Issue Type: Bug
>Reporter: Kunal Kapoor
>Assignee: Kunal Kapoor
>Priority: Minor
> Fix For: 1.3.0
>
>  Time Spent: 3h 40m
>  Remaining Estimate: 0h
>
> Problem:
>  # Load data into maintable
>  # create datamap parallelly
> preaggregate table will not have any data while data load is successful for 
> main table. This will make the pre-aggregate table inconsistent



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] carbondata issue #1863: [CI][Streaming] Reduce the execution time of TestStr...

[ravipesala]

Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/1863
  
LGTM


---

[GitHub] carbondata pull request #1850: [CARBONDATA-2069] Restrict create datamap whe...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/carbondata/pull/1850


---

[GitHub] carbondata issue #1499: [WIP][CARBONDATA-1235]Add Lucene Datamap

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/1499
  
Build Failed  with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/3188/



---

[GitHub] carbondata pull request #1868: [CARBONDATA-2082][CARBONDATA-1516] Timeseries...

[xubo245]

Github user xubo245 closed the pull request at:

https://github.com/apache/carbondata/pull/1868


---

[GitHub] carbondata issue #1850: [CARBONDATA-2069] Restrict create datamap when load ...

[ravipesala]

Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/1850
  
LGTM


---

[jira] [Resolved] (CARBONDATA-2063) Tests should not depend on each other

[Liang Chen (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CARBONDATA-2063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Liang Chen resolved CARBONDATA-2063.

   Resolution: Fixed
Fix Version/s: 1.3.0

> Tests should not depend on each other
> -
>
> Key: CARBONDATA-2063
> URL: https://issues.apache.org/jira/browse/CARBONDATA-2063
> Project: CarbonData
>  Issue Type: Bug
>  Components: test
>Affects Versions: 1.3.0
>Reporter: xuchuanyin
>Assignee: xuchuanyin
>Priority: Minor
> Fix For: 1.3.0
>
>  Time Spent: 3h 40m
>  Remaining Estimate: 0h
>
> Tests should not depend on each other, otherwise tests will fail in 
> developer's IDE if they don't execute all the tests at once.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] carbondata pull request #1843: [CARBONDATA-2063][Tests] Fix bugs in interdep...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/carbondata/pull/1843


---

[GitHub] carbondata issue #1843: [CARBONDATA-2063][Tests] Fix bugs in interdependent ...

[chenliang613]

Github user chenliang613 commented on the issue:

https://github.com/apache/carbondata/pull/1843
  
LGTM


---

[jira] [Resolved] (CARBONDATA-2089) Test cases is incorrect because it always run success no matter whether the SQL thrown exception

[Liang Chen (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CARBONDATA-2089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Liang Chen resolved CARBONDATA-2089.

Resolution: Fixed

> Test cases is incorrect because it always run success no matter whether the 
> SQL thrown exception
> 
>
> Key: CARBONDATA-2089
> URL: https://issues.apache.org/jira/browse/CARBONDATA-2089
> Project: CarbonData
>  Issue Type: Bug
>  Components: test
>Affects Versions: 1.3.0
>Reporter: xubo245
>Assignee: xubo245
>Priority: Major
> Fix For: 1.3.0
>
>  Time Spent: 3h
>  Remaining Estimate: 0h
>
> Test cases is incorrect because it always run success no matter whether the 
> SQL thrown exception,
> if the sql run success, then assert(false), and then catch exception and 
> assert(true), so it always true.
> Test cases is incorrect,like:
> {code:java}
>   test("test timeseries create table seven") {
> try {
>   sql(
> "create datamap agg0 on table mainTable using 'preaggregate' 
> DMPROPERTIES ('timeseries.eventTime'='dataTime', 
> 'timeseries.hierarchy'='hour=1,day=1,year=1,month=1') as select dataTime, 
> sum(age) from mainTable group by dataTime")
>   assert(false)
> } catch {
>   case _:Exception =>
> assert(true)
> }
>   }
> {code}
> It will always run success.
> SDV have many similar test case, we need to fix it!



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] carbondata issue #1843: [CARBONDATA-2063][Tests] Fix bugs in interdependent ...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/1843
  
Build Success with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/3186/



---

[GitHub] carbondata pull request #1866: [CARBONDATA-2089] Test cases is incorrect bec...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/carbondata/pull/1866


---

[GitHub] carbondata issue #1843: [CARBONDATA-2063][Tests] Fix bugs in interdependent ...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/1843
  
Build Success with Spark 2.2.1, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/1952/



---

[GitHub] carbondata issue #1866: [CARBONDATA-2089] Test cases is incorrect because it...

[chenliang613]

Github user chenliang613 commented on the issue:

https://github.com/apache/carbondata/pull/1866
  
LGTM


---

[GitHub] carbondata issue #1359: [CARBONDATA-1480]Min Max Index Example for DataMap

[ravipesala]

Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/1359
  
SDV Build Fail , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/3154/



---

[GitHub] carbondata issue #1499: [WIP][CARBONDATA-1235]Add Lucene Datamap

[ravipesala]

Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/1499
  
SDV Build Fail , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/3153/



---

[GitHub] carbondata issue #1510: [CARBONDATA-1543] Supported DataMap chooser and expr...

[ravipesala]

Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/1510
  
SDV Build Fail , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/3152/



---

[GitHub] carbondata issue #1499: [WIP][CARBONDATA-1235]Add Lucene Datamap

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/1499
  
Build Failed with Spark 2.2.1, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/1954/



---

[GitHub] carbondata issue #1359: [CARBONDATA-1480]Min Max Index Example for DataMap

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/1359
  
Build Failed with Spark 2.2.1, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/1955/



---

[GitHub] carbondata issue #1510: [CARBONDATA-1543] Supported DataMap chooser and expr...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/1510
  
Build Failed with Spark 2.2.1, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/1953/



---

[GitHub] carbondata issue #1865: [WIP] [CARBONDATA-2088][CARBONDATA-1516] It better u...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/1865
  
Build Failed  with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/3185/



---

[jira] [Resolved] (CARBONDATA-2066) uniqwithoutheader.csv file gets deleted when "duplicate values" test case is run

[Liang Chen (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CARBONDATA-2066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Liang Chen resolved CARBONDATA-2066.

   Resolution: Fixed
 Assignee: Raghunandan S
Fix Version/s: 1.3.0

> uniqwithoutheader.csv file gets deleted when "duplicate values" test case is 
> run
> 
>
> Key: CARBONDATA-2066
> URL: https://issues.apache.org/jira/browse/CARBONDATA-2066
> Project: CarbonData
>  Issue Type: Bug
>Reporter: Raghunandan
>Assignee: Raghunandan S
>Priority: Minor
> Fix For: 1.3.0
>
>  Time Spent: 1h 20m
>  Remaining Estimate: 0h
>
> when "duplicate values" UT is run, uniqwithoutheader.csv file used to load 
> into hive gets deleted.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] carbondata issue #1865: [WIP] [CARBONDATA-2088][CARBONDATA-1516] It better u...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/1865
  
Build Success with Spark 2.2.1, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/1951/



---

[GitHub] carbondata pull request #1845: [CARBONDATA-2066] Add Local keyword when load...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/carbondata/pull/1845


---

[jira] [Resolved] (CARBONDATA-1945) documentation for bad records action needs to be updated

[Liang Chen (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CARBONDATA-1945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Liang Chen resolved CARBONDATA-1945.

   Resolution: Fixed
Fix Version/s: 1.3.0

> documentation for bad records action needs to be updated
> 
>
> Key: CARBONDATA-1945
> URL: https://issues.apache.org/jira/browse/CARBONDATA-1945
> Project: CarbonData
>  Issue Type: Improvement
>  Components: docs
>Affects Versions: 1.3.0
> Environment: spark 2.1
>Reporter: Vandana Yadav
>Assignee: Vandana Yadav
>Priority: Trivial
> Fix For: 1.3.0
>
> Attachments: 2000_UniqData.csv
>
>  Time Spent: 1h
>  Remaining Estimate: 0h
>
> We should update bad record action's default value as while creating and 
> loading data with some empty or bad record it is giving exception 
> Error: java.lang.Exception: Data load failed due to bad record:
> Steps to reproduce:
> 1) Create Table:
> CREATE TABLE uniqdata (CUST_ID int,CUST_NAME String,ACTIVE_EMUI_VERSION 
> string, DOB timestamp, DOJ timestamp, BIGINT_COLUMN1 bigint,BIGINT_COLUMN2 
> bigint,DECIMAL_COLUMN1 decimal(30,10), DECIMAL_COLUMN2 
> decimal(36,10),Double_COLUMN1 double, Double_COLUMN2 double,INTEGER_COLUMN1 
> int) STORED BY 'org.apache.carbondata.format' TBLPROPERTIES 
> ("TABLE_BLOCKSIZE"= "256 MB")
> 2) Load data:
> LOAD DATA inpath 'hdfs://localhost:54310/Data/uniqdata/2000_UniqData.csv' 
> INTO table uniqdata options('DELIMITER'=',', 'FILEHEADER'='CUST_ID, 
> CUST_NAME, ACTIVE_EMUI_VERSION, DOB, DOJ, BIGINT_COLUMN1, BIGINT_COLUMN2, 
> DECIMAL_COLUMN1, DECIMAL_COLUMN2, Double_COLUMN1, Double_COLUMN2, 
> INTEGER_COLUMN1','TIMESTAMPFORMAT'='-MM-dd HH:mm:ss');
> 3) Output: 
> Expected output: data should be loaded successfully.
> Actual Output: Error: java.lang.Exception: Data load failed due to bad 
> record: The value with column name dob and column data type TIMESTAMP is not 
> a valid TIMESTAMP type. (state=,code=0)



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] carbondata issue #1843: [CARBONDATA-2063][Tests] Fix bugs in interdependent ...

[chenliang613]

Github user chenliang613 commented on the issue:

https://github.com/apache/carbondata/pull/1843
  
retest this please


---

[GitHub] carbondata pull request #1842: [CARBONDATA-1945] updated bad records handlin...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/carbondata/pull/1842


---

[GitHub] carbondata issue #1842: [CARBONDATA-1945] updated bad records handling for i...

[chenliang613]

Github user chenliang613 commented on the issue:

https://github.com/apache/carbondata/pull/1842
  
LGTM


---

[GitHub] carbondata issue #1865: [WIP] [CARBONDATA-2088][CARBONDATA-1516] It better u...

[ravipesala]

Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/1865
  
SDV Build Fail , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/3151/



---

[GitHub] carbondata issue #1872: [TEST-FRAMEWORK] Added test framework to ensure proc...

[ravipesala]

Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/1872
  
SDV Build Fail , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/3150/



---

[GitHub] carbondata issue #1872: [TEST-FRAMEWORK] Added test framework to ensure proc...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/1872
  
Build Failed with Spark 2.2.1, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/1950/



---

[GitHub] carbondata issue #1872: [TEST-FRAMEWORK] Added test framework to ensure proc...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/1872
  
Build Failed  with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/3184/



---

[GitHub] carbondata pull request #1872: [TEST-FRAMEWORK] Added test framework to ensu...

[ravipesala]

GitHub user ravipesala opened a pull request:

https://github.com/apache/carbondata/pull/1872

[TEST-FRAMEWORK] Added test framework to ensure processMeta and processData 
methods from Carbon Commands

Problem : 
Existing Carbon command framework suppose to ensure that  processData 
should not access any hive metastore operations and processMetaData and 
undoMetaData should not access any file operations.
But there is no test framework to ensure this command calls, so it is very 
difficult for developers to follow and ensure this framework.

Solution : 
Added a test framework in carbon using mocks to ensure following calls
1. Find all the implemented classes of MetadataCommand, DataCommand and 
AtomicRunnableCommand.
2. processMetaData or undoMetaData can access hive metastore but not any 
file operations.
3. processData can access file operations but not any hive metastore 

This framework is enabled only when carbon hive schema metastore 
(spark.carbon.hive.schema.store) is enabled. 
After running tests using this framework I find out around 254 tests are 
failing. These tests are mostly related to  Autocompaction, IUD, Alter table, 
partitioning and complex types.
CI Link : 
http://136.243.101.176:8080/job/ApacheCarbondataSpark2.1Hadoop2.7.2new/116/

Be sure to do all of the following checklist to help us incorporate 
your contribution quickly and easily:

 - [X] Any interfaces changed?
 
 - [X] Any backward compatibility impacted?
 
 - [X] Document update required?

 - [X] Testing done
  
 - [X] For large changes, please consider breaking it into sub-tasks under 
an umbrella JIRA. 



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/ravipesala/incubator-carbondata instrument

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/carbondata/pull/1872.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1872


commit fe841a30c35b2f2fbb1d5911db35a5695f045da9
Author: ravipesala 
Date:   2018-01-22T11:11:25Z

Added test framework to ensure processMeta and processData methods from 
Carbon Commands




---

[GitHub] carbondata issue #1764: [CARBONDATA-1987] Make package name and directory pa...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/1764
  
Build Failed with Spark 2.2.1, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/1949/



---

[GitHub] carbondata issue #1764: [CARBONDATA-1987] Make package name and directory pa...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/1764
  
Build Success with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/3183/



---

[GitHub] carbondata issue #1865: [WIP] [CARBONDATA-2088][CARBONDATA-1516] It better u...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/1865
  
Build Failed  with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/3182/



---

[GitHub] carbondata issue #1865: [WIP] [CARBONDATA-2088][CARBONDATA-1516] It better u...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/1865
  
Build Failed with Spark 2.2.1, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/1948/



---

[GitHub] carbondata issue #1871: [CARBONDATA-2089]SQL exception is masked due to asse...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/1871
  
Build Success with Spark 2.2.1, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/1947/



---

[GitHub] carbondata issue #1871: [CARBONDATA-2089]SQL exception is masked due to asse...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/1871
  
Build Success with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/3181/



---

[GitHub] carbondata issue #1764: [CARBONDATA-1987] Make package name and directory pa...

[ravipesala]

Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/1764
  
SDV Build Fail , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/3149/



---

[GitHub] carbondata issue #1865: [WIP] [CARBONDATA-2088][CARBONDATA-1516] It better u...

[ravipesala]

Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/1865
  
SDV Build Fail , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/3148/



---

[GitHub] carbondata issue #1871: [CARBONDATA-2089]SQL exception is masked due to asse...

[ravipesala]

Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/1871
  
SDV Build Fail , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/3147/



---

[GitHub] carbondata pull request #1871: [CARBONDATA-2089]SQL exception is masked due ...

[sraghunandan]

GitHub user sraghunandan opened a pull request:

https://github.com/apache/carbondata/pull/1871

[CARBONDATA-2089]SQL exception is masked due to assert(false) inside try 
catch and exception always asserting true



Be sure to do all of the following checklist to help us incorporate 
your contribution quickly and easily:

 - [ x] Any interfaces changed?
 No
 - [ x] Any backward compatibility impacted?
 No
 - [x ] Document update required?
No
 - [x ] Testing done
Please provide details on 
- Whether new unit test cases have been added or why no new tests 
are required?
- How it is tested? Please attach test report.
- Is it a performance related change? Please attach the performance 
test report.
- Any additional information to help reviewers in testing this 
change.
   Test cases have been corrected
 - [ x] For large changes, please consider breaking it into sub-tasks under 
an umbrella JIRA. 
NA


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/sraghunandan/carbondata-1 
correct_try_catch_in_sdv

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/carbondata/pull/1871.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1871


commit e9c62a4ff1fe76b8343dbaf62a5573afea0348ad
Author: Raghunandan S 
Date:   2018-01-27T15:19:47Z

SQL exception is masked due to assert(false) inside try catch and exception 
always asserting true




---

[GitHub] carbondata issue #1866: [CARBONDATA-2089] Test cases is incorrect because it...

[xubo245]

Github user xubo245 commented on the issue:

https://github.com/apache/carbondata/pull/1866
  
I fix all similar UT test cases error, please review it @jackylk 
@kumarvishal09 @sraghunandan 


---

[GitHub] carbondata issue #1866: [CARBONDATA-2089] Test cases is incorrect because it...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/1866
  
Build Success with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/3180/



---

[GitHub] carbondata issue #1866: [CARBONDATA-2089] Test cases is incorrect because it...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/1866
  
Build Success with Spark 2.2.1, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/1946/



---

[GitHub] carbondata issue #1865: [WIP] [CARBONDATA-2088][CARBONDATA-1516] It better u...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/1865
  
Build Success with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/3179/



---

[GitHub] carbondata issue #1866: [CARBONDATA-2089] Test cases is incorrect because it...

[xubo245]

Github user xubo245 commented on the issue:

https://github.com/apache/carbondata/pull/1866
  
retest this please


---

[GitHub] carbondata issue #1866: [CARBONDATA-2089] Test cases is incorrect because it...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/1866
  
Build Failed with Spark 2.2.1, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/1944/



---

[GitHub] carbondata issue #1865: [CARBONDATA-2088][CARBONDATA-1516] It better using '...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/1865
  
Build Success with Spark 2.2.1, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/1945/



---

[GitHub] carbondata issue #1866: [CARBONDATA-2089] Test cases is incorrect because it...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/1866
  
Build Failed  with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/3178/



---

[GitHub] carbondata issue #1865: [CARBONDATA-2088][CARBONDATA-1516] It better using '...

[ravipesala]

Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/1865
  
SDV Build Fail , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/3146/



---

[GitHub] carbondata issue #1866: [CARBONDATA-2089] Test cases is incorrect because it...

[ravipesala]

Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/1866
  
SDV Build Fail , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/3145/



---

[GitHub] carbondata pull request #1865: [CARBONDATA-2088][CARBONDATA-1516] It better ...

[xubo245]

GitHub user xubo245 reopened a pull request:

https://github.com/apache/carbondata/pull/1865

 [CARBONDATA-2088][CARBONDATA-1516] It better using 'timeseries_cube'  
instead of preaggregate for timeseries pre-aggregate table


Be sure to do all of the following checklist to help us incorporate 
your contribution quickly and easily:

 - [ ] Any interfaces changed?
 change using 'timeseries_cube'  instead of using  preaggregat
change timeseries.eventTime  and  timeseries.hierarchy
 - [ ] Any backward compatibility impacted?
 NA
 - [ ] Document update required?
NA
 - [ ] Testing done
Add new test case
   
 - [ ] For large changes, please consider breaking it into sub-tasks under 
an umbrella JIRA. 
NA


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/xubo245/carbondata usingTimeseries

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/carbondata/pull/1865.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1865


commit 707be22b2a084e24e2d56d2bd85bd09b2d9ff5d5
Author: xubo245 <601450868@...>
Date:   2018-01-26T09:47:46Z

 [CARBONDATA-2088][CARBONDATA-1516] It better using 'timeseries_cube' 
instead of preaggregate got timeseries pre-aggregate table

commit bf42301283d67905eb7e3845040cf91ab5ba42c3
Author: xubo245 <601450868@...>
Date:   2018-01-26T12:45:04Z

fix error

commit 60f259f40594b7cc5e80166b58b31b3e9a602f17
Author: xubo245 <601450868@...>
Date:   2018-01-27T02:49:24Z

fix findbugs

commit b1fb16d6022faab7c873708459b94b0224b4a9bd
Author: xubo245 <601450868@...>
Date:   2018-01-27T03:43:53Z

drop table to fix CI test error




---

[GitHub] carbondata pull request #1848: [WIP][CARBONDATA-2068][CARBONDATA-1516][Tests...

[xubo245]

Github user xubo245 closed the pull request at:

https://github.com/apache/carbondata/pull/1848


---

[GitHub] carbondata issue #1866: [CARBONDATA-2089] Test cases is incorrect because it...

[ravipesala]

Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/1866
  
SDV Build Fail , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/3144/



---

[GitHub] carbondata issue #1858: [CARBONDATA-2075][CARBONDATA-1516] It should throw e...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/1858
  
Build Failed with Spark 2.2.1, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/1942/



---

[GitHub] carbondata issue #1858: [CARBONDATA-2075][CARBONDATA-1516] It should throw e...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/1858
  
Build Failed  with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/3176/



---

[GitHub] carbondata issue #1858: [CARBONDATA-2075][CARBONDATA-1516] It should throw e...

[xubo245]

Github user xubo245 commented on the issue:

https://github.com/apache/carbondata/pull/1858
  
retest this please


---

[GitHub] carbondata issue #1863: [CI][Streaming] Reduce the execution time of TestStr...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/1863
  
Build Success with Spark 2.2.1, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/1941/



---

[jira] [Updated] (CARBONDATA-2085) It's different between load twice and create datamap with load again after load data and create datamap

[xubo245 (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CARBONDATA-2085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

xubo245 updated CARBONDATA-2085:

Description: 
It's different between two test case

test case 1: load twice and create datamap , and then query
test case 2:load once , create datamap and load again, and then query

{code:java}
+  test("load data into mainTable after create timeseries datamap on table 1") {
 +sql("drop table if exists mainTable")
 +sql(
 +  """
 +| CREATE TABLE mainTable(
 +|   mytime timestamp,
 +|   name string,
 +|   age int)
 +| STORED BY 'org.apache.carbondata.format'
 +  """.stripMargin)
 +
 +sql(s"LOAD DATA LOCAL INPATH '$resourcesPath/timeseriestest.csv' into 
table mainTable")
 +
 +sql(
 +  """
 +| create datamap agg0 on table mainTable
 +| using 'preaggregate'
 +| DMPROPERTIES (
 +|   'timeseries.eventTime'='mytime',
 +|   
'timeseries.hierarchy'='second=1,minute=1,hour=1,day=1,month=1,year=1')
 +| as select mytime, sum(age)
 +| from mainTable
 +| group by mytime""".stripMargin)
 +
 +sql(s"LOAD DATA LOCAL INPATH '$resourcesPath/timeseriestest.csv' into 
table mainTable")
 +val df = sql(
 +  """
 +| select
 +|   timeseries(mytime,'minute') as minuteLevel,
 +|   sum(age) as sum
 +| from mainTable
 +| where timeseries(mytime,'minute')>='2016-02-23 01:01:00'
 +| group by
 +|   timeseries(mytime,'minute')
 +| order by
 +|   timeseries(mytime,'minute')
 +  """.stripMargin)
 +
 +// only for test, it need remove before merge
 +df.show()
 +sql("select * from maintable_agg0_minute").show(100)
 +
 +checkAnswer(df,
 +  Seq(Row(Timestamp.valueOf("2016-02-23 01:01:00"), 120),
 +Row(Timestamp.valueOf("2016-02-23 01:02:00"), 280)))
 +
 +  }
 +
 +  test("load data into mainTable after create timeseries datamap on table 2") 
{
 +sql("drop table if exists mainTable")
 +sql(
 +  """
 +| CREATE TABLE mainTable(
 +|   mytime timestamp,
 +|   name string,
 +|   age int)
 +| STORED BY 'org.apache.carbondata.format'
 +  """.stripMargin)
 +
 +sql(s"LOAD DATA LOCAL INPATH '$resourcesPath/timeseriestest.csv' into 
table mainTable")
 +sql(s"LOAD DATA LOCAL INPATH '$resourcesPath/timeseriestest.csv' into 
table mainTable")
 +sql(
 +  """
 +| create datamap agg0 on table mainTable
 +| using 'preaggregate'
 +| DMPROPERTIES (
 +|   'timeseries.eventTime'='mytime',
 +|   
'timeseries.hierarchy'='second=1,minute=1,hour=1,day=1,month=1,year=1')
 +| as select mytime, sum(age)
 +| from mainTable
 +| group by mytime""".stripMargin)
 +
 +
 +val df = sql(
 +  """
 +| select
 +|   timeseries(mytime,'minute') as minuteLevel,
 +|   sum(age) as sum
 +| from mainTable
 +| where timeseries(mytime,'minute')>='2016-02-23 01:01:00'
 +| group by
 +|   timeseries(mytime,'minute')
 +| order by
 +|   timeseries(mytime,'minute')
 +  """.stripMargin)
 +
 +// only for test, it need remove before merge
 +df.show()
 +sql("select * from maintable_agg0_minute").show(100)
 +
 +
 +checkAnswer(df,
 +  Seq(Row(Timestamp.valueOf("2016-02-23 01:01:00"), 120),
 +Row(Timestamp.valueOf("2016-02-23 01:02:00"), 280)))
 +  }
 +
{code}


test case 1 and 2 should success , but test case 1 fail 


  was:
It's different between two test case

test case 1: load twice and create datamap , and then query
test case 2:load once , create datamap and load again, and then query

{code:java}
+  test("load data into mainTable after create timeseries datamap on table 1") {
 +sql("drop table if exists mainTable")
 +sql(
 +  """
 +| CREATE TABLE mainTable(
 +|   mytime timestamp,
 +|   name string,
 +|   age int)
 +| STORED BY 'org.apache.carbondata.format'
 +  """.stripMargin)
 +
 +sql(s"LOAD DATA LOCAL INPATH '$resourcesPath/timeseriestest.csv' into 
table mainTable")
 +
 +sql(
 +  """
 +| create datamap agg0 on table mainTable
 +| using 'preaggregate'
 +| DMPROPERTIES (
 +|   'timeseries.eventTime'='mytime',
 +|   
'timeseries.hierarchy'='second=1,minute=1,hour=1,day=1,month=1,year=1')
 +| as select mytime, sum(age)
 +| from mainTable
 +| group by mytime""".stripMargin)
 +
 +sql(s"LOAD DATA LOCAL INPATH '$resourcesPath/timeseriestest.csv' into 
table mainTable")
 +val df = sql(
 +  """
 +| select
 +|   timeseries(mytime,'minute') as minuteLevel,
 +|   sum(age) as sum
 +| from mainTable
 +| where 

[jira] [Updated] (CARBONDATA-2085) It's different between load twice and create datamap with load again after load data and create datamap

[xubo245 (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CARBONDATA-2085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

xubo245 updated CARBONDATA-2085:

Description: 
It's different between two test case

test case 1: load twice and create datamap , and then query
test case 2:load once , create datamap and load again, and then query

{code:java}
+  test("load data into mainTable after create timeseries datamap on table 1") {
 +sql("drop table if exists mainTable")
 +sql(
 +  """
 +| CREATE TABLE mainTable(
 +|   mytime timestamp,
 +|   name string,
 +|   age int)
 +| STORED BY 'org.apache.carbondata.format'
 +  """.stripMargin)
 +
 +sql(s"LOAD DATA LOCAL INPATH '$resourcesPath/timeseriestest.csv' into 
table mainTable")
 +
 +sql(
 +  """
 +| create datamap agg0 on table mainTable
 +| using 'preaggregate'
 +| DMPROPERTIES (
 +|   'timeseries.eventTime'='mytime',
 +|   
'timeseries.hierarchy'='second=1,minute=1,hour=1,day=1,month=1,year=1')
 +| as select mytime, sum(age)
 +| from mainTable
 +| group by mytime""".stripMargin)
 +
 +sql(s"LOAD DATA LOCAL INPATH '$resourcesPath/timeseriestest.csv' into 
table mainTable")
 +val df = sql(
 +  """
 +| select
 +|   timeseries(mytime,'minute') as minuteLevel,
 +|   sum(age) as sum
 +| from mainTable
 +| where timeseries(mytime,'minute')>='2016-02-23 01:01:00'
 +| group by
 +|   timeseries(mytime,'minute')
 +| order by
 +|   timeseries(mytime,'minute')
 +  """.stripMargin)
 +
 +// only for test, it need remove before merge
 +df.show()
 +sql("select * from maintable_agg0_minute").show(100)
 +
 +checkAnswer(df,
 +  Seq(Row(Timestamp.valueOf("2016-02-23 01:01:00"), 120),
 +Row(Timestamp.valueOf("2016-02-23 01:02:00"), 280)))
 +
 +  }
 +
 +  test("load data into mainTable after create timeseries datamap on table 2") 
{
 +sql("drop table if exists mainTable")
 +sql(
 +  """
 +| CREATE TABLE mainTable(
 +|   mytime timestamp,
 +|   name string,
 +|   age int)
 +| STORED BY 'org.apache.carbondata.format'
 +  """.stripMargin)
 +
 +sql(s"LOAD DATA LOCAL INPATH '$resourcesPath/timeseriestest.csv' into 
table mainTable")
 +sql(s"LOAD DATA LOCAL INPATH '$resourcesPath/timeseriestest.csv' into 
table mainTable")
 +sql(
 +  """
 +| create datamap agg0 on table mainTable
 +| using 'preaggregate'
 +| DMPROPERTIES (
 +|   'timeseries.eventTime'='mytime',
 +|   
'timeseries.hierarchy'='second=1,minute=1,hour=1,day=1,month=1,year=1')
 +| as select mytime, sum(age)
 +| from mainTable
 +| group by mytime""".stripMargin)
 +
 +
 +val df = sql(
 +  """
 +| select
 +|   timeseries(mytime,'minute') as minuteLevel,
 +|   sum(age) as sum
 +| from mainTable
 +| where timeseries(mytime,'minute')>='2016-02-23 01:01:00'
 +| group by
 +|   timeseries(mytime,'minute')
 +| order by
 +|   timeseries(mytime,'minute')
 +  """.stripMargin)
 +
 +// only for test, it need remove before merge
 +df.show()
 +sql("select * from maintable_agg0_minute").show(100)
 +
 +
 +checkAnswer(df,
 +  Seq(Row(Timestamp.valueOf("2016-02-23 01:01:00"), 120),
 +Row(Timestamp.valueOf("2016-02-23 01:02:00"), 280)))
 +  }
 +
{code}



  was:
It's different between load twice and create datamap with load again after load 
data and create datamap


{code:java}
 test("test if timeseries load twice is successful on table creation") {
sql("drop table if exists mainTable")
sql(
  """
| CREATE TABLE mainTable(
|   mytime timestamp,
|   name string,
|   age int)
| STORED BY 'org.apache.carbondata.format'
  """.stripMargin)
sql(s"LOAD DATA LOCAL INPATH '$resourcesPath/timeseriestest.csv' into table 
mainTable")
sql(s"LOAD DATA LOCAL INPATH '$resourcesPath/timeseriestest.csv' into table 
mainTable")
sql(
  """
| create datamap agg0 on table mainTable
| using 'preaggregate'
| DMPROPERTIES (
|   'timeseries.eventTime'='mytime',
|   
'timeseries.hierarchy'='second=1,minute=1,hour=1,day=1,month=1,year=1')
| as select mytime, sum(age)
| from mainTable
| group by mytime
  """.stripMargin)
checkAnswer(sql("select * from maintable_agg0_second"),
  Seq(Row(Timestamp.valueOf("2016-02-23 01:01:30.0"), 20),
Row(Timestamp.valueOf("2016-02-23 01:01:40.0"), 40),
Row(Timestamp.valueOf("2016-02-23 01:01:50.0"), 60),
Row(Timestamp.valueOf("2016-02-23 01:02:30.0"), 80),
Row(Timestamp.valueOf("2016-02-23 01:02:40.0"), 100),

[GitHub] carbondata issue #1863: [CI][Streaming] Reduce the execution time of TestStr...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/1863
  
Build Success with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/3175/



---

[GitHub] carbondata pull request #1870: [WIP][CARBONDATA-2085][CARBONDATA-1516] Times...

[xubo245]

Github user xubo245 closed the pull request at:

https://github.com/apache/carbondata/pull/1870


---

[GitHub] carbondata pull request #1870: [CARBONDATA-2085][CARBONDATA-1516] Timeseries...

[xubo245]

GitHub user xubo245 opened a pull request:

https://github.com/apache/carbondata/pull/1870

[CARBONDATA-2085][CARBONDATA-1516] Timeseries pre-aggregate table is error 
when load data after create Timeseries pre-aggregate table


Be sure to do all of the following checklist to help us incorporate 
your contribution quickly and easily:

 - [ ] Any interfaces changed?
 
 - [ ] Any backward compatibility impacted?
 
 - [ ] Document update required?

 - [ ] Testing done
Please provide details on 
- Whether new unit test cases have been added or why no new tests 
are required?
- How it is tested? Please attach test report.
- Is it a performance related change? Please attach the performance 
test report.
- Any additional information to help reviewers in testing this 
change.
   
 - [ ] For large changes, please consider breaking it into sub-tasks under 
an umbrella JIRA. 



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/xubo245/carbondata loadAfterCreateDataMap

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/carbondata/pull/1870.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1870


commit 4e77ab3ea45540c5bebc54d2f234f63465b081f1
Author: xubo245 <601450868@...>
Date:   2018-01-27T10:16:17Z

[CARBONDATA-2085][CARBONDATA-1516] Timeseries pre-aggregate table is error 
when load data after create Timeseries pre-aggregate table




---

[GitHub] carbondata issue #1845: [CARBONDATA-2066] Add Local keyword when loading dat...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/1845
  
Build Success with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/3174/



---

[GitHub] carbondata issue #1845: [CARBONDATA-2066] Add Local keyword when loading dat...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/1845
  
Build Success with Spark 2.2.1, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/1940/



---

[GitHub] carbondata issue #1858: [CARBONDATA-2075][CARBONDATA-1516] It should throw n...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/1858
  
Build Failed with Spark 2.2.1, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/1939/



---

[GitHub] carbondata issue #1858: [CARBONDATA-2075][CARBONDATA-1516] It should throw n...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/1858
  
Build Failed  with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/3173/



---

[GitHub] carbondata issue #1863: [CI][Streaming] Reduce the execution time of TestStr...

[ravipesala]

Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/1863
  
SDV Build Fail , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/3143/



---

[GitHub] carbondata issue #1858: [CARBONDATA-2075][CARBONDATA-1516] It should throw n...

[ravipesala]

Github user ravipesala commented on the issue:

https://github.com/apache/carbondata/pull/1858
  
SDV Build Fail , Please check CI 
http://144.76.159.231:8080/job/ApacheSDVTests/3142/



---

[GitHub] carbondata issue #1869: [WIP][CARBONDATA-2083][CARBONDATA-1516] Timeseries p...

[xubo245]

Github user xubo245 commented on the issue:

https://github.com/apache/carbondata/pull/1869
  
don't support in this version


---

[GitHub] carbondata pull request #1869: [WIP][CARBONDATA-2083][CARBONDATA-1516] Times...

[xubo245]

GitHub user xubo245 reopened a pull request:

https://github.com/apache/carbondata/pull/1869

[WIP][CARBONDATA-2083][CARBONDATA-1516] Timeseries pre-aggregate table 
should support hour !=1 and so on



Be sure to do all of the following checklist to help us incorporate 
your contribution quickly and easily:

 - [ ] Any interfaces changed?
 support hour  !=1
 - [ ] Any backward compatibility impacted?
 NA
 - [ ] Document update required?
NA
 - [ ] Testing done
Please provide details on 
- Whether new unit test cases have been added or why no new tests 
are required?
- How it is tested? Please attach test report.
- Is it a performance related change? Please attach the performance 
test report.
- Any additional information to help reviewers in testing this 
change.
   
 - [ ] For large changes, please consider breaking it into sub-tasks under 
an umbrella JIRA. 
NA


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/xubo245/carbondata supportTimeSeriesOtherValue

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/carbondata/pull/1869.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1869


commit cf543ee38627c6ffde613c9908b870c966be56a8
Author: xubo245 <601450868@...>
Date:   2018-01-27T06:41:49Z

[CARBONDATA-2083][CARBONDATA-1516] Timeseries pre-aggregate table should 
support hour !=1 and so on




---

[GitHub] carbondata pull request #1869: [WIP][CARBONDATA-2083][CARBONDATA-1516] Times...

[xubo245]

Github user xubo245 closed the pull request at:

https://github.com/apache/carbondata/pull/1869


---

[GitHub] carbondata pull request #1869: [WIP][CARBONDATA-2083][CARBONDATA-1516] Times...

[xubo245]

GitHub user xubo245 reopened a pull request:

https://github.com/apache/carbondata/pull/1869

[WIP][CARBONDATA-2083][CARBONDATA-1516] Timeseries pre-aggregate table 
should support hour !=1 and so on



Be sure to do all of the following checklist to help us incorporate 
your contribution quickly and easily:

 - [ ] Any interfaces changed?
 support hour  !=1
 - [ ] Any backward compatibility impacted?
 NA
 - [ ] Document update required?
NA
 - [ ] Testing done
Please provide details on 
- Whether new unit test cases have been added or why no new tests 
are required?
- How it is tested? Please attach test report.
- Is it a performance related change? Please attach the performance 
test report.
- Any additional information to help reviewers in testing this 
change.
   
 - [ ] For large changes, please consider breaking it into sub-tasks under 
an umbrella JIRA. 
NA


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/xubo245/carbondata supportTimeSeriesOtherValue

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/carbondata/pull/1869.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1869


commit cf543ee38627c6ffde613c9908b870c966be56a8
Author: xubo245 <601450868@...>
Date:   2018-01-27T06:41:49Z

[CARBONDATA-2083][CARBONDATA-1516] Timeseries pre-aggregate table should 
support hour !=1 and so on




---

[GitHub] carbondata pull request #1869: [WIP][CARBONDATA-2083][CARBONDATA-1516] Times...

[xubo245]

Github user xubo245 closed the pull request at:

https://github.com/apache/carbondata/pull/1869


---

[GitHub] carbondata pull request #1859: [CARBONDATA-2077][CARBONDATA-1516] it should ...

[xubo245]

Github user xubo245 closed the pull request at:

https://github.com/apache/carbondata/pull/1859


---

[GitHub] carbondata pull request #1869: [WIP][CARBONDATA-2083][CARBONDATA-1516] Times...

[xubo245]

Github user xubo245 closed the pull request at:

https://github.com/apache/carbondata/pull/1869


---

[GitHub] carbondata issue #1859: [CARBONDATA-2077][CARBONDATA-1516] it should throw n...

[xubo245]

Github user xubo245 commented on the issue:

https://github.com/apache/carbondata/pull/1859
  
it's combined to https://github.com/apache/carbondata/pull/1858


---

[GitHub] carbondata issue #1845: [CARBONDATA-2066] Add Local keyword when loading dat...

[sraghunandan]

Github user sraghunandan commented on the issue:

https://github.com/apache/carbondata/pull/1845
  
retest this please


---

[GitHub] carbondata pull request #1864: [CARBONDATA-2086][CARBONDATA-1516] Create dat...

[xubo245]

Github user xubo245 closed the pull request at:

https://github.com/apache/carbondata/pull/1864


---

[GitHub] carbondata pull request #1865: [CARBONDATA-2088][CARBONDATA-1516] It better ...

[xubo245]

Github user xubo245 closed the pull request at:

https://github.com/apache/carbondata/pull/1865


---

[jira] [Closed] (CARBONDATA-2084) Timeseries pre-aggregate table should support min and max when the create datamap don't as select max and min

[xubo245 (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CARBONDATA-2084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

xubo245 closed CARBONDATA-2084.
---
Resolution: Invalid

> Timeseries pre-aggregate table should support min and max when the create 
> datamap don't as select max and min
> -
>
> Key: CARBONDATA-2084
> URL: https://issues.apache.org/jira/browse/CARBONDATA-2084
> Project: CarbonData
>  Issue Type: Bug
>  Components: core, spark-integration
>Affects Versions: 1.3.0
>Reporter: xubo245
>Priority: Major
> Fix For: 1.3.0
>
>
> {code:java}
>   override def beforeAll: Unit = {
> SparkUtil4Test.createTaskMockUp(sqlContext)
> CarbonProperties.getInstance()
>   .addProperty(CarbonCommonConstants.CARBON_TIMESTAMP_FORMAT,
> CarbonCommonConstants.CARBON_TIMESTAMP_DEFAULT_FORMAT)
> sql("drop table if exists mainTable")
> sql("CREATE TABLE mainTable(mytime timestamp, name string, age int) 
> STORED BY 'org.apache.carbondata.format'")
> sql("create datamap agg0 on table mainTable using 'preaggregate' 
> DMPROPERTIES ('timeseries.eventTime'='mytime', 
> 'timeseries.hierarchy'='second=1,minute=1,hour=1,day=1,month=1,year=1') as 
> select mytime, sum(age) from mainTable group by mytime")
> sql(s"LOAD DATA LOCAL INPATH '$resourcesPath/timeseriestest.csv' into 
> table mainTable")
>   }
> {code}
> {code:java}
>   // TODO: support max
>   ignore("test PreAggregate table selection 38: filter < and >=, max") {
> val df = sql(
>   """
> | select
> |   timeseries(mytime,'second') as secondLevel,
> |   max(age) as maxValue
> | from mainTable
> | where timeseries(mytime,'second')<'2016-02-23 01:02:00' and 
> timeseries(mytime,'second')>='2016-02-23 01:01:00'
> | group by
> |   timeseries(mytime,'second')
> | order by
> |   timeseries(mytime,'second')
>   """.stripMargin)
> df.show()
> checkAnswer(df, Seq(Row(Timestamp.valueOf("2016-02-23 01:01:50"), 30)))
>   }
>   ignore("test PreAggregate table selection 39: filter < and >=, min") {
> val df = sql(
>   """
> | select
> |   timeseries(mytime,'second') as secondLevel,
> |   min(age) as minValue
> | from mainTable
> | where timeseries(mytime,'second')<'2016-02-23 01:02:00' and 
> timeseries(mytime,'second')>='2016-02-23 01:01:00'
> | group by
> |   timeseries(mytime,'second')
> | order by
> |   timeseries(mytime,'second')
>   """.stripMargin)
> df.show()
> checkAnswer(df, Seq(Row(Timestamp.valueOf("2016-02-23 01:01:30"), 10)))
>   }
> {code}
> detail please see my branch: 
> https://github.com/xubo245/carbondata/tree/usingTimeseries



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Closed] (CARBONDATA-2087) Order rule should keep consistent for create datamap

[xubo245 (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CARBONDATA-2087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

xubo245 closed CARBONDATA-2087.
---
Resolution: Won't Fix

no need to fix, because it will be changed.

> Order rule should keep consistent for create datamap
> 
>
> Key: CARBONDATA-2087
> URL: https://issues.apache.org/jira/browse/CARBONDATA-2087
> Project: CarbonData
>  Issue Type: Bug
>  Components: core, spark-integration
>Affects Versions: 1.3.0
>Reporter: xubo245
>Priority: Minor
> Fix For: 1.3.0
>
>
> It success:
> {code:java}
>   test("test timeseries create table 18: support day=1,year=1,month=1") {
> try {
>   sql(
> """create datamap agg1 on table mainTable
>   | using 'preaggregate'
>   | DMPROPERTIES (
>   |'timeseries.eventTime'='dataTime',
>   |'timeseries.hierarchy'='day=1,year=1,month=1')
>   | as select dataTime, sum(age) from mainTable
>   | group by dataTime
> """.stripMargin)
>   checkExistence(sql("show tables"), false, "maintable_agg1_hour")
>   checkExistence(sql("show tables"), true, "maintable_agg1_day", 
> "maintable_agg1_month", "maintable_agg1_year")
> } catch {
>   case _: Exception =>
> assert(false)
> }
>   }
> {code}
> but it fail:
> {code:java}
>   test("test timeseries create table 21: don't support year=1,month=1") {
> try {
>   sql(
> """create datamap agg1 on table mainTable
>   |using 'preaggregate'
>   |DMPROPERTIES (
>   |   'timeseries.eventTime'='dataTime',
>   |   'timeseries.hierarchy'='year=1,month=1')
>   |as select dataTime, sum(age) from mainTable
>   |group by dataTime""".stripMargin)
>   assert(false)
> } catch {
>   case e: MalformedCarbonCommandException =>
> assert(e.getMessage.contains("year=1,month=1 is in wrong order"))
> } finally {
>   checkExistence(sql("show tables"), false, "maintable_agg1_year")
> }
>   }
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Closed] (CARBONDATA-2083) Timeseries pre-aggregate table should support hour != 1 , others are the same

[xubo245 (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CARBONDATA-2083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

xubo245 closed CARBONDATA-2083.
---
Resolution: Won't Fix

no need to support in this version

> Timeseries pre-aggregate table should support hour != 1 , others are the same 
> --
>
> Key: CARBONDATA-2083
> URL: https://issues.apache.org/jira/browse/CARBONDATA-2083
> Project: CarbonData
>  Issue Type: Bug
>  Components: core, spark-integration
>Affects Versions: 1.3.0
>Reporter: xubo245
>Priority: Major
> Fix For: 1.3.0
>
>  Time Spent: 40m
>  Remaining Estimate: 0h
>
> test case find the error:
> {code:java}
>   test("test timeseries create table 13: don't support hour=2") {
> try {
>   sql(
> """create datamap agg1 on table mainTable using 'preaggregate'
>   |DMPROPERTIES (
>   |   'timeseries.eventTime'='dataTime',
>   |   'timeseries.hierarchy'='hour=2')
>   |as select dataTime, sum(age) from mainTable
>   |group by dataTime
> """.stripMargin)
>   assert(false)
> } catch {
>   case e: MalformedCarbonCommandException =>
> assert(e.getMessage.contains("Unsupported Value for 
> hierarchy:hour=2"))
> } finally {
>   checkExistence(sql("show tables"), false, "maintable_agg1_hour")
> }
>   }
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] carbondata issue #1863: [CI][Streaming] Reduce the execution time of TestStr...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/1863
  
Build Success with Spark 2.2.1, Please check CI 
http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/1938/



---

[GitHub] carbondata issue #1863: [CI][Streaming] Reduce the execution time of TestStr...

[CarbonDataQA]

Github user CarbonDataQA commented on the issue:

https://github.com/apache/carbondata/pull/1863
  
Build Failed  with Spark 2.1.0, Please check CI 
http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/3172/



---