GitHub user xubo245 opened a pull request:
https://github.com/apache/carbondata/pull/2673
[WIP] Test Carbonstore
Be sure to do all of the following checklist to help us incorporate
your contribution quickly and easily:
- [ ] Any interfaces changed?
- [ ] Any backward compatibility impacted?
- [ ] Document update required?
- [ ] Testing done
Please provide details on
- Whether new unit test cases have been added or why no new tests
are required?
- How it is tested? Please attach test report.
- Is it a performance related change? Please attach the performance
test report.
- Any additional information to help reviewers in testing this
change.
- [ ] For large changes, please consider breaking it into sub-tasks under
an umbrella JIRA.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/xubo245/carbondata carbonstoreCS
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/carbondata/pull/2673.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2673
----
commit e80126a208a173225c9899df8e4a9f177a07636d
Author: Jacky Li <jacky.likun@...>
Date: 2018-06-13T15:57:00Z
[CARBONDATA-2609] Change RPC implementation to Hadoop RPC framework
This closes #2372
commit 437556bb8d44380836956cc341718bb56dce5cdf
Author: QiangCai <qiangcai@...>
Date: 2018-07-03T12:21:18Z
[CARBONDATA-2690][CarbonStore] implement RESTful API: create table, load
data and select
This PR adds:
1.basic framework
rewrite the carbon store's Master, Worker and Scheduler code in Java
2.RESTful API
support create a table by using file meta store
support load data to a table in single work
support select data with a filter
This closes #2440
commit 01c623d67dc0384674b2e489a88d819f687d6ac6
Author: xuchuanyin <xuchuanyin@...>
Date: 2018-06-13T01:03:28Z
[CARBONDATA-2613] Support csv based carbon table
1. create csv based carbon table using
CREATE TABLE fact_table (col1 bigint, col2 string, ..., col100 string)
STORED BY 'CarbonData'
TBLPROPERTIES(
'foramt'='csv',
'csv.delimiter'=',',
'csv.header'='col1,col2,col100')
2. Load data to this table using
ALTER TABLE fact_table ADD SEGMENT LOCATION 'path/to/data1'
This closes #2374
commit 464e948d55f43a71d82c5fb654f4e1ea66c1ef65
Author: Jacky Li <jacky.likun@...>
Date: 2018-07-09T04:23:49Z
[CARBONDATA-2705][CarbonStore] CarbonStore Java API and Implementation
Support two implementations:
1.LocalCarbonStore for usage in local mode
2.DistributedCarbonStore leveraging multiple server (Master and Workers)
via RPC
This closes #2473
commit 4cd7f5881b5595cf2b321dcb295a270162e53623
Author: Jacky Li <jacky.likun@...>
Date: 2018-07-10T13:20:45Z
[CARBONDATA-2688][CarbonStore] Support SQL in REST API
Support SQL interface in Horizon service.
Support REST client for SQL
This closes #2481
commit 72bdef53c6582e2b8127430675c1324c73ddc825
Author: Jacky Li <jacky.likun@...>
Date: 2018-07-18T02:14:43Z
[REBASE] Rebasing with master branch and Fixing rebase conflict
commit a2d5b098d0764c61044039cc519601995347ae6d
Author: Ajith <ajith2489@...>
Date: 2018-07-12T03:47:22Z
[CARBONDATA-2736][CARBONSTORE] Kafka integration with Carbon StreamSQL
Modification in this PR:
1.Pass source table properties to streamReader.load()
2.Do not pass schema when sparkSession.readStream
3.Remove querySchema validation against sink as dataFrame made from kafka
source will not have schema ( its written in value column of schema )
4.Extract the dataframe from kafka source which contain actual data schema
@ writeStream
This closes #2495
commit 8a63df595a10e2dbf03f08b108859f339ea4fe7a
Author: Ajith <ajith2489@...>
Date: 2018-07-18T11:18:54Z
[CARBONDATA-2752][CARBONSTORE] Carbon provide Zeppelin support
This closes #2522
commit 65249c0aaa5ff50452f5df24477d0f26e06a70b4
Author: QiangCai <qiangcai@...>
Date: 2018-07-19T06:50:38Z
[CARBONDATA-2767][CarbonStore] Fix task locality issue
If the Spark cluster and the Hadoop cluster are two different machine
cluster, the Spark tasks will run in RACK_LOCAL mode.
This closes #2528
commit f75a5c861ce00df9e00c4dbe50433d8fcf07ad0f
Author: QiangCai <qiangcai@...>
Date: 2018-07-24T03:18:59Z
[CARBONDATA-2776][CarbonStore] Support ingesting data from Kafka service
This closes #2544
commit 7443cd4da0d0d4d457456490a80995cdea7f7019
Author: Jacky Li <jacky.likun@...>
Date: 2018-07-31T18:16:26Z
[CARBONDATA-2825][CARBONDATA-2828] CarbonStore and InternalCarbonStore API
This closes #2589
commit 87706e04887383b58eca42a4477abf14969a63d8
Author: Ajith <ajith2489@...>
Date: 2018-08-12T11:24:27Z
[CARBONDATA-2826] support select using distributed carbon store
Provides select support with select columns pruning and filter pushdown
using new RDD for distributed carbon store
This closes 2631
commit 19e55a3a21cc3e4ab091863ee320917e05764a98
Author: Jacky Li <jacky.likun@...>
Date: 2018-08-17T08:03:10Z
[HOTFIX] fix findbugs and pom dependency
commit 1233c61de3ed5ad577b962766f9e94ab9d62e2a1
Author: xubo245 <xubo29@...>
Date: 2018-08-23T02:27:56Z
Test carbon performance
1.suppport explain in zeppelin
2.PageViewsProducer
3.change the dis dependency
4.add time for horizon
carbonstore for cs
add time
optimize
CI pass
commit 6deba93b73cd06df6330ca36c90e40060eb2c381
Author: xubo245 <xubo29@...>
Date: 2018-08-30T02:18:28Z
Rebase to carbondata master
1.TODO: S3 AK/SK configure
----
---
