GitHub user xubo245 reopened a pull request:

    https://github.com/apache/carbondata/pull/2073

    [CARBONDATA-2260] CarbonThriftServer should support store carbon table on S3

    CarbonThriftServer should support store carbon table on S3
    
    
    Be sure to do all of the following checklist to help us incorporate 
    your contribution quickly and easily:
    
     - [x] Any interfaces changed?
     
     - [x] Any backward compatibility impacted?
     
     - [x] Document update required?
    
     - [x] Testing done
            Step 1: 
              start carbonThriftServer by  
org.apache.carbondata.examples.CarbonThriftServerExample
              User should use themselves storePath, AK, SK and endPoint
          Step 2: 
              start beeline like:  beeline -u jdbc:hive2://IP:10000
             IP and port should provide
           Now it support running with local, HDFS and S3 mode
    Test example:
    
    
    ```
    root@ecs-909c:~# beeline -u jdbc:hive2://192.168.0.206:10000
    Connecting to jdbc:hive2://192.168.0.206:10000
    2018-03-17 17:44:29 INFO  Utils:310 - Supplied authorities: 
192.168.0.206:10000
    2018-03-17 17:44:29 INFO  Utils:397 - Resolved authority: 
192.168.0.206:10000
    2018-03-17 17:44:29 INFO  HiveConnection:203 - Will try to open client 
transport with JDBC Uri: jdbc:hive2://192.168.0.206:10000
    Connected to: Spark SQL (version 2.2.1)
    Driver: Hive JDBC (version 1.2.1.spark2)
    Transaction isolation: TRANSACTION_REPEATABLE_READ
    Beeline version 1.2.1.spark2 by Apache Hive
    0: jdbc:hive2://192.168.0.206:10000>  Drop table if exists local_table2;
    +---------+--+
    | Result  |
    +---------+--+
    +---------+--+
    No rows selected (21.205 seconds)
    0: jdbc:hive2://192.168.0.206:10000>
    0: jdbc:hive2://192.168.0.206:10000> CREATE TABLE if not exists 
local_table2( shortField SHORT, intField INT, bigintField LONG, doubleField 
DOUBLE, stringField STRING, timestampField TIMESTAMP, decimalField 
DECIMAL(18,2), dateField DATE, charField CHAR(5), floatField FLOAT ) STORED BY 
'carbondata' LOCATION 's3a://carbonstore/local_table2'  
TBLPROPERTIES('SORT_COLUMNS'='');
    +---------+--+
    | Result  |
    +---------+--+
    +---------+--+
    No rows selected (1.296 seconds)
    0: jdbc:hive2://192.168.0.206:10000>
    0: jdbc:hive2://192.168.0.206:10000>
    0: jdbc:hive2://192.168.0.206:10000> LOAD DATA LOCAL INPATH 
'/huawei/xubo/git/carbondata2/examples/spark2/src/main/resources/data1.csv' 
INTO TABLE local_table2 OPTIONS('HEADER'='true');
    +---------+--+
    | Result  |
    +---------+--+
    +---------+--+
    No rows selected (3.339 seconds)
    0: jdbc:hive2://192.168.0.206:10000>
    0: jdbc:hive2://192.168.0.206:10000>  select * from local_table2;
    
+-------------+-----------+--------------+--------------+--------------+------------------------+---------------+-------------+------------+-------------+--+
    | shortfield  | intfield  | bigintfield  | doublefield  | stringfield  |    
 timestampfield     | decimalfield  |  datefield  | charfield  | floatfield  |
    
+-------------+-----------+--------------+--------------+--------------+------------------------+---------------+-------------+------------+-------------+--+
    | 1           | 10        | 1100         | 48.4         | spark        | 
2015-04-23 12:01:01.0  | 1.23          | 2015-04-23  | aaa        | 2.5         
|
    | 5           | 17        | 1140         | 43.4         | spark        | 
2015-07-27 12:01:02.0  | 3.45          | 2015-07-27  | bbb        | 2.5         
|
    | 1           | 11        | 1100         | 44.4         | flink        | 
2015-05-23 12:01:03.0  | 23.23         | 2015-05-23  | ccc        | 2.5         
|
    | 1           | 10        | 1150         | 43.4         | spark        | 
2015-07-24 12:01:04.0  | 254.12        | 2015-07-24  | ddd        | 2.5         
|
    | 1           | 10        | 1100         | 47.4         | spark        | 
2015-07-23 12:01:05.0  | 876.14        | 2015-07-23  | eeee       | 3.5         
|
    | 3           | 14        | 1160         | 43.4         | hive         | 
2015-07-26 12:01:06.0  | 3454.32       | 2015-07-26  | ff         | 2.5         
|
    | 2           | 10        | 1100         | 43.4         | impala       | 
2015-07-23 12:01:07.0  | 456.98        | 2015-07-23  | ggg        | 2.5         
|
    | 1           | 10        | 1100         | 43.4         | spark        | 
2015-05-23 12:01:08.0  | 32.53         | 2015-05-23  | hhh        | 2.5         
|
    | 4           | 16        | 1130         | 42.4         | impala       | 
2015-07-23 12:01:09.0  | 67.23         | 2015-07-23  | iii        | 2.5         
|
    | 1           | 10        | 1100         | 43.4         | spark        | 
2015-07-23 12:01:10.0  | 832.23        | 2015-07-23  | jjj        | 2.5         
|
    
+-------------+-----------+--------------+--------------+--------------+------------------------+---------------+-------------+------------+-------------+--+
    10 rows selected (0.959 seconds)
    0: jdbc:hive2://192.168.0.206:10000>
    
    
    ```
           
     - [x] For large changes, please consider breaking it into sub-tasks under 
an umbrella JIRA. 
    


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/xubo245/carbondata 
CARBONDATA-2260-carbonthriftserver-support-s3

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/carbondata/pull/2073.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #2073
    
----
commit c59fe8d687dac498fccf341b972d2a0c5ddadbf5
Author: root <601450868@...>
Date:   2018-03-17T09:45:34Z

    [CARBONDATA-2260] CarbonThriftServer should support store carbon table on S3
    
    UPDATE SCALA STYLE
    
    FIX scala style
    
    fix scalastyle
    
    support run this class with local, hdfs and s2 mode

----


---

Reply via email to