----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/10857/ -----------------------------------------------------------
Review request for hive. Description ------- Draft patch for review. based on phase 1 mentioned in design doc. Deviates slighlty from doc in the follow ways... 1) adds a couple of (temporary) rest calls to enable/disable streaming on a table. Later these will be replaced with support in DDL. 2) Also also HTTP methods are GET for easy testing with web browser 3) Authentication disabled on the new streaming HTTP methods Usage Examples on db named 'sdb' & table named 'log' : 1) Setup db & table with single partition column 'date': hcat -e "create database sdb; use sdb; create table log(msg string, region string) partitioned by (date string) ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' LINES TERMINATED BY '\n' STORED AS TEXTFILE; " 2) To check streaming status: http://localhost:50111/templeton/v1/streaming/status?database=sdb&table=log 3) Enable Streaming: http://localhost:50111/templeton/v1/streaming/enable?database=sdb&table=log&col=date&value=1000 4) Get Chunk File to write to: http://localhost:50111/templeton/v1/streaming/chunkget?database=sdb&table=log&schema=blah&format=blah&record_separator=blah&field_separator=blah 5) Commit Chunk File: http://localhost:50111/templeton/v1/streaming/chunkcommit?database=sdb&table=log&chunkfile=/user/hive/streaming/tmp/sdb/log/2 6) Abort Chunk File: http://localhost:50111/templeton/v1/streaming/chunkabort?database=sdb&table=log&chunkfile=/user/hive/streaming/tmp/sdb/log/3 7) Roll Partition: http://localhost:50111/templeton/v1/streaming/partitionroll?database=sdb&table=log&partition_column=date&partition_value=3000 This addresses bug HIVE-4196. https://issues.apache.org/jira/browse/HIVE-4196 Diffs ----- common/src/java/org/apache/hadoop/hive/conf/HiveConf.java c61d95b hcatalog/webhcat/svr/src/main/java/org/apache/hcatalog/templeton/HcatStreamingDelegator.java PRE-CREATION hcatalog/webhcat/svr/src/main/java/org/apache/hcatalog/templeton/Server.java 29ac4b3 metastore/if/hive_metastore.thrift c2051f4 metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.h 7b31d28 metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp 3d69472 metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore_server.skeleton.cpp 3b90b44 metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/SkewedInfo.java d8d6e71 metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/SkewedValueList.java 030b54a metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ThriftHiveMetastore.java 5929cda metastore/src/gen/thrift/gen-php/metastore/ThriftHiveMetastore.php a69d214 metastore/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore-remote 6fd2cce metastore/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore.py 9b856e5 metastore/src/gen/thrift/gen-rb/thrift_hive_metastore.rb 25aa30c metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java dc14084 metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java cef50f4 metastore/src/java/org/apache/hadoop/hive/metastore/IMetaStoreClient.java a2d6b1b metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 2079337 metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java 233fb46 metastore/src/model/org/apache/hadoop/hive/metastore/model/MTable.java 2a78ce9 metastore/src/model/package.jdo a84d2bf metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java 00eb0b4 Diff: https://reviews.apache.org/r/10857/diff/ Testing ------- Manual testing only Thanks, Roshan Naik
