----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/26988/#review58542 -----------------------------------------------------------
Nice work! This looks great! The only thing I think we need to change is the execution engine level. It's possible for a single HS2 to have both MR and Tez (and Spark) tasks running. For example, in seperate sessions I can set the execution engine and run some queries in MR and some in tez. Thus I think each job id needs a paired execution engine. - Brock Noland On Oct. 23, 2014, 6:31 a.m., Mohit Sabharwal wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/26988/ > ----------------------------------------------------------- > > (Updated Oct. 23, 2014, 6:31 a.m.) > > > Review request for hive. > > > Bugs: HIVE-8568 > https://issues.apache.org/jira/browse/HIVE-8568 > > > Repository: hive-git > > > Description > ------- > > HIVE-8568 : Add HS2 API to fetch Job IDs for a given query > > This patch adds a Thrift call to fetch Job IDs corresponding to all > running MR tasks. The response includes a list of job IDs and an > enum indicating the execution engine. Throws an exception if no > jobIDs were fetched. > > Currently only returns MR job IDs. Support for Tez job IDs is left > as a todo item for a future commit. > > The client may need to make the call multiple times since the jobs > may not have started running. Exposed RunningJob associated with > ExecDriver as a public method, so that the Driver can access the > jobIDs corresponding to all running tasks. > > > Diffs > ----- > > > itests/hive-unit-hadoop2/src/test/java/org/apache/hive/TestThriftGetJobIDs.java > PRE-CREATION > ql/src/java/org/apache/hadoop/hive/ql/Driver.java > e25450531a71ef4ae4c6d9ea1788e618189a17cb > ql/src/java/org/apache/hadoop/hive/ql/DriverContext.java > c7d3b6652f89cf7b6507f35176962ff3287d112d > ql/src/java/org/apache/hadoop/hive/ql/exec/mr/ExecDriver.java > 4e3df75c614fe7232e670201f2560c7ccd1db41c > ql/src/java/org/apache/hadoop/hive/ql/thrift/JobIDSet.java PRE-CREATION > service/if/TCLIService.thrift 4024bb3f412440fb7533f2e2d8ebc9a7cdc0776d > service/src/gen/thrift/gen-cpp/TCLIService.h > 030475b25188c5d2494da4de0bd6edc1ae807eca > service/src/gen/thrift/gen-cpp/TCLIService.cpp > 209ce63ae1ffd593de81e8e0a8e73218afe3cd79 > service/src/gen/thrift/gen-cpp/TCLIService_server.skeleton.cpp > 988bb4c11ddb717f585e0ba2fb4773ec5fff77e6 > service/src/gen/thrift/gen-cpp/TCLIService_types.h > f32dc3c90caedba86d943a9295a2f246a7b0ec90 > service/src/gen/thrift/gen-cpp/TCLIService_types.cpp > 326d25b8b7d814f7bbdfab7dde805be4834493dc > > service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TCLIService.java > 54851b8d513179e3618ee5a974941bb6a72378b6 > > service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TExecutionEngine.java > PRE-CREATION > > service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TGetJobIDsReq.java > PRE-CREATION > > service/src/gen/thrift/gen-javabean/org/apache/hive/service/cli/thrift/TGetJobIDsResp.java > PRE-CREATION > service/src/gen/thrift/gen-php/TCLIService.php > d2462967c4ee40c46bdeb6c8e24e22e63f3567e3 > service/src/gen/thrift/gen-py/TCLIService/TCLIService-remote > f6ff43f021524adb1a179595bfdd9260e98bff28 > service/src/gen/thrift/gen-py/TCLIService/TCLIService.py > ebc65746ccd7e58c0878f426b429274d4b59ed0b > service/src/gen/thrift/gen-py/TCLIService/ttypes.py > 6cd64d0386f1e7c73eee7d9868c387c4942a5f9f > service/src/gen/thrift/gen-rb/t_c_l_i_service.rb > fd1ca9aa13f3db170caf310ebb1ee1bac9f70b63 > service/src/gen/thrift/gen-rb/t_c_l_i_service_types.rb > c731544888f7480b0b1af70440ce1697e8597c12 > service/src/java/org/apache/hive/service/cli/CLIService.java > f5751f1305d7dd4c1f74af5a3a4f94f018b7a38f > service/src/java/org/apache/hive/service/cli/CLIServiceClient.java > 3155c238ff688bfea16b0aaeea950599bb659b5b > service/src/java/org/apache/hive/service/cli/EmbeddedCLIServiceClient.java > 9cad5be198c063115a8e90c67b1c2fd910ca8bc6 > service/src/java/org/apache/hive/service/cli/ICLIService.java > c9cc1f4da56f1cd10f6348ea2b9e17e203b87664 > service/src/java/org/apache/hive/service/cli/operation/Operation.java > acb95cb015395f4a1a9280c3b0c719228e584df7 > > service/src/java/org/apache/hive/service/cli/operation/OperationManager.java > a57b6e5d322ac312636c19633cee44f711b653df > service/src/java/org/apache/hive/service/cli/operation/SQLOperation.java > 8cabf7ee2945296774d31925a2bce46a7320d668 > service/src/java/org/apache/hive/service/cli/session/HiveSession.java > 6359a5b879928e8726017520f9a733d6b11decd4 > service/src/java/org/apache/hive/service/cli/session/HiveSessionImpl.java > fa28a6b6a4acb61d8b442ed13b0421e1e0f13368 > service/src/java/org/apache/hive/service/cli/thrift/ThriftCLIService.java > a0a6e183bbd05cd61ba97f187d66b286c145969c > > service/src/java/org/apache/hive/service/cli/thrift/ThriftCLIServiceClient.java > 1af45398b895cd7616c5627d318422e14b81e734 > > service/src/test/org/apache/hive/service/cli/thrift/ThriftCLIServiceTest.java > 630cfc9124abf7a8871b613b967141d0447eb18e > > Diff: https://reviews.apache.org/r/26988/diff/ > > > Testing > ------- > > Added unit test that issues async execute statements and follows > it up with getJobID calls. > > Did not add to ThriftCLIServiceTest since test needs a miniMR cluster > to run jobs that generate jobIDs. > > > Thanks, > > Mohit Sabharwal > >
