Marcel Kornacker has uploaded a new patch set (#3). Change subject: IMPALA-3902: Scheduler improvements for running multiple fragment instances on a single backend ......................................................................
IMPALA-3902: Scheduler improvements for running multiple fragment instances on a single backend This is an extension of the scheduler and coordinator for multi-threaded execution. It mainly removes the assumption of having a single fragment instance per host. The approach taken here is to create parallel data structures and control flow functions, where necessary, and otherwise to leave the existing single-instance logic in place. The parallel structures' and functions' names are prefixed with "Mt" to facilitate the enventual clean-up. Changes to data structures: - QuerySchedule: per-instance and per-fragment structs with complete execution parameters (instead of partially relying on TQueryExecRequest); the per-instance execution parameter struct is a child of the per-fragment parameter struct - explicit fragent id, with range 0..#fragments-1 (instead of relying on an index into an array in TQueryExecRequest) Excluded: - runtime filter handling - anything related to RM Change-Id: I240445199e22f009f4e72fdb8754eb8d77e3d680 --- M be/src/common/global-types.h M be/src/runtime/coordinator.cc M be/src/runtime/coordinator.h M be/src/scheduling/query-resource-mgr.cc M be/src/scheduling/query-resource-mgr.h M be/src/scheduling/query-schedule.cc M be/src/scheduling/query-schedule.h M be/src/scheduling/simple-scheduler.cc M be/src/scheduling/simple-scheduler.h M be/src/service/query-exec-state.cc M be/src/service/query-options.cc M be/src/service/query-options.h M common/thrift/ExecStats.thrift M common/thrift/Frontend.thrift M common/thrift/ImpalaInternalService.thrift M common/thrift/ImpalaService.thrift M common/thrift/Planner.thrift M common/thrift/Types.thrift M fe/src/main/java/com/cloudera/impala/common/TreeNode.java M fe/src/main/java/com/cloudera/impala/planner/Planner.java M fe/src/main/java/com/cloudera/impala/service/Frontend.java M fe/src/test/java/com/cloudera/impala/planner/PlannerTestBase.java 22 files changed, 1,339 insertions(+), 431 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/54/4054/3 -- To view, visit http://gerrit.cloudera.org:8080/4054 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: newpatchset Gerrit-Change-Id: I240445199e22f009f4e72fdb8754eb8d77e3d680 Gerrit-PatchSet: 3 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Marcel Kornacker <[email protected]>
