Joe McDonnell has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/22214 )
Change subject: IMPALA-11979: Add 'scheduling_seed' to customize consistent scheduling behavior ...................................................................... IMPALA-11979: Add 'scheduling_seed' to customize consistent scheduling behavior This adds a startup parameter --scheduling_seed which is a string identifier of an executor within an executor group. It should be unique within an executor group, but it could be reused across executors groups on the same system. This is used for scan range scheduling for remote filesystems, so this can be used to make the scheduling deterministic across multiple executor groups or when an executor group gets restarted on machines with different IP addresses. For example, the 3rd executor in an executor group of size 8 might use "executor_3_of_8" for its scheduling seed. If there are multiple executors groups of size 8, the 3rd in each can use that scheduling seed. Testing: - Ran core job Change-Id: Ie01c7d119cc88766082dbfca3ff685354d01f71f Reviewed-on: http://gerrit.cloudera.org:8080/22214 Reviewed-by: Joe McDonnell <[email protected]> Tested-by: Impala Public Jenkins <[email protected]> --- M be/src/experiments/hash-ring-util.cc M be/src/scheduling/executor-group.cc M be/src/scheduling/hash-ring-test.cc M be/src/scheduling/hash-ring.cc M be/src/scheduling/hash-ring.h M be/src/service/impala-server.cc M common/protobuf/statestore_service.proto 7 files changed, 55 insertions(+), 9 deletions(-) Approvals: Joe McDonnell: Looks good to me, approved Impala Public Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/22214 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Ie01c7d119cc88766082dbfca3ff685354d01f71f Gerrit-Change-Number: 22214 Gerrit-PatchSet: 6 Gerrit-Owner: Joe McDonnell <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Jason Fehr <[email protected]> Gerrit-Reviewer: Joe McDonnell <[email protected]> Gerrit-Reviewer: Pranav Lodha <[email protected]> Gerrit-Reviewer: Riza Suminto <[email protected]>
