Henry Robinson has uploaded a new change for review. http://gerrit.cloudera.org:8080/4907
Change subject: IMPALA-4415: Fix unassigned scan range of size 1 ...................................................................... IMPALA-4415: Fix unassigned scan range of size 1 ComputeScanRangeAssignment() computes the per-node scan-range load as the total number of bytes to be scanned / number of nodes, and casts that to a float. However, that computation suffers from a precision issue where the average * num_nodes may be 1 byte less than the total. The scan range assignment loop continues until at least average * num_nodes bytes have been assigned. If the last scan range has only 1 byte, it will not be assigned (if it has 2 or more bytes, it will be considered for assignment before the loop exit condition is met). The fix is to make sure that all instances are assigned in the last iteration of the assignment loop, even if the per-node threshold is already met. Testing: No local repro was found - only S3 and LocalFS builds have it. A unit test requires a lot of infrastructure from simple-scheduler-util that doesn't exist yet (e.g. ComputeScanRangeAssignment() is not called). S3 and LocalFS full test builds are in progress. Change-Id: Id3af767ee9d121ca62ac383ef9e696a18dc903d6 --- M be/src/scheduling/simple-scheduler.cc 1 file changed, 14 insertions(+), 7 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/07/4907/1 -- To view, visit http://gerrit.cloudera.org:8080/4907 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Id3af767ee9d121ca62ac383ef9e696a18dc903d6 Gerrit-PatchSet: 1 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Henry Robinson <[email protected]>
