David Rorke has posted comments on this change. ( http://gerrit.cloudera.org:8080/19532 )
Change subject: IMPALA-11904: Data cache support dumping for reloading ...................................................................... Patch Set 5: (1 comment) http://gerrit.cloudera.org:8080/#/c/19532/5/be/src/scheduling/executor-group.cc File be/src/scheduling/executor-group.cc: http://gerrit.cloudera.org:8080/#/c/19532/5/be/src/scheduling/executor-group.cc@108 PS5, Line 108: auto cmp = [](const BackendDescriptorPB& a, const BackendDescriptorPB& b) { : return a.address().port() < b.address().port(); : }; : std::sort(be_descs.begin(), be_descs.end(), cmp); This seems fine for providing consistent scheduling across multiple backends within a host, but I think there are a couple other cases that may need to be handled: * Number of executors in the group changes after restart leading to different scan range scheduling (maybe we should just not support reload and clear any dumped cache state in this case?). * IP addresses of executors in the group change after restart leading to different hashing and different scan range scheduling (IMPALA-11979). Note IP addresses will change on restart in a kubernetes environment. -- To view, visit http://gerrit.cloudera.org:8080/19532 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Id867f4fc7343898e4906332c3caa40eb57a03101 Gerrit-Change-Number: 19532 Gerrit-PatchSet: 5 Gerrit-Owner: Anonymous Coward <[email protected]> Gerrit-Reviewer: Anonymous Coward <[email protected]> Gerrit-Reviewer: David Rorke <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Joe McDonnell <[email protected]> Gerrit-Comment-Date: Mon, 20 Mar 2023 16:37:34 +0000 Gerrit-HasComments: Yes
