Impala Public Jenkins has submitted this change and it was merged. Change subject: IMPALA-5377: Impala may crash if given a fragment instance while restarting ......................................................................
IMPALA-5377: Impala may crash if given a fragment instance while restarting If an Impala daemon restarts quickly enough so that the statestore does not detect a failure, coordinators will still issue fragment instances to that daemon. There's a race in that situation, where the BE service port was opened before ExecEnv::StartServices() had initialized the process-wide memtracker; trying to run the finstance would then crash the process. This patch inverts the order of ExecEnv::StartServices() and starting the backend and client servers. Testing: reproduced by manually injecting sleeps after be_server->Start() and running a query on the cluster. After the patch, sleeping *before or after* be_server->Start() did not trigger a crash, and the query failed as expected if it was executed in that narrow window. No automated testing added: this is a hard bug to hit without sleeps and would need some repeated kill-and-restart loops to have a reasonable chance of triggering. Change-Id: I1d53b36304cb86c43e110e10cf76a39976ae3bd5 Reviewed-on: http://gerrit.cloudera.org:8080/7036 Reviewed-by: Dan Hecht <[email protected]> Tested-by: Impala Public Jenkins --- M be/src/service/impalad-main.cc 1 file changed, 10 insertions(+), 7 deletions(-) Approvals: Impala Public Jenkins: Verified Dan Hecht: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/7036 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: merged Gerrit-Change-Id: I1d53b36304cb86c43e110e10cf76a39976ae3bd5 Gerrit-PatchSet: 3 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Henry Robinson <[email protected]> Gerrit-Reviewer: Alex Behm <[email protected]> Gerrit-Reviewer: Dan Hecht <[email protected]> Gerrit-Reviewer: Henry Robinson <[email protected]> Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong <[email protected]>
