Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/13306 )
Change subject: IMPALA-1653: Don't close hiveserver2 session when connection is closed ...................................................................... Patch Set 5: (11 comments) http://gerrit.cloudera.org:8080/#/c/13306/5//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/13306/5//COMMIT_MSG@27 PS5, Line 27: disconnected_esession_timeout typo http://gerrit.cloudera.org:8080/#/c/13306/5/be/src/service/impala-server.cc File be/src/service/impala-server.cc: http://gerrit.cloudera.org:8080/#/c/13306/5/be/src/service/impala-server.cc@211 PS5, Line 211: DEFINE_int32(disconnected_session_timeout, 60 * 60, "The time, in seconds, that a " I think we could set this lower. One data point is that hue sets the query timeout at 5m and the session timeout at 15m: https://github.com/cloudera/hue/blob/dadf7b58654848e5870fdd9f1e4b80ab0ec5fafb/apps/impala/src/impala/conf.py#L82 http://gerrit.cloudera.org:8080/#/c/13306/5/be/src/service/impala-server.cc@1393 PS5, Line 1393: if (session_type == TSessionType::HIVESERVER2) { Probably worth documenting why this is necessary, i.e. we might be using this session from this connection for the first time. http://gerrit.cloudera.org:8080/#/c/13306/5/be/src/service/impala-server.cc@2053 PS5, Line 2053: sessions_to_close Maybe disconnected_sessions? http://gerrit.cloudera.org:8080/#/c/13306/5/be/src/service/impala-server.cc@2064 PS5, Line 2064: sessions_to_close = it->second; move() to avoid copying set? http://gerrit.cloudera.org:8080/#/c/13306/5/be/src/service/impala-server.cc@2085 PS5, Line 2085: DCHECK DCHECK_EQ http://gerrit.cloudera.org:8080/#/c/13306/5/be/src/service/impala-server.cc@2127 PS5, Line 2127: session_timeout_cv_.WaitFor(timeout_lock, MICROS_PER_SEC); Not your change, but I feel like this was a bit overengineered. If I understood correctly all this machinery is added to avoid sweeping over the map once per second if there's no session in the system that might need to be expired. http://gerrit.cloudera.org:8080/#/c/13306/5/be/src/service/impala-server.cc@2150 PS5, Line 2150: if (session_state->connections.size() == 0) { Why is this mutually exclusive with the idle check? I think if this hits the idle timeout before the disconnected session timeout, we should still expire it. http://gerrit.cloudera.org:8080/#/c/13306/5/be/src/service/impala-server.cc@2192 PS5, Line 2192: // Remove any timed out disconnected sessions from the map. Missing an and/or? http://gerrit.cloudera.org:8080/#/c/13306/5/tests/custom_cluster/test_hs2.py File tests/custom_cluster/test_hs2.py: http://gerrit.cloudera.org:8080/#/c/13306/5/tests/custom_cluster/test_hs2.py@40 PS5, Line 40: def test_disconnected_session_timeout(self): Would be good to also test interaction with the idle timeout - i.e. if there's a low idle timeout and a high disconnected timeout, sessions still get expired. http://gerrit.cloudera.org:8080/#/c/13306/5/tests/hs2/test_hs2.py File tests/hs2/test_hs2.py: http://gerrit.cloudera.org:8080/#/c/13306/5/tests/hs2/test_hs2.py@621 PS5, Line 621: reamins remains -- To view, visit http://gerrit.cloudera.org:8080/13306 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia4555cd9b73db5b4dde92cd4fac4f9bfa3664d78 Gerrit-Change-Number: 13306 Gerrit-PatchSet: 5 Gerrit-Owner: Thomas Marshall <[email protected]> Gerrit-Reviewer: Andrew Sherman <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Michael Ho <[email protected]> Gerrit-Reviewer: Thomas Marshall <[email protected]> Gerrit-Reviewer: Tim Armstrong <[email protected]> Gerrit-Comment-Date: Tue, 21 May 2019 00:11:32 +0000 Gerrit-HasComments: Yes
