Sahil Takiar has posted comments on this change. ( http://gerrit.cloudera.org:8080/15378 )
Change subject: IMPALA-9466: impala-shell client retry for hs2-http protocol ...................................................................... Patch Set 6: (1 comment) http://gerrit.cloudera.org:8080/#/c/15378/6/shell/impala_client.py File shell/impala_client.py: http://gerrit.cloudera.org:8080/#/c/15378/6/shell/impala_client.py@933 PS6, Line 933: _do_hs2_rpc > I think thats a valid point. The goal really is to retry in case of failure do we know what is causing the 5xx errors? are the 5xx errors a symptom of the connection drops, or are they caused by something else? I wish it were as simple as calling `self.transport.isOpen()`, but looking at the Thrift code: https://github.com/apache/thrift/blob/master/lib/py/src/transport/THttpClient.py#L126 - all it does is check if close has been called or not I don't know of a straightforward of detecting if the connection has been closed by the server. There are several articles online reporting similar problems: https://williamsbdev.com/posts/no-http-response-exceptions/ http://stefan.buettcher.org/cs/conn_closed.html https://stackoverflow.com/questions/10558791/apache-httpclient-interim-error-nohttpresponseexception I'm seeing a few different solutions online: * Check if the underlying socket is in the CLOSE_WAIT state. I'm not sure how to do this from Python / Thrift * Write/read some dummy data on the HTTP connection before using it - seems like a lot of overhead * Re-cycle idle connections; if a connection hasn't been used in over "x" minutes, then close it and re-connect (perhaps lazily) Re-cyling idle connections seems like a useful improvement regardless Open to further suggestions / input on this. Most of this is based on a bit of googling. -- To view, visit http://gerrit.cloudera.org:8080/15378 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I0da9e9e8d34a340eaf763397cc095ff6260d65d5 Gerrit-Change-Number: 15378 Gerrit-PatchSet: 6 Gerrit-Owner: Abhishek Rawat <[email protected]> Gerrit-Reviewer: Abhishek Rawat <[email protected]> Gerrit-Reviewer: David Knupp <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Sahil Takiar <[email protected]> Gerrit-Comment-Date: Thu, 12 Mar 2020 02:52:49 +0000 Gerrit-HasComments: Yes
