[
https://issues.apache.org/jira/browse/IMPALA-9383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17039731#comment-17039731
]
ASF subversion and git services commented on IMPALA-9383:
---------------------------------------------------------
Commit f216e68089e49af970674600fc1ff05b759d641b in impala's branch
refs/heads/master from Thomas Tauber-Marshall
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=f216e68 ]
IMPALA-9383: Fix hang in hs2-http server with large, chunks request
The issue is that in each call to THttpTransport::read(), it always
starts by calling refill(), which tries to read more data off the
socket, but for chunked requests each call to read() only processes a
single chunk.
So, if more than one chunk is read off the socket at a time (which
happens with large requests due to an algorithm that increases the
amount of data read off the socket with each read), you can end up
with more chunks still needing to be processed but no more data to
read off the socket, and the next call to THttpTransport::read() will
hang when it calls refill().
The solution is to not always call refill() at the beginning of each
call to THttpTransport::read(). This works because the functions that
actually process the data, readLine() and readContent() will call
refill() themselves anyways if they actually need more data.
Testing:
- Added a BE test that uses curl to send chunked requests to the hs2
http server.
Change-Id: If822fb9cd2d7f2b0f5e36879fd7ccbba9a217ad0
Reviewed-on: http://gerrit.cloudera.org:8080/15218
Reviewed-by: Impala Public Jenkins <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> HS2 HTTP server hangs on large chunked requests
> -----------------------------------------------
>
> Key: IMPALA-9383
> URL: https://issues.apache.org/jira/browse/IMPALA-9383
> Project: IMPALA
> Issue Type: Bug
> Reporter: Thomas Tauber-Marshall
> Assignee: Thomas Tauber-Marshall
> Priority: Critical
>
> There's a bug in THttpTransport that causes it to hang when sent a large,
> chunked request.
> The issue is that in each call to THttpTransport::read(), it always starts by
> calling refill() which reads more data off the socket, but for chunked
> requests each call to read() only processes a single chunk.
> So, if more than one chunk is read off the socket at a time, you can end up
> with more chunks still needing to be processed but no more data to read off
> the socket, and the next call to THttpTransport::read() will hang when it
> calls refill().
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
