bryancall opened a new issue, #13777:
URL: https://github.com/apache/trafficserver/issues/13777

   ## Summary
   
   When ATS talks HTTP/2 to the origin and answers an HTTP/1.1 client with a 
chunked response, the response can hang without its terminating chunk. The 
client gets the whole body but never gets `0\r\n\r\n`, so it waits until 
`transaction_active_timeout_out` ends the transaction.
   
   It only affects responses larger than the consumer buffer's water mark, 
about 32 KiB.
   
   ## Cause
   
   1. `HttpTunnel::_throttle_chunked_producer` (added in f75672ec81, "Fix 
ChunkHandler Flow Control") calls `p->read_vio->disable()` when a consumer's 
buffer goes over its high water mark.
   2. If the producer is an outbound `Http2Stream` and the origin's final DATA 
frame with END_STREAM arrives while the read VIO is disabled, 
`Http2ConnectionState::rcv_data_frame` gates on `stream->is_read_enabled()` and 
never signals `VC_EVENT_READ_COMPLETE`. The pure-END_STREAM path (empty DATA 
frame) does the same. `Http2Stream::signal_read_event` also returns early when 
`read_vio.is_disabled()`.
   3. When the tunnel drains and re-enables the producer, 
`Http2Stream::reenable(VIO::READ)` only calls 
`connection_state.restart_receiving(this)`, which updates the flow-control 
window. Nothing re-delivers the completion the stream dropped.
   4. The tunnel never sees READ_COMPLETE for the producer, so the chunking 
consumer never writes the final chunk.
   
   While the transaction sits like this, `Http2Stream::main_event_handler` logs 
`HTTP/2 unknown case of <event> event` once a second. The timeout event arrives 
with `_sm` set and no read or write work left, so it falls into the `unknown 
case` branch.
   
   ## Observed
   
   On a 10.0.x build with HTTP/2 to origin enabled 
(`proxy.config.ssl.client.alpn_protocols: h2,http/1.1`). The code involved is 
the same on master.
   
   - Size threshold, for the origin where most stuck streams occurred: none of 
about 90,000 responses under 32 KiB hung. 34 of about 4,100 over 32 KiB did, 
and the smallest was 34,301 bytes.
   - Duration: 83 HTTP/2 transactions took over 524 s in 30 minutes, against 0 
on HTTP/1.1 origins on the same host.
   - Logs: one `HTTP/2 unknown case of ... event` warning per second per stuck 
stream.
   
   ## Suggested fix
   
   Make a disabled read VIO remember a completion it couldn't deliver. For 
example, in `Http2Stream::reenable(VIO::READ)`, if `receive_end_stream` is set 
and the buffered data has all been handed over, signal `VC_EVENT_READ_COMPLETE` 
(through `update_read_request` or `signal_read_event`) as well as calling 
`restart_receiving`.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to