Yuxuan Wang created THRIFT-5490:
-----------------------------------
Summary: Improve memory efficiency in go THeader implementation
Key: THRIFT-5490
URL: https://issues.apache.org/jira/browse/THRIFT-5490
Project: Thrift
Issue Type: Improvement
Components: Go - Library
Reporter: Yuxuan Wang
Currently in go's THeaderTransport implementation, due to THeader's framed
nature, there are 2 buffers used, one for read and one for write. The 2 buffers
are part of THeaderTransport and have the same lifecycle with the transport
itself.
This caused an issue with idle connections (e.g. connections sitting in a
pool). For idle connections, their buffers were grew to handle the
request/response, but never shrink, even when the connection is sitting in
idle. So when a connection pool is used, every idle connection sitting in the
pool has a memory profile of largest read + largest write they have ever had in
their whole lifecycle, and when we have a server handling large
request/response payloads with a lot of idle connections, the memory footprint
can grow significant.
We should use a resource pool, and return the buffers to the pool after done
reading/writing, to better manage memory footprint.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
