ss77892 opened a new pull request, #11051: URL: https://github.com/apache/ozone/pull/11051
## What changes were proposed in this pull request? HDDS-16208. Cache out-of-order streaming ReadBlock responses instead of discarding them to avoid client poll timeout StreamBlockInputStream assumes that streaming ReadBlock responses arrive in offset order. However, concurrent chunk readers on the datanode serve pre-read requests in non-deterministic order, so a response covering a later offset can arrive before the one covering the current position. When that happened, the reader dropped the out-of-order response after taking it from the response queue. Since the datanode never resends that data, the client was then stuck waiting for a chunk that would never arrive again, and the read eventually failed with a poll timeout. This PR makes the client keep such responses instead of discarding them: - StreamingReader gains a pendingProtos TreeMap keyed by block offset. When read() polls a response whose offset is ahead of the current position, the response is cached there instead of being dropped. - On every read(), the cache is checked first (pollPendingProtos()): the smallest cached entry covering the current position is served directly, without re-requesting the data from the datanode. Entries the reader has already advanced past are dropped. - seek() clears cached entries whose data ends at or before the new position (clearPendingProtosBelow()), so a chunk the reader seeked past is never served and does not leak memory. ## What is the link to the Apache JIRA https://issues.apache.org/jira/browse/HDDS-16208 ## How was this patch tested? Two UTs has been added Real cluster with HBase on top of Ozone FS with ReadBlock enabled under YCSB workloads. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
