peterxcli opened a new pull request, #12042:
URL: https://github.com/apache/ozone/pull/12042

   ## What changes were proposed in this pull request?
   
   For each stream `ReadBlock` request, the datanode allocates a heap buffer of 
the full response size. The client asks for 1 MiB responses by default 
(`ozone.client.stream.read.response-data-size`), so a 4 KiB positioned read 
still makes the datanode allocate and clear 1 MiB. At about 2,000 positioned 
reads per second, that is about 2 GB of heap per second, and it makes other 
reads on the same datanode slower.
   
   The buffer has had the full response size since HDDS-13975. The cost showed 
up during the measurements for HDDS-17399 (#11724).
   
   With this change:
   
   - `BlockReadCursor.responseDataSize()` is at most the length of the range 
that the request reads. `KeyValueHandler.readBlockImpl` already allocates its 
buffer with this value, so the handler does not change.
   - The range starts and ends on checksum boundaries, so one checksum interval 
always fits in the buffer.
   - A request that is larger than the response size gets the same buffer and 
the same responses as before. The cursor hands out the same ranges in all cases.
   
   ### Benchmark
   
   freon `ocpr` on one host, each process pinned to its own CPUs, 256 MiB key, 
stream read path. One sequential reader runs together with positioned reads. 
The table shows the speed of the sequential reader in MB/s.
   
   | Positioned reads | Before | After |
   |---|---|---|
   | None | 1,158 | 1,135 |
   | 8 threads x 64 KiB, same stream | 971 | 1,048 |
   | 8 threads x 4 KiB, same stream | 977 | 1,096 |
   | 2 threads x 4 KiB, other process and connection | 982 | 1,073 |
   
   Without positioned reads, the difference is run-to-run noise. The positioned 
reads also become faster with a small buffer: when the client asks for 16 KiB 
responses instead of 1 MiB, positioned reads from another connection go from 
2,045 to 2,320 per second (about 14%).
   
   ## What is the link to the Apache JIRA
   
   https://issues.apache.org/jira/browse/HDDS-17801
   
   ## How was this patch tested?
   
   - `TestBlockReadCursor#testResponseDataSizeIsAtMostTheRequestedRange`: a 
small request gets a buffer of the size of its range, also in a short last 
chunk and in a chunk without checksums. A zero-length request gets an empty 
buffer. Requests with more than one response keep the response size and the 
same ranges.
   - `TestKeyValueHandler#testReadBlockBufferIsAtMostTheRequestedRange`: reads 
through the handler and checks the capacity of the buffer that it passes to the 
block file. Two small reads get 64 KiB and 20 bytes instead of 1 MiB. A read 
with two responses gets a buffer of the response size and returns the correct 
data.
   - Both tests fail without the change.
   - The other tests in `TestBlockReadCursor`, `TestKeyValueHandler` and 
`TestGrpcXceiverService` pass.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to