Chu Cheng Li created HDDS-17801:
-----------------------------------

             Summary: Limit the stream ReadBlock buffer on the datanode to the 
requested range
                 Key: HDDS-17801
                 URL: https://issues.apache.org/jira/browse/HDDS-17801
             Project: Apache Ozone
          Issue Type: Improvement
          Components: Ozone Datanode
            Reporter: Chu Cheng Li
            Assignee: Chu Cheng Li


h3. Problem
* For each stream {{ReadBlock}} request, {{KeyValueHandler.readBlockImpl}} 
allocates a heap buffer of the full response size.
* The client asks for 1 MiB responses by default 
({{ozone.client.stream.read.response-data-size}}). Thus, a 4 KiB positioned 
read allocates 1 MiB on the datanode.
* At 2,000 positioned reads per second, the datanode allocates and clears about 
2 GB of heap per second. This makes other reads on the same datanode slower.

h3. Changes
* {{BlockReadCursor.responseDataSize()}} is now at most the length of the range 
that the request reads. {{readBlockImpl}} allocates its buffer with this size.
* The range starts and ends on checksum boundaries. Thus, one checksum interval 
always fits in the buffer.
* A request that is larger than the response size gets the same buffer and the 
same responses as before.

h3. Tests
* A small request gets a buffer of the size of its range. This includes a range 
in a short last chunk and a range in a chunk without checksums.
* A zero-length request gets an empty buffer.
* A request with more than one response gets a buffer of the response size and 
returns the correct data.

h3. Benchmark
freon {{ocpr}} on one host, each process pinned to its own CPUs, 256 MiB key, 
stream read path. One sequential reader runs together with positioned reads. 
The table shows the speed of the sequential reader in MB/s.
||Positioned reads||Before||After||
|None|1,158|1,135|
|8 threads x 64 KiB, same stream|971|1,048|
|8 threads x 4 KiB, same stream|977|1,096|
|2 threads x 4 KiB, other process and connection|982|1,073|
* Without positioned reads, the difference is run-to-run noise.
* The positioned reads also become faster with a small buffer. When the client 
asks for 16 KiB responses instead of 1 MiB, positioned reads from another 
connection go from 2,045 to 2,320 per second (about 14%).

Found during the measurements for HDDS-17399. The buffer has the full response 
size since HDDS-13975.



--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to