anmolanmol1234 commented on code in PR #8830:
URL: https://github.com/apache/hadoop/pull/8830#discussion_r4522583049
##########
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/ReadBufferManagerV2.java:
##########
@@ -890,43 +1119,128 @@ private ReadBuffer clearFromReadAheadQueue(final String
eTag,
*/
private int getBlockFromCompletedQueue(final String eTag, final long
position,
final int length, final byte[] buffer) throws IOException {
- ReadBuffer buf = getBufferFromCompletedQueue(eTag, position);
+ // Find any child buffer from the window that covers the requested position
+ ReadBuffer anyChild = getBufferFromCompletedQueue(eTag, position);
- if (buf == null) {
+ if (anyChild == null) {
+ // No buffer covers this position - cache miss
+ printTraceLog(
+ "No buffer in completed queue covers position: {} for eTag: {}",
+ position, eTag);
return 0;
}
- buf.startReading(); // atomic increment of refCount.
+ // Get the parent and its children
+ ReadBuffer parent = anyChild.getParentBuffer();
+
+ if (parent == null) {
+ // This is a standalone buffer (not part of a parent/child structure)
+ // Fall back to simple single-buffer copy
+ return copyFromSingleBuffer(anyChild, position, length, buffer);
+ }
+
+ List<ReadBuffer> children = parent.getChildBufferList();
+ // Increment refCount on ALL children first to prevent eviction
+ for (ReadBuffer child : children) {
+ child.startReading();
+ }
+
+ try {
+ for (ReadBuffer child : children) {
+ if (child.getStatus() == ReadBufferStatus.READ_FAILED) {
Review Comment:
getBufferFromCompletedQueue now returns any completed child whose parent
window covers the position. The loop here only checks for READ_FAILED, then
copies from the shared array on the assumption that all children succeeded. A
sibling can still be NOT_AVAILABLE in the read-ahead queue or in progress.
waitForProcess only waits when it finds an in-progress buffer, so it may not
wait when siblings are only queued. That lets the copy return unfilled bytes.
Please require every child to be AVAILABLE before copying, and otherwise return
0 so the caller does a direct read.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
