szetszwo commented on code in PR #10415:
URL: https://github.com/apache/ozone/pull/10415#discussion_r3357592089
##########
hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/storage/StreamBlockInputStream.java:
##########
@@ -208,8 +214,11 @@ public synchronized void seek(long pos) throws IOException
{
if (pos == position) {
return;
}
- LOG.debug("{}: seek {} -> {}", this, position, pos);
- closeStream();
+ LOG.debug("{}: seek {} -> {}", getName(streamingReader), position, pos);
+ if (streamingReader != null && readBuffer != null) {
+ streamingReader.offerToQueue(readBuffer.getProto());
Review Comment:
> Then the read buffer will pull potentally one of those chunks off the
queue into the buffer, not necessairly all of them - is that correct?
It is correct -- the protos are in the queue and they will be pulled one by
one.
> Do we need to ensure the queue is empty before pushing data read from the
queue back onto it?
No, because we will drop it if the proto is outside the required range; see
the change below:
```diff
ByteBuffer readFromQueue() throws IOException {
final long blockOffset = readBlock.getOffset();
final long pos = getPos();
if (pos < blockOffset) {
- // This should not happen, and if it does, we have a bug.
- setFailedAndThrow(new IllegalStateException(
- this + ": out of order, position " + pos + " < block offset " +
blockOffset));
+ // This can happen after seek, just drop the buffer
+ return null;
}
final long offset = pos - blockOffset;
if (offset > 0) {
dataBuffer.position(Math.toIntExact(Math.min(offset,
dataBuffer.limit())));
}
LOG.debug("{}: return response positon {}, length {} (block offset
{}, length {})",
name, pos, dataBuffer.remaining(), blockOffset, data.size());
return dataBuffer;
}
```
> ... how do we know where to read from the queue? ...
The proto has the offset and length information. The readFromQueue() method
will check and set the dataBuffer before returning it; see above.
> Then we seek to offset 100MB - all that queued data needs to be dropped
...
That's correct.
> ... if we seek instread to 20MB, then on the queue is some data we need.
Continue with your example, the code will drop the first 20MB and use the
remaining 12MB.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
