[
https://issues.apache.org/jira/browse/HDDS-16722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chu Cheng Li updated HDDS-16722:
--------------------------------
Description:
HDDS-9904 adds a block read through the Ratis data stream. This read is an
alternative to the gRPC streaming read. This sub-task adds the datanode part.
This sub-task contains a part of https://github.com/apache/ozone/pull/10743.
That PR is too large for one review.
*Changes*
* {{ContainerStateMachine}} implements {{transferTo(Message,
WritableByteChannel)}} for ReadBlock requests.
** It gives the request to the streaming read of the dispatcher.
** It writes each response to the read-only stream as one reply. A reply has
three parts, in this order: the length of the metadata, the response without
the data, and the data.
* {{ClosedContainerReadResolver}} serves the reads of closed containers. It is
a Ratis {{DataStreamApi.Resolver}} (RATIS-2603).
** SCM gives the read pipeline of a closed container a random ID. No Raft group
has this ID. Thus, a read-only stream on this pipeline fails with a group
mismatch.
** The resolver finds the container by its ID, not by the Raft group. It serves
the read only when the container is closed and the request names this datanode.
* A new {{DatanodeVersion}}, {{RATIS_DATASTREAM_READ_BLOCK_SUPPORT}}, tells
clients which datanodes can serve this read.
*Tests*
* Unit tests examine the replies and the resolver.
* An end-to-end test reads a block with a plain Ratis client.
was:
HDDS-9904 adds a streaming block read over the Ratis data stream, as an
alternative to the gRPC streaming read. This sub-task covers the datanode side.
It is split out of https://github.com/apache/ozone/pull/10743, which was too
large to review in one piece.
* {{ContainerStateMachine}} implements {{transferTo(Message,
WritableByteChannel)}} for ReadBlock. It runs the request through the
dispatcher's streaming read and writes each response to the read-only stream as
one reply: the length of the metadata, the response without its data, then the
data.
* Read pipelines of closed containers have random IDs with no Raft group behind
them, so a read-only stream opened on one fails with a group mismatch.
{{ClosedContainerReadResolver}}, a Ratis {{DataStreamApi.Resolver}}
(RATIS-2603), serves these reads by container instead, when the container is
closed and the request names this datanode.
* A new {{DatanodeVersion}}, {{RATIS_DATASTREAM_READ_BLOCK_SUPPORT}}, tells
clients which datanodes can serve it.
Testing: unit tests for the replies and the resolver, and an end-to-end test
that reads a block with a plain Ratis client.
> Serve ReadBlock over Ratis read-only data streams on the datanode
> -----------------------------------------------------------------
>
> Key: HDDS-16722
> URL: https://issues.apache.org/jira/browse/HDDS-16722
> Project: Apache Ozone
> Issue Type: Sub-task
> Components: Ozone Datanode
> Reporter: Chu Cheng Li
> Assignee: Chu Cheng Li
> Priority: Major
>
> HDDS-9904 adds a block read through the Ratis data stream. This read is an
> alternative to the gRPC streaming read. This sub-task adds the datanode part.
> This sub-task contains a part of https://github.com/apache/ozone/pull/10743.
> That PR is too large for one review.
> *Changes*
> * {{ContainerStateMachine}} implements {{transferTo(Message,
> WritableByteChannel)}} for ReadBlock requests.
> ** It gives the request to the streaming read of the dispatcher.
> ** It writes each response to the read-only stream as one reply. A reply has
> three parts, in this order: the length of the metadata, the response without
> the data, and the data.
> * {{ClosedContainerReadResolver}} serves the reads of closed containers. It
> is a Ratis {{DataStreamApi.Resolver}} (RATIS-2603).
> ** SCM gives the read pipeline of a closed container a random ID. No Raft
> group has this ID. Thus, a read-only stream on this pipeline fails with a
> group mismatch.
> ** The resolver finds the container by its ID, not by the Raft group. It
> serves the read only when the container is closed and the request names this
> datanode.
> * A new {{DatanodeVersion}}, {{RATIS_DATASTREAM_READ_BLOCK_SUPPORT}}, tells
> clients which datanodes can serve this read.
> *Tests*
> * Unit tests examine the replies and the resolver.
> * An end-to-end test reads a block with a plain Ratis client.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
