[
https://issues.apache.org/jira/browse/HDDS-14223?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ivan Andika updated HDDS-14223:
-------------------------------
Description:
Streaming Write Pipeline Approach 2 was written in the initial Ozone Streaming
Write Pipeline design doc ([^Ozone Write Streaming.pdf]) to improve the
performance compared to phase 1.
The idea is that instead of sending PutBlock through the AsyncApi#send (which
will send to the leader instead of the primary), we will send PutBlock
(metadata) as part of the streaming (i.e. DataStreamOutput#writeAsync), shared
with WriteChunk (data) writeAsync calls. We can even force a SYNC as part of
the PutBlock (i.e. PutBlock ensures that the data have been persistently stored
in the underlying storage) instead of having a separate configuration for
syncing.
We have two patches HDDS-6500 and HDDS-6137 that added support for sending
PutBlock during stream close. However, there are some issues in the
implementation as mentioned in HDDS-12007.
Furthermore, we are still mixing Approach 1 (Separate Metadata and Data) which
is called during flush boundary + close with Approach 2 which is called only
during close. In my opinion, we should stick with one approach (either Approach
1 or Approach 2, but not both).
There are a few things to revisit
* We might need to introduce a new Ratis internal mechanism to handle a write
"commit" (e.g. PutBlock will write to a separate buffer (not directly to the
StateMachine.DataChannel) to be processed separately by another
StateMachine.DataStream API like commitStream), or we can reuse
StandardWriteOption.SYNC as a wait to commit.
* We need to see whether we actually need to use Raft at all or can Ratis
streaming be used to implement a general write pipeline (i.e. Chain
replication, CRAQ, HDFS write pipeline), etc
* Currently our client implementation (BlockDataStreamOutput,
BlockDataStreamOutputEntry, BlockDataStreamOutputEntryPool, etc) are based on
the Write Pipeline V1 since it was easier to port last time. We need to see
whether we need to rework the entire client output stream implementation to
achieve better performance.
was:
Streaming Write Pipeline Approach 2 was written in the initial Ozone Streaming
Write Pipeline design doc ([^Ozone Write Streaming.pdf]) to improve the
performance compared to phase 1.
The idea is that instead of sending PutBlock through the AsyncApi#send (which
will send to the leader instead of the primary), we will send PutBlock
(metadata) as part of the streaming (i.e. DataStreamOutput#writeAsync), shared
with WriteChunk (data) writeAsync calls. We can even force a SYNC as part of
the PutBlock (i.e. PutBlock ensures that the data have been persistently stored
in the underlying storage) instead of having a separate configuration for
syncing.
We have two patches HDDS-6500 and HDDS-6137 that added support for sending
PutBlock during stream close. However, there are some issues in the
implementation as mentioned in HDDS-12007.
Furthermore, we are still mixing Approach 1 (Separate Metadata and Data) which
is called during flush boundary + close with Approach 2 which is called only
during close. In my opinion, we should stick with one approach (either Approach
1 or Approach 2, but not both).
There are a few things to revisit
* We might need to introduce a new Ratis internal mechanism to handle a write
"commit" (e.g. PutBlock will write be written to a separate buffer (not
directly to the StateMachine.DataChannel) to be processed separately by another
StateMachine.DataStream API like commitStream), or we can reuse
StandardWriteOption.SYNC as a wait to commit.
* We need to see whether we actually need to use Raft at all or can Ratis
streaming be used to implement a general write pipeline (i.e. Chain
replication, CRAQ, HDFS write pipeline), etc
* Currently our client implementation (BlockDataStreamOutput,
BlockDataStreamOutputEntry, BlockDataStreamOutputEntryPool, etc) are based on
the Write Pipeline V1 since it was easier to port last time. We need to see
whether we need to rework the entire client output stream implementation to
achieve better performance.
> Revisit Approach 2: Streaming Path for both Data and MetaData
> --------------------------------------------------------------
>
> Key: HDDS-14223
> URL: https://issues.apache.org/jira/browse/HDDS-14223
> Project: Apache Ozone
> Issue Type: Sub-task
> Reporter: Ivan Andika
> Assignee: Ivan Andika
> Priority: Major
>
> Streaming Write Pipeline Approach 2 was written in the initial Ozone
> Streaming Write Pipeline design doc ([^Ozone Write Streaming.pdf]) to improve
> the performance compared to phase 1.
> The idea is that instead of sending PutBlock through the AsyncApi#send (which
> will send to the leader instead of the primary), we will send PutBlock
> (metadata) as part of the streaming (i.e. DataStreamOutput#writeAsync),
> shared with WriteChunk (data) writeAsync calls. We can even force a SYNC as
> part of the PutBlock (i.e. PutBlock ensures that the data have been
> persistently stored in the underlying storage) instead of having a separate
> configuration for syncing.
> We have two patches HDDS-6500 and HDDS-6137 that added support for sending
> PutBlock during stream close. However, there are some issues in the
> implementation as mentioned in HDDS-12007.
> Furthermore, we are still mixing Approach 1 (Separate Metadata and Data)
> which is called during flush boundary + close with Approach 2 which is called
> only during close. In my opinion, we should stick with one approach (either
> Approach 1 or Approach 2, but not both).
> There are a few things to revisit
> * We might need to introduce a new Ratis internal mechanism to handle a
> write "commit" (e.g. PutBlock will write to a separate buffer (not directly
> to the StateMachine.DataChannel) to be processed separately by another
> StateMachine.DataStream API like commitStream), or we can reuse
> StandardWriteOption.SYNC as a wait to commit.
> * We need to see whether we actually need to use Raft at all or can Ratis
> streaming be used to implement a general write pipeline (i.e. Chain
> replication, CRAQ, HDFS write pipeline), etc
> * Currently our client implementation (BlockDataStreamOutput,
> BlockDataStreamOutputEntry, BlockDataStreamOutputEntryPool, etc) are based on
> the Write Pipeline V1 since it was easier to port last time. We need to see
> whether we need to rework the entire client output stream implementation to
> achieve better performance.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
