chungen0126 commented on PR #10828: URL: https://github.com/apache/ozone/pull/10828#issuecomment-5053857711
> This solution creates a race condition, ... > > What is the race condition? > > ... Changes in this PR allows graceful fail back to non-stream writes. > > The change in this PR is very complicated. It does not worth to implement fallback for just changing the conf from disable streaming to enable streaming. Thanks @yandrey321 for working on this. I believe this fallback is necessary for now, for the following reasons: Imagine a scenario where the cluster inevitably contains a mix of DataNodes that support Ratis streaming and those that do not. This ultimately leads to two issues: 1. When creating a pipeline, SCM pipeline manager treats DataNodes regardless of whether they support Ratis streaming as identical, meaning we always run the risk of creating a pipeline that does not support Ratis streaming. 2. When selecting a pipeline, there is no awareness of whether the DataNodes within that pipeline support Ratis streaming. As a result, we might still end up selecting a non-streaming pipeline and failing during writes. 3. Even if we have enough DataNodes supporting Ratis streaming, the above two reasons will ultimately force us to fall back to non-streaming writes. My perspective: I view this as a short cup to keep writes functioning properly, while we still need to close and update the pipeline states for those whose DataNode port information is outdated. However, I think we should create a ticket to ensure that either pipeline creation or pipeline selection is aware of the write mode. This way, the client can accurately target pipelines that support Ratis streaming. @yandrey321 , @szetszwo , what are your thoughts on this? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
