sumitagrawl commented on PR #5812: URL: https://github.com/apache/ozone/pull/5812#issuecomment-1875651049
> There is no impact and I'm not saying that we shouldn't proceed with the changes. But there could be better solutions that also keep consistency. For example, we can trigger a pipeline sync before adding new containers with batch. @xBis7 Related to above points, SCM do not have this problem as pipeline is created before container creation. Pipeline is added to recon in below 2 cases, both confirm with SCM before adding pipeline to Recon: 1. Pipeline sync task triggered at 5 min to ensure scm and recon pipelines are sync 2. ReconPipelineReportHandler: DN trigger Pipeline Report every 1 min, and pipeline is added to recon using this But container add is triggered through ICR by DN for every container to Recon. For Ratis, pipeline is created and marked open in Pipeline Report from DN, so Recon do not face issue for ratis as SCM also wait for same event. For EC, problem is observed as SCM directly mark as OPEN and no way to sync this to recon. Now, ICR for container add can be more frequent and for every ICR, triggering pipeline task sync will not be appropriate. This will keep buffering the ICR at Recon for large cluster. Now for container add, it always sync with SCM to get container with pipeline, and to avoid buffering, its optimised to add the pipeline details obtained with container itself. This is safe and consistent as pipeline is obtained from SCM itself. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
