timmylicheng commented on pull request #929: URL: https://github.com/apache/hadoop-ozone/pull/929#issuecomment-633343824
> Thanks @xiaoyuyao for your suggestion, i if put the limit check to RatisPipelineProvider, all the ratis pipeline creation will be limit, my key point is limit the pipeline creation from BackgroundPipelineCreator, and a bigger limit check for other pipeline creation, for example, the putkey rpc call request a unknown pipeline, like 4 replication, if it cannot find the unknown type and replication factor pipeline, it will create pipeline sync, without the limit for BackgroundPipelineCreator, there must be no quota for the unknown type or replication factor pipeline. @maobaolong You still have FACTOR to help you classify different behaviors. RatisPipelineProvider is the best place if you would complex logic for different Ratis pipelines. If you just want to skip creation, you can take a look at skipCreation function in BackgroundPipelineCreator. But I would you suggest to have a complete strategy to allocate different factor's pipelines. At the end of the day, your purpose is to support multiple Ratis pipeline with different factors. This would include preallocation of pipelines, resource split and potential pipeline eviction. It may be more than just stop creating pipelines to leave quota to other pipelines with different factors. It's hard to rely on configs to know the best setup of pipeline quota. And it's almost impossible to maintain to balance of pipeline quota if you consider cluster expansion or shrinkage. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
