bharatviswa504 commented on a change in pull request #2569:
URL: https://github.com/apache/ozone/pull/2569#discussion_r695603286
##########
File path:
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/ContainerManagerImpl.java
##########
@@ -176,33 +176,64 @@ public ContainerInfo getContainer(final ContainerID id)
public ContainerInfo allocateContainer(
final ReplicationConfig replicationConfig, final String owner)
throws IOException {
+ // Acquire pipeline manager lock, to avoid any updates to pipeline
+ // while allocate container happens. This is to avoid scenario like
+ // mentioned in HDDS-5655.
+ pipelineManager.acquireReadLock();
lock.lock();
+ List<Pipeline> pipelines;
+ Pipeline pipeline;
+ ContainerInfo containerInfo = null;
try {
- final List<Pipeline> pipelines = pipelineManager
+ pipelines = pipelineManager
.getPipelines(replicationConfig, Pipeline.PipelineState.OPEN);
+ if (!pipelines.isEmpty()) {
+ pipeline = pipelines.get(random.nextInt(pipelines.size()));
+ containerInfo = createContainer(pipeline, owner);
+ }
+ } finally {
+ lock.unlock();
+ pipelineManager.releaseReadLock();
+ }
- final Pipeline pipeline;
- if (pipelines.isEmpty()) {
- try {
- pipeline = pipelineManager.createPipeline(replicationConfig);
- pipelineManager.waitPipelineReady(pipeline.getId(), 0);
- } catch (IOException e) {
- scmContainerManagerMetrics.incNumFailureCreateContainers();
+ if (pipelines.isEmpty()) {
+ try {
+ pipeline = pipelineManager.createPipeline(replicationConfig);
+ pipelineManager.waitPipelineReady(pipeline.getId(), 0);
+ } catch (IOException e) {
+ scmContainerManagerMetrics.incNumFailureCreateContainers();
+ throw new IOException("Could not allocate container. Cannot get any" +
+ " matching pipeline for replicationConfig: " + replicationConfig
+ + ", State:PipelineState.OPEN", e);
+ }
+ pipelineManager.acquireReadLock();
+ lock.lock();
+ try {
+ pipelines = pipelineManager
+ .getPipelines(replicationConfig, Pipeline.PipelineState.OPEN);
+ if (!pipelines.isEmpty()) {
+ pipeline = pipelines.get(random.nextInt(pipelines.size()));
+ containerInfo = createContainer(pipeline, owner);
+ } else {
throw new IOException("Could not allocate container. Cannot get any"
+
" matching pipeline for replicationConfig: " + replicationConfig
- + ", State:PipelineState.OPEN", e);
+ + ", State:PipelineState.OPEN");
}
- } else {
- pipeline = pipelines.get(random.nextInt(pipelines.size()));
- }
- final ContainerInfo containerInfo = allocateContainer(pipeline, owner);
- if (LOG.isTraceEnabled()) {
- LOG.trace("New container allocated: {}", containerInfo);
+ } finally {
+ lock.unlock();
+ pipelineManager.releaseReadLock();
Review comment:
We cannot do that, that is the reason for releasing lock and reacquire.
As if pipeline list is empty, we create pipeline and wait for ready. Here if
we acquire pipeline lock, When DN reports the pipeline report handler needs
pipelinemanager lock to update pipelinestate.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
