[jira] [Commented] (HDDS-1517) AllocateBlock call fails with ContainerNotFoundException
[ https://issues.apache.org/jira/browse/HDDS-1517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16845825#comment-16845825 ] Hudson commented on HDDS-1517: -- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #16586 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/16586/]) HDDS-1517. AllocateBlock call fails with ContainerNotFoundException (shashikant: rev a315913c48f475a31065de48a441c7faae89ab15) * (edit) hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/container/TestSCMContainerManager.java * (edit) hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/SCMContainerManager.java * (edit) hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/ContainerStateManager.java * (edit) hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/block/TestBlockManager.java > AllocateBlock call fails with ContainerNotFoundException > > > Key: HDDS-1517 > URL: https://issues.apache.org/jira/browse/HDDS-1517 > Project: Hadoop Distributed Data Store > Issue Type: Bug > Components: SCM >Affects Versions: 0.5.0 >Reporter: Shashikant Banerjee >Assignee: Shashikant Banerjee >Priority: Major > Labels: pull-request-available > Fix For: 0.4.1 > > Attachments: HDDS-1517.000.patch > > Time Spent: 1h 40m > Remaining Estimate: 0h > > In allocateContainer call, the container is first added to pipelineStateMap > and then added to container cache. If two allocate blocks execute > concurrently, it might happen that one find the container to exist in the > pipelineStateMap but the container is yet to be updated in the container > cache, hence failing with CONTAINER_NOT_FOUND exception. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDDS-1517) AllocateBlock call fails with ContainerNotFoundException
[ https://issues.apache.org/jira/browse/HDDS-1517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16845808#comment-16845808 ] Shashikant Banerjee commented on HDDS-1517: --- Thanks [~jnp] for the review. I have committed this change to trunk. > AllocateBlock call fails with ContainerNotFoundException > > > Key: HDDS-1517 > URL: https://issues.apache.org/jira/browse/HDDS-1517 > Project: Hadoop Distributed Data Store > Issue Type: Bug > Components: SCM >Affects Versions: 0.5.0 >Reporter: Shashikant Banerjee >Assignee: Shashikant Banerjee >Priority: Major > Labels: pull-request-available > Fix For: 0.4.1 > > Attachments: HDDS-1517.000.patch > > Time Spent: 1h 40m > Remaining Estimate: 0h > > In allocateContainer call, the container is first added to pipelineStateMap > and then added to container cache. If two allocate blocks execute > concurrently, it might happen that one find the container to exist in the > pipelineStateMap but the container is yet to be updated in the container > cache, hence failing with CONTAINER_NOT_FOUND exception. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDDS-1517) AllocateBlock call fails with ContainerNotFoundException
[ https://issues.apache.org/jira/browse/HDDS-1517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16845578#comment-16845578 ] Jitendra Nath Pandey commented on HDDS-1517: +1 for the latest patch in the PR. > AllocateBlock call fails with ContainerNotFoundException > > > Key: HDDS-1517 > URL: https://issues.apache.org/jira/browse/HDDS-1517 > Project: Hadoop Distributed Data Store > Issue Type: Bug > Components: SCM >Affects Versions: 0.5.0 >Reporter: Shashikant Banerjee >Assignee: Shashikant Banerjee >Priority: Major > Labels: pull-request-available > Fix For: 0.4.1 > > Attachments: HDDS-1517.000.patch > > Time Spent: 1.5h > Remaining Estimate: 0h > > In allocateContainer call, the container is first added to pipelineStateMap > and then added to container cache. If two allocate blocks execute > concurrently, it might happen that one find the container to exist in the > pipelineStateMap but the container is yet to be updated in the container > cache, hence failing with CONTAINER_NOT_FOUND exception. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDDS-1517) AllocateBlock call fails with ContainerNotFoundException
[ https://issues.apache.org/jira/browse/HDDS-1517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16842325#comment-16842325 ] Shashikant Banerjee commented on HDDS-1517: --- Thanks [~jnp], as discussed i have updated the patch in the pull request. > AllocateBlock call fails with ContainerNotFoundException > > > Key: HDDS-1517 > URL: https://issues.apache.org/jira/browse/HDDS-1517 > Project: Hadoop Distributed Data Store > Issue Type: Bug > Components: SCM >Affects Versions: 0.5.0 >Reporter: Shashikant Banerjee >Assignee: Shashikant Banerjee >Priority: Major > Labels: pull-request-available > Fix For: 0.4.1 > > Attachments: HDDS-1517.000.patch > > Time Spent: 0.5h > Remaining Estimate: 0h > > In allocateContainer call, the container is first added to pipelineStateMap > and then added to container cache. If two allocate blocks execute > concurrently, it might happen that one find the container to exist in the > pipelineStateMap but the container is yet to be updated in the container > cache, hence failing with CONTAINER_NOT_FOUND exception. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDDS-1517) AllocateBlock call fails with ContainerNotFoundException
[ https://issues.apache.org/jira/browse/HDDS-1517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16841586#comment-16841586 ] Jitendra Nath Pandey commented on HDDS-1517: The patch moves addition of container to pipelineStateMap after its addition to container cache. Now a thread may first find the container in the cache but not in pipelineStateMap. How is the race condition addressed? Do we guarantee that a thread will never look in pipelineStateMap before it looks in container cache? > AllocateBlock call fails with ContainerNotFoundException > > > Key: HDDS-1517 > URL: https://issues.apache.org/jira/browse/HDDS-1517 > Project: Hadoop Distributed Data Store > Issue Type: Bug > Components: SCM >Affects Versions: 0.5.0 >Reporter: Shashikant Banerjee >Assignee: Shashikant Banerjee >Priority: Major > Labels: pull-request-available > Fix For: 0.4.1 > > Attachments: HDDS-1517.000.patch > > Time Spent: 0.5h > Remaining Estimate: 0h > > In allocateContainer call, the container is first added to pipelineStateMap > and then added to container cache. If two allocate blocks execute > concurrently, it might happen that one find the container to exist in the > pipelineStateMap but the container is yet to be updated in the container > cache, hence failing with CONTAINER_NOT_FOUND exception. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDDS-1517) AllocateBlock call fails with ContainerNotFoundException
[
https://issues.apache.org/jira/browse/HDDS-1517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16840058#comment-16840058
]
Hadoop QA commented on HDDS-1517:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
24s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} yetus {color} | {color:red} 0m 7s{color}
| {color:red} Unprocessed flag(s): --jenkins --skip-dir {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce base:
https://builds.apache.org/job/PreCommit-HDDS-Build/2689/artifact/out/Dockerfile
|
| JIRA Issue | HDDS-1517 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12968752/HDDS-1517.000.patch |
| Console output |
https://builds.apache.org/job/PreCommit-HDDS-Build/2689/console |
| versions | git=2.7.4 |
| Powered by | Apache Yetus 0.11.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> AllocateBlock call fails with ContainerNotFoundException
>
>
> Key: HDDS-1517
> URL: https://issues.apache.org/jira/browse/HDDS-1517
> Project: Hadoop Distributed Data Store
> Issue Type: Bug
> Components: SCM
>Affects Versions: 0.5.0
>Reporter: Shashikant Banerjee
>Assignee: Shashikant Banerjee
>Priority: Major
> Fix For: 0.5.0
>
> Attachments: HDDS-1517.000.patch
>
>
> In allocateContainer call, the container is first added to pipelineStateMap
> and then added to container cache. If two allocate blocks execute
> concurrently, it might happen that one find the container to exist in the
> pipelineStateMap but the container is yet to be updated in the container
> cache, hence failing with CONTAINER_NOT_FOUND exception.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDDS-1517) AllocateBlock call fails with ContainerNotFoundException
[ https://issues.apache.org/jira/browse/HDDS-1517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16840044#comment-16840044 ] Shashikant Banerjee commented on HDDS-1517: --- Patch v0 adds the fix. I will open up a pull request and add a new patch which also will add test to verify the fix. > AllocateBlock call fails with ContainerNotFoundException > > > Key: HDDS-1517 > URL: https://issues.apache.org/jira/browse/HDDS-1517 > Project: Hadoop Distributed Data Store > Issue Type: Bug > Components: SCM >Affects Versions: 0.5.0 >Reporter: Shashikant Banerjee >Assignee: Shashikant Banerjee >Priority: Major > Fix For: 0.5.0 > > Attachments: HDDS-1517.000.patch > > > In allocateContainer call, the container is first added to pipelineStateMap > and then added to container cache. If two allocate blocks execute > concurrently, it might happen that one find the container to exist in the > pipelineStateMap but the container is yet to be updated in the container > cache, hence failing with CONTAINER_NOT_FOUND exception. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDDS-1517) AllocateBlock call fails with ContainerNotFoundException
[ https://issues.apache.org/jira/browse/HDDS-1517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16837488#comment-16837488 ] Jitendra Nath Pandey commented on HDDS-1517: Is it related to HDDS-1374? > AllocateBlock call fails with ContainerNotFoundException > > > Key: HDDS-1517 > URL: https://issues.apache.org/jira/browse/HDDS-1517 > Project: Hadoop Distributed Data Store > Issue Type: Bug > Components: SCM >Affects Versions: 0.5.0 >Reporter: Shashikant Banerjee >Assignee: Shashikant Banerjee >Priority: Major > Fix For: 0.5.0 > > > In allocateContainer call, the container is first added to pipelineStateMap > and then added to container cache. If two allocate blocks execute > concurrently, it might happen that one find the container to exist in the > pipelineStateMap but the container is yet to be updated in the container > cache, hence failing with CONTAINER_NOT_FOUND exception. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
