[jira] [Updated] (YARN-9292) Implement logic to keep docker image consistent in application that uses :latest tag
[ https://issues.apache.org/jira/browse/YARN-9292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Yang updated YARN-9292: Attachment: YARN-9292.008.patch > Implement logic to keep docker image consistent in application that uses > :latest tag > > > Key: YARN-9292 > URL: https://issues.apache.org/jira/browse/YARN-9292 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Eric Yang >Assignee: Eric Yang >Priority: Major > Attachments: YARN-9292.001.patch, YARN-9292.002.patch, > YARN-9292.003.patch, YARN-9292.004.patch, YARN-9292.005.patch, > YARN-9292.006.patch, YARN-9292.007.patch, YARN-9292.008.patch > > > Docker image with latest tag can run in YARN cluster without any validation > in node managers. If a image with latest tag is changed during containers > launch. It might produce inconsistent results between nodes. This is surfaced > toward end of development for YARN-9184 to keep docker image consistent > within a job. One of the ideas to keep :latest tag consistent for a job, is > to use docker image command to figure out the image id and use image id to > propagate to rest of the container requests. There are some challenges to > overcome: > # The latest tag does not exist on the node where first container starts. > The first container will need to download the latest image, and find image > ID. This can introduce lag time for other containers to start. > # If image id is used to start other container, container-executor may have > problems to check if the image is coming from a trusted source. Both image > name and ID must be supply through .cmd file to container-executor. However, > hacker can supply incorrect image id and defeat container-executor security > checks. > If we can over come those challenges, it maybe possible to keep docker image > consistent with one application. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9292) Implement logic to keep docker image consistent in application that uses :latest tag
[ https://issues.apache.org/jira/browse/YARN-9292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Yang updated YARN-9292: Attachment: YARN-9292.007.patch > Implement logic to keep docker image consistent in application that uses > :latest tag > > > Key: YARN-9292 > URL: https://issues.apache.org/jira/browse/YARN-9292 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Eric Yang >Assignee: Eric Yang >Priority: Major > Attachments: YARN-9292.001.patch, YARN-9292.002.patch, > YARN-9292.003.patch, YARN-9292.004.patch, YARN-9292.005.patch, > YARN-9292.006.patch, YARN-9292.007.patch > > > Docker image with latest tag can run in YARN cluster without any validation > in node managers. If a image with latest tag is changed during containers > launch. It might produce inconsistent results between nodes. This is surfaced > toward end of development for YARN-9184 to keep docker image consistent > within a job. One of the ideas to keep :latest tag consistent for a job, is > to use docker image command to figure out the image id and use image id to > propagate to rest of the container requests. There are some challenges to > overcome: > # The latest tag does not exist on the node where first container starts. > The first container will need to download the latest image, and find image > ID. This can introduce lag time for other containers to start. > # If image id is used to start other container, container-executor may have > problems to check if the image is coming from a trusted source. Both image > name and ID must be supply through .cmd file to container-executor. However, > hacker can supply incorrect image id and defeat container-executor security > checks. > If we can over come those challenges, it maybe possible to keep docker image > consistent with one application. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9292) Implement logic to keep docker image consistent in application that uses :latest tag
[ https://issues.apache.org/jira/browse/YARN-9292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Yang updated YARN-9292: Target Version/s: 3.3.0 > Implement logic to keep docker image consistent in application that uses > :latest tag > > > Key: YARN-9292 > URL: https://issues.apache.org/jira/browse/YARN-9292 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Eric Yang >Assignee: Eric Yang >Priority: Major > Attachments: YARN-9292.001.patch, YARN-9292.002.patch, > YARN-9292.003.patch, YARN-9292.004.patch, YARN-9292.005.patch, > YARN-9292.006.patch > > > Docker image with latest tag can run in YARN cluster without any validation > in node managers. If a image with latest tag is changed during containers > launch. It might produce inconsistent results between nodes. This is surfaced > toward end of development for YARN-9184 to keep docker image consistent > within a job. One of the ideas to keep :latest tag consistent for a job, is > to use docker image command to figure out the image id and use image id to > propagate to rest of the container requests. There are some challenges to > overcome: > # The latest tag does not exist on the node where first container starts. > The first container will need to download the latest image, and find image > ID. This can introduce lag time for other containers to start. > # If image id is used to start other container, container-executor may have > problems to check if the image is coming from a trusted source. Both image > name and ID must be supply through .cmd file to container-executor. However, > hacker can supply incorrect image id and defeat container-executor security > checks. > If we can over come those challenges, it maybe possible to keep docker image > consistent with one application. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9292) Implement logic to keep docker image consistent in application that uses :latest tag
[ https://issues.apache.org/jira/browse/YARN-9292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Yang updated YARN-9292: Attachment: YARN-9292.006.patch > Implement logic to keep docker image consistent in application that uses > :latest tag > > > Key: YARN-9292 > URL: https://issues.apache.org/jira/browse/YARN-9292 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Eric Yang >Assignee: Eric Yang >Priority: Major > Attachments: YARN-9292.001.patch, YARN-9292.002.patch, > YARN-9292.003.patch, YARN-9292.004.patch, YARN-9292.005.patch, > YARN-9292.006.patch > > > Docker image with latest tag can run in YARN cluster without any validation > in node managers. If a image with latest tag is changed during containers > launch. It might produce inconsistent results between nodes. This is surfaced > toward end of development for YARN-9184 to keep docker image consistent > within a job. One of the ideas to keep :latest tag consistent for a job, is > to use docker image command to figure out the image id and use image id to > propagate to rest of the container requests. There are some challenges to > overcome: > # The latest tag does not exist on the node where first container starts. > The first container will need to download the latest image, and find image > ID. This can introduce lag time for other containers to start. > # If image id is used to start other container, container-executor may have > problems to check if the image is coming from a trusted source. Both image > name and ID must be supply through .cmd file to container-executor. However, > hacker can supply incorrect image id and defeat container-executor security > checks. > If we can over come those challenges, it maybe possible to keep docker image > consistent with one application. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9292) Implement logic to keep docker image consistent in application that uses :latest tag
[ https://issues.apache.org/jira/browse/YARN-9292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Yang updated YARN-9292: Attachment: YARN-9292.005.patch > Implement logic to keep docker image consistent in application that uses > :latest tag > > > Key: YARN-9292 > URL: https://issues.apache.org/jira/browse/YARN-9292 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Eric Yang >Assignee: Eric Yang >Priority: Major > Attachments: YARN-9292.001.patch, YARN-9292.002.patch, > YARN-9292.003.patch, YARN-9292.004.patch, YARN-9292.005.patch > > > Docker image with latest tag can run in YARN cluster without any validation > in node managers. If a image with latest tag is changed during containers > launch. It might produce inconsistent results between nodes. This is surfaced > toward end of development for YARN-9184 to keep docker image consistent > within a job. One of the ideas to keep :latest tag consistent for a job, is > to use docker image command to figure out the image id and use image id to > propagate to rest of the container requests. There are some challenges to > overcome: > # The latest tag does not exist on the node where first container starts. > The first container will need to download the latest image, and find image > ID. This can introduce lag time for other containers to start. > # If image id is used to start other container, container-executor may have > problems to check if the image is coming from a trusted source. Both image > name and ID must be supply through .cmd file to container-executor. However, > hacker can supply incorrect image id and defeat container-executor security > checks. > If we can over come those challenges, it maybe possible to keep docker image > consistent with one application. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9292) Implement logic to keep docker image consistent in application that uses :latest tag
[ https://issues.apache.org/jira/browse/YARN-9292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Yang updated YARN-9292: Attachment: YARN-9292.004.patch > Implement logic to keep docker image consistent in application that uses > :latest tag > > > Key: YARN-9292 > URL: https://issues.apache.org/jira/browse/YARN-9292 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Eric Yang >Assignee: Eric Yang >Priority: Major > Attachments: YARN-9292.001.patch, YARN-9292.002.patch, > YARN-9292.003.patch, YARN-9292.004.patch > > > Docker image with latest tag can run in YARN cluster without any validation > in node managers. If a image with latest tag is changed during containers > launch. It might produce inconsistent results between nodes. This is surfaced > toward end of development for YARN-9184 to keep docker image consistent > within a job. One of the ideas to keep :latest tag consistent for a job, is > to use docker image command to figure out the image id and use image id to > propagate to rest of the container requests. There are some challenges to > overcome: > # The latest tag does not exist on the node where first container starts. > The first container will need to download the latest image, and find image > ID. This can introduce lag time for other containers to start. > # If image id is used to start other container, container-executor may have > problems to check if the image is coming from a trusted source. Both image > name and ID must be supply through .cmd file to container-executor. However, > hacker can supply incorrect image id and defeat container-executor security > checks. > If we can over come those challenges, it maybe possible to keep docker image > consistent with one application. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9292) Implement logic to keep docker image consistent in application that uses :latest tag
[ https://issues.apache.org/jira/browse/YARN-9292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Yang updated YARN-9292: Attachment: YARN-9292.003.patch > Implement logic to keep docker image consistent in application that uses > :latest tag > > > Key: YARN-9292 > URL: https://issues.apache.org/jira/browse/YARN-9292 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Eric Yang >Assignee: Eric Yang >Priority: Major > Attachments: YARN-9292.001.patch, YARN-9292.002.patch, > YARN-9292.003.patch > > > Docker image with latest tag can run in YARN cluster without any validation > in node managers. If a image with latest tag is changed during containers > launch. It might produce inconsistent results between nodes. This is surfaced > toward end of development for YARN-9184 to keep docker image consistent > within a job. One of the ideas to keep :latest tag consistent for a job, is > to use docker image command to figure out the image id and use image id to > propagate to rest of the container requests. There are some challenges to > overcome: > # The latest tag does not exist on the node where first container starts. > The first container will need to download the latest image, and find image > ID. This can introduce lag time for other containers to start. > # If image id is used to start other container, container-executor may have > problems to check if the image is coming from a trusted source. Both image > name and ID must be supply through .cmd file to container-executor. However, > hacker can supply incorrect image id and defeat container-executor security > checks. > If we can over come those challenges, it maybe possible to keep docker image > consistent with one application. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9292) Implement logic to keep docker image consistent in application that uses :latest tag
[ https://issues.apache.org/jira/browse/YARN-9292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Yang updated YARN-9292: Attachment: YARN-9292.002.patch > Implement logic to keep docker image consistent in application that uses > :latest tag > > > Key: YARN-9292 > URL: https://issues.apache.org/jira/browse/YARN-9292 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Eric Yang >Assignee: Eric Yang >Priority: Major > Attachments: YARN-9292.001.patch, YARN-9292.002.patch > > > Docker image with latest tag can run in YARN cluster without any validation > in node managers. If a image with latest tag is changed during containers > launch. It might produce inconsistent results between nodes. This is surfaced > toward end of development for YARN-9184 to keep docker image consistent > within a job. One of the ideas to keep :latest tag consistent for a job, is > to use docker image command to figure out the image id and use image id to > propagate to rest of the container requests. There are some challenges to > overcome: > # The latest tag does not exist on the node where first container starts. > The first container will need to download the latest image, and find image > ID. This can introduce lag time for other containers to start. > # If image id is used to start other container, container-executor may have > problems to check if the image is coming from a trusted source. Both image > name and ID must be supply through .cmd file to container-executor. However, > hacker can supply incorrect image id and defeat container-executor security > checks. > If we can over come those challenges, it maybe possible to keep docker image > consistent with one application. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9292) Implement logic to keep docker image consistent in application that uses :latest tag
[ https://issues.apache.org/jira/browse/YARN-9292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Yang updated YARN-9292: Attachment: YARN-9292.001.patch > Implement logic to keep docker image consistent in application that uses > :latest tag > > > Key: YARN-9292 > URL: https://issues.apache.org/jira/browse/YARN-9292 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Eric Yang >Assignee: Eric Yang >Priority: Major > Attachments: YARN-9292.001.patch > > > Docker image with latest tag can run in YARN cluster without any validation > in node managers. If a image with latest tag is changed during containers > launch. It might produce inconsistent results between nodes. This is surfaced > toward end of development for YARN-9184 to keep docker image consistent > within a job. One of the ideas to keep :latest tag consistent for a job, is > to use docker image command to figure out the image id and use image id to > propagate to rest of the container requests. There are some challenges to > overcome: > # The latest tag does not exist on the node where first container starts. > The first container will need to download the latest image, and find image > ID. This can introduce lag time for other containers to start. > # If image id is used to start other container, container-executor may have > problems to check if the image is coming from a trusted source. Both image > name and ID must be supply through .cmd file to container-executor. However, > hacker can supply incorrect image id and defeat container-executor security > checks. > If we can over come those challenges, it maybe possible to keep docker image > consistent with one application. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
