[
https://issues.apache.org/jira/browse/YARN-3854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16645555#comment-16645555
]
Chandni Singh commented on YARN-3854:
-------------------------------------
{quote}It would be nice to have an "untouchable" list though. Basically images
that won't get deleted even if they would qualify to be deleted using LRU.
{quote}
+1
{quote}It would also be nice to be able to specify images to delete cluster
wide with some sort of API. Just put the image into the LRU list regardless of
where it would have been.
{quote}
I agree this is useful, however to me this seems like out of scope for this
effort. A way to do this is to add an API to RM which will then ask all the NMs
to put the images in the LRU. This will require changes on the RM side and the
API layer between RM and NMs.
IMO it would be a good enhancement later.
{quote}I'm worried that the image deletion is going to be quite rough and
error-prone. Docker containers sometimes stick around in the Exited or Dead
states or keep running beyond when they should. That will cause the delete to
perpetually fail since the image is still in use, even though it isn't really
in use, at least not in a useful sense. Do we just keep going around and around
and trying to delete the same images even if we keep failing?
{quote}
I can think of few things to make it a little better when an image is deleted
# Find out all the existing containers for the image
# If there is a way to identify Exited or Dead State of the container, and all
the containers are in this state then forcefully delete the image.
# If the above is not possible or there are containers that keep running, we
retry deleting configured number of times. Once this configured number of times
fails, throw an alert.
{quote}This isn't true on the whole. {{docker run}} only does a {{docker pull}}
if the image doesn't exist locally.
{quote}
That's right. I will make it explicit in the doc that when the image doesn't
exist locally, then {{docker pull}} is done which takes longer.
> Add localization support for docker images
> ------------------------------------------
>
> Key: YARN-3854
> URL: https://issues.apache.org/jira/browse/YARN-3854
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: yarn
> Reporter: Sidharta Seethana
> Assignee: Chandni Singh
> Priority: Major
> Labels: Docker
> Attachments: Localization Support For Docker Images.pdf,
> YARN-3854-branch-2.8.001.patch,
> YARN-3854_Localization_support_for_Docker_image_v1.pdf,
> YARN-3854_Localization_support_for_Docker_image_v2.pdf,
> YARN-3854_Localization_support_for_Docker_image_v3.pdf
>
>
> We need the ability to localize docker images when those images aren't
> already available locally. There are various approaches that could be used
> here with different trade-offs/issues : image archives on HDFS + docker load
> , docker pull during the localization phase or (automatic) docker pull
> during the run/launch phase.
> We also need the ability to clean-up old/stale, unused images.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
