Re: [openstack-dev] [glance] Option to skip deleting images in use?
I think that's horrible idea. How do we do that store independent with the linking dependencies? We should not depend universal use case like this on limited subset of backends, specially non-OpenStack ones. Glance (nor Nova) should never depend having direct access to the actual medium where the images are stored. I think this is school book example for something called database. Well arguable if this should be tracked at Glance or Nova, but definitely not a dirty hack expecting specific backend characteristics. As mentioned before the protected image property is to ensure that the image does not get deleted, that is also easy to track when the images are queried. Perhaps the record needs to track the original state of protected flag, image id and use count. 3 column table and couple of API calls. Lets not at least make it any more complicated than it needs to be if such functionality is desired. - Erno From: Nikhil Komawar [mailto:nikhil.koma...@rackspace.com] Sent: 17 December 2014 20:34 To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [glance] Option to skip deleting images in use? Guess that's a implementation detail. Depends on the way you go about using what's available now, I suppose. Thanks, -Nikhil From: Chris St. Pierre [chris.a.st.pie...@gmail.com] Sent: Wednesday, December 17, 2014 2:07 PM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [glance] Option to skip deleting images in use? I was assuming atomic increment/decrement operations, in which case I'm not sure I see the race conditions. Or is atomism assuming too much? On Wed, Dec 17, 2014 at 11:59 AM, Nikhil Komawar nikhil.koma...@rackspace.commailto:nikhil.koma...@rackspace.com wrote: That looks like a decent alternative if it works. However, it would be too racy unless we we implement a test-and-set for such properties or there is a different job which queues up these requests and perform sequentially for each tenant. Thanks, -Nikhil From: Chris St. Pierre [chris.a.st.pie...@gmail.commailto:chris.a.st.pie...@gmail.com] Sent: Wednesday, December 17, 2014 10:23 AM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [glance] Option to skip deleting images in use? That's unfortunately too simple. You run into one of two cases: 1. If the job automatically removes the protected attribute when an image is no longer in use, then you lose the ability to use protected on images that are not in use. I.e., there's no way to say, nothing is currently using this image, but please keep it around. (This seems particularly useful for snapshots, for instance.) 2. If the job does not automatically remove the protected attribute, then an image would be protected if it had ever been in use; to delete an image, you'd have to manually un-protect it, which is a workflow that quite explicitly defeats the whole purpose of flagging images as protected when they're in use. It seems like flagging an image as *not* in use is actually a fairly difficult problem, since it requires consensus among all components that might be using images. The only solution that readily occurs to me would be to add something like a filesystem link count to images in Glance. Then when Nova spawns an instance, it increments the usage count; when the instance is destroyed, the usage count is decremented. And similarly with other components that use images. An image could only be deleted when its usage count was zero. There are ample opportunities to get out of sync there, but it's at least a sketch of something that might work, and isn't *too* horribly hackish. Thoughts? On Tue, Dec 16, 2014 at 6:11 PM, Vishvananda Ishaya vishvana...@gmail.commailto:vishvana...@gmail.com wrote: A simple solution that wouldn't require modification of glance would be a cron job that lists images and snapshots and marks them protected while they are in use. Vish On Dec 16, 2014, at 3:19 PM, Collins, Sean sean_colli...@cable.comcast.commailto:sean_colli...@cable.comcast.com wrote: On Tue, Dec 16, 2014 at 05:12:31PM EST, Chris St. Pierre wrote: No, I'm looking to prevent images that are in use from being deleted. In use and protected are disjoint sets. I have seen multiple cases of images (and snapshots) being deleted while still in use in Nova, which leads to some very, shall we say, interesting bugs and support problems. I do think that we should try and determine a way forward on this, they are indeed disjoint sets. Setting an image as protected is a proactive measure, we should try and figure out a way to keep tenants from shooting themselves in the foot if possible. -- Sean M. Collins ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.orgmailto:OpenStack-dev@lists.openstack.org
Re: [openstack-dev] [glance] Option to skip deleting images in use?
I wasn't suggesting that we *actually* use filesystem link count, and make hard links or whatever for every time the image is used. That's prima facie absurd, for a lot more reasons that you point out. I was suggesting a new database field that tracks the number of times an image is in use, by *analogy* with filesystem link counts. (If I wanted to be unnecessarily abrasive I might say, This is a textbook example of something called an analogy, but I'm not interested in being unnecessarily abrasive.) Overloading the protected flag is *still* a terrible hack. Even if we tracked the initial state of protected and restored that state when an image went out of use, that would negate the ability to make an image protected while it was in use and expect that change to remain in place. So that just violates the principle of least surprise. Of course, we could have glance modify the original_protected_state flag when that flag is non-null and the user changes the actual protected flag, but this is just layering hacks upon hacks. By actually tracking the number of times an image is in use, we can preserve the ability to protect images *and* avoid deleting images in use. On Thu, Dec 18, 2014 at 5:27 AM, Kuvaja, Erno kuv...@hp.com wrote: I think that’s horrible idea. How do we do that store independent with the linking dependencies? We should not depend universal use case like this on limited subset of backends, specially non-OpenStack ones. Glance (nor Nova) should never depend having direct access to the actual medium where the images are stored. I think this is school book example for something called database. Well arguable if this should be tracked at Glance or Nova, but definitely not a dirty hack expecting specific backend characteristics. As mentioned before the protected image property is to ensure that the image does not get deleted, that is also easy to track when the images are queried. Perhaps the record needs to track the original state of protected flag, image id and use count. 3 column table and couple of API calls. Lets not at least make it any more complicated than it needs to be if such functionality is desired. - Erno *From:* Nikhil Komawar [mailto:nikhil.koma...@rackspace.com] *Sent:* 17 December 2014 20:34 *To:* OpenStack Development Mailing List (not for usage questions) *Subject:* Re: [openstack-dev] [glance] Option to skip deleting images in use? Guess that's a implementation detail. Depends on the way you go about using what's available now, I suppose. Thanks, -Nikhil -- *From:* Chris St. Pierre [chris.a.st.pie...@gmail.com] *Sent:* Wednesday, December 17, 2014 2:07 PM *To:* OpenStack Development Mailing List (not for usage questions) *Subject:* Re: [openstack-dev] [glance] Option to skip deleting images in use? I was assuming atomic increment/decrement operations, in which case I'm not sure I see the race conditions. Or is atomism assuming too much? On Wed, Dec 17, 2014 at 11:59 AM, Nikhil Komawar nikhil.koma...@rackspace.com wrote: That looks like a decent alternative if it works. However, it would be too racy unless we we implement a test-and-set for such properties or there is a different job which queues up these requests and perform sequentially for each tenant. Thanks, -Nikhil -- *From:* Chris St. Pierre [chris.a.st.pie...@gmail.com] *Sent:* Wednesday, December 17, 2014 10:23 AM *To:* OpenStack Development Mailing List (not for usage questions) *Subject:* Re: [openstack-dev] [glance] Option to skip deleting images in use? That's unfortunately too simple. You run into one of two cases: 1. If the job automatically removes the protected attribute when an image is no longer in use, then you lose the ability to use protected on images that are not in use. I.e., there's no way to say, nothing is currently using this image, but please keep it around. (This seems particularly useful for snapshots, for instance.) 2. If the job does not automatically remove the protected attribute, then an image would be protected if it had ever been in use; to delete an image, you'd have to manually un-protect it, which is a workflow that quite explicitly defeats the whole purpose of flagging images as protected when they're in use. It seems like flagging an image as *not* in use is actually a fairly difficult problem, since it requires consensus among all components that might be using images. The only solution that readily occurs to me would be to add something like a filesystem link count to images in Glance. Then when Nova spawns an instance, it increments the usage count; when the instance is destroyed, the usage count is decremented. And similarly with other components that use images. An image could only be deleted when its usage count was zero. There are ample opportunities to get out of sync there, but it's at least
Re: [openstack-dev] [glance] Option to skip deleting images in use?
On 12/18/2014 02:08 PM, Chris St. Pierre wrote: I wasn't suggesting that we *actually* use filesystem link count, and make hard links or whatever for every time the image is used. That's prima facie absurd, for a lot more reasons that you point out. I was suggesting a new database field that tracks the number of times an image is in use, by *analogy* with filesystem link counts. (If I wanted to be unnecessarily abrasive I might say, This is a textbook example of something called an analogy, but I'm not interested in being unnecessarily abrasive.) Overloading the protected flag is *still* a terrible hack. Even if we tracked the initial state of protected and restored that state when an image went out of use, that would negate the ability to make an image I guess I don't understand what you consider to be overloading of the protected flag. The original purpose of the protected flag was to protect images from being deleted. Best, -jay protected while it was in use and expect that change to remain in place. So that just violates the principle of least surprise. Of course, we could have glance modify the original_protected_state flag when that flag is non-null and the user changes the actual protected flag, but this is just layering hacks upon hacks. By actually tracking the number of times an image is in use, we can preserve the ability to protect images *and* avoid deleting images in use. On Thu, Dec 18, 2014 at 5:27 AM, Kuvaja, Erno kuv...@hp.com mailto:kuv...@hp.com wrote: I think that’s horrible idea. How do we do that store independent with the linking dependencies? __ __ We should not depend universal use case like this on limited subset of backends, specially non-OpenStack ones. Glance (nor Nova) should never depend having direct access to the actual medium where the images are stored. I think this is school book example for something called database. Well arguable if this should be tracked at Glance or Nova, but definitely not a dirty hack expecting specific backend characteristics. __ __ As mentioned before the protected image property is to ensure that the image does not get deleted, that is also easy to track when the images are queried. Perhaps the record needs to track the original state of protected flag, image id and use count. 3 column table and couple of API calls. Lets not at least make it any more complicated than it needs to be if such functionality is desired. __ __ __-__Erno __ __ *From:*Nikhil Komawar [mailto:nikhil.koma...@rackspace.com mailto:nikhil.koma...@rackspace.com] *Sent:* 17 December 2014 20:34 *To:* OpenStack Development Mailing List (not for usage questions) *Subject:* Re: [openstack-dev] [glance] Option to skip deleting images in use? __ __ Guess that's a implementation detail. Depends on the way you go about using what's available now, I suppose. __ __ Thanks, -Nikhil *From:*Chris St. Pierre [chris.a.st.pie...@gmail.com mailto:chris.a.st.pie...@gmail.com] *Sent:* Wednesday, December 17, 2014 2:07 PM *To:* OpenStack Development Mailing List (not for usage questions) *Subject:* Re: [openstack-dev] [glance] Option to skip deleting images in use? I was assuming atomic increment/decrement operations, in which case I'm not sure I see the race conditions. Or is atomism assuming too much? __ __ On Wed, Dec 17, 2014 at 11:59 AM, Nikhil Komawar nikhil.koma...@rackspace.com mailto:nikhil.koma...@rackspace.com wrote: That looks like a decent alternative if it works. However, it would be too racy unless we we implement a test-and-set for such properties or there is a different job which queues up these requests and perform sequentially for each tenant. __ __ Thanks, -Nikhil *From:*Chris St. Pierre [chris.a.st.pie...@gmail.com mailto:chris.a.st.pie...@gmail.com] *Sent:* Wednesday, December 17, 2014 10:23 AM *To:* OpenStack Development Mailing List (not for usage questions) *Subject:* Re: [openstack-dev] [glance] Option to skip deleting images in use? That's unfortunately too simple. You run into one of two cases: __ __ 1. If the job automatically removes the protected attribute when an image is no longer in use, then you lose the ability to use protected on images that are not in use. I.e., there's no way to say, nothing is currently using this image, but please keep it around. (This seems particularly useful for snapshots, for instance.) __ __ 2. If the job does
Re: [openstack-dev] [glance] Option to skip deleting images in use?
Presumably to prevent images from being deleted for arbitrary reasons that are left to the administrator(s) of each individual implementation of OpenStack, though. Using the protected flag to prevent images that are in use from being deleted obviates the ability to use it for arbitrary protection. That is, it can either be used as a general purpose flag to prevent deletion of an image; or it can be used as a flag for images that are in use and thus must not be deleted; but it cannot be used for both. (At least, not without a wild and woolly network of hacks to ensure that it can serve both purposes.) Given the general-purpose nature of the flag, I don't think that something that should be taken away from the administrators. And yet, it's very desirable to prevent deletion of images that are in use. I think both of these things should be supported, at the same time on the same installation. I do not consider it a solution to the problem that images can be deleted in use to take the protected flag away from arbitrary, bespoke use. On Thu, Dec 18, 2014 at 6:44 PM, Jay Pipes jaypi...@gmail.com wrote: On 12/18/2014 02:08 PM, Chris St. Pierre wrote: I wasn't suggesting that we *actually* use filesystem link count, and make hard links or whatever for every time the image is used. That's prima facie absurd, for a lot more reasons that you point out. I was suggesting a new database field that tracks the number of times an image is in use, by *analogy* with filesystem link counts. (If I wanted to be unnecessarily abrasive I might say, This is a textbook example of something called an analogy, but I'm not interested in being unnecessarily abrasive.) Overloading the protected flag is *still* a terrible hack. Even if we tracked the initial state of protected and restored that state when an image went out of use, that would negate the ability to make an image I guess I don't understand what you consider to be overloading of the protected flag. The original purpose of the protected flag was to protect images from being deleted. Best, -jay protected while it was in use and expect that change to remain in place. So that just violates the principle of least surprise. Of course, we could have glance modify the original_protected_state flag when that flag is non-null and the user changes the actual protected flag, but this is just layering hacks upon hacks. By actually tracking the number of times an image is in use, we can preserve the ability to protect images *and* avoid deleting images in use. On Thu, Dec 18, 2014 at 5:27 AM, Kuvaja, Erno kuv...@hp.com mailto:kuv...@hp.com wrote: I think that’s horrible idea. How do we do that store independent with the linking dependencies? __ __ We should not depend universal use case like this on limited subset of backends, specially non-OpenStack ones. Glance (nor Nova) should never depend having direct access to the actual medium where the images are stored. I think this is school book example for something called database. Well arguable if this should be tracked at Glance or Nova, but definitely not a dirty hack expecting specific backend characteristics. __ __ As mentioned before the protected image property is to ensure that the image does not get deleted, that is also easy to track when the images are queried. Perhaps the record needs to track the original state of protected flag, image id and use count. 3 column table and couple of API calls. Lets not at least make it any more complicated than it needs to be if such functionality is desired. __ __ __-__Erno __ __ *From:*Nikhil Komawar [mailto:nikhil.koma...@rackspace.com mailto:nikhil.koma...@rackspace.com] *Sent:* 17 December 2014 20:34 *To:* OpenStack Development Mailing List (not for usage questions) *Subject:* Re: [openstack-dev] [glance] Option to skip deleting images in use? __ __ Guess that's a implementation detail. Depends on the way you go about using what's available now, I suppose. __ __ Thanks, -Nikhil *From:*Chris St. Pierre [chris.a.st.pie...@gmail.com mailto:chris.a.st.pie...@gmail.com] *Sent:* Wednesday, December 17, 2014 2:07 PM *To:* OpenStack Development Mailing List (not for usage questions) *Subject:* Re: [openstack-dev] [glance] Option to skip deleting images in use? I was assuming atomic increment/decrement operations, in which case I'm not sure I see the race conditions. Or is atomism assuming too much? __ __ On Wed, Dec 17, 2014 at 11:59 AM, Nikhil Komawar nikhil.koma...@rackspace.com mailto:nikhil.koma...@rackspace.com wrote: That looks like a decent alternative if it works. However
Re: [openstack-dev] [glance] Option to skip deleting images in use?
That's unfortunately too simple. You run into one of two cases: 1. If the job automatically removes the protected attribute when an image is no longer in use, then you lose the ability to use protected on images that are not in use. I.e., there's no way to say, nothing is currently using this image, but please keep it around. (This seems particularly useful for snapshots, for instance.) 2. If the job does not automatically remove the protected attribute, then an image would be protected if it had ever been in use; to delete an image, you'd have to manually un-protect it, which is a workflow that quite explicitly defeats the whole purpose of flagging images as protected when they're in use. It seems like flagging an image as *not* in use is actually a fairly difficult problem, since it requires consensus among all components that might be using images. The only solution that readily occurs to me would be to add something like a filesystem link count to images in Glance. Then when Nova spawns an instance, it increments the usage count; when the instance is destroyed, the usage count is decremented. And similarly with other components that use images. An image could only be deleted when its usage count was zero. There are ample opportunities to get out of sync there, but it's at least a sketch of something that might work, and isn't *too* horribly hackish. Thoughts? On Tue, Dec 16, 2014 at 6:11 PM, Vishvananda Ishaya vishvana...@gmail.com wrote: A simple solution that wouldn’t require modification of glance would be a cron job that lists images and snapshots and marks them protected while they are in use. Vish On Dec 16, 2014, at 3:19 PM, Collins, Sean sean_colli...@cable.comcast.com wrote: On Tue, Dec 16, 2014 at 05:12:31PM EST, Chris St. Pierre wrote: No, I'm looking to prevent images that are in use from being deleted. In use and protected are disjoint sets. I have seen multiple cases of images (and snapshots) being deleted while still in use in Nova, which leads to some very, shall we say, interesting bugs and support problems. I do think that we should try and determine a way forward on this, they are indeed disjoint sets. Setting an image as protected is a proactive measure, we should try and figure out a way to keep tenants from shooting themselves in the foot if possible. -- Sean M. Collins ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Chris St. Pierre ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [glance] Option to skip deleting images in use?
That looks like a decent alternative if it works. However, it would be too racy unless we we implement a test-and-set for such properties or there is a different job which queues up these requests and perform sequentially for each tenant. Thanks, -Nikhil From: Chris St. Pierre [chris.a.st.pie...@gmail.com] Sent: Wednesday, December 17, 2014 10:23 AM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [glance] Option to skip deleting images in use? That's unfortunately too simple. You run into one of two cases: 1. If the job automatically removes the protected attribute when an image is no longer in use, then you lose the ability to use protected on images that are not in use. I.e., there's no way to say, nothing is currently using this image, but please keep it around. (This seems particularly useful for snapshots, for instance.) 2. If the job does not automatically remove the protected attribute, then an image would be protected if it had ever been in use; to delete an image, you'd have to manually un-protect it, which is a workflow that quite explicitly defeats the whole purpose of flagging images as protected when they're in use. It seems like flagging an image as *not* in use is actually a fairly difficult problem, since it requires consensus among all components that might be using images. The only solution that readily occurs to me would be to add something like a filesystem link count to images in Glance. Then when Nova spawns an instance, it increments the usage count; when the instance is destroyed, the usage count is decremented. And similarly with other components that use images. An image could only be deleted when its usage count was zero. There are ample opportunities to get out of sync there, but it's at least a sketch of something that might work, and isn't *too* horribly hackish. Thoughts? On Tue, Dec 16, 2014 at 6:11 PM, Vishvananda Ishaya vishvana...@gmail.commailto:vishvana...@gmail.com wrote: A simple solution that wouldn’t require modification of glance would be a cron job that lists images and snapshots and marks them protected while they are in use. Vish On Dec 16, 2014, at 3:19 PM, Collins, Sean sean_colli...@cable.comcast.commailto:sean_colli...@cable.comcast.com wrote: On Tue, Dec 16, 2014 at 05:12:31PM EST, Chris St. Pierre wrote: No, I'm looking to prevent images that are in use from being deleted. In use and protected are disjoint sets. I have seen multiple cases of images (and snapshots) being deleted while still in use in Nova, which leads to some very, shall we say, interesting bugs and support problems. I do think that we should try and determine a way forward on this, they are indeed disjoint sets. Setting an image as protected is a proactive measure, we should try and figure out a way to keep tenants from shooting themselves in the foot if possible. -- Sean M. Collins ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.orgmailto:OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.orgmailto:OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Chris St. Pierre ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [glance] Option to skip deleting images in use?
I was assuming atomic increment/decrement operations, in which case I'm not sure I see the race conditions. Or is atomism assuming too much? On Wed, Dec 17, 2014 at 11:59 AM, Nikhil Komawar nikhil.koma...@rackspace.com wrote: That looks like a decent alternative if it works. However, it would be too racy unless we we implement a test-and-set for such properties or there is a different job which queues up these requests and perform sequentially for each tenant. Thanks, -Nikhil -- *From:* Chris St. Pierre [chris.a.st.pie...@gmail.com] *Sent:* Wednesday, December 17, 2014 10:23 AM *To:* OpenStack Development Mailing List (not for usage questions) *Subject:* Re: [openstack-dev] [glance] Option to skip deleting images in use? That's unfortunately too simple. You run into one of two cases: 1. If the job automatically removes the protected attribute when an image is no longer in use, then you lose the ability to use protected on images that are not in use. I.e., there's no way to say, nothing is currently using this image, but please keep it around. (This seems particularly useful for snapshots, for instance.) 2. If the job does not automatically remove the protected attribute, then an image would be protected if it had ever been in use; to delete an image, you'd have to manually un-protect it, which is a workflow that quite explicitly defeats the whole purpose of flagging images as protected when they're in use. It seems like flagging an image as *not* in use is actually a fairly difficult problem, since it requires consensus among all components that might be using images. The only solution that readily occurs to me would be to add something like a filesystem link count to images in Glance. Then when Nova spawns an instance, it increments the usage count; when the instance is destroyed, the usage count is decremented. And similarly with other components that use images. An image could only be deleted when its usage count was zero. There are ample opportunities to get out of sync there, but it's at least a sketch of something that might work, and isn't *too* horribly hackish. Thoughts? On Tue, Dec 16, 2014 at 6:11 PM, Vishvananda Ishaya vishvana...@gmail.com wrote: A simple solution that wouldn’t require modification of glance would be a cron job that lists images and snapshots and marks them protected while they are in use. Vish On Dec 16, 2014, at 3:19 PM, Collins, Sean sean_colli...@cable.comcast.com wrote: On Tue, Dec 16, 2014 at 05:12:31PM EST, Chris St. Pierre wrote: No, I'm looking to prevent images that are in use from being deleted. In use and protected are disjoint sets. I have seen multiple cases of images (and snapshots) being deleted while still in use in Nova, which leads to some very, shall we say, interesting bugs and support problems. I do think that we should try and determine a way forward on this, they are indeed disjoint sets. Setting an image as protected is a proactive measure, we should try and figure out a way to keep tenants from shooting themselves in the foot if possible. -- Sean M. Collins ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Chris St. Pierre ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Chris St. Pierre ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [glance] Option to skip deleting images in use?
Guess that's a implementation detail. Depends on the way you go about using what's available now, I suppose. Thanks, -Nikhil From: Chris St. Pierre [chris.a.st.pie...@gmail.com] Sent: Wednesday, December 17, 2014 2:07 PM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [glance] Option to skip deleting images in use? I was assuming atomic increment/decrement operations, in which case I'm not sure I see the race conditions. Or is atomism assuming too much? On Wed, Dec 17, 2014 at 11:59 AM, Nikhil Komawar nikhil.koma...@rackspace.commailto:nikhil.koma...@rackspace.com wrote: That looks like a decent alternative if it works. However, it would be too racy unless we we implement a test-and-set for such properties or there is a different job which queues up these requests and perform sequentially for each tenant. Thanks, -Nikhil From: Chris St. Pierre [chris.a.st.pie...@gmail.commailto:chris.a.st.pie...@gmail.com] Sent: Wednesday, December 17, 2014 10:23 AM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [glance] Option to skip deleting images in use? That's unfortunately too simple. You run into one of two cases: 1. If the job automatically removes the protected attribute when an image is no longer in use, then you lose the ability to use protected on images that are not in use. I.e., there's no way to say, nothing is currently using this image, but please keep it around. (This seems particularly useful for snapshots, for instance.) 2. If the job does not automatically remove the protected attribute, then an image would be protected if it had ever been in use; to delete an image, you'd have to manually un-protect it, which is a workflow that quite explicitly defeats the whole purpose of flagging images as protected when they're in use. It seems like flagging an image as *not* in use is actually a fairly difficult problem, since it requires consensus among all components that might be using images. The only solution that readily occurs to me would be to add something like a filesystem link count to images in Glance. Then when Nova spawns an instance, it increments the usage count; when the instance is destroyed, the usage count is decremented. And similarly with other components that use images. An image could only be deleted when its usage count was zero. There are ample opportunities to get out of sync there, but it's at least a sketch of something that might work, and isn't *too* horribly hackish. Thoughts? On Tue, Dec 16, 2014 at 6:11 PM, Vishvananda Ishaya vishvana...@gmail.commailto:vishvana...@gmail.com wrote: A simple solution that wouldn’t require modification of glance would be a cron job that lists images and snapshots and marks them protected while they are in use. Vish On Dec 16, 2014, at 3:19 PM, Collins, Sean sean_colli...@cable.comcast.commailto:sean_colli...@cable.comcast.com wrote: On Tue, Dec 16, 2014 at 05:12:31PM EST, Chris St. Pierre wrote: No, I'm looking to prevent images that are in use from being deleted. In use and protected are disjoint sets. I have seen multiple cases of images (and snapshots) being deleted while still in use in Nova, which leads to some very, shall we say, interesting bugs and support problems. I do think that we should try and determine a way forward on this, they are indeed disjoint sets. Setting an image as protected is a proactive measure, we should try and figure out a way to keep tenants from shooting themselves in the foot if possible. -- Sean M. Collins ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.orgmailto:OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.orgmailto:OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Chris St. Pierre ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.orgmailto:OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Chris St. Pierre ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [glance] Option to skip deleting images in use?
Currently, with delay_delete enabled, the Glance scrubber happily deletes whatever images you ask it to. That includes images that are currently in use by Nova guests, which can really hose things. It'd be nice to have an option to tell the scrubber to skip deletion of images that are currently in use, which is fairly trivial to check for and provides a nice measure of protection. Without delay_delete enabled, checking for images in use likely takes too much time, so this would be limited to just images that are scrubbed with delay_delete. I wanted to bring this up here before I go to the trouble of writing a spec for it, particularly since it doesn't appear that glance currently talks to Nova as a client at all. Is this something that folks would be interested in having? Thanks! -- Chris St. Pierre ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [glance] Option to skip deleting images in use?
Just set the images to is_public=False as an admin and they'll disappear from everyone except the admin. -jay On 12/16/2014 03:09 PM, Chris St. Pierre wrote: Currently, with delay_delete enabled, the Glance scrubber happily deletes whatever images you ask it to. That includes images that are currently in use by Nova guests, which can really hose things. It'd be nice to have an option to tell the scrubber to skip deletion of images that are currently in use, which is fairly trivial to check for and provides a nice measure of protection. Without delay_delete enabled, checking for images in use likely takes too much time, so this would be limited to just images that are scrubbed with delay_delete. I wanted to bring this up here before I go to the trouble of writing a spec for it, particularly since it doesn't appear that glance currently talks to Nova as a client at all. Is this something that folks would be interested in having? Thanks! -- Chris St. Pierre ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [glance] Option to skip deleting images in use?
The goal here is protection against deletion of in-use images, not a workaround that can be executed by an admin. For instance, someone without admin still can't do that, and someone with a fat finger can still delete images in use. Don't lose your data is a fine workaround for taking backups, but most of us take backups anyway. Same deal. On Tue, Dec 16, 2014 at 2:30 PM, Jay Pipes jaypi...@gmail.com wrote: Just set the images to is_public=False as an admin and they'll disappear from everyone except the admin. -jay On 12/16/2014 03:09 PM, Chris St. Pierre wrote: Currently, with delay_delete enabled, the Glance scrubber happily deletes whatever images you ask it to. That includes images that are currently in use by Nova guests, which can really hose things. It'd be nice to have an option to tell the scrubber to skip deletion of images that are currently in use, which is fairly trivial to check for and provides a nice measure of protection. Without delay_delete enabled, checking for images in use likely takes too much time, so this would be limited to just images that are scrubbed with delay_delete. I wanted to bring this up here before I go to the trouble of writing a spec for it, particularly since it doesn't appear that glance currently talks to Nova as a client at all. Is this something that folks would be interested in having? Thanks! -- Chris St. Pierre ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Chris St. Pierre ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [glance] Option to skip deleting images in use?
On 12/16/2014 04:23 PM, Chris St. Pierre wrote: The goal here is protection against deletion of in-use images, not a workaround that can be executed by an admin. For instance, someone without admin still can't do that, and someone with a fat finger can still delete images in use. Then set the protected property on the image, which prevents it from being deleted. From the glance CLI image-update help output: --is-protected [True|False] Prevent image from being deleted. Don't lose your data is a fine workaround for taking backups, but most of us take backups anyway. Same deal. On Tue, Dec 16, 2014 at 2:30 PM, Jay Pipes jaypi...@gmail.com mailto:jaypi...@gmail.com wrote: Just set the images to is_public=False as an admin and they'll disappear from everyone except the admin. -jay On 12/16/2014 03:09 PM, Chris St. Pierre wrote: Currently, with delay_delete enabled, the Glance scrubber happily deletes whatever images you ask it to. That includes images that are currently in use by Nova guests, which can really hose things. It'd be nice to have an option to tell the scrubber to skip deletion of images that are currently in use, which is fairly trivial to check for and provides a nice measure of protection. Without delay_delete enabled, checking for images in use likely takes too much time, so this would be limited to just images that are scrubbed with delay_delete. I wanted to bring this up here before I go to the trouble of writing a spec for it, particularly since it doesn't appear that glance currently talks to Nova as a client at all. Is this something that folks would be interested in having? Thanks! -- Chris St. Pierre _ OpenStack-dev mailing list OpenStack-dev@lists.openstack.__org mailto:OpenStack-dev@lists.openstack.org http://lists.openstack.org/__cgi-bin/mailman/listinfo/__openstack-dev http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev _ OpenStack-dev mailing list OpenStack-dev@lists.openstack.__org mailto:OpenStack-dev@lists.openstack.org http://lists.openstack.org/__cgi-bin/mailman/listinfo/__openstack-dev http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Chris St. Pierre ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [glance] Option to skip deleting images in use?
+1 Thanks, -Nikhil From: Jay Pipes [jaypi...@gmail.com] Sent: Tuesday, December 16, 2014 4:33 PM To: openstack-dev@lists.openstack.org Subject: Re: [openstack-dev] [glance] Option to skip deleting images in use? On 12/16/2014 04:23 PM, Chris St. Pierre wrote: The goal here is protection against deletion of in-use images, not a workaround that can be executed by an admin. For instance, someone without admin still can't do that, and someone with a fat finger can still delete images in use. Then set the protected property on the image, which prevents it from being deleted. From the glance CLI image-update help output: --is-protected [True|False] Prevent image from being deleted. Don't lose your data is a fine workaround for taking backups, but most of us take backups anyway. Same deal. On Tue, Dec 16, 2014 at 2:30 PM, Jay Pipes jaypi...@gmail.com mailto:jaypi...@gmail.com wrote: Just set the images to is_public=False as an admin and they'll disappear from everyone except the admin. -jay On 12/16/2014 03:09 PM, Chris St. Pierre wrote: Currently, with delay_delete enabled, the Glance scrubber happily deletes whatever images you ask it to. That includes images that are currently in use by Nova guests, which can really hose things. It'd be nice to have an option to tell the scrubber to skip deletion of images that are currently in use, which is fairly trivial to check for and provides a nice measure of protection. Without delay_delete enabled, checking for images in use likely takes too much time, so this would be limited to just images that are scrubbed with delay_delete. I wanted to bring this up here before I go to the trouble of writing a spec for it, particularly since it doesn't appear that glance currently talks to Nova as a client at all. Is this something that folks would be interested in having? Thanks! -- Chris St. Pierre _ OpenStack-dev mailing list OpenStack-dev@lists.openstack.__org mailto:OpenStack-dev@lists.openstack.org http://lists.openstack.org/__cgi-bin/mailman/listinfo/__openstack-dev http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev _ OpenStack-dev mailing list OpenStack-dev@lists.openstack.__org mailto:OpenStack-dev@lists.openstack.org http://lists.openstack.org/__cgi-bin/mailman/listinfo/__openstack-dev http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Chris St. Pierre ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [glance] Option to skip deleting images in use?
Hi Chris, Are you looking for the 'protected' attribute? You can mark an image with 'protected'=True, then the image can't be deleted by accidentally. On 17/12/14 10:23, Chris St. Pierre wrote: The goal here is protection against deletion of in-use images, not a workaround that can be executed by an admin. For instance, someone without admin still can't do that, and someone with a fat finger can still delete images in use. Don't lose your data is a fine workaround for taking backups, but most of us take backups anyway. Same deal. On Tue, Dec 16, 2014 at 2:30 PM, Jay Pipes jaypi...@gmail.com mailto:jaypi...@gmail.com wrote: Just set the images to is_public=False as an admin and they'll disappear from everyone except the admin. -jay On 12/16/2014 03:09 PM, Chris St. Pierre wrote: Currently, with delay_delete enabled, the Glance scrubber happily deletes whatever images you ask it to. That includes images that are currently in use by Nova guests, which can really hose things. It'd be nice to have an option to tell the scrubber to skip deletion of images that are currently in use, which is fairly trivial to check for and provides a nice measure of protection. Without delay_delete enabled, checking for images in use likely takes too much time, so this would be limited to just images that are scrubbed with delay_delete. I wanted to bring this up here before I go to the trouble of writing a spec for it, particularly since it doesn't appear that glance currently talks to Nova as a client at all. Is this something that folks would be interested in having? Thanks! -- Chris St. Pierre ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org mailto:OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org mailto:OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Chris St. Pierre ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Cheers Best regards, Fei Long Wang (王飞龙) -- Senior Cloud Software Engineer Tel: +64-48032246 Email: flw...@catalyst.net.nz Catalyst IT Limited Level 6, Catalyst House, 150 Willis Street, Wellington -- ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [glance] Option to skip deleting images in use?
No, I'm looking to prevent images that are in use from being deleted. In use and protected are disjoint sets. On Tue, Dec 16, 2014 at 3:36 PM, Fei Long Wang feil...@catalyst.net.nz wrote: Hi Chris, Are you looking for the 'protected' attribute? You can mark an image with 'protected'=True, then the image can't be deleted by accidentally. On 17/12/14 10:23, Chris St. Pierre wrote: The goal here is protection against deletion of in-use images, not a workaround that can be executed by an admin. For instance, someone without admin still can't do that, and someone with a fat finger can still delete images in use. Don't lose your data is a fine workaround for taking backups, but most of us take backups anyway. Same deal. On Tue, Dec 16, 2014 at 2:30 PM, Jay Pipes jaypi...@gmail.com wrote: Just set the images to is_public=False as an admin and they'll disappear from everyone except the admin. -jay On 12/16/2014 03:09 PM, Chris St. Pierre wrote: Currently, with delay_delete enabled, the Glance scrubber happily deletes whatever images you ask it to. That includes images that are currently in use by Nova guests, which can really hose things. It'd be nice to have an option to tell the scrubber to skip deletion of images that are currently in use, which is fairly trivial to check for and provides a nice measure of protection. Without delay_delete enabled, checking for images in use likely takes too much time, so this would be limited to just images that are scrubbed with delay_delete. I wanted to bring this up here before I go to the trouble of writing a spec for it, particularly since it doesn't appear that glance currently talks to Nova as a client at all. Is this something that folks would be interested in having? Thanks! -- Chris St. Pierre ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Chris St. Pierre ___ OpenStack-dev mailing listOpenStack-dev@lists.openstack.orghttp://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Cheers Best regards, Fei Long Wang (王飞龙) -- Senior Cloud Software Engineer Tel: +64-48032246 Email: flw...@catalyst.net.nz Catalyst IT Limited Level 6, Catalyst House, 150 Willis Street, Wellington -- ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Chris St. Pierre ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [glance] Option to skip deleting images in use?
Hi Chris, Apologies for not having heard your use case completely. From the description as well as the information you've provided; it is my recommendation to use a protected property in Glance for the Image entity that is in use. You can then use it in the service of your choice (Nova, Cinder) for not deleting the same. It is that service which shall have more accurate information as well as be source of truth for the in-use state of the Image entity. Making a call out to different service (except backend stores) is out of the scope of Glance. (Nova is the client of Glance and we would like to avoid the circular dependency mess there!) Hope it helps. Please let me know if you need more information. Thanks and Regards, -Nikhil From: Chris St. Pierre [chris.a.st.pie...@gmail.com] Sent: Tuesday, December 16, 2014 5:12 PM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [glance] Option to skip deleting images in use? No, I'm looking to prevent images that are in use from being deleted. In use and protected are disjoint sets. On Tue, Dec 16, 2014 at 3:36 PM, Fei Long Wang feil...@catalyst.net.nzmailto:feil...@catalyst.net.nz wrote: Hi Chris, Are you looking for the 'protected' attribute? You can mark an image with 'protected'=True, then the image can't be deleted by accidentally. On 17/12/14 10:23, Chris St. Pierre wrote: The goal here is protection against deletion of in-use images, not a workaround that can be executed by an admin. For instance, someone without admin still can't do that, and someone with a fat finger can still delete images in use. Don't lose your data is a fine workaround for taking backups, but most of us take backups anyway. Same deal. On Tue, Dec 16, 2014 at 2:30 PM, Jay Pipes jaypi...@gmail.commailto:jaypi...@gmail.com wrote: Just set the images to is_public=False as an admin and they'll disappear from everyone except the admin. -jay On 12/16/2014 03:09 PM, Chris St. Pierre wrote: Currently, with delay_delete enabled, the Glance scrubber happily deletes whatever images you ask it to. That includes images that are currently in use by Nova guests, which can really hose things. It'd be nice to have an option to tell the scrubber to skip deletion of images that are currently in use, which is fairly trivial to check for and provides a nice measure of protection. Without delay_delete enabled, checking for images in use likely takes too much time, so this would be limited to just images that are scrubbed with delay_delete. I wanted to bring this up here before I go to the trouble of writing a spec for it, particularly since it doesn't appear that glance currently talks to Nova as a client at all. Is this something that folks would be interested in having? Thanks! -- Chris St. Pierre ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.orgmailto:OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.orgmailto:OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Chris St. Pierre ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.orgmailto:OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Cheers Best regards, Fei Long Wang (王飞龙) -- Senior Cloud Software Engineer Tel: +64-48032246tel:%2B64-48032246 Email: flw...@catalyst.net.nzmailto:flw...@catalyst.net.nz Catalyst IT Limited Level 6, Catalyst House, 150 Willis Street, Wellington -- ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.orgmailto:OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Chris St. Pierre ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [glance] Option to skip deleting images in use?
On Tue, Dec 16, 2014 at 05:12:31PM EST, Chris St. Pierre wrote: No, I'm looking to prevent images that are in use from being deleted. In use and protected are disjoint sets. I have seen multiple cases of images (and snapshots) being deleted while still in use in Nova, which leads to some very, shall we say, interesting bugs and support problems. I do think that we should try and determine a way forward on this, they are indeed disjoint sets. Setting an image as protected is a proactive measure, we should try and figure out a way to keep tenants from shooting themselves in the foot if possible. -- Sean M. Collins ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [glance] Option to skip deleting images in use?
A simple solution that wouldn’t require modification of glance would be a cron job that lists images and snapshots and marks them protected while they are in use. Vish On Dec 16, 2014, at 3:19 PM, Collins, Sean sean_colli...@cable.comcast.com wrote: On Tue, Dec 16, 2014 at 05:12:31PM EST, Chris St. Pierre wrote: No, I'm looking to prevent images that are in use from being deleted. In use and protected are disjoint sets. I have seen multiple cases of images (and snapshots) being deleted while still in use in Nova, which leads to some very, shall we say, interesting bugs and support problems. I do think that we should try and determine a way forward on this, they are indeed disjoint sets. Setting an image as protected is a proactive measure, we should try and figure out a way to keep tenants from shooting themselves in the foot if possible. -- Sean M. Collins ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
