Re: [openstack-dev] [Nova] Reconciling flavors and block device mappings
On 08/29/2016 05:53 AM, Andrew Laski wrote: Personally I believe the cat is out of the bag on bdms overriding flavors and we should just commit to that path and make it work well. And for deployers who rely on flavors being the source of truth maybe we provide them a policy check or some other mechanism to control acceptable bdm values so that they can rely on flavor packing. Yes, I agree with everything you wrote above. -jay __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Reconciling flavors and block device mappings
> -Original Message- > From: Sylvain Bauza [mailto:sba...@redhat.com] > Sent: Monday, August 29, 2016 2:43 PM > To: OpenStack Development Mailing List (not for usage questions) d...@lists.openstack.org> > Subject: Re: [openstack-dev] [Nova] Reconciling flavors and block device > mappings > >> While : > >> #1 the first change about setting root_gb equals 0 in RequestSpec for > >> making sure BFV instances are correctly using the DiskFilter is fine > >> by me having it merged with a TODO/FIXME saying that the code would > >> be expired once the scheduler uses the resource providers, #2, then > >> the second patch about trying to look at the BDMs for DiskFilter is > >> very wrong by me, because the Compute API shouldn't accept IMHO to > >> ask for a flavor *and* a BDM with a related disk size different from > >> the flavor. AFAIT, we should return a 40x (probably 409 or 400) for > >> that instead of accepting it silently. > > > > Well, a flavor is always required when launching an instance. I wasn't > > aware until recently that one could "override" the root GB (or > > eph/swap) sizes in the API. Apparently, the API allows it, though, > > even though the code ignores whatever was passed as the override > > value. If the API supports it, I think the code should probably be > > changed to override the size values to whatever the user entered, no? > > > I'm very sad to see this possible. FWIW, I think that the flavor should > always be > the source of truth for knowing the asked disk size (if no, imagine all the > conditionals in code...) and we should somehow reconcile what the flavor > provided and what the user explicitly asked for. > > Having a possibility to trample the flavor size seems to be a very bad UX (I > guess > most of the operators don't think about that when understanding why this > instance could have a different size from the related flavor) hence me > thinking > we should rather discuss on a possible new microversion for returning a 400 > when both sizes are not identical. Hi, when it comes to more short-term I think that many comments of John/Tim/Andrew/Jay/Silvain are not against the fact that patches will get merged. somehow I disagree with Silvain on the part of 'returning 400 in case both sizes are not identical' as mid-term direction That will break the scheduling for those who now use root disk = 0. And using root disk=0 for such instances should be not only acceptable workaround but also acceptable in mid- long- term IMHO. I disagree because [1] gives definition of root disk as ' Virtual root disk size in gigabytes. This is an ephemeral disk that the base image is copied into. When booting from a persistent volume it is not used' So there it is written 'root disk is an ephemeral disk', 'when booting from a persistent volume it is not used' Using boot from volume [which should be referred 'using BDM, as more technically learnt, according to what I learnt in this thread] thus shouldn't have relation with root disk. And not-yet-merged [2] gives direction "Therefore 0 should only be used for volume booted instances or for testing purposes." So operators should not wonder [Sylvain] why its disk sizes could be different from the related flavor. I have no objections on improving flavors (/flavor specs) or having other mechanism to control the size of the (Cinder) volume being used with VM. It sounds for me more Cinder area than Nova, but maybe I am wrong. I would say, that would be used to complement volume quota(s) Introducing something like max.instances could be part of addressing that use cases/possibly with other use cases. That however sounds like a new blueprint. BR, Konstantin [1] http://docs.openstack.org/admin-guide/compute-flavors.html [2] https://review.openstack.org/#/c/339034/ > The other option I could see could be to have the nested flavor in the > RequestSpec be reconciling those two objects being different from the user- > provided flavor (eg. say a flavor with swap=10G and a BDM with swap=1G, then > RequestSpec.flavor would have swap=1G) but given we don't expose the > RequestSpec objects on the API level, that still leaves operators possibly > confused. > > -Sylvain > > > > -jay > > > > __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Reconciling flavors and block device mappings
On Mon, Aug 29, 2016, at 08:42 AM, Sylvain Bauza wrote: > > > Le 29/08/2016 14:20, Jay Pipes a écrit : > > On 08/29/2016 05:11 AM, Sylvain Bauza wrote: > >> Le 29/08/2016 13:25, Jay Pipes a écrit : > >>> On 08/26/2016 09:20 AM, Ed Leafe wrote: > On Aug 25, 2016, at 3:19 PM, Andrew Laskiwrote: > > > One other thing to note is that while a flavor constrains how much > > local > > disk is used it does not constrain volume size at all. So a user can > > specify an ephemeral/swap disk <= to what the flavor provides but can > > have an arbitrary sized root disk if it's a remote volume. > > This kind of goes to the heart of the argument against flavors being > the sole source of truth for a request. As cloud evolves, we keep > packing more and more stuff into a concept that was originally meant > to only divide up resources that came bundled together (CPU, RAM, and > local disk). This hasn’t been a good solution for years, and the > sooner we start accepting that a request can be much more complex > than a flavor can adequately express, the better. > > If we have decided that remote volumes are a good thing (I don’t > think there’s any argument there), then we should treat that part of > the request as being as fundamental as a flavor. We need to make the > scheduler smarter so that it doesn’t rely on flavor as being the only > source of truth. > > The first step to improving Nova is admitting we have a problem. :) > >>> > >>> FWIW, I agree with you on the above. The issue I had with the proposed > >>> patches was that they would essentially be a hack for a short period > >>> of time until the resource providers work standardized the way that > >>> DISK_GB resources were tracked -- including for providers of shared > >>> disk storage. > >>> > >>> I've long felt that flavors as a concept should be, as Chris so > >>> adeptly wrote, "UI furniture" and should be decomposed into their > >>> requisite lists of resource amounts, required traits and preferred > >>> traits and that those decomposed parts are what should be passed to > >>> the Compute API, not a flavor ID. > >>> > >>> But again, we're actively changing all this code in the resource > >>> providers and qualitative traits patches so I warned about adding more > >>> code that was essentially just a short-lived hack. I'd be OK adding > >>> the hack code if there were some big bright WARNINGs placed in there > >>> that likely the code would be removed in Ocata. > >>> > >> > >> While : > >> #1 the first change about setting root_gb equals 0 in RequestSpec for > >> making sure BFV instances are correctly using the DiskFilter is fine by > >> me having it merged with a TODO/FIXME saying that the code would be > >> expired once the scheduler uses the resource providers, > >> #2, then the second patch about trying to look at the BDMs for > >> DiskFilter is very wrong by me, because the Compute API shouldn't accept > >> IMHO to ask for a flavor *and* a BDM with a related disk size different > >> from the flavor. AFAIT, we should return a 40x (probably 409 or 400) for > >> that instead of accepting it silently. > > > > Well, a flavor is always required when launching an instance. I wasn't > > aware until recently that one could "override" the root GB (or > > eph/swap) sizes in the API. Apparently, the API allows it, though, > > even though the code ignores whatever was passed as the override > > value. If the API supports it, I think the code should probably be > > changed to override the size values to whatever the user entered, no? > > > I'm very sad to see this possible. FWIW, I think that the flavor should > always be the source of truth for knowing the asked disk size (if no, > imagine all the conditionals in code...) and we should somehow reconcile > what the flavor provided and what the user explicitly asked for. At the heart of all of this is the fact that we've allowed people to believe two different things: flavors are the source of truth, bdms can be used to override flavors. But bdms only half override flavors because they don't affect scheduling so people who believe the latter are understandably trying to fix that. But we can't have it both ways, so we need to have consensus about what we're supporting and then make it work fully. Personally I believe the cat is out of the bag on bdms overriding flavors and we should just commit to that path and make it work well. And for deployers who rely on flavors being the source of truth maybe we provide them a policy check or some other mechanism to control acceptable bdm values so that they can rely on flavor packing. > > Having a possibility to trample the flavor size seems to be a very bad > UX (I guess most of the operators don't think about that when > understanding why this instance could have a different size from the > related flavor) hence me thinking we
Re: [openstack-dev] [Nova] Reconciling flavors and block device mappings
On Mon, Aug 29, 2016, at 07:25 AM, Jay Pipes wrote: > On 08/26/2016 09:20 AM, Ed Leafe wrote: > > On Aug 25, 2016, at 3:19 PM, Andrew Laskiwrote: > > > >> One other thing to note is that while a flavor constrains how much local > >> disk is used it does not constrain volume size at all. So a user can > >> specify an ephemeral/swap disk <= to what the flavor provides but can > >> have an arbitrary sized root disk if it's a remote volume. > > > > This kind of goes to the heart of the argument against flavors being the > > sole source of truth for a request. As cloud evolves, we keep packing more > > and more stuff into a concept that was originally meant to only divide up > > resources that came bundled together (CPU, RAM, and local disk). This > > hasn’t been a good solution for years, and the sooner we start accepting > > that a request can be much more complex than a flavor can adequately > > express, the better. > > > > If we have decided that remote volumes are a good thing (I don’t think > > there’s any argument there), then we should treat that part of the request > > as being as fundamental as a flavor. We need to make the scheduler smarter > > so that it doesn’t rely on flavor as being the only source of truth. > > > > The first step to improving Nova is admitting we have a problem. :) > > FWIW, I agree with you on the above. The issue I had with the proposed > patches was that they would essentially be a hack for a short period of > time until the resource providers work standardized the way that DISK_GB > resources were tracked -- including for providers of shared disk storage. > > I've long felt that flavors as a concept should be, as Chris so adeptly > wrote, "UI furniture" and should be decomposed into their requisite > lists of resource amounts, required traits and preferred traits and that > those decomposed parts are what should be passed to the Compute API, not > a flavor ID. > > But again, we're actively changing all this code in the resource > providers and qualitative traits patches so I warned about adding more > code that was essentially just a short-lived hack. I'd be OK adding the > hack code if there were some big bright WARNINGs placed in there that > likely the code would be removed in Ocata. I'd like to clarify that there are two parts to the patches proposed. One part is about bdm overrides influencing the scheduler, and the other part is about proper resource tracking. I've attempted to punt on the resource tracking part in this thread because I agree that we have a proper solution on the way and the current proposals are workarounds. There is some value in the workarounds though as they could be backported to Mitaka. > > -jay > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: > openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Reconciling flavors and block device mappings
Le 29/08/2016 14:43, Andrew Laski a écrit : On Mon, Aug 29, 2016, at 08:20 AM, Jay Pipes wrote: On 08/29/2016 05:11 AM, Sylvain Bauza wrote: Le 29/08/2016 13:25, Jay Pipes a écrit : On 08/26/2016 09:20 AM, Ed Leafe wrote: On Aug 25, 2016, at 3:19 PM, Andrew Laskiwrote: One other thing to note is that while a flavor constrains how much local disk is used it does not constrain volume size at all. So a user can specify an ephemeral/swap disk <= to what the flavor provides but can have an arbitrary sized root disk if it's a remote volume. This kind of goes to the heart of the argument against flavors being the sole source of truth for a request. As cloud evolves, we keep packing more and more stuff into a concept that was originally meant to only divide up resources that came bundled together (CPU, RAM, and local disk). This hasn’t been a good solution for years, and the sooner we start accepting that a request can be much more complex than a flavor can adequately express, the better. If we have decided that remote volumes are a good thing (I don’t think there’s any argument there), then we should treat that part of the request as being as fundamental as a flavor. We need to make the scheduler smarter so that it doesn’t rely on flavor as being the only source of truth. The first step to improving Nova is admitting we have a problem. :) FWIW, I agree with you on the above. The issue I had with the proposed patches was that they would essentially be a hack for a short period of time until the resource providers work standardized the way that DISK_GB resources were tracked -- including for providers of shared disk storage. I've long felt that flavors as a concept should be, as Chris so adeptly wrote, "UI furniture" and should be decomposed into their requisite lists of resource amounts, required traits and preferred traits and that those decomposed parts are what should be passed to the Compute API, not a flavor ID. But again, we're actively changing all this code in the resource providers and qualitative traits patches so I warned about adding more code that was essentially just a short-lived hack. I'd be OK adding the hack code if there were some big bright WARNINGs placed in there that likely the code would be removed in Ocata. While : #1 the first change about setting root_gb equals 0 in RequestSpec for making sure BFV instances are correctly using the DiskFilter is fine by me having it merged with a TODO/FIXME saying that the code would be expired once the scheduler uses the resource providers, #2, then the second patch about trying to look at the BDMs for DiskFilter is very wrong by me, because the Compute API shouldn't accept IMHO to ask for a flavor *and* a BDM with a related disk size different from the flavor. AFAIT, we should return a 40x (probably 409 or 400) for that instead of accepting it silently. Well, a flavor is always required when launching an instance. I wasn't aware until recently that one could "override" the root GB (or eph/swap) sizes in the API. Apparently, the API allows it, though, even though the code ignores whatever was passed as the override value. If the API supports it, I think the code should probably be changed to override the size values to whatever the user entered, no? That's the question here. There's a lot of desire to have the overrides be the values used in scheduling, since currently the flavor values are used, but making that change potentially impacts how flavors are packed onto hosts for some deployers. The main thing I want to get out of this thread is if that's okay. The sense I get so far is that it's okay to make the change to have bdm values be passed to the scheduler, via RequestSpec modifications, and deployers can rely upon CPU/RAM constraints to determine packing. So the main thing to do is have good release notes about the change. Like I said just before, that could be an option but given we don't expose an instance-related RequestSpec, it means that litterally operators showing an instance would wonder why its disk sizes could be different from the related flavor. -Sylvain -jay __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Reconciling flavors and block device mappings
On Mon, Aug 29, 2016, at 08:20 AM, Jay Pipes wrote: > On 08/29/2016 05:11 AM, Sylvain Bauza wrote: > > Le 29/08/2016 13:25, Jay Pipes a écrit : > >> On 08/26/2016 09:20 AM, Ed Leafe wrote: > >>> On Aug 25, 2016, at 3:19 PM, Andrew Laskiwrote: > >>> > One other thing to note is that while a flavor constrains how much > local > disk is used it does not constrain volume size at all. So a user can > specify an ephemeral/swap disk <= to what the flavor provides but can > have an arbitrary sized root disk if it's a remote volume. > >>> > >>> This kind of goes to the heart of the argument against flavors being > >>> the sole source of truth for a request. As cloud evolves, we keep > >>> packing more and more stuff into a concept that was originally meant > >>> to only divide up resources that came bundled together (CPU, RAM, and > >>> local disk). This hasn’t been a good solution for years, and the > >>> sooner we start accepting that a request can be much more complex > >>> than a flavor can adequately express, the better. > >>> > >>> If we have decided that remote volumes are a good thing (I don’t > >>> think there’s any argument there), then we should treat that part of > >>> the request as being as fundamental as a flavor. We need to make the > >>> scheduler smarter so that it doesn’t rely on flavor as being the only > >>> source of truth. > >>> > >>> The first step to improving Nova is admitting we have a problem. :) > >> > >> FWIW, I agree with you on the above. The issue I had with the proposed > >> patches was that they would essentially be a hack for a short period > >> of time until the resource providers work standardized the way that > >> DISK_GB resources were tracked -- including for providers of shared > >> disk storage. > >> > >> I've long felt that flavors as a concept should be, as Chris so > >> adeptly wrote, "UI furniture" and should be decomposed into their > >> requisite lists of resource amounts, required traits and preferred > >> traits and that those decomposed parts are what should be passed to > >> the Compute API, not a flavor ID. > >> > >> But again, we're actively changing all this code in the resource > >> providers and qualitative traits patches so I warned about adding more > >> code that was essentially just a short-lived hack. I'd be OK adding > >> the hack code if there were some big bright WARNINGs placed in there > >> that likely the code would be removed in Ocata. > >> > > > > While : > > #1 the first change about setting root_gb equals 0 in RequestSpec for > > making sure BFV instances are correctly using the DiskFilter is fine by > > me having it merged with a TODO/FIXME saying that the code would be > > expired once the scheduler uses the resource providers, > > #2, then the second patch about trying to look at the BDMs for > > DiskFilter is very wrong by me, because the Compute API shouldn't accept > > IMHO to ask for a flavor *and* a BDM with a related disk size different > > from the flavor. AFAIT, we should return a 40x (probably 409 or 400) for > > that instead of accepting it silently. > > Well, a flavor is always required when launching an instance. I wasn't > aware until recently that one could "override" the root GB (or eph/swap) > sizes in the API. Apparently, the API allows it, though, even though the > code ignores whatever was passed as the override value. If the API > supports it, I think the code should probably be changed to override the > size values to whatever the user entered, no? That's the question here. There's a lot of desire to have the overrides be the values used in scheduling, since currently the flavor values are used, but making that change potentially impacts how flavors are packed onto hosts for some deployers. The main thing I want to get out of this thread is if that's okay. The sense I get so far is that it's okay to make the change to have bdm values be passed to the scheduler, via RequestSpec modifications, and deployers can rely upon CPU/RAM constraints to determine packing. So the main thing to do is have good release notes about the change. > > -jay > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: > openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Reconciling flavors and block device mappings
Le 29/08/2016 14:20, Jay Pipes a écrit : On 08/29/2016 05:11 AM, Sylvain Bauza wrote: Le 29/08/2016 13:25, Jay Pipes a écrit : On 08/26/2016 09:20 AM, Ed Leafe wrote: On Aug 25, 2016, at 3:19 PM, Andrew Laskiwrote: One other thing to note is that while a flavor constrains how much local disk is used it does not constrain volume size at all. So a user can specify an ephemeral/swap disk <= to what the flavor provides but can have an arbitrary sized root disk if it's a remote volume. This kind of goes to the heart of the argument against flavors being the sole source of truth for a request. As cloud evolves, we keep packing more and more stuff into a concept that was originally meant to only divide up resources that came bundled together (CPU, RAM, and local disk). This hasn’t been a good solution for years, and the sooner we start accepting that a request can be much more complex than a flavor can adequately express, the better. If we have decided that remote volumes are a good thing (I don’t think there’s any argument there), then we should treat that part of the request as being as fundamental as a flavor. We need to make the scheduler smarter so that it doesn’t rely on flavor as being the only source of truth. The first step to improving Nova is admitting we have a problem. :) FWIW, I agree with you on the above. The issue I had with the proposed patches was that they would essentially be a hack for a short period of time until the resource providers work standardized the way that DISK_GB resources were tracked -- including for providers of shared disk storage. I've long felt that flavors as a concept should be, as Chris so adeptly wrote, "UI furniture" and should be decomposed into their requisite lists of resource amounts, required traits and preferred traits and that those decomposed parts are what should be passed to the Compute API, not a flavor ID. But again, we're actively changing all this code in the resource providers and qualitative traits patches so I warned about adding more code that was essentially just a short-lived hack. I'd be OK adding the hack code if there were some big bright WARNINGs placed in there that likely the code would be removed in Ocata. While : #1 the first change about setting root_gb equals 0 in RequestSpec for making sure BFV instances are correctly using the DiskFilter is fine by me having it merged with a TODO/FIXME saying that the code would be expired once the scheduler uses the resource providers, #2, then the second patch about trying to look at the BDMs for DiskFilter is very wrong by me, because the Compute API shouldn't accept IMHO to ask for a flavor *and* a BDM with a related disk size different from the flavor. AFAIT, we should return a 40x (probably 409 or 400) for that instead of accepting it silently. Well, a flavor is always required when launching an instance. I wasn't aware until recently that one could "override" the root GB (or eph/swap) sizes in the API. Apparently, the API allows it, though, even though the code ignores whatever was passed as the override value. If the API supports it, I think the code should probably be changed to override the size values to whatever the user entered, no? I'm very sad to see this possible. FWIW, I think that the flavor should always be the source of truth for knowing the asked disk size (if no, imagine all the conditionals in code...) and we should somehow reconcile what the flavor provided and what the user explicitly asked for. Having a possibility to trample the flavor size seems to be a very bad UX (I guess most of the operators don't think about that when understanding why this instance could have a different size from the related flavor) hence me thinking we should rather discuss on a possible new microversion for returning a 400 when both sizes are not identical. The other option I could see could be to have the nested flavor in the RequestSpec be reconciling those two objects being different from the user-provided flavor (eg. say a flavor with swap=10G and a BDM with swap=1G, then RequestSpec.flavor would have swap=1G) but given we don't expose the RequestSpec objects on the API level, that still leaves operators possibly confused. -Sylvain -jay __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Reconciling flavors and block device mappings
On 08/29/2016 05:11 AM, Sylvain Bauza wrote: Le 29/08/2016 13:25, Jay Pipes a écrit : On 08/26/2016 09:20 AM, Ed Leafe wrote: On Aug 25, 2016, at 3:19 PM, Andrew Laskiwrote: One other thing to note is that while a flavor constrains how much local disk is used it does not constrain volume size at all. So a user can specify an ephemeral/swap disk <= to what the flavor provides but can have an arbitrary sized root disk if it's a remote volume. This kind of goes to the heart of the argument against flavors being the sole source of truth for a request. As cloud evolves, we keep packing more and more stuff into a concept that was originally meant to only divide up resources that came bundled together (CPU, RAM, and local disk). This hasn’t been a good solution for years, and the sooner we start accepting that a request can be much more complex than a flavor can adequately express, the better. If we have decided that remote volumes are a good thing (I don’t think there’s any argument there), then we should treat that part of the request as being as fundamental as a flavor. We need to make the scheduler smarter so that it doesn’t rely on flavor as being the only source of truth. The first step to improving Nova is admitting we have a problem. :) FWIW, I agree with you on the above. The issue I had with the proposed patches was that they would essentially be a hack for a short period of time until the resource providers work standardized the way that DISK_GB resources were tracked -- including for providers of shared disk storage. I've long felt that flavors as a concept should be, as Chris so adeptly wrote, "UI furniture" and should be decomposed into their requisite lists of resource amounts, required traits and preferred traits and that those decomposed parts are what should be passed to the Compute API, not a flavor ID. But again, we're actively changing all this code in the resource providers and qualitative traits patches so I warned about adding more code that was essentially just a short-lived hack. I'd be OK adding the hack code if there were some big bright WARNINGs placed in there that likely the code would be removed in Ocata. While : #1 the first change about setting root_gb equals 0 in RequestSpec for making sure BFV instances are correctly using the DiskFilter is fine by me having it merged with a TODO/FIXME saying that the code would be expired once the scheduler uses the resource providers, #2, then the second patch about trying to look at the BDMs for DiskFilter is very wrong by me, because the Compute API shouldn't accept IMHO to ask for a flavor *and* a BDM with a related disk size different from the flavor. AFAIT, we should return a 40x (probably 409 or 400) for that instead of accepting it silently. Well, a flavor is always required when launching an instance. I wasn't aware until recently that one could "override" the root GB (or eph/swap) sizes in the API. Apparently, the API allows it, though, even though the code ignores whatever was passed as the override value. If the API supports it, I think the code should probably be changed to override the size values to whatever the user entered, no? -jay __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Reconciling flavors and block device mappings
Le 29/08/2016 13:25, Jay Pipes a écrit : On 08/26/2016 09:20 AM, Ed Leafe wrote: On Aug 25, 2016, at 3:19 PM, Andrew Laskiwrote: One other thing to note is that while a flavor constrains how much local disk is used it does not constrain volume size at all. So a user can specify an ephemeral/swap disk <= to what the flavor provides but can have an arbitrary sized root disk if it's a remote volume. This kind of goes to the heart of the argument against flavors being the sole source of truth for a request. As cloud evolves, we keep packing more and more stuff into a concept that was originally meant to only divide up resources that came bundled together (CPU, RAM, and local disk). This hasn’t been a good solution for years, and the sooner we start accepting that a request can be much more complex than a flavor can adequately express, the better. If we have decided that remote volumes are a good thing (I don’t think there’s any argument there), then we should treat that part of the request as being as fundamental as a flavor. We need to make the scheduler smarter so that it doesn’t rely on flavor as being the only source of truth. The first step to improving Nova is admitting we have a problem. :) FWIW, I agree with you on the above. The issue I had with the proposed patches was that they would essentially be a hack for a short period of time until the resource providers work standardized the way that DISK_GB resources were tracked -- including for providers of shared disk storage. I've long felt that flavors as a concept should be, as Chris so adeptly wrote, "UI furniture" and should be decomposed into their requisite lists of resource amounts, required traits and preferred traits and that those decomposed parts are what should be passed to the Compute API, not a flavor ID. But again, we're actively changing all this code in the resource providers and qualitative traits patches so I warned about adding more code that was essentially just a short-lived hack. I'd be OK adding the hack code if there were some big bright WARNINGs placed in there that likely the code would be removed in Ocata. While : #1 the first change about setting root_gb equals 0 in RequestSpec for making sure BFV instances are correctly using the DiskFilter is fine by me having it merged with a TODO/FIXME saying that the code would be expired once the scheduler uses the resource providers, #2, then the second patch about trying to look at the BDMs for DiskFilter is very wrong by me, because the Compute API shouldn't accept IMHO to ask for a flavor *and* a BDM with a related disk size different from the flavor. AFAIT, we should return a 40x (probably 409 or 400) for that instead of accepting it silently. -Sylvain -jay __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Reconciling flavors and block device mappings
On 08/26/2016 09:20 AM, Ed Leafe wrote: On Aug 25, 2016, at 3:19 PM, Andrew Laskiwrote: One other thing to note is that while a flavor constrains how much local disk is used it does not constrain volume size at all. So a user can specify an ephemeral/swap disk <= to what the flavor provides but can have an arbitrary sized root disk if it's a remote volume. This kind of goes to the heart of the argument against flavors being the sole source of truth for a request. As cloud evolves, we keep packing more and more stuff into a concept that was originally meant to only divide up resources that came bundled together (CPU, RAM, and local disk). This hasn’t been a good solution for years, and the sooner we start accepting that a request can be much more complex than a flavor can adequately express, the better. If we have decided that remote volumes are a good thing (I don’t think there’s any argument there), then we should treat that part of the request as being as fundamental as a flavor. We need to make the scheduler smarter so that it doesn’t rely on flavor as being the only source of truth. The first step to improving Nova is admitting we have a problem. :) FWIW, I agree with you on the above. The issue I had with the proposed patches was that they would essentially be a hack for a short period of time until the resource providers work standardized the way that DISK_GB resources were tracked -- including for providers of shared disk storage. I've long felt that flavors as a concept should be, as Chris so adeptly wrote, "UI furniture" and should be decomposed into their requisite lists of resource amounts, required traits and preferred traits and that those decomposed parts are what should be passed to the Compute API, not a flavor ID. But again, we're actively changing all this code in the resource providers and qualitative traits patches so I warned about adding more code that was essentially just a short-lived hack. I'd be OK adding the hack code if there were some big bright WARNINGs placed in there that likely the code would be removed in Ocata. -jay __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Reconciling flavors and block device mappings
Le 26/08/2016 12:33, Chris Dent a écrit : On Thu, 25 Aug 2016, Sylvain Bauza wrote: Of course, long-term, we could try to see how to have composite flavors for helping users to not create a whole handful of flavors for quite the same user requests, but that would still be flavors (or the name for saying a flavor composition). long-term flavors should be a piece of UI furniture that is present in a human-oriented-non-nova UI/API that provides raw information to the computers-talking-to-computers API that is provided by nova. But that's very long term. Here, I didn't wanted to discuss on the long-term strategy about what could be a "composite" flavor (even if I tend to agree with you on the above) but rather explaining that "flavor" (aka. the concept for user-provided piece of information self-defining the request constraints) should be kept as the only source of truth. TBH, I very much dislike the fact that we can at the API level set a very different BDM size from the one the flavor gave (for the same volume type). In CLI, that's even worst, we just consider "ephemeral" and "swap" as being things totally unrelated to a flavor *facepalm* __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Reconciling flavors and block device mappings
From: John Griffith [mailto:john.griffi...@gmail.com] Sent: Friday, August 26, 2016 10:21 PM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [Nova] Reconciling flavors and block device mappings On Fri, Aug 26, 2016 at 10:20 AM, Ed Leafe <e...@leafe.com<mailto:e...@leafe.com>> wrote: On Aug 25, 2016, at 3:19 PM, Andrew Laski <and...@lascii.com<mailto:and...@lascii.com>> wrote: > One other thing to note is that while a flavor constrains how much local > disk is used it does not constrain volume size at all. So a user can > specify an ephemeral/swap disk <= to what the flavor provides but can > have an arbitrary sized root disk if it's a remote volume. > This kind of goes to the heart of the argument against flavors being the sole > source of truth for a request. As cloud evolves, we keep packing more and > more stuff into a concept that was originally meant to only divide up > resources that came bundled together (CPU, RAM, and local disk). This hasn’t > been a good solution for years, and the sooner we start accepting that a > request can be much more complex than a flavor can adequately express, the > better. > If we have decided that remote volumes are a good thing (I don’t think > there’s any argument there), then we should treat that part of the request as > being as fundamental as a flavor. We need to make the scheduler smarter so > that it doesn’t rely on > flavor as being the only source of truth. > +1 We have done extensive testing with patch [1] and ensured that it’s not breaking anything. IMO this patch should be the best solution till now and there should not be any issues in accepting the same. Please review the patch and provide your opinions so that we can take appropriate actions to get this resolved. [1] https://review.openstack.org/#/c/200870/ Thank you, Abhishek Kekane The first step to improving Nova is admitting we have a problem. :) -- Ed Leafe __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe<http://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ Disclaimer: This email and any attachments are sent in strictest confidence for the sole use of the addressee and may contain legally privileged, confidential, and proprietary data. If you are not the intended recipient, please advise the sender by replying promptly to this email and then delete and destroy this email and any attachments without any further use, copying or forwarding. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Reconciling flavors and block device mappings
On Fri, Aug 26, 2016 at 10:20 AM, Ed Leafewrote: > On Aug 25, 2016, at 3:19 PM, Andrew Laski wrote: > > > One other thing to note is that while a flavor constrains how much local > > disk is used it does not constrain volume size at all. So a user can > > specify an ephemeral/swap disk <= to what the flavor provides but can > > have an arbitrary sized root disk if it's a remote volume. > > This kind of goes to the heart of the argument against flavors being the > sole source of truth for a request. As cloud evolves, we keep packing more > and more stuff into a concept that was originally meant to only divide up > resources that came bundled together (CPU, RAM, and local disk). This > hasn’t been a good solution for years, and the sooner we start accepting > that a request can be much more complex than a flavor can adequately > express, the better. > > If we have decided that remote volumes are a good thing (I don’t think > there’s any argument there), then we should treat that part of the request > as being as fundamental as a flavor. We need to make the scheduler smarter > so that it doesn’t rely on flavor as being the only source of truth. > +1 > > The first step to improving Nova is admitting we have a problem. :) > > > -- Ed Leafe > > > > > > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Reconciling flavors and block device mappings
On Aug 25, 2016, at 3:19 PM, Andrew Laskiwrote: > One other thing to note is that while a flavor constrains how much local > disk is used it does not constrain volume size at all. So a user can > specify an ephemeral/swap disk <= to what the flavor provides but can > have an arbitrary sized root disk if it's a remote volume. This kind of goes to the heart of the argument against flavors being the sole source of truth for a request. As cloud evolves, we keep packing more and more stuff into a concept that was originally meant to only divide up resources that came bundled together (CPU, RAM, and local disk). This hasn’t been a good solution for years, and the sooner we start accepting that a request can be much more complex than a flavor can adequately express, the better. If we have decided that remote volumes are a good thing (I don’t think there’s any argument there), then we should treat that part of the request as being as fundamental as a flavor. We need to make the scheduler smarter so that it doesn’t rely on flavor as being the only source of truth. The first step to improving Nova is admitting we have a problem. :) -- Ed Leafe __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Reconciling flavors and block device mappings
On Thu, 25 Aug 2016, Sylvain Bauza wrote: Of course, long-term, we could try to see how to have composite flavors for helping users to not create a whole handful of flavors for quite the same user requests, but that would still be flavors (or the name for saying a flavor composition). long-term flavors should be a piece of UI furniture that is present in a human-oriented-non-nova UI/API that provides raw information to the computers-talking-to-computers API that is provided by nova. But that's very long term. -- Chris Dent ┬─┬ノ( º _ ºノ)https://anticdent.org/ freenode: cdent tw: @anticdent__ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Reconciling flavors and block device mappings
On Thu, 25 Aug 2016, Andrew Laski wrote: Allow block device mappings, when present, to mostly determine instance packing. By that I mean that the scheduler only takes into account local disk that would be consumed, but we add additional configuration to Nova which limits the number of instance that can be placed on a host. This is a compromise solution but I fear that a single int value does not meet the needs of deployers wishing to limit instances on a host. They want it to take into account cpu allocations and ram and disk, in short a flavor :) When you say "add additional configuration" do you mean "add more things to nova.conf"? If so, then please don't do that. There is far too much of that. -- Chris Dent ┬─┬ノ( º _ ºノ)https://anticdent.org/ freenode: cdent tw: @anticdent__ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Reconciling flavors and block device mappings
Le 25/08/2016 16:19, Andrew Laski a écrit : Cross posting to gather some operator feedback. There have been a couple of contentious patches gathering attention recently about how to handle the case where a block device mapping supersedes flavor information. Before moving forward on either of those I think we should have a discussion about how best to handle the general case, and how to handle any changes in behavior that results from that. There are two cases presented: 1. A user boots an instance using a Cinder volume as a root disk, however the flavor specifies root_gb = x where x > 0. The current behavior in Nova is that the scheduler is given the flavor root_gb info to take into account during scheduling. This may disqualify some hosts from receiving the instance even though that disk space is not necessary because the root disk is a remote volume. https://review.openstack.org/#/c/200870/ 2. A user boots an instance and uses the block device mapping parameters to specify a swap or ephemeral disk size that is less than specified on the flavor. This leads to the same problem as above, the scheduler is provided information that doesn't match the actual disk space to be consumed. https://review.openstack.org/#/c/352522/ Now the issue: while it's easy enough to provide proper information to the scheduler on what the actual disk consumption will be when using block device mappings that undermines one of the purposes of flavors which is to control instance packing on hosts. So the outstanding question is to what extent should users have the ability to use block device mappings to bypass flavor constraints? One other thing to note is that while a flavor constrains how much local disk is used it does not constrain volume size at all. So a user can specify an ephemeral/swap disk <= to what the flavor provides but can have an arbitrary sized root disk if it's a remote volume. Some possibilities: Completely allow block device mappings, when present, to determine instance packing. This is what the patches above propose and there's a strong desire for this behavior from some folks. But changes how many instances may fit on a host which could be undesirable to some. That would completely (as you mentioned) tramples the fact that Nova uses flavors as quantitative resource user queries and would create some kind of conditional whether we should check if a BDM is there and also overriding the flavor disk values. Please, I think we should only have one single source of truth for knowing the user disk request, which are flavors. Of course, long-term, we could try to see how to have composite flavors for helping users to not create a whole handful of flavors for quite the same user requests, but that would still be flavors (or the name for saying a flavor composition). Keep the status quo. It's clear that is undesirable based on the bug reports and proposed patches above. The status quo is not good as well. Given that we contract on BDM sizes in the API, we should somehow respect that contract and either accept the request (and honor it) or refuse it gracefully (for example, say if a flavor swap value doesn't match the swap BDM size you asked for) Allow block device mappings, when present, to mostly determine instance packing. By that I mean that the scheduler only takes into account local disk that would be consumed, but we add additional configuration to Nova which limits the number of instance that can be placed on a host. This is a compromise solution but I fear that a single int value does not meet the needs of deployers wishing to limit instances on a host. They want it to take into account cpu allocations and ram and disk, in short a flavor :) If we consider that a flavor is the only source of truth, it means that another possibility would be to say that when an user requests both a flavor and a BDM, we would need to reconcile those two into one single flavor that would be part of the RequestSpec object. That wouldn't be the flavor the user asked, sure, but we would respect the quantitative resource values he wanted. -Sylvain And of course there may be some other unconsidered solution. That's where you, dear reader, come in. Thoughts? -Andrew __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev