Hi Meg, Let's take it to a separate thread. I will explain the overcommit aspects to you
--- Regards, Dmitriy On Tue, Jun 24, 2014 at 11:00 PM, Meg McRoberts <[email protected]> wrote: > How does one know that ones overcommit ratio is too high? Are there > monitoring tools > that would show CPU utilization for all Compute nodes in the environment? > And, if one > set the RAM overcommit rate to something other than 1:1, would it be easy > to tell whether > one was CPU or memory-bound? > > I suppose if users were complaining about performance, this would be a > good place to look, > but this is certainly not the only factor that could affect VM > performance... > > > On Tue, Jun 24, 2014 at 12:49 PM, Dmitriy Novakovskiy < > [email protected]> wrote: > >> David, >> >> *I created an etherpad with the additional requirements >>> (https://etherpad.openstack.org/p/cpu-overcommit-setting >>> <https://etherpad.openstack.org/p/cpu-overcommit-setting>) but don’t have >>> rights to set it as the URL for the specification. If you or an admin >>> could do that, it’d be appreciated.* >> >> >> Done - added your etherpad as a full spec to BP: >> https://blueprints.launchpad.net/fuel/+spec/cpu-overcommit-setting >> >> *With these expectations, you would need to have the parameter be on the >>> Settings tab not (just) the Deployment Wizard. * >> >> >> Got it, makes sense. It should be on both screens. >> >> * Also, currently the only way to reactivate the Settings Tab today and >>> “re-apply” is to reset the environment – I.e. delete the existing >>> environment and re-deploy with the modified settings/parameters. That may >>> be a bit heavy handed for just changing the overcommit value, so we may >>> need a second blueprint that would enable settings to be changed from Fuel >>> without deleting and redeploying the environment… * >>> >> >> Got it. Well, IMO it calls for us to think through some key concepts, >> what Fuel should/should not be able to do and enforce... Ability to adjust >> CPU overcommit rate on post-deployment phases will be useful, but not >> pri-1. I just thought that UI+engine already supports it. >> >> --- >> Regards, >> Dmitriy >> >> >> On Tue, Jun 24, 2014 at 10:36 PM, David Easter <[email protected]> >> wrote: >> >>> Thanks. With these expectations, you would need to have the parameter >>> be on the Settings tab not (just) the Deployment Wizard. Having it only in >>> the deployment wizard would mean you couldn’t modify it later and that it >>> wouldn’t be reflected in the UI. Also, currently the only way to >>> reactivate the Settings Tab today and “re-apply” is to reset the >>> environment – I.e. delete the existing environment and re-deploy with the >>> modified settings/parameters. That may be a bit heavy handed for just >>> changing the overcommit value, so we may need a second blueprint that would >>> enable settings to be changed from Fuel without deleting and redeploying >>> the environment… >>> >>> I created an etherpad with the additional requirements ( >>> https://etherpad.openstack.org/p/cpu-overcommit-setting) but don’t have >>> rights to set it as the URL for the specification. If you or an admin >>> could do that, it’d be appreciated. >>> >>> Thanks, >>> >>> - David J. Easter >>> Director of Product Management, Mirantis >>> >>> From: Dmitriy Novakovskiy <[email protected]> >>> Date: Tuesday, June 24, 2014 at 11:54 AM >>> To: David Easter <[email protected]> >>> Cc: Bogdan Dobrelya <[email protected]>, fuel-dev < >>> [email protected]>, Sergii Golovatiuk < >>> [email protected]> >>> >>> Subject: Re: [Fuel-dev] overcommit rates >>> >>> David, >>> >>> 1. Yes >>> 2. Yes, same as other Settings which Fuel can "re-apply" today (syslog >>> server, scheduler type, etc). As Sergey pointed out, there's no danger in >>> it, and there should not be any additional technical effort to implement. >>> The only thing is - we need to add a note on UI and/or in documentation to >>> explain to user that changing overcommit rate to lower will not >>> automatically offload Compute nodes that are already over-provisioned, >>> he/she will still need to migrate excessive VMs away:) >>> 3. No, no monitoring. Fuel master should be the only source of "truth" >>> for those settings which are exposed to user on Web UI. >>> >>> >>> --- >>> Regards, >>> Dmitriy >>> >>> >>> On Tue, Jun 24, 2014 at 8:35 PM, Sergii Golovatiuk < >>> [email protected]> wrote: >>> >>>> Hi, >>>> >>>> IMHO, >>>> >>>> 1. Yes >>>> 2. In theory, it's possible to change/update values after deployment >>>> time. It's not a dangerous operation. If node is full, it won't be selected >>>> for new VM starts. >>>> 3. If it's changed manually, it will be reset on next puppet run. >>>> >>>> ~Sergii >>>> >>>> >>>> On Tue, Jun 24, 2014 at 5:21 PM, David Easter <[email protected]> >>>> wrote: >>>> >>>>> Dmitriy, >>>>> >>>>> Once set through the UI, I have a few questions on expectations >>>>> post-deployment: >>>>> >>>>> 1. Would the value selected need to be shown within the Fuel UI >>>>> post-deployment to let the administrator know what value was selected >>>>> per >>>>> environment? >>>>> 2. Is there an expectation that the value can be changed in a >>>>> running environment, or would this only be expected at deployment time? >>>>> 3. If the value were changed in the environment manually, would be >>>>> expected that the Fuel Master Node would need to monitor for such >>>>> changes? >>>>> >>>>> Thanks, >>>>> >>>>> - David J. Easter >>>>> Director of Product Management, Mirantis >>>>> >>>>> From: Dmitriy Novakovskiy <[email protected]> >>>>> Date: Tuesday, June 24, 2014 at 4:37 AM >>>>> To: Bogdan Dobrelya <[email protected]> >>>>> Cc: fuel-dev <[email protected]> >>>>> Subject: Re: [Fuel-dev] overcommit rates >>>>> >>>>> All, >>>>> >>>>> I've submitted a blueprint on this: >>>>> https://blueprints.launchpad.net/fuel/+spec/cpu-overcommit-setting >>>>> >>>>> Bogdan, >>>>> >>>>> Nice and creative approach, but IMO the question of "how many >>>>> instances to run" and "what overcommit to use" belongs more to >>>>> planning/architecture workshop/BOM calculator >>>>> <http://www.mirantis.com/openstack-services/bom-calculator/> phase. >>>>> Not only amount of VMs to run on given amount of Compute nodes counts here >>>>> - also the capability of hypervisor to "juggle" oversubscribed cores (poor >>>>> in KVM, better in ESXi), the kind of environment (Prod, Dev/Test), the >>>>> level of tolerance to VM failure, etc. Fuel just need to prompt+allow user >>>>> to apply decisions made at planning phase to deployment phase. >>>>> >>>>> >>>>> >>>>> --- >>>>> Regards, >>>>> Dmitriy >>>>> >>>>> >>>>> On Tue, Jun 24, 2014 at 2:22 PM, Bogdan Dobrelya < >>>>> [email protected]> wrote: >>>>> >>>>>> On 06/24/2014 01:50 PM, Dmitriy Novakovskiy wrote: >>>>>> > Andrey, Dmitriy >>>>>> > >>>>>> > I've just realized that CPU overcommitment setting is defined not >>>>>> for >>>>>> > Controllers but for Compute nodes, and may differ on per-node >>>>>> basis. So, >>>>>> > the 2nd "harm" example that I exposed a few emails ago (with failed >>>>>> and >>>>>> > redeployed controller resulting in inconsistent overcommitment >>>>>> behavior) >>>>>> > is irrelevant. >>>>>> > >>>>>> > With this in mind, 3rd example is even worse :) >>>>>> > >>>>>> > 3) User goes through architecture workshop and capacity planning >>>>>> > exercise, decides on CPU overcommit rates he/she will use for >>>>>> OpenStack >>>>>> > cloud. User deploys the cloud with, let's say, 20+ compute nodes. >>>>>> Now, >>>>>> > w/o CPU overcommit setting on GUI, User will: >>>>>> > - get a cloud that's inconsistent with decisions made during >>>>>> planning >>>>>> > - be not aware of this - GUI did not point user to a need of >>>>>> specifying >>>>>> > this setting >>>>>> > - be frustrated and put under risk - if the cloud is deployed with >>>>>> real >>>>>> > workloads the compute nodes will soon become over provisioned and >>>>>> > unreliable due to default Fuel setting (prod KVM clouds often use >>>>>> > super-soft overcommit setting, or no overcommit at all) >>>>>> > - finally - *user will have to MANUALLY specify and maintain >>>>>> overcommit >>>>>> > setting across 20+ compute nodes.* >>>>>> > * >>>>>> > * >>>>>> > Doesn't sound like a good UX from cloud lifecycle management app :) >>>>>> - >>>>>> > Fuel will quickly be ditched in favor of config management tool. >>>>>> > >>>>>> > So, my proposal on CPU overcommitment setting is following: >>>>>> > >>>>>> > - Expose it in Wizard on hypervisor selection screen >>>>>> > - To simplify things initially - implement the setting as a >>>>>> drop-down >>>>>> > list, with 2 options: >>>>>> > -- No CPU overcommitment (1:1) >>>>>> > -- Light CPU overcommitment (3:1) >>>>>> > -- Heavy CPU overcommitment (6:1) --- default setting in BOM >>>>>> calculator >>>>>> >>>>>> Good point. We could as well evaluate the suggested ratio based on the >>>>>> user outputs and roles allocation, e.g.: >>>>>> 1) Q(wizard):"How many running instances your cloud has to maintain?" >>>>>> -> >>>>>> user specifies 10000 >>>>>> 2) and there are 100 compute nodes assigned in UI >>>>>> >>>>>> then, >>>>>> - the CPU overcommitment should be evaluated as 10000/100 : 1 >>>>>> - if the value exceeds the default maximum (16) for OSt, the warning >>>>>> should be issued in UI notifications (please provide more compute >>>>>> nodes >>>>>> in order to maintain the desired capacity for running instances) >>>>>> >>>>>> > >>>>>> > >>>>>> > --- >>>>>> > Regards, >>>>>> > Dmitriy >>>>>> > >>>>>> > >>>>>> > On Tue, Jun 24, 2014 at 1:23 PM, Andrey Danin <[email protected] >>>>>> > <mailto:[email protected]>> wrote: >>>>>> > >>>>>> > I think, we can add to openstack.yaml a special tag "advanced" >>>>>> to >>>>>> > every advanced parameter, and UI can hide all such parameters >>>>>> for >>>>>> > every group under the button "show more options". Also we can >>>>>> add >>>>>> > the "expand all hidden options" button at the top of the tab. I >>>>>> > don't think it's hard to implement. >>>>>> > >>>>>> > Let's wait an opinion of UI team. >>>>>> > >>>>>> > On Jun 24, 2014 1:58 PM, "Meg McRoberts" < >>>>>> [email protected] >>>>>> > <mailto:[email protected]>> wrote: >>>>>> > >>>>>> > Jesse and others make a very good point. The Settings >>>>>> screen >>>>>> > has already become a real mess >>>>>> > that needs to be split up and organized a bit. I'm told >>>>>> that >>>>>> > the serious restructuring should come >>>>>> > after we implement the plug-ins scheme (currently planned >>>>>> for >>>>>> > 6.0). And each section definitely >>>>>> > needs an "Advanced" section for configurations that aren't >>>>>> routine. >>>>>> > >>>>>> > So do we wait to restructure the "Settings" screen until the >>>>>> > plug-ins scheme is implemented? I >>>>>> > think breaking it up into sub-sections would make it more >>>>>> usable >>>>>> > in the meantime but I don't know >>>>>> > if it's worth the effort to do that ahead of the plug-ins. >>>>>> > >>>>>> > >>>>>> > On Tue, Jun 24, 2014 at 2:23 AM, Jesse Pretorius >>>>>> > <[email protected] <mailto: >>>>>> [email protected]>> >>>>>> > wrote: >>>>>> > >>>>>> > On 23 June 2014 23:02, Dmitry Borodaenko >>>>>> > <[email protected] <mailto: >>>>>> [email protected]>> >>>>>> > wrote: >>>>>> > >>>>>> > Adding a configuration variable that can easily be >>>>>> > changed after >>>>>> > deployment is completed into Fuel is a slippery >>>>>> slope, >>>>>> > you risk >>>>>> > spamming the already overcrowded Fuel settings page >>>>>> with >>>>>> > dozens of >>>>>> > trivial (and not so trivial) options from every >>>>>> > OpenStack component. >>>>>> > >>>>>> > >>>>>> > While I agree, the fact of the matter is that Fuel >>>>>> > facilitates the deployment of configuration to all >>>>>> nodes in >>>>>> > the environment in a consistent manner. This is >>>>>> essential to >>>>>> > deploy a reliable and supportable environment. >>>>>> > >>>>>> > In order to maintain the simplicity of the UI, but also >>>>>> > provide access to OpenStack's customisable >>>>>> configuration, >>>>>> > why not do something like add an 'advanced' button into >>>>>> the >>>>>> > UI. Under there, the additional options can be >>>>>> provided, and >>>>>> > things that are already 'Advanced' (like the scheduler, >>>>>> the >>>>>> > Ceph replication factor, etc) can move under the >>>>>> 'Advanced' >>>>>> > button for each section. >>>>>> > >>>>>> > An additional very beneficial feature in the Advanced >>>>>> > section would be to allow any arbitrary key-value pairs >>>>>> for >>>>>> > any settings that a user may wish to deploy into the >>>>>> > configuration files for the appropriate system (eg: >>>>>> > nova.conf). The user can be warned that these will not >>>>>> be >>>>>> > validated and that they are used at the user's own risk. >>>>>> > This can allow any number of OpenStack underlying >>>>>> features >>>>>> > to be used without requiring a specific UI for it. >>>>>> > >>>>>> > -- >>>>>> > Mailing list: https://launchpad.net/~fuel-dev >>>>>> > Post to : [email protected] >>>>>> > <mailto:[email protected]> >>>>>> > Unsubscribe : https://launchpad.net/~fuel-dev >>>>>> > More help : https://help.launchpad.net/ListHelp >>>>>> > >>>>>> > >>>>>> > >>>>>> > -- >>>>>> > Mailing list: https://launchpad.net/~fuel-dev >>>>>> > Post to : [email protected] >>>>>> > <mailto:[email protected]> >>>>>> > Unsubscribe : https://launchpad.net/~fuel-dev >>>>>> > More help : https://help.launchpad.net/ListHelp >>>>>> > >>>>>> > >>>>>> > -- >>>>>> > Mailing list: https://launchpad.net/~fuel-dev >>>>>> > Post to : [email protected] >>>>>> > <mailto:[email protected]> >>>>>> > Unsubscribe : https://launchpad.net/~fuel-dev >>>>>> > More help : https://help.launchpad.net/ListHelp >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> >>>>>> >>>>>> -- >>>>>> Best regards, >>>>>> Bogdan Dobrelya, >>>>>> Skype #bogdando_at_yahoo.com >>>>>> Irc #bogdando >>>>>> >>>>> >>>>> -- Mailing list: https://launchpad.net/~fuel-dev Post to : >>>>> [email protected] Unsubscribe : >>>>> https://launchpad.net/~fuel-dev More help : >>>>> https://help.launchpad.net/ListHelp >>>>> >>>>> -- >>>>> Mailing list: https://launchpad.net/~fuel-dev >>>>> Post to : [email protected] >>>>> Unsubscribe : https://launchpad.net/~fuel-dev >>>>> More help : https://help.launchpad.net/ListHelp >>>>> >>>>> >>>> >>> >> >> -- >> Mailing list: https://launchpad.net/~fuel-dev >> Post to : [email protected] >> Unsubscribe : https://launchpad.net/~fuel-dev >> More help : https://help.launchpad.net/ListHelp >> >> >
-- Mailing list: https://launchpad.net/~fuel-dev Post to : [email protected] Unsubscribe : https://launchpad.net/~fuel-dev More help : https://help.launchpad.net/ListHelp
