Re: [openstack-dev] [TripleO] config options, defaults, oh my!
Clint Byrum said on Thu, Apr 10, 2014 at 09:45:17AM -0700: Now you've described it, you're right, I'm not interested in TripleO or TripleO milestones. I am interested in using os-*-config, Heat and tripleo-image-elements to produce pure OpenStack deployments from 3 to 3000 nodes, for varying workloads. That is precisely what TripleO wants as that first milestone too. What is the difference between what I said we want (a default OpenStack deployment) and what you just said (a pure OpenStack deployment)? At what point did anybody suggest to you that TripleO doesn't want a highly scalable deployment of OpenStack? I'm not sure why you wouldn't just put the configuration you need in the upstream TripleO as the default configuration. Even more, why wouldn't you put these configurations in as the defaults for upstream Nova/Glance/Cinder/Neutron/etc? The trouble is there can only be one default configuration, whereas there are many potential kinds of deployment. If the default deployment is highly scalable, it will need a minimum number of nodes to start up. EG a highly scalable Logstash infrastructure requires at least 8 nodes to itself before you add any compute nodes. This won't be useful to those who want to start with a trial deployment and scale up from there. I like your Heat idea very much, where the user can easily supply whole files or filesets. This allows the user to choose: 1) What software is installed on which nodes, by element composition 2) The contents of configuration files, by Heat passthrough of a fileset 3) Per-instance variables plus a few tweak-prone knobs, via Heat parameters Thus we don't have to enshrine a single configuration or add a Heat parameter for every single config option. And TripleO can publish at first one but later several default Openstack deployments, each suitable for a different scale. Alexis -- Nova Engineer, HP Cloud. AKA lealexis, lxsli. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [TripleO] config options, defaults, oh my!
Clint Byrum said on Wed, Apr 09, 2014 at 12:05:35PM -0700: Excerpts from Duncan Thomas's message of 2014-04-09 11:11:06 -0700: I think this is dangerous thinking - the config you want depends so hugely on your intended workload and available hardware that trying any strong view of what an Openstack deployment should look like into the deployment tool forever forces that deployment tool to be a minor, niche product that *has* to be replaced by something more expressive in order to be widely usable. The config you want for a primary hadoop shop is totally different to what you'd want for primary web-host shop is somewhat different to what you'd want for a public/generic cloud, etc. Things like AZ support, neutron model, cinder back-end choice, H/A model etc are dictated by scale and use-cases. If you only want your config tool to deal with one deployment type, that tool becomes pretty much irrelevant to the totality of the Openstack effort, and should be replaced by something more layered/openminded. +1 to Duncan. While I can understand the urge to get something simple working first before tackling more complex usecases, it's pointless if in doing so you make those usecases even tougher. I can certainly understand how one might mistake TripleO as a deployment tool. It is no such thing. OpenStack is the deployment suite, with the tools being Nova, Glance, Neutron, Heat, diskimage-builder, os-*-config, etc. TripleO is a _program_, in the sense of an effort to gather collaborative forces, to deploy OpenStack using these tools. Now you've described it, you're right, I'm not interested in TripleO or TripleO milestones. I am interested in using os-*-config, Heat and tripleo-image-elements to produce pure OpenStack deployments from 3 to 3000 nodes, for varying workloads. To achieve this end I need tools which offer complete control of the system configuration. I'm very happy to work with TripleO to enhance our shared tools but if in pursuit of TripleO milestones those tools are hobbled, it's a problem. Alexis -- Nova Engineer, HP Cloud. AKA lealexis, lxsli. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [TripleO] config options, defaults, oh my!
Clint Byrum said on Wed, Apr 09, 2014 at 01:33:18PM -0700: This assumes that we don't want system integrators to contribute to TripleO, which is the opposite of how things are. We absolutely do, and in fact, that is part of the point of having a program around OpenStack deployment. Let's get system integrators into OpenStack's CI system and let's get a few of the most important scenarios into the gate of OpenStack. As a system integrator, do you want to say to your customers that you start with an unusable set of tools that the community tests individually, or do you want to say that you start with the deployment that the community tests directly on every commit, and then enhance based on individual customer need? I absolutely do want system integrators to contribute to TripleO. As you've described the TripleO mission, its entire reason to be is to provide an integrated OpenStack system. At the same time, I'd like those contributions not to be muddled in with the core tooling. This separation makes the base elements far easier to write and also means we can iterate on the TripleO configuration without necessarily affecting third-party system integrators. For example: * I produce a 'logstash' element which simply installs + starts that service. The configuration is purely what's needed to make it start. * I add logstash.conf to the tripleo-compute-config and tripleo-controller-config elements, describing how Logstash should be configured in the TripleO default deployment. * This is tested by TripleO CI. * As a service provider, I decide I want a logstash cluster to improve scalability. I write my own logstash.conf files and put them in mycorp-compute-config and mycorp-controller-config elements. Then I rebuild images using both the tripleo-*-config and mycorp-*-config elements. This allows me to use the tested TripleO deployment but with my own Logstash configuration. Later if I choose I can reconfigure other parts of the system, using templates from the tripleo-*-config elements as a starting point. It is worth noting that Heat has grown the ability to grab a local file and inject it into your template at runtime. I think it would actually make sense to have os-apply-config enhanced to be able to override whole template files based on something like this: resources: server1: metadata: template_overrides: /etc/nova/nova.conf: get_file [ my_special_nova.conf.template ] In that, we achieve what you want, but we can do so without rebuilding the whole image. Yeah this is a good enhancement. It'd be more convenient and rule out a class of gotchas if you could provide a tarball or something rather than having to individually specify every file. If we did this then instead of producing tripleo-*-config elements in my example above, the TripleO default deployment configuration would take the form of two tarballs for Heat (compute + control). The source for these could be maintained in a separate repository, EG tripleo-configuration. Alexis -- Nova Engineer, HP Cloud. AKA lealexis, lxsli. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [TripleO] config options, defaults, oh my!
Miller, Mark M (EB SW Cloud - RD - Corvallis) said on Wed, Apr 09, 2014 at 10:30:26PM +: Does anyone have a flowchart of the cloud build/configure process including interactions between the various components/stages of TripleO and Heat? Not to my (very limited) knowledge. If you read the following, you should be off to a good start: * https://github.com/openstack/tripleo-incubator/blob/master/scripts/devtest_overcloud.sh * https://github.com/openstack/diskimage-builder/blob/master/README.md * https://github.com/openstack/diskimage-builder/blob/master/bin/disk-image-create * https://github.com/openstack/tripleo-image-elements/tree/master/elements/os-apply-config * https://github.com/openstack/os-refresh-config * https://github.com/openstack/os-apply-config * http://docs.openstack.org/developer/heat/template_guide/index.html So essentially: * An image is built from elements using diskimage-create * `heat stack-create` kicks off the whole process, using inputs EG overcloud.yaml and testenv.json. * (try `heat resource-metadata overcloud NotCompute0Config`, iirc Heat has evaluated functions by this time) * Once the instance starts, os-collect-config runs periodically inside + downloads the Heat metadata into a JSON file (/var/lib/os-collect-config/cfn.json iirc) * After o-c-c, os-refresh-config runs. At level 50 of configure.d, os-apply-config happens. The JSON is used to evaluate Mustache templates. Hope that helps! Alexis -- Nova Engineer, HP Cloud. AKA lealexis, lxsli. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [TripleO] config options, defaults, oh my!
Thank you for the leads. I will look them up. Mark -Original Message- From: Lee, Alexis Sent: Thursday, April 10, 2014 3:58 AM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [TripleO] config options, defaults, oh my! Miller, Mark M (EB SW Cloud - RD - Corvallis) said on Wed, Apr 09, 2014 at 10:30:26PM +: Does anyone have a flowchart of the cloud build/configure process including interactions between the various components/stages of TripleO and Heat? Not to my (very limited) knowledge. If you read the following, you should be off to a good start: * https://github.com/openstack/tripleo-incubator/blob/master/scripts/devtest_overcloud.sh * https://github.com/openstack/diskimage-builder/blob/master/README.md * https://github.com/openstack/diskimage-builder/blob/master/bin/disk-image-create * https://github.com/openstack/tripleo-image-elements/tree/master/elements/os-apply-config * https://github.com/openstack/os-refresh-config * https://github.com/openstack/os-apply-config * http://docs.openstack.org/developer/heat/template_guide/index.html So essentially: * An image is built from elements using diskimage-create * `heat stack-create` kicks off the whole process, using inputs EG overcloud.yaml and testenv.json. * (try `heat resource-metadata overcloud NotCompute0Config`, iirc Heat has evaluated functions by this time) * Once the instance starts, os-collect-config runs periodically inside + downloads the Heat metadata into a JSON file (/var/lib/os-collect-config/cfn.json iirc) * After o-c-c, os-refresh-config runs. At level 50 of configure.d, os-apply-config happens. The JSON is used to evaluate Mustache templates. Hope that helps! Alexis -- Nova Engineer, HP Cloud. AKA lealexis, lxsli. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [TripleO] config options, defaults, oh my!
Excerpts from Alexis Lee's message of 2014-04-10 02:35:32 -0700: Clint Byrum said on Wed, Apr 09, 2014 at 12:05:35PM -0700: Excerpts from Duncan Thomas's message of 2014-04-09 11:11:06 -0700: I think this is dangerous thinking - the config you want depends so hugely on your intended workload and available hardware that trying any strong view of what an Openstack deployment should look like into the deployment tool forever forces that deployment tool to be a minor, niche product that *has* to be replaced by something more expressive in order to be widely usable. The config you want for a primary hadoop shop is totally different to what you'd want for primary web-host shop is somewhat different to what you'd want for a public/generic cloud, etc. Things like AZ support, neutron model, cinder back-end choice, H/A model etc are dictated by scale and use-cases. If you only want your config tool to deal with one deployment type, that tool becomes pretty much irrelevant to the totality of the Openstack effort, and should be replaced by something more layered/openminded. +1 to Duncan. While I can understand the urge to get something simple working first before tackling more complex usecases, it's pointless if in doing so you make those usecases even tougher. I'm not sure I understand where that has actually happened. TripleO's target is massive clouds, because OpenStack is about massive clouds. As I said, this is not about a demo, it is about a real usable cloud. I can certainly understand how one might mistake TripleO as a deployment tool. It is no such thing. OpenStack is the deployment suite, with the tools being Nova, Glance, Neutron, Heat, diskimage-builder, os-*-config, etc. TripleO is a _program_, in the sense of an effort to gather collaborative forces, to deploy OpenStack using these tools. Now you've described it, you're right, I'm not interested in TripleO or TripleO milestones. I am interested in using os-*-config, Heat and tripleo-image-elements to produce pure OpenStack deployments from 3 to 3000 nodes, for varying workloads. That is precisely what TripleO wants as that first milestone too. What is the difference between what I said we want (a default OpenStack deployment) and what you just said (a pure OpenStack deployment)? At what point did anybody suggest to you that TripleO doesn't want a highly scalable deployment of OpenStack? To achieve this end I need tools which offer complete control of the system configuration. I'm very happy to work with TripleO to enhance our shared tools but if in pursuit of TripleO milestones those tools are hobbled, it's a problem. I'm not sure why you wouldn't just put the configuration you need in the upstream TripleO as the default configuration. Even more, why wouldn't you put these configurations in as the defaults for upstream Nova/Glance/Cinder/Neutron/etc? ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [TripleO] config options, defaults, oh my!
Robert Collins said on Wed, Apr 09, 2014 at 01:58:59AM +1200:
I like this - something like
nova:
config:
- section: default
values:
- option: 'compute_manager'
value: 'ironic.nova.compute.manager.ClusterComputeManager'
- section: cells
values:
- option: 'driver'
value: nova.cells.rpc_driver.CellsRPCDriver
should be able to represent most? all (it can handle repeating items)
oslo.config settings and render it easily:
{{#config}}
{{#comment}} repeats for each section {{/comment}}
[{{section}}]
{{#values}}
{{option}}={{value}}
{{/values}}
{{/config}}
Hello,
I've gone some distance down this road:
https://review.openstack.org/#/c/83353/6/elements/nova/os-apply-config/etc/nova/log.conf
https://review.openstack.org/#/c/83422/6/logstash-source.yaml
I wouldn't call the result - encoding a complete config file into Heat
metadata - pretty. And this isn't completely genericised either.
It'd be much better if TripleO image elements focused on installing and
starting services and allowed system integrators to define the
configuration. In one place, in plain text files, the UNIX way. I've
appended my proposal to Rob's etherpad here:
https://etherpad.openstack.org/p/tripleo-config-passthrough
Soon-to-be outdated copy appended here:
Hi Rob, I have some serious concerns about the above approaches. For the
sake of argument, let's suppose we want to write a file that looks like
a Heat template. How would you write a Mustache template that handles
that level of nesting? Even if you accomplish that, how readable do you
think the metadata to fill out that template would look?
I see the system integration process emerging like this:
* Figure out what files you want + what you want in them
* Slice and dice that into metadata
* Write some fairly complicated templates to reconstitute the metadata
* Get out more or less what you started with
I'd like to propose an alternative method where Heat and TripleO barely
touch the config. The system integrator writes an image element per
node-flavour, EG mycorp-compute-config. If they choose, they could
write more (EG for specific hardware) limited only by their
devtest-equivalent's ability to allocate those. This element contains a
99-os-apply-config directory, the templates from which overwrite any
templates from normal os-apply-config directories in other elements.
os-apply-config/install.d/99-install-config-templates will need to be
patched for this to be possible, but this is very little work in
comparison to the alternatives. I could also support simply an
os-apply-config.override directory, if a full numbered set of dirs seems
overkill, but in this case normal elements would have to be forbidden
from using it (and people being as they are, someone would). The
templates in that directory are 99% plain config files, laid out in a
single filesystem structure exactly as the system integrator wants them.
The only templated values which need to be supplied by Heat are those
which vary per-instance.
If we do this, tripleo-image-elements should focus on installing and
starting services. They should only include a minimal viable
configuration for demo purposes. This should greatly reduce the amount
of work required to produce a new element. Also the number of Heat
parameters used by any element (only per-instance would be necessary,
anything further is a convenience).
Some usecases where this approach is superior:
* Files where order is important, EG squid.conf
* Files which multiple elements want to touch, EG nova.conf
* When the system integrator wants to add config unforeseen by the
appropriate element or where using an element would be
heavyweight. EG to configure the MOTD or add a global vimrc.
* Easy to add hardware-specific configuration
Final thoughts - the mycorp-compute-config element might need to do a
bit of chmod + chown'ing as well as just providing 99-os-apply-config.
If OpenStack wants to provide a complete off-the-shelf configured
solution, we could provide a system integrator element which expresses
OpenStack opinion on what that solution should look like. In fact we
could provide several, suitable to different scales.
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [TripleO] config options, defaults, oh my!
On 8 April 2014 18:25, Clint Byrum cl...@fewbar.com wrote: Excerpts from Jay Dobies's message of 2014-04-08 06:40:07 -0700: I've always assumed TripleO is very low-level. Put another way, non-prescriptive. It's not going to push an agenda that says you should be doing things a certain way, but rather gives you more than enough rope to hang yourself (just makes it easier). And I've always looked at TripleO as the opposite. It is a default deployment of OpenStack. That is why we look at having to set a non-default option as a bug. That is why we only currently offer one set of Heat templates. Of course I want to see it more widely configurable, as I understand that there will be whole sections of the OpenStack interested user base that won't want an ovs overlay network. There will be shops that simply refuse to use MySQL, or want to put swift proxies on their own nodes, etc. etc. But if we can't all agree on a widely usable set of defaults, and deploy that, then I think OpenStack, not just TripleO, is forever going to be a framework on which proprietary solutions are built, rather than a whole open source platform. I think this is dangerous thinking - the config you want depends so hugely on your intended workload and available hardware that trying any strong view of what an Openstack deployment should look like into the deployment tool forever forces that deployment tool to be a minor, niche product that *has* to be replaced by something more expressive in order to be widely usable. The config you want for a primary hadoop shop is totally different to what you'd want for primary web-host shop is somewhat different to what you'd want for a public/generic cloud, etc. Things like AZ support, neutron model, cinder back-end choice, H/A model etc are dictated by scale and use-cases. If you only want your config tool to deal with one deployment type, that tool becomes pretty much irrelevant to the totality of the Openstack effort, and should be replaced by something more layered/openminded. This isn't to say we must boil the ocean right now and make everything available, but rather that decisions should take the long view into account. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [TripleO] config options, defaults, oh my!
Excerpts from Duncan Thomas's message of 2014-04-09 11:11:06 -0700: On 8 April 2014 18:25, Clint Byrum cl...@fewbar.com wrote: Excerpts from Jay Dobies's message of 2014-04-08 06:40:07 -0700: I've always assumed TripleO is very low-level. Put another way, non-prescriptive. It's not going to push an agenda that says you should be doing things a certain way, but rather gives you more than enough rope to hang yourself (just makes it easier). And I've always looked at TripleO as the opposite. It is a default deployment of OpenStack. That is why we look at having to set a non-default option as a bug. That is why we only currently offer one set of Heat templates. Of course I want to see it more widely configurable, as I understand that there will be whole sections of the OpenStack interested user base that won't want an ovs overlay network. There will be shops that simply refuse to use MySQL, or want to put swift proxies on their own nodes, etc. etc. But if we can't all agree on a widely usable set of defaults, and deploy that, then I think OpenStack, not just TripleO, is forever going to be a framework on which proprietary solutions are built, rather than a whole open source platform. I think this is dangerous thinking - the config you want depends so hugely on your intended workload and available hardware that trying any strong view of what an Openstack deployment should look like into the deployment tool forever forces that deployment tool to be a minor, niche product that *has* to be replaced by something more expressive in order to be widely usable. The config you want for a primary hadoop shop is totally different to what you'd want for primary web-host shop is somewhat different to what you'd want for a public/generic cloud, etc. Things like AZ support, neutron model, cinder back-end choice, H/A model etc are dictated by scale and use-cases. If you only want your config tool to deal with one deployment type, that tool becomes pretty much irrelevant to the totality of the Openstack effort, and should be replaced by something more layered/openminded. I can certainly understand how one might mistake TripleO as a deployment tool. It is no such thing. OpenStack is the deployment suite, with the tools being Nova, Glance, Neutron, Heat, diskimage-builder, os-*-config, etc. TripleO is a _program_, in the sense of an effort to gather collaborative forces, to deploy OpenStack using these tools. So any concern that you have that these tools will end up being niche tools will in fact affect all of OpenStack. The niche that we're aiming at is the broadest base of users that OpenStack has. That would be the ones who we have driven the defaults toward. If there is no group of users that can use the defaults, then our first deployment will not have a large uptake beyond OpenStack's CI itself. However, there is nothing about this first goal of deploying a default OpenStack cloud that prevents us from then widening its purpose for more and more sets of users. This isn't to say we must boil the ocean right now and make everything available, but rather that decisions should take the long view into account. I don't think any of us suggested sacrificing the long term view for the short term milestone. What I said is that what we're aiming at now as a _milestone_ is not a widely configurable cloud, but a default cloud. And things that are done to support widening the tools' focus slow progress toward the current milestone. I get a sense that people are feeling adversarial toward this focus, rather than collaborative, and that troubles me. It may be my fault, so I want to make it very clear that I _do_ welcome any and all contribution. If people are willing to put in the time to get things done in TripleO, then that is hugely valuable. I'm only suggesting that if you have a choice in what to do next, I suggest that it be things that get us closer to our current milestones. Thanks everyone for your thoughts on this. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [TripleO] config options, defaults, oh my!
Excerpts from Alexis Lee's message of 2014-04-09 06:44:20 -0700:
Robert Collins said on Wed, Apr 09, 2014 at 01:58:59AM +1200:
I like this - something like
nova:
config:
- section: default
values:
- option: 'compute_manager'
value: 'ironic.nova.compute.manager.ClusterComputeManager'
- section: cells
values:
- option: 'driver'
value: nova.cells.rpc_driver.CellsRPCDriver
should be able to represent most? all (it can handle repeating items)
oslo.config settings and render it easily:
{{#config}}
{{#comment}} repeats for each section {{/comment}}
[{{section}}]
{{#values}}
{{option}}={{value}}
{{/values}}
{{/config}}
Hello,
I've gone some distance down this road:
https://review.openstack.org/#/c/83353/6/elements/nova/os-apply-config/etc/nova/log.conf
https://review.openstack.org/#/c/83422/6/logstash-source.yaml
I wouldn't call the result - encoding a complete config file into Heat
metadata - pretty. And this isn't completely genericised either.
I find your templates pretty easy to read. Mustache was chosen because it
has almost no logic, and thus is _extremely_ easy to follow and predict.
It'd be much better if TripleO image elements focused on installing and
starting services and allowed system integrators to define the
configuration. In one place, in plain text files, the UNIX way. I've
appended my proposal to Rob's etherpad here:
https://etherpad.openstack.org/p/tripleo-config-passthrough
This assumes that we don't want system integrators to contribute to
TripleO, which is the opposite of how things are. We absolutely do,
and in fact, that is part of the point of having a program around
OpenStack deployment. Let's get system integrators into OpenStack's CI
system and let's get a few of the most important scenarios into the gate
of OpenStack.
As a system integrator, do you want to say to your customers that you
start with an unusable set of tools that the community tests individually,
or do you want to say that you start with the deployment that the
community tests directly on every commit, and then enhance based on
individual customer need?
Soon-to-be outdated copy appended here:
Hi Rob, I have some serious concerns about the above approaches. For the
sake of argument, let's suppose we want to write a file that looks like
a Heat template. How would you write a Mustache template that handles
that level of nesting? Even if you accomplish that, how readable do you
think the metadata to fill out that template would look?
I see the system integration process emerging like this:
* Figure out what files you want + what you want in them
* Slice and dice that into metadata
* Write some fairly complicated templates to reconstitute the metadata
* Get out more or less what you started with
I'd like to propose an alternative method where Heat and TripleO barely
touch the config. The system integrator writes an image element per
node-flavour, EG mycorp-compute-config. If they choose, they could
write more (EG for specific hardware) limited only by their
devtest-equivalent's ability to allocate those. This element contains a
99-os-apply-config directory, the templates from which overwrite any
templates from normal os-apply-config directories in other elements.
os-apply-config/install.d/99-install-config-templates will need to be
patched for this to be possible, but this is very little work in
comparison to the alternatives. I could also support simply an
os-apply-config.override directory, if a full numbered set of dirs seems
overkill, but in this case normal elements would have to be forbidden
from using it (and people being as they are, someone would). The
templates in that directory are 99% plain config files, laid out in a
single filesystem structure exactly as the system integrator wants them.
The only templated values which need to be supplied by Heat are those
which vary per-instance.
So one reason I'd rather see overrides done generically as heat
parameters/metadata/etc. is that it may not be entirely clear when a
user needs to override from a default, and having to distribute a new
image to do that is not necessarily the best user experience, especially
if one is tinkering.
If we do this, tripleo-image-elements should focus on installing and
starting services. They should only include a minimal viable
configuration for demo purposes. This should greatly reduce the amount
of work required to produce a new element. Also the number of Heat
parameters used by any element (only per-instance would be necessary,
anything further is a convenience).
OpenStack is not for demo purposes, it is for production usage. There
is no reason we cannot and should not build a deployment that can be
used directly in the most common configuration, and
Re: [openstack-dev] [TripleO] config options, defaults, oh my!
On 10 April 2014 08:33, Clint Byrum cl...@fewbar.com wrote: This is exactly what we're doing. We're just suggesting exposing variations in the Heat templates, rather than in the elements. It is worth noting that Heat has grown the ability to grab a local file and inject it into your template at runtime. I think it would actually make sense to have os-apply-config enhanced to be able to override whole template files based on something like this: resources: server1: metadata: template_overrides: /etc/nova/nova.conf: get_file [ my_special_nova.conf.template ] In that, we achieve what you want, but we can do so without rebuilding the whole image. This makes me a little nervous: its much easier to break os-collect-config by forcing os-apply-config to break hard this way, than through bad metadata. I think I'm ok with the sentiment, but nervous about impl. -Rob -- Robert Collins rbtcoll...@hp.com Distinguished Technologist HP Converged Cloud ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [TripleO] config options, defaults, oh my!
Does anyone have a flowchart of the cloud build/configure process including interactions between the various components/stages of TripleO and Heat? -Original Message- From: Robert Collins [mailto:robe...@robertcollins.net] Sent: Wednesday, April 09, 2014 2:29 PM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [TripleO] config options, defaults, oh my! On 10 April 2014 08:33, Clint Byrum cl...@fewbar.com wrote: This is exactly what we're doing. We're just suggesting exposing variations in the Heat templates, rather than in the elements. It is worth noting that Heat has grown the ability to grab a local file and inject it into your template at runtime. I think it would actually make sense to have os-apply-config enhanced to be able to override whole template files based on something like this: resources: server1: metadata: template_overrides: /etc/nova/nova.conf: get_file [ my_special_nova.conf.template ] In that, we achieve what you want, but we can do so without rebuilding the whole image. This makes me a little nervous: its much easier to break os-collect-config by forcing os-apply-config to break hard this way, than through bad metadata. I think I'm ok with the sentiment, but nervous about impl. -Rob -- Robert Collins rbtcoll...@hp.com Distinguished Technologist HP Converged Cloud ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [TripleO] config options, defaults, oh my!
+1 to this: nova: config: default.compute_manager: ironic.nova.compute.manager.ClusterComputeManager cells.driver: nova.cells.rpc_driver.CellsRPCDriver Adding a generic mechanism like this and having everything configurable seems like a best option to me. On 04/08/2014 01:51 AM, Dan Prince wrote: - Original Message - From: Robert Collins robe...@robertcollins.net To: OpenStack Development Mailing List openstack-dev@lists.openstack.org Sent: Monday, April 7, 2014 4:00:30 PM Subject: [openstack-dev] [TripleO] config options, defaults, oh my! So one interesting thing from the influx of new reviews is lots of patches exposing all the various plumbing bits of OpenStack. This is good in some ways (yay, we can configure more stuff), but in some ways its kindof odd - like - its not clear when https://review.openstack.org/#/c/83122/ is needed. I'm keen to expose things that are really needed, but i'm not sure that /all/ options are needed - what do folk think? I think we can learn much from some of the more mature configuration management tools in the community on this front. Using puppet as an example here (although I'm sure other tools may do similar things as well)... Take configuration of the Nova API server. There is a direct configuration parameter for 'neutron_metadata_proxy_shared_secret' in the Puppet nova::api class. This parameter is exposed in the class (sort of the equivalent of a TripleO element) directly because it is convenient and many users may want to customize the value. There are however hundreds of Nova config options and most of them aren't exposed as parameters in the various Nova puppet classes. For these it is possible to define a nova_config resource to configure *any* nova.conf parameter in an ad hoc style for your own installation tuning purposes. I could see us using a similar model in TripleO where our elements support configuring common config elements directly, but we also allow people to tune extra undocumented options for their own use. There is always going to be a need for this as people need to tune things for their own installations with options that may not be appropriate for the common set of elements. Standardizing this mechanism across many of the OpenStack service elements would also make a lot of sense. Today we have this for Nova: nova: verbose: False - Print more verbose output (set logging level to INFO instead of default WARNING level). debug: False - Print debugging output (set logging level to DEBUG instead of default WARNING level). baremetal: pxe_deploy_timeout: 1200 . I could see us adding a generic mechanism like this to overlay with the existing (documented) data structure: nova: config: default.compute_manager: ironic.nova.compute.manager.ClusterComputeManager cells.driver: nova.cells.rpc_driver.CellsRPCDriver And in this manner a user might be able to add *any* supported config param to the element. Also, some things really should be higher order operations - like the neutron callback to nova right - that should be either set to timeout in nova configured in neutron, *or* set in both sides appropriately, never one-half or the other. I think we need to sort out our approach here to be systematic quite quickly to deal with these reviews. I totally agree. I was also planning to email the list about this very issue this week :) My email subject was going to be TripleO templates... an upstream maintenance problem. For the existing reviews today I think we should be somewhat selective about what parameters we expose as top level within the elements. That said we are missing some rather fundamental features to allow users to configure undocumented parameters as well. So we need to solve this problem quickly because there are certainly some configuration corner that users will need. As is today we are missing some rather fundamental features in os-apply-config and the elements to be able to pull this off. What we really need is a generic INI style template generator. Or perhaps we could use something like augeus or even devstacks simple ini editing functions to pull this off. In any case the idea would be that we allow users to inject their own undocumented config parameters into the various service config files. Or perhaps we could auto-generate mustache templates based off of the upstream sample config files. Many approaches would work here I think... Here's an attempt to do so - this could become a developers guide patch. Config options in TripleO == Non-API driven configuration falls into four categories: A - fixed at buildtime (e.g. ld.so path) B - cluster state derived C - local machine derived D - deployer choices For A, it should be entirely done within the elements concerned. For B, the heat template should accept parameters to choose the desired config (e.g. the Neutron-Nova example able
Re: [openstack-dev] [TripleO] config options, defaults, oh my!
-Original Message- From: Robert Collins [mailto:robe...@robertcollins.net] Sent: 07 April 2014 21:01 To: OpenStack Development Mailing List Subject: [openstack-dev] [TripleO] config options, defaults, oh my! So one interesting thing from the influx of new reviews is lots of patches exposing all the various plumbing bits of OpenStack. This is good in some ways (yay, we can configure more stuff), but in some ways its kindof odd - like - its not clear when https://review.openstack.org/#/c/83122/ is needed. I'm keen to expose things that are really needed, but i'm not sure that /all/ options are needed - what do folk think? I'm very wary of trying to make the decision in TripleO of what should and shouldn't be configurable in some other project.For sure the number of config options in Nova is a problem, and one that's been discussed many times at summits. However I think you could also make the case/assumption for any service that the debate about having a config option has already been held within that service as part of the review that merged that option in the code - re-running the debate about whether something should be configurable via TripleO feels like some sort of policing function on configurability above and beyond what the experts in that service have already considered, and that doesn't feel right to me. Right now TripleO has a very limited view of what can be configured, based as I understand on primarily what's needed for its CI job. As more folks who have real deployments start to look at using TripleO its inevitable that they are going to want to enable the settings that are important to them to be configured. I can't imagine that anyone is going to add a configuration value for the sake of it, so can't we start with the perspective that we are slowly exposing the set of values that do need to be configured ? Also, some things really should be higher order operations - like the neutron callback to nova right - that should be either set to timeout in nova configured in neutron, *or* set in both sides appropriately, never one-half or the other. I think we need to sort out our approach here to be systematic quite quickly to deal with these reviews. Here's an attempt to do so - this could become a developers guide patch. Config options in TripleO == Non-API driven configuration falls into four categories: A - fixed at buildtime (e.g. ld.so path) B - cluster state derived C - local machine derived D - deployer choices For A, it should be entirely done within the elements concerned. For B, the heat template should accept parameters to choose the desired config (e.g. the Neutron-Nova example able) but then express the config in basic primitives in the instance metadata. For C, elements should introspect the machine (e.g. memory size to determine mysql memory footprint) inside os-refresh-config scripts; longer term we should make this an input layer to os-collect-config. For D, we need a sensible parameter in the heat template and probably direct mapping down to instance metadata. I understand the split, but all of the reviews in question seem to be in D, so I'm not sure this helps much. But we have a broader question - when should something be configurable at all? In my mind we have these scenarios: 1) There is a single right answer 2) There are many right answers An example of 1) would be any test-only option like failure injection - the production value is always 'off'. For 2), hypervisor driver is a great example - anything other than qemu is a valid production value :) But, it seems to me that these cases actually subdivide further - 1a) single right answer, and the default is the right answer 1b) single right answer and it is not the default 2a) many right answers, and the default is the most/nearly most common one 2b) many right answers, and the default is either not one of them or is a corner case So my proposal here - what I'd like to do as we add all these config options to TripleO is to take the care to identify which of A/B/C/D they are and code them appropriately, and if the option is one of 1b) or 2b) make sure there is a bug in the relevant project about the fact that we're having to override a default. If the option is really a case of 1a) I'm not sure we want it configurable at all. I'm not convinced that anyone is in a position to judge that there is a single right answer - I know the values that are right for my deployments, but I'm not arrogant enough to say that they universally applicable.You only have to see the wide range of Openstack Deployments presented at every summit to know that that there a lot of different use cases out there. My worry is that if we try to have that debate in the context of a TripleO review, then we'll just spin between opinions rather than make the rapid progress towards getting the needed
Re: [openstack-dev] [TripleO] config options, defaults, oh my!
On 04/08/2014 06:03 AM, Day, Phil wrote: -Original Message- From: Robert Collins [mailto:robe...@robertcollins.net] Sent: 07 April 2014 21:01 To: OpenStack Development Mailing List Subject: [openstack-dev] [TripleO] config options, defaults, oh my! So one interesting thing from the influx of new reviews is lots of patches exposing all the various plumbing bits of OpenStack. This is good in some ways (yay, we can configure more stuff), but in some ways its kindof odd - like - its not clear when https://review.openstack.org/#/c/83122/ is needed. I'm keen to expose things that are really needed, but i'm not sure that /all/ options are needed - what do folk think? I'm very wary of trying to make the decision in TripleO of what should and shouldn't be configurable in some other project.For sure the number of config options in Nova is a problem, and one that's been discussed many times at summits. However I think you could also make the case/assumption for any service that the debate about having a config option has already been held within that service as part of the review that merged that option in the code - re-running the debate about whether something should be configurable via TripleO feels like some sort of policing function on configurability above and beyond what the experts in that service have already considered, and that doesn't feel right to me. Right now TripleO has a very limited view of what can be configured, based as I understand on primarily what's needed for its CI job. As more folks who have real deployments start to look at using TripleO its inevitable that they are going to want to enable the settings that are important to them to be configured. I can't imagine that anyone is going to add a configuration value for the sake of it, so can't we start with the perspective that we are slowly exposing the set of values that do need to be configured ? I think Phil is dead on. I'll also share the devstack experience here. Until we provided the way for arbitrary pass through we were basically getting a few patches every week that were let me configure this variable in the configs over and over again. I 100% agree that the config parameter space is huge and out of control, and I actively challenge when people add new config options in Nova, however those are knobs that people are using. If you limit what's allowed to be configured, you limit the use of the tool. Like the old adage about the fact that everyone only uses 10% of the functionality of MS Word (sadly they don't all use the same 10%). There was a really good proposal on the table by Mark MC a few cycles ago about annotating the config options in projects with things like 'debug', 'tuning', so that it would be clear what variables we expected people to change, and what variables we assume only experts would change. I think if there is desire to push on the config explosion, that would be the right place to do it. -Sean -- Sean Dague Samsung Research America s...@dague.net / sean.da...@samsung.com http://dague.net signature.asc Description: OpenPGP digital signature ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [TripleO] config options, defaults, oh my!
On Tue, Apr 8, 2014 at 6:03 AM, Day, Phil philip@hp.com wrote: -Original Message- From: Robert Collins [mailto:robe...@robertcollins.net] Sent: 07 April 2014 21:01 To: OpenStack Development Mailing List Subject: [openstack-dev] [TripleO] config options, defaults, oh my! So one interesting thing from the influx of new reviews is lots of patches exposing all the various plumbing bits of OpenStack. This is good in some ways (yay, we can configure more stuff), but in some ways its kindof odd - like - its not clear when https://review.openstack.org/#/c/83122/ is needed. I'm keen to expose things that are really needed, but i'm not sure that /all/ options are needed - what do folk think? I'm very wary of trying to make the decision in TripleO of what should and shouldn't be configurable in some other project.For sure the number of config options in Nova is a problem, and one that's been discussed many times at summits. However I think you could also make the case/assumption for any service that the debate about having a config option has already been held within that service as part of the review that merged that option in the code - re-running the debate about whether something should be configurable via TripleO feels like some sort of policing function on configurability above and beyond what the experts in that service have already considered, and that doesn't feel right to me. This captures very well what I was thinking as well. I just don't think it should be a question in TripleO if a config option should be exposed or not. I don't think any of the reviews we've had recently, or are in the queue, folks have added just for the sake of exposing more options. I assume it's because they actually wanted to tweak those options. So my proposal here - what I'd like to do as we add all these config options to TripleO is to take the care to identify which of A/B/C/D they are and code them appropriately, and if the option is one of 1b) or 2b) make sure there is a bug in the relevant project about the fact that we're having to override a default. If the option is really a case of 1a) I'm not sure we want it configurable at all. If the root cause of the problem is that every change to exploit a configurable feature of a deployed service needs an explicit matching change in TripleO, maybe that's a degree of tight coupling that needs to be addressed by having a more generic mechanism ? Indeed, I think the implementation we have for configuration right now that uses templates is a bit of a maintenance burden as Dan said. Every time you want to expose a new configuration option, you need a change to tripleo-image-elements to add it to the template, and corresponding change to tripleo-heat-templates so that the option can be set via Heat metadata. This is similar I think (if not the same) as what Dan is proposing, but I'd really like to see something along these lines instead: - an implementation that used sample or generated config files for all the OpenStack projects instead of templates - a way for elements to specify their configuration for TripleO when we can't use upstream defaults (json/yaml structure, etc). This structure would get backed into the image for os-collect-config to pick up and consume. Also a way for other elements to override others, e.g., Nova element has some default configs, then the Ironic might need to override some of those. - enhance os-apply-config to have it support additional ways (iniset, crudini, augeas, whatever) to modify config files instead of just templatizing so that we can apply that baked-in config and additional config from Heat when not using templates. That solves the question of what to expose in the elements. Since we're using sample/generated config files from the projects, everything is exposed by default. When the upstream project adds a new config option, it automatically gets exposed. Half of the ongoing review burden is gone right there, with no developer guide needed either. Also part of the win...we'd have polished config files that look like upstreams, except for the values of course :-). And, this also helps get us going down the road of making the elements more discoverable in terms of what can be configured and what the values are for TripleO, something that I know Tuskar is very interested in having. The question then becomes about what to do about exposing these options in the Heat templates. We could keep the templates as we have today, and just add parameters for the options that we know are useful to us for the CD and CI use cases. We could then document how a user would go about adding additional parameters to the templates that they wanted exposed. Perhaps even give them a way to merge their custom template snippets that add Parameters into those from tripleo-heat-templates. And, in fact, I think in most cases it *wouldn't* be a case of users wanting to expose stuff so
Re: [openstack-dev] [TripleO] config options, defaults, oh my!
Hi On 8 Apr 2014, at 11:20, Sean Dague s...@dague.net wrote: I think Phil is dead on. I'll also share the devstack experience here. Until we provided the way for arbitrary pass through we were basically getting a few patches every week that were let me configure this variable in the configs over and over again. +1 We can't be in the business of prescribing what users can/can't configure in the daemons they are using us to deploy. Cheers, Chris ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [TripleO] config options, defaults, oh my!
Excerpts from Day, Phil's message of 2014-04-08 03:03:41 -0700: -Original Message- From: Robert Collins [mailto:robe...@robertcollins.net] Sent: 07 April 2014 21:01 To: OpenStack Development Mailing List Subject: [openstack-dev] [TripleO] config options, defaults, oh my! So one interesting thing from the influx of new reviews is lots of patches exposing all the various plumbing bits of OpenStack. This is good in some ways (yay, we can configure more stuff), but in some ways its kindof odd - like - its not clear when https://review.openstack.org/#/c/83122/ is needed. I'm keen to expose things that are really needed, but i'm not sure that /all/ options are needed - what do folk think? I'm very wary of trying to make the decision in TripleO of what should and shouldn't be configurable in some other project.For sure the number of config options in Nova is a problem, and one that's been discussed many times at summits. However I think you could also make the case/assumption for any service that the debate about having a config option has already been held within that service as part of the review that merged that option in the code - re-running the debate about whether something should be configurable via TripleO feels like some sort of policing function on configurability above and beyond what the experts in that service have already considered, and that doesn't feel right to me. Right now TripleO has a very limited view of what can be configured, based as I understand on primarily what's needed for its CI job. As more folks who have real deployments start to look at using TripleO its inevitable that they are going to want to enable the settings that are important to them to be configured. I can't imagine that anyone is going to add a configuration value for the sake of it, so can't we start with the perspective that we are slowly exposing the set of values that do need to be configured ? snip So my proposal here - what I'd like to do as we add all these config options to TripleO is to take the care to identify which of A/B/C/D they are and code them appropriately, and if the option is one of 1b) or 2b) make sure there is a bug in the relevant project about the fact that we're having to override a default. If the option is really a case of 1a) I'm not sure we want it configurable at all. I'm not convinced that anyone is in a position to judge that there is a single right answer - I know the values that are right for my deployments, but I'm not arrogant enough to say that they universally applicable. You only have to see the wide range of Openstack Deployments presented at every summit to know that that there a lot of different use cases out there. My worry is that if we try to have that debate in the context of a TripleO review, then we'll just spin between opinions rather than make the rapid progress towards getting the needed degree of configurability. So I would rule out 1a and 1b as options.Again it feels like any debate about this really belongs back in the projects that TripleO is deploying rather than TripleO itself. To me, TripleO isn't just OpenStack CI on a large scale. And TripleO isn't here's a generic set of tools, go build a cloud. At least, not yet, likely some day. To me, TripleO is first _a_ deployment of OpenStack. Meaning, we can consider our first milestone reached when we have produced a consumable, high scale deployment of OpenStack using OpenStack to operate it. Contributions that do not push us toward that milestone have an impact on it. At the very least they distract our reviewers, and at most, they actively delay that milestone by adding complexity resulting in slower velocity toward it. I don't mean to say that they're not welcome, as I think they add an incredible amount of value; we end up with a virtuous circle as we gather users and contribution that is not directly related to our immediate goals. However, what I'm suggesting is that generic universal configurability of all of the services isn't the immediate goal. Many of these configuration knobs, while eventually necessary to widen the usability of TripleO in a similar fashion to the rest of the OpenStack, will delay even the most narrow initial usability of TripleO. I'm also not a great fan of considering a change in the default value (in either TripleO or the original project) as an alternative -whether the original default was a good choice or not there is a high chance that someone is relying on it - so any change in a default needs to go through a deprication period so that folks have a chance to explicitly configure to keep the setting if they need to. That patterns reasonably well established in most of the projects, and as folks are now starting to use TripleO I think it needs to have the same discipline. It's a pain for sure, but it's the only way to keep stable systems for people
Re: [openstack-dev] [TripleO] config options, defaults, oh my!
On 8 April 2014 11:51, Dan Prince dpri...@redhat.com wrote:
- Original Message -
From: Robert Collins robe...@robertcollins.net
To: OpenStack Development Mailing List openstack-dev@lists.openstack.org
Sent: Monday, April 7, 2014 4:00:30 PM
Subject: [openstack-dev] [TripleO] config options, defaults, oh my!
So one interesting thing from the influx of new reviews is lots of
patches exposing all the various plumbing bits of OpenStack. This is
good in some ways (yay, we can configure more stuff), but in some ways
its kindof odd - like - its not clear when
https://review.openstack.org/#/c/83122/ is needed.
I'm keen to expose things that are really needed, but i'm not sure
that /all/ options are needed - what do folk think?
I think we can learn much from some of the more mature configuration
management tools in the community on this front. Using puppet as an example
here (although I'm sure other tools may do similar things as well)... Take
configuration of the Nova API server. There is a direct configuration
parameter for 'neutron_metadata_proxy_shared_secret' in the Puppet nova::api
class. This parameter is exposed in the class (sort of the equivalent of a
TripleO element) directly because it is convenient and many users may want to
customize the value. There are however hundreds of Nova config options and
most of them aren't exposed as parameters in the various Nova puppet classes.
For these it is possible to define a nova_config resource to configure *any*
nova.conf parameter in an ad hoc style for your own installation tuning
purposes.
I could see us using a similar model in TripleO where our elements support
configuring common config elements directly, but we also allow people to tune
extra undocumented options for their own use. There is always going to be a
need for this as people need to tune things for their own installations with
options that may not be appropriate for the common set of elements.
Standardizing this mechanism across many of the OpenStack service elements
would also make a lot of sense. Today we have this for Nova:
nova:
verbose: False
- Print more verbose output (set logging level to INFO instead of default
WARNING level).
debug: False
- Print debugging output (set logging level to DEBUG instead of default
WARNING level).
baremetal:
pxe_deploy_timeout: 1200
.
I could see us adding a generic mechanism like this to overlay with the
existing (documented) data structure:
nova:
config:
default.compute_manager:
ironic.nova.compute.manager.ClusterComputeManager
cells.driver: nova.cells.rpc_driver.CellsRPCDriver
And in this manner a user might be able to add *any* supported config param
to the element.
I like this - something like
nova:
config:
- section: default
values:
- option: 'compute_manager'
value: 'ironic.nova.compute.manager.ClusterComputeManager'
- section: cells
values:
- option: 'driver'
value: nova.cells.rpc_driver.CellsRPCDriver
should be able to represent most? all (it can handle repeating items)
oslo.config settings and render it easily:
{{#config}}
{{#comment}} repeats for each section {{/comment}}
[{{section}}]
{{#values}}
{{option}}={{value}}
{{/values}}
{{/config}}
Also, some things
really should be higher order operations - like the neutron callback
to nova right - that should be either set to timeout in nova
configured in neutron, *or* set in both sides appropriately, never
one-half or the other.
I think we need to sort out our approach here to be systematic quite
quickly to deal with these reviews.
I totally agree. I was also planning to email the list about this very issue
this week :) My email subject was going to be TripleO templates... an
upstream maintenance problem.
For the existing reviews today I think we should be somewhat selective about
what parameters we expose as top level within the elements. That said we are
missing some rather fundamental features to allow users to configure
undocumented parameters as well. So we need to solve this problem quickly
because there are certainly some configuration corner that users will need.
As is today we are missing some rather fundamental features in
os-apply-config and the elements to be able to pull this off. What we really
need is a generic INI style template generator. Or perhaps we could use
something like augeus or even devstacks simple ini editing functions to pull
this off. In any case the idea would be that we allow users to inject their
own undocumented config parameters into the various service config files. Or
perhaps we could auto-generate mustache templates based off of the upstream
sample config files. Many approaches would work here I think...
I agree that there are many approaches here - I think the sketch above
may be sufficient for right now.
-Rob
--
Robert Collins
Re: [openstack-dev] [TripleO] config options, defaults, oh my!
On 9 April 2014 00:48, Chris Jones c...@tenshu.net wrote: Hi On 8 Apr 2014, at 11:20, Sean Dague s...@dague.net wrote: I think Phil is dead on. I'll also share the devstack experience here. Until we provided the way for arbitrary pass through we were basically getting a few patches every week that were let me configure this variable in the configs over and over again. +1 We can't be in the business of prescribing what users can/can't configure in the daemons they are using us to deploy. I think this points to a misapprehension about what I was saying that I think other folk in the thread have had too - and I'm not going to try to reply to each individually :). The question isn't about prescribing limits - of course anyone can configure anything. The question is /how/. Do we model it? Do we punt and pass everything through? Do we model some stuff? -lots- of complexity in these setups is tied entirely to 'all servers running X need Y' style questions which is where Heats value /should/ shine through - but we don't want every deployer to have to learn Heat on day one of their deploy - so we need a way to: - deliver a great out of the box experience - let higher order configuration - cluster aware - be done well - whilst also surfacing the plumbing as needed. Right now we have no differentiation between plumbing exposure and semantically modelled configuration, and I think thats a problem. I *loved* Dan's answer :) -Rob -- Robert Collins rbtcoll...@hp.com Distinguished Technologist HP Converged Cloud ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [TripleO] config options, defaults, oh my!
I'm very wary of trying to make the decision in TripleO of what should and shouldn't be configurable in some other project.For sure the number of config options in Nova is a problem, and one that's been discussed many times at summits. However I think you could also make the case/assumption for any service that the debate about having a config option has already been held within that service as part of the review that merged that option in the code - re-running the debate about whether something should be configurable via TripleO feels like some sort of policing function on configurability above and beyond what the experts in that service have already considered, and that doesn't feel right to me. My general feeling is that I agree with this sentiment. In my experience on management tools, there's always someone who wants to turn the one knob I forgot to expose. And that's been on significantly simpler projects than OpenStack; the complexity and scale of the features means there's potentially a ton of tweaking to be done. More generally, this starts to drift into the bigger question of what TripleO is. The notion of defaults or limiting configuration exposure is for prescriptive purposes. You can change X because we think it's going to have a major impact. If we don't expose Y, it's because we're driving the user to not want to change it. I've always assumed TripleO is very low-level. Put another way, non-prescriptive. It's not going to push an agenda that says you should be doing things a certain way, but rather gives you more than enough rope to hang yourself (just makes it easier). The question of how to make things easier to grok for a new user lies in a different area. Either documentation (basic v. advanced user guide sort of thing) or potentially in the Tuskar GUI. More configuration options means Tuskar's life is more difficult, but to me, that's where we add in the notion of You almost definitely want to configure these things, but if you're really insane you can look at this other set of stuff to configure. So I think we need to have a way of specifying everything. And we need to have that way not kill us in the process. I like the proposed idea of an open-ended config area. It's us acknowledging that we're sitting on top of a dozen other projects. Admittedly, I don't fully understand Slagle's proposal, but the idea of pulling in samples from other projects and not making us acknowledge every configuration option is also appealing. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [TripleO] config options, defaults, oh my!
Excerpts from Jay Dobies's message of 2014-04-08 06:40:07 -0700: I'm very wary of trying to make the decision in TripleO of what should and shouldn't be configurable in some other project.For sure the number of config options in Nova is a problem, and one that's been discussed many times at summits. However I think you could also make the case/assumption for any service that the debate about having a config option has already been held within that service as part of the review that merged that option in the code - re-running the debate about whether something should be configurable via TripleO feels like some sort of policing function on configurability above and beyond what the experts in that service have already considered, and that doesn't feel right to me. My general feeling is that I agree with this sentiment. In my experience on management tools, there's always someone who wants to turn the one knob I forgot to expose. And that's been on significantly simpler projects than OpenStack; the complexity and scale of the features means there's potentially a ton of tweaking to be done. More generally, this starts to drift into the bigger question of what TripleO is. The notion of defaults or limiting configuration exposure is for prescriptive purposes. You can change X because we think it's going to have a major impact. If we don't expose Y, it's because we're driving the user to not want to change it. I've always assumed TripleO is very low-level. Put another way, non-prescriptive. It's not going to push an agenda that says you should be doing things a certain way, but rather gives you more than enough rope to hang yourself (just makes it easier). And I've always looked at TripleO as the opposite. It is a default deployment of OpenStack. That is why we look at having to set a non-default option as a bug. That is why we only currently offer one set of Heat templates. Of course I want to see it more widely configurable, as I understand that there will be whole sections of the OpenStack interested user base that won't want an ovs overlay network. There will be shops that simply refuse to use MySQL, or want to put swift proxies on their own nodes, etc. etc. But if we can't all agree on a widely usable set of defaults, and deploy that, then I think OpenStack, not just TripleO, is forever going to be a framework on which proprietary solutions are built, rather than a whole open source platform. The question of how to make things easier to grok for a new user lies in a different area. Either documentation (basic v. advanced user guide sort of thing) or potentially in the Tuskar GUI. More configuration options means Tuskar's life is more difficult, but to me, that's where we add in the notion of You almost definitely want to configure these things, but if you're really insane you can look at this other set of stuff to configure. So I think we need to have a way of specifying everything. And we need to have that way not kill us in the process. I like the proposed idea of an open-ended config area. It's us acknowledging that we're sitting on top of a dozen other projects. Admittedly, I don't fully understand Slagle's proposal, but the idea of pulling in samples from other projects and not making us acknowledge every configuration option is also appealing. Despite our differing views on what TripleO is today, I agree with the need for an open ended config section. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [TripleO] config options, defaults, oh my!
- Original Message - From: Robert Collins robe...@robertcollins.net To: OpenStack Development Mailing List openstack-dev@lists.openstack.org Sent: Monday, April 7, 2014 4:00:30 PM Subject: [openstack-dev] [TripleO] config options, defaults, oh my! So one interesting thing from the influx of new reviews is lots of patches exposing all the various plumbing bits of OpenStack. This is good in some ways (yay, we can configure more stuff), but in some ways its kindof odd - like - its not clear when https://review.openstack.org/#/c/83122/ is needed. I'm keen to expose things that are really needed, but i'm not sure that /all/ options are needed - what do folk think? I think we can learn much from some of the more mature configuration management tools in the community on this front. Using puppet as an example here (although I'm sure other tools may do similar things as well)... Take configuration of the Nova API server. There is a direct configuration parameter for 'neutron_metadata_proxy_shared_secret' in the Puppet nova::api class. This parameter is exposed in the class (sort of the equivalent of a TripleO element) directly because it is convenient and many users may want to customize the value. There are however hundreds of Nova config options and most of them aren't exposed as parameters in the various Nova puppet classes. For these it is possible to define a nova_config resource to configure *any* nova.conf parameter in an ad hoc style for your own installation tuning purposes. I could see us using a similar model in TripleO where our elements support configuring common config elements directly, but we also allow people to tune extra undocumented options for their own use. There is always going to be a need for this as people need to tune things for their own installations with options that may not be appropriate for the common set of elements. Standardizing this mechanism across many of the OpenStack service elements would also make a lot of sense. Today we have this for Nova: nova: verbose: False - Print more verbose output (set logging level to INFO instead of default WARNING level). debug: False - Print debugging output (set logging level to DEBUG instead of default WARNING level). baremetal: pxe_deploy_timeout: 1200 . I could see us adding a generic mechanism like this to overlay with the existing (documented) data structure: nova: config: default.compute_manager: ironic.nova.compute.manager.ClusterComputeManager cells.driver: nova.cells.rpc_driver.CellsRPCDriver And in this manner a user might be able to add *any* supported config param to the element. Also, some things really should be higher order operations - like the neutron callback to nova right - that should be either set to timeout in nova configured in neutron, *or* set in both sides appropriately, never one-half or the other. I think we need to sort out our approach here to be systematic quite quickly to deal with these reviews. I totally agree. I was also planning to email the list about this very issue this week :) My email subject was going to be TripleO templates... an upstream maintenance problem. For the existing reviews today I think we should be somewhat selective about what parameters we expose as top level within the elements. That said we are missing some rather fundamental features to allow users to configure undocumented parameters as well. So we need to solve this problem quickly because there are certainly some configuration corner that users will need. As is today we are missing some rather fundamental features in os-apply-config and the elements to be able to pull this off. What we really need is a generic INI style template generator. Or perhaps we could use something like augeus or even devstacks simple ini editing functions to pull this off. In any case the idea would be that we allow users to inject their own undocumented config parameters into the various service config files. Or perhaps we could auto-generate mustache templates based off of the upstream sample config files. Many approaches would work here I think... Here's an attempt to do so - this could become a developers guide patch. Config options in TripleO == Non-API driven configuration falls into four categories: A - fixed at buildtime (e.g. ld.so path) B - cluster state derived C - local machine derived D - deployer choices For A, it should be entirely done within the elements concerned. For B, the heat template should accept parameters to choose the desired config (e.g. the Neutron-Nova example able) but then express the config in basic primitives in the instance metadata. For C, elements should introspect the machine (e.g. memory size to determine mysql memory footprint) inside os-refresh-config scripts; longer term we should make this an input layer to os-collect-config. For D, we need
