Re: [openstack-dev] [all] sample config files should be ignored in git...

2014-03-27 Thread Michael Chapman
On Thu, Mar 27, 2014 at 4:10 PM, Robert Collins
robe...@robertcollins.netwrote:

 On 27 March 2014 17:30, Tom Fifield t...@openstack.org wrote:

  Does anyone disagree?
 
  /me raises hand
 
  When I was an operator, I regularly referred to the sample config files
  in the git repository.
 
  If there weren't generated versions of the sample config in the repo, I
  would probably grep the code (not an ideal user experience!). Running
  some random script that I don't know about the existence and might
  depend on having something else installed of is probably not something
  that would happen.

 So, I think its important you have sample configs to refer to.

 Do they need to be in the git repo?

 Note that because libraries now export config options (which is the
 root of this problem!) you cannot ever know from the source all the
 options for a service - you *must* know the library versions you are
 running, to interrogate them for their options.

 We can - and should - have a discussion about the appropriateness of
 the layering leak we have today, but in the meantime this is breaking
 multiple projects every time any shared library that uses oslo.config
 changes any config option... so we need to solve the workflow aspect.

 How about we make a copy of the latest config for each project for
 each series - e.g. trunk of everything, Icehouse of servers with trunk
 of everything else, etc and make that easily acccessible?

 -Rob

 --
 Robert Collins rbtcoll...@hp.com
 Distinguished Technologist
 HP Converged Cloud

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



There are already some samples in the 'configuration reference' section of
docs.openstack, eg:

http://docs.openstack.org/havana/config-reference/content/ch_configuring-openstack-identity.html#sample-configuration-files

However the compute and image sections opt for a formatted table, and the
network section is more like an installation guide.

If the samples are to be removed from github, perhaps our configuration
reference section could be first and foremost the set of sample
configuration files for each project + plugin, rather than them being
merely a part of the reference doc as it currently exists.

I fairly consistently refer to the github copies of the samples. They also
allow me to refer to specific lines of the config when explaining concepts
over text. I am not against their removal, but if we were to remove them
I'd disappointed if I had to search very far on docs.openstack.org to get
to them, and I would want the raw files instead of something formatted.

 - Michael
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [all] sample config files should be ignored in git...

2014-03-27 Thread Andreas Jaeger
On 03/27/2014 06:10 AM, Robert Collins wrote:
 On 27 March 2014 17:30, Tom Fifield t...@openstack.org wrote:
 
 Does anyone disagree?

 /me raises hand

 When I was an operator, I regularly referred to the sample config files
 in the git repository.

 If there weren't generated versions of the sample config in the repo, I
 would probably grep the code (not an ideal user experience!). Running
 some random script that I don't know about the existence and might
 depend on having something else installed of is probably not something
 that would happen.
 
 So, I think its important you have sample configs to refer to.
 
 Do they need to be in the git repo?
 
 Note that because libraries now export config options (which is the
 root of this problem!) you cannot ever know from the source all the
 options for a service - you *must* know the library versions you are
 running, to interrogate them for their options.

And how shall we document this properly in the manuals?

 We can - and should - have a discussion about the appropriateness of
 the layering leak we have today, but in the meantime this is breaking
 multiple projects every time any shared library that uses oslo.config
 changes any config option... so we need to solve the workflow aspect.

Please together with the documentation team.

 How about we make a copy of the latest config for each project for
 each series - e.g. trunk of everything, Icehouse of servers with trunk
 of everything else, etc and make that easily acccessible?


Andreas
-- 
 Andreas Jaeger aj@{suse.com,opensuse.org} Twitter: jaegerandi
  SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
   GF: Jeff Hawn,Jennifer Guild,Felix Imendörffer,HRB16746 (AG Nürnberg)
GPG fingerprint = 93A3 365E CE47 B889 DF7F  FED1 389A 563C C272 A126

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [all] sample config files should be ignored in git...

2014-03-27 Thread Flavio Percoco

On 27/03/14 18:10 +1300, Robert Collins wrote:

On 27 March 2014 17:30, Tom Fifield t...@openstack.org wrote:


Does anyone disagree?


/me raises hand

When I was an operator, I regularly referred to the sample config files
in the git repository.

If there weren't generated versions of the sample config in the repo, I
would probably grep the code (not an ideal user experience!). Running
some random script that I don't know about the existence and might
depend on having something else installed of is probably not something
that would happen.


So, I think its important you have sample configs to refer to.

Do they need to be in the git repo?

Note that because libraries now export config options (which is the
root of this problem!) you cannot ever know from the source all the
options for a service - you *must* know the library versions you are
running, to interrogate them for their options.

We can - and should - have a discussion about the appropriateness of
the layering leak we have today, but in the meantime this is breaking
multiple projects every time any shared library that uses oslo.config
changes any config option... so we need to solve the workflow aspect.

How about we make a copy of the latest config for each project for
each series - e.g. trunk of everything, Icehouse of servers with trunk
of everything else, etc and make that easily acccessible?


I'd agree with the original proposal if - and only if - something like
what Robert proposed here is done.

I'd say the config file could be generated for each milestone cut and
live in the milestone branch.

As Tom pointed out, referring to the sample configs is very useful
from many points of view (operations, support, development etc).

Cheers,
Flavio

--
@flaper87
Flavio Percoco


pgp5FItgZgpGb.pgp
Description: PGP signature
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [all] sample config files should be ignored in git...

2014-03-27 Thread Dirk Müller
Hi,

 When I was an operator, I regularly referred to the sample config files
 in the git repository.

The sample config files in git repository are tremendeously useful for
any operator and OpenStack Packager. Having them generateable with a
tox line is very cumbersome.

As a minimum those config files should be part of the sdist tarball
(aka generated during sdist time).

 Do they need to be in the git repo?

IMHO yes, they should go alongside the code change.

 Note that because libraries now export config options (which is the
 root of this problem!) you cannot ever know from the source all the
 options for a service - you *must* know the library versions you are
 running, to interrogate them for their options.

The problem is that we hammer in all the libraries configuration
option into the main config file. if we'd have include support and
we'd just include the libraries config options that are generated as a
separate file (and possibly autogenerated) this problem would not
occur, and it would avoid the gate breakages.


Thanks,
Dirk

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [all] sample config files should be ignored in git...

2014-03-27 Thread Doug Hellmann
On Thu, Mar 27, 2014 at 5:21 AM, Dirk Müller d...@dmllr.de wrote:

 Hi,

  When I was an operator, I regularly referred to the sample config files
  in the git repository.

 The sample config files in git repository are tremendeously useful for
 any operator and OpenStack Packager. Having them generateable with a
 tox line is very cumbersome.


 As a minimum those config files should be part of the sdist tarball
 (aka generated during sdist time).

  Do they need to be in the git repo?

 IMHO yes, they should go alongside the code change.

  Note that because libraries now export config options (which is the
  root of this problem!) you cannot ever know from the source all the
  options for a service - you *must* know the library versions you are
  running, to interrogate them for their options.

 The problem is that we hammer in all the libraries configuration
 option into the main config file. if we'd have include support and
 we'd just include the libraries config options that are generated as a
 separate file (and possibly autogenerated) this problem would not
 occur, and it would avoid the gate breakages.


Do we need an include directive? We could use the existing --config-dir
option to read more than one file at runtime.

Doug





 Thanks,
 Dirk

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [all] sample config files should be ignored in git...

2014-03-27 Thread Joe Gordon
On Wed, Mar 26, 2014 at 11:53 PM, Michael Chapman wop...@gmail.com wrote:

 On Thu, Mar 27, 2014 at 4:10 PM, Robert Collins robe...@robertcollins.net
  wrote:

 On 27 March 2014 17:30, Tom Fifield t...@openstack.org wrote:

  Does anyone disagree?
 
  /me raises hand
 
  When I was an operator, I regularly referred to the sample config files
  in the git repository.
 
  If there weren't generated versions of the sample config in the repo, I
  would probably grep the code (not an ideal user experience!). Running
  some random script that I don't know about the existence and might
  depend on having something else installed of is probably not something
  that would happen.

 So, I think its important you have sample configs to refer to.

 Do they need to be in the git repo?

 Note that because libraries now export config options (which is the
 root of this problem!) you cannot ever know from the source all the
 options for a service - you *must* know the library versions you are
 running, to interrogate them for their options.

 We can - and should - have a discussion about the appropriateness of
 the layering leak we have today, but in the meantime this is breaking
 multiple projects every time any shared library that uses oslo.config
 changes any config option... so we need to solve the workflow aspect.

 How about we make a copy of the latest config for each project for
 each series - e.g. trunk of everything, Icehouse of servers with trunk
 of everything else, etc and make that easily acccessible?

 -Rob

 --
 Robert Collins rbtcoll...@hp.com
 Distinguished Technologist
 HP Converged Cloud

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



 There are already some samples in the 'configuration reference' section of
 docs.openstack, eg:


 http://docs.openstack.org/havana/config-reference/content/ch_configuring-openstack-identity.html#sample-configuration-files

 However the compute and image sections opt for a formatted table, and the
 network section is more like an installation guide.

 If the samples are to be removed from github, perhaps our configuration
 reference section could be first and foremost the set of sample
 configuration files for each project + plugin, rather than them being
 merely a part of the reference doc as it currently exists.

 I fairly consistently refer to the github copies of the samples. They also
 allow me to refer to specific lines of the config when explaining concepts
 over text. I am not against their removal, but if we were to remove them
 I'd disappointed if I had to search very far on docs.openstack.org to get
 to them, and I would want the raw files instead of something formatted.


++, This sounds like a good approach, If we make the config samples very
easy to find on docs.openstack.org and make them automatically regenerate
to be up to date then the workflow for everyone who used to use the in tree
samples only changes slightly, just look at at new website for the same
thing.




  - Michael

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [all] sample config files should be ignored in git...

2014-03-27 Thread Joe Gordon
On Thu, Mar 27, 2014 at 2:21 AM, Dirk Müller d...@dmllr.de wrote:

 Hi,

  When I was an operator, I regularly referred to the sample config files
  in the git repository.

 The sample config files in git repository are tremendeously useful for
 any operator and OpenStack Packager. Having them generateable with a
 tox line is very cumbersome.


Why is it cumbersome? We do the same thing.


 As a minimum those config files should be part of the sdist tarball
 (aka generated during sdist time).

  Do they need to be in the git repo?

 IMHO yes, they should go alongside the code change.


Why? We don't include any other automatically generated files (or at least
try not too).

What about if you could just go to docs.openstack.org and find them with a
single click?



  Note that because libraries now export config options (which is the
  root of this problem!) you cannot ever know from the source all the
  options for a service - you *must* know the library versions you are
  running, to interrogate them for their options.

 The problem is that we hammer in all the libraries configuration
 option into the main config file. if we'd have include support and
 we'd just include the libraries config options that are generated as a
 separate file (and possibly autogenerated) this problem would not
 occur, and it would avoid the gate breakages.


 Thanks,
 Dirk

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [all] sample config files should be ignored in git...

2014-03-27 Thread Sean Dague
On 03/27/2014 02:54 PM, Joe Gordon wrote:
 
 
 
 On Thu, Mar 27, 2014 at 2:21 AM, Dirk Müller d...@dmllr.de
 mailto:d...@dmllr.de wrote:
 
 Hi,
 
  When I was an operator, I regularly referred to the sample config
 files
  in the git repository.
 
 The sample config files in git repository are tremendeously useful for
 any operator and OpenStack Packager. Having them generateable with a
 tox line is very cumbersome.
 
 
 Why is it cumbersome? We do the same thing.

Because we've already got a working tox environment. Which includes
knowing, in advance that you can't just pip install tox (as 1.7.x is
broken), and that you need to have postgresql and mysql and libffi dev
packages installed, and a C compiler.

Start with a pristine Linux it is a lot manual steps you have to go
through to get a working config out of tox -e genconfig.

So I think it's a fair concern that we did just move a burden back onto
users because we dug a hole by letting libraries declare arbitrary
required variables in our config files.

-Sean

-- 
Sean Dague
Samsung Research America
s...@dague.net / sean.da...@samsung.com
http://dague.net



signature.asc
Description: OpenPGP digital signature
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [all] sample config files should be ignored in git...

2014-03-27 Thread Joe Gordon
On Thu, Mar 27, 2014 at 1:11 PM, Sean Dague s...@dague.net wrote:

 On 03/27/2014 02:54 PM, Joe Gordon wrote:
 
 
 
  On Thu, Mar 27, 2014 at 2:21 AM, Dirk Müller d...@dmllr.de
  mailto:d...@dmllr.de wrote:
 
  Hi,
 
   When I was an operator, I regularly referred to the sample config
  files
   in the git repository.
 
  The sample config files in git repository are tremendeously useful
 for
  any operator and OpenStack Packager. Having them generateable with a
  tox line is very cumbersome.
 
 
  Why is it cumbersome? We do the same thing.

 Because we've already got a working tox environment. Which includes
 knowing, in advance that you can't just pip install tox (as 1.7.x is
 broken), and that you need to have postgresql and mysql and libffi dev
 packages installed, and a C compiler.

 Start with a pristine Linux it is a lot manual steps you have to go
 through to get a working config out of tox -e genconfig.

 So I think it's a fair concern that we did just move a burden back onto
 users because we dug a hole by letting libraries declare arbitrary
 required variables in our config files.


Good answer.



 -Sean

 --
 Sean Dague
 Samsung Research America
 s...@dague.net / sean.da...@samsung.com
 http://dague.net


 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [all] sample config files should be ignored in git...

2014-03-27 Thread Clint Byrum
Excerpts from Sean Dague's message of 2014-03-27 13:11:57 -0700:
 On 03/27/2014 02:54 PM, Joe Gordon wrote:
  
  
  
  On Thu, Mar 27, 2014 at 2:21 AM, Dirk Müller d...@dmllr.de
  mailto:d...@dmllr.de wrote:
  
  Hi,
  
   When I was an operator, I regularly referred to the sample config
  files
   in the git repository.
  
  The sample config files in git repository are tremendeously useful for
  any operator and OpenStack Packager. Having them generateable with a
  tox line is very cumbersome.
  
  
  Why is it cumbersome? We do the same thing.
 
 Because we've already got a working tox environment. Which includes
 knowing, in advance that you can't just pip install tox (as 1.7.x is
 broken), and that you need to have postgresql and mysql and libffi dev
 packages installed, and a C compiler.
 
 Start with a pristine Linux it is a lot manual steps you have to go
 through to get a working config out of tox -e genconfig.
 
 So I think it's a fair concern that we did just move a burden back onto
 users because we dug a hole by letting libraries declare arbitrary
 required variables in our config files.
 

This is pretty standard in the open source world. Git trees do not have
all of the things that the user needs. Git trees have all the things
that the project provides.

If this were autotools we'd have people run 'autoreconf' and/or 'make
doc'. That would likely involve installing autotools, and might also
require some libraries to be present to build tools that are used to
generate things.

I would have a hard time supporting users who don't read the README
before trying to make use of a git tree to use/install/configure a piece
of software. As long as those steps are spelled out, and the releases
contain this generated file, I'm +1 on removing it from git.

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [all] sample config files should be ignored in git...

2014-03-27 Thread Tom Fifield

On 28/03/14 04:58, Joe Gordon wrote:




On Thu, Mar 27, 2014 at 1:11 PM, Sean Dague s...@dague.net
mailto:s...@dague.net wrote:

On 03/27/2014 02:54 PM, Joe Gordon wrote:
 
 
 
  On Thu, Mar 27, 2014 at 2:21 AM, Dirk Müller d...@dmllr.de
mailto:d...@dmllr.de
  mailto:d...@dmllr.de mailto:d...@dmllr.de wrote:
 
  Hi,
 
   When I was an operator, I regularly referred to the sample
config
  files
   in the git repository.
 
  The sample config files in git repository are tremendeously
useful for
  any operator and OpenStack Packager. Having them generateable
with a
  tox line is very cumbersome.
 
 
  Why is it cumbersome? We do the same thing.

Because we've already got a working tox environment. Which includes
knowing, in advance that you can't just pip install tox (as 1.7.x is
broken), and that you need to have postgresql and mysql and libffi dev
packages installed, and a C compiler.

Start with a pristine Linux it is a lot manual steps you have to go
through to get a working config out of tox -e genconfig.

So I think it's a fair concern that we did just move a burden back onto
users because we dug a hole by letting libraries declare arbitrary
required variables in our config files.


Good answer.


+1


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] [all] sample config files should be ignored in git...

2014-03-26 Thread Clint Byrum
This is an issue that affects all of our git repos. If you are using
oslo.config, you will likely also be using the sample config generator.

However, for some reason we are all checking this generated file in.
This makes no sense, as we humans are not editting it, and it often
picks up config files from other things like libraries (keystoneclient
in particular). This has lead to breakage in the gate a few times for
Heat, perhaps for others as well.

I move that we all rm this file from our git trees, and start generating
it as part of the install/dist process (I have no idea how to do
this..). This would require:

- rm sample files and add them to .gitignore in all trees
- Removing check_uptodate.sh from all trees/tox.ini's
- Generating file during dist/install process.

Does anyone disagree?

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [all] sample config files should be ignored in git...

2014-03-26 Thread Russell Bryant
On 03/26/2014 02:10 PM, Clint Byrum wrote:
 This is an issue that affects all of our git repos. If you are using
 oslo.config, you will likely also be using the sample config generator.
 
 However, for some reason we are all checking this generated file in.
 This makes no sense, as we humans are not editting it, and it often
 picks up config files from other things like libraries (keystoneclient
 in particular). This has lead to breakage in the gate a few times for
 Heat, perhaps for others as well.
 
 I move that we all rm this file from our git trees, and start generating
 it as part of the install/dist process (I have no idea how to do
 this..). This would require:
 
 - rm sample files and add them to .gitignore in all trees
 - Removing check_uptodate.sh from all trees/tox.ini's
 - Generating file during dist/install process.
 
 Does anyone disagree?

This has been done in Nova, except we don't have it generated during
install.  We just have instructions and a tox target that will do it if
you choose to.

https://git.openstack.org/cgit/openstack/nova/tree/etc/nova/README-nova.conf.txt

Related, adding instructions to generate without tox:
https://review.openstack.org/#/c/82533/

-- 
Russell Bryant

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [all] sample config files should be ignored in git...

2014-03-26 Thread Zane Bitter

On 26/03/14 14:10, Clint Byrum wrote:

This is an issue that affects all of our git repos. If you are using
oslo.config, you will likely also be using the sample config generator.

However, for some reason we are all checking this generated file in.
This makes no sense, as we humans are not editting it, and it often
picks up config files from other things like libraries (keystoneclient
in particular). This has lead to breakage in the gate a few times for
Heat, perhaps for others as well.


Just to put the other side of this... the latest change to oslo.config 
has produced a *completely broken* config file in Heat (due to the fix 
for bug #1262148 landing in oslo.config - see  bug #1288586 for gory 
details).


The fact that we have to make a change to a file in the repository that 
goes through code review means that we are able to see that. If it were 
silently generated with no human intervention, we would be shipping 
garbage right now.



That said, the fact that config files have to match to pass the gate, as 
they do currently, also makes it very hard to actually fix the bug. So 
I'm not sure what the right answer is here.


cheers,
Zane.


I move that we all rm this file from our git trees, and start generating
it as part of the install/dist process (I have no idea how to do
this..). This would require:

- rm sample files and add them to .gitignore in all trees
- Removing check_uptodate.sh from all trees/tox.ini's
- Generating file during dist/install process.

Does anyone disagree?

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [all] sample config files should be ignored in git...

2014-03-26 Thread Anne Gentle
On Wed, Mar 26, 2014 at 1:10 PM, Clint Byrum cl...@fewbar.com wrote:

 This is an issue that affects all of our git repos. If you are using
 oslo.config, you will likely also be using the sample config generator.

 However, for some reason we are all checking this generated file in.
 This makes no sense, as we humans are not editting it, and it often
 picks up config files from other things like libraries (keystoneclient
 in particular). This has lead to breakage in the gate a few times for
 Heat, perhaps for others as well.

 I move that we all rm this file from our git trees, and start generating
 it as part of the install/dist process (I have no idea how to do
 this..). This would require:

 - rm sample files and add them to .gitignore in all trees
 - Removing check_uptodate.sh from all trees/tox.ini's
 - Generating file during dist/install process.

 Does anyone disagree?


The documentation currently points to the latest copy of the generated
files when they're available. I'd like to continue having those generated
and looked at by humans in reviews.

I think if you asked non-devs, such as deployers, you'd find wider uses.
Can you poll another group in addition to this mailing list?

Thanks,
Anne



 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [all] sample config files should be ignored in git...

2014-03-26 Thread Clint Byrum
Excerpts from Anne Gentle's message of 2014-03-26 11:49:29 -0700:
 On Wed, Mar 26, 2014 at 1:10 PM, Clint Byrum cl...@fewbar.com wrote:
 
  This is an issue that affects all of our git repos. If you are using
  oslo.config, you will likely also be using the sample config generator.
 
  However, for some reason we are all checking this generated file in.
  This makes no sense, as we humans are not editting it, and it often
  picks up config files from other things like libraries (keystoneclient
  in particular). This has lead to breakage in the gate a few times for
  Heat, perhaps for others as well.
 
  I move that we all rm this file from our git trees, and start generating
  it as part of the install/dist process (I have no idea how to do
  this..). This would require:
 
  - rm sample files and add them to .gitignore in all trees
  - Removing check_uptodate.sh from all trees/tox.ini's
  - Generating file during dist/install process.
 
  Does anyone disagree?
 
 
 The documentation currently points to the latest copy of the generated
 files when they're available. I'd like to continue having those generated
 and looked at by humans in reviews.
 
 I think if you asked non-devs, such as deployers, you'd find wider uses.
 Can you poll another group in addition to this mailing list?
 

The way I see it, the generated sample config file is itself
documentation. Perhaps that file should actually go where the docs go,
rather than sitting in the git tree.

With other libraries causing changes anyway, we're not really reviewing
every change anyway, otherwise I wouldn't have sent this message. We
weren't able to review what keystoneclient did before it broke our
gate. Keystoneclient isn't going to review the matrix of dependent repos
for breakage there either.

We do review the code changes that lead to the relevant changes in our
samples, and that _should_ be enough. It works the same with all of
our other code-born documentation (such as the Heat template guide). So
I'm comfortable saying that reviewers should be able to catch obvious
things that would break the sample configs from the code alone, in the
same way that reviewers would find such an error in other such generated
documentation.

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [all] sample config files should be ignored in git...

2014-03-26 Thread Clint Byrum
Excerpts from Steven Hardy's message of 2014-03-26 11:27:58 -0700:
 On Wed, Mar 26, 2014 at 11:10:04AM -0700, Clint Byrum wrote:
  This is an issue that affects all of our git repos. If you are using
  oslo.config, you will likely also be using the sample config generator.
  
  However, for some reason we are all checking this generated file in.
  This makes no sense, as we humans are not editting it, and it often
  picks up config files from other things like libraries (keystoneclient
  in particular). This has lead to breakage in the gate a few times for
  Heat, perhaps for others as well.
  
  I move that we all rm this file from our git trees, and start generating
  it as part of the install/dist process (I have no idea how to do
  this..). This would require:
  
  - rm sample files and add them to .gitignore in all trees
  - Removing check_uptodate.sh from all trees/tox.ini's
  - Generating file during dist/install process.
  
  Does anyone disagree?
 
 So this sounds like a great idea in theory, I'd love to stop getting
 surprise gate breakage every keystoneclient release becuause of minor
 changes to keystone_authtoken.
 
 My main concern is we're replacing suprise breakage due to keystoneclient
 with surprise breakage due to oslo.config, since that version is not
 capped.
 
 E.g look at this review I just posted (too hastily) - generate_sample.sh
 has done something crazy and generated a totally broken config:
 
 https://review.openstack.org/#/c/83151/
 
 I'm not quite clear on why that's broken, but it does highlight one of the
 problems with relying on autogeneration with no review.  I guess we'll get
 to review the logs of the broken gate tests instead :\
 
 I'd love to hear ideas on how we can do this in an automated way which
 won't be really unstable/unreliable.

This seems like a real bug, and one that would hopefully be handled
through the normal bug fixing cycle that includes ensuring test
coverage. No?

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [all] sample config files should be ignored in git...

2014-03-26 Thread Robert Collins
On 27 March 2014 17:30, Tom Fifield t...@openstack.org wrote:

 Does anyone disagree?

 /me raises hand

 When I was an operator, I regularly referred to the sample config files
 in the git repository.

 If there weren't generated versions of the sample config in the repo, I
 would probably grep the code (not an ideal user experience!). Running
 some random script that I don't know about the existence and might
 depend on having something else installed of is probably not something
 that would happen.

So, I think its important you have sample configs to refer to.

Do they need to be in the git repo?

Note that because libraries now export config options (which is the
root of this problem!) you cannot ever know from the source all the
options for a service - you *must* know the library versions you are
running, to interrogate them for their options.

We can - and should - have a discussion about the appropriateness of
the layering leak we have today, but in the meantime this is breaking
multiple projects every time any shared library that uses oslo.config
changes any config option... so we need to solve the workflow aspect.

How about we make a copy of the latest config for each project for
each series - e.g. trunk of everything, Icehouse of servers with trunk
of everything else, etc and make that easily acccessible?

-Rob

-- 
Robert Collins rbtcoll...@hp.com
Distinguished Technologist
HP Converged Cloud

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev