Re: [openstack-dev] [all] sample config files should be ignored in git...

[Tom Fifield]

On 28/03/14 04:58, Joe Gordon wrote:




On Thu, Mar 27, 2014 at 1:11 PM, Sean Dague mailto:s...@dague.net>> wrote:

On 03/27/2014 02:54 PM, Joe Gordon wrote:
 >
 >
 >
 > On Thu, Mar 27, 2014 at 2:21 AM, Dirk Müller mailto:d...@dmllr.de>
 > >> wrote:
 >
 > Hi,
 >
 > >> When I was an operator, I regularly referred to the sample
config
 > files
 > >> in the git repository.
 >
 > The sample config files in git repository are tremendeously
useful for
 > any operator and OpenStack Packager. Having them generateable
with a
 > tox line is very cumbersome.
 >
 >
 > Why is it cumbersome? We do the same thing.

Because we've already got a working tox environment. Which includes
knowing, in advance that you can't just pip install tox (as 1.7.x is
broken), and that you need to have postgresql and mysql and libffi dev
packages installed, and a C compiler.

Start with a pristine Linux it is a lot manual steps you have to go
through to get a working config out of tox -e genconfig.

So I think it's a fair concern that we did just move a burden back onto
users because we dug a hole by letting libraries declare arbitrary
required variables in our config files.


Good answer.


+1


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] sample config files should be ignored in git...

[Clint Byrum]

Excerpts from Sean Dague's message of 2014-03-27 13:11:57 -0700:
> On 03/27/2014 02:54 PM, Joe Gordon wrote:
> > 
> > 
> > 
> > On Thu, Mar 27, 2014 at 2:21 AM, Dirk Müller  > > wrote:
> > 
> > Hi,
> > 
> > >> When I was an operator, I regularly referred to the sample config
> > files
> > >> in the git repository.
> > 
> > The sample config files in git repository are tremendeously useful for
> > any operator and OpenStack Packager. Having them generateable with a
> > tox line is very cumbersome.
> > 
> > 
> > Why is it cumbersome? We do the same thing.
> 
> Because we've already got a working tox environment. Which includes
> knowing, in advance that you can't just pip install tox (as 1.7.x is
> broken), and that you need to have postgresql and mysql and libffi dev
> packages installed, and a C compiler.
> 
> Start with a pristine Linux it is a lot manual steps you have to go
> through to get a working config out of tox -e genconfig.
> 
> So I think it's a fair concern that we did just move a burden back onto
> users because we dug a hole by letting libraries declare arbitrary
> required variables in our config files.
> 

This is pretty standard in the open source world. Git trees do not have
all of the things that the user needs. Git trees have all the things
that the project provides.

If this were autotools we'd have people run 'autoreconf' and/or 'make
doc'. That would likely involve installing autotools, and might also
require some libraries to be present to build tools that are used to
generate things.

I would have a hard time supporting users who don't read the README
before trying to make use of a git tree to use/install/configure a piece
of software. As long as those steps are spelled out, and the releases
contain this generated file, I'm +1 on removing it from git.

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] sample config files should be ignored in git...

[Joe Gordon]

On Thu, Mar 27, 2014 at 1:11 PM, Sean Dague  wrote:

> On 03/27/2014 02:54 PM, Joe Gordon wrote:
> >
> >
> >
> > On Thu, Mar 27, 2014 at 2:21 AM, Dirk Müller  > > wrote:
> >
> > Hi,
> >
> > >> When I was an operator, I regularly referred to the sample config
> > files
> > >> in the git repository.
> >
> > The sample config files in git repository are tremendeously useful
> for
> > any operator and OpenStack Packager. Having them generateable with a
> > tox line is very cumbersome.
> >
> >
> > Why is it cumbersome? We do the same thing.
>
> Because we've already got a working tox environment. Which includes
> knowing, in advance that you can't just pip install tox (as 1.7.x is
> broken), and that you need to have postgresql and mysql and libffi dev
> packages installed, and a C compiler.
>
> Start with a pristine Linux it is a lot manual steps you have to go
> through to get a working config out of tox -e genconfig.
>
> So I think it's a fair concern that we did just move a burden back onto
> users because we dug a hole by letting libraries declare arbitrary
> required variables in our config files.
>

Good answer.


>
> -Sean
>
> --
> Sean Dague
> Samsung Research America
> s...@dague.net / sean.da...@samsung.com
> http://dague.net
>
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] sample config files should be ignored in git...

[Sean Dague]

On 03/27/2014 02:54 PM, Joe Gordon wrote:
> 
> 
> 
> On Thu, Mar 27, 2014 at 2:21 AM, Dirk Müller  > wrote:
> 
> Hi,
> 
> >> When I was an operator, I regularly referred to the sample config
> files
> >> in the git repository.
> 
> The sample config files in git repository are tremendeously useful for
> any operator and OpenStack Packager. Having them generateable with a
> tox line is very cumbersome.
> 
> 
> Why is it cumbersome? We do the same thing.

Because we've already got a working tox environment. Which includes
knowing, in advance that you can't just pip install tox (as 1.7.x is
broken), and that you need to have postgresql and mysql and libffi dev
packages installed, and a C compiler.

Start with a pristine Linux it is a lot manual steps you have to go
through to get a working config out of tox -e genconfig.

So I think it's a fair concern that we did just move a burden back onto
users because we dug a hole by letting libraries declare arbitrary
required variables in our config files.

-Sean

-- 
Sean Dague
Samsung Research America
s...@dague.net / sean.da...@samsung.com
http://dague.net



signature.asc
Description: OpenPGP digital signature
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] sample config files should be ignored in git...

[Joe Gordon]

On Thu, Mar 27, 2014 at 2:21 AM, Dirk Müller  wrote:

> Hi,
>
> >> When I was an operator, I regularly referred to the sample config files
> >> in the git repository.
>
> The sample config files in git repository are tremendeously useful for
> any operator and OpenStack Packager. Having them generateable with a
> tox line is very cumbersome.
>
>
Why is it cumbersome? We do the same thing.


> As a minimum those config files should be part of the sdist tarball
> (aka generated during sdist time).
>
> > Do they need to be in the git repo?
>
> IMHO yes, they should go alongside the code change.
>

Why? We don't include any other automatically generated files (or at least
try not too).

What about if you could just go to docs.openstack.org and find them with a
single click?


>
> > Note that because libraries now export config options (which is the
> > root of this problem!) you cannot ever know from the source all the
> > options for a service - you *must* know the library versions you are
> > running, to interrogate them for their options.
>
> The problem is that we hammer in all the libraries configuration
> option into the main config file. if we'd have "include" support and
> we'd just include the libraries config options that are generated as a
> separate file (and possibly autogenerated) this problem would not
> occur, and it would avoid the gate breakages.
>
>
> Thanks,
> Dirk
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] sample config files should be ignored in git...

[Joe Gordon]

On Wed, Mar 26, 2014 at 11:53 PM, Michael Chapman  wrote:

> On Thu, Mar 27, 2014 at 4:10 PM, Robert Collins  > wrote:
>
>> On 27 March 2014 17:30, Tom Fifield  wrote:
>>
>> >> Does anyone disagree?
>> >
>> > /me raises hand
>> >
>> > When I was an operator, I regularly referred to the sample config files
>> > in the git repository.
>> >
>> > If there weren't generated versions of the sample config in the repo, I
>> > would probably grep the code (not an ideal user experience!). Running
>> > some random script that I don't know about the existence and might
>> > depend on having something else installed of is probably not something
>> > that would happen.
>>
>> So, I think its important you have sample configs to refer to.
>>
>> Do they need to be in the git repo?
>>
>> Note that because libraries now export config options (which is the
>> root of this problem!) you cannot ever know from the source all the
>> options for a service - you *must* know the library versions you are
>> running, to interrogate them for their options.
>>
>> We can - and should - have a discussion about the appropriateness of
>> the layering leak we have today, but in the meantime this is breaking
>> multiple projects every time any shared library that uses oslo.config
>> changes any config option... so we need to solve the workflow aspect.
>>
>> How about we make a copy of the latest config for each project for
>> each series - e.g. trunk of everything, Icehouse of servers with trunk
>> of everything else, etc and make that easily acccessible?
>>
>> -Rob
>>
>> --
>> Robert Collins 
>> Distinguished Technologist
>> HP Converged Cloud
>>
>> ___
>> OpenStack-dev mailing list
>> OpenStack-dev@lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
>
> There are already some samples in the 'configuration reference' section of
> docs.openstack, eg:
>
>
> http://docs.openstack.org/havana/config-reference/content/ch_configuring-openstack-identity.html#sample-configuration-files
>
> However the compute and image sections opt for a formatted table, and the
> network section is more like an installation guide.
>
> If the samples are to be removed from github, perhaps our configuration
> reference section could be first and foremost the set of sample
> configuration files for each project + plugin, rather than them being
> merely a part of the reference doc as it currently exists.
>
> I fairly consistently refer to the github copies of the samples. They also
> allow me to refer to specific lines of the config when explaining concepts
> over text. I am not against their removal, but if we were to remove them
> I'd disappointed if I had to search very far on docs.openstack.org to get
> to them, and I would want the raw files instead of something formatted.
>

++, This sounds like a good approach, If we make the config samples very
easy to find on docs.openstack.org and make them automatically regenerate
to be up to date then the workflow for everyone who used to use the in tree
samples only changes slightly, just look at at new website for the same
thing.


>
>
>  - Michael
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] sample config files should be ignored in git...

[Doug Hellmann]

On Thu, Mar 27, 2014 at 5:21 AM, Dirk Müller  wrote:

> Hi,
>
> >> When I was an operator, I regularly referred to the sample config files
> >> in the git repository.
>
> The sample config files in git repository are tremendeously useful for
> any operator and OpenStack Packager. Having them generateable with a
> tox line is very cumbersome.
>

> As a minimum those config files should be part of the sdist tarball
> (aka generated during sdist time).
>
> > Do they need to be in the git repo?
>
> IMHO yes, they should go alongside the code change.
>
> > Note that because libraries now export config options (which is the
> > root of this problem!) you cannot ever know from the source all the
> > options for a service - you *must* know the library versions you are
> > running, to interrogate them for their options.
>
> The problem is that we hammer in all the libraries configuration
> option into the main config file. if we'd have "include" support and
> we'd just include the libraries config options that are generated as a
> separate file (and possibly autogenerated) this problem would not
> occur, and it would avoid the gate breakages.
>

Do we need an "include" directive? We could use the existing --config-dir
option to read more than one file at runtime.

Doug



>
>
> Thanks,
> Dirk
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] sample config files should be ignored in git...

[Dirk Müller]

Hi,

>> When I was an operator, I regularly referred to the sample config files
>> in the git repository.

The sample config files in git repository are tremendeously useful for
any operator and OpenStack Packager. Having them generateable with a
tox line is very cumbersome.

As a minimum those config files should be part of the sdist tarball
(aka generated during sdist time).

> Do they need to be in the git repo?

IMHO yes, they should go alongside the code change.

> Note that because libraries now export config options (which is the
> root of this problem!) you cannot ever know from the source all the
> options for a service - you *must* know the library versions you are
> running, to interrogate them for their options.

The problem is that we hammer in all the libraries configuration
option into the main config file. if we'd have "include" support and
we'd just include the libraries config options that are generated as a
separate file (and possibly autogenerated) this problem would not
occur, and it would avoid the gate breakages.


Thanks,
Dirk

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] sample config files should be ignored in git...

[Flavio Percoco]

On 27/03/14 18:10 +1300, Robert Collins wrote:

On 27 March 2014 17:30, Tom Fifield  wrote:


Does anyone disagree?


/me raises hand

When I was an operator, I regularly referred to the sample config files
in the git repository.

If there weren't generated versions of the sample config in the repo, I
would probably grep the code (not an ideal user experience!). Running
some random script that I don't know about the existence and might
depend on having something else installed of is probably not something
that would happen.


So, I think its important you have sample configs to refer to.

Do they need to be in the git repo?

Note that because libraries now export config options (which is the
root of this problem!) you cannot ever know from the source all the
options for a service - you *must* know the library versions you are
running, to interrogate them for their options.

We can - and should - have a discussion about the appropriateness of
the layering leak we have today, but in the meantime this is breaking
multiple projects every time any shared library that uses oslo.config
changes any config option... so we need to solve the workflow aspect.

How about we make a copy of the latest config for each project for
each series - e.g. trunk of everything, Icehouse of servers with trunk
of everything else, etc and make that easily acccessible?


I'd agree with the original proposal if - and only if - something like
what Robert proposed here is done.

I'd say the config file could be generated for each milestone cut and
live in the milestone branch.

As Tom pointed out, referring to the sample configs is very useful
from many points of view (operations, support, development etc).

Cheers,
Flavio

--
@flaper87
Flavio Percoco


pgp5FItgZgpGb.pgp
Description: PGP signature
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] sample config files should be ignored in git...

[Andreas Jaeger]

On 03/27/2014 06:10 AM, Robert Collins wrote:
> On 27 March 2014 17:30, Tom Fifield  wrote:
> 
>>> Does anyone disagree?
>>
>> /me raises hand
>>
>> When I was an operator, I regularly referred to the sample config files
>> in the git repository.
>>
>> If there weren't generated versions of the sample config in the repo, I
>> would probably grep the code (not an ideal user experience!). Running
>> some random script that I don't know about the existence and might
>> depend on having something else installed of is probably not something
>> that would happen.
> 
> So, I think its important you have sample configs to refer to.
> 
> Do they need to be in the git repo?
> 
> Note that because libraries now export config options (which is the
> root of this problem!) you cannot ever know from the source all the
> options for a service - you *must* know the library versions you are
> running, to interrogate them for their options.

And how shall we document this properly in the manuals?

> We can - and should - have a discussion about the appropriateness of
> the layering leak we have today, but in the meantime this is breaking
> multiple projects every time any shared library that uses oslo.config
> changes any config option... so we need to solve the workflow aspect.

Please together with the documentation team.

> How about we make a copy of the latest config for each project for
> each series - e.g. trunk of everything, Icehouse of servers with trunk
> of everything else, etc and make that easily acccessible?


Andreas
-- 
 Andreas Jaeger aj@{suse.com,opensuse.org} Twitter: jaegerandi
  SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
   GF: Jeff Hawn,Jennifer Guild,Felix Imendörffer,HRB16746 (AG Nürnberg)
GPG fingerprint = 93A3 365E CE47 B889 DF7F  FED1 389A 563C C272 A126

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] sample config files should be ignored in git...

[Michael Chapman]

On Thu, Mar 27, 2014 at 4:10 PM, Robert Collins
wrote:

> On 27 March 2014 17:30, Tom Fifield  wrote:
>
> >> Does anyone disagree?
> >
> > /me raises hand
> >
> > When I was an operator, I regularly referred to the sample config files
> > in the git repository.
> >
> > If there weren't generated versions of the sample config in the repo, I
> > would probably grep the code (not an ideal user experience!). Running
> > some random script that I don't know about the existence and might
> > depend on having something else installed of is probably not something
> > that would happen.
>
> So, I think its important you have sample configs to refer to.
>
> Do they need to be in the git repo?
>
> Note that because libraries now export config options (which is the
> root of this problem!) you cannot ever know from the source all the
> options for a service - you *must* know the library versions you are
> running, to interrogate them for their options.
>
> We can - and should - have a discussion about the appropriateness of
> the layering leak we have today, but in the meantime this is breaking
> multiple projects every time any shared library that uses oslo.config
> changes any config option... so we need to solve the workflow aspect.
>
> How about we make a copy of the latest config for each project for
> each series - e.g. trunk of everything, Icehouse of servers with trunk
> of everything else, etc and make that easily acccessible?
>
> -Rob
>
> --
> Robert Collins 
> Distinguished Technologist
> HP Converged Cloud
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>


There are already some samples in the 'configuration reference' section of
docs.openstack, eg:

http://docs.openstack.org/havana/config-reference/content/ch_configuring-openstack-identity.html#sample-configuration-files

However the compute and image sections opt for a formatted table, and the
network section is more like an installation guide.

If the samples are to be removed from github, perhaps our configuration
reference section could be first and foremost the set of sample
configuration files for each project + plugin, rather than them being
merely a part of the reference doc as it currently exists.

I fairly consistently refer to the github copies of the samples. They also
allow me to refer to specific lines of the config when explaining concepts
over text. I am not against their removal, but if we were to remove them
I'd disappointed if I had to search very far on docs.openstack.org to get
to them, and I would want the raw files instead of something formatted.

 - Michael
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] sample config files should be ignored in git...

[Robert Collins]

On 27 March 2014 17:30, Tom Fifield  wrote:

>> Does anyone disagree?
>
> /me raises hand
>
> When I was an operator, I regularly referred to the sample config files
> in the git repository.
>
> If there weren't generated versions of the sample config in the repo, I
> would probably grep the code (not an ideal user experience!). Running
> some random script that I don't know about the existence and might
> depend on having something else installed of is probably not something
> that would happen.

So, I think its important you have sample configs to refer to.

Do they need to be in the git repo?

Note that because libraries now export config options (which is the
root of this problem!) you cannot ever know from the source all the
options for a service - you *must* know the library versions you are
running, to interrogate them for their options.

We can - and should - have a discussion about the appropriateness of
the layering leak we have today, but in the meantime this is breaking
multiple projects every time any shared library that uses oslo.config
changes any config option... so we need to solve the workflow aspect.

How about we make a copy of the latest config for each project for
each series - e.g. trunk of everything, Icehouse of servers with trunk
of everything else, etc and make that easily acccessible?

-Rob

-- 
Robert Collins 
Distinguished Technologist
HP Converged Cloud

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] sample config files should be ignored in git...

[Tom Fifield]

On 27/03/14 02:10, Clint Byrum wrote:
> This is an issue that affects all of our git repos. If you are using
> oslo.config, you will likely also be using the sample config generator.
> 
> However, for some reason we are all checking this generated file in.
> This makes no sense, as we humans are not editting it, and it often
> picks up config files from other things like libraries (keystoneclient
> in particular). This has lead to breakage in the gate a few times for
> Heat, perhaps for others as well.
> 
> I move that we all rm this file from our git trees, and start generating
> it as part of the install/dist process (I have no idea how to do
> this..). This would require:
> 
> - rm sample files and add them to .gitignore in all trees
> - Removing check_uptodate.sh from all trees/tox.ini's
> - Generating file during dist/install process.
> 
> Does anyone disagree?

/me raises hand

When I was an operator, I regularly referred to the sample config files
in the git repository.

If there weren't generated versions of the sample config in the repo, I
would probably grep the code (not an ideal user experience!). Running
some random script that I don't know about the existence and might
depend on having something else installed of is probably not something
that would happen.


Regards,


Tom

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] sample config files should be ignored in git...

[Clint Byrum]

Excerpts from Steven Hardy's message of 2014-03-26 11:27:58 -0700:
> On Wed, Mar 26, 2014 at 11:10:04AM -0700, Clint Byrum wrote:
> > This is an issue that affects all of our git repos. If you are using
> > oslo.config, you will likely also be using the sample config generator.
> > 
> > However, for some reason we are all checking this generated file in.
> > This makes no sense, as we humans are not editting it, and it often
> > picks up config files from other things like libraries (keystoneclient
> > in particular). This has lead to breakage in the gate a few times for
> > Heat, perhaps for others as well.
> > 
> > I move that we all rm this file from our git trees, and start generating
> > it as part of the install/dist process (I have no idea how to do
> > this..). This would require:
> > 
> > - rm sample files and add them to .gitignore in all trees
> > - Removing check_uptodate.sh from all trees/tox.ini's
> > - Generating file during dist/install process.
> > 
> > Does anyone disagree?
> 
> So this sounds like a great idea in theory, I'd love to stop getting
> surprise gate breakage every keystoneclient release becuause of minor
> changes to keystone_authtoken.
> 
> My main concern is we're replacing suprise breakage due to keystoneclient
> with surprise breakage due to oslo.config, since that version is not
> capped.
> 
> E.g look at this review I just posted (too hastily) - generate_sample.sh
> has done something crazy and generated a totally broken config:
> 
> https://review.openstack.org/#/c/83151/
> 
> I'm not quite clear on why that's broken, but it does highlight one of the
> problems with relying on autogeneration with no review.  I guess we'll get
> to review the logs of the broken gate tests instead :\
> 
> I'd love to hear ideas on how we can do this in an automated way which
> won't be really unstable/unreliable.

This seems like a real bug, and one that would hopefully be handled
through the normal bug fixing cycle that includes ensuring test
coverage. No?

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] sample config files should be ignored in git...

[Clint Byrum]

Excerpts from Anne Gentle's message of 2014-03-26 11:49:29 -0700:
> On Wed, Mar 26, 2014 at 1:10 PM, Clint Byrum  wrote:
> 
> > This is an issue that affects all of our git repos. If you are using
> > oslo.config, you will likely also be using the sample config generator.
> >
> > However, for some reason we are all checking this generated file in.
> > This makes no sense, as we humans are not editting it, and it often
> > picks up config files from other things like libraries (keystoneclient
> > in particular). This has lead to breakage in the gate a few times for
> > Heat, perhaps for others as well.
> >
> > I move that we all rm this file from our git trees, and start generating
> > it as part of the install/dist process (I have no idea how to do
> > this..). This would require:
> >
> > - rm sample files and add them to .gitignore in all trees
> > - Removing check_uptodate.sh from all trees/tox.ini's
> > - Generating file during dist/install process.
> >
> > Does anyone disagree?
> >
> 
> The documentation currently points to the latest copy of the generated
> files when they're available. I'd like to continue having those generated
> and looked at by humans in reviews.
> 
> I think if you asked non-devs, such as deployers, you'd find wider uses.
> Can you poll another group in addition to this mailing list?
> 

The way I see it, the generated sample config file is itself
documentation. Perhaps that file should actually go where the docs go,
rather than sitting in the git tree.

With other libraries causing changes anyway, we're not really reviewing
every change anyway, otherwise I wouldn't have sent this message. We
weren't able to review what keystoneclient did before it broke our
gate. Keystoneclient isn't going to review the matrix of dependent repos
for breakage there either.

We do review the code changes that lead to the relevant changes in our
samples, and that _should_ be enough. It works the same with all of
our other code-born documentation (such as the Heat template guide). So
I'm comfortable saying that reviewers should be able to catch obvious
things that would break the sample configs from the code alone, in the
same way that reviewers would find such an error in other such generated
documentation.

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] sample config files should be ignored in git...

[Anne Gentle]

On Wed, Mar 26, 2014 at 1:10 PM, Clint Byrum  wrote:

> This is an issue that affects all of our git repos. If you are using
> oslo.config, you will likely also be using the sample config generator.
>
> However, for some reason we are all checking this generated file in.
> This makes no sense, as we humans are not editting it, and it often
> picks up config files from other things like libraries (keystoneclient
> in particular). This has lead to breakage in the gate a few times for
> Heat, perhaps for others as well.
>
> I move that we all rm this file from our git trees, and start generating
> it as part of the install/dist process (I have no idea how to do
> this..). This would require:
>
> - rm sample files and add them to .gitignore in all trees
> - Removing check_uptodate.sh from all trees/tox.ini's
> - Generating file during dist/install process.
>
> Does anyone disagree?
>

The documentation currently points to the latest copy of the generated
files when they're available. I'd like to continue having those generated
and looked at by humans in reviews.

I think if you asked non-devs, such as deployers, you'd find wider uses.
Can you poll another group in addition to this mailing list?

Thanks,
Anne


>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] sample config files should be ignored in git...

[Zane Bitter]

On 26/03/14 14:10, Clint Byrum wrote:

This is an issue that affects all of our git repos. If you are using
oslo.config, you will likely also be using the sample config generator.

However, for some reason we are all checking this generated file in.
This makes no sense, as we humans are not editting it, and it often
picks up config files from other things like libraries (keystoneclient
in particular). This has lead to breakage in the gate a few times for
Heat, perhaps for others as well.


Just to put the other side of this... the latest change to oslo.config 
has produced a *completely broken* config file in Heat (due to the fix 
for bug #1262148 landing in oslo.config - see  bug #1288586 for gory 
details).


The fact that we have to make a change to a file in the repository that 
goes through code review means that we are able to see that. If it were 
silently generated with no human intervention, we would be shipping 
garbage right now.



That said, the fact that config files have to match to pass the gate, as 
they do currently, also makes it very hard to actually fix the bug. So 
I'm not sure what the right answer is here.


cheers,
Zane.


I move that we all rm this file from our git trees, and start generating
it as part of the install/dist process (I have no idea how to do
this..). This would require:

- rm sample files and add them to .gitignore in all trees
- Removing check_uptodate.sh from all trees/tox.ini's
- Generating file during dist/install process.

Does anyone disagree?

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] sample config files should be ignored in git...

[Steven Hardy]

On Wed, Mar 26, 2014 at 11:10:04AM -0700, Clint Byrum wrote:
> This is an issue that affects all of our git repos. If you are using
> oslo.config, you will likely also be using the sample config generator.
> 
> However, for some reason we are all checking this generated file in.
> This makes no sense, as we humans are not editting it, and it often
> picks up config files from other things like libraries (keystoneclient
> in particular). This has lead to breakage in the gate a few times for
> Heat, perhaps for others as well.
> 
> I move that we all rm this file from our git trees, and start generating
> it as part of the install/dist process (I have no idea how to do
> this..). This would require:
> 
> - rm sample files and add them to .gitignore in all trees
> - Removing check_uptodate.sh from all trees/tox.ini's
> - Generating file during dist/install process.
> 
> Does anyone disagree?

So this sounds like a great idea in theory, I'd love to stop getting
surprise gate breakage every keystoneclient release becuause of minor
changes to keystone_authtoken.

My main concern is we're replacing suprise breakage due to keystoneclient
with surprise breakage due to oslo.config, since that version is not
capped.

E.g look at this review I just posted (too hastily) - generate_sample.sh
has done something crazy and generated a totally broken config:

https://review.openstack.org/#/c/83151/

I'm not quite clear on why that's broken, but it does highlight one of the
problems with relying on autogeneration with no review.  I guess we'll get
to review the logs of the broken gate tests instead :\

I'd love to hear ideas on how we can do this in an automated way which
won't be really unstable/unreliable.

Steve

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] sample config files should be ignored in git...

[Russell Bryant]

On 03/26/2014 02:10 PM, Clint Byrum wrote:
> This is an issue that affects all of our git repos. If you are using
> oslo.config, you will likely also be using the sample config generator.
> 
> However, for some reason we are all checking this generated file in.
> This makes no sense, as we humans are not editting it, and it often
> picks up config files from other things like libraries (keystoneclient
> in particular). This has lead to breakage in the gate a few times for
> Heat, perhaps for others as well.
> 
> I move that we all rm this file from our git trees, and start generating
> it as part of the install/dist process (I have no idea how to do
> this..). This would require:
> 
> - rm sample files and add them to .gitignore in all trees
> - Removing check_uptodate.sh from all trees/tox.ini's
> - Generating file during dist/install process.
> 
> Does anyone disagree?

This has been done in Nova, except we don't have it generated during
install.  We just have instructions and a tox target that will do it if
you choose to.

https://git.openstack.org/cgit/openstack/nova/tree/etc/nova/README-nova.conf.txt

Related, adding instructions to generate without tox:
https://review.openstack.org/#/c/82533/

-- 
Russell Bryant

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [all] sample config files should be ignored in git...

[Clint Byrum]

This is an issue that affects all of our git repos. If you are using
oslo.config, you will likely also be using the sample config generator.

However, for some reason we are all checking this generated file in.
This makes no sense, as we humans are not editting it, and it often
picks up config files from other things like libraries (keystoneclient
in particular). This has lead to breakage in the gate a few times for
Heat, perhaps for others as well.

I move that we all rm this file from our git trees, and start generating
it as part of the install/dist process (I have no idea how to do
this..). This would require:

- rm sample files and add them to .gitignore in all trees
- Removing check_uptodate.sh from all trees/tox.ini's
- Generating file during dist/install process.

Does anyone disagree?

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev