Re: [openstack-dev] [TripleO] Time to break backwards compatibility for *cloud-password file location?

2014-08-19 Thread James Polley
After some more discussion and several attempts at different ways to solve
this problem, here's the message I've just added to
https://review.openstack.org/#/c/97657/


I spoke to xu haiwei about this change a few days ago and agreed to have a
stab at switching from using environment variables to command-line flags,
in line with the comments above. The best way i could see to do this was to
store the password file location in $TE_DATAFILE. I wanted to check if this
was a sensible idea so I jumped into #tripleo

This led to some discussion in-channel where we (bnemec, lifeless, myself)
tried to step back, look at what problem we're trying to solve, and look
for a good way to solve it.

We think, if our understanding is correct, that we're trying to do a few
different things here:

1) The current logic (look in $CWD for passwords first, then look in
$TRIPLEO_ROOT) is confusing, especially for someone reading the code - we
think it's not particularly clear what's happening or why
2) We (who were discussing it at the time) think that we (the broader
TripleO team) want to make sure that users can drive multiple stacks - eg,
running multiple overclouds out of one undercloud - and have different
credentials for each. We (who were discussing it at the time) don't
actually know of anyone who does this, so we're only speculating that this
something that some people *might* find useful - if we could talk to
someone who needs this functionality, it would help us make sure we're
doing something useful for them
3) Looking to the future, these passwords should eventually be stored in
Barbican or something similar. In the shorter term, it would probably be a
good idea to move the passwords into $TE_DATAFILE. However, the work
required to do that today is non-trivial as we'd need to find all the
places the passwords are currently used and change the way the password is
looked up.

Looking at the history of this change, it seems that it was originally
trying to solve (1) by making the logic for finding passwords simpler and
cleaner. However, that broke (2); in trying to make (2) work again, this
change got made more complex, even though we haven't found anyone who
actually relies on this functionality working. We seem to have a solution
(put the filenames into $TE_DATAFILE) which achieves (1) and preserves the
ability to do (2) - but any work we do on it will have to be tweaked again
(probably soon) when we work on (3)

So, assuming I've understood the motivations of this change correctly,
here's what I plan to do:

1) Wait for people to tell me I've misunderstood the intent of this change
(this is a non-blocking step, other steps will proceed in parallel with
waiting)
2) Raise https://bugs.launchpad.net/tripleo/+bug/1358600 about getting a
solution for (3) (which will enable (2) and help with (1) )
3) Raise a new CR (I12d013196b0c533a0550167b9b8209e590e3bd49) which
attempts to add some docs into the scripts to clarify the existing logic

I'll copy this message onto the mailing list as well - if we've
misunderstood what's needed here, or these plans don't meet your needs,
please follow up there with some more details.


On Thu, Jul 3, 2014 at 9:10 AM, James Polley j...@jamezpolley.com wrote:




 On Thu, Jul 3, 2014 at 5:33 AM, Sullivan, Jon Paul 
 jonpaul.sulli...@hp.com wrote:

  -Original Message-
  From: Giulio Fidente [mailto:gfide...@redhat.com]
  Sent: 01 July 2014 13:08
  Subject: Re: [openstack-dev] [TripleO] Time to break backwards
  compatibility for *cloud-password file location?
 
  On 06/25/2014 11:25 AM, mar...@redhat.com wrote:
   On 25/06/14 10:52, James Polley wrote:
   Until https://review.openstack.org/#/c/83250/, the setup-*-password
   scripts used to drop password files into $CWD, which meant that if
   you ran the script from a different location next time, your old
   passwords wouldn't be found.
  
   https://review.openstack.org/#/c/83250/ changed this so that the
   default behaviour is to put the password files in $TRIPLEO_ROOT; but
   for backwards compatibility we left the script checking to see if
   there's a file in the current directory, and using that file in
   preference to $TRIPLEO_ROOT if it exists.
  
   However, this behaviour is still confusing to people. I'm not
   entirely clear on why it's confusing (it makes perfect sense to
   me...) but I imagine it's because we still have the problem that the
   code works fine if run from one directory, but run from a different
  directory it can't find passwords.
  
   There are two open patches which would break backwards compatibility
   and only ever use the files in $TRIPLEO_ROOT:
  
   https://review.openstack.org/#/c/93981/
   https://review.openstack.org/#/c/97657/
  
   The latter review is under more active development, and has
   suggestions that the directory containing the password files should
   be parameterised, defaulting to $TRIPLEO_ROOT. This would still break
   for anyone who relies on the password files being

Re: [openstack-dev] [TripleO] Time to break backwards compatibility for *cloud-password file location?

2014-07-02 Thread Sullivan, Jon Paul
 -Original Message-
 From: Giulio Fidente [mailto:gfide...@redhat.com]
 Sent: 01 July 2014 13:08
 Subject: Re: [openstack-dev] [TripleO] Time to break backwards
 compatibility for *cloud-password file location?
 
 On 06/25/2014 11:25 AM, mar...@redhat.com wrote:
  On 25/06/14 10:52, James Polley wrote:
  Until https://review.openstack.org/#/c/83250/, the setup-*-password
  scripts used to drop password files into $CWD, which meant that if
  you ran the script from a different location next time, your old
  passwords wouldn't be found.
 
  https://review.openstack.org/#/c/83250/ changed this so that the
  default behaviour is to put the password files in $TRIPLEO_ROOT; but
  for backwards compatibility we left the script checking to see if
  there's a file in the current directory, and using that file in
  preference to $TRIPLEO_ROOT if it exists.
 
  However, this behaviour is still confusing to people. I'm not
  entirely clear on why it's confusing (it makes perfect sense to
  me...) but I imagine it's because we still have the problem that the
  code works fine if run from one directory, but run from a different
 directory it can't find passwords.
 
  There are two open patches which would break backwards compatibility
  and only ever use the files in $TRIPLEO_ROOT:
 
  https://review.openstack.org/#/c/93981/
  https://review.openstack.org/#/c/97657/
 
  The latter review is under more active development, and has
  suggestions that the directory containing the password files should
  be parameterised, defaulting to $TRIPLEO_ROOT. This would still break
  for anyone who relies on the password files being in the directory
  they run the script from, but at least there would be a fairly easy
 fix for them.
 
 
  How about we:
 
  * parameterize as suggested by Fabio in the review @
  https://review.openstack.org/#/c/97657/

+1

 
  * move setting of this param to more visible location (setup, like
  devtest_variables or testenv). We can then give this better visibility
  in the dev/test autodocs with a warning about the 'old' behaviour

+1

 
  * add a deprecation warning to the code that reads from
  $CWD/tripleo-overcloud-passwords to say that this will now need to be
  set as a parameter in ... wherever. How long is a good period for
 this?
 
 +1

+1

Would it make sense to copy the passwords across such that the users behaviour 
is not changed were they to delete their old passwords file.  The deprecation 
warning would read that they can set VAR to point to the passwords file they 
are currently using, or delete passwords file to pick up the new default 
location of VAR (which has defaulted to TRIPLEO_ROOT)

 
 actually, I have probably being the first suggesting that we should
 parametrize the path to the password files so I want to add my
 motivations here
 
 the big win that I see here is that people may want to customize only
 some of the passwords, for example, the undercloud admin
 
 the script creating the password files is *already* capable of pushing
 in the file only new passwords, without regenerating passwords which
 could have been manually set in there already
 
 this basically implements the 'feature' I mentioned except people just
 doesn't know it!
 
 so I'd like we to expose this as a feature, from the early stages as
 Marios suggests too, maybe from devtest_variables
 --
 Giulio Fidente
 GPG KEY: 08D733BA
 
 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Thanks, 
Jon-Paul Sullivan ☺ Cloud Services - @hpcloud

Postal Address: Hewlett-Packard Galway Limited, Ballybrit Business Park, Galway.
Registered Office: Hewlett-Packard Galway Limited, 63-74 Sir John Rogerson's 
Quay, Dublin 2. 
Registered Number: 361933
 
The contents of this message and any attachments to it are confidential and may 
be legally privileged. If you have received this message in error you should 
delete it from your system immediately and advise the sender.

To any recipient of this message within HP, unless otherwise stated, you should 
consider this message and attachments as HP CONFIDENTIAL.
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [TripleO] Time to break backwards compatibility for *cloud-password file location?

2014-07-02 Thread James Polley
On Thu, Jul 3, 2014 at 5:33 AM, Sullivan, Jon Paul jonpaul.sulli...@hp.com
wrote:

  -Original Message-
  From: Giulio Fidente [mailto:gfide...@redhat.com]
  Sent: 01 July 2014 13:08
  Subject: Re: [openstack-dev] [TripleO] Time to break backwards
  compatibility for *cloud-password file location?
 
  On 06/25/2014 11:25 AM, mar...@redhat.com wrote:
   On 25/06/14 10:52, James Polley wrote:
   Until https://review.openstack.org/#/c/83250/, the setup-*-password
   scripts used to drop password files into $CWD, which meant that if
   you ran the script from a different location next time, your old
   passwords wouldn't be found.
  
   https://review.openstack.org/#/c/83250/ changed this so that the
   default behaviour is to put the password files in $TRIPLEO_ROOT; but
   for backwards compatibility we left the script checking to see if
   there's a file in the current directory, and using that file in
   preference to $TRIPLEO_ROOT if it exists.
  
   However, this behaviour is still confusing to people. I'm not
   entirely clear on why it's confusing (it makes perfect sense to
   me...) but I imagine it's because we still have the problem that the
   code works fine if run from one directory, but run from a different
  directory it can't find passwords.
  
   There are two open patches which would break backwards compatibility
   and only ever use the files in $TRIPLEO_ROOT:
  
   https://review.openstack.org/#/c/93981/
   https://review.openstack.org/#/c/97657/
  
   The latter review is under more active development, and has
   suggestions that the directory containing the password files should
   be parameterised, defaulting to $TRIPLEO_ROOT. This would still break
   for anyone who relies on the password files being in the directory
   they run the script from, but at least there would be a fairly easy
  fix for them.
  
  
   How about we:
  
   * parameterize as suggested by Fabio in the review @
   https://review.openstack.org/#/c/97657/

 +1

  
   * move setting of this param to more visible location (setup, like
   devtest_variables or testenv). We can then give this better visibility
   in the dev/test autodocs with a warning about the 'old' behaviour

 +1

  
   * add a deprecation warning to the code that reads from
   $CWD/tripleo-overcloud-passwords to say that this will now need to be
   set as a parameter in ... wherever. How long is a good period for
  this?
 
  +1

 +1

 Would it make sense to copy the passwords across such that the users
 behaviour is not changed were they to delete their old passwords file.  The
 deprecation warning would read that they can set VAR to point to the
 passwords file they are currently using, or delete passwords file to pick
 up the new default location of VAR (which has defaulted to TRIPLEO_ROOT)


This sounds like something I was trying in the first 10 revisions of
https://review.openstack.org/#/c/83250. I ended up ditching it because it
seemed like the logic was getting too complex.

Eg, if someone has multiple sets of password files, this is fine when we
see the first set  - we just copy it to $TRIPLEO_ROOT and print the
deprecation warning.

But later on if we see a second file - do we clobber the one in
$TRIPLEO_ROOT? Do we skip the copy and just print the deprecation warning,
maybe with an addendum to point out that we've seen two different files? Do
we diff the one in $CWD and the one in $TRIPLEO_ROOT to check if they're
the same?

At the time I was working on 83250 I decided it was simplest to not make
any attempt to clean up, but maybe it's time to revisit that decision.


 actually, I have probably being the first suggesting that we should
 parametrize the path to the password files so I want to add my
 motivations here

 the big win that I see here is that people may want to customize only
 some of the passwords, for example, the undercloud admin

 the script creating the password files is *already* capable of pushing
 in the file only new passwords, without regenerating passwords which
 could have been manually set in there already

 this basically implements the 'feature' I mentioned except people just
 doesn't know it!

 so I'd like we to expose this as a feature, from the early stages as
 Marios suggests too, maybe from devtest_variables
 --
 Giulio Fidente
 GPG KEY: 08D733BA

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Thanks,
 Jon-Paul Sullivan ☺ Cloud Services - @hpcloud

 Postal Address: Hewlett-Packard Galway Limited, Ballybrit Business Park,
 Galway.
 Registered Office: Hewlett-Packard Galway Limited, 63-74 Sir John
 Rogerson's Quay, Dublin 2.
 Registered Number: 361933

 The contents of this message and any attachments to it are confidential
 and may be legally privileged. If you have received this message in error
 you should delete it from your system immediately and advise the sender

Re: [openstack-dev] [TripleO] Time to break backwards compatibility for *cloud-password file location?

2014-07-01 Thread Giulio Fidente

On 06/25/2014 11:25 AM, mar...@redhat.com wrote:

On 25/06/14 10:52, James Polley wrote:

Until https://review.openstack.org/#/c/83250/, the setup-*-password scripts
used to drop password files into $CWD, which meant that if you ran the
script from a different location next time, your old passwords wouldn't be
found.

https://review.openstack.org/#/c/83250/ changed this so that the default
behaviour is to put the password files in $TRIPLEO_ROOT; but for backwards
compatibility we left the script checking to see if there's a file in the
current directory, and using that file in preference to $TRIPLEO_ROOT if it
exists.

However, this behaviour is still confusing to people. I'm not entirely
clear on why it's confusing (it makes perfect sense to me...) but I imagine
it's because we still have the problem that the code works fine if run from
one directory, but run from a different directory it can't find passwords.

There are two open patches which would break backwards compatibility and
only ever use the files in $TRIPLEO_ROOT:

https://review.openstack.org/#/c/93981/
https://review.openstack.org/#/c/97657/

The latter review is under more active development, and has suggestions
that the directory containing the password files should be parameterised,
defaulting to $TRIPLEO_ROOT. This would still break for anyone who relies
on the password files being in the directory they run the script from, but
at least there would be a fairly easy fix for them.



How about we:

* parameterize as suggested by Fabio in the review @
https://review.openstack.org/#/c/97657/

* move setting of this param to more visible location (setup, like
devtest_variables or testenv). We can then give this better visibility
in the dev/test autodocs with a warning about the 'old' behaviour

* add a deprecation warning to the code that reads from
$CWD/tripleo-overcloud-passwords to say that this will now need to be
set as a parameter in ... wherever. How long is a good period for this?


+1

actually, I have probably being the first suggesting that we should 
parametrize the path to the password files so I want to add my 
motivations here


the big win that I see here is that people may want to customize only 
some of the passwords, for example, the undercloud admin


the script creating the password files is *already* capable of pushing 
in the file only new passwords, without regenerating passwords which 
could have been manually set in there already


this basically implements the 'feature' I mentioned except people just 
doesn't know it!


so I'd like we to expose this as a feature, from the early stages as 
Marios suggests too, maybe from devtest_variables

--
Giulio Fidente
GPG KEY: 08D733BA

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [TripleO] Time to break backwards compatibility for *cloud-password file location?

2014-06-25 Thread mar...@redhat.com
On 25/06/14 10:52, James Polley wrote:
 Until https://review.openstack.org/#/c/83250/, the setup-*-password scripts
 used to drop password files into $CWD, which meant that if you ran the
 script from a different location next time, your old passwords wouldn't be
 found.
 
 https://review.openstack.org/#/c/83250/ changed this so that the default
 behaviour is to put the password files in $TRIPLEO_ROOT; but for backwards
 compatibility we left the script checking to see if there's a file in the
 current directory, and using that file in preference to $TRIPLEO_ROOT if it
 exists.
 
 However, this behaviour is still confusing to people. I'm not entirely
 clear on why it's confusing (it makes perfect sense to me...) but I imagine
 it's because we still have the problem that the code works fine if run from
 one directory, but run from a different directory it can't find passwords.
 
 There are two open patches which would break backwards compatibility and
 only ever use the files in $TRIPLEO_ROOT:
 
 https://review.openstack.org/#/c/93981/
 https://review.openstack.org/#/c/97657/
 
 The latter review is under more active development, and has suggestions
 that the directory containing the password files should be parameterised,
 defaulting to $TRIPLEO_ROOT. This would still break for anyone who relies
 on the password files being in the directory they run the script from, but
 at least there would be a fairly easy fix for them.
 

How about we:

* parameterize as suggested by Fabio in the review @
https://review.openstack.org/#/c/97657/

* move setting of this param to more visible location (setup, like
devtest_variables or testenv). We can then give this better visibility
in the dev/test autodocs with a warning about the 'old' behaviour

* add a deprecation warning to the code that reads from
$CWD/tripleo-overcloud-passwords to say that this will now need to be
set as a parameter in ... wherever. How long is a good period for this?

I don't think we make much in terms of backwards compatibility
guarantees right now do we?

marios







 To help decide if it's time to break backwards compatibility in this case
 (and if so, how), I'd love to see some more comments on 97657. If we don't
 want to break backwards compatibility, maybe comments about a better way to
 handle the ambiguity would be helpful.
 
 
 
 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev