Re: [openstack-dev] [TripleO] Time to break backwards compatibility for *cloud-password file location?
After some more discussion and several attempts at different ways to solve this problem, here's the message I've just added to https://review.openstack.org/#/c/97657/ I spoke to xu haiwei about this change a few days ago and agreed to have a stab at switching from using environment variables to command-line flags, in line with the comments above. The best way i could see to do this was to store the password file location in $TE_DATAFILE. I wanted to check if this was a sensible idea so I jumped into #tripleo This led to some discussion in-channel where we (bnemec, lifeless, myself) tried to step back, look at what problem we're trying to solve, and look for a good way to solve it. We think, if our understanding is correct, that we're trying to do a few different things here: 1) The current logic (look in $CWD for passwords first, then look in $TRIPLEO_ROOT) is confusing, especially for someone reading the code - we think it's not particularly clear what's happening or why 2) We (who were discussing it at the time) think that we (the broader TripleO team) want to make sure that users can drive multiple stacks - eg, running multiple overclouds out of one undercloud - and have different credentials for each. We (who were discussing it at the time) don't actually know of anyone who does this, so we're only speculating that this something that some people *might* find useful - if we could talk to someone who needs this functionality, it would help us make sure we're doing something useful for them 3) Looking to the future, these passwords should eventually be stored in Barbican or something similar. In the shorter term, it would probably be a good idea to move the passwords into $TE_DATAFILE. However, the work required to do that today is non-trivial as we'd need to find all the places the passwords are currently used and change the way the password is looked up. Looking at the history of this change, it seems that it was originally trying to solve (1) by making the logic for finding passwords simpler and cleaner. However, that broke (2); in trying to make (2) work again, this change got made more complex, even though we haven't found anyone who actually relies on this functionality working. We seem to have a solution (put the filenames into $TE_DATAFILE) which achieves (1) and preserves the ability to do (2) - but any work we do on it will have to be tweaked again (probably soon) when we work on (3) So, assuming I've understood the motivations of this change correctly, here's what I plan to do: 1) Wait for people to tell me I've misunderstood the intent of this change (this is a non-blocking step, other steps will proceed in parallel with waiting) 2) Raise https://bugs.launchpad.net/tripleo/+bug/1358600 about getting a solution for (3) (which will enable (2) and help with (1) ) 3) Raise a new CR (I12d013196b0c533a0550167b9b8209e590e3bd49) which attempts to add some docs into the scripts to clarify the existing logic I'll copy this message onto the mailing list as well - if we've misunderstood what's needed here, or these plans don't meet your needs, please follow up there with some more details. On Thu, Jul 3, 2014 at 9:10 AM, James Polley j...@jamezpolley.com wrote: On Thu, Jul 3, 2014 at 5:33 AM, Sullivan, Jon Paul jonpaul.sulli...@hp.com wrote: -Original Message- From: Giulio Fidente [mailto:gfide...@redhat.com] Sent: 01 July 2014 13:08 Subject: Re: [openstack-dev] [TripleO] Time to break backwards compatibility for *cloud-password file location? On 06/25/2014 11:25 AM, mar...@redhat.com wrote: On 25/06/14 10:52, James Polley wrote: Until https://review.openstack.org/#/c/83250/, the setup-*-password scripts used to drop password files into $CWD, which meant that if you ran the script from a different location next time, your old passwords wouldn't be found. https://review.openstack.org/#/c/83250/ changed this so that the default behaviour is to put the password files in $TRIPLEO_ROOT; but for backwards compatibility we left the script checking to see if there's a file in the current directory, and using that file in preference to $TRIPLEO_ROOT if it exists. However, this behaviour is still confusing to people. I'm not entirely clear on why it's confusing (it makes perfect sense to me...) but I imagine it's because we still have the problem that the code works fine if run from one directory, but run from a different directory it can't find passwords. There are two open patches which would break backwards compatibility and only ever use the files in $TRIPLEO_ROOT: https://review.openstack.org/#/c/93981/ https://review.openstack.org/#/c/97657/ The latter review is under more active development, and has suggestions that the directory containing the password files should be parameterised, defaulting to $TRIPLEO_ROOT. This would still break for anyone who relies on the password files being
Re: [openstack-dev] [TripleO] Time to break backwards compatibility for *cloud-password file location?
-Original Message- From: Giulio Fidente [mailto:gfide...@redhat.com] Sent: 01 July 2014 13:08 Subject: Re: [openstack-dev] [TripleO] Time to break backwards compatibility for *cloud-password file location? On 06/25/2014 11:25 AM, mar...@redhat.com wrote: On 25/06/14 10:52, James Polley wrote: Until https://review.openstack.org/#/c/83250/, the setup-*-password scripts used to drop password files into $CWD, which meant that if you ran the script from a different location next time, your old passwords wouldn't be found. https://review.openstack.org/#/c/83250/ changed this so that the default behaviour is to put the password files in $TRIPLEO_ROOT; but for backwards compatibility we left the script checking to see if there's a file in the current directory, and using that file in preference to $TRIPLEO_ROOT if it exists. However, this behaviour is still confusing to people. I'm not entirely clear on why it's confusing (it makes perfect sense to me...) but I imagine it's because we still have the problem that the code works fine if run from one directory, but run from a different directory it can't find passwords. There are two open patches which would break backwards compatibility and only ever use the files in $TRIPLEO_ROOT: https://review.openstack.org/#/c/93981/ https://review.openstack.org/#/c/97657/ The latter review is under more active development, and has suggestions that the directory containing the password files should be parameterised, defaulting to $TRIPLEO_ROOT. This would still break for anyone who relies on the password files being in the directory they run the script from, but at least there would be a fairly easy fix for them. How about we: * parameterize as suggested by Fabio in the review @ https://review.openstack.org/#/c/97657/ +1 * move setting of this param to more visible location (setup, like devtest_variables or testenv). We can then give this better visibility in the dev/test autodocs with a warning about the 'old' behaviour +1 * add a deprecation warning to the code that reads from $CWD/tripleo-overcloud-passwords to say that this will now need to be set as a parameter in ... wherever. How long is a good period for this? +1 +1 Would it make sense to copy the passwords across such that the users behaviour is not changed were they to delete their old passwords file. The deprecation warning would read that they can set VAR to point to the passwords file they are currently using, or delete passwords file to pick up the new default location of VAR (which has defaulted to TRIPLEO_ROOT) actually, I have probably being the first suggesting that we should parametrize the path to the password files so I want to add my motivations here the big win that I see here is that people may want to customize only some of the passwords, for example, the undercloud admin the script creating the password files is *already* capable of pushing in the file only new passwords, without regenerating passwords which could have been manually set in there already this basically implements the 'feature' I mentioned except people just doesn't know it! so I'd like we to expose this as a feature, from the early stages as Marios suggests too, maybe from devtest_variables -- Giulio Fidente GPG KEY: 08D733BA ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev Thanks, Jon-Paul Sullivan ☺ Cloud Services - @hpcloud Postal Address: Hewlett-Packard Galway Limited, Ballybrit Business Park, Galway. Registered Office: Hewlett-Packard Galway Limited, 63-74 Sir John Rogerson's Quay, Dublin 2. Registered Number: 361933 The contents of this message and any attachments to it are confidential and may be legally privileged. If you have received this message in error you should delete it from your system immediately and advise the sender. To any recipient of this message within HP, unless otherwise stated, you should consider this message and attachments as HP CONFIDENTIAL. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [TripleO] Time to break backwards compatibility for *cloud-password file location?
On Thu, Jul 3, 2014 at 5:33 AM, Sullivan, Jon Paul jonpaul.sulli...@hp.com wrote: -Original Message- From: Giulio Fidente [mailto:gfide...@redhat.com] Sent: 01 July 2014 13:08 Subject: Re: [openstack-dev] [TripleO] Time to break backwards compatibility for *cloud-password file location? On 06/25/2014 11:25 AM, mar...@redhat.com wrote: On 25/06/14 10:52, James Polley wrote: Until https://review.openstack.org/#/c/83250/, the setup-*-password scripts used to drop password files into $CWD, which meant that if you ran the script from a different location next time, your old passwords wouldn't be found. https://review.openstack.org/#/c/83250/ changed this so that the default behaviour is to put the password files in $TRIPLEO_ROOT; but for backwards compatibility we left the script checking to see if there's a file in the current directory, and using that file in preference to $TRIPLEO_ROOT if it exists. However, this behaviour is still confusing to people. I'm not entirely clear on why it's confusing (it makes perfect sense to me...) but I imagine it's because we still have the problem that the code works fine if run from one directory, but run from a different directory it can't find passwords. There are two open patches which would break backwards compatibility and only ever use the files in $TRIPLEO_ROOT: https://review.openstack.org/#/c/93981/ https://review.openstack.org/#/c/97657/ The latter review is under more active development, and has suggestions that the directory containing the password files should be parameterised, defaulting to $TRIPLEO_ROOT. This would still break for anyone who relies on the password files being in the directory they run the script from, but at least there would be a fairly easy fix for them. How about we: * parameterize as suggested by Fabio in the review @ https://review.openstack.org/#/c/97657/ +1 * move setting of this param to more visible location (setup, like devtest_variables or testenv). We can then give this better visibility in the dev/test autodocs with a warning about the 'old' behaviour +1 * add a deprecation warning to the code that reads from $CWD/tripleo-overcloud-passwords to say that this will now need to be set as a parameter in ... wherever. How long is a good period for this? +1 +1 Would it make sense to copy the passwords across such that the users behaviour is not changed were they to delete their old passwords file. The deprecation warning would read that they can set VAR to point to the passwords file they are currently using, or delete passwords file to pick up the new default location of VAR (which has defaulted to TRIPLEO_ROOT) This sounds like something I was trying in the first 10 revisions of https://review.openstack.org/#/c/83250. I ended up ditching it because it seemed like the logic was getting too complex. Eg, if someone has multiple sets of password files, this is fine when we see the first set - we just copy it to $TRIPLEO_ROOT and print the deprecation warning. But later on if we see a second file - do we clobber the one in $TRIPLEO_ROOT? Do we skip the copy and just print the deprecation warning, maybe with an addendum to point out that we've seen two different files? Do we diff the one in $CWD and the one in $TRIPLEO_ROOT to check if they're the same? At the time I was working on 83250 I decided it was simplest to not make any attempt to clean up, but maybe it's time to revisit that decision. actually, I have probably being the first suggesting that we should parametrize the path to the password files so I want to add my motivations here the big win that I see here is that people may want to customize only some of the passwords, for example, the undercloud admin the script creating the password files is *already* capable of pushing in the file only new passwords, without regenerating passwords which could have been manually set in there already this basically implements the 'feature' I mentioned except people just doesn't know it! so I'd like we to expose this as a feature, from the early stages as Marios suggests too, maybe from devtest_variables -- Giulio Fidente GPG KEY: 08D733BA ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev Thanks, Jon-Paul Sullivan ☺ Cloud Services - @hpcloud Postal Address: Hewlett-Packard Galway Limited, Ballybrit Business Park, Galway. Registered Office: Hewlett-Packard Galway Limited, 63-74 Sir John Rogerson's Quay, Dublin 2. Registered Number: 361933 The contents of this message and any attachments to it are confidential and may be legally privileged. If you have received this message in error you should delete it from your system immediately and advise the sender
Re: [openstack-dev] [TripleO] Time to break backwards compatibility for *cloud-password file location?
On 25/06/14 10:52, James Polley wrote: Until https://review.openstack.org/#/c/83250/, the setup-*-password scripts used to drop password files into $CWD, which meant that if you ran the script from a different location next time, your old passwords wouldn't be found. https://review.openstack.org/#/c/83250/ changed this so that the default behaviour is to put the password files in $TRIPLEO_ROOT; but for backwards compatibility we left the script checking to see if there's a file in the current directory, and using that file in preference to $TRIPLEO_ROOT if it exists. However, this behaviour is still confusing to people. I'm not entirely clear on why it's confusing (it makes perfect sense to me...) but I imagine it's because we still have the problem that the code works fine if run from one directory, but run from a different directory it can't find passwords. There are two open patches which would break backwards compatibility and only ever use the files in $TRIPLEO_ROOT: https://review.openstack.org/#/c/93981/ https://review.openstack.org/#/c/97657/ The latter review is under more active development, and has suggestions that the directory containing the password files should be parameterised, defaulting to $TRIPLEO_ROOT. This would still break for anyone who relies on the password files being in the directory they run the script from, but at least there would be a fairly easy fix for them. How about we: * parameterize as suggested by Fabio in the review @ https://review.openstack.org/#/c/97657/ * move setting of this param to more visible location (setup, like devtest_variables or testenv). We can then give this better visibility in the dev/test autodocs with a warning about the 'old' behaviour * add a deprecation warning to the code that reads from $CWD/tripleo-overcloud-passwords to say that this will now need to be set as a parameter in ... wherever. How long is a good period for this? I don't think we make much in terms of backwards compatibility guarantees right now do we? marios To help decide if it's time to break backwards compatibility in this case (and if so, how), I'd love to see some more comments on 97657. If we don't want to break backwards compatibility, maybe comments about a better way to handle the ambiguity would be helpful. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
