Re: [Openstack] [Blueprint automatic-secure-key-generation] Automatic SECURE_KEY generation
Hi guys, according from your answers, I believe we're still trying to tackle slightly different issues, so I'd like step back a bit and try to sum up our discussion. Please correct me where I am wrong. I think we all agree that the SECRET_KEY should be protected as much as possible, via sensible file permissions, restricted access to the dashboard machine(s)/VM(s), HTTPS and all that jazz. One should also set SESSION_COOKIE_SECURE, SESSION_COOKIE_HTTPONLY and CSRF_COOKIE_SECURE appropriately. We also agree that a key has to be generated somehow and no default key should be provided (given the ability to abuse it easily). So the only remaining question is about _when_ to actually set the key. You both seem to want a key generated automatically for development environments (and thus, for run_tests.sh), possible by also asking the user if he wants that or not. That's a worthy thing to do and I'll try to address this in the way Gabriel proposed (see below). So only production deployments are left. There, I see (roughly) two scenarios that may occur in practice: - One dashboard instance on one machine/VM served by one or multiple python interpreters (through Apache+mod_wsgi or whatever) - Multiple dashboard instances, split over multiple machines, where each may use one or more interpreters In both cases, we would have to make sure that all dashboard instances have the _same_ SECRET_KEY available before starting up (to the latest at the time they read their settings.py). Besides letting this be done by the admin manually, there's the option to automate this. Obviously, this may be different depending on how dashboard(s) are deployed: 1. Centralized tools like crowbar/chef/puppet There you can set/generate the key on the central controller / admin node and push to all dashboard nodes and be done (simplified, I know). 2. Packages Generally, you can generate a key upon installation, which would be unique for one machine. Won't work for multiple-machine deployments. 3. Appliance images Similarly, you can generate the key while booting up. You can't use the packaging approach as there is no package installation phase. Again, won't work in the multiple-machine scenario. 4. Deploy from tarballs / git / ... Not a real option, IMHO. So, multiple-machine deployments will only work via centralized mechanisms (this seems to match current practice) whereas single-machine deployments do have some room for further automation. My original idea was to provide an option to simplify 2. and 3. and pretend that the admin does the right thing in case 1. and hopefully 4. (like it is done currently for all cases). BTW, it should also work in case 1. if the PATH in get_from_file(PATH) points to an NFS export, but that's another story ;-) So how about this: Only provide extra convenience / security for the developer (but ask him) and set nothing by default in production, but only document how one could do it in local_settings.py.example. Either way, sorry for the lengthy mail :-) On 06/19/2012 09:45 PM, Gabriel Hurley wrote: That's looking pretty good, Sascha. May I suggest: 1. Instead of using the temporary lockfile, let's actually move slightly back towards your original approach and check for the existence of a known secret key file (added to .gitignore of course) which the get_secret_key function can check for. This is actually how get_secret_key() stil works, the only difference to what you have in mind is that it also generates the key-file if it doesn't exist. The lockfile is used in order to avoid a potential race of multiple Python processes each trying to create the secret key file if it doesn't exist. If the key is already available, this whole thing would not be needed, true. The only thing that is missing are more secure file permissions for the key-file (I would address that). 2. Split the code that generates the secret key into a separate function generate_secret_key which can be used at will. 2. In the run_tests.sh script (meant for developers, really not used in production deployments), have it check for a valid secret key in either the settings file itself or the known secret key file as part of the sanity check for the environment. If a valid secret key is not available, have it ask the user if they'd like to automatically generate a secret key (using generate_secret_key). Make sure this also respects the -q/--quiet flag for non-interactive runs like Jenkins. 3. Drop lines 25-29 in the local_settings.py.example in your branch, and uncomment line 31. I liked removing the example SECRET_KEY that exists in the example settings file like in your first patch. Even leaving it in as a comment encourages people to simply uncomment it and use the recycled insecure key. Let's take that back out and just keep your explanatory text, which is spot-on. Those points are all sound and I'll do as you propose. 4. Modify the get_secret_key function so that if a valid key is
Re: [Openstack] [Blueprint automatic-secure-key-generation] Automatic SECURE_KEY generation
Sascha, That's all fine; my only reasoning on step 4 was simply that we could provide a more detailed, controlled error message by checking for the key ourselves. I actually don't feel strongly on that point. - Gabriel -Original Message- From: Sascha Peilicke [mailto:sasc...@suse.de] Sent: Wednesday, June 20, 2012 9:05 AM To: Gabriel Hurley; openstack@lists.launchpad.net Cc: Paul McMillan Subject: Re: [Openstack] [Blueprint automatic-secure-key-generation] Automatic SECURE_KEY generation Hi guys, according from your answers, I believe we're still trying to tackle slightly different issues, so I'd like step back a bit and try to sum up our discussion. Please correct me where I am wrong. I think we all agree that the SECRET_KEY should be protected as much as possible, via sensible file permissions, restricted access to the dashboard machine(s)/VM(s), HTTPS and all that jazz. One should also set SESSION_COOKIE_SECURE, SESSION_COOKIE_HTTPONLY and CSRF_COOKIE_SECURE appropriately. We also agree that a key has to be generated somehow and no default key should be provided (given the ability to abuse it easily). So the only remaining question is about _when_ to actually set the key. You both seem to want a key generated automatically for development environments (and thus, for run_tests.sh), possible by also asking the user if he wants that or not. That's a worthy thing to do and I'll try to address this in the way Gabriel proposed (see below). So only production deployments are left. There, I see (roughly) two scenarios that may occur in practice: - One dashboard instance on one machine/VM served by one or multiple python interpreters (through Apache+mod_wsgi or whatever) - Multiple dashboard instances, split over multiple machines, where each may use one or more interpreters In both cases, we would have to make sure that all dashboard instances have the _same_ SECRET_KEY available before starting up (to the latest at the time they read their settings.py). Besides letting this be done by the admin manually, there's the option to automate this. Obviously, this may be different depending on how dashboard(s) are deployed: 1. Centralized tools like crowbar/chef/puppet There you can set/generate the key on the central controller / admin node and push to all dashboard nodes and be done (simplified, I know). 2. Packages Generally, you can generate a key upon installation, which would be unique for one machine. Won't work for multiple-machine deployments. 3. Appliance images Similarly, you can generate the key while booting up. You can't use the packaging approach as there is no package installation phase. Again, won't work in the multiple-machine scenario. 4. Deploy from tarballs / git / ... Not a real option, IMHO. So, multiple-machine deployments will only work via centralized mechanisms (this seems to match current practice) whereas single-machine deployments do have some room for further automation. My original idea was to provide an option to simplify 2. and 3. and pretend that the admin does the right thing in case 1. and hopefully 4. (like it is done currently for all cases). BTW, it should also work in case 1. if the PATH in get_from_file(PATH) points to an NFS export, but that's another story ;-) So how about this: Only provide extra convenience / security for the developer (but ask him) and set nothing by default in production, but only document how one could do it in local_settings.py.example. Either way, sorry for the lengthy mail :-) On 06/19/2012 09:45 PM, Gabriel Hurley wrote: That's looking pretty good, Sascha. May I suggest: 1. Instead of using the temporary lockfile, let's actually move slightly back towards your original approach and check for the existence of a known secret key file (added to .gitignore of course) which the get_secret_key function can check for. This is actually how get_secret_key() stil works, the only difference to what you have in mind is that it also generates the key-file if it doesn't exist. The lockfile is used in order to avoid a potential race of multiple Python processes each trying to create the secret key file if it doesn't exist. If the key is already available, this whole thing would not be needed, true. The only thing that is missing are more secure file permissions for the key-file (I would address that). 2. Split the code that generates the secret key into a separate function generate_secret_key which can be used at will. 2. In the run_tests.sh script (meant for developers, really not used in production deployments), have it check for a valid secret key in either the settings file itself or the known secret key file as part of the sanity check for the environment. If a valid secret key is not available, have it ask the user if they'd like to automatically generate a secret key (using
Re: [Openstack] [Blueprint automatic-secure-key-generation] Automatic SECURE_KEY generation
On 06/19/2012 04:55 AM, Paul McMillan wrote: Ah, you're thinking of a setup where there are multiple dashboard VMs behind a load-balancer serving requests. Indeed, there the dashboard instances should either share the SECRET_KEY or the load-balancer has to make sure that all requests of a given session are redirected to the same dashboard instance. I'm concerned that anything which automatically generates a secret key will cause further problems down the line for other users. For example, you've clearly experienced what happens when you use more than one worker and generate a per-process key. Imagine trying to debug that same problem on a multi-system cloud (with a load balancer that usually routes people to the same place, but not always). If you aren't forced to learn about this setting during deployment, you are faced with a nearly impossible problem of users just sometimes get logged out. I feel like this patch is merely kicking the can down the road just a little past where your particular project needs it to be, without thinking about the bigger picture. I'm sorry about that, but that was definitely not the intent. Inherently you are right, there are just far to many possible setups to get them all right. Thus it's a valid choice to defer such decisions to the one doing the setup. But trying to ease the process can't be that wrong either. That's the whole point why distributions don't only provide packages that merely include pristine upstream tarballs. We try (and sometimes fail to) provide useful defaults that are at least useful to get started. I'm sure you're not seriously suggesting that a large-scale production deployment of openstack will be served entirely by a single point of failure dashboard server. But shouldn't local_settings.py still take preference over settings.py? Thus the admin could still set a specific SECRET_KEY in local_settings.py regardless of the default (auto-generated) one. So I only would have to fix the patch by not removing the documentation about SECRET_KEY from local_settings.py, right? I agree with Gabriel. Horizon should ship with no secret key (so it won't start until you set one). At most, it should automatically generate a key on a per-process basis, or possibly as part of run_tests, so that getting started with development is easy. Under no circumstances should it attempt to read the mind of an admin doing a production deployment, because it will invariably do the wrong thing more often than the right. As a security issue, it's important that admins READ THE DOCUMENTATION. Preventing the project from starting until they address the issue is one good way. Ok, I've adjusted the patch to reflect that. So there's no default anymore but some more documentation about the options instead: https://github.com/saschpe/horizon/compare/bp/automatic-secure-key-generation This should now match Gabriel's proposal, so would that be ok for you? Unfortunately, this is only relevant for securing production deployments. Nobody cares if a developer instance is setup securely ;-) I beg to differ. Opening trivially exploitable holes in development machines (especially laptops) can be extremely damaging. (you did read the docs about the consequences of disclosing a secret key, right?) Actually, this statement wasn't meant to be overly serious, therefore the smiley. I only tried to clarify that production was the primary concern, sorry again. If we don't force developers and end-users to read the documentation, particularly around security features, they will get them wrong. Best Guess security isn't a business I want to be in. No doubts about that and still they do. Hopefully, the current patch is acceptable to you know. -- With kind regards, Sascha Peilicke SUSE Linux GmbH, Maxfeldstr. 5, D-90409 Nuernberg, Germany GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer HRB 16746 (AG Nürnberg) signature.asc Description: OpenPGP digital signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [Blueprint automatic-secure-key-generation] Automatic SECURE_KEY generation
That's looking pretty good, Sascha. May I suggest: 1. Instead of using the temporary lockfile, let's actually move slightly back towards your original approach and check for the existence of a known secret key file (added to .gitignore of course) which the get_secret_key function can check for. 2. Split the code that generates the secret key into a separate function generate_secret_key which can be used at will. 2. In the run_tests.sh script (meant for developers, really not used in production deployments), have it check for a valid secret key in either the settings file itself or the known secret key file as part of the sanity check for the environment. If a valid secret key is not available, have it ask the user if they'd like to automatically generate a secret key (using generate_secret_key). Make sure this also respects the -q/--quiet flag for non-interactive runs like Jenkins. 3. Drop lines 25-29 in the local_settings.py.example in your branch, and uncomment line 31. I liked removing the example SECRET_KEY that exists in the example settings file like in your first patch. Even leaving it in as a comment encourages people to simply uncomment it and use the recycled insecure key. Let's take that back out and just keep your explanatory text, which is spot-on. 4. Modify the get_secret_key function so that if a valid key is not available (e.g. this is production and we didn't use run_tests.sh to install the environment) it dies with a very explanatory error message indicating that a secret key must be set. I know that's a lot to take in, but I think it walks the cleanest line between making it super-easy for developers while making sure everyone is aware of what's going on. If you'll do that and push the review to Gerrit I'll be totally in support of it. Thanks! - Gabriel -Original Message- From: openstack-bounces+gabriel.hurley=nebula@lists.launchpad.net [mailto:openstack- bounces+gabriel.hurley=nebula@lists.launchpad.net] On Behalf Of Sascha Peilicke Sent: Tuesday, June 19, 2012 2:04 AM To: openstack@lists.launchpad.net Subject: Re: [Openstack] [Blueprint automatic-secure-key-generation] Automatic SECURE_KEY generation On 06/19/2012 04:55 AM, Paul McMillan wrote: Ah, you're thinking of a setup where there are multiple dashboard VMs behind a load-balancer serving requests. Indeed, there the dashboard instances should either share the SECRET_KEY or the load-balancer has to make sure that all requests of a given session are redirected to the same dashboard instance. I'm concerned that anything which automatically generates a secret key will cause further problems down the line for other users. For example, you've clearly experienced what happens when you use more than one worker and generate a per-process key. Imagine trying to debug that same problem on a multi-system cloud (with a load balancer that usually routes people to the same place, but not always). If you aren't forced to learn about this setting during deployment, you are faced with a nearly impossible problem of users just sometimes get logged out. I feel like this patch is merely kicking the can down the road just a little past where your particular project needs it to be, without thinking about the bigger picture. I'm sorry about that, but that was definitely not the intent. Inherently you are right, there are just far to many possible setups to get them all right. Thus it's a valid choice to defer such decisions to the one doing the setup. But trying to ease the process can't be that wrong either. That's the whole point why distributions don't only provide packages that merely include pristine upstream tarballs. We try (and sometimes fail to) provide useful defaults that are at least useful to get started. I'm sure you're not seriously suggesting that a large-scale production deployment of openstack will be served entirely by a single point of failure dashboard server. But shouldn't local_settings.py still take preference over settings.py? Thus the admin could still set a specific SECRET_KEY in local_settings.py regardless of the default (auto-generated) one. So I only would have to fix the patch by not removing the documentation about SECRET_KEY from local_settings.py, right? I agree with Gabriel. Horizon should ship with no secret key (so it won't start until you set one). At most, it should automatically generate a key on a per-process basis, or possibly as part of run_tests, so that getting started with development is easy. Under no circumstances should it attempt to read the mind of an admin doing a production deployment, because it will invariably do the wrong thing more often than the right. As a security issue, it's important that admins READ THE DOCUMENTATION. Preventing the project from starting until they address the issue is one good way. Ok, I've adjusted the patch to reflect
Re: [Openstack] [Blueprint automatic-secure-key-generation] Automatic SECURE_KEY generation
On 06/15/2012 11:08 PM, Gabriel Hurley wrote: To the points Sascha raised, and in particular in response to the code method suggested here (https://github.com/saschpe/horizon/commit/1414d538d65d2d3deb981db0ab9e888a3c96a149) I think we are largely in agreement except for one point: I have no problem with this approach for development, or even for distro packaging, but I strongly dislike this automatic generation for a production deployment. Particularly there are issues about distributed deployments where the installs need to share a common secret key to allow for shared data signing validation, etc. I would rather not obfuscate these very real and very relevant concerns for productions deployments for the sake of making everything else a little easier. Especially because it will lead to very hard-to-diagnose bugs because people aren't aware of the magic happening behind the scenes. Ah, you're thinking of a setup where there are multiple dashboard VMs behind a load-balancer serving requests. Indeed, there the dashboard instances should either share the SECRET_KEY or the load-balancer has to make sure that all requests of a given session are redirected to the same dashboard instance. But shouldn't local_settings.py still take preference over settings.py? Thus the admin could still set a specific SECRET_KEY in local_settings.py regardless of the default (auto-generated) one. So I only would have to fix the patch by not removing the documentation about SECRET_KEY from local_settings.py, right? As such, I'd rather have the code you wrote be part of the environment build/run_tests code such that there's an *optional* tool to do this, but it doesn't hide legitimate security and functionality concerns. Unfortunately, this is only relevant for securing production deployments. Nobody cares if a developer instance is setup securely ;-) I'm also cc'ing Paul McMillan, Django's resident security expert, to get his take on this. Have you already got an answer from Paul? He wan't in the CC list, actually. - Gabriel -Original Message- From: Sascha Peilicke [mailto:sasc...@suse.de] Sent: Friday, June 15, 2012 12:38 AM To: Gabriel Hurley Cc: openstack@lists.launchpad.net Subject: Re: [Blueprint automatic-secure-key-generation] Automatic SECURE_KEY generation Hi Grabriel, let's discuss the blueprint [1] on the list. On 06/14/2012 10:36 PM, Gabriel Hurley wrote: Blueprint changed by Gabriel Hurley: Whiteboard set to: After discussing this with both the Horizon core team and Django's security czar/core committer Paul McMillan, we've decided the best way to proceed with this is as follows: * Remove the default SECRET_KEY so it cannot be shared causing security problems. * For development, add a few lines to auto-generate a SECRET_KEY if one isn't present. Ok, nothing to add, this is what the patch actually does [2]. * For production, document that a SECRET_KEY is required, how to generate one, etc. The question to me is, why this should matter to the admin at all. Security-wise, the only thing that matters is that the SECRET_KEY is unique per dashboard installation and set before the first start. Whether this is done by the admin or by the patch to discuss doesn't really matter. However, even if documented appropriately, setting up a complete OpenStack deployment isn't exactly a piece of cake. Having to remember yet another config value is error-prone IMO and easily forgotten. Actually, Django already does a really good job in documenting this, but we stumbled upon this because all of our internal development clouds had the same (default) SECRET_KEY. Seems like we just forgot ;-) * Work with the distros to make sure they properly generate a unique SECRET_KEY for each install. This is why I started the whole topic, as there are cases where this is just impractical. But let's take it slowly, current OpenStack rpm packages for openSUSE / SLES generate a SECRET_KEY as a %post scriptlet (i.e. just after package installation). This doesn't hurt and is probably what you had in mind. However, this only works if there is actually a package to install. Unfortunately, this isn't the case when the dashboard is deployed from an appliance image. Of course, you could check and set the SECRET_KEY after successfully booting up the appliance image via a script (if it is not a snapshot of an already booted system) or you could just do it when the Django app is actually started. The latter seems more practical to me. And lastly, when it's part of the horizon code-base, the issue could be solved for all deployments. Footnotes: [1] https://blueprints.launchpad.net/horizon/+spec/automatic-secure-key- generation [2] https://github.com/saschpe/horizon/commit/1414d538d65d2d3deb981db0a b9e888a3c96a149 -- With kind regards, Sascha Peilicke SUSE Linux GmbH, Maxfeldstr. 5, D-90409 Nuernberg, Germany GF: Jeff Hawn,
Re: [Openstack] [Blueprint automatic-secure-key-generation] Automatic SECURE_KEY generation
Hi Grabriel, let's discuss the blueprint [1] on the list. On 06/14/2012 10:36 PM, Gabriel Hurley wrote: Blueprint changed by Gabriel Hurley: Whiteboard set to: After discussing this with both the Horizon core team and Django's security czar/core committer Paul McMillan, we've decided the best way to proceed with this is as follows: * Remove the default SECRET_KEY so it cannot be shared causing security problems. * For development, add a few lines to auto-generate a SECRET_KEY if one isn't present. Ok, nothing to add, this is what the patch actually does [2]. * For production, document that a SECRET_KEY is required, how to generate one, etc. The question to me is, why this should matter to the admin at all. Security-wise, the only thing that matters is that the SECRET_KEY is unique per dashboard installation and set before the first start. Whether this is done by the admin or by the patch to discuss doesn't really matter. However, even if documented appropriately, setting up a complete OpenStack deployment isn't exactly a piece of cake. Having to remember yet another config value is error-prone IMO and easily forgotten. Actually, Django already does a really good job in documenting this, but we stumbled upon this because all of our internal development clouds had the same (default) SECRET_KEY. Seems like we just forgot ;-) * Work with the distros to make sure they properly generate a unique SECRET_KEY for each install. This is why I started the whole topic, as there are cases where this is just impractical. But let's take it slowly, current OpenStack rpm packages for openSUSE / SLES generate a SECRET_KEY as a %post scriptlet (i.e. just after package installation). This doesn't hurt and is probably what you had in mind. However, this only works if there is actually a package to install. Unfortunately, this isn't the case when the dashboard is deployed from an appliance image. Of course, you could check and set the SECRET_KEY after successfully booting up the appliance image via a script (if it is not a snapshot of an already booted system) or you could just do it when the Django app is actually started. The latter seems more practical to me. And lastly, when it's part of the horizon code-base, the issue could be solved for all deployments. Footnotes: [1] https://blueprints.launchpad.net/horizon/+spec/automatic-secure-key-generation [2] https://github.com/saschpe/horizon/commit/1414d538d65d2d3deb981db0ab9e888a3c96a149 -- With kind regards, Sascha Peilicke SUSE Linux GmbH, Maxfeldstr. 5, D-90409 Nuernberg, Germany GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer HRB 16746 (AG Nürnberg) signature.asc Description: OpenPGP digital signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [Blueprint automatic-secure-key-generation] Automatic SECURE_KEY generation
To the points Sascha raised, and in particular in response to the code method suggested here (https://github.com/saschpe/horizon/commit/1414d538d65d2d3deb981db0ab9e888a3c96a149) I think we are largely in agreement except for one point: I have no problem with this approach for development, or even for distro packaging, but I strongly dislike this automatic generation for a production deployment. Particularly there are issues about distributed deployments where the installs need to share a common secret key to allow for shared data signing validation, etc. I would rather not obfuscate these very real and very relevant concerns for productions deployments for the sake of making everything else a little easier. Especially because it will lead to very hard-to-diagnose bugs because people aren't aware of the magic happening behind the scenes. As such, I'd rather have the code you wrote be part of the environment build/run_tests code such that there's an *optional* tool to do this, but it doesn't hide legitimate security and functionality concerns. I'm also cc'ing Paul McMillan, Django's resident security expert, to get his take on this. - Gabriel -Original Message- From: Sascha Peilicke [mailto:sasc...@suse.de] Sent: Friday, June 15, 2012 12:38 AM To: Gabriel Hurley Cc: openstack@lists.launchpad.net Subject: Re: [Blueprint automatic-secure-key-generation] Automatic SECURE_KEY generation Hi Grabriel, let's discuss the blueprint [1] on the list. On 06/14/2012 10:36 PM, Gabriel Hurley wrote: Blueprint changed by Gabriel Hurley: Whiteboard set to: After discussing this with both the Horizon core team and Django's security czar/core committer Paul McMillan, we've decided the best way to proceed with this is as follows: * Remove the default SECRET_KEY so it cannot be shared causing security problems. * For development, add a few lines to auto-generate a SECRET_KEY if one isn't present. Ok, nothing to add, this is what the patch actually does [2]. * For production, document that a SECRET_KEY is required, how to generate one, etc. The question to me is, why this should matter to the admin at all. Security-wise, the only thing that matters is that the SECRET_KEY is unique per dashboard installation and set before the first start. Whether this is done by the admin or by the patch to discuss doesn't really matter. However, even if documented appropriately, setting up a complete OpenStack deployment isn't exactly a piece of cake. Having to remember yet another config value is error-prone IMO and easily forgotten. Actually, Django already does a really good job in documenting this, but we stumbled upon this because all of our internal development clouds had the same (default) SECRET_KEY. Seems like we just forgot ;-) * Work with the distros to make sure they properly generate a unique SECRET_KEY for each install. This is why I started the whole topic, as there are cases where this is just impractical. But let's take it slowly, current OpenStack rpm packages for openSUSE / SLES generate a SECRET_KEY as a %post scriptlet (i.e. just after package installation). This doesn't hurt and is probably what you had in mind. However, this only works if there is actually a package to install. Unfortunately, this isn't the case when the dashboard is deployed from an appliance image. Of course, you could check and set the SECRET_KEY after successfully booting up the appliance image via a script (if it is not a snapshot of an already booted system) or you could just do it when the Django app is actually started. The latter seems more practical to me. And lastly, when it's part of the horizon code-base, the issue could be solved for all deployments. Footnotes: [1] https://blueprints.launchpad.net/horizon/+spec/automatic-secure-key- generation [2] https://github.com/saschpe/horizon/commit/1414d538d65d2d3deb981db0a b9e888a3c96a149 -- With kind regards, Sascha Peilicke SUSE Linux GmbH, Maxfeldstr. 5, D-90409 Nuernberg, Germany GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer HRB 16746 (AG Nürnberg) ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
