Re: [openstack-dev] [TripleO][Heat] instance_user fallout, keeping the 'heat-admin' user working

[Steven Hardy]

On Wed, Sep 02, 2015 at 08:32:25PM -0400, James Slagle wrote:
> On Wed, Sep 2, 2015 at 4:22 PM, Dan Prince  wrote:
> > We had an IRC discussion today about the 'heat-admin' user in #tripleo.
> >
> > Upstream Heat recently reverted the 'instance_user' config file option
> > which we relied on in TripleO to standardize the default (admin) user
> > on our nodes. It is my understanding that Heat would prefer not to
> > maintain this option because it causes subtle compatibility issues
> > across the OpenStack and AWS APIs and the interactions between cloud
> > -init version, etc. So it was deprecated in Icehouse... and recently
> > removed in [1].
> >
> > We could just go with the default distro user (centos, fedora, ubuntu,
> > etc.) but it would be really nice to standardize on a user name for
> > maintenance should anyone every spin up a cloud using multiple distros
> > or something.
> >
> > So a couple of options. We could just go on and update our templates
> > like this [2]. This actually seems pretty clean to me, but it would
> > require anybody who has created custom firstboot scripts to do the same
> > (we have proposed docker patches with firstboot scripts that need
> > similar updates).
> 
> Yea, that's the main reason I'm not fond of this approach. It really
> feels like cluttering up the firstboot interface, in that anyone who
> wants to plugin in their own config there has to remember to also
> include this snippet. It leads to copying/pasting around yaml, which I
> don't think is a great pattern going forward.

Yeah, I agree, so I proposed an alternative approach:

https://review.openstack.org/#/c/220057/

This leaves the existing NodeUserData interface intact, and introduces a
new NodeAdminUserData interface (which is specifically for injecting the
admin user, and leaves the existing interface alone).

> It would be nice to have a cleaner separation between the interfaces
> that we offer to users and those that need to be reserved/used for
> TripleO's own purposes.

Agreed, that's what I was aiming for with my patch above, we should
probably work towards more clearly documenting what's supported "external"
vs internal interfaces tho.

> I'm not sure of a better solution though other than a native
> SoftwareDeployment resource in the templates directly that creates a
> known user and reads the ssh keys from the user data (via a script).

I think it's better to avoid using SoftwareDeployment resources for this
if possible, because you really want the user configured as early as
possible for easier debugging e.g when credentials or network are broken
and SoftwareDeployments don't work because occ can't reach heat.

> Or, what about baking in some static configuration for cloud-init into
> our images that creates the known user?

That could work, but you'd still need a user-data script to collect the SSH
key from the nova metadata server, because we won't know that at
image-build time.

You're right tho, this probably could be built into the image, it just
seems a little more flexible to expose it in the templates.

> > Alternately, we could propose that Heat revert the instance_user
> > feature or some version of it. We've been using that for a year or two
> > now and it has actually been fairly nice to set the default that way.
> 
> I really liked having the one consistent user no matter the cloud
> image you deployed from as well. I'm not sure we could successfully
> persuade it to go back in though given it was deprecated in Icehouse.

So, I don't think this is impossible, but let me try to explain/remember
the reasons for removing it:

1. Many users disliked that we injected a non-default user and wanted
transparency, e.g just let them configure the users they want via
cloud-init vs "hidden" configuration injected via heat.  This actually
originated with Clint, back in 2013[1] and was also flagged as impacting
TripleO back in 2014.

2. We have historically to maintain code to inject this user which works
over a wide range of cloud-init versions [2] This has proven to be very
difficult, for various reasons, including:
 - It's impossible to find a cloud-config syntax which works over all
   versions of cloud-init we (heat) expect to support (e.g Ubuntu 12.04 LTS
   still ships 0.6.3 which doesn't support cloud-config syntax *at all*,
   and it's expected to be supported until 2017).
 - The alternative to cloud-config was an approach which injects a boothook
   to create the user, but this ran into issues on systems with selinux
   which were difficult/impossible to resolve in a way compatible with
   0.6.3.

So, IIRC, we reached the conclusion that it made sense to just punt the
decision of what users to create, and how to create them to the template
author, who will presumably know if they want a user created, and if so
what version of cloud-init they expect to support in their images
(something the deployers and/or template authors know but heat maintainers
can't possibly 

Re: [openstack-dev] [TripleO][Heat] instance_user fallout, keeping the 'heat-admin' user working

[James Slagle]

On Thu, Sep 3, 2015 at 6:46 AM, Steven Hardy  wrote:
> On Wed, Sep 02, 2015 at 08:32:25PM -0400, James Slagle wrote:
>> On Wed, Sep 2, 2015 at 4:22 PM, Dan Prince  wrote:
>> > We had an IRC discussion today about the 'heat-admin' user in #tripleo.
>> >
>> > Upstream Heat recently reverted the 'instance_user' config file option
>> > which we relied on in TripleO to standardize the default (admin) user
>> > on our nodes. It is my understanding that Heat would prefer not to
>> > maintain this option because it causes subtle compatibility issues
>> > across the OpenStack and AWS APIs and the interactions between cloud
>> > -init version, etc. So it was deprecated in Icehouse... and recently
>> > removed in [1].
>> >
>> > We could just go with the default distro user (centos, fedora, ubuntu,
>> > etc.) but it would be really nice to standardize on a user name for
>> > maintenance should anyone every spin up a cloud using multiple distros
>> > or something.
>> >
>> > So a couple of options. We could just go on and update our templates
>> > like this [2]. This actually seems pretty clean to me, but it would
>> > require anybody who has created custom firstboot scripts to do the same
>> > (we have proposed docker patches with firstboot scripts that need
>> > similar updates).
>>
>> Yea, that's the main reason I'm not fond of this approach. It really
>> feels like cluttering up the firstboot interface, in that anyone who
>> wants to plugin in their own config there has to remember to also
>> include this snippet. It leads to copying/pasting around yaml, which I
>> don't think is a great pattern going forward.
>
> Yeah, I agree, so I proposed an alternative approach:
>
> https://review.openstack.org/#/c/220057/
>
> This leaves the existing NodeUserData interface intact, and introduces a
> new NodeAdminUserData interface (which is specifically for injecting the
> admin user, and leaves the existing interface alone).

This looks like a nice solution, and it addresses the concern. Thanks
for pulling it together.

>
>> It would be nice to have a cleaner separation between the interfaces
>> that we offer to users and those that need to be reserved/used for
>> TripleO's own purposes.
>
> Agreed, that's what I was aiming for with my patch above, we should
> probably work towards more clearly documenting what's supported "external"
> vs internal interfaces tho.
>
>> I'm not sure of a better solution though other than a native
>> SoftwareDeployment resource in the templates directly that creates a
>> known user and reads the ssh keys from the user data (via a script).
>
> I think it's better to avoid using SoftwareDeployment resources for this
> if possible, because you really want the user configured as early as
> possible for easier debugging e.g when credentials or network are broken
> and SoftwareDeployments don't work because occ can't reach heat.
>
>> Or, what about baking in some static configuration for cloud-init into
>> our images that creates the known user?
>
> That could work, but you'd still need a user-data script to collect the SSH
> key from the nova metadata server, because we won't know that at
> image-build time.
>
> You're right tho, this probably could be built into the image, it just
> seems a little more flexible to expose it in the templates.
>
>> > Alternately, we could propose that Heat revert the instance_user
>> > feature or some version of it. We've been using that for a year or two
>> > now and it has actually been fairly nice to set the default that way.
>>
>> I really liked having the one consistent user no matter the cloud
>> image you deployed from as well. I'm not sure we could successfully
>> persuade it to go back in though given it was deprecated in Icehouse.
>
> So, I don't think this is impossible, but let me try to explain/remember
> the reasons for removing it:
>
> 1. Many users disliked that we injected a non-default user and wanted
> transparency, e.g just let them configure the users they want via
> cloud-init vs "hidden" configuration injected via heat.  This actually
> originated with Clint, back in 2013[1] and was also flagged as impacting
> TripleO back in 2014.
>
> 2. We have historically to maintain code to inject this user which works
> over a wide range of cloud-init versions [2] This has proven to be very
> difficult, for various reasons, including:
>  - It's impossible to find a cloud-config syntax which works over all
>versions of cloud-init we (heat) expect to support (e.g Ubuntu 12.04 LTS
>still ships 0.6.3 which doesn't support cloud-config syntax *at all*,
>and it's expected to be supported until 2017).
>  - The alternative to cloud-config was an approach which injects a boothook
>to create the user, but this ran into issues on systems with selinux
>which were difficult/impossible to resolve in a way compatible with
>0.6.3.
>
> So, IIRC, we reached the conclusion that it made sense to just punt the
> 

Re: [openstack-dev] [TripleO][Heat] instance_user fallout, keeping the 'heat-admin' user working

[Dan Prince]

On Wed, 2015-09-02 at 20:32 -0400, James Slagle wrote:
> On Wed, Sep 2, 2015 at 4:22 PM, Dan Prince 
> wrote:
> > We had an IRC discussion today about the 'heat-admin' user in
> > #tripleo.
> > 
> > Upstream Heat recently reverted the 'instance_user' config file
> > option
> > which we relied on in TripleO to standardize the default (admin)
> > user
> > on our nodes. It is my understanding that Heat would prefer not to
> > maintain this option because it causes subtle compatibility issues
> > across the OpenStack and AWS APIs and the interactions between
> > cloud
> > -init version, etc. So it was deprecated in Icehouse... and
> > recently
> > removed in [1].
> > 
> > We could just go with the default distro user (centos, fedora,
> > ubuntu,
> > etc.) but it would be really nice to standardize on a user name for
> > maintenance should anyone every spin up a cloud using multiple
> > distros
> > or something.
> > 
> > So a couple of options. We could just go on and update our
> > templates
> > like this [2]. This actually seems pretty clean to me, but it would
> > require anybody who has created custom firstboot scripts to do the
> > same
> > (we have proposed docker patches with firstboot scripts that need
> > similar updates).
> 
> Yea, that's the main reason I'm not fond of this approach. It really
> feels like cluttering up the firstboot interface, in that anyone who
> wants to plugin in their own config there has to remember to also
> include this snippet. It leads to copying/pasting around yaml, which
> I
> don't think is a great pattern going forward.
> 

Well, ideally we'll have a minimal number of first boot scripts... in
tree there are only 3 that I can see right now and the meaningful code
is actually quite small. Just this:

  user_config:
type: OS::Heat::CloudConfig
properties:
  cloud_config:
user: 'heat-admin'

Any out of tree firstboot scripts would need the same implementation
fixes though. It is those that primarily concern me. Still, if we were
to validate this somehow and failfast that could be a fair compromise.

> It would be nice to have a cleaner separation between the interfaces
> that we offer to users and those that need to be reserved/used for
> TripleO's own purposes.

Agree. The ability to compose userdata via OS::Heat::MultipartMime does
already exist somewhat. Perhaps we could fashion a better way to
include a global user config (from a  heat-admin-user-data.yaml or
something). Haven't tried this yet...

> 
> I'm not sure of a better solution though other than a native
> SoftwareDeployment resource in the templates directly that creates a
> known user and reads the ssh keys from the user data (via a script).

This seems like duplication of a feature we get for free w/ cloud-init
though right?

If we wanted to validate the user exists a script (executed via a
SoftwareDeployment) could be useful for that.

> 
> Or, what about baking in some static configuration for cloud-init
> into
> our images that creates the known user?

Agree this would work but it might be undesirable for images like
Atomic which some would like to avoid repackaging for use via TripleO.
At least that is an idea on the table with the Docker implementation...
as in we don't actually create a custom TripleO version of Atomic. It
just works out of the box with the stock version.

> 
> > Alternately, we could propose that Heat revert the instance_user
> > feature or some version of it. We've been using that for a year or
> > two
> > now and it has actually been fairly nice to set the default that
> > way.
> 
> I really liked having the one consistent user no matter the cloud
> image you deployed from as well. I'm not sure we could successfully
> persuade it to go back in though given it was deprecated in Icehouse.

Agree. I miss the old feature quite a bit. Just wanted to give the new
idea a try first before posted a revert...

> 
> > 
> > Thoughts?
> > 
> > 
> > [1] https://review.openstack.org/103928
> > 
> > [2] https://review.openstack.org/#/c/219861/
> > 
> > ___
> > ___
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsu
> > bscribe
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
> 
> 

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [TripleO][Heat] instance_user fallout, keeping the 'heat-admin' user working

[James Slagle]

On Wed, Sep 2, 2015 at 4:22 PM, Dan Prince  wrote:
> We had an IRC discussion today about the 'heat-admin' user in #tripleo.
>
> Upstream Heat recently reverted the 'instance_user' config file option
> which we relied on in TripleO to standardize the default (admin) user
> on our nodes. It is my understanding that Heat would prefer not to
> maintain this option because it causes subtle compatibility issues
> across the OpenStack and AWS APIs and the interactions between cloud
> -init version, etc. So it was deprecated in Icehouse... and recently
> removed in [1].
>
> We could just go with the default distro user (centos, fedora, ubuntu,
> etc.) but it would be really nice to standardize on a user name for
> maintenance should anyone every spin up a cloud using multiple distros
> or something.
>
> So a couple of options. We could just go on and update our templates
> like this [2]. This actually seems pretty clean to me, but it would
> require anybody who has created custom firstboot scripts to do the same
> (we have proposed docker patches with firstboot scripts that need
> similar updates).

Yea, that's the main reason I'm not fond of this approach. It really
feels like cluttering up the firstboot interface, in that anyone who
wants to plugin in their own config there has to remember to also
include this snippet. It leads to copying/pasting around yaml, which I
don't think is a great pattern going forward.

It would be nice to have a cleaner separation between the interfaces
that we offer to users and those that need to be reserved/used for
TripleO's own purposes.

I'm not sure of a better solution though other than a native
SoftwareDeployment resource in the templates directly that creates a
known user and reads the ssh keys from the user data (via a script).

Or, what about baking in some static configuration for cloud-init into
our images that creates the known user?

> Alternately, we could propose that Heat revert the instance_user
> feature or some version of it. We've been using that for a year or two
> now and it has actually been fairly nice to set the default that way.

I really liked having the one consistent user no matter the cloud
image you deployed from as well. I'm not sure we could successfully
persuade it to go back in though given it was deprecated in Icehouse.

>
> Thoughts?
>
>
> [1] https://review.openstack.org/103928
>
> [2] https://review.openstack.org/#/c/219861/
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



-- 
-- James Slagle
--

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev