Re: [openstack-dev] OpenStack support for Amazon Concepts - was Re: cloud-init IPv6 support

[Joshua Harlow]

And here is the code that does this (for cloudinit 0.7.x):

https://bazaar.launchpad.net/~cloud-init-dev/cloud-init/trunk/view/head:/cloudinit/sources/helpers/openstack.py

This same code is used by the config drive datasource (the one that 
makes a disk/iso) in cloudinit and the http endpoint based datasource in 
cloudinit (the one that exports /openstack/ on the metadata server).


https://bazaar.launchpad.net/~cloud-init-dev/cloud-init/trunk/view/head:/cloudinit/sources/helpers/openstack.py#L320 
(for the config drive subclass) and 
https://bazaar.launchpad.net/~cloud-init-dev/cloud-init/trunk/view/head:/cloudinit/sources/helpers/openstack.py#L419 
(for the http endpoint based subclass).


Note that in the following: 
https://bazaar.launchpad.net/~cloud-init-dev/cloud-init/trunk/view/head:/cloudinit/sources/DataSourceOpenStack.py#L77 
you can already provide a different metadata url (say one that uses 
ipv6) that will override the default "http://169.254.169.254; defined in 
there. This can be done in a few various ways but feel free to jump on 
#cloud-init IRC channel and ask if interested or find me on IRC 
somewhere (since I'm the main one who worked on all the above code).


-Josh

Fox, Kevin M wrote:

No, we already extend the metadata server with our own stuff. See
/openstack/ on the metadata server. Cloudinit even supports the
extensions. Supporting ipv6 as well as v4 is the same. Why does it
matter if aws doesnt currently support it? They can support it if they
want in the future and reuse code, or do their own thing and have to
convince cloudinit to support there way too. But why should that hold
back the openstack metadata server now? Lets lead rather then follow.

Thanks,
Kevin *
*

*From:* Sean M. Collins
*Sent:* Saturday, September 05, 2015 3:19:48 PM
*To:* OpenStack Development Mailing List (not for usage questions)
*Cc:* Fox, Kevin M; PAUL CARVER
*Subject:* OpenStack support for Amazon Concepts - was Re:
[openstack-dev] cloud-init IPv6 support

On Fri, Sep 04, 2015 at 04:20:23PM EDT, Kevin Benton wrote:

 Right, it depends on your perspective of who 'owns' the API. Is it
 cloud-init or EC2?

 At this point I would argue that cloud-init is in control because it would
 be a large undertaking to switch all of the AMI's on Amazon to something
 else. However, I know Sean disagrees with me on this point so I'll let him
 reply here.



Here's my take:

Cloud-Init is a *client* of the Metadata API. The OpenStack Metadata API
in both the Neutron and Nova projects should all the details of the
Metadata API that is documented at:

http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-instance-metadata.html

This means that this is a compatibility layer that OpenStack has
implemented so that users can use appliances, applications, and
operating system images in both Amazon EC2 and an OpenStack environment.

Yes, we can make changes to cloud-init. However, there is no guarantee
that all users of the Metadata API are exclusively using cloud-init as
their client. It is highly unlikely that people are rolling their own
Metadata API clients, but it's a contract we've made with users. This
includes transport level details like the IP address that the service
listens on.

The Metadata API is an established API that Amazon introduced years ago,
and we shouldn't be "improving" APIs that we don't control. If Amazon
were to introduce IPv6 support the Metadata API tomorrow, we would
naturally implement it exactly the way they implemented it in EC2. We'd
honor the contract that Amazon made with its users, in our Metadata API,
since it is a compatibility layer.

However, since they haven't defined transport level details of the
Metadata API, regarding IPv6 - we can't take it upon ourselves to pick a
solution. It is not our API.

The nice thing about config-drive is that we've created a new mechanism
for bootstrapping instances - by replacing the transport level details
of the API. Rather than being a link-local address that instances access
over HTTP, it's a device that guests can mount and read. The actual
contents of the drive may have a similar schema as the Metadata API, but
I think at this point we've made enough of a differentiation between the
EC2 Metadata API and config-drive that I believe the contents of the
actual drive that the instance mounts can be changed without breaking
user expectations - since config-drive was developed by the OpenStack
community. The point being that we call it "config-drive" in
conversation and our docs. Users understand that config-drive is a
different feature.

I've had this same conversation about the Security Group API that we
have. We've named it the same thing as the Amazon API, but then went and
made all the fields different, inexplicably. Thankfully, it's just the
names of the fields, rather than being huge conceptual changes.

http://lists.openstack.org/pipermail/opens

Re: [openstack-dev] OpenStack support for Amazon Concepts - was Re: cloud-init IPv6 support

[Fox, Kevin M]

No, we already extend the metadata server with our own stuff. See /openstack/ 
on the metadata server. Cloudinit even supports the extensions. Supporting ipv6 
as well as v4 is the same. Why does it matter if aws doesnt currently support 
it? They can support it if they want in the future and reuse code, or do their 
own thing and have to convince cloudinit to support there way too. But why 
should that hold back the openstack metadata server now? Lets lead rather then 
follow.

Thanks,
Kevin


From: Sean M. Collins
Sent: Saturday, September 05, 2015 3:19:48 PM
To: OpenStack Development Mailing List (not for usage questions)
Cc: Fox, Kevin M; PAUL CARVER
Subject: OpenStack support for Amazon Concepts - was Re: [openstack-dev] 
cloud-init IPv6 support

On Fri, Sep 04, 2015 at 04:20:23PM EDT, Kevin Benton wrote:
> Right, it depends on your perspective of who 'owns' the API. Is it
> cloud-init or EC2?
>
> At this point I would argue that cloud-init is in control because it would
> be a large undertaking to switch all of the AMI's on Amazon to something
> else. However, I know Sean disagrees with me on this point so I'll let him
> reply here.


Here's my take:

Cloud-Init is a *client* of the Metadata API. The OpenStack Metadata API
in both the Neutron and Nova projects should all the details of the
Metadata API that is documented at:

http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-instance-metadata.html

This means that this is a compatibility layer that OpenStack has
implemented so that users can use appliances, applications, and
operating system images in both Amazon EC2 and an OpenStack environment.

Yes, we can make changes to cloud-init. However, there is no guarantee
that all users of the Metadata API are exclusively using cloud-init as
their client. It is highly unlikely that people are rolling their own
Metadata API clients, but it's a contract we've made with users. This
includes transport level details like the IP address that the service
listens on.

The Metadata API is an established API that Amazon introduced years ago,
and we shouldn't be "improving" APIs that we don't control. If Amazon
were to introduce IPv6 support the Metadata API tomorrow, we would
naturally implement it exactly the way they implemented it in EC2. We'd
honor the contract that Amazon made with its users, in our Metadata API,
since it is a compatibility layer.

However, since they haven't defined transport level details of the
Metadata API, regarding IPv6 - we can't take it upon ourselves to pick a
solution. It is not our API.

The nice thing about config-drive is that we've created a new mechanism
for bootstrapping instances - by replacing the transport level details
of the API. Rather than being a link-local address that instances access
over HTTP, it's a device that guests can mount and read. The actual
contents of the drive may have a similar schema as the Metadata API, but
I think at this point we've made enough of a differentiation between the
EC2 Metadata API and config-drive that I believe the contents of the
actual drive that the instance mounts can be changed without breaking
user expectations - since config-drive was developed by the OpenStack
community. The point being that we call it "config-drive" in
conversation and our docs. Users understand that config-drive is a
different feature.

I've had this same conversation about the Security Group API that we
have. We've named it the same thing as the Amazon API, but then went and
made all the fields different, inexplicably. Thankfully, it's just the
names of the fields, rather than being huge conceptual changes.

http://lists.openstack.org/pipermail/openstack-dev/2015-June/068319.html

Basically, I believe that OpenStack should create APIs that are
community driven and owned, and that we should only emulate
non-community APIs where appropriate, and explicitly state that we only
are emulating them. Putting improvements in APIs that came from
somewhere else, instead of creating new OpenStack branded APIs is a lost
opportunity to differentiate OpenStack from other projects, as well as
Amazon AWS.

Thanks for reading, and have a great holiday.

--
Sean M. Collins
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] OpenStack support for Amazon Concepts - was Re: cloud-init IPv6 support

[Fox, Kevin M]

Yeah, we heve been trying to work through how to make instance users work with 
config drive and its staticness makes the problem very difficult. It just 
trades one problem for another.

Thanks,
Kevin


From: Jim Rollenhagen
Sent: Sunday, September 06, 2015 5:02:24 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] OpenStack support for Amazon Concepts - was Re: 
cloud-init IPv6 support



> On Sep 6, 2015, at 09:43, Monty Taylor <mord...@inaugust.com> wrote:
>
>> On 09/05/2015 06:19 PM, Sean M. Collins wrote:
>>> On Fri, Sep 04, 2015 at 04:20:23PM EDT, Kevin Benton wrote:
>>> Right, it depends on your perspective of who 'owns' the API. Is it
>>> cloud-init or EC2?
>>>
>>> At this point I would argue that cloud-init is in control because it would
>>> be a large undertaking to switch all of the AMI's on Amazon to something
>>> else. However, I know Sean disagrees with me on this point so I'll let him
>>> reply here.
>>
>>
>> Here's my take:
>>
>> Cloud-Init is a *client* of the Metadata API. The OpenStack Metadata API
>> in both the Neutron and Nova projects should all the details of the
>> Metadata API that is documented at:
>>
>> http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-instance-metadata.html
>>
>> This means that this is a compatibility layer that OpenStack has
>> implemented so that users can use appliances, applications, and
>> operating system images in both Amazon EC2 and an OpenStack environment.
>>
>> Yes, we can make changes to cloud-init. However, there is no guarantee
>> that all users of the Metadata API are exclusively using cloud-init as
>> their client. It is highly unlikely that people are rolling their own
>> Metadata API clients, but it's a contract we've made with users. This
>> includes transport level details like the IP address that the service
>> listens on.
>>
>> The Metadata API is an established API that Amazon introduced years ago,
>> and we shouldn't be "improving" APIs that we don't control. If Amazon
>> were to introduce IPv6 support the Metadata API tomorrow, we would
>> naturally implement it exactly the way they implemented it in EC2. We'd
>> honor the contract that Amazon made with its users, in our Metadata API,
>> since it is a compatibility layer.
>>
>> However, since they haven't defined transport level details of the
>> Metadata API, regarding IPv6 - we can't take it upon ourselves to pick a
>> solution. It is not our API.
>>
>> The nice thing about config-drive is that we've created a new mechanism
>> for bootstrapping instances - by replacing the transport level details
>> of the API. Rather than being a link-local address that instances access
>> over HTTP, it's a device that guests can mount and read. The actual
>> contents of the drive may have a similar schema as the Metadata API, but
>> I think at this point we've made enough of a differentiation between the
>> EC2 Metadata API and config-drive that I believe the contents of the
>> actual drive that the instance mounts can be changed without breaking
>> user expectations - since config-drive was developed by the OpenStack
>> community. The point being that we call it "config-drive" in
>> conversation and our docs. Users understand that config-drive is a
>> different feature.
>
> Another great part about config-drive is that it's scalable. At infra's 
> application scale, we take pains to disable anyting in our images that might 
> want to contact the metadata API because we're essentially a DDOS on it.

So, I tend to think a simple API service like this should never be hard to 
scale. Put a bunch of hosts behind a load balancer, boom, done. Even 1000 
requests/s shouldn't be hard, though it may require many hosts, and that's far 
beyond what infra would hit today.

The one problem I have with config-drive is that it is static. I'd love for 
systems like cloud-init, glean, etc, to be able to see changes to mounted 
disks, attached networks, etc. Attaching things after the fact isn't uncommon, 
and to make the user config the thing is a terrible experience. :(

// jim

>
> config-drive being local to the hypervisor host makes it MUCH more stable at 
> scale.
>
> cloud-init supports config-drive
>
> If it were up to me, nobody would be enablig the metadata API in new 
> deployments.
>
> I totally agree that we should not make changes in the metadata API.
>
>> I've had this same conversation about the Security Group API that we
>> have. We've named it the same thing as the Amazon API, but then went 

Re: [openstack-dev] OpenStack support for Amazon Concepts - was Re: cloud-init IPv6 support

[Fox, Kevin M]

We have the whole /openstack namespace. We can extend it as far as we like.
Again, why would aws choosing to go a different way then openstack when 
openstack did something first be an openstack problem? We're not even talking 
about a big change. Just making the same md server available on a second ip.

Thanks,
Kevin


From: Sean M. Collins
Sent: Sunday, September 06, 2015 7:34:54 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] OpenStack support for Amazon Concepts - was Re: 
cloud-init IPv6 support

On Sun, Sep 06, 2015 at 04:25:43PM EDT, Kevin Benton wrote:
> So it's been pointed out that http://169.254.169.254/openstack is completed
> OpenStack invented. I don't quite understand how that's not violating the
> contract you said we have with end users about EC2 compatibility under the
> restriction of 'no new stuff'.

I think that is a violation. I don't think that allows us to make more
changes, just because we've broken the contract once, so a second
infraction is less significant.

> If we added an IPv6 endpoint that the metadata service listens on, it would
> just be another place that non cloud-init clients don't know how to talk
> to. It's not going to break our compatibility with any clients that connect
> to the IPv4 address.

No, but if Amazon were to make a decision about how to implement IPv6 in
EC2 and how to make the Metadata API service work with IPv6 we'd be
supporting two implementations - the one we came up with and one for
supporting the way Amazon implemented it.

--
Sean M. Collins

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] OpenStack support for Amazon Concepts - was Re: cloud-init IPv6 support

[Kevin Benton]

The contract we have is to maintain compatibility. As long as a client
written for the AWS API continues to work, I don't think we are violating
anything. Offering one API isn't a promise not to offer an alternative way
to access the same information.
On Sep 6, 2015 7:37 PM, "Sean M. Collins"  wrote:

> On Sun, Sep 06, 2015 at 04:25:43PM EDT, Kevin Benton wrote:
> > So it's been pointed out that http://169.254.169.254/openstack is
> completed
> > OpenStack invented. I don't quite understand how that's not violating the
> > contract you said we have with end users about EC2 compatibility under
> the
> > restriction of 'no new stuff'.
>
> I think that is a violation. I don't think that allows us to make more
> changes, just because we've broken the contract once, so a second
> infraction is less significant.
>
> > If we added an IPv6 endpoint that the metadata service listens on, it
> would
> > just be another place that non cloud-init clients don't know how to talk
> > to. It's not going to break our compatibility with any clients that
> connect
> > to the IPv4 address.
>
> No, but if Amazon were to make a decision about how to implement IPv6 in
> EC2 and how to make the Metadata API service work with IPv6 we'd be
> supporting two implementations - the one we came up with and one for
> supporting the way Amazon implemented it.
>
> --
> Sean M. Collins
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] OpenStack support for Amazon Concepts - was Re: cloud-init IPv6 support

[John Garbutt]

On 7 September 2015 at 03:34, Sean M. Collins  wrote:
> On Sun, Sep 06, 2015 at 04:25:43PM EDT, Kevin Benton wrote:
>> So it's been pointed out that http://169.254.169.254/openstack is completed
>> OpenStack invented. I don't quite understand how that's not violating the
>> contract you said we have with end users about EC2 compatibility under the
>> restriction of 'no new stuff'.
>
> I think that is a violation. I don't think that allows us to make more
> changes, just because we've broken the contract once, so a second
> infraction is less significant.

I see the OpenStack part of the metadata service a different
interface, that happens to be accessed in a similar way to EC2.

>> If we added an IPv6 endpoint that the metadata service listens on, it would
>> just be another place that non cloud-init clients don't know how to talk
>> to. It's not going to break our compatibility with any clients that connect
>> to the IPv4 address.
>
> No, but if Amazon were to make a decision about how to implement IPv6 in
> EC2 and how to make the Metadata API service work with IPv6 we'd be
> supporting two implementations - the one we came up with and one for
> supporting the way Amazon implemented it.

Yes, thats the cost of moving first.
Honestly, I would assume we end up implementing two access routes, if
we support IPv6 first.

Thanks,
John

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] OpenStack support for Amazon Concepts - was Re: cloud-init IPv6 support

[John Garbutt]

On 7 September 2015 at 01:02, Jim Rollenhagen  wrote:
>> On Sep 6, 2015, at 09:43, Monty Taylor  wrote:
>>> On 09/05/2015 06:19 PM, Sean M. Collins wrote:
 On Fri, Sep 04, 2015 at 04:20:23PM EDT, Kevin Benton wrote:
 Right, it depends on your perspective of who 'owns' the API. Is it
 cloud-init or EC2?

 At this point I would argue that cloud-init is in control because it would
 be a large undertaking to switch all of the AMI's on Amazon to something
 else. However, I know Sean disagrees with me on this point so I'll let him
 reply here.
>>>
>>>
>>> Here's my take:
>>>
>>> Cloud-Init is a *client* of the Metadata API. The OpenStack Metadata API
>>> in both the Neutron and Nova projects should all the details of the
>>> Metadata API that is documented at:
>>>
>>> http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-instance-metadata.html
>>>
>>> This means that this is a compatibility layer that OpenStack has
>>> implemented so that users can use appliances, applications, and
>>> operating system images in both Amazon EC2 and an OpenStack environment.
>>>
>>> Yes, we can make changes to cloud-init. However, there is no guarantee
>>> that all users of the Metadata API are exclusively using cloud-init as
>>> their client. It is highly unlikely that people are rolling their own
>>> Metadata API clients, but it's a contract we've made with users. This
>>> includes transport level details like the IP address that the service
>>> listens on.
>>>
>>> The Metadata API is an established API that Amazon introduced years ago,
>>> and we shouldn't be "improving" APIs that we don't control. If Amazon
>>> were to introduce IPv6 support the Metadata API tomorrow, we would
>>> naturally implement it exactly the way they implemented it in EC2. We'd
>>> honor the contract that Amazon made with its users, in our Metadata API,
>>> since it is a compatibility layer.
>>>
>>> However, since they haven't defined transport level details of the
>>> Metadata API, regarding IPv6 - we can't take it upon ourselves to pick a
>>> solution. It is not our API.
>>>
>>> The nice thing about config-drive is that we've created a new mechanism
>>> for bootstrapping instances - by replacing the transport level details
>>> of the API. Rather than being a link-local address that instances access
>>> over HTTP, it's a device that guests can mount and read. The actual
>>> contents of the drive may have a similar schema as the Metadata API, but
>>> I think at this point we've made enough of a differentiation between the
>>> EC2 Metadata API and config-drive that I believe the contents of the
>>> actual drive that the instance mounts can be changed without breaking
>>> user expectations - since config-drive was developed by the OpenStack
>>> community. The point being that we call it "config-drive" in
>>> conversation and our docs. Users understand that config-drive is a
>>> different feature.
>>
>> Another great part about config-drive is that it's scalable. At infra's 
>> application scale, we take pains to disable anyting in our images that might 
>> want to contact the metadata API because we're essentially a DDOS on it.
>
> So, I tend to think a simple API service like this should never be hard to 
> scale. Put a bunch of hosts behind a load balancer, boom, done. Even 1000 
> requests/s shouldn't be hard, though it may require many hosts, and that's 
> far beyond what infra would hit today.
>
> The one problem I have with config-drive is that it is static. I'd love for 
> systems like cloud-init, glean, etc, to be able to see changes to mounted 
> disks, attached networks, etc. Attaching things after the fact isn't 
> uncommon, and to make the user config the thing is a terrible experience. :(

While I would love to avoid the complexity of the metadata service,
its dynamic nature is the key bit you loose with config drive.

For example, our mechanism for passwords (sure, I wish everyone used
keys) uses the openstack metadata service as a two way communication
system. Add VIF is probably a better example.

Thanks,
John

PS
If we get the per instance keystone token 'injection' working
correctly using purely config drive, then instances could just
authenticate to the regular API, avoiding the need for the metadata
service in its current form, but thats probably a red herring at this
point in time.

>> config-drive being local to the hypervisor host makes it MUCH more stable at 
>> scale.
>>
>> cloud-init supports config-drive
>>
>> If it were up to me, nobody would be enablig the metadata API in new 
>> deployments.
>>
>> I totally agree that we should not make changes in the metadata API.
>>
>>> I've had this same conversation about the Security Group API that we
>>> have. We've named it the same thing as the Amazon API, but then went and
>>> made all the fields different, inexplicably. Thankfully, it's just the
>>> names of the fields, rather than being huge 

Re: [openstack-dev] OpenStack support for Amazon Concepts - was Re: cloud-init IPv6 support

[Sean M. Collins]

On Sun, Sep 06, 2015 at 04:25:43PM EDT, Kevin Benton wrote:
> So it's been pointed out that http://169.254.169.254/openstack is completed
> OpenStack invented. I don't quite understand how that's not violating the
> contract you said we have with end users about EC2 compatibility under the
> restriction of 'no new stuff'.

I think that is a violation. I don't think that allows us to make more
changes, just because we've broken the contract once, so a second
infraction is less significant. 

> If we added an IPv6 endpoint that the metadata service listens on, it would
> just be another place that non cloud-init clients don't know how to talk
> to. It's not going to break our compatibility with any clients that connect
> to the IPv4 address.

No, but if Amazon were to make a decision about how to implement IPv6 in
EC2 and how to make the Metadata API service work with IPv6 we'd be
supporting two implementations - the one we came up with and one for
supporting the way Amazon implemented it.

-- 
Sean M. Collins

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] OpenStack support for Amazon Concepts - was Re: cloud-init IPv6 support

[Jim Rollenhagen]

> On Sep 6, 2015, at 09:43, Monty Taylor  wrote:
> 
>> On 09/05/2015 06:19 PM, Sean M. Collins wrote:
>>> On Fri, Sep 04, 2015 at 04:20:23PM EDT, Kevin Benton wrote:
>>> Right, it depends on your perspective of who 'owns' the API. Is it
>>> cloud-init or EC2?
>>> 
>>> At this point I would argue that cloud-init is in control because it would
>>> be a large undertaking to switch all of the AMI's on Amazon to something
>>> else. However, I know Sean disagrees with me on this point so I'll let him
>>> reply here.
>> 
>> 
>> Here's my take:
>> 
>> Cloud-Init is a *client* of the Metadata API. The OpenStack Metadata API
>> in both the Neutron and Nova projects should all the details of the
>> Metadata API that is documented at:
>> 
>> http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-instance-metadata.html
>> 
>> This means that this is a compatibility layer that OpenStack has
>> implemented so that users can use appliances, applications, and
>> operating system images in both Amazon EC2 and an OpenStack environment.
>> 
>> Yes, we can make changes to cloud-init. However, there is no guarantee
>> that all users of the Metadata API are exclusively using cloud-init as
>> their client. It is highly unlikely that people are rolling their own
>> Metadata API clients, but it's a contract we've made with users. This
>> includes transport level details like the IP address that the service
>> listens on.
>> 
>> The Metadata API is an established API that Amazon introduced years ago,
>> and we shouldn't be "improving" APIs that we don't control. If Amazon
>> were to introduce IPv6 support the Metadata API tomorrow, we would
>> naturally implement it exactly the way they implemented it in EC2. We'd
>> honor the contract that Amazon made with its users, in our Metadata API,
>> since it is a compatibility layer.
>> 
>> However, since they haven't defined transport level details of the
>> Metadata API, regarding IPv6 - we can't take it upon ourselves to pick a
>> solution. It is not our API.
>> 
>> The nice thing about config-drive is that we've created a new mechanism
>> for bootstrapping instances - by replacing the transport level details
>> of the API. Rather than being a link-local address that instances access
>> over HTTP, it's a device that guests can mount and read. The actual
>> contents of the drive may have a similar schema as the Metadata API, but
>> I think at this point we've made enough of a differentiation between the
>> EC2 Metadata API and config-drive that I believe the contents of the
>> actual drive that the instance mounts can be changed without breaking
>> user expectations - since config-drive was developed by the OpenStack
>> community. The point being that we call it "config-drive" in
>> conversation and our docs. Users understand that config-drive is a
>> different feature.
> 
> Another great part about config-drive is that it's scalable. At infra's 
> application scale, we take pains to disable anyting in our images that might 
> want to contact the metadata API because we're essentially a DDOS on it.

So, I tend to think a simple API service like this should never be hard to 
scale. Put a bunch of hosts behind a load balancer, boom, done. Even 1000 
requests/s shouldn't be hard, though it may require many hosts, and that's far 
beyond what infra would hit today. 

The one problem I have with config-drive is that it is static. I'd love for 
systems like cloud-init, glean, etc, to be able to see changes to mounted 
disks, attached networks, etc. Attaching things after the fact isn't uncommon, 
and to make the user config the thing is a terrible experience. :(

// jim 

> 
> config-drive being local to the hypervisor host makes it MUCH more stable at 
> scale.
> 
> cloud-init supports config-drive
> 
> If it were up to me, nobody would be enablig the metadata API in new 
> deployments.
> 
> I totally agree that we should not make changes in the metadata API.
> 
>> I've had this same conversation about the Security Group API that we
>> have. We've named it the same thing as the Amazon API, but then went and
>> made all the fields different, inexplicably. Thankfully, it's just the
>> names of the fields, rather than being huge conceptual changes.
>> 
>> http://lists.openstack.org/pipermail/openstack-dev/2015-June/068319.html
>> 
>> Basically, I believe that OpenStack should create APIs that are
>> community driven and owned, and that we should only emulate
>> non-community APIs where appropriate, and explicitly state that we only
>> are emulating them. Putting improvements in APIs that came from
>> somewhere else, instead of creating new OpenStack branded APIs is a lost
>> opportunity to differentiate OpenStack from other projects, as well as
>> Amazon AWS.
>> 
>> Thanks for reading, and have a great holiday.
> 
> I could not possibly agree more if our brains were physically fused.
> 
> __
> 

Re: [openstack-dev] OpenStack support for Amazon Concepts - was Re: cloud-init IPv6 support

[Kevin Benton]

So it's been pointed out that http://169.254.169.254/openstack is completed
OpenStack invented. I don't quite understand how that's not violating the
contract you said we have with end users about EC2 compatibility under the
restriction of 'no new stuff'.

If we added an IPv6 endpoint that the metadata service listens on, it would
just be another place that non cloud-init clients don't know how to talk
to. It's not going to break our compatibility with any clients that connect
to the IPv4 address.
On Sep 5, 2015 3:22 PM, "Sean M. Collins"  wrote:

> On Fri, Sep 04, 2015 at 04:20:23PM EDT, Kevin Benton wrote:
> > Right, it depends on your perspective of who 'owns' the API. Is it
> > cloud-init or EC2?
> >
> > At this point I would argue that cloud-init is in control because it
> would
> > be a large undertaking to switch all of the AMI's on Amazon to something
> > else. However, I know Sean disagrees with me on this point so I'll let
> him
> > reply here.
>
>
> Here's my take:
>
> Cloud-Init is a *client* of the Metadata API. The OpenStack Metadata API
> in both the Neutron and Nova projects should all the details of the
> Metadata API that is documented at:
>
>
> http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-instance-metadata.html
>
> This means that this is a compatibility layer that OpenStack has
> implemented so that users can use appliances, applications, and
> operating system images in both Amazon EC2 and an OpenStack environment.
>
> Yes, we can make changes to cloud-init. However, there is no guarantee
> that all users of the Metadata API are exclusively using cloud-init as
> their client. It is highly unlikely that people are rolling their own
> Metadata API clients, but it's a contract we've made with users. This
> includes transport level details like the IP address that the service
> listens on.
>
> The Metadata API is an established API that Amazon introduced years ago,
> and we shouldn't be "improving" APIs that we don't control. If Amazon
> were to introduce IPv6 support the Metadata API tomorrow, we would
> naturally implement it exactly the way they implemented it in EC2. We'd
> honor the contract that Amazon made with its users, in our Metadata API,
> since it is a compatibility layer.
>
> However, since they haven't defined transport level details of the
> Metadata API, regarding IPv6 - we can't take it upon ourselves to pick a
> solution. It is not our API.
>
> The nice thing about config-drive is that we've created a new mechanism
> for bootstrapping instances - by replacing the transport level details
> of the API. Rather than being a link-local address that instances access
> over HTTP, it's a device that guests can mount and read. The actual
> contents of the drive may have a similar schema as the Metadata API, but
> I think at this point we've made enough of a differentiation between the
> EC2 Metadata API and config-drive that I believe the contents of the
> actual drive that the instance mounts can be changed without breaking
> user expectations - since config-drive was developed by the OpenStack
> community. The point being that we call it "config-drive" in
> conversation and our docs. Users understand that config-drive is a
> different feature.
>
> I've had this same conversation about the Security Group API that we
> have. We've named it the same thing as the Amazon API, but then went and
> made all the fields different, inexplicably. Thankfully, it's just the
> names of the fields, rather than being huge conceptual changes.
>
> http://lists.openstack.org/pipermail/openstack-dev/2015-June/068319.html
>
> Basically, I believe that OpenStack should create APIs that are
> community driven and owned, and that we should only emulate
> non-community APIs where appropriate, and explicitly state that we only
> are emulating them. Putting improvements in APIs that came from
> somewhere else, instead of creating new OpenStack branded APIs is a lost
> opportunity to differentiate OpenStack from other projects, as well as
> Amazon AWS.
>
> Thanks for reading, and have a great holiday.
>
> --
> Sean M. Collins
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] OpenStack support for Amazon Concepts - was Re: cloud-init IPv6 support

[Monty Taylor]

On 09/05/2015 06:19 PM, Sean M. Collins wrote:

On Fri, Sep 04, 2015 at 04:20:23PM EDT, Kevin Benton wrote:

Right, it depends on your perspective of who 'owns' the API. Is it
cloud-init or EC2?

At this point I would argue that cloud-init is in control because it would
be a large undertaking to switch all of the AMI's on Amazon to something
else. However, I know Sean disagrees with me on this point so I'll let him
reply here.



Here's my take:

Cloud-Init is a *client* of the Metadata API. The OpenStack Metadata API
in both the Neutron and Nova projects should all the details of the
Metadata API that is documented at:

http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-instance-metadata.html

This means that this is a compatibility layer that OpenStack has
implemented so that users can use appliances, applications, and
operating system images in both Amazon EC2 and an OpenStack environment.

Yes, we can make changes to cloud-init. However, there is no guarantee
that all users of the Metadata API are exclusively using cloud-init as
their client. It is highly unlikely that people are rolling their own
Metadata API clients, but it's a contract we've made with users. This
includes transport level details like the IP address that the service
listens on.

The Metadata API is an established API that Amazon introduced years ago,
and we shouldn't be "improving" APIs that we don't control. If Amazon
were to introduce IPv6 support the Metadata API tomorrow, we would
naturally implement it exactly the way they implemented it in EC2. We'd
honor the contract that Amazon made with its users, in our Metadata API,
since it is a compatibility layer.

However, since they haven't defined transport level details of the
Metadata API, regarding IPv6 - we can't take it upon ourselves to pick a
solution. It is not our API.

The nice thing about config-drive is that we've created a new mechanism
for bootstrapping instances - by replacing the transport level details
of the API. Rather than being a link-local address that instances access
over HTTP, it's a device that guests can mount and read. The actual
contents of the drive may have a similar schema as the Metadata API, but
I think at this point we've made enough of a differentiation between the
EC2 Metadata API and config-drive that I believe the contents of the
actual drive that the instance mounts can be changed without breaking
user expectations - since config-drive was developed by the OpenStack
community. The point being that we call it "config-drive" in
conversation and our docs. Users understand that config-drive is a
different feature.


Another great part about config-drive is that it's scalable. At infra's 
application scale, we take pains to disable anyting in our images that 
might want to contact the metadata API because we're essentially a DDOS 
on it.


config-drive being local to the hypervisor host makes it MUCH more 
stable at scale.


cloud-init supports config-drive

If it were up to me, nobody would be enablig the metadata API in new 
deployments.


I totally agree that we should not make changes in the metadata API.


I've had this same conversation about the Security Group API that we
have. We've named it the same thing as the Amazon API, but then went and
made all the fields different, inexplicably. Thankfully, it's just the
names of the fields, rather than being huge conceptual changes.

http://lists.openstack.org/pipermail/openstack-dev/2015-June/068319.html

Basically, I believe that OpenStack should create APIs that are
community driven and owned, and that we should only emulate
non-community APIs where appropriate, and explicitly state that we only
are emulating them. Putting improvements in APIs that came from
somewhere else, instead of creating new OpenStack branded APIs is a lost
opportunity to differentiate OpenStack from other projects, as well as
Amazon AWS.

Thanks for reading, and have a great holiday.



I could not possibly agree more if our brains were physically fused.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev