On 2020-11-19 00:07, Tomasz Chmielewski wrote:
On 2020-11-18 23:50, Tomasz Chmielewski wrote:
That's a weird one!
In AWS, there is a concept of "instance metadata" - a webserver which
lets you fetch some instance metadata using http:
https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instancedata-data-retrieval.html
For example, you can run this (from both AWS/EC2 instance and LXD
container running inside a AWS/EC2 instance), it will return some
metadata:
curl -v http://169.254.169.254/latest/meta-data/
Now, some of these requests time out when executed from a LXD
container running inside a AWS/EC2 - but work perfectly from the very
same AWS/EC2 instance.
For example, this request works fine from AWS/EC2 instance (ignore the
output - HTTP connection works just fine):
root@aws-instance:~# curl -v http://169.254.169.254/latest/api/token
* Trying 169.254.169.254...
* TCP_NODELAY set
* Connected to 169.254.169.254 (169.254.169.254) port 80 (#0)
GET /latest/api/token HTTP/1.1
Host: 169.254.169.254
User-Agent: curl/7.58.0
Accept: */*
< HTTP/1.1 405 Not Allowed
< Allow: OPTIONS, PUT
< Content-Length: 0
< Date: Wed, 18 Nov 2020 22:41:46 GMT
< Server: EC2ws
< Connection: close
< Content-Type: text/plain
<
* Closing connection 0
However, when executed from within a LXD container running inside the
very same AWS/EC2 instance - it times out!
root@lxd-container:~# curl -v http://169.254.169.254/latest/api/token
* Trying 169.254.169.254...
* TCP_NODELAY set
* Connected to 169.254.169.254 (169.254.169.254) port 80 (#0)
GET /latest/api/token HTTP/1.1
Host: 169.254.169.254
User-Agent: curl/7.58.0
Accept: */*
Even more weirdly, these work inside the container:
curl -v http://169.254.169.254/latest/api/
curl -v http://169.254.169.254/latest/api/t
curl -v http://169.254.169.254/latest/api/to
curl -v http://169.254.169.254/latest/api/tok
curl -v http://169.254.169.254/latest/api/toke
And this times out:
curl -v http://169.254.169.254/latest/api/token
Does anyone know why? tcpdump doesn't give me many clues (TTL?).
A somewhat related post (with docker having a similar issue):
https://rtfm.co.ua/en/aws-eksctl-put-http-169-254-169-254-latest-api-token-net-http-request-canceled-2/
But, I'm no closer in getting a similar workaround for LXD.
If someone's struggling with a similar issue - here is a fix:
aws ec2 modify-instance-metadata-options --instance-id i-abcdefghijklmn
--http-put-response-hop-limit 2 --http-endpoint enabled
Tomasz Chmielewski
https://lxadm.com
_______________________________________________
lxc-users mailing list
lxc-users@lists.linuxcontainers.org
http://lists.linuxcontainers.org/listinfo/lxc-users
