[openstack-dev] [Nova][Docker] Environment variables
Hi All, I have submitted a new blueprint which addresses the a common pattern in the docker world. A usual pattern in the docker world is to use environment variables to configure a container. docker run -e SQL_URL=postgres://user:password@/db my-app The nova docker driver doesn't support to set any environment variables. To work around this issue I used cloud-init which works fine. But this approach has of course the drawback that a) I have to install the cloud init service. and b) my docker container doesn't work outside of openstack. I propose to allow a user to set docker environment variables via nova instance metadata. The metadata key should have a prefix like ENV_ which can be used to determine all environment variables. The prefix should be removed and the remainder key and vaule will be injected. The metadata can unfortunately not be set in horizon but can be used from the nova command line tool and from heat. Example heat: myapp: Type: OS::Nova::Server Properties: flavor: m1.small image: my-app:latest meta-data: - ENV_SQL_URL: postgres://user:password@/db - ENV_SOMETHING_ELSE: Value Let me know what you think about that. Blueprint: https://blueprints.launchpad.net/nova/+spec/docker-env-via-meta-data regards, Daniel ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova][Docker] Environment variables
Hi Russell, You actually propose to extend the whole nova stack to support environment variables. Would any other driver benefit from this API extension? Is that what you imagine? nova --env SQL_URL=postgres://user:password --image Regarding the discussion you mentioned. Are there any public resources to read. I kind of missed it. Most likely it was before I was part of this community :) thanks, Daniel On Mon, Dec 16, 2013 at 4:07 PM, Russell Bryant rbry...@redhat.com wrote: On 12/16/2013 09:27 AM, Daniel Kuffner wrote: Hi All, I have submitted a new blueprint which addresses the a common pattern in the docker world. A usual pattern in the docker world is to use environment variables to configure a container. docker run -e SQL_URL=postgres://user:password@/db my-app The nova docker driver doesn't support to set any environment variables. To work around this issue I used cloud-init which works fine. But this approach has of course the drawback that a) I have to install the cloud init service. and b) my docker container doesn't work outside of openstack. I propose to allow a user to set docker environment variables via nova instance metadata. The metadata key should have a prefix like ENV_ which can be used to determine all environment variables. The prefix should be removed and the remainder key and vaule will be injected. The metadata can unfortunately not be set in horizon but can be used from the nova command line tool and from heat. Example heat: myapp: Type: OS::Nova::Server Properties: flavor: m1.small image: my-app:latest meta-data: - ENV_SQL_URL: postgres://user:password@/db - ENV_SOMETHING_ELSE: Value Let me know what you think about that. Blueprint: https://blueprints.launchpad.net/nova/+spec/docker-env-via-meta-data Thanks for starting the discussion. More people should do this for their blueprints. :-) One of the things we should be striving for is to provide as consistent of an experience as we can across drivers. Right now, we have the metadata service and config drive, and neither of those are driver specific. In the case of config drive, whether it's used or not is exposed through the API. As you point out, the meta-data service does technically work with the docker driver. I don't think we should support environment variables like this automatically. Instead, I think it would be more appropriate to add an API extension for specifying env vars. That way the behavior is more explicit and communicated through the API. The env vars would be passed through all of the appropriate plumbing and down to drivers that are able to support it. This is all also assuming that containers support is staying in Nova and not a new service. That discussion seems to have stalled. Is anyone still pushing on that? Any updates? -- Russell Bryant ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova][Docker] Environment variables
Hi Chuck, yes please, I'm eager to see what you have. :) Daniel On Mon, Dec 16, 2013 at 4:12 PM, Chuck Short chuck.sh...@canonical.com wrote: Hi Russel, I have something that is pushing it for to stay in nova (at least the compute drivers). I should have a gerrit branch for people to review soon. Regards chuck On Mon, Dec 16, 2013 at 10:07 AM, Russell Bryant rbry...@redhat.com wrote: On 12/16/2013 09:27 AM, Daniel Kuffner wrote: Hi All, I have submitted a new blueprint which addresses the a common pattern in the docker world. A usual pattern in the docker world is to use environment variables to configure a container. docker run -e SQL_URL=postgres://user:password@/db my-app The nova docker driver doesn't support to set any environment variables. To work around this issue I used cloud-init which works fine. But this approach has of course the drawback that a) I have to install the cloud init service. and b) my docker container doesn't work outside of openstack. I propose to allow a user to set docker environment variables via nova instance metadata. The metadata key should have a prefix like ENV_ which can be used to determine all environment variables. The prefix should be removed and the remainder key and vaule will be injected. The metadata can unfortunately not be set in horizon but can be used from the nova command line tool and from heat. Example heat: myapp: Type: OS::Nova::Server Properties: flavor: m1.small image: my-app:latest meta-data: - ENV_SQL_URL: postgres://user:password@/db - ENV_SOMETHING_ELSE: Value Let me know what you think about that. Blueprint: https://blueprints.launchpad.net/nova/+spec/docker-env-via-meta-data Thanks for starting the discussion. More people should do this for their blueprints. :-) One of the things we should be striving for is to provide as consistent of an experience as we can across drivers. Right now, we have the metadata service and config drive, and neither of those are driver specific. In the case of config drive, whether it's used or not is exposed through the API. As you point out, the meta-data service does technically work with the docker driver. I don't think we should support environment variables like this automatically. Instead, I think it would be more appropriate to add an API extension for specifying env vars. That way the behavior is more explicit and communicated through the API. The env vars would be passed through all of the appropriate plumbing and down to drivers that are able to support it. This is all also assuming that containers support is staying in Nova and not a new service. That discussion seems to have stalled. Is anyone still pushing on that? Any updates? -- Russell Bryant ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova][Docker] Environment variables
That would be great, I have also a couple of change request waiting for approval. Would be good to know if it has any relevance in the future. https://review.openstack.org/#/c/59824/ https://review.openstack.org/#/c/62182/ https://review.openstack.org/#/c/62183/ https://review.openstack.org/#/c/62220/ Daniel On Mon, Dec 16, 2013 at 4:17 PM, Russell Bryant rbry...@redhat.com wrote: On 12/16/2013 10:12 AM, Chuck Short wrote: I have something that is pushing it for to stay in nova (at least the compute drivers). I should have a gerrit branch for people to review soon. OK. Do you have any design notes for whatever you're proposing? That would probably be easier to review and discuss. -- Russell Bryant ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova][Docker] Environment variables
Actually Daniel P. Barrange comment is interesting. He states that a configuration per instance would also be beneficial for Cinder. The configuration is essentially needed to change the bootstrap of the image. If you look at a docker image in abstract way then that is the same thing - a configuration to bootstrap the image differently. From user point of view it could look like: nova boot --image-meta key=value Daniel On Mon, Dec 16, 2013 at 10:04 PM, Dan Smith d...@danplanet.com wrote: eg use a 'env_' prefix for glance image attributes We've got a couple of cases now where we want to overrides these same things on a per-instance basis. Kernel command line args is one other example. Other hardware overrides like disk/net device types are another possibility Rather than invent new extensions for each, I think we should have a way to pass arbitrary attributes alon with the boot API call, that a driver would handle in much the same way as they do for glance image properties. Basically think of it as a way to custom any image property per instance created. Personally, I think having a bunch of special case magic namespaces (even if documented) is less desirable than a proper API to do something like this. Especially a namespace that someone else could potentially use legitimately that would conflict. To me, this feels a lot like what I'm worried this effort will turn into, which is making containers support in Nova look like a bolt-on thing with a bunch of specialness required to make it behave. Anyone remember this bolt-on gem? nova boot --block-device-mapping vda=965453c9-02b5-4d5b-8ec0-3164a89bf6f4:::0 --flavor=m1.tiny --image=6415797a-7c03-45fe-b490-f9af99d2bae0 BFV I found that one amidst hundreds of forum threads of people confused about what incantation of magic they were supposed to do to make it actually boot from volume. Just MHO. --Dan ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Openstack][Nova][Docker] Devstack with docker driver
Hi Swapnil, Looks like the docker-registry image is broken, since it cannot find run.sh inside the container. 2014/01/09 06:36:15 Unable to locate ./docker-registry/run.sh Maybe you could try to remove and re-import image docker rmi docker-registry and then execute ./tools/docker/install_docker.sh again. On Thu, Jan 9, 2014 at 7:42 AM, Swapnil Kulkarni swapnilkulkarni2...@gmail.com wrote: Hi Eric, I tried running the 'docker run' command without -d and it gets following error $ sudo docker run -d=false -p 5042:5000 -e SETTINGS_FLAVOR=openstack -e OS_USERNAME=admin -e OS_PASSWORD=password -e OS_TENANT_NAME=admin -e OS_GLANCE_URL=http://127.0.0.1:9292 -e OS_AUTH_URL=http://127.0.0.1:35357/v2.0 docker-registry ./docker-registry/run.sh lxc-start: No such file or directory - stat(/proc/16438/root/dev//console) 2014/01/09 06:36:15 Unable to locate ./docker-registry/run.sh On the other hand, If I run the failing command just after stack.sh fails with -d, it works fine, sudo docker run -d -p 5042:5000 -e SETTINGS_FLAVOR=openstack -e OS_USERNAME=admin -e OS_PASSWORD=password -e OS_TENANT_NAME=admin -e OS_GLANCE_URL=http://127.0.0.1:9292 -e OS_AUTH_URL=http://127.0.0.1:35357/v2.0 docker-registry ./docker-registry/run.sh 5b737f8d2282114c1a0cfc4f25bc7c9ef8c5da7e0d8fa7ed9ccee0be81cddafc Best Regards, Swapnil On Wed, Jan 8, 2014 at 8:29 PM, Eric Windisch e...@windisch.us wrote: On Tue, Jan 7, 2014 at 11:13 PM, Swapnil Kulkarni swapnilkulkarni2...@gmail.com wrote: Let me know in case I can be of any help getting this resolved. Please try running the failing 'docker run' command manually and without the '-d' argument. I've been able to reproduce an error myself, but wish to confirm that this matches the error you're seeing. Regards, Eric Windisch ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Openstack][Nova][Docker] Devstack with docker driver
Hello, I just had the same problem today with a openstack installation form packstack. The problem was the firewall so I had to open 9292, 35357 iptables -I INPUT -p tcp --dport 9292 -j ACCEPT iptables -I INPUT -p tcp --dport 35357 -j ACCEPT Daniel On Thu, Jan 9, 2014 at 3:24 PM, Swapnil Kulkarni swapnilkulkarni2...@gmail.com wrote: Hi Daniel, I think it was some proxy issue prohibiting from the download. I downloaded the correct docker-registry.tar.gz and stack.sh completed but with some issues remain related to image availability. The docker push is getting HTTP code 500 while uploading metadata: invalid character '' looking for beginning of value error. I did a search for the error and found [1] on ask.openstack which states firewall could be an issue, which is not I guess in my situation. Any ideas? [1] https://ask.openstack.org/en/question/8320/docker-push-error/ Best Regards, Swapnil On Thu, Jan 9, 2014 at 7:37 PM, Daniel Kuffner daniel.kuff...@gmail.com wrote: The tar file seem to be corrupt, the tools script downloads it to: ./files/docker-registry.tar.gz On Thu, Jan 9, 2014 at 11:17 AM, Swapnil Kulkarni swapnilkulkarni2...@gmail.com wrote: Hi Daniel, I removed the existing images and executed ./tools/docker/install_docker.sh. I am facing new issue related to docker-registry, Error: exit status 2: tar: This does not look like a tar archive is the size 945 bytes correct for docker-registry image? Best Regards, Swapnil On Thu, Jan 9, 2014 at 1:35 PM, Daniel Kuffner daniel.kuff...@gmail.com wrote: Hi Swapnil, Looks like the docker-registry image is broken, since it cannot find run.sh inside the container. 2014/01/09 06:36:15 Unable to locate ./docker-registry/run.sh Maybe you could try to remove and re-import image docker rmi docker-registry and then execute ./tools/docker/install_docker.sh again. On Thu, Jan 9, 2014 at 7:42 AM, Swapnil Kulkarni swapnilkulkarni2...@gmail.com wrote: Hi Eric, I tried running the 'docker run' command without -d and it gets following error $ sudo docker run -d=false -p 5042:5000 -e SETTINGS_FLAVOR=openstack -e OS_USERNAME=admin -e OS_PASSWORD=password -e OS_TENANT_NAME=admin -e OS_GLANCE_URL=http://127.0.0.1:9292 -e OS_AUTH_URL=http://127.0.0.1:35357/v2.0 docker-registry ./docker-registry/run.sh lxc-start: No such file or directory - stat(/proc/16438/root/dev//console) 2014/01/09 06:36:15 Unable to locate ./docker-registry/run.sh On the other hand, If I run the failing command just after stack.sh fails with -d, it works fine, sudo docker run -d -p 5042:5000 -e SETTINGS_FLAVOR=openstack -e OS_USERNAME=admin -e OS_PASSWORD=password -e OS_TENANT_NAME=admin -e OS_GLANCE_URL=http://127.0.0.1:9292 -e OS_AUTH_URL=http://127.0.0.1:35357/v2.0 docker-registry ./docker-registry/run.sh 5b737f8d2282114c1a0cfc4f25bc7c9ef8c5da7e0d8fa7ed9ccee0be81cddafc Best Regards, Swapnil On Wed, Jan 8, 2014 at 8:29 PM, Eric Windisch e...@windisch.us wrote: On Tue, Jan 7, 2014 at 11:13 PM, Swapnil Kulkarni swapnilkulkarni2...@gmail.com wrote: Let me know in case I can be of any help getting this resolved. Please try running the failing 'docker run' command manually and without the '-d' argument. I've been able to reproduce an error myself, but wish to confirm that this matches the error you're seeing. Regards, Eric Windisch ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [nova][network] Gateway in network_info seems not to be correct
Hi All, I'm currently trying out the docker in FlatDHCPManager multi_host mode. I figured out that my instance are not pingable. The reason for that is that they get the wrong gateway assigned. Currently we use the gateway field provided by the network_info object. That works great in a single node setup. But in a multi_host setup it still delivers (in my case) 10.0.0.1. Which is wrong since it must be dnsmasq IP address of the compute node. Should a driver use the DNS ip (dnsmasq) address in the multi_host mode as gateway? Why is the gateway delivered by the nova not correct (bug)? Any insides? Corresponding bug: please see also: https://bugs.launchpad.net/nova/+bug/1279509 thank you, Daniel ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Openstack][Nova][Docker] Devstack with docker driver
Hi, what is the error reported by docker? Can you post the docker registry log? What version of docker do you use? I assume you use devstack master branch? thank you, Daniel On Tue, Mar 11, 2014 at 1:19 PM, urgensherpa sherpa.ur...@outlook.com wrote: Hello!, i can run docker containers and push it to docker io but i failed to push it for local glance.and get the same error mentioned here. Could you please show some more light on how you resolved it. i started settingup openstack and docker using devstack. here is my localrc FLOATING_RANGE=192.168.140.0/27 FIXED_RANGE=10.11.12.0/24 FIXED_NETWORK_SIZE=256 FLAT_INTERFACE=eth1 ADMIN_PASSWORD=g MYSQL_PASSWORD=g RABBIT_PASSWORD=g SERVICE_PASSWORD=g SERVICE_TOKEN=g SCHEDULER=nova.scheduler.filter_scheduler.FilterScheduler VIRT_DRIVER=docker SCREEN_LOGDIR=$DEST/logs/screen --- the machine im testing is on vmware ubuntu 13.01 with two nics assuming eth0 connected to internet and eth1 to local network. --- -- View this message in context: http://openstack.10931.n7.nabble.com/Openstack-Nova-Docker-Devstack-with-docker-driver-tp28361p34845.html Sent from the Developer mailing list archive at Nabble.com. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [Nova][Docker] What are the plans or thoughts about providing volumes aka folder mounts
Hi All, We are using in our company for a prototype the docker hypervisor on openstack. We have the need to mount a folder inside of a container. To achieve this goal I have implemented a hack which allows to specify a folder mount via nova metadata. For example a heat template could look like: my-container: Type: OS::Nova::Server Properties: flavor: m1.large image: my-image:latest metadata: Volumes: /host/path:/guest/path This approach is of course not perfect and even a security risk (which is in our case no issue since we are not going to provide a public cloud). Any other ideas or plans how to provide the volume/folder mount in the future? regards, Daniel ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova][Docker] What are the plans or thoughts about providing volumes aka folder mounts
Yes agree it not very cloud like. Thanks for pointing out the Manila project, didn't know about it. On Fri, Dec 6, 2013 at 5:25 PM, Russell Bryant rbry...@redhat.com wrote: On 12/06/2013 10:54 AM, Daniel Kuffner wrote: Hi All, We are using in our company for a prototype the docker hypervisor on openstack. We have the need to mount a folder inside of a container. To achieve this goal I have implemented a hack which allows to specify a folder mount via nova metadata. For example a heat template could look like: my-container: Type: OS::Nova::Server Properties: flavor: m1.large image: my-image:latest metadata: Volumes: /host/path:/guest/path This approach is of course not perfect and even a security risk (which is in our case no issue since we are not going to provide a public cloud). Any other ideas or plans how to provide the volume/folder mount in the future? I think *directly* specifying a host path isn't very cloudy. We don't really have an abstraction appropriate for this yet. Manila [1] (filesystem aaS) seems to be the closest thing. Perhaps some work in Manila and some Nova+Manila integration would be the right direction here. Thoughts? [1] https://wiki.openstack.org/wiki/Manila_Overview -- Russell Bryant ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
