Re: [openstack-dev] [kolla] Kolla Blueprints

[Steven Dake]

On 10/01/2014 07:08 PM, Angus Lees wrote:

On Wed, 1 Oct 2014 09:05:23 PM Fox, Kevin M wrote:

Has anyone figured out a way of having a floating ip like feature with
docker so that you can have rabbitmq, mysql, or ceph mon's at fixed ip's
and be able to migrate them around from physical host to physical host and
still have them at fixed locations that you can easily put in static config
files?

This is part of the additional functionality kubernetes adds on top of docker.

kubernetes uses a "proxy" on every host which knows about all the published
"services".  The services share a port space (ie: every service has to have a
unique port assigned), and the proxies know where to forward requests to find
one of the backends for that service.

docker communicates parameters via environment variables and has a few
standard environment variables that are used for "links" to other containers.
Kubernetes also uses these link env variables but points them at the proxies
instead of directly to the other containers.  Since oslo.config can't look up
environment variables directly (that's something I'd like to add), I have a
simple shell one-liner that expands environment variables in the relevant
config files before starting the openstack server.

As a concrete example: I configure a keystone service in my kubernetes config
and in my static config files I use values like:

identity_uri = http://$ENV[KEYSTONE_PORT_5000_TCP_ADDR]:
$ENV[KEYSTONE_PORT_5000_TCP_PORT]/v2.0

docker/kubernetes sets those env variables to refer to the proxy on the local
host and the port number from my service config - this information is static
for the lifetime of that docker instance.  The proxy will reroute the requests
dynamically to wherever the actual instances are running right now.

I hope that's enough detail - I encourage you to read the kubernetes docs
since they have diagrams, etc that will make it much clearer than the above.

Really nice explanation.

+2!

Regards,
-steve


  - Gus


Maybe iptables rules? Maybe adding another bridge? Maybe just disabling the
docker network stack all together and binding the service to a fixed,
static address on the host?

Also, I ran across:
http://jperrin.github.io/centos/2014/09/25/centos-docker-and-systemd/ and
it does seem to work. I was able to get openssh-server and keystone to work
in the same container without needing to write custom start/stop scripts.
This kind of setup would make a nova compute container much, much easier.

Thanks,
Kevin

From: Steven Dake [sd...@redhat.com]
Sent: Wednesday, October 01, 2014 8:04 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [kolla] Kolla Blueprints

On 09/30/2014 09:55 AM, Chmouel Boudjnah wrote:

On Tue, Sep 30, 2014 at 6:41 PM, Steven Dake
mailto:sd...@redhat.com>> wrote:

I've done a first round of prioritization.  I think key things we need
people to step up for are nova and rabbitmq containers.

For the developers, please take a moment to pick a specific blueprint to
work on.  If your already working on something, this hsould help to prevent
duplicate work :)


As I understand in the current implementations[1]  the containers are
configured with a mix of shell scripts using crudini and other shell
command. Is it the way to configure the containers? and is a deployment
tool like Ansible (or others) is something that is planned to be used in
the future?

Chmouel

Chmouel,

I am not really sure what the best solution to configure the containers.  It
is clear to me the current shell scripts are fragile in nature and do not
handle container restart properly.  The idea of using Puppet or Ansible as
a CM tool has been discussed with no resolution.  At the moment, I'm
satisified with a somewhat hacky solution if we can get the containers
operational.

Regards,
-steve




[1] from https://github.com/jlabocki/superhappyfunshow/



___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org<mailto:OpenStack-dev@lists.openstack.org>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [kolla] Kolla Blueprints

[Angus Lees]

On Wed, 1 Oct 2014 09:05:23 PM Fox, Kevin M wrote:
> Has anyone figured out a way of having a floating ip like feature with
> docker so that you can have rabbitmq, mysql, or ceph mon's at fixed ip's
> and be able to migrate them around from physical host to physical host and
> still have them at fixed locations that you can easily put in static config
> files?

This is part of the additional functionality kubernetes adds on top of docker.

kubernetes uses a "proxy" on every host which knows about all the published 
"services".  The services share a port space (ie: every service has to have a 
unique port assigned), and the proxies know where to forward requests to find 
one of the backends for that service.

docker communicates parameters via environment variables and has a few 
standard environment variables that are used for "links" to other containers.  
Kubernetes also uses these link env variables but points them at the proxies 
instead of directly to the other containers.  Since oslo.config can't look up 
environment variables directly (that's something I'd like to add), I have a 
simple shell one-liner that expands environment variables in the relevant 
config files before starting the openstack server.

As a concrete example: I configure a keystone service in my kubernetes config 
and in my static config files I use values like:

   identity_uri = http://$ENV[KEYSTONE_PORT_5000_TCP_ADDR]:
$ENV[KEYSTONE_PORT_5000_TCP_PORT]/v2.0

docker/kubernetes sets those env variables to refer to the proxy on the local 
host and the port number from my service config - this information is static 
for the lifetime of that docker instance.  The proxy will reroute the requests 
dynamically to wherever the actual instances are running right now.

I hope that's enough detail - I encourage you to read the kubernetes docs 
since they have diagrams, etc that will make it much clearer than the above.

 - Gus

> Maybe iptables rules? Maybe adding another bridge? Maybe just disabling the
> docker network stack all together and binding the service to a fixed,
> static address on the host?
> 
> Also, I ran across:
> http://jperrin.github.io/centos/2014/09/25/centos-docker-and-systemd/ and
> it does seem to work. I was able to get openssh-server and keystone to work
> in the same container without needing to write custom start/stop scripts.
> This kind of setup would make a nova compute container much, much easier.
> 
> Thanks,
> Kevin
> 
> From: Steven Dake [sd...@redhat.com]
> Sent: Wednesday, October 01, 2014 8:04 AM
> To: OpenStack Development Mailing List (not for usage questions)
> Subject: Re: [openstack-dev] [kolla] Kolla Blueprints
> 
> On 09/30/2014 09:55 AM, Chmouel Boudjnah wrote:
> 
> On Tue, Sep 30, 2014 at 6:41 PM, Steven Dake
> mailto:sd...@redhat.com>> wrote:
> 
> I've done a first round of prioritization.  I think key things we need
> people to step up for are nova and rabbitmq containers.
> 
> For the developers, please take a moment to pick a specific blueprint to
> work on.  If your already working on something, this hsould help to prevent
> duplicate work :)
> 
> 
> As I understand in the current implementations[1]  the containers are
> configured with a mix of shell scripts using crudini and other shell
> command. Is it the way to configure the containers? and is a deployment
> tool like Ansible (or others) is something that is planned to be used in
> the future?
> 
> Chmouel
> 
> Chmouel,
> 
> I am not really sure what the best solution to configure the containers.  It
> is clear to me the current shell scripts are fragile in nature and do not
> handle container restart properly.  The idea of using Puppet or Ansible as
> a CM tool has been discussed with no resolution.  At the moment, I'm
> satisified with a somewhat hacky solution if we can get the containers
> operational.
> 
> Regards,
> -steve
> 
> 
> 
> 
> [1] from https://github.com/jlabocki/superhappyfunshow/
> 
> 
> 
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org<mailto:OpenStack-dev@lists.openstack.org>
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

-- 
 - Gus

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [kolla] Kolla Blueprints

[Vishvananda Ishaya]

On Oct 1, 2014, at 2:05 PM, Fox, Kevin M  wrote:

> Has anyone figured out a way of having a floating ip like feature with docker 
> so that you can have rabbitmq, mysql, or ceph mon's at fixed ip's and be able 
> to migrate them around from physical host to physical host and still have 
> them at fixed locations that you can easily put in static config files?

There are[1] many[2] ways[3] to do this, but in general I don’t think they pass 
the “too much magic” sniff test. I think the standard docker approach of 
passing in the necessary ips via environment variables is probably the most 
user friendly option. Containers are light-weight enough to restart if the data 
changes.

[1] https://github.com/coreos/flannel
[2] https://github.com/vishvananda/wormhole
[3] https://github.com/openshift/geard/blob/master/docs/linking.md

Vish

> Maybe iptables rules? Maybe adding another bridge? Maybe just disabling the 
> docker network stack all together and binding the service to a fixed, static 
> address on the host?
> 
> Also, I ran across: 
> http://jperrin.github.io/centos/2014/09/25/centos-docker-and-systemd/ and it 
> does seem to work. I was able to get openssh-server and keystone to work in 
> the same container without needing to write custom start/stop scripts. This 
> kind of setup would make a nova compute container much, much easier.
> 
> Thanks,
> Kevin
> From: Steven Dake [sd...@redhat.com]
> Sent: Wednesday, October 01, 2014 8:04 AM
> To: OpenStack Development Mailing List (not for usage questions)
> Subject: Re: [openstack-dev] [kolla] Kolla Blueprints
> 
> On 09/30/2014 09:55 AM, Chmouel Boudjnah wrote:
>> 
>> On Tue, Sep 30, 2014 at 6:41 PM, Steven Dake  wrote:
>> 
>> I've done a first round of prioritization.  I think key things we need 
>> people to step up for are nova and rabbitmq containers.
>> 
>> For the developers, please take a moment to pick a specific blueprint to 
>> work on.  If your already working on something, this hsould help to prevent 
>> duplicate work :)
>> 
>> 
>> As I understand in the current implementations[1]  the containers are 
>> configured with a mix of shell scripts using crudini and other shell 
>> command. Is it the way to configure the containers? and is a deployment tool 
>> like Ansible (or others) is something that is planned to be used in the 
>> future?
>> 
>> Chmouel
>> 
> Chmouel,
> 
> I am not really sure what the best solution to configure the containers.  It 
> is clear to me the current shell scripts are fragile in nature and do not 
> handle container restart properly.  The idea of using Puppet or Ansible as a 
> CM tool has been discussed with no resolution.  At the moment, I'm satisified 
> with a somewhat hacky solution if we can get the containers operational.
> 
> Regards,
> -steve
> 
> 
> 
>> 
>> [1] from https://github.com/jlabocki/superhappyfunshow/
>> 
>> 
>> ___
>> OpenStack-dev mailing list
>> OpenStack-dev@lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



signature.asc
Description: Message signed with OpenPGP using GPGMail
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [kolla] Kolla Blueprints

[Fox, Kevin M]

Has anyone figured out a way of having a floating ip like feature with docker 
so that you can have rabbitmq, mysql, or ceph mon's at fixed ip's and be able 
to migrate them around from physical host to physical host and still have them 
at fixed locations that you can easily put in static config files?

Maybe iptables rules? Maybe adding another bridge? Maybe just disabling the 
docker network stack all together and binding the service to a fixed, static 
address on the host?

Also, I ran across: 
http://jperrin.github.io/centos/2014/09/25/centos-docker-and-systemd/ and it 
does seem to work. I was able to get openssh-server and keystone to work in the 
same container without needing to write custom start/stop scripts. This kind of 
setup would make a nova compute container much, much easier.

Thanks,
Kevin

From: Steven Dake [sd...@redhat.com]
Sent: Wednesday, October 01, 2014 8:04 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [kolla] Kolla Blueprints

On 09/30/2014 09:55 AM, Chmouel Boudjnah wrote:

On Tue, Sep 30, 2014 at 6:41 PM, Steven Dake 
mailto:sd...@redhat.com>> wrote:

I've done a first round of prioritization.  I think key things we need people 
to step up for are nova and rabbitmq containers.

For the developers, please take a moment to pick a specific blueprint to work 
on.  If your already working on something, this hsould help to prevent 
duplicate work :)


As I understand in the current implementations[1]  the containers are 
configured with a mix of shell scripts using crudini and other shell command. 
Is it the way to configure the containers? and is a deployment tool like 
Ansible (or others) is something that is planned to be used in the future?

Chmouel

Chmouel,

I am not really sure what the best solution to configure the containers.  It is 
clear to me the current shell scripts are fragile in nature and do not handle 
container restart properly.  The idea of using Puppet or Ansible as a CM tool 
has been discussed with no resolution.  At the moment, I'm satisified with a 
somewhat hacky solution if we can get the containers operational.

Regards,
-steve




[1] from https://github.com/jlabocki/superhappyfunshow/



___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org<mailto:OpenStack-dev@lists.openstack.org>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [kolla] Kolla Blueprints

[Clint Byrum]

Excerpts from Steven Dake's message of 2014-10-01 08:04:38 -0700:
> On 09/30/2014 09:55 AM, Chmouel Boudjnah wrote:
> >
> > On Tue, Sep 30, 2014 at 6:41 PM, Steven Dake  > > wrote:
> >
> >
> > I've done a first round of prioritization.  I think key things we
> > need people to step up for are nova and rabbitmq containers.
> >
> > For the developers, please take a moment to pick a specific
> > blueprint to work on.  If your already working on something, this
> > hsould help to prevent duplicate work :)
> >
> >
> >
> > As I understand in the current implementations[1]  the containers are 
> > configured with a mix of shell scripts using crudini and other shell 
> > command. Is it the way to configure the containers? and is a 
> > deployment tool like Ansible (or others) is something that is planned 
> > to be used in the future?
> >
> > Chmouel
> >
> Chmouel,
> 
> I am not really sure what the best solution to configure the 
> containers.  It is clear to me the current shell scripts are fragile in 
> nature and do not handle container restart properly.  The idea of using 
> Puppet or Ansible as a CM tool has been discussed with no resolution.  
> At the moment, I'm satisified with a somewhat hacky solution if we can 
> get the containers operational.

What if we modified diskimage-builder to create docker containers out
of the existing tripleo-image-elements?

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [kolla] Kolla Blueprints

[Steven Dake]

On 09/30/2014 09:55 AM, Chmouel Boudjnah wrote:


On Tue, Sep 30, 2014 at 6:41 PM, Steven Dake > wrote:



I've done a first round of prioritization.  I think key things we
need people to step up for are nova and rabbitmq containers.

For the developers, please take a moment to pick a specific
blueprint to work on.  If your already working on something, this
hsould help to prevent duplicate work :)



As I understand in the current implementations[1]  the containers are 
configured with a mix of shell scripts using crudini and other shell 
command. Is it the way to configure the containers? and is a 
deployment tool like Ansible (or others) is something that is planned 
to be used in the future?


Chmouel


Chmouel,

I am not really sure what the best solution to configure the 
containers.  It is clear to me the current shell scripts are fragile in 
nature and do not handle container restart properly.  The idea of using 
Puppet or Ansible as a CM tool has been discussed with no resolution.  
At the moment, I'm satisified with a somewhat hacky solution if we can 
get the containers operational.


Regards,
-steve





[1] from https://github.com/jlabocki/superhappyfunshow/


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [kolla] Kolla Blueprints

[Chmouel Boudjnah]

On Tue, Sep 30, 2014 at 6:41 PM, Steven Dake  wrote:

>
> I've done a first round of prioritization.  I think key things we need
> people to step up for are nova and rabbitmq containers.
>
> For the developers, please take a moment to pick a specific blueprint to
> work on.  If your already working on something, this hsould help to prevent
> duplicate work :)



As I understand in the current implementations[1]  the containers are
configured with a mix of shell scripts using crudini and other shell
command. Is it the way to configure the containers? and is a deployment
tool like Ansible (or others) is something that is planned to be used in
the future?

Chmouel


[1] from https://github.com/jlabocki/superhappyfunshow/
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [kolla] Kolla Blueprints

[Steven Dake]

On 09/30/2014 06:32 AM, Ryan Hallisey wrote:

Hi all,

The blueprints have been setup for Kolla: https://blueprints.launchpad.net/kolla

Currently, there are blueprints for all of the openstack services and a few 
supporting services.
They are, nova, swift, cinder, neutron, horizon, keystone, glance, ceilometer, 
heat, trove,
zaqar, sahara, mysql, and rabbitmq.

Feel free to take ownership of a service!

Thanks,
Ryan Hallisey

Ryan,

Thanks.

I've done a first round of prioritization.  I think key things we need 
people to step up for are nova and rabbitmq containers.


For the developers, please take a moment to pick a specific blueprint to 
work on.  If your already working on something, this hsould help to 
prevent duplicate work :)


Thanks
-0steve


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [kolla] Kolla Blueprints

[Ryan Hallisey]

Hi all,

The blueprints have been setup for Kolla: https://blueprints.launchpad.net/kolla

Currently, there are blueprints for all of the openstack services and a few 
supporting services.
They are, nova, swift, cinder, neutron, horizon, keystone, glance, ceilometer, 
heat, trove, 
zaqar, sahara, mysql, and rabbitmq.

Feel free to take ownership of a service!

Thanks,
Ryan Hallisey

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev