subject:"Re\\\: \\\[openstack\\\-dev\\\] \\\[kolla\\\] on Dockerfile patterns"


On 10/14/2014 01:12 PM, Clint Byrum wrote:

Excerpts from Lars Kellogg-Stedman's message of 2014-10-14 12:50:48 -0700:

On Tue, Oct 14, 2014 at 03:25:56PM -0400, Jay Pipes wrote:

I think the above strategy is spot on. Unfortunately, that's not how the
Docker ecosystem works.

I'm not sure I agree here, but again nobody is forcing you to use this
tool.


operating system that the image is built for. I see you didn't respond to my
point that in your openstack-containers environment, you end up with Debian
*and* Fedora images, since you use the official MySQL dockerhub image. And
therefore you will end up needing to know sysadmin specifics (such as how
network interfaces are set up) on multiple operating system distributions.

I missed that part, but ideally you don't *care* about the
distribution in use.  All you care about is the application.  Your
container environment (docker itself, or maybe a higher level
abstraction) sets up networking for you, and away you go.

If you have to perform system administration tasks inside your
containers, my general feeling is that something is wrong.


Speaking as a curmudgeon ops guy from back in the day.. the reason
I choose the OS I do is precisely because it helps me _when something
is wrong_. And the best way an OS can help me is to provide excellent
debugging tools, and otherwise move out of the way.

When something _is_ wrong and I want to attach GDB to mysqld in said
container, I could build a new container with debugging tools installed,
but that may lose the very system state that I'm debugging. So I need to
run things inside the container like apt-get or yum to install GDB.. and
at some point you start to realize that having a whole OS is actually a
good thing even if it means needing to think about a few more things up
front, such as which OS will I use? and what tools do I need installed
in my containers?

What I mean to say is, just grabbing off the shelf has unstated
consequences.
The biggest gain of containers is they are hermetically sealed. They 
turn hundreds of packages (the dependencies and OS files) into 1 
interface with one operation:

Start with a defined variatic environment.

This idea rocks if you can put up with the pain that debugging something 
that is busted is very difficult.   Accessing logs is not tidy and 
debugging in the ways that true experienced folk know how to (with gdb 
attach for example) just isn't possible.


It also requires that you rely on a completely stateless model (until 
persistent storage is implemented in k8s I guess) and completely 
idempotent model.  I really like the idea of time to upgrade, lets roll 
new images across the cluster.  This model is very powerful but comes 
with some pain.


Regards
-steve



Sure, Docker isn't any more limiting than using a VM or bare hardware, but
if you use the official Docker images, it is more limiting, no?

No more so than grabbing a virtual appliance rather than building a
system yourself.

In other words: sure, it's less flexible, but possibly it's faster to
get started, which is especially useful if your primary goal is not
be a database administrator but is actually write an application
that uses a database backend.

I think there are uses cases for both official and customized
images.


In the case of Kolla, we're deploying OpenStack, not just some new
application that uses a database backend. I think the bar is a bit
higher for operations than end-user applications, since it sits below
the abstractions, much closer to the metal.

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [kolla] on Dockerfile patterns

2014-10-15 Thread Vishvananda Ishaya


On Oct 14, 2014, at 1:21 PM, Lars Kellogg-Stedman l...@redhat.com wrote:

 On Tue, Oct 14, 2014 at 04:06:22PM -0400, Jay Pipes wrote:
 I understand that general feeling, but system administration tasks like
 debugging networking issues or determining and grepping log file locations
 or diagnosing packaging issues for OpenStack services or performing database
 logfile maintenance and backups don't just go away because you're using
 containers, right?
 
 They don't go away, but they're not necessarily things that you would
 do inside your container.
 
 Any state (e.g., database tables) that has a lifetime different from
 that of your container should be stored outside of the container
 proper.  In docker, this would be a volume (in a cloud environment,
 this would be something like EBS or a Cinder volume).
 
 Ideally, your container-optimized applications logs to stdout/stderr.
 If you have multiple processes, they each run in a separate container.
 
 Backups take advantage of the data volumes you've associated with your
 container.  E.g., spawning a new container using the docker
 --volumes-from option to access that data for backup purposes.
 
 If you really need to get inside a container for diagnostic purposes,
 then you use something like nsenter, nsinit, or the forthcoming
 docker exec”.

“something like” isn’t good enough here. There must be a standard way
to do this stuff or people will continue to build fat containers with
all of their pet tools inside. This means containers will just be
another incarnation of virtualization.

Vish

 
 
 they very much seem to be developed from the point of view of application
 developers, and not so much from the point of view of operators who need to
 maintain and support those applications.
 
 I think it's entirely accurate to say that they are
 application-centric, much like services such as Heroku, OpenShift,
 etc.
 
 -- 
 Lars Kellogg-Stedman l...@redhat.com | larsks @ {freenode,twitter,github}
 Cloud Engineering / OpenStack  | http://blog.oddbit.com/
 
 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



signature.asc
Description: Message signed with OpenPGP using GPGMail
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [kolla] on Dockerfile patterns

2014-10-15 Thread Vishvananda Ishaya


On Oct 14, 2014, at 1:12 PM, Clint Byrum cl...@fewbar.com wrote:

 Excerpts from Lars Kellogg-Stedman's message of 2014-10-14 12:50:48 -0700:
 On Tue, Oct 14, 2014 at 03:25:56PM -0400, Jay Pipes wrote:
 I think the above strategy is spot on. Unfortunately, that's not how the
 Docker ecosystem works.
 
 I'm not sure I agree here, but again nobody is forcing you to use this
 tool.
 
 operating system that the image is built for. I see you didn't respond to my
 point that in your openstack-containers environment, you end up with Debian
 *and* Fedora images, since you use the official MySQL dockerhub image. And
 therefore you will end up needing to know sysadmin specifics (such as how
 network interfaces are set up) on multiple operating system distributions.
 
 I missed that part, but ideally you don't *care* about the
 distribution in use.  All you care about is the application.  Your
 container environment (docker itself, or maybe a higher level
 abstraction) sets up networking for you, and away you go.
 
 If you have to perform system administration tasks inside your
 containers, my general feeling is that something is wrong.
 
 
 Speaking as a curmudgeon ops guy from back in the day.. the reason
 I choose the OS I do is precisely because it helps me _when something
 is wrong_. And the best way an OS can help me is to provide excellent
 debugging tools, and otherwise move out of the way.
 
 When something _is_ wrong and I want to attach GDB to mysqld in said
 container, I could build a new container with debugging tools installed,
 but that may lose the very system state that I'm debugging. So I need to
 run things inside the container like apt-get or yum to install GDB.. and
 at some point you start to realize that having a whole OS is actually a
 good thing even if it means needing to think about a few more things up
 front, such as which OS will I use? and what tools do I need installed
 in my containers?
 
 What I mean to say is, just grabbing off the shelf has unstated
 consequences.

If this is how people are going to use and think about containers, I would
submit they are a huge waste of time. The performance value they offer is
dramatically outweighed by the flexibilty and existing tooling that exists
for virtual machines. As I state in my blog post[1] if we really want to
get value from containers, we must convert to the single application per
container view. This means having standard ways of doing the above either
on the host machine or in a debugging container that is as easy (or easier)
than the workflow you mention. There are not good ways to do this yet, and
the community hand-waves it away, saying things like, well you could …”.
You could isn’t good enough. The result is that a lot of people that are
using containers today are doing fat containers with a full os.

Vish

[1] 
https://medium.com/@vishvananda/standard-components-not-standard-containers-c30567f23da6
 
 Sure, Docker isn't any more limiting than using a VM or bare hardware, but
 if you use the official Docker images, it is more limiting, no?
 
 No more so than grabbing a virtual appliance rather than building a
 system yourself.  
 
 In other words: sure, it's less flexible, but possibly it's faster to
 get started, which is especially useful if your primary goal is not
 be a database administrator but is actually write an application
 that uses a database backend.
 
 I think there are uses cases for both official and customized
 images.
 
 
 In the case of Kolla, we're deploying OpenStack, not just some new
 application that uses a database backend. I think the bar is a bit
 higher for operations than end-user applications, since it sits below
 the abstractions, much closer to the metal.
 
 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



signature.asc
Description: Message signed with OpenPGP using GPGMail
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [kolla] on Dockerfile patterns

2014-10-15 Thread David Vossel



- Original Message -
 I'm not arguing that everything should be managed by one systemd, I'm just
 saying, for certain types of containers, a single docker container with
 systemd in it might be preferable to trying to slice it unnaturally into
 several containers.
 
 Systemd has invested a lot of time/effort to be able to relaunch failed
 services, support spawning and maintaining unix sockets and services across
 them, etc, that you'd have to push out of and across docker containers. All
 of that can be done, but why reinvent the wheel? Like you said, pacemaker
 can be made to make it all work, but I have yet to see a way to deploy
 pacemaker services anywhere near as easy as systemd+yum makes it. (Thanks be
 to redhat. :)
 
 The answer seems to be, its not dockerish. Thats ok. I just wanted to
 understand the issue for what it is. If there is a really good reason for
 not wanting to do it, or that its just not the way things are done. I've
 had kind of the opposite feeling regarding docker containers. Docker use to
 do very bad things when killing the container. nasty if you wanted your
 database not to go corrupt. killing pid 1 is a bit sketchy then forcing the
 container down after 10 seconds was particularly bad. having something like
 systemd in place allows the database to be notified, then shutdown properly.
 Sure you can script up enough shell to make this work, but you have to do
 some difficult code, over and over again... Docker has gotten better more
 recently but it still makes me a bit nervous using it for statefull things.
 
 As for recovery, systemd can do the recovery too. I'd argue at this point in
 time, I'd expect systemd recovery to probably work better then some custom

yes, systemd can do recovery and that is part of the problem. From my 
perspective
there should be one resource management system. Whether that be pacemaker, 
kubernetes,
or some other distributed system, it doesn't matter.  If you are mixing systemd
with these other external distributed orchestration/management tools you have 
containers
that are silently failing/recovering without the management layer having any 
clue.

centralized recovery. There's one tool responsible for detecting and invoking 
recovery.
Everything else in the system is designed to make that possible.

If we want to put a process in the container to manage multiple services, we'd 
need
the ability to escalate failures to the distributed management tool.  Systemd 
could
work if it was given the ability to act more as a watchdog after starting 
services than
invoke recovery. If systemd could be configured to die (or potentially 
gracefully cleanup
the containers resource's before dieing) whenever a failure is detected, then 
systemd
might make sense. 

I'm approaching this from a system management point of view. Running systemd in 
your
one off container that you're managing manually does not have the same 
drawbacks.
I don't have a vendetta against systemd or anything, I just think it's a step 
backwards
to put systemd in containers. I see little value in having containers become 
lightweight
virtual machines. Containers have much more to offer.

-- Vossel



 shell scripts when it comes to doing the right thing recovering at bring up.
 The other thing is, recovery is not just about pid1 going away. often it
 sticks around and other badness is going on. Its A way to know things are
 bad, but you can't necessarily rely on it to know the container's healty.
 You need more robust checks for that.
 
 Thanks,
 Kevin
 
 
 From: David Vossel [dvos...@redhat.com]
 Sent: Tuesday, October 14, 2014 4:52 PM
 To: OpenStack Development Mailing List (not for usage questions)
 Subject: Re: [openstack-dev] [kolla] on Dockerfile patterns
 
 - Original Message -
  Ok, why are you so down on running systemd in a container?
 
 It goes against the grain.
 
 From a distributed systems view, we gain quite a bit of control by
 maintaining
 one service per container. Containers can be re-organised and re-purposed
 dynamically.
 If we have systemd trying to manage an entire stack of resources within a
 container,
 we lose this control.
 
 From my perspective a containerized application stack needs to be managed
 externally
 by whatever is orchestrating the containers to begin with. When we take a
 step back
 and look at how we actually want to deploy containers, systemd doesn't make
 much sense.
 It actually limits us in the long run.
 
 Also... recovery. Using systemd to manage a stack of resources within a
 single container
 makes it difficult for whatever is externally enforcing the availability of
 that container
 to detect the health of the container.  As it is now, the actual service is
 pid 1 of a
 container. If that service dies, the container dies. If systemd is pid 1,
 there can
 be all kinds of chaos occurring within the container, but the external
 distributed
 orchestration system won't have a clue

Re: [openstack-dev] [kolla] on Dockerfile patterns

2014-10-15 Thread David Vossel



- Original Message -
 On Tue, 2014-10-14 at 19:52 -0400, David Vossel wrote:
  
  - Original Message -
   Ok, why are you so down on running systemd in a container?
  
  It goes against the grain.
  
  From a distributed systems view, we gain quite a bit of control by
  maintaining
  one service per container. Containers can be re-organised and re-purposed
  dynamically.
  If we have systemd trying to manage an entire stack of resources within a
  container,
  we lose this control.
  
  From my perspective a containerized application stack needs to be managed
  externally
  by whatever is orchestrating the containers to begin with. When we take a
  step back
  and look at how we actually want to deploy containers, systemd doesn't make
  much sense.
  It actually limits us in the long run.
  
  Also... recovery. Using systemd to manage a stack of resources within a
  single container
  makes it difficult for whatever is externally enforcing the availability of
  that container
  to detect the health of the container.  As it is now, the actual service is
  pid 1 of a
  container. If that service dies, the container dies. If systemd is pid 1,
  there can
  be all kinds of chaos occurring within the container, but the external
  distributed
  orchestration system won't have a clue (unless it invokes some custom
  health monitoring
  tools within the container itself, which will likely be the case someday.)
 
 I don't really think this is a good argument.  If you're using docker,
 docker is the management and orchestration system for the containers.

no, docker is a local tool for pulling images and launching containers.
Docker is not the distributed resource manager in charge of overseeing
what machines launch what containers and how those containers are linked
together.

 There's no dogmatic answer to the question should you run init in the
 container.

an init daemon might make sense to put in some containers where we have
a tightly coupled resource stack. There could be a use case where it would
make more sense to put these resources in a single container.

I don't think systemd is a good solution for the init daemon though. Systemd
attempts to handle recovery itself as if it has the entire view of the 
system. With containers, the system view exists outside of the containers.
If we put an internal init daemon within the containers, that daemon needs
to escalate internal failures. The easiest way to do this is to
have init die if it encounters a resource failure (init is pid 1, pid 1 exiting
causes container to exit, container exiting gets the attention of whatever
is managing the containers)

 The reason for not running init inside a container managed by docker is
 that you want the template to be thin for ease of orchestration and
 transfer, so you want to share as much as possible with the host.  The
 more junk you put into the container, the fatter and less agile it
 becomes, so you should probably share the init system with the host in
 this paradigm.

I don't think the local init system and containers should have anything
to do with one another.  I said this in a previous reply, I'm approaching
this problem from a distributed management perspective. The host's
init daemon only has a local view of the world. 

 
 Conversely, containers can be used to virtualize full operating systems.
 This isn't the standard way of doing docker, but LXC and OpenVZ by
 default do containers this way.  For this type of container, because you
 have a full OS running inside the container, you have to also have
 systemd (assuming it's the init system) running within the container.

sure, if you want to do this use systemd. I don't understand the use case
where this makes any sense though. For me this falls in the yeah you can do it,
but why? category.

-- Vossel

 
 James
 
 
 
 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [kolla] on Dockerfile patterns

2014-10-15 Thread Lars Kellogg-Stedman

On Wed, Oct 15, 2014 at 07:52:56AM -0700, Vishvananda Ishaya wrote:
 There must be a standard way
 to do this stuff or people will continue to build fat containers with
 all of their pet tools inside. This means containers will just be
 another incarnation of virtualization.

I wouldn't spend time worrying about that.  docker exec will be the
standard way as soon as it lands in a release version, which I think
will be happening imminently with 1.3.

-- 
Lars Kellogg-Stedman l...@redhat.com | larsks @ {freenode,twitter,github}
Cloud Engineering / OpenStack  | http://blog.oddbit.com/



pgp6GOmzu0krD.pgp
Description: PGP signature
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [kolla] on Dockerfile patterns

2014-10-15 Thread Clint Byrum

Excerpts from Vishvananda Ishaya's message of 2014-10-15 07:52:34 -0700:
 
 On Oct 14, 2014, at 1:12 PM, Clint Byrum cl...@fewbar.com wrote:
 
  Excerpts from Lars Kellogg-Stedman's message of 2014-10-14 12:50:48 -0700:
  On Tue, Oct 14, 2014 at 03:25:56PM -0400, Jay Pipes wrote:
  I think the above strategy is spot on. Unfortunately, that's not how the
  Docker ecosystem works.
  
  I'm not sure I agree here, but again nobody is forcing you to use this
  tool.
  
  operating system that the image is built for. I see you didn't respond to 
  my
  point that in your openstack-containers environment, you end up with 
  Debian
  *and* Fedora images, since you use the official MySQL dockerhub image. 
  And
  therefore you will end up needing to know sysadmin specifics (such as how
  network interfaces are set up) on multiple operating system distributions.
  
  I missed that part, but ideally you don't *care* about the
  distribution in use.  All you care about is the application.  Your
  container environment (docker itself, or maybe a higher level
  abstraction) sets up networking for you, and away you go.
  
  If you have to perform system administration tasks inside your
  containers, my general feeling is that something is wrong.
  
  
  Speaking as a curmudgeon ops guy from back in the day.. the reason
  I choose the OS I do is precisely because it helps me _when something
  is wrong_. And the best way an OS can help me is to provide excellent
  debugging tools, and otherwise move out of the way.
  
  When something _is_ wrong and I want to attach GDB to mysqld in said
  container, I could build a new container with debugging tools installed,
  but that may lose the very system state that I'm debugging. So I need to
  run things inside the container like apt-get or yum to install GDB.. and
  at some point you start to realize that having a whole OS is actually a
  good thing even if it means needing to think about a few more things up
  front, such as which OS will I use? and what tools do I need installed
  in my containers?
  
  What I mean to say is, just grabbing off the shelf has unstated
  consequences.
 
 If this is how people are going to use and think about containers, I would
 submit they are a huge waste of time. The performance value they offer is
 dramatically outweighed by the flexibilty and existing tooling that exists
 for virtual machines. As I state in my blog post[1] if we really want to
 get value from containers, we must convert to the single application per
 container view. This means having standard ways of doing the above either
 on the host machine or in a debugging container that is as easy (or easier)
 than the workflow you mention. There are not good ways to do this yet, and
 the community hand-waves it away, saying things like, well you could …”.
 You could isn’t good enough. The result is that a lot of people that are
 using containers today are doing fat containers with a full os.
 

I think we really agree.

What the container universe hasn't worked out is all the stuff that the
distros have worked out for a long time now: consistency.

I think it would be a good idea for containers' filesystem contents to
be a whole distro. What's at question in this thread is what should be
running. If we can just chroot into the container's FS and run apt-get/yum
install our tools, and then nsenter and attach to the running process,
then huzzah: I think we have best of both worlds.

To the container makers: consider that things can and will go wrong,
and the answer may already exist as a traditional tool, and not be
restart the container.

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [kolla] on Dockerfile patterns

2014-10-15 Thread David Vossel



- Original Message -
 Excerpts from Vishvananda Ishaya's message of 2014-10-15 07:52:34 -0700:
  
  On Oct 14, 2014, at 1:12 PM, Clint Byrum cl...@fewbar.com wrote:
  
   Excerpts from Lars Kellogg-Stedman's message of 2014-10-14 12:50:48
   -0700:
   On Tue, Oct 14, 2014 at 03:25:56PM -0400, Jay Pipes wrote:
   I think the above strategy is spot on. Unfortunately, that's not how
   the
   Docker ecosystem works.
   
   I'm not sure I agree here, but again nobody is forcing you to use this
   tool.
   
   operating system that the image is built for. I see you didn't respond
   to my
   point that in your openstack-containers environment, you end up with
   Debian
   *and* Fedora images, since you use the official MySQL dockerhub
   image. And
   therefore you will end up needing to know sysadmin specifics (such as
   how
   network interfaces are set up) on multiple operating system
   distributions.
   
   I missed that part, but ideally you don't *care* about the
   distribution in use.  All you care about is the application.  Your
   container environment (docker itself, or maybe a higher level
   abstraction) sets up networking for you, and away you go.
   
   If you have to perform system administration tasks inside your
   containers, my general feeling is that something is wrong.
   
   
   Speaking as a curmudgeon ops guy from back in the day.. the reason
   I choose the OS I do is precisely because it helps me _when something
   is wrong_. And the best way an OS can help me is to provide excellent
   debugging tools, and otherwise move out of the way.
   
   When something _is_ wrong and I want to attach GDB to mysqld in said
   container, I could build a new container with debugging tools installed,
   but that may lose the very system state that I'm debugging. So I need to
   run things inside the container like apt-get or yum to install GDB.. and
   at some point you start to realize that having a whole OS is actually a
   good thing even if it means needing to think about a few more things up
   front, such as which OS will I use? and what tools do I need installed
   in my containers?
   
   What I mean to say is, just grabbing off the shelf has unstated
   consequences.
  
  If this is how people are going to use and think about containers, I would
  submit they are a huge waste of time. The performance value they offer is
  dramatically outweighed by the flexibilty and existing tooling that exists
  for virtual machines. As I state in my blog post[1] if we really want to
  get value from containers, we must convert to the single application per
  container view. This means having standard ways of doing the above either
  on the host machine or in a debugging container that is as easy (or easier)
  than the workflow you mention. There are not good ways to do this yet, and
  the community hand-waves it away, saying things like, well you could …”.
  You could isn’t good enough. The result is that a lot of people that are
  using containers today are doing fat containers with a full os.
  
 
 I think we really agree.
 
 What the container universe hasn't worked out is all the stuff that the
 distros have worked out for a long time now: consistency.

I agree we need consistency. I have an idea. What if we developed an entrypoint
script standard...

Something like LSB init scripts except tailored towards the container use case.
The primary difference would be that the 'start' action of this new standard
wouldn't fork. Instead 'start' would be pid 1. The 'status' could be checked
externally by calling the exact same entry point script to invoke the 'status'
function.

This standard would lock us into the 'one service per container' concept while
giving us the ability to standardize on how the container is launched and 
monitored.

If we all conformed to something like this, docker could even extend the 
standard
so health checks could be performed using the docker cli tool.

docker status container id

Internally docker would just be doing a nsenter into the container and calling
the internal status function in our init script standard.

We already have docker start container and docker stop container. Being able
to generically call something like docker status container and have that 
translate
into some service specific command on the inside of the container would be kind 
of
neat.

Tools like kubernetes could use this functionality to poll a container's health 
and
be able to detect issues occurring within the container that don't necessarily
involve the container's service failing.

Does anyone else have any interest in this? I have quite a bit of of init 
script type
standard experience. It would be trivial for me to define something like this 
for us
to begin discussing.

-- Vossel

 I think it would be a good idea for containers' filesystem contents to
 be a whole distro. What's at question in this thread is what should be
 running. If we can just chroot into the container's FS

Re: [openstack-dev] [kolla] on Dockerfile patterns

2014-10-15 Thread Lars Kellogg-Stedman

On Wed, Oct 15, 2014 at 01:50:08PM -0400, David Vossel wrote:
 Something like LSB init scripts except tailored towards the container use 
 case.
 The primary difference would be that the 'start' action of this new standard
 wouldn't fork. Instead 'start' would be pid 1. The 'status' could be checked
 externally by calling the exact same entry point script to invoke the 'status'
 function.

With the 1.3 release, which introduces docker exec, you could just
about get there.  Rather than attempting to introspect the container
to find the entrypoint script -- which might not even exist -- I would
say standardize on some top level paths (e.g., '/status') that can be
used to run a status check, and leave the implementation of those
paths up to the image (maybe they're scripts, maybe they're binaries,
just as long as they are executable).

Then you check would boil down to:

  docker exec container id /status

The reason why I am trying to avoid assuming some specially
constructed entrypoint script is that many images will simply not have
one -- they simply provide an initial command via CMD. Adding a
/status script or similar in this case is very simple:

   FROM original_image
   ADD status /status

Doing this via an entrypoint script would be a bit more complicated:

- You would have to determine whether or not the original image had an
  existing entrypoint script.
- If so you would need to wrap it or replicate the functionality.
- Some images may have entrypoint scripts that already provide
  subcommand like functionality (docker run someimage keyword,
  where keyword is parsed by the entrypoint script) and might not be
  compatible with an entrypoint-based status check.

Otherwise, I think establishing a best practice mechanism for executing
in-container checks is a great idea.

-- 
Lars Kellogg-Stedman l...@redhat.com | larsks @ {freenode,twitter,github}
Cloud Engineering / OpenStack  | http://blog.oddbit.com/



pgpD1P2Vb3KMF.pgp
Description: PGP signature
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [kolla] on Dockerfile patterns


On 10/14/2014 04:52 PM, David Vossel wrote:


- Original Message -

Ok, why are you so down on running systemd in a container?

It goes against the grain.

 From a distributed systems view, we gain quite a bit of control by maintaining
one service per container. Containers can be re-organised and re-purposed 
dynamically.
If we have systemd trying to manage an entire stack of resources within a 
container,
we lose this control.

 From my perspective a containerized application stack needs to be managed 
externally
by whatever is orchestrating the containers to begin with. When we take a step 
back
and look at how we actually want to deploy containers, systemd doesn't make 
much sense.
It actually limits us in the long run.

Also... recovery. Using systemd to manage a stack of resources within a single 
container
makes it difficult for whatever is externally enforcing the availability of 
that container
to detect the health of the container.  As it is now, the actual service is pid 
1 of a
container. If that service dies, the container dies. If systemd is pid 1, there 
can
be all kinds of chaos occurring within the container, but the external 
distributed
orchestration system won't have a clue (unless it invokes some custom health 
monitoring
tools within the container itself, which will likely be the case someday.)
I tend to agree systemd makes healthchecking and recovery escalation 
more difficult/impossible.  At a minimum to do escalation with systemd, 
the external orch system (k8s) needs to run code in the container to 
healthcheck the services.  This can be done today with k8s, but I fail 
to see why it is necessary to involve the complexity of systemd.  The 
systemd system is pretty sweet for multiple processes in one container, 
but systemd doesn't meld well with the one application per container 
model that appears to be best practice at this time.


Regards,
-steve


-- Vossel



Pacemaker works, but its kind of a pain to setup compared just yum installing
a few packages and setting init to systemd. There are some benefits for
sure, but if you have to force all the docker components onto the same
physical machine anyway, why bother with the extra complexity?

Thanks,
Kevin


From: David Vossel [dvos...@redhat.com]
Sent: Tuesday, October 14, 2014 3:14 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [kolla] on Dockerfile patterns

- Original Message -

Same thing works with cloud init too...


I've been waiting on systemd working inside a container for a while. it
seems
to work now.

oh no...


The idea being its hard to write a shell script to get everything up and
running with all the interactions that may need to happen. The init
system's
already designed for that. Take a nova-compute docker container for
example,
you probably need nova-compute, libvirt, neutron-openvswitch-agent, and the
celiometer-agent all backed in. Writing a shell script to get it all
started
and shut down properly would be really ugly.

You could split it up into 4 containers and try and ensure they are
coscheduled and all the pieces are able to talk to each other, but why?
Putting them all in one container with systemd starting the subprocesses is
much easier and shouldn't have many drawbacks. The components code is
designed and tested assuming the pieces are all together.

What you need is a dependency model that is enforced outside of the
containers. Something
that manages the order containers are started/stopped/recovered in. This
allows
you to isolate your containers with 1 service per container, yet still
express that
container with service A needs to start before container with service B.

Pacemaker does this easily. There's even a docker resource-agent for
Pacemaker now.
https://github.com/ClusterLabs/resource-agents/blob/master/heartbeat/docker

-- Vossel

ps. don't run systemd in a container... If you think you should, talk to me
first.


You can even add a ssh server in there easily too and then ansible in to do
whatever other stuff you want to do to the container like add other
monitoring and such

Ansible or puppet or whatever should work better in this arrangement too
since existing code assumes you can just systemctl start foo;

Kevin

From: Lars Kellogg-Stedman [l...@redhat.com]
Sent: Tuesday, October 14, 2014 12:10 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [kolla] on Dockerfile patterns

On Tue, Oct 14, 2014 at 02:45:30PM -0400, Jay Pipes wrote:

With Docker, you are limited to the operating system of whatever the
image
uses.

See, that's the part I disagree with.  What I was saying about ansible
and puppet in my email is that I think the right thing to do is take
advantage of those tools:

   FROM ubuntu

   RUN apt-get install ansible
   COPY my_ansible_config.yaml /my_ansible_config.yaml
   RUN ansible

Re: [openstack-dev] [kolla] on Dockerfile patterns

On 10/14/2014 05:44 PM, Angus Lees wrote:

On Tue, 14 Oct 2014 07:51:54 AM Steven Dake wrote:

Angus,

On 10/13/2014 08:51 PM, Angus Lees wrote:

I've been reading a bunch of the existing Dockerfiles, and I have two
humble requests:

1. It would be good if the interesting code came from python
sdist/bdists
rather than rpms.

This will make it possible to rebuild the containers using code from a
private branch or even unsubmitted code, without having to go through a
redhat/rpm release process first.

I care much less about where the python dependencies come from. Pulling
them from rpms rather than pip/pypi seems like a very good idea, given
the relative difficulty of caching pypi content and we also pull in the
required C, etc libraries for free.

With this in place, I think I could drop my own containers and switch to
reusing kolla's for building virtual testing environments. This would
make me happy.

I've captured this requirement here:
https://blueprints.launchpad.net/kolla/+spec/run-from-master

I also believe it would be interesting to run from master or a stable
branch for CD. Unfortunately I'm still working on the nova-compute
docker code, but if someone comes along and picks up that blueprint, i
expect it will get implemented :) Maybe that could be you.

Yeah I've already got a bunch of working containers that pull from master[1],
but I've been thinking I should change that to use an externally supplied
bdist. The downside is you quickly end up wanting a docker container to build
your deployment docker container. I gather this is quite a common thing to
do, but I haven't found the time to script it up yet.

[1] https://github.com/anguslees/kube-openstack/tree/master/docker

I could indeed work on this, and I guess I was gauging the level of enthusiasm
within kolla for such a change. I don't want to take time away from the
alternative I have that already does what I need only to push uphill to get it
integrated :/
There would be no uphill push. For milestone #2, I am already going to
reorganize the docker directory to support centos+rdo as an alternative
to fedora+rdo. Fedora+master is just another directory in this model
(or Ubuntu + master if you want that choice as well). IMO the more
choice about deployment platforms the better, especially a master model
(or more likely a stable branch model).

Regards
-steve

2. I think we should separate out run the server from do once-off
setup.

Currently the containers run a start.sh that typically sets up the
database, runs the servers, creates keystone users and sets up the
keystone catalog. In something like k8s, the container will almost
certainly be run multiple times in parallel and restarted numerous times,
so all those other steps go against the service-oriented k8s ideal and
are at-best wasted.

The server may not start before the configuration of the server is
complete. I guess I don't quite understand what you indicate here when
you say we have separate pods that do initial database setup/migrate.
Do you mean have dependencies in some way, or for eg:

glance-registry-setup-pod.yaml - the glance registry pod descriptor
which sets up the db and keystone
glance-registry-pod.yaml - the glance registry pod descriptor which
starts the application and waits for db/keystone setup

and start these two pods as part of the same selector (glance-registry)?

That idea sounds pretty appealing although probably won't be ready to go
for milestone #1.

So the way I do it now, I have a replicationController that starts/manages
(eg) nova-api pods[2]. I separately have a nova-db-sync pod[3] that basically
just runs nova-manage db sync.

I then have a simple shell script[4] that starts them all at the same time.
The nova-api pods crash and get restarted a few times until the database has
been appropriately configured by the nova-db-sync pod, and then they're fine and
start serving.

When nova-db-sync exits successfully, the pod just sits in state terminated
thanks to restartPolicy: onFailure. Sometime later I can delete the
terminated nova-db-sync pod, but it's also harmless if I just leave it or even
if it gets occasionally re-run as part of some sort of update.

[2]
https://github.com/anguslees/kube-openstack/blob/master/kubernetes-in/nova-api-repcon.yaml
[3]
https://github.com/anguslees/kube-openstack/blob/master/kubernetes-in/nova-db-sync-pod.yaml
[4] https://github.com/anguslees/kube-openstack/blob/master/kubecfg-create.sh

I'm open to whether we want to make these as lightweight/independent as
possible (every daemon in an individual container), or limit it to one per
project (eg: run nova-api, nova-conductor,

Re: [openstack-dev] [kolla] on Dockerfile patterns


On 10/14/2014 06:10 PM, Clint Byrum wrote:

Excerpts from Fox, Kevin M's message of 2014-10-14 17:40:16 -0700:

I'm not arguing that everything should be managed by one systemd, I'm
just saying, for certain types of containers, a single docker container
with systemd in it might be preferable to trying to slice it unnaturally
into several containers.


Can you be more concrete? Most of the time things that need to be in
the same machine tend to have some kind of controller already. Meanwhile
it is worth noting that you can have one _image_, but several containers
running from that one image. So if you're trying to run a few pieces of
Neutron, for instance, you can have multiple containers each from that
one neutron image.


Systemd has invested a lot of time/effort to be able to relaunch failed
services, support spawning and maintaining unix sockets and services
across them, etc, that you'd have to push out of and across docker
containers. All of that can be done, but why reinvent the wheel? Like you
said, pacemaker can be made to make it all work, but I have yet to see
a way to deploy pacemaker services anywhere near as easy as systemd+yum
makes it. (Thanks be to redhat. :)


There are some of us who are rather annoyed that systemd tries to do
this in such a naive way and assumes everyone will want that kind of
management. It's the same naiveté that leads people to think if they
make their app server systemd service depend on their mysql systemd
service that this will eliminate startup problems. Once you have more
than one server, it doesn't work.

Kubernetes adds a distributed awareness of the containers that makes it
uniquely positioned to do most of those jobs much better than systemd
can.


The answer seems to be, its not dockerish. Thats ok. I just wanted to
understand the issue for what it is. If there is a really good reason for
not wanting to do it, or that its just not the way things are done. I've
had kind of the opposite feeling regarding docker containers. Docker use
to do very bad things when killing the container. nasty if you wanted
your database not to go corrupt. killing pid 1 is a bit sketchy then
forcing the container down after 10 seconds was particularly bad. having
something like systemd in place allows the database to be notified, then
shutdown properly. Sure you can script up enough shell to make this work,
but you have to do some difficult code, over and over again... Docker
has gotten better more recently but it still makes me a bit nervous
using it for statefull things.


What I think David was saying was that the process you want to run under
systemd is the pid 1 of the container. So if killing that would be bad,
it would also be bad to stop the systemd service, which would do the
same thing: send it SIGTERM. If that causes all hell to break loose, the
stateful thing isn't worth a dime, because it isn't crash safe.


As for recovery, systemd can do the recovery too. I'd argue at this
point in time, I'd expect systemd recovery to probably work better
then some custom shell scripts when it comes to doing the right thing
recovering at bring up. The other thing is, recovery is not just about
pid1 going away. often it sticks around and other badness is going
on. Its A way to know things are bad, but you can't necessarily rely on
it to know the container's healty. You need more robust checks for that.

I think one thing people like about Kubernetes is that when a container
crashes, and needs to be brought back up, it may actually be brought
up on a different, less busy, more healthy host. I could be wrong, or
that might be in the FUTURE section. But the point is, recovery and
start-up are not things that always want to happen on the same box.
yes FUTURE as  t his doesn't exist today.  There is a significant runway 
to enhancing the availability management of kubernetes natively.


Regards
-steve


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [kolla] on Dockerfile patterns

2014-10-15 Thread James Bottomley

On Wed, 2014-10-15 at 11:24 -0400, David Vossel wrote:
 
 - Original Message -
  On Tue, 2014-10-14 at 19:52 -0400, David Vossel wrote:
   
   - Original Message -
Ok, why are you so down on running systemd in a container?
   
   It goes against the grain.
   
   From a distributed systems view, we gain quite a bit of control by
   maintaining
   one service per container. Containers can be re-organised and 
   re-purposed
   dynamically.
   If we have systemd trying to manage an entire stack of resources within a
   container,
   we lose this control.
   
   From my perspective a containerized application stack needs to be managed
   externally
   by whatever is orchestrating the containers to begin with. When we take a
   step back
   and look at how we actually want to deploy containers, systemd doesn't 
   make
   much sense.
   It actually limits us in the long run.
   
   Also... recovery. Using systemd to manage a stack of resources within a
   single container
   makes it difficult for whatever is externally enforcing the availability 
   of
   that container
   to detect the health of the container.  As it is now, the actual service 
   is
   pid 1 of a
   container. If that service dies, the container dies. If systemd is pid 1,
   there can
   be all kinds of chaos occurring within the container, but the external
   distributed
   orchestration system won't have a clue (unless it invokes some custom
   health monitoring
   tools within the container itself, which will likely be the case someday.)
  
  I don't really think this is a good argument.  If you're using docker,
  docker is the management and orchestration system for the containers.
 
 no, docker is a local tool for pulling images and launching containers.
 Docker is not the distributed resource manager in charge of overseeing
 what machines launch what containers and how those containers are linked
 together.

Well, neither is systemd: fleet management has a variety of solution.

  There's no dogmatic answer to the question should you run init in the
  container.
 
 an init daemon might make sense to put in some containers where we have
 a tightly coupled resource stack. There could be a use case where it would
 make more sense to put these resources in a single container.
 
 I don't think systemd is a good solution for the init daemon though. Systemd
 attempts to handle recovery itself as if it has the entire view of the 
 system. With containers, the system view exists outside of the containers.
 If we put an internal init daemon within the containers, that daemon needs
 to escalate internal failures. The easiest way to do this is to
 have init die if it encounters a resource failure (init is pid 1, pid 1 
 exiting
 causes container to exit, container exiting gets the attention of whatever
 is managing the containers)

I won't comment on what init should be.  However, init should not be
running in application containers, as I have said because it complicates
the situation.  Application containers are more compelling the simpler
they are constructed because they're easier to describe in xml +
templates.

  The reason for not running init inside a container managed by docker is
  that you want the template to be thin for ease of orchestration and
  transfer, so you want to share as much as possible with the host.  The
  more junk you put into the container, the fatter and less agile it
  becomes, so you should probably share the init system with the host in
  this paradigm.
 
 I don't think the local init system and containers should have anything
 to do with one another.  I said this in a previous reply, I'm approaching
 this problem from a distributed management perspective. The host's
 init daemon only has a local view of the world. 

If the container is an OS container, what you run inside is a full OS
stack; the only sharing is the kernel, so you get whatever the distro is
using as init and for some of them, that's systemd.  You have no choice
for OS containers.

  
  Conversely, containers can be used to virtualize full operating systems.
  This isn't the standard way of doing docker, but LXC and OpenVZ by
  default do containers this way.  For this type of container, because you
  have a full OS running inside the container, you have to also have
  systemd (assuming it's the init system) running within the container.
 
 sure, if you want to do this use systemd. I don't understand the use case
 where this makes any sense though. For me this falls in the yeah you can do 
 it,
 but why? category.

It's the standard Service Provider use case: containers are used as
dense, lightweight Virtual Environments for Virtual Private Servers.
The customer can be provisioned with whatever OS they like, but you
still get 3x the density and the huge elasticity improvements of
containers.

James



___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org

Re: [openstack-dev] [kolla] on Dockerfile patterns

2014-10-15 Thread Angus Lees

On Wed, 15 Oct 2014 09:51:03 AM Clint Byrum wrote:
 Excerpts from Vishvananda Ishaya's message of 2014-10-15 07:52:34 -0700:
  On Oct 14, 2014, at 1:12 PM, Clint Byrum cl...@fewbar.com wrote:
   Excerpts from Lars Kellogg-Stedman's message of 2014-10-14 12:50:48 
-0700:
   On Tue, Oct 14, 2014 at 03:25:56PM -0400, Jay Pipes wrote:
   I think the above strategy is spot on. Unfortunately, that's not how
   the
   Docker ecosystem works.
   
   I'm not sure I agree here, but again nobody is forcing you to use this
   tool.
   
   operating system that the image is built for. I see you didn't respond
   to my point that in your openstack-containers environment, you end up
   with Debian *and* Fedora images, since you use the official MySQL
   dockerhub image. And therefore you will end up needing to know
   sysadmin specifics (such as how network interfaces are set up) on
   multiple operating system distributions.  
   I missed that part, but ideally you don't *care* about the
   distribution in use.  All you care about is the application.  Your
   container environment (docker itself, or maybe a higher level
   abstraction) sets up networking for you, and away you go.
   
   If you have to perform system administration tasks inside your
   containers, my general feeling is that something is wrong.
   
   Speaking as a curmudgeon ops guy from back in the day.. the reason
   I choose the OS I do is precisely because it helps me _when something
   is wrong_. And the best way an OS can help me is to provide excellent
   debugging tools, and otherwise move out of the way.
   
   When something _is_ wrong and I want to attach GDB to mysqld in said
   container, I could build a new container with debugging tools installed,
   but that may lose the very system state that I'm debugging. So I need to
   run things inside the container like apt-get or yum to install GDB.. and
   at some point you start to realize that having a whole OS is actually a
   good thing even if it means needing to think about a few more things up
   front, such as which OS will I use? and what tools do I need
   installed
   in my containers?
   
   What I mean to say is, just grabbing off the shelf has unstated
   consequences.
  
  If this is how people are going to use and think about containers, I would
  submit they are a huge waste of time. The performance value they offer is
  dramatically outweighed by the flexibilty and existing tooling that exists
  for virtual machines. As I state in my blog post[1] if we really want to
  get value from containers, we must convert to the single application per
  container view. This means having standard ways of doing the above either
  on the host machine or in a debugging container that is as easy (or
  easier)
  than the workflow you mention. There are not good ways to do this yet, and
  the community hand-waves it away, saying things like, well you could …”.
  You could isn’t good enough. The result is that a lot of people that are
  using containers today are doing fat containers with a full os.
 
 I think we really agree.
 
 What the container universe hasn't worked out is all the stuff that the
 distros have worked out for a long time now: consistency.
 
 I think it would be a good idea for containers' filesystem contents to
 be a whole distro. What's at question in this thread is what should be
 running. If we can just chroot into the container's FS and run apt-get/yum
 install our tools, and then nsenter and attach to the running process,
 then huzzah: I think we have best of both worlds.

Erm, yes that's exactly what you can do with containers (docker, lxc, and 
presumably any other use of containers with a private/ephemeral filesystem).

To others here:

There's a lot of strongly-held opinions being expressed on this thread, and a 
number of them appear to be based on misinformation or a lack of understanding 
of the technology and goals.  I'm happy to talk over IRC/VC/whatever to anyone 
about why I think this sort of stuff is worth pursuing (and I assume there are 
plenty of others too).  I'd also suggest reading docs and/or in-person at your 
local docker/devops meetup would be a more efficient method of learning rather 
than this to-and-fro on the os-dev mailing list...

 To the container makers: consider that things can and will go wrong,
 and the answer may already exist as a traditional tool, and not be
 restart the container.
 
 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

-- 
 - Gus

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [kolla] on Dockerfile patterns

2014-10-15 Thread Angus Lees

On Wed, 15 Oct 2014 12:40:16 AM Fox, Kevin M wrote:
 Systemd has invested a lot of time/effort to be able to relaunch failed
 services, support spawning and maintaining unix sockets and services across
 them, etc, that you'd have to push out of and across docker containers. All
 of that can be done, but why reinvent the wheel? Like you said, pacemaker
 can be made to make it all work, but I have yet to see a way to deploy
 pacemaker services anywhere near as easy as systemd+yum makes it. (Thanks
 be to redhat. :)

You should also consider fleet, if you want a systemd-approach to containers. 
 
It's basically a cluster-wide systemd that often (but doesn't have to) 
start/restart docker containers on various hosts.

I tried it for a short while and it isn't bad.  The verbosity and 
repetitiveness of the systemd files was a bit annoying, but that would be easy 
to script away.  I did like how simple it was, and the ability to express 
dependencies between systemd entities.

Note that fleet is essentially systemd managing containers from the outside - 
not running systemd inside the container.  So in many ways it's a 
repeat/reinforcement of the same conversation we're already having.

-- 
 - Gus

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [kolla] on Dockerfile patterns

2014-10-15 Thread Clint Byrum

Excerpts from Angus Lees's message of 2014-10-15 17:30:52 -0700:
 On Wed, 15 Oct 2014 09:51:03 AM Clint Byrum wrote:
  Excerpts from Vishvananda Ishaya's message of 2014-10-15 07:52:34 -0700:
   On Oct 14, 2014, at 1:12 PM, Clint Byrum cl...@fewbar.com wrote:
Excerpts from Lars Kellogg-Stedman's message of 2014-10-14 12:50:48 
 -0700:
On Tue, Oct 14, 2014 at 03:25:56PM -0400, Jay Pipes wrote:
I think the above strategy is spot on. Unfortunately, that's not how
the
Docker ecosystem works.

I'm not sure I agree here, but again nobody is forcing you to use this
tool.

operating system that the image is built for. I see you didn't respond
to my point that in your openstack-containers environment, you end up
with Debian *and* Fedora images, since you use the official MySQL
dockerhub image. And therefore you will end up needing to know
sysadmin specifics (such as how network interfaces are set up) on
multiple operating system distributions.  
I missed that part, but ideally you don't *care* about the
distribution in use.  All you care about is the application.  Your
container environment (docker itself, or maybe a higher level
abstraction) sets up networking for you, and away you go.

If you have to perform system administration tasks inside your
containers, my general feeling is that something is wrong.

Speaking as a curmudgeon ops guy from back in the day.. the reason
I choose the OS I do is precisely because it helps me _when something
is wrong_. And the best way an OS can help me is to provide excellent
debugging tools, and otherwise move out of the way.

When something _is_ wrong and I want to attach GDB to mysqld in said
container, I could build a new container with debugging tools installed,
but that may lose the very system state that I'm debugging. So I need to
run things inside the container like apt-get or yum to install GDB.. and
at some point you start to realize that having a whole OS is actually a
good thing even if it means needing to think about a few more things up
front, such as which OS will I use? and what tools do I need
installed
in my containers?

What I mean to say is, just grabbing off the shelf has unstated
consequences.
   
   If this is how people are going to use and think about containers, I would
   submit they are a huge waste of time. The performance value they offer is
   dramatically outweighed by the flexibilty and existing tooling that exists
   for virtual machines. As I state in my blog post[1] if we really want to
   get value from containers, we must convert to the single application per
   container view. This means having standard ways of doing the above either
   on the host machine or in a debugging container that is as easy (or
   easier)
   than the workflow you mention. There are not good ways to do this yet, and
   the community hand-waves it away, saying things like, well you could …”.
   You could isn’t good enough. The result is that a lot of people that are
   using containers today are doing fat containers with a full os.
  
  I think we really agree.
  
  What the container universe hasn't worked out is all the stuff that the
  distros have worked out for a long time now: consistency.
  
  I think it would be a good idea for containers' filesystem contents to
  be a whole distro. What's at question in this thread is what should be
  running. If we can just chroot into the container's FS and run apt-get/yum
  install our tools, and then nsenter and attach to the running process,
  then huzzah: I think we have best of both worlds.
 
 Erm, yes that's exactly what you can do with containers (docker, lxc, and 
 presumably any other use of containers with a private/ephemeral filesystem).
 

The point I was trying to make is that this case was not being addressed
by the don't run init in a container crowd. I am in that crowd, and
thus, wanted to make the point: this is how I think it will work when
I do finally get around to trying containers for a real workload.

 To others here:
 
 There's a lot of strongly-held opinions being expressed on this thread, and a 
 number of them appear to be based on misinformation or a lack of 
 understanding 
 of the technology and goals.  I'm happy to talk over IRC/VC/whatever to 
 anyone 
 about why I think this sort of stuff is worth pursuing (and I assume there 
 are 
 plenty of others too).  I'd also suggest reading docs and/or in-person at 
 your 
 local docker/devops meetup would be a more efficient method of learning 
 rather 
 than this to-and-fro on the os-dev mailing list...
 

I think you may have been mistaken about the purpose of this list. It is
for openstack developers to discuss exactly the things you're talking
about not discussing on this list. You've done a great job of helping us
clear these things up. Don't turn around and tell us now don't ever do

Re: [openstack-dev] [kolla] on Dockerfile patterns

2014-10-14 Thread Chris Dent


On Tue, 14 Oct 2014, Angus Lees wrote:


2. I think we should separate out run the server from do once-off setup.


Yes! Otherwise it feels like the entire point of using containers
and dockerfiles is rather lost.

--
Chris Dent tw:@anticdent freenode:cdent
https://tank.peermore.com/tanks/cdent

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [kolla] on Dockerfile patterns

On Tue, Oct 14, 2014 at 02:51:15PM +1100, Angus Lees wrote:
 1. It would be good if the interesting code came from python sdist/bdists 
 rather than rpms.

I agree in principal, although starting from packages right now lets
us ignore a whole host of issues.  Possibly we'll hit that change down
the road.

 2. I think we should separate out run the server from do once-off setup.
 
 Currently the containers run a start.sh that typically sets up the database, 
 runs the servers, creates keystone users and sets up the keystone catalog.  
 In 
 something like k8s, the container will almost certainly be run multiple times 
 in parallel and restarted numerous times, so all those other steps go against 
 the service-oriented k8s ideal and are at-best wasted.

All the existing containers [*] are designed to be idempotent, which I
think is not a bad model.  Even if we move initial configuration out
of the service containers I think that is a goal we want to preserve.

I pursued exactly the model you suggest on my own when working on an
ansible-driven workflow for setting things up:

  https://github.com/larsks/openstack-containers

Ansible made it easy to support one-off batch containers which, as
you say, aren't exactly supported in Kubernetes.  I like your
(ab?)use of restartPolicy; I think that's worth pursuing.

[*] That work, which includes rabbitmq, mariadb, keystone, and glance.

 I'm open to whether we want to make these as lightweight/independent as 
 possible (every daemon in an individual container), or limit it to one per 
 project (eg: run nova-api, nova-conductor, nova-scheduler, etc all in one 
 container).

My goal is one-service-per-container, because that generally makes the
question of process supervision and log collection a *host* problem
rather than a *container* problem. It also makes it easier to scale an
individual service, if that becomes necessary.

-- 
Lars Kellogg-Stedman l...@redhat.com | larsks @ {freenode,twitter,github}
Cloud Engineering / OpenStack  | http://blog.oddbit.com/



pgp5UySJo2RQn.pgp
Description: PGP signature
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [kolla] on Dockerfile patterns

2014-10-14 Thread Steven Dake

Angus,

On 10/13/2014 08:51 PM, Angus Lees wrote:

I've been reading a bunch of the existing Dockerfiles, and I have two humble
requests:

1. It would be good if the interesting code came from python sdist/bdists
rather than rpms.

This will make it possible to rebuild the containers using code from a private
branch or even unsubmitted code, without having to go through a redhat/rpm
release process first.

I care much less about where the python dependencies come from. Pulling them
from rpms rather than pip/pypi seems like a very good idea, given the relative
difficulty of caching pypi content and we also pull in the required C, etc
libraries for free.

With this in place, I think I could drop my own containers and switch to
reusing kolla's for building virtual testing environments. This would make me
happy.

I've captured this requirement here:
https://blueprints.launchpad.net/kolla/+spec/run-from-master

2. I think we should separate out run the server from do once-off setup.

Currently the containers run a start.sh that typically sets up the database,
runs the servers, creates keystone users and sets up the keystone catalog. In
something like k8s, the container will almost certainly be run multiple times
in parallel and restarted numerous times, so all those other steps go against
the service-oriented k8s ideal and are at-best wasted.

I suggest making the container contain the deployed code and offer a few thin
scripts/commands for entrypoints. The main replicationController/pod _just_
starts the server, and then we have separate pods (or perhaps even non-k8s
container invocations) that do initial database setup/migrate, and post-
install keystone setup.
The server may not start before the configuration of the server is
complete. I guess I don't quite understand what you indicate here when
you say we have separate pods that do initial database setup/migrate.
Do you mean have dependencies in some way, or for eg:

and start these two pods as part of the same selector (glance-registry)?

That idea sounds pretty appealing although probably won't be ready to go
for milestone #1.

Regards,
-steve

I'm open to whether we want to make these as lightweight/independent as
possible (every daemon in an individual container), or limit it to one per
project (eg: run nova-api, nova-conductor, nova-scheduler, etc all in one
container). I think the differences are run-time scalability and resource-
attribution vs upfront coding effort and are not hugely significant either way.

Post-install catalog setup we can combine into one cross-service setup like
tripleO does[1]. Although k8s doesn't have explicit support for batch tasks
currently, I'm doing the pre-install setup in restartPolicy: onFailure pods
currently and it seems to work quite well[2].

(I'm saying post install catalog setup, but really keystone catalog can
happen at any point pre/post aiui.)

[1]
https://github.com/openstack/tripleo-incubator/blob/master/scripts/setup-endpoints
[2]
https://github.com/anguslees/kube-openstack/blob/master/kubernetes-in/nova-db-sync-pod.yaml

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [kolla] on Dockerfile patterns

Absolutely this needs splitting out. I ran into an issue a few years ago with 
this antipattern with the mythtv folks. The myth client on my laptop got 
upgraded and it was overly helpful in that it connected directly to the 
database and upgraded the schema for me, breaking the server, and all the other 
clients, and forcing an unplanned upgrade of everything else. :/

From: Chris Dent [chd...@redhat.com]
Sent: Tuesday, October 14, 2014 3:54 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [kolla] on Dockerfile patterns

On Tue, 14 Oct 2014, Angus Lees wrote:

 2. I think we should separate out run the server from do once-off setup.

Yes! Otherwise it feels like the entire point of using containers
and dockerfiles is rather lost.

--
Chris Dent tw:@anticdent freenode:cdent
https://tank.peermore.com/tanks/cdent

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [kolla] on Dockerfile patterns


On 10/14/2014 10:49 AM, Lars Kellogg-Stedman wrote:

On Tue, Oct 14, 2014 at 02:51:15PM +1100, Angus Lees wrote:

1. It would be good if the interesting code came from python sdist/bdists
rather than rpms.


I agree in principal, although starting from packages right now lets
us ignore a whole host of issues.  Possibly we'll hit that change down
the road.


2. I think we should separate out run the server from do once-off setup.

Currently the containers run a start.sh that typically sets up the database,
runs the servers, creates keystone users and sets up the keystone catalog.  In
something like k8s, the container will almost certainly be run multiple times
in parallel and restarted numerous times, so all those other steps go against
the service-oriented k8s ideal and are at-best wasted.


All the existing containers [*] are designed to be idempotent, which I
think is not a bad model.  Even if we move initial configuration out
of the service containers I think that is a goal we want to preserve.

I pursued exactly the model you suggest on my own when working on an
ansible-driven workflow for setting things up:

   https://github.com/larsks/openstack-containers

Ansible made it easy to support one-off batch containers which, as
you say, aren't exactly supported in Kubernetes.  I like your
(ab?)use of restartPolicy; I think that's worth pursuing.


I agree that Ansible makes it easy to support one-off batch containers. 
Ansible rocks.


Which brings me to my question (admittedly, I am a Docker n00b, so 
please forgive me for the question)...


Can I use your Dockerfiles to build Ubuntu/Debian images instead of only 
Fedora images? Seems to me that the image-based Docker system makes the 
resulting container quite brittle -- since a) you can't use 
configuration management systems like Ansible to choose which operating 
system or package management tools you wish to use, and b) any time you 
may a change to the image, you need to regenerate the image from a new 
Dockerfile and, presumably, start a new container with the new image, 
shut down the old container, and then change all the other containers 
that were linked with the old container to point to the new one. All of 
this would be a simple apt-get upgrade -y for things like security 
updates, which for the most part, wouldn't require any such container 
rebuilds.


So... what am I missing with this? What makes Docker images more ideal 
than straight up LXC containers and using Ansible to control 
upgrades/changes to configuration of the software on those containers?


Again, sorry for the n00b question!

Best,
-jay


[*] That work, which includes rabbitmq, mariadb, keystone, and glance.


I'm open to whether we want to make these as lightweight/independent as
possible (every daemon in an individual container), or limit it to one per
project (eg: run nova-api, nova-conductor, nova-scheduler, etc all in one
container).


My goal is one-service-per-container, because that generally makes the
question of process supervision and log collection a *host* problem
rather than a *container* problem. It also makes it easier to scale an
individual service, if that becomes necessary.



___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [kolla] on Dockerfile patterns

On Tue, Oct 14, 2014 at 12:33:42PM -0400, Jay Pipes wrote:
 Can I use your Dockerfiles to build Ubuntu/Debian images instead of only
 Fedora images?

Not easily, no.

 Seems to me that the image-based Docker system makes the
 resulting container quite brittle -- since a) you can't use configuration
 management systems like Ansible to choose which operating system or package
 management tools you wish to use...

While that's true, it seems like a non-goal.  You're not starting with
a virtual machine and a blank disk here, you're starting from an
existing filesystem.

I'm not sure I understand your use case enough to give you a more
useful reply.

 So... what am I missing with this? What makes Docker images more ideal than
 straight up LXC containers and using Ansible to control upgrades/changes to
 configuration of the software on those containers?

I think that in general that Docker images are more share-able, and
the layered model makes building components on top of a base image
both easy and reasonably efficient in terms of time and storage.

I think that Ansible makes a great tool for managing configuration
inside Docker containers, and you could easily use it as part of the
image build process.  Right now, people using Docker are basically
writing shell scripts to perform system configuration, which is like a
20 year step back in time.  Using a more structured mechanism for
doing this is a great idea, and one that lots of people are pursuing.
I have looked into using Puppet as part of both the build and runtime
configuration process, but I haven't spent much time on it yet.

A key goal for Docker images is generally that images are immutable,
or at least stateless.  You don't yum upgrade or apt-get upgrade
in a container; you generate a new image with new packages/code/etc.
This makes it trivial to revert to a previous version of a deployment,
and clearly separates the build the image process from the run the
application process.

I like this model.

-- 
Lars Kellogg-Stedman l...@redhat.com | larsks @ {freenode,twitter,github}
Cloud Engineering / OpenStack  | http://blog.oddbit.com/



pgpF7zc0dsOpi.pgp
Description: PGP signature
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [kolla] on Dockerfile patterns


On 10/14/2014 01:28 PM, Lars Kellogg-Stedman wrote:

On Tue, Oct 14, 2014 at 12:33:42PM -0400, Jay Pipes wrote:

Can I use your Dockerfiles to build Ubuntu/Debian images instead of only
Fedora images?


Not easily, no.


Seems to me that the image-based Docker system makes the
resulting container quite brittle -- since a) you can't use configuration
management systems like Ansible to choose which operating system or package
management tools you wish to use...


While that's true, it seems like a non-goal.  You're not starting with
a virtual machine and a blank disk here, you're starting from an
existing filesystem.

I'm not sure I understand your use case enough to give you a more
useful reply.


Sorry, I'm trying hard to describe some of this. I have limited 
vocabulary to use in this new space :)


I guess what I am saying is that there is a wealth of existing 
configuration management provenance that installs and configures 
application packages. These configuration management modules/tools are 
written to entirely take away the multi-operating-system, 
multi-package-manager problems.


Instead of having two Dockerfiles, one that only works on Fedora that 
does something like:


FROM fedora20
RUN yum -y install python-pbr

and one that only works on Debian:

FROM debian:wheezy
RUN apt-get install -y python-pbr

Configuration management tools like Ansible already work cross-operating 
system, and allow you to express what gets installed regardless of the 
operating system of the disk image:


tasks:
  - name: install PBR Debian
apt: name=python-pbr state=present
when: ansible_os_family == Debian
  - name: install PBR RH
yum: name=python-pbr state=present
when: ansible_os_family == RedHat

Heck, in Chef, you wouldn't even need the when: switch logic, since Chef 
knows which package management system to use depending on the operating 
system.


With Docker, you are limited to the operating system of whatever the 
image uses.


This means, for things like your openstack-containers Ansible+Docker 
environment (which is wicked cool, BTW), you have containers running 
Fedora20 for everything except MySQL, which due to the fact that you are 
using the official [1] MySQL image on dockerhub, is only a 
Debian:Wheezy image.


This means you now have to know the system administrative comments and 
setup for two operating systems ... or go find a Fedora20 image for 
mysql somewhere.


It just seems to me that Docker is re-inventing a whole bunch of stuff 
that configuration management tools like Ansible, Puppet, Chef, and 
Saltstack have gotten good at over the years.


[1] Is there an official MySQL docker image? I found 553 Dockerhub 
repositories for MySQL images...



So... what am I missing with this? What makes Docker images more ideal than
straight up LXC containers and using Ansible to control upgrades/changes to
configuration of the software on those containers?


I think that in general that Docker images are more share-able, and
the layered model makes building components on top of a base image
both easy and reasonably efficient in terms of time and storage.


By layered model, are you referring to the bottom layer being the Docker 
image and then upper layers being stuff managed by a configuration 
management system?



I think that Ansible makes a great tool for managing configuration
inside Docker containers, and you could easily use it as part of the
image build process.  Right now, people using Docker are basically
writing shell scripts to perform system configuration, which is like a
20 year step back in time.


Right, I've noticed that :)

  Using a more structured mechanism for

doing this is a great idea, and one that lots of people are pursuing.
I have looked into using Puppet as part of both the build and runtime
configuration process, but I haven't spent much time on it yet.


Oh, I don't think Puppet is any better than Ansible for these things.


A key goal for Docker images is generally that images are immutable,
or at least stateless.  You don't yum upgrade or apt-get upgrade
in a container; you generate a new image with new packages/code/etc.
This makes it trivial to revert to a previous version of a deployment,
and clearly separates the build the image process from the run the
application process.


OK, so bear with me more on this, please... :)

Let's say I build Docker images for, say, my nova-conductor container, 
and I build the image by doing a devstack-style install from git repos 
method. Then, I build another nova-conductor container from a newer 
revision in source control.


How would I go about essentially transferring the ownership of the RPC 
exchanges that the original nova-conductor container managed to the new 
nova-conductor container? Would it be as simple as shutting down the old 
container and starting up the new nova-conductor container using things 
like --link rabbitmq:rabbitmq in the startup docker line?


Genuinely curious and inspired by this

Re: [openstack-dev] [kolla] on Dockerfile patterns

On Tue, Oct 14, 2014 at 02:45:30PM -0400, Jay Pipes wrote:
 With Docker, you are limited to the operating system of whatever the image
 uses.

See, that's the part I disagree with.  What I was saying about ansible
and puppet in my email is that I think the right thing to do is take
advantage of those tools:

  FROM ubuntu

  RUN apt-get install ansible
  COPY my_ansible_config.yaml /my_ansible_config.yaml
  RUN ansible /my_ansible_config.yaml

Or:

  FROM Fedora

  RUN yum install ansible
  COPY my_ansible_config.yaml /my_ansible_config.yaml
  RUN ansible /my_ansible_config.yaml

Put the minimal instructions in your dockerfile to bootstrap your
preferred configuration management tool. This is exactly what you
would do when booting, say, a Nova instance into an openstack
environment: you can provide a shell script to cloud-init that would
install whatever packages are required to run your config management
tool, and then run that tool.

Once you have bootstrapped your cm environment you can take advantage
of all those distribution-agnostic cm tools.

In other words, using docker is no more limiting than using a vm or
bare hardware that has been installed with your distribution of
choice.

 [1] Is there an official MySQL docker image? I found 553 Dockerhub
 repositories for MySQL images...

Yes, it's called mysql.  It is in fact one of the official images
highlighted on https://registry.hub.docker.com/.

 I have looked into using Puppet as part of both the build and runtime
 configuration process, but I haven't spent much time on it yet.
 
 Oh, I don't think Puppet is any better than Ansible for these things.

I think it's pretty clear that I was not suggesting it was better than
ansible.  That is hardly relevant to this discussion.  I was only
saying that is what *I* have looked at, and I was agreeing that *any*
configuration management system is probably better than writing shells
cript.

 How would I go about essentially transferring the ownership of the RPC
 exchanges that the original nova-conductor container managed to the new
 nova-conductor container? Would it be as simple as shutting down the old
 container and starting up the new nova-conductor container using things like
 --link rabbitmq:rabbitmq in the startup docker line?

I think that you would not necessarily rely on --link for this sort of
thing.  Under kubernetes, you would use a service definition, in
which kubernetes maintains a proxy that directs traffic to the
appropriate place as containers are created and destroyed.

Outside of kubernetes, you would use some other service discovery
mechanism; there are many available (etcd, consul, serf, etc).

But this isn't particularly a docker problem.  This is the same
problem you would face running the same software on top of a cloud
environment in which you cannot predict things like ip addresses a
priori.

-- 
Lars Kellogg-Stedman l...@redhat.com | larsks @ {freenode,twitter,github}
Cloud Engineering / OpenStack  | http://blog.oddbit.com/



pgp9SM_Y1OfTe.pgp
Description: PGP signature
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [kolla] on Dockerfile patterns


On 10/14/2014 03:10 PM, Lars Kellogg-Stedman wrote:

On Tue, Oct 14, 2014 at 02:45:30PM -0400, Jay Pipes wrote:

With Docker, you are limited to the operating system of whatever the image
uses.


See, that's the part I disagree with.  What I was saying about ansible
and puppet in my email is that I think the right thing to do is take
advantage of those tools:

   FROM ubuntu

   RUN apt-get install ansible
   COPY my_ansible_config.yaml /my_ansible_config.yaml
   RUN ansible /my_ansible_config.yaml

Or:

   FROM Fedora

   RUN yum install ansible
   COPY my_ansible_config.yaml /my_ansible_config.yaml
   RUN ansible /my_ansible_config.yaml

Put the minimal instructions in your dockerfile to bootstrap your
preferred configuration management tool. This is exactly what you
would do when booting, say, a Nova instance into an openstack
environment: you can provide a shell script to cloud-init that would
install whatever packages are required to run your config management
tool, and then run that tool.


I think the above strategy is spot on. Unfortunately, that's not how the 
Docker ecosystem works. Everything is based on images, and you get the 
operating system that the image is built for. I see you didn't respond 
to my point that in your openstack-containers environment, you end up 
with Debian *and* Fedora images, since you use the official MySQL 
dockerhub image. And therefore you will end up needing to know sysadmin 
specifics (such as how network interfaces are set up) on multiple 
operating system distributions.



Once you have bootstrapped your cm environment you can take advantage
of all those distribution-agnostic cm tools.

In other words, using docker is no more limiting than using a vm or
bare hardware that has been installed with your distribution of
choice.


Sure, Docker isn't any more limiting than using a VM or bare hardware, 
but if you use the official Docker images, it is more limiting, no?



How would I go about essentially transferring the ownership of the RPC
exchanges that the original nova-conductor container managed to the new
nova-conductor container? Would it be as simple as shutting down the old
container and starting up the new nova-conductor container using things like
--link rabbitmq:rabbitmq in the startup docker line?


I think that you would not necessarily rely on --link for this sort of
thing.  Under kubernetes, you would use a service definition, in
which kubernetes maintains a proxy that directs traffic to the
appropriate place as containers are created and destroyed.

Outside of kubernetes, you would use some other service discovery
mechanism; there are many available (etcd, consul, serf, etc).

But this isn't particularly a docker problem.  This is the same
problem you would face running the same software on top of a cloud
environment in which you cannot predict things like ip addresses a
priori.


Gotcha. I'm still reading through the Kubernetes docs to better 
understand the role that it plays in relation to Docker. Hopefully I'll 
have a better grip on this stuff by summit time :) Thanks again for your 
very helpful replies.


Best,
-jay

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [kolla] on Dockerfile patterns

2014-10-14 Thread Chris Dent


On Tue, 14 Oct 2014, Jay Pipes wrote:

This means you now have to know the system administrative comments and setup 
for two operating systems ... or go find a Fedora20 image for mysql 
somewhere.


For sake of conversation and devil's advocacy let me ask, in
response to this paragraph, why [do you] have to know [...]?

If you've got a docker container that is running mysql, IME that's _all_
it should be doing and any (post-setup) management you are doing of it
should be happening by creating a new container and trashing the one
that is not right, not manipulating the existing container. The
operating system should be as close to invisible as you can get it.

Everything in the Dockerfile should be about getting the service to
a fully installe and configured state and it should close with the
one command or entrypoint it is going to run/use, in the mysql case
that's one of the various ways to get mysqld happening.

If the goal is to use Docker and to have automation I think it would
be better to automate the generation of suitably
layered/hierarchical Dockerfiles (using whatever tool of choice),
not having Dockerfiles which then use config tools to populate out
the image. If such config tools are necessary in the container it is
pretty much guaranteed that the container is being overburdened with
too many responsibilities.

[1] Is there an official MySQL docker image? I found 553 Dockerhub 
repositories for MySQL images...


  https://github.com/docker-library/mysql

Docker-library appears to be the place for official things.

By layered model, are you referring to the bottom layer being the Docker 
image and then upper layers being stuff managed by a configuration management 
system?


I assume it's the layering afforded by union file systems: Makes
building images based on other images cheap and fast. The cheapness
and fastness is one of the reasons why expressive Dockerfiles[1] are
important: each line is a separate checkpointed image.


[1] That is, Dockerfiles which do the install and config work rather
than calling on other stuff to do the work.

--
Chris Dent tw:@anticdent freenode:cdent
https://tank.peermore.com/tanks/cdent

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [kolla] on Dockerfile patterns

Same thing works with cloud init too...


I've been waiting on systemd working inside a container for a while. it seems 
to work now.

The idea being its hard to write a shell script to get everything up and 
running with all the interactions that may need to happen. The init system's 
already designed for that. Take a nova-compute docker container for example, 
you probably need nova-compute, libvirt, neutron-openvswitch-agent, and the 
celiometer-agent all backed in. Writing a shell script to get it all started 
and shut down properly would be really ugly.

You could split it up into 4 containers and try and ensure they are coscheduled 
and all the pieces are able to talk to each other, but why? Putting them all in 
one container with systemd starting the subprocesses is much easier and 
shouldn't have many drawbacks. The components code is designed and tested 
assuming the pieces are all together.

You can even add a ssh server in there easily too and then ansible in to do 
whatever other stuff you want to do to the container like add other monitoring 
and such

Ansible or puppet or whatever should work better in this arrangement too since 
existing code assumes you can just systemctl start foo;

Kevin

From: Lars Kellogg-Stedman [l...@redhat.com]
Sent: Tuesday, October 14, 2014 12:10 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [kolla] on Dockerfile patterns

On Tue, Oct 14, 2014 at 02:45:30PM -0400, Jay Pipes wrote:
 With Docker, you are limited to the operating system of whatever the image
 uses.

See, that's the part I disagree with.  What I was saying about ansible
and puppet in my email is that I think the right thing to do is take
advantage of those tools:

  FROM ubuntu

  RUN apt-get install ansible
  COPY my_ansible_config.yaml /my_ansible_config.yaml
  RUN ansible /my_ansible_config.yaml

Or:

  FROM Fedora

  RUN yum install ansible
  COPY my_ansible_config.yaml /my_ansible_config.yaml
  RUN ansible /my_ansible_config.yaml

Put the minimal instructions in your dockerfile to bootstrap your
preferred configuration management tool. This is exactly what you
would do when booting, say, a Nova instance into an openstack
environment: you can provide a shell script to cloud-init that would
install whatever packages are required to run your config management
tool, and then run that tool.

Once you have bootstrapped your cm environment you can take advantage
of all those distribution-agnostic cm tools.

In other words, using docker is no more limiting than using a vm or
bare hardware that has been installed with your distribution of
choice.

 [1] Is there an official MySQL docker image? I found 553 Dockerhub
 repositories for MySQL images...

Yes, it's called mysql.  It is in fact one of the official images
highlighted on https://registry.hub.docker.com/.

 I have looked into using Puppet as part of both the build and runtime
 configuration process, but I haven't spent much time on it yet.

 Oh, I don't think Puppet is any better than Ansible for these things.

I think it's pretty clear that I was not suggesting it was better than
ansible.  That is hardly relevant to this discussion.  I was only
saying that is what *I* have looked at, and I was agreeing that *any*
configuration management system is probably better than writing shells
cript.

 How would I go about essentially transferring the ownership of the RPC
 exchanges that the original nova-conductor container managed to the new
 nova-conductor container? Would it be as simple as shutting down the old
 container and starting up the new nova-conductor container using things like
 --link rabbitmq:rabbitmq in the startup docker line?

I think that you would not necessarily rely on --link for this sort of
thing.  Under kubernetes, you would use a service definition, in
which kubernetes maintains a proxy that directs traffic to the
appropriate place as containers are created and destroyed.

Outside of kubernetes, you would use some other service discovery
mechanism; there are many available (etcd, consul, serf, etc).

But this isn't particularly a docker problem.  This is the same
problem you would face running the same software on top of a cloud
environment in which you cannot predict things like ip addresses a
priori.

--
Lars Kellogg-Stedman l...@redhat.com | larsks @ {freenode,twitter,github}
Cloud Engineering / OpenStack  | http://blog.oddbit.com/


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [kolla] on Dockerfile patterns

On Tue, Oct 14, 2014 at 03:25:56PM -0400, Jay Pipes wrote:
 I think the above strategy is spot on. Unfortunately, that's not how the
 Docker ecosystem works.

I'm not sure I agree here, but again nobody is forcing you to use this
tool.

 operating system that the image is built for. I see you didn't respond to my
 point that in your openstack-containers environment, you end up with Debian
 *and* Fedora images, since you use the official MySQL dockerhub image. And
 therefore you will end up needing to know sysadmin specifics (such as how
 network interfaces are set up) on multiple operating system distributions.

I missed that part, but ideally you don't *care* about the
distribution in use.  All you care about is the application.  Your
container environment (docker itself, or maybe a higher level
abstraction) sets up networking for you, and away you go.

If you have to perform system administration tasks inside your
containers, my general feeling is that something is wrong.

 Sure, Docker isn't any more limiting than using a VM or bare hardware, but
 if you use the official Docker images, it is more limiting, no?

No more so than grabbing a virtual appliance rather than building a
system yourself.  

In other words: sure, it's less flexible, but possibly it's faster to
get started, which is especially useful if your primary goal is not
be a database administrator but is actually write an application
that uses a database backend.

I think there are uses cases for both official and customized
images.

-- 
Lars Kellogg-Stedman l...@redhat.com | larsks @ {freenode,twitter,github}
Cloud Engineering / OpenStack  | http://blog.oddbit.com/



pgpKaSDODdjVy.pgp
Description: PGP signature
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [kolla] on Dockerfile patterns


On 10/14/2014 03:50 PM, Lars Kellogg-Stedman wrote:

On Tue, Oct 14, 2014 at 03:25:56PM -0400, Jay Pipes wrote:

I think the above strategy is spot on. Unfortunately, that's not how the
Docker ecosystem works.


I'm not sure I agree here, but again nobody is forcing you to use this
tool.


I know that. I'm not slamming Docker. I'm trying to better understand 
the Docker and Kubernetes systems.



operating system that the image is built for. I see you didn't respond to my
point that in your openstack-containers environment, you end up with Debian
*and* Fedora images, since you use the official MySQL dockerhub image. And
therefore you will end up needing to know sysadmin specifics (such as how
network interfaces are set up) on multiple operating system distributions.


I missed that part, but ideally you don't *care* about the
distribution in use.  All you care about is the application.  Your
container environment (docker itself, or maybe a higher level
abstraction) sets up networking for you, and away you go.

If you have to perform system administration tasks inside your
containers, my general feeling is that something is wrong.


I understand that general feeling, but system administration tasks like 
debugging networking issues or determining and grepping log file 
locations or diagnosing packaging issues for OpenStack services or 
performing database logfile maintenance and backups don't just go away 
because you're using containers, right? If there are multiple operating 
systems in use in these containers, it makes the life of an admin more 
cumbersome, IMO.


I guess all I'm saying is that, from what I can tell, Docker and 
Kubernetes and all the application/service-centric worldviews are cool 
and all, but they very much seem to be developed from the point of view 
of application developers, and not so much from the point of view of 
operators who need to maintain and support those applications.


I'm still super-interested in the topic, just thought I'd point that out.

Best,
-jay

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [kolla] on Dockerfile patterns

2014-10-14 Thread Clint Byrum

Excerpts from Lars Kellogg-Stedman's message of 2014-10-14 12:50:48 -0700:
 On Tue, Oct 14, 2014 at 03:25:56PM -0400, Jay Pipes wrote:
  I think the above strategy is spot on. Unfortunately, that's not how the
  Docker ecosystem works.
 
 I'm not sure I agree here, but again nobody is forcing you to use this
 tool.
 
  operating system that the image is built for. I see you didn't respond to my
  point that in your openstack-containers environment, you end up with Debian
  *and* Fedora images, since you use the official MySQL dockerhub image. And
  therefore you will end up needing to know sysadmin specifics (such as how
  network interfaces are set up) on multiple operating system distributions.
 
 I missed that part, but ideally you don't *care* about the
 distribution in use.  All you care about is the application.  Your
 container environment (docker itself, or maybe a higher level
 abstraction) sets up networking for you, and away you go.
 
 If you have to perform system administration tasks inside your
 containers, my general feeling is that something is wrong.
 

Speaking as a curmudgeon ops guy from back in the day.. the reason
I choose the OS I do is precisely because it helps me _when something
is wrong_. And the best way an OS can help me is to provide excellent
debugging tools, and otherwise move out of the way.

When something _is_ wrong and I want to attach GDB to mysqld in said
container, I could build a new container with debugging tools installed,
but that may lose the very system state that I'm debugging. So I need to
run things inside the container like apt-get or yum to install GDB.. and
at some point you start to realize that having a whole OS is actually a
good thing even if it means needing to think about a few more things up
front, such as which OS will I use? and what tools do I need installed
in my containers?

What I mean to say is, just grabbing off the shelf has unstated
consequences.

  Sure, Docker isn't any more limiting than using a VM or bare hardware, but
  if you use the official Docker images, it is more limiting, no?
 
 No more so than grabbing a virtual appliance rather than building a
 system yourself.  
 
 In other words: sure, it's less flexible, but possibly it's faster to
 get started, which is especially useful if your primary goal is not
 be a database administrator but is actually write an application
 that uses a database backend.
 
 I think there are uses cases for both official and customized
 images.
 

In the case of Kolla, we're deploying OpenStack, not just some new
application that uses a database backend. I think the bar is a bit
higher for operations than end-user applications, since it sits below
the abstractions, much closer to the metal.

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [kolla] on Dockerfile patterns

On Tue, Oct 14, 2014 at 04:06:22PM -0400, Jay Pipes wrote:
 I understand that general feeling, but system administration tasks like
 debugging networking issues or determining and grepping log file locations
 or diagnosing packaging issues for OpenStack services or performing database
 logfile maintenance and backups don't just go away because you're using
 containers, right?

They don't go away, but they're not necessarily things that you would
do inside your container.

Any state (e.g., database tables) that has a lifetime different from
that of your container should be stored outside of the container
proper.  In docker, this would be a volume (in a cloud environment,
this would be something like EBS or a Cinder volume).

Ideally, your container-optimized applications logs to stdout/stderr.
If you have multiple processes, they each run in a separate container.

Backups take advantage of the data volumes you've associated with your
container.  E.g., spawning a new container using the docker
--volumes-from option to access that data for backup purposes.

If you really need to get inside a container for diagnostic purposes,
then you use something like nsenter, nsinit, or the forthcoming
docker exec.


 they very much seem to be developed from the point of view of application
 developers, and not so much from the point of view of operators who need to
 maintain and support those applications.

I think it's entirely accurate to say that they are
application-centric, much like services such as Heroku, OpenShift,
etc.

-- 
Lars Kellogg-Stedman l...@redhat.com | larsks @ {freenode,twitter,github}
Cloud Engineering / OpenStack  | http://blog.oddbit.com/



pgp29hOhB_K2x.pgp
Description: PGP signature
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [kolla] on Dockerfile patterns

2014-10-14 Thread Ian Main

Angus Lees wrote:
I've been reading a bunch of the existing Dockerfiles, and I have two humble
requests:

1. It would be good if the interesting code came from python sdist/bdists
rather than rpms.

This will make it possible to rebuild the containers using code from a
private
branch or even unsubmitted code, without having to go through a redhat/rpm
release process first.

I care much less about where the python dependencies come from. Pulling them
from rpms rather than pip/pypi seems like a very good idea, given the
relative
difficulty of caching pypi content and we also pull in the required C, etc
libraries for free.

With this in place, I think I could drop my own containers and switch to
reusing kolla's for building virtual testing environments. This would make
me
happy.

2. I think we should separate out run the server from do once-off setup.

Currently the containers run a start.sh that typically sets up the database,
runs the servers, creates keystone users and sets up the keystone catalog.
In
something like k8s, the container will almost certainly be run multiple times
in parallel and restarted numerous times, so all those other steps go against
the service-oriented k8s ideal and are at-best wasted.

(I'm saying post install catalog setup, but really keystone catalog can
happen at any point pre/post aiui.)

[1]
https://github.com/openstack/tripleo-incubator/blob/master/scripts/setup-endpoints
[2]
https://github.com/anguslees/kube-openstack/blob/master/kubernetes-in/nova-db-sync-pod.yaml

--
- Gus

One thing I've learned is to not perform software updates within a container.
A number of the containers I've seen do software updates on startup but I've
seen this break dependencies in containers a few times making them unusable.
This detracts from the ability to have a completely controlled environment
within a container with proven software versions that play nicely together.

Ian

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [kolla] on Dockerfile patterns

2014-10-14 Thread David Vossel



- Original Message -
 Same thing works with cloud init too...
 
 
 I've been waiting on systemd working inside a container for a while. it seems
 to work now.

oh no...

 The idea being its hard to write a shell script to get everything up and
 running with all the interactions that may need to happen. The init system's
 already designed for that. Take a nova-compute docker container for example,
 you probably need nova-compute, libvirt, neutron-openvswitch-agent, and the
 celiometer-agent all backed in. Writing a shell script to get it all started
 and shut down properly would be really ugly.

 You could split it up into 4 containers and try and ensure they are
 coscheduled and all the pieces are able to talk to each other, but why?
 Putting them all in one container with systemd starting the subprocesses is
 much easier and shouldn't have many drawbacks. The components code is
 designed and tested assuming the pieces are all together.

What you need is a dependency model that is enforced outside of the containers. 
Something
that manages the order containers are started/stopped/recovered in. This allows
you to isolate your containers with 1 service per container, yet still express 
that
container with service A needs to start before container with service B.

Pacemaker does this easily. There's even a docker resource-agent for Pacemaker 
now.
https://github.com/ClusterLabs/resource-agents/blob/master/heartbeat/docker

-- Vossel

ps. don't run systemd in a container... If you think you should, talk to me 
first.

 
 You can even add a ssh server in there easily too and then ansible in to do
 whatever other stuff you want to do to the container like add other
 monitoring and such
 
 Ansible or puppet or whatever should work better in this arrangement too
 since existing code assumes you can just systemctl start foo;
 
 Kevin
 
 From: Lars Kellogg-Stedman [l...@redhat.com]
 Sent: Tuesday, October 14, 2014 12:10 PM
 To: OpenStack Development Mailing List (not for usage questions)
 Subject: Re: [openstack-dev] [kolla] on Dockerfile patterns
 
 On Tue, Oct 14, 2014 at 02:45:30PM -0400, Jay Pipes wrote:
  With Docker, you are limited to the operating system of whatever the image
  uses.
 
 See, that's the part I disagree with.  What I was saying about ansible
 and puppet in my email is that I think the right thing to do is take
 advantage of those tools:
 
   FROM ubuntu
 
   RUN apt-get install ansible
   COPY my_ansible_config.yaml /my_ansible_config.yaml
   RUN ansible /my_ansible_config.yaml
 
 Or:
 
   FROM Fedora
 
   RUN yum install ansible
   COPY my_ansible_config.yaml /my_ansible_config.yaml
   RUN ansible /my_ansible_config.yaml
 
 Put the minimal instructions in your dockerfile to bootstrap your
 preferred configuration management tool. This is exactly what you
 would do when booting, say, a Nova instance into an openstack
 environment: you can provide a shell script to cloud-init that would
 install whatever packages are required to run your config management
 tool, and then run that tool.
 
 Once you have bootstrapped your cm environment you can take advantage
 of all those distribution-agnostic cm tools.
 
 In other words, using docker is no more limiting than using a vm or
 bare hardware that has been installed with your distribution of
 choice.
 
  [1] Is there an official MySQL docker image? I found 553 Dockerhub
  repositories for MySQL images...
 
 Yes, it's called mysql.  It is in fact one of the official images
 highlighted on https://registry.hub.docker.com/.
 
  I have looked into using Puppet as part of both the build and runtime
  configuration process, but I haven't spent much time on it yet.
 
  Oh, I don't think Puppet is any better than Ansible for these things.
 
 I think it's pretty clear that I was not suggesting it was better than
 ansible.  That is hardly relevant to this discussion.  I was only
 saying that is what *I* have looked at, and I was agreeing that *any*
 configuration management system is probably better than writing shells
 cript.
 
  How would I go about essentially transferring the ownership of the RPC
  exchanges that the original nova-conductor container managed to the new
  nova-conductor container? Would it be as simple as shutting down the old
  container and starting up the new nova-conductor container using things
  like
  --link rabbitmq:rabbitmq in the startup docker line?
 
 I think that you would not necessarily rely on --link for this sort of
 thing.  Under kubernetes, you would use a service definition, in
 which kubernetes maintains a proxy that directs traffic to the
 appropriate place as containers are created and destroyed.
 
 Outside of kubernetes, you would use some other service discovery
 mechanism; there are many available (etcd, consul, serf, etc).
 
 But this isn't particularly a docker problem.  This is the same
 problem you would face running the same software on top of a cloud

Re: [openstack-dev] [kolla] on Dockerfile patterns

Ok, why are you so down on running systemd in a container?

Pacemaker works, but its kind of a pain to setup compared just yum installing a 
few packages and setting init to systemd. There are some benefits for sure, but 
if you have to force all the docker components onto the same physical machine 
anyway, why bother with the extra complexity?

Thanks,
Kevin


From: David Vossel [dvos...@redhat.com]
Sent: Tuesday, October 14, 2014 3:14 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [kolla] on Dockerfile patterns

- Original Message -
 Same thing works with cloud init too...


 I've been waiting on systemd working inside a container for a while. it seems
 to work now.

oh no...

 The idea being its hard to write a shell script to get everything up and
 running with all the interactions that may need to happen. The init system's
 already designed for that. Take a nova-compute docker container for example,
 you probably need nova-compute, libvirt, neutron-openvswitch-agent, and the
 celiometer-agent all backed in. Writing a shell script to get it all started
 and shut down properly would be really ugly.

 You could split it up into 4 containers and try and ensure they are
 coscheduled and all the pieces are able to talk to each other, but why?
 Putting them all in one container with systemd starting the subprocesses is
 much easier and shouldn't have many drawbacks. The components code is
 designed and tested assuming the pieces are all together.

What you need is a dependency model that is enforced outside of the containers. 
Something
that manages the order containers are started/stopped/recovered in. This allows
you to isolate your containers with 1 service per container, yet still express 
that
container with service A needs to start before container with service B.

Pacemaker does this easily. There's even a docker resource-agent for Pacemaker 
now.
https://github.com/ClusterLabs/resource-agents/blob/master/heartbeat/docker

-- Vossel

ps. don't run systemd in a container... If you think you should, talk to me 
first.


 You can even add a ssh server in there easily too and then ansible in to do
 whatever other stuff you want to do to the container like add other
 monitoring and such

 Ansible or puppet or whatever should work better in this arrangement too
 since existing code assumes you can just systemctl start foo;

 Kevin
 
 From: Lars Kellogg-Stedman [l...@redhat.com]
 Sent: Tuesday, October 14, 2014 12:10 PM
 To: OpenStack Development Mailing List (not for usage questions)
 Subject: Re: [openstack-dev] [kolla] on Dockerfile patterns

 On Tue, Oct 14, 2014 at 02:45:30PM -0400, Jay Pipes wrote:
  With Docker, you are limited to the operating system of whatever the image
  uses.

 See, that's the part I disagree with.  What I was saying about ansible
 and puppet in my email is that I think the right thing to do is take
 advantage of those tools:

   FROM ubuntu

   RUN apt-get install ansible
   COPY my_ansible_config.yaml /my_ansible_config.yaml
   RUN ansible /my_ansible_config.yaml

 Or:

   FROM Fedora

   RUN yum install ansible
   COPY my_ansible_config.yaml /my_ansible_config.yaml
   RUN ansible /my_ansible_config.yaml

 Put the minimal instructions in your dockerfile to bootstrap your
 preferred configuration management tool. This is exactly what you
 would do when booting, say, a Nova instance into an openstack
 environment: you can provide a shell script to cloud-init that would
 install whatever packages are required to run your config management
 tool, and then run that tool.

 Once you have bootstrapped your cm environment you can take advantage
 of all those distribution-agnostic cm tools.

 In other words, using docker is no more limiting than using a vm or
 bare hardware that has been installed with your distribution of
 choice.

  [1] Is there an official MySQL docker image? I found 553 Dockerhub
  repositories for MySQL images...

 Yes, it's called mysql.  It is in fact one of the official images
 highlighted on https://registry.hub.docker.com/.

  I have looked into using Puppet as part of both the build and runtime
  configuration process, but I haven't spent much time on it yet.
 
  Oh, I don't think Puppet is any better than Ansible for these things.

 I think it's pretty clear that I was not suggesting it was better than
 ansible.  That is hardly relevant to this discussion.  I was only
 saying that is what *I* have looked at, and I was agreeing that *any*
 configuration management system is probably better than writing shells
 cript.

  How would I go about essentially transferring the ownership of the RPC
  exchanges that the original nova-conductor container managed to the new
  nova-conductor container? Would it be as simple as shutting down the old
  container and starting up the new nova-conductor container using things
  like

Re: [openstack-dev] [kolla] on Dockerfile patterns

2014-10-14 Thread David Vossel



- Original Message -
 Ok, why are you so down on running systemd in a container?

It goes against the grain.

From a distributed systems view, we gain quite a bit of control by maintaining
one service per container. Containers can be re-organised and re-purposed 
dynamically.
If we have systemd trying to manage an entire stack of resources within a 
container,
we lose this control.

From my perspective a containerized application stack needs to be managed 
externally
by whatever is orchestrating the containers to begin with. When we take a step 
back
and look at how we actually want to deploy containers, systemd doesn't make 
much sense.
It actually limits us in the long run.

Also... recovery. Using systemd to manage a stack of resources within a single 
container
makes it difficult for whatever is externally enforcing the availability of 
that container
to detect the health of the container.  As it is now, the actual service is pid 
1 of a
container. If that service dies, the container dies. If systemd is pid 1, there 
can
be all kinds of chaos occurring within the container, but the external 
distributed
orchestration system won't have a clue (unless it invokes some custom health 
monitoring
tools within the container itself, which will likely be the case someday.)

-- Vossel


 Pacemaker works, but its kind of a pain to setup compared just yum installing
 a few packages and setting init to systemd. There are some benefits for
 sure, but if you have to force all the docker components onto the same
 physical machine anyway, why bother with the extra complexity?

 Thanks,
 Kevin
 
 
 From: David Vossel [dvos...@redhat.com]
 Sent: Tuesday, October 14, 2014 3:14 PM
 To: OpenStack Development Mailing List (not for usage questions)
 Subject: Re: [openstack-dev] [kolla] on Dockerfile patterns
 
 - Original Message -
  Same thing works with cloud init too...
 
 
  I've been waiting on systemd working inside a container for a while. it
  seems
  to work now.
 
 oh no...
 
  The idea being its hard to write a shell script to get everything up and
  running with all the interactions that may need to happen. The init
  system's
  already designed for that. Take a nova-compute docker container for
  example,
  you probably need nova-compute, libvirt, neutron-openvswitch-agent, and the
  celiometer-agent all backed in. Writing a shell script to get it all
  started
  and shut down properly would be really ugly.
 
  You could split it up into 4 containers and try and ensure they are
  coscheduled and all the pieces are able to talk to each other, but why?
  Putting them all in one container with systemd starting the subprocesses is
  much easier and shouldn't have many drawbacks. The components code is
  designed and tested assuming the pieces are all together.
 
 What you need is a dependency model that is enforced outside of the
 containers. Something
 that manages the order containers are started/stopped/recovered in. This
 allows
 you to isolate your containers with 1 service per container, yet still
 express that
 container with service A needs to start before container with service B.
 
 Pacemaker does this easily. There's even a docker resource-agent for
 Pacemaker now.
 https://github.com/ClusterLabs/resource-agents/blob/master/heartbeat/docker
 
 -- Vossel
 
 ps. don't run systemd in a container... If you think you should, talk to me
 first.
 
 
  You can even add a ssh server in there easily too and then ansible in to do
  whatever other stuff you want to do to the container like add other
  monitoring and such
 
  Ansible or puppet or whatever should work better in this arrangement too
  since existing code assumes you can just systemctl start foo;
 
  Kevin
  
  From: Lars Kellogg-Stedman [l...@redhat.com]
  Sent: Tuesday, October 14, 2014 12:10 PM
  To: OpenStack Development Mailing List (not for usage questions)
  Subject: Re: [openstack-dev] [kolla] on Dockerfile patterns
 
  On Tue, Oct 14, 2014 at 02:45:30PM -0400, Jay Pipes wrote:
   With Docker, you are limited to the operating system of whatever the
   image
   uses.
 
  See, that's the part I disagree with.  What I was saying about ansible
  and puppet in my email is that I think the right thing to do is take
  advantage of those tools:
 
FROM ubuntu
 
RUN apt-get install ansible
COPY my_ansible_config.yaml /my_ansible_config.yaml
RUN ansible /my_ansible_config.yaml
 
  Or:
 
FROM Fedora
 
RUN yum install ansible
COPY my_ansible_config.yaml /my_ansible_config.yaml
RUN ansible /my_ansible_config.yaml
 
  Put the minimal instructions in your dockerfile to bootstrap your
  preferred configuration management tool. This is exactly what you
  would do when booting, say, a Nova instance into an openstack
  environment: you can provide a shell script to cloud-init that would
  install whatever packages are required to run

Re: [openstack-dev] [kolla] on Dockerfile patterns

2014-10-14 Thread Charles Crouch



- Original Message -
 Ok, why are you so down on running systemd in a container?
 
 Pacemaker works, but its kind of a pain to setup compared just yum installing
 a few packages and setting init to systemd. There are some benefits for
 sure, but if you have to force all the docker components onto the same
 physical machine anyway, why bother with the extra complexity?

The other benefit of running systemd in a container is that, at least
conceptually, it makes the migration path from VMs to docker containers
that much easier. If you've got an existing investment in using system
tools to run your services, it would be awfully nice to be able to keep
using that, at least in the short term, as you get your feet wet with
docker. Once you've seen the light and decided that pacemaker is a much
better choice than systemd then you can go port everything over.

I know that using systemd is not the docker-way but it is a way. 
And I think enabling people to gain some benefits of using containers without 
making them completely re-architect how they manage their services
has some value. 

To me this is akin to letting people run their legacy/non-cloudy apps on 
an IaaS. Are you going to get all the benefits that come from re-architecting 
your app to be autoscaleable and robust in the face of instance outages?
Nope. But are you going to get some benefits? Yep

Cheers
Charles

 
 Thanks,
 Kevin
 
 
 From: David Vossel [dvos...@redhat.com]
 Sent: Tuesday, October 14, 2014 3:14 PM
 To: OpenStack Development Mailing List (not for usage questions)
 Subject: Re: [openstack-dev] [kolla] on Dockerfile patterns
 
 - Original Message -
  Same thing works with cloud init too...
 
 
  I've been waiting on systemd working inside a container for a while. it
  seems
  to work now.
 
 oh no...
 
  The idea being its hard to write a shell script to get everything up and
  running with all the interactions that may need to happen. The init
  system's
  already designed for that. Take a nova-compute docker container for
  example,
  you probably need nova-compute, libvirt, neutron-openvswitch-agent, and the
  celiometer-agent all backed in. Writing a shell script to get it all
  started
  and shut down properly would be really ugly.
 
  You could split it up into 4 containers and try and ensure they are
  coscheduled and all the pieces are able to talk to each other, but why?
  Putting them all in one container with systemd starting the subprocesses is
  much easier and shouldn't have many drawbacks. The components code is
  designed and tested assuming the pieces are all together.
 
 What you need is a dependency model that is enforced outside of the
 containers. Something
 that manages the order containers are started/stopped/recovered in. This
 allows
 you to isolate your containers with 1 service per container, yet still
 express that
 container with service A needs to start before container with service B.
 
 Pacemaker does this easily. There's even a docker resource-agent for
 Pacemaker now.
 https://github.com/ClusterLabs/resource-agents/blob/master/heartbeat/docker
 
 -- Vossel
 
 ps. don't run systemd in a container... If you think you should, talk to me
 first.
 
 
  You can even add a ssh server in there easily too and then ansible in to do
  whatever other stuff you want to do to the container like add other
  monitoring and such
 
  Ansible or puppet or whatever should work better in this arrangement too
  since existing code assumes you can just systemctl start foo;
 
  Kevin
  
  From: Lars Kellogg-Stedman [l...@redhat.com]
  Sent: Tuesday, October 14, 2014 12:10 PM
  To: OpenStack Development Mailing List (not for usage questions)
  Subject: Re: [openstack-dev] [kolla] on Dockerfile patterns
 
  On Tue, Oct 14, 2014 at 02:45:30PM -0400, Jay Pipes wrote:
   With Docker, you are limited to the operating system of whatever the
   image
   uses.
 
  See, that's the part I disagree with.  What I was saying about ansible
  and puppet in my email is that I think the right thing to do is take
  advantage of those tools:
 
FROM ubuntu
 
RUN apt-get install ansible
COPY my_ansible_config.yaml /my_ansible_config.yaml
RUN ansible /my_ansible_config.yaml
 
  Or:
 
FROM Fedora
 
RUN yum install ansible
COPY my_ansible_config.yaml /my_ansible_config.yaml
RUN ansible /my_ansible_config.yaml
 
  Put the minimal instructions in your dockerfile to bootstrap your
  preferred configuration management tool. This is exactly what you
  would do when booting, say, a Nova instance into an openstack
  environment: you can provide a shell script to cloud-init that would
  install whatever packages are required to run your config management
  tool, and then run that tool.
 
  Once you have bootstrapped your cm environment you can take advantage
  of all those distribution-agnostic cm tools.
 
  In other words, using

Re: [openstack-dev] [kolla] on Dockerfile patterns