On 09/25/2014 12:01 AM, Clint Byrum wrote:
Excerpts from Mike Spreitzer's message of 2014-09-24 22:01:54 -0700:
Clint Byrum <cl...@fewbar.com> wrote on 09/25/2014 12:13:53 AM:

Excerpts from Mike Spreitzer's message of 2014-09-24 20:49:20 -0700:
Steven Dake <sd...@redhat.com> wrote on 09/24/2014 11:02:49 PM:
Does TripleO require container functionality that is not available
when using the Docker driver for Nova?

As far as I can tell, the quantitative handling of capacities and
demands in Kubernetes is much inferior to what Nova does today.

Yes, TripleO needs to manage baremetal and containers from a single
host. Nova and Neutron do not offer this as a feature unfortunately.
In what sense would Kubernetes "manage baremetal" (at all)?
By "from a single host" do you mean that a client on one host
can manage remote baremetal and containers?

I can see that Kubernetes allows a client on one host to get
containers placed remotely --- but so does the Docker driver for Nova.

I mean that one box would need to host Ironic, Docker, and Nova, for
the purposes of deploying OpenStack. We call it the "undercloud", or
sometimes the "Deployment Cloud".

It's not necessarily something that Nova/Neutron cannot do by design,
but it doesn't work now.

As far as use cases go, the main use case is to run a specific
Docker container on a specific Kubernetes "minion" bare metal host.
Clint, in another branch of this email tree you referred to
"the VMs that host Kubernetes".  How does that square with
Steve's text that seems to imply bare metal minions?

That was in a more general context, discussing using Kubernetes for
general deployment. Could have just as easily have said "hosts",
"machines", or "instances".

I can see that some people have had much more detailed design
discussions than I have yet found.  Perhaps it would be helpful
to share an organized presentation of the design thoughts in
more detail.

I personally have not had any detailed discussions about this before it
was announced. I've just dug into the design and some of the code of
Kubernetes because it is quite interesting to me.

If TripleO already knows it wants to run a specific Docker image
on a specific host then TripleO does not need a scheduler.

TripleO does not ever specify destination host, because Nova does not
allow that, nor should it. It does want to isolate failure domains so
that all three Galera nodes aren't on the same PDU, but we've not really
gotten to the point where we can do that yet.
So I am still not clear on what Steve is trying to say is the main use
Kubernetes is even farther from balancing among PDUs than Nova is.
At least Nova has a framework in which this issue can be posed and solved.
I mean a framework that actually can carry the necessary information.
The Kubernetes scheduler interface is extremely impoverished in the
information it passes and it uses GO structs --- which, like C structs,
can not be subclassed.
I don't think this is totally clear yet. The thing that Steven seems to be
trying to solve is deploying OpenStack using docker, and Kubernetes may
very well be a better choice than Nova for this. There are some really
nice features, and a lot of the benefits we've been citing about image
based deployments are realized in docker without the pain of a full OS
image to redeploy all the time.

This is precisely the problem I want to solve. I looked at Nova+Docker as a solution, and it seems to me the runway to get to a successful codebase is longer with more risk. That is why this is an experiment to see if a Kubernetes based approach would work. if at the end of the day we throw out Kubernetes as a scheduler once we have the other problems solved and reimplement Kubernetes in Nova+Docker, I think that would be an acceptable outcome, but not something I want to *start* with but *finish* with.


The structs vs. classes argument is completely out of line and has
nothing to do with where Kubernetes might go in the future. It's like
saying because cars use internal combustion engines they are limited. It
is just a facet of how it works today.

Nova's filter scheduler includes a fatal bug that bites when balancing and
you want more than
one element per area, see https://bugs.launchpad.net/nova/+bug/1373478.
However: (a) you might not need more than one element per area and
(b) fixing that bug is a much smaller job than expanding the mind of K8s.

Perhaps. I am quite a fan of set based design, and Kubernetes is a
narrowly focused single implementation solution, where Nova is a broadly
focused abstraction layer for VM's. I think it is worthwhile to push
a bit into the Kubernetes space and see whether the limitations are
important or not.

OpenStack-dev mailing list

OpenStack-dev mailing list

Reply via email to