Carlos, please sync up with me to get my full rant on the deficiencies of the Cloud API. I can point you to our "best" implementations (which sadly for the community are closed source - not my call)
FWIW the current Cloud API is IMHO a crappy extension point and it is almost impossible for anyone to implement correctly. There is, to my knowledge, no correct implementation of the Jenkins Cloud API. The "best" implementations I am aware of are currently closed source but IMHO it's no big loss to the community as even those have issues. The changes I made in 1.607 correct for some of the issues, but w.r.t. the Cloud API it's more lipstick on a pig. On my backlog is to replace the Cloud API with a new and much easier to implement extension point. I gave myself the hooks to do that in either 1.592 or 1.588 and it should be possible to implement as a plugin rather than waiting for changes in core. Every single cloud implementation has issues as soon as you start to put it under load. With oc-cloud I am able to mask a large chunk of the issues by virtue of doing the required coordination from Operations Center and by doing the state management in a local transaction database... but all of that is far far too much overhead to ask every cloud implementation to do. Jenkins *should* be doing it for the cloud implementer... In short, I believe what the docker-plugin is doing is "a severely sub-optimal path" and it only gets away with it when under light-medium load. I can cause it to "go bad" with some of my stress tests that I used to develop the fixes for 1.607 On 23 January 2015 at 17:33, Carlos Sanchez <car...@apache.org> wrote: > Hi! > > I'm creating the Kubernetes plugin [1] to start slaves in Kubernetes or > Google Container Engine, which basically means launching Docker containers > across multiple hosts with just one API. > Some background: Kubernetes allows launching individual containers (Pods) > or creating a ReplicationController, which accepts a Pod template and a > number of how many containers you want to run, and can be resized > dynamically, a pool of containers. > > Initially I though about implementing the plugin using a > ReplicationController and resizing it as new slaves are needed, but seems > that Jenkins operation doesn't fit well with this idea. It would prevent > killing specific slaves, it would just be a pool. So I went ahead and moved > to launch Pods one by one. > > The issue I have now is about initialization time, where the container is > not accepting connections until the image is downloaded and started. The > Docker plugin has this comment [2] > > // Docker instances may have a long init script. If we declare > // the provisioning complete by returning without the connect > // operation, NodeProvisioner may decide that it still wants > // one more instance, because it sees that (1) all the slaves > // are offline (because it's still being launched) and > // (2) there's no capacity provisioned yet. > // > // deferring the completion of provisioning until the launch > // goes successful prevents this problem. > slave.toComputer().connect(false).get(); > > Should I do the same? > > There is also the issue that Kubernetes would be better suited for swarm > slaves. Opening ssh or ports in each pod is more complicated than just > doing the connection to the master from the container. > > Thanks! > > [1] > https://github.com/carlossg/jenkins-kubernetes-plugin/blob/master/src/main/java/org/csanchez/jenkins/plugins/kubernetes/KubernetesCloud.java > [2] > https://github.com/jenkinsci/docker-plugin/blob/master/src/main/java/com/nirima/jenkins/plugins/docker/DockerCloud.java#L239 > > -- > You received this message because you are subscribed to the Google Groups > "Jenkins Developers" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to jenkinsci-dev+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/jenkinsci-dev/5e641038-b83b-4901-83a8-4bb54255498a%40googlegroups.com > <https://groups.google.com/d/msgid/jenkinsci-dev/5e641038-b83b-4901-83a8-4bb54255498a%40googlegroups.com?utm_medium=email&utm_source=footer> > . > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "Jenkins Developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-dev+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CA%2BnPnMyQQZ%3DqKjpr_-3APS%2BeQU1ggBVemKUeW-AM0eKSEVhwKQ%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
