Carlos, please sync up with me to get my full rant on the deficiencies of the Cloud API. I can point you to our "best" implementations (which sadly for the community are closed source - not my call)
FWIW the current Cloud API is IMHO a crappy extension point and it is almost impossible for anyone to implement correctly. There is, to my knowledge, no correct implementation of the Jenkins Cloud API. The "best" implementations I am aware of are currently closed source but IMHO it's no big loss to the community as even those have issues. The changes I made in 1.607 correct for some of the issues, but w.r.t. the Cloud API it's more lipstick on a pig. On my backlog is to replace the Cloud API with a new and much easier to implement extension point. I gave myself the hooks to do that in either 1.592 or 1.588 and it should be possible to implement as a plugin rather than waiting for changes in core. Every single cloud implementation has issues as soon as you start to put it under load. With oc-cloud I am able to mask a large chunk of the issues by virtue of doing the required coordination from Operations Center and by doing the state management in a local transaction database... but all of that is far far too much overhead to ask every cloud implementation to do. Jenkins *should* be doing it for the cloud implementer... In short, I believe what the docker-plugin is doing is "a severely sub-optimal path" and it only gets away with it when under light-medium load. I can cause it to "go bad" with some of my stress tests that I used to develop the fixes for 1.607 On 23 January 2015 at 17:33, Carlos Sanchez <[email protected]> wrote: > Hi! > > I'm creating the Kubernetes plugin [1] to start slaves in Kubernetes or > Google Container Engine, which basically means launching Docker containers > across multiple hosts with just one API. > Some background: Kubernetes allows launching individual containers (Pods) > or creating a ReplicationController, which accepts a Pod template and a > number of how many containers you want to run, and can be resized > dynamically, a pool of containers. > > Initially I though about implementing the plugin using a > ReplicationController and resizing it as new slaves are needed, but seems > that Jenkins operation doesn't fit well with this idea. It would prevent > killing specific slaves, it would just be a pool. So I went ahead and moved > to launch Pods one by one. > > The issue I have now is about initialization time, where the container is > not accepting connections until the image is downloaded and started. The > Docker plugin has this comment [2] > > // Docker instances may have a long init script. If we declare > // the provisioning complete by returning without the connect > // operation, NodeProvisioner may decide that it still wants > // one more instance, because it sees that (1) all the slaves > // are offline (because it's still being launched) and > // (2) there's no capacity provisioned yet. > // > // deferring the completion of provisioning until the launch > // goes successful prevents this problem. > slave.toComputer().connect(false).get(); > > Should I do the same? > > There is also the issue that Kubernetes would be better suited for swarm > slaves. Opening ssh or ports in each pod is more complicated than just > doing the connection to the master from the container. > > Thanks! > > [1] > https://github.com/carlossg/jenkins-kubernetes-plugin/blob/master/src/main/java/org/csanchez/jenkins/plugins/kubernetes/KubernetesCloud.java > [2] > https://github.com/jenkinsci/docker-plugin/blob/master/src/main/java/com/nirima/jenkins/plugins/docker/DockerCloud.java#L239 > > -- > You received this message because you are subscribed to the Google Groups > "Jenkins Developers" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/jenkinsci-dev/5e641038-b83b-4901-83a8-4bb54255498a%40googlegroups.com > <https://groups.google.com/d/msgid/jenkinsci-dev/5e641038-b83b-4901-83a8-4bb54255498a%40googlegroups.com?utm_medium=email&utm_source=footer> > . > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "Jenkins Developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CA%2BnPnMyQQZ%3DqKjpr_-3APS%2BeQU1ggBVemKUeW-AM0eKSEVhwKQ%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
