Hi Tyson Thanks for reaching out, having Mesos integration in OpenWhisk is something I have heard some users asking about and any help to figure out how to best integrate are very welcome.
As you already aware OpenWhisk Invoker is a critical path component optimized for performance, to get those container pause and unpause in single digit milliseconds that's the reason for the recent change from using docker engine down to runc Here is my initial notes, take into account I don't have first hand experience with Mesos, which is why is good to have folks like you join :-) I think extending in a way that a different mode can be use would be useful to plug a different orchestration methodology. For log collection we are looking into sending logs into ELK (Elastic search) there is some early investigation going on, so I see after this integration, mesos can be told to which ELK to send the logs, and the Invoker to get the results. In terms of scala vs. nodejs, I would recommend to use scala with the java client, since there could be some components being integrated into Invoker and code sharing. How you want to follow up, maybe you have something to show us or maybe setup a hangout call to meet I think Dragos from Adobe, might have a lot more feedback than me since I think he has first hand experience --Carlos On Sat, May 27, 2017 at 2:44 PM Tyson Norris <[email protected]> wrote: > Hi All - > I’ve been considering how we can leverage mesos resource scheduling within > OpenWhisk, and wanted to share some ideas and solicit some feedback. > > In general, I’ve been experimenting with an approach that replaces the > DockerContainer implementation with a MesosTask. The MesosTask will have > effectively the same lifecycle (ignoring pause/resume functions) and the > same interface (HTTP api to the container), the main difference being that > the container will be deployed to some arbitrary host in the cluster. > > There are a few broad topics around this, including: > - docker client usage needs to be better isolated (e.g. cleaning up > existing containers, reconciling containers already running at time of > invoker startup, etc). I think this will be mostly straightforward. > - log collection - I’m not sure the best approach here, but one is to > completely decouple log collection from activation execution, and then > provide a mesos-specific impl > - container state tracking + load balancing - obviously there is potential > for conflict if 2 invokers schedule activations on the same container in > the cluster (since the state tracking of the container would still be in > the invoker). This would imply some extension to the ContainerPool as well. > - mesos framework - we’ve discussed internally a bit, and some preferences > I have are: leverage the mesos http api (avoid the older jni libs if > possible) and provide an independent framework application which provides a > (simpler) http api that is consumed by the invoker (if mesos integration > is enabled in the deployment). This way the framework deployment can be > isolated from controller/invoker deployment, and interaction with docker > containers is mostly the same (except for logging). > - There are a few options of implementing mesos http api clients like: > rxjava client, and nodejs client, but I've not seen any good Scala client > to date, so we may either provide a scala app using the rxjava client, or > create.a new client in Scala. > > Let me know if you have any thoughts around any of these, and I will share > more details as they come. > > Thanks > Tyson > > >
