Thank you for your answer and interest in this! Just a small note about scheduler logic. Mozilla guys created Kazoo - pure python implementation of Zookeeper protocol (which isn't trivial with all retries and timeouts) and it seems to work. So - nothing extraordinary, just careful programming :) But, yes, let's start with executors though. I don't quite understand why would we need to change libprocess. I haven't dug too deep into sources (and I'm no C++ guy) - but to me it seems that protocol is protobuf messages inside HTTP POST body. I just couldn't find yet where to does executor driver send them (UNIX socket? TCP?). Wouldn't simply documenting it be enough for a start? I'll gladly create JIRA, just want to have better understanding of this undertaking.
On Mon, Jan 13, 2014 at 7:25 AM, Vinod Kone <[email protected]> wrote: > Hey Vetoshkin, > > Thanks for bringing up the question. As Dave alluded to we have discussed > the pros and cons of getting away from language bindings in the past. > > One of the advantages of us proving language bindings and frameworks > linking to it is that we have to implement non-trivial scheduler/executor > specific logic in a single place, the driver (sched.cpp and exec.cpp > respectively). These drivers are not simply wrappers that call in to the > scheduler/executor after getting a message from Mesos. > > For example the scheduler driver does authentication and mesos master > discovery on behalf of the scheduler. If we were to get rid of the driver, > every framework (or language) has to cook up their own solutions for these > in addition to handling the transport of the protocol messages, which makes > writing a new framework rather tedious. > > That said the executor driver unlike the scheduler driver, has very little > logic in it besides wrapping the executor. So we could conceivably get rid > of the executor bindings in favor of providing pure language > implementations. This requires some (substantial?) refactoring of > libprocess (mesos's communication framework) to send/receive HTTP wrapped > protobuf messages. Do you mind creating a ticket to track this work? > > HTH, > > > On Sat, Jan 11, 2014 at 7:01 PM, Dave Lester <[email protected] > >wrote: > > > HI Vetoshkin, > > > > At face value, I think the goal of making language bindings unnecessary > in > > favor of adopting a protocol makes sense -- I'd love to see Mesos expand > > its support for framework languages, and this could potentially help make > > that easier. > > > > If I remember correctly, Vinod (cc'd) was discussing a similar idea a few > > months back. Perhaps he'd like to weigh in. > > > > Dave > > > > > > On Thu, Jan 9, 2014 at 10:09 PM, Vetoshkin Nikita < > > [email protected]> wrote: > > > >> Hi! > >> Have you ever considered providing a protocol for implementing Executor > in > >> foreign language additionally to writing various bindings yourself? > >> I think that protocol isn't that hard as you already use protobuf to > >> exchange information, but I haven't dug deep in the sources. Documenting > >> wire protocol could open ways to implement Executors in pure python or > >> golang or any other language. It could provide some benefits: > >> * in Python one could use gevent which is pretty popular > >> * golang has it's own eventloop builtin > >> * pure language binding could save a lot of trouble bridging with > >> unfriendly C++ > >> * building and using pure language client could be performed using > >> native > >> tools like `pip install` or `go get` without need to establish c++ dev > >> environment. > >> Going further you could open Framework<->Master protocol to allow crazy > >> stuff like using all fancy Netty, Akka, Finagle, whatever... network > >> tools, > >> frameworks and libraries to talk to Mesos. > >> > >> Sorry to bother if it's a wrong place or there's already an answer in > the > >> wild. Thanks in advance! > >> > > > > >
