[
https://issues.apache.org/jira/browse/MESOS-930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13884398#comment-13884398
]
Nikita Vetoshkin commented on MESOS-930:
----------------------------------------
Stepping back and giving this thought another spin is great. No need to hurry.
You're right about goals. One thing I'd like to add to (3). Bindings should be
maintainable by developers of the language. Thus maintaining pure Python or
pure Go bindings is much easier for people dealing with Python or Go. E.g. I'm
experiencing deadlock somewhere in example scheduler in Python, event Ctrl+C
doesn't help - process just keeps running. I've attached gdb - I'm seeing a
bunch of threads waiting on mutex and one - on epoll. It's hard for python
developer to go deeper and debug this. I'll create JIRA on this issue a bit
later, I promise :).
I'd also like to argue about reimplementing the same thing:
# A lot of protocols are reimplemented in every language or even several
implementations in one language.
Take e.g. HTTP. There are pure Python modules as well as bindings to
libcurl. There are several drivers for PostgreSQL.
# Zookeeper clients.
I think you would agree that zookeeper protocol and client behaviour isn't
a walk in the park. I read that Chubby client was written is C++ and Java guys
end up using it via subprocess or something.
First, there were Python bindings for C client. Guys from Mozilla used it
early versions of Kazoo. Later they re-implemented Zookeeper protocol in pure
Python. And, after some time of trials (in which I took part), they got pretty
stable client, which can be (and is) used in production. I googled a bit -
there already exists pure Go Zookeeper client.
There are pure language bindings for Apache Kafka. Pretty young project too.
They simply cannot provide bindings, because reference implementation is
written in Scala. Forcing everyone to carry JVM in order to produce or consume
would be too much :) Instead they focused on protocol. And open source brought
implementations.
# "because whenever we want to add driver-side implementation, we would need to
ensure all of the bindings are updated."
I don't think is true. Even now you cannot break Slave<->Executor protocol,
because that would force rebuilding all executors and maybe even changing
custom executor implementations. Every change should be backwards compatible.
So go on, add something to the protocol, 3rd party implementations will catch
up eventually if they need.
If you really need to break things - this still will demand work in every
binding or maybe on user side too. Publishing new version of protocol can make
you free from this - people, if they really need and want, will implement in
the way they like. It can be Go, it can Akka, all sorts of event loops in
Python (tornado, twisted, gevent, tulip, whatever).
> Provide slave<->executor protocol
> ---------------------------------
>
> Key: MESOS-930
> URL: https://issues.apache.org/jira/browse/MESOS-930
> Project: Mesos
> Issue Type: Improvement
> Components: general
> Reporter: Nikita Vetoshkin
> Priority: Minor
>
> This ticket is the result of the discussion started in mailing list
> (http://www.mail-archive.com/[email protected]/msg05477.html).
> It would be great if Mesos provided protocol for slave<->executor
> communication additionally to currently provided c++ based language bindings.
> ocumenting
> wire protocol could open ways to implement Executors in pure python or
> golang or any other language. It could provide some benefits:
> * in Python one could use gevent which is pretty popular
> * golang has it's own eventloop builtin
> * pure language binding could save a lot of trouble bridging with
> unfriendly C++
> * building and using pure language client could be performed using native
> tools like `pip install` or `go get` without need to establish c++ dev
> environment.
> Before moving any further we need to decide, if this is actually a good thing
> to do. According to discussion in mail list - looks like it's worth doing.
> So next logical thing is to decide:
> * should protocol be something utterly new
> * should we document current protocol used by libprocess
> * should libprocess protocol be brushed a little before documenting it for
> external implementation
> While waiting for discussion I'd like to start documenting current protocol.
> Where can one do it in a way suitable for comments? RB?
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
