Hey Tim, I can see your point, and am finding it hard to think of any compelling arguments against the issue of fragmentation, but I do have a few thoughts.p
That said, I would strongly suggest taking ease-of-use and language specific code structures into consideration. A huge monolithic build system might not be a good thing either, if I'm not mistaken that's why twitter built Pants. Spark is actually a great example here, it's going to be a huge pain to publish PySpark to PYPI because of the way they structure the code, unless you force users to use a bleeding edge version of setuptools to be able to install the software. In the case of PySpark (and other libraries that require compiled dependencies, see Hadoofus on github which I collaborated on this exact issue). It's a nightmare. Projects that work well with python setuptools are projects that are just python, from my experience. That said, it's only a nightmare when you *have* precompiled dependencies that need to be part of the build process. This is no longer the case with the new mesos bindings, so why make it so hard? Take Go as another example (this is similar to installing pip dependencies from github too) - a user can simply plug in the path to a repository and away they go. It's easy, and will rapidly speed up adoption IMO. This isn't something that can easily be done if it's not in it's own repo, and the Mesos repository is pretty huge now. My opinion is largely from a users perspective. However, I would ask the question - how often does the framework API change in such a way that it breaks compatibility? Will there be a need to orchestrate releases among 20 language bindings to get a release of the core out, how often? Would it be easier for developers to implement a change and also make that change across all languages at the same time, is that even really going to happen? It's also worth considering release cycles, with all bindings being built into the core, it requires them all the be release together (or it's a git tag pain). Given that lots of the bindings are going to be (and already are) community driven, and only a few people are in charge of the Mesos release cycle (taking at least a few weeks for a release to come out) the pace for each binding has to be the same, and there's no autonomy. My personal feeling is that develop user experience isn't thought about enough is these sorts of situations, and not having a good experience either to use or work on the code is a pain and can slow down adoption. Would be interested to hear what you all think, or if you completely disagree :-) Tom. On Tuesday, 15 July 2014, Tim St Clair <[email protected]> wrote: > So... your response basically capitulates to the fragmentation argument: > > "Yes we will have binding strewn about of questionable quality that may, > or may not, work with core." > > The point that I'm trying to make is, fragmentation *is not* a good thing. > > > -------------------------------------- > Case in point - The Hadoop Ecosystem (fragmentation) > > In order for anyone to make a salient stack of any measure, vendors have > to knit together components into a stack which can then be consumed by the > masses. > > -------------------------------------- > Counterpoint - Spark (curating) libraries > > Spark bundles 1st order interface libraries as part of a curated core. > You are guaranteed that the core will inter-operate, and PySpark is given > 1st class standing. > > -------------------------------------- > > This is a bad idea, unless there is a plan to hedge the risk. > > -Tim > > > ----- Original Message ----- > > From: "yifan" <[email protected] <javascript:;>> > > To: [email protected] <javascript:;> > > Sent: Monday, July 14, 2014 7:10:34 PM > > Subject: Re: Mesos language bindings in the wild > > > > Hi Tim, > > > > I found that in zookeeper, they also separate the bindings from the core. > > > > https://cwiki.apache.org/confluence/display/ZOOKEEPER/ZKClientBindings > > > > So, IMHO, I think it should be the maintainer's responsibility to keep > > the binding in healthy state, with clear documentation of which version > > of the mesos core they supports. > > > > Yifan > > > > > > > > On 07/14/2014 11:30 AM, Tim St Clair wrote: > > > So I fear the fragmentation that can occur if we provide native > bindings > > > outside of the core, unless there is some mechanism for testing, & a > well > > > established versioning scheme. > > > > > > IMHO, priority inversion on 'versioning' should come before bindings to > > > ensure we adhere to policy. > > > > > > Thoughts? > > > > > > -Tim > > > > > > > > > ----- Original Message ----- > > >> From: "Tom Arnfeld" <[email protected] <javascript:;>> > > >> To: [email protected] <javascript:;> > > >> Cc: [email protected] <javascript:;> > > >> Sent: Friday, July 11, 2014 10:22:59 AM > > >> Subject: Re: Mesos language bindings in the wild > > >> > > >> Very exciting. I'd vote +1 for splitting them out. Especially if you > > >> look at the common way of using Go imports, just stick the project on > > >> GitHub and import it directly using "github.com/mesos/mesos-go" or > > >> similar. > > >> > > >> I guess one argument is that you have more fragmentation of the code > > >> (e.g every library has it's own copy of the protos) but I'm not sure > > >> that's a bad thing. > > >> > > >> Just my two cents. Looking forward to this! > > >> > > >>> On 11 Jul 2014, at 16:59, Thomas Rampelberg <[email protected] > <javascript:;>> wrote: > > >>> > > >>> I've started preparing the python bindings to hopefully take this > > >>> route ( https://reviews.apache.org/r/23224/ would love some reviews! > > >>> ). In fact, there is already a native python implementation of both > > >>> libprocess and the framework apis! ( > https://github.com/wickman/pesos/ > > >>> , https://github.com/wickman/compactor ). > > >>> > > >>> What are the benefits of bindings being part of the project source > > >>> itself instead of having blessed implementations like mesos-python > > >>> where the source and versioning becomes separate? I've been running > > >>> into difficulties making automake and python's build tools play > nicely > > >>> together. It seems like there'd be more flexibility in general by > > >>> splitting them out. > > >>> > > >>> > > >>>> On Thu, Jul 10, 2014 at 3:57 PM, Niklas Nielsen < > [email protected] <javascript:;>> > > >>>> wrote: > > >>>> I just wanted to clarify - native, meaning _no_ dependency to > libmesos > > >>>> and > > >>>> native to its language (only Go, only Python and so on) i.e. use the > > >>>> low-level API. > > >>>> > > >>>> Sorry for the confusion, > > >>>> Niklas > > >>>> > > >>>> > > >>>>> On 10 July 2014 15:55, Dominic Hamon <[email protected] > <javascript:;>> wrote: > > >>>>> > > >>>>> In my dream world, we wouldn't need any native bindings. I can > imagine > > >>>>> having example frameworks or starter frameworks that use the > low-level > > >>>>> API > > >>>>> (the wire protocol with protocol buffers for message passing), but > > >>>>> nothing > > >>>>> like we have that needs C or JNI, etc. > > >>>>> > > >>>>> > > >>>>> > > >>>>> > > >>>>> On Thu, Jul 10, 2014 at 3:26 PM, Niklas Nielsen < > [email protected] <javascript:;>> > > >>>>> wrote: > > >>>>> > > >>>>>> Hi all, > > >>>>>> > > >>>>>> I wanted to start a discussion around the language bindings in the > > >>>>>> wild > > >>>>>> (Go, Haskell, native Python, Go, Java and so on) and possibly get > to a > > >>>>>> strategy where we start bringing those into Mesos proper. As most > > >>>>>> things > > >>>>>> points towards, it will probably make sense to focus on the native > > >>>>>> "bindings" leveraging the low-level API. To name one candidate to > > >>>>>> start > > >>>>>> with, we are especially interested in getting Go native support in > > >>>>>> Mesos > > >>>>>> proper (and in a solid state). So Vladimir, we'd be super > thrilled to > > >>>>> start > > >>>>>> collaborating with you on your current work. > > >>>>>> > > >>>>>> We are interested to hear what thoughts you all might have on > this. > > >>>>>> > > >>>>>> Thanks, > > >>>>>> Niklas > > > > > > -- > > Gu Yifan > > > > > > -- > Cheers, > Timothy St. Clair > Red Hat Inc. >
