It seems as though you're trying to send a 'RegisterFrameworkMessage' but you're actually encoding a 'FrameworkInfo'?
The error message indicates that your binary data is not matching the expected format. On Fri, May 9, 2014 at 4:29 AM, Vladimir Vivien <vladimir.viv...@gmail.com>wrote: > Ben M, > Re built mesos from master. Made the suggested changes for headers. > Getting HTTP response back now. > > Still getting Error, even though getting 201 back (see below) > "... protobuf.hpp:400] Initialization errors: framework.user, > framework.name". > > I am sending protobuf encoded binary data. > I am using protobuf 2.5 + a Go protobuf lib. > > The String representation is > "user:"test" name:"gomes" id:<value:"gomes-framework-1" >" > > HTTP Request > POST /master/mesos.internal.RegisterFrameworkMessage HTTP/1.1 > Host: 127.0.0.1:5050 > User-Agent: Go 1.1 package http > Content-Length: 34 > Connection: Keep-Alive > Content-Type: application/x-protobuf > Libprocess-From: scheduler(1)@127.0.0.1:8080 > Accept-Encoding: gzip > > HTTP Response > 2014/05/09 07:27:11 Response Body: > HTTP/1.1 202 Accepted > Content-Length: 0 > Date: Fri, 09 May 2014 11:27:11 GMT > > > > On Tue, May 6, 2014 at 4:06 PM, Benjamin Mahler <benjamin.mah...@gmail.com > > wrote: > >> Can you build this off of the latest master code? The reviews have been >> committed! >> >> You should now use "Libprocess-From" instead of "User-Agent". >> >> E.g. >> >> Libprocess-From: scheduler@127.0.0.1:8080 >> >> Also, what is your "string" representation of the protobuf? You're >> serializing it using a protobuf library, right? It should be binary data. >> >> >> On Sun, May 4, 2014 at 7:57 AM, Vladimir Vivien < >> vladimir.viv...@gmail.com> wrote: >> >>> Slow progress from Go front: >>> I am at a point where I am testing this approach using Go. >>> >>> HTTP Sent: >>> POST /master/mesos.internal.RegisterFrameworkMessage HTTP/1.1 >>> Host: 127.0.0.1:5050 >>> User-Agent: libprocess/scheduler(1)@127.0.0.1:8080 >>> Content-Length: 34 >>> Connection: Keep-Alive >>> Content-Type: application/x-protobuf >>> Accept-Encoding: gzip >>> >>> String representation of protobuf sent: >>> user:"test" name:"gomes" id:<value:"gomes-framework-1" > >>> >>> Master console response >>> W0504 10:53:48.263000 3960 protobuf.hpp:399] Initialization errors: >>> framework.user, framework.name >>> >>> Before I delve in to CPP to see what's going on, hope someone can let me >>> know what I am doing wrong. >>> >>> PS. I havent applied the two patches mentioned here. >>> >>> vladimir.vivien >>> >>> >>> >>> On Wed, Apr 30, 2014 at 1:59 AM, Benjamin Mahler < >>> benjamin.mah...@gmail.com> wrote: >>> >>>> Brian, I'm curious whether most python HTTP libraries support using >>>> custom HTTP request headers. >>>> >>>> We'll land benh's patches to send 202 responses in 0.19.0: >>>> >>>> https://reviews.apache.org/r/20276/ >>>> https://reviews.apache.org/r/20277/ >>>> >>>> Once these get committed, you should be able to get something working >>>> nicely. This will allow libraries like "pesos" to use a custom header >>>> "Libprocess-From" to receive 202 responses for messages. >>>> >>>> We'd also like to get some more foundation laid for the lower-level >>>> API, I'll let benh comment on what else is needed here: >>>> >>>> https://reviews.apache.org/r/20309/ >>>> >>>> >>>> On Sun, Apr 27, 2014 at 9:23 PM, Vetoshkin Nikita < >>>> nikita.vetosh...@gmail.com> wrote: >>>> >>>>> That's great news indeed! >>>>> Maybe there should be a place on mesos site mentioning such efforts? >>>>> >>>>> >>>>> On Mon, Apr 28, 2014 at 5:24 AM, Brian Wickman <wick...@gmail.com> >>>>> wrote: >>>>> >>>>> > To my knowledge there is no Zookeeper Group implementation for Go >>>>> yet. >>>>> > There do however appear to be Go zookeeper >>>>> > bindings<https://github.com/samuel/go-zookeeper>. >>>>> > From my day of spelunking through code, it seems that probably the >>>>> most >>>>> > challenging thing will be getting a ZookeeperMasterDetector >>>>> implementation >>>>> > working correctly to enable Scheduler "high availability" mode for >>>>> these >>>>> > other languages. >>>>> > >>>>> > Java/Python/C++ have the benefit of preexisting Group implementations >>>>> > (java< >>>>> > >>>>> https://github.com/twitter/commons/blob/master/src/java/com/twitter/common/zookeeper/Group.java >>>>> > > >>>>> > , python< >>>>> > >>>>> https://github.com/twitter/commons/blob/master/src/python/twitter/common/zookeeper/group/kazoo_group.py >>>>> > > >>>>> > , c++< >>>>> > >>>>> https://git-wip-us.apache.org/repos/asf?p=mesos.git;a=blob;f=src/zookeeper/group.cpp;h=70972980dd25618c76eb36e0a31d060c096e36dc;hb=HEAD >>>>> > >) >>>>> > which are the foundation for ZK master detection. An intermediate >>>>> option >>>>> > could be to pull the ZookeeperMasterDetectorProcess out into a >>>>> standalone >>>>> > binary and have the native implementation talk to it as a sort of >>>>> proxy to >>>>> > figure out who the leading master is. But that's not nearly as >>>>> satisfying >>>>> > as a pure language implementation. >>>>> > >>>>> > ~brian >>>>> > >>>>> > >>>>> > On Sun, Apr 27, 2014 at 3:22 PM, Vladimir Vivien >>>>> > <vladimir.viv...@gmail.com>wrote: >>>>> > >>>>> > > It must be the right time for this. I started the same effort for >>>>> Go. >>>>> > > I am following Kevin's example and Ben's well-documented email (in >>>>> this >>>>> > > thread). >>>>> > > Very early but should have some proof-of-concept working soon. >>>>> > > >>>>> > > https://github.com/vladimirvivien/ionos >>>>> > > >>>>> > > >>>>> > > On Sun, Apr 27, 2014 at 6:07 PM, Tom Arnfeld <t...@duedil.com> >>>>> wrote: >>>>> > > >>>>> > >> (Sorry to jump into this thread) >>>>> > >> >>>>> > >> I have to say, this is very exciting! It only became apparent to >>>>> me how >>>>> > >> painful it is having to compile the mesos egg into a Python >>>>> framework >>>>> > the >>>>> > >> past week >>>>> > >> while writing one, especially when running the framework on a >>>>> different >>>>> > >> architecture to the one running the executor! >>>>> > >> >>>>> > >> Looking forward to being able to use this. :-) >>>>> > >> >>>>> > >> It’d be awesome if the API was near identical, so one could switch >>>>> > >> between `mesos` and `pesos` easily… >>>>> > >> >>>>> > >> Tom. >>>>> > >> >>>>> > >> On 27 Apr 2014, at 22:42, Brian Wickman <wick...@gmail.com> >>>>> wrote: >>>>> > >> >>>>> > >> > And I've started a skeleton implementation of the Mesos >>>>> framework API >>>>> > at >>>>> > >> > https://github.com/wickman/pesos >>>>> > >> > >>>>> > >> > While I vendored the translated protobuf, it would be great to >>>>> tackle >>>>> > >> > https://issues.apache.org/jira/browse/MESOS-857 at some point. >>>>> > >> > >>>>> > >> > >>>>> > >> > On Sun, Apr 27, 2014 at 12:50 PM, Brian Wickman < >>>>> wick...@gmail.com> >>>>> > >> wrote: >>>>> > >> > >>>>> > >> >> I've implemented a pure python version of the basic >>>>> > >> >> Process/ProtobufProcess mechanics and wire protocol. I >>>>> haven't gone >>>>> > >> so far >>>>> > >> >> as Kevin and tried to register a framework or maybe talk to a >>>>> > >> replicated >>>>> > >> >> log, but it shouldn't be much more work: >>>>> > >> >> >>>>> > >> >> https://github.com/wickman/compactor >>>>> > >> >> >>>>> > >> >> Zero documentation but you can read the tests for a general >>>>> idea of >>>>> > >> what's >>>>> > >> >> going on. >>>>> > >> >> >>>>> > >> >> cheers, >>>>> > >> >> brian >>>>> > >> >> >>>>> > >> >> >>>>> > >> >> >>>>> > >> >> On Sun, Apr 13, 2014 at 10:28 PM, Kevin Sweeney < >>>>> > >> kevin.t.swee...@gmail.com >>>>> > >> >>> wrote: >>>>> > >> >> >>>>> > >> >>> I've got the start of a JVM verson on github - it can >>>>> currently >>>>> > >> register >>>>> > >> >>> a framework and parse the response. Client-side I still need >>>>> to >>>>> > >> figure out >>>>> > >> >>> how to properly configure Keep-Alive. Servlet can dispatch >>>>> messages >>>>> > to >>>>> > >> >>> message handlers in a type-safe way and returns a 202 for >>>>> messages >>>>> > >> it's >>>>> > >> >>> going to handle. Code's a mess currently. >>>>> > >> >>> >>>>> > >> >>> https://github.com/kevints/mesos-framework-api >>>>> > >> >>> >>>>> > >> >>> >>>>> > >> >>> On Fri, Apr 11, 2014 at 7:12 PM, Benjamin Hindman < >>>>> > >> b...@eecs.berkeley.edu >>>>> > >> >>>> wrote: >>>>> > >> >>> >>>>> > >> >>>> First, my apologies for getting to this party so late. It's >>>>> great >>>>> > to >>>>> > >> see >>>>> > >> >>>> people interested in helping create native-language Mesos >>>>> > libraries. >>>>> > >> >>>> >>>>> > >> >>>> Vladimir: my presentation was definitely referring to the the >>>>> > >> low-level >>>>> > >> >>>> protocol between master, framework (scheduler), slave, and >>>>> > >> executors. I'll >>>>> > >> >>>> do my best here to clarify how the current protocol works >>>>> and what >>>>> > >> we need >>>>> > >> >>>> to do to get it to the point where we can write >>>>> native-language >>>>> > >> libraries. >>>>> > >> >>>> (Eventually it would be great to move some of this into >>>>> > >> documentation as >>>>> > >> >>>> necessary.) >>>>> > >> >>>> >>>>> > >> >>>> As Nikita pointed out, the protocol is currently >>>>> "HTTP-like". As my >>>>> > >> >>>> presentation describes, think actors and one-way message >>>>> passing >>>>> > when >>>>> > >> >>>> considering how the protocol works. >>>>> > >> >>>> >>>>> > >> >>>> To send a message an actor POSTs an HTTP request where the >>>>> actor >>>>> > >> that is >>>>> > >> >>>> supposed to receive the message is the first component of the >>>>> > >> request path >>>>> > >> >>>> and the name of the message is the remaining part of the >>>>> path. To >>>>> > >> >>>> distinguish one of these "messages" from a normal HTTP >>>>> request we >>>>> > >> look to >>>>> > >> >>>> see if the 'User-Agent' is 'libprocess/...'. For example: >>>>> > >> >>>> >>>>> > >> >>>> POST /master/mesos.internal.RegisterFrameworkMessage HTTP/1.1 >>>>> > >> >>>> User-Agent: libprocess/scheduler(1)@10.0.1.7:53523 >>>>> > >> >>>> >>>>> > >> >>>> ... represents a message with the name >>>>> > >> >>>> 'mesos.internal.RegisterFrameworkMessage' destined for the >>>>> actor >>>>> > >> 'master' >>>>> > >> >>>> coming from the actor 'scheduler(1)' at 10.0.1.7:53523. If >>>>> the >>>>> > >> 'master' >>>>> > >> >>>> actor were to send a message back it would look something >>>>> like >>>>> > this: >>>>> > >> >>>> >>>>> > >> >>>> POST /scheduler(1)/mesos.internal.FrameworkRegisteredMessage >>>>> > HTTP/1.1 >>>>> > >> >>>> User-Agent: libprocess/master@10.0.1.7:5050 >>>>> > >> >>>> >>>>> > >> >>>> So, one-way message passing via HTTP POST. >>>>> > >> >>>> >>>>> > >> >>>> The message data is captured as the body of the HTTP request >>>>> (which >>>>> > >> can >>>>> > >> >>>> be specified using _either_ Content-Length or a >>>>> Transfer-Encoding, >>>>> > >> and as >>>>> > >> >>>> Nikita points out we use chunked transfer encoding >>>>> internally). The >>>>> > >> data is >>>>> > >> >>>> arbitrary and the actor ultimately decides how it wants to >>>>> "parse" >>>>> > >> it. In >>>>> > >> >>>> Mesos, 99% of our messages use serialized protobufs, but we >>>>> also >>>>> > >> send a few >>>>> > >> >>>> messages with just arbitrary data. All this really means is >>>>> that >>>>> > >> knowing >>>>> > >> >>>> the actor and message name is not enough, you also need to >>>>> know >>>>> > what >>>>> > >> the >>>>> > >> >>>> body type is supposed to be for that message. In the future >>>>> we'll >>>>> > >> probably >>>>> > >> >>>> enable messages with either JSON or serialized protobuf[1] >>>>> ... for >>>>> > >> now, >>>>> > >> >>>> just serialized protobuf. >>>>> > >> >>>> >>>>> > >> >>>> Okay, so where does this break down when trying to do this >>>>> > >> >>>> language-natively? I've had some of this in the works and >>>>> this >>>>> > >> conversation >>>>> > >> >>>> has motivated me to publish some reviews addressing the >>>>> issues: >>>>> > >> >>>> >>>>> > >> >>>> (1) We'll need to return a response if one plans to use a >>>>> native >>>>> > HTTP >>>>> > >> >>>> library since it'll expect request/response. >>>>> > >> >>>> https://reviews.apache.org/r/20276 introduces responding >>>>> with a >>>>> > '202 >>>>> > >> >>>> Accepted' for these messages (from the HTTP specification, a >>>>> '202 >>>>> > >> >>>> Accepted': "The request has been accepted for processing, >>>>> but the >>>>> > >> >>>> processing has not been completed. The request might or >>>>> might not >>>>> > >> >>>> eventually be acted upon, as it might be disallowed when >>>>> processing >>>>> > >> >>>> actually takes place. There is no facility for re-sending a >>>>> status >>>>> > >> code >>>>> > >> >>>> from an asynchronous operation such as this."). >>>>> > >> >>>> >>>>> > >> >>>> (2) Most HTTP libraries will set their 'User-Agent' >>>>> themselves, so >>>>> > >> >>>> https://reviews.apache.org/r/20277 introduces a >>>>> 'libprocess-from' >>>>> > >> >>>> header that works similar to User-Agent. There is still some >>>>> > cleanup >>>>> > >> I'd >>>>> > >> >>>> love to do around stringification of PIDs (the underlying >>>>> type >>>>> > Mesos >>>>> > >> uses >>>>> > >> >>>> for remote actors, inspired by Erlang). Until then, the >>>>> > >> 'libprocess-from' >>>>> > >> >>>> string is unfortunately esoteric (see the test). >>>>> > >> >>>> >>>>> > >> >>>> The combination of these two patches should make sending and >>>>> > >> receiving >>>>> > >> >>>> messages straightforward. However, we still plan to expose >>>>> the >>>>> > >> low-level >>>>> > >> >>>> Event and Call protobuf messages and that will be the >>>>> preferred >>>>> > >> approach >>>>> > >> >>>> for building a native-language library. Follow along at >>>>> > >> >>>> https://issues.apache.org/jira/browse/MESOS-1127 for more >>>>> details. >>>>> > >> (To >>>>> > >> >>>> be clear, you'd still be able to implement native-language >>>>> > libraries >>>>> > >> with >>>>> > >> >>>> the patches above but we'll be deprecating the protobufs >>>>> you'd be >>>>> > >> using in >>>>> > >> >>>> favor of Event and Call protobufs instead. If you're eager >>>>> to get >>>>> > >> that >>>>> > >> >>>> going before Event and Call are committed I'm happy to >>>>> discuss the >>>>> > >> existing >>>>> > >> >>>> protobufs in more detail.) >>>>> > >> >>>> >>>>> > >> >>>> I hope this helps. >>>>> > >> >>>> >>>>> > >> >>>> Ben. >>>>> > >> >>>> >>>>> > >> >>>> >>>>> > >> >>>> On Fri, Apr 11, 2014 at 4:54 AM, Vladimir Vivien < >>>>> > >> >>>> vladimir.viv...@gmail.com> wrote: >>>>> > >> >>>> >>>>> > >> >>>>> Nikita >>>>> > >> >>>>> Thanks for the JIRA. >>>>> > >> >>>>> >>>>> > >> >>>>> >>>>> > >> >>>>> On Wed, Apr 9, 2014 at 2:16 PM, Vetoshkin Nikita < >>>>> > >> >>>>> nikita.vetosh...@gmail.com >>>>> > >> >>>>>> wrote: >>>>> > >> >>>>> >>>>> > >> >>>>>> BTW, there is also somehow related ticket >>>>> > >> >>>>>> https://issues.apache.org/jira/browse/MESOS-930 >>>>> > >> >>>>>> >>>>> > >> >>>>>> >>>>> > >> >>>>>> On Wed, Apr 9, 2014 at 9:54 PM, Benjamin Mahler >>>>> > >> >>>>>> <benjamin.mah...@gmail.com>wrote: >>>>> > >> >>>>>> >>>>> > >> >>>>>>>> >>>>> > >> >>>>>>>> I thought the low-level api being referred in the >>>>> > >> >>>>>>>> video had to do with communication between master and >>>>> > >> >>>>>> framework|executor >>>>> > >> >>>>>>>> for scheduling. But, it's really administrative. I >>>>> thought >>>>> > that >>>>> > >> >>>>> would >>>>> > >> >>>>>>>> have been an opportunity for a Go binding that did not >>>>> require >>>>> > >> >>>>> the C++ >>>>> > >> >>>>>>>> libraries. >>>>> > >> >>>>>>>> >>>>> > >> >>>>>>> >>>>> > >> >>>>>>> Vladimir, the low-level API referred to in the talk is >>>>> exactly >>>>> > >> what >>>>> > >> >>>>>> you're >>>>> > >> >>>>>>> interpreting, it is for communication between master and >>>>> > >> scheduler, >>>>> > >> >>>>> and >>>>> > >> >>>>>>> slave and executor. You could definitely build pure go >>>>> bindings >>>>> > as >>>>> > >> >>>>> you >>>>> > >> >>>>>>> described, just not with JSON. >>>>> > >> >>>>>>> >>>>> > >> >>>>>>> Forget I mentioned anything about the administrative >>>>> endpoints >>>>> > and >>>>> > >> >>>>> JSON, >>>>> > >> >>>>>> as >>>>> > >> >>>>>>> I see that's leading to confusion. ;) >>>>> > >> >>>>>>> >>>>> > >> >>>>>>> On Wed, Apr 9, 2014 at 3:39 AM, Vladimir Vivien >>>>> > >> >>>>>>> <vladimir.viv...@gmail.com>wrote: >>>>> > >> >>>>>>> >>>>> > >> >>>>>>>> Ben, >>>>> > >> >>>>>>>> Thank you for clarifying. I thought the low-level api >>>>> being >>>>> > >> >>>>> referred in >>>>> > >> >>>>>>> the >>>>> > >> >>>>>>>> video had to do with communication between master and >>>>> > >> >>>>>> framework|executor >>>>> > >> >>>>>>>> for scheduling. But, it's really administrative. I >>>>> thought >>>>> > that >>>>> > >> >>>>> would >>>>> > >> >>>>>>>> have been an opportunity for a Go binding that did not >>>>> require >>>>> > >> >>>>> the C++ >>>>> > >> >>>>>>>> libraries. >>>>> > >> >>>>>>>> >>>>> > >> >>>>>>>> Thanks anyway. >>>>> > >> >>>>>>>> >>>>> > >> >>>>>>>> >>>>> > >> >>>>>>>> >>>>> > >> >>>>>>>> >>>>> > >> >>>>>>>> >>>>> > >> >>>>>>>> >>>>> > >> >>>>>>>> On Tue, Apr 8, 2014 at 4:52 PM, Benjamin Mahler >>>>> > >> >>>>>>>> <benjamin.mah...@gmail.com>wrote: >>>>> > >> >>>>>>>> >>>>> > >> >>>>>>>>> Sorry, I was not referring to implementing a scheduler >>>>> via >>>>> > JSON >>>>> > >> >>>>>> instead >>>>> > >> >>>>>>>> of >>>>> > >> >>>>>>>>> protobuf, in theory that would be possible but there >>>>> has been >>>>> > no >>>>> > >> >>>>>>> planning >>>>> > >> >>>>>>>>> in this area. Sorry for the confusion. >>>>> > >> >>>>>>>>> >>>>> > >> >>>>>>>>> I was referring to administrative endpoints. For >>>>> example, >>>>> > >> >>>>> kicking a >>>>> > >> >>>>>>>>> framework out or telling the master a slave is needs to >>>>> be >>>>> > >> >>>>> repaired. >>>>> > >> >>>>>>>> These >>>>> > >> >>>>>>>>> endpoints may rely on the ability to convert JSON to >>>>> internal >>>>> > >> >>>>>>> protobufs. >>>>> > >> >>>>>>>>> >>>>> > >> >>>>>>>>> Can you clarify what you're looking to do? Are you >>>>> looking to >>>>> > >> >>>>>> implement >>>>> > >> >>>>>>>> an >>>>> > >> >>>>>>>>> API in Go that communicates with JSON instead of >>>>> serialized >>>>> > >> >>>>> protobuf? >>>>> > >> >>>>>>>>> >>>>> > >> >>>>>>>>> On Tue, Apr 8, 2014 at 1:19 PM, Vladimir Vivien >>>>> > >> >>>>>>>>> <vladimir.viv...@gmail.com>wrote: >>>>> > >> >>>>>>>>> >>>>> > >> >>>>>>>>>> Ben, >>>>> > >> >>>>>>>>>> That is exactly what I am asking. >>>>> > >> >>>>>>>>>> Is that something coming up soon, is there a JIRA I >>>>> can look >>>>> > >> >>>>> at? >>>>> > >> >>>>>>>>>> I wanna get early start on a native json Go api or >>>>> even help >>>>> > >> >>>>> out if >>>>> > >> >>>>>>>>>> possible. >>>>> > >> >>>>>>>>>> >>>>> > >> >>>>>>>>>> >>>>> > >> >>>>>>>>>> On Tue, Apr 8, 2014 at 3:25 PM, Benjamin Mahler >>>>> > >> >>>>>>>>>> <benjamin.mah...@gmail.com>wrote: >>>>> > >> >>>>>>>>>> >>>>> > >> >>>>>>>>>>> +vinod, benh >>>>> > >> >>>>>>>>>>> >>>>> > >> >>>>>>>>>>> Hey Vladimir, there will be some authenticated REST >>>>> > >> >>>>> endpoints at >>>>> > >> >>>>>>> some >>>>> > >> >>>>>>>>>>> point, there is some work in this area underway. >>>>> > >> >>>>>>>>>>> >>>>> > >> >>>>>>>>>>> We have the ability to encode protobuf messages as >>>>> JSON, so >>>>> > >> >>>>> the >>>>> > >> >>>>>>> plan >>>>> > >> >>>>>>>>> was >>>>> > >> >>>>>>>>>> to >>>>> > >> >>>>>>>>>>> have any REST endpoints directly use JSON to send us >>>>> > >> >>>>> protobuf >>>>> > >> >>>>>>>> messages. >>>>> > >> >>>>>>>>>> I'm >>>>> > >> >>>>>>>>>>> not sure if this is what you're asking though? >>>>> > >> >>>>>>>>>>> >>>>> > >> >>>>>>>>>>> >>>>> > >> >>>>>>>>>>> On Tue, Apr 8, 2014 at 11:13 AM, Vetoshkin Nikita < >>>>> > >> >>>>>>>>>>> nikita.vetosh...@gmail.com> wrote: >>>>> > >> >>>>>>>>>>> >>>>> > >> >>>>>>>>>>>> I'm not a mesos guy, just very curious. But in my >>>>> opinion >>>>> > >> >>>>> - I >>>>> > >> >>>>>>> doubt >>>>> > >> >>>>>>>>> it, >>>>> > >> >>>>>>>>>>>> HTTP is synchronous request-response protocol. Mesos >>>>> needs >>>>> > >> >>>>>>>> something >>>>> > >> >>>>>>>>>> more >>>>> > >> >>>>>>>>>>>> robust for message passing. Websockets anyone? :) >>>>> > >> >>>>>>>>>>>> >>>>> > >> >>>>>>>>>>>> >>>>> > >> >>>>>>>>>>>> On Tue, Apr 8, 2014 at 10:08 PM, Vladimir Vivien >>>>> > >> >>>>>>>>>>>> <vladimir.viv...@gmail.com>wrote: >>>>> > >> >>>>>>>>>>>> >>>>> > >> >>>>>>>>>>>>> Ben / Nikita >>>>> > >> >>>>>>>>>>>>> Thanks for the pointers. >>>>> > >> >>>>>>>>>>>>> So, (without digging yet) is it a fair summary to >>>>> say >>>>> > >> >>>>> that >>>>> > >> >>>>>>>>> libprocess >>>>> > >> >>>>>>>>>>>> wraps >>>>> > >> >>>>>>>>>>>>> protobufs-encoded calls and push them over HTTP to >>>>> > >> >>>>>>> master/slaves >>>>> > >> >>>>>>>> ? >>>>> > >> >>>>>>>>>> Will >>>>> > >> >>>>>>>>>>>>> protobuf (eventually) be supplanted by direct HTTP >>>>> via >>>>> > >> >>>>> REST >>>>> > >> >>>>>> or >>>>> > >> >>>>>>>>>> similar >>>>> > >> >>>>>>>>>>> ? >>>>> > >> >>>>>>>>>>>>> >>>>> > >> >>>>>>>>>>>>> >>>>> > >> >>>>>>>>>>>>> On Mon, Apr 7, 2014 at 2:54 PM, Vetoshkin Nikita < >>>>> > >> >>>>>>>>>>>>> nikita.vetosh...@gmail.com >>>>> > >> >>>>>>>>>>>>>> wrote: >>>>> > >> >>>>>>>>>>>>> >>>>> > >> >>>>>>>>>>>>>> Or, just to get to know - you can take tcpdump and >>>>> > >> >>>>> take a >>>>> > >> >>>>>>> look >>>>> > >> >>>>>>>> :) >>>>> > >> >>>>>>>>>>>>>> >>>>> > >> >>>>>>>>>>>>>> I personally wouldn't call that HTTP. Something >>>>> > >> >>>>> "HTTP-like" >>>>> > >> >>>>>>>> would >>>>> > >> >>>>>>>>>>>>> describe >>>>> > >> >>>>>>>>>>>>>> it better. Because it's not request-response. It's >>>>> > >> >>>>> just >>>>> > >> >>>>>>> message >>>>> > >> >>>>>>>>>>>> passing, >>>>> > >> >>>>>>>>>>>>> no >>>>> > >> >>>>>>>>>>>>>> need to wait for the answer - send new message one >>>>> > >> >>>>> after >>>>> > >> >>>>>>>> another. >>>>> > >> >>>>>>>>>>> Every >>>>> > >> >>>>>>>>>>>>>> message is POST with address and message type >>>>> encoded >>>>> > >> >>>>> in >>>>> > >> >>>>>> URI: >>>>> > >> >>>>>>>>> POST >>>>> > >> >>>>>>>>>>>>>> /executor(1)/mesos.internal.RunTaskMessage. Sender >>>>> is >>>>> > >> >>>>>> encoded >>>>> > >> >>>>>>>> in >>>>> > >> >>>>>>>>>>>>> User-Agent >>>>> > >> >>>>>>>>>>>>>> header, e.g: libprocess/slave(1)@127.0.0.1:5051. >>>>> Body >>>>> > >> >>>>>>> contains >>>>> > >> >>>>>>>>>>>> protobuf >>>>> > >> >>>>>>>>>>>>>> message, Transfer-Encoding is always "chunked". >>>>> > >> >>>>>>>>>>>>>> >>>>> > >> >>>>>>>>>>>>>> >>>>> > >> >>>>>>>>>>>>>> On Mon, Apr 7, 2014 at 10:42 PM, Benjamin Mahler >>>>> > >> >>>>>>>>>>>>>> <benjamin.mah...@gmail.com>wrote: >>>>> > >> >>>>>>>>>>>>>> >>>>> > >> >>>>>>>>>>>>>>> Unfortunately you will need to learn this by >>>>> > >> >>>>> looking at >>>>> > >> >>>>>> the >>>>> > >> >>>>>>>>> code >>>>> > >> >>>>>>>>>> in >>>>> > >> >>>>>>>>>>>>>>> libprocess, as the message passing format is not >>>>> > >> >>>>>> explicitly >>>>> > >> >>>>>>>>>>>> documented >>>>> > >> >>>>>>>>>>>>> at >>>>> > >> >>>>>>>>>>>>>>> the current time. >>>>> > >> >>>>>>>>>>>>>>> >>>>> > >> >>>>>>>>>>>>>>> Start with calls like ProtobufProcess::send() and >>>>> > >> >>>>> dig >>>>> > >> >>>>>> your >>>>> > >> >>>>>>>> way >>>>> > >> >>>>>>>>>>> down. >>>>> > >> >>>>>>>>>>>>>>> >>>>> > >> >>>>>>>>>>>>>>> >>>>> > >> >>>>>>>>>>>>>>> On Sat, Apr 5, 2014 at 7:52 AM, Vladimir Vivien >>>>> > >> >>>>>>>>>>>>>>> <vladimir.viv...@gmail.com>wrote: >>>>> > >> >>>>>>>>>>>>>>> >>>>> > >> >>>>>>>>>>>>>>>> I was watching this video from >>>>> > >> >>>>>>>>>>>>>>>> >>>>> https://www.youtube.com/watch?v=n5GT7OFSh58fromBen >>>>> > >> >>>>>>> where >>>>> > >> >>>>>>>> he >>>>> > >> >>>>>>>>>>>> talked >>>>> > >> >>>>>>>>>>>>>>>> about the wire protocol for Mesos being done in >>>>> > >> >>>>>>>>>>>>>>>> HTTP. >>>>> > >> >>>>>>>>>>>>>>>> >>>>> > >> >>>>>>>>>>>>>>>> Where can I learn about the low-level wire >>>>> > >> >>>>> protocol >>>>> > >> >>>>>>> either >>>>> > >> >>>>>>>> in >>>>> > >> >>>>>>>>>>>>>>> documentation >>>>> > >> >>>>>>>>>>>>>>>> or browsing through the code. >>>>> > >> >>>>>>>>>>>>>>>> >>>>> > >> >>>>>>>>>>>>>>>> Thanks. >>>>> > >> >>>>>>>>>>>>>>>> >>>>> > >> >>>>>>>>>>>>>>>> -- >>>>> > >> >>>>>>>>>>>>>>>> Vladimir Vivien >>>>> > >> >>>>>>>>>>>>>>>> >>>>> > >> >>>>>>>>>>>>>>> >>>>> > >> >>>>>>>>>>>>>> >>>>> > >> >>>>>>>>>>>>> >>>>> > >> >>>>>>>>>>>>> >>>>> > >> >>>>>>>>>>>>> >>>>> > >> >>>>>>>>>>>>> -- >>>>> > >> >>>>>>>>>>>>> Vladimir Vivien >>>>> > >> >>>>>>>>>>>>> >>>>> > >> >>>>>>>>>>>> >>>>> > >> >>>>>>>>>>> >>>>> > >> >>>>>>>>>> >>>>> > >> >>>>>>>>>> >>>>> > >> >>>>>>>>>> >>>>> > >> >>>>>>>>>> -- >>>>> > >> >>>>>>>>>> Vladimir Vivien >>>>> > >> >>>>>>>>>> >>>>> > >> >>>>>>>>> >>>>> > >> >>>>>>>> >>>>> > >> >>>>>>>> >>>>> > >> >>>>>>>> >>>>> > >> >>>>>>>> -- >>>>> > >> >>>>>>>> Vladimir Vivien >>>>> > >> >>>>>>>> >>>>> > >> >>>>>>> >>>>> > >> >>>>>> >>>>> > >> >>>>> >>>>> > >> >>>>> >>>>> > >> >>>>> >>>>> > >> >>>>> -- >>>>> > >> >>>>> Vladimir Vivien >>>>> > >> >>>>> >>>>> > >> >>>> >>>>> > >> >>>> >>>>> > >> >>> >>>>> > >> >> >>>>> > >> >>>>> > >> >>>>> > > >>>>> > > >>>>> > > -- >>>>> > > Vladimir Vivien >>>>> > > >>>>> > >>>>> >>>> >>>> >>> >>> >>> -- >>> Vladimir Vivien >>> >> >> > > > -- > Vladimir Vivien >
