Awesome - we'll be running RC2 through our CI env and let you know the outcome soon as we know. Thanks!
*Marco Massenzio* *Distributed Systems Engineerhttp://codetrips.com <http://codetrips.com>* On Tue, Sep 1, 2015 at 11:42 AM, Vinod Kone <[email protected]> wrote: > My only concern is that if we decide to change the protobuf -> json > conversion for int64 (from json number to string), we should do that in the > scheduler http api as well (Resource protobuf uses int64 for ports). > > But since the scheduler http api is labeled beta for 0.24, we can still > change the semantics in 0.25. > > So, I'll go ahead and call the vote for RC2 today. > > On Tue, Sep 1, 2015 at 11:05 AM, Marco Massenzio <[email protected]> > wrote: > >> Cool - I'll ping Joseph on that one. >> >> (the -1 from Nik was related to the known ROOT test issues that -if >> memory serves- we agreed were non-blocking: I'll follow up with him too) >> >> *Marco Massenzio* >> >> *Distributed Systems Engineerhttp://codetrips.com <http://codetrips.com>* >> >> On Tue, Sep 1, 2015 at 10:49 AM, Vinod Kone <[email protected]> wrote: >> >>> Thanks for the nudge Marco. >>> >>> There was a binding -1 from Niklas. >>> >>> I'm planning to cut an RC2. The cherry picks I've selected so far are in >>> MESOS-2562 <https://issues.apache.org/jira/browse/MESOS-2562>. >>> >>> The only one I'm currently waiting on to get a resolution for is >>> https://issues.apache.org/jira/browse/MESOS-3345. >>> >>> On Tue, Sep 1, 2015 at 10:44 AM, Marco Massenzio <[email protected]> >>> wrote: >>> >>>> Hey guys, >>>> >>>> just a quick note to bring back the conversation on track to the >>>> 0.24-RC1 release. >>>> Is my understanding correct that there are currently no binding -1's? >>>> >>>> @Vinod: what do you think, are we good to release? >>>> >>>> Thanks! >>>> >>>> *Marco Massenzio* >>>> >>>> *Distributed Systems Engineerhttp://codetrips.com >>>> <http://codetrips.com>* >>>> >>>> On Tue, Sep 1, 2015 at 1:49 AM, Dario Rexin <[email protected]> wrote: >>>> >>>>> One more question. From the Mesos code it doesn’t look like events are >>>>> being split or combined, so given I have a client that gives me access to >>>>> the individual chunks, is it safe to assume that each chunk contains >>>>> exactly one event? Because that would make parsing the events a lot easier >>>>> for me. >>>>> >>>>> Thanks, >>>>> Dario >>>>> >>>>> On Sep 1, 2015, at 8:42 AM, [email protected] wrote: >>>>> >>>>> Hi Vinod, >>>>> >>>>> thanks for the explanation, I got it now. >>>>> >>>>> Thanks, >>>>> Dario >>>>> >>>>> On 31.08.2015, at 23:47, Vinod Kone <[email protected]> wrote: >>>>> >>>>> I think you might be confused with the HTTP chunked encoding and >>>>> RecordIO encoding. Most HTTP client libraries dechunk the stream before >>>>> presenting it to the application. So the application needs to know the >>>>> encoding of the dechunked data to be able to process it. >>>>> >>>>> In Mesos's case, the server (master here) can encode it in JSON or >>>>> Protobuf. We wanted to have a consistent way to encode both these formats >>>>> and Record-IO format was the one we settled on. Note that this format is >>>>> also used by the Twitter streaming API >>>>> <https://dev.twitter.com/streaming/overview/processing> (see >>>>> delimited messages section). >>>>> >>>>> HTH, >>>>> >>>>> On Mon, Aug 31, 2015 at 2:09 PM, Dario Rexin <[email protected]> >>>>> wrote: >>>>> >>>>>> Hi Vino, >>>>>> >>>>>> On Aug 31, 2015, at 9:36 PM, Vinod Kone <[email protected]> wrote: >>>>>> >>>>>> Hi Dario, >>>>>> >>>>>> Can you test with "curl --no-buffer" option? Looks like your stdout >>>>>> might be line-buffered. >>>>>> >>>>>> >>>>>> that did the trick, thanks! >>>>>> >>>>>> >>>>>> The reason we used record-io formatting is to be consistent in how we >>>>>> stream protobuf and json encoded data. >>>>>> >>>>>> >>>>>> How does simple chunked encoding prevent you from doing this? >>>>>> >>>>>> Thanks, >>>>>> Dario >>>>>> >>>>>> On Fri, Aug 28, 2015 at 2:04 PM, <[email protected]> wrote: >>>>>> >>>>>>> Anand, >>>>>>> >>>>>>> thanks for the explanation. I didn't think about the case when you >>>>>>> have to split a message, now it makes sense. >>>>>>> >>>>>>> But the case I observed with curl is still weird. Even when >>>>>>> splitting a message, it should still receive both parts almost at the >>>>>>> same >>>>>>> time. Do you have any idea why it could behave like this? >>>>>>> >>>>>>> On 28.08.2015, at 21:31, Anand Mazumdar <[email protected]> wrote: >>>>>>> >>>>>>> Dario, >>>>>>> >>>>>>> Most HTTP libraries/parsers ( including one that Mesos uses >>>>>>> internally ) provide a way to specify a default size of each chunk. If a >>>>>>> Mesos Event is too big , it would get split into smaller chunks and >>>>>>> vice-versa. >>>>>>> >>>>>>> -anand >>>>>>> >>>>>>> On Aug 28, 2015, at 11:51 AM, [email protected] wrote: >>>>>>> >>>>>>> Anand, >>>>>>> >>>>>>> in the example from my first mail you can see that curl prints the >>>>>>> size of a message and then waits for the next message and only when it >>>>>>> receives that message it will print the prior message plus the size of >>>>>>> the >>>>>>> next message, but not the actual message. >>>>>>> >>>>>>> What's the benefit of encoding multiple messages in a single chunk? >>>>>>> You could simply create a single chunk per event. >>>>>>> >>>>>>> Cheers, >>>>>>> Dario >>>>>>> >>>>>>> On 28.08.2015, at 19:43, Anand Mazumdar <[email protected]> wrote: >>>>>>> >>>>>>> Dario, >>>>>>> >>>>>>> Can you shed a bit more light on what you still find puzzling about >>>>>>> the CURL behavior after my explanation ? >>>>>>> >>>>>>> PS: A single HTTP chunk can have 0 or more Mesos (Scheduler API) >>>>>>> Events. So in your example, the first chunk had complete information >>>>>>> about >>>>>>> the first “event”, followed by partial information about the subsequent >>>>>>> event from another chunk. >>>>>>> >>>>>>> As for the benefit of using RecordIO format here, how else do you >>>>>>> think we could have de-marcated two events in the response ? >>>>>>> >>>>>>> -anand >>>>>>> >>>>>>> >>>>>>> On Aug 28, 2015, at 10:01 AM, [email protected] wrote: >>>>>>> >>>>>>> Anand, >>>>>>> >>>>>>> thanks for the explanation. I'm still a little puzzled why curl >>>>>>> behaves so strange. I will check how other client behave as soon as I >>>>>>> have >>>>>>> a chance. >>>>>>> >>>>>>> Vinod, >>>>>>> >>>>>>> what exactly is the benefit of using recordio here? Doesn't it make >>>>>>> the content-type somewhat wrong? If I send 'Accept: application/json' >>>>>>> and >>>>>>> receive 'Content-Type: application/json', I actually expect to receive >>>>>>> only >>>>>>> json in the message. >>>>>>> >>>>>>> Thanks, >>>>>>> Dario >>>>>>> >>>>>>> On 28.08.2015, at 18:13, Vinod Kone <[email protected]> wrote: >>>>>>> >>>>>>> I'm happy to add the "\n" after the event (note it's different from >>>>>>> chunk) if that makes CURL play nicer. I'm not sure about the "\r" part >>>>>>> though? Is that a nice to have or does it have some other benefit? >>>>>>> >>>>>>> The design doc is not set in the stone since this has not been >>>>>>> released yet. So definitely want to do the right/easy thing. >>>>>>> >>>>>>> On Fri, Aug 28, 2015 at 7:53 AM, Anand Mazumdar <[email protected] >>>>>>> > wrote: >>>>>>> >>>>>>>> Dario, >>>>>>>> >>>>>>>> Thanks for the detailed explanation and for trying out the new API. >>>>>>>> However, this is not a bug. The output from CURL is the encoding used >>>>>>>> by >>>>>>>> Mesos for the events stream. From the user doc >>>>>>>> <https://github.com/apache/mesos/blob/master/docs/scheduler_http_api.md> >>>>>>>> : >>>>>>>> >>>>>>>> *"Master encodes each Event in RecordIO format, i.e., string >>>>>>>> representation of length of the event in bytes followed by JSON or >>>>>>>> binary >>>>>>>> Protobuf (possibly compressed) encoded event. Note that the value of >>>>>>>> length will never be ‘0’ and the size of the length will be the size of >>>>>>>> unsigned integer (i.e., 64 bits). Also, note that the RecordIO encoding >>>>>>>> should be decoded by the scheduler whereas the underlying HTTP chunked >>>>>>>> encoding is typically invisible at the application (scheduler) layer.“* >>>>>>>> >>>>>>>> If you run CURL with tracing enabled i.e. —trace, the output would >>>>>>>> be something similar to this: >>>>>>>> >>>>>>>> <= Recv header, 2 bytes (0x2) >>>>>>>> 0000: 0d 0a .. >>>>>>>> <= Recv data, 115 bytes (0x73) >>>>>>>> 0000: 36 64 0d 0a 31 30 35 0a 7b 22 73 75 62 73 63 72 >>>>>>>> 6d..105.{"subscr >>>>>>>> 0010: 69 62 65 64 22 3a 7b 22 66 72 61 6d 65 77 6f 72 >>>>>>>> ibed":{"framewor >>>>>>>> 0020: 6b 5f 69 64 22 3a 7b 22 76 61 6c 75 65 22 3a 22 >>>>>>>> k_id":{"value":" >>>>>>>> 0030: 32 30 31 35 30 38 32 35 2d 31 30 33 30 31 38 2d >>>>>>>> 20150825-103018- >>>>>>>> 0040: 33 38 36 33 38 37 31 34 39 38 2d 35 30 35 30 2d >>>>>>>> 3863871498-5050- >>>>>>>> 0050: 31 31 38 35 2d 30 30 31 30 22 7d 7d 2c 22 74 79 >>>>>>>> 1185-0010"}},"ty >>>>>>>> 0060: 70 65 22 3a 22 53 55 42 53 43 52 49 42 45 44 22 >>>>>>>> pe":"SUBSCRIBED" >>>>>>>> 0070: 7d 0d 0a }.. >>>>>>>> <others >>>>>>>> >>>>>>>> In the output above, the chunks are correctly delimited by ‘CRLF' >>>>>>>> (0d 0a) as per the HTTP RFC. As mentioned earlier, the output that you >>>>>>>> observe on stdout with CURL is of the Record-IO encoding used for the >>>>>>>> events stream ( and is not related to the RFC ): >>>>>>>> >>>>>>>> event = event-size LF >>>>>>>> event-data >>>>>>>> >>>>>>>> Looking forward to more bug reports as you try out the new API ! >>>>>>>> >>>>>>>> -anand >>>>>>>> >>>>>>>> On Aug 28, 2015, at 12:56 AM, Dario Rexin <[email protected]> >>>>>>>> wrote: >>>>>>>> >>>>>>>> -1 (non-binding) >>>>>>>> >>>>>>>> I found a breaking bug in the new HTTP API. The messages do not >>>>>>>> conform to the HTTP standard for chunked transfer encoding. in RFC 2616 >>>>>>>> Sec. 3 (http://www.w3.org/Protocols/rfc2616/rfc2616-sec3.html) a >>>>>>>> chunk is defined as: >>>>>>>> >>>>>>>> chunk = chunk-size [ chunk-extension ] CRLF >>>>>>>> chunk-data CRLF >>>>>>>> >>>>>>>> >>>>>>>> The HTTP API currently sends a chunk as: >>>>>>>> >>>>>>>> chunk = chunk-size LF >>>>>>>> chunk-data >>>>>>>> >>>>>>>> >>>>>>>> A standard conform HTTP client like curl can’t correctly interpret >>>>>>>> the data as a complete chunk. In curl it currently looks like this: >>>>>>>> >>>>>>>> 104 >>>>>>>> >>>>>>>> {"subscribed":{"framework_id":{"value":"20150820-114552-16777343-5050-43704-0000"}},"type":"SUBSCRIBED"}20 >>>>>>>> {"type":"HEARTBEAT”}666 >>>>>>>> …. waiting … >>>>>>>> >>>>>>>> {"offers":{"offers":[{"agent_id":{"value":"20150820-114552-16777343-5050-43704-S0"},"framework_id":{"value":"20150820-114552-16777343-5050-43704-0000"},"hostname":"localhost","id":{"value":"20150820-114552-16777343-5050-43704-O0"},"resources":[{"name":"cpus","role":"*","scalar":{"value":8},"type":"SCALAR"},{"name":"mem","role":"*","scalar":{"value":15360},"type":"SCALAR"},{"name":"disk","role":"*","scalar":{"value":2965448},"type":"SCALAR"},{"name":"ports","ranges":{"range":[{"begin":31000,"end":32000}]},"role":"*","type":"RANGES"}],"url":{"address":{"hostname":"localhost","ip":"127.0.0.1","port":5051},"path":"\/slave(1)","scheme":"http"}}]},"type":"OFFERS”}20 >>>>>>>> … waiting … >>>>>>>> {"type":"HEARTBEAT”}20 >>>>>>>> … waiting … >>>>>>>> >>>>>>>> It will receive a couple of messages after successful registration >>>>>>>> with the master and the last thing printed is a number (in this case >>>>>>>> 666). >>>>>>>> Then after some time it will print the first offers message followed >>>>>>>> by the >>>>>>>> number 20. The explanation for this behavior is, that curl can’t >>>>>>>> interpret >>>>>>>> the data it gets from Mesos as a complete chunk and waits for the >>>>>>>> missing >>>>>>>> data. So it prints what it thinks is a chunk (a message followed by the >>>>>>>> size of the next messsage) and keeps the rest of the message until >>>>>>>> another >>>>>>>> message arrives and so on. The fix for this is to terminate both >>>>>>>> lines, the >>>>>>>> message size and the message data, with CRLF. >>>>>>>> >>>>>>>> Cheers, >>>>>>>> Dario >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>> >>>>> >>>> >>> >> >
