> On June 4, 2013, 12:33 a.m., Ben Mahler wrote: > > Good idea, but seems like a JSON endpoint would be better (this is how we > > expose statistics in mesos)? > > Brenden Matthews wrote: > That's fine, but then I'd have to do a bunch of JSON serialization in the > Hadoop JobTracker, which seems overkill to me.
My concern is twofold: (1) An http endpoint seems nicer than a file, IMO. One can just curl the endpoint and each response will contain the latest data, rather than a periodically update file where each read returns a checkpointed version of the scheduler state. (2) The idiomatic way of exposing state in mesos is through http endpoints, so it would be nice to stay consistent there. I realize that it's tough to add that into the hadoop scheduler given that we don't have any abstractions in place to assist in exporting stats over an endpoint. But I'm open to options! :) - Ben ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/11165/#review21384 ----------------------------------------------------------- On June 11, 2013, 7:33 p.m., Brenden Matthews wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/11165/ > ----------------------------------------------------------- > > (Updated June 11, 2013, 7:33 p.m.) > > > Review request for mesos. > > > Description > ------- > > Write Mesos JobTracker stats to state file. > > Review: https://reviews.apache.org/r/11165/ > > > Diffs > ----- > > hadoop/mesos/src/java/org/apache/hadoop/mapred/MesosScheduler.java > afe401f5265e3d9494af7eace42eec45943184a3 > > Diff: https://reviews.apache.org/r/11165/diff/ > > > Testing > ------- > > Used in production at airbnb. > > make -j10 check && cd hadoop && make hadoop-2.0.0-mr1-cdh4.2.1 && make > hadoop-0.20.205.0 && make hadoop-0.20.2-cdh3u3 > > > Thanks, > > Brenden Matthews > >
