> On Oct. 15, 2014, 5:02 p.m., Timothy Chen wrote: > > src/master/master.cpp, line 4494 > > <https://reviews.apache.org/r/25184/diff/3/?file=716687#file716687line4494> > > > > Sorry not to keep knit picking, but our style in mesos is to have > > comments end with periods. > > > > We usually don't include the jira ticket number as well. Please just > > remove the line ("MESOS-1746") > > Timothy Chen wrote: > Or Tim if you want to just fix the comment and commit it :) It might be > easier
I'll do that. - Timothy ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25184/#review56729 ----------------------------------------------------------- On Oct. 15, 2014, 2:23 a.m., Chengwei Yang wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/25184/ > ----------------------------------------------------------- > > (Updated Oct. 15, 2014, 2:23 a.m.) > > > Review request for mesos, Adam B and Timothy St. Clair. > > > Bugs: MESOS-1746 > https://issues.apache.org/jira/browse/MESOS-1746 > > > Repository: mesos-git > > > Description > ------- > > There was a bug found that Spark use TaskStatus.data to transfer computed > result and mesos-master RES memory keeps increasing fast and finally will be > killed by OOM killer. > > > Diffs > ----- > > src/master/master.cpp cb46cec0674b3aa031450c5b4f48f4f8bb92767d > > Diff: https://reviews.apache.org/r/25184/diff/ > > > Testing > ------- > > tested with spark. It's very easy to reproduce this issue (100%) with spark, > when spark use mesos as resource manager, its executor driver will put result > into TaskStatus. For example, a result of a single task like below. > > 14/08/22 13:29:18 INFO Executor: Serialized size of result for 248 is 17573033 > > It's about 16MB large, and a stage of spark generally consist of maybe > hundreds of task and finished in tens of seconds, this will put mesos get > killed by OOM killer soon. > > > Thanks, > > Chengwei Yang > >
