----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25184/#review56729 -----------------------------------------------------------
src/master/master.cpp <https://reviews.apache.org/r/25184/#comment97115> Sorry not to keep knit picking, but our style in mesos is to have comments end with periods. We usually don't include the jira ticket number as well. Please just remove the line ("MESOS-1746") - Timothy Chen On Oct. 15, 2014, 2:23 a.m., Chengwei Yang wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/25184/ > ----------------------------------------------------------- > > (Updated Oct. 15, 2014, 2:23 a.m.) > > > Review request for mesos, Adam B and Timothy St. Clair. > > > Bugs: MESOS-1746 > https://issues.apache.org/jira/browse/MESOS-1746 > > > Repository: mesos-git > > > Description > ------- > > There was a bug found that Spark use TaskStatus.data to transfer computed > result and mesos-master RES memory keeps increasing fast and finally will be > killed by OOM killer. > > > Diffs > ----- > > src/master/master.cpp cb46cec0674b3aa031450c5b4f48f4f8bb92767d > > Diff: https://reviews.apache.org/r/25184/diff/ > > > Testing > ------- > > tested with spark. It's very easy to reproduce this issue (100%) with spark, > when spark use mesos as resource manager, its executor driver will put result > into TaskStatus. For example, a result of a single task like below. > > 14/08/22 13:29:18 INFO Executor: Serialized size of result for 248 is 17573033 > > It's about 16MB large, and a stage of spark generally consist of maybe > hundreds of task and finished in tens of seconds, this will put mesos get > killed by OOM killer soon. > > > Thanks, > > Chengwei Yang > >
