----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/29507/#review89242 -----------------------------------------------------------
Ship it! Thanks Adam, looks good, just one suggestion related to naming the field in the connection message. docs/configuration.md (line 373) <https://reviews.apache.org/r/29507/#comment141828> How about "will be removed" to keep the terminology consistent with our other slave removal flag names, instead of "will be asked to shutdown"? docs/configuration.md (lines 556 - 558) <https://reviews.apache.org/r/29507/#comment141848> Ditto here, let's say "removed" instead of "asked to shutdown". docs/upgrades.md (line 13) <https://reviews.apache.org/r/29507/#comment141850> How about: "Slaves should be upgraded to 0.23.x before changing this flag." Just because the re-register comment here seems a bit confusing, people using this timeout might not know that the slaves use it to trigger re-registrations..? Up to you. src/master/flags.cpp (lines 387 - 391) <https://reviews.apache.org/r/29507/#comment141857> "1s" and "15min" will not parse as Durations, but "1secs" and "15mins" will, so how about just stringifying directly, so we are guaranteed to show them a parseable string: ``` if (value < Seconds(1) || values > Minutes(15)) { return Error("Expected --slave_ping_timeout to be within " + stringify(Seconds(1)) + " and " + stringify(Minutes(15))); } return None(); ``` Also, it's too bad that we have to include the flag names here, I was a bit surprised that the load error coming from flags does not already include which flag's validation failed! I left a comment for benh here: https://reviews.apache.org/r/34943/ src/master/flags.cpp (line 401) <https://reviews.apache.org/r/29507/#comment141858> (1) s/const// since we don't bother taking in native types as const arguments in our existing style. (2) s/uint32_t/size_t/ since we've generally been using 'size_t' for non-negative countable things like this. src/master/master.cpp (line 129) <https://reviews.apache.org/r/29507/#comment141859> Ditto above: s/uint32_t/size_t/ and s/const// Just to be consistent with the majority of our code. src/master/master.cpp (line 269) <https://reviews.apache.org/r/29507/#comment141860> Here, just s/uint32_t/size_t src/master/master.cpp (lines 3154 - 3157) <https://reviews.apache.org/r/29507/#comment141862> Hm.. you have a ping_timeout in the message, but that doesn't correspond to the ping timeout? It seems a bit misleading that it corresponds to the shutdown timeout. Two suggestions: (1) Rename 'ping_timeout_seconds' to 'total_ping_timeout_seconds' to help us realize that this isn't the singular ping timeout, along with a small note on the message related to this. (2) Add max_ping_timeouts to the connection message, and set both fields directly from the flags. (1) seems simpler to me for now. src/master/master.cpp (line 3234) <https://reviews.apache.org/r/29507/#comment141863> Feel free to omit 'const' for locally scoped variables like this, we should probably be careful about proliferating 'const', there are a ton we can add to the existing source but it will probably get too verbose and inconsistent :( src/messages/messages.proto (lines 296 - 298) <https://reviews.apache.org/r/29507/#comment141865> Per the above comment, how about: ``` message MasterSlaveConnection { // If no pings are received within the total timeout, // the master will remove the slave. optional double total_ping_timeout_seconds = 1; } ``` src/slave/slave.cpp (line 976) <https://reviews.apache.org/r/29507/#comment141866> Revert this change? Looks odd that TERMINATING has the same two lines but doesn't have the newline. - Ben Mahler On June 22, 2015, 10:03 a.m., Adam B wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/29507/ > ----------------------------------------------------------- > > (Updated June 22, 2015, 10:03 a.m.) > > > Review request for mesos, Ben Mahler and Niklas Nielsen. > > > Bugs: MESOS-2110 > https://issues.apache.org/jira/browse/MESOS-2110 > > > Repository: mesos > > > Description > ------- > > Added new --slave_ping_timeout and --max_slave_ping_timeouts flags > to mesos-master to supplement the DEFAULT_SLAVE_PING_TIMEOUT (15secs) > and DEFAULT_MAX_SLAVE_PING_TIMEOUTS (5). > > These can be extended if slaves are expected/allowed to be down for > longer than a minute or two. > > Slave will receive master's ping timeout in SlaveRe[re]gisteredMessage. > > Beware that this affects recovery from network timeouts as well as > actual slave node/process failover. > > Also fixed the log message in recoveredSlavesTimeout() to correctly > reference flags.slave_reregister_timeout instead of the unrelated > ping timeouts. > > > Diffs > ----- > > docs/configuration.md aaf65bf > docs/upgrades.md 355307a > src/master/constants.hpp 072d59c > src/master/constants.cpp 997b792 > src/master/flags.hpp 55ed3a9 > src/master/flags.cpp 4377715 > src/master/master.cpp 0135c15 > src/messages/messages.proto 1c8d79e > src/slave/constants.hpp 84927e5 > src/slave/constants.cpp d8d2f98 > src/slave/slave.hpp f1cf3b8 > src/slave/slave.cpp 40c0c33 > src/tests/partition_tests.cpp f7ee3ab > src/tests/slave_recovery_tests.cpp c036e9c > src/tests/slave_tests.cpp 5030198 > > Diff: https://reviews.apache.org/r/29507/diff/ > > > Testing > ------- > > Manually tested slave failover/shutdown with master using different > --slave_ping_timeout and --max_slave_ping_timeouts. > Ran unit tests with shorter non-default values for ping timeouts. > `make check` with new unit tests: ShortPingTimeoutUnreachableMaster and > ShortPingTimeoutUnreachableSlave > > > Thanks, > > Adam B > >
