----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/17685/#review33824 -----------------------------------------------------------
Jie and I did a thorough review. The changes mostly look great, however we found some bugs around onDiscard handling in collect.hpp and protobuf.hpp. Also, Subprocess is now unnecessarily complicated, we can remove the outer Promise and directly return the reaper's Future. (Since the reaper will not respond to onDiscard, it *must* ensure the pid is reaped). 3rdparty/libprocess/3rdparty/stout/tests/os/signals_tests.cpp <https://reviews.apache.org/r/17685/#comment63517> Why this change? (Maybe add a note) 3rdparty/libprocess/include/process/collect.hpp <https://reviews.apache.org/r/17685/#comment63511> This looks like a bug, the future will remain pending since the Promise is not discarded! 3rdparty/libprocess/include/process/collect.hpp <https://reviews.apache.org/r/17685/#comment63512> Ditto here. 3rdparty/libprocess/include/process/future.hpp <https://reviews.apache.org/r/17685/#comment63439> We should really have documentation for the discard semantics you describe in this change! 3rdparty/libprocess/include/process/future.hpp <https://reviews.apache.org/r/17685/#comment63452> What if users want to pass in an lvalue? Should we not be providing the overload for that? (Rather than having the const& only for C++03). 3rdparty/libprocess/include/process/future.hpp <https://reviews.apache.org/r/17685/#comment63441> Why does this need to be public? (Maybe add a comment). 3rdparty/libprocess/include/process/future.hpp <https://reviews.apache.org/r/17685/#comment63442> Maybe mention that it is because onDiscarded requires a void function..? 3rdparty/libprocess/include/process/future.hpp <https://reviews.apache.org/r/17685/#comment63446> We should kill this comment now. 3rdparty/libprocess/include/process/future.hpp <https://reviews.apache.org/r/17685/#comment63449> It seems useful to comment on the propagation flow 'downwards' and 'upwards' and explicitly call out the relationship between: 1. Promise::associate vs. 2. Promise::set, Promise::fail, Promise::discard We should not call something from 1 after 2, or something from 2 after 1. (This TODO only addresses 2 after 1, right?). 3rdparty/libprocess/include/process/future.hpp <https://reviews.apache.org/r/17685/#comment63448> We understood this part, but can you add a little note around why an onDiscard of 'future' is not propagated here for posterity? 3rdparty/libprocess/include/process/future.hpp <https://reviews.apache.org/r/17685/#comment63450> Documentation on these semantics in the API would be nice :) (discard returns false if already called, or the future is no longer pending) 3rdparty/libprocess/include/process/protobuf.hpp <https://reviews.apache.org/r/17685/#comment63516> This looks like a bug! The future will remain pending. A correct solution would be to discard the promise _and_ self terminate. 3rdparty/libprocess/src/process.cpp <https://reviews.apache.org/r/17685/#comment63526> It doesn't look like this discard will be noticed by those holding the Promise for this, and hence the response will never transition out of pending. Can you double check this? 3rdparty/libprocess/src/process.cpp <https://reviews.apache.org/r/17685/#comment63457> Re: discard vs discarded Consider: // Stop polling if a discard occurs on our future. 3rdparty/libprocess/src/process.cpp <https://reviews.apache.org/r/17685/#comment63459> Can you add a comment about the cycle you're breaking here (with WeakFuture)? 3rdparty/libprocess/src/process.cpp <https://reviews.apache.org/r/17685/#comment63434> Because we're using 'discard' and 'discarded' to mean different things, how about saying: // Make sure we stop polling if a discard occurs on our future. 3rdparty/libprocess/src/process.cpp <https://reviews.apache.org/r/17685/#comment63437> We're thinking we can avoid the friend function by having a wrapper continuation for discard that takes a 'Shared' promise, and we also pass a 'Shared' promise in the watcher data. This avoids the issue with the promise being deleted by the watcher. Does this make sense? 3rdparty/libprocess/src/process.cpp <https://reviews.apache.org/r/17685/#comment63466> Seems ok for now, but returning number of bytes seems most useful to the caller. 3rdparty/libprocess/src/process.cpp <https://reviews.apache.org/r/17685/#comment63467> Maybe a comment about the cycle you're breaking with WeakFuture? 3rdparty/libprocess/src/process.cpp <https://reviews.apache.org/r/17685/#comment63471> Shouldn't this avoid the call to io::write if promise->future().hasDiscard() is true? (This is basically the same check you're doing for the io::read call). We thought, maybe you wanted to write anything you've read, which essentially keeps splice somewhat consistent (avoiding dropping any read data). But, this doesn't seem satisfied in the code here anyway, since the write future would immediately be discarded. This means the write operation might stop before writing the read data. 3rdparty/libprocess/src/process.cpp <https://reviews.apache.org/r/17685/#comment63468> Ditto about cycle. 3rdparty/libprocess/src/process.cpp <https://reviews.apache.org/r/17685/#comment63473> Ditto here for checking promise->future().hasDiscard() before calling io::write. 3rdparty/libprocess/src/tests/io_tests.cpp <https://reviews.apache.org/r/17685/#comment63478> Maybe spell out that you're expecting a failure in the 128 byte equality check if the second io::write was not successfully discarded? 3rdparty/libprocess/src/tests/io_tests.cpp <https://reviews.apache.org/r/17685/#comment63480> Would be nice to also test the discard semantics in between read and write by using a full pipe on the write end. 3rdparty/libprocess/src/tests/process_tests.cpp <https://reviews.apache.org/r/17685/#comment63481> Should probably update this comment now to reflect discard vs discarded propagation. 3rdparty/libprocess/src/tests/process_tests.cpp <https://reviews.apache.org/r/17685/#comment63482> Better names? discard1, discard2, discard3, discard4, discard5 are not the most descriptive :) Or a comment above the test to describe what is being tested. 3rdparty/libprocess/src/tests/process_tests.cpp <https://reviews.apache.org/r/17685/#comment63485> How about adding a: ASSERT_TRUE(future.hasDiscard()); 3rdparty/libprocess/src/tests/process_tests.cpp <https://reviews.apache.org/r/17685/#comment63486> ditto for checking hasDiscard() as well 3rdparty/libprocess/src/tests/process_tests.cpp <https://reviews.apache.org/r/17685/#comment63490> Perhaps spell out that this is because the .then() chain should not continue once a discard occurs on the result? 3rdparty/libprocess/src/tests/process_tests.cpp <https://reviews.apache.org/r/17685/#comment63505> Couldn't we use a Promise or (atomic boolean in C++11) here to avoid the need for a volatile boolean? Do you need volatile? Won't the await() provide a full memory barrier? 3rdparty/libprocess/src/tests/process_tests.cpp <https://reviews.apache.org/r/17685/#comment63509> Maybe some comments? :) - Ben Mahler On Feb. 5, 2014, 11:40 p.m., Benjamin Hindman wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/17685/ > ----------------------------------------------------------- > > (Updated Feb. 5, 2014, 11:40 p.m.) > > > Review request for mesos, Adam B, Ben Mahler, Ian Downes, Jie Yu, Niklas > Nielsen, TILL TOENSHOFF, Vinod Kone, and Jiang Yan Xu. > > > Repository: mesos-git > > > Description > ------- > > Previously, invoking Future::discard caused a future to transition > into the DISCARDED state which meant that any callbacks associated wit > such a transition (i.e., Future::onAny, Future::onDiscarded) got > executed _even if the computation associated with the future had not > yet completed_! This meant certain patterns where difficult if not all > together impossible. Consider: > > ExpensiveResource* resource = new ExpensiveResource(); > > Future<int> future = function() > .then([=] () { ... *resource ...; }) > .onDiscarded([=] () { delete resource; }); > > future.discard(); > > In the above code, it's possible that the onDiscarded lambda would > execute _concurrently_ with the lambda in the then! With the new > semantics, a discard is more advisorial: rather than transition a > future it instead allows a subsequent computation to decide whether or > not to cancel (or even begin) some computation. For example, the > Future::then implementation uses the "discard" information to decide > whether or not to execute the next lambda after the previous future > has been satisfied. > > A few new helpers have been added to support this refactor: > > Future::onDiscard(callback): > > Adds a callback for when Future::discard was executed on this future. > > Future::hasDiscard(): > > Checks to see if Future::discard was executed on this future. > > Promise::discard(): > > Transitions this promises underlying future to DISCARDED. > > > Diffs > ----- > > 3rdparty/libprocess/3rdparty/stout/tests/os/signals_tests.cpp > 66caa0490189cb1aff3af1631fbcf7150658b6cc > 3rdparty/libprocess/include/process/collect.hpp > ff8ab341a588d26f9678aea7f53e892d292725ec > 3rdparty/libprocess/include/process/deferred.hpp > a25080f7cfc164108e89660d4c38de8b290aee02 > 3rdparty/libprocess/include/process/future.hpp > aaf17af2eef0f1bf7062aa04469f1fa27bb4cfbe > 3rdparty/libprocess/include/process/protobuf.hpp > 7500601026a58c95325ffe4410a161d59ff7817e > 3rdparty/libprocess/src/process.cpp > 2e7764a8e0badec704b8610f3f72f0bd16cc9612 > 3rdparty/libprocess/src/tests/http_tests.cpp > 62ab64d81c11ab7260620c91100b7e1f29b81e33 > 3rdparty/libprocess/src/tests/io_tests.cpp > eab786432debf06615572c4486cdf08bac91d927 > 3rdparty/libprocess/src/tests/process_tests.cpp > be288c353dce78ef641708646979e2e4f3f2c504 > > Diff: https://reviews.apache.org/r/17685/diff/ > > > Testing > ------- > > make check > > > Thanks, > > Benjamin Hindman > >
