Mesos 0.20.1 release status
We are targetting 17 tickets. It include improvements and bug fixes. 4 of them are still in progress. http://s.apache.org/mesos-0.20.1-unresolved-issues I'll cut the tag for RC1 and send for voting, once these issues are reviewed/submitted or 9/15 @6pm PDT, whichever comes first! The open issues (if any) will be moved to next release, at that point in time. In case you got questions, let me know. Thank you, -- Regards, Bhuvan Arumugam www.livecipher.com
Re: Review Request 25566: Minor cleanups to the Master code.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25566/#review53449 --- Ship it! src/master/master.cpp https://reviews.apache.org/r/25566/#comment93108 - Vinod Kone On Sept. 12, 2014, 2:01 a.m., Ben Mahler wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25566/ --- (Updated Sept. 12, 2014, 2:01 a.m.) Review request for mesos and Vinod Kone. Repository: mesos-git Description --- (1) Updated the Slave struct to avoid counting resources. Rather, when asked, compute resources based on the tasks. This makes it easier to do the resource accounting in https://reviews.apache.org/r/25567/ where we hold on to terminal tasks. (2) Cleaned up the task removal logging, to be inside removeTask(Task*). (3) Consistently use utils::copy instead of keys() / values() when a copy is required to iterate correctly, to make it more explicit to the reader. Diffs - src/master/http.cpp 6dd11fe5297ea68331b5e9f23a6d8590edecedc4 src/master/master.hpp b4926001178ebb00b34b0b7e03f491d4a800afc2 src/master/master.cpp d5db24ef3c2d2501aa5852b62d50a425bc0ad925 Diff: https://reviews.apache.org/r/25566/diff/ Testing --- no functional change make check Thanks, Ben Mahler
Re: Review Request 25567: Hold on to unacknowledged terminal tasks in the Master.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25567/#review53450 --- Ship it! src/master/master.hpp https://reviews.apache.org/r/25567/#comment93109 s/updateTaskState/updateTask/ since you are not just updating the state? src/master/master.hpp https://reviews.apache.org/r/25567/#comment93110 s/a/the/ - Vinod Kone On Sept. 12, 2014, 2:01 a.m., Ben Mahler wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25567/ --- (Updated Sept. 12, 2014, 2:01 a.m.) Review request for mesos, Niklas Nielsen and Vinod Kone. Bugs: MESOS-1410 https://issues.apache.org/jira/browse/MESOS-1410 Repository: mesos-git Description --- This is the MESOS-1410 which fixes MESOS-1389. The idea here is that the master needs to hold on to those tasks that are terminal, but have yet to be acknowledged by the scheduler. Otherwise, reconciliation requests could lead to TASK_LOST updates **before** a framework receives a terminal TASK_FINISHED/TASK_FAILED/etc update for the task. Now the master needs to: (1) Remove tasks when an acknowledgement arrives. (2) Recover resources when the task becomes terminal. (3) Omit resources for terminal tasks in the http statistics. Diffs - src/master/master.hpp b4926001178ebb00b34b0b7e03f491d4a800afc2 src/master/master.cpp d5db24ef3c2d2501aa5852b62d50a425bc0ad925 Diff: https://reviews.apache.org/r/25567/diff/ Testing --- make check Added new tests in https://reviews.apache.org/r/25568/ Thanks, Ben Mahler
Re: Mesos 0.20.1 release status
It's already past 9/16 6 pm PDT? Tim On Tue, Sep 16, 2014 at 2:19 AM, Bhuvan Arumugam bhu...@apache.org wrote: We are targetting 17 tickets. It include improvements and bug fixes. 4 of them are still in progress. http://s.apache.org/mesos-0.20.1-unresolved-issues I'll cut the tag for RC1 and send for voting, once these issues are reviewed/submitted or 9/15 @6pm PDT, whichever comes first! The open issues (if any) will be moved to next release, at that point in time. In case you got questions, let me know. Thank you, -- Regards, Bhuvan Arumugam www.livecipher.com
Re: Review Request 25568: Added tests for terminal unacknowledged tasks in the Master.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25568/#review53484 --- src/tests/master_tests.cpp https://reviews.apache.org/r/25568/#comment93123 Hmm... This is not passed to StartSlave()? src/tests/master_tests.cpp https://reviews.apache.org/r/25568/#comment93124 s/task terminal/task in terminal state/ src/tests/master_tests.cpp https://reviews.apache.org/r/25568/#comment93126 The Ignore subsequent offers comment is incorrect here. src/tests/master_tests.cpp https://reviews.apache.org/r/25568/#comment93127 What is the guarantee that offers2 will be made after task's resources are recovered? src/tests/reconciliation_tests.cpp https://reviews.apache.org/r/25568/#comment93128 s/ensure/ensures/ src/tests/reconciliation_tests.cpp https://reviews.apache.org/r/25568/#comment93129 s/task terminal/task in terminal state/ src/tests/reconciliation_tests.cpp https://reviews.apache.org/r/25568/#comment93131 Why do you want to make sure that master received the reconcile tasks message? Isn't the receipt of the update guarantee that? Actually, thinking a bit more, how do you guarantee that the second update was due to reconcile tasks and not a retry by the slave? - Vinod Kone On Sept. 12, 2014, 2:01 a.m., Ben Mahler wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25568/ --- (Updated Sept. 12, 2014, 2:01 a.m.) Review request for mesos, Niklas Nielsen and Vinod Kone. Bugs: MESOS-1410 https://issues.apache.org/jira/browse/MESOS-1410 Repository: mesos-git Description --- Added two tests: (1) Ensure that reconciliation works for terminal unacknowledged tasks. (2) Ensure that resources are released for terminal unacknowledged tasks. Diffs - src/master/master.cpp d5db24ef3c2d2501aa5852b62d50a425bc0ad925 src/tests/master_tests.cpp 3d080b2efad5a210353d4cef4c827380d5138d1a src/tests/reconciliation_tests.cpp 1c9e73b0ee99a8a33f663f992b0c9770e83b98c5 Diff: https://reviews.apache.org/r/25568/diff/ Testing --- make check, ran these new tests in repetition Thanks, Ben Mahler
Re: Mesos 0.20.1 release status
On Mon, Sep 15, 2014 at 11:19 PM, Bhuvan Arumugam bhu...@apache.org wrote: I'll cut the tag for RC1 and send for voting, once these issues are reviewed/submitted or 9/15 @6pm PDT, whichever comes first! The open issues (if any) will be moved to next release, at that point in time. You want Adam to do this part because it needs committer access. Thanks for the ticket wrangling by the way!
Re: Mesos 0.20.1 release status
Bhuvan, Thanks a ton! Are you going to do a cut, or cherry-pick on top of 0.20.0? - Jie On Mon, Sep 15, 2014 at 11:19 PM, Bhuvan Arumugam bhu...@apache.org wrote: We are targetting 17 tickets. It include improvements and bug fixes. 4 of them are still in progress. http://s.apache.org/mesos-0.20.1-unresolved-issues I'll cut the tag for RC1 and send for voting, once these issues are reviewed/submitted or 9/15 @6pm PDT, whichever comes first! The open issues (if any) will be moved to next release, at that point in time. In case you got questions, let me know. Thank you, -- Regards, Bhuvan Arumugam www.livecipher.com
Re: Review Request 25569: Refactor test environment validations
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25569/ --- (Updated Sept. 16, 2014, 7:07 a.m.) Review request for mesos and Ben Mahler. Summary (updated) - Refactor test environment validations Repository: mesos-git Description (updated) --- Review: https://reviews.apache.org/r/25569 Refactor how validation is done in the environment by defining a standard test filter interface, and only execute the validation once. Diffs (updated) - src/tests/environment.cpp 2274251aaf653d83c2d03ef2186763978067a747 Diff: https://reviews.apache.org/r/25569/diff/ Testing --- make check Thanks, Timothy Chen
Re: Review Request 25569: Refactor test environment validations
On Sept. 12, 2014, 11:39 p.m., Ben Mahler wrote: Thanks for following up! Not your fault, but the current design of enable() seems a bit unfortunate, because we will print things excessively unless we use static variables as you've done here. What about the following instead? (1) Add a method called 'disabled()' that returns a list of disabled test names. (2) Inside 'disabled()', we can print the disablement messages once at the beginning, then loop over the tests to construct the list of disabled test names to return. (3) Inside 'Environment::Environment()', we use disabled() to construct the full 'disabled' string. Does that seem cleaner? The static variables seem good for now, modulo my comment below. Consider adding a TODO to clean this up if you go this route! Ok I've done a refactor based on your comments, let me know what you think. - Timothy --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25569/#review53246 --- On Sept. 16, 2014, 7:07 a.m., Timothy Chen wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25569/ --- (Updated Sept. 16, 2014, 7:07 a.m.) Review request for mesos and Ben Mahler. Repository: mesos-git Description --- Review: https://reviews.apache.org/r/25569 Refactor how validation is done in the environment by defining a standard test filter interface, and only execute the validation once. Diffs - src/tests/environment.cpp 2274251aaf653d83c2d03ef2186763978067a747 Diff: https://reviews.apache.org/r/25569/diff/ Testing --- make check Thanks, Timothy Chen
Re: Review Request 25569: Refactor test environment validations
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25569/#review53491 --- Bad patch! Reviews applied: [25569] Failed command: ./support/mesos-style.py Error: Checking 504 files using filter --filter=-,+build/class,+build/deprecated,+build/endif_comment,+readability/todo,+readability/namespace,+runtime/vlog,+whitespace/blank_line,+whitespace/comma,+whitespace/end_of_line,+whitespace/ending_newline,+whitespace/forcolon,+whitespace/indent,+whitespace/line_length,+whitespace/tab,+whitespace/todo src/tests/environment.cpp:279: Lines should be = 80 characters long [whitespace/line_length] [2] Total errors found: 1 - Mesos ReviewBot On Sept. 16, 2014, 7:07 a.m., Timothy Chen wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25569/ --- (Updated Sept. 16, 2014, 7:07 a.m.) Review request for mesos and Ben Mahler. Repository: mesos-git Description --- Review: https://reviews.apache.org/r/25569 Refactor how validation is done in the environment by defining a standard test filter interface, and only execute the validation once. Diffs - src/tests/environment.cpp 2274251aaf653d83c2d03ef2186763978067a747 Diff: https://reviews.apache.org/r/25569/diff/ Testing --- make check Thanks, Timothy Chen
Re: Review Request 25250: Mark running tasks killed during framework shutdown.
On Sept. 15, 2014, 4:38 p.m., Benjamin Hindman wrote: src/master/master.cpp, line 4010 https://reviews.apache.org/r/25250/diff/4/?file=682254#file682254line4010 I suggest we use TASK_LOST here instead. We definitely want a terminal state like TASK_KILLED, but we've reserved TASK_KILLED for when a framework has actually intiated the kill itself, and thus I'd prefer not to overload the semantics. This might be a good candidate for a new task state, e.g., TASK_REMOVED, which has been discussed in the past, but I can't recall if there is a JIRA for that or not. If not, it would be great to have you create one Alex so we can have a discussion about how to introduce new task states (and maybe even a way to introduce sub-states that framework writers themselves could customize). I used to have `TASK_LOST` here, but my understanding is that `TASK_LOST` is used for abnormal situations, i.e. when the task is not finished not because of scheduler's direct command, but because of some external reasons. I agree, that a new task state is a very good solution. We have [this ticket](https://issues.apache.org/jira/browse/MESOS-343), one solution for which would be to introduce something like `TaskStatusExplained` or a protobuf message for every task. But maybe for this situation something like `TASK_ABANDONED` would be rather descriptive. - Alexander --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25250/#review53350 --- On Sept. 7, 2014, 6:35 p.m., Alexander Rukletsov wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25250/ --- (Updated Sept. 7, 2014, 6:35 p.m.) Review request for mesos, Benjamin Hindman and Till Toenshoff. Bugs: MESOS-1736 https://issues.apache.org/jira/browse/MESOS-1736 Repository: mesos-git Description --- When a framework is shut down e.g. by calling driver.stop() from the scheduler, running tasks are marked KILLED before migrating them to completed. Diffs - src/master/master.cpp c6393b2 src/tests/master_tests.cpp 3d080b2 Diff: https://reviews.apache.org/r/25250/diff/ Testing --- make check (OS X) Thanks, Alexander Rukletsov
Review Request 25695: Update to enable systemd control of mesos services
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25695/ --- Review request for mesos, Jie Yu and Vinod Kone. Bugs: MESOS-1195 https://issues.apache.org/jira/browse/MESOS-1195 Repository: mesos-git Description --- This update enables support for systemd co-managed cgroup controllers Diffs - configure.ac c4b4391 src/linux/cgroups.cpp 5093b4c src/slave/containerizer/isolators/cgroups/cpushare.cpp b1cad47 Diff: https://reviews.apache.org/r/25695/diff/ Testing --- systemctl start mesos-master mesos-slave several runs of 'mesos execute' to verify creation and cleanup. systemctl stop mesos-master mesos-slave make check Thanks, Timothy St. Clair
Re: Review Request 25523: Add Docker pull to docker abstraction
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25523/#review53528 --- src/docker/docker.hpp https://reviews.apache.org/r/25523/#comment93215 From an interface perspective I'd prefer that we created an 'Image' abstraction just like we have 'Container', and have Docker::pull return FutureImage where doing a discard on the future is equivalent to cancel. See https://reviews.apache.org/r/24588 for the initial but incomplete version that I had done before 0.20.0. - Benjamin Hindman On Sept. 11, 2014, 6:44 a.m., Timothy Chen wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25523/ --- (Updated Sept. 11, 2014, 6:44 a.m.) Review request for mesos and Benjamin Hindman. Repository: mesos-git Description --- Review: https://reviews.apache.org/r/25523 Diffs - src/docker/docker.hpp e7adedb93272209231a3a9aefecfd6ccc7802ff5 src/docker/docker.cpp af51ac9058382aede61b09e06e312ad2ce6de03e src/slave/containerizer/docker.cpp 0febbac5df4126f6c8d9a06dd0ba1668d041b34a src/tests/docker_tests.cpp 826a8c1ef1b3089d416e5775fa2cf4e5cb0c26d1 Diff: https://reviews.apache.org/r/25523/diff/ Testing --- make check Thanks, Timothy Chen
Re: Review Request 25270: Enable bridge network in Mesos
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25270/#review53531 --- Ship it! src/tests/docker_containerizer_tests.cpp https://reviews.apache.org/r/25270/#comment93221 Awesome test! Can we add a comment above that explains in text how this differs from the other bridged test? In particular, it would be nice for people to understand that you have two tests here, both testing bridged networking, one which tests for an executor, the other which tests for a task and also makes sure the port mapping works. Thanks Tim! src/tests/docker_containerizer_tests.cpp https://reviews.apache.org/r/25270/#comment93227 Let's move this down to where it's actually used. ;-) src/tests/docker_containerizer_tests.cpp https://reviews.apache.org/r/25270/#comment93218 Separating these stanzas would make them easier to read (as we do when setting up other protobufs). src/tests/docker_tests.cpp https://reviews.apache.org/r/25270/#comment93225 Just a random question, what happens if /mnt/mesos/sandbox doesn't exist? src/tests/docker_tests.cpp https://reviews.apache.org/r/25270/#comment93223 Please split these up like above! src/tests/docker_tests.cpp https://reviews.apache.org/r/25270/#comment93224 Please use AWAIT_EXPECT_FAILED, otherwise when this code actually becomes asynchronous this test is going to start non-deterministically failing. - Benjamin Hindman On Sept. 11, 2014, 5:48 p.m., Timothy Chen wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25270/ --- (Updated Sept. 11, 2014, 5:48 p.m.) Review request for mesos, Benjamin Hindman, Jie Yu, and Timothy St. Clair. Bugs: MESOS-1621 https://issues.apache.org/jira/browse/MESOS-1621 Repository: mesos-git Description --- Review: https://reviews.apache.org/r/25270 Diffs - include/mesos/mesos.proto dea51f94d130c131421c43e7fd774ceb8941f501 src/docker/docker.cpp af51ac9058382aede61b09e06e312ad2ce6de03e src/slave/slave.cpp 1b3dc7370a2441e4159aa5ee552b64ca5e511e96 src/tests/docker_containerizer_tests.cpp 8654f9c787bd207f6a7b821651e0c083bea9dc8a src/tests/docker_tests.cpp 826a8c1ef1b3089d416e5775fa2cf4e5cb0c26d1 Diff: https://reviews.apache.org/r/25270/diff/ Testing --- make check Thanks, Timothy Chen
Re: Review Request 25403: Override entrypoint when shell enabled in Docker
On Sept. 9, 2014, 6:50 p.m., Benjamin Hindman wrote: src/docker/docker.cpp, line 337 https://reviews.apache.org/r/25403/diff/1/?file=680701#file680701line337 Why not move this up above as well? Timothy Chen wrote: The Docker cli --entrypoint only allows you to put in a single string, but we actually need a array of entrypoint entries (which is what docker inspect returns). I tried --entrypoint=/bin/sh -c on the cli and it immediately failed. Therefore, I have to run this in the cli: docker run --entrypoint=/bin/sh busybox -c ls Ah, I see, honestly this isn't obvious from the code and it's nasty that it's linked but not well scoped (so copy/paste errors are likely) so explaining why you're adding the '-c' there when you add it would probably save someone a headache in the future! - Benjamin --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25403/#review52767 --- On Sept. 11, 2014, 12:40 a.m., Timothy Chen wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25403/ --- (Updated Sept. 11, 2014, 12:40 a.m.) Review request for mesos, Benjamin Hindman and Jie Yu. Bugs: MESOS-1770 https://issues.apache.org/jira/browse/MESOS-1770 Repository: mesos-git Description --- Review: https://reviews.apache.org/r/25403 Diffs - src/docker/docker.cpp af51ac9058382aede61b09e06e312ad2ce6de03e Diff: https://reviews.apache.org/r/25403/diff/ Testing --- make check Thanks, Timothy Chen
Re: Review Request 25695: Update to enable systemd control of mesos services
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25695/#review53534 --- Bad patch! Reviews applied: [25695] Failed command: ./configure Error: checking build system type... x86_64-unknown-linux-gnu checking host system type... x86_64-unknown-linux-gnu checking target system type... x86_64-unknown-linux-gnu checking for a BSD-compatible install... /usr/bin/install -c checking whether build environment is sane... yes checking for a thread-safe mkdir -p... /bin/mkdir -p checking for gawk... no checking for mawk... mawk checking whether make sets $(MAKE)... yes checking whether make supports nested variables... yes checking for style of include used by make... GNU checking for gcc... gcc checking whether the C compiler works... yes checking for C compiler default output file name... a.out checking for suffix of executables... checking whether we are cross compiling... no checking for suffix of object files... o checking whether we are using the GNU C compiler... yes checking whether gcc accepts -g... yes checking for gcc option to accept ISO C89... none needed checking whether gcc understands -c and -o together... yes checking dependency style of gcc... gcc3 checking for ar... ar checking the archiver (ar) interface... ar checking how to print strings... printf checking for a sed that does not truncate output... /bin/sed checking for grep that handles long lines and -e... /bin/grep checking for egrep... /bin/grep -E checking for fgrep... /bin/grep -F checking for ld used by gcc... /usr/bin/ld checking if the linker (/usr/bin/ld) is GNU ld... yes checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B checking the name lister (/usr/bin/nm -B) interface... BSD nm checking whether ln -s works... yes checking the maximum length of command line arguments... 1572864 checking whether the shell understands some XSI constructs... yes checking whether the shell understands +=... yes checking how to convert x86_64-unknown-linux-gnu file names to x86_64-unknown-linux-gnu format... func_convert_file_noop checking how to convert x86_64-unknown-linux-gnu file names to toolchain format... func_convert_file_noop checking for /usr/bin/ld option to reload object files... -r checking for objdump... objdump checking how to recognize dependent libraries... pass_all checking for dlltool... no checking how to associate runtime and link libraries... printf %s\n checking for g++... g++ checking whether we are using the GNU C++ compiler... yes checking whether g++ accepts -g... yes checking dependency style of g++... gcc3 checking for archiver @FILE support... @ checking for strip... strip checking for ranlib... ranlib checking command to parse /usr/bin/nm -B output from gcc object... ok checking for sysroot... no checking for mt... mt checking if mt is a manifest tool... no checking how to run the C preprocessor... gcc -E checking for ANSI C header files... yes checking for sys/types.h... yes checking for sys/stat.h... yes checking for stdlib.h... yes checking for string.h... yes checking for memory.h... yes checking for strings.h... yes checking for inttypes.h... yes checking for stdint.h... yes checking for unistd.h... yes checking for dlfcn.h... yes checking for objdir... .libs checking if gcc supports -fno-rtti -fno-exceptions... no checking for gcc option to produce PIC... -fPIC -DPIC checking if gcc PIC flag -fPIC -DPIC works... yes checking if gcc static flag -static works... yes checking if gcc supports -c -o file.o... yes checking if gcc supports -c -o file.o... (cached) yes checking whether the gcc linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes checking whether -lc should be explicitly linked in... no checking dynamic linker characteristics... GNU/Linux ld.so checking how to hardcode library paths into programs... immediate checking whether stripping libraries is possible... yes checking if libtool supports shared libraries... yes checking whether to build shared libraries... yes checking whether to build static libraries... yes checking how to run the C++ preprocessor... g++ -E checking for ld used by g++... /usr/bin/ld -m elf_x86_64 checking if the linker (/usr/bin/ld -m elf_x86_64) is GNU ld... yes checking whether the g++ linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes checking for g++ option to produce PIC... -fPIC -DPIC checking if g++ PIC flag -fPIC -DPIC works... yes checking if g++ static flag -static works... yes checking if g++ supports -c -o file.o... yes checking if g++ supports -c -o file.o... (cached) yes checking whether the g++ linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes checking dynamic linker characteristics... (cached) GNU/Linux ld.so checking how to hardcode library paths into programs... immediate configure: creating ./config.lt config.lt: creating libtool
Re: Review Request 25403: Override entrypoint when shell enabled in Docker
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25403/#review53533 --- Ship it! But I re-opened the '-c' issue so that you can add a comment above argv.push_back(-c) so that it's clear. Thanks Tim! - Benjamin Hindman On Sept. 11, 2014, 12:40 a.m., Timothy Chen wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25403/ --- (Updated Sept. 11, 2014, 12:40 a.m.) Review request for mesos, Benjamin Hindman and Jie Yu. Bugs: MESOS-1770 https://issues.apache.org/jira/browse/MESOS-1770 Repository: mesos-git Description --- Review: https://reviews.apache.org/r/25403 Diffs - src/docker/docker.cpp af51ac9058382aede61b09e06e312ad2ce6de03e Diff: https://reviews.apache.org/r/25403/diff/ Testing --- make check Thanks, Timothy Chen
Re: Review Request 24776: Add docker containerizer destroy tests
On Sept. 9, 2014, 6:15 p.m., Benjamin Hindman wrote: Why did you need to mock DockerContainerizerProcess in order to write these tests? Couldn't you have just used the existing MockDockerContainerizer? Timothy Chen wrote: I wanted to simulate having destroy called in a pull/fetching state, so I thought the only way to do so is to mock the process since the callbacks are on DockerContainerizerProcess and not the Containerizer, so the callbacks for fetch and pull blocks and I can call destroy in that state and verify it was able to destroy. Yup yup, you're correct, my apologies! - Benjamin --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/24776/#review52758 --- On Sept. 15, 2014, 8:33 p.m., Timothy Chen wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/24776/ --- (Updated Sept. 15, 2014, 8:33 p.m.) Review request for mesos, Benjamin Hindman and Jie Yu. Repository: mesos-git Description --- Review: https://reviews.apache.org/r/24776 Diffs - src/slave/containerizer/docker.hpp fbbd45d77e5f2f74ca893552f85eb893b3dd948f src/slave/containerizer/docker.cpp 0febbac5df4126f6c8d9a06dd0ba1668d041b34a src/tests/docker_containerizer_tests.cpp 8654f9c787bd207f6a7b821651e0c083bea9dc8a Diff: https://reviews.apache.org/r/24776/diff/ Testing --- make check Thanks, Timothy Chen
Re: Review Request 25569: Refactor test environment validations
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25569/#review53541 --- src/tests/environment.cpp https://reviews.apache.org/r/25569/#comment93236 vectorprocess::OwnedTestFilter testFilters; we will eventually move process::Owned to unique_ptr so please start using it :) - Dominic Hamon On Sept. 16, 2014, 12:07 a.m., Timothy Chen wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25569/ --- (Updated Sept. 16, 2014, 12:07 a.m.) Review request for mesos and Ben Mahler. Repository: mesos-git Description --- Review: https://reviews.apache.org/r/25569 Refactor how validation is done in the environment by defining a standard test filter interface, and only execute the validation once. Diffs - src/tests/environment.cpp 2274251aaf653d83c2d03ef2186763978067a747 Diff: https://reviews.apache.org/r/25569/diff/ Testing --- make check Thanks, Timothy Chen
Re: Completed tasks remains in TASK_RUNNING when framework is disconnected
Okay - that only solves half of the problem for us: users will still see their frameworks as running even though they completed but it is a first step. Let's continue the discussion in a JIRA ticket; I'll create one shortly. Thanks for helping out! Niklas On 15 September 2014 18:17, Benjamin Mahler benjamin.mah...@gmail.com wrote: On Mon, Sep 15, 2014 at 3:11 PM, Niklas Nielsen nik...@mesosphere.io wrote: Thanks for your input Ben! (Comments inlined) On 15 September 2014 12:35, Benjamin Mahler benjamin.mah...@gmail.com wrote: To ensure that the architecture of mesos remains a scalable one, we want to persist state in the leaves of the system as much as possible. This is why the master has never persisted tasks, task states, or status updates. Note that status updates can contain arbitrarily large amounts of data at the current time (example impact of this: https://issues.apache.org/jira/browse/MESOS-1746). Adam, I think solution (2) you listed above of including a terminal indicator in the _private_ message between slave and master would easily allow us to release the resources at the correct time. We would still hold the correct task state in the master, and would maintain the status update stream invariants for frameworks (guaranteed ordered delivery). This would be simpler to implement with my recent change here, because you no longer have to remove the task to release the resources: https://reviews.apache.org/r/25568/ We should still hold the correct task state; meaning the actual state of the task on the slave? Correct, as in, just maintaining the existing task state semantics in the master (for correctness of reconciliation / status update ordering). Then the auxiliary field should represent last known status, which may not necessarily be terminal. For example, a staging update followed by a running update while the framework is disconnected will show as staging still - or am I missing something? It would still show as staging, the running update would never be sent to the master since the slave is not receiving acknowledgements. But if there were a terminal task, then the slave would be setting the auxiliary field in the StatusUpdateMessage and the master will know to release the resources. + backwards compatibility Longer term, adding pipelining of status updates would be a nice improvement (similar to what you listed in (1) above). But as Vinod said, it will require care to ensure we maintain the stream invariants for frameworks (i.e. probably need to send multiple updates in 1 message). How does this sound? Sounds great - we would love to help out with (2). Would you be up for shepherding such a change? Yep! The changes needed should be based off of https://reviews.apache.org/r/25568/ since it changes the resource releasing in the master. On Thu, Sep 11, 2014 at 12:02 PM, Adam Bordelon a...@mesosphere.io wrote: Definitely relevant. If the master could be trusted to persist all the task status updates, then they could be queued up at the master instead of the slave once the master has acknowledged its receipt. Then the master could have the most up-to-date task state and can recover the resources as soon as it receives a terminal update, even if the framework is disconnected and unable to receive/ack the status updates. Then, once the framework reconnects, the master will be responsible for sending its queued status updates. We will still need a queue on the slave side, but only for updates that the master has not persisted and ack'ed, primarily during the scenario when the slave is disconnected from the master. On Thu, Sep 11, 2014 at 10:17 AM, Vinod Kone vinodk...@gmail.com wrote: The semantics of these changes would have an impact on the upcoming task reconciliation. @BenM: Can you chime in here on how this fits into the task reconciliation work that you've been leading? On Wed, Sep 10, 2014 at 6:12 PM, Adam Bordelon a...@mesosphere.io wrote: I agree with Niklas that if the executor has sent a terminal status update to the slave, then the task is done and the master should be able to recover those resources. Only sending the oldest status update to the master, especially in the case of framework failover, prevents these resources from being recovered in a timely manner. I see a couple of options for getting around this, each with their own disadvantages. 1) Send the entire status update stream to the master. Once the master sees the terminal status update, it will removeTask and recover the resources. Future resends of the update will be forwarded to the scheduler, but the master will ignore (with warning and
Re: Review Request 25695: Update to enable systemd control of mesos services
On Sept. 16, 2014, 3:38 p.m., Mesos ReviewBot wrote: Bad patch! Reviews applied: [25695] Failed command: ./configure Error: checking build system type... x86_64-unknown-linux-gnu checking host system type... x86_64-unknown-linux-gnu checking target system type... x86_64-unknown-linux-gnu checking for a BSD-compatible install... /usr/bin/install -c checking whether build environment is sane... yes checking for a thread-safe mkdir -p... /bin/mkdir -p checking for gawk... no checking for mawk... mawk checking whether make sets $(MAKE)... yes checking whether make supports nested variables... yes checking for style of include used by make... GNU checking for gcc... gcc checking whether the C compiler works... yes checking for C compiler default output file name... a.out checking for suffix of executables... checking whether we are cross compiling... no checking for suffix of object files... o checking whether we are using the GNU C compiler... yes checking whether gcc accepts -g... yes checking for gcc option to accept ISO C89... none needed checking whether gcc understands -c and -o together... yes checking dependency style of gcc... gcc3 checking for ar... ar checking the archiver (ar) interface... ar checking how to print strings... printf checking for a sed that does not truncate output... /bin/sed checking for grep that handles long lines and -e... /bin/grep checking for egrep... /bin/grep -E checking for fgrep... /bin/grep -F checking for ld used by gcc... /usr/bin/ld checking if the linker (/usr/bin/ld) is GNU ld... yes checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B checking the name lister (/usr/bin/nm -B) interface... BSD nm checking whether ln -s works... yes checking the maximum length of command line arguments... 1572864 checking whether the shell understands some XSI constructs... yes checking whether the shell understands +=... yes checking how to convert x86_64-unknown-linux-gnu file names to x86_64-unknown-linux-gnu format... func_convert_file_noop checking how to convert x86_64-unknown-linux-gnu file names to toolchain format... func_convert_file_noop checking for /usr/bin/ld option to reload object files... -r checking for objdump... objdump checking how to recognize dependent libraries... pass_all checking for dlltool... no checking how to associate runtime and link libraries... printf %s\n checking for g++... g++ checking whether we are using the GNU C++ compiler... yes checking whether g++ accepts -g... yes checking dependency style of g++... gcc3 checking for archiver @FILE support... @ checking for strip... strip checking for ranlib... ranlib checking command to parse /usr/bin/nm -B output from gcc object... ok checking for sysroot... no checking for mt... mt checking if mt is a manifest tool... no checking how to run the C preprocessor... gcc -E checking for ANSI C header files... yes checking for sys/types.h... yes checking for sys/stat.h... yes checking for stdlib.h... yes checking for string.h... yes checking for memory.h... yes checking for strings.h... yes checking for inttypes.h... yes checking for stdint.h... yes checking for unistd.h... yes checking for dlfcn.h... yes checking for objdir... .libs checking if gcc supports -fno-rtti -fno-exceptions... no checking for gcc option to produce PIC... -fPIC -DPIC checking if gcc PIC flag -fPIC -DPIC works... yes checking if gcc static flag -static works... yes checking if gcc supports -c -o file.o... yes checking if gcc supports -c -o file.o... (cached) yes checking whether the gcc linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes checking whether -lc should be explicitly linked in... no checking dynamic linker characteristics... GNU/Linux ld.so checking how to hardcode library paths into programs... immediate checking whether stripping libraries is possible... yes checking if libtool supports shared libraries... yes checking whether to build shared libraries... yes checking whether to build static libraries... yes checking how to run the C++ preprocessor... g++ -E checking for ld used by g++... /usr/bin/ld -m elf_x86_64 checking if the linker (/usr/bin/ld -m elf_x86_64) is GNU ld... yes checking whether the g++ linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes checking for g++ option to produce PIC... -fPIC -DPIC checking if g++ PIC flag -fPIC -DPIC works... yes checking if g++ static flag -static works... yes checking if g++ supports -c -o file.o... yes checking if g++ supports -c -o file.o... (cached) yes checking whether the g++ linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes checking dynamic linker characteristics... (cached) GNU/Linux ld.so checking how to hardcode library paths into programs... immediate configure: creating
Re: Review Request 25597: Added a version checker class to stout.
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25597/#review53553
---
Looking much better, thanks Kapil!!
3rdparty/libprocess/3rdparty/stout/include/stout/version.hpp
https://reviews.apache.org/r/25597/#comment93251
What about s/versionStrings/split/ as the variable name?
3rdparty/libprocess/3rdparty/stout/include/stout/version.hpp
https://reviews.apache.org/r/25597/#comment93244
Any tests for the error cases?
For example, a test for 0.1.2.3 would cause this code to crash, because
it will go over the end of the array.
3rdparty/libprocess/3rdparty/stout/include/stout/version.hpp
https://reviews.apache.org/r/25597/#comment93246
Can you include the parse error here?
e.g.
```
return Error(Invalid version component ' + versionStrings[i] + ': +
numify.error());
```
When we return an error string, we typically do not include the input
because it often leads to double logging, consider what a caller might log:
```
TryVersion version = Version::parse(v);
if (version.isError()) {
LOG(ERROR) Failed to parse version ' v ':
version.error();
}
```
In this case, the best thing we can do is to return an Error of the
_reason_ the version was unparsable.
3rdparty/libprocess/3rdparty/stout/include/stout/version.hpp
https://reviews.apache.org/r/25597/#comment93247
Another newline here :)
3rdparty/libprocess/3rdparty/stout/include/stout/version.hpp
https://reviews.apache.org/r/25597/#comment93248
Missing ostream include?
3rdparty/libprocess/3rdparty/stout/tests/version_tests.cpp
https://reviews.apache.org/r/25597/#comment93249
Some error cases would be great here!
- Ben Mahler
On Sept. 16, 2014, 5:11 a.m., Kapil Arya wrote:
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25597/
---
(Updated Sept. 16, 2014, 5:11 a.m.)
Review request for mesos, Adam B and Niklas Nielsen.
Repository: mesos-git
Description
---
Currently there is no facility in Mesos for checking compatibility of various
Mesos components that could have been built at different times with
potentially different Mesos versions. This requirement is especially
important for doing various compatibility checks between Mesos and Mesos
modules (WIP).
- Features major, minor, and patch numbers.
- Convenience functions for comparing two versions.
Diffs
-
3rdparty/libprocess/3rdparty/Makefile.am
db9766d70adb9076946cd2b467c55636fe5f7235
3rdparty/libprocess/3rdparty/stout/Makefile.am
b6464de53c3873ecd0b62a08ca9aac530043ffb9
3rdparty/libprocess/3rdparty/stout/include/Makefile.am
6fa5b741bdd7f089ba93bf6fea43b9f39f8f0edb
3rdparty/libprocess/3rdparty/stout/include/stout/os.hpp
5bbf829b3fa5d09a92e1d64c52c1fc7eed73fc91
3rdparty/libprocess/3rdparty/stout/include/stout/version.hpp PRE-CREATION
3rdparty/libprocess/3rdparty/stout/tests/version_tests.cpp PRE-CREATION
Diff: https://reviews.apache.org/r/25597/diff/
Testing
---
Added a stout test and ran make check
Thanks,
Kapil Arya
Re: Review Request 25695: Update to enable systemd control of mesos services
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25695/ --- (Updated Sept. 16, 2014, 6:19 p.m.) Review request for mesos, Jie Yu and Vinod Kone. Changes --- minor update to pass on missing dependency check. Bugs: MESOS-1195 https://issues.apache.org/jira/browse/MESOS-1195 Repository: mesos-git Description --- This update enables support for systemd co-managed cgroup controllers Diffs (updated) - configure.ac c4b4391 src/linux/cgroups.cpp 5093b4c src/slave/containerizer/isolators/cgroups/cpushare.cpp b1cad47 Diff: https://reviews.apache.org/r/25695/diff/ Testing --- systemctl start mesos-master mesos-slave several runs of 'mesos execute' to verify creation and cleanup. systemctl stop mesos-master mesos-slave make check Thanks, Timothy St. Clair
Re: Review Request 25663: MESOS-1392: MasterDetector now returns a None when it cannot read the content of the ZNode it has detected.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25663/#review53558 --- src/master/detector.cpp https://reviews.apache.org/r/25663/#comment93256 Any way to explicitly ignore the no node case? Here we're ignoring all failures, which is a bit scary, since it's not obvious why the more serious errors would be caught elsewhere. I haven't put a lot of thought into this, could we return a FutureOptionstring data? Or should we consider wrapping our things in some kind of ZKOperationstring which lets us look at the various error cases? Happy to chat further! - Ben Mahler On Sept. 15, 2014, 9:24 p.m., Jiang Yan Xu wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25663/ --- (Updated Sept. 15, 2014, 9:24 p.m.) Review request for mesos and Ben Mahler. Bugs: MESOS-1392 https://issues.apache.org/jira/browse/MESOS-1392 Repository: mesos-git Description --- See summary. Diffs - src/master/detector.cpp 6436b8ee7e1ab6451a6b999a1cfbb2f79190e6ca Diff: https://reviews.apache.org/r/25663/diff/ Testing --- make check. Thanks, Jiang Yan Xu
Re: Review Request 25597: Added a version checker class to stout.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25597/ --- (Updated Sept. 16, 2014, 2:48 p.m.) Review request for mesos, Adam B and Niklas Nielsen. Changes --- Added back the accessor methods major(), minor(), and patch(). More error checks. Addressed Ben's comments. Repository: mesos-git Description --- Currently there is no facility in Mesos for checking compatibility of various Mesos components that could have been built at different times with potentially different Mesos versions. This requirement is especially important for doing various compatibility checks between Mesos and Mesos modules (WIP). - Features major, minor, and patch numbers. - Convenience functions for comparing two versions. Diffs (updated) - 3rdparty/libprocess/3rdparty/Makefile.am db9766d70adb9076946cd2b467c55636fe5f7235 3rdparty/libprocess/3rdparty/stout/Makefile.am b6464de53c3873ecd0b62a08ca9aac530043ffb9 3rdparty/libprocess/3rdparty/stout/include/Makefile.am 6fa5b741bdd7f089ba93bf6fea43b9f39f8f0edb 3rdparty/libprocess/3rdparty/stout/include/stout/os.hpp 5bbf829b3fa5d09a92e1d64c52c1fc7eed73fc91 3rdparty/libprocess/3rdparty/stout/include/stout/version.hpp PRE-CREATION 3rdparty/libprocess/3rdparty/stout/tests/version_tests.cpp PRE-CREATION Diff: https://reviews.apache.org/r/25597/diff/ Testing --- Added a stout test and ran make check Thanks, Kapil Arya
Re: Review Request 25597: Added a version checker class to stout.
On Sept. 15, 2014, 6:46 p.m., Ben Mahler wrote:
3rdparty/libprocess/3rdparty/stout/include/stout/version.hpp, lines 32-34
https://reviews.apache.org/r/25597/diff/3/?file=688423#file688423line32
The program will crash if the split is non-numeric, because you'll call
.get() on a numify operation, are you intending this to occur?
A comment would be nice!
Kapil Arya wrote:
Good catch! The question now is how to handle the situation where a
string such as 1.a.2 is passed on? Should it be considered an error or
should it be interpreted as 1.0.0? If it should be considered an error,
how should be change the contructor to reflect that?
Ben Mahler wrote:
You can add a 'parse' method akin to what was done in duration.hpp:
```
static TryVersion parse(const std::string version);
```
This makes the error explicit, and we no longer need the (major(),
minor(), patch()) accessor methods, since the member variables would be const.
Kapil Arya wrote:
There is one trouble with making major/minor as const fields. Glibc
defines major and minor as macros and that breaks the compilation when these
fields are initialized in a contructor as : major(majorVersion),
minor(minorVersion), patch(patchVersion). The preprocessor expands them to
gnu_dev_major and gnu_dev_minor respectively.
The alternative solution is to go back to accessor functions and mark
these fields as private. Another alternative is to rename the const fields as
majorVersion, minorVersion, and patchVersion.
Is there any preference here?
I see, seems a bit tricky to skirt around the macro issue:
```
// This caller is ok?
int major = version.major();
// The constructor argument is ok?
Version::Version(int major, int minor, int patch) {}
```
What if we remove the accessors, and make them const private as 'major_',
'minor_', 'patch_'?
I'm curious why we'd need access to the version components, rather than just
using the comparison operators or the stringify operator.
I could also envision some compatibility operators to ensure that it's the same
major version if = 1.0, or same minor version if 1.0.
How about we hide them for now and defer the decision for later should the need
arise?
- Ben
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25597/#review53370
---
On Sept. 16, 2014, 6:48 p.m., Kapil Arya wrote:
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25597/
---
(Updated Sept. 16, 2014, 6:48 p.m.)
Review request for mesos, Adam B and Niklas Nielsen.
Repository: mesos-git
Description
---
Currently there is no facility in Mesos for checking compatibility of various
Mesos components that could have been built at different times with
potentially different Mesos versions. This requirement is especially
important for doing various compatibility checks between Mesos and Mesos
modules (WIP).
- Features major, minor, and patch numbers.
- Convenience functions for comparing two versions.
Diffs
-
3rdparty/libprocess/3rdparty/Makefile.am
db9766d70adb9076946cd2b467c55636fe5f7235
3rdparty/libprocess/3rdparty/stout/Makefile.am
b6464de53c3873ecd0b62a08ca9aac530043ffb9
3rdparty/libprocess/3rdparty/stout/include/Makefile.am
6fa5b741bdd7f089ba93bf6fea43b9f39f8f0edb
3rdparty/libprocess/3rdparty/stout/include/stout/os.hpp
5bbf829b3fa5d09a92e1d64c52c1fc7eed73fc91
3rdparty/libprocess/3rdparty/stout/include/stout/version.hpp PRE-CREATION
3rdparty/libprocess/3rdparty/stout/tests/version_tests.cpp PRE-CREATION
Diff: https://reviews.apache.org/r/25597/diff/
Testing
---
Added a stout test and ran make check
Thanks,
Kapil Arya
Re: Review Request 25695: Update to enable systemd control of mesos services
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25695/#review53571 --- Patch looks great! Reviews applied: [25695] All tests passed. - Mesos ReviewBot On Sept. 16, 2014, 6:19 p.m., Timothy St. Clair wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25695/ --- (Updated Sept. 16, 2014, 6:19 p.m.) Review request for mesos, Jie Yu and Vinod Kone. Bugs: MESOS-1195 https://issues.apache.org/jira/browse/MESOS-1195 Repository: mesos-git Description --- This update enables support for systemd co-managed cgroup controllers Diffs - configure.ac c4b4391 src/linux/cgroups.cpp 5093b4c src/slave/containerizer/isolators/cgroups/cpushare.cpp b1cad47 Diff: https://reviews.apache.org/r/25695/diff/ Testing --- systemctl start mesos-master mesos-slave several runs of 'mesos execute' to verify creation and cleanup. systemctl stop mesos-master mesos-slave make check Thanks, Timothy St. Clair
Re: Review Request 25597: Added a version checker class to stout.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25597/#review53569 --- Almost there!! I think we can get away with not needing to expose major, minor, and patch for now. If you look at the use-case for os::Release, we only needed the comparison operator. That would avoid the majorVersion clunkiness, what do you think? 3rdparty/libprocess/3rdparty/stout/include/stout/version.hpp https://reviews.apache.org/r/25597/#comment93271 We've tried to stick with // style comments, can you change this one? 3rdparty/libprocess/3rdparty/stout/include/stout/version.hpp https://reviews.apache.org/r/25597/#comment93276 Can you move this down to where it is used? 3rdparty/libprocess/3rdparty/stout/include/stout/version.hpp https://reviews.apache.org/r/25597/#comment93275 s/. M/; m/ I don't think you need the int's here on the stringify calls. Also, we don't use periods at the end of any of our log lines. There are two reasons for this, (1) the code needed is clumsy in some situations, and (2) it's tricky with nested errors. For example, who's responsibility is it here to end with a period? If the caller does the following we end up with two periods! ``` LOG(ERROR) Failed to parse: error.get() .; ``` 3rdparty/libprocess/3rdparty/stout/tests/version_tests.cpp https://reviews.apache.org/r/25597/#comment93269 Two newlines between top level definitions. 3rdparty/libprocess/3rdparty/stout/tests/version_tests.cpp https://reviews.apache.org/r/25597/#comment93270 Ditto here. - Ben Mahler On Sept. 16, 2014, 6:48 p.m., Kapil Arya wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25597/ --- (Updated Sept. 16, 2014, 6:48 p.m.) Review request for mesos, Adam B and Niklas Nielsen. Repository: mesos-git Description --- Currently there is no facility in Mesos for checking compatibility of various Mesos components that could have been built at different times with potentially different Mesos versions. This requirement is especially important for doing various compatibility checks between Mesos and Mesos modules (WIP). - Features major, minor, and patch numbers. - Convenience functions for comparing two versions. Diffs - 3rdparty/libprocess/3rdparty/Makefile.am db9766d70adb9076946cd2b467c55636fe5f7235 3rdparty/libprocess/3rdparty/stout/Makefile.am b6464de53c3873ecd0b62a08ca9aac530043ffb9 3rdparty/libprocess/3rdparty/stout/include/Makefile.am 6fa5b741bdd7f089ba93bf6fea43b9f39f8f0edb 3rdparty/libprocess/3rdparty/stout/include/stout/os.hpp 5bbf829b3fa5d09a92e1d64c52c1fc7eed73fc91 3rdparty/libprocess/3rdparty/stout/include/stout/version.hpp PRE-CREATION 3rdparty/libprocess/3rdparty/stout/tests/version_tests.cpp PRE-CREATION Diff: https://reviews.apache.org/r/25597/diff/ Testing --- Added a stout test and ran make check Thanks, Kapil Arya
Re: Review Request 25597: Added a version checker class to stout.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25597/#review53572 --- Bad patch! Reviews applied: [25597] Failed command: ./support/mesos-style.py Error: Checking 506 files using filter --filter=-,+build/class,+build/deprecated,+build/endif_comment,+readability/todo,+readability/namespace,+runtime/vlog,+whitespace/blank_line,+whitespace/comma,+whitespace/end_of_line,+whitespace/ending_newline,+whitespace/forcolon,+whitespace/indent,+whitespace/line_length,+whitespace/tab,+whitespace/todo 3rdparty/libprocess/3rdparty/stout/include/stout/version.hpp:40: Line ends in whitespace. Consider deleting these extra spaces. [whitespace/end_of_line] [4] Total errors found: 1 - Mesos ReviewBot On Sept. 16, 2014, 6:48 p.m., Kapil Arya wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25597/ --- (Updated Sept. 16, 2014, 6:48 p.m.) Review request for mesos, Adam B and Niklas Nielsen. Repository: mesos-git Description --- Currently there is no facility in Mesos for checking compatibility of various Mesos components that could have been built at different times with potentially different Mesos versions. This requirement is especially important for doing various compatibility checks between Mesos and Mesos modules (WIP). - Features major, minor, and patch numbers. - Convenience functions for comparing two versions. Diffs - 3rdparty/libprocess/3rdparty/Makefile.am db9766d70adb9076946cd2b467c55636fe5f7235 3rdparty/libprocess/3rdparty/stout/Makefile.am b6464de53c3873ecd0b62a08ca9aac530043ffb9 3rdparty/libprocess/3rdparty/stout/include/Makefile.am 6fa5b741bdd7f089ba93bf6fea43b9f39f8f0edb 3rdparty/libprocess/3rdparty/stout/include/stout/os.hpp 5bbf829b3fa5d09a92e1d64c52c1fc7eed73fc91 3rdparty/libprocess/3rdparty/stout/include/stout/version.hpp PRE-CREATION 3rdparty/libprocess/3rdparty/stout/tests/version_tests.cpp PRE-CREATION Diff: https://reviews.apache.org/r/25597/diff/ Testing --- Added a stout test and ran make check Thanks, Kapil Arya
Hadoop on Mesos
Hey everyone, I've been working on a potential extension to Hadoop on Mesos which allows the framework to potentially release allocated (but idle) TaskTracker slots if they are doing nothing. This helps release resources hadoop is allocated but not using, to increase overall cluster utilisation when multiple frameworks are involved. This scenario most commonly appears when you have a large job with an expensive reduce phase. While the reducers are running the map slots are completely idle, and therefore are unable to be offered to other frameworks that could make use of the resources. However, there are various intricacies of doing this, and it's quite hacky, largely because it requires we change the number of available slots on a TaskTracker while it's running. I'd be interested to hear anyones thoughts on the idea... Especially those that worked on the hadoop framework early on! Pull request is here https://github.com/mesos/hadoop/pull/33 and another issue related to this solution can be found here https://github.com/mesos/hadoop/issues/32. Cheers, Tom.
Re: Review Request 25597: Added a version checker class to stout.
On Sept. 15, 2014, 2:46 p.m., Ben Mahler wrote:
3rdparty/libprocess/3rdparty/stout/include/stout/version.hpp, lines 32-34
https://reviews.apache.org/r/25597/diff/3/?file=688423#file688423line32
The program will crash if the split is non-numeric, because you'll call
.get() on a numify operation, are you intending this to occur?
A comment would be nice!
Kapil Arya wrote:
Good catch! The question now is how to handle the situation where a
string such as 1.a.2 is passed on? Should it be considered an error or
should it be interpreted as 1.0.0? If it should be considered an error,
how should be change the contructor to reflect that?
Ben Mahler wrote:
You can add a 'parse' method akin to what was done in duration.hpp:
```
static TryVersion parse(const std::string version);
```
This makes the error explicit, and we no longer need the (major(),
minor(), patch()) accessor methods, since the member variables would be const.
Kapil Arya wrote:
There is one trouble with making major/minor as const fields. Glibc
defines major and minor as macros and that breaks the compilation when these
fields are initialized in a contructor as : major(majorVersion),
minor(minorVersion), patch(patchVersion). The preprocessor expands them to
gnu_dev_major and gnu_dev_minor respectively.
The alternative solution is to go back to accessor functions and mark
these fields as private. Another alternative is to rename the const fields as
majorVersion, minorVersion, and patchVersion.
Is there any preference here?
Ben Mahler wrote:
I see, seems a bit tricky to skirt around the macro issue:
```
// This caller is ok?
int major = version.major();
// The constructor argument is ok?
Version::Version(int major, int minor, int patch) {}
```
What if we remove the accessors, and make them const private as 'major_',
'minor_', 'patch_'?
I'm curious why we'd need access to the version components, rather than
just using the comparison operators or the stringify operator.
I could also envision some compatibility operators to ensure that it's
the same major version if = 1.0, or same minor version if 1.0.
How about we hide them for now and defer the decision for later should
the need arise?
I can't think of any reason for explicitly accessing the version components so
I agree that we should hide them for now.
Along the same lines, the next question (came up with discussion with Bernd) is
that why shouldn't we have arbitrary number of components rather than just
having major, minor, and patch? Shouldn't we just keep a vector of components
that is allowed to grow to arbitrary lengths? We certainly see more than just
major/minor/patch in Linux kernel version for example. The counter argument is
that a lot of systems (including Linux) are trying to get to a simpler
versioning scheme with less components :).
- Kapil
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25597/#review53370
---
On Sept. 16, 2014, 2:48 p.m., Kapil Arya wrote:
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25597/
---
(Updated Sept. 16, 2014, 2:48 p.m.)
Review request for mesos, Adam B and Niklas Nielsen.
Repository: mesos-git
Description
---
Currently there is no facility in Mesos for checking compatibility of various
Mesos components that could have been built at different times with
potentially different Mesos versions. This requirement is especially
important for doing various compatibility checks between Mesos and Mesos
modules (WIP).
- Features major, minor, and patch numbers.
- Convenience functions for comparing two versions.
Diffs
-
3rdparty/libprocess/3rdparty/Makefile.am
db9766d70adb9076946cd2b467c55636fe5f7235
3rdparty/libprocess/3rdparty/stout/Makefile.am
b6464de53c3873ecd0b62a08ca9aac530043ffb9
3rdparty/libprocess/3rdparty/stout/include/Makefile.am
6fa5b741bdd7f089ba93bf6fea43b9f39f8f0edb
3rdparty/libprocess/3rdparty/stout/include/stout/os.hpp
5bbf829b3fa5d09a92e1d64c52c1fc7eed73fc91
3rdparty/libprocess/3rdparty/stout/include/stout/version.hpp PRE-CREATION
3rdparty/libprocess/3rdparty/stout/tests/version_tests.cpp PRE-CREATION
Diff: https://reviews.apache.org/r/25597/diff/
Testing
---
Added a stout test and ran make check
Thanks,
Kapil Arya
Re: Review Request 25597: Added a version checker class to stout.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25597/ --- (Updated Sept. 16, 2014, 4:39 p.m.) Review request for mesos, Adam B and Niklas Nielsen. Changes --- Remove accessor methods. Make component private consts. Repository: mesos-git Description --- Currently there is no facility in Mesos for checking compatibility of various Mesos components that could have been built at different times with potentially different Mesos versions. This requirement is especially important for doing various compatibility checks between Mesos and Mesos modules (WIP). - Features major, minor, and patch numbers. - Convenience functions for comparing two versions. Diffs (updated) - 3rdparty/libprocess/3rdparty/Makefile.am db9766d70adb9076946cd2b467c55636fe5f7235 3rdparty/libprocess/3rdparty/stout/Makefile.am b6464de53c3873ecd0b62a08ca9aac530043ffb9 3rdparty/libprocess/3rdparty/stout/include/Makefile.am 6fa5b741bdd7f089ba93bf6fea43b9f39f8f0edb 3rdparty/libprocess/3rdparty/stout/include/stout/os.hpp 5bbf829b3fa5d09a92e1d64c52c1fc7eed73fc91 3rdparty/libprocess/3rdparty/stout/include/stout/version.hpp PRE-CREATION 3rdparty/libprocess/3rdparty/stout/tests/version_tests.cpp PRE-CREATION Diff: https://reviews.apache.org/r/25597/diff/ Testing --- Added a stout test and ran make check Thanks, Kapil Arya
Re: Review Request 25597: Added a version checker class to stout.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25597/ --- (Updated Sept. 16, 2014, 4:40 p.m.) Review request for mesos, Adam B and Niklas Nielsen. Changes --- Fixed typo from revision 9. Repository: mesos-git Description --- Currently there is no facility in Mesos for checking compatibility of various Mesos components that could have been built at different times with potentially different Mesos versions. This requirement is especially important for doing various compatibility checks between Mesos and Mesos modules (WIP). - Features major, minor, and patch numbers. - Convenience functions for comparing two versions. Diffs (updated) - 3rdparty/libprocess/3rdparty/Makefile.am db9766d70adb9076946cd2b467c55636fe5f7235 3rdparty/libprocess/3rdparty/stout/Makefile.am b6464de53c3873ecd0b62a08ca9aac530043ffb9 3rdparty/libprocess/3rdparty/stout/include/Makefile.am 6fa5b741bdd7f089ba93bf6fea43b9f39f8f0edb 3rdparty/libprocess/3rdparty/stout/include/stout/os.hpp 5bbf829b3fa5d09a92e1d64c52c1fc7eed73fc91 3rdparty/libprocess/3rdparty/stout/include/stout/version.hpp PRE-CREATION 3rdparty/libprocess/3rdparty/stout/tests/version_tests.cpp PRE-CREATION Diff: https://reviews.apache.org/r/25597/diff/ Testing --- Added a stout test and ran make check Thanks, Kapil Arya
Re: Review Request 25597: Added a version checker class to stout.
On Sept. 15, 2014, 6:46 p.m., Ben Mahler wrote:
3rdparty/libprocess/3rdparty/stout/include/stout/version.hpp, lines 32-34
https://reviews.apache.org/r/25597/diff/3/?file=688423#file688423line32
The program will crash if the split is non-numeric, because you'll call
.get() on a numify operation, are you intending this to occur?
A comment would be nice!
Kapil Arya wrote:
Good catch! The question now is how to handle the situation where a
string such as 1.a.2 is passed on? Should it be considered an error or
should it be interpreted as 1.0.0? If it should be considered an error,
how should be change the contructor to reflect that?
Ben Mahler wrote:
You can add a 'parse' method akin to what was done in duration.hpp:
```
static TryVersion parse(const std::string version);
```
This makes the error explicit, and we no longer need the (major(),
minor(), patch()) accessor methods, since the member variables would be const.
Kapil Arya wrote:
There is one trouble with making major/minor as const fields. Glibc
defines major and minor as macros and that breaks the compilation when these
fields are initialized in a contructor as : major(majorVersion),
minor(minorVersion), patch(patchVersion). The preprocessor expands them to
gnu_dev_major and gnu_dev_minor respectively.
The alternative solution is to go back to accessor functions and mark
these fields as private. Another alternative is to rename the const fields as
majorVersion, minorVersion, and patchVersion.
Is there any preference here?
Ben Mahler wrote:
I see, seems a bit tricky to skirt around the macro issue:
```
// This caller is ok?
int major = version.major();
// The constructor argument is ok?
Version::Version(int major, int minor, int patch) {}
```
What if we remove the accessors, and make them const private as 'major_',
'minor_', 'patch_'?
I'm curious why we'd need access to the version components, rather than
just using the comparison operators or the stringify operator.
I could also envision some compatibility operators to ensure that it's
the same major version if = 1.0, or same minor version if 1.0.
How about we hide them for now and defer the decision for later should
the need arise?
Kapil Arya wrote:
I can't think of any reason for explicitly accessing the version
components so I agree that we should hide them for now.
Along the same lines, the next question (came up with discussion with
Bernd) is that why shouldn't we have arbitrary number of components rather
than just having major, minor, and patch? Shouldn't we just keep a vector of
components that is allowed to grow to arbitrary lengths? We certainly see
more than just major/minor/patch in Linux kernel version for example. The
counter argument is that a lot of systems (including Linux) are trying to get
to a simpler versioning scheme with less components :).
Definitely! Since the first use case is replacing os::Release, we can leave a
TODO on Version to deal with more than 3 components.
- Ben
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25597/#review53370
---
On Sept. 16, 2014, 8:40 p.m., Kapil Arya wrote:
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25597/
---
(Updated Sept. 16, 2014, 8:40 p.m.)
Review request for mesos, Adam B and Niklas Nielsen.
Repository: mesos-git
Description
---
Currently there is no facility in Mesos for checking compatibility of various
Mesos components that could have been built at different times with
potentially different Mesos versions. This requirement is especially
important for doing various compatibility checks between Mesos and Mesos
modules (WIP).
- Features major, minor, and patch numbers.
- Convenience functions for comparing two versions.
Diffs
-
3rdparty/libprocess/3rdparty/Makefile.am
db9766d70adb9076946cd2b467c55636fe5f7235
3rdparty/libprocess/3rdparty/stout/Makefile.am
b6464de53c3873ecd0b62a08ca9aac530043ffb9
3rdparty/libprocess/3rdparty/stout/include/Makefile.am
6fa5b741bdd7f089ba93bf6fea43b9f39f8f0edb
3rdparty/libprocess/3rdparty/stout/include/stout/os.hpp
5bbf829b3fa5d09a92e1d64c52c1fc7eed73fc91
3rdparty/libprocess/3rdparty/stout/include/stout/version.hpp PRE-CREATION
3rdparty/libprocess/3rdparty/stout/tests/version_tests.cpp PRE-CREATION
Diff: https://reviews.apache.org/r/25597/diff/
Testing
---
Added a stout test and ran make check
Thanks,
Kapil Arya
Re: Review Request 25035: Fix for MESOS-1688
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25035/ --- (Updated Sept. 16, 2014, 9:05 nachm.) Review request for mesos and Vinod Kone. Changes --- Adjusted CHANGELOG and comments for integration with 0.21.0 instead 0.20.1. Improved warning log. Bugs: MESOS-1688 https://issues.apache.org/jira/browse/MESOS-1688 Repository: mesos-git Description --- As already explained in JIRA MESOS-1688, there are schedulers allocating memory only for the executor and not for tasks. For tasks only CPU resources are allocated in this case. Such a scheduler does not get offered any idle CPUs if the slave has nearly used up all memory. This can easily lead to a dead lock (in the application, not in Mesos). Simple example: 1. Scheduler allocates all memory of a slave for an executor 2. Scheduler launches a task for this executor (allocating 1 CPU) 3. Task finishes: 1 CPU , 0 MB memory allocatable. 4. No offers are made, as no memory is left. Scheduler will wait for offers forever. Dead lock in the application. To fix this problem, offers must be made if CPU resources are allocatable without considering allocatable memory Diffs (updated) - CHANGELOG a822cc4 src/common/resources.cpp edf36b1 src/master/constants.cpp faa1503 src/master/hierarchical_allocator_process.hpp 34f8cd6 src/master/master.cpp 18464ba src/tests/allocator_tests.cpp 774528a Diff: https://reviews.apache.org/r/25035/diff/ Testing --- Deployed patched Mesos 0.19.1 on a small cluster with 3 slaves and tested running multiple parallel Spark jobs in fine-grained mode to saturate allocatable memory. The jobs run fine now. This load always caused a dead lock in all Spark jobs within one minute with the unpatched Mesos. Thanks, Martin Weindel
Re: Review Request 25597: Added a version checker class to stout.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25597/#review53595 --- Ship it! Thanks Kapil, looks great! I will get this committed for you shortly, I'll just add a TODO per your comments on more than 3 version components and I'll remove the single quotes per my comment below. 3rdparty/libprocess/3rdparty/stout/include/stout/version.hpp https://reviews.apache.org/r/25597/#comment93304 Looks like the single quotes here are unnecessary since these are just numbers? We could say: maximum 3 components allowed instead of using the colon, does that read a little better? - Ben Mahler On Sept. 16, 2014, 8:40 p.m., Kapil Arya wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25597/ --- (Updated Sept. 16, 2014, 8:40 p.m.) Review request for mesos, Adam B and Niklas Nielsen. Repository: mesos-git Description --- Currently there is no facility in Mesos for checking compatibility of various Mesos components that could have been built at different times with potentially different Mesos versions. This requirement is especially important for doing various compatibility checks between Mesos and Mesos modules (WIP). - Features major, minor, and patch numbers. - Convenience functions for comparing two versions. Diffs - 3rdparty/libprocess/3rdparty/Makefile.am db9766d70adb9076946cd2b467c55636fe5f7235 3rdparty/libprocess/3rdparty/stout/Makefile.am b6464de53c3873ecd0b62a08ca9aac530043ffb9 3rdparty/libprocess/3rdparty/stout/include/Makefile.am 6fa5b741bdd7f089ba93bf6fea43b9f39f8f0edb 3rdparty/libprocess/3rdparty/stout/include/stout/os.hpp 5bbf829b3fa5d09a92e1d64c52c1fc7eed73fc91 3rdparty/libprocess/3rdparty/stout/include/stout/version.hpp PRE-CREATION 3rdparty/libprocess/3rdparty/stout/tests/version_tests.cpp PRE-CREATION Diff: https://reviews.apache.org/r/25597/diff/ Testing --- Added a stout test and ran make check Thanks, Kapil Arya
Re: Review Request 25035: Fix for MESOS-1688
On Sept. 15, 2014, 9:02 nachm., Vinod Kone wrote: CHANGELOG, lines 1-9 https://reviews.apache.org/r/25035/diff/7/?file=688718#file688718line1 Thinking a bit more about this and talking to others. Adding deprecations in a bug fix release is bit weird. 2 options. 1) We can land this feature in 0.21.0 and not 0.20.1. That way we will do deprecation warning in 0.21.0 and disallow cpu/mem only executors in 0.22.0. This is the most straightforward. 2) Land this in 0.20.1, but the deprecation warning, in changelog (and ResourceUsageChecker?), happens in 0.21.0. The disallowing hapens in 0.22.0. This is bit weird but not too bad if you absolutely need this in 0.20.1. Considering 0.21.0 would happen in a month or so, I prefer #1. Does that work for you? For me it only matters to fix the problem in the near future. So I adjusted the patch for integration with 0.21.0. - Martin --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25035/#review53362 --- On Sept. 16, 2014, 9:05 nachm., Martin Weindel wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25035/ --- (Updated Sept. 16, 2014, 9:05 nachm.) Review request for mesos and Vinod Kone. Bugs: MESOS-1688 https://issues.apache.org/jira/browse/MESOS-1688 Repository: mesos-git Description --- As already explained in JIRA MESOS-1688, there are schedulers allocating memory only for the executor and not for tasks. For tasks only CPU resources are allocated in this case. Such a scheduler does not get offered any idle CPUs if the slave has nearly used up all memory. This can easily lead to a dead lock (in the application, not in Mesos). Simple example: 1. Scheduler allocates all memory of a slave for an executor 2. Scheduler launches a task for this executor (allocating 1 CPU) 3. Task finishes: 1 CPU , 0 MB memory allocatable. 4. No offers are made, as no memory is left. Scheduler will wait for offers forever. Dead lock in the application. To fix this problem, offers must be made if CPU resources are allocatable without considering allocatable memory Diffs - CHANGELOG a822cc4 src/common/resources.cpp edf36b1 src/master/constants.cpp faa1503 src/master/hierarchical_allocator_process.hpp 34f8cd6 src/master/master.cpp 18464ba src/tests/allocator_tests.cpp 774528a Diff: https://reviews.apache.org/r/25035/diff/ Testing --- Deployed patched Mesos 0.19.1 on a small cluster with 3 slaves and tested running multiple parallel Spark jobs in fine-grained mode to saturate allocatable memory. The jobs run fine now. This load always caused a dead lock in all Spark jobs within one minute with the unpatched Mesos. Thanks, Martin Weindel
Re: Review Request 25597: Added a version checker class to stout.
On Sept. 16, 2014, 9:05 p.m., Ben Mahler wrote: Thanks Kapil, looks great! I will get this committed for you shortly, I'll just add a TODO per your comments on more than 3 version components and I'll remove the single quotes per my comment below. Committed! - Ben --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25597/#review53595 --- On Sept. 16, 2014, 8:40 p.m., Kapil Arya wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25597/ --- (Updated Sept. 16, 2014, 8:40 p.m.) Review request for mesos, Adam B and Niklas Nielsen. Repository: mesos-git Description --- Currently there is no facility in Mesos for checking compatibility of various Mesos components that could have been built at different times with potentially different Mesos versions. This requirement is especially important for doing various compatibility checks between Mesos and Mesos modules (WIP). - Features major, minor, and patch numbers. - Convenience functions for comparing two versions. Diffs - 3rdparty/libprocess/3rdparty/Makefile.am db9766d70adb9076946cd2b467c55636fe5f7235 3rdparty/libprocess/3rdparty/stout/Makefile.am b6464de53c3873ecd0b62a08ca9aac530043ffb9 3rdparty/libprocess/3rdparty/stout/include/Makefile.am 6fa5b741bdd7f089ba93bf6fea43b9f39f8f0edb 3rdparty/libprocess/3rdparty/stout/include/stout/os.hpp 5bbf829b3fa5d09a92e1d64c52c1fc7eed73fc91 3rdparty/libprocess/3rdparty/stout/include/stout/version.hpp PRE-CREATION 3rdparty/libprocess/3rdparty/stout/tests/version_tests.cpp PRE-CREATION Diff: https://reviews.apache.org/r/25597/diff/ Testing --- Added a stout test and ran make check Thanks, Kapil Arya
Re: Review Request 25035: Fix for MESOS-1688
On Sept. 15, 2014, 3:23 p.m., Timothy St. Clair wrote: src/master/hierarchical_allocator_process.hpp, line 837 https://reviews.apache.org/r/25035/diff/7/?file=688721#file688721line837 What happens in the case where all CPUs are taken but memory is available? It looks like it will return (true), but this should not be possible. I think you want to give an offer in the case where there are CPU resources available, but memory is consumed by the executor. Vinod Kone wrote: Giving memory only resources is ok as long as it is used for a task and not an executor. See my comments above. Could you please add a detailed comment in the code above the mod, as on 1st inspection it leaves me still feeling unsettled. - Timothy --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25035/#review53343 --- On Sept. 16, 2014, 9:05 p.m., Martin Weindel wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25035/ --- (Updated Sept. 16, 2014, 9:05 p.m.) Review request for mesos and Vinod Kone. Bugs: MESOS-1688 https://issues.apache.org/jira/browse/MESOS-1688 Repository: mesos-git Description --- As already explained in JIRA MESOS-1688, there are schedulers allocating memory only for the executor and not for tasks. For tasks only CPU resources are allocated in this case. Such a scheduler does not get offered any idle CPUs if the slave has nearly used up all memory. This can easily lead to a dead lock (in the application, not in Mesos). Simple example: 1. Scheduler allocates all memory of a slave for an executor 2. Scheduler launches a task for this executor (allocating 1 CPU) 3. Task finishes: 1 CPU , 0 MB memory allocatable. 4. No offers are made, as no memory is left. Scheduler will wait for offers forever. Dead lock in the application. To fix this problem, offers must be made if CPU resources are allocatable without considering allocatable memory Diffs - CHANGELOG a822cc4 src/common/resources.cpp edf36b1 src/master/constants.cpp faa1503 src/master/hierarchical_allocator_process.hpp 34f8cd6 src/master/master.cpp 18464ba src/tests/allocator_tests.cpp 774528a Diff: https://reviews.apache.org/r/25035/diff/ Testing --- Deployed patched Mesos 0.19.1 on a small cluster with 3 slaves and tested running multiple parallel Spark jobs in fine-grained mode to saturate allocatable memory. The jobs run fine now. This load always caused a dead lock in all Spark jobs within one minute with the unpatched Mesos. Thanks, Martin Weindel
Re: Review Request 25035: Fix for MESOS-1688
On Sept. 15, 2014, 3:23 nachm., Timothy St. Clair wrote: src/master/hierarchical_allocator_process.hpp, line 837 https://reviews.apache.org/r/25035/diff/7/?file=688721#file688721line837 What happens in the case where all CPUs are taken but memory is available? It looks like it will return (true), but this should not be possible. I think you want to give an offer in the case where there are CPU resources available, but memory is consumed by the executor. Vinod Kone wrote: Giving memory only resources is ok as long as it is used for a task and not an executor. See my comments above. Timothy St. Clair wrote: Could you please add a detailed comment in the code above the mod, as on 1st inspection it leaves me still feeling unsettled. I agree with Vinod. An executor may make use of additional offered memory, e.g for expanding a cache. In this scenario, the already allocated CPU resources are sufficient. - Martin --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25035/#review53343 --- On Sept. 16, 2014, 9:05 nachm., Martin Weindel wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25035/ --- (Updated Sept. 16, 2014, 9:05 nachm.) Review request for mesos and Vinod Kone. Bugs: MESOS-1688 https://issues.apache.org/jira/browse/MESOS-1688 Repository: mesos-git Description --- As already explained in JIRA MESOS-1688, there are schedulers allocating memory only for the executor and not for tasks. For tasks only CPU resources are allocated in this case. Such a scheduler does not get offered any idle CPUs if the slave has nearly used up all memory. This can easily lead to a dead lock (in the application, not in Mesos). Simple example: 1. Scheduler allocates all memory of a slave for an executor 2. Scheduler launches a task for this executor (allocating 1 CPU) 3. Task finishes: 1 CPU , 0 MB memory allocatable. 4. No offers are made, as no memory is left. Scheduler will wait for offers forever. Dead lock in the application. To fix this problem, offers must be made if CPU resources are allocatable without considering allocatable memory Diffs - CHANGELOG a822cc4 src/common/resources.cpp edf36b1 src/master/constants.cpp faa1503 src/master/hierarchical_allocator_process.hpp 34f8cd6 src/master/master.cpp 18464ba src/tests/allocator_tests.cpp 774528a Diff: https://reviews.apache.org/r/25035/diff/ Testing --- Deployed patched Mesos 0.19.1 on a small cluster with 3 slaves and tested running multiple parallel Spark jobs in fine-grained mode to saturate allocatable memory. The jobs run fine now. This load always caused a dead lock in all Spark jobs within one minute with the unpatched Mesos. Thanks, Martin Weindel
Re: Mesos 0.20.1 release status
Tim, sorry. I meant today, 9/16 @6pm PDT. Vinod, yes, Adam is helping me to push CHANGELOG. I'll work with him to create tag, mvn push, etc. Jie, i'm hoping to go with tags. I'll create a tag for 0.20.1-rc1. For new RC builds (if any), i'll create new tag from previous RC and cherry-pick the bug fixes. Team, if you are working on any of these tickets, please follow-up with reviewers to +1 the patch. If these patches are not merged before 6pm, it may miss this release. I'd prefer not to delay the release schedule, considering next major release 0.21.0 will be out in 3-4 weeks. http://s.apache.org/mesos-0.20.1-unresolved-issues On Mon, Sep 15, 2014 at 11:35 PM, Timothy Chen tnac...@gmail.com wrote: It's already past 9/16 6 pm PDT? Tim On Tue, Sep 16, 2014 at 2:19 AM, Bhuvan Arumugam bhu...@apache.org wrote: We are targetting 17 tickets. It include improvements and bug fixes. 4 of them are still in progress. http://s.apache.org/mesos-0.20.1-unresolved-issues I'll cut the tag for RC1 and send for voting, once these issues are reviewed/submitted or 9/15 @6pm PDT, whichever comes first! The open issues (if any) will be moved to next release, at that point in time. In case you got questions, let me know. Thank you, -- Regards, Bhuvan Arumugam www.livecipher.com -- Regards, Bhuvan Arumugam www.livecipher.com
Re: Review Request 25597: Added a version checker class to stout.
On Sept. 16, 2014, 5:05 p.m., Ben Mahler wrote: Thanks Kapil, looks great! I will get this committed for you shortly, I'll just add a TODO per your comments on more than 3 version components and I'll remove the single quotes per my comment below. Ben Mahler wrote: Committed! Thanks Ben! - Kapil --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25597/#review53595 --- On Sept. 16, 2014, 4:40 p.m., Kapil Arya wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25597/ --- (Updated Sept. 16, 2014, 4:40 p.m.) Review request for mesos, Adam B and Niklas Nielsen. Repository: mesos-git Description --- Currently there is no facility in Mesos for checking compatibility of various Mesos components that could have been built at different times with potentially different Mesos versions. This requirement is especially important for doing various compatibility checks between Mesos and Mesos modules (WIP). - Features major, minor, and patch numbers. - Convenience functions for comparing two versions. Diffs - 3rdparty/libprocess/3rdparty/Makefile.am db9766d70adb9076946cd2b467c55636fe5f7235 3rdparty/libprocess/3rdparty/stout/Makefile.am b6464de53c3873ecd0b62a08ca9aac530043ffb9 3rdparty/libprocess/3rdparty/stout/include/Makefile.am 6fa5b741bdd7f089ba93bf6fea43b9f39f8f0edb 3rdparty/libprocess/3rdparty/stout/include/stout/os.hpp 5bbf829b3fa5d09a92e1d64c52c1fc7eed73fc91 3rdparty/libprocess/3rdparty/stout/include/stout/version.hpp PRE-CREATION 3rdparty/libprocess/3rdparty/stout/tests/version_tests.cpp PRE-CREATION Diff: https://reviews.apache.org/r/25597/diff/ Testing --- Added a stout test and ran make check Thanks, Kapil Arya
Re: Review Request 25035: Fix for MESOS-1688
On Sept. 15, 2014, 3:23 p.m., Timothy St. Clair wrote: src/master/hierarchical_allocator_process.hpp, line 837 https://reviews.apache.org/r/25035/diff/7/?file=688721#file688721line837 What happens in the case where all CPUs are taken but memory is available? It looks like it will return (true), but this should not be possible. I think you want to give an offer in the case where there are CPU resources available, but memory is consumed by the executor. Vinod Kone wrote: Giving memory only resources is ok as long as it is used for a task and not an executor. See my comments above. Timothy St. Clair wrote: Could you please add a detailed comment in the code above the mod, as on 1st inspection it leaves me still feeling unsettled. Martin Weindel wrote: I agree with Vinod. An executor may make use of additional offered memory, e.g for expanding a cache. In this scenario, the already allocated CPU resources are sufficient. More generally, I think resources for executors should be required, and resources for tasks should be optional. If a task doesn't need to specify CPU, then as a corollary, it doesn't need to specify memory. - Ben --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25035/#review53343 --- On Sept. 16, 2014, 9:05 p.m., Martin Weindel wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25035/ --- (Updated Sept. 16, 2014, 9:05 p.m.) Review request for mesos and Vinod Kone. Bugs: MESOS-1688 https://issues.apache.org/jira/browse/MESOS-1688 Repository: mesos-git Description --- As already explained in JIRA MESOS-1688, there are schedulers allocating memory only for the executor and not for tasks. For tasks only CPU resources are allocated in this case. Such a scheduler does not get offered any idle CPUs if the slave has nearly used up all memory. This can easily lead to a dead lock (in the application, not in Mesos). Simple example: 1. Scheduler allocates all memory of a slave for an executor 2. Scheduler launches a task for this executor (allocating 1 CPU) 3. Task finishes: 1 CPU , 0 MB memory allocatable. 4. No offers are made, as no memory is left. Scheduler will wait for offers forever. Dead lock in the application. To fix this problem, offers must be made if CPU resources are allocatable without considering allocatable memory Diffs - CHANGELOG a822cc4 src/common/resources.cpp edf36b1 src/master/constants.cpp faa1503 src/master/hierarchical_allocator_process.hpp 34f8cd6 src/master/master.cpp 18464ba src/tests/allocator_tests.cpp 774528a Diff: https://reviews.apache.org/r/25035/diff/ Testing --- Deployed patched Mesos 0.19.1 on a small cluster with 3 slaves and tested running multiple parallel Spark jobs in fine-grained mode to saturate allocatable memory. The jobs run fine now. This load always caused a dead lock in all Spark jobs within one minute with the unpatched Mesos. Thanks, Martin Weindel
Re: Review Request 25549: Basic filesystem isolator for Linux.
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25549/#review53402
---
A few style issues. I'll let Vinod to give a final ship it.
src/slave/containerizer/isolators/filesystem/shared.hpp
https://reviews.apache.org/r/25549/#comment93052
Style: comments should have 70 char limit each line.
src/slave/containerizer/isolators/filesystem/shared.cpp
https://reviews.apache.org/r/25549/#comment93056
Do not use snake case:)
src/slave/containerizer/isolators/filesystem/shared.cpp
https://reviews.apache.org/r/25549/#comment93053
This shoudl fit in one line
src/slave/containerizer/isolators/filesystem/shared.cpp
https://reviews.apache.org/r/25549/#comment93054
Captialize the comment.
src/slave/containerizer/isolators/filesystem/shared.cpp
https://reviews.apache.org/r/25549/#comment93055
Ditto.
src/slave/containerizer/isolators/filesystem/shared.cpp
https://reviews.apache.org/r/25549/#comment93057
Ditto, snake_case
src/slave/containerizer/isolators/filesystem/shared.cpp
https://reviews.apache.org/r/25549/#comment93058
Add a comment about why -n is used here.
src/slave/slave.cpp
https://reviews.apache.org/r/25549/#comment93050
Style: move { to the above line.
src/slave/slave.cpp
https://reviews.apache.org/r/25549/#comment93051
Ditto.
src/tests/isolator_tests.cpp
https://reviews.apache.org/r/25549/#comment93060
Add a blank line
- Jie Yu
On Sept. 15, 2014, 8:01 p.m., Ian Downes wrote:
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25549/
---
(Updated Sept. 15, 2014, 8:01 p.m.)
Review request for mesos, Ben Mahler, Jie Yu, and Vinod Kone.
Bugs: MESOS-1586
https://issues.apache.org/jira/browse/MESOS-1586
Repository: mesos-git
Description
---
Does not report usage or enforce quota but can create 'private' directories
for each container which mask parts of the shared host filesystem.
This review replaces https://reviews.apache.org/r/24178/ because of some file
renaming. I addressed all comments from earlier reviews.
Diffs
-
include/mesos/mesos.proto dea51f94d130c131421c43e7fd774ceb8941f501
src/Makefile.am 9b973e5503e30180045e270220987ba647da8038
src/common/parse.hpp e6153d8a1f25bc9ddbe1e391306beeacfc8d5ff6
src/common/type_utils.hpp 480c0883fe6ed7f6a9daf77d83ebb077da2e66ee
src/slave/containerizer/isolators/filesystem/shared.hpp PRE-CREATION
src/slave/containerizer/isolators/filesystem/shared.cpp PRE-CREATION
src/slave/containerizer/linux_launcher.cpp
d5ef1d6aa762cf81a3e8384552d97fe95b9cbd95
src/slave/containerizer/mesos/containerizer.cpp
9d083294caa5c5a47ba3ceaa1b57346144cb795c
src/slave/flags.hpp 21e00212bc402674eaea73b44b3f91df477a7213
src/slave/slave.cpp 1b3dc7370a2441e4159aa5ee552b64ca5e511e96
src/tests/isolator_tests.cpp c38f87632cb6984543cb3767dbd656cde7459610
src/tests/mesos.hpp 957e2233cc11c438fd80d3b6d1907a1223093104
Diff: https://reviews.apache.org/r/25549/diff/
Testing
---
make check # added a test
Thanks,
Ian Downes
Re: Review Request 25597: Added a version checker class to stout.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25597/#review53608 --- Patch looks great! Reviews applied: [25597] All tests passed. - Mesos ReviewBot On Sept. 16, 2014, 8:40 p.m., Kapil Arya wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25597/ --- (Updated Sept. 16, 2014, 8:40 p.m.) Review request for mesos, Adam B and Niklas Nielsen. Repository: mesos-git Description --- Currently there is no facility in Mesos for checking compatibility of various Mesos components that could have been built at different times with potentially different Mesos versions. This requirement is especially important for doing various compatibility checks between Mesos and Mesos modules (WIP). - Features major, minor, and patch numbers. - Convenience functions for comparing two versions. Diffs - 3rdparty/libprocess/3rdparty/Makefile.am db9766d70adb9076946cd2b467c55636fe5f7235 3rdparty/libprocess/3rdparty/stout/Makefile.am b6464de53c3873ecd0b62a08ca9aac530043ffb9 3rdparty/libprocess/3rdparty/stout/include/Makefile.am 6fa5b741bdd7f089ba93bf6fea43b9f39f8f0edb 3rdparty/libprocess/3rdparty/stout/include/stout/os.hpp 5bbf829b3fa5d09a92e1d64c52c1fc7eed73fc91 3rdparty/libprocess/3rdparty/stout/include/stout/version.hpp PRE-CREATION 3rdparty/libprocess/3rdparty/stout/tests/version_tests.cpp PRE-CREATION Diff: https://reviews.apache.org/r/25597/diff/ Testing --- Added a stout test and ran make check Thanks, Kapil Arya
Re: Review Request 25569: Refactor test environment validations
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25569/ --- (Updated Sept. 16, 2014, 10:35 p.m.) Review request for mesos and Ben Mahler. Repository: mesos-git Description (updated) --- Review: https://reviews.apache.org/r/25569 Diffs (updated) - src/tests/environment.cpp 2274251aaf653d83c2d03ef2186763978067a747 Diff: https://reviews.apache.org/r/25569/diff/ Testing --- make check Thanks, Timothy Chen
Re: Review Request 25403: Override entrypoint when shell enabled in Docker
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25403/ --- (Updated Sept. 16, 2014, 10:37 p.m.) Review request for mesos, Benjamin Hindman and Jie Yu. Bugs: MESOS-1770 https://issues.apache.org/jira/browse/MESOS-1770 Repository: mesos-git Description --- Review: https://reviews.apache.org/r/25403 Diffs (updated) - src/docker/docker.cpp af51ac9058382aede61b09e06e312ad2ce6de03e Diff: https://reviews.apache.org/r/25403/diff/ Testing --- make check Thanks, Timothy Chen
Build failed in Jenkins: Mesos-Trunk-Ubuntu-Build-Out-Of-Src-Disable-Java-Disable-Python-Disable-Webui #2374
See https://builds.apache.org/job/Mesos-Trunk-Ubuntu-Build-Out-Of-Src-Disable-Java-Disable-Python-Disable-Webui/2374/changes Changes: [bmahler] Added a Version class to stout. -- [...truncated 58686 lines...] I0916 23:01:05.386067 21051 replica.cpp:676] Persisted action at 0 I0916 23:01:05.386090 21051 replica.cpp:661] Replica learned NOP action at position 0 I0916 23:01:05.083233 21042 leveldb.cpp:343] Persisting action (16 bytes) to leveldb took 121548ns I0916 23:01:05.386147 21042 replica.cpp:676] Persisted action at 0 I0916 23:01:05.386168 21042 replica.cpp:661] Replica learned NOP action at position 0 I0916 23:01:05.386436 21040 coordinator.cpp:340] Coordinator attempting to write APPEND action at position 1 I0916 23:01:05.386842 21048 replica.cpp:508] Replica received write request for position 1 I0916 23:01:05.386904 21040 replica.cpp:508] Replica received write request for position 1 I0916 23:01:05.387473 21048 leveldb.cpp:343] Persisting action (27 bytes) to leveldb took 603894ns I0916 23:01:05.387475 21040 leveldb.cpp:343] Persisting action (27 bytes) to leveldb took 538731ns I0916 23:01:05.387511 21048 replica.cpp:676] Persisted action at 1 I0916 23:01:05.387524 21040 replica.cpp:676] Persisted action at 1 I0916 23:01:05.387969 21043 replica.cpp:655] Replica received learned notice for position 1 I0916 23:01:05.387998 21053 replica.cpp:655] Replica received learned notice for position 1 I0916 23:01:05.388157 21043 leveldb.cpp:343] Persisting action (29 bytes) to leveldb took 165034ns I0916 23:01:05.388177 21043 replica.cpp:676] Persisted action at 1 I0916 23:01:05.388188 21043 replica.cpp:661] Replica learned APPEND action at position 1 I0916 23:01:05.388212 21053 leveldb.cpp:343] Persisting action (29 bytes) to leveldb took 190343ns I0916 23:01:05.388236 21053 replica.cpp:676] Persisted action at 1 I0916 23:01:05.388245 21053 replica.cpp:661] Replica learned APPEND action at position 1 I0916 23:01:05.390319 21026 leveldb.cpp:176] Opened db in 1.951015ms I0916 23:01:05.391607 21026 leveldb.cpp:183] Compacted db in 1.261904ms I0916 23:01:05.391633 21026 leveldb.cpp:198] Created db iterator in 4244ns I0916 23:01:05.391650 21026 leveldb.cpp:204] Seeked to beginning of db in 7069ns I0916 23:01:05.391669 21026 leveldb.cpp:273] Iterated through 1 keys in the db in 8220ns I0916 23:01:05.391685 21026 replica.cpp:741] Replica recovered with log positions 0 - 0 with 1 holes and 0 unlearned I0916 23:01:05.392328 21047 replica.cpp:474] Replica received implicit promise request with proposal 1 I0916 23:01:05.392354 21047 replica.cpp:479] Replica denying promise request with proposal 1 I0916 23:01:05.392408 21043 replica.cpp:474] Replica received implicit promise request with proposal 1 I0916 23:01:05.392943 21043 leveldb.cpp:306] Persisting metadata (8 bytes) to leveldb took 510019ns I0916 23:01:05.392963 21043 replica.cpp:342] Persisted promised to 1 I0916 23:01:05.393698 21055 replica.cpp:474] Replica received implicit promise request with proposal 2 I0916 23:01:05.393705 21045 replica.cpp:474] Replica received implicit promise request with proposal 2 I0916 23:01:05.394114 21045 leveldb.cpp:306] Persisting metadata (8 bytes) to leveldb took 380162ns I0916 23:01:05.394114 21055 leveldb.cpp:306] Persisting metadata (8 bytes) to leveldb took 390574ns I0916 23:01:05.394136 21045 replica.cpp:342] Persisted promised to 2 I0916 23:01:05.394152 21055 replica.cpp:342] Persisted promised to 2 I0916 23:01:05.394424 21053 coordinator.cpp:230] Coordinator attemping to fill missing position I0916 23:01:05.395113 21046 replica.cpp:375] Replica received explicit promise request for position 0 with proposal 3 I0916 23:01:05.395136 21055 replica.cpp:375] Replica received explicit promise request for position 0 with proposal 3 I0916 23:01:05.395153 21046 leveldb.cpp:438] Reading position from leveldb took 16040ns I0916 23:01:05.395339 21055 leveldb.cpp:343] Persisting action (8 bytes) to leveldb took 181686ns I0916 23:01:05.395359 21055 replica.cpp:676] Persisted action at 0 I0916 23:01:05.395392 21046 leveldb.cpp:343] Persisting action (16 bytes) to leveldb took 215312ns I0916 23:01:05.395414 21046 replica.cpp:676] Persisted action at 0 I0916 23:01:05.395781 21047 replica.cpp:655] Replica received learned notice for position 0 I0916 23:01:05.395792 21049 replica.cpp:655] Replica received learned notice for position 0 I0916 23:01:05.395972 21047 leveldb.cpp:343] Persisting action (16 bytes) to leveldb took 163947ns I0916 23:01:05.395992 21047 replica.cpp:676] Persisted action at 0 I0916 23:01:05.396003 21047 replica.cpp:661] Replica learned NOP action at position 0 I0916 23:01:05.396059 21049 leveldb.cpp:343] Persisting action (16 bytes) to leveldb took 198163ns I0916 23:01:05.396105 21049 replica.cpp:676] Persisted action at 0 I0916 23:01:05.396118 21049 replica.cpp:661] Replica learned NOP action at position 0 I0916 23:01:05.396561 21053
Re: Review Request 25035: Fix for MESOS-1688
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25035/#review53621 --- Patch looks great! Reviews applied: [25035] All tests passed. - Mesos ReviewBot On Sept. 16, 2014, 9:05 p.m., Martin Weindel wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25035/ --- (Updated Sept. 16, 2014, 9:05 p.m.) Review request for mesos and Vinod Kone. Bugs: MESOS-1688 https://issues.apache.org/jira/browse/MESOS-1688 Repository: mesos-git Description --- As already explained in JIRA MESOS-1688, there are schedulers allocating memory only for the executor and not for tasks. For tasks only CPU resources are allocated in this case. Such a scheduler does not get offered any idle CPUs if the slave has nearly used up all memory. This can easily lead to a dead lock (in the application, not in Mesos). Simple example: 1. Scheduler allocates all memory of a slave for an executor 2. Scheduler launches a task for this executor (allocating 1 CPU) 3. Task finishes: 1 CPU , 0 MB memory allocatable. 4. No offers are made, as no memory is left. Scheduler will wait for offers forever. Dead lock in the application. To fix this problem, offers must be made if CPU resources are allocatable without considering allocatable memory Diffs - CHANGELOG a822cc4 src/common/resources.cpp edf36b1 src/master/constants.cpp faa1503 src/master/hierarchical_allocator_process.hpp 34f8cd6 src/master/master.cpp 18464ba src/tests/allocator_tests.cpp 774528a Diff: https://reviews.apache.org/r/25035/diff/ Testing --- Deployed patched Mesos 0.19.1 on a small cluster with 3 slaves and tested running multiple parallel Spark jobs in fine-grained mode to saturate allocatable memory. The jobs run fine now. This load always caused a dead lock in all Spark jobs within one minute with the unpatched Mesos. Thanks, Martin Weindel
Re: Review Request 25270: Enable bridge network in Mesos
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25270/ --- (Updated Sept. 16, 2014, 11:04 p.m.) Review request for drill, Benjamin Hindman, Jie Yu, and Timothy St. Clair. Bugs: MESOS-1621 https://issues.apache.org/jira/browse/MESOS-1621 Repository: mesos-git Description --- Review: https://reviews.apache.org/r/25270 Diffs (updated) - include/mesos/mesos.proto dea51f94d130c131421c43e7fd774ceb8941f501 src/docker/docker.cpp af51ac9058382aede61b09e06e312ad2ce6de03e src/slave/slave.cpp 1b3dc7370a2441e4159aa5ee552b64ca5e511e96 src/tests/docker_containerizer_tests.cpp 8654f9c787bd207f6a7b821651e0c083bea9dc8a src/tests/docker_tests.cpp 826a8c1ef1b3089d416e5775fa2cf4e5cb0c26d1 Diff: https://reviews.apache.org/r/25270/diff/ Testing --- make check Thanks, Timothy Chen
Re: Review Request 25270: Enable bridge network in Mesos
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25270/ --- (Updated Sept. 16, 2014, 11:04 p.m.) Review request for mesos, Benjamin Hindman, Jie Yu, and Timothy St. Clair. Bugs: MESOS-1621 https://issues.apache.org/jira/browse/MESOS-1621 Repository: mesos-git Description --- Review: https://reviews.apache.org/r/25270 Diffs - include/mesos/mesos.proto dea51f94d130c131421c43e7fd774ceb8941f501 src/docker/docker.cpp af51ac9058382aede61b09e06e312ad2ce6de03e src/slave/slave.cpp 1b3dc7370a2441e4159aa5ee552b64ca5e511e96 src/tests/docker_containerizer_tests.cpp 8654f9c787bd207f6a7b821651e0c083bea9dc8a src/tests/docker_tests.cpp 826a8c1ef1b3089d416e5775fa2cf4e5cb0c26d1 Diff: https://reviews.apache.org/r/25270/diff/ Testing --- make check Thanks, Timothy Chen
Build failed in Jenkins: mesos-reviewbot #1587
See https://builds.apache.org/job/mesos-reviewbot/1587/changes Changes: [bmahler] Added a Version class to stout. -- [...truncated 5517 lines...] Removing 3rdparty/libprocess/config.status Removing 3rdparty/libprocess/config.sub Removing 3rdparty/libprocess/configure Removing 3rdparty/libprocess/depcomp Removing 3rdparty/libprocess/include/Makefile Removing 3rdparty/libprocess/include/Makefile.in Removing 3rdparty/libprocess/libtool Removing 3rdparty/libprocess/ltmain.sh Removing 3rdparty/libprocess/m4/libtool.m4 Removing 3rdparty/libprocess/m4/ltoptions.m4 Removing 3rdparty/libprocess/m4/ltsugar.m4 Removing 3rdparty/libprocess/m4/ltversion.m4 Removing 3rdparty/libprocess/m4/lt~obsolete.m4 Removing 3rdparty/libprocess/missing Removing Makefile Removing Makefile.in Removing aclocal.m4 Removing ar-lib Removing autom4te.cache/ Removing bin/gdb-mesos-local.sh Removing bin/gdb-mesos-master.sh Removing bin/gdb-mesos-slave.sh Removing bin/gdb-mesos-tests.sh Removing bin/lldb-mesos-local.sh Removing bin/lldb-mesos-master.sh Removing bin/lldb-mesos-slave.sh Removing bin/lldb-mesos-tests.sh Removing bin/mesos-local-flags.sh Removing bin/mesos-local.sh Removing bin/mesos-master-flags.sh Removing bin/mesos-master.sh Removing bin/mesos-slave-flags.sh Removing bin/mesos-slave.sh Removing bin/mesos-tests-flags.sh Removing bin/mesos-tests.sh Removing bin/mesos.sh Removing bin/valgrind-mesos-local.sh Removing bin/valgrind-mesos-master.sh Removing bin/valgrind-mesos-slave.sh Removing bin/valgrind-mesos-tests.sh Removing compile Removing config.guess Removing config.log Removing config.lt Removing config.status Removing config.sub Removing configure Removing depcomp Removing ec2/Makefile Removing ec2/Makefile.in Removing include/mesos/mesos.hpp Removing install-sh Removing libtool Removing ltmain.sh Removing m4/libtool.m4 Removing m4/ltoptions.m4 Removing m4/ltsugar.m4 Removing m4/ltversion.m4 Removing m4/lt~obsolete.m4 Skipping repository mesos-0.20.0/_build/3rdparty/leveldb Removing mesos-0.21.0.tar.gz Removing mesos.pc Removing missing Removing mpi/mpiexec-mesos Removing src/.deps/ Removing src/Makefile Removing src/Makefile.in Removing src/authorizer/.deps/ Removing src/cli/.deps/ Removing src/common/.deps/ Removing src/containerizer/ Removing src/deploy/mesos-daemon.sh Removing src/deploy/mesos-start-cluster.sh Removing src/deploy/mesos-start-masters.sh Removing src/deploy/mesos-start-slaves.sh Removing src/deploy/mesos-stop-cluster.sh Removing src/deploy/mesos-stop-masters.sh Removing src/deploy/mesos-stop-slaves.sh Removing src/docker/.deps/ Removing src/examples/.deps/ Removing src/examples/java/test-exception-framework Removing src/examples/java/test-executor Removing src/examples/java/test-framework Removing src/examples/java/test-log Removing src/examples/java/test-multiple-executors-framework Removing src/examples/python/test-containerizer Removing src/examples/python/test-executor Removing src/examples/python/test-framework Removing src/exec/.deps/ Removing src/files/.deps/ Removing src/health-check/.deps/ Removing src/java/generated/org/apache/mesos/MesosNativeLibrary.java Removing src/java/jni/.deps/ Removing src/java/mesos.pom Removing src/jvm/.deps/ Removing src/jvm/org/apache/.deps/ Removing src/launcher/.deps/ Removing src/linux/.deps/ Removing src/linux/routing/.deps/ Removing src/linux/routing/filter/.deps/ Removing src/linux/routing/link/.deps/ Removing src/linux/routing/queueing/.deps/ Removing src/local/.deps/ Removing src/log/.deps/ Removing src/log/tool/.deps/ Removing src/logging/.deps/ Removing src/master/.deps/ Removing src/messages/.deps/ Removing src/python/interface/setup.py Removing src/python/native/ext_modules.py Removing src/python/native/setup.py Removing src/python/setup.py Removing src/sasl/.deps/ Removing src/sched/.deps/ Removing src/scheduler/.deps/ Removing src/slave/.deps/ Removing src/slave/containerizer/.deps/ Removing src/slave/containerizer/isolators/cgroups/.deps/ Removing src/slave/containerizer/isolators/network/.deps/ Removing src/slave/containerizer/mesos/.deps/ Removing src/state/.deps/ Removing src/tests/.deps/ Removing src/tests/common/.deps/ Removing src/usage/.deps/ Removing src/zookeeper/.deps/ + git reset --hard HEAD HEAD is now at 9b58956 Added a Version class to stout. + date Tue Sep 16 22:23:11 UTC 2014 + ./support/verify-reviews.py mesos-review mesos-review42 1 Checking if review: 25565 needs verification Skipping blocking review 25565 Checking if review: 25079 needs verification Latest diff timestamp: 2014-08-28 16:24:06 Latest review timestamp: 2014-08-30 06:14:56 Checking if review: 25551 needs verification Latest diff timestamp: 2014-09-11 19:54:05 Latest review timestamp: 2014-09-13 00:02:43 Checking if review: 25525 needs verification Latest diff timestamp: 2014-09-13 00:27:28 Latest review timestamp: 2014-09-13 20:02:01 Latest dependency change timestamp: 2014-09-13 00:33:32 Checking if review: 25623 needs
Re: Review Request 25569: Refactor test environment validations
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25569/#review53627 --- Patch looks great! Reviews applied: [25569] All tests passed. - Mesos ReviewBot On Sept. 16, 2014, 10:35 p.m., Timothy Chen wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25569/ --- (Updated Sept. 16, 2014, 10:35 p.m.) Review request for mesos and Ben Mahler. Repository: mesos-git Description --- Review: https://reviews.apache.org/r/25569 Diffs - src/tests/environment.cpp 2274251aaf653d83c2d03ef2186763978067a747 Diff: https://reviews.apache.org/r/25569/diff/ Testing --- make check Thanks, Timothy Chen
Jenkins build is back to normal : mesos-reviewbot #1588
See https://builds.apache.org/job/mesos-reviewbot/1588/
Re: Review Request 25695: Update to enable systemd control of mesos services
On Sept. 16, 2014, 7:27 p.m., Mesos ReviewBot wrote: Patch looks great! Reviews applied: [25695] All tests passed. I'll let @jieyu comment shepherd this, but I think we should not rush this into 0.20.1 because this is not a bug in 0.20.0. - Vinod --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25695/#review53571 --- On Sept. 16, 2014, 6:19 p.m., Timothy St. Clair wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25695/ --- (Updated Sept. 16, 2014, 6:19 p.m.) Review request for mesos, Jie Yu and Vinod Kone. Bugs: MESOS-1195 https://issues.apache.org/jira/browse/MESOS-1195 Repository: mesos-git Description --- This update enables support for systemd co-managed cgroup controllers Diffs - configure.ac c4b4391 src/linux/cgroups.cpp 5093b4c src/slave/containerizer/isolators/cgroups/cpushare.cpp b1cad47 Diff: https://reviews.apache.org/r/25695/diff/ Testing --- systemctl start mesos-master mesos-slave several runs of 'mesos execute' to verify creation and cleanup. systemctl stop mesos-master mesos-slave make check Thanks, Timothy St. Clair
Re: Build failed in Jenkins: Mesos-Trunk-Ubuntu-Build-Out-Of-Src-Disable-Java-Disable-Python-Disable-Webui #2374
+Tim Chen Can you take a look at the health check failure? https://issues.apache.org/jira/browse/MESOS-1802 I'll take a look at the Registrar test: https://issues.apache.org/jira/browse/MESOS-1803 On Tue, Sep 16, 2014 at 4:01 PM, Apache Jenkins Server jenk...@builds.apache.org wrote: See https://builds.apache.org/job/Mesos-Trunk-Ubuntu-Build-Out-Of-Src-Disable-Java-Disable-Python-Disable-Webui/2374/changes Changes: [bmahler] Added a Version class to stout. -- [...truncated 58686 lines...] I0916 23:01:05.386067 21051 replica.cpp:676] Persisted action at 0 I0916 23:01:05.386090 21051 replica.cpp:661] Replica learned NOP action at position 0 I0916 23:01:05.083233 21042 leveldb.cpp:343] Persisting action (16 bytes) to leveldb took 121548ns I0916 23:01:05.386147 21042 replica.cpp:676] Persisted action at 0 I0916 23:01:05.386168 21042 replica.cpp:661] Replica learned NOP action at position 0 I0916 23:01:05.386436 21040 coordinator.cpp:340] Coordinator attempting to write APPEND action at position 1 I0916 23:01:05.386842 21048 replica.cpp:508] Replica received write request for position 1 I0916 23:01:05.386904 21040 replica.cpp:508] Replica received write request for position 1 I0916 23:01:05.387473 21048 leveldb.cpp:343] Persisting action (27 bytes) to leveldb took 603894ns I0916 23:01:05.387475 21040 leveldb.cpp:343] Persisting action (27 bytes) to leveldb took 538731ns I0916 23:01:05.387511 21048 replica.cpp:676] Persisted action at 1 I0916 23:01:05.387524 21040 replica.cpp:676] Persisted action at 1 I0916 23:01:05.387969 21043 replica.cpp:655] Replica received learned notice for position 1 I0916 23:01:05.387998 21053 replica.cpp:655] Replica received learned notice for position 1 I0916 23:01:05.388157 21043 leveldb.cpp:343] Persisting action (29 bytes) to leveldb took 165034ns I0916 23:01:05.388177 21043 replica.cpp:676] Persisted action at 1 I0916 23:01:05.388188 21043 replica.cpp:661] Replica learned APPEND action at position 1 I0916 23:01:05.388212 21053 leveldb.cpp:343] Persisting action (29 bytes) to leveldb took 190343ns I0916 23:01:05.388236 21053 replica.cpp:676] Persisted action at 1 I0916 23:01:05.388245 21053 replica.cpp:661] Replica learned APPEND action at position 1 I0916 23:01:05.390319 21026 leveldb.cpp:176] Opened db in 1.951015ms I0916 23:01:05.391607 21026 leveldb.cpp:183] Compacted db in 1.261904ms I0916 23:01:05.391633 21026 leveldb.cpp:198] Created db iterator in 4244ns I0916 23:01:05.391650 21026 leveldb.cpp:204] Seeked to beginning of db in 7069ns I0916 23:01:05.391669 21026 leveldb.cpp:273] Iterated through 1 keys in the db in 8220ns I0916 23:01:05.391685 21026 replica.cpp:741] Replica recovered with log positions 0 - 0 with 1 holes and 0 unlearned I0916 23:01:05.392328 21047 replica.cpp:474] Replica received implicit promise request with proposal 1 I0916 23:01:05.392354 21047 replica.cpp:479] Replica denying promise request with proposal 1 I0916 23:01:05.392408 21043 replica.cpp:474] Replica received implicit promise request with proposal 1 I0916 23:01:05.392943 21043 leveldb.cpp:306] Persisting metadata (8 bytes) to leveldb took 510019ns I0916 23:01:05.392963 21043 replica.cpp:342] Persisted promised to 1 I0916 23:01:05.393698 21055 replica.cpp:474] Replica received implicit promise request with proposal 2 I0916 23:01:05.393705 21045 replica.cpp:474] Replica received implicit promise request with proposal 2 I0916 23:01:05.394114 21045 leveldb.cpp:306] Persisting metadata (8 bytes) to leveldb took 380162ns I0916 23:01:05.394114 21055 leveldb.cpp:306] Persisting metadata (8 bytes) to leveldb took 390574ns I0916 23:01:05.394136 21045 replica.cpp:342] Persisted promised to 2 I0916 23:01:05.394152 21055 replica.cpp:342] Persisted promised to 2 I0916 23:01:05.394424 21053 coordinator.cpp:230] Coordinator attemping to fill missing position I0916 23:01:05.395113 21046 replica.cpp:375] Replica received explicit promise request for position 0 with proposal 3 I0916 23:01:05.395136 21055 replica.cpp:375] Replica received explicit promise request for position 0 with proposal 3 I0916 23:01:05.395153 21046 leveldb.cpp:438] Reading position from leveldb took 16040ns I0916 23:01:05.395339 21055 leveldb.cpp:343] Persisting action (8 bytes) to leveldb took 181686ns I0916 23:01:05.395359 21055 replica.cpp:676] Persisted action at 0 I0916 23:01:05.395392 21046 leveldb.cpp:343] Persisting action (16 bytes) to leveldb took 215312ns I0916 23:01:05.395414 21046 replica.cpp:676] Persisted action at 0 I0916 23:01:05.395781 21047 replica.cpp:655] Replica received learned notice for position 0 I0916 23:01:05.395792 21049 replica.cpp:655] Replica received learned notice for position 0 I0916 23:01:05.395972 21047 leveldb.cpp:343] Persisting action (16 bytes) to leveldb took 163947ns I0916 23:01:05.395992 21047 replica.cpp:676] Persisted action at 0 I0916 23:01:05.396003
Re: Review Request 25523: Add Docker pull to docker abstraction
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25523/ --- (Updated Sept. 17, 2014, 1:06 a.m.) Review request for drill and Benjamin Hindman. Repository: mesos-git Description --- Review: https://reviews.apache.org/r/25523 Diffs (updated) - src/docker/docker.hpp e7adedb93272209231a3a9aefecfd6ccc7802ff5 src/docker/docker.cpp af51ac9058382aede61b09e06e312ad2ce6de03e src/slave/containerizer/docker.cpp 0febbac5df4126f6c8d9a06dd0ba1668d041b34a src/tests/docker_tests.cpp 826a8c1ef1b3089d416e5775fa2cf4e5cb0c26d1 Diff: https://reviews.apache.org/r/25523/diff/ Testing --- make check Thanks, Timothy Chen
Re: Review Request 25523: Add Docker pull to docker abstraction
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25523/ --- (Updated Sept. 17, 2014, 1:07 a.m.) Review request for mesos and Benjamin Hindman. Repository: mesos-git Description --- Review: https://reviews.apache.org/r/25523 Diffs - src/docker/docker.hpp e7adedb93272209231a3a9aefecfd6ccc7802ff5 src/docker/docker.cpp af51ac9058382aede61b09e06e312ad2ce6de03e src/slave/containerizer/docker.cpp 0febbac5df4126f6c8d9a06dd0ba1668d041b34a src/tests/docker_tests.cpp 826a8c1ef1b3089d416e5775fa2cf4e5cb0c26d1 Diff: https://reviews.apache.org/r/25523/diff/ Testing --- make check Thanks, Timothy Chen
Re: Build failed in Jenkins: Mesos-Trunk-Ubuntu-Build-Out-Of-Src-Disable-Java-Disable-Python-Disable-Webui #2374
Hi Ben, Sounds good, I'll take a look sometime tonight or tomorrow. Tim On Tue, Sep 16, 2014 at 8:34 PM, Benjamin Mahler benjamin.mah...@gmail.com wrote: +Tim Chen Can you take a look at the health check failure? https://issues.apache.org/jira/browse/MESOS-1802 I'll take a look at the Registrar test: https://issues.apache.org/jira/browse/MESOS-1803 On Tue, Sep 16, 2014 at 4:01 PM, Apache Jenkins Server jenk...@builds.apache.org wrote: See https://builds.apache.org/job/Mesos-Trunk-Ubuntu-Build-Out-Of-Src-Disable-Java-Disable-Python-Disable-Webui/2374/changes Changes: [bmahler] Added a Version class to stout. -- [...truncated 58686 lines...] I0916 23:01:05.386067 21051 replica.cpp:676] Persisted action at 0 I0916 23:01:05.386090 21051 replica.cpp:661] Replica learned NOP action at position 0 I0916 23:01:05.083233 21042 leveldb.cpp:343] Persisting action (16 bytes) to leveldb took 121548ns I0916 23:01:05.386147 21042 replica.cpp:676] Persisted action at 0 I0916 23:01:05.386168 21042 replica.cpp:661] Replica learned NOP action at position 0 I0916 23:01:05.386436 21040 coordinator.cpp:340] Coordinator attempting to write APPEND action at position 1 I0916 23:01:05.386842 21048 replica.cpp:508] Replica received write request for position 1 I0916 23:01:05.386904 21040 replica.cpp:508] Replica received write request for position 1 I0916 23:01:05.387473 21048 leveldb.cpp:343] Persisting action (27 bytes) to leveldb took 603894ns I0916 23:01:05.387475 21040 leveldb.cpp:343] Persisting action (27 bytes) to leveldb took 538731ns I0916 23:01:05.387511 21048 replica.cpp:676] Persisted action at 1 I0916 23:01:05.387524 21040 replica.cpp:676] Persisted action at 1 I0916 23:01:05.387969 21043 replica.cpp:655] Replica received learned notice for position 1 I0916 23:01:05.387998 21053 replica.cpp:655] Replica received learned notice for position 1 I0916 23:01:05.388157 21043 leveldb.cpp:343] Persisting action (29 bytes) to leveldb took 165034ns I0916 23:01:05.388177 21043 replica.cpp:676] Persisted action at 1 I0916 23:01:05.388188 21043 replica.cpp:661] Replica learned APPEND action at position 1 I0916 23:01:05.388212 21053 leveldb.cpp:343] Persisting action (29 bytes) to leveldb took 190343ns I0916 23:01:05.388236 21053 replica.cpp:676] Persisted action at 1 I0916 23:01:05.388245 21053 replica.cpp:661] Replica learned APPEND action at position 1 I0916 23:01:05.390319 21026 leveldb.cpp:176] Opened db in 1.951015ms I0916 23:01:05.391607 21026 leveldb.cpp:183] Compacted db in 1.261904ms I0916 23:01:05.391633 21026 leveldb.cpp:198] Created db iterator in 4244ns I0916 23:01:05.391650 21026 leveldb.cpp:204] Seeked to beginning of db in 7069ns I0916 23:01:05.391669 21026 leveldb.cpp:273] Iterated through 1 keys in the db in 8220ns I0916 23:01:05.391685 21026 replica.cpp:741] Replica recovered with log positions 0 - 0 with 1 holes and 0 unlearned I0916 23:01:05.392328 21047 replica.cpp:474] Replica received implicit promise request with proposal 1 I0916 23:01:05.392354 21047 replica.cpp:479] Replica denying promise request with proposal 1 I0916 23:01:05.392408 21043 replica.cpp:474] Replica received implicit promise request with proposal 1 I0916 23:01:05.392943 21043 leveldb.cpp:306] Persisting metadata (8 bytes) to leveldb took 510019ns I0916 23:01:05.392963 21043 replica.cpp:342] Persisted promised to 1 I0916 23:01:05.393698 21055 replica.cpp:474] Replica received implicit promise request with proposal 2 I0916 23:01:05.393705 21045 replica.cpp:474] Replica received implicit promise request with proposal 2 I0916 23:01:05.394114 21045 leveldb.cpp:306] Persisting metadata (8 bytes) to leveldb took 380162ns I0916 23:01:05.394114 21055 leveldb.cpp:306] Persisting metadata (8 bytes) to leveldb took 390574ns I0916 23:01:05.394136 21045 replica.cpp:342] Persisted promised to 2 I0916 23:01:05.394152 21055 replica.cpp:342] Persisted promised to 2 I0916 23:01:05.394424 21053 coordinator.cpp:230] Coordinator attemping to fill missing position I0916 23:01:05.395113 21046 replica.cpp:375] Replica received explicit promise request for position 0 with proposal 3 I0916 23:01:05.395136 21055 replica.cpp:375] Replica received explicit promise request for position 0 with proposal 3 I0916 23:01:05.395153 21046 leveldb.cpp:438] Reading position from leveldb took 16040ns I0916 23:01:05.395339 21055 leveldb.cpp:343] Persisting action (8 bytes) to leveldb took 181686ns I0916 23:01:05.395359 21055 replica.cpp:676] Persisted action at 0 I0916 23:01:05.395392 21046 leveldb.cpp:343] Persisting action (16 bytes) to leveldb took 215312ns I0916 23:01:05.395414 21046 replica.cpp:676] Persisted action at 0 I0916 23:01:05.395781 21047 replica.cpp:655] Replica received learned notice for position 0 I0916 23:01:05.395792 21049 replica.cpp:655] Replica received learned notice for position 0 I0916 23:01:05.395972 21047
Re: Review Request 25403: Override entrypoint when shell enabled in Docker
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25403/#review53637 --- Patch looks great! Reviews applied: [25403] All tests passed. - Mesos ReviewBot On Sept. 16, 2014, 10:37 p.m., Timothy Chen wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25403/ --- (Updated Sept. 16, 2014, 10:37 p.m.) Review request for mesos, Benjamin Hindman and Jie Yu. Bugs: MESOS-1770 https://issues.apache.org/jira/browse/MESOS-1770 Repository: mesos-git Description --- Review: https://reviews.apache.org/r/25403 Diffs - src/docker/docker.cpp af51ac9058382aede61b09e06e312ad2ce6de03e Diff: https://reviews.apache.org/r/25403/diff/ Testing --- make check Thanks, Timothy Chen
Re: Mesos 0.20.1 release status
Release update: We have finalized the issue/commits that will make it for this release. We are waiting to merge these 2 patches, before we cut 0.20.1 RC1: https://reviews.apache.org/r/25403/ https://reviews.apache.org/r/25523/ On Tue, Sep 16, 2014 at 2:14 PM, Bhuvan Arumugam bhu...@apache.org wrote: Tim, sorry. I meant today, 9/16 @6pm PDT. Vinod, yes, Adam is helping me to push CHANGELOG. I'll work with him to create tag, mvn push, etc. Jie, i'm hoping to go with tags. I'll create a tag for 0.20.1-rc1. For new RC builds (if any), i'll create new tag from previous RC and cherry-pick the bug fixes. Team, if you are working on any of these tickets, please follow-up with reviewers to +1 the patch. If these patches are not merged before 6pm, it may miss this release. I'd prefer not to delay the release schedule, considering next major release 0.21.0 will be out in 3-4 weeks. http://s.apache.org/mesos-0.20.1-unresolved-issues On Mon, Sep 15, 2014 at 11:35 PM, Timothy Chen tnac...@gmail.com wrote: It's already past 9/16 6 pm PDT? Tim On Tue, Sep 16, 2014 at 2:19 AM, Bhuvan Arumugam bhu...@apache.org wrote: We are targetting 17 tickets. It include improvements and bug fixes. 4 of them are still in progress. http://s.apache.org/mesos-0.20.1-unresolved-issues I'll cut the tag for RC1 and send for voting, once these issues are reviewed/submitted or 9/15 @6pm PDT, whichever comes first! The open issues (if any) will be moved to next release, at that point in time. In case you got questions, let me know. Thank you, -- Regards, Bhuvan Arumugam www.livecipher.com -- Regards, Bhuvan Arumugam www.livecipher.com -- Regards, Bhuvan Arumugam www.livecipher.com
Build failed in Jenkins: mesos-reviewbot #1590
See https://builds.apache.org/job/mesos-reviewbot/1590/changes Changes: [vinodkone] Enabled bridge network for Docker Containerizer. -- [...truncated 3883 lines...] make check-am make[3]: Entering directory `https://builds.apache.org/job/mesos-reviewbot/ws/mesos-0.21.0/_build/src' test ../.. = .. || \ (/bin/mkdir -p python/src/mesos cp -pf ../../src/python/src/mesos/__init__.py python/src/mesos/__init__.py) test ../.. = .. || \ (/bin/mkdir -p python/interface/src/mesos cp -pf ../../src/python/interface/src/mesos/__init__.py python/interface/src/mesos/__init__.py) test ../.. = .. || \ (/bin/mkdir -p python/interface/src/mesos/interface cp -pf ../../src/python/interface/src/mesos/interface/__init__.py python/interface/src/mesos/interface/__init__.py) test ../.. = .. || \ (/bin/mkdir -p python/native/src/mesos cp -pf ../../src/python/native/src/mesos/__init__.py python/native/src/mesos/__init__.py) test ../.. = .. || \ (/bin/mkdir -p python/native/src/mesos/native cp -pf ../../src/python/native/src/mesos/native/__init__.py python/native/src/mesos/native/__init__.py) test ../.. = .. || \ (/bin/mkdir -p python/native/src/mesos/native cp -pf ../../src/python/native/src/mesos/native/mesos_executor_driver_impl.cpp python/native/src/mesos/native/mesos_executor_driver_impl.cpp) test ../.. = .. || \ (/bin/mkdir -p python/native/src/mesos/native cp -pf ../../src/python/native/src/mesos/native/mesos_executor_driver_impl.hpp python/native/src/mesos/native/mesos_executor_driver_impl.hpp) test ../.. = .. || \ (/bin/mkdir -p python/native/src/mesos/native cp -pf ../../src/python/native/src/mesos/native/mesos_scheduler_driver_impl.cpp python/native/src/mesos/native/mesos_scheduler_driver_impl.cpp) test ../.. = .. || \ (/bin/mkdir -p python/native/src/mesos/native cp -pf ../../src/python/native/src/mesos/native/mesos_scheduler_driver_impl.hpp python/native/src/mesos/native/mesos_scheduler_driver_impl.hpp) test ../.. = .. || \ (/bin/mkdir -p python/native/src/mesos/native cp -pf ../../src/python/native/src/mesos/native/module.cpp python/native/src/mesos/native/module.cpp) test ../.. = .. || \ (/bin/mkdir -p python/native/src/mesos/native cp -pf ../../src/python/native/src/mesos/native/module.hpp python/native/src/mesos/native/module.hpp) test ../.. = .. || \ (/bin/mkdir -p python/native/src/mesos/native cp -pf ../../src/python/native/src/mesos/native/proxy_executor.cpp python/native/src/mesos/native/proxy_executor.cpp) test ../.. = .. || \ (/bin/mkdir -p python/native/src/mesos/native cp -pf ../../src/python/native/src/mesos/native/proxy_executor.hpp python/native/src/mesos/native/proxy_executor.hpp) test ../.. = .. || \ (/bin/mkdir -p python/native/src/mesos/native cp -pf ../../src/python/native/src/mesos/native/proxy_scheduler.cpp python/native/src/mesos/native/proxy_scheduler.cpp) test ../.. = .. || \ (/bin/mkdir -p python/native/src/mesos/native cp -pf ../../src/python/native/src/mesos/native/proxy_scheduler.hpp python/native/src/mesos/native/proxy_scheduler.hpp) running bdist_egg running egg_info writing requirements to src/mesos.interface.egg-info/requires.txt writing src/mesos.interface.egg-info/PKG-INFO writing namespace_packages to src/mesos.interface.egg-info/namespace_packages.txt writing top-level names to src/mesos.interface.egg-info/top_level.txt writing dependency_links to src/mesos.interface.egg-info/dependency_links.txt writing requirements to src/mesos.interface.egg-info/requires.txt writing src/mesos.interface.egg-info/PKG-INFO writing namespace_packages to src/mesos.interface.egg-info/namespace_packages.txt writing top-level names to src/mesos.interface.egg-info/top_level.txt writing dependency_links to src/mesos.interface.egg-info/dependency_links.txt reading manifest file 'src/mesos.interface.egg-info/SOURCES.txt' writing manifest file 'src/mesos.interface.egg-info/SOURCES.txt' installing library code to build/bdist.linux-x86_64/egg running install_lib running build_py creating build/bdist.linux-x86_64/egg creating build/bdist.linux-x86_64/egg/mesos creating build/bdist.linux-x86_64/egg/mesos/interface copying build/lib.linux-x86_64-2.7/mesos/interface/__init__.py - build/bdist.linux-x86_64/egg/mesos/interface copying build/lib.linux-x86_64-2.7/mesos/interface/containerizer_pb2.py -
Build failed in Jenkins: mesos-reviewbot #1591
See https://builds.apache.org/job/mesos-reviewbot/1591/ -- [...truncated 3870 lines...] make[6]: Leaving directory `https://builds.apache.org/job/mesos-reviewbot/ws/mesos-0.21.0/_build/3rdparty/libprocess' make[5]: Leaving directory `https://builds.apache.org/job/mesos-reviewbot/ws/mesos-0.21.0/_build/3rdparty/libprocess' Making check in include make[5]: Entering directory `https://builds.apache.org/job/mesos-reviewbot/ws/mesos-0.21.0/_build/3rdparty/libprocess/include' make[5]: Nothing to be done for `check'. make[5]: Leaving directory `https://builds.apache.org/job/mesos-reviewbot/ws/mesos-0.21.0/_build/3rdparty/libprocess/include' make[4]: Leaving directory `https://builds.apache.org/job/mesos-reviewbot/ws/mesos-0.21.0/_build/3rdparty/libprocess' make[4]: Entering directory `https://builds.apache.org/job/mesos-reviewbot/ws/mesos-0.21.0/_build/3rdparty' make[4]: Nothing to be done for `check-am'. make[4]: Leaving directory `https://builds.apache.org/job/mesos-reviewbot/ws/mesos-0.21.0/_build/3rdparty' make[3]: Leaving directory `https://builds.apache.org/job/mesos-reviewbot/ws/mesos-0.21.0/_build/3rdparty' make[2]: Leaving directory `https://builds.apache.org/job/mesos-reviewbot/ws/mesos-0.21.0/_build/3rdparty' Making check in src make[2]: Entering directory `https://builds.apache.org/job/mesos-reviewbot/ws/mesos-0.21.0/_build/src' make check-am make[3]: Entering directory `https://builds.apache.org/job/mesos-reviewbot/ws/mesos-0.21.0/_build/src' test ../.. = .. || \ (/bin/mkdir -p python/src/mesos cp -pf ../../src/python/src/mesos/__init__.py python/src/mesos/__init__.py) test ../.. = .. || \ (/bin/mkdir -p python/interface/src/mesos cp -pf ../../src/python/interface/src/mesos/__init__.py python/interface/src/mesos/__init__.py) test ../.. = .. || \ (/bin/mkdir -p python/interface/src/mesos/interface cp -pf ../../src/python/interface/src/mesos/interface/__init__.py python/interface/src/mesos/interface/__init__.py) test ../.. = .. || \ (/bin/mkdir -p python/native/src/mesos cp -pf ../../src/python/native/src/mesos/__init__.py python/native/src/mesos/__init__.py) test ../.. = .. || \ (/bin/mkdir -p python/native/src/mesos/native cp -pf ../../src/python/native/src/mesos/native/__init__.py python/native/src/mesos/native/__init__.py) test ../.. = .. || \ (/bin/mkdir -p python/native/src/mesos/native cp -pf ../../src/python/native/src/mesos/native/mesos_executor_driver_impl.cpp python/native/src/mesos/native/mesos_executor_driver_impl.cpp) test ../.. = .. || \ (/bin/mkdir -p python/native/src/mesos/native cp -pf ../../src/python/native/src/mesos/native/mesos_executor_driver_impl.hpp python/native/src/mesos/native/mesos_executor_driver_impl.hpp) test ../.. = .. || \ (/bin/mkdir -p python/native/src/mesos/native cp -pf ../../src/python/native/src/mesos/native/mesos_scheduler_driver_impl.cpp python/native/src/mesos/native/mesos_scheduler_driver_impl.cpp) test ../.. = .. || \ (/bin/mkdir -p python/native/src/mesos/native cp -pf ../../src/python/native/src/mesos/native/mesos_scheduler_driver_impl.hpp python/native/src/mesos/native/mesos_scheduler_driver_impl.hpp) test ../.. = .. || \ (/bin/mkdir -p python/native/src/mesos/native cp -pf ../../src/python/native/src/mesos/native/module.cpp python/native/src/mesos/native/module.cpp) test ../.. = .. || \ (/bin/mkdir -p python/native/src/mesos/native cp -pf ../../src/python/native/src/mesos/native/module.hpp python/native/src/mesos/native/module.hpp) test ../.. = .. || \ (/bin/mkdir -p python/native/src/mesos/native cp -pf ../../src/python/native/src/mesos/native/proxy_executor.cpp python/native/src/mesos/native/proxy_executor.cpp) test ../.. = .. || \ (/bin/mkdir -p python/native/src/mesos/native cp -pf ../../src/python/native/src/mesos/native/proxy_executor.hpp python/native/src/mesos/native/proxy_executor.hpp) test ../.. = .. || \ (/bin/mkdir -p python/native/src/mesos/native cp -pf ../../src/python/native/src/mesos/native/proxy_scheduler.cpp python/native/src/mesos/native/proxy_scheduler.cpp) test ../.. = .. || \ (/bin/mkdir -p python/native/src/mesos/native cp -pf ../../src/python/native/src/mesos/native/proxy_scheduler.hpp python/native/src/mesos/native/proxy_scheduler.hpp) running bdist_egg running bdist_egg running egg_info writing requirements to
Build failed in Jenkins: Mesos-Trunk-Ubuntu-Build-In-Src-Set-JAVA_HOME #2106
See https://builds.apache.org/job/Mesos-Trunk-Ubuntu-Build-In-Src-Set-JAVA_HOME/2106/changes Changes: [vinodkone] Enabled bridge network for Docker Containerizer. -- [...truncated 3121 lines...] byte-compiling build/bdist.linux-x86_64/egg/google/protobuf/descriptor_pool.py to descriptor_pool.pyc byte-compiling build/bdist.linux-x86_64/egg/google/protobuf/text_format.py to text_format.pyc byte-compiling build/bdist.linux-x86_64/egg/google/protobuf/service.py to service.pyc byte-compiling build/bdist.linux-x86_64/egg/google/protobuf/descriptor.py to descriptor.pyc byte-compiling build/bdist.linux-x86_64/egg/google/__init__.py to __init__.pyc creating build/bdist.linux-x86_64/egg/EGG-INFO copying protobuf.egg-info/PKG-INFO - build/bdist.linux-x86_64/egg/EGG-INFO copying protobuf.egg-info/SOURCES.txt - build/bdist.linux-x86_64/egg/EGG-INFO copying protobuf.egg-info/dependency_links.txt - build/bdist.linux-x86_64/egg/EGG-INFO copying protobuf.egg-info/namespace_packages.txt - build/bdist.linux-x86_64/egg/EGG-INFO copying protobuf.egg-info/requires.txt - build/bdist.linux-x86_64/egg/EGG-INFO copying protobuf.egg-info/top_level.txt - build/bdist.linux-x86_64/egg/EGG-INFO zip_safe flag not set; analyzing archive contents... creating dist creating 'dist/protobuf-2.5.0-py2.7.egg' and adding 'build/bdist.linux-x86_64/egg' to it removing 'build/bdist.linux-x86_64/egg' (and everything under it) running bdist_egg running egg_info creating src/mesos.egg-info writing requirements to src/mesos.egg-info/requires.txt writing src/mesos.egg-info/PKG-INFO writing namespace_packages to src/mesos.egg-info/namespace_packages.txt writing top-level names to src/mesos.egg-info/top_level.txt writing dependency_links to src/mesos.egg-info/dependency_links.txt writing requirements to src/mesos.egg-info/requires.txt writing src/mesos.egg-info/PKG-INFO writing namespace_packages to src/mesos.egg-info/namespace_packages.txt writing top-level names to src/mesos.egg-info/top_level.txt writing dependency_links to src/mesos.egg-info/dependency_links.txt writing manifest file 'src/mesos.egg-info/SOURCES.txt' reading manifest file 'src/mesos.egg-info/SOURCES.txt' writing manifest file 'src/mesos.egg-info/SOURCES.txt' installing library code to build/bdist.linux-x86_64/egg running install_lib running build_py creating build creating build/lib.linux-x86_64-2.7 creating build/lib.linux-x86_64-2.7/mesos copying src/mesos/__init__.py - build/lib.linux-x86_64-2.7/mesos creating build/bdist.linux-x86_64 creating build/bdist.linux-x86_64/egg creating build/bdist.linux-x86_64/egg/mesos copying build/lib.linux-x86_64-2.7/mesos/__init__.py - build/bdist.linux-x86_64/egg/mesos byte-compiling build/bdist.linux-x86_64/egg/mesos/__init__.py to __init__.pyc creating build/bdist.linux-x86_64/egg/EGG-INFO copying src/mesos.egg-info/PKG-INFO - build/bdist.linux-x86_64/egg/EGG-INFO copying src/mesos.egg-info/SOURCES.txt - build/bdist.linux-x86_64/egg/EGG-INFO copying src/mesos.egg-info/dependency_links.txt - build/bdist.linux-x86_64/egg/EGG-INFO copying src/mesos.egg-info/namespace_packages.txt - build/bdist.linux-x86_64/egg/EGG-INFO copying src/mesos.egg-info/requires.txt - build/bdist.linux-x86_64/egg/EGG-INFO copying src/mesos.egg-info/top_level.txt - build/bdist.linux-x86_64/egg/EGG-INFO zip_safe flag not set; analyzing archive contents... mesos.__init__: module references __path__ creating dist creating 'dist/mesos-0.21.0-py2.7.egg' and adding 'build/bdist.linux-x86_64/egg' to it removing 'build/bdist.linux-x86_64/egg' (and everything under it) running bdist_egg running egg_info creating src/mesos.interface.egg-info writing requirements to src/mesos.interface.egg-info/requires.txt writing src/mesos.interface.egg-info/PKG-INFO writing namespace_packages to src/mesos.interface.egg-info/namespace_packages.txt writing top-level names to src/mesos.interface.egg-info/top_level.txt writing dependency_links to src/mesos.interface.egg-info/dependency_links.txt writing requirements to src/mesos.interface.egg-info/requires.txt writing src/mesos.interface.egg-info/PKG-INFO writing namespace_packages to src/mesos.interface.egg-info/namespace_packages.txt writing top-level names to src/mesos.interface.egg-info/top_level.txt writing dependency_links to src/mesos.interface.egg-info/dependency_links.txt writing manifest file 'src/mesos.interface.egg-info/SOURCES.txt' reading manifest file 'src/mesos.interface.egg-info/SOURCES.txt' writing manifest file 'src/mesos.interface.egg-info/SOURCES.txt' installing library code to build/bdist.linux-x86_64/egg running install_lib running build_py creating build creating build/lib.linux-x86_64-2.7 creating build/lib.linux-x86_64-2.7/mesos copying src/mesos/__init__.py - build/lib.linux-x86_64-2.7/mesos creating build/lib.linux-x86_64-2.7/mesos/interface copying src/mesos/interface/__init__.py - build/lib.linux-x86_64-2.7/mesos/interface copying
Build failed in Jenkins: Mesos-Ubuntu-distcheck #334
See https://builds.apache.org/job/Mesos-Ubuntu-distcheck/334/changes Changes: [vinodkone] Enabled bridge network for Docker Containerizer. -- [...truncated 3865 lines...] libtool: link: g++ -g -g2 -O2 -Wno-unused-local-typedefs -std=c++11 -o tests tests-decoder_tests.o tests-encoder_tests.o tests-http_tests.o tests-io_tests.o tests-main.o tests-mutex_tests.o tests-metrics_tests.o tests-owned_tests.o tests-process_tests.o tests-queue_tests.o tests-reap_tests.o tests-sequence_tests.o tests-shared_tests.o tests-statistics_tests.o tests-subprocess_tests.o tests-system_tests.o tests-timeseries_tests.o tests-time_tests.o 3rdparty/.libs/libgmock.a ./.libs/libprocess.a https://builds.apache.org/job/Mesos-Ubuntu-distcheck/ws/build/mesos-0.21.0/_build/3rdparty/libprocess/3rdparty/glog-0.3.3/.libs/libglog.a https://builds.apache.org/job/Mesos-Ubuntu-distcheck/ws/build/mesos-0.21.0/_build/3rdparty/libprocess/3rdparty/libev-4.15/.libs/libev.a 3rdparty/glog-0.3.3/.libs/libglog.a -lpthread 3rdparty/.libs/libry_http_parser.a 3rdparty/libev-4.15/.libs/libev.a -lm -lz -lrt make[6]: Leaving directory `https://builds.apache.org/job/Mesos-Ubuntu-distcheck/ws/build/mesos-0.21.0/_build/3rdparty/libprocess' make check-local make[6]: Entering directory `https://builds.apache.org/job/Mesos-Ubuntu-distcheck/ws/build/mesos-0.21.0/_build/3rdparty/libprocess' ./tests WARNING: Logging before InitGoogleLogging() is written to STDERR I0917 03:00:19.049882 29960 process.cpp:1771] libprocess is initialized on 67.195.81.190:43272 for 16 cpus Note: Google Test filter = [==] Running 0 tests from 0 test cases. [==] 0 tests from 0 test cases ran. (0 ms total) [ PASSED ] 0 tests. YOU HAVE 3 DISABLED TESTS make[6]: Leaving directory `https://builds.apache.org/job/Mesos-Ubuntu-distcheck/ws/build/mesos-0.21.0/_build/3rdparty/libprocess' make[5]: Leaving directory `https://builds.apache.org/job/Mesos-Ubuntu-distcheck/ws/build/mesos-0.21.0/_build/3rdparty/libprocess' Making check in include make[5]: Entering directory `https://builds.apache.org/job/Mesos-Ubuntu-distcheck/ws/build/mesos-0.21.0/_build/3rdparty/libprocess/include' make[5]: Nothing to be done for `check'. make[5]: Leaving directory `https://builds.apache.org/job/Mesos-Ubuntu-distcheck/ws/build/mesos-0.21.0/_build/3rdparty/libprocess/include' make[4]: Leaving directory `https://builds.apache.org/job/Mesos-Ubuntu-distcheck/ws/build/mesos-0.21.0/_build/3rdparty/libprocess' make[4]: Entering directory `https://builds.apache.org/job/Mesos-Ubuntu-distcheck/ws/build/mesos-0.21.0/_build/3rdparty' make[4]: Nothing to be done for `check-am'. make[4]: Leaving directory `https://builds.apache.org/job/Mesos-Ubuntu-distcheck/ws/build/mesos-0.21.0/_build/3rdparty' make[3]: Leaving directory `https://builds.apache.org/job/Mesos-Ubuntu-distcheck/ws/build/mesos-0.21.0/_build/3rdparty' make[2]: Leaving directory `https://builds.apache.org/job/Mesos-Ubuntu-distcheck/ws/build/mesos-0.21.0/_build/3rdparty' Making check in src make[2]: Entering directory `https://builds.apache.org/job/Mesos-Ubuntu-distcheck/ws/build/mesos-0.21.0/_build/src' make check-am make[3]: Entering directory `https://builds.apache.org/job/Mesos-Ubuntu-distcheck/ws/build/mesos-0.21.0/_build/src' test ../.. = .. || \ (/bin/mkdir -p python/src/mesos cp -pf ../../src/python/src/mesos/__init__.py python/src/mesos/__init__.py) test ../.. = .. || \ (/bin/mkdir -p python/interface/src/mesos cp -pf ../../src/python/interface/src/mesos/__init__.py python/interface/src/mesos/__init__.py) test ../.. = .. || \ (/bin/mkdir -p python/interface/src/mesos/interface cp -pf ../../src/python/interface/src/mesos/interface/__init__.py python/interface/src/mesos/interface/__init__.py) test ../.. = .. || \ (/bin/mkdir -p python/native/src/mesos cp -pf ../../src/python/native/src/mesos/__init__.py python/native/src/mesos/__init__.py) test ../.. = .. || \ (/bin/mkdir -p python/native/src/mesos/native cp -pf ../../src/python/native/src/mesos/native/__init__.py python/native/src/mesos/native/__init__.py) test ../.. = .. || \ (/bin/mkdir -p python/native/src/mesos/native cp -pf ../../src/python/native/src/mesos/native/mesos_executor_driver_impl.cpp python/native/src/mesos/native/mesos_executor_driver_impl.cpp) test ../.. = .. || \ (/bin/mkdir -p python/native/src/mesos/native cp -pf ../../src/python/native/src/mesos/native/mesos_executor_driver_impl.hpp python/native/src/mesos/native/mesos_executor_driver_impl.hpp) test ../.. = .. || \ (/bin/mkdir -p python/native/src/mesos/native cp -pf
