Re: Review Request 17345: Replaced Log::Writer constructor with explicit Log::Writer::start.
On March 4, 2014, 2:11 a.m., Ben Mahler wrote: src/tests/log_tests.cpp, lines 1462-1464 https://reviews.apache.org/r/17345/diff/2/?file=497565#file497565line1462 Do you need to store 'start'? AWAIT_READY(writer.start()); Yes, I should check that start returned some position! Fixed. - Benjamin --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/17345/#review36061 --- On Feb. 19, 2014, 8:03 a.m., Benjamin Hindman wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/17345/ --- (Updated Feb. 19, 2014, 8:03 a.m.) Review request for mesos and Jie Yu. Repository: mesos-git Description --- See summary. Diffs - src/java/jni/org_apache_mesos_Log.cpp 36c636d155c1581eeb7734cdbc5b6fac4ca42440 src/log/log.hpp 1f0b30ddf8709cf64db7989775c8b0e926af99b5 src/log/log.cpp e83f822af86a2389e2b1abab9489713cb59838c2 src/log/tool/benchmark.cpp 7d5a032ec4f1652ce80f2486989c01cb6f29a5d4 src/tests/log_tests.cpp 7e6dbdee0a9b316e49fda5fd36a9f76331265a24 Diff: https://reviews.apache.org/r/17345/diff/ Testing --- make check Thanks, Benjamin Hindman
Re: Review Request 17343: Exposed coordinator demotion.
On March 3, 2014, 10:49 p.m., Ben Mahler wrote: src/tests/log_tests.cpp, line 700 https://reviews.apache.org/r/17343/diff/2/?file=497560#file497560line700 AWAIT_READY uses a default of 10 Seconds, any reason for the explicit duration here? AWAIT_READY used to have a lower duration and when it was increased these weren't changed. :( Alas, I added this to https://reviews.apache.org/r/18263. - Benjamin --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/17343/#review36048 --- On Feb. 19, 2014, 8:02 a.m., Benjamin Hindman wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/17343/ --- (Updated Feb. 19, 2014, 8:02 a.m.) Review request for mesos and Jie Yu. Repository: mesos-git Description --- See summary Diffs - src/log/coordinator.hpp 35b68e938c5f2ca76bbdf8dbb0184ef1686d94f1 src/log/coordinator.cpp daa1ca8e719798743e46cb1f7b3f1cdd569b5f6d src/log/log.hpp 1f0b30ddf8709cf64db7989775c8b0e926af99b5 src/log/log.cpp e83f822af86a2389e2b1abab9489713cb59838c2 src/state/zookeeper.hpp d1d1fedf27987aeaf9fbdee678d3b3848d05620a src/tests/log_tests.cpp 7e6dbdee0a9b316e49fda5fd36a9f76331265a24 Diff: https://reviews.apache.org/r/17343/diff/ Testing --- make check Thanks, Benjamin Hindman
Re: Review Request 18945: Add --version flag to mesos master and slave
On March 9, 2014, 5:56 p.m., Benjamin Hindman wrote: Submitted, thanks Nikita! - Benjamin --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18945/#review36604 --- On March 9, 2014, 8:32 p.m., Nikita Vetoshkin wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18945/ --- (Updated March 9, 2014, 8:32 p.m.) Review request for mesos. Bugs: MESOS-1068 https://issues.apache.org/jira/browse/MESOS-1068 Repository: mesos-git Description --- Subj. Looks like a bit copy-paste, please advise where to put this to be shared between slave and master code. I can see some useful defines like -DPACKAGE_STRING during build, but I don't see if it's used somewhere. Diffs - src/master/flags.hpp 159b2de5878927613ba94f81005dba601f072026 src/master/main.cpp 4c74a1b387dc99aa223cb9cf8a096d3b4a126a0a src/slave/flags.hpp e4d98a53cbfb7f9ca828f17e82d492274cb9969d src/slave/main.cpp a498a6ae6a79c7155c07a5d6dc2d6c9dc8ae060f Diff: https://reviews.apache.org/r/18945/diff/ Testing --- make check passes, manually tested --version option on master and slave Thanks, Nikita Vetoshkin
[jira] [Resolved] (MESOS-1068) No --version command line parameter
[ https://issues.apache.org/jira/browse/MESOS-1068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benjamin Hindman resolved MESOS-1068. - Resolution: Fixed Assignee: Nikita Vetoshkin commit e7dc7b9656f3079433a3240236b2012e94135a8b Author: Nikita Vetoshkin nikita.vetosh...@gmail.com Date: Sun Mar 9 23:41:26 2014 -0700 Add --version flag to mesos master and slave. Review: https://reviews.apache.org/r/18945 No --version command line parameter --- Key: MESOS-1068 URL: https://issues.apache.org/jira/browse/MESOS-1068 Project: Mesos Issue Type: Improvement Components: master Affects Versions: 0.16.0 Environment: Ubuntu 13.10 Reporter: Ted M. Young Assignee: Nikita Vetoshkin Labels: newbie Fix For: 0.19.0 There's no version indication in either the Mesos web UI or from the command line. It'd be great if there was a --version argument that returned this information. -- This message was sent by Atlassian JIRA (v6.2#6252)
Jenkins build is back to normal : Mesos-Trunk-Ubuntu-Build-In-Src-Set-JAVA_HOME #1686
See https://builds.apache.org/job/Mesos-Trunk-Ubuntu-Build-In-Src-Set-JAVA_HOME/1686/changes
Review Request 18952: Refactored Promise to be thread-safe.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18952/ --- Review request for mesos, Ben Mahler and Jie Yu. Repository: mesos-git Description --- In particular, doing associations is strictly safer now. Diffs - 3rdparty/libprocess/include/process/future.hpp 27b0970bf1d1ae1b977ddfc2de5ee858f1031bf5 Diff: https://reviews.apache.org/r/18952/diff/ Testing --- make check Thanks, Benjamin Hindman
Review Request 18951: Simplified future await latch semantics.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18951/ --- Review request for mesos, Ben Mahler and Jie Yu. Repository: mesos-git Description --- Simplify. Diffs - 3rdparty/libprocess/include/process/future.hpp 27b0970bf1d1ae1b977ddfc2de5ee858f1031bf5 Diff: https://reviews.apache.org/r/18951/diff/ Testing --- make check Thanks, Benjamin Hindman
Review Request 18950: Added an asynchronous mutex.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18950/ --- Review request for mesos, Ben Mahler, Jie Yu, and Vinod Kone. Repository: mesos-git Description --- This is a simpler alternative to using something like 'Sequence', although it requires that one explicitly perform an action after the serialized work is done using something like Future::onAny which now works correctly thanks to the updated discard semantics. Diffs - 3rdparty/libprocess/Makefile.am 3c6219eb6e76306463b3710ab7e50ec8b75d3d76 3rdparty/libprocess/include/process/future.hpp 27b0970bf1d1ae1b977ddfc2de5ee858f1031bf5 3rdparty/libprocess/include/process/internal.hpp PRE-CREATION 3rdparty/libprocess/include/process/mutex.hpp PRE-CREATION 3rdparty/libprocess/src/tests/mutex_tests.cpp PRE-CREATION Diff: https://reviews.apache.org/r/18950/diff/ Testing --- make check Thanks, Benjamin Hindman
Review Request 18954: Added Future::after.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18954/ --- Review request for mesos, Ben Mahler, Jie Yu, and Vinod Kone. Repository: mesos-git Description --- This is to avoid a standard pattern seen with doing a 'delay' after some asynchronous function call has been made and we're waiting on it's completion via Future::then, onAny, etc. This allows one to do: someAsynchronousFunction() .then(defer(self(), Self::_continuation1)) .then(defer(self(), Self::_continuation2)) .after(Seconds(30), defer(self(), Self::_timeout, lambda::_1)); Diffs - 3rdparty/libprocess/include/process/future.hpp 27b0970bf1d1ae1b977ddfc2de5ee858f1031bf5 3rdparty/libprocess/include/process/http.hpp 7f549ba3476ecf5dec0db21d57ee58bcd73d5996 3rdparty/libprocess/src/tests/process_tests.cpp e899aed7dbe6e8645484fe0ba69521e9fb0fdad2 Diff: https://reviews.apache.org/r/18954/diff/ Testing --- make check Thanks, Benjamin Hindman
Re: Review Request 18950: Added an asynchronous mutex.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18950/ --- (Updated March 10, 2014, 8:03 a.m.) Review request for mesos, Ben Mahler, Jie Yu, and Vinod Kone. Repository: mesos-git Description --- This is a simpler alternative to using something like 'Sequence', although it requires that one explicitly perform an action after the serialized work is done using something like Future::onAny which now works correctly thanks to the updated discard semantics. Diffs - 3rdparty/libprocess/Makefile.am 3c6219eb6e76306463b3710ab7e50ec8b75d3d76 3rdparty/libprocess/include/process/future.hpp 27b0970bf1d1ae1b977ddfc2de5ee858f1031bf5 3rdparty/libprocess/include/process/internal.hpp PRE-CREATION 3rdparty/libprocess/include/process/mutex.hpp PRE-CREATION 3rdparty/libprocess/src/tests/mutex_tests.cpp PRE-CREATION Diff: https://reviews.apache.org/r/18950/diff/ Testing --- make check Thanks, Benjamin Hindman
Review Request 18953: Improved thread-safety of Latch.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18953/ --- Review request for mesos and Ben Mahler. Repository: mesos-git Description --- See summary. Diffs - 3rdparty/libprocess/include/process/latch.hpp 5170aa8397ca832a511c55fdc4abfe4678b02f89 3rdparty/libprocess/src/latch.cpp a6f1256ff8860e9a624c0f200ea53912048fd6e3 Diff: https://reviews.apache.org/r/18953/diff/ Testing --- make check Thanks, Benjamin Hindman
Re: Review Request 18952: Refactored Promise to be thread-safe.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18952/ --- (Updated March 10, 2014, 8:03 a.m.) Review request for mesos, Ben Mahler and Jie Yu. Repository: mesos-git Description --- In particular, doing associations is strictly safer now. Diffs - 3rdparty/libprocess/include/process/future.hpp 27b0970bf1d1ae1b977ddfc2de5ee858f1031bf5 Diff: https://reviews.apache.org/r/18952/diff/ Testing --- make check Thanks, Benjamin Hindman
Review Request 18954: Added Future::after.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18954/ --- Review request for mesos, Ben Mahler, Jie Yu, and Vinod Kone. Repository: mesos-git Description --- This is to avoid a standard pattern seen with doing a 'delay' after some asynchronous function call has been made and we're waiting on it's completion via Future::then, onAny, etc. This allows one to do: someAsynchronousFunction() .then(defer(self(), Self::_continuation1)) .then(defer(self(), Self::_continuation2)) .after(Seconds(30), defer(self(), Self::_timeout, lambda::_1)); Diffs - 3rdparty/libprocess/include/process/future.hpp 27b0970bf1d1ae1b977ddfc2de5ee858f1031bf5 3rdparty/libprocess/include/process/http.hpp 7f549ba3476ecf5dec0db21d57ee58bcd73d5996 3rdparty/libprocess/src/tests/process_tests.cpp e899aed7dbe6e8645484fe0ba69521e9fb0fdad2 Diff: https://reviews.apache.org/r/18954/diff/ Testing --- make check Thanks, Benjamin Hindman
Review Request 18953: Improved thread-safety of Latch.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18953/ --- Review request for mesos and Ben Mahler. Repository: mesos-git Description --- See summary. Diffs - 3rdparty/libprocess/include/process/latch.hpp 5170aa8397ca832a511c55fdc4abfe4678b02f89 3rdparty/libprocess/src/latch.cpp a6f1256ff8860e9a624c0f200ea53912048fd6e3 Diff: https://reviews.apache.org/r/18953/diff/ Testing --- make check Thanks, Benjamin Hindman
[jira] [Created] (MESOS-1076) Add namespacing to cgroups to enforce the expected structure in slave/containerizer/isolators/cgroups/cpushare.cpp
Archana kumari created MESOS-1076: - Summary: Add namespacing to cgroups to enforce the expected structure in slave/containerizer/isolators/cgroups/cpushare.cpp Key: MESOS-1076 URL: https://issues.apache.org/jira/browse/MESOS-1076 Project: Mesos Issue Type: Improvement Components: general, slave Reporter: Archana kumari The point of the issue is to provide structural information from the cgroups API -- This message was sent by Atlassian JIRA (v6.2#6252)
Jenkins build is back to normal : Mesos-Trunk-Ubuntu-Build-Out-Of-Src-Set-JAVA_HOME #1957
See https://builds.apache.org/job/Mesos-Trunk-Ubuntu-Build-Out-Of-Src-Set-JAVA_HOME/1957/changes
[jira] [Commented] (MESOS-1076) Add namespacing to cgroups to enforce the expected structure in slave/containerizer/isolators/cgroups/cpushare.cpp
[ https://issues.apache.org/jira/browse/MESOS-1076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13925543#comment-13925543 ] Archana kumari commented on MESOS-1076: --- I found the namespace cgroups in https://github.com/apache/mesos/blob/bcd4dc19e10c4b54b90fae713d14477b849147c2/src/linux/cgroups.hpp#L431 The stat function present here is present inside the namepace cgroups now, so I need it to put it inside cpu and memory namepace with the modifications pointed by you and also add a cpuacct namespace in the file with a stat function to read the cpuacct.stat, and then access stat in https://github.com/apache/mesos/blob/master/src/slave/containerizer/isolators/cgroups/cpushare.cpp#L386 using the structure cgroups::cpu::stat() and like wise.. Anyone please advice me if I am correct.. Add namespacing to cgroups to enforce the expected structure in slave/containerizer/isolators/cgroups/cpushare.cpp Key: MESOS-1076 URL: https://issues.apache.org/jira/browse/MESOS-1076 Project: Mesos Issue Type: Improvement Components: general, slave Reporter: Archana kumari Original Estimate: 168h Remaining Estimate: 168h The point of the issue is to provide structural information from the cgroups API -- This message was sent by Atlassian JIRA (v6.2#6252)
OPW - Contributor request
Hello, My name is Isabel Jimenez and I would like to apply to the Outreach Program for Women with Mesos, my username in JIRA is ijimenez. I would like to be added as a contributor so I can assign myself an Issue an propose a patch for it. Thank you, Isabel
Review Request 18957: Added --without-included-leveldb flag
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18957/ --- Review request for mesos, Adam B, Benjamin Hindman, Ben Mahler, Niklas Nielsen, Timothy St. Clair, and Vinod Kone. Bugs: MESOS-1071 https://issues.apache.org/jira/browse/MESOS-1071 Repository: mesos-git Description --- Allows preventing the use of the currently bundled leveldb in favor of a preinstalled version. Diffs - 3rdparty/Makefile.am 8e1d915 configure.ac 390f11b src/Makefile.am 384b312 src/python/setup.py.in 02f00ef Diff: https://reviews.apache.org/r/18957/diff/ Testing --- ../configure make check OSX: brew installed leveldb comes with activated libsnappy ../configure --without-included-leveldb CPPFLAGS=-I/usr/local/include LDFLAGS=-L/usr/local/lib LIBS=-lleveldb -lsnappy make check linux: manually installed leveldb without libsnappy ../configure --without-included-leveldb CPPFLAGS=-I/usr/local/include LDFLAGS=-L/usr/local/lib LIBS=-lleveldb make check Thanks, Till Toenshoff
[jira] [Commented] (MESOS-1071) Enable building against installed third-party dependencies.
[ https://issues.apache.org/jira/browse/MESOS-1071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13925636#comment-13925636 ] Till Toenshoff commented on MESOS-1071: --- As small step in the hopefully correct direction, I have just proposed https://reviews.apache.org/r/18957/ to allow for --without-included-leveldb. Enable building against installed third-party dependencies. --- Key: MESOS-1071 URL: https://issues.apache.org/jira/browse/MESOS-1071 Project: Mesos Issue Type: Improvement Components: build Reporter: Benjamin Hindman Most of our third-party dependencies are included in the project and statically linked into our resulting binaries and libraries. We would like to enable building Mesos but using system installed dependencies instead. In certain circumstances this is more difficult because we've actually needed to patch these libraries (either for C++11 or to alter semantics). Rather than eliminating our internal copies of these third-party dependencies the first step should be to just enable using external (i.e., system installed) dependencies. We already do this for ZooKeeper by allowing people to use the --without-included-zookeeper flag during compilation. We should do this for other libraries as well. In fact, for the libraries that we have not patched (and even for some that we have patched) we should check to see if an appropriate system installed dependency exists and preferentially use that unless --with-included-dependency is explicitly used. Note that this issue represents a stepping stone to removing our third-party dependencies from our repository. -- This message was sent by Atlassian JIRA (v6.2#6252)
Re: Review Request 18957: Added --without-included-leveldb flag
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18957/#review36634 --- Patch looks great! Reviews applied: [18957] All tests passed. - Mesos ReviewBot On March 10, 2014, 10:36 a.m., Till Toenshoff wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18957/ --- (Updated March 10, 2014, 10:36 a.m.) Review request for mesos, Adam B, Benjamin Hindman, Ben Mahler, Niklas Nielsen, Timothy St. Clair, and Vinod Kone. Bugs: MESOS-1071 https://issues.apache.org/jira/browse/MESOS-1071 Repository: mesos-git Description --- Allows preventing the use of the currently bundled leveldb in favor of a preinstalled version. Diffs - 3rdparty/Makefile.am 8e1d915 configure.ac 390f11b src/Makefile.am 384b312 src/python/setup.py.in 02f00ef Diff: https://reviews.apache.org/r/18957/diff/ Testing --- ../configure make check OSX: brew installed leveldb comes with activated libsnappy ../configure --without-included-leveldb CPPFLAGS=-I/usr/local/include LDFLAGS=-L/usr/local/lib LIBS=-lleveldb -lsnappy make check linux: manually installed leveldb without libsnappy ../configure --without-included-leveldb CPPFLAGS=-I/usr/local/include LDFLAGS=-L/usr/local/lib LIBS=-lleveldb make check Thanks, Till Toenshoff
Re: Review Request 18954: Added Future::after.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18954/#review36635 --- Bad patch! Reviews applied: [18954] Failed command: git apply --index 18954.patch Error: error: patch failed: 3rdparty/libprocess/include/process/future.hpp:8 error: 3rdparty/libprocess/include/process/future.hpp: patch does not apply - Mesos ReviewBot On March 10, 2014, 8:03 a.m., Benjamin Hindman wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18954/ --- (Updated March 10, 2014, 8:03 a.m.) Review request for mesos, Ben Mahler, Jie Yu, and Vinod Kone. Repository: mesos-git Description --- This is to avoid a standard pattern seen with doing a 'delay' after some asynchronous function call has been made and we're waiting on it's completion via Future::then, onAny, etc. This allows one to do: someAsynchronousFunction() .then(defer(self(), Self::_continuation1)) .then(defer(self(), Self::_continuation2)) .after(Seconds(30), defer(self(), Self::_timeout, lambda::_1)); Diffs - 3rdparty/libprocess/include/process/future.hpp 27b0970bf1d1ae1b977ddfc2de5ee858f1031bf5 3rdparty/libprocess/include/process/http.hpp 7f549ba3476ecf5dec0db21d57ee58bcd73d5996 3rdparty/libprocess/src/tests/process_tests.cpp e899aed7dbe6e8645484fe0ba69521e9fb0fdad2 Diff: https://reviews.apache.org/r/18954/diff/ Testing --- make check Thanks, Benjamin Hindman
Re: Review Request 18953: Improved thread-safety of Latch.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18953/#review36636 --- Bad patch! Reviews applied: [18952] Failed command: git apply --index 18952.patch Error: error: patch failed: 3rdparty/libprocess/include/process/future.hpp:473 error: 3rdparty/libprocess/include/process/future.hpp: patch does not apply - Mesos ReviewBot On March 10, 2014, 8:03 a.m., Benjamin Hindman wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18953/ --- (Updated March 10, 2014, 8:03 a.m.) Review request for mesos and Ben Mahler. Repository: mesos-git Description --- See summary. Diffs - 3rdparty/libprocess/include/process/latch.hpp 5170aa8397ca832a511c55fdc4abfe4678b02f89 3rdparty/libprocess/src/latch.cpp a6f1256ff8860e9a624c0f200ea53912048fd6e3 Diff: https://reviews.apache.org/r/18953/diff/ Testing --- make check Thanks, Benjamin Hindman
Re: Review Request 18950: Added an asynchronous mutex.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18950/#review36637 --- Patch looks great! Reviews applied: [18950] All tests passed. - Mesos ReviewBot On March 10, 2014, 8:03 a.m., Benjamin Hindman wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18950/ --- (Updated March 10, 2014, 8:03 a.m.) Review request for mesos, Ben Mahler, Jie Yu, and Vinod Kone. Repository: mesos-git Description --- This is a simpler alternative to using something like 'Sequence', although it requires that one explicitly perform an action after the serialized work is done using something like Future::onAny which now works correctly thanks to the updated discard semantics. Diffs - 3rdparty/libprocess/Makefile.am 3c6219eb6e76306463b3710ab7e50ec8b75d3d76 3rdparty/libprocess/include/process/future.hpp 27b0970bf1d1ae1b977ddfc2de5ee858f1031bf5 3rdparty/libprocess/include/process/internal.hpp PRE-CREATION 3rdparty/libprocess/include/process/mutex.hpp PRE-CREATION 3rdparty/libprocess/src/tests/mutex_tests.cpp PRE-CREATION Diff: https://reviews.apache.org/r/18950/diff/ Testing --- make check Thanks, Benjamin Hindman
Re: Review Request 18594: Added liveProcesses to process tree abstraction.
On Feb. 28, 2014, 6:43 p.m., Ben Mahler wrote: Did you still want this now that r/18604/ is submitted? Niklas Nielsen wrote: Till has been using them in a reworked version of signal escalation in PluggableContainerizer::destroy(). We wanted to give the external program a chance to clean up before sigkilling. He is currently working on it; I'll drop them if he decides to go with a different approach :) This indeed has been rather valuable for the PluggableContainerizer, hence it would be great if we got this committed. In my tests, this approach worked flawlessly. - Till --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18594/#review35813 --- On Feb. 28, 2014, 12:55 a.m., Niklas Nielsen wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18594/ --- (Updated Feb. 28, 2014, 12:55 a.m.) Review request for mesos and Ben Mahler. Repository: mesos-git Description --- liveProcesses() returns a count of the existing processes in a process tree. This is used by up coming signal escalation. Diffs - 3rdparty/libprocess/3rdparty/stout/include/stout/os/process.hpp 26f4fbe Diff: https://reviews.apache.org/r/18594/diff/ Testing --- Functional testing with signal escalation code and make check. Thanks, Niklas Nielsen
Re: Review Request 18951: Simplified future await latch semantics.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18951/#review36639 --- Bad patch! Reviews applied: [18951] Failed command: git apply --index 18951.patch Error: error: patch failed: 3rdparty/libprocess/include/process/future.hpp:13 error: 3rdparty/libprocess/include/process/future.hpp: patch does not apply - Mesos ReviewBot On March 10, 2014, 8:02 a.m., Benjamin Hindman wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18951/ --- (Updated March 10, 2014, 8:02 a.m.) Review request for mesos, Ben Mahler and Jie Yu. Repository: mesos-git Description --- Simplify. Diffs - 3rdparty/libprocess/include/process/future.hpp 27b0970bf1d1ae1b977ddfc2de5ee858f1031bf5 Diff: https://reviews.apache.org/r/18951/diff/ Testing --- make check Thanks, Benjamin Hindman
Re: Review Request 18595: Added overloaded killtree for process trees.
On March 7, 2014, 3:35 a.m., Ben Mahler wrote: 3rdparty/libprocess/3rdparty/stout/include/stout/os/killtree.hpp, line 51 https://reviews.apache.org/r/18595/diff/1/?file=506546#file506546line51 This seems a little strange, because we'll be trying to signal multiple times: root - signaled 1 time child - signaled 2 times grandchild - signaled 3 times ... Nth level pid - signaled N times Also, how will this be used? Let's say you use this for a 5 minute signal escalation on the tree originally returned by killtree(pid_t, int, bool, bool). 5 minutes later, how do you know the pids are still valid and you don't accidentally kill unrelated processes? Ben Mahler wrote: Actually it's probably more productive to review r/18597 first. :) Niklas Nielsen wrote: This RR was mostly to address killing orphan processes or in other words, have a way to kill the processes that belong to a process tree root which might shutdown before its children. With the current killtree implementation, it seems to me that this would fail (the pid can't be resolved to a process and returns). Till mentioned that we might end up killing new processes same pids, which is a problem. I am not sure how we can address both; I'll take a second stab at it tomorrow :) Yep, I am a bit concerned about pid-reuse. As all our options are poll-based (this approach in connection with RR18594 and a delayed-continuation) as well as process::reaper, this approach seemed a bit safer in regard to the polling period. process::reaper uses 1 second whereas the use of this can be freely adjusted to smaller intervals (see implementation of PluggableContainerizer's destroy escalation). In the end however, all solutions we have so far appear to carry the risk of killing unrelated (reused) pids. For the signal repetition on childs within the same pid session or group, that indeed might be a great point and me might want to address it. - Till --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18595/#review36486 --- On Feb. 28, 2014, 12:54 a.m., Niklas Nielsen wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18595/ --- (Updated Feb. 28, 2014, 12:54 a.m.) Review request for mesos and Ben Mahler. Repository: mesos-git Description --- New killtree(ProcessTree tree, int signal) traverse process tree and sends a signal to all pids. This is done regardless of presence and state of process. Patch is used by up coming signal escalation. Diffs - 3rdparty/libprocess/3rdparty/stout/include/stout/os/killtree.hpp 1f45897 Diff: https://reviews.apache.org/r/18595/diff/ Testing --- Functional testing with signal escalation code and make check. Thanks, Niklas Nielsen
Re: Review Request 18957: Added --without-included-leveldb flag
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18957/#review36642 --- So IMHO, --without-XYZ will become an onerous pattern to follow when building all of Mesos against system dependencies. It might make more sense to create a --proper=yes flag, where it searches the system for the dependencies, and fails if it can not find them. - Timothy St. Clair On March 10, 2014, 10:36 a.m., Till Toenshoff wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18957/ --- (Updated March 10, 2014, 10:36 a.m.) Review request for mesos, Adam B, Benjamin Hindman, Ben Mahler, Niklas Nielsen, Timothy St. Clair, and Vinod Kone. Bugs: MESOS-1071 https://issues.apache.org/jira/browse/MESOS-1071 Repository: mesos-git Description --- Allows preventing the use of the currently bundled leveldb in favor of a preinstalled version. Diffs - 3rdparty/Makefile.am 8e1d915 configure.ac 390f11b src/Makefile.am 384b312 src/python/setup.py.in 02f00ef Diff: https://reviews.apache.org/r/18957/diff/ Testing --- ../configure make check OSX: brew installed leveldb comes with activated libsnappy ../configure --without-included-leveldb CPPFLAGS=-I/usr/local/include LDFLAGS=-L/usr/local/lib LIBS=-lleveldb -lsnappy make check linux: manually installed leveldb without libsnappy ../configure --without-included-leveldb CPPFLAGS=-I/usr/local/include LDFLAGS=-L/usr/local/lib LIBS=-lleveldb make check Thanks, Till Toenshoff
Re: Review Request 18957: Added --without-included-leveldb flag
On March 10, 2014, 3:10 p.m., Timothy St. Clair wrote: So IMHO, --without-XYZ will become an onerous pattern to follow when building all of Mesos against system dependencies. It might make more sense to create a --proper=yes flag, where it searches the system for the dependencies, and fails if it can not find them. You can still leverage the CPPFLAGS LDFLAGS as before. - Timothy --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18957/#review36642 --- On March 10, 2014, 10:36 a.m., Till Toenshoff wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18957/ --- (Updated March 10, 2014, 10:36 a.m.) Review request for mesos, Adam B, Benjamin Hindman, Ben Mahler, Niklas Nielsen, Timothy St. Clair, and Vinod Kone. Bugs: MESOS-1071 https://issues.apache.org/jira/browse/MESOS-1071 Repository: mesos-git Description --- Allows preventing the use of the currently bundled leveldb in favor of a preinstalled version. Diffs - 3rdparty/Makefile.am 8e1d915 configure.ac 390f11b src/Makefile.am 384b312 src/python/setup.py.in 02f00ef Diff: https://reviews.apache.org/r/18957/diff/ Testing --- ../configure make check OSX: brew installed leveldb comes with activated libsnappy ../configure --without-included-leveldb CPPFLAGS=-I/usr/local/include LDFLAGS=-L/usr/local/lib LIBS=-lleveldb -lsnappy make check linux: manually installed leveldb without libsnappy ../configure --without-included-leveldb CPPFLAGS=-I/usr/local/include LDFLAGS=-L/usr/local/lib LIBS=-lleveldb make check Thanks, Till Toenshoff
Re: Review Request 17567: Pluggable Containerizer
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/17567/ --- (Updated March 10, 2014, 3:15 p.m.) Review request for mesos, Adam B, Benjamin Hindman, Ben Mahler, Ian Downes, Niklas Nielsen, and Vinod Kone. Changes --- Rebased and removed committed dependencies from the list. Bugs: MESOS-816 https://issues.apache.org/jira/browse/MESOS-816 Repository: mesos-git Description --- This patch adds the so-called pluggable containerizer. This containerizer delegates all containerizer calls directly to an external, pluggable containerizer program (which can be specified on start-up). Few calls have internal fall-back implementations such as wait(), destroy() and usage(). The protocol for the interactions with the external program is as follows: COMMAND (ADDITIONAL-PARAMETERS) INPUT-PROTO RESULT-PROTO launch (ContainerID, --mesos-executor, path) TaskInfo PluggableStatus update (ContainerID) ResourceArray PluggableStatus usage (ContainerID) ResourceStatistics wait (ContainerID) PluggableTermination destroy (ContainerID) PluggableStatus When protocol buffers need to be provided, the Mesos side of the pluggable containerizer implementation will serialize the protos on stdin and vice-versa for reading protos on stdout as drafted in the above scheme. Diffs (updated) - configure.ac a84b960 include/mesos/mesos.proto 37f8a7f src/Makefile.am 61d832b src/examples/python/test-containerizer.in PRE-CREATION src/examples/python/test_containerizer.py PRE-CREATION src/slave/containerizer/containerizer.cpp d0a1023 src/slave/containerizer/pluggable_containerizer.hpp PRE-CREATION src/slave/containerizer/pluggable_containerizer.cpp PRE-CREATION src/slave/flags.hpp e4d98a5 src/tests/pluggable_containerizer_test.cpp PRE-CREATION Diff: https://reviews.apache.org/r/17567/diff/ Testing --- make check and functional testing. Thanks, Till Toenshoff
Re: Review Request 17567: Pluggable Containerizer
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/17567/ --- (Updated March 10, 2014, 3:18 p.m.) Review request for mesos, Adam B, Benjamin Hindman, Ben Mahler, Ian Downes, Niklas Nielsen, and Vinod Kone. Changes --- Rebased. Bugs: MESOS-816 https://issues.apache.org/jira/browse/MESOS-816 Repository: mesos-git Description --- This patch adds the so-called pluggable containerizer. This containerizer delegates all containerizer calls directly to an external, pluggable containerizer program (which can be specified on start-up). Few calls have internal fall-back implementations such as wait(), destroy() and usage(). The protocol for the interactions with the external program is as follows: COMMAND (ADDITIONAL-PARAMETERS) INPUT-PROTO RESULT-PROTO launch (ContainerID, --mesos-executor, path) TaskInfo PluggableStatus update (ContainerID) ResourceArray PluggableStatus usage (ContainerID) ResourceStatistics wait (ContainerID) PluggableTermination destroy (ContainerID) PluggableStatus When protocol buffers need to be provided, the Mesos side of the pluggable containerizer implementation will serialize the protos on stdin and vice-versa for reading protos on stdout as drafted in the above scheme. Diffs (updated) - configure.ac 390f11b include/mesos/mesos.proto 37f8a7f src/Makefile.am 384b312 src/slave/containerizer/containerizer.cpp d0a1023 src/slave/flags.hpp c9a627b Diff: https://reviews.apache.org/r/17567/diff/ Testing --- make check and functional testing. Thanks, Till Toenshoff
Re: Review Request 17567: Pluggable Containerizer
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/17567/ --- (Updated March 10, 2014, 3:22 p.m.) Review request for mesos, Adam B, Benjamin Hindman, Ben Mahler, Ian Downes, Niklas Nielsen, and Vinod Kone. Changes --- Readded uncommitted dependencies. Review-Board keeps forgetting them once the diff is updated. Sorry for this spam. Bugs: MESOS-816 https://issues.apache.org/jira/browse/MESOS-816 Repository: mesos-git Description --- This patch adds the so-called pluggable containerizer. This containerizer delegates all containerizer calls directly to an external, pluggable containerizer program (which can be specified on start-up). Few calls have internal fall-back implementations such as wait(), destroy() and usage(). The protocol for the interactions with the external program is as follows: COMMAND (ADDITIONAL-PARAMETERS) INPUT-PROTO RESULT-PROTO launch (ContainerID, --mesos-executor, path) TaskInfo PluggableStatus update (ContainerID) ResourceArray PluggableStatus usage (ContainerID) ResourceStatistics wait (ContainerID) PluggableTermination destroy (ContainerID) PluggableStatus When protocol buffers need to be provided, the Mesos side of the pluggable containerizer implementation will serialize the protos on stdin and vice-versa for reading protos on stdout as drafted in the above scheme. Diffs - configure.ac 390f11b include/mesos/mesos.proto 37f8a7f src/Makefile.am 384b312 src/slave/containerizer/containerizer.cpp d0a1023 src/slave/flags.hpp c9a627b Diff: https://reviews.apache.org/r/17567/diff/ Testing --- make check and functional testing. Thanks, Till Toenshoff
Re: Review Request 18957: Added --without-included-leveldb flag
On March 10, 2014, 3:10 p.m., Timothy St. Clair wrote: So IMHO, --without-XYZ will become an onerous pattern to follow when building all of Mesos against system dependencies. It might make more sense to create a --proper=yes flag, where it searches the system for the dependencies, and fails if it can not find them. Timothy St. Clair wrote: You can still leverage the CPPFLAGS LDFLAGS as before. Or perhaps --proper could enable --without-*, that seems like the cleanest solution. - Timothy --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18957/#review36642 --- On March 10, 2014, 10:36 a.m., Till Toenshoff wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18957/ --- (Updated March 10, 2014, 10:36 a.m.) Review request for mesos, Adam B, Benjamin Hindman, Ben Mahler, Niklas Nielsen, Timothy St. Clair, and Vinod Kone. Bugs: MESOS-1071 https://issues.apache.org/jira/browse/MESOS-1071 Repository: mesos-git Description --- Allows preventing the use of the currently bundled leveldb in favor of a preinstalled version. Diffs - 3rdparty/Makefile.am 8e1d915 configure.ac 390f11b src/Makefile.am 384b312 src/python/setup.py.in 02f00ef Diff: https://reviews.apache.org/r/18957/diff/ Testing --- ../configure make check OSX: brew installed leveldb comes with activated libsnappy ../configure --without-included-leveldb CPPFLAGS=-I/usr/local/include LDFLAGS=-L/usr/local/lib LIBS=-lleveldb -lsnappy make check linux: manually installed leveldb without libsnappy ../configure --without-included-leveldb CPPFLAGS=-I/usr/local/include LDFLAGS=-L/usr/local/lib LIBS=-lleveldb make check Thanks, Till Toenshoff
Re: Review Request 18957: Added --without-included-leveldb flag
On March 10, 2014, 3:10 p.m., Timothy St. Clair wrote: So IMHO, --without-XYZ will become an onerous pattern to follow when building all of Mesos against system dependencies. It might make more sense to create a --proper=yes flag, where it searches the system for the dependencies, and fails if it can not find them. Timothy St. Clair wrote: You can still leverage the CPPFLAGS LDFLAGS as before. Timothy St. Clair wrote: Or perhaps --proper could enable --without-*, that seems like the cleanest solution. I do not have a strong opinion on this particular detail. We might want to discuss this on the linked JIRA ticket ;) - Till --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18957/#review36642 --- On March 10, 2014, 10:36 a.m., Till Toenshoff wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18957/ --- (Updated March 10, 2014, 10:36 a.m.) Review request for mesos, Adam B, Benjamin Hindman, Ben Mahler, Niklas Nielsen, Timothy St. Clair, and Vinod Kone. Bugs: MESOS-1071 https://issues.apache.org/jira/browse/MESOS-1071 Repository: mesos-git Description --- Allows preventing the use of the currently bundled leveldb in favor of a preinstalled version. Diffs - 3rdparty/Makefile.am 8e1d915 configure.ac 390f11b src/Makefile.am 384b312 src/python/setup.py.in 02f00ef Diff: https://reviews.apache.org/r/18957/diff/ Testing --- ../configure make check OSX: brew installed leveldb comes with activated libsnappy ../configure --without-included-leveldb CPPFLAGS=-I/usr/local/include LDFLAGS=-L/usr/local/lib LIBS=-lleveldb -lsnappy make check linux: manually installed leveldb without libsnappy ../configure --without-included-leveldb CPPFLAGS=-I/usr/local/include LDFLAGS=-L/usr/local/lib LIBS=-lleveldb make check Thanks, Till Toenshoff
Re: Review Request 17567: Pluggable Containerizer
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/17567/#review36651 --- Bad patch! Reviews applied: [18595, 18594, 18311, 18403, 17567] Failed command: ./bootstrap Error: autoreconf: Entering directory `.' autoreconf: configure.ac: not using Gettext autoreconf: running: aclocal --warnings=all -I m4 autoreconf: configure.ac: tracing configure.ac:44: warning: back quotes and double quotes must not be escaped in: unrecognized option: $[1] configure.ac:44: Try \`$[0] --help' for more information. aclocal.m4:625: LT_OUTPUT is expanded from... configure.ac:44: the top level configure.ac:44: warning: back quotes and double quotes must not be escaped in: unrecognized argument: $[1] configure.ac:44: Try \`$[0] --help' for more information. aclocal.m4:625: LT_OUTPUT is expanded from... configure.ac:44: the top level configure.ac:260: warning: The macro `AC_LANG_SAVE' is obsolete. configure.ac:260: You should run autoupdate. ../../lib/autoconf/lang.m4:126: AC_LANG_SAVE is expanded from... m4/acx_pthread.m4:63: ACX_PTHREAD is expanded from... configure.ac:260: the top level configure.ac:260: warning: The macro `AC_LANG_C' is obsolete. configure.ac:260: You should run autoupdate. ../../lib/autoconf/c.m4:73: AC_LANG_C is expanded from... m4/acx_pthread.m4:63: ACX_PTHREAD is expanded from... configure.ac:260: the top level configure.ac:260: warning: The macro `AC_TRY_LINK' is obsolete. configure.ac:260: You should run autoupdate. ../../lib/autoconf/general.m4:2688: AC_TRY_LINK is expanded from... m4/acx_pthread.m4:63: ACX_PTHREAD is expanded from... configure.ac:260: the top level configure.ac:260: warning: The macro `AC_LANG_RESTORE' is obsolete. configure.ac:260: You should run autoupdate. ../../lib/autoconf/lang.m4:135: AC_LANG_RESTORE is expanded from... m4/acx_pthread.m4:63: ACX_PTHREAD is expanded from... configure.ac:260: the top level configure.ac:465: warning: The macro `AC_PYTHON_DEVEL' is obsolete. configure.ac:465: You should run autoupdate. m4/ax_python_devel.m4:72: AC_PYTHON_DEVEL is expanded from... configure.ac:465: the top level autoreconf: configure.ac: adding subdirectory 3rdparty/libprocess to autoreconf autoreconf: Entering directory `3rdparty/libprocess' configure.ac:28: warning: back quotes and double quotes must not be escaped in: unrecognized option: $[1] configure.ac:28: Try \`$[0] --help' for more information. aclocal.m4:625: LT_OUTPUT is expanded from... configure.ac:28: the top level configure.ac:28: warning: back quotes and double quotes must not be escaped in: unrecognized argument: $[1] configure.ac:28: Try \`$[0] --help' for more information. aclocal.m4:625: LT_OUTPUT is expanded from... configure.ac:28: the top level configure.ac:132: warning: The macro `AC_LANG_SAVE' is obsolete. configure.ac:132: You should run autoupdate. ../../lib/autoconf/lang.m4:126: AC_LANG_SAVE is expanded from... m4/acx_pthread.m4:63: ACX_PTHREAD is expanded from... configure.ac:132: the top level configure.ac:132: warning: The macro `AC_LANG_C' is obsolete. configure.ac:132: You should run autoupdate. ../../lib/autoconf/c.m4:73: AC_LANG_C is expanded from... m4/acx_pthread.m4:63: ACX_PTHREAD is expanded from... configure.ac:132: the top level configure.ac:132: warning: The macro `AC_TRY_LINK' is obsolete. configure.ac:132: You should run autoupdate. ../../lib/autoconf/general.m4:2688: AC_TRY_LINK is expanded from... m4/acx_pthread.m4:63: ACX_PTHREAD is expanded from... configure.ac:132: the top level configure.ac:132: warning: The macro `AC_LANG_RESTORE' is obsolete. configure.ac:132: You should run autoupdate. ../../lib/autoconf/lang.m4:135: AC_LANG_RESTORE is expanded from... m4/acx_pthread.m4:63: ACX_PTHREAD is expanded from... configure.ac:132: the top level autoreconf: configure.ac: adding subdirectory 3rdparty/stout to autoreconf autoreconf: Entering directory `3rdparty/stout' autoreconf: configure.ac: not using Libtool autoreconf: running: /usr/bin/autoconf --warnings=all autoreconf: configure.ac: not using Autoheader autoreconf: running: automake --add-missing --copy --no-force --warnings=all configure.ac:10: installing `./missing' autoreconf: Leaving directory `3rdparty/stout' autoreconf: running: libtoolize --install --copy libtoolize: putting auxiliary files in `.'. libtoolize: copying file `./config.guess' libtoolize: copying file `./config.sub' libtoolize: copying file `./ltmain.sh' libtoolize: putting macros in AC_CONFIG_MACRO_DIR, `m4'. libtoolize: copying file `m4/libtool.m4' libtoolize: You should add the contents of `m4/libtool.m4' to `aclocal.m4'. libtoolize: copying file `m4/ltoptions.m4' libtoolize: You should add the contents of `m4/ltoptions.m4' to `aclocal.m4'. libtoolize: copying file `m4/ltsugar.m4' libtoolize: You should add the contents of `m4/ltsugar.m4' to `aclocal.m4'. libtoolize: copying file `m4/ltversion.m4'
[jira] [Commented] (MESOS-1071) Enable building against installed third-party dependencies.
[ https://issues.apache.org/jira/browse/MESOS-1071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13925825#comment-13925825 ] Timothy St. Clair commented on MESOS-1071: -- How does this sound? Prefer system detection where possible, but fallback to bundled version if not found and --with-XYZ=path is not set. This would always check the system 1st, and fallback *only* where needed. e.g. --with-leveldb=some_prefixed_path 1.) if with_leveldb == some valid prefix path append -I -L flags 2.) AC_CHECK_HEADERS([leveldb/db.h], ['some positive test TBD'], [ if --with-leveldb FAIL else use bundled. ]) 3.) Adjust flags so all scripts use -lleveldb sans path insertion. Thoughts? Enable building against installed third-party dependencies. --- Key: MESOS-1071 URL: https://issues.apache.org/jira/browse/MESOS-1071 Project: Mesos Issue Type: Improvement Components: build Reporter: Benjamin Hindman Most of our third-party dependencies are included in the project and statically linked into our resulting binaries and libraries. We would like to enable building Mesos but using system installed dependencies instead. In certain circumstances this is more difficult because we've actually needed to patch these libraries (either for C++11 or to alter semantics). Rather than eliminating our internal copies of these third-party dependencies the first step should be to just enable using external (i.e., system installed) dependencies. We already do this for ZooKeeper by allowing people to use the --without-included-zookeeper flag during compilation. We should do this for other libraries as well. In fact, for the libraries that we have not patched (and even for some that we have patched) we should check to see if an appropriate system installed dependency exists and preferentially use that unless --with-included-dependency is explicitly used. Note that this issue represents a stepping stone to removing our third-party dependencies from our repository. -- This message was sent by Atlassian JIRA (v6.2#6252)
Re: Review Request 17567: Pluggable Containerizer
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/17567/ --- (Updated March 10, 2014, 4:10 p.m.) Review request for mesos, Adam B, Benjamin Hindman, Ben Mahler, Ian Downes, Niklas Nielsen, and Vinod Kone. Changes --- Now includes all needed files. Bugs: MESOS-816 https://issues.apache.org/jira/browse/MESOS-816 Repository: mesos-git Description --- This patch adds the so-called pluggable containerizer. This containerizer delegates all containerizer calls directly to an external, pluggable containerizer program (which can be specified on start-up). Few calls have internal fall-back implementations such as wait(), destroy() and usage(). The protocol for the interactions with the external program is as follows: COMMAND (ADDITIONAL-PARAMETERS) INPUT-PROTO RESULT-PROTO launch (ContainerID, --mesos-executor, path) TaskInfo PluggableStatus update (ContainerID) ResourceArray PluggableStatus usage (ContainerID) ResourceStatistics wait (ContainerID) PluggableTermination destroy (ContainerID) PluggableStatus When protocol buffers need to be provided, the Mesos side of the pluggable containerizer implementation will serialize the protos on stdin and vice-versa for reading protos on stdout as drafted in the above scheme. Diffs (updated) - configure.ac 390f11b include/mesos/mesos.proto 37f8a7f src/Makefile.am 384b312 src/examples/python/test-containerizer.in PRE-CREATION src/examples/python/test_containerizer.py PRE-CREATION src/slave/containerizer/containerizer.cpp d0a1023 src/slave/containerizer/pluggable_containerizer.hpp PRE-CREATION src/slave/containerizer/pluggable_containerizer.cpp PRE-CREATION src/slave/flags.hpp c9a627b src/tests/pluggable_containerizer_test.cpp PRE-CREATION Diff: https://reviews.apache.org/r/17567/diff/ Testing --- make check and functional testing. Thanks, Till Toenshoff
Re: Review Request 17567: Pluggable Containerizer
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/17567/#review36654 --- Bad patch! Reviews applied: [18595, 18594, 18311, 18403, 17567] Failed command: make -j3 check GTEST_FILTER='' /dev/null Error: ev.c:1531:31: warning: 'ev_default_loop_ptr' initialized and declared 'extern' [enabled by default] ev.c: In function 'evpipe_write': ev.c:2160:17: warning: ignoring return value of 'write', declared with attribute warn_unused_result [-Wunused-result] ev.c:2172:17: warning: ignoring return value of 'write', declared with attribute warn_unused_result [-Wunused-result] ev.c: In function 'pipecb': ev.c:2193:16: warning: ignoring return value of 'read', declared with attribute warn_unused_result [-Wunused-result] ev.c:2207:16: warning: ignoring return value of 'read', declared with attribute warn_unused_result [-Wunused-result] In file included from /usr/include/c++/4.6/ext/hash_set:61:0, from src/glog/stl_logging.h:54, from src/stl_logging_unittest.cc:34: /usr/include/c++/4.6/backward/backward_warning.h:33:2: warning: #warning This file includes at least one deprecated or antiquated header which may be removed without further notice at a future date. Please use a non-deprecated interface with equivalent functionality instead. For a listing of replacement headers and interfaces, consult the file backward_warning.h. To disable this warning use -Wno-deprecated. [-Wcpp] In file included from src/utilities.h:73:0, from src/googletest.h:38, from src/stl_logging_unittest.cc:48: src/base/mutex.h:137:0: warning: _XOPEN_SOURCE redefined [enabled by default] /usr/include/features.h:166:0: note: this is the location of the previous definition In file included from src/tests/process_tests.cpp:12:0: ./include/process/collect.hpp: In function 'process::Futureprocess::FutureT process::await(const process::FutureT, const Optionprocess::Timeout) [with T = int]': src/tests/process_tests.cpp:1085:57: instantiated from here ./include/process/collect.hpp:340:51: error: no matching function for call to 'bind(unresolved overloaded function type, const process::Futureint)' ./include/process/collect.hpp:340:51: note: candidates are: /usr/include/c++/4.6/tr1/functional:1372:5: note: templateclass _Result, class _Functor, class ... _ArgTypes std::tr1::_Bind_result_Result, typename std::tr1::_Maybe_wrap_member_pointer_Functor::type(_ArgTypes ...) std::tr1::bind(_Functor, _ArgTypes ...) /usr/include/c++/4.6/tr1/functional:1359:5: note: templateclass _Functor, class ... _ArgTypes std::tr1::_Bindtypename std::tr1::_Maybe_wrap_member_pointer_Functor::type(_ArgTypes ...) std::tr1::bind(_Functor, _ArgTypes ...) make[5]: *** [tests-process_tests.o] Error 1 make[5]: *** Waiting for unfinished jobs make[4]: *** [check-am] Error 2 make[3]: *** [check-recursive] Error 1 make[2]: *** [check-recursive] Error 1 make[1]: *** [check] Error 2 make: *** [check-recursive] Error 1 - Mesos ReviewBot On March 10, 2014, 4:10 p.m., Till Toenshoff wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/17567/ --- (Updated March 10, 2014, 4:10 p.m.) Review request for mesos, Adam B, Benjamin Hindman, Ben Mahler, Ian Downes, Niklas Nielsen, and Vinod Kone. Bugs: MESOS-816 https://issues.apache.org/jira/browse/MESOS-816 Repository: mesos-git Description --- This patch adds the so-called pluggable containerizer. This containerizer delegates all containerizer calls directly to an external, pluggable containerizer program (which can be specified on start-up). Few calls have internal fall-back implementations such as wait(), destroy() and usage(). The protocol for the interactions with the external program is as follows: COMMAND (ADDITIONAL-PARAMETERS) INPUT-PROTO RESULT-PROTO launch (ContainerID, --mesos-executor, path) TaskInfo PluggableStatus update (ContainerID) ResourceArray PluggableStatus usage (ContainerID) ResourceStatistics wait (ContainerID) PluggableTermination destroy (ContainerID) PluggableStatus When protocol buffers need to be provided, the Mesos side of the pluggable containerizer implementation will serialize the protos on stdin and vice-versa for reading protos on stdout as drafted in the above scheme. Diffs - configure.ac 390f11b include/mesos/mesos.proto 37f8a7f src/Makefile.am 384b312 src/examples/python/test-containerizer.in PRE-CREATION src/examples/python/test_containerizer.py PRE-CREATION src/slave/containerizer/containerizer.cpp d0a1023 src/slave/containerizer/pluggable_containerizer.hpp PRE-CREATION
Re: Review Request 18595: Added overloaded killtree for process trees.
On March 6, 2014, 7:35 p.m., Ben Mahler wrote: 3rdparty/libprocess/3rdparty/stout/include/stout/os/killtree.hpp, line 51 https://reviews.apache.org/r/18595/diff/1/?file=506546#file506546line51 This seems a little strange, because we'll be trying to signal multiple times: root - signaled 1 time child - signaled 2 times grandchild - signaled 3 times ... Nth level pid - signaled N times Also, how will this be used? Let's say you use this for a 5 minute signal escalation on the tree originally returned by killtree(pid_t, int, bool, bool). 5 minutes later, how do you know the pids are still valid and you don't accidentally kill unrelated processes? Ben Mahler wrote: Actually it's probably more productive to review r/18597 first. :) Niklas Nielsen wrote: This RR was mostly to address killing orphan processes or in other words, have a way to kill the processes that belong to a process tree root which might shutdown before its children. With the current killtree implementation, it seems to me that this would fail (the pid can't be resolved to a process and returns). Till mentioned that we might end up killing new processes same pids, which is a problem. I am not sure how we can address both; I'll take a second stab at it tomorrow :) Till Toenshoff wrote: Yep, I am a bit concerned about pid-reuse. As all our options are poll-based (this approach in connection with RR18594 and a delayed-continuation) as well as process::reaper, this approach seemed a bit safer in regard to the polling period. process::reaper uses 1 second whereas the use of this can be freely adjusted to smaller intervals (see implementation of PluggableContainerizer's destroy escalation). In the end however, all solutions we have so far appear to carry the risk of killing unrelated (reused) pids. For the signal repetition on childs within the same pid session or group, that indeed might be a great point and me might want to address it. Till Toenshoff wrote: Just found some well-founded information on pid-reuse on linux: http://stackoverflow.com/questions/11323410/linux-pid-recycling Thanks for the pointer on pid-reuse! I think a general issue is whether it is safe (or whether we should) interfere with further shutdown of the process root's children. AFAIK We can't safely reap them as processes deeper in the tree might reap and/or ignore SIGCHLD? Another way would be to signal the process groups after the delay (if the root is no longer around). However, group id reuse will be a similar problem. It would significantly simplify escalation, if we sent SIGKILL to the process root (and its children) after a delay (potentially leaving behind orphan processes if the root exited before children did). - Niklas --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18595/#review36486 --- On Feb. 27, 2014, 4:54 p.m., Niklas Nielsen wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18595/ --- (Updated Feb. 27, 2014, 4:54 p.m.) Review request for mesos and Ben Mahler. Repository: mesos-git Description --- New killtree(ProcessTree tree, int signal) traverse process tree and sends a signal to all pids. This is done regardless of presence and state of process. Patch is used by up coming signal escalation. Diffs - 3rdparty/libprocess/3rdparty/stout/include/stout/os/killtree.hpp 1f45897 Diff: https://reviews.apache.org/r/18595/diff/ Testing --- Functional testing with signal escalation code and make check. Thanks, Niklas Nielsen
Re: Review Request 18952: Refactored Promise to be thread-safe.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18952/#review36656 --- 3rdparty/libprocess/include/process/future.hpp https://reviews.apache.org/r/18952/#comment67690 While we are writing to associated under lock, we aren't reading under lock. Should lock be a rwlock? or should this be read/written under RCU semantics? Apologies if I'm missing something about how Promise is implemented that makes this unnecessary. - Dominic Hamon On March 10, 2014, 1:03 a.m., Benjamin Hindman wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18952/ --- (Updated March 10, 2014, 1:03 a.m.) Review request for mesos, Ben Mahler and Jie Yu. Repository: mesos-git Description --- In particular, doing associations is strictly safer now. Diffs - 3rdparty/libprocess/include/process/future.hpp 27b0970bf1d1ae1b977ddfc2de5ee858f1031bf5 Diff: https://reviews.apache.org/r/18952/diff/ Testing --- make check Thanks, Benjamin Hindman
Re: Review Request 18954: Added Future::after.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18954/#review36659 --- 3rdparty/libprocess/include/process/future.hpp https://reviews.apache.org/r/18954/#comment67693 i know it doesn't match the include order, but it might be more maintainable as we move towards C++11 to put this as the #else stanza above. 3rdparty/libprocess/include/process/future.hpp https://reviews.apache.org/r/18954/#comment67694 is the callback responsible for checking the future then? would it be better to avoid calling the callback if the future has been discarded to avoid boilerplate/mistakes? 3rdparty/libprocess/include/process/future.hpp https://reviews.apache.org/r/18954/#comment67695 you could use lambda::function in both the C++11 and boost cases to make future cleanup easier. 3rdparty/libprocess/src/tests/process_tests.cpp https://reviews.apache.org/r/18954/#comment67698 maybe this should be a separate test, to avoid having to do cleanup in between and accidentally introducing dependencies. - Dominic Hamon On March 10, 2014, 1:03 a.m., Benjamin Hindman wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18954/ --- (Updated March 10, 2014, 1:03 a.m.) Review request for mesos, Ben Mahler, Jie Yu, and Vinod Kone. Repository: mesos-git Description --- This is to avoid a standard pattern seen with doing a 'delay' after some asynchronous function call has been made and we're waiting on it's completion via Future::then, onAny, etc. This allows one to do: someAsynchronousFunction() .then(defer(self(), Self::_continuation1)) .then(defer(self(), Self::_continuation2)) .after(Seconds(30), defer(self(), Self::_timeout, lambda::_1)); Diffs - 3rdparty/libprocess/include/process/future.hpp 27b0970bf1d1ae1b977ddfc2de5ee858f1031bf5 3rdparty/libprocess/include/process/http.hpp 7f549ba3476ecf5dec0db21d57ee58bcd73d5996 3rdparty/libprocess/src/tests/process_tests.cpp e899aed7dbe6e8645484fe0ba69521e9fb0fdad2 Diff: https://reviews.apache.org/r/18954/diff/ Testing --- make check Thanks, Benjamin Hindman
[jira] [Commented] (MESOS-1071) Enable building against installed third-party dependencies.
[ https://issues.apache.org/jira/browse/MESOS-1071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13925899#comment-13925899 ] Vinod Kone commented on MESOS-1071: --- I would prefer to also have an option to always use bundled dependencies irrespective of system installed ones. In fact this could be the default so as to not break users that are already expect this behavior. After a deprecation period we could change the default to prefer system installed dependencies. Enable building against installed third-party dependencies. --- Key: MESOS-1071 URL: https://issues.apache.org/jira/browse/MESOS-1071 Project: Mesos Issue Type: Improvement Components: build Reporter: Benjamin Hindman Most of our third-party dependencies are included in the project and statically linked into our resulting binaries and libraries. We would like to enable building Mesos but using system installed dependencies instead. In certain circumstances this is more difficult because we've actually needed to patch these libraries (either for C++11 or to alter semantics). Rather than eliminating our internal copies of these third-party dependencies the first step should be to just enable using external (i.e., system installed) dependencies. We already do this for ZooKeeper by allowing people to use the --without-included-zookeeper flag during compilation. We should do this for other libraries as well. In fact, for the libraries that we have not patched (and even for some that we have patched) we should check to see if an appropriate system installed dependency exists and preferentially use that unless --with-included-dependency is explicitly used. Note that this issue represents a stepping stone to removing our third-party dependencies from our repository. -- This message was sent by Atlassian JIRA (v6.2#6252)
Re: Review Request 18973: Upgraded to gmock 1.7 from gmock 1.6
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18973/#review3 --- configure.ac https://reviews.apache.org/r/18973/#comment67709 How does this work when users disable cxx11? - Vinod Kone On March 10, 2014, 5:23 p.m., Dominic Hamon wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18973/ --- (Updated March 10, 2014, 5:23 p.m.) Review request for mesos, Benjamin Hindman and Vinod Kone. Bugs: MESOS-1044 and MESOS-750 https://issues.apache.org/jira/browse/MESOS-1044 https://issues.apache.org/jira/browse/MESOS-750 Repository: mesos-git Description --- see summary. Diffs - 3rdparty/libprocess/3rdparty/gmock-1.6.0.tar.gz d45d9894b0214f5f02a88f6da5c258327110efd8 3rdparty/libprocess/3rdparty/gmock-1.7.0.tar.gz PRE-CREATION 3rdparty/libprocess/3rdparty/versions.am 97727537778511ca5a10be4f3c25cd21d919 3rdparty/libprocess/configure.ac 88aeaada170fbc846794e00047cd5aa9835f9fcf LICENSE b45416f1cc1bb98e8edb1ebdc078b05e6c514439 configure.ac 91ddf57b3836bcdb31e5f7cf1e64cf8d14743374 Diff: https://reviews.apache.org/r/18973/diff/ Testing --- clean build make check Thanks, Dominic Hamon
Re: Review Request 18973: Upgraded to gmock 1.7 from gmock 1.6
On March 10, 2014, 10:33 a.m., Vinod Kone wrote: configure.ac, line 628 https://reviews.apache.org/r/18973/diff/1/?file=515205#file515205line628 How does this work when users disable cxx11? The same way it did before - this flag enables GTEST to be more intelligent about trying to use tuple/tr1::tuple. When C++11 is disabled, it checks the libc version to determine what should be available. It's much more sophisticated in this version. I will run tests under more compiler versions to make sure. - Dominic --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18973/#review3 --- On March 10, 2014, 10:23 a.m., Dominic Hamon wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18973/ --- (Updated March 10, 2014, 10:23 a.m.) Review request for mesos, Benjamin Hindman and Vinod Kone. Bugs: MESOS-1044 and MESOS-750 https://issues.apache.org/jira/browse/MESOS-1044 https://issues.apache.org/jira/browse/MESOS-750 Repository: mesos-git Description --- see summary. Diffs - 3rdparty/libprocess/3rdparty/gmock-1.6.0.tar.gz d45d9894b0214f5f02a88f6da5c258327110efd8 3rdparty/libprocess/3rdparty/gmock-1.7.0.tar.gz PRE-CREATION 3rdparty/libprocess/3rdparty/versions.am 97727537778511ca5a10be4f3c25cd21d919 3rdparty/libprocess/configure.ac 88aeaada170fbc846794e00047cd5aa9835f9fcf LICENSE b45416f1cc1bb98e8edb1ebdc078b05e6c514439 configure.ac 91ddf57b3836bcdb31e5f7cf1e64cf8d14743374 Diff: https://reviews.apache.org/r/18973/diff/ Testing --- clean build make check Thanks, Dominic Hamon
[jira] [Assigned] (MESOS-550) Python compilation fails when trying to compile without included zookeeper libs
[ https://issues.apache.org/jira/browse/MESOS-550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Till Toenshoff reassigned MESOS-550: Assignee: Till Toenshoff Python compilation fails when trying to compile without included zookeeper libs --- Key: MESOS-550 URL: https://issues.apache.org/jira/browse/MESOS-550 Project: Mesos Issue Type: Bug Components: python api Affects Versions: 0.12.0 Reporter: Julien Eid Assignee: Till Toenshoff Hey, I'm trying to compile Mesos against a separate Zookeeper install for packaging. The python setup.py only links against the .libs folder and doesn't check LDFLAGS for a different location for the lib. Relevant lines /src/python/setup.py: os.path.join(abs_top_builddir, zookeeper, '.libs', 'libzookeeper_mt.a') -- This message was sent by Atlassian JIRA (v6.2#6252)
Review Request 18974: Factor out method to build command for fetcher.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18974/ --- Review request for mesos, Ian Downes and Vinod Kone. Bugs: MESOS-1050 and MESOS-1063 https://issues.apache.org/jira/browse/MESOS-1050 https://issues.apache.org/jira/browse/MESOS-1063 Repository: mesos-git Description --- see summary Diffs - src/Makefile.am 384b3122b61294401ba4a894c06e985d9fc2fb1e src/slave/containerizer/mesos_containerizer.cpp 9bf9829fb38f17d9a2a0d3ab33d45d34cd5d3ea5 src/tests/containerizer_tests.cpp PRE-CREATION Diff: https://reviews.apache.org/r/18974/diff/ Testing --- make check Thanks, Dominic Hamon
Re: building meso-0.17.0 failing on maverick
On Sat, Mar 8, 2014 at 6:23 PM, haja gecko haja.ge...@gmail.com wrote: gcc-4.8: error: unrecognized command line option '-Wshorten-64-to-32' This doesn't seem to be a valid gcc option. AFAICT, this is not being set in our Makefiles. Do you have some alias that is setting this option? ➜ mesos git:(master) gcc-4.8 -Wshorten-64-to-32 gcc-4.8: error: unrecognized command line option '-Wshorten-64-to-32' gcc-4.8: fatal error: no input files compilation terminated. ➜ mesos git:(master) ack shorten-64-to-32 * ➜
Re: Review Request 18954: Added Future::after.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18954/ --- (Updated March 10, 2014, 5:59 p.m.) Review request for mesos, Ben Mahler, Jie Yu, and Vinod Kone. Repository: mesos-git Description --- This is to avoid a standard pattern seen with doing a 'delay' after some asynchronous function call has been made and we're waiting on it's completion via Future::then, onAny, etc. This allows one to do: someAsynchronousFunction() .then(defer(self(), Self::_continuation1)) .then(defer(self(), Self::_continuation2)) .after(Seconds(30), defer(self(), Self::_timeout, lambda::_1)); Diffs (updated) - 3rdparty/libprocess/include/process/future.hpp 27b0970bf1d1ae1b977ddfc2de5ee858f1031bf5 3rdparty/libprocess/include/process/http.hpp 7f549ba3476ecf5dec0db21d57ee58bcd73d5996 3rdparty/libprocess/src/tests/process_tests.cpp e899aed7dbe6e8645484fe0ba69521e9fb0fdad2 Diff: https://reviews.apache.org/r/18954/diff/ Testing --- make check Thanks, Benjamin Hindman
Re: Review Request 18381: Added authentication support for slaves.
On March 6, 2014, 4:07 p.m., Vinod Kone wrote: src/master/flags.hpp, lines 109-119 https://reviews.apache.org/r/18381/diff/2/?file=511055#file511055line109 have defaults as false? simply do a validation in master that they don't conflict. Got rid of the Options and put --authenticate as it was, but renamed Flags::authenticate to Flags::authenticate_frameworks for clarity. Added a TODO for future deprecation and transition to --authenticate_frameworks. On March 6, 2014, 4:07 p.m., Vinod Kone wrote: src/master/master.cpp, lines 293-305 https://reviews.apache.org/r/18381/diff/2/?file=511057#file511057line293 Hmm. This is getting complicated. Can you actually not do the flag deprecation stuff in this review. Lets do that in a subsequent review. IOW, keep authenticate flag as is and simply add authenticate_slaves flag. You can add a TODO for the rename. Understood. Left --authenticate as it was, but renamed Flags::authenticate to Flags::authenticate_frameworks for clarity. On March 6, 2014, 4:07 p.m., Vinod Kone wrote: src/master/master.cpp, line 297 https://reviews.apache.org/r/18381/diff/2/?file=511057#file511057line297 s/frameworks but not slaves/ ? Master only allowing authenticated frameworks but not slaves to register sounds awkward to me, and inaccurate since any slave could register, regardless of whether it wants to authenticate. So I made frameworks and slaves separate log lines, e.g.: Master only allowing authenticated frameworks to register Master allowing unauthenticated slaves to register On March 6, 2014, 4:07 p.m., Vinod Kone wrote: src/sasl/authenticatee.hpp, lines 41-66 https://reviews.apache.org/r/18381/diff/2/?file=511059#file511059line41 Why this refactoring? When I tried to include authenticatee.hpp in slave.cpp, I got multiple definition linker errors like: ./.libs/libmesos_no_3rdparty.a(libmesos_no_3rdparty_la-slave.o): In function `mesos::internal::sasl::Authenticatee::~Authenticatee()': /home/me/dev/mesos/build/src/../../src/sasl/authenticatee.hpp:390: multiple definition of `mesos::internal::sasl::Authenticatee::~Authenticatee()' ./.libs/libmesos_no_3rdparty.a(libmesos_no_3rdparty_la-sched.o):/home/me/dev/mesos/build/src/../../src/sasl/authenticatee.hpp:390: first defined here There are a few ways to fix this: 1) Move the implementations of Authenticatee's methods into the class definition (as I had done). 2) Move the implementations into an authenticatee.cpp. 3) Mark these implementation methods as 'inline'. I have now marked the implementations as inline, which is much less disruptive. On March 6, 2014, 4:07 p.m., Vinod Kone wrote: src/master/master.cpp, lines 2563-2604 https://reviews.apache.org/r/18381/diff/2/?file=511057#file511057line2563 There's a lot of code duplication here. I'm not convinced that differentiating framework/slave authentication has any benefits. Either way we need to figure out to factor out some of this common code. After merging *.frameworks/slaves back into authenticating/authenticated/authenticators, most of the duplication was removed. There is now just the single authenticate() and _authenticate, as before. The only distinct code for frameworks vs. slaves is looping through master.frameworks and master.slaves to call deactivate(Framework) or deactivate(Slave) on each, plus a little logging. If we wanted to iterate through both frameworks and slaves to look for a matching pid to deactivate, we wouldn't even need the authenticatee type in AuthenticateMessage. However, I feel that it is important for the master to know what it is trying to authenticate the pid as, especially as we get to implementing role authorization. The other alternative is that we could have a separate AuthenticateSlaveMessage and leave AuthenticateMessage for frameworks; then it would be easier to add new fields (FrameworkID/SlaveID?) that might not apply to all kinds of AuthenticateMessages. Thoughts? On March 6, 2014, 4:07 p.m., Vinod Kone wrote: src/slave/slave.cpp, lines 86-116 https://reviews.apache.org/r/18381/diff/2/?file=511063#file511063line86 Both master and slave use this. We should really refactor this into a common function that both can use. We could probably stick it in sasl/common.hpp? Made a first pass at refactoring this. There's still the difference that master expects multiple credential lines whereas slave only expects a single credential line, so I have separate readCredential/readCredentials functions. That combined with the different errors/logging makes the current sasl/common.hpp not as neat as I'd like it to be. Ideas for improvement? - Adam --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18381/#review36419 --- On March
Re: Review Request 18973: Upgraded to gmock 1.7 from gmock 1.6
On March 10, 2014, 10:33 a.m., Vinod Kone wrote: configure.ac, line 628 https://reviews.apache.org/r/18973/diff/1/?file=515205#file515205line628 How does this work when users disable cxx11? Dominic Hamon wrote: The same way it did before - this flag enables GTEST to be more intelligent about trying to use tuple/tr1::tuple. When C++11 is disabled, it checks the libc version to determine what should be available. It's much more sophisticated in this version. I will run tests under more compiler versions to make sure. This fails to build under g++4.4 _with_ c++11. https://groups.google.com/d/topic/googlemock/XvPDUENM5JM/discussion So we need g++ 4.7 to use gmock 1.7 in c++11 which is a shame as it massively simplifies the tuple issue. Discarding this issue. - Dominic --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18973/#review3 --- On March 10, 2014, 10:23 a.m., Dominic Hamon wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18973/ --- (Updated March 10, 2014, 10:23 a.m.) Review request for mesos, Benjamin Hindman and Vinod Kone. Bugs: MESOS-1044 and MESOS-750 https://issues.apache.org/jira/browse/MESOS-1044 https://issues.apache.org/jira/browse/MESOS-750 Repository: mesos-git Description --- see summary. Diffs - 3rdparty/libprocess/3rdparty/gmock-1.6.0.tar.gz d45d9894b0214f5f02a88f6da5c258327110efd8 3rdparty/libprocess/3rdparty/gmock-1.7.0.tar.gz PRE-CREATION 3rdparty/libprocess/3rdparty/versions.am 97727537778511ca5a10be4f3c25cd21d919 3rdparty/libprocess/configure.ac 88aeaada170fbc846794e00047cd5aa9835f9fcf LICENSE b45416f1cc1bb98e8edb1ebdc078b05e6c514439 configure.ac 91ddf57b3836bcdb31e5f7cf1e64cf8d14743374 Diff: https://reviews.apache.org/r/18973/diff/ Testing --- clean build make check Thanks, Dominic Hamon
Re: Review Request 18974: Factor out method to build command for fetcher.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18974/#review36669 --- src/slave/containerizer/mesos_containerizer.cpp https://reviews.apache.org/r/18974/#comment67713 new line. src/slave/containerizer/mesos_containerizer.cpp https://reviews.apache.org/r/18974/#comment67715 What about checking this for emptyness too? Actually, instead of checking for emptyness we should make these flags optional (as the TODO in slave/flags.hpp suggests) and just check isSome() here. Better to do this fix in a different review. src/slave/containerizer/mesos_containerizer.cpp https://reviews.apache.org/r/18974/#comment67727 s/buildCommand/setenv/ ? src/tests/containerizer_tests.cpp https://reviews.apache.org/r/18974/#comment67722 // Forward declaration. src/tests/containerizer_tests.cpp https://reviews.apache.org/r/18974/#comment67723 s/std::/string/string/ src/tests/containerizer_tests.cpp https://reviews.apache.org/r/18974/#comment67724 ditto. src/tests/containerizer_tests.cpp https://reviews.apache.org/r/18974/#comment67725 ditto. src/tests/containerizer_tests.cpp https://reviews.apache.org/r/18974/#comment67717 new line. src/tests/containerizer_tests.cpp https://reviews.apache.org/r/18974/#comment67718 new line. src/tests/containerizer_tests.cpp https://reviews.apache.org/r/18974/#comment67721 ws. src/tests/containerizer_tests.cpp https://reviews.apache.org/r/18974/#comment67720 new line. src/tests/containerizer_tests.cpp https://reviews.apache.org/r/18974/#comment67719 ws. - Vinod Kone On March 10, 2014, 5:41 p.m., Dominic Hamon wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18974/ --- (Updated March 10, 2014, 5:41 p.m.) Review request for mesos, Ian Downes and Vinod Kone. Bugs: MESOS-1050 and MESOS-1063 https://issues.apache.org/jira/browse/MESOS-1050 https://issues.apache.org/jira/browse/MESOS-1063 Repository: mesos-git Description --- see summary Diffs - src/Makefile.am 384b3122b61294401ba4a894c06e985d9fc2fb1e src/slave/containerizer/mesos_containerizer.cpp 9bf9829fb38f17d9a2a0d3ab33d45d34cd5d3ea5 src/tests/containerizer_tests.cpp PRE-CREATION Diff: https://reviews.apache.org/r/18974/diff/ Testing --- make check Thanks, Dominic Hamon
[jira] [Commented] (MESOS-1076) Add namespacing to cgroups to enforce the expected structure in slave/containerizer/isolators/cgroups/cpushare.cpp
[ https://issues.apache.org/jira/browse/MESOS-1076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13925982#comment-13925982 ] Benjamin Mahler commented on MESOS-1076: That sounds about right, when you're ready with some changes you can send a review on ReviewBoard. That will help generate discussion around your code! Add namespacing to cgroups to enforce the expected structure in slave/containerizer/isolators/cgroups/cpushare.cpp Key: MESOS-1076 URL: https://issues.apache.org/jira/browse/MESOS-1076 Project: Mesos Issue Type: Improvement Components: general, slave Reporter: Archana kumari Original Estimate: 168h Remaining Estimate: 168h The point of the issue is to provide structural information from the cgroups API -- This message was sent by Atlassian JIRA (v6.2#6252)
Re: Review Request 18954: Added Future::after.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18954/ --- (Updated March 10, 2014, 6:13 p.m.) Review request for mesos, Ben Mahler, Jie Yu, and Vinod Kone. Repository: mesos-git Description --- This is to avoid a standard pattern seen with doing a 'delay' after some asynchronous function call has been made and we're waiting on it's completion via Future::then, onAny, etc. This allows one to do: someAsynchronousFunction() .then(defer(self(), Self::_continuation1)) .then(defer(self(), Self::_continuation2)) .after(Seconds(30), defer(self(), Self::_timeout, lambda::_1)); Diffs - 3rdparty/libprocess/include/process/future.hpp 27b0970bf1d1ae1b977ddfc2de5ee858f1031bf5 3rdparty/libprocess/include/process/http.hpp 7f549ba3476ecf5dec0db21d57ee58bcd73d5996 3rdparty/libprocess/src/tests/process_tests.cpp e899aed7dbe6e8645484fe0ba69521e9fb0fdad2 Diff: https://reviews.apache.org/r/18954/diff/ Testing --- make check Thanks, Benjamin Hindman
Re: Review Request 18381: Added authentication support for slaves.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18381/ --- (Updated March 10, 2014, 11:19 a.m.) Review request for mesos and Vinod Kone. Changes --- Updated in response to more of Vinod's feedback. + Removed --authenticate_frameworks in favor of keeping --authenticate and adding a TODO. + Merged frameworks/slaves into common authenticating/authenticated/authenticators collections, removed duplicate code. + Moved credentials file reading/parsing into sasl/common.hpp. + Fixed authenticatee.hpp refactoring with 'inline'. Remaining issues: - Move deactivate(Slave) changes into a new review; make this one dependent on that. - File new JIRAs for doxygen and new authentication bug. Bugs: MESOS-804 https://issues.apache.org/jira/browse/MESOS-804 Repository: mesos-git Description --- Added authentication support for slaves. Fixes MESOS-804. Open Issues: - Should AuthenticateMessage be replaced with AuthenticateFrameworkMessage, or specify an Authenticatee type as coded here? - removeSlave vs. deactivate(Slave): Some uses of removeSlave might benefit from just deactivating if checkpointing is enabled. - We currently deactivate a registered slave/framework when a new authenticate message comes in, even if the new authentication message is a failure/fake. Will file a new JIRA for this security hole. - When multiple entries for the same principal exist in the credentials file, only the last entry is used. Acceptable behavior, but shouldn't this be documented? Diffs (updated) - src/master/flags.hpp 159b2de src/master/master.hpp 49a3e15 src/master/master.cpp f7ba9aa src/messages/messages.proto c26a3d0 src/sasl/authenticatee.hpp 42a4eba src/sasl/common.hpp PRE-CREATION src/sched/sched.cpp 00f6307 src/slave/flags.hpp e4d98a5 src/slave/slave.hpp 01b80df src/slave/slave.cpp b350df4 src/tests/authentication_tests.cpp 127c5e6 src/tests/cluster.hpp d1bf680 src/tests/mesos.cpp 96adeac src/tests/sasl_tests.cpp 945426d src/tests/slave_recovery_tests.cpp 40a9599 Diff: https://reviews.apache.org/r/18381/diff/ Testing --- make check; manually tested flatfile slave authentication success/failure. Added new slave authentication unit tests in authentication_tests.cpp. Thanks, Adam B
Re: Review Request 18370: Refactor Cluster::Master::start methods
On March 6, 2014, 7:52 p.m., Benjamin Hindman wrote: src/tests/cluster.hpp, line 108 https://reviews.apache.org/r/18370/diff/5/?file=507531#file507531line108 I'm late to this party, but why did we change this to be called MasterInfo? This clashes with another type we have called MasterInfo and upon a cursory look seems unnecessary. Moreover if we needed this change we didn't change the Slave struct below to be called SlaveInfo for consistency. Vinod Kone wrote: Charlie can you follow up on benh's comment by sending a new patch? You can resolve the above issue once you do so. Thanks. Charlie Carson wrote: Hey Ben - first I'm agreeable and I'm happy to make the corresponding change to Slave, but I think we need to decide on what the blessed change IS. I'll give you the history --begin history I sent the review, w/ no name change and it confused Dominic b/c we have a Master class which REALLY represents the resources allocated by the Start method (and therefore that need to be released by the Stop method). It's confusing, b/c you have Cluster::Masters::Master which itself has a member which is of type mesos::internal::Master. so, in the same methods, you have two different Master types and in fact assign one to a member of the other. anyways, I explained to Dominic what was going on and suggested naming it MasterDependents since that's more accurately what it is. I didn't love that name, but no one suggested a better one, at the time. Vinod saw that patch and suggested MasterInfo. -end of history 1. agreed that whatever we do should be consistent between Master Slave dependents objects 2. agreed that MasterInfo isn't really descriptive 3. personally, I prefer anything besides Master (or Slave), b/c Master already is a well-known concept, and it's being used SxS in the same method. 4. from a design pov, I'm inclined to say that this shouldn't even be a struct. Ideally, it's a simple list of things to cleanup. I'll need to verify that is safe to do (ordering) and I'm not sure how much boiler-plating it will require. IOW, if all these guys inherited from IDeleteMe, then I'd make a listIDeleteMe* and as we allocated things we'd push them onto the list and delete them in reverse order. since there's no common super class we'll need some template magic (at a minimum boost::variant with a visitor to do the delete). It's possible that boost has something better I will take a look. so, possible fixes in order of my preference: 1) switch to list of things to delete (will send a code review unless someone says don't bother) 2) change MasterInfo to better represent Resources Allocated by Start that need to be deleted by Stop (again, a name suggestion is very welcome here :-) ). whatever we pick, make Slave match. 3) leave MasterInfo and change Slave - SlaveInfo 4) go back to Master Slave (and document at the declaration what it's for and at the use points that it's not actual master) Make sense? again, I'm fine to do any of the above (and in fact, at this point I've done 4, 2, 3 :-) ) I just don't want a concrete decision. if you feel strongly let me know and that's what I'll do. Charlie Carson wrote: I just DO want a concrete decision :-) Benjamin Hindman wrote: Awesome, in the interest of not adding too much more here, I think the name Dependents really wasn't that bad! It's explicitly descriptive to say the least! The reason a struct existed was because originally we wanted to be able to expose different components of a single started master (or slave), so we would keep everything around. Eventually this became just the list of things to cleanup. I think in the future with C++11 we'll be able to do the deleting even easier than making things inherit from an IDeleteMe interface, so I'd prefer not to do all that for now. Hey Charlie. Mind opening a new review for the changes since this is already submitted? Thanks. - Vinod --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18370/#review36415 --- On March 1, 2014, 1:15 a.m., Charlie Carson wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18370/ --- (Updated March 1, 2014, 1:15 a.m.) Review request for mesos and Vinod Kone. Bugs: MESOS-1024 https://issues.apache.org/jira/browse/MESOS-1024 Repository: mesos-git Description --- Refactor Cluster::Master::start methods There are currently two overloads
Re: Review Request 18295: Removed unnecessary includes and inline definitions.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18295/ --- (Updated March 10, 2014, 11:27 a.m.) Review request for mesos, Benjamin Hindman, Ben Mahler, and Vinod Kone. Changes --- Fixed header include order and renamed date_utils to date. Bugs: MESOS-1022 https://issues.apache.org/jira/browse/MESOS-1022 Repository: mesos-git Description --- See summary. Diffs (updated) - include/mesos/executor.hpp 7bc8ecac3f13431e6f7dd45deb06fecf6e0f5d8a include/mesos/resources.hpp 59f495cdb2a2d55a6c436767789ee579cd5c2f97 include/mesos/scheduler.hpp 85db1117122441b5d9fb013380ea37aa0acc84ea include/mesos/values.hpp 64c138ca6c23ec450e42e3eecf3276e6fc305644 src/Makefile.am 384b3122b61294401ba4a894c06e985d9fc2fb1e src/common/attributes.hpp a08cf188a6d9c27386fd10440d8944f0d7b6fa08 src/common/attributes.cpp 851c92a88804370313bf052c85f99d483fd67416 src/common/build.hpp 115e9cc3ec1e28d4622c7884bcbac904b24b95dd src/common/build.cpp eed08eda4a0db572992187347a2199054ade26d2 src/common/date_utils.hpp 085c1ce685316b3b96b7564c3613bf775a485054 src/common/date_utils.cpp 22c7dacb0f5a265fb595cf8cd0486530d8c28d5c src/common/http.hpp 717bbe99f2ec51a2d1b90730eb19cd54cc82b897 src/common/lock.cpp 11c8e8c50d806271c36c2ec20633acf06447c37f src/common/protobuf_utils.hpp 0f653414bc1bb2b632ec8cd9c8bd7202a53d42e1 src/common/protobuf_utils.cpp PRE-CREATION src/common/resources.cpp 61c5bda9ed7718e87c405822cf3de4795c51f38f src/common/thread.hpp 7e487241e419adb6a64109a949ee80a965771b25 src/common/thread.cpp PRE-CREATION src/common/type_utils.hpp 784a808a5e0b78dab24a4806d6f1f9491a2d6d44 src/common/type_utils.cpp PRE-CREATION src/common/values.cpp 15583fde45946cfd860d13297a03a927f0523115 src/master/http.cpp a019f15615d028f3d3628b2611709fc8a3a81bf8 src/master/master.hpp 49a3e151d53b786027184c14b96371016a926497 src/master/master.cpp 2a403336791737d874c26928c9fffea08ac246d8 src/master/repairer.cpp 151b4ed7ac0b8dacd175b97d372c1f867cd280f6 src/slave/constants.hpp d237383ff8d00f4731b689a2a1592fdd36f09a4d src/slave/containerizer/cgroups_launcher.cpp 39f0e4c3baaed3403da160ba63dada4a53d9af09 src/slave/containerizer/isolator.hpp fc6c9ab10ff535ddd8390aeacf2151ac2d5174a6 src/slave/containerizer/isolators/posix.hpp 7fbc6ddd9aa5518870bf938c6ead5eb72d3ec524 src/slave/containerizer/launcher.cpp fb0c461a00ec21e52e2f008b81cb6d545f537c99 src/slave/flags.hpp e4d98a53cbfb7f9ca828f17e82d492274cb9969d src/slave/gc.hpp 7b6fb8365b2e793c019907ecdfbce7413d72f8b4 src/slave/gc.cpp 3720255704171eaa054c96a790ecd5e6e5304b33 src/slave/http.cpp 594032da1b2edd47961cd8201acd093b22fa30ae src/slave/main.cpp a498a6ae6a79c7155c07a5d6dc2d6c9dc8ae060f src/slave/monitor.hpp c042bc1af31aedf7d2991741d3f8cb70ef353b40 src/slave/monitor.cpp 1c02986e22bc1dcbc2f07de546bf865d34c41c89 src/slave/paths.hpp 41bb73d6359ee6122ccc2170512311e1ad4fcc4c src/slave/paths.cpp PRE-CREATION src/slave/slave.hpp 01b80dfc1dd55edd6d2f722ff1d4f1bf4d96f28e src/slave/slave.cpp 6abb95df4c2339c1e961d2a4ee75d3512c3d23a8 src/slave/state.hpp 22f569def3c000856190deff4d55e636afb9e00e src/slave/state.cpp 21d1fb7fbe695bfc3f80ce17c413261f98e6e0b4 src/slave/status_update_manager.hpp 24e3882ad1969c20a64daf90e408618c310e9a81 src/slave/status_update_manager.cpp 9db53e8b2a6440b7eebe3bc61912b170bde7a473 src/tests/monitor_tests.cpp 4b950e14bd94cdfa21212268b56bebdc1200078d src/tests/slave_recovery_tests.cpp 40a9599787918b78790462e81729ec7ac2395509 Diff: https://reviews.apache.org/r/18295/diff/ Testing --- No functional change. Ran clean build and make check. Timed build: before change - 26m30s after change - 19m33s diff stats: 44 files changed, 1067 insertions(+), 1074 deletions(-) Thanks, Dominic Hamon
[jira] [Created] (MESOS-1077) Registrar tests are flaky.
Benjamin Mahler created MESOS-1077:
--
Summary: Registrar tests are flaky.
Key: MESOS-1077
URL: https://issues.apache.org/jira/browse/MESOS-1077
Project: Mesos
Issue Type: Bug
Reporter: Benjamin Mahler
Assignee: Benjamin Mahler
Fix For: 0.19.0
Looks like the tests are using an empty MasterInfo.
{noformat}
[--] 4 tests from RegistrarTest
[ RUN ] RegistrarTest.recover
I0307 15:58:37.984900 25438 registrar.cpp:216] Recovering registrar
I0307 15:58:37.989827 25441 registrar.cpp:242] Successfully recovered registrar
I0307 15:58:37.989867 25441 registrar.cpp:378] Attempting to update the
'registry'
I0307 15:58:37.992431 25438 registrar.cpp:413] Successfully updated 'registry'
I0307 15:58:37.992526 25438 registrar.cpp:378] Attempting to update the
'registry'
I0307 15:58:37.994709 25441 registrar.cpp:413] Successfully updated 'registry'
I0307 15:58:37.994753 25441 registrar.cpp:378] Attempting to update the
'registry'
I0307 15:58:37.996723 25443 registrar.cpp:413] Successfully updated 'registry'
[ OK ] RegistrarTest.recover (19 ms)
[ RUN ] RegistrarTest.admit
I0307 15:58:38.004277 25441 registrar.cpp:216] Recovering registrar
I0307 15:58:38.007971 25439 registrar.cpp:242] Successfully recovered registrar
I0307 15:58:38.008002 25439 registrar.cpp:378] Attempting to update the
'registry'
[libprotobuf FATAL google/protobuf/message_lite.cc:273] CHECK failed:
IsInitialized(): Can't serialize message of type mesos.internal.Registry
because it is missing required fields: master.info.id, master.info.ip,
master.info.port
libprocess: registrar@192.168.122.164:55109 terminating due to CHECK failed:
IsInitialized(): Can't serialize message of type mesos.internal.Registry
because it is missing required fields: master.info.id, master.info.ip,
master.info.port
2014-03-07
15:58:40,536:25417(0x7fd7c9443700):ZOO_ERROR@handle_socket_error_msg@1697:
Socket [127.0.0.1:47993] zk retcode=-4, errno=111(Connection refused): server
refused to accept the client
2014-03-07
15:58:43,872:25417(0x7fd7c9443700):ZOO_ERROR@handle_socket_error_msg@1697:
Socket [127.0.0.1:47993] zk retcode=-4, errno=111(Connection refused): server
refused to accept the client
2014-03-07
15:58:47,206:25417(0x7fd7c9443700):ZOO_ERROR@handle_socket_error_msg@1697:
Socket [127.0.0.1:47993] zk retcode=-4, errno=111(Connection refused): server
refused to accept the client
tests/registrar_tests.cpp:122: Failure
Failed to wait 10secs for registrar.recover(MasterInfo())
[ FAILED ] RegistrarTest.admit (10004 ms)
[ RUN ] RegistrarTest.readmit
I0307 15:58:48.006923 25437 registrar.cpp:216] Recovering registrar
I0307 15:58:48.013659 25439 registrar.cpp:242] Successfully recovered registrar
I0307 15:58:48.013696 25439 registrar.cpp:378] Attempting to update the
'registry'
[libprotobuf FATAL google/protobuf/message_lite.cc:273] CHECK failed:
IsInitialized(): Can't serialize message of type mesos.internal.Registry
because it is missing required fields: master.info.id, master.info.ip,
master.info.port
libprocess: registrar@192.168.122.164:55109 terminating due to CHECK failed:
IsInitialized(): Can't serialize message of type mesos.internal.Registry
because it is missing required fields: master.info.id, master.info.ip,
master.info.port
2014-03-07
15:58:50,543:25417(0x7fd7c9443700):ZOO_ERROR@handle_socket_error_msg@1697:
Socket [127.0.0.1:47993] zk retcode=-4, errno=111(Connection refused): server
refused to accept the client
2014-03-07
15:58:53,880:25417(0x7fd7c9443700):ZOO_ERROR@handle_socket_error_msg@1697:
Socket [127.0.0.1:47993] zk retcode=-4, errno=111(Connection refused): server
refused to accept the client
2014-03-07
15:58:57,217:25417(0x7fd7c9443700):ZOO_ERROR@handle_socket_error_msg@1697:
Socket [127.0.0.1:47993] zk retcode=-4, errno=111(Connection refused): server
refused to accept the client
tests/registrar_tests.cpp:142: Failure
Failed to wait 10secs for registrar.recover(MasterInfo())
[ FAILED ] RegistrarTest.readmit (10001 ms)
[ RUN ] RegistrarTest.remove
I0307 15:58:58.007901 25441 registrar.cpp:216] Recovering registrar
I0307 15:58:58.014595 25440 registrar.cpp:242] Successfully recovered registrar
I0307 15:58:58.014668 25440 registrar.cpp:378] Attempting to update the
'registry'
[libprotobuf FATAL google/protobuf/message_lite.cc:273] CHECK failed:
IsInitialized(): Can't serialize message of type mesos.internal.Registry
because it is missing required fields: master.info.id, master.info.ip,
master.info.port
libprocess: registrar@192.168.122.164:55109 terminating due to CHECK failed:
IsInitialized(): Can't serialize message of type mesos.internal.Registry
because it is missing required fields: master.info.id, master.info.ip,
master.info.port
2014-03-07
15:59:00,554:25417(0x7fd7c9443700):ZOO_ERROR@handle_socket_error_msg@1697:
Socket
[jira] [Commented] (MESOS-610) Split slave specific tests out of master_tests
[ https://issues.apache.org/jira/browse/MESOS-610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13926019#comment-13926019 ] Bernd Mathiske commented on MESOS-610: -- Patch has been submitted. Split slave specific tests out of master_tests -- Key: MESOS-610 URL: https://issues.apache.org/jira/browse/MESOS-610 Project: Mesos Issue Type: Improvement Components: test Reporter: Vinod Kone Assignee: Bernd Mathiske Fix For: 0.19.0 master_tests.cpp is getting too unwieldy and is ripe for a split. we should create slave_tests.cpp and move slave related tests there. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (MESOS-1078) JNI calls hasNext on ArrayList instead of iterator.
[ https://issues.apache.org/jira/browse/MESOS-1078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13926023#comment-13926023 ] Bernd Mathiske commented on MESOS-1078: --- Have patch. Will submit shortly. JNI calls hasNext on ArrayList instead of iterator. --- Key: MESOS-1078 URL: https://issues.apache.org/jira/browse/MESOS-1078 Project: Mesos Issue Type: Bug Reporter: Niklas Quarfot Nielsen Assignee: Bernd Mathiske We experienced a crash in the JNI code while using requestResources(). It looks like hasNext() is called on the ArrayList instead of the iterator. When using the Clojure bindings, this caused an exception which propagated into the JNI code. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (MESOS-1078) JNI calls hasNext on ArrayList instead of iterator.
Niklas Quarfot Nielsen created MESOS-1078: - Summary: JNI calls hasNext on ArrayList instead of iterator. Key: MESOS-1078 URL: https://issues.apache.org/jira/browse/MESOS-1078 Project: Mesos Issue Type: Bug Reporter: Niklas Quarfot Nielsen Assignee: Bernd Mathiske We experienced a crash in the JNI code while using requestResources(). It looks like hasNext() is called on the ArrayList instead of the iterator. When using the Clojure bindings, this caused an exception which propagated into the JNI code. -- This message was sent by Atlassian JIRA (v6.2#6252)
Re: Review Request 18974: Factor out method to build command for fetcher.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18974/ --- (Updated March 10, 2014, 11:42 a.m.) Review request for mesos, Ian Downes and Vinod Kone. Bugs: MESOS-1050 and MESOS-1063 https://issues.apache.org/jira/browse/MESOS-1050 https://issues.apache.org/jira/browse/MESOS-1063 Repository: mesos-git Description --- see summary Diffs (updated) - src/Makefile.am 384b3122b61294401ba4a894c06e985d9fc2fb1e src/slave/containerizer/mesos_containerizer.cpp 9bf9829fb38f17d9a2a0d3ab33d45d34cd5d3ea5 src/tests/containerizer_tests.cpp PRE-CREATION Diff: https://reviews.apache.org/r/18974/diff/ Testing --- make check Thanks, Dominic Hamon
Re: Review Request 18953: Improved thread-safety of Latch.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18953/#review36679 --- Ship it! Ship It! - Vinod Kone On March 10, 2014, 8:03 a.m., Benjamin Hindman wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18953/ --- (Updated March 10, 2014, 8:03 a.m.) Review request for mesos and Ben Mahler. Repository: mesos-git Description --- See summary. Diffs - 3rdparty/libprocess/include/process/latch.hpp 5170aa8397ca832a511c55fdc4abfe4678b02f89 3rdparty/libprocess/src/latch.cpp a6f1256ff8860e9a624c0f200ea53912048fd6e3 Diff: https://reviews.apache.org/r/18953/diff/ Testing --- make check Thanks, Benjamin Hindman
Review Request 18976: One-line fix for MESOS-1078
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18976/ --- Review request for mesos. Bugs: MESOS-1078 https://issues.apache.org/jira/browse/MESOS-1078 Repository: mesos-git Description --- Apparently a typical copy-paste mistake of the programmer of this code. Diffs - src/java/jni/org_apache_mesos_MesosSchedulerDriver.cpp d2369ac Diff: https://reviews.apache.org/r/18976/diff/ Testing --- Please allow me to submit this without writing a unit test against it. That would be overkill IMHO for such an obvious fix. We have an internal app at Mesosphere that we cannot share unfortunately that used to crash and now works. Thanks, Bernd Mathiske
Re: Review Request 18976: One-line fix for MESOS-1078: JNI calls hasNext on ArrayList instead of iterator
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18976/ --- (Updated March 10, 2014, 11:46 a.m.) Review request for mesos. Changes --- Expanded summary Summary (updated) - One-line fix for MESOS-1078: JNI calls hasNext on ArrayList instead of iterator Bugs: MESOS-1078 https://issues.apache.org/jira/browse/MESOS-1078 Repository: mesos-git Description --- Apparently a typical copy-paste mistake of the programmer of this code. Diffs - src/java/jni/org_apache_mesos_MesosSchedulerDriver.cpp d2369ac Diff: https://reviews.apache.org/r/18976/diff/ Testing --- Please allow me to submit this without writing a unit test against it. That would be overkill IMHO for such an obvious fix. We have an internal app at Mesosphere that we cannot share unfortunately that used to crash and now works. Thanks, Bernd Mathiske
[jira] [Assigned] (MESOS-1076) Add namespacing to cgroups to enforce the expected structure in slave/containerizer/isolators/cgroups/cpushare.cpp
[ https://issues.apache.org/jira/browse/MESOS-1076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Archana kumari reassigned MESOS-1076: - Assignee: Archana kumari Add namespacing to cgroups to enforce the expected structure in slave/containerizer/isolators/cgroups/cpushare.cpp Key: MESOS-1076 URL: https://issues.apache.org/jira/browse/MESOS-1076 Project: Mesos Issue Type: Improvement Components: general, slave Reporter: Archana kumari Assignee: Archana kumari Original Estimate: 168h Remaining Estimate: 168h The point of the issue is to provide structural information from the cgroups API -- This message was sent by Atlassian JIRA (v6.2#6252)
Re: Review Request 18976: One-line fix for MESOS-1078: JNI calls hasNext on ArrayList instead of iterator
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18976/#review36683 --- Ship it! Ship It! - Benjamin Hindman On March 10, 2014, 6:46 p.m., Bernd Mathiske wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18976/ --- (Updated March 10, 2014, 6:46 p.m.) Review request for mesos. Bugs: MESOS-1078 https://issues.apache.org/jira/browse/MESOS-1078 Repository: mesos-git Description --- Apparently a typical copy-paste mistake of the programmer of this code. Diffs - src/java/jni/org_apache_mesos_MesosSchedulerDriver.cpp d2369ac Diff: https://reviews.apache.org/r/18976/diff/ Testing --- Please allow me to submit this without writing a unit test against it. That would be overkill IMHO for such an obvious fix. We have an internal app at Mesosphere that we cannot share unfortunately that used to crash and now works. Thanks, Bernd Mathiske
Review Request 18977: Fixed the flaky Registrar tests.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18977/ --- Review request for mesos and Vinod Kone. Bugs: MESOS-1077 https://issues.apache.org/jira/browse/MESOS-1077 Repository: mesos-git Description --- See MESOS-1077. MasterInfo has required fields and some of the tests were not setting them. Diffs - src/tests/registrar_tests.cpp 8620e8a5c06b77c7a4ae56308436eb24af90bf6b Diff: https://reviews.apache.org/r/18977/diff/ Testing --- make check with 50 repetitions Thanks, Ben Mahler
Re: Review Request 18977: Fixed the flaky Registrar tests.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18977/#review36685 --- Ship it! src/tests/registrar_tests.cpp https://reviews.apache.org/r/18977/#comment67739 do you want to use protobuf::createMasterInfo()? - Vinod Kone On March 10, 2014, 6:53 p.m., Ben Mahler wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18977/ --- (Updated March 10, 2014, 6:53 p.m.) Review request for mesos and Vinod Kone. Bugs: MESOS-1077 https://issues.apache.org/jira/browse/MESOS-1077 Repository: mesos-git Description --- See MESOS-1077. MasterInfo has required fields and some of the tests were not setting them. Diffs - src/tests/registrar_tests.cpp 8620e8a5c06b77c7a4ae56308436eb24af90bf6b Diff: https://reviews.apache.org/r/18977/diff/ Testing --- make check with 50 repetitions Thanks, Ben Mahler
Re: Review Request 18295: Removed unnecessary includes and inline definitions.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18295/#review36687 --- Patch looks great! Reviews applied: [18295] All tests passed. - Mesos ReviewBot On March 10, 2014, 6:27 p.m., Dominic Hamon wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18295/ --- (Updated March 10, 2014, 6:27 p.m.) Review request for mesos, Benjamin Hindman, Ben Mahler, and Vinod Kone. Bugs: MESOS-1022 https://issues.apache.org/jira/browse/MESOS-1022 Repository: mesos-git Description --- See summary. Diffs - include/mesos/executor.hpp 7bc8ecac3f13431e6f7dd45deb06fecf6e0f5d8a include/mesos/resources.hpp 59f495cdb2a2d55a6c436767789ee579cd5c2f97 include/mesos/scheduler.hpp 85db1117122441b5d9fb013380ea37aa0acc84ea include/mesos/values.hpp 64c138ca6c23ec450e42e3eecf3276e6fc305644 src/Makefile.am 384b3122b61294401ba4a894c06e985d9fc2fb1e src/common/attributes.hpp a08cf188a6d9c27386fd10440d8944f0d7b6fa08 src/common/attributes.cpp 851c92a88804370313bf052c85f99d483fd67416 src/common/build.hpp 115e9cc3ec1e28d4622c7884bcbac904b24b95dd src/common/build.cpp eed08eda4a0db572992187347a2199054ade26d2 src/common/date_utils.hpp 085c1ce685316b3b96b7564c3613bf775a485054 src/common/date_utils.cpp 22c7dacb0f5a265fb595cf8cd0486530d8c28d5c src/common/http.hpp 717bbe99f2ec51a2d1b90730eb19cd54cc82b897 src/common/lock.cpp 11c8e8c50d806271c36c2ec20633acf06447c37f src/common/protobuf_utils.hpp 0f653414bc1bb2b632ec8cd9c8bd7202a53d42e1 src/common/protobuf_utils.cpp PRE-CREATION src/common/resources.cpp 61c5bda9ed7718e87c405822cf3de4795c51f38f src/common/thread.hpp 7e487241e419adb6a64109a949ee80a965771b25 src/common/thread.cpp PRE-CREATION src/common/type_utils.hpp 784a808a5e0b78dab24a4806d6f1f9491a2d6d44 src/common/type_utils.cpp PRE-CREATION src/common/values.cpp 15583fde45946cfd860d13297a03a927f0523115 src/master/http.cpp a019f15615d028f3d3628b2611709fc8a3a81bf8 src/master/master.hpp 49a3e151d53b786027184c14b96371016a926497 src/master/master.cpp 2a403336791737d874c26928c9fffea08ac246d8 src/master/repairer.cpp 151b4ed7ac0b8dacd175b97d372c1f867cd280f6 src/slave/constants.hpp d237383ff8d00f4731b689a2a1592fdd36f09a4d src/slave/containerizer/cgroups_launcher.cpp 39f0e4c3baaed3403da160ba63dada4a53d9af09 src/slave/containerizer/isolator.hpp fc6c9ab10ff535ddd8390aeacf2151ac2d5174a6 src/slave/containerizer/isolators/posix.hpp 7fbc6ddd9aa5518870bf938c6ead5eb72d3ec524 src/slave/containerizer/launcher.cpp fb0c461a00ec21e52e2f008b81cb6d545f537c99 src/slave/flags.hpp e4d98a53cbfb7f9ca828f17e82d492274cb9969d src/slave/gc.hpp 7b6fb8365b2e793c019907ecdfbce7413d72f8b4 src/slave/gc.cpp 3720255704171eaa054c96a790ecd5e6e5304b33 src/slave/http.cpp 594032da1b2edd47961cd8201acd093b22fa30ae src/slave/main.cpp a498a6ae6a79c7155c07a5d6dc2d6c9dc8ae060f src/slave/monitor.hpp c042bc1af31aedf7d2991741d3f8cb70ef353b40 src/slave/monitor.cpp 1c02986e22bc1dcbc2f07de546bf865d34c41c89 src/slave/paths.hpp 41bb73d6359ee6122ccc2170512311e1ad4fcc4c src/slave/paths.cpp PRE-CREATION src/slave/slave.hpp 01b80dfc1dd55edd6d2f722ff1d4f1bf4d96f28e src/slave/slave.cpp 6abb95df4c2339c1e961d2a4ee75d3512c3d23a8 src/slave/state.hpp 22f569def3c000856190deff4d55e636afb9e00e src/slave/state.cpp 21d1fb7fbe695bfc3f80ce17c413261f98e6e0b4 src/slave/status_update_manager.hpp 24e3882ad1969c20a64daf90e408618c310e9a81 src/slave/status_update_manager.cpp 9db53e8b2a6440b7eebe3bc61912b170bde7a473 src/tests/monitor_tests.cpp 4b950e14bd94cdfa21212268b56bebdc1200078d src/tests/slave_recovery_tests.cpp 40a9599787918b78790462e81729ec7ac2395509 Diff: https://reviews.apache.org/r/18295/diff/ Testing --- No functional change. Ran clean build and make check. Timed build: before change - 26m30s after change - 19m33s diff stats: 44 files changed, 1067 insertions(+), 1074 deletions(-) Thanks, Dominic Hamon
[jira] [Commented] (MESOS-1071) Enable building against installed third-party dependencies.
[ https://issues.apache.org/jira/browse/MESOS-1071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13926066#comment-13926066 ] Timothy St. Clair commented on MESOS-1071: -- How about having the slow migration path similar to c++11. Default at first would be to build with a option --enable-proper=no, with the option to input as all the rest of the checks come into play, the transition can occur once it has matured. Enable building against installed third-party dependencies. --- Key: MESOS-1071 URL: https://issues.apache.org/jira/browse/MESOS-1071 Project: Mesos Issue Type: Improvement Components: build Reporter: Benjamin Hindman Most of our third-party dependencies are included in the project and statically linked into our resulting binaries and libraries. We would like to enable building Mesos but using system installed dependencies instead. In certain circumstances this is more difficult because we've actually needed to patch these libraries (either for C++11 or to alter semantics). Rather than eliminating our internal copies of these third-party dependencies the first step should be to just enable using external (i.e., system installed) dependencies. We already do this for ZooKeeper by allowing people to use the --without-included-zookeeper flag during compilation. We should do this for other libraries as well. In fact, for the libraries that we have not patched (and even for some that we have patched) we should check to see if an appropriate system installed dependency exists and preferentially use that unless --with-included-dependency is explicitly used. Note that this issue represents a stepping stone to removing our third-party dependencies from our repository. -- This message was sent by Atlassian JIRA (v6.2#6252)
Re: Review Request 18952: Refactored Promise to be thread-safe.
On March 10, 2014, 6:42 p.m., Vinod Kone wrote: 3rdparty/libprocess/include/process/future.hpp, lines 642-646 https://reviews.apache.org/r/18952/diff/1/?file=514950#file514950line642 IIUC, this is changing the semantics of what happens when you call set/fail/discard on a promise that is already associated. No? You should not have been calling set/fail/discard on a future that was already associated, this enforces as much. - Benjamin --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18952/#review36678 --- On March 10, 2014, 7:18 p.m., Benjamin Hindman wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18952/ --- (Updated March 10, 2014, 7:18 p.m.) Review request for mesos, Ben Mahler and Jie Yu. Repository: mesos-git Description --- In particular, doing associations is strictly safer now. Diffs - 3rdparty/libprocess/include/process/future.hpp 27b0970bf1d1ae1b977ddfc2de5ee858f1031bf5 Diff: https://reviews.apache.org/r/18952/diff/ Testing --- make check Thanks, Benjamin Hindman
Re: Review Request 18977: Fixed the flaky Registrar tests.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18977/ --- (Updated March 10, 2014, 7:20 p.m.) Review request for mesos, Benjamin Hindman and Vinod Kone. Changes --- Used protobuf::createMasterInfo(). Bugs: MESOS-1077 https://issues.apache.org/jira/browse/MESOS-1077 Repository: mesos-git Description --- See MESOS-1077. MasterInfo has required fields and some of the tests were not setting them. Diffs (updated) - src/tests/registrar_tests.cpp 8620e8a5c06b77c7a4ae56308436eb24af90bf6b Diff: https://reviews.apache.org/r/18977/diff/ Testing --- make check with 50 repetitions Thanks, Ben Mahler
Re: Review Request 18977: Fixed the flaky Registrar tests.
On March 10, 2014, 6:57 p.m., Dominic Hamon wrote: src/tests/registrar_tests.cpp, line 53 https://reviews.apache.org/r/18977/diff/1/?file=515319#file515319line53 you could initialize the master in the constructor. Originally I couldn't do this because I would then want master to be 'const', now that I have createMasterInfo() I can use the copy constructor and have a 'const' 'master'. There are some tests that create mutable data in the constructor, which I was originally trying to avoid in this case (see paths_tests.cpp). But I'd like to stick with a fresh mutable master per test, since I can envision recovery tests that will want to re-assign 'master' without affecting other tests. - Ben --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18977/#review36686 --- On March 10, 2014, 6:53 p.m., Ben Mahler wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18977/ --- (Updated March 10, 2014, 6:53 p.m.) Review request for mesos and Vinod Kone. Bugs: MESOS-1077 https://issues.apache.org/jira/browse/MESOS-1077 Repository: mesos-git Description --- See MESOS-1077. MasterInfo has required fields and some of the tests were not setting them. Diffs - src/tests/registrar_tests.cpp 8620e8a5c06b77c7a4ae56308436eb24af90bf6b Diff: https://reviews.apache.org/r/18977/diff/ Testing --- make check with 50 repetitions Thanks, Ben Mahler
Re: Review Request 18386: Option reference cleanup in mesos.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18386/#review36691 --- src/slave/containerizer/mesos_containerizer.cpp https://reviews.apache.org/r/18386/#comment67747 ditto here src/slave/status_update_manager.cpp https://reviews.apache.org/r/18386/#comment67746 Why don't we do the same update here as we did in slave.cpp? That is, we can pull out the option, CHECK_SOME and just use run.get() rather than needing the temporary '_run' variable. - Ben Mahler On March 5, 2014, 12:31 a.m., Dominic Hamon wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18386/ --- (Updated March 5, 2014, 12:31 a.m.) Review request for mesos and Ben Mahler. Bugs: MESOS-1008 https://issues.apache.org/jira/browse/MESOS-1008 Repository: mesos-git Description --- See summary Diffs - src/linux/fs.hpp 1d86dd0d24c3daae957b5eec387638d1e8e6d7db src/linux/fs.cpp e5f4f9a16becd4e5960d0cbb7f988736188b2426 src/log/log.cpp 7f855f25d97e0caeafa7708951c4ec51ddbc3de4 src/sched/sched.cpp 00f6307e539d3176185266095c2424a58ea1d426 src/slave/containerizer/mesos_containerizer.cpp 6d990cb1045bb4e68668ad0710eeb2ab5c9bbdb5 src/slave/slave.cpp b350df45c631a8976011eb88435728b6d7623848 src/slave/status_update_manager.cpp 9db53e8b2a6440b7eebe3bc61912b170bde7a473 Diff: https://reviews.apache.org/r/18386/diff/ Testing --- make check 'grep' for cases where Options are reassigned after references are taken. Thanks, Dominic Hamon
Re: building meso-0.17.0 failing on maverick
I believe that we are doing that since commit c3505e319d759e68425aa3b7177fc4377bf8db77 had landed. That patch should warn users during the configuration phase when invalid python / c++ compiler setups are detected. Haja, could you please send your config.log directly to me (toensh...@me.com) as I missed your original post and the archives seem to have removed your attachment. On Mar 10, 2014, at 7:43 PM, Benjamin Mahler benjamin.mah...@gmail.com wrote: Is there anything we can do to enforce this compatibility matrix at configure time? -- Forwarded message -- From: Till Toenshoff toensh...@me.com Date: Mon, Mar 10, 2014 at 11:38 AM Subject: Re: building meso-0.17.0 failing on maverick To: dev@mesos.apache.org Hey there - I did unfortunately miss the initial question, hence I have to do a bit of guess-work here. Forgive me if that is entirely off. That flag is not explicitly set by Mesos, as Vinod mentioned. It is however implicitly done by distutils extraction the cflags out of the existing python installation. So in the end, Mesos does in fact use that flag at least for the Python egg compilation. You can check that by running: python-config —cflags I would expect something like the following as the result: -I/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -I/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -fno-strict-aliasing -fno-common -dynamic -arch x86_64 -arch i386 -g -Os -pipe -fno-common -fno-strict-aliasing -fwrapv -mno-fused-madd -DENABLE_DTRACE -DMACOSX -DNDEBUG -Wall -Wstrict-prototypes -Wshorten-64-to-32 -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE As you can see, there is this -Wshorten-64-to-32” coming from and it is used when building the python egg. As you are using gcc to compile and not clang, the problem pops up. If you really have to use gcc, then you will have to install an alternative Python distribution - I would recommend using homebrew or anything else. As an alternative, use clang as that is AFAIK fully supported since Mesos 0.17.0. The simplified compatibility matrix is as follows: apple-python brew-python clang X- gcc- X Hope that helps. There is some more background on this issue on http://stackoverflow.com/questions/20733512/mesos-examplestest-pythonframework-check-fails-on-osx cheerio! Till On Mar 10, 2014, at 6:50 PM, Vinod Kone vinodk...@gmail.com wrote: On Sat, Mar 8, 2014 at 6:23 PM, haja gecko haja.ge...@gmail.com wrote: gcc-4.8: error: unrecognized command line option '-Wshorten-64-to-32' This doesn't seem to be a valid gcc option. AFAICT, this is not being set in our Makefiles. Do you have some alias that is setting this option? ➜ mesos git:(master) gcc-4.8 -Wshorten-64-to-32 gcc-4.8: error: unrecognized command line option '-Wshorten-64-to-32' gcc-4.8: fatal error: no input files compilation terminated. ➜ mesos git:(master) ack shorten-64-to-32 * ➜
[jira] [Updated] (MESOS-1071) Enable building against installed third-party dependencies.
[ https://issues.apache.org/jira/browse/MESOS-1071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Timothy St. Clair updated MESOS-1071: - Attachment: modified_tillt.patch Slight modification to [~tillt] 's original patch, which should enable. Enable building against installed third-party dependencies. --- Key: MESOS-1071 URL: https://issues.apache.org/jira/browse/MESOS-1071 Project: Mesos Issue Type: Improvement Components: build Reporter: Benjamin Hindman Attachments: modified_tillt.patch Most of our third-party dependencies are included in the project and statically linked into our resulting binaries and libraries. We would like to enable building Mesos but using system installed dependencies instead. In certain circumstances this is more difficult because we've actually needed to patch these libraries (either for C++11 or to alter semantics). Rather than eliminating our internal copies of these third-party dependencies the first step should be to just enable using external (i.e., system installed) dependencies. We already do this for ZooKeeper by allowing people to use the --without-included-zookeeper flag during compilation. We should do this for other libraries as well. In fact, for the libraries that we have not patched (and even for some that we have patched) we should check to see if an appropriate system installed dependency exists and preferentially use that unless --with-included-dependency is explicitly used. Note that this issue represents a stepping stone to removing our third-party dependencies from our repository. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (MESOS-1071) Enable building against installed third-party dependencies.
[ https://issues.apache.org/jira/browse/MESOS-1071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13926151#comment-13926151 ] Till Toenshoff commented on MESOS-1071: --- I'll go ahead and acquire your work, present it as mine and win :D thanks! Will update my RR. Enable building against installed third-party dependencies. --- Key: MESOS-1071 URL: https://issues.apache.org/jira/browse/MESOS-1071 Project: Mesos Issue Type: Improvement Components: build Reporter: Benjamin Hindman Attachments: modified_tillt.patch Most of our third-party dependencies are included in the project and statically linked into our resulting binaries and libraries. We would like to enable building Mesos but using system installed dependencies instead. In certain circumstances this is more difficult because we've actually needed to patch these libraries (either for C++11 or to alter semantics). Rather than eliminating our internal copies of these third-party dependencies the first step should be to just enable using external (i.e., system installed) dependencies. We already do this for ZooKeeper by allowing people to use the --without-included-zookeeper flag during compilation. We should do this for other libraries as well. In fact, for the libraries that we have not patched (and even for some that we have patched) we should check to see if an appropriate system installed dependency exists and preferentially use that unless --with-included-dependency is explicitly used. Note that this issue represents a stepping stone to removing our third-party dependencies from our repository. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (MESOS-1071) Enable building against installed third-party dependencies.
[ https://issues.apache.org/jira/browse/MESOS-1071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13926157#comment-13926157 ] Timothy St. Clair commented on MESOS-1071: -- It's open source, tis the name of the game ;-) Enable building against installed third-party dependencies. --- Key: MESOS-1071 URL: https://issues.apache.org/jira/browse/MESOS-1071 Project: Mesos Issue Type: Improvement Components: build Reporter: Benjamin Hindman Attachments: modified_tillt.patch Most of our third-party dependencies are included in the project and statically linked into our resulting binaries and libraries. We would like to enable building Mesos but using system installed dependencies instead. In certain circumstances this is more difficult because we've actually needed to patch these libraries (either for C++11 or to alter semantics). Rather than eliminating our internal copies of these third-party dependencies the first step should be to just enable using external (i.e., system installed) dependencies. We already do this for ZooKeeper by allowing people to use the --without-included-zookeeper flag during compilation. We should do this for other libraries as well. In fact, for the libraries that we have not patched (and even for some that we have patched) we should check to see if an appropriate system installed dependency exists and preferentially use that unless --with-included-dependency is explicitly used. Note that this issue represents a stepping stone to removing our third-party dependencies from our repository. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Resolved] (MESOS-1078) JNI calls hasNext on ArrayList instead of iterator.
[ https://issues.apache.org/jira/browse/MESOS-1078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Niklas Quarfot Nielsen resolved MESOS-1078. --- Resolution: Fixed Fixed in https://reviews.apache.org/r/18976/ JNI calls hasNext on ArrayList instead of iterator. --- Key: MESOS-1078 URL: https://issues.apache.org/jira/browse/MESOS-1078 Project: Mesos Issue Type: Bug Reporter: Niklas Quarfot Nielsen Assignee: Bernd Mathiske We experienced a crash in the JNI code while using requestResources(). It looks like hasNext() is called on the ArrayList instead of the iterator. When using the Clojure bindings, this caused an exception which propagated into the JNI code. -- This message was sent by Atlassian JIRA (v6.2#6252)
Re: Review Request 18386: Option reference cleanup in mesos.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18386/ --- (Updated March 10, 2014, 1:35 p.m.) Review request for mesos and Ben Mahler. Bugs: MESOS-1008 https://issues.apache.org/jira/browse/MESOS-1008 Repository: mesos-git Description --- See summary Diffs (updated) - src/linux/fs.hpp 1d86dd0d24c3daae957b5eec387638d1e8e6d7db src/linux/fs.cpp e5f4f9a16becd4e5960d0cbb7f988736188b2426 src/log/log.cpp 7f855f25d97e0caeafa7708951c4ec51ddbc3de4 src/sched/sched.cpp 00f6307e539d3176185266095c2424a58ea1d426 src/slave/containerizer/mesos_containerizer.cpp 6d990cb1045bb4e68668ad0710eeb2ab5c9bbdb5 src/slave/slave.cpp b350df45c631a8976011eb88435728b6d7623848 src/slave/status_update_manager.cpp 9db53e8b2a6440b7eebe3bc61912b170bde7a473 Diff: https://reviews.apache.org/r/18386/diff/ Testing --- make check 'grep' for cases where Options are reassigned after references are taken. Thanks, Dominic Hamon
Re: Review Request 18752: MESOS-610: Split slave specific tests out of master_tests
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18752/ --- (Updated March 10, 2014, 8:37 p.m.) Review request for mesos and Vinod Kone. Changes --- assigning to myself for review. -- Vinod Bugs: MESOS-610 https://issues.apache.org/jira/browse/MESOS-610 Repository: mesos-git Description --- MESOS-610: Split slave specific tests out of master_tests. Not that most of the tests in master_test.cpp are very master-specific to begin with. Nor are any of them terribly slave-specific. Typically, any such tests concern the whole Mesos conglomerate including the scheduler driver. The submitted patch suggests a more or less plausible split, but other divisions are easily thinkable. I'd also be happy with keeping them all in one file and renaming that file, maybe to mesos_tests.cpp? Diffs - src/Makefile.am 61d832b src/tests/master_tests.cpp 42c5a77 src/tests/slave_tests.cpp PRE-CREATION Diff: https://reviews.apache.org/r/18752/diff/ Testing --- Compiled and ran all tests (make check) on Mac OS 10.9 / Clang 3.3 and Ubuntu 13.10 / gcc 4.8.1. Thanks, Bernd Mathiske
[jira] [Updated] (MESOS-1076) Add namespacing to cgroups to enforce the expected structure in slave/containerizer/isolators/cgroups/cpushare.cpp
[ https://issues.apache.org/jira/browse/MESOS-1076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Archana kumari updated MESOS-1076: -- Summary: Add namespacing to cgroups to enforce the expected structure in slave/containerizer/isolators/cgroups/cpushare.cpp (was: Add namespacing to cgroups to enforce the expected structure in slave/containerizer/isolators/cgroups/cpushare.cpp) Add namespacing to cgroups to enforce the expected structure in slave/containerizer/isolators/cgroups/cpushare.cpp --- Key: MESOS-1076 URL: https://issues.apache.org/jira/browse/MESOS-1076 Project: Mesos Issue Type: Improvement Components: general, slave Reporter: Archana kumari Assignee: Archana kumari Original Estimate: 168h Remaining Estimate: 168h The point of the issue is to provide structural information from the cgroups API -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (MESOS-816) Allow delegation to shell scripts for isolation
[
https://issues.apache.org/jira/browse/MESOS-816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13926181#comment-13926181
]
Nikita Vetoshkin commented on MESOS-816:
Just looking at slave-containerizer interface. Would it be a good idea to
wrap all responses into some kind of {{Option}}? i.e. to give the containerizer
process to tell if it failed to fulfil the action because of error e.g.
{{Permission denied}}, {{No such file or directory}}, {{No docker installed}}.
Otherwise it could be very difficult to debug errors as containerizer should
not have any logs.
Allow delegation to shell scripts for isolation
---
Key: MESOS-816
URL: https://issues.apache.org/jira/browse/MESOS-816
Project: Mesos
Issue Type: Improvement
Components: isolation, slave
Reporter: Jason Dusek
Priority: Minor
Attachments: mesos-shell-isolator.jpg
Original Estimate: 72h
Remaining Estimate: 72h
Being able to delegate isolation to shell scripts could make it easier to
leverage the machinery provided by the LXC tools, LibVirt, VirtualBox, Docker
and similar containerization systems.
Why go through command line tools for isolation? We have seen many requests
for isolation, covering a wide variety of scenarios:
- Setups requiring multiple versions of the same language (Ruby 1.8, Ruby
1.9).
- Setups requiring installation and configuration of RPM-packaged
applications.
- Build-and-test setups, where sharing the environment of the host would
impact reproducibility.
- Integration of 3rd party, service-oriented applications.
- Launching applications with Docker.
- Launching multiple instances of a Mesos framework that, like Hadoop, has
significant system setup and dependencies.
To cover these and other use cases, it seems reasonable to allow Mesos to
delegate to external programs for isolation:
- It makes it easier to experiment with new containerization tools.
- It allows for site administrators to customize containerization, or even
implement new containerization mechanisms, without impacting their ability to
keep pace with Mesos development.
- Many external programs exist for containerization -- Docker, LXC tools,
LibVirt -- which handle a great deal of the book-keeping around finding and
efficiently cloning disk images and setting up the guest system (its
hostname, TTYs, /dev/*, /proc).
The scenarios listed above can be understood in terms of three use cases:
- The containerized system service scenario, wherein an application,
installed with RPM or a similar tool, is started and managed by the init
system within a container. Percona MySQL is an example of such an application.
- The containerized application scenario, wherein an application is installed
or unpacked and then configured and launched in a single command. For
example, running a custom Rails app with bundle install bundle exec rails.
- The containerized framework/executor scenario, wherein the application is
Spark, Hadoop or another Mesos framework/executor pair.
One way to achieve this could be to introduce an External Isolator, which
works in parallel with the existing process/posix and cgroups isolators. The
responsibility of this isolator would be to act as a thin layer to external
isolators. Calls for task launching, stopping or any other resource change
would be serialized and passed to the external isolators by the Mesos
External Isolator.
Allowing for pluggable isolators invites the possibility of having different
isolators per task. For applications using containers, it's reasonable that
each application or framework can specify a different base image; and this
would be an option passed to the corresponding isolator. One can also imagine
specialized frameworks that need to disable isolation entirely. For example,
a system backup framework that would specify a null isolator to allow it to
snapshot interesting data on each slave and transfer it to a sanctioned
storage location.
However, for users and framework authors to specify isolators would both be
harmful to portability and would make isolation their problem, no longer
something handled transparently by Mesos. Furthermore, it would have the
unintended effect of putting them at odds with site administrators, who would
also specify isolators -- as a command line option for each slave.
Allowing tasks to carry a more abstract notion of container with them would
allow for most application level scenarios we've outlined above.
Theoretically, more than one isolator might be able to handle a given
container. For example if, the container is specified as an ISO and a
distro LiveCD is provided, one could imagine a Docker isolator, LXC isolator
or Virtualbox isolator handling it.
Re: Review Request 15542: Offer an execvp like interface for running tasks.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/15542/#review36700 --- Hey Jason, Are the outstanding issues still pending or do you have an up-coming patch? :) - Niklas Nielsen On Jan. 21, 2014, 2:36 p.m., Jason Dusek wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/15542/ --- (Updated Jan. 21, 2014, 2:36 p.m.) Review request for mesos, Benjamin Hindman, Ben Mahler, and Niklas Nielsen. Repository: mesos-git Description --- Offer an execvp like interface for running tasks. Review: https://reviews.apache.org/r/15542 Diffs - include/mesos/mesos.proto 655f86757487ddbe551fdcf53eb793e773ecdd34 src/examples/python/test_framework.py deca48e779ae099424fa73bb9a8ac5c419c5faf1 src/launcher/executor.cpp b73ab479500a7347a38ba53acecfab9229f1080d src/launcher/launcher.cpp d5ab66704429a95eeb8eda5188e33d8e691221af src/launcher/main.cpp de64609905ee63096c0173fe7e64a1eafea5d6bf src/sched/sched.cpp f9028e81d81c6229d07737df2136077bf902a372 src/slave/process_isolator.cpp 0bc698f04f7c8eaad166dc9d646e13310129dd01 Diff: https://reviews.apache.org/r/15542/diff/ Testing --- Ran Python test executor and `make check`. Thanks, Jason Dusek
[jira] [Commented] (MESOS-610) Split slave specific tests out of master_tests
[ https://issues.apache.org/jira/browse/MESOS-610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13926202#comment-13926202 ] Vinod Kone commented on MESOS-610: -- Can paste the review url :) Split slave specific tests out of master_tests -- Key: MESOS-610 URL: https://issues.apache.org/jira/browse/MESOS-610 Project: Mesos Issue Type: Improvement Components: test Reporter: Vinod Kone Assignee: Bernd Mathiske Fix For: 0.19.0 master_tests.cpp is getting too unwieldy and is ripe for a split. we should create slave_tests.cpp and move slave related tests there. -- This message was sent by Atlassian JIRA (v6.2#6252)
Re: Review Request 18946: Moved JNI code to separate library
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18946/#review36705 --- So I looked over the patch, and it appears there will be a new install target but it doesn't appear to be shared or versioned? e.g. (no) -version-info 0:0:0 -release $(PACKAGE_VERSION) -shared Is there a reason? - Timothy St. Clair On March 10, 2014, 3:27 a.m., Till Toenshoff wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18946/ --- (Updated March 10, 2014, 3:27 a.m.) Review request for mesos, Adam B, Ben Mahler, Niklas Nielsen, and Vinod Kone. Bugs: MESOS-855 https://issues.apache.org/jira/browse/MESOS-855 Repository: mesos-git Description --- Introduced a new environment variable (MESOS_NATIVE_JAVA_LIBRARY). That variable points towards libmesos_java. libmesos_java contains the JNI- specific code (formally part of libmesos) and dynamically links against libmesos. A typical java-based framework relies on mesos.jar to do the loading but may use some extra logic in its startup to make sure MESOS_NATIVE[_JAVA]_LIBRARY is set/valid. That extra-logic would need to be adapted to use the new environment variable instead of the old one. Diffs - bin/mesos-slave-flags.sh.in dc73aef src/Makefile.am 384b312 src/java/generated/org/apache/mesos/MesosNativeLibrary.java.in 231d1e2 Diff: https://reviews.apache.org/r/18946/diff/ Testing --- make check and functional testing with external, java based frameworks Thanks, Till Toenshoff
Re: Review Request 18595: Added overloaded killtree for process trees.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18595/#review36706 --- Umm this is what PID namespaces is for. FWIW a fork bomb process could escape this logic, but not escape a namespace. http://timothysc.github.io/blog/2013/02/22/perprocess/ - Timothy St. Clair On Feb. 28, 2014, 12:54 a.m., Niklas Nielsen wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18595/ --- (Updated Feb. 28, 2014, 12:54 a.m.) Review request for mesos and Ben Mahler. Repository: mesos-git Description --- New killtree(ProcessTree tree, int signal) traverse process tree and sends a signal to all pids. This is done regardless of presence and state of process. Patch is used by up coming signal escalation. Diffs - 3rdparty/libprocess/3rdparty/stout/include/stout/os/killtree.hpp 1f45897 Diff: https://reviews.apache.org/r/18595/diff/ Testing --- Functional testing with signal escalation code and make check. Thanks, Niklas Nielsen
Re: Review Request 15542: Offer an execvp like interface for running tasks.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/15542/#review36708 --- Bad patch! Reviews applied: [15542] Failed command: git apply --index 15542.patch Error: error: patch failed: include/mesos/mesos.proto:109 error: include/mesos/mesos.proto: patch does not apply error: src/launcher/launcher.cpp: does not exist in index error: src/launcher/main.cpp: does not exist in index error: src/slave/process_isolator.cpp: does not exist in index - Mesos ReviewBot On Jan. 21, 2014, 10:36 p.m., Jason Dusek wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/15542/ --- (Updated Jan. 21, 2014, 10:36 p.m.) Review request for mesos, Benjamin Hindman, Ben Mahler, and Niklas Nielsen. Repository: mesos-git Description --- Offer an execvp like interface for running tasks. Review: https://reviews.apache.org/r/15542 Diffs - include/mesos/mesos.proto 655f86757487ddbe551fdcf53eb793e773ecdd34 src/examples/python/test_framework.py deca48e779ae099424fa73bb9a8ac5c419c5faf1 src/launcher/executor.cpp b73ab479500a7347a38ba53acecfab9229f1080d src/launcher/launcher.cpp d5ab66704429a95eeb8eda5188e33d8e691221af src/launcher/main.cpp de64609905ee63096c0173fe7e64a1eafea5d6bf src/sched/sched.cpp f9028e81d81c6229d07737df2136077bf902a372 src/slave/process_isolator.cpp 0bc698f04f7c8eaad166dc9d646e13310129dd01 Diff: https://reviews.apache.org/r/15542/diff/ Testing --- Ran Python test executor and `make check`. Thanks, Jason Dusek
[jira] [Commented] (MESOS-969) resource allocation (memory) beyond availability
[ https://issues.apache.org/jira/browse/MESOS-969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13926282#comment-13926282 ] Ashutosh Jain commented on MESOS-969: - As per our discussion, I have made changes in slave/main.cpp and make check tends to completion with one failed test. what is the next step ? link to log of make check - https://www.dropbox.com/s/xi0517c5crgo80o/makecheck_1.txt resource allocation (memory) beyond availability Key: MESOS-969 URL: https://issues.apache.org/jira/browse/MESOS-969 Project: Mesos Issue Type: Bug Affects Versions: 0.14.2, 0.18.0 Reporter: Ashutosh Jain Assignee: Ashutosh Jain Priority: Minor I am running my mesos 0.14.2 instance in virtual-box. I had allocated 1466 MB of memory to it. 1. I created the server at 127.0.1.1:5050. ./mesos-master.sh 2. then I added the system itself as slave and allocated 2000 MB of memory using ./mesos-slave.sh --master=127.0.1.1:5050 --resources=mem:2000 and the resources were happily allocated to it. (allocated memory was being shown in the GUI at 127.0.1.1:5050 in the browser) I even tried with 2 of memory and still the same results . basically it not being checked whether the resources are even available or not . -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (MESOS-610) Split slave specific tests out of master_tests
[ https://issues.apache.org/jira/browse/MESOS-610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13926285#comment-13926285 ] Bernd Mathiske commented on MESOS-610: -- https://reviews.apache.org/r/18752/ Split slave specific tests out of master_tests -- Key: MESOS-610 URL: https://issues.apache.org/jira/browse/MESOS-610 Project: Mesos Issue Type: Improvement Components: test Reporter: Vinod Kone Assignee: Bernd Mathiske Fix For: 0.19.0 master_tests.cpp is getting too unwieldy and is ripe for a split. we should create slave_tests.cpp and move slave related tests there. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (MESOS-969) resource allocation (memory) beyond availability
[ https://issues.apache.org/jira/browse/MESOS-969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13926293#comment-13926293 ] Adam B commented on MESOS-969: -- I was also seeing the PythonFramework test fail (due to lingering launcher objects) after the recent containerizer changes. Try a 'make clean' (and possibly 'make distclean') to clear that out. When your changes are ready, submit a patch in ReviewBoard and post the link here. resource allocation (memory) beyond availability Key: MESOS-969 URL: https://issues.apache.org/jira/browse/MESOS-969 Project: Mesos Issue Type: Bug Affects Versions: 0.14.2, 0.18.0 Reporter: Ashutosh Jain Assignee: Ashutosh Jain Priority: Minor I am running my mesos 0.14.2 instance in virtual-box. I had allocated 1466 MB of memory to it. 1. I created the server at 127.0.1.1:5050. ./mesos-master.sh 2. then I added the system itself as slave and allocated 2000 MB of memory using ./mesos-slave.sh --master=127.0.1.1:5050 --resources=mem:2000 and the resources were happily allocated to it. (allocated memory was being shown in the GUI at 127.0.1.1:5050 in the browser) I even tried with 2 of memory and still the same results . basically it not being checked whether the resources are even available or not . -- This message was sent by Atlassian JIRA (v6.2#6252)
Re: Review Request 11975: Use hostname instead of IP for redirection.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/11975/#review36710 --- Is this patch still relevant or can it be dropped? - Niklas Nielsen On Oct. 22, 2013, 12:03 p.m., Brenden Matthews wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/11975/ --- (Updated Oct. 22, 2013, 12:03 p.m.) Review request for mesos. Repository: mesos Description --- Use hostname instead of IP for redirection. In the Web UI, the redirection does not work for hosts where the local interface address is not publicly accessible. For example, with EC2 the redirection will not work. This change writes the Master's hostname into ZooKeeper, thereby allowing the redirection to work with private IP addresses. Review: https://reviews.apache.org/r/11975/ Diffs - src/detector/detector.hpp b0e66888050c1987b7200cdbf21ebe5e2e55e6c0 src/detector/detector.cpp 12deefa0b9df3f4946d80f500caaa5199b8ea28e src/local/local.cpp e4b5ec5b3dfae6dd89746353d3caaa8d7e117840 src/master/http.cpp ca66d6782023d303c8acc89ef9cf421cfca8d0dc src/master/main.cpp 19fcb9f09d8fc03a0719aada7d216b575eb3069b src/master/master.hpp 8ec0c17c71b7b4679d4f712d0fb742d420c9152d src/master/master.cpp b0a2757af3ec83ead53374504fe24d3a8f7673ad src/messages/messages.proto 19d4b387b50884f9f4a70efb3e9b739f846abf57 src/sched/sched.cpp 5718fe9280d7e1124ffef2de9e4d47b347f0600a src/slave/main.cpp 750a12766bde64059bfd4635ea077cbd43cb4301 src/tests/cluster.hpp f743bb3251af81fb9d8afd51de4df6efcf289bb9 src/tests/master_detector_tests.cpp 2d140ba1a364a7af4d643951d6016ac17dd10526 src/webui/master/static/js/controllers.js c553e358e9f7598033e55a2e8dfff167d8282f7b Diff: https://reviews.apache.org/r/11975/diff/ Testing --- make check Thanks, Brenden Matthews
Re: Review Request 11975: Use hostname instead of IP for redirection.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/11975/#review36712 --- Bad patch! Reviews applied: [11975] Failed command: git apply --index 11975.patch Error: error: src/detector/detector.hpp: does not exist in index error: src/detector/detector.cpp: does not exist in index error: patch failed: src/local/local.cpp:124 error: src/local/local.cpp: patch does not apply error: patch failed: src/master/http.cpp:276 error: src/master/http.cpp: patch does not apply error: patch failed: src/master/main.cpp:123 error: src/master/main.cpp: patch does not apply error: patch failed: src/master/master.hpp:82 error: src/master/master.hpp: patch does not apply error: patch failed: src/master/master.cpp:327 error: src/master/master.cpp: patch does not apply error: patch failed: src/messages/messages.proto:332 error: src/messages/messages.proto: patch does not apply error: patch failed: src/sched/sched.cpp:111 error: src/sched/sched.cpp: patch does not apply error: patch failed: src/slave/main.cpp:132 error: src/slave/main.cpp: patch does not apply error: patch failed: src/tests/cluster.hpp:228 error: src/tests/cluster.hpp: patch does not apply error: src/tests/master_detector_tests.cpp: does not exist in index error: patch failed: src/webui/master/static/js/controllers.js:124 error: src/webui/master/static/js/controllers.js: patch does not apply - Mesos ReviewBot On Oct. 22, 2013, 7:03 p.m., Brenden Matthews wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/11975/ --- (Updated Oct. 22, 2013, 7:03 p.m.) Review request for mesos. Repository: mesos Description --- Use hostname instead of IP for redirection. In the Web UI, the redirection does not work for hosts where the local interface address is not publicly accessible. For example, with EC2 the redirection will not work. This change writes the Master's hostname into ZooKeeper, thereby allowing the redirection to work with private IP addresses. Review: https://reviews.apache.org/r/11975/ Diffs - src/detector/detector.hpp b0e66888050c1987b7200cdbf21ebe5e2e55e6c0 src/detector/detector.cpp 12deefa0b9df3f4946d80f500caaa5199b8ea28e src/local/local.cpp e4b5ec5b3dfae6dd89746353d3caaa8d7e117840 src/master/http.cpp ca66d6782023d303c8acc89ef9cf421cfca8d0dc src/master/main.cpp 19fcb9f09d8fc03a0719aada7d216b575eb3069b src/master/master.hpp 8ec0c17c71b7b4679d4f712d0fb742d420c9152d src/master/master.cpp b0a2757af3ec83ead53374504fe24d3a8f7673ad src/messages/messages.proto 19d4b387b50884f9f4a70efb3e9b739f846abf57 src/sched/sched.cpp 5718fe9280d7e1124ffef2de9e4d47b347f0600a src/slave/main.cpp 750a12766bde64059bfd4635ea077cbd43cb4301 src/tests/cluster.hpp f743bb3251af81fb9d8afd51de4df6efcf289bb9 src/tests/master_detector_tests.cpp 2d140ba1a364a7af4d643951d6016ac17dd10526 src/webui/master/static/js/controllers.js c553e358e9f7598033e55a2e8dfff167d8282f7b Diff: https://reviews.apache.org/r/11975/diff/ Testing --- make check Thanks, Brenden Matthews
Re: Review Request 13675: Added a slave load hint for scheduler sorting.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/13675/#review36713 --- https://reviews.apache.org/r/17325 expose system load for both masters and slaves. So you should be able to get statistics with the offer.hostname at hand. Do you still want to hang statistics off the offer? - Niklas Nielsen On Aug. 26, 2013, 3:38 p.m., Brenden Matthews wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/13675/ --- (Updated Aug. 26, 2013, 3:38 p.m.) Review request for mesos. Repository: mesos-git Description --- Added a slave load hint for scheduler sorting. With this change, the offers will include a slave load hint based on the 5 minute load average for that slave (returned from getloadavg()). Frameworks can use this hint to select among otherwise similar offers, as an optimization. Review: https://reviews.apache.org/r/13675 Diffs - include/mesos/mesos.proto bbf1d31a8a51c58d7118a0d61f24e47f7b6f1b11 src/master/master.hpp 6bd899894e7b0cd5178d52f930a60be11f836976 src/master/master.cpp dc155ba7225ed194525f6ae2a6b96ca8dbdfa432 src/slave/slave.cpp 3e2c6007ca2fbf210a76af376bb5bac5fa32594f Diff: https://reviews.apache.org/r/13675/diff/ Testing --- make check Thanks, Brenden Matthews
Re: What happens when I call reconcileTasks and database divergence
This is a great question and is the primary motivation behind: https://issues.apache.org/jira/browse/MESOS-295 To guarantee frameworks can maintain a consistent view of their tasks (without a custom reconciliation mechanism, as is used in Aurora), we will be implementing the Registrar to persist a small amount of state in the Master: https://issues.apache.org/jira/browse/MESOS-764 I'm also planning to write a document describing how to properly implement state reconciliation from a framework developer's perspective, once the Registrar is released. After discussing with other committers, it's likely that the Registrar will be released as follows to provide the smoothest upgrade path: 1. Initial Registrar release. This will, by default, provide the same semantics as before. (--strict=false). 2. Subsequent release. This will, by default, provide strict semantics. (--strict=true). Only when we're operating in a --strict manner can frameworks fully reconcile state against the Master. The design doc may shed some light here, but some things are out-of-date (including --strict, which does not correspond to what I've described here): https://cwiki.apache.org/confluence/display/MESOS/Registrar+Design+Document I'll update the doc in the coming weeks, let me know if you have other questions! On Sun, Mar 9, 2014 at 9:47 PM, Vinod Kone vinodk...@gmail.com wrote: Hey David, You might want to look at Aurora and Marathon to see how they do state reconciliation. We are working on a new feature, adding persistent state to master (MESOS-764) https://issues.apache.org/jira/browse/MESOS-764, that should make reconciliation even easier.
[jira] [Assigned] (MESOS-1008) Reduce copying in stout / libprocess primitives {Try, Option, Result, Future}.
[
https://issues.apache.org/jira/browse/MESOS-1008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dominic Hamon reassigned MESOS-1008:
Assignee: Dominic Hamon (was: Benjamin Mahler)
Reduce copying in stout / libprocess primitives {Try, Option, Result, Future}.
--
Key: MESOS-1008
URL: https://issues.apache.org/jira/browse/MESOS-1008
Project: Mesos
Issue Type: Improvement
Reporter: Benjamin Mahler
Assignee: Dominic Hamon
The following will discuss {{Try}}, but the same applies to {{Option}} and
{{Result}}.
Currently retrieving the value from a {{Try}} requires a copy:
{code}
T get() const { ... return *t; }
{code}
Instead, we can return a const:
{code}
const T get() const { ... return *t; }
{code}
For existing callers, this should be fairly seamless:
{code}
const T t = try.get(); // No change needed.
T t = try.get(); // No change needed, T is already required to be copyable.
try.get().mutator(); // Will no longer compile, but we should not allow
this anyway!
{code}
{code}
const T t = try.get();
try = T(); // t is now garbage!
t.foo(); // No longer works.
{code}
The last case is the most concerning as this mistake cannot be caught at
compile-time. We could remove the assignment operators, but this seems overly
restrictive. We could also guard (via {{CHECK}}) the assignment operator
after a {{get()}} operation, but of course, we're also preventing valid
{{Try}} usage if we go this route. The best path may simply be to leave the
assignment operators as is (many callers already make copies).
We can also add C++11 support for move to pilfer the {{Try}}, to avoid copies
in the caller entirely:
{code}
T move() const { ... } // After a move(), we must guard Try operations.
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
Re: Review Request 18386: Option reference cleanup in mesos.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18386/#review36726 --- Patch looks great! Reviews applied: [18386] All tests passed. - Mesos ReviewBot On March 10, 2014, 8:35 p.m., Dominic Hamon wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18386/ --- (Updated March 10, 2014, 8:35 p.m.) Review request for mesos and Ben Mahler. Bugs: MESOS-1008 https://issues.apache.org/jira/browse/MESOS-1008 Repository: mesos-git Description --- See summary Diffs - src/linux/fs.hpp 1d86dd0d24c3daae957b5eec387638d1e8e6d7db src/linux/fs.cpp e5f4f9a16becd4e5960d0cbb7f988736188b2426 src/log/log.cpp 7f855f25d97e0caeafa7708951c4ec51ddbc3de4 src/sched/sched.cpp 00f6307e539d3176185266095c2424a58ea1d426 src/slave/containerizer/mesos_containerizer.cpp 6d990cb1045bb4e68668ad0710eeb2ab5c9bbdb5 src/slave/slave.cpp b350df45c631a8976011eb88435728b6d7623848 src/slave/status_update_manager.cpp 9db53e8b2a6440b7eebe3bc61912b170bde7a473 Diff: https://reviews.apache.org/r/18386/diff/ Testing --- make check 'grep' for cases where Options are reassigned after references are taken. Thanks, Dominic Hamon
[jira] [Commented] (MESOS-471) Improve documentation about some slave flags (e.g., resources, attributes)
[ https://issues.apache.org/jira/browse/MESOS-471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13926400#comment-13926400 ] Jörn Franke commented on MESOS-471: --- will include all configuration option from the latest source code Improve documentation about some slave flags (e.g., resources, attributes) -- Key: MESOS-471 URL: https://issues.apache.org/jira/browse/MESOS-471 Project: Mesos Issue Type: Improvement Components: documentation Reporter: Vinod Kone Assignee: Jörn Franke Priority: Trivial -- This message was sent by Atlassian JIRA (v6.2#6252)
Re: Review Request 18381: Added authentication support for slaves.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18381/#review36727 --- Bad patch! Reviews applied: [18381] Failed command: git apply --index 18381.patch Error: error: patch failed: src/slave/flags.hpp:191 error: src/slave/flags.hpp: patch does not apply error: patch failed: src/tests/cluster.hpp:245 error: src/tests/cluster.hpp: patch does not apply - Mesos ReviewBot On March 10, 2014, 6:19 p.m., Adam B wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18381/ --- (Updated March 10, 2014, 6:19 p.m.) Review request for mesos and Vinod Kone. Bugs: MESOS-804 https://issues.apache.org/jira/browse/MESOS-804 Repository: mesos-git Description --- Added authentication support for slaves. Fixes MESOS-804. Open Issues: - Should AuthenticateMessage be replaced with AuthenticateFrameworkMessage, or specify an Authenticatee type as coded here? - removeSlave vs. deactivate(Slave): Some uses of removeSlave might benefit from just deactivating if checkpointing is enabled. - We currently deactivate a registered slave/framework when a new authenticate message comes in, even if the new authentication message is a failure/fake. Will file a new JIRA for this security hole. - When multiple entries for the same principal exist in the credentials file, only the last entry is used. Acceptable behavior, but shouldn't this be documented? Diffs - src/master/flags.hpp 159b2de src/master/master.hpp 49a3e15 src/master/master.cpp f7ba9aa src/messages/messages.proto c26a3d0 src/sasl/authenticatee.hpp 42a4eba src/sasl/common.hpp PRE-CREATION src/sched/sched.cpp 00f6307 src/slave/flags.hpp e4d98a5 src/slave/slave.hpp 01b80df src/slave/slave.cpp b350df4 src/tests/authentication_tests.cpp 127c5e6 src/tests/cluster.hpp d1bf680 src/tests/mesos.cpp 96adeac src/tests/sasl_tests.cpp 945426d src/tests/slave_recovery_tests.cpp 40a9599 Diff: https://reviews.apache.org/r/18381/diff/ Testing --- make check; manually tested flatfile slave authentication success/failure. Added new slave authentication unit tests in authentication_tests.cpp. Thanks, Adam B
Review Request 18988: Improved gate.hpp documentation.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18988/ --- Review request for mesos, Benjamin Hindman, Ben Mahler, and Vinod Kone. Repository: mesos-git Description --- A trivial patch. Added some documentation as I was reading the class hopefully to help libprocess newbies. Diffs - 3rdparty/libprocess/src/gate.hpp 954f6208d4e14b95742e2dc67c564e4ed2e0a96e Diff: https://reviews.apache.org/r/18988/diff/ Testing --- N/A Thanks, Jiang Yan Xu
