Re: Review Request 25079: Replaced macro expansion with variadic template
On Sept. 12, 2014, 7:03 p.m., Ben Mahler wrote: Thanks for doing this! A few higher level comments: (1) We have strings::join in stout. Have you considered implementing this TLineHelper as a generic Joiner in stout/strings.hpp? Seems like it belongs there instead of a newline joiner in this file. Let's also be sure to not expose it in the higher level namespace. (2) We tend to leverage support/post-reviews.py to break changes apart, for example, the addition of the variadic template and the breaking up of help into a .cpp file could have been done in two separate reviews to speed up the review / commit process. We could get the latter change committed quickly! As much as possible, breaking changes apart into a chain of commits will make your life easier. :) (3) From the discussion on MESOS-750, it seems that gcc 4.4 is the minimum supported compiler version. FWICT, this means that within mesos we can certainly assume variadic templates. My hunch is that we don't want to continue writing both C++11/non-C++11 code paths in stout/libprocess due to the complexity and the maintenance burden, but that's a decision we should seek benh's input on. For now, the #ifdef is certainly a safe way to go. Cody Maloney wrote: (1) There are actually a number string joiners throughout the codebase, and lots of unnecessary constant string - std::string - appending. Things like the newline adder add a lot of extra compile time while not actually giving less typing than people inserting '\n' by hand (On top of the runtime behavior doing less unnecessary computation)... (3) I've worked with variadic templates on gcc 4.6 - 4.9, and each until about 4.7 has slight changes in behavior / bugfixes. We really need a bot moving into C++11 land to guarantee that all the compilers we say we support work. Ben Mahler wrote: (1) Good point, we can make strings::join variadic for C++11 and it can do the job for us in help.hpp, to avoid further proliferation of string joiners. :) (3) Ok, let's keep the #ifdef for now! I have submitted r25789 and will modify this patch to use the strings::join provided from there. - Joris --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25079/#review53198 --- On Aug. 28, 2014, 4:24 p.m., Patrick Reilly wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25079/ --- (Updated Aug. 28, 2014, 4:24 p.m.) Review request for mesos, Adam B and Benjamin Hindman. Bugs: MESOS-1734 https://issues.apache.org/jira/browse/MESOS-1734 Repository: mesos-git Description --- Reduce compile time: - replacing a macro expansion with a variadic template - moving implementation from help.hpp to help.cpp Diffs - 3rdparty/libprocess/Makefile.am edbe54b 3rdparty/libprocess/include/process/help.hpp 4333b5b 3rdparty/libprocess/src/help.cpp PRE-CREATION Diff: https://reviews.apache.org/r/25079/diff/ Testing --- Ran make check. Thanks, Patrick Reilly
Re: Review Request 25789: Variadic strings join for c++11 and above
On Sept. 18, 2014, 8:11 p.m., Dominic Hamon wrote: 3rdparty/libprocess/3rdparty/stout/include/stout/strings.hpp, line 190 https://reviews.apache.org/r/25789/diff/1/?file=693848#file693848line190 given that we test for variadic template support in configure, do you think we still need this? I'm on board with not checking this anymore as long as everyone is on board with that. - Joris --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25789/#review53862 --- On Sept. 18, 2014, 7:58 p.m., Joris Van Remoortere wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25789/ --- (Updated Sept. 18, 2014, 7:58 p.m.) Review request for mesos and Benjamin Hindman. Repository: mesos-git Description --- Add Variadic strings join for c++11 and above. There is a second version of the variadic join which takes a reference to a stringstream as a parameter. This is handy when strings::join is just a part of a larger string manipulation. Diffs - 3rdparty/libprocess/3rdparty/stout/include/stout/strings.hpp a1702cd 3rdparty/libprocess/3rdparty/stout/tests/strings_tests.cpp 51008e5 Diff: https://reviews.apache.org/r/25789/diff/ Testing --- Ran make check for stout. Added test cases for join as these were missing. Thanks, Joris Van Remoortere
Re: Review Request 25789: Variadic strings join for c++11 and above
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25789/ --- (Updated Sept. 18, 2014, 10:22 p.m.) Review request for mesos and Benjamin Hindman. Changes --- Deal with a prototype collision between the Iterable Join and variadic join by requiring multiple arguments. Refactor the helper class pattern to basic function recursion within a helper namespace. Repository: mesos-git Description --- Add Variadic strings join for c++11 and above. There is a second version of the variadic join which takes a reference to a stringstream as a parameter. This is handy when strings::join is just a part of a larger string manipulation. Diffs (updated) - 3rdparty/libprocess/3rdparty/stout/include/stout/strings.hpp a1702cd 3rdparty/libprocess/3rdparty/stout/tests/strings_tests.cpp 51008e5 Diff: https://reviews.apache.org/r/25789/diff/ Testing --- Ran make check for stout. Added test cases for join as these were missing. Thanks, Joris Van Remoortere
Review Request 25798: Refactor Libprocess Help to reduce compile time.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25798/ --- Review request for mesos, Benjamin Hindman and Niklas Nielsen. Repository: mesos-git Description --- Follow up from 25079. Reduces compile time overall by ~1/3rd. Move implementation of libprocess help into cpp file. Use strings::join to refactor macro expansion that was injecting lots of overhead even when the expanded functions were not used. Diffs - 3rdparty/libprocess/Makefile.am edbe54b 3rdparty/libprocess/include/process/help.hpp 4333b5b 3rdparty/libprocess/src/help.cpp PRE-CREATION Diff: https://reviews.apache.org/r/25798/diff/ Testing --- make check on 3rdparty support/mesos-style.py Thanks, Joris Van Remoortere
Re: Review Request 25789: Variadic strings join for c++11 and above
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25789/#review53891 --- It seems we have mixed use of const char * and std::string in the usage of join. Dealing with this and writing test for this now. - Joris Van Remoortere On Sept. 18, 2014, 10:22 p.m., Joris Van Remoortere wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25789/ --- (Updated Sept. 18, 2014, 10:22 p.m.) Review request for mesos and Benjamin Hindman. Repository: mesos-git Description --- Add Variadic strings join for c++11 and above. There is a second version of the variadic join which takes a reference to a stringstream as a parameter. This is handy when strings::join is just a part of a larger string manipulation. Diffs - 3rdparty/libprocess/3rdparty/stout/include/stout/strings.hpp a1702cd 3rdparty/libprocess/3rdparty/stout/tests/strings_tests.cpp 51008e5 Diff: https://reviews.apache.org/r/25789/diff/ Testing --- Ran make check for stout. Added test cases for join as these were missing. Thanks, Joris Van Remoortere
Re: Review Request 25789: Variadic strings join for c++11 and above
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25789/ --- (Updated Sept. 18, 2014, 11:10 p.m.) Review request for mesos and Benjamin Hindman. Changes --- Dealt with mixed use between (const char *) and (std::string) in a single join() call. Accompanying unit test. Repository: mesos-git Description --- Add Variadic strings join for c++11 and above. There is a second version of the variadic join which takes a reference to a stringstream as a parameter. This is handy when strings::join is just a part of a larger string manipulation. Diffs (updated) - 3rdparty/libprocess/3rdparty/stout/include/stout/strings.hpp a1702cd 3rdparty/libprocess/3rdparty/stout/tests/strings_tests.cpp 51008e5 Diff: https://reviews.apache.org/r/25789/diff/ Testing --- Ran make check for stout. Added test cases for join as these were missing. Thanks, Joris Van Remoortere
Re: Review Request 25798: Refactor Libprocess Help to reduce compile time.
On Sept. 18, 2014, 11:14 p.m., Niklas Nielsen wrote: 3rdparty/libprocess/include/process/help.hpp, line 120 https://reviews.apache.org/r/25798/diff/1/?file=693978#file693978line120 std::endl? Cody Maloney wrote: std::endl is a flush + '\n'. Note the whole codebase only uses '\n' newlines, so this works well for now. If we wanted to support changing line endings based on platform, there would be a lot more infrastructure needed. std::endl is actually a manipulator that also implied flushing. It can not be passed as the delimiter to join, so the line would look inconsistent: strings::join(ss, \n, std::forwardT(args)...) std::endl; - Joris --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25798/#review53889 --- On Sept. 18, 2014, 10:53 p.m., Joris Van Remoortere wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25798/ --- (Updated Sept. 18, 2014, 10:53 p.m.) Review request for mesos, Benjamin Hindman and Niklas Nielsen. Repository: mesos-git Description --- Follow up from 25079. Reduces compile time overall by ~1/3rd. Move implementation of libprocess help into cpp file. Use strings::join to refactor macro expansion that was injecting lots of overhead even when the expanded functions were not used. Diffs - 3rdparty/libprocess/Makefile.am edbe54b 3rdparty/libprocess/include/process/help.hpp 4333b5b 3rdparty/libprocess/src/help.cpp PRE-CREATION Diff: https://reviews.apache.org/r/25798/diff/ Testing --- make check on 3rdparty support/mesos-style.py Thanks, Joris Van Remoortere
Re: Review Request 25798: Refactor Libprocess Help to reduce compile time.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25798/ --- (Updated Sept. 19, 2014, 12:26 a.m.) Review request for mesos, Benjamin Hindman and Niklas Nielsen. Changes --- fixing style issues. Repository: mesos-git Description --- Follow up from 25079. Reduces compile time overall by ~1/3rd. Move implementation of libprocess help into cpp file. Use strings::join to refactor macro expansion that was injecting lots of overhead even when the expanded functions were not used. Diffs (updated) - 3rdparty/libprocess/Makefile.am edbe54b 3rdparty/libprocess/include/process/help.hpp 4333b5b 3rdparty/libprocess/src/help.cpp PRE-CREATION Diff: https://reviews.apache.org/r/25798/diff/ Testing --- make check on 3rdparty support/mesos-style.py Thanks, Joris Van Remoortere
Re: Review Request 25789: Variadic strings join for c++11 and above
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25789/ --- (Updated Sept. 19, 2014, 12:36 a.m.) Review request for mesos and Benjamin Hindman. Changes --- remove explicit inlines. Fix style issues. Repository: mesos-git Description --- Add Variadic strings join for c++11 and above. There is a second version of the variadic join which takes a reference to a stringstream as a parameter. This is handy when strings::join is just a part of a larger string manipulation. Diffs (updated) - 3rdparty/libprocess/3rdparty/stout/include/stout/strings.hpp a1702cd 3rdparty/libprocess/3rdparty/stout/tests/strings_tests.cpp 51008e5 Diff: https://reviews.apache.org/r/25789/diff/ Testing --- Ran make check for stout. Added test cases for join as these were missing. Thanks, Joris Van Remoortere
Re: Review Request 25789: Variadic strings join for c++11 and above
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25789/ --- (Updated Sept. 19, 2014, 6:37 p.m.) Review request for mesos and Benjamin Hindman. Changes --- Add overloads of an internal::append function to support mixed type strings::join(). There are special overloads for string ands const char * to maintain performance. Added a test for mixed type strings::join(). Fixed style issues. Repository: mesos-git Description --- Add Variadic strings join for c++11 and above. There is a second version of the variadic join which takes a reference to a stringstream as a parameter. This is handy when strings::join is just a part of a larger string manipulation. Diffs (updated) - 3rdparty/libprocess/3rdparty/stout/include/stout/strings.hpp a1702cd 3rdparty/libprocess/3rdparty/stout/tests/strings_tests.cpp 51008e5 Diff: https://reviews.apache.org/r/25789/diff/ Testing --- Ran make check for stout. Added test cases for join as these were missing. Thanks, Joris Van Remoortere
Re: Review Request 25789: Variadic strings join for c++11 and above
On Sept. 19, 2014, 10:25 a.m., Benjamin Hindman wrote: 3rdparty/libprocess/3rdparty/stout/include/stout/strings.hpp, line 205 https://reviews.apache.org/r/25789/diff/4/?file=694225#file694225line205 At first I was expecting strings::join to just be variadic on std::string (like the original strings::join functions). We have a 'stringify' operation that we use which ultimately just uses operator , will we get the same result with the std::string() conversion as we do with stringify? Irregardless, let's make sure that we have tests which are joining more than just strings if we expect to get more than just strings, and that the semantics are expected with stringify! I made the variadic strings::join() more generic in that it now uses a helper function called append(). This in the general case falls back to stringify, but binds first to special versions for string and const char * that have low overhead. - Joris --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25789/#review53949 --- On Sept. 19, 2014, 6:37 p.m., Joris Van Remoortere wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25789/ --- (Updated Sept. 19, 2014, 6:37 p.m.) Review request for mesos and Benjamin Hindman. Repository: mesos-git Description --- Add Variadic strings join for c++11 and above. There is a second version of the variadic join which takes a reference to a stringstream as a parameter. This is handy when strings::join is just a part of a larger string manipulation. Diffs - 3rdparty/libprocess/3rdparty/stout/include/stout/strings.hpp a1702cd 3rdparty/libprocess/3rdparty/stout/tests/strings_tests.cpp 51008e5 Diff: https://reviews.apache.org/r/25789/diff/ Testing --- Ran make check for stout. Added test cases for join as these were missing. Thanks, Joris Van Remoortere
Re: Review Request 25798: Refactor Libprocess Help to reduce compile time.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25798/ --- (Updated Sept. 19, 2014, 7:01 p.m.) Review request for mesos, Benjamin Hindman and Niklas Nielsen. Changes --- Simplify strings::join() invocation. Repository: mesos-git Description --- Follow up from 25079. Reduces compile time overall by ~1/3rd. Move implementation of libprocess help into cpp file. Use strings::join to refactor macro expansion that was injecting lots of overhead even when the expanded functions were not used. Diffs (updated) - 3rdparty/libprocess/Makefile.am edbe54b 3rdparty/libprocess/include/process/help.hpp 4333b5b 3rdparty/libprocess/src/help.cpp PRE-CREATION Diff: https://reviews.apache.org/r/25798/diff/ Testing --- make check on 3rdparty support/mesos-style.py Thanks, Joris Van Remoortere
Review Request 25847: Refactor Libprocess: class Node
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25847/ --- Review request for mesos and Niklas Nielsen. Repository: mesos-git Description --- Move class Node out of process.cpp and into its own header. Part of refactoring process.cpp. Diffs - 3rdparty/libprocess/include/Makefile.am 09f6e41 3rdparty/libprocess/include/process/node.hpp PRE-CREATION 3rdparty/libprocess/src/process.cpp 3ac56c7 Diff: https://reviews.apache.org/r/25847/diff/ Testing --- make check in libprocess support/mesos-stype.py Thanks, Joris Van Remoortere
Re: Review Request 25847: Refactor Libprocess: class Node
On Sept. 19, 2014, 8:32 p.m., Dominic Hamon wrote: 3rdparty/libprocess/include/process/node.hpp, line 32 https://reviews.apache.org/r/25847/diff/1/?file=697014#file697014line32 uint64_t? start getting ipv6 support in... ;) Nikita Vetoshkin wrote: Still won't fit in ipv6 though :( We can use code__uint128_t/code ; however, I am not sure if this is supported accross all our required compilers. The larger issue is that the uint32_t type is currently baked into the protobufs, so this would be a larger compatibility change. - Joris --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25847/#review54018 --- On Sept. 19, 2014, 8:31 p.m., Joris Van Remoortere wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25847/ --- (Updated Sept. 19, 2014, 8:31 p.m.) Review request for mesos and Niklas Nielsen. Repository: mesos-git Description --- Move class Node out of process.cpp and into its own header. Part of refactoring process.cpp. Diffs - 3rdparty/libprocess/include/Makefile.am 09f6e41 3rdparty/libprocess/include/process/node.hpp PRE-CREATION 3rdparty/libprocess/src/process.cpp 3ac56c7 Diff: https://reviews.apache.org/r/25847/diff/ Testing --- make check in libprocess support/mesos-stype.py Thanks, Joris Van Remoortere
Re: Review Request 25847: Refactor Libprocess: class Node
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25847/ --- (Updated Sept. 19, 2014, 10:19 p.m.) Review request for mesos and Niklas Nielsen. Changes --- fix style issue. Repository: mesos-git Description --- Move class Node out of process.cpp and into its own header. Part of refactoring process.cpp. Diffs (updated) - 3rdparty/libprocess/include/Makefile.am 09f6e41 3rdparty/libprocess/include/process/node.hpp PRE-CREATION 3rdparty/libprocess/src/process.cpp 3ac56c7 Diff: https://reviews.apache.org/r/25847/diff/ Testing --- make check in libprocess support/mesos-stype.py Thanks, Joris Van Remoortere
Review Request 25868: Refactor Libprocess: class ProcessReference
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25868/ --- Review request for mesos and Niklas Nielsen. Repository: mesos-git Description --- Move class ProcessReference out of process.cpp and into its own header. Part of refactoring process.cpp. Diffs - 3rdparty/libprocess/include/Makefile.am 542ae1c 3rdparty/libprocess/include/process/process_reference.hpp PRE-CREATION 3rdparty/libprocess/src/process.cpp 8adc809 Diff: https://reviews.apache.org/r/25868/diff/ Testing --- make check in 3rdparty/libprocess support/mesos-style.py Thanks, Joris Van Remoortere
Re: Review Request 25868: Refactor Libprocess: class ProcessReference
On Sept. 20, 2014, 2:24 p.m., Dominic Hamon wrote: 3rdparty/libprocess/include/process/process_reference.hpp, line 9 https://reviews.apache.org/r/25868/diff/1/?file=698470#file698470line9 i wonder if we can still need this at all. would just having a std::shared_ptrSomeProcess cover the use-cases? I looked at this before moving it out. std::shared_ptr is meants to act in an RAII fashion, with the option to define special destructor (0 ref count) behavior. In the case of ProcessReference there seems to be specific management going on around termination of processes. for example a spin loop waiting for a 0 ref count. Right now I am trying to clean up / refactor things out of process.cpp in small patches in order to simplify my proposal for MESOS-1330. I can revisit whether we can replace this with shared_ptr once I have isolated the event management system. - Joris --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25868/#review54077 --- On Sept. 20, 2014, 12:58 a.m., Joris Van Remoortere wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25868/ --- (Updated Sept. 20, 2014, 12:58 a.m.) Review request for mesos and Niklas Nielsen. Repository: mesos-git Description --- Move class ProcessReference out of process.cpp and into its own header. Part of refactoring process.cpp. Diffs - 3rdparty/libprocess/include/Makefile.am 542ae1c 3rdparty/libprocess/include/process/process_reference.hpp PRE-CREATION 3rdparty/libprocess/src/process.cpp 8adc809 Diff: https://reviews.apache.org/r/25868/diff/ Testing --- make check in 3rdparty/libprocess support/mesos-style.py Thanks, Joris Van Remoortere
Re: Review Request 25789: Variadic strings join
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25789/ --- (Updated Sept. 22, 2014, 5:30 p.m.) Review request for mesos and Benjamin Hindman. Changes --- Remove the for c++11 and above comments since variadic templates are now part of the configuration check. Summary (updated) - Variadic strings join Repository: mesos-git Description (updated) --- Add Variadic strings join. There is a second version of the variadic join which takes a reference to a stringstream as a parameter. This is handy when strings::join is just a part of a larger string manipulation. Diffs - 3rdparty/libprocess/3rdparty/stout/include/stout/strings.hpp a1702cd 3rdparty/libprocess/3rdparty/stout/tests/strings_tests.cpp 51008e5 Diff: https://reviews.apache.org/r/25789/diff/ Testing --- Ran make check for stout. Added test cases for join as these were missing. Thanks, Joris Van Remoortere
Re: Review Request 25868: Refactor Libprocess: class ProcessReference
On Sept. 20, 2014, 7:22 p.m., Ben Mahler wrote: Thanks Joris! Any reason you want this exposed in the public include/ folder of libprocess, as opposed to an internal header inside src/? Don't think we'd want this in the public includes. Niklas Nielsen wrote: It seems to be a bit unconsistent whether things have gone in include/ or not. For example, the socket abstraction is only used by libprocess internally and is in include/. The node class ended up there too (my / our fault) - so if we decide to move the private headers to src/, we need to move that alongside doing a scan for the headers we only use inside libprocess. Hi Ben, I am refactoring certain classes from process.cpp into the include directory because my goal is to allow pluggable implementations (modules) of the event-management abstraction. My thought process is that we should have the classes required to implement this abstraction available in the public include dir. Since this is an open source project, it's not impossible for us to have some of these class declarations / definitions in the src directory, and the module implementer to be required to build it with the src tree pulled. I am in the process of making a branch available as a prototype for MESOS-1330; which will make it more apparent where I am going. What are your thoughts on the rules behind the public include dir, and what requirements we can impose on module implementers? - Joris --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25868/#review54086 --- On Sept. 20, 2014, 12:58 a.m., Joris Van Remoortere wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25868/ --- (Updated Sept. 20, 2014, 12:58 a.m.) Review request for mesos and Niklas Nielsen. Repository: mesos-git Description --- Move class ProcessReference out of process.cpp and into its own header. Part of refactoring process.cpp. Diffs - 3rdparty/libprocess/include/Makefile.am 542ae1c 3rdparty/libprocess/include/process/process_reference.hpp PRE-CREATION 3rdparty/libprocess/src/process.cpp 8adc809 Diff: https://reviews.apache.org/r/25868/diff/ Testing --- make check in 3rdparty/libprocess support/mesos-style.py Thanks, Joris Van Remoortere
Re: Review Request 25868: Refactor Libprocess: class ProcessReference
On Sept. 20, 2014, 7:22 p.m., Ben Mahler wrote: Thanks Joris! Any reason you want this exposed in the public include/ folder of libprocess, as opposed to an internal header inside src/? Don't think we'd want this in the public includes. Niklas Nielsen wrote: It seems to be a bit unconsistent whether things have gone in include/ or not. For example, the socket abstraction is only used by libprocess internally and is in include/. The node class ended up there too (my / our fault) - so if we decide to move the private headers to src/, we need to move that alongside doing a scan for the headers we only use inside libprocess. Joris Van Remoortere wrote: Hi Ben, I am refactoring certain classes from process.cpp into the include directory because my goal is to allow pluggable implementations (modules) of the event-management abstraction. My thought process is that we should have the classes required to implement this abstraction available in the public include dir. Since this is an open source project, it's not impossible for us to have some of these class declarations / definitions in the src directory, and the module implementer to be required to build it with the src tree pulled. I am in the process of making a branch available as a prototype for MESOS-1330; which will make it more apparent where I am going. What are your thoughts on the rules behind the public include dir, and what requirements we can impose on module implementers? Ben Mahler wrote: For the public include directory, if the abstraction is generally useful (e.g. Node, Socket is less clearly useful but looks ok) then it's fine to include these in the public headers. `ProcessReference` however, is not something that the library user can leverage, and it's dangerous to expose: use of it can lead to ProcessManager::cleanup spinning forever! _My thought process is that we should have the classes required to implement this abstraction available in the public include dir._ Are you interested in applying the same philosophy to the mesos code base? That would grow the public headers substantially. _Since this is an open source project, it's not impossible for us to have some of these class declarations / definitions in the src directory, and the module implementer to be required to build it with the src tree pulled._ Seems like a reasonable thing for module implementers IMHO, otherwise we may have to expose a lot of internals in the public include folder of both libprocess and mesos. Thanks for your feedback Ben. I'm fine with requiring module implementers to pull the src; I just did not want to force the issue without a discussion. If everyone else is ok with this then I will keep things in the src dirs. - Joris --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25868/#review54086 --- On Sept. 20, 2014, 12:58 a.m., Joris Van Remoortere wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25868/ --- (Updated Sept. 20, 2014, 12:58 a.m.) Review request for mesos and Niklas Nielsen. Repository: mesos-git Description --- Move class ProcessReference out of process.cpp and into its own header. Part of refactoring process.cpp. Diffs - 3rdparty/libprocess/include/Makefile.am 542ae1c 3rdparty/libprocess/include/process/process_reference.hpp PRE-CREATION 3rdparty/libprocess/src/process.cpp 8adc809 Diff: https://reviews.apache.org/r/25868/diff/ Testing --- make check in 3rdparty/libprocess support/mesos-style.py Thanks, Joris Van Remoortere
Re: Review Request 25868: Refactor Libprocess: class ProcessReference
On Sept. 20, 2014, 7:22 p.m., Ben Mahler wrote: Thanks Joris! Any reason you want this exposed in the public include/ folder of libprocess, as opposed to an internal header inside src/? Don't think we'd want this in the public includes. Niklas Nielsen wrote: It seems to be a bit unconsistent whether things have gone in include/ or not. For example, the socket abstraction is only used by libprocess internally and is in include/. The node class ended up there too (my / our fault) - so if we decide to move the private headers to src/, we need to move that alongside doing a scan for the headers we only use inside libprocess. Joris Van Remoortere wrote: Hi Ben, I am refactoring certain classes from process.cpp into the include directory because my goal is to allow pluggable implementations (modules) of the event-management abstraction. My thought process is that we should have the classes required to implement this abstraction available in the public include dir. Since this is an open source project, it's not impossible for us to have some of these class declarations / definitions in the src directory, and the module implementer to be required to build it with the src tree pulled. I am in the process of making a branch available as a prototype for MESOS-1330; which will make it more apparent where I am going. What are your thoughts on the rules behind the public include dir, and what requirements we can impose on module implementers? Ben Mahler wrote: For the public include directory, if the abstraction is generally useful (e.g. Node, Socket is less clearly useful but looks ok) then it's fine to include these in the public headers. `ProcessReference` however, is not something that the library user can leverage, and it's dangerous to expose: use of it can lead to ProcessManager::cleanup spinning forever! _My thought process is that we should have the classes required to implement this abstraction available in the public include dir._ Are you interested in applying the same philosophy to the mesos code base? That would grow the public headers substantially. _Since this is an open source project, it's not impossible for us to have some of these class declarations / definitions in the src directory, and the module implementer to be required to build it with the src tree pulled._ Seems like a reasonable thing for module implementers IMHO, otherwise we may have to expose a lot of internals in the public include folder of both libprocess and mesos. Joris Van Remoortere wrote: Thanks for your feedback Ben. I'm fine with requiring module implementers to pull the src; I just did not want to force the issue without a discussion. If everyone else is ok with this then I will keep things in the src dirs. Here is a prototype for the event management refactor I would like to do for MESOS-1330: https://github.com/mesosphere/mesos/tree/abstract-event-manager I will modify this diff to move the process_reference into src as discussed. (I have followed the same pattern in the above prototype). - Joris --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25868/#review54086 --- On Sept. 20, 2014, 12:58 a.m., Joris Van Remoortere wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25868/ --- (Updated Sept. 20, 2014, 12:58 a.m.) Review request for mesos and Niklas Nielsen. Repository: mesos-git Description --- Move class ProcessReference out of process.cpp and into its own header. Part of refactoring process.cpp. Diffs - 3rdparty/libprocess/include/Makefile.am 542ae1c 3rdparty/libprocess/include/process/process_reference.hpp PRE-CREATION 3rdparty/libprocess/src/process.cpp 8adc809 Diff: https://reviews.apache.org/r/25868/diff/ Testing --- make check in 3rdparty/libprocess support/mesos-style.py Thanks, Joris Van Remoortere
Re: Review Request 25868: Refactor Libprocess: class ProcessReference
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25868/ --- (Updated Sept. 23, 2014, 1:18 a.m.) Review request for mesos and Niklas Nielsen. Changes --- Move ProcessReference to src dir from include dir. Repository: mesos-git Description --- Move class ProcessReference out of process.cpp and into its own header. Part of refactoring process.cpp. Diffs (updated) - 3rdparty/libprocess/Makefile.am edbe54b 3rdparty/libprocess/src/process.cpp 8adc809 3rdparty/libprocess/src/process_reference.hpp PRE-CREATION Diff: https://reviews.apache.org/r/25868/diff/ Testing --- make check in 3rdparty/libprocess support/mesos-style.py Thanks, Joris Van Remoortere
Review Request 25922: Explore disk io isolation in cgroups
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25922/ --- Review request for mesos, Adam B and Benjamin Hindman. Repository: mesos-git Description --- Following from: r25105 Currently there is no disk isolation in place and this affects an executor to be starved of disk when another disk heavy operation such as copying a multi gigabyte file is being performed by another executor. Diffs - include/mesos/mesos.proto be45494 include/mesos/resources.hpp 0e37170 src/Makefile.am 9b973e5 src/common/resources.cpp e9a0c85 src/linux/cgroups.hpp abf31df src/linux/cgroups.cpp 5093b4c src/slave/containerizer/isolators/cgroups/blkio.hpp PRE-CREATION src/slave/containerizer/isolators/cgroups/blkio.cpp PRE-CREATION src/slave/containerizer/mesos/containerizer.cpp 9d08329 src/tests/isolator_tests.cpp c38f876 Diff: https://reviews.apache.org/r/25922/diff/ Testing --- make check sudo bin/mesos-tests.sh --gtest_filter=*BlkIO* support/mesos-style.py Thanks, Joris Van Remoortere
Re: Review Request 25789: Variadic strings join
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25789/ --- (Updated Sept. 23, 2014, 5:30 p.m.) Review request for mesos and Benjamin Hindman. Changes --- fix style issues. Repository: mesos-git Description --- Add Variadic strings join. There is a second version of the variadic join which takes a reference to a stringstream as a parameter. This is handy when strings::join is just a part of a larger string manipulation. Diffs (updated) - 3rdparty/libprocess/3rdparty/stout/include/stout/strings.hpp a1702cd 3rdparty/libprocess/3rdparty/stout/tests/strings_tests.cpp 51008e5 Diff: https://reviews.apache.org/r/25789/diff/ Testing --- Ran make check for stout. Added test cases for join as these were missing. Thanks, Joris Van Remoortere
Re: Review Request 25789: Variadic strings join
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25789/ --- (Updated Sept. 25, 2014, 5:10 p.m.) Review request for mesos and Benjamin Hindman. Changes --- fix style issues. Repository: mesos-git Description --- Add Variadic strings join. There is a second version of the variadic join which takes a reference to a stringstream as a parameter. This is handy when strings::join is just a part of a larger string manipulation. Diffs (updated) - 3rdparty/libprocess/3rdparty/stout/include/stout/strings.hpp a1702cd 3rdparty/libprocess/3rdparty/stout/tests/strings_tests.cpp 51008e5 Diff: https://reviews.apache.org/r/25789/diff/ Testing --- Ran make check for stout. Added test cases for join as these were missing. Thanks, Joris Van Remoortere
Re: Review Request 25789: Variadic strings join
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25789/ --- (Updated Sept. 25, 2014, 6:25 p.m.) Review request for mesos and Benjamin Hindman. Changes --- fix style issues. Repository: mesos-git Description --- Add Variadic strings join. There is a second version of the variadic join which takes a reference to a stringstream as a parameter. This is handy when strings::join is just a part of a larger string manipulation. Diffs (updated) - 3rdparty/libprocess/3rdparty/stout/include/stout/strings.hpp a1702cd 3rdparty/libprocess/3rdparty/stout/tests/strings_tests.cpp 51008e5 Diff: https://reviews.apache.org/r/25789/diff/ Testing --- Ran make check for stout. Added test cases for join as these were missing. Thanks, Joris Van Remoortere
Review Request 26056: Prevent multiple definitions of synchronized header.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/26056/ --- Review request for mesos and Niklas Nielsen. Repository: mesos-git Description --- Defend against multiple definitions of synchronized. Add conditional include. Diffs - 3rdparty/libprocess/src/synchronized.hpp 577a5fc Diff: https://reviews.apache.org/r/26056/diff/ Testing --- make check in 3rdparty/libprocess support/mesos-style.py Thanks, Joris Van Remoortere
Review Request 26060: Refactor Libprocess: class HttpProxy
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/26060/ --- Review request for mesos and Niklas Nielsen. Repository: mesos-git Description --- Move http_proxy out of process.cpp, which forces the bare scaffolding for the event_manager abstraction dependency injection. For now: SocketManager - EventManager - internal::EventManager. In this step we only virtualized (raised) the mandatory send() functions. internal::EventManager exists to inject only the functions that http_proxy depends on. Diffs - 3rdparty/libprocess/Makefile.am 616618e 3rdparty/libprocess/src/event_manager.hpp PRE-CREATION 3rdparty/libprocess/src/event_manager_base.hpp PRE-CREATION 3rdparty/libprocess/src/http_proxy.hpp PRE-CREATION 3rdparty/libprocess/src/http_proxy.cpp PRE-CREATION 3rdparty/libprocess/src/process.cpp e46e6b1 Diff: https://reviews.apache.org/r/26060/diff/ Testing --- make check support/mesos-style.py Thanks, Joris Van Remoortere
Re: Review Request 26056: Normalize enforce 3rdparty/libprocess conditional include style.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/26056/ --- (Updated Sept. 25, 2014, 11:04 p.m.) Review request for mesos and Niklas Nielsen. Changes --- Changing from synchronized.hpp specific to normalizing all of 3rdparty/libprocess conditional includes. Summary (updated) - Normalize enforce 3rdparty/libprocess conditional include style. Repository: mesos-git Description (updated) --- Normalize 3rdparty/libprocess conditional include style. Diffs (updated) - 3rdparty/libprocess/include/process/metrics/metrics.hpp 0a97f12 3rdparty/libprocess/include/process/tuples/details.hpp 80b0539 3rdparty/libprocess/src/config.hpp cbaf41d 3rdparty/libprocess/src/fatal.hpp 38646f3 3rdparty/libprocess/src/net.hpp 1ba3dbb 3rdparty/libprocess/src/process_reference.hpp 4964107 3rdparty/libprocess/src/synchronized.hpp 577a5fc Diff: https://reviews.apache.org/r/26056/diff/ Testing --- make check in 3rdparty/libprocess support/mesos-style.py Thanks, Joris Van Remoortere
Review Request 26150: Libprocess Benchmark
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/26150/ --- Review request for mesos and Niklas Nielsen. Bugs: MESOS-1840 https://issues.apache.org/jira/browse/MESOS-1840 Repository: mesos-git Description --- A benchmark for libprocess. It forks num_proc times and launched num_threads libprocess processes in each child. They are aware of the master's (parent) address and all play ping pong with it. This benchmark measures throughput in terms of the number of RPCs handled per second using persistent (linked) connections. A new test file (benchmarks) is introduced because we want to fork before libprocess is initialized. This allows us to prevent short-circuiting of message passing between processes under the same ProcessManager. This way we force the execution path of the underlying event management system. Diffs - 3rdparty/libprocess/Makefile.am 616618e 3rdparty/libprocess/src/tests/benchmarks.cpp PRE-CREATION Diff: https://reviews.apache.org/r/26150/diff/ Testing --- make check in 3rdparty/libprocess support/mesos-style.py Thanks, Joris Van Remoortere
Review Request 26476: Remove dynamic allocation from Option.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/26476/ --- Review request for mesos, Benjamin Hindman and Niklas Nielsen. Repository: mesos-git Description --- Remove dynamic allocations from Option class. Diffs - 3rdparty/libprocess/3rdparty/stout/include/stout/option.hpp 47fe92c Diff: https://reviews.apache.org/r/26476/diff/ Testing --- make check support/mesos-style.py valgrind (reduced allocation count) Thanks, Joris Van Remoortere
Re: Review Request 26476: Remove dynamic allocation from Option.
On Oct. 9, 2014, 3:45 a.m., Dominic Hamon wrote: 3rdparty/libprocess/3rdparty/stout/include/stout/option.hpp, line 131 https://reviews.apache.org/r/26476/diff/1/?file=716336#file716336line131 This makes Option arbitrarily large which could be problematic where we copy it (we can't assume move semantics). I don't understand the benefit of this change. We have so many dynamic allocations throughout the code-base, it seems like a strange place to focus attention. In the original implementation of Option, a large T would still be deep copied; it would just be done on the heap. To avoid large copies one should use a reference counted abstraction such as shared_ptr (e.g. Optionstd::shared_ptrT or std::shared_ptrOptionT). Option is meant to augment a type with 1 extra bit of (nullable / unknownable, whichever you prefer) state. Tackling Option is one way of reducing a significant number of dynamic allocations as it is a heavily used library. Option is also something that is commonly assumed to be a light-weight abstraction; so it is less of a surprise if it doesn't have an underlying dynamic allocation. - Joris --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/26476/#review55945 --- On Oct. 9, 2014, 1:35 a.m., Joris Van Remoortere wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/26476/ --- (Updated Oct. 9, 2014, 1:35 a.m.) Review request for mesos, Benjamin Hindman and Niklas Nielsen. Repository: mesos-git Description --- Remove dynamic allocations from Option class. Diffs - 3rdparty/libprocess/3rdparty/stout/include/stout/option.hpp 47fe92c Diff: https://reviews.apache.org/r/26476/diff/ Testing --- make check support/mesos-style.py valgrind (reduced allocation count) Thanks, Joris Van Remoortere
Re: Review Request 26476: Remove dynamic allocation from Option.
On Oct. 9, 2014, 4:41 a.m., Jie Yu wrote: Flying by. You may wanna take a look at: https://github.com/facebook/folly/blob/master/folly/Optional.h Not sure if we can use unstricted union? Does g++44 supports that? According to https://gcc.gnu.org/projects/cxx0x.html unrestricted unions are supported from gcc4.6+. In-place new is an old c-ism. - Joris --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/26476/#review55951 --- On Oct. 9, 2014, 1:35 a.m., Joris Van Remoortere wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/26476/ --- (Updated Oct. 9, 2014, 1:35 a.m.) Review request for mesos, Benjamin Hindman and Niklas Nielsen. Repository: mesos-git Description --- Remove dynamic allocations from Option class. Diffs - 3rdparty/libprocess/3rdparty/stout/include/stout/option.hpp 47fe92c Diff: https://reviews.apache.org/r/26476/diff/ Testing --- make check support/mesos-style.py valgrind (reduced allocation count) Thanks, Joris Van Remoortere
Review Request 26533: Memory cleanup: libprocess finalize
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/26533/ --- Review request for mesos, Benjamin Hindman and Niklas Nielsen. Repository: mesos-git Description --- Introduce a finalize() function to match the initialize() function in process.hpp. Use this at the unit test runner for libprocess to start getting more valuable valgrind feedback. - Added cleanup of remaining Processes in ~ProcessManager() as a start. Diffs - 3rdparty/libprocess/include/process/process.hpp 270ca28 3rdparty/libprocess/src/process.cpp d30ed63 3rdparty/libprocess/src/tests/main.cpp 0a3ba4e Diff: https://reviews.apache.org/r/26533/diff/ Testing --- make check support/mesos-style valgrind 3rdparty/libprocess/tests (to see a reduction in definitely lost blocks) Thanks, Joris Van Remoortere
Re: Review Request 26533: Memory cleanup: libprocess finalize
On Oct. 10, 2014, 5:04 p.m., Dominic Hamon wrote: 3rdparty/libprocess/src/process.cpp, line 1453 https://reviews.apache.org/r/26533/diff/1/?file=717191#file717191line1453 should these be std::unique_ptr instead? (i'm currently adding support for this to the libprocess configure check). Indeed. I have some follow up patches that will try to tackle the actual clean shutdown. This initial patch it to try and introduce a place to close the loop. - Joris --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/26533/#review56166 --- On Oct. 10, 2014, 1:02 a.m., Joris Van Remoortere wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/26533/ --- (Updated Oct. 10, 2014, 1:02 a.m.) Review request for mesos, Benjamin Hindman and Niklas Nielsen. Repository: mesos-git Description --- Introduce a finalize() function to match the initialize() function in process.hpp. Use this at the unit test runner for libprocess to start getting more valuable valgrind feedback. - Added cleanup of remaining Processes in ~ProcessManager() as a start. Diffs - 3rdparty/libprocess/include/process/process.hpp 270ca28 3rdparty/libprocess/src/process.cpp d30ed63 3rdparty/libprocess/src/tests/main.cpp 0a3ba4e Diff: https://reviews.apache.org/r/26533/diff/ Testing --- make check support/mesos-style valgrind 3rdparty/libprocess/tests (to see a reduction in definitely lost blocks) Thanks, Joris Van Remoortere
Re: Review Request 26533: Memory cleanup: libprocess finalize
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/26533/ --- (Updated Oct. 10, 2014, 9:39 p.m.) Review request for mesos, Benjamin Hindman and Niklas Nielsen. Changes --- fix style issues. Repository: mesos-git Description --- Introduce a finalize() function to match the initialize() function in process.hpp. Use this at the unit test runner for libprocess to start getting more valuable valgrind feedback. - Added cleanup of remaining Processes in ~ProcessManager() as a start. Diffs (updated) - 3rdparty/libprocess/include/process/process.hpp 270ca28 3rdparty/libprocess/src/process.cpp d30ed63 3rdparty/libprocess/src/tests/main.cpp 0a3ba4e Diff: https://reviews.apache.org/r/26533/diff/ Testing --- make check support/mesos-style valgrind 3rdparty/libprocess/tests (to see a reduction in definitely lost blocks) Thanks, Joris Van Remoortere
Review Request 26578: Memory cleanup: libprocess gc finalize
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/26578/ --- Review request for mesos, Benjamin Hindman and Niklas Nielsen. Repository: mesos-git Description --- Implement the finalize() virtual override for Libprocess' garbage collector. This terminates all processes registered with the garbage collector when the garbage collector terminates. Diffs - 3rdparty/libprocess/include/process/gc.hpp e83c636 Diff: https://reviews.apache.org/r/26578/diff/ Testing --- make check support/mesos-style.py Thanks, Joris Van Remoortere
Review Request 26580: Memory cleanup: libprocess join worker threads
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/26580/ --- Review request for mesos, Benjamin Hindman and Niklas Nielsen. Repository: mesos-git Description --- During process::finalize() join on the worker threads that were dispatched, after the gc has had a chance to terminate. Diffs - 3rdparty/libprocess/src/process.cpp 85fb995 Diff: https://reviews.apache.org/r/26580/diff/ Testing --- make check support/mesos-style.py Thanks, Joris Van Remoortere
Review Request 26583: Memory cleanup: libprocess delete garbage collector process
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/26583/ --- Review request for mesos, Benjamin Hindman and Niklas Nielsen. Repository: mesos-git Description --- Wrap the libprocess garbage collector process with a unique ptr. Reset the ptr during finalization. Diffs - 3rdparty/libprocess/src/process.cpp 85fb995 Diff: https://reviews.apache.org/r/26583/diff/ Testing --- make check Thanks, Joris Van Remoortere
Re: Review Request 26580: Memory cleanup: libprocess join worker threads
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/26580/ --- (Updated Oct. 10, 2014, 11:38 p.m.) Review request for mesos, Benjamin Hindman and Niklas Nielsen. Changes --- Introduce finalize() on ProcessManager so that we can terminate outstanding processes without destroying our runqueue. Repository: mesos-git Description (updated) --- Terminate the gc; Finalize the ProcessManager (picking up -idealy 0- left-over processes); Join on the worker threads that were dispatched. Diffs (updated) - 3rdparty/libprocess/src/process.cpp 85fb995 Diff: https://reviews.apache.org/r/26580/diff/ Testing --- make check support/mesos-style.py Thanks, Joris Van Remoortere
Re: Review Request 26583: Memory cleanup: libprocess delete garbage collector process
On Oct. 11, 2014, 12:15 a.m., Dominic Hamon wrote: 3rdparty/libprocess/src/process.cpp, line 1635 https://reviews.apache.org/r/26583/diff/1/?file=717970#file717970line1635 should do the same for this while you're here. Since help is covered by the garbage collector it doesn't make sense to wrap it with a unique_ptr; otherwise we would have to .release() it to prevent a double free. - Joris --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/26583/#review56262 --- On Oct. 10, 2014, 11:12 p.m., Joris Van Remoortere wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/26583/ --- (Updated Oct. 10, 2014, 11:12 p.m.) Review request for mesos, Benjamin Hindman and Niklas Nielsen. Repository: mesos-git Description --- Wrap the libprocess garbage collector process with a unique ptr. Reset the ptr during finalization. Diffs - 3rdparty/libprocess/src/process.cpp 85fb995 Diff: https://reviews.apache.org/r/26583/diff/ Testing --- make check Thanks, Joris Van Remoortere
Re: Review Request 26578: Memory cleanup: libprocess gc finalize
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/26578/ --- (Updated Oct. 13, 2014, 5:29 p.m.) Review request for mesos, Benjamin Hindman and Niklas Nielsen. Changes --- Use std::unqiue_ptr in libprocess gc. Rewrite exited() logic to use find() erase to reduce map lookups. Repository: mesos-git Description --- Implement the finalize() virtual override for Libprocess' garbage collector. This terminates all processes registered with the garbage collector when the garbage collector terminates. Diffs (updated) - 3rdparty/libprocess/include/process/gc.hpp e83c636 Diff: https://reviews.apache.org/r/26578/diff/ Testing --- make check support/mesos-style.py Thanks, Joris Van Remoortere
Re: Review Request 26580: Memory cleanup: libprocess join worker threads
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/26580/ --- (Updated Oct. 13, 2014, 6:10 p.m.) Review request for mesos, Benjamin Hindman and Niklas Nielsen. Changes --- using std::unique_ptr for list of worker pthreads. Repository: mesos-git Description --- Terminate the gc; Finalize the ProcessManager (picking up -idealy 0- left-over processes); Join on the worker threads that were dispatched. Diffs (updated) - 3rdparty/libprocess/src/process.cpp 85fb995 Diff: https://reviews.apache.org/r/26580/diff/ Testing --- make check support/mesos-style.py Thanks, Joris Van Remoortere
Re: Review Request 26150: Libprocess Benchmark
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/26150/ --- (Updated Oct. 13, 2014, 7:52 p.m.) Review request for mesos and Niklas Nielsen. Changes --- Fix style issues. Back-port some c++11. Bugs: MESOS-1840 https://issues.apache.org/jira/browse/MESOS-1840 Repository: mesos-git Description --- A benchmark for libprocess. It forks num_proc times and launched num_threads libprocess processes in each child. They are aware of the master's (parent) address and all play ping pong with it. This benchmark measures throughput in terms of the number of RPCs handled per second using persistent (linked) connections. A new test file (benchmarks) is introduced because we want to fork before libprocess is initialized. This allows us to prevent short-circuiting of message passing between processes under the same ProcessManager. This way we force the execution path of the underlying event management system. Diffs (updated) - 3rdparty/libprocess/Makefile.am 616618e 3rdparty/libprocess/src/tests/benchmarks.cpp PRE-CREATION Diff: https://reviews.apache.org/r/26150/diff/ Testing --- make check in 3rdparty/libprocess support/mesos-style.py Thanks, Joris Van Remoortere
Re: Review Request 26150: Libprocess Benchmark
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/26150/ --- (Updated Oct. 14, 2014, 9:27 p.m.) Review request for mesos and Niklas Nielsen. Changes --- Swap out std::thread, std::mutex, std::condition_variable from a simpler Actor / Latch approach. Fix style issues. Bugs: MESOS-1840 https://issues.apache.org/jira/browse/MESOS-1840 Repository: mesos-git Description --- A benchmark for libprocess. It forks num_proc times and launched num_threads libprocess processes in each child. They are aware of the master's (parent) address and all play ping pong with it. This benchmark measures throughput in terms of the number of RPCs handled per second using persistent (linked) connections. A new test file (benchmarks) is introduced because we want to fork before libprocess is initialized. This allows us to prevent short-circuiting of message passing between processes under the same ProcessManager. This way we force the execution path of the underlying event management system. Diffs (updated) - 3rdparty/libprocess/Makefile.am 616618e 3rdparty/libprocess/src/tests/benchmarks.cpp PRE-CREATION Diff: https://reviews.apache.org/r/26150/diff/ Testing --- make check in 3rdparty/libprocess support/mesos-style.py Thanks, Joris Van Remoortere
Re: Review Request 26150: Libprocess Benchmark
On Oct. 13, 2014, 9:47 p.m., Niklas Nielsen wrote: 3rdparty/libprocess/src/tests/benchmarks.cpp, line 25 https://reviews.apache.org/r/26150/diff/3/?file=719696#file719696line25 This is the first time we use std::thread - do you have some references that it is supported across our graced compilers? Same for mutex Removed use of thread. On Oct. 13, 2014, 9:47 p.m., Niklas Nielsen wrote: 3rdparty/libprocess/src/tests/benchmarks.cpp, line 95 https://reviews.apache.org/r/26150/diff/3/?file=719696#file719696line95 Was this the lock that depended on Dominic's configure patch? If so, can you add it as a dependency? Remove use of mutex / unique_lock; use Latch instead. On Oct. 13, 2014, 9:47 p.m., Niklas Nielsen wrote: 3rdparty/libprocess/src/tests/benchmarks.cpp, line 97 https://reviews.apache.org/r/26150/diff/3/?file=719696#file719696line97 Can we use the latch (latch.hpp) abstraction here? Yep. Done. - Joris --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/26150/#review56442 --- On Oct. 14, 2014, 9:27 p.m., Joris Van Remoortere wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/26150/ --- (Updated Oct. 14, 2014, 9:27 p.m.) Review request for mesos and Niklas Nielsen. Bugs: MESOS-1840 https://issues.apache.org/jira/browse/MESOS-1840 Repository: mesos-git Description --- A benchmark for libprocess. It forks num_proc times and launched num_threads libprocess processes in each child. They are aware of the master's (parent) address and all play ping pong with it. This benchmark measures throughput in terms of the number of RPCs handled per second using persistent (linked) connections. A new test file (benchmarks) is introduced because we want to fork before libprocess is initialized. This allows us to prevent short-circuiting of message passing between processes under the same ProcessManager. This way we force the execution path of the underlying event management system. Diffs - 3rdparty/libprocess/Makefile.am 616618e 3rdparty/libprocess/src/tests/benchmarks.cpp PRE-CREATION Diff: https://reviews.apache.org/r/26150/diff/ Testing --- make check in 3rdparty/libprocess support/mesos-style.py Thanks, Joris Van Remoortere
Re: Review Request 26150: Libprocess Benchmark
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/26150/ --- (Updated Oct. 14, 2014, 9:30 p.m.) Review request for mesos and Niklas Nielsen. Changes --- Remove unused code. Bugs: MESOS-1840 https://issues.apache.org/jira/browse/MESOS-1840 Repository: mesos-git Description --- A benchmark for libprocess. It forks num_proc times and launched num_threads libprocess processes in each child. They are aware of the master's (parent) address and all play ping pong with it. This benchmark measures throughput in terms of the number of RPCs handled per second using persistent (linked) connections. A new test file (benchmarks) is introduced because we want to fork before libprocess is initialized. This allows us to prevent short-circuiting of message passing between processes under the same ProcessManager. This way we force the execution path of the underlying event management system. Diffs (updated) - 3rdparty/libprocess/Makefile.am 616618e 3rdparty/libprocess/src/tests/benchmarks.cpp PRE-CREATION Diff: https://reviews.apache.org/r/26150/diff/ Testing --- make check in 3rdparty/libprocess support/mesos-style.py Thanks, Joris Van Remoortere
Re: Review Request 26580: Memory cleanup: libprocess join worker threads
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/26580/ --- (Updated Oct. 14, 2014, 9:44 p.m.) Review request for mesos, Benjamin Hindman and Niklas Nielsen. Changes --- Address Dominic's issues. Repository: mesos-git Description --- Terminate the gc; Finalize the ProcessManager (picking up -idealy 0- left-over processes); Join on the worker threads that were dispatched. Diffs (updated) - 3rdparty/libprocess/src/process.cpp 85fb995 Diff: https://reviews.apache.org/r/26580/diff/ Testing --- make check support/mesos-style.py Thanks, Joris Van Remoortere
Re: Review Request 26150: Libprocess Benchmark
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/26150/ --- (Updated Oct. 15, 2014, 8:42 p.m.) Review request for mesos and Niklas Nielsen. Changes --- Fix pipe bug. Fix null terminated buffer bug. Bugs: MESOS-1840 https://issues.apache.org/jira/browse/MESOS-1840 Repository: mesos-git Description --- A benchmark for libprocess. It forks num_proc times and launched num_threads libprocess processes in each child. They are aware of the master's (parent) address and all play ping pong with it. This benchmark measures throughput in terms of the number of RPCs handled per second using persistent (linked) connections. A new test file (benchmarks) is introduced because we want to fork before libprocess is initialized. This allows us to prevent short-circuiting of message passing between processes under the same ProcessManager. This way we force the execution path of the underlying event management system. Diffs (updated) - 3rdparty/libprocess/Makefile.am 616618e 3rdparty/libprocess/src/tests/benchmarks.cpp PRE-CREATION Diff: https://reviews.apache.org/r/26150/diff/ Testing --- make check in 3rdparty/libprocess support/mesos-style.py Thanks, Joris Van Remoortere
Re: Review Request 26476: Remove dynamic allocation from Option.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/26476/ --- (Updated Oct. 15, 2014, 9:29 p.m.) Review request for mesos, Benjamin Hindman and Niklas Nielsen. Changes --- fix style issues. Repository: mesos-git Description --- Remove dynamic allocations from Option class. Diffs (updated) - 3rdparty/libprocess/3rdparty/stout/include/stout/option.hpp 47fe92c Diff: https://reviews.apache.org/r/26476/diff/ Testing --- make check support/mesos-style.py valgrind (reduced allocation count) Thanks, Joris Van Remoortere
Re: Review Request 26578: Memory cleanup: libprocess gc finalize
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/26578/ --- (Updated Oct. 15, 2014, 9:37 p.m.) Review request for mesos, Benjamin Hindman and Niklas Nielsen. Changes --- fix style issue. Repository: mesos-git Description --- Implement the finalize() virtual override for Libprocess' garbage collector. This terminates all processes registered with the garbage collector when the garbage collector terminates. Diffs (updated) - 3rdparty/libprocess/include/process/gc.hpp e83c636 Diff: https://reviews.apache.org/r/26578/diff/ Testing --- make check support/mesos-style.py Thanks, Joris Van Remoortere
Review Request 26783: Fix race condition in Latch
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/26783/
---
Review request for mesos, Benjamin Hindman and Niklas Nielsen.
Repository: mesos-git
Description
---
There is a race condition in Latch. The same pid can be terminated by 2
seperate threads simultaneously.
Diffs
-
3rdparty/libprocess/src/latch.cpp 89185ec
Diff: https://reviews.apache.org/r/26783/diff/
Testing
---
make check
The following test could segfault before: (after this patch it does not)
```
auto do_launch = []() {
std::mutex mut;
std::condition_variable cond;
bool ready = true;
Latch *latch = new Latch();
const size_t num_iter = 1;
std::thread t1([]() {
for (size_t i = 0; i num_iter; ++i) {
{
std::unique_lockstd::mutex lock(mut);
while (!ready) {
cond.wait(lock);
}
ready = false;
}
latch-trigger();
}
});
std::thread t2([]() {
for (size_t i = 0; i num_iter; ++i) {
latch-await();
delete latch;
latch = new Latch();
std::lock_guardstd::mutex lock(mut);
ready = true;
cond.notify_one();
}
});
t1.join();
t2.join();
};
const size_t nthread = 16;
std::vectorstd::thread tvec;
for (size_t i = 0; i nthread; ++i) {
tvec.emplace_back(do_launch);
}
for (auto t : tvec) {
t.join();
}
```
Thanks,
Joris Van Remoortere
Re: Review Request 27025: Libprocess Socket missing zookeeper group include
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/27025/ --- (Updated Oct. 22, 2014, 6:19 p.m.) Review request for mesos, Benjamin Hindman and Niklas Nielsen. Repository: mesos-git Description --- Zookeeper/group.hpp was missing an include. This was not obvious until we reduced un-necessary includes in libprocess. Diffs - src/zookeeper/group.hpp dc600b1 Diff: https://reviews.apache.org/r/27025/diff/ Testing --- Thanks, Joris Van Remoortere
Re: Review Request 27024: Libprocess Socket reduce over-include
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/27024/ --- (Updated Oct. 22, 2014, 6:19 p.m.) Review request for mesos, Benjamin Hindman and Niklas Nielsen. Repository: mesos-git Description --- Timeout only uses clock. Reduce inclusion to clock as opposed to all of process.hpp. This helps prevent circular dependency when we include future in socket.hpp. Diffs - 3rdparty/libprocess/include/process/timeout.hpp 40906cb Diff: https://reviews.apache.org/r/27024/diff/ Testing --- Thanks, Joris Van Remoortere
Re: Review Request 26583: Memory cleanup: libprocess delete garbage collector process
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/26583/ --- (Updated Oct. 22, 2014, 6:28 p.m.) Review request for mesos, Benjamin Hindman and Niklas Nielsen. Repository: mesos-git Description --- Wrap the libprocess garbage collector process with a unique ptr. Reset the ptr during finalization. Diffs - 3rdparty/libprocess/src/process.cpp 85fb995 Diff: https://reviews.apache.org/r/26583/diff/ Testing --- make check Thanks, Joris Van Remoortere
Re: Review Request 26150: Libprocess Benchmark
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/26150/ --- (Updated Oct. 22, 2014, 6:35 p.m.) Review request for mesos and Niklas Nielsen. Changes --- fix whitespace style issue. Bugs: MESOS-1840 https://issues.apache.org/jira/browse/MESOS-1840 Repository: mesos-git Description --- A benchmark for libprocess. It forks num_proc times and launched num_threads libprocess processes in each child. They are aware of the master's (parent) address and all play ping pong with it. This benchmark measures throughput in terms of the number of RPCs handled per second using persistent (linked) connections. A new test file (benchmarks) is introduced because we want to fork before libprocess is initialized. This allows us to prevent short-circuiting of message passing between processes under the same ProcessManager. This way we force the execution path of the underlying event management system. Diffs (updated) - 3rdparty/libprocess/Makefile.am 616618e 3rdparty/libprocess/src/tests/benchmarks.cpp PRE-CREATION Diff: https://reviews.apache.org/r/26150/diff/ Testing --- make check in 3rdparty/libprocess support/mesos-style.py Thanks, Joris Van Remoortere
Re: Review Request 26783: Fix race condition in Latch
On Oct. 17, 2014, 6:46 p.m., Niklas Nielsen wrote:
Hey Joris,
Can you file a JIRA ticket on the issue?
Also, do you know why multiple threads where calling the destructor?
It is not the case that multiple threads are calling the destructor. It is the
case that a thread joining (await) on the latch could then destroy the latch
after it has joined. This condition can happen before the terminate(pid) in
trigger() is fully completed. In this case terminate(pid) is called twice which
can cause a segfault. The extra atomic check helps prevent the 2nd terminate.
- Joris
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/26783/#review57188
---
On Oct. 22, 2014, 6:55 p.m., Joris Van Remoortere wrote:
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/26783/
---
(Updated Oct. 22, 2014, 6:55 p.m.)
Review request for mesos, Benjamin Hindman and Niklas Nielsen.
Bugs: MESOS-1968
https://issues.apache.org/jira/browse/MESOS-1968
Repository: mesos-git
Description
---
There is a race condition in Latch. The same pid can be terminated by 2
seperate threads simultaneously.
Diffs
-
3rdparty/libprocess/src/latch.cpp 89185ec
Diff: https://reviews.apache.org/r/26783/diff/
Testing
---
make check
The following test could segfault before: (after this patch it does not)
```
auto do_launch = []() {
std::mutex mut;
std::condition_variable cond;
bool ready = true;
Latch *latch = new Latch();
const size_t num_iter = 1;
std::thread t1([]() {
for (size_t i = 0; i num_iter; ++i) {
{
std::unique_lockstd::mutex lock(mut);
while (!ready) {
cond.wait(lock);
}
ready = false;
}
latch-trigger();
}
});
std::thread t2([]() {
for (size_t i = 0; i num_iter; ++i) {
latch-await();
delete latch;
latch = new Latch();
std::lock_guardstd::mutex lock(mut);
ready = true;
cond.notify_one();
}
});
t1.join();
t2.join();
};
const size_t nthread = 16;
std::vectorstd::thread tvec;
for (size_t i = 0; i nthread; ++i) {
tvec.emplace_back(do_launch);
}
for (auto t : tvec) {
t.join();
}
```
Thanks,
Joris Van Remoortere
Re: Review Request 26783: Fix race condition in Latch
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/26783/
---
(Updated Oct. 22, 2014, 6:55 p.m.)
Review request for mesos, Benjamin Hindman and Niklas Nielsen.
Changes
---
Created JIRA
Bugs: MESOS-1968
https://issues.apache.org/jira/browse/MESOS-1968
Repository: mesos-git
Description
---
There is a race condition in Latch. The same pid can be terminated by 2
seperate threads simultaneously.
Diffs
-
3rdparty/libprocess/src/latch.cpp 89185ec
Diff: https://reviews.apache.org/r/26783/diff/
Testing
---
make check
The following test could segfault before: (after this patch it does not)
```
auto do_launch = []() {
std::mutex mut;
std::condition_variable cond;
bool ready = true;
Latch *latch = new Latch();
const size_t num_iter = 1;
std::thread t1([]() {
for (size_t i = 0; i num_iter; ++i) {
{
std::unique_lockstd::mutex lock(mut);
while (!ready) {
cond.wait(lock);
}
ready = false;
}
latch-trigger();
}
});
std::thread t2([]() {
for (size_t i = 0; i num_iter; ++i) {
latch-await();
delete latch;
latch = new Latch();
std::lock_guardstd::mutex lock(mut);
ready = true;
cond.notify_one();
}
});
t1.join();
t2.join();
};
const size_t nthread = 16;
std::vectorstd::thread tvec;
for (size_t i = 0; i nthread; ++i) {
tvec.emplace_back(do_launch);
}
for (auto t : tvec) {
t.join();
}
```
Thanks,
Joris Van Remoortere
Re: Review Request 26476: Remove dynamic allocation from Option.
On Oct. 16, 2014, 12:23 a.m., Ben Mahler wrote: A few higher level questions: (1) What motivated this? Concretely, which performance aspect of Mesos is this improving? In the past, we eliminated copies of our Option,Try,Future,Result family because we found that the copying of large Option,Try,Future,ResultRegistry types was degrading performance. In general there should be a well understood benefit, especially when we're increasing the esotericism of the code in the name of performance. :) (2) When is this better? When is this worse? It looks like None() options are now more expensive? Did you measure Option performance with any benchmarks? (3) Curious why you introduce the new public reset() method, since most callers use `option = None()`. Would be great to avoid introducing another way to do it. 1. The motivation for this is 3-fold: a. Reduce dynamic allocations. These can cause latency jitter as process lifetime grows. This kind of jitter can make it hard to grasp the upper bound of latency on certain operations under locks. This modification only moves the allocated space of T, it does not reduce or increase the number of actual construction / move calls unless the new move constructor is used. b. The commonly understood implication of Optional / Option / Nullable is that it augments the type field by 1 bit in order to allow representation of an unknown or null state. This is handy in cases where a type such as int64_t fully utilizes its 64 bit storage space, and representing unknown would otherwise require us to steal a number (such as INT64_MAX). This class should not take on the additional responsibility of managing memory for the augmented type. c. It can be very deceptive to a newcomer when Optionint64_t does a dynamic allocation. Intuitively you would not expect a type such as int64_t to do a dynamic allocation or be expensive to copy. Naturally OptionBigType would be expected to be expensive to copy, and so a developer would be more inclined to do something like std::shared_ptrOptionBigType. 2. This change has the biggest impact for Options of small types such as Optionint or OptionSmallStruct. The stack allocation vs. dynamic allocation for small objects can be a 2-3 orders of magnitude cost difference. This is worse when have Options of large types such as OptionMegabyteBuffer; but only in that this *could* be allocated on the stack; it is expected that one not do so (and rather use indirection around or inside the Option). You are correct that None() options are now more expensive, but only in memory size (and only when the type is larger than 4 bytes). In my experience we do not allocate a large amount of options on a single stack (if we had a large collection of them they would be in some container that is itself dynamically allocated). I do not have a benchmark for this specific change, although I have done it in the past for other projects. I can write one if you like. 3. I will make the reset() function private :-) - Joris --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/26476/#review56846 --- On Oct. 15, 2014, 9:29 p.m., Joris Van Remoortere wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/26476/ --- (Updated Oct. 15, 2014, 9:29 p.m.) Review request for mesos, Benjamin Hindman and Niklas Nielsen. Repository: mesos-git Description --- Remove dynamic allocations from Option class. Diffs - 3rdparty/libprocess/3rdparty/stout/include/stout/option.hpp 47fe92c Diff: https://reviews.apache.org/r/26476/diff/ Testing --- make check support/mesos-style.py valgrind (reduced allocation count) Thanks, Joris Van Remoortere
Re: Review Request 26476: Remove dynamic allocation from Option.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/26476/ --- (Updated Oct. 22, 2014, 7:34 p.m.) Review request for mesos, Benjamin Hindman and Niklas Nielsen. Changes --- Make reset private. Repository: mesos-git Description --- Remove dynamic allocations from Option class. Diffs (updated) - 3rdparty/libprocess/3rdparty/stout/include/stout/option.hpp 47fe92c Diff: https://reviews.apache.org/r/26476/diff/ Testing --- make check support/mesos-style.py valgrind (reduced allocation count) Thanks, Joris Van Remoortere
Re: Review Request 27024: Libprocess Socket reduce over-include
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/27024/ --- (Updated Oct. 22, 2014, 8:05 p.m.) Review request for mesos, Benjamin Hindman and Niklas Nielsen. Changes --- filled out testing done. Repository: mesos-git Description --- Timeout only uses clock. Reduce inclusion to clock as opposed to all of process.hpp. This helps prevent circular dependency when we include future in socket.hpp. Diffs - 3rdparty/libprocess/include/process/timeout.hpp 40906cb Diff: https://reviews.apache.org/r/27024/diff/ Testing (updated) --- make check with this patch and 27025 applied. Thanks, Joris Van Remoortere
Re: Review Request 26150: Libprocess Benchmark
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/26150/ --- (Updated Oct. 23, 2014, 6:50 p.m.) Review request for mesos and Niklas Nielsen. Changes --- fix minor style issues. Bugs: MESOS-1840 https://issues.apache.org/jira/browse/MESOS-1840 Repository: mesos-git Description --- A benchmark for libprocess. It forks num_proc times and launched num_threads libprocess processes in each child. They are aware of the master's (parent) address and all play ping pong with it. This benchmark measures throughput in terms of the number of RPCs handled per second using persistent (linked) connections. A new test file (benchmarks) is introduced because we want to fork before libprocess is initialized. This allows us to prevent short-circuiting of message passing between processes under the same ProcessManager. This way we force the execution path of the underlying event management system. Diffs (updated) - 3rdparty/libprocess/Makefile.am 616618e 3rdparty/libprocess/src/tests/benchmarks.cpp PRE-CREATION Diff: https://reviews.apache.org/r/26150/diff/ Testing --- make check in 3rdparty/libprocess support/mesos-style.py Thanks, Joris Van Remoortere
Re: Review Request 26150: Libprocess Benchmark
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/26150/ --- (Updated Oct. 23, 2014, 8:40 p.m.) Review request for mesos and Niklas Nielsen. Changes --- cleaning up description. Bugs: MESOS-1840 https://issues.apache.org/jira/browse/MESOS-1840 Repository: mesos-git Description (updated) --- A benchmark for libprocess. It forks and launches multiple libprocess processes in each child. They are aware of the master's (parent) address and all play ping pong with it. This benchmark measures throughput in terms of the number of RPCs handled per second using persistent (linked) connections. A new test file (benchmarks) is introduced because we want to fork before libprocess is initialized. This allows us to prevent short-circuiting of message passing between processes under the same ProcessManager. This way we force the execution path of the underlying event management system. Diffs - 3rdparty/libprocess/Makefile.am 616618e 3rdparty/libprocess/src/tests/benchmarks.cpp PRE-CREATION Diff: https://reviews.apache.org/r/26150/diff/ Testing --- make check in 3rdparty/libprocess support/mesos-style.py Thanks, Joris Van Remoortere
Review Request 27111: Block IO Isolator: usage metrics
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/27111/ --- Review request for mesos, Benjamin Hindman and Ian Downes. Bugs: MESOS-1977 https://issues.apache.org/jira/browse/MESOS-1977 Repository: mesos-git Description --- Implement a block io isolator that just publishes read / write, bytes / operations per second. A split of r25922 as suggested by Ian. Diffs - include/mesos/mesos.proto 6b93e90 src/Makefile.am 2617f77 src/linux/cgroups.hpp abf31df src/linux/cgroups.cpp 62df4b7 src/linux/fs.hpp ac8b5f4 src/linux/fs.cpp b01d14c src/slave/containerizer/isolators/cgroups/blkio.hpp PRE-CREATION src/slave/containerizer/isolators/cgroups/blkio.cpp PRE-CREATION src/slave/containerizer/mesos/containerizer.cpp 9d08329 src/tests/isolator_tests.cpp 52b38a3 Diff: https://reviews.apache.org/r/27111/diff/ Testing --- make check sudo ./mesos-tests --gtest_filter=BlkIOIsolatorTest* Thanks, Joris Van Remoortere
Review Request 27113: Libprocess benchmark cleanup
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/27113/ --- Review request for mesos, Ben Mahler and Niklas Nielsen. Bugs: MESOS-1980 https://issues.apache.org/jira/browse/MESOS-1980 Repository: mesos-git Description --- Add a comment to BenchmarkProcess. Remove setLink() as it is not strictly necessary. Diffs - 3rdparty/libprocess/src/tests/benchmarks.cpp 79a650b Diff: https://reviews.apache.org/r/27113/diff/ Testing --- make check Thanks, Joris Van Remoortere
Re: Review Request 27111: Block IO Isolator: usage metrics
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/27111/ --- (Updated Oct. 23, 2014, 9:58 p.m.) Review request for mesos, Benjamin Hindman and Ian Downes. Changes --- fix whitespace issues. Bugs: MESOS-1977 https://issues.apache.org/jira/browse/MESOS-1977 Repository: mesos-git Description --- Implement a block io isolator that just publishes read / write, bytes / operations per second. A split of r25922 as suggested by Ian. Diffs (updated) - include/mesos/mesos.proto 6b93e90 src/Makefile.am 2617f77 src/linux/cgroups.hpp abf31df src/linux/cgroups.cpp 62df4b7 src/linux/fs.hpp ac8b5f4 src/linux/fs.cpp b01d14c src/slave/containerizer/isolators/cgroups/blkio.hpp PRE-CREATION src/slave/containerizer/isolators/cgroups/blkio.cpp PRE-CREATION src/slave/containerizer/mesos/containerizer.cpp 9d08329 src/tests/isolator_tests.cpp 52b38a3 Diff: https://reviews.apache.org/r/27111/diff/ Testing --- make check sudo ./mesos-tests --gtest_filter=BlkIOIsolatorTest* Thanks, Joris Van Remoortere
Review Request 27116: Fix Libprocess benchmark OSX
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/27116/ --- Review request for mesos, Niklas Nielsen and Till Toenshoff. Repository: mesos-git Description --- pipe2 does not work on osx. Replace with a helper function that sets up the pipes using pipe() and then stout/os::cloexec(). Diffs - 3rdparty/libprocess/src/tests/benchmarks.cpp 79a650b Diff: https://reviews.apache.org/r/27116/diff/ Testing --- make check Thanks, Joris Van Remoortere
Re: Review Request 27113: Libprocess benchmark cleanup
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/27113/ --- (Updated Oct. 24, 2014, 3:09 a.m.) Review request for mesos, Ben Mahler and Niklas Nielsen. Changes --- add std::unique_ptr configure check as dependency. Bugs: MESOS-1980 https://issues.apache.org/jira/browse/MESOS-1980 Repository: mesos-git Description --- Add a comment to BenchmarkProcess. Remove setLink() as it is not strictly necessary. Diffs - 3rdparty/libprocess/src/tests/benchmarks.cpp 79a650b Diff: https://reviews.apache.org/r/27113/diff/ Testing --- make check Thanks, Joris Van Remoortere
Re: Review Request 27113: Libprocess benchmark cleanup
On Oct. 23, 2014, 10:18 p.m., Ben Mahler wrote: 3rdparty/libprocess/src/tests/benchmarks.cpp, lines 40-53 https://reviews.apache.org/r/27113/diff/1/?file=731328#file731328line40 I think we need to follow the same pattern as the mesos benchmarks, that is, they are included within the test binary, not as separate mains. The problem with including this in the regular libprocess test suite is that we require libprocess to be initialized after the forks have been instantiated. Once we can safely terminate the global libprocess state, we could merge this with the regular tests, until then it needs to be in a separate binary (or forcibly run before the other tests). - Joris --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/27113/#review58124 --- On Oct. 24, 2014, 3:09 a.m., Joris Van Remoortere wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/27113/ --- (Updated Oct. 24, 2014, 3:09 a.m.) Review request for mesos, Ben Mahler and Niklas Nielsen. Bugs: MESOS-1980 https://issues.apache.org/jira/browse/MESOS-1980 Repository: mesos-git Description --- Add a comment to BenchmarkProcess. Remove setLink() as it is not strictly necessary. Diffs - 3rdparty/libprocess/src/tests/benchmarks.cpp 79a650b Diff: https://reviews.apache.org/r/27113/diff/ Testing --- make check Thanks, Joris Van Remoortere
Re: Review Request 27113: Libprocess benchmark cleanup
On Oct. 23, 2014, 10:18 p.m., Ben Mahler wrote: 3rdparty/libprocess/src/tests/benchmarks.cpp, line 149 https://reviews.apache.org/r/27113/diff/1/?file=731328#file731328line149 Why is this a set, don't you only need to know whether you're linked? (i.e. boolean). This is just 1:1 client:server in terms of messaging, right? It's a bit confusing as to why there is 1 'other', but many 'linkedPorts'.. There are many clients connecting to a single server. The server keeps track of which clients it has linked with to ensure that it links back to all of them. - Joris --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/27113/#review58124 --- On Oct. 24, 2014, 3:09 a.m., Joris Van Remoortere wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/27113/ --- (Updated Oct. 24, 2014, 3:09 a.m.) Review request for mesos, Ben Mahler and Niklas Nielsen. Bugs: MESOS-1980 https://issues.apache.org/jira/browse/MESOS-1980 Repository: mesos-git Description --- Add a comment to BenchmarkProcess. Remove setLink() as it is not strictly necessary. Diffs - 3rdparty/libprocess/src/tests/benchmarks.cpp 79a650b Diff: https://reviews.apache.org/r/27113/diff/ Testing --- make check Thanks, Joris Van Remoortere
Re: Review Request 27113: Libprocess benchmark cleanup
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/27113/ --- (Updated Oct. 24, 2014, 8:46 p.m.) Review request for mesos, Ben Mahler and Niklas Nielsen. Changes --- address bmahler's issues. Bugs: MESOS-1980 https://issues.apache.org/jira/browse/MESOS-1980 Repository: mesos-git Description --- Add a comment to BenchmarkProcess. Remove setLink() as it is not strictly necessary. Diffs (updated) - 3rdparty/libprocess/src/tests/benchmarks.cpp 3177a8e Diff: https://reviews.apache.org/r/27113/diff/ Testing --- make check Thanks, Joris Van Remoortere
Re: Review Request 27111: Block IO Isolator: usage metrics
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/27111/ --- (Updated Oct. 25, 2014, 5:48 p.m.) Review request for mesos, Benjamin Hindman and Ian Downes. Changes --- Re-organize blkio stats. Make blkio unit test setup async-signal-safe. Bugs: MESOS-1977 https://issues.apache.org/jira/browse/MESOS-1977 Repository: mesos-git Description --- Implement a block io isolator that just publishes read / write, bytes / operations per second. A split of r25922 as suggested by Ian. Diffs (updated) - include/mesos/mesos.proto 6b93e90 src/Makefile.am 2617f77 src/linux/cgroups.hpp abf31df src/linux/cgroups.cpp 62df4b7 src/linux/fs.hpp ac8b5f4 src/linux/fs.cpp b01d14c src/slave/containerizer/isolators/cgroups/blkio.hpp PRE-CREATION src/slave/containerizer/isolators/cgroups/blkio.cpp PRE-CREATION src/slave/containerizer/mesos/containerizer.cpp 9d08329 src/tests/isolator_tests.cpp 52b38a3 Diff: https://reviews.apache.org/r/27111/diff/ Testing --- make check sudo ./mesos-tests --gtest_filter=BlkIOIsolatorTest* Thanks, Joris Van Remoortere
Re: Review Request 27111: Block IO Isolator: usage metrics
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/27111/ --- (Updated Oct. 25, 2014, 5:53 p.m.) Review request for mesos, Benjamin Hindman and Ian Downes. Changes --- Add async-signal-safe comments. Fix cosmetics. Bugs: MESOS-1977 https://issues.apache.org/jira/browse/MESOS-1977 Repository: mesos-git Description --- Implement a block io isolator that just publishes read / write, bytes / operations per second. A split of r25922 as suggested by Ian. Diffs (updated) - include/mesos/mesos.proto 6b93e90 src/Makefile.am 2617f77 src/linux/cgroups.hpp abf31df src/linux/cgroups.cpp 62df4b7 src/linux/fs.hpp ac8b5f4 src/linux/fs.cpp b01d14c src/slave/containerizer/isolators/cgroups/blkio.hpp PRE-CREATION src/slave/containerizer/isolators/cgroups/blkio.cpp PRE-CREATION src/slave/containerizer/mesos/containerizer.cpp 9d08329 src/tests/isolator_tests.cpp 52b38a3 Diff: https://reviews.apache.org/r/27111/diff/ Testing --- make check sudo ./mesos-tests --gtest_filter=BlkIOIsolatorTest* Thanks, Joris Van Remoortere
Re: Review Request 27111: Block IO Isolator: usage metrics
On Oct. 25, 2014, 1:02 a.m., Ian Downes wrote: src/slave/containerizer/isolators/cgroups/blkio.cpp, line 252 https://reviews.apache.org/r/27111/diff/2/?file=731349#file731349line252 What's the difference between blkio.throttle.io_service_bytes and blkio.io_service_bytes? From my experimentation: blkio.io_service_bytes does not actually report the bytes read/written, whereas blkio.throttle.io_service_bytes does. - Joris --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/27111/#review58418 --- On Oct. 25, 2014, 5:53 p.m., Joris Van Remoortere wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/27111/ --- (Updated Oct. 25, 2014, 5:53 p.m.) Review request for mesos, Benjamin Hindman and Ian Downes. Bugs: MESOS-1977 https://issues.apache.org/jira/browse/MESOS-1977 Repository: mesos-git Description --- Implement a block io isolator that just publishes read / write, bytes / operations per second. A split of r25922 as suggested by Ian. Diffs - include/mesos/mesos.proto 6b93e90 src/Makefile.am 2617f77 src/linux/cgroups.hpp abf31df src/linux/cgroups.cpp 62df4b7 src/linux/fs.hpp ac8b5f4 src/linux/fs.cpp b01d14c src/slave/containerizer/isolators/cgroups/blkio.hpp PRE-CREATION src/slave/containerizer/isolators/cgroups/blkio.cpp PRE-CREATION src/slave/containerizer/mesos/containerizer.cpp 9d08329 src/tests/isolator_tests.cpp 52b38a3 Diff: https://reviews.apache.org/r/27111/diff/ Testing --- make check sudo ./mesos-tests --gtest_filter=BlkIOIsolatorTest* Thanks, Joris Van Remoortere
Re: Review Request 27023: Libprocess Socket std::shared_ptr
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/27023/ --- (Updated Oct. 29, 2014, 4:31 a.m.) Review request for mesos, Benjamin Hindman and Niklas Nielsen. Changes --- Add get() function which instantiates the impl if it was not already (e.g. implicit int conversion after default Socket() constructor). Add create() function which returns a new shared_ptr (RVO) to make the get() function a good candidate for inlining. Repository: mesos-git Description --- Use std::shared_ptr to do reference counting for libprocess rather than manually doing it. Diffs (updated) - 3rdparty/libprocess/include/process/socket.hpp 6683881 Diff: https://reviews.apache.org/r/27023/diff/ Testing (updated) --- make check Thanks, Joris Van Remoortere
Re: Review Request 27023: Libprocess Socket std::shared_ptr
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/27023/ --- (Updated Oct. 29, 2014, 4:43 a.m.) Review request for mesos, Benjamin Hindman and Niklas Nielsen. Changes --- don't introduce rvalue impl constructor yet. alphabetize include. Repository: mesos-git Description --- Use std::shared_ptr to do reference counting for libprocess rather than manually doing it. Diffs (updated) - 3rdparty/libprocess/include/process/socket.hpp 6683881 Diff: https://reviews.apache.org/r/27023/diff/ Testing --- make check Thanks, Joris Van Remoortere
Re: Review Request 27023: Libprocess Socket std::shared_ptr
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/27023/ --- (Updated Oct. 29, 2014, 5:10 a.m.) Review request for mesos, Benjamin Hindman and Niklas Nielsen. Changes --- fix brace on new line Repository: mesos-git Description --- Use std::shared_ptr to do reference counting for libprocess rather than manually doing it. Diffs (updated) - 3rdparty/libprocess/include/process/socket.hpp 6683881 Diff: https://reviews.apache.org/r/27023/diff/ Testing --- make check Thanks, Joris Van Remoortere
Re: Review Request 27054: Libprocess Socket: introduce connect()
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/27054/ --- (Updated Oct. 29, 2014, 5:18 a.m.) Review request for mesos, Benjamin Hindman and Niklas Nielsen. Changes --- rebase on 27023 Bugs: MESOS-1330 https://issues.apache.org/jira/browse/MESOS-1330 Repository: mesos-git Description --- Introduce the connect(const Node node) call to Socket. Use it in link() and send(). Diffs (updated) - 3rdparty/libprocess/include/process/socket.hpp 6683881 3rdparty/libprocess/src/process.cpp 85fb995 Diff: https://reviews.apache.org/r/27054/diff/ Testing --- make check Thanks, Joris Van Remoortere
Review Request 27350: Libprocess: Introduce mutex configure check
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/27350/ --- Review request for mesos, Benjamin Hindman and Dominic Hamon. Bugs: MESOS-2009 https://issues.apache.org/jira/browse/MESOS-2009 Repository: mesos-git Description --- Introduce mutex to the configure check for Libprocess Diffs - 3rdparty/libprocess/m4/ax_cxx_compile_stdcxx_11.m4 bf2bd19 Diff: https://reviews.apache.org/r/27350/diff/ Testing --- make check on gcc-4.4 make check on gcc-4.9 Thanks, Joris Van Remoortere
Re: Review Request 27054: Libprocess Socket: introduce connect()
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/27054/ --- (Updated Oct. 29, 2014, 9:47 p.m.) Review request for mesos, Benjamin Hindman and Niklas Nielsen. Changes --- Depends on mutex configure check for libprocess. Bugs: MESOS-1330 https://issues.apache.org/jira/browse/MESOS-1330 Repository: mesos-git Description --- Introduce the connect(const Node node) call to Socket. Use it in link() and send(). Diffs - 3rdparty/libprocess/include/process/socket.hpp 6683881 3rdparty/libprocess/src/process.cpp 85fb995 Diff: https://reviews.apache.org/r/27054/diff/ Testing --- make check Thanks, Joris Van Remoortere
Review Request 27351: Libprocess: Introduce enable_shared_from_this configure check
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/27351/ --- Review request for mesos, Benjamin Hindman and Dominic Hamon. Bugs: MESOS-2010 https://issues.apache.org/jira/browse/MESOS-2010 Repository: mesos-git Description --- Introduce enable_shared_from_this to the configure check for Libprocess Diffs - 3rdparty/libprocess/m4/ax_cxx_compile_stdcxx_11.m4 bf2bd19 Diff: https://reviews.apache.org/r/27351/diff/ Testing --- make check on gcc-4.4 make check on gcc-4.9 Thanks, Joris Van Remoortere
Re: Review Request 27351: Libprocess: Introduce enable_shared_from_this configure check
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/27351/ --- (Updated Oct. 29, 2014, 10:23 p.m.) Review request for mesos, Benjamin Hindman and Dominic Hamon. Changes --- Add comment for scope. Rename struct. Bugs: MESOS-2010 https://issues.apache.org/jira/browse/MESOS-2010 Repository: mesos-git Description --- Introduce enable_shared_from_this to the configure check for Libprocess Diffs (updated) - 3rdparty/libprocess/m4/ax_cxx_compile_stdcxx_11.m4 bf2bd19 Diff: https://reviews.apache.org/r/27351/diff/ Testing --- make check on gcc-4.4 make check on gcc-4.9 Thanks, Joris Van Remoortere
Review Request 27354: Introduce mutex configure check
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/27354/ --- Review request for mesos, Benjamin Hindman and Dominic Hamon. Bugs: MESOS-2011 https://issues.apache.org/jira/browse/MESOS-2011 Repository: mesos-git Description --- * Introduce mutex to the configure check * Document availability of mutex in style guide Diffs - docs/mesos-c++-style-guide.md f50354c m4/ax_cxx_compile_stdcxx_11.m4 72a1080 Diff: https://reviews.apache.org/r/27354/diff/ Testing --- make check on gcc-4.4 make check on gcc-4.9 Thanks, Joris Van Remoortere
Re: Review Request 27054: Libprocess Socket: introduce connect()
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/27054/ --- (Updated Oct. 29, 2014, 10:42 p.m.) Review request for mesos, Benjamin Hindman and Niklas Nielsen. Changes --- Depends on enable_shared_from_this configure check for libprocess. Bugs: MESOS-1330 https://issues.apache.org/jira/browse/MESOS-1330 Repository: mesos-git Description --- Introduce the connect(const Node node) call to Socket. Use it in link() and send(). Diffs - 3rdparty/libprocess/include/process/socket.hpp 6683881 3rdparty/libprocess/src/process.cpp 85fb995 Diff: https://reviews.apache.org/r/27054/diff/ Testing --- make check Thanks, Joris Van Remoortere
Review Request 27356: Introduce enable_shared_from_this configure check
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/27356/ --- Review request for mesos, Benjamin Hindman and Dominic Hamon. Bugs: MESOS-2012 https://issues.apache.org/jira/browse/MESOS-2012 Repository: mesos-git Description --- * Introduce enable_shared_from_this to the configure check * Document availability of enable_shared_from_this in style guide Diffs - docs/mesos-c++-style-guide.md f50354c m4/ax_cxx_compile_stdcxx_11.m4 72a1080 Diff: https://reviews.apache.org/r/27356/diff/ Testing --- make check on gcc-4.4 make check on gcc-4.9 Thanks, Joris Van Remoortere
Re: Review Request 27413: Rename the Node class in Endpoint and replace the ip and port pairs from the UPID class and process namespace with this concept.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/27413/#review59361 --- This has been discussed before here: https://reviews.apache.org/r/25847/. Niklas suggested we not rename the class as the naming was originally inspired by Erlang http://www.erlang.org/doc/reference_manual/distributed.html. Maybe this deserves a JIRA discussion rather than reviewboard. - Joris Van Remoortere On Oct. 31, 2014, 3:45 a.m., Evelina Dumitrescu wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/27413/ --- (Updated Oct. 31, 2014, 3:45 a.m.) Review request for mesos, Dominic Hamon and switched to 'mcypark'. Bugs: MESOS-2019 https://issues.apache.org/jira/browse/MESOS-2019 Repository: mesos-git Description --- The Node class abstracts the endpoint notion: it defines the association of the ip and port for a connection. A node defines more the host(which can have multiple active endpoints). At the moment, the Node class is used to keep a mapping from a socket to the ip port pair in the process namespace. I want to propose to rename this class in Endpoint and also extend its use by replacing the ip port fields from the UPID class and process namespace with this type. Diffs - 3rdparty/libprocess/include/Makefile.am 542ae1c39283087da6fef69be29c811ac35f00f4 3rdparty/libprocess/include/process/endpoint.hpp PRE-CREATION 3rdparty/libprocess/include/process/node.hpp 7a96894456683be60595a8f96f85df37947ff5b6 3rdparty/libprocess/include/process/pid.hpp 23453229c2a868a62b143416de33cf630268a64e 3rdparty/libprocess/include/process/process.hpp 81a1f7a3fece78bb7df1e86850572922c8704ea4 3rdparty/libprocess/src/http.cpp 4ef00d11de4ff4e7eb4e44aac204147799f85ac2 3rdparty/libprocess/src/pid.cpp 20ff25c0959c247687955535f02e0397653af52a 3rdparty/libprocess/src/process.cpp 85fb9958342f0bcdde322d9c55333126e6f86668 3rdparty/libprocess/src/tests/benchmarks.cpp 3177a8ecac6adf8b52c3160587dc3a4fe2707d26 3rdparty/libprocess/src/tests/http_tests.cpp a1c3685c88d268c7ecdc2dad22608198dc0b295d 3rdparty/libprocess/src/tests/metrics_tests.cpp 33539e45bab8f79afa773a325465016f01fd3b75 3rdparty/libprocess/src/tests/process_tests.cpp b985fb77ea05fae5c0b144ea48814acc7bb5135b src/common/protobuf_utils.cpp a9b65e328c4c62bff7fbf5633dda25d742d79019 src/master/master.cpp 762d2ff6c168ac212f70b43275692a77496a7fcd src/sched/sched.cpp 0fb8c7bda75545389f8024489b3c76ae115111f4 src/scheduler/scheduler.cpp fb88a3e029e97ba33eae5d50503be5ed9c9533e6 src/slave/http.cpp 965d78c3e3ea2e3c922f62aabf6fc87c5cf2ab8d src/slave/slave.cpp 96fb5f7385b0762d46d8129f7e43207bd6311644 src/tests/files_tests.cpp a696aa22d56b37ee70c6e64c81a849da6d436451 src/tests/gc_tests.cpp f7747e2999cbd64ab0cd0321cca0021489dc3346 src/tests/logging_tests.cpp 58e9b3399c3de28b1c5b3a8ec8fdf85b7a31c6d5 src/tests/master_contender_detector_tests.cpp 8f7fb18f0ca985364565fc4cd50f113b1461fbe7 src/tests/master_tests.cpp 2e525749247626c05efb2f54a707599facb114b6 src/tests/monitor_tests.cpp 4b950e14bd94cdfa21212268b56bebdc1200078d Diff: https://reviews.apache.org/r/27413/diff/ Testing --- make check Thanks, Evelina Dumitrescu
Re: Unable to install Mesos on Ubuntu 14.04. Error during 'make'
That seems like you might have a connectivity / firewall issue. Can you try just doing a wget on that artifact? On Fri, Oct 31, 2014 at 1:56 PM, Sweta Rani swetarani3...@gmail.com wrote: I tried it again but failed with following errors : make[1]: Entering directory `/home/proxima/mesos/build/src' Building mesos-0.21.0.jar ... [INFO] Scanning for projects... Downloading: http://repo.maven.apache.org/maven2/org/apache/apache/11/apache-11.pom [ERROR] The build could not read 1 project - [Help 1] [ERROR] [ERROR] The project org.apache.mesos:mesos:0.21.0 (/home/proxima/mesos/build/src/java/mesos.pom) has 1 error [ERROR] Non-resolvable parent POM: Could not transfer artifact org.apache:apache:pom:11 from/to central ( http://repo.maven.apache.org/maven2): Connection to http://repo.maven.apache.org refused and 'parent.relativePath' points at wrong local POM @ line 18, column 11: Connection refused - [Help 2] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/ProjectBuildingException [ERROR] [Help 2] http://cwiki.apache.org/confluence/display/MAVEN/UnresolvableModelException make[1]: *** [java/target/mesos-0.21.0.jar] Error 1 make[1]: Leaving directory `/home/proxima/mesos/build/src' make: *** [all-recursive] Error 1 On Fri Oct 31 2014 at 1:09:15 PM Sweta Rani swetarani3...@gmail.com wrote: yaa.. followed the steps as directed in the documentation *Sweta Rani* *3rd Year Undergraduate* *Department of Electrical Engineering* *IIT Kharagpur http://www.iitkgp.ac.in/* On Fri, Oct 31, 2014 at 3:14 AM, Dominic Hamon dha...@twopensource.com wrote: Let's tackle the easy stuff: did you run bootstrap and configure as per the getting started doc? On Oct 30, 2014 2:04 PM, Sweta Rani swetarani3...@gmail.com wrote: Tried all the suggestions and tips given in JIRA and different google forums but still the error remains same. Making all in . make[1]: Entering directory `/home/proxima/mesos/build' make[1]: Nothing to be done for `all-am'. make[1]: Leaving directory `/home/proxima/mesos/build' Making all in 3rdparty make[1]: Entering directory `/home/proxima/mesos/build/3rdparty' make all-recursive make[2]: Entering directory `/home/proxima/mesos/build/3rdparty' Making all in libprocess make[3]: Entering directory `/home/proxima/mesos/build/3rdparty/libprocess' Making all in 3rdparty make[4]: Entering directory `/home/proxima/mesos/build/3rdparty/libprocess/3rdparty' make all-recursive make[5]: Entering directory `/home/proxima/mesos/build/3rdparty/libprocess/3rdparty' Making all in stout make[6]: Entering directory `/home/proxima/mesos/build/3rdparty/libprocess/3rdparty/stout' Making all in . make[7]: Entering directory `/home/proxima/mesos/build/3rdparty/libprocess/3rdparty/stout' make[7]: Nothing to be done for `all-am'. make[7]: Leaving directory `/home/proxima/mesos/build/3rdparty/libprocess/3rdparty/stout' Making all in include make[7]: Entering directory `/home/proxima/mesos/build/3rdparty/libprocess/3rdparty/stout/include' make[7]: Nothing to be done for `all'. make[7]: Leaving directory `/home/proxima/mesos/build/3rdparty/libprocess/3rdparty/stout/include' make[6]: Leaving directory `/home/proxima/mesos/build/3rdparty/libprocess/3rdparty/stout' make[6]: Entering directory `/home/proxima/mesos/build/3rdparty/libprocess/3rdparty' make[6]: Nothing to be done for `all-am'. make[6]: Leaving directory `/home/proxima/mesos/build/3rdparty/libprocess/3rdparty' make[5]: Leaving directory `/home/proxima/mesos/build/3rdparty/libprocess/3rdparty' make[4]: Leaving directory `/home/proxima/mesos/build/3rdparty/libprocess/3rdparty' Making all in . make[4]: Entering directory `/home/proxima/mesos/build/3rdparty/libprocess' make[4]: Nothing to be done for `all-am'. make[4]: Leaving directory `/home/proxima/mesos/build/3rdparty/libprocess' Making all in include make[4]: Entering directory `/home/proxima/mesos/build/3rdparty/libprocess/include' make[4]: Nothing to be done for `all'. make[4]: Leaving directory `/home/proxima/mesos/build/3rdparty/libprocess/include' make[3]: Leaving directory `/home/proxima/mesos/build/3rdparty/libprocess' make[3]: Entering directory `/home/proxima/mesos/build/3rdparty' make[3]: Nothing to be done for `all-am'. make[3]: Leaving directory `/home/proxima/mesos/build/3rdparty' make[2]: Leaving directory `/home/proxima/mesos/build/3rdparty' make[1]: Leaving directory `/home/proxima/mesos/build/3rdparty' Making all in src make[1]: Entering directory
Re: Unable to install Mesos on Ubuntu 14.04. Error during 'make'
I think this suggests you're fetching through a proxy: *Proxy request sent, awaiting response... 200 OK* When I wget this is my output: wget http://repo.maven.apache.org/maven2/org/apache/apache/11/apache-11.pom --2014-10-31 15:05:32-- http://repo.maven.apache.org/maven2/org/apache/apache/11/apache-11.pom Resolving repo.maven.apache.org (repo.maven.apache.org)... 23.235.47.215 Connecting to repo.maven.apache.org (repo.maven.apache.org)|23.235.47.215|:80... connected. *HTTP request sent, awaiting response... 200 OK* I think you might need to set up your proxy along these lines: http://maven.apache.org/guides/mini/guide-proxies.html ... but I am not familiar with Maven. Joris On Fri, Oct 31, 2014 at 2:36 PM, Sweta Rani swetarani3...@gmail.com wrote: Tried wget for all the given https, it was successful but still after that make was not successful and make check says YOU HAVE 3 DISABLED TESTS proxima@Centauri:~/mesos/build$ wget http://repo.maven.apache.org/maven2/org/apache/apache/11/apache-11.pom --2014-11-01 02:55:51-- http://repo.maven.apache.org/maven2/org/apache/apache/11/apache-11.pom Connecting to 10.3.100.207:8080... connected. Proxy request sent, awaiting response... 200 OK Length: 14811 (14K) [text/xml] Saving to: ‘apache-11.pom’ *Joris Van Remoortere* On Sat Nov 01 2014 at 2:36:27 AM Joris Van Remoortere jo...@mesosphere.io wrote: That seems like you might have a connectivity / firewall issue. Can you try just doing a wget on that artifact? On Fri, Oct 31, 2014 at 1:56 PM, Sweta Rani swetarani3...@gmail.com wrote: I tried it again but failed with following errors : make[1]: Entering directory `/home/proxima/mesos/build/src' Building mesos-0.21.0.jar ... [INFO] Scanning for projects... Downloading: http://repo.maven.apache.org/maven2/org/apache/apache/11/apache-11.pom [ERROR] The build could not read 1 project - [Help 1] [ERROR] [ERROR] The project org.apache.mesos:mesos:0.21.0 (/home/proxima/mesos/build/src/java/mesos.pom) has 1 error [ERROR] Non-resolvable parent POM: Could not transfer artifact org.apache:apache:pom:11 from/to central ( http://repo.maven.apache.org/maven2): Connection to http://repo.maven.apache.org refused and 'parent.relativePath' points at wrong local POM @ line 18, column 11: Connection refused - [Help 2] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/ProjectBuil dingException [ERROR] [Help 2] http://cwiki.apache.org/confluence/display/MAVEN/Unresolvabl eModelException make[1]: *** [java/target/mesos-0.21.0.jar] Error 1 make[1]: Leaving directory `/home/proxima/mesos/build/src' make: *** [all-recursive] Error 1 On Fri Oct 31 2014 at 1:09:15 PM Sweta Rani swetarani3...@gmail.com wrote: yaa.. followed the steps as directed in the documentation *Sweta Rani* *3rd Year Undergraduate* *Department of Electrical Engineering* *IIT Kharagpur http://www.iitkgp.ac.in/* On Fri, Oct 31, 2014 at 3:14 AM, Dominic Hamon dha...@twopensource.com wrote: Let's tackle the easy stuff: did you run bootstrap and configure as per the getting started doc? On Oct 30, 2014 2:04 PM, Sweta Rani swetarani3...@gmail.com wrote: Tried all the suggestions and tips given in JIRA and different google forums but still the error remains same. Making all in . make[1]: Entering directory `/home/proxima/mesos/build' make[1]: Nothing to be done for `all-am'. make[1]: Leaving directory `/home/proxima/mesos/build' Making all in 3rdparty make[1]: Entering directory `/home/proxima/mesos/build/3rdparty' make all-recursive make[2]: Entering directory `/home/proxima/mesos/build/3rdparty' Making all in libprocess make[3]: Entering directory `/home/proxima/mesos/build/3rdparty/libprocess' Making all in 3rdparty make[4]: Entering directory `/home/proxima/mesos/build/3rdparty/libprocess/3rdparty' make all-recursive make[5]: Entering directory `/home/proxima/mesos/build/3rdparty/libprocess/3rdparty' Making all in stout make[6]: Entering directory `/home/proxima/mesos/build/3rdparty/libprocess/3rdparty/stout' Making all in . make[7]: Entering directory `/home/proxima/mesos/build/3rdparty/libprocess/3rdparty/stout' make[7]: Nothing to be done for `all-am'. make[7]: Leaving directory `/home/proxima/mesos/build/3rdparty/libprocess/3rdparty/stout' Making all in include make[7]: Entering directory `/home/proxima/mesos/build/3rdparty/libprocess/3rdparty/stou t
Review Request 27461: Change process reparent test to be recursive search for init
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/27461/ --- Review request for mesos and Ian Downes. Bugs: MESOS-2025 https://issues.apache.org/jira/browse/MESOS-2025 Repository: mesos-git Description --- Reparenting does not always assign pid 1 (/sbin/init). If there is a user init such as init --user with some other pid, this will be the new parent. Modify os_tests to check up the parent tree, and succeed if there is a path to pid 1 without zombies along the way. This is not the cleanest fix, but I'm having trouble finding a way to find the appropriate init to check for. Diffs - 3rdparty/libprocess/3rdparty/stout/tests/os_tests.cpp 3f39017 Diff: https://reviews.apache.org/r/27461/diff/ Testing --- make check Thanks, Joris Van Remoortere
Re: Review Request 27461: Change process reparent test to be recursive search for init
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/27461/ --- (Updated Nov. 1, 2014, 3:03 a.m.) Review request for mesos and Ian Downes. Changes --- make loop logic easier on the eye. Bugs: MESOS-2025 https://issues.apache.org/jira/browse/MESOS-2025 Repository: mesos-git Description --- Reparenting does not always assign pid 1 (/sbin/init). If there is a user init such as init --user with some other pid, this will be the new parent. Modify os_tests to check up the parent tree, and succeed if there is a path to pid 1 without zombies along the way. This is not the cleanest fix, but I'm having trouble finding a way to find the appropriate init to check for. Diffs (updated) - 3rdparty/libprocess/3rdparty/stout/tests/os_tests.cpp 3f39017 Diff: https://reviews.apache.org/r/27461/diff/ Testing --- make check Thanks, Joris Van Remoortere
Re: Unable to install Mesos on Ubuntu 14.04. Error during 'make'
Did you verify that the settings.xml file was parsed using mvn -X? Specifically after the first few lines you will see something along the lines of: [DEBUG] Reading global settings from /usr/share/maven/conf/settings.xml [DEBUG] Reading user settings from /home/abc/.m2/settings.xml [DEBUG] Using local repository at /home/abc/.m2/repository Make sure your settings file is in the right location. On Mon, Nov 3, 2014 at 11:51 AM, Sweta Rani swetarani3...@gmail.com wrote: I use campus proxy configuration for internet. But I am unable to run make in installing Mesos on my Ubuntu -14.04, as it ends up with ERROR(1). Upon searching, I created settings.xml file inside .m2 with the required configuration but failed again. Please help me out, as it is greatly hindering me from my work. On Sat Nov 01 2014 at 3:38:57 AM Joris Van Remoortere jo...@mesosphere.io wrote: I think this suggests you're fetching through a proxy: *Proxy request sent, awaiting response... 200 OK* When I wget this is my output: wget http://repo.maven.apache.org/maven2/org/apache/apache/11/ apache-11.pom --2014-10-31 http://repo.maven.apache.org/maven2/org/apache/apache/11/apache-11.pom--2014-10-31 15:05:32-- http://repo.maven.apache.org/maven2/org/apache/apache/11/apache-11.pom Resolving repo.maven.apache.org (repo.maven.apache.org)... 23.235.47.215 Connecting to repo.maven.apache.org (repo.maven.apache.org)|23.235.47.215|:80... connected. *HTTP request sent, awaiting response... 200 OK* I think you might need to set up your proxy along these lines: http://maven.apache.org/guides/mini/guide-proxies.html ... but I am not familiar with Maven. Joris On Fri, Oct 31, 2014 at 2:36 PM, Sweta Rani swetarani3...@gmail.com wrote: Tried wget for all the given https, it was successful but still after that make was not successful and make check says YOU HAVE 3 DISABLED TESTS proxima@Centauri:~/mesos/build$ wget http://repo.maven.apache.org/maven2/org/apache/apache/11/apache-11.pom --2014-11-01 02:55:51-- http://repo.maven.apache.org/maven2/org/apache/apache/11/apache-11.pom Connecting to 10.3.100.207:8080... connected. Proxy request sent, awaiting response... 200 OK Length: 14811 (14K) [text/xml] Saving to: ‘apache-11.pom’ *Joris Van Remoortere* On Sat Nov 01 2014 at 2:36:27 AM Joris Van Remoortere jo...@mesosphere.io wrote: That seems like you might have a connectivity / firewall issue. Can you try just doing a wget on that artifact? On Fri, Oct 31, 2014 at 1:56 PM, Sweta Rani swetarani3...@gmail.com wrote: I tried it again but failed with following errors : make[1]: Entering directory `/home/proxima/mesos/build/src' Building mesos-0.21.0.jar ... [INFO] Scanning for projects... Downloading: http://repo.maven.apache.org/maven2/org/apache/apache/11/ apache-11.pom [ERROR] The build could not read 1 project - [Help 1] [ERROR] [ERROR] The project org.apache.mesos:mesos:0.21.0 (/home/proxima/mesos/build/src/java/mesos.pom) has 1 error [ERROR] Non-resolvable parent POM: Could not transfer artifact org.apache:apache:pom:11 from/to central ( http://repo.maven.apache.org/maven2): Connection to http://repo.maven.apache.org refused and 'parent.relativePath' points at wrong local POM @ line 18, column 11: Connection refused - [Help 2] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/ProjectBuil dingException [ERROR] [Help 2] http://cwiki.apache.org/confluence/display/MAVEN/Unresolvabl eModelException make[1]: *** [java/target/mesos-0.21.0.jar] Error 1 make[1]: Leaving directory `/home/proxima/mesos/build/src' make: *** [all-recursive] Error 1 On Fri Oct 31 2014 at 1:09:15 PM Sweta Rani swetarani3...@gmail.com wrote: yaa.. followed the steps as directed in the documentation *Sweta Rani* *3rd Year Undergraduate* *Department of Electrical Engineering* *IIT Kharagpur http://www.iitkgp.ac.in/* On Fri, Oct 31, 2014 at 3:14 AM, Dominic Hamon dha...@twopensource.com wrote: Let's tackle the easy stuff: did you run bootstrap and configure as per the getting started doc? On Oct 30, 2014 2:04 PM, Sweta Rani swetarani3...@gmail.com wrote: Tried all the suggestions and tips given in JIRA and different google forums but still the error remains same. Making all in . make[1]: Entering
Re: Review Request 27461: fix OsTest.killtreeNoRoot: check for reparent and not zombie
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/27461/ --- (Updated Nov. 4, 2014, 12:17 a.m.) Review request for mesos and Ian Downes. Changes --- Modify checks to address Ian's comment. Summary (updated) - fix OsTest.killtreeNoRoot: check for reparent and not zombie Bugs: MESOS-2025 https://issues.apache.org/jira/browse/MESOS-2025 Repository: mesos-git Description (updated) --- Reparenting does not always assign pid 1 (/sbin/init). If there is a user init such as init --user with some other pid, this will be the new parent. Modify os_tests to check that the subtree has been reparented to a process different from its original parent (a.k.a. child) and that it is not a zombie. Diffs (updated) - 3rdparty/libprocess/3rdparty/stout/tests/os_tests.cpp 3f39017 Diff: https://reviews.apache.org/r/27461/diff/ Testing --- make check Thanks, Joris Van Remoortere
Re: Review Request 27461: fix OsTest.killtreeNoRoot: check for reparent and not zombie
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/27461/ --- (Updated Nov. 4, 2014, 12:36 a.m.) Review request for mesos and Ian Downes. Changes --- Clarifying comment as per Ian's request. Bugs: MESOS-2025 https://issues.apache.org/jira/browse/MESOS-2025 Repository: mesos-git Description --- Reparenting does not always assign pid 1 (/sbin/init). If there is a user init such as init --user with some other pid, this will be the new parent. Modify os_tests to check that the subtree has been reparented to a process different from its original parent (a.k.a. child) and that it is not a zombie. Diffs (updated) - 3rdparty/libprocess/3rdparty/stout/tests/os_tests.cpp 3f39017 Diff: https://reviews.apache.org/r/27461/diff/ Testing --- make check Thanks, Joris Van Remoortere
Re: Review Request 27461: fix OsTest.killtreeNoRoot: check for reparent and not zombie
On Nov. 4, 2014, 12:37 a.m., Adam B wrote: 3rdparty/libprocess/3rdparty/stout/tests/os_tests.cpp, line 661 https://reviews.apache.org/r/27461/diff/3/?file=747878#file747878line661 Couldn't this be grandchild.children.front()? we'd have to keep a reference to the object rather than the implicit conversion to pid_t for that to work. - Joris --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/27461/#review59692 --- On Nov. 4, 2014, 12:36 a.m., Joris Van Remoortere wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/27461/ --- (Updated Nov. 4, 2014, 12:36 a.m.) Review request for mesos and Ian Downes. Bugs: MESOS-2025 https://issues.apache.org/jira/browse/MESOS-2025 Repository: mesos-git Description --- Reparenting does not always assign pid 1 (/sbin/init). If there is a user init such as init --user with some other pid, this will be the new parent. Modify os_tests to check that the subtree has been reparented to a process different from its original parent (a.k.a. child) and that it is not a zombie. Diffs - 3rdparty/libprocess/3rdparty/stout/tests/os_tests.cpp 3f39017 Diff: https://reviews.apache.org/r/27461/diff/ Testing --- make check Thanks, Joris Van Remoortere
Re: Review Request 27461: fix OsTest.killtreeNoRoot: check for reparent and not zombie
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/27461/ --- (Updated Nov. 4, 2014, 12:49 a.m.) Review request for mesos and Ian Downes. Changes --- Address Adam's comments. Bugs: MESOS-2025 https://issues.apache.org/jira/browse/MESOS-2025 Repository: mesos-git Description --- Reparenting does not always assign pid 1 (/sbin/init). If there is a user init such as init --user with some other pid, this will be the new parent. Modify os_tests to check that the subtree has been reparented to a process different from its original parent (a.k.a. child) and that it is not a zombie. Diffs (updated) - 3rdparty/libprocess/3rdparty/stout/tests/os_tests.cpp 3f39017 Diff: https://reviews.apache.org/r/27461/diff/ Testing --- make check Thanks, Joris Van Remoortere
Re: Unable to install Mesos on Ubuntu 14.04. Error during 'make'
You need to make the directory and set the permissions to the user you are running the master as. For example: sudo mkdir /var/lib/mesos sudo chown jmlvanre /var/lib/mesos [jmlvanre] run mesos master... On Tue, Nov 4, 2014 at 2:07 PM, Sweta Rani swetarani3...@gmail.com wrote: Thanks for your help. I have installed mesos but unable to start mesos master by the command ./bin/mesos-master.sh --ip=127.0.0.1 --work_dir=/var/lib/mesos. The file exists as mesos-master.sh.in in the bin folder. It reports: Failed to create work directory '/var/lib/mesos': Permission denied *Sweta Rani* On Tue, Nov 4, 2014 at 2:50 PM, Adam Bordelon a...@mesosphere.io wrote: As an alternate workaround, if you don't need the Java support (you're not running a JVM framework like Spark/Hadoop/Marathon), then you could use `configure --disable-java` to skip all the maven/pom build steps. By disabling Java, you could at least check that the rest of Mesos builds and runs fine, which would still work with C++, Python, or Go frameworks, or even any Java frameworks using the pure Java bindings (jesos). On Mon, Nov 3, 2014 at 2:01 PM, Joris Van Remoortere jo...@mesosphere.io wrote: Did you verify that the settings.xml file was parsed using mvn -X? Specifically after the first few lines you will see something along the lines of: [DEBUG] Reading global settings from /usr/share/maven/conf/settings.xml [DEBUG] Reading user settings from /home/abc/.m2/settings.xml [DEBUG] Using local repository at /home/abc/.m2/repository Make sure your settings file is in the right location. On Mon, Nov 3, 2014 at 11:51 AM, Sweta Rani swetarani3...@gmail.com wrote: I use campus proxy configuration for internet. But I am unable to run make in installing Mesos on my Ubuntu -14.04, as it ends up with ERROR(1). Upon searching, I created settings.xml file inside .m2 with the required configuration but failed again. Please help me out, as it is greatly hindering me from my work. On Sat Nov 01 2014 at 3:38:57 AM Joris Van Remoortere jo...@mesosphere.io wrote: I think this suggests you're fetching through a proxy: *Proxy request sent, awaiting response... 200 OK* When I wget this is my output: wget http://repo.maven.apache.org/maven2/org/apache/apache/11/ apache-11.pom --2014-10-31 http://repo.maven.apache.org/maven2/org/apache/apache/11/apache-11.pom--2014-10-31 15:05:32-- http://repo.maven.apache.org/maven2/org/apache/apache/11/apache-11.pom Resolving repo.maven.apache.org (repo.maven.apache.org)... 23.235.47.215 Connecting to repo.maven.apache.org (repo.maven.apache.org)|23.235.47.215|:80... connected. *HTTP request sent, awaiting response... 200 OK* I think you might need to set up your proxy along these lines: http://maven.apache.org/guides/mini/guide-proxies.html ... but I am not familiar with Maven. Joris On Fri, Oct 31, 2014 at 2:36 PM, Sweta Rani swetarani3...@gmail.com wrote: Tried wget for all the given https, it was successful but still after that make was not successful and make check says YOU HAVE 3 DISABLED TESTS proxima@Centauri:~/mesos/build$ wget http://repo.maven.apache.org/maven2/org/apache/apache/11/apache-11.pom --2014-11-01 02:55:51-- http://repo.maven.apache.org/maven2/org/apache/apache/11/apache-11.pom Connecting to 10.3.100.207:8080... connected. Proxy request sent, awaiting response... 200 OK Length: 14811 (14K) [text/xml] Saving to: ‘apache-11.pom’ *Joris Van Remoortere* On Sat Nov 01 2014 at 2:36:27 AM Joris Van Remoortere jo...@mesosphere.io wrote: That seems like you might have a connectivity / firewall issue. Can you try just doing a wget on that artifact? On Fri, Oct 31, 2014 at 1:56 PM, Sweta Rani swetarani3...@gmail.com wrote: I tried it again but failed with following errors : make[1]: Entering directory `/home/proxima/mesos/build/src' Building mesos-0.21.0.jar ... [INFO] Scanning for projects... Downloading: http://repo.maven.apache.org/maven2/org/apache/apache/11/ apache-11.pom [ERROR] The build could not read 1 project - [Help 1] [ERROR] [ERROR] The project org.apache.mesos:mesos:0.21.0 (/home/proxima/mesos/build/src/java/mesos.pom) has 1 error [ERROR] Non-resolvable parent POM: Could not transfer artifact org.apache:apache:pom:11 from/to central ( http://repo.maven.apache.org/maven2): Connection to http://repo.maven.apache.org refused and 'parent.relativePath' points at wrong local POM @ line 18, column 11
Re: Unable to install Mesos on Ubuntu 14.04. Error during 'make'
If you run: mesos-slave.sh --help Then you will see all the options you can pass to the slave. One of them is --port=VALUE Joris On Thu, Nov 6, 2014 at 6:39 AM, Sweta Rani swetarani3...@gmail.com wrote: How can I change port for mesos-slave from default 5051 to 5052? *Sweta Rani* On Thu, Nov 6, 2014 at 7:58 PM, Sweta Rani swetarani3...@gmail.com wrote: Alex, Done with master but unable to run slave using command ./bin/mesos-slave.sh --master=127.0.0.1:5050 Also the port 5051 is allocated with random work '-'. See this: proxima@proxima:~/mesos/build$ netstat -ltnp | grep ':5051' (Not all processes could be identified, non-owned process info will not be shown, you would have to be root to see it all.) tcp0 0 0.0.0.0:50510.0.0.0:* LISTEN - proxima@proxima:~/mesos/build$ sudo kill -9 - kill: failed to parse argument: '-' proxima@proxima:~/mesos/build$ netstat -ltnp | grep ':5051' (Not all processes could be identified, non-owned process info will not be shown, you would have to be root to see it all.) tcp0 0 0.0.0.0:50510.0.0.0:* LISTEN - proxima@:~/mesos/build$ ./bin/mesos-slave.sh --master=127.0.0.1:5050 WARNING: Logging before InitGoogleLogging() is written to STDERR F1106 19:51:22.737992 6367 process.cpp:1532] Failed to initialize, bind 0.0.0.0:5051: Address already in use [98] *** Check failure stack trace: *** Aborted (core dumped) *Sweta Rani* On Thu, Nov 6, 2014 at 7:07 PM, Alex Rukletsov a...@mesosphere.io wrote: Sweta, have you tried another port? On Thu, Nov 6, 2014 at 2:24 PM, Sweta Rani swetarani3...@gmail.com wrote: Joris Done with above commands, but now it gives following error: WARNING: Logging before InitGoogleLogging() is written to STDERR F1106 18:50:48.117947 9706 process.cpp:1532] Failed to initialize, bind 127.0.0.1:5050: Address already in use [98] *** Check failure stack trace: *** I tried to solve it by http://stackoverflow.com/questions/10745878/ubuntu-error-with-apache-98address-already-in-use but it didnt work for me. *Sweta Rani* On Wed, Nov 5, 2014 at 3:50 AM, Joris Van Remoortere jo...@mesosphere.io wrote: You need to make the directory and set the permissions to the user you are running the master as. For example: sudo mkdir /var/lib/mesos sudo chown jmlvanre /var/lib/mesos [jmlvanre] run mesos master... On Tue, Nov 4, 2014 at 2:07 PM, Sweta Rani swetarani3...@gmail.com wrote: Thanks for your help. I have installed mesos but unable to start mesos master by the command ./bin/mesos-master.sh --ip=127.0.0.1 --work_dir=/var/lib/mesos. The file exists as mesos-master.sh.in in the bin folder. It reports: Failed to create work directory '/var/lib/mesos': Permission denied *Sweta Rani* On Tue, Nov 4, 2014 at 2:50 PM, Adam Bordelon a...@mesosphere.io wrote: As an alternate workaround, if you don't need the Java support (you're not running a JVM framework like Spark/Hadoop/Marathon), then you could use `configure --disable-java` to skip all the maven/pom build steps. By disabling Java, you could at least check that the rest of Mesos builds and runs fine, which would still work with C++, Python, or Go frameworks, or even any Java frameworks using the pure Java bindings (jesos). On Mon, Nov 3, 2014 at 2:01 PM, Joris Van Remoortere jo...@mesosphere.io wrote: Did you verify that the settings.xml file was parsed using mvn -X? Specifically after the first few lines you will see something along the lines of: [DEBUG] Reading global settings from /usr/share/maven/conf/settings.xml [DEBUG] Reading user settings from /home/abc/.m2/settings.xml [DEBUG] Using local repository at /home/abc/.m2/repository Make sure your settings file is in the right location. On Mon, Nov 3, 2014 at 11:51 AM, Sweta Rani swetarani3...@gmail.com wrote: I use campus proxy configuration for internet. But I am unable to run make in installing Mesos on my Ubuntu -14.04, as it ends up with ERROR(1). Upon searching, I created settings.xml file inside .m2 with the required configuration but failed again. Please help me out, as it is greatly hindering me from my work. On Sat Nov 01 2014 at 3:38:57 AM Joris Van Remoortere jo...@mesosphere.io wrote: I think this suggests you're fetching through a proxy: *Proxy request sent, awaiting response... 200 OK* When I wget this is my output: wget http://repo.maven.apache.org/maven2
Re: Review Request 26476: Remove dynamic allocation from Option.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/26476/ --- (Updated Nov. 6, 2014, 7:13 p.m.) Review request for mesos, Benjamin Hindman and Niklas Nielsen. Changes --- Add the JIRA issue for this. Bugs: MESOS-1991 https://issues.apache.org/jira/browse/MESOS-1991 Repository: mesos-git Description --- Remove dynamic allocations from Option class. Diffs - 3rdparty/libprocess/3rdparty/stout/include/stout/option.hpp 47fe92c Diff: https://reviews.apache.org/r/26476/diff/ Testing --- make check support/mesos-style.py valgrind (reduced allocation count) Thanks, Joris Van Remoortere
Re: SVN : Connection Refused
http://stackoverflow.com/questions/1491180/how-to-configure-a-http-proxy-for-svn On Fri, Nov 7, 2014 at 8:10 AM, Sweta Rani swetarani3...@gmail.com wrote: Upon running svn checkout http://svn.apache.org/repos/asf/mesos/site mesos-last-attempt, it gives following error: svn: E000111: Unable to connect to a repository at URL ' http://svn.apache.org/repos/asf/mesos/site' svn: E000111: Error running context: Connection refused I am using proxy so I tried even: export HTTP_PROXY=http://10.3.100.207:8080; But still after this it didn't work. Thanks, Sweta
Re: Review Request 27496: Replaced Timer::create/cancel with Clock::timer/cancel.
On Nov. 12, 2014, 1:04 a.m., Ben Mahler wrote: 3rdparty/libprocess/include/process/timer.hpp, line 53 https://reviews.apache.org/r/27496/diff/1/?file=747095#file747095line53 Looks like you need an include for clock.hpp..? Does this compile if Clock is not transitively included? Friendship is an implicit forward declaration within the given namespace, so in this case the include is not required. - Joris --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/27496/#review60911 --- On Nov. 11, 2014, 4:52 p.m., Benjamin Hindman wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/27496/ --- (Updated Nov. 11, 2014, 4:52 p.m.) Review request for mesos, Ben Mahler, Jie Yu, Joris Van Remoortere, Niklas Nielsen, and Vinod Kone. Repository: mesos-git Description --- See summary. Diffs - 3rdparty/libprocess/include/process/c++11/delay.hpp 5f686db1df50829a5aad76eb91ea6a86e8969c1d 3rdparty/libprocess/include/process/clock.hpp eb157cac996c48fedc69002b9d80adaba1ed85d1 3rdparty/libprocess/include/process/delay.hpp 487f652c9e9b7c8c3aa8b4edc9e59789cffd8da9 3rdparty/libprocess/include/process/future.hpp 46ae16b0bbce79005f5ed8711be663eeeb8f4bcf 3rdparty/libprocess/include/process/timer.hpp e2f55636b520d43a8c9efa28ad4fe5f75c4f5573 3rdparty/libprocess/src/process.cpp 85fb9958342f0bcdde322d9c55333126e6f86668 Diff: https://reviews.apache.org/r/27496/diff/ Testing --- make check Thanks, Benjamin Hindman
