[jira] [Created] (MESOS-2932) There is a typo in docs/docker-containerizer.md file
Chen Zhiwei created MESOS-2932: -- Summary: There is a typo in docs/docker-containerizer.md file Key: MESOS-2932 URL: https://issues.apache.org/jira/browse/MESOS-2932 Project: Mesos Issue Type: Bug Components: documentation Affects Versions: 0.22.1 Reporter: Chen Zhiwei Priority: Minor A docker image currently supports having a entrypoint and/or a default command. The `a entrypoint` should be `an entrypoint`. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-2925) Invalid usage of ATOMIC_FLAG_INIT in member initialization
[ https://issues.apache.org/jira/browse/MESOS-2925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14600641#comment-14600641 ] Paul Brett commented on MESOS-2925: --- Thanks Michael - updated code is posted on reviewboard. > Invalid usage of ATOMIC_FLAG_INIT in member initialization > -- > > Key: MESOS-2925 > URL: https://issues.apache.org/jira/browse/MESOS-2925 > Project: Mesos > Issue Type: Bug > Components: libprocess >Affects Versions: 0.23.0 >Reporter: Paul Brett >Assignee: Paul Brett > > The C++ specification states: > The macro ATOMIC_FLAG_INIT shall be defined in such a way that it can be used > to initialize an object of type atomic_flag to the clear state. The macro can > be used in the form: "atomic_flag guard = ATOMIC_FLAG_INIT; "It is > unspecified whether the macro can be used in other initialization contexts." > Clang catches this (although reports it erroneously as a braced scaled init > issue) and refuses to compile libprocess. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (MESOS-2925) Invalid usage of ATOMIC_FLAG_INIT in member initialization
[ https://issues.apache.org/jira/browse/MESOS-2925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Brett reassigned MESOS-2925: - Assignee: Paul Brett > Invalid usage of ATOMIC_FLAG_INIT in member initialization > -- > > Key: MESOS-2925 > URL: https://issues.apache.org/jira/browse/MESOS-2925 > Project: Mesos > Issue Type: Bug > Components: libprocess >Affects Versions: 0.23.0 >Reporter: Paul Brett >Assignee: Paul Brett > > The C++ specification states: > The macro ATOMIC_FLAG_INIT shall be defined in such a way that it can be used > to initialize an object of type atomic_flag to the clear state. The macro can > be used in the form: "atomic_flag guard = ATOMIC_FLAG_INIT; "It is > unspecified whether the macro can be used in other initialization contexts." > Clang catches this (although reports it erroneously as a braced scaled init > issue) and refuses to compile libprocess. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-2931) Add explanation of master bootstrap process to Operational Guide
[ https://issues.apache.org/jira/browse/MESOS-2931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Nugent updated MESOS-2931: - Description: When Mesos starts up, masters come up in an empty bootstrap state. End users may find that they experience failures without an apparent cause. The documentation for the operational guide should lay out the requirements for the bootstrap process and its behavior. See MESOS-2148 for an example of an issue filed when someone wasn't aware of the bootstrap process. was: When Mesos starts up, masters come up in an empty bootstrap state. End users may find that they experience failures without an apparent cause. The documentation for the operational guide should lay out the requirements for the bootstrap process and its behavior. See MESOS-2148 for an example of an issue filed when someone wasn't aware of the problem. > Add explanation of master bootstrap process to Operational Guide > > > Key: MESOS-2931 > URL: https://issues.apache.org/jira/browse/MESOS-2931 > Project: Mesos > Issue Type: Documentation > Components: documentation >Affects Versions: 0.22.1 >Reporter: Daniel Nugent >Priority: Minor > > When Mesos starts up, masters come up in an empty bootstrap state. End users > may find that they experience failures without an apparent cause. The > documentation for the operational guide should lay out the requirements for > the bootstrap process and its behavior. > See MESOS-2148 for an example of an issue filed when someone wasn't aware of > the bootstrap process. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (MESOS-2931) Add explanation of master bootstrap process to Operational Guide
Daniel Nugent created MESOS-2931: Summary: Add explanation of master bootstrap process to Operational Guide Key: MESOS-2931 URL: https://issues.apache.org/jira/browse/MESOS-2931 Project: Mesos Issue Type: Documentation Components: documentation Affects Versions: 0.22.1 Reporter: Daniel Nugent Priority: Minor When Mesos starts up, masters come up in an empty bootstrap state. End users may find that they experience failures without an apparent cause. The documentation for the operational guide should lay out the requirements for the bootstrap process and its behavior. See MESOS-2148 for an example of an issue filed when someone wasn't aware of the problem. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-2925) Invalid usage of ATOMIC_FLAG_INIT in member initialization
[
https://issues.apache.org/jira/browse/MESOS-2925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14600512#comment-14600512
]
Michael Park commented on MESOS-2925:
-
[~pbrett] I gave it a {{ShipIt}} on the review but how about we use the inline
member initializer syntax instead?
{code}
class Obj
{
public:
std::atomic_flag lock = ATOMIC_FLAG_INIT;
};
{code}
> Invalid usage of ATOMIC_FLAG_INIT in member initialization
> --
>
> Key: MESOS-2925
> URL: https://issues.apache.org/jira/browse/MESOS-2925
> Project: Mesos
> Issue Type: Bug
> Components: libprocess
>Affects Versions: 0.23.0
>Reporter: Paul Brett
>
> The C++ specification states:
> The macro ATOMIC_FLAG_INIT shall be defined in such a way that it can be used
> to initialize an object of type atomic_flag to the clear state. The macro can
> be used in the form: "atomic_flag guard = ATOMIC_FLAG_INIT; "It is
> unspecified whether the macro can be used in other initialization contexts."
> Clang catches this (although reports it erroneously as a braced scaled init
> issue) and refuses to compile libprocess.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (MESOS-2928) Update stout to #include headers for symbols we rely on
[ https://issues.apache.org/jira/browse/MESOS-2928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14600455#comment-14600455 ] Paul Brett edited comment on MESOS-2928 at 6/25/15 12:58 AM: - https://reviews.apache.org/r/35861/ was (Author: pbrett): https://reviews.apache.org/r/35860/ > Update stout to #include headers for symbols we rely on > --- > > Key: MESOS-2928 > URL: https://issues.apache.org/jira/browse/MESOS-2928 > Project: Mesos > Issue Type: Bug >Reporter: Paul Brett >Assignee: Paul Brett > > Update mesos to #include headers for symbols we rely on -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-2928) Update stout to #include headers for symbols we rely on
[ https://issues.apache.org/jira/browse/MESOS-2928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14600455#comment-14600455 ] Paul Brett commented on MESOS-2928: --- https://reviews.apache.org/r/35860/ > Update stout to #include headers for symbols we rely on > --- > > Key: MESOS-2928 > URL: https://issues.apache.org/jira/browse/MESOS-2928 > Project: Mesos > Issue Type: Bug >Reporter: Paul Brett >Assignee: Paul Brett > > Update mesos to #include headers for symbols we rely on -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-2832) Enable configuring Mesos with environment variables without having them leak to tasks launched
[ https://issues.apache.org/jira/browse/MESOS-2832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benjamin Hindman updated MESOS-2832: Story Points: 8 > Enable configuring Mesos with environment variables without having them leak > to tasks launched > -- > > Key: MESOS-2832 > URL: https://issues.apache.org/jira/browse/MESOS-2832 > Project: Mesos > Issue Type: Wish >Reporter: Cody Maloney >Assignee: Benjamin Hindman >Priority: Critical > Labels: mesosphere > Fix For: 0.23.0 > > > Currently if mesos is configured with environment variables (MESOS_MODULES), > those show up in every task which is launched unless the executor explicitly > cleans them up. > If the task being launched happens to be something libprocess / mesos based, > this can often prevent the task from starting up (A scheduler has issues > loading a module intended for the slave). > There are also cases where it would be nice to be able to change what the > PATH is that tasks launch with (the host may have more in the path than tasks > are supposed to / allowed to depend upon). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (MESOS-2832) Enable configuring Mesos with environment variables without having them leak to tasks launched
[ https://issues.apache.org/jira/browse/MESOS-2832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benjamin Hindman resolved MESOS-2832. - Resolution: Fixed Fix Version/s: 0.23.0 commit 46dc9979e9cc38d36dc7300db13af39bdfbfd52e Author: Benjamin Hindman Date: Mon Jun 15 00:29:20 2015 -0700 Added 'executor_environment_variables' flag to slave. This new flag, 'executor_environment_variables', let's an operator specify the environment variables that should be passed to an executor, and thus, any of it's tasks. By default, an executor will inherit the environment variables of the slave. Review: https://reviews.apache.org/r/35567 > Enable configuring Mesos with environment variables without having them leak > to tasks launched > -- > > Key: MESOS-2832 > URL: https://issues.apache.org/jira/browse/MESOS-2832 > Project: Mesos > Issue Type: Wish >Reporter: Cody Maloney >Assignee: Benjamin Hindman >Priority: Critical > Labels: mesosphere > Fix For: 0.23.0 > > > Currently if mesos is configured with environment variables (MESOS_MODULES), > those show up in every task which is launched unless the executor explicitly > cleans them up. > If the task being launched happens to be something libprocess / mesos based, > this can often prevent the task from starting up (A scheduler has issues > loading a module intended for the slave). > There are also cases where it would be nice to be able to change what the > PATH is that tasks launch with (the host may have more in the path than tasks > are supposed to / allowed to depend upon). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-2551) C++ Scheduler library should send Call messages to Master
[ https://issues.apache.org/jira/browse/MESOS-2551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod Kone updated MESOS-2551: -- https://reviews.apache.org/r/35855/ https://reviews.apache.org/r/35856/ https://reviews.apache.org/r/35857/ https://reviews.apache.org/r/35858/ TODO: SUBSCRIBE call. > C++ Scheduler library should send Call messages to Master > - > > Key: MESOS-2551 > URL: https://issues.apache.org/jira/browse/MESOS-2551 > Project: Mesos > Issue Type: Story >Reporter: Vinod Kone >Assignee: Vinod Kone > Labels: tech-debt, twitter > > Currently, the C++ library sends different messages to Master instead of a > single Call message. To vet the new Call API it should send Call messages. > Master should be updated to handle all types of Calls. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-2340) Publish JSON in ZK instead of serialized MasterInfo
[ https://issues.apache.org/jira/browse/MESOS-2340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14600428#comment-14600428 ] Vinod Kone commented on MESOS-2340: --- 0.23.0 and 0.24.0 components (scheduler, master) should always be able to talk to each other. > Publish JSON in ZK instead of serialized MasterInfo > --- > > Key: MESOS-2340 > URL: https://issues.apache.org/jira/browse/MESOS-2340 > Project: Mesos > Issue Type: Improvement > Components: leader election >Reporter: Zameer Manji >Assignee: Marco Massenzio > > Currently to discover the master a client needs the ZK node location and > access to the MasterInfo protobuf so it can deserialize the binary blob in > the node. > I think it would be nice to publish JSON (like Twitter's ServerSets) so > clients are not tied to protobuf to do service discovery. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (MESOS-2930) Allow the Resource Estimator to express over-allocation of revocable resources.
Benjamin Mahler created MESOS-2930: -- Summary: Allow the Resource Estimator to express over-allocation of revocable resources. Key: MESOS-2930 URL: https://issues.apache.org/jira/browse/MESOS-2930 Project: Mesos Issue Type: Improvement Components: slave Reporter: Benjamin Mahler Currently the resource estimator returns the amount of oversubscription resources that are available, since resources cannot be negative, this allows the resource estimator to express the following: (1) Return empty resources: We are fully allocated for oversubscription resources. (2) Return non-empty resources: We are under-allocated for oversubscription resources. In other words, some are available. However, there is an additional situation that we cannot express: (3) Analogous to returning non-empty "negative" resources: We are over-allocated for oversubscription resources. Do not re-offer any of the over-allocated oversubscription resources that are recovered. Without (3), the slave can only shrink the total pool of oversubscription resources by returning (1) as resources are recovered, until the pool is shrunk to the desired size. However, this approach is only best-effort, it's possible for a framework to launch more tasks in the window of time (15 seconds by default) that the slave polls the estimator. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-2340) Publish JSON in ZK instead of serialized MasterInfo
[ https://issues.apache.org/jira/browse/MESOS-2340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14600410#comment-14600410 ] Charles Allen commented on MESOS-2340: -- Is rolling back from 0.24 to 0.23 during a failed rolling upgrade or immediately after something that is intended to be supported? > Publish JSON in ZK instead of serialized MasterInfo > --- > > Key: MESOS-2340 > URL: https://issues.apache.org/jira/browse/MESOS-2340 > Project: Mesos > Issue Type: Improvement > Components: leader election >Reporter: Zameer Manji >Assignee: Marco Massenzio > > Currently to discover the master a client needs the ZK node location and > access to the MasterInfo protobuf so it can deserialize the binary blob in > the node. > I think it would be nice to publish JSON (like Twitter's ServerSets) so > clients are not tied to protobuf to do service discovery. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (MESOS-2926) Extend mesos-style.py/cpplint.py to check #include files
Paul Brett created MESOS-2926: - Summary: Extend mesos-style.py/cpplint.py to check #include files Key: MESOS-2926 URL: https://issues.apache.org/jira/browse/MESOS-2926 Project: Mesos Issue Type: Bug Reporter: Paul Brett Assignee: Paul Brett cpplint.py provides the capability to enforce the style guide requirements for #including everything you use and ordering files based on type but it does not work for mesos because we do use #include <...> for project files where it expects #include "...". We should update the style checker to support our include usage and then turn it on by default in the commit hook. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (MESOS-2927) Update mesos to #include headers for symbols we rely on
Paul Brett created MESOS-2927: - Summary: Update mesos to #include headers for symbols we rely on Key: MESOS-2927 URL: https://issues.apache.org/jira/browse/MESOS-2927 Project: Mesos Issue Type: Bug Reporter: Paul Brett Assignee: Paul Brett Update mesos to #include headers for symbols we rely on -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (MESOS-2929) Update libprocess to #include headers for symbols we rely on
Paul Brett created MESOS-2929: - Summary: Update libprocess to #include headers for symbols we rely on Key: MESOS-2929 URL: https://issues.apache.org/jira/browse/MESOS-2929 Project: Mesos Issue Type: Bug Reporter: Paul Brett Assignee: Paul Brett -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (MESOS-2928) Update stout to #include headers for symbols we rely on
Paul Brett created MESOS-2928: - Summary: Update stout to #include headers for symbols we rely on Key: MESOS-2928 URL: https://issues.apache.org/jira/browse/MESOS-2928 Project: Mesos Issue Type: Bug Reporter: Paul Brett Assignee: Paul Brett Update mesos to #include headers for symbols we rely on -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-1187) precision errors with allocation calculations
[ https://issues.apache.org/jira/browse/MESOS-1187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14600359#comment-14600359 ] Ian Downes commented on MESOS-1187: --- Was fixed-point considered? > precision errors with allocation calculations > - > > Key: MESOS-1187 > URL: https://issues.apache.org/jira/browse/MESOS-1187 > Project: Mesos > Issue Type: Bug > Components: allocation, master >Reporter: aniruddha sathaye > > As allocations are stored/transmitted as doubles many a times precision > errors creep in. > we have seen erroneous share calculations happen only because of floating > point arithmetic. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-2824) Support pre-fetching default container image on slave startup
[ https://issues.apache.org/jira/browse/MESOS-2824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Xu updated MESOS-2824: -- Assignee: (was: Yan Xu) > Support pre-fetching default container image on slave startup > - > > Key: MESOS-2824 > URL: https://issues.apache.org/jira/browse/MESOS-2824 > Project: Mesos > Issue Type: Improvement > Components: isolation >Affects Versions: 0.23.0 >Reporter: Ian Downes >Priority: Minor > Labels: twitter > > Default container images can be specified with the --default_container_info > flag to the slave. This may be a large image that will take a long time to > initially fetch/hash/extract when the first container is provisioned. Add > optional support to start fetching the image when the slave starts and > consider not registering until the fetch is complete. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-2824) Support pre-fetching default container image on slave startup
[ https://issues.apache.org/jira/browse/MESOS-2824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14600356#comment-14600356 ] Yan Xu commented on MESOS-2824: --- We ([~vinodkone], [~idownes] and myself) discussed the pre-fetching strategies but it seemed unacceptable to have either - the slave to be blocked for extended period of time (minutes) which delays the communication between the executor and scheduler, or - the first task that uses this image to be blocked for a long time to wait for the container image to be ready. Pre-fetching, either started automatically or upon explicit requests is a good idea but we'll probably not do it on slave startup. > Support pre-fetching default container image on slave startup > - > > Key: MESOS-2824 > URL: https://issues.apache.org/jira/browse/MESOS-2824 > Project: Mesos > Issue Type: Improvement > Components: isolation >Affects Versions: 0.23.0 >Reporter: Ian Downes >Assignee: Yan Xu >Priority: Minor > Labels: twitter > > Default container images can be specified with the --default_container_info > flag to the slave. This may be a large image that will take a long time to > initially fetch/hash/extract when the first container is provisioned. Add > optional support to start fetching the image when the slave starts and > consider not registering until the fetch is complete. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-2340) Publish JSON in ZK instead of serialized MasterInfo
[ https://issues.apache.org/jira/browse/MESOS-2340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14600350#comment-14600350 ] Marco Massenzio commented on MESOS-2340: Update sent to [the user mailing list](http://www.mail-archive.com/user@mesos.apache.org/msg03691.html) > Publish JSON in ZK instead of serialized MasterInfo > --- > > Key: MESOS-2340 > URL: https://issues.apache.org/jira/browse/MESOS-2340 > Project: Mesos > Issue Type: Improvement > Components: leader election >Reporter: Zameer Manji >Assignee: Marco Massenzio > > Currently to discover the master a client needs the ZK node location and > access to the MasterInfo protobuf so it can deserialize the binary blob in > the node. > I think it would be nice to publish JSON (like Twitter's ServerSets) so > clients are not tied to protobuf to do service discovery. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-898) Introduce CMake as an alternative build system.
[ https://issues.apache.org/jira/browse/MESOS-898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14600293#comment-14600293 ] Alex Clemmer commented on MESOS-898: [~vi...@twitter.com] I would be very, very disappointed in any Apache project that did not have community involvement in a decision like this. > Introduce CMake as an alternative build system. > --- > > Key: MESOS-898 > URL: https://issues.apache.org/jira/browse/MESOS-898 > Project: Mesos > Issue Type: Epic > Components: build >Reporter: Timothy St. Clair >Assignee: Alex Clemmer > Labels: build > > This is a rather substantial undertaking, so I would want upstream > debate+buy-in prior to full commitment. The basic premise is: upstream > rebundles several of its dependencies in part to tightly control its stack. > This is not out of the norm, but in order to be picked up by distribution > channels it needs to built against system dependencies, and rebundling is > strictly forbidden. Given that the mesos primary target platform are > data-center distributions such as RHEL/CENTOS/SL it makes sense to still have > bundling support for those who do not have dependencies in their channels > "yet". This is where cmake can be win with it's uber macros > (http://www.cmake.org/cmake/help/v2.8.8/cmake.html#module:ExternalProject). > I do not know of any equivalent in the autotools world, other then to brew > your own solution. I've done this type of work in the past, and completely > transformed condor and would leverage a lot of the work that was done there. > I currently have a tracking branch where I've started this work, but before I > go off into the woods, it makes sense to have a debate in public. > The primary benefits are: > 1. Enable downstream channels to easily distro without carrying a large patch > sets. > 2. Still support existing "non-proper" distribution methods. > 3. Harden / future proof dependent interfaces. > Side Benefits: > Audit current build mechanics. > - Presently the language specific binding are not installed. (.py & .jar) > - make -jX currently fails > - optionally look in arm support. > Costs: > 1. Time > 2. Potential temporary destabilization > 3. Infrastructure around build+test may need to change. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-898) Introduce CMake as an alternative build system.
[ https://issues.apache.org/jira/browse/MESOS-898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14600279#comment-14600279 ] Vinod Kone commented on MESOS-898: -- You know that this JIRA is public right :) Also, since this support is being added to Apache Mesos, we expect the community to be in the know during development of changes like these. Regarding cross platform support off the bat, I'll trust your instincts on this. From what I saw in your commits, windows support didn't seem to complicate things. So, it's ok with me. Regarding CMake idioms, [~tstclair] (Mesos committer), might be able to help you out here. > Introduce CMake as an alternative build system. > --- > > Key: MESOS-898 > URL: https://issues.apache.org/jira/browse/MESOS-898 > Project: Mesos > Issue Type: Epic > Components: build >Reporter: Timothy St. Clair >Assignee: Alex Clemmer > Labels: build > > This is a rather substantial undertaking, so I would want upstream > debate+buy-in prior to full commitment. The basic premise is: upstream > rebundles several of its dependencies in part to tightly control its stack. > This is not out of the norm, but in order to be picked up by distribution > channels it needs to built against system dependencies, and rebundling is > strictly forbidden. Given that the mesos primary target platform are > data-center distributions such as RHEL/CENTOS/SL it makes sense to still have > bundling support for those who do not have dependencies in their channels > "yet". This is where cmake can be win with it's uber macros > (http://www.cmake.org/cmake/help/v2.8.8/cmake.html#module:ExternalProject). > I do not know of any equivalent in the autotools world, other then to brew > your own solution. I've done this type of work in the past, and completely > transformed condor and would leverage a lot of the work that was done there. > I currently have a tracking branch where I've started this work, but before I > go off into the woods, it makes sense to have a debate in public. > The primary benefits are: > 1. Enable downstream channels to easily distro without carrying a large patch > sets. > 2. Still support existing "non-proper" distribution methods. > 3. Harden / future proof dependent interfaces. > Side Benefits: > Audit current build mechanics. > - Presently the language specific binding are not installed. (.py & .jar) > - make -jX currently fails > - optionally look in arm support. > Costs: > 1. Time > 2. Potential temporary destabilization > 3. Infrastructure around build+test may need to change. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-898) Introduce CMake as an alternative build system.
[ https://issues.apache.org/jira/browse/MESOS-898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14600263#comment-14600263 ] Cody Maloney commented on MESOS-898: I would suggest that with the move to CMake we switch to using a raw upstream packaged version of boost. There isn't a lot we gain by stripping out some of the headers, and it adds a lot more complexity. CMake has a lot of stuff ready-made for finding, downloading boost if and only if it isn't present on the host machine, isn't of the right version, etc. Forcing rebuilding all of that logic/code so that we can remove some files in a tarball which shouldn't be embedded inside the repository anyways seems like not the best idea. > Introduce CMake as an alternative build system. > --- > > Key: MESOS-898 > URL: https://issues.apache.org/jira/browse/MESOS-898 > Project: Mesos > Issue Type: Epic > Components: build >Reporter: Timothy St. Clair >Assignee: Alex Clemmer > Labels: build > > This is a rather substantial undertaking, so I would want upstream > debate+buy-in prior to full commitment. The basic premise is: upstream > rebundles several of its dependencies in part to tightly control its stack. > This is not out of the norm, but in order to be picked up by distribution > channels it needs to built against system dependencies, and rebundling is > strictly forbidden. Given that the mesos primary target platform are > data-center distributions such as RHEL/CENTOS/SL it makes sense to still have > bundling support for those who do not have dependencies in their channels > "yet". This is where cmake can be win with it's uber macros > (http://www.cmake.org/cmake/help/v2.8.8/cmake.html#module:ExternalProject). > I do not know of any equivalent in the autotools world, other then to brew > your own solution. I've done this type of work in the past, and completely > transformed condor and would leverage a lot of the work that was done there. > I currently have a tracking branch where I've started this work, but before I > go off into the woods, it makes sense to have a debate in public. > The primary benefits are: > 1. Enable downstream channels to easily distro without carrying a large patch > sets. > 2. Still support existing "non-proper" distribution methods. > 3. Harden / future proof dependent interfaces. > Side Benefits: > Audit current build mechanics. > - Presently the language specific binding are not installed. (.py & .jar) > - make -jX currently fails > - optionally look in arm support. > Costs: > 1. Time > 2. Potential temporary destabilization > 3. Infrastructure around build+test may need to change. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-1187) precision errors with allocation calculations
[ https://issues.apache.org/jira/browse/MESOS-1187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14600257#comment-14600257 ] Jie Yu commented on MESOS-1187: --- This is a simple test to reproduce: https://reviews.apache.org/r/35849/ > precision errors with allocation calculations > - > > Key: MESOS-1187 > URL: https://issues.apache.org/jira/browse/MESOS-1187 > Project: Mesos > Issue Type: Bug > Components: allocation, master >Reporter: aniruddha sathaye > > As allocations are stored/transmitted as doubles many a times precision > errors creep in. > we have seen erroneous share calculations happen only because of floating > point arithmetic. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-898) Introduce CMake as an alternative build system.
[
https://issues.apache.org/jira/browse/MESOS-898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14600254#comment-14600254
]
Alex Clemmer commented on MESOS-898:
[~vinodkone]:
{quote}
--> Looks like you are adding windows support too (I'm guessing that's your
main motivation to work on this? Stoked to see this btw!). It's probably better
to work on WIN support in a separate patch.
{quote}
First, yes, I'm attempting to contribute Windows (and Windows container)
support to the Mesos project, and yes, this is the first big step. But, please
don't spread it around to too many people just yet -- to the extent possible, I
am hoping to avoid impacting Mesosphere's ability to effectively "launch" the
feature. :)
To your suggestion that we split Windows support into a different patch: it
seems like you're saying it might be better to first contribute a baseline
CMake-based build system, and then extend that to support Windows later. (That
is, I assume you're not talking about how to package up the changes to the C++
code that we will need in order to support Windows, which I assume we all agree
belong in a different patch.) Is that all right?
I'm actually hoping to make x-plat support out of the box a core goal of this
new build system. My rationale is basically twofold: (1) I am basically certain
that I will end up developing a very different build system if we don't make
x-plat support a priority right out of the gate (right now, I'm testing that
each iteration works on both Linux and Windows, which definitely guides the
design a lot), and (2) I've taken care to clearly mark off the Windows-specific
stuff so that we can pull it out easily later if this Windows stuff doesn't
work out. More particularly, I am suggesting this course of action because I
think it will result in the least aggregate work in expectation.
That all said, I understand the hesitation to put stuff into the codebase if it
isn't used, and I'm totally willing to be convinced that I'm wrong. Let me know
if you see something I don't.
To your other issues, that boost issue that you and [~xujyan] is good to know,
so thanks for pointing it out. And, I was actually hoping that one of you had
good knowledge of CMake idioms. :) If we don't have someone with that expertise
on deck, I agree we need to go find it before merge.
> Introduce CMake as an alternative build system.
> ---
>
> Key: MESOS-898
> URL: https://issues.apache.org/jira/browse/MESOS-898
> Project: Mesos
> Issue Type: Epic
> Components: build
>Reporter: Timothy St. Clair
>Assignee: Alex Clemmer
> Labels: build
>
> This is a rather substantial undertaking, so I would want upstream
> debate+buy-in prior to full commitment. The basic premise is: upstream
> rebundles several of its dependencies in part to tightly control its stack.
> This is not out of the norm, but in order to be picked up by distribution
> channels it needs to built against system dependencies, and rebundling is
> strictly forbidden. Given that the mesos primary target platform are
> data-center distributions such as RHEL/CENTOS/SL it makes sense to still have
> bundling support for those who do not have dependencies in their channels
> "yet". This is where cmake can be win with it's uber macros
> (http://www.cmake.org/cmake/help/v2.8.8/cmake.html#module:ExternalProject).
> I do not know of any equivalent in the autotools world, other then to brew
> your own solution. I've done this type of work in the past, and completely
> transformed condor and would leverage a lot of the work that was done there.
> I currently have a tracking branch where I've started this work, but before I
> go off into the woods, it makes sense to have a debate in public.
> The primary benefits are:
> 1. Enable downstream channels to easily distro without carrying a large patch
> sets.
> 2. Still support existing "non-proper" distribution methods.
> 3. Harden / future proof dependent interfaces.
> Side Benefits:
> Audit current build mechanics.
> - Presently the language specific binding are not installed. (.py & .jar)
> - make -jX currently fails
> - optionally look in arm support.
> Costs:
> 1. Time
> 2. Potential temporary destabilization
> 3. Infrastructure around build+test may need to change.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (MESOS-2889) Add SSL switch to python configuration
[ https://issues.apache.org/jira/browse/MESOS-2889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Artem Harutyunyan reassigned MESOS-2889: Assignee: Artem Harutyunyan (was: Joris Van Remoortere) > Add SSL switch to python configuration > -- > > Key: MESOS-2889 > URL: https://issues.apache.org/jira/browse/MESOS-2889 > Project: Mesos > Issue Type: Bug >Reporter: Joris Van Remoortere >Assignee: Artem Harutyunyan > Labels: mesosphere > > The python egg requires explicit dependencies for SSL. Add these to the > python configuration if ssl is enabled. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Reopened] (MESOS-1187) precision errors with allocation calculations
[ https://issues.apache.org/jira/browse/MESOS-1187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jie Yu reopened MESOS-1187: --- I was able to reproduce. 0.1 + 0.1 + 0.1 - 0.1 - 0.1 = 0.09892 < 0.1 That would cause issues if the code relies on the fact that 0.1 + 0.1 + 0.1 - 0.1 - 0.1 == 0.1 > precision errors with allocation calculations > - > > Key: MESOS-1187 > URL: https://issues.apache.org/jira/browse/MESOS-1187 > Project: Mesos > Issue Type: Bug > Components: allocation, master >Reporter: aniruddha sathaye >Priority: Minor > > As allocations are stored/transmitted as doubles many a times precision > errors creep in. > we have seen erroneous share calculations happen only because of floating > point arithmetic. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-1187) precision errors with allocation calculations
[ https://issues.apache.org/jira/browse/MESOS-1187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jie Yu updated MESOS-1187: -- Priority: Major (was: Minor) > precision errors with allocation calculations > - > > Key: MESOS-1187 > URL: https://issues.apache.org/jira/browse/MESOS-1187 > Project: Mesos > Issue Type: Bug > Components: allocation, master >Reporter: aniruddha sathaye > > As allocations are stored/transmitted as doubles many a times precision > errors creep in. > we have seen erroneous share calculations happen only because of floating > point arithmetic. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-999) Slave should wait() and start executor registration timeout after launch
[ https://issues.apache.org/jira/browse/MESOS-999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Xu updated MESOS-999: - Assignee: (was: Yan Xu) > Slave should wait() and start executor registration timeout after launch > - > > Key: MESOS-999 > URL: https://issues.apache.org/jira/browse/MESOS-999 > Project: Mesos > Issue Type: Bug > Components: isolation >Affects Versions: 0.18.0 >Reporter: Ian Downes >Priority: Minor > > The current code will start launch a container and wait on it before the > launch is complete. We should do this only after the container has > successfully launched. Likewise for the executor registration timeout. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-999) Slave should wait() and start executor registration timeout after launch
[
https://issues.apache.org/jira/browse/MESOS-999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14600134#comment-14600134
]
Yan Xu commented on MESOS-999:
--
So we ended up not taking on this.
Our motivation for addressing this was because with docker and other new
containerization efforts such as MESOS-2386, some considerable amount of time
can be spent on pulling and preparing the container images before the executor
is launched so we don't want the slave to kill it due to a small
{{--executor_registration_timeout}}.
While implementing this we realized that adding a slave level launch timeout
flag doesn't solve the fundamental problem of the long image preparation having
a noticeable impact on the slave and its tasks. We should instead working
towards a solution that minimizes such impact.
The {{--executor_registration_timeout}} flag was originally introduced to
account for the time required to fetch the executor so for now giving it a
large value is the reasonable way for tasks that use container images.
Ultimately only the task knows what the expected preparation time is so such
timeouts should probably go to ExecutorInfo or TaskInfo.
I feel like this ticket as it is currently phrased could be closed as {{Won't
Fix}}. Do you agree [~idownes]?
> Slave should wait() and start executor registration timeout after launch
> -
>
> Key: MESOS-999
> URL: https://issues.apache.org/jira/browse/MESOS-999
> Project: Mesos
> Issue Type: Bug
> Components: isolation
>Affects Versions: 0.18.0
>Reporter: Ian Downes
>Assignee: Yan Xu
>Priority: Minor
>
> The current code will start launch a container and wait on it before the
> launch is complete. We should do this only after the container has
> successfully launched. Likewise for the executor registration timeout.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MESOS-999) Slave should wait() and start executor registration timeout after launch
[ https://issues.apache.org/jira/browse/MESOS-999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yan Xu updated MESOS-999: - Labels: (was: twitter) > Slave should wait() and start executor registration timeout after launch > - > > Key: MESOS-999 > URL: https://issues.apache.org/jira/browse/MESOS-999 > Project: Mesos > Issue Type: Bug > Components: isolation >Affects Versions: 0.18.0 >Reporter: Ian Downes >Priority: Minor > > The current code will start launch a container and wait on it before the > launch is complete. We should do this only after the container has > successfully launched. Likewise for the executor registration timeout. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-2925) Invalid usage of ATOMIC_FLAG_INIT in member initialization
[ https://issues.apache.org/jira/browse/MESOS-2925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14600121#comment-14600121 ] Paul Brett commented on MESOS-2925: --- [~jvanremoortere] - I think the init macro in the initializer list use looks much better but the compiler warns against it because the behavior is undefined and therefore unsafe. BTW, I'm using clang on Linux, so I don't know if the proposed Apple tweak would help me. > Invalid usage of ATOMIC_FLAG_INIT in member initialization > -- > > Key: MESOS-2925 > URL: https://issues.apache.org/jira/browse/MESOS-2925 > Project: Mesos > Issue Type: Bug > Components: libprocess >Affects Versions: 0.23.0 >Reporter: Paul Brett > > The C++ specification states: > The macro ATOMIC_FLAG_INIT shall be defined in such a way that it can be used > to initialize an object of type atomic_flag to the clear state. The macro can > be used in the form: "atomic_flag guard = ATOMIC_FLAG_INIT; "It is > unspecified whether the macro can be used in other initialization contexts." > Clang catches this (although reports it erroneously as a braced scaled init > issue) and refuses to compile libprocess. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-2925) Invalid usage of ATOMIC_FLAG_INIT in member initialization
[ https://issues.apache.org/jira/browse/MESOS-2925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14600104#comment-14600104 ] Paul Brett commented on MESOS-2925: --- Up for review at https://reviews.apache.org/r/35841/ > Invalid usage of ATOMIC_FLAG_INIT in member initialization > -- > > Key: MESOS-2925 > URL: https://issues.apache.org/jira/browse/MESOS-2925 > Project: Mesos > Issue Type: Bug > Components: libprocess >Affects Versions: 0.23.0 >Reporter: Paul Brett > > The C++ specification states: > The macro ATOMIC_FLAG_INIT shall be defined in such a way that it can be used > to initialize an object of type atomic_flag to the clear state. The macro can > be used in the form: "atomic_flag guard = ATOMIC_FLAG_INIT; "It is > unspecified whether the macro can be used in other initialization contexts." > Clang catches this (although reports it erroneously as a braced scaled init > issue) and refuses to compile libprocess. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-2637) Consolidate 'foo', 'bar', ... string constants in test and example code
[ https://issues.apache.org/jira/browse/MESOS-2637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Niklas Quarfot Nielsen updated MESOS-2637: -- Shepherd: (was: Niklas Quarfot Nielsen) > Consolidate 'foo', 'bar', ... string constants in test and example code > --- > > Key: MESOS-2637 > URL: https://issues.apache.org/jira/browse/MESOS-2637 > Project: Mesos > Issue Type: Bug > Components: technical debt >Reporter: Niklas Quarfot Nielsen >Assignee: Colin Williams > > We are using 'foo', 'bar', ... string constants and pairs in > src/tests/master_tests.cpp, src/tests/slave_tests.cpp, > src/tests/hook_tests.cpp and src/examples/test_hook_module.cpp for label and > hooks tests. These values should be stored in local variables to avoid the > possibility of assignment getting out of sync with checking for that same > value. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-2637) Consolidate 'foo', 'bar', ... string constants in test and example code
[ https://issues.apache.org/jira/browse/MESOS-2637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14600087#comment-14600087 ] Colin Williams commented on MESOS-2637: --- I'll do that, thanks for your help so far. On Jun 24, 2015 2:19 PM, "Niklas Quarfot Nielsen (JIRA)" > Consolidate 'foo', 'bar', ... string constants in test and example code > --- > > Key: MESOS-2637 > URL: https://issues.apache.org/jira/browse/MESOS-2637 > Project: Mesos > Issue Type: Bug > Components: technical debt >Reporter: Niklas Quarfot Nielsen >Assignee: Colin Williams > > We are using 'foo', 'bar', ... string constants and pairs in > src/tests/master_tests.cpp, src/tests/slave_tests.cpp, > src/tests/hook_tests.cpp and src/examples/test_hook_module.cpp for label and > hooks tests. These values should be stored in local variables to avoid the > possibility of assignment getting out of sync with checking for that same > value. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (MESOS-2925) Invalid usage of ATOMIC_FLAG_INIT in member initialization
[
https://issues.apache.org/jira/browse/MESOS-2925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14600084#comment-14600084
]
Joris Van Remoortere edited comment on MESOS-2925 at 6/24/15 8:26 PM:
--
Hi [~pbrett]. Clang 3.7 does error out on OSX due to the braced scaled
initializer warning. According to [~tillt] this is something that Apple tends
to tweak before they provide the stable releases of clang for OSX?
AFAIK the macro expands to either \{false\} or \{0\}. I don't see why it is
harmful to keep using this to initialize the locks. I find the code in the
review request harder to read due to the noise. What do you think?
was (Author: jvanremoortere):
Hi [~pbrett]. Clang 3.7 does error out on OSX due to the braced scaled
initializer warning. According to [~tillt] this is something that Apple tends
to tweak before they provide the stable releases of clang for OSX?
AFAIK the macro expands to either `{false}` or {0}. I don't see why it is
harmful to keep using this to initialize the locks. I find the code in the
review request harder to read due to the noise. What do you think?
> Invalid usage of ATOMIC_FLAG_INIT in member initialization
> --
>
> Key: MESOS-2925
> URL: https://issues.apache.org/jira/browse/MESOS-2925
> Project: Mesos
> Issue Type: Bug
> Components: libprocess
>Affects Versions: 0.23.0
>Reporter: Paul Brett
>
> The C++ specification states:
> The macro ATOMIC_FLAG_INIT shall be defined in such a way that it can be used
> to initialize an object of type atomic_flag to the clear state. The macro can
> be used in the form: "atomic_flag guard = ATOMIC_FLAG_INIT; "It is
> unspecified whether the macro can be used in other initialization contexts."
> Clang catches this (although reports it erroneously as a braced scaled init
> issue) and refuses to compile libprocess.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-2925) Invalid usage of ATOMIC_FLAG_INIT in member initialization
[
https://issues.apache.org/jira/browse/MESOS-2925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14600084#comment-14600084
]
Joris Van Remoortere commented on MESOS-2925:
-
Hi [~pbrett]. Clang 3.7 does error out on OSX due to the braced scaled
initializer warning. According to [~tillt] this is something that Apple tends
to tweak before they provide the stable releases of clang for OSX?
AFAIK the macro expands to either `{false}` or {0}. I don't see why it is
harmful to keep using this to initialize the locks. I find the code in the
review request harder to read due to the noise. What do you think?
> Invalid usage of ATOMIC_FLAG_INIT in member initialization
> --
>
> Key: MESOS-2925
> URL: https://issues.apache.org/jira/browse/MESOS-2925
> Project: Mesos
> Issue Type: Bug
> Components: libprocess
>Affects Versions: 0.23.0
>Reporter: Paul Brett
>
> The C++ specification states:
> The macro ATOMIC_FLAG_INIT shall be defined in such a way that it can be used
> to initialize an object of type atomic_flag to the clear state. The macro can
> be used in the form: "atomic_flag guard = ATOMIC_FLAG_INIT; "It is
> unspecified whether the macro can be used in other initialization contexts."
> Clang catches this (although reports it erroneously as a braced scaled init
> issue) and refuses to compile libprocess.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (MESOS-2925) Invalid usage of ATOMIC_FLAG_INIT in member initialization
Paul Brett created MESOS-2925: - Summary: Invalid usage of ATOMIC_FLAG_INIT in member initialization Key: MESOS-2925 URL: https://issues.apache.org/jira/browse/MESOS-2925 Project: Mesos Issue Type: Bug Components: libprocess Affects Versions: 0.23.0 Reporter: Paul Brett The C++ specification states: The macro ATOMIC_FLAG_INIT shall be defined in such a way that it can be used to initialize an object of type atomic_flag to the clear state. The macro can be used in the form: "atomic_flag guard = ATOMIC_FLAG_INIT; "It is unspecified whether the macro can be used in other initialization contexts." Clang catches this (although reports it erroneously as a braced scaled init issue) and refuses to compile libprocess. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-2551) C++ Scheduler library should send Call messages to Master
[ https://issues.apache.org/jira/browse/MESOS-2551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marco Massenzio updated MESOS-2551: --- Labels: tech-debt twitter (was: tech-debt) > C++ Scheduler library should send Call messages to Master > - > > Key: MESOS-2551 > URL: https://issues.apache.org/jira/browse/MESOS-2551 > Project: Mesos > Issue Type: Story >Reporter: Vinod Kone >Assignee: Vinod Kone > Labels: tech-debt, twitter > > Currently, the C++ library sends different messages to Master instead of a > single Call message. To vet the new Call API it should send Call messages. > Master should be updated to handle all types of Calls. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (MESOS-2551) C++ Scheduler library should send Call messages to Master
[ https://issues.apache.org/jira/browse/MESOS-2551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14599969#comment-14599969 ] Marco Massenzio edited comment on MESOS-2551 at 6/24/15 7:08 PM: - Thanks, [~vi...@twitter.com] for taking this up. I've marked it in progress, as I'm assuming you're working on it? [~vi...@twitter.com] - can you please keep [~adam-mesos] and me posted on progress: 0.23RC is coming up fast and we'd like to have this in too, if we possibly can. (Just for historical records purposes, [this review](https://reviews.apache.org/r/35806/) was originally posted by [~ijimenez]) was (Author: marco-mesos): Thanks, [~vi...@twitter.com] for taking this up. Just for historical records purposes, [this review](https://reviews.apache.org/r/35806/) was originally posted. [~vi...@twitter.com] - can you please keep [~adam-mesos] and me posted on progress: 0.23RC is coming up fast and we'd like to have this in too, if we possibly can. > C++ Scheduler library should send Call messages to Master > - > > Key: MESOS-2551 > URL: https://issues.apache.org/jira/browse/MESOS-2551 > Project: Mesos > Issue Type: Story >Reporter: Vinod Kone >Assignee: Vinod Kone > Labels: tech-debt, twitter > > Currently, the C++ library sends different messages to Master instead of a > single Call message. To vet the new Call API it should send Call messages. > Master should be updated to handle all types of Calls. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-2551) C++ Scheduler library should send Call messages to Master
[ https://issues.apache.org/jira/browse/MESOS-2551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14599969#comment-14599969 ] Marco Massenzio commented on MESOS-2551: Thanks, [~vi...@twitter.com] for taking this up. Just for historical records purposes, [this review](https://reviews.apache.org/r/35806/) was originally posted. [~vi...@twitter.com] - can you please keep [~adam-mesos] and me posted on progress: 0.23RC is coming up fast and we'd like to have this in too, if we possibly can. > C++ Scheduler library should send Call messages to Master > - > > Key: MESOS-2551 > URL: https://issues.apache.org/jira/browse/MESOS-2551 > Project: Mesos > Issue Type: Story >Reporter: Vinod Kone >Assignee: Vinod Kone > Labels: tech-debt > > Currently, the C++ library sends different messages to Master instead of a > single Call message. To vet the new Call API it should send Call messages. > Master should be updated to handle all types of Calls. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (MESOS-2621) Create documentation for observability metrics
[ https://issues.apache.org/jira/browse/MESOS-2621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod Kone resolved MESOS-2621. --- Resolution: Fixed Assignee: Ricardo Cervera-Navarro Fix Version/s: 0.23.0 commit f16d73852623ee05cc13d2757115f7815e608964 Author: Ricardo Cervera-Navarro Date: Wed Jun 24 12:01:39 2015 -0700 Added documentation on monitoring metrics and alerts. Review: https://reviews.apache.org/r/33241 > Create documentation for observability metrics > -- > > Key: MESOS-2621 > URL: https://issues.apache.org/jira/browse/MESOS-2621 > Project: Mesos > Issue Type: Documentation >Reporter: Ricardo Cervera-Navarro >Assignee: Ricardo Cervera-Navarro >Priority: Minor > Fix For: 0.23.0 > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-898) Introduce CMake as an alternative build system.
[
https://issues.apache.org/jira/browse/MESOS-898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14599955#comment-14599955
]
Yan Xu commented on MESOS-898:
--
{quote}
--> Looks like you are pulling in the whole of boost? While it's not clear, we
bundle the boost ourselves. It basically includes only the boost headers that
we need. There should be a README somewhere that explains the process.
{quote}
Yeah it's super unclear. The README is hidden within the tarball.
https://github.com/apache/mesos/blob/master/3rdparty/libprocess/3rdparty/boost-1.53.0.tar.gz
We should move it out.
> Introduce CMake as an alternative build system.
> ---
>
> Key: MESOS-898
> URL: https://issues.apache.org/jira/browse/MESOS-898
> Project: Mesos
> Issue Type: Epic
> Components: build
>Reporter: Timothy St. Clair
>Assignee: Alex Clemmer
> Labels: build
>
> This is a rather substantial undertaking, so I would want upstream
> debate+buy-in prior to full commitment. The basic premise is: upstream
> rebundles several of its dependencies in part to tightly control its stack.
> This is not out of the norm, but in order to be picked up by distribution
> channels it needs to built against system dependencies, and rebundling is
> strictly forbidden. Given that the mesos primary target platform are
> data-center distributions such as RHEL/CENTOS/SL it makes sense to still have
> bundling support for those who do not have dependencies in their channels
> "yet". This is where cmake can be win with it's uber macros
> (http://www.cmake.org/cmake/help/v2.8.8/cmake.html#module:ExternalProject).
> I do not know of any equivalent in the autotools world, other then to brew
> your own solution. I've done this type of work in the past, and completely
> transformed condor and would leverage a lot of the work that was done there.
> I currently have a tracking branch where I've started this work, but before I
> go off into the woods, it makes sense to have a debate in public.
> The primary benefits are:
> 1. Enable downstream channels to easily distro without carrying a large patch
> sets.
> 2. Still support existing "non-proper" distribution methods.
> 3. Harden / future proof dependent interfaces.
> Side Benefits:
> Audit current build mechanics.
> - Presently the language specific binding are not installed. (.py & .jar)
> - make -jX currently fails
> - optionally look in arm support.
> Costs:
> 1. Time
> 2. Potential temporary destabilization
> 3. Infrastructure around build+test may need to change.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-898) Introduce CMake as an alternative build system.
[ https://issues.apache.org/jira/browse/MESOS-898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14599952#comment-14599952 ] Vinod Kone commented on MESOS-898: -- [~hausdorff] Took a look at your code. Looks really good and well documented! Couple of things: --> Looks like you are adding windows support too (I'm guessing that's your main motivation to work on this? Stoked to see this btw!). It's probably better to work on WIN support in a separate patch. --> Looks like you are pulling in the whole of boost? While it's not clear, we bundle the boost ourselves. It basically includes only the boost headers that we need. There should be a README somewhere that explains the process. --> It's worthwhile to find someone who understands CMake idioms to take a look to address some of the workarounds you had to do (e.g., global patch command etc). All in all, great job! Looking forward to use CMake. > Introduce CMake as an alternative build system. > --- > > Key: MESOS-898 > URL: https://issues.apache.org/jira/browse/MESOS-898 > Project: Mesos > Issue Type: Epic > Components: build >Reporter: Timothy St. Clair >Assignee: Alex Clemmer > Labels: build > > This is a rather substantial undertaking, so I would want upstream > debate+buy-in prior to full commitment. The basic premise is: upstream > rebundles several of its dependencies in part to tightly control its stack. > This is not out of the norm, but in order to be picked up by distribution > channels it needs to built against system dependencies, and rebundling is > strictly forbidden. Given that the mesos primary target platform are > data-center distributions such as RHEL/CENTOS/SL it makes sense to still have > bundling support for those who do not have dependencies in their channels > "yet". This is where cmake can be win with it's uber macros > (http://www.cmake.org/cmake/help/v2.8.8/cmake.html#module:ExternalProject). > I do not know of any equivalent in the autotools world, other then to brew > your own solution. I've done this type of work in the past, and completely > transformed condor and would leverage a lot of the work that was done there. > I currently have a tracking branch where I've started this work, but before I > go off into the woods, it makes sense to have a debate in public. > The primary benefits are: > 1. Enable downstream channels to easily distro without carrying a large patch > sets. > 2. Still support existing "non-proper" distribution methods. > 3. Harden / future proof dependent interfaces. > Side Benefits: > Audit current build mechanics. > - Presently the language specific binding are not installed. (.py & .jar) > - make -jX currently fails > - optionally look in arm support. > Costs: > 1. Time > 2. Potential temporary destabilization > 3. Infrastructure around build+test may need to change. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-2919) Framework can overcommit oversubscribable resources during master failover.
[
https://issues.apache.org/jira/browse/MESOS-2919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jie Yu updated MESOS-2919:
--
Story Points: 3
> Framework can overcommit oversubscribable resources during master failover.
> ---
>
> Key: MESOS-2919
> URL: https://issues.apache.org/jira/browse/MESOS-2919
> Project: Mesos
> Issue Type: Bug
>Reporter: Jie Yu
>Assignee: Jie Yu
>Priority: Critical
> Labels: twitter
>
> This is due to a bug in the hierarchical allocator. Here is the sequence of
> events:
> 1) slave uses a fixed resource estimator which advertise 4 revocable cpus
> 2) a framework A launches a task that uses all the 4 revocable cpus
> 3) master fails over
> 4) slave re-registers with the new master, and sends UpdateSlaveMessage with
> 4 revocable cpus as oversubscribed resources
> 5) framework A hasn't registered yet, therefore, the slave's available
> resources will be 4 revocable cpus
> 6) framework A registered and will receive an additional 4 revocable cpus. So
> it can launch another task with 4 revocable cpus (that means 8 total!)
> The problem is due to the way we calculate 'allocated' resource in allocator
> when 'updateSlave'. If the framework is not registered, the 'allocation'
> below is not accurate (check that if block in 'addSlave').
> {code}
> template
> void
> HierarchicalAllocatorProcess::updateSlave(
> const SlaveID& slaveId,
> const Resources& oversubscribed)
> {
> CHECK(initialized);
> CHECK(slaves.contains(slaveId));
> // Check that all the oversubscribed resources are revocable.
> CHECK_EQ(oversubscribed, oversubscribed.revocable());
> // Update the total resources.
> // First remove the old oversubscribed resources from the total.
> slaves[slaveId].total -= slaves[slaveId].total.revocable();
> // Now add the new estimate of oversubscribed resources.
> slaves[slaveId].total += oversubscribed;
> // Now, update the total resources in the role sorter.
> roleSorter->update(
> slaveId,
> slaves[slaveId].total.unreserved());
> // Calculate the current allocation of oversubscribed resources.
> Resources allocation;
> foreachkey (const std::string& role, roles) {
> allocation += roleSorter->allocation(role, slaveId).revocable();
> }
> // Update the available resources.
> // First remove the old oversubscribed resources from available.
> slaves[slaveId].available -= slaves[slaveId].available.revocable();
> // Now add the new estimate of available oversubscribed resources.
> slaves[slaveId].available += oversubscribed - allocation;
> LOG(INFO) << "Slave " << slaveId << " (" << slaves[slaveId].hostname
> << ") updated with oversubscribed resources " << oversubscribed
> << " (total: " << slaves[slaveId].total
> << ", available: " << slaves[slaveId].available << ")";
> allocate(slaveId);
> }
> template
> void
> HierarchicalAllocatorProcess::addSlave(
> const SlaveID& slaveId,
> const SlaveInfo& slaveInfo,
> const Resources& total,
> const hashmap& used)
> {
> CHECK(initialized);
> CHECK(!slaves.contains(slaveId));
> roleSorter->add(slaveId, total.unreserved());
> foreachpair (const FrameworkID& frameworkId,
>const Resources& allocated,
>used) {
> if (frameworks.contains(frameworkId)) {
> const std::string& role = frameworks[frameworkId].role;
> // TODO(bmahler): Validate that the reserved resources have the
> // framework's role.
> roleSorter->allocated(role, slaveId, allocated.unreserved());
> frameworkSorters[role]->add(slaveId, allocated);
> frameworkSorters[role]->allocated(
> frameworkId.value(), slaveId, allocated);
> }
> }
> ...
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-2919) Framework can overcommit oversubscribable resources during master failover.
[
https://issues.apache.org/jira/browse/MESOS-2919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14599902#comment-14599902
]
Jie Yu commented on MESOS-2919:
---
https://reviews.apache.org/r/35836/
> Framework can overcommit oversubscribable resources during master failover.
> ---
>
> Key: MESOS-2919
> URL: https://issues.apache.org/jira/browse/MESOS-2919
> Project: Mesos
> Issue Type: Bug
>Reporter: Jie Yu
>Assignee: Jie Yu
>Priority: Critical
> Labels: twitter
>
> This is due to a bug in the hierarchical allocator. Here is the sequence of
> events:
> 1) slave uses a fixed resource estimator which advertise 4 revocable cpus
> 2) a framework A launches a task that uses all the 4 revocable cpus
> 3) master fails over
> 4) slave re-registers with the new master, and sends UpdateSlaveMessage with
> 4 revocable cpus as oversubscribed resources
> 5) framework A hasn't registered yet, therefore, the slave's available
> resources will be 4 revocable cpus
> 6) framework A registered and will receive an additional 4 revocable cpus. So
> it can launch another task with 4 revocable cpus (that means 8 total!)
> The problem is due to the way we calculate 'allocated' resource in allocator
> when 'updateSlave'. If the framework is not registered, the 'allocation'
> below is not accurate (check that if block in 'addSlave').
> {code}
> template
> void
> HierarchicalAllocatorProcess::updateSlave(
> const SlaveID& slaveId,
> const Resources& oversubscribed)
> {
> CHECK(initialized);
> CHECK(slaves.contains(slaveId));
> // Check that all the oversubscribed resources are revocable.
> CHECK_EQ(oversubscribed, oversubscribed.revocable());
> // Update the total resources.
> // First remove the old oversubscribed resources from the total.
> slaves[slaveId].total -= slaves[slaveId].total.revocable();
> // Now add the new estimate of oversubscribed resources.
> slaves[slaveId].total += oversubscribed;
> // Now, update the total resources in the role sorter.
> roleSorter->update(
> slaveId,
> slaves[slaveId].total.unreserved());
> // Calculate the current allocation of oversubscribed resources.
> Resources allocation;
> foreachkey (const std::string& role, roles) {
> allocation += roleSorter->allocation(role, slaveId).revocable();
> }
> // Update the available resources.
> // First remove the old oversubscribed resources from available.
> slaves[slaveId].available -= slaves[slaveId].available.revocable();
> // Now add the new estimate of available oversubscribed resources.
> slaves[slaveId].available += oversubscribed - allocation;
> LOG(INFO) << "Slave " << slaveId << " (" << slaves[slaveId].hostname
> << ") updated with oversubscribed resources " << oversubscribed
> << " (total: " << slaves[slaveId].total
> << ", available: " << slaves[slaveId].available << ")";
> allocate(slaveId);
> }
> template
> void
> HierarchicalAllocatorProcess::addSlave(
> const SlaveID& slaveId,
> const SlaveInfo& slaveInfo,
> const Resources& total,
> const hashmap& used)
> {
> CHECK(initialized);
> CHECK(!slaves.contains(slaveId));
> roleSorter->add(slaveId, total.unreserved());
> foreachpair (const FrameworkID& frameworkId,
>const Resources& allocated,
>used) {
> if (frameworks.contains(frameworkId)) {
> const std::string& role = frameworks[frameworkId].role;
> // TODO(bmahler): Validate that the reserved resources have the
> // framework's role.
> roleSorter->allocated(role, slaveId, allocated.unreserved());
> frameworkSorters[role]->add(slaveId, allocated);
> frameworkSorters[role]->allocated(
> frameworkId.value(), slaveId, allocated);
> }
> }
> ...
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-2637) Consolidate 'foo', 'bar', ... string constants in test and example code
[ https://issues.apache.org/jira/browse/MESOS-2637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14599886#comment-14599886 ] Niklas Quarfot Nielsen commented on MESOS-2637: --- Hi [~lackita] - Did you get to look at this? I will be out for the next two weeks from tomorrow. If you are planning on getting reviews/feedback in the mean time - can you ping one of the other committers? Thanks! Niklas > Consolidate 'foo', 'bar', ... string constants in test and example code > --- > > Key: MESOS-2637 > URL: https://issues.apache.org/jira/browse/MESOS-2637 > Project: Mesos > Issue Type: Bug > Components: technical debt >Reporter: Niklas Quarfot Nielsen >Assignee: Colin Williams > > We are using 'foo', 'bar', ... string constants and pairs in > src/tests/master_tests.cpp, src/tests/slave_tests.cpp, > src/tests/hook_tests.cpp and src/examples/test_hook_module.cpp for label and > hooks tests. These values should be stored in local variables to avoid the > possibility of assignment getting out of sync with checking for that same > value. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (MESOS-898) Introduce CMake as an alternative build system.
[ https://issues.apache.org/jira/browse/MESOS-898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14599866#comment-14599866 ] Alex Clemmer edited comment on MESOS-898 at 6/24/15 6:17 PM: - Only a few people have given a quick once-over look to the [prototype CMake-based build system for libprocess|https://github.com/hausdorff/mesos/commit/073d42cea1bed8e7fb801f59685f52d3be06e510?diff=unified], but they have been pretty positive, so I would like to target EOD tomorrow (i.e., Th, June 24th) for posting something to ReviewBoard -- I don't want to let the commit hang too long, but I am also new to the community, and I want to give people a chance to build consensus that this commit is at least directionally correct. :) was (Author: hausdorff): Only a few people have given a quick once-over look to the prototype [CMake-based build system for libprocess|https://github.com/hausdorff/mesos/commit/073d42cea1bed8e7fb801f59685f52d3be06e510?diff=unified], but they have been pretty positive, so I would like to target EOD tomorrow (i.e., Th, June 24th) for posting something to ReviewBoard -- I don't want to let the commit hang too long, but I am also new to the community, and I want to give people a chance to build consensus that this commit is at least directionally correct. :) > Introduce CMake as an alternative build system. > --- > > Key: MESOS-898 > URL: https://issues.apache.org/jira/browse/MESOS-898 > Project: Mesos > Issue Type: Epic > Components: build >Reporter: Timothy St. Clair >Assignee: Alex Clemmer > Labels: build > > This is a rather substantial undertaking, so I would want upstream > debate+buy-in prior to full commitment. The basic premise is: upstream > rebundles several of its dependencies in part to tightly control its stack. > This is not out of the norm, but in order to be picked up by distribution > channels it needs to built against system dependencies, and rebundling is > strictly forbidden. Given that the mesos primary target platform are > data-center distributions such as RHEL/CENTOS/SL it makes sense to still have > bundling support for those who do not have dependencies in their channels > "yet". This is where cmake can be win with it's uber macros > (http://www.cmake.org/cmake/help/v2.8.8/cmake.html#module:ExternalProject). > I do not know of any equivalent in the autotools world, other then to brew > your own solution. I've done this type of work in the past, and completely > transformed condor and would leverage a lot of the work that was done there. > I currently have a tracking branch where I've started this work, but before I > go off into the woods, it makes sense to have a debate in public. > The primary benefits are: > 1. Enable downstream channels to easily distro without carrying a large patch > sets. > 2. Still support existing "non-proper" distribution methods. > 3. Harden / future proof dependent interfaces. > Side Benefits: > Audit current build mechanics. > - Presently the language specific binding are not installed. (.py & .jar) > - make -jX currently fails > - optionally look in arm support. > Costs: > 1. Time > 2. Potential temporary destabilization > 3. Infrastructure around build+test may need to change. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (MESOS-898) Introduce CMake as an alternative build system.
[ https://issues.apache.org/jira/browse/MESOS-898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14599866#comment-14599866 ] Alex Clemmer edited comment on MESOS-898 at 6/24/15 6:13 PM: - Only a few people have given a quick once-over look to the prototype [CMake-based build system for libprocess|https://github.com/hausdorff/mesos/commit/073d42cea1bed8e7fb801f59685f52d3be06e510?diff=unified], but they have been pretty positive, so I would like to target EOD tomorrow (i.e., Th, June 24th) for posting something to ReviewBoard -- I don't want to let the commit hang too long, but I am also new to the community, and I want to give people a chance to build consensus that this commit is at least directionally correct. :) was (Author: hausdorff): Only a few people have given a quick once-over look to the prototype [CMake-based build system for libprocess|https://github.com/hausdorff/mesos/commit/073d42cea1bed8e7fb801f59685f52d3be06e510?diff=unified], but they have been pretty positive, so I would like to target EOD tomorrow (i.e., Th, June 24th) for posting something to ReviewBoard. This slight delay is mainly because I am new to the community, and I want to give people a chance to build consensus that this commit is at least directionally correct. :) > Introduce CMake as an alternative build system. > --- > > Key: MESOS-898 > URL: https://issues.apache.org/jira/browse/MESOS-898 > Project: Mesos > Issue Type: Epic > Components: build >Reporter: Timothy St. Clair >Assignee: Alex Clemmer > Labels: build > > This is a rather substantial undertaking, so I would want upstream > debate+buy-in prior to full commitment. The basic premise is: upstream > rebundles several of its dependencies in part to tightly control its stack. > This is not out of the norm, but in order to be picked up by distribution > channels it needs to built against system dependencies, and rebundling is > strictly forbidden. Given that the mesos primary target platform are > data-center distributions such as RHEL/CENTOS/SL it makes sense to still have > bundling support for those who do not have dependencies in their channels > "yet". This is where cmake can be win with it's uber macros > (http://www.cmake.org/cmake/help/v2.8.8/cmake.html#module:ExternalProject). > I do not know of any equivalent in the autotools world, other then to brew > your own solution. I've done this type of work in the past, and completely > transformed condor and would leverage a lot of the work that was done there. > I currently have a tracking branch where I've started this work, but before I > go off into the woods, it makes sense to have a debate in public. > The primary benefits are: > 1. Enable downstream channels to easily distro without carrying a large patch > sets. > 2. Still support existing "non-proper" distribution methods. > 3. Harden / future proof dependent interfaces. > Side Benefits: > Audit current build mechanics. > - Presently the language specific binding are not installed. (.py & .jar) > - make -jX currently fails > - optionally look in arm support. > Costs: > 1. Time > 2. Potential temporary destabilization > 3. Infrastructure around build+test may need to change. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-898) Introduce CMake as an alternative build system.
[ https://issues.apache.org/jira/browse/MESOS-898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14599866#comment-14599866 ] Alex Clemmer commented on MESOS-898: Only a few people have given a quick once-over look to the prototype [CMake-based build system for libprocess|https://github.com/hausdorff/mesos/commit/073d42cea1bed8e7fb801f59685f52d3be06e510?diff=unified], but they have been pretty positive, so I would like to target EOD tomorrow (i.e., Th, June 24th) for posting something to ReviewBoard. This slight delay is mainly because I am new to the community, and I want to give people a chance to build consensus that this commit is at least directionally correct. :) > Introduce CMake as an alternative build system. > --- > > Key: MESOS-898 > URL: https://issues.apache.org/jira/browse/MESOS-898 > Project: Mesos > Issue Type: Epic > Components: build >Reporter: Timothy St. Clair >Assignee: Alex Clemmer > Labels: build > > This is a rather substantial undertaking, so I would want upstream > debate+buy-in prior to full commitment. The basic premise is: upstream > rebundles several of its dependencies in part to tightly control its stack. > This is not out of the norm, but in order to be picked up by distribution > channels it needs to built against system dependencies, and rebundling is > strictly forbidden. Given that the mesos primary target platform are > data-center distributions such as RHEL/CENTOS/SL it makes sense to still have > bundling support for those who do not have dependencies in their channels > "yet". This is where cmake can be win with it's uber macros > (http://www.cmake.org/cmake/help/v2.8.8/cmake.html#module:ExternalProject). > I do not know of any equivalent in the autotools world, other then to brew > your own solution. I've done this type of work in the past, and completely > transformed condor and would leverage a lot of the work that was done there. > I currently have a tracking branch where I've started this work, but before I > go off into the woods, it makes sense to have a debate in public. > The primary benefits are: > 1. Enable downstream channels to easily distro without carrying a large patch > sets. > 2. Still support existing "non-proper" distribution methods. > 3. Harden / future proof dependent interfaces. > Side Benefits: > Audit current build mechanics. > - Presently the language specific binding are not installed. (.py & .jar) > - make -jX currently fails > - optionally look in arm support. > Costs: > 1. Time > 2. Potential temporary destabilization > 3. Infrastructure around build+test may need to change. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-2795) Introduce filesystem provisioner abstraction to Mesos containerizer
[ https://issues.apache.org/jira/browse/MESOS-2795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14599834#comment-14599834 ] Ian Downes commented on MESOS-2795: --- ContainerImage protobuf: https://reviews.apache.org/r/34136 Add support for container image provisioners. This includes the Provisioner interface and support for provisioners in the MesosContainerizer, including checkpointing and recovery of the optional container rootfs. https://reviews.apache.org/r/34137 > Introduce filesystem provisioner abstraction to Mesos containerizer > --- > > Key: MESOS-2795 > URL: https://issues.apache.org/jira/browse/MESOS-2795 > Project: Mesos > Issue Type: Improvement > Components: isolation >Affects Versions: 0.22.1 >Reporter: Ian Downes >Assignee: Ian Downes > Labels: twitter > > Optional filesystem provisioner component for the Mesos containerizer that > can provision per-container filesystems. > This is different to a filesystem isolators because it just provisions a root > filesystem for a container and doesn't actually do any isolation (e.g., > through a mount namespace + pivot or chroot). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (MESOS-898) Introduce CMake as an alternative build system.
[ https://issues.apache.org/jira/browse/MESOS-898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14599706#comment-14599706 ] Alex Clemmer edited comment on MESOS-898 at 6/24/15 4:47 PM: - I'd like to start this conversation with (what I think is) the simplest step forward. I've put together a prototype CMake-based build system that covers libprocess and the third-party libraries, which successfully builds on several flavors of Linux. It is the [last commit of this branch|https://github.com/hausdorff/mesos/commits/test_cmake]. In general, I'm hoping that this will help focus the discussion around concrete things that I need to fix or do less stupidly, to accomplish our goals in this project. Some notes: * I DON'T KNOW CMAKE (OR AUTOCONF), so if it looks like I'm doing something silly, it's because I am! * It's pretty well-commented. I'm hoping it's not too hard to navigate. * It is build to be x-plat at the outset. So, you should be able to take this file and generate makefiles or, like, nmake files. * This commit spans Mesos, libprocess, and stout (rather than separating them out) mainly to make it easy to consume -- for the "real" solution, I will happily split them out appropriately. * So far, this prototype commit follows [~haosd...@gmail.com]'s supposition that we want to build the CMake system incrementally and in parallel (though I want to note that stout is a header-only library, and does not need to be built). * I agree that with [~tstclair] that it is probably best that we coalesce libprocess and stout into one sort of "systems layer", but given the progress here so far, I'd like to postpone that discussion for another time. I'm willing to rewrite this part of the build system in the event that those changes materialize. Some things on the roadmap if you all like the progress so far: [x] Third-party dependencies are downloaded, configured, and built when you run `make` (or whatever). [ ] Support for system installations of the third-party dependencies (i.e., we should allow users to use glog if it's already installed on their machine) [x] Tests build and run on multiple flavors of Linux [ ] Benchmarks build and run on multiple flavors of Linux n.b., for issue 2 I have not decided what the "right" way of communicating the system dependencies to CMake is. In the autoconf solution this comes from `configure`, but CMake will not have a configure step because it needs to run not only on POSIX machines. EDIT: sorry, folks, for double posting this. I don't interact with JIRA much. was (Author: hausdorff): I'd like to start this conversation with (what I think is) the simplest step forward. I've put together a prototype CMake-based build system that covers libprocess and the third-party libraries, which successfully builds on several flavors of Linux. It is the [last commit of this branch|https://github.com/hausdorff/mesos/commits/test_cmake]. In general, I'm hoping that this will help focus the discussion around concrete things that I need to fix or do less stupidly, to accomplish our goals in this project. Some notes: * I DON'T KNOW CMAKE (OR AUTOCONF), so if it looks like I'm doing something silly, it's because I am! * It's pretty well-commented. I'm hoping it's not too hard to navigate. * It is build to be x-plat at the outset. So, you should be able to take this file and generate makefiles or, like, nmake files. * This commit spans Mesos, libprocess, and stout (rather than separating them out) mainly to make it easy to consume -- for the "real" solution, I will happily split them out appropriately. * So far, this prototype commit follows [~haosd...@gmail.com]'s supposition that we want to build the CMake system incrementally and in parallel (though I want to note that stout is a header-only library, and does not need to be built). Some things on the roadmap if you all like the progress so far: [x] Third-party dependencies are downloaded, configured, and built when you run `make` (or whatever). [ ] Support for system installations of the third-party dependencies (i.e., we should allow users to use glog if it's already installed on their machine) [x] Tests build and run on multiple flavors of Linux [ ] Benchmarks build and run on multiple flavors of Linux n.b., for issue 2 I have not decided what the "right" way of communicating the system dependencies to CMake is. In the autoconf solution this comes from `configure`, but CMake will not have a configure step because it needs to run not only on POSIX machines. EDIT: sorry, folks, for double posting this. I don't interact with JIRA much. > Introduce CMake as an alternative build system. > --- > > Key: MESOS-898 > URL: https://issues.apache.org/jira/browse/MESOS-898 > Project: Mesos > Issue Typ
[jira] [Comment Edited] (MESOS-898) Introduce CMake as an alternative build system.
[ https://issues.apache.org/jira/browse/MESOS-898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14599706#comment-14599706 ] Alex Clemmer edited comment on MESOS-898 at 6/24/15 4:43 PM: - I'd like to start this conversation with (what I think is) the simplest step forward. I've put together a prototype CMake-based build system that covers libprocess and the third-party libraries, which successfully builds on several flavors of Linux. It is the [last commit of this branch|https://github.com/hausdorff/mesos/commits/test_cmake]. In general, I'm hoping that this will help focus the discussion around concrete things that I need to fix or do less stupidly, to accomplish our goals in this project. Some notes: * I DON'T KNOW CMAKE (OR AUTOCONF), so if it looks like I'm doing something silly, it's because I am! * It's pretty well-commented. I'm hoping it's not too hard to navigate. * It is build to be x-plat at the outset. So, you should be able to take this file and generate makefiles or, like, nmake files. * This commit spans Mesos, libprocess, and stout (rather than separating them out) mainly to make it easy to consume -- for the "real" solution, I will happily split them out appropriately. * So far, this prototype commit follows [~haosd...@gmail.com]'s supposition that we want to build the CMake system incrementally and in parallel (though I want to note that stout is a header-only library, and does not need to be built). Some things on the roadmap if you all like the progress so far: [x] Third-party dependencies are downloaded, configured, and built when you run `make` (or whatever). [ ] Support for system installations of the third-party dependencies (i.e., we should allow users to use glog if it's already installed on their machine) [x] Tests build and run on multiple flavors of Linux [ ] Benchmarks build and run on multiple flavors of Linux n.b., for issue 2 I have not decided what the "right" way of communicating the system dependencies to CMake is. In the autoconf solution this comes from `configure`, but CMake will not have a configure step because it needs to run not only on POSIX machines. EDIT: sorry, folks, for double posting this. I don't interact with JIRA much. was (Author: hausdorff): I'd like to start this conversation with (what I think is) the simplest step forward. I've put together a prototype CMake-based build system that covers libprocess and the third-party libraries, which successfully builds on several flavors of Linux. It is the [last commit of this branch|https://github.com/hausdorff/mesos/commits/test_cmake]. In general, I'm hoping that this will help focus the discussion around concrete things that I need to fix or do less stupidly, to accomplish our goals in this project. Some notes: * I DON'T KNOW CMAKE (OR AUTOCONF), so if it looks like I'm doing something silly, it's because I am! * It's pretty well-commented. I'm hoping it's not too hard to navigate. * It is build to be x-plat at the outset. So, you should be able to take this file and generate makefiles or, like, nmake files. Some things on the roadmap if you all like the progress so far: [x] Third-party dependencies are downloaded, configured, and built when you run `make` (or whatever). [ ] Support for system installations of the third-party dependencies (i.e., we should allow users to use glog if it's already installed on their machine) [x] Tests build and run on multiple flavors of Linux [ ] Benchmarks build and run on multiple flavors of Linux n.b., for issue 2 I have not decided what the "right" way of communicating the system dependencies to CMake is. In the autoconf solution this comes from `configure`, but CMake will not have a configure step because it needs to run not only on POSIX machines. EDIT: sorry, folks, for double posting this. I don't interact with JIRA much. > Introduce CMake as an alternative build system. > --- > > Key: MESOS-898 > URL: https://issues.apache.org/jira/browse/MESOS-898 > Project: Mesos > Issue Type: Epic > Components: build >Reporter: Timothy St. Clair >Assignee: Alex Clemmer > Labels: build > > This is a rather substantial undertaking, so I would want upstream > debate+buy-in prior to full commitment. The basic premise is: upstream > rebundles several of its dependencies in part to tightly control its stack. > This is not out of the norm, but in order to be picked up by distribution > channels it needs to built against system dependencies, and rebundling is > strictly forbidden. Given that the mesos primary target platform are > data-center distributions such as RHEL/CENTOS/SL it makes sense to still have > bundling support for those who do not have dependencies in their channels
[jira] [Comment Edited] (MESOS-898) Introduce CMake as an alternative build system.
[ https://issues.apache.org/jira/browse/MESOS-898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14599706#comment-14599706 ] Alex Clemmer edited comment on MESOS-898 at 6/24/15 4:39 PM: - I'd like to start this conversation with (what I think is) the simplest step forward. I've put together a prototype CMake-based build system that covers libprocess and the third-party libraries, which successfully builds on several flavors of Linux. It is the [last commit of this branch|https://github.com/hausdorff/mesos/commits/test_cmake]. In general, I'm hoping that this will help focus the discussion around concrete things that I need to fix or do less stupidly, to accomplish our goals in this project. Some notes: * I DON'T KNOW CMAKE (OR AUTOCONF), so if it looks like I'm doing something silly, it's because I am! * It's pretty well-commented. I'm hoping it's not too hard to navigate. * It is build to be x-plat at the outset. So, you should be able to take this file and generate makefiles or, like, nmake files. Some things on the roadmap if you all like the progress so far: [x] Third-party dependencies are downloaded, configured, and built when you run `make` (or whatever). [ ] Support for system installations of the third-party dependencies (i.e., we should allow users to use glog if it's already installed on their machine) [x] Tests build and run on multiple flavors of Linux [ ] Benchmarks build and run on multiple flavors of Linux n.b., for issue 2 I have not decided what the "right" way of communicating the system dependencies to CMake is. In the autoconf solution this comes from `configure`, but CMake will not have a configure step because it needs to run not only on POSIX machines. EDIT: sorry, folks, for screwing up the posting of this the first time. I don't interact with JIRA much. was (Author: hausdorff): I'd like to start this conversation with (what I think is) the simplest step forward. I've put together a prototype CMake-based build system that covers libprocess and the third-party libraries, which successfully builds on several flavors of Linux. It is the [last commit of this branch|https://github.com/hausdorff/mesos/commits/test_cmake]. In general, I'm hoping that this will help focus the discussion around concrete things that I need to fix or do less stupidly, to accomplish our goals in this project. Some notes: * I DON'T KNOW CMAKE (OR AUTOCONF), so if it looks like I'm doing something silly, it's because I am! * It's pretty well-commented. I'm hoping it's not too hard to navigate. * It is build to be x-plat at the outset. So, you should be able to take this file and generate makefiles or, like, nmake files. Some things on the roadmap if you all like the progress so far: [x] Third-party dependencies are downloaded, configured, and built when you run `make` (or whatever). [ ] Support for system installations of the third-party dependencies (i.e., we should allow users to use glog if it's already installed on their machine) [x] Tests build and run on multiple flavors of Linux [ ] Benchmarks build and run on multiple flavors of Linux n.b., for issue 2 I have not decided what the "right" way of communicating the system dependencies to CMake is. In the autoconf solution this comes from `configure`, but CMake will not have a configure step because it needs to run not only on POSIX machines. > Introduce CMake as an alternative build system. > --- > > Key: MESOS-898 > URL: https://issues.apache.org/jira/browse/MESOS-898 > Project: Mesos > Issue Type: Epic > Components: build >Reporter: Timothy St. Clair >Assignee: Alex Clemmer > Labels: build > > This is a rather substantial undertaking, so I would want upstream > debate+buy-in prior to full commitment. The basic premise is: upstream > rebundles several of its dependencies in part to tightly control its stack. > This is not out of the norm, but in order to be picked up by distribution > channels it needs to built against system dependencies, and rebundling is > strictly forbidden. Given that the mesos primary target platform are > data-center distributions such as RHEL/CENTOS/SL it makes sense to still have > bundling support for those who do not have dependencies in their channels > "yet". This is where cmake can be win with it's uber macros > (http://www.cmake.org/cmake/help/v2.8.8/cmake.html#module:ExternalProject). > I do not know of any equivalent in the autotools world, other then to brew > your own solution. I've done this type of work in the past, and completely > transformed condor and would leverage a lot of the work that was done there. > I currently have a tracking branch where I've started this work, but before I > go off into the wo
[jira] [Commented] (MESOS-898) Introduce CMake as an alternative build system.
[ https://issues.apache.org/jira/browse/MESOS-898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14599706#comment-14599706 ] Alex Clemmer commented on MESOS-898: I'd like to start this conversation with (what I think is) the simplest step forward. I've put together a prototype CMake-based build system that covers libprocess and the third-party libraries, which successfully builds on several flavors of Linux. It is the [last commit of this branch|https://github.com/hausdorff/mesos/commits/test_cmake]. In general, I'm hoping that this will help focus the discussion around concrete things that I need to fix or do less stupidly, to accomplish our goals in this project. Some notes: * I DON'T KNOW CMAKE (OR AUTOCONF), so if it looks like I'm doing something silly, it's because I am! * It's pretty well-commented. I'm hoping it's not too hard to navigate. * It is build to be x-plat at the outset. So, you should be able to take this file and generate makefiles or, like, nmake files. Some things on the roadmap if you all like the progress so far: [x] Third-party dependencies are downloaded, configured, and built when you run `make` (or whatever). [ ] Support for system installations of the third-party dependencies (i.e., we should allow users to use glog if it's already installed on their machine) [x] Tests build and run on multiple flavors of Linux [ ] Benchmarks build and run on multiple flavors of Linux n.b., for issue 2 I have not decided what the "right" way of communicating the system dependencies to CMake is. In the autoconf solution this comes from `configure`, but CMake will not have a configure step because it needs to run not only on POSIX machines. > Introduce CMake as an alternative build system. > --- > > Key: MESOS-898 > URL: https://issues.apache.org/jira/browse/MESOS-898 > Project: Mesos > Issue Type: Epic > Components: build >Reporter: Timothy St. Clair >Assignee: Alex Clemmer > Labels: build > > This is a rather substantial undertaking, so I would want upstream > debate+buy-in prior to full commitment. The basic premise is: upstream > rebundles several of its dependencies in part to tightly control its stack. > This is not out of the norm, but in order to be picked up by distribution > channels it needs to built against system dependencies, and rebundling is > strictly forbidden. Given that the mesos primary target platform are > data-center distributions such as RHEL/CENTOS/SL it makes sense to still have > bundling support for those who do not have dependencies in their channels > "yet". This is where cmake can be win with it's uber macros > (http://www.cmake.org/cmake/help/v2.8.8/cmake.html#module:ExternalProject). > I do not know of any equivalent in the autotools world, other then to brew > your own solution. I've done this type of work in the past, and completely > transformed condor and would leverage a lot of the work that was done there. > I currently have a tracking branch where I've started this work, but before I > go off into the woods, it makes sense to have a debate in public. > The primary benefits are: > 1. Enable downstream channels to easily distro without carrying a large patch > sets. > 2. Still support existing "non-proper" distribution methods. > 3. Harden / future proof dependent interfaces. > Side Benefits: > Audit current build mechanics. > - Presently the language specific binding are not installed. (.py & .jar) > - make -jX currently fails > - optionally look in arm support. > Costs: > 1. Time > 2. Potential temporary destabilization > 3. Infrastructure around build+test may need to change. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (MESOS-898) Introduce CMake as an alternative build system.
[ https://issues.apache.org/jira/browse/MESOS-898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14599706#comment-14599706 ] Alex Clemmer edited comment on MESOS-898 at 6/24/15 4:40 PM: - I'd like to start this conversation with (what I think is) the simplest step forward. I've put together a prototype CMake-based build system that covers libprocess and the third-party libraries, which successfully builds on several flavors of Linux. It is the [last commit of this branch|https://github.com/hausdorff/mesos/commits/test_cmake]. In general, I'm hoping that this will help focus the discussion around concrete things that I need to fix or do less stupidly, to accomplish our goals in this project. Some notes: * I DON'T KNOW CMAKE (OR AUTOCONF), so if it looks like I'm doing something silly, it's because I am! * It's pretty well-commented. I'm hoping it's not too hard to navigate. * It is build to be x-plat at the outset. So, you should be able to take this file and generate makefiles or, like, nmake files. Some things on the roadmap if you all like the progress so far: [x] Third-party dependencies are downloaded, configured, and built when you run `make` (or whatever). [ ] Support for system installations of the third-party dependencies (i.e., we should allow users to use glog if it's already installed on their machine) [x] Tests build and run on multiple flavors of Linux [ ] Benchmarks build and run on multiple flavors of Linux n.b., for issue 2 I have not decided what the "right" way of communicating the system dependencies to CMake is. In the autoconf solution this comes from `configure`, but CMake will not have a configure step because it needs to run not only on POSIX machines. EDIT: sorry, folks, for double posting this. I don't interact with JIRA much. was (Author: hausdorff): I'd like to start this conversation with (what I think is) the simplest step forward. I've put together a prototype CMake-based build system that covers libprocess and the third-party libraries, which successfully builds on several flavors of Linux. It is the [last commit of this branch|https://github.com/hausdorff/mesos/commits/test_cmake]. In general, I'm hoping that this will help focus the discussion around concrete things that I need to fix or do less stupidly, to accomplish our goals in this project. Some notes: * I DON'T KNOW CMAKE (OR AUTOCONF), so if it looks like I'm doing something silly, it's because I am! * It's pretty well-commented. I'm hoping it's not too hard to navigate. * It is build to be x-plat at the outset. So, you should be able to take this file and generate makefiles or, like, nmake files. Some things on the roadmap if you all like the progress so far: [x] Third-party dependencies are downloaded, configured, and built when you run `make` (or whatever). [ ] Support for system installations of the third-party dependencies (i.e., we should allow users to use glog if it's already installed on their machine) [x] Tests build and run on multiple flavors of Linux [ ] Benchmarks build and run on multiple flavors of Linux n.b., for issue 2 I have not decided what the "right" way of communicating the system dependencies to CMake is. In the autoconf solution this comes from `configure`, but CMake will not have a configure step because it needs to run not only on POSIX machines. EDIT: sorry, folks, for screwing up the posting of this the first time. I don't interact with JIRA much. > Introduce CMake as an alternative build system. > --- > > Key: MESOS-898 > URL: https://issues.apache.org/jira/browse/MESOS-898 > Project: Mesos > Issue Type: Epic > Components: build >Reporter: Timothy St. Clair >Assignee: Alex Clemmer > Labels: build > > This is a rather substantial undertaking, so I would want upstream > debate+buy-in prior to full commitment. The basic premise is: upstream > rebundles several of its dependencies in part to tightly control its stack. > This is not out of the norm, but in order to be picked up by distribution > channels it needs to built against system dependencies, and rebundling is > strictly forbidden. Given that the mesos primary target platform are > data-center distributions such as RHEL/CENTOS/SL it makes sense to still have > bundling support for those who do not have dependencies in their channels > "yet". This is where cmake can be win with it's uber macros > (http://www.cmake.org/cmake/help/v2.8.8/cmake.html#module:ExternalProject). > I do not know of any equivalent in the autotools world, other then to brew > your own solution. I've done this type of work in the past, and completely > transformed condor and would leverage a lot of the work that was done there. > I currently have a
[jira] [Issue Comment Deleted] (MESOS-898) Introduce CMake as an alternative build system.
[ https://issues.apache.org/jira/browse/MESOS-898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Clemmer updated MESOS-898: --- Comment: was deleted (was: I'd like to start this conversation with (what I think is) the simplest step forward. I've put together a prototype CMake-based build system that covers libprocess and the third-party libraries, which successfully builds on several flavors of Linux. It is the [last commit of this branch|https://github.com/hausdorff/mesos/commits/test_cmake]. In general, I'm hoping that this will help focus the discussion around concrete things that I need to fix or do less stupidly, to accomplish our goals in this project. Some notes: * I DON'T KNOW CMAKE (OR AUTOCONF), so if it looks like I'm doing something silly, it's because I am! * It's pretty well-commented. I'm hoping it's not too hard to navigate. * It is build to be x-plat at the outset. So, you should be able to take this file and generate makefiles or, like, nmake files. Some things on the roadmap if you all like the progress so far: [x] Third-party dependencies are downloaded, configured, and built when you run `make` (or whatever). [ ] Support for system installations of the third-party dependencies (i.e., we should allow users to use glog if it's already installed on their machine) [x] Tests build and run on multiple flavors of Linux [ ] Benchmarks build and run on multiple flavors of Linux n.b., for issue 2 I have not decided what the "right" way of communicating the system dependencies to CMake is. In the autoconf solution this comes from `configure`, but CMake will not have a configure step because it needs to run not only on POSIX machines.) > Introduce CMake as an alternative build system. > --- > > Key: MESOS-898 > URL: https://issues.apache.org/jira/browse/MESOS-898 > Project: Mesos > Issue Type: Epic > Components: build >Reporter: Timothy St. Clair >Assignee: Alex Clemmer > Labels: build > > This is a rather substantial undertaking, so I would want upstream > debate+buy-in prior to full commitment. The basic premise is: upstream > rebundles several of its dependencies in part to tightly control its stack. > This is not out of the norm, but in order to be picked up by distribution > channels it needs to built against system dependencies, and rebundling is > strictly forbidden. Given that the mesos primary target platform are > data-center distributions such as RHEL/CENTOS/SL it makes sense to still have > bundling support for those who do not have dependencies in their channels > "yet". This is where cmake can be win with it's uber macros > (http://www.cmake.org/cmake/help/v2.8.8/cmake.html#module:ExternalProject). > I do not know of any equivalent in the autotools world, other then to brew > your own solution. I've done this type of work in the past, and completely > transformed condor and would leverage a lot of the work that was done there. > I currently have a tracking branch where I've started this work, but before I > go off into the woods, it makes sense to have a debate in public. > The primary benefits are: > 1. Enable downstream channels to easily distro without carrying a large patch > sets. > 2. Still support existing "non-proper" distribution methods. > 3. Harden / future proof dependent interfaces. > Side Benefits: > Audit current build mechanics. > - Presently the language specific binding are not installed. (.py & .jar) > - make -jX currently fails > - optionally look in arm support. > Costs: > 1. Time > 2. Potential temporary destabilization > 3. Infrastructure around build+test may need to change. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-898) Introduce CMake as an alternative build system.
[ https://issues.apache.org/jira/browse/MESOS-898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14599705#comment-14599705 ] Alex Clemmer commented on MESOS-898: I'd like to start this conversation with (what I think is) the simplest step forward. I've put together a prototype CMake-based build system that covers libprocess and the third-party libraries, which successfully builds on several flavors of Linux. It is the [last commit of this branch|https://github.com/hausdorff/mesos/commits/test_cmake]. In general, I'm hoping that this will help focus the discussion around concrete things that I need to fix or do less stupidly, to accomplish our goals in this project. Some notes: * I DON'T KNOW CMAKE (OR AUTOCONF), so if it looks like I'm doing something silly, it's because I am! * It's pretty well-commented. I'm hoping it's not too hard to navigate. * It is build to be x-plat at the outset. So, you should be able to take this file and generate makefiles or, like, nmake files. Some things on the roadmap if you all like the progress so far: [x] Third-party dependencies are downloaded, configured, and built when you run `make` (or whatever). [ ] Support for system installations of the third-party dependencies (i.e., we should allow users to use glog if it's already installed on their machine) [x] Tests build and run on multiple flavors of Linux [ ] Benchmarks build and run on multiple flavors of Linux n.b., for issue 2 I have not decided what the "right" way of communicating the system dependencies to CMake is. In the autoconf solution this comes from `configure`, but CMake will not have a configure step because it needs to run not only on POSIX machines. > Introduce CMake as an alternative build system. > --- > > Key: MESOS-898 > URL: https://issues.apache.org/jira/browse/MESOS-898 > Project: Mesos > Issue Type: Epic > Components: build >Reporter: Timothy St. Clair >Assignee: Alex Clemmer > Labels: build > > This is a rather substantial undertaking, so I would want upstream > debate+buy-in prior to full commitment. The basic premise is: upstream > rebundles several of its dependencies in part to tightly control its stack. > This is not out of the norm, but in order to be picked up by distribution > channels it needs to built against system dependencies, and rebundling is > strictly forbidden. Given that the mesos primary target platform are > data-center distributions such as RHEL/CENTOS/SL it makes sense to still have > bundling support for those who do not have dependencies in their channels > "yet". This is where cmake can be win with it's uber macros > (http://www.cmake.org/cmake/help/v2.8.8/cmake.html#module:ExternalProject). > I do not know of any equivalent in the autotools world, other then to brew > your own solution. I've done this type of work in the past, and completely > transformed condor and would leverage a lot of the work that was done there. > I currently have a tracking branch where I've started this work, but before I > go off into the woods, it makes sense to have a debate in public. > The primary benefits are: > 1. Enable downstream channels to easily distro without carrying a large patch > sets. > 2. Still support existing "non-proper" distribution methods. > 3. Harden / future proof dependent interfaces. > Side Benefits: > Audit current build mechanics. > - Presently the language specific binding are not installed. (.py & .jar) > - make -jX currently fails > - optionally look in arm support. > Costs: > 1. Time > 2. Potential temporary destabilization > 3. Infrastructure around build+test may need to change. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-2664) Modernize the codebase to C++11
[
https://issues.apache.org/jira/browse/MESOS-2664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14599623#comment-14599623
]
Michael Park commented on MESOS-2664:
-
[~bmahler] I think {{-Wswitch}} _almost_ captures what we want.
Yes, to both (a) and (b). The missing part for {{-Wswitch}} for us is: (c) If
we covered all the cases *and* have a {{default}}, it would be nice for that to
also be a warning. When we add a new enum case, I think what we want is a
warning saying "you need to handle this new enum value here!" rather than
silently going into the {{default}} case. An example from our codebase:
{code}
switch (volume.mode()) {
case Volume::RW: volumeConfig += ":rw"; break;
case Volume::RO: volumeConfig += ":ro"; break;
default:
LOG(FATAL) << "Unknown Volume mode: " << volume.mode();
break;
}
{code}
I think we can capture what we want with {{-Wswitch}} + a style guide around
the use {{default}}. What do you think?
P.S. We currently have {{-Wall}} turned on, which includes {{-Wswitch}} :)
> Modernize the codebase to C++11
> ---
>
> Key: MESOS-2664
> URL: https://issues.apache.org/jira/browse/MESOS-2664
> Project: Mesos
> Issue Type: Epic
> Components: technical debt
>Reporter: Michael Park
>Assignee: Michael Park
> Labels: mesosphere
>
> Since [this
> commit|https://github.com/apache/mesos/commit/0f5c78fad3423181f7227027eb42d162811514e7],
> we officially require GCC-4.8+ and Clang-3.5+. This means that we now have
> full C++11 support and therefore can start to modernize our codebase to be
> more readable, safer and efficient!
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-2545) Developer guide for libprocess
[ https://issues.apache.org/jira/browse/MESOS-2545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14599613#comment-14599613 ] Bernd Mathiske commented on MESOS-2545: --- commit 98f85e1f81b86502013fdf1783f9d8ec65eae3d0 Author: Joerg Schad Date: Wed Jun 24 17:47:30 2015 +0200 Remove (almost all) html from libprocess Developer Guide. Review: https://reviews.apache.org/r/35568 > Developer guide for libprocess > -- > > Key: MESOS-2545 > URL: https://issues.apache.org/jira/browse/MESOS-2545 > Project: Mesos > Issue Type: Documentation > Components: libprocess >Reporter: Bernd Mathiske >Assignee: Joerg Schad > Labels: documentation, mesosphere > > Create a developer guide for libprocess that explains the philosophy behind > it and explains the most important features as well as the prevalent use > patterns in Mesos with examples. > This could be similar to stout/README.md. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-2855) Update operational guide to include growing from standalone to high availability
[
https://issues.apache.org/jira/browse/MESOS-2855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14599603#comment-14599603
]
Michael Schenck commented on MESOS-2855:
https://reviews.apache.org/r/35361/
> Update operational guide to include growing from standalone to high
> availability
>
>
> Key: MESOS-2855
> URL: https://issues.apache.org/jira/browse/MESOS-2855
> Project: Mesos
> Issue Type: Documentation
>Reporter: Michael Schenck
>Assignee: Michael Schenck
> Labels: documentation
>
> The [Operational
> Guide|http://mesos.apache.org/documentation/latest/operational-guide/] covers
> increasing quorum size from {{--quorum=2}}, but does not cover how to move
> from a _standalone_ master to a high availability configuration.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (MESOS-2924) Allow simple construction via initializer list on hashset.
Till Toenshoff created MESOS-2924:
-
Summary: Allow simple construction via initializer list on hashset.
Key: MESOS-2924
URL: https://issues.apache.org/jira/browse/MESOS-2924
Project: Mesos
Issue Type: Improvement
Reporter: Till Toenshoff
{{hashmap}} already has a initializer-list constructor, {{hashset}} should
offer the same.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MESOS-2924) Allow simple construction via initializer list on hashset.
[
https://issues.apache.org/jira/browse/MESOS-2924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Till Toenshoff updated MESOS-2924:
--
Priority: Minor (was: Major)
> Allow simple construction via initializer list on hashset.
> --
>
> Key: MESOS-2924
> URL: https://issues.apache.org/jira/browse/MESOS-2924
> Project: Mesos
> Issue Type: Improvement
>Reporter: Till Toenshoff
>Priority: Minor
>
> {{hashmap}} already has a initializer-list constructor, {{hashset}} should
> offer the same.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (MESOS-2923) fetcher.cpp - problem with certificates..?
Tomasz Mieszkowski created MESOS-2923:
-
Summary: fetcher.cpp - problem with certificates..?
Key: MESOS-2923
URL: https://issues.apache.org/jira/browse/MESOS-2923
Project: Mesos
Issue Type: Bug
Affects Versions: 0.22.1, 0.22.0
Environment: Ubuntu 14.04 (build + test)
Reporter: Tomasz Mieszkowski
Mesos 0.22.0/0.22.1 built and installed from sources accordingly to the
instructions given [here|http://mesos.apache.org/gettingstarted/] has some
problem with certificates.
Every time I try to deploy something that requires downloading any resource via
HTTPS (with URI specified via Marathon), such deployment fails and I get this
message in failed app's sandbox:
{code}
E0617 09:58:44.339409 12380 fetcher.cpp:138] Error downloading resource:
Problem with the SSL CA cert (path? access rights?)
{code}
Trying to download the same resource on the same slave with {{curl}} or
{{wget}} works without problems.
Moreover, when I install exactly the same version of Mesos from Mesosphere's
debs on identical machines (i.e., set up by the same Ansible scripts),
everything works fine as well.
I guess it must be something related to the way how Mesos is built - maybe some
missing switch for {{configure}} or {{make}}..?
Any ideas..?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-2123) Document changes in C++ Resources API in CHANGELOG.
[ https://issues.apache.org/jira/browse/MESOS-2123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14599208#comment-14599208 ] Adam B commented on MESOS-2123: --- Also note that state.json's slave "resources" field now includes checkpointed resources (persistent volumes, dynamic reservations), which may also be worth mentioning as an API change. See https://issues.apache.org/jira/browse/MESOS-692 for more info. > Document changes in C++ Resources API in CHANGELOG. > --- > > Key: MESOS-2123 > URL: https://issues.apache.org/jira/browse/MESOS-2123 > Project: Mesos > Issue Type: Task >Reporter: Jie Yu > Labels: twitter > > With the refactor introduced in MESOS-1974, we need to document those API > changes in CHANGELOG. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-2884) Allow isolators to specify required namespaces
[ https://issues.apache.org/jira/browse/MESOS-2884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14599202#comment-14599202 ] Adam B commented on MESOS-2884: --- [~karya] These RRs have been committed. Is there anything else left to do for this ticket? commit 1a82a3fb2bc717c468218384190a115b770f88c3 Author: Kapil Arya Date: Tue Jun 23 12:33:41 2015 -0700 Updated LinuxLauncher to receive list of namespaces. MesosContainerizer looks up the list of required namespaces by calling Isolator::namespaces() for all enabled isolators and passes on this value to LinuxLauncher. Review: https://reviews.apache.org/r/35586 commit 2143ae0315990ed663bf5810a801adeacff3a986 Author: Kapil Arya Date: Tue Jun 23 12:32:32 2015 -0700 Updated Isolator to return required namespaces. This would enable the MesosContainerizer to pass on a list of namespaces to LinuxLauncher instead of having LinuxLauncher guess it from the isolation flags. Review: https://reviews.apache.org/r/35585 > Allow isolators to specify required namespaces > -- > > Key: MESOS-2884 > URL: https://issues.apache.org/jira/browse/MESOS-2884 > Project: Mesos > Issue Type: Task > Components: isolation >Reporter: Kapil Arya >Assignee: Kapil Arya > Labels: mesosphere > > Currently, the LinuxLauncher looks into SlaveFlags to compute the namespaces > that should be enabled when launching the executor. This means that a custom > Isolator module doesn't have any way to specify dependency on a set of > namespaces. > The proposed solution is to extend the Isolator interface to also export the > namespaces dependency. This way the MesosContainerizer can directly query all > loaded Isolators (inbuilt and custom modules) to compute the set of > namespaces required by the executor. This set of namespaces is then passed on > to the LinuxLauncher. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-2673) Follow Google Style Guide for header file include order completely.
[ https://issues.apache.org/jira/browse/MESOS-2673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14599195#comment-14599195 ] Adam B commented on MESOS-2673: --- [~js84] If, as described in the gist, this Epic describes not just updating the style guide, but also making the changes in stout/libprocess/mesos, then I highly doubt that we will complete this Epic in time for 0.23. Could you please retarget to 0.24 if appropriate? Thanks! > Follow Google Style Guide for header file include order completely. > --- > > Key: MESOS-2673 > URL: https://issues.apache.org/jira/browse/MESOS-2673 > Project: Mesos > Issue Type: Improvement >Reporter: Joerg Schad >Assignee: Joerg Schad >Priority: Minor > Labels: mesosphere > Fix For: 0.23.0 > > > The header include order for Mesos actually follows the Google Styleguide but > omits step 1 without mentioning this exception in the Mesos styleguide. This > proposal suggests to adapt to the include order explained in the Google > Styleguide i.e. include the direct headers first in the .cpp files > implementing them. > A gist of the proposal can be found here: > https://gist.github.com/joerg84/65cb9611d24b2e35b69b > The corresponding Review Board review can be found here: > https://reviews.apache.org/r/33646/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-2350) Add support for MesosContainerizerLaunch to chroot to a specified path
[ https://issues.apache.org/jira/browse/MESOS-2350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14599187#comment-14599187 ] Adam B commented on MESOS-2350: --- Auxiliary reviews were committed. Still waiting on the "Main review": https://reviews.apache.org/r/31444/ commit eafef12b3b0f6a6e48f1d7e623f3a81c4834938c Author: Ian Downes Date: Wed Feb 25 11:43:53 2015 -0800 Accept dummy arguments for fs::mount(). Review: https://reviews.apache.org/r/31443 commit 55869a5c33fd96202164021e795b473ede3415fe Author: Ian Downes Date: Wed Feb 25 11:21:11 2015 -0800 Add Linux pivot_root() wrapper. Review: https://reviews.apache.org/r/31442 commit b37c1ba55a977cbc19bf1e2d9e418cb6a8d16169 Author: Ian Downes Date: Wed Feb 25 11:17:24 2015 -0800 Add os::mknod() wrapper. Review: https://reviews.apache.org/r/31441 commit 7d82bb72e9b9b9d64ac65e6aa8e6193f2438fb1f Author: Ian Downes Date: Tue Mar 3 14:42:48 2015 -0800 Add os::stat::rdev(). Review: https://reviews.apache.org/r/31438/ commit 0f9cf3ec425cceececd40cad906be67279064f9f Author: Ian Downes Date: Tue Mar 3 14:42:25 2015 -0800 Add os::stat::mode(). Review: https://reviews.apache.org/r/31440/ commit 5beab91afcdf23b4a5184881bd994c623a3fd2c5 Author: Ian Downes Date: Wed Feb 25 14:20:44 2015 -0800 Add os::chroot() wrapper. Review: https://reviews.apache.org/r/31439 > Add support for MesosContainerizerLaunch to chroot to a specified path > -- > > Key: MESOS-2350 > URL: https://issues.apache.org/jira/browse/MESOS-2350 > Project: Mesos > Issue Type: Improvement > Components: isolation >Affects Versions: 0.21.1, 0.22.0 >Reporter: Ian Downes >Assignee: Ian Downes > Labels: twitter > > In preparation for the MesosContainerizer to support a filesystem isolator > the MesosContainerizerLauncher must support chrooting. Optionally, it should > also configure the chroot environment by (re-)mounting special filesystems > such as /proc and /sys and making device nodes such as /dev/zero, etc., such > that the chroot environment is functional. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-2349) Provide a way to execute an arbitrary process in a MesosContainerizer container context
[
https://issues.apache.org/jira/browse/MESOS-2349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14599181#comment-14599181
]
Adam B commented on MESOS-2349:
---
[~idownes] Have you started progress on this yet? If not, I doubt it will make
it into 0.23, so we should retarget it.
> Provide a way to execute an arbitrary process in a MesosContainerizer
> container context
> ---
>
> Key: MESOS-2349
> URL: https://issues.apache.org/jira/browse/MESOS-2349
> Project: Mesos
> Issue Type: Improvement
> Components: isolation
>Affects Versions: 0.21.1, 0.22.0
>Reporter: Ian Downes
>Assignee: Ian Downes
> Labels: twitter
>
> Include a separate binary that when provided with a container_id, path to an
> executable, and optional arguments will find the container context, enter it,
> and exec the executable.
> e.g.,
> {noformat}
> mesos-container-exec --container_id=abc123 [--] /path/to/executable [arg1 ...]
> {noformat}
> This need only support (initially) containers created with the
> MesosContainerizer and will support all isolators shipped with Mesos, i.e.,
> it should find and enter the cgroups and namespaces for the running executor
> of the specified container.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MESOS-2888) Add SSL socket tests
[ https://issues.apache.org/jira/browse/MESOS-2888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adam B updated MESOS-2888: -- Target Version/s: 0.23.0 > Add SSL socket tests > > > Key: MESOS-2888 > URL: https://issues.apache.org/jira/browse/MESOS-2888 > Project: Mesos > Issue Type: Improvement > Components: libprocess >Reporter: Joris Van Remoortere >Assignee: Joris Van Remoortere > Labels: libprocess, mesosphere, ssl, tests > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-1791) Introduce Master / Offer Resource Reservations aka Quota
[ https://issues.apache.org/jira/browse/MESOS-1791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Rukletsov updated MESOS-1791: --- Epic Name: Quota (was: Offer Reservations) Component/s: replicated log master allocation Summary: Introduce Master / Offer Resource Reservations aka Quota (was: Introduce Master / Offer Resource Reservations) > Introduce Master / Offer Resource Reservations aka Quota > > > Key: MESOS-1791 > URL: https://issues.apache.org/jira/browse/MESOS-1791 > Project: Mesos > Issue Type: Epic > Components: allocation, master, replicated log >Reporter: Tom Arnfeld > Labels: mesosphere > > Currently Mesos supports the ability to reserve resources (for a given role) > on a per-slave basis, as introduced in MESOS-505. This allows you to almost > statically partition off a set of resources on a set of machines, to > guarantee certain types of frameworks get some resources. > This is very useful, though it is also very useful to be able to control > these reservations through the master (instead of per-slave) for when I don't > care which nodes I get on, as long as I get X cpu and Y RAM, or Z sets of > (X,Y). > I'm not sure what structure this could take, but apparently it has already > been discussed. Would this be a CLI flag? Could there be a (authenticated) > web interface to control these reservations? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-2774) SIGSEGV received during process::MessageEncoder::encode()
[
https://issues.apache.org/jira/browse/MESOS-2774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14599149#comment-14599149
]
Klaus Ma commented on MESOS-2774:
-
Where can I download the core file?
> SIGSEGV received during process::MessageEncoder::encode()
> -
>
> Key: MESOS-2774
> URL: https://issues.apache.org/jira/browse/MESOS-2774
> Project: Mesos
> Issue Type: Bug
>Affects Versions: 0.23.0
>Reporter: Yan Xu
>Priority: Critical
>
> Observed in production.
> {noformat:title=}
> Program terminated with signal 11, Segmentation fault.
> #0 0x7fd6ded54b2e in _itoa_word () from /lib64/libc.so.6
> (gdb) bt
> #0 0x7fd6ded54b2e in _itoa_word () from /lib64/libc.so.6
> #1 0x7fd6ded57f13 in vfprintf () from /lib64/libc.so.6
> #2 0x7fd6ded75b49 in vsprintf () from /lib64/libc.so.6
> #3 0x7fd6ded5fe38 in sprintf () from /lib64/libc.so.6
> #4 0x7fd6dedf473e in inet_ntop () from /lib64/libc.so.6
> #5 0x7fd6e058aed6 in operator<< (stream=..., pid=Unhandled dwarf
> expression opcode 0xf3
> ) at ./3rdparty/stout/include/stout/ip.hpp:225
> #6 operator<< (stream=..., pid=Unhandled dwarf expression opcode 0xf3
> ) at ./include/process/address.hpp:106
> #7 process::operator<< (stream=..., pid=Unhandled dwarf expression opcode
> 0xf3
> ) at src/pid.cpp:65
> #8 0x7fd6e05ab29a in process::MessageEncoder::encode
> (message=0x7fd6b80e2d70) at src/encoder.hpp:123
> #9 0x7fd6e05ad1b5 in process::MessageEncoder::MessageEncoder
> (this=0x7fd6b813aed0, s=..., _message=0x7fd6b80e2d70) at src/encoder.hpp:98
> #10 0x7fd6e05a60cc in process::SocketManager::send (this=0x124d100,
> message=0x7fd6b80e2d70) at src/process.cpp:1583
> #11 0x7fd6e05a6750 in process::ProcessBase::send (this=0x7fd6c00097d0,
> to=..., name="PONG", data=0x0, length=0) at src/process.cpp:2679
> #12 0x7fd6e0155f87 in mesos::internal::slave::Slave::pingOld
> (this=0x7fd6c0008f20, from=..., body=Unhandled dwarf expression opcode 0xf3
> ) at slave/slave.cpp:2896
> #13 0x7fd6e05a68cb in operator() (this=0x7fd6c00097d0, event=...) at
> /opt/rh/devtoolset-2/root/usr/include/c++/4.8.2/functional:2464
> #14 process::ProcessBase::visit (this=0x7fd6c00097d0, event=...) at
> src/process.cpp:2688
> #15 0x7fd6e019120d in
> ProtobufProcess::visit (this=0x7fd6c0008f20,
> event=...) at ../3rdparty/libprocess/include/process/protobuf.hpp:94
> #16 0x7fd6e05a009a in process::ProcessManager::resume (this=0x124d3b0,
> process=0x7fd6c00097d0) at src/process.cpp:2175
> #17 0x7fd6e05a035c in process::schedule (arg=Unhandled dwarf expression
> opcode 0xf3
> ) at src/process.cpp:613
> #18 0x7fd6df5f583d in start_thread () from /lib64/libpthread.so.0
> #19 0x7fd6dede7fcd in clone () from /lib64/libc.so.6
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-2664) Modernize the codebase to C++11
[ https://issues.apache.org/jira/browse/MESOS-2664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14599114#comment-14599114 ] Adam B commented on MESOS-2664: --- Retargeting this epic to 0.24, since we won't complete all of the issues in time for 0.23. The issues that have been resolved will be listed individually in the CHANGELOG, but we'll have to wait at least one more release before we can claim to have "modernized the codebase to C++11". Great work so far though! > Modernize the codebase to C++11 > --- > > Key: MESOS-2664 > URL: https://issues.apache.org/jira/browse/MESOS-2664 > Project: Mesos > Issue Type: Epic > Components: technical debt >Reporter: Michael Park >Assignee: Michael Park > Labels: mesosphere > > Since [this > commit|https://github.com/apache/mesos/commit/0f5c78fad3423181f7227027eb42d162811514e7], > we officially require GCC-4.8+ and Clang-3.5+. This means that we now have > full C++11 support and therefore can start to modernize our codebase to be > more readable, safer and efficient! -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-2664) Modernize the codebase to C++11
[ https://issues.apache.org/jira/browse/MESOS-2664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adam B updated MESOS-2664: -- Target Version/s: 0.24.0 (was: 0.23.0) > Modernize the codebase to C++11 > --- > > Key: MESOS-2664 > URL: https://issues.apache.org/jira/browse/MESOS-2664 > Project: Mesos > Issue Type: Epic > Components: technical debt >Reporter: Michael Park >Assignee: Michael Park > Labels: mesosphere > > Since [this > commit|https://github.com/apache/mesos/commit/0f5c78fad3423181f7227027eb42d162811514e7], > we officially require GCC-4.8+ and Clang-3.5+. This means that we now have > full C++11 support and therefore can start to modernize our codebase to be > more readable, safer and efficient! -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-2670) Update existing lambdas to meet style guide
[ https://issues.apache.org/jira/browse/MESOS-2670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14599107#comment-14599107 ] Adam B commented on MESOS-2670: --- [~jvanremoortere] [~benjaminhindman] Now that the above patches have been committed, is there anything left to do before we can close this ticket? > Update existing lambdas to meet style guide > --- > > Key: MESOS-2670 > URL: https://issues.apache.org/jira/browse/MESOS-2670 > Project: Mesos > Issue Type: Task >Reporter: Joris Van Remoortere >Assignee: haosdent > Labels: c++11 > > There are already some lambdas in C++11 specific files. Modify these to meet > the updated style guide. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-2757) Can't Use -> operator with Option
[ https://issues.apache.org/jira/browse/MESOS-2757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14599102#comment-14599102 ] Adam B commented on MESOS-2757: --- [~jvanremoortere] This review was discarded (no reason given). Could you please explain why and update this ticket if it's no longer "reviewable"? Is it still an issue? > Can't Use -> operator with Option > > > Key: MESOS-2757 > URL: https://issues.apache.org/jira/browse/MESOS-2757 > Project: Mesos > Issue Type: Improvement > Components: stout >Reporter: Joris Van Remoortere >Assignee: Joris Van Remoortere > Labels: c++11, mesosphere, option, stout > > Let's add operator overloads to Option to allow access to the underlying T > using the `->` operator. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-1832) Slave should accept PingSlaveMessage but not "PING" message.
[ https://issues.apache.org/jira/browse/MESOS-1832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adam B updated MESOS-1832: -- Target Version/s: 0.24.0 (was: 0.23.0) > Slave should accept PingSlaveMessage but not "PING" message. > > > Key: MESOS-1832 > URL: https://issues.apache.org/jira/browse/MESOS-1832 > Project: Mesos > Issue Type: Task >Reporter: Vinod Kone >Assignee: Adam B > Labels: mesosphere > > Slave handles both "PING" message and PingSlaveMessage in until 0.22.0 for > backwards compatibility (https://reviews.apache.org/r/25867/). > In 0.23.0, slave no longer needs handle "PING". -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-1831) Master should send PingSlaveMessage instead of "PING"
[ https://issues.apache.org/jira/browse/MESOS-1831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adam B updated MESOS-1831: -- Target Version/s: 0.24.0 (was: 0.23.0) > Master should send PingSlaveMessage instead of "PING" > - > > Key: MESOS-1831 > URL: https://issues.apache.org/jira/browse/MESOS-1831 > Project: Mesos > Issue Type: Task >Reporter: Vinod Kone >Assignee: Adam B > Labels: mesosphere > > In 0.21.0 master sends "PING" message with an embedded PingSlaveMessage for > backwards compatibility (https://reviews.apache.org/r/25867/). > In 0.22.0, master should send PingSlaveMessage directly instead of "PING". -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (MESOS-2894) web UI shows "YYYY" for year instead of year
[
https://issues.apache.org/jira/browse/MESOS-2894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Adam B resolved MESOS-2894.
---
Resolution: Fixed
Fix Version/s: 0.23.0
commit d86341b216dc9bb0f97663007c04d147b663f427
Author: haosdent huang
Date: Wed Jun 24 01:29:57 2015 -0700
Fixed web UI showing "" for year instead of year.
Review: https://reviews.apache.org/r/35782
> web UI shows "" for year instead of year
>
>
> Key: MESOS-2894
> URL: https://issues.apache.org/jira/browse/MESOS-2894
> Project: Mesos
> Issue Type: Bug
> Components: webui
>Reporter: brian wickman
>Assignee: haosdent
>Priority: Minor
> Labels: newbie
> Fix For: 0.23.0
>
> Attachments: MESOS-2894-after.png, MESOS-2894-before.png
>
>
> The directory listing formats on when it should probably format on :
> {noformat}
> drwxr-xr-x3 ads twitter 4 KBJun 18 15:27 .pex
> drwxr-xr-x2 ads twitter 4 KBDec 01 aurora
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-1807) Disallow executors with cpu only or memory only resources
[ https://issues.apache.org/jira/browse/MESOS-1807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14599058#comment-14599058 ] Adam B commented on MESOS-1807: --- Thanks for the explanation, Vinod. I'm targeting this out of 0.23 since nobody is actively working on it. > Disallow executors with cpu only or memory only resources > - > > Key: MESOS-1807 > URL: https://issues.apache.org/jira/browse/MESOS-1807 > Project: Mesos > Issue Type: Improvement >Reporter: Vinod Kone > Labels: newbie > > Currently master allows executors to be launched with either only cpus or > only memory but we shouldn't allow that. > This is because executor is an actual unix process that is launched by the > slave. If an executor doesn't specify cpus, what should do the cpu limits be > for that executor when there are no tasks running on it? If no cpu limits are > set then it might starve other executors/tasks on the slave violating > isolation guarantees. Same goes with memory. Moreover, the current > containerizer/isolator code will throw failures when using such an executor, > e.g., when the last task on the executor finishes and Containerizer::update() > is called with 0 cpus or 0 mem. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-1807) Disallow executors with cpu only or memory only resources
[ https://issues.apache.org/jira/browse/MESOS-1807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adam B updated MESOS-1807: -- Target Version/s: 0.24.0 (was: 0.23.0) > Disallow executors with cpu only or memory only resources > - > > Key: MESOS-1807 > URL: https://issues.apache.org/jira/browse/MESOS-1807 > Project: Mesos > Issue Type: Improvement >Reporter: Vinod Kone > Labels: newbie > > Currently master allows executors to be launched with either only cpus or > only memory but we shouldn't allow that. > This is because executor is an actual unix process that is launched by the > slave. If an executor doesn't specify cpus, what should do the cpu limits be > for that executor when there are no tasks running on it? If no cpu limits are > set then it might starve other executors/tasks on the slave violating > isolation guarantees. Same goes with memory. Moreover, the current > containerizer/isolator code will throw failures when using such an executor, > e.g., when the last task on the executor finishes and Containerizer::update() > is called with 0 cpus or 0 mem. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-2572) Add memory statistics tests.
[ https://issues.apache.org/jira/browse/MESOS-2572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14599051#comment-14599051 ] Adam B commented on MESOS-2572: --- [~jieyu] [~idownes] Do either of you have time to review these patches? Looks like [~chzhcn] updated them recently after Ian's latest reviews. They've been sitting around for a while now, and they won't make it into 0.23 without some attention. > Add memory statistics tests. > > > Key: MESOS-2572 > URL: https://issues.apache.org/jira/browse/MESOS-2572 > Project: Mesos > Issue Type: Task >Reporter: Chi Zhang >Assignee: Chi Zhang > Labels: twitter > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-2154) Port CFS support to Docker Containerizer
[ https://issues.apache.org/jira/browse/MESOS-2154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14599049#comment-14599049 ] Adam B commented on MESOS-2154: --- Pushing this out to 0.24, since nobody seems to be working on it. > Port CFS support to Docker Containerizer > > > Key: MESOS-2154 > URL: https://issues.apache.org/jira/browse/MESOS-2154 > Project: Mesos > Issue Type: Improvement > Components: docker, isolation >Affects Versions: 0.21.0 > Environment: Linux (Ubuntu 14.04.1) >Reporter: Andrew Ortman >Priority: Minor > > Port the CFS support the Mesos Containerizer has to the Docker Containerizer. > Whenever the --cgroup_enable_cfs flag is set, the Docker Containerizer should > update the cfs_period_us and cfs_quota_us values to allow hard CPU capping on > the container. > Current workaround is to pass those values as LXC configuration parameters -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-2154) Port CFS support to Docker Containerizer
[ https://issues.apache.org/jira/browse/MESOS-2154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adam B updated MESOS-2154: -- Target Version/s: 0.24.0 (was: 0.23.0) > Port CFS support to Docker Containerizer > > > Key: MESOS-2154 > URL: https://issues.apache.org/jira/browse/MESOS-2154 > Project: Mesos > Issue Type: Improvement > Components: docker, isolation >Affects Versions: 0.21.0 > Environment: Linux (Ubuntu 14.04.1) >Reporter: Andrew Ortman >Priority: Minor > > Port the CFS support the Mesos Containerizer has to the Docker Containerizer. > Whenever the --cgroup_enable_cfs flag is set, the Docker Containerizer should > update the cfs_period_us and cfs_quota_us values to allow hard CPU capping on > the container. > Current workaround is to pass those values as LXC configuration parameters -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-2632) Remove capture by reference of temporaries in mesos
[ https://issues.apache.org/jira/browse/MESOS-2632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14599045#comment-14599045 ] Adam B commented on MESOS-2632: --- [~jvanremoortere] Do you think we'll get this into 0.23, or should we push it out to the next release? > Remove capture by reference of temporaries in mesos > --- > > Key: MESOS-2632 > URL: https://issues.apache.org/jira/browse/MESOS-2632 > Project: Mesos > Issue Type: Task > Components: technical debt >Reporter: Joris Van Remoortere > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MESOS-2894) web UI shows "YYYY" for year instead of year
[
https://issues.apache.org/jira/browse/MESOS-2894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
haosdent updated MESOS-2894:
Attachment: MESOS-2894-after.png
MESOS-2894-before.png
> web UI shows "" for year instead of year
>
>
> Key: MESOS-2894
> URL: https://issues.apache.org/jira/browse/MESOS-2894
> Project: Mesos
> Issue Type: Bug
> Components: webui
>Reporter: brian wickman
>Assignee: haosdent
>Priority: Minor
> Labels: newbie
> Attachments: MESOS-2894-after.png, MESOS-2894-before.png
>
>
> The directory listing formats on when it should probably format on :
> {noformat}
> drwxr-xr-x3 ads twitter 4 KBJun 18 15:27 .pex
> drwxr-xr-x2 ads twitter 4 KBDec 01 aurora
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MESOS-2894) web UI shows "YYYY" for year instead of year
[
https://issues.apache.org/jira/browse/MESOS-2894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Adam B updated MESOS-2894:
--
Assignee: haosdent
> web UI shows "" for year instead of year
>
>
> Key: MESOS-2894
> URL: https://issues.apache.org/jira/browse/MESOS-2894
> Project: Mesos
> Issue Type: Bug
> Components: webui
>Reporter: brian wickman
>Assignee: haosdent
>Priority: Minor
> Labels: newbie
>
> The directory listing formats on when it should probably format on :
> {noformat}
> drwxr-xr-x3 ads twitter 4 KBJun 18 15:27 .pex
> drwxr-xr-x2 ads twitter 4 KBDec 01 aurora
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-2894) web UI shows "YYYY" for year instead of year
[
https://issues.apache.org/jira/browse/MESOS-2894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14598994#comment-14598994
]
Adam B commented on MESOS-2894:
---
Review: https://reviews.apache.org/r/35782
> web UI shows "" for year instead of year
>
>
> Key: MESOS-2894
> URL: https://issues.apache.org/jira/browse/MESOS-2894
> Project: Mesos
> Issue Type: Bug
> Components: webui
>Reporter: brian wickman
>Assignee: haosdent
>Priority: Minor
> Labels: newbie
>
> The directory listing formats on when it should probably format on :
> {noformat}
> drwxr-xr-x3 ads twitter 4 KBJun 18 15:27 .pex
> drwxr-xr-x2 ads twitter 4 KBDec 01 aurora
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
