[jira] [Commented] (MESOS-898) Introduce CMake as an alternative build system.
[ https://issues.apache.org/jira/browse/MESOS-898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17149785#comment-17149785 ] Andrew Schwartzmeyer commented on MESOS-898: [~asekretenko] > Introduce CMake as an alternative build system. > --- > > Key: MESOS-898 > URL: https://issues.apache.org/jira/browse/MESOS-898 > Project: Mesos > Issue Type: Epic > Components: build >Reporter: Timothy St. Clair >Assignee: Joseph Wu >Priority: Major > Labels: build, cmake, mesosphere > > This is a rather substantial undertaking, so I would want upstream > debate+buy-in prior to full commitment. The basic premise is: upstream > rebundles several of its dependencies in part to tightly control its stack. > This is not out of the norm, but in order to be picked up by distribution > channels it needs to built against system dependencies, and rebundling is > strictly forbidden. Given that the mesos primary target platform are > data-center distributions such as RHEL/CENTOS/SL it makes sense to still have > bundling support for those who do not have dependencies in their channels > "yet". This is where cmake can be win with it's uber macros > (http://www.cmake.org/cmake/help/v2.8.8/cmake.html#module:ExternalProject). > I do not know of any equivalent in the autotools world, other then to brew > your own solution. I've done this type of work in the past, and completely > transformed condor and would leverage a lot of the work that was done there. > I currently have a tracking branch where I've started this work, but before I > go off into the woods, it makes sense to have a debate in public. > The primary benefits are: > 1. Enable downstream channels to easily distro without carrying a large patch > sets. > 2. Still support existing "non-proper" distribution methods. > 3. Harden / future proof dependent interfaces. > Side Benefits: > Audit current build mechanics. > - Presently the language specific binding are not installed. (.py & .jar) > - make -jX currently fails > - optionally look in arm support. > Costs: > 1. Time > 2. Potential temporary destabilization > 3. Infrastructure around build+test may need to change. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (MESOS-898) Introduce CMake as an alternative build system.
[ https://issues.apache.org/jira/browse/MESOS-898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17149461#comment-17149461 ] Andrei Sekretenko commented on MESOS-898: - Now we have an installable cmake build that can be used in production. I'm closing this epic as Resolved, finally. Moved the issues needed to achieve full feature-parity with cmake into a new epic https://issues.apache.org/jira/browse/MESOS-10145 and other, non-essential ones into an old epic on cmake improvements: https://issues.apache.org/jira/browse/MESOS-7866 > Introduce CMake as an alternative build system. > --- > > Key: MESOS-898 > URL: https://issues.apache.org/jira/browse/MESOS-898 > Project: Mesos > Issue Type: Epic > Components: build >Reporter: Timothy St. Clair >Assignee: Joseph Wu >Priority: Major > Labels: build, cmake, mesosphere > > This is a rather substantial undertaking, so I would want upstream > debate+buy-in prior to full commitment. The basic premise is: upstream > rebundles several of its dependencies in part to tightly control its stack. > This is not out of the norm, but in order to be picked up by distribution > channels it needs to built against system dependencies, and rebundling is > strictly forbidden. Given that the mesos primary target platform are > data-center distributions such as RHEL/CENTOS/SL it makes sense to still have > bundling support for those who do not have dependencies in their channels > "yet". This is where cmake can be win with it's uber macros > (http://www.cmake.org/cmake/help/v2.8.8/cmake.html#module:ExternalProject). > I do not know of any equivalent in the autotools world, other then to brew > your own solution. I've done this type of work in the past, and completely > transformed condor and would leverage a lot of the work that was done there. > I currently have a tracking branch where I've started this work, but before I > go off into the woods, it makes sense to have a debate in public. > The primary benefits are: > 1. Enable downstream channels to easily distro without carrying a large patch > sets. > 2. Still support existing "non-proper" distribution methods. > 3. Harden / future proof dependent interfaces. > Side Benefits: > Audit current build mechanics. > - Presently the language specific binding are not installed. (.py & .jar) > - make -jX currently fails > - optionally look in arm support. > Costs: > 1. Time > 2. Potential temporary destabilization > 3. Infrastructure around build+test may need to change. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (MESOS-898) Introduce CMake as an alternative build system.
[ https://issues.apache.org/jira/browse/MESOS-898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14632686#comment-14632686 ] Alex Clemmer commented on MESOS-898: [~vinodkone]: Could I get you to double-check my implementation of the .so versioning semantics? You can see on line #25 of this file[1], I set the current SOVERSION to 0. I chose this because of this line[2] in `configure.ac`, but the truth is I don't know AutoConf, and I am only guessing that this is where the SOVERSION is set. If this is wrong, feel free to ping this thread, or add some comments on the review here[3]. [1] https://reviews.apache.org/r/36513/diff/1?file=1012823#file1012823line25 [2] https://github.com/hausdorff/mesos/blob/cmake_rb_wip1/configure.ac#L21 [3] https://reviews.apache.org/r/36513/ Introduce CMake as an alternative build system. --- Key: MESOS-898 URL: https://issues.apache.org/jira/browse/MESOS-898 Project: Mesos Issue Type: Epic Components: build Reporter: Timothy St. Clair Assignee: Alex Clemmer Labels: build This is a rather substantial undertaking, so I would want upstream debate+buy-in prior to full commitment. The basic premise is: upstream rebundles several of its dependencies in part to tightly control its stack. This is not out of the norm, but in order to be picked up by distribution channels it needs to built against system dependencies, and rebundling is strictly forbidden. Given that the mesos primary target platform are data-center distributions such as RHEL/CENTOS/SL it makes sense to still have bundling support for those who do not have dependencies in their channels yet. This is where cmake can be win with it's uber macros (http://www.cmake.org/cmake/help/v2.8.8/cmake.html#module:ExternalProject). I do not know of any equivalent in the autotools world, other then to brew your own solution. I've done this type of work in the past, and completely transformed condor and would leverage a lot of the work that was done there. I currently have a tracking branch where I've started this work, but before I go off into the woods, it makes sense to have a debate in public. The primary benefits are: 1. Enable downstream channels to easily distro without carrying a large patch sets. 2. Still support existing non-proper distribution methods. 3. Harden / future proof dependent interfaces. Side Benefits: Audit current build mechanics. - Presently the language specific binding are not installed. (.py .jar) - make -jX currently fails - optionally look in arm support. Costs: 1. Time 2. Potential temporary destabilization 3. Infrastructure around build+test may need to change. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-898) Introduce CMake as an alternative build system.
[ https://issues.apache.org/jira/browse/MESOS-898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14628493#comment-14628493 ] Alex Clemmer commented on MESOS-898: Hey folks. [~tstclair] gave some useful feedback, and I posted 2 commits to RB. Happy to discuss it here or there. The reviews are at: (1) https://reviews.apache.org/r/36514/ (2) https://reviews.apache.org/r/36514/ Introduce CMake as an alternative build system. --- Key: MESOS-898 URL: https://issues.apache.org/jira/browse/MESOS-898 Project: Mesos Issue Type: Epic Components: build Reporter: Timothy St. Clair Assignee: Alex Clemmer Labels: build This is a rather substantial undertaking, so I would want upstream debate+buy-in prior to full commitment. The basic premise is: upstream rebundles several of its dependencies in part to tightly control its stack. This is not out of the norm, but in order to be picked up by distribution channels it needs to built against system dependencies, and rebundling is strictly forbidden. Given that the mesos primary target platform are data-center distributions such as RHEL/CENTOS/SL it makes sense to still have bundling support for those who do not have dependencies in their channels yet. This is where cmake can be win with it's uber macros (http://www.cmake.org/cmake/help/v2.8.8/cmake.html#module:ExternalProject). I do not know of any equivalent in the autotools world, other then to brew your own solution. I've done this type of work in the past, and completely transformed condor and would leverage a lot of the work that was done there. I currently have a tracking branch where I've started this work, but before I go off into the woods, it makes sense to have a debate in public. The primary benefits are: 1. Enable downstream channels to easily distro without carrying a large patch sets. 2. Still support existing non-proper distribution methods. 3. Harden / future proof dependent interfaces. Side Benefits: Audit current build mechanics. - Presently the language specific binding are not installed. (.py .jar) - make -jX currently fails - optionally look in arm support. Costs: 1. Time 2. Potential temporary destabilization 3. Infrastructure around build+test may need to change. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-898) Introduce CMake as an alternative build system.
[ https://issues.apache.org/jira/browse/MESOS-898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14625301#comment-14625301 ] Alex Clemmer commented on MESOS-898: [~vinodkone]: Yep, per [~tstclair]'s request, we are working through things in GitHub before we put it into the RB. He took a week for vacation, and has subsequently been very busy, which is why this issue resolution time has been somewhat protracted -- but, considering Tim's expertise, I think it's definitely worth taking a bit extra time (an a somewhat non-traditional review process) to make sure he's good with what's going on here. The current state is the first commit in this branch, if you'd like to have a look: https://github.com/hausdorff/mesos/commits/cmake_rb (Do note that I rebase this consistently to keep all the little changes in one review, so that it's easy to comment on and stuff.) I will post here when we're ready to push to RB. Sorry again that this is taking longer than expected. Introduce CMake as an alternative build system. --- Key: MESOS-898 URL: https://issues.apache.org/jira/browse/MESOS-898 Project: Mesos Issue Type: Epic Components: build Reporter: Timothy St. Clair Assignee: Alex Clemmer Labels: build This is a rather substantial undertaking, so I would want upstream debate+buy-in prior to full commitment. The basic premise is: upstream rebundles several of its dependencies in part to tightly control its stack. This is not out of the norm, but in order to be picked up by distribution channels it needs to built against system dependencies, and rebundling is strictly forbidden. Given that the mesos primary target platform are data-center distributions such as RHEL/CENTOS/SL it makes sense to still have bundling support for those who do not have dependencies in their channels yet. This is where cmake can be win with it's uber macros (http://www.cmake.org/cmake/help/v2.8.8/cmake.html#module:ExternalProject). I do not know of any equivalent in the autotools world, other then to brew your own solution. I've done this type of work in the past, and completely transformed condor and would leverage a lot of the work that was done there. I currently have a tracking branch where I've started this work, but before I go off into the woods, it makes sense to have a debate in public. The primary benefits are: 1. Enable downstream channels to easily distro without carrying a large patch sets. 2. Still support existing non-proper distribution methods. 3. Harden / future proof dependent interfaces. Side Benefits: Audit current build mechanics. - Presently the language specific binding are not installed. (.py .jar) - make -jX currently fails - optionally look in arm support. Costs: 1. Time 2. Potential temporary destabilization 3. Infrastructure around build+test may need to change. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-898) Introduce CMake as an alternative build system.
[ https://issues.apache.org/jira/browse/MESOS-898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14621456#comment-14621456 ] Vinod Kone commented on MESOS-898: -- Any updates on this [~hausdorff]? Introduce CMake as an alternative build system. --- Key: MESOS-898 URL: https://issues.apache.org/jira/browse/MESOS-898 Project: Mesos Issue Type: Epic Components: build Reporter: Timothy St. Clair Assignee: Alex Clemmer Labels: build This is a rather substantial undertaking, so I would want upstream debate+buy-in prior to full commitment. The basic premise is: upstream rebundles several of its dependencies in part to tightly control its stack. This is not out of the norm, but in order to be picked up by distribution channels it needs to built against system dependencies, and rebundling is strictly forbidden. Given that the mesos primary target platform are data-center distributions such as RHEL/CENTOS/SL it makes sense to still have bundling support for those who do not have dependencies in their channels yet. This is where cmake can be win with it's uber macros (http://www.cmake.org/cmake/help/v2.8.8/cmake.html#module:ExternalProject). I do not know of any equivalent in the autotools world, other then to brew your own solution. I've done this type of work in the past, and completely transformed condor and would leverage a lot of the work that was done there. I currently have a tracking branch where I've started this work, but before I go off into the woods, it makes sense to have a debate in public. The primary benefits are: 1. Enable downstream channels to easily distro without carrying a large patch sets. 2. Still support existing non-proper distribution methods. 3. Harden / future proof dependent interfaces. Side Benefits: Audit current build mechanics. - Presently the language specific binding are not installed. (.py .jar) - make -jX currently fails - optionally look in arm support. Costs: 1. Time 2. Potential temporary destabilization 3. Infrastructure around build+test may need to change. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-898) Introduce CMake as an alternative build system.
[ https://issues.apache.org/jira/browse/MESOS-898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14601522#comment-14601522 ] Timothy St. Clair commented on MESOS-898: - [~hausdorff] comments made, feel free to ping me on irc later today on the topic. Introduce CMake as an alternative build system. --- Key: MESOS-898 URL: https://issues.apache.org/jira/browse/MESOS-898 Project: Mesos Issue Type: Epic Components: build Reporter: Timothy St. Clair Assignee: Alex Clemmer Labels: build This is a rather substantial undertaking, so I would want upstream debate+buy-in prior to full commitment. The basic premise is: upstream rebundles several of its dependencies in part to tightly control its stack. This is not out of the norm, but in order to be picked up by distribution channels it needs to built against system dependencies, and rebundling is strictly forbidden. Given that the mesos primary target platform are data-center distributions such as RHEL/CENTOS/SL it makes sense to still have bundling support for those who do not have dependencies in their channels yet. This is where cmake can be win with it's uber macros (http://www.cmake.org/cmake/help/v2.8.8/cmake.html#module:ExternalProject). I do not know of any equivalent in the autotools world, other then to brew your own solution. I've done this type of work in the past, and completely transformed condor and would leverage a lot of the work that was done there. I currently have a tracking branch where I've started this work, but before I go off into the woods, it makes sense to have a debate in public. The primary benefits are: 1. Enable downstream channels to easily distro without carrying a large patch sets. 2. Still support existing non-proper distribution methods. 3. Harden / future proof dependent interfaces. Side Benefits: Audit current build mechanics. - Presently the language specific binding are not installed. (.py .jar) - make -jX currently fails - optionally look in arm support. Costs: 1. Time 2. Potential temporary destabilization 3. Infrastructure around build+test may need to change. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-898) Introduce CMake as an alternative build system.
[
https://issues.apache.org/jira/browse/MESOS-898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14600254#comment-14600254
]
Alex Clemmer commented on MESOS-898:
[~vinodkone]:
{quote}
-- Looks like you are adding windows support too (I'm guessing that's your
main motivation to work on this? Stoked to see this btw!). It's probably better
to work on WIN support in a separate patch.
{quote}
First, yes, I'm attempting to contribute Windows (and Windows container)
support to the Mesos project, and yes, this is the first big step. But, please
don't spread it around to too many people just yet -- to the extent possible, I
am hoping to avoid impacting Mesosphere's ability to effectively launch the
feature. :)
To your suggestion that we split Windows support into a different patch: it
seems like you're saying it might be better to first contribute a baseline
CMake-based build system, and then extend that to support Windows later. (That
is, I assume you're not talking about how to package up the changes to the C++
code that we will need in order to support Windows, which I assume we all agree
belong in a different patch.) Is that all right?
I'm actually hoping to make x-plat support out of the box a core goal of this
new build system. My rationale is basically twofold: (1) I am basically certain
that I will end up developing a very different build system if we don't make
x-plat support a priority right out of the gate (right now, I'm testing that
each iteration works on both Linux and Windows, which definitely guides the
design a lot), and (2) I've taken care to clearly mark off the Windows-specific
stuff so that we can pull it out easily later if this Windows stuff doesn't
work out. More particularly, I am suggesting this course of action because I
think it will result in the least aggregate work in expectation.
That all said, I understand the hesitation to put stuff into the codebase if it
isn't used, and I'm totally willing to be convinced that I'm wrong. Let me know
if you see something I don't.
To your other issues, that boost issue that you and [~xujyan] is good to know,
so thanks for pointing it out. And, I was actually hoping that one of you had
good knowledge of CMake idioms. :) If we don't have someone with that expertise
on deck, I agree we need to go find it before merge.
Introduce CMake as an alternative build system.
---
Key: MESOS-898
URL: https://issues.apache.org/jira/browse/MESOS-898
Project: Mesos
Issue Type: Epic
Components: build
Reporter: Timothy St. Clair
Assignee: Alex Clemmer
Labels: build
This is a rather substantial undertaking, so I would want upstream
debate+buy-in prior to full commitment. The basic premise is: upstream
rebundles several of its dependencies in part to tightly control its stack.
This is not out of the norm, but in order to be picked up by distribution
channels it needs to built against system dependencies, and rebundling is
strictly forbidden. Given that the mesos primary target platform are
data-center distributions such as RHEL/CENTOS/SL it makes sense to still have
bundling support for those who do not have dependencies in their channels
yet. This is where cmake can be win with it's uber macros
(http://www.cmake.org/cmake/help/v2.8.8/cmake.html#module:ExternalProject).
I do not know of any equivalent in the autotools world, other then to brew
your own solution. I've done this type of work in the past, and completely
transformed condor and would leverage a lot of the work that was done there.
I currently have a tracking branch where I've started this work, but before I
go off into the woods, it makes sense to have a debate in public.
The primary benefits are:
1. Enable downstream channels to easily distro without carrying a large patch
sets.
2. Still support existing non-proper distribution methods.
3. Harden / future proof dependent interfaces.
Side Benefits:
Audit current build mechanics.
- Presently the language specific binding are not installed. (.py .jar)
- make -jX currently fails
- optionally look in arm support.
Costs:
1. Time
2. Potential temporary destabilization
3. Infrastructure around build+test may need to change.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-898) Introduce CMake as an alternative build system.
[ https://issues.apache.org/jira/browse/MESOS-898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14600263#comment-14600263 ] Cody Maloney commented on MESOS-898: I would suggest that with the move to CMake we switch to using a raw upstream packaged version of boost. There isn't a lot we gain by stripping out some of the headers, and it adds a lot more complexity. CMake has a lot of stuff ready-made for finding, downloading boost if and only if it isn't present on the host machine, isn't of the right version, etc. Forcing rebuilding all of that logic/code so that we can remove some files in a tarball which shouldn't be embedded inside the repository anyways seems like not the best idea. Introduce CMake as an alternative build system. --- Key: MESOS-898 URL: https://issues.apache.org/jira/browse/MESOS-898 Project: Mesos Issue Type: Epic Components: build Reporter: Timothy St. Clair Assignee: Alex Clemmer Labels: build This is a rather substantial undertaking, so I would want upstream debate+buy-in prior to full commitment. The basic premise is: upstream rebundles several of its dependencies in part to tightly control its stack. This is not out of the norm, but in order to be picked up by distribution channels it needs to built against system dependencies, and rebundling is strictly forbidden. Given that the mesos primary target platform are data-center distributions such as RHEL/CENTOS/SL it makes sense to still have bundling support for those who do not have dependencies in their channels yet. This is where cmake can be win with it's uber macros (http://www.cmake.org/cmake/help/v2.8.8/cmake.html#module:ExternalProject). I do not know of any equivalent in the autotools world, other then to brew your own solution. I've done this type of work in the past, and completely transformed condor and would leverage a lot of the work that was done there. I currently have a tracking branch where I've started this work, but before I go off into the woods, it makes sense to have a debate in public. The primary benefits are: 1. Enable downstream channels to easily distro without carrying a large patch sets. 2. Still support existing non-proper distribution methods. 3. Harden / future proof dependent interfaces. Side Benefits: Audit current build mechanics. - Presently the language specific binding are not installed. (.py .jar) - make -jX currently fails - optionally look in arm support. Costs: 1. Time 2. Potential temporary destabilization 3. Infrastructure around build+test may need to change. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-898) Introduce CMake as an alternative build system.
[ https://issues.apache.org/jira/browse/MESOS-898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14600293#comment-14600293 ] Alex Clemmer commented on MESOS-898: [~vi...@twitter.com] I would be very, very disappointed in any Apache project that did not have community involvement in a decision like this. Introduce CMake as an alternative build system. --- Key: MESOS-898 URL: https://issues.apache.org/jira/browse/MESOS-898 Project: Mesos Issue Type: Epic Components: build Reporter: Timothy St. Clair Assignee: Alex Clemmer Labels: build This is a rather substantial undertaking, so I would want upstream debate+buy-in prior to full commitment. The basic premise is: upstream rebundles several of its dependencies in part to tightly control its stack. This is not out of the norm, but in order to be picked up by distribution channels it needs to built against system dependencies, and rebundling is strictly forbidden. Given that the mesos primary target platform are data-center distributions such as RHEL/CENTOS/SL it makes sense to still have bundling support for those who do not have dependencies in their channels yet. This is where cmake can be win with it's uber macros (http://www.cmake.org/cmake/help/v2.8.8/cmake.html#module:ExternalProject). I do not know of any equivalent in the autotools world, other then to brew your own solution. I've done this type of work in the past, and completely transformed condor and would leverage a lot of the work that was done there. I currently have a tracking branch where I've started this work, but before I go off into the woods, it makes sense to have a debate in public. The primary benefits are: 1. Enable downstream channels to easily distro without carrying a large patch sets. 2. Still support existing non-proper distribution methods. 3. Harden / future proof dependent interfaces. Side Benefits: Audit current build mechanics. - Presently the language specific binding are not installed. (.py .jar) - make -jX currently fails - optionally look in arm support. Costs: 1. Time 2. Potential temporary destabilization 3. Infrastructure around build+test may need to change. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-898) Introduce CMake as an alternative build system.
[ https://issues.apache.org/jira/browse/MESOS-898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14600279#comment-14600279 ] Vinod Kone commented on MESOS-898: -- You know that this JIRA is public right :) Also, since this support is being added to Apache Mesos, we expect the community to be in the know during development of changes like these. Regarding cross platform support off the bat, I'll trust your instincts on this. From what I saw in your commits, windows support didn't seem to complicate things. So, it's ok with me. Regarding CMake idioms, [~tstclair] (Mesos committer), might be able to help you out here. Introduce CMake as an alternative build system. --- Key: MESOS-898 URL: https://issues.apache.org/jira/browse/MESOS-898 Project: Mesos Issue Type: Epic Components: build Reporter: Timothy St. Clair Assignee: Alex Clemmer Labels: build This is a rather substantial undertaking, so I would want upstream debate+buy-in prior to full commitment. The basic premise is: upstream rebundles several of its dependencies in part to tightly control its stack. This is not out of the norm, but in order to be picked up by distribution channels it needs to built against system dependencies, and rebundling is strictly forbidden. Given that the mesos primary target platform are data-center distributions such as RHEL/CENTOS/SL it makes sense to still have bundling support for those who do not have dependencies in their channels yet. This is where cmake can be win with it's uber macros (http://www.cmake.org/cmake/help/v2.8.8/cmake.html#module:ExternalProject). I do not know of any equivalent in the autotools world, other then to brew your own solution. I've done this type of work in the past, and completely transformed condor and would leverage a lot of the work that was done there. I currently have a tracking branch where I've started this work, but before I go off into the woods, it makes sense to have a debate in public. The primary benefits are: 1. Enable downstream channels to easily distro without carrying a large patch sets. 2. Still support existing non-proper distribution methods. 3. Harden / future proof dependent interfaces. Side Benefits: Audit current build mechanics. - Presently the language specific binding are not installed. (.py .jar) - make -jX currently fails - optionally look in arm support. Costs: 1. Time 2. Potential temporary destabilization 3. Infrastructure around build+test may need to change. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-898) Introduce CMake as an alternative build system.
[ https://issues.apache.org/jira/browse/MESOS-898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14599706#comment-14599706 ] Alex Clemmer commented on MESOS-898: I'd like to start this conversation with (what I think is) the simplest step forward. I've put together a prototype CMake-based build system that covers libprocess and the third-party libraries, which successfully builds on several flavors of Linux. It is the [last commit of this branch|https://github.com/hausdorff/mesos/commits/test_cmake]. In general, I'm hoping that this will help focus the discussion around concrete things that I need to fix or do less stupidly, to accomplish our goals in this project. Some notes: * I DON'T KNOW CMAKE (OR AUTOCONF), so if it looks like I'm doing something silly, it's because I am! * It's pretty well-commented. I'm hoping it's not too hard to navigate. * It is build to be x-plat at the outset. So, you should be able to take this file and generate makefiles or, like, nmake files. Some things on the roadmap if you all like the progress so far: [x] Third-party dependencies are downloaded, configured, and built when you run `make` (or whatever). [ ] Support for system installations of the third-party dependencies (i.e., we should allow users to use glog if it's already installed on their machine) [x] Tests build and run on multiple flavors of Linux [ ] Benchmarks build and run on multiple flavors of Linux n.b., for issue 2 I have not decided what the right way of communicating the system dependencies to CMake is. In the autoconf solution this comes from `configure`, but CMake will not have a configure step because it needs to run not only on POSIX machines. Introduce CMake as an alternative build system. --- Key: MESOS-898 URL: https://issues.apache.org/jira/browse/MESOS-898 Project: Mesos Issue Type: Epic Components: build Reporter: Timothy St. Clair Assignee: Alex Clemmer Labels: build This is a rather substantial undertaking, so I would want upstream debate+buy-in prior to full commitment. The basic premise is: upstream rebundles several of its dependencies in part to tightly control its stack. This is not out of the norm, but in order to be picked up by distribution channels it needs to built against system dependencies, and rebundling is strictly forbidden. Given that the mesos primary target platform are data-center distributions such as RHEL/CENTOS/SL it makes sense to still have bundling support for those who do not have dependencies in their channels yet. This is where cmake can be win with it's uber macros (http://www.cmake.org/cmake/help/v2.8.8/cmake.html#module:ExternalProject). I do not know of any equivalent in the autotools world, other then to brew your own solution. I've done this type of work in the past, and completely transformed condor and would leverage a lot of the work that was done there. I currently have a tracking branch where I've started this work, but before I go off into the woods, it makes sense to have a debate in public. The primary benefits are: 1. Enable downstream channels to easily distro without carrying a large patch sets. 2. Still support existing non-proper distribution methods. 3. Harden / future proof dependent interfaces. Side Benefits: Audit current build mechanics. - Presently the language specific binding are not installed. (.py .jar) - make -jX currently fails - optionally look in arm support. Costs: 1. Time 2. Potential temporary destabilization 3. Infrastructure around build+test may need to change. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-898) Introduce CMake as an alternative build system.
[ https://issues.apache.org/jira/browse/MESOS-898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14599705#comment-14599705 ] Alex Clemmer commented on MESOS-898: I'd like to start this conversation with (what I think is) the simplest step forward. I've put together a prototype CMake-based build system that covers libprocess and the third-party libraries, which successfully builds on several flavors of Linux. It is the [last commit of this branch|https://github.com/hausdorff/mesos/commits/test_cmake]. In general, I'm hoping that this will help focus the discussion around concrete things that I need to fix or do less stupidly, to accomplish our goals in this project. Some notes: * I DON'T KNOW CMAKE (OR AUTOCONF), so if it looks like I'm doing something silly, it's because I am! * It's pretty well-commented. I'm hoping it's not too hard to navigate. * It is build to be x-plat at the outset. So, you should be able to take this file and generate makefiles or, like, nmake files. Some things on the roadmap if you all like the progress so far: [x] Third-party dependencies are downloaded, configured, and built when you run `make` (or whatever). [ ] Support for system installations of the third-party dependencies (i.e., we should allow users to use glog if it's already installed on their machine) [x] Tests build and run on multiple flavors of Linux [ ] Benchmarks build and run on multiple flavors of Linux n.b., for issue 2 I have not decided what the right way of communicating the system dependencies to CMake is. In the autoconf solution this comes from `configure`, but CMake will not have a configure step because it needs to run not only on POSIX machines. Introduce CMake as an alternative build system. --- Key: MESOS-898 URL: https://issues.apache.org/jira/browse/MESOS-898 Project: Mesos Issue Type: Epic Components: build Reporter: Timothy St. Clair Assignee: Alex Clemmer Labels: build This is a rather substantial undertaking, so I would want upstream debate+buy-in prior to full commitment. The basic premise is: upstream rebundles several of its dependencies in part to tightly control its stack. This is not out of the norm, but in order to be picked up by distribution channels it needs to built against system dependencies, and rebundling is strictly forbidden. Given that the mesos primary target platform are data-center distributions such as RHEL/CENTOS/SL it makes sense to still have bundling support for those who do not have dependencies in their channels yet. This is where cmake can be win with it's uber macros (http://www.cmake.org/cmake/help/v2.8.8/cmake.html#module:ExternalProject). I do not know of any equivalent in the autotools world, other then to brew your own solution. I've done this type of work in the past, and completely transformed condor and would leverage a lot of the work that was done there. I currently have a tracking branch where I've started this work, but before I go off into the woods, it makes sense to have a debate in public. The primary benefits are: 1. Enable downstream channels to easily distro without carrying a large patch sets. 2. Still support existing non-proper distribution methods. 3. Harden / future proof dependent interfaces. Side Benefits: Audit current build mechanics. - Presently the language specific binding are not installed. (.py .jar) - make -jX currently fails - optionally look in arm support. Costs: 1. Time 2. Potential temporary destabilization 3. Infrastructure around build+test may need to change. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-898) Introduce CMake as an alternative build system.
[ https://issues.apache.org/jira/browse/MESOS-898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14599866#comment-14599866 ] Alex Clemmer commented on MESOS-898: Only a few people have given a quick once-over look to the prototype [CMake-based build system for libprocess|https://github.com/hausdorff/mesos/commit/073d42cea1bed8e7fb801f59685f52d3be06e510?diff=unified], but they have been pretty positive, so I would like to target EOD tomorrow (i.e., Th, June 24th) for posting something to ReviewBoard. This slight delay is mainly because I am new to the community, and I want to give people a chance to build consensus that this commit is at least directionally correct. :) Introduce CMake as an alternative build system. --- Key: MESOS-898 URL: https://issues.apache.org/jira/browse/MESOS-898 Project: Mesos Issue Type: Epic Components: build Reporter: Timothy St. Clair Assignee: Alex Clemmer Labels: build This is a rather substantial undertaking, so I would want upstream debate+buy-in prior to full commitment. The basic premise is: upstream rebundles several of its dependencies in part to tightly control its stack. This is not out of the norm, but in order to be picked up by distribution channels it needs to built against system dependencies, and rebundling is strictly forbidden. Given that the mesos primary target platform are data-center distributions such as RHEL/CENTOS/SL it makes sense to still have bundling support for those who do not have dependencies in their channels yet. This is where cmake can be win with it's uber macros (http://www.cmake.org/cmake/help/v2.8.8/cmake.html#module:ExternalProject). I do not know of any equivalent in the autotools world, other then to brew your own solution. I've done this type of work in the past, and completely transformed condor and would leverage a lot of the work that was done there. I currently have a tracking branch where I've started this work, but before I go off into the woods, it makes sense to have a debate in public. The primary benefits are: 1. Enable downstream channels to easily distro without carrying a large patch sets. 2. Still support existing non-proper distribution methods. 3. Harden / future proof dependent interfaces. Side Benefits: Audit current build mechanics. - Presently the language specific binding are not installed. (.py .jar) - make -jX currently fails - optionally look in arm support. Costs: 1. Time 2. Potential temporary destabilization 3. Infrastructure around build+test may need to change. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-898) Introduce CMake as an alternative build system.
[
https://issues.apache.org/jira/browse/MESOS-898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14599955#comment-14599955
]
Yan Xu commented on MESOS-898:
--
{quote}
-- Looks like you are pulling in the whole of boost? While it's not clear, we
bundle the boost ourselves. It basically includes only the boost headers that
we need. There should be a README somewhere that explains the process.
{quote}
Yeah it's super unclear. The README is hidden within the tarball.
https://github.com/apache/mesos/blob/master/3rdparty/libprocess/3rdparty/boost-1.53.0.tar.gz
We should move it out.
Introduce CMake as an alternative build system.
---
Key: MESOS-898
URL: https://issues.apache.org/jira/browse/MESOS-898
Project: Mesos
Issue Type: Epic
Components: build
Reporter: Timothy St. Clair
Assignee: Alex Clemmer
Labels: build
This is a rather substantial undertaking, so I would want upstream
debate+buy-in prior to full commitment. The basic premise is: upstream
rebundles several of its dependencies in part to tightly control its stack.
This is not out of the norm, but in order to be picked up by distribution
channels it needs to built against system dependencies, and rebundling is
strictly forbidden. Given that the mesos primary target platform are
data-center distributions such as RHEL/CENTOS/SL it makes sense to still have
bundling support for those who do not have dependencies in their channels
yet. This is where cmake can be win with it's uber macros
(http://www.cmake.org/cmake/help/v2.8.8/cmake.html#module:ExternalProject).
I do not know of any equivalent in the autotools world, other then to brew
your own solution. I've done this type of work in the past, and completely
transformed condor and would leverage a lot of the work that was done there.
I currently have a tracking branch where I've started this work, but before I
go off into the woods, it makes sense to have a debate in public.
The primary benefits are:
1. Enable downstream channels to easily distro without carrying a large patch
sets.
2. Still support existing non-proper distribution methods.
3. Harden / future proof dependent interfaces.
Side Benefits:
Audit current build mechanics.
- Presently the language specific binding are not installed. (.py .jar)
- make -jX currently fails
- optionally look in arm support.
Costs:
1. Time
2. Potential temporary destabilization
3. Infrastructure around build+test may need to change.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-898) Introduce CMake as an alternative build system.
[
https://issues.apache.org/jira/browse/MESOS-898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14597899#comment-14597899
]
haosdent commented on MESOS-898:
LLVM and KDE move to cmake several years ago. In fact, I try to use
[am2cmake|http://www.cmake.org/Wiki/CMake#automake.2Fautotools.2Fautoconf] to
convert current automake files one month ago. But the convert result is ugly, I
think it's better to convert manually.
I think the largest advantage to use CMake is: CMake could help you [generate
IDE project
files|http://www.cmake.org/cmake/help/v3.0/manual/cmake-generators.7.html].
After add CMake support, we just need execute {code}cmake -G Your IDE
name{code} IDE name could be visual studio, xcode, CodeBlocks, Eclipse,
CodeLite, QtCreator or other support IDE here. And then it could be imported by
you ide.
Introduce CMake as an alternative build system.
---
Key: MESOS-898
URL: https://issues.apache.org/jira/browse/MESOS-898
Project: Mesos
Issue Type: Epic
Components: build
Reporter: Timothy St. Clair
Assignee: Alex Clemmer
Labels: build
This is a rather substantial undertaking, so I would want upstream
debate+buy-in prior to full commitment. The basic premise is: upstream
rebundles several of its dependencies in part to tightly control its stack.
This is not out of the norm, but in order to be picked up by distribution
channels it needs to built against system dependencies, and rebundling is
strictly forbidden. Given that the mesos primary target platform are
data-center distributions such as RHEL/CENTOS/SL it makes sense to still have
bundling support for those who do not have dependencies in their channels
yet. This is where cmake can be win with it's uber macros
(http://www.cmake.org/cmake/help/v2.8.8/cmake.html#module:ExternalProject).
I do not know of any equivalent in the autotools world, other then to brew
your own solution. I've done this type of work in the past, and completely
transformed condor and would leverage a lot of the work that was done there.
I currently have a tracking branch where I've started this work, but before I
go off into the woods, it makes sense to have a debate in public.
The primary benefits are:
1. Enable downstream channels to easily distro without carrying a large patch
sets.
2. Still support existing non-proper distribution methods.
3. Harden / future proof dependent interfaces.
Side Benefits:
Audit current build mechanics.
- Presently the language specific binding are not installed. (.py .jar)
- make -jX currently fails
- optionally look in arm support.
Costs:
1. Time
2. Potential temporary destabilization
3. Infrastructure around build+test may need to change.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-898) Introduce CMake as an alternative build system.
[ https://issues.apache.org/jira/browse/MESOS-898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14597917#comment-14597917 ] Timothy St. Clair commented on MESOS-898: - imho, stout and libprocess should be subsumed by the main code-base for simplification. The notion that they are separable and could be versioned independently, I think has been proven false. Doing this 1st, would greatly simplify any build system changes. Introduce CMake as an alternative build system. --- Key: MESOS-898 URL: https://issues.apache.org/jira/browse/MESOS-898 Project: Mesos Issue Type: Epic Components: build Reporter: Timothy St. Clair Assignee: Alex Clemmer Labels: build This is a rather substantial undertaking, so I would want upstream debate+buy-in prior to full commitment. The basic premise is: upstream rebundles several of its dependencies in part to tightly control its stack. This is not out of the norm, but in order to be picked up by distribution channels it needs to built against system dependencies, and rebundling is strictly forbidden. Given that the mesos primary target platform are data-center distributions such as RHEL/CENTOS/SL it makes sense to still have bundling support for those who do not have dependencies in their channels yet. This is where cmake can be win with it's uber macros (http://www.cmake.org/cmake/help/v2.8.8/cmake.html#module:ExternalProject). I do not know of any equivalent in the autotools world, other then to brew your own solution. I've done this type of work in the past, and completely transformed condor and would leverage a lot of the work that was done there. I currently have a tracking branch where I've started this work, but before I go off into the woods, it makes sense to have a debate in public. The primary benefits are: 1. Enable downstream channels to easily distro without carrying a large patch sets. 2. Still support existing non-proper distribution methods. 3. Harden / future proof dependent interfaces. Side Benefits: Audit current build mechanics. - Presently the language specific binding are not installed. (.py .jar) - make -jX currently fails - optionally look in arm support. Costs: 1. Time 2. Potential temporary destabilization 3. Infrastructure around build+test may need to change. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
