James,
It is incredibly challenging to build everything from scratch.
To build mesos from 100% sources as you propose you'll need to find **all**
transitive dependencies first. Each in turn will have transitive deps.
In fact, if you look at the build system the reason leveldb.patch exist is
to attempt and reduce that complexity.
```
+# Removed tests for TCMALLOC and SNAPPY to prevent additional dependencies.
```
Most linux packages exist **exactly** for that reason. It is not easy to
build native dependencies and have it work for your operating system.
That's what apt-* is for ubuntu for exapmle.
Now building **just mesos** from sources is quite trivial.
Here is my entire ansible build for my local dev box
```
- name: configure mesos
shell: "LIBS=-lglog LDFLAGS=-L/usr/local/lib
{{bolt_cache_dir}}/{{mesos_dir}}/configure
--with-glog=/usr/local
--disable-python
--disable-java"
args:
creates: '{{bolt_cache_dir}}/{{mesos_dir}}/build/Makefile'
chdir: '{{bolt_cache_dir}}/{{mesos_dir}}/build'
- name: Make mesos
sudo: True
shell:
"ldconfig && make -j{{ansible_processor_vcpus}}"
args:
chdir: '{{bolt_cache_dir}}/{{mesos_dir}}/build'
creates: '{{bolt_cache_dir}}/{{mesos_dir}}/build/src/libmesos.la'
```
If you ever participated in large scale native builds, you might know that
compiler flags and linker flags have a huge effect on whether the software
will work at runtime (i.e: symbols stripped away, extern functions not
being exported for weird reasons, ABI incompatibility, etc).
To be honest, building everything from source sounds impractical unless you
have a team of developers optimizing it for your platform. Your
dependencies would be an explosion of build system system deps - CMAKE,
autoconf, Scons, at the top of my head for transitive deps.
As I said, that is exactly my ansible task for building mesos locally. Hope
it helps.
-ag
Sincerely,
Alexander Gallego
On Sun, Feb 22, 2015 at 3:11 PM, CCAAT <[email protected]> wrote:
> On 02/22/2015 06:35 AM, [email protected] wrote:
>
> Last friday i put some Ansible scripts on github for provisoning a multi
>> AZ cluster on AWS.
>> You could have a look at it
>> https://github.com/roybos/aws-mesos-marathon-cluster and maybe it helps
>> you.
>>
>> It basically creates a VPC within an AWS region and setups 1 node in 3
>> AZ's.
>> All nodes are currently equal(mesos master and slave are on the same
>> machine) which is fine for smaller clusters for let's say 3 to 30 nodes.
>>
>> Roy Bos
>>
>
> Very cool. I'm new to ansible, but I like what I've learns so far.
> What would be cool is if some thing like this ansible example was for
> just 3 (generic linux ) system that are installed from sources. No
> binaries, no distro specific packages, just pure, raw sources. That
> way it would provide a baseline for installation on all sorts of linux
> (embedded, vm, container, uncommon_distro, different architectures like
> arm64 etc etc) based systems.
>
>
> Any ansible guides for generic mesos installs from sources would be
> of keen interests to many folks.
>
> After that in-house from sources methodology is viable, I'm quite certain
> companies will want to augment their local (in-house) cluster resources,
> with resources from vendors, particularly in a dynamic mode of utilization.
> Therefore, within the in-house resources mixed (supplemented) with vendor
> resources is the future of cluster computing. ymmv.
>
> Sadly, I see little progress on this open systems approach, and that
> concerns me greatly for the viability of the mesos cluster. Is it indeed
> going to be limited to large corporations trying to sell something nobody
> wants; or are there going to be open source methodologies developed that
> streamline the installation of mesos
> inside of a medium size company with modest resources.
>
> Ansible is the key technology here combined with building up a mesos
> cluster FROM 100% SOURCES.
>
>
> Put that in your presentation at mesos con(artists?)
>
>
> James
>
>
>
>
>
