Mesos & related codes from 100% sources:
I understand your sentiment. However, I disagree, strongly. I have
been using gentoo for a long time. It has THE solution to 'dependency
hell' built into how how ebuilds are constructed. [1]
I have mesos working and spark (mostly). These other codes that
are necessary just take a bit of effort. It's a cultural approach
rather than accumulating binaries. As (mesos) clustering matures
it's going to be increasingly difficult to debug problems, without
building from sources, imho. A little pain now, great rewards later.
Keeping projects on the open source track does not prevent anyone
(vendors included) from packaging up for quick evaluational types
of installs, or formal usage.
But, if that is all the community develops, then after a while it does
become too much of an effort. At that point, it is no longer an open
source project, but a binary offering. That may bode well for the
short term business interests of some, but long term, it will kill
the project. ymmv.
For example, the ability to diagnose deep problems with resource
utilization will necessitate kernel level debugging with such tools
as ftrace, trace-cmd and Kernelshark, as discussed in these REDhat
documents: [2]
I'd be most interested in associating with folks that have a desire to
build from 100% sources. Afterall, openstack, slurm and many other
packages do this and they are quite easy to support (as there are
ebuilds for them) in gentoo. Often other groups use gentoo ebuilds
as a guide to source compilations on other distros. Gentoo was forked
to create ChromeOS, which was forked again to create CoreOS. Some other
linux distros employ this semantic of building from sources, but, gentoo
linux is the most robust, imho. This source build approach also will
tremendously facilitate porting of mesos and related codes to different
and newer architectures, such as Arm64.
So shall we start with an ansible playbook for building mesos from
sources? WE can add other codes from there and let folks pick and choose
to build as they like. Surely this is not even a difficult task if
organized at the developer level with some miniscule level of
support/encouragement by the developers of mesos. Perhaps a ticket is
needed for comment? This sort of organization will attract more coders
and still encourage a basis for vendors to package up binaries
for more focused needs, including targeted platform support. After all
this is exactly what CoreOS is doing: [3,4,5]
Comments are most welcome!
James
[1] https://devmanual.gentoo.org/
[2]
https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_MRG/1.3/html/Realtime_Tuning_Guide/sect-Realtime_Tuning_Guide-Realtime_Specific_Tuning-Latency_Tracing_Using_trace_cmd.html
[3]
https://coreos.com/docs/sdk-distributors/sdk/building-development-images/
[4] https://chromium.googlesource.com/chromiumos/overlays/portage/
[5] https://en.wikipedia.org/wiki/CoreOS
On 02/22/2015 03:38 PM, Alexander Gallego wrote:
James,
It is incredibly challenging to build everything from scratch.
To build mesos from 100% sources as you propose you'll need to find
**all** transitive dependencies first. Each in turn will have transitive
deps.
In fact, if you look at the build system the reason leveldb.patch exist
is to attempt and reduce that complexity.
```
+# Removed tests for TCMALLOC and SNAPPY to prevent additional dependencies.
```
Most linux packages exist **exactly** for that reason. It is not easy to
build native dependencies and have it work for your operating system.
That's what apt-* is for ubuntu for exapmle.
Now building **just mesos** from sources is quite trivial.
Here is my entire ansible build for my local dev box
```
- name: configure mesos
shell: "LIBS=-lglog LDFLAGS=-L/usr/local/lib
{{bolt_cache_dir}}/{{mesos_dir}}/configure
--with-glog=/usr/local
--disable-python
--disable-java"
args:
creates: '{{bolt_cache_dir}}/{{mesos_dir}}/build/Makefile'
chdir: '{{bolt_cache_dir}}/{{mesos_dir}}/build'
- name: Make mesos
sudo: True
shell:
"ldconfig && make -j{{ansible_processor_vcpus}}"
args:
chdir: '{{bolt_cache_dir}}/{{mesos_dir}}/build'
creates: '{{bolt_cache_dir}}/{{mesos_dir}}/build/src/libmesos.la
<http://libmesos.la>'
```
If you ever participated in large scale native builds, you might know
that compiler flags and linker flags have a huge effect on whether the
software will work at runtime (i.e: symbols stripped away, extern
functions not being exported for weird reasons, ABI incompatibility, etc).
To be honest, building everything from source sounds impractical unless
you have a team of developers optimizing it for your platform. Your
dependencies would be an explosion of build system system deps - CMAKE,
autoconf, Scons, at the top of my head for transitive deps.
As I said, that is exactly my ansible task for building mesos locally.
Hope it helps.
-ag
Sincerely,
Alexander Gallego
On Sun, Feb 22, 2015 at 3:11 PM, CCAAT <[email protected]
<mailto:[email protected]>> wrote:
On 02/22/2015 06:35 AM, [email protected] <mailto:[email protected]> wrote:
Last friday i put some Ansible scripts on github for provisoning
a multi
AZ cluster on AWS.
You could have a look at it
https://github.com/roybos/aws-__mesos-marathon-cluster
<https://github.com/roybos/aws-mesos-marathon-cluster> and maybe
it helps you.
It basically creates a VPC within an AWS region and setups 1
node in 3 AZ's.
All nodes are currently equal(mesos master and slave are on the same
machine) which is fine for smaller clusters for let's say 3 to
30 nodes.
Roy Bos
Very cool. I'm new to ansible, but I like what I've learns so far.
What would be cool is if some thing like this ansible example was for
just 3 (generic linux ) system that are installed from sources. No
binaries, no distro specific packages, just pure, raw sources. That
way it would provide a baseline for installation on all sorts of linux
(embedded, vm, container, uncommon_distro, different architectures
like arm64 etc etc) based systems.
Any ansible guides for generic mesos installs from sources would be
of keen interests to many folks.
After that in-house from sources methodology is viable, I'm quite
certain companies will want to augment their local (in-house)
cluster resources, with resources from vendors, particularly in a
dynamic mode of utilization. Therefore, within the in-house
resources mixed (supplemented) with vendor resources is the future
of cluster computing. ymmv.
Sadly, I see little progress on this open systems approach, and that
concerns me greatly for the viability of the mesos cluster. Is it
indeed going to be limited to large corporations trying to sell
something nobody wants; or are there going to be open source
methodologies developed that streamline the installation of mesos
inside of a medium size company with modest resources.
Ansible is the key technology here combined with building up a mesos
cluster FROM 100% SOURCES.
Put that in your presentation at mesos con(artists?)
James
