Hello,
Well I did not want to "hijack a thread" but since several folks are
dancing right next to my current quandary, I figured I'd start another,
companion thread closely related to several of the current threads.
So, I'm building up (2) clusters of (3) machines each; the first
difference is one set will be running systemd and the other set will be
running openrc. The purpose is to help flush out differences related to
how system/kernel based resources are managed by systemd vs openrc, aid
in diagnosing deep memory (OOM) issues and test various kernel
optimizations.
That said, what I'm working on is installing btrfs as the file system on
each pair of mirrored (Raid 1) drives on each system. Later on with
btrfs, I can migrate to (Raid 10) or (Raid 6) when R6 becomes stable.
What I' proposing to do is use a Distributed File System (DFS) like
lustre on top of btrfs for these (2) mesos + spark clusters. Later on
I might try FhGFS as well as gluster after lustre.
With btrfs, it's pretty straight forward to combine several disk into
one "pool" of space. If anyone has setup btrfs for mesos, and
suggestions are most welcome?
Or can I just install mesos on the cluster-btrfs without a DFS ?
Pros/Cons of these approaches? I'm really trying to avoid the legacy
codes such as Hadoop and HDFS.
Suggestions and comments are most welcome.
James
