On 14/08/15 18:54, Dean Hildebrand wrote:
Thanks for the replies Simon...
Chris, are you using -v to give the container access to the nfs subdir
(and hence to a gpfs subdir) (and hence achieve a level of
multi-tenancy)?
-v option to what?
Even without containers, I wonder if this could allow
users to run their own VMs as root as well...and preventing them from
becoming root on gpfs...
I'd love for you to share your experience (mgmt and perf) with this
architecture once you get it up and running.
A quick and dirty test:
From a VM:
-bash-4.1$ time dd if=/dev/zero of=cjwtestfile2 bs=1M count=10240
real 0m20.411s 0m22.137s 0m21.431s 0m21.730s 0m22.056s 0m21.759s
user 0m0.005s 0m0.007s 0m0.006s 0m0.003s 0m0.002s 0m0.004s
sys 0m11.710s 0m10.615s 0m10.399s 0m10.474s 0m10.682s 0m9.965s
From the underlying hypervisor.
real 0m11.138s 0m9.813s 0m9.761s 0m9.793s 0m9.773s 0m9.723s
user 0m0.006s 0m0.013s 0m0.009s 0m0.008s 0m0.008s 0m0.009s
sys 0m5.447s 0m5.529s 0m5.802s 0m5.580s 0m6.190s 0m5.516s
So there's a factor of just over 2 slowdown.
As it's still 500MB/s, I think it's good enough for now.
The machine has a 10Gbit/s network connection and both hypervisor and VM
are running SL6.
Some side benefits of this
architecture that we have been thinking about as well is that it allows
both the containers and VMs to be somewhat ephemeral, while the gpfs
continues to run in the hypervisor...
Indeed. This is another advantage.
If we were running Debian, it would be possible to export part of a
filesystem to a VM. Which would presumably work. In redhat derived OSs
(we are currently using Scientific Linux), I don't believe it is - hence
exporting via NFS.
To ensure VMotion works relatively smoothly, just ensure each VM is
given a hostname to mount that always routes back to the localhost nfs
server on each machine...and I think things should work relatively
smoothly. Note you'll need to maintain the same set of nfs exports
across the entire cluster as well, so taht when a VM moves to another
machine it immediately has an available export to mount.
Yes, we are doing this.
Simon alludes to potential problems at the NFS layer on live migration.
Otherwise, yes indeed the setup should be fine. I'm not familiar enough
with the details of NFS - but I have heard NFS described as "a stateless
filesystem with state". It's the stateful bits I'm concerned about.
Chris
Dean Hildebrand
IBM Almaden Research Center
Inactive hide details for "Simon Thompson (Research Computing - IT
Services)" ---08/13/2015 07:33:16 AM--->I've set up a couple"Simon
Thompson (Research Computing - IT Services)" ---08/13/2015 07:33:16
AM--->I've set up a couple of VM hosts to export some of its GPFS
filesystem >via NFS to machines on that
From: "Simon Thompson (Research Computing - IT Services)"
<[email protected]>
To: gpfsug main discussion list <[email protected]>
Date: 08/13/2015 07:33 AM
Subject: Re: [gpfsug-discuss] Reexporting GPFS via NFS on VM host
Sent by: [email protected]
------------------------------------------------------------------------
>I've set up a couple of VM hosts to export some of its GPFS filesystem
>via NFS to machines on that VM host[1,2].
Provided all your sockets no the VM host are licensed.
>Is live migration of VMs likely to work?
>
>Live migration isn't a hard requirement, but if it will work, it could
>make our life easier.
Live migration using a GPFS file-system on the hypervisor node should work
(subject to the usual caveats of live migration).
Whether live migration and your VM instances would still be able to NFS
mount (assuming loopback address?) if they moved to a different
hypervisor, pass, you might get weird NFS locks. And if they are still
mounting from the original VM host, then you are not doing what the FAQ
says you can do.
Simon
_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at gpfsug.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss
_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at gpfsug.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss
_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at gpfsug.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss
