Hi,
about your DB usage of the Lustre FS, I would strongly advise you to follow
Andreas advice and test first.
Note also that you must use a recent Percona-released MySQL to avoid a nasty
POSIX-related bug in Innodb that crashes MySQL...
In my case, performance showed to be dramatically insufficent. small random I/O
and meta-data performances are a killer for MySQL.
I could probably optimze in a number of other ways that what I did, or throw a
whole lot of additional hardware at it, but after several month of tweaking
(both lustre and mysql), we've just got away from it and returned to plain
direct-attached storage (and sharding).
Nevertheless, I took the time to run post-mortem benchs with mysql+lustre (for
other untested optimzations, notably RAID configs of the underlying storage).
And also with standard tools (dd, sgp_dd, bonnie++) on other DFS.
To make a long story short :
1/ our usage of MySQL is not compatible with Lustre. You should check your is !
2/ Lustre is the best-performing DFS I could test, given then hardware we had
(tested PVFS2/OrangeFS, GlusterFS, FHGFS/FraunhoferFS)
(NB : FHGFS is second-best by far. OrangeFS showed stability issues, Gluster is
stable but is a joke...)
Hardware :
- a pair of HA MDS/MGS with a shared MDT on a iscsi array of RAID10 15k disks
(note : dedicated gigE switch)
- 3 OSSes with 2 RAID1 'os' disks + 6 RAIDed 7.2K SAS disks (originally was a
RAID10)
- connected with DDR IB (+ gigE for 'legacy' clients)
Hope that helps.
François Chassaing Directeur Technique - CTO
weborama.com - f...@weborama.com
- Mail Original -
De: Andreas Dilger adil...@whamcloud.com
À: Jon Yeargers yearg...@ohsu.edu
Cc: lustre-discuss@lists.lustre.org
Envoyé: Jeudi 8 Novembre 2012 02h19:30 GMT +01:00 Amsterdam / Berlin / Berne /
Rome / Stockholm / Vienne
Objet: Re: [Lustre-discuss] Lustre - usage / best practices
On 2012-11-04, at 8:54 AM, Jon Yeargers wrote:
I’ll soon be retiring a number of old file servers and have the opportunity
to move to a DFS based system. Im wondering about using Lustre for ‘mixed
purposes’ and what the recommended setup is.
More specifically: lets say there are three groups that want to use the
space. One group is primarily storing Tb of data in large numbers of flat
files.
So, perfect use case for Lustre.
A second group is using a DB like MySQL that has fewer but larger files.
This winds up being small random IO to the file. Not a common workload for
Lustre, so I would really recommend testing it out on a small system first.
A third may have some other process. Would I divide up my DFS space somehow
or is it ok for all three groups to comingle? Would I do something on the
Lustre clients to force them apart or just rely on Linux security to restrict
access to the client folders (on the lustre client)?
The Lustre servers have essentially the same security model as NFS today, so
they depend on the clients to enforce user identification. If malicious users
can configure their own client nodes and connect them to the filesystem, then
they could access any files in the filesystem. If the client nodes are secure,
then Lustre will properly enforce user access permissions/ACLs to the files.
I can imagine creating a single Lustre client and then NIS mounting the
various volumes on separate machines for each group. This seems like adding
an extra step but perhaps it’s the normal pattern?
Do you mean NFS when you wrote NIS? I'm not sure why this would be better,
but I can definitely think of reasons why it would be worse.
Cheers, Andreas
--
Andreas Dilger Whamcloud, Inc.
Principal Lustre Engineerhttp://www.whamcloud.com/
___
Lustre-discuss mailing list
Lustre-discuss@lists.lustre.org
http://lists.lustre.org/mailman/listinfo/lustre-discuss
___
Lustre-discuss mailing list
Lustre-discuss@lists.lustre.org
http://lists.lustre.org/mailman/listinfo/lustre-discuss
