Vic, Sergi, you can not compare Lustre and GPFS without providing a clear usecase as otherwise you compare apple with oranges. the reason for this is quite simple, Lustre plays well in pretty much one usecase - HPC, GPFS on the other hand is used in many forms of deployments from Storage for Virtual Machines, HPC, Scale-Out NAS, Solutions in digital media, to hosting some of the biggest, most business critical Transactional database installations in the world. you look at 2 products with completely different usability spectrum, functions and features unless as said above you narrow it down to a very specific usecase with a lot of details. even just HPC has a very large spectrum and not everybody is working in a single directory, which is the main scale point for Lustre compared to GPFS and the reason is obvious, if you have only 1 active metadata server (which is what 99% of all lustre systems run) some operations like single directory contention is simpler to make fast, but only up to the limit of your one node, but what happens when you need to go beyond that and only a real distributed architecture can support your workload ? for example look at most chip design workloads, which is a form of HPC, it is something thats extremely metadata and small file dominated, you talk about 100's of millions (in some cases even billions) of files, majority of them <4k, the rest larger files , majority of it with random access patterns that benefit from massive client side caching and distributed data coherency models supported by GPFS token manager infrastructure across 10's or 100's of metadata server and 1000's of compute nodes. you also need to look at the rich feature set GPFS provides, which not all may be important for some environments but are for others like Snapshot, Clones, Hierarchical Storage Management (ILM) , Local Cache acceleration (LROC), Global Namespace Wan Integration (AFM), Encryption, etc just to name a few.
Sven ------------------------------------------ Sven Oehme Scalable Storage Research email: [email protected] Phone: +1 (408) 824-8904 IBM Almaden Research Lab ------------------------------------------ From: Vic Cornell <[email protected]> To: gpfsug main discussion list <[email protected]> Date: 08/08/2014 10:16 AM Subject: Re: [gpfsug-discuss] GPFS and Lustre on same node Sent by: [email protected] Disclaimers - I work for DDN - we sell lustre and GPFS. I know GPFS much better than I know Lustre. The biggest difference we find between GPFS and Lustre is that GPFS - can usually achieve 90% of the bandwidth available to a single client with a single thread. Lustre needs multiple parallel streams to saturate - say an Infiniband connection. Lustre is often faster than GPFS and often has superior metadata performance - particularly where lots of files are created in a single directory. GPFS can support Windows - Lustre cannot. I think GPFS is better integrated and easier to deploy than Lustre - some people disagree with me. Regards, Vic On 8 Aug 2014, at 14:14, Sergi Moré Codina <[email protected]> wrote: > Hi all, > > About main differences between GPFS and Lustre, here you have some bits from our experience: > > -Reliability: GPFS its been proved to be more stable and reliable. Also offers more flexibility in terms of fail-over. It have no restriction in number of servers. As far as I know, an NSD can have as many secondary servers as you want (we are using 8). > > -Metadata: In Lustre each file system is restricted to two servers. No restriction in GPFS. > > -Updates: In GPFS you can update the whole storage cluster without stopping production, one server at a time. > > -Server/Client role: As Jeremy said, in GPFS every server act as a client as well. Useful for administrative tasks. > > -Troubleshooting: Problems with GPFS are easier to track down. Logs are more clear, and offers better tools than Lustre. > > -Support: No problems at all with GPFS support. It is true that it could take time to go up within all support levels, but we always got a good solution. Quite different in terms of hardware. IBM support quality has drop a lot since about last year an a half. Really slow and tedious process to get replacements. Moreover, we keep receiving bad "certified reutilitzed parts" hardware, which slow the whole process even more. > > > These are the main differences I would stand out after some years of experience with both file systems, but do not take it as a fact. > > PD: Salvatore, I would suggest you to contact Jordi Valls. He joined EBI a couple of months ago, and has experience working with both file systems here at BSC. > > Best Regards, > Sergi. > > > On 08/08/2014 01:40 PM, Jeremy Robst wrote: >> On Fri, 8 Aug 2014, Salvatore Di Nardo wrote: >> >>> Now, skipping all this GSS rant, which have nothing to do with the file >>> system anyway and going back to my question: >>> >>> Could someone point the main differences between GPFS and Lustre? >> >> I'm looking at making the same decision here - to buy GPFS or to roll >> our own Lustre configuration. I'm in the process of setting up test >> systems, and so far the main difference seems to be in the that in GPFS >> each server sees the full filesystem, and so you can run other >> applications (e.g backup) on a GPFS server whereas the Luste OSS (object >> storage servers) see only a portion of the storage (the filesystem is >> striped across the OSSes), so you need a Lustre client to mount the full >> filesystem for things like backup. >> >> However I have very little practical experience of either and would also >> be interested in any comments. >> >> Thanks >> >> Jeremy >> >> >> _______________________________________________ >> gpfsug-discuss mailing list >> gpfsug-discuss at gpfsug.org >> http://gpfsug.org/mailman/listinfo/gpfsug-discuss >> > > > -- > > ------------------------------------------------------------------------ > > Sergi More Codina > Barcelona Supercomputing Center > Centro Nacional de Supercomputacion > WWW: http://www.bsc.es Tel: +34-93-405 42 27 > e-mail: [email protected] Fax: +34-93-413 77 21 > > ------------------------------------------------------------------------ > > WARNING / LEGAL TEXT: This message is intended only for the use of the > individual or entity to which it is addressed and may contain > information which is privileged, confidential, proprietary, or exempt > from disclosure under applicable law. If you are not the intended > recipient or the person responsible for delivering the message to the > intended recipient, you are strictly prohibited from disclosing, > distributing, copying, or in any way using this message. If you have > received this communication in error, please notify the sender and > destroy and delete any copies you may have received. > > http://www.bsc.es/disclaimer.htm > > > _______________________________________________ > gpfsug-discuss mailing list > gpfsug-discuss at gpfsug.org > http://gpfsug.org/mailman/listinfo/gpfsug-discuss _______________________________________________ gpfsug-discuss mailing list gpfsug-discuss at gpfsug.org http://gpfsug.org/mailman/listinfo/gpfsug-discuss
_______________________________________________ gpfsug-discuss mailing list gpfsug-discuss at gpfsug.org http://gpfsug.org/mailman/listinfo/gpfsug-discuss
