One difference: I believe metadata performance is still notably lower with a ZFS based MDT. I know the gap is closing with changes LLNL and Intel have made, but last I heard it was still significant. Someone else may be able to give current numbers.
Of course, you can do an ldiskfs MDT and ZFS OSTs. And since the MDT is much smaller than OSTs, one of the central advantages of ZFS - it allows use of an efficient software RAID, so you can use cheaper hardware options - is less relevant there. (Because the disk hardware is a smaller % of the cost of a good MDS/MDT.) - Patrick ________________________________________ From: lustre-discuss [[email protected]] on behalf of Stearman, Marc [[email protected]] Sent: Wednesday, January 27, 2016 10:46 AM To: [email protected] Cc: Lustre discussion Subject: Re: [lustre-discuss] How to configure OST and OSS on separate nodes Yes, you can run ZFS instead of ldiskfs. A number of sites, LLNL and SDSC included, have moved over to ZFS only Lustre file systems. There have been a number of presentations given at the Lustre User Group (LUG) about the advantages of ZFS. Please look here http://opensfs.org/resources/presentations/ and look at the past few years of talks with ZFS in the title. I will be giving a tutorial on how to Install and Configure a ZFS file system at the Lustre Ecosystem Workshop in March, but that may be a bit too late depending on your needs. -Marc ---- D. Marc Stearman Lustre Operations Lead [email protected] Office: 925-423-9670 Mobile: 925-216-7516 > On Jan 27, 2016, at 2:00 AM, [email protected] wrote: > > Hi, > > Now, it cleared my confusion. So, we really should go for having storage as > separate, we will fix with this option. > > And besides, I have completely ignored ZFS till now. Can I go for ZFS instead > of ldiskfs? What’s the main difference I can find? > > -- > Regards, > Jeevan. > > From: lustre-discuss [mailto:[email protected]] On > Behalf Of Oliver Mangold > Sent: Wednesday, January 27, 2016 3:25 PM > To: [email protected] > Subject: Re: [lustre-discuss] How to configure OST and OSS on separate nodes > > On 27.01.2016 10:36, [email protected] wrote: > But my question is: Can I use OST on client and OSS on separate server (if it > is possible to run OST and OSS on separate servers) > > I don't understand what you mean. OSS is just a name for the machine which > runs one or multiple OSTs. So by definition OSTs always run on OSSes. > > The reason for this is we have tens of servers with internal hard disk of 4T > each which we want to use it for OSTs, but all these servers are also should > be clients (they are the grid execution hosts). > Sounds like a bad idea to me to do that. Beside the mentioned memory pressure > problems, you also don't have a failover configuration for the OSTs, which is > strongly recommended. Otherwise one downed server will stall all or most of > your active jobs. > > So, I am thinking of having separate OSSs and use the servers itself as OSTs. > Also, do we need to install lustre kernel in all those OSTs? Because, all > these servers are running with kernel 2.6.32-573.12.1.el6.x86_64, which the > standard lustre packages won’t be supporting. So, we need to roll back to > 32-504_lustre kernel and again we need to work on setting yum repos, > installing packages, resolving dependencies required for our tools..it’s > really a lot of work. > > The last I know is, you need a patched kernel when using ldiskfs as the > backing filesystem, but you can use a patchless kernel with ZFS. > > > The information contained in this electronic message and any attachments to > this message are intended for the exclusive use of the addressee(s) and may > contain proprietary, confidential or privileged information. If you are not > the intended recipient, you should not disseminate, distribute or copy this > e-mail. Please notify the sender immediately and destroy all copies of this > message and any attachments. WARNING: Computer viruses can be transmitted via > email. The recipient should check this email and any attachments for the > presence of viruses. The company accepts no liability for any damage caused > by any virus transmitted by this email. > www.wipro.com_______________________________________________ > lustre-discuss mailing list > [email protected] > http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org _______________________________________________ lustre-discuss mailing list [email protected] http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org _______________________________________________ lustre-discuss mailing list [email protected] http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
