On 2010-02-10, at 07:39, Roger Spellman wrote: > Thank you. Based on the kernel version string, we had assumed that > SLES > was closer to the latest kernel.org release than RHEL. That appears > not > to be the case. > > Just curious, why the limit is now 16T? This works nicely for 2T > drives > in an 8+2 RAID 6. But, is there a reason that the limit couldn't be > much higher, say 64T or 256T?
Two reasons for this: - primarily, the upstream e2fsprogs does not yet have full support for >16TB filesystems, and while experimental patches exist there are still bugs being found occasionally in that code - there is a certain amount of testing that we need to do before we can say that Lustre supports that configuration That said, with 1.8.2 it is still possible to format the filesystem with the experimental 64-bit e2fsprogs, and mount the OSTs with "-o force_over_16tb" and test this out yourselves. Feedback is of course welcome. I would suggest running "llverfs" on the mounted Lustre filesystem (or other tool which can post-facto verify the data being written) to completely fill an OST, probably unmount/remount it to clear any cache, and then read the data back and ensure that you are getting the correct data back. Running an "e2fsck -f" on the OST would also help verify the on-disk filesystem structure. At some point we will likely conduct this same testing and "officially" support this configuration, but it wasn't done for 1.8.2. At some point, the e2fsck overhead of a large ext4/ldiskfs filesystem becomes too high to support huge configurations (e.g. larger than, say, 128TB if even that). While ext4 and e2fsprogs have gotten a lot of improvements to speed up e2fsck time, there is a limit to what can be done with this. >> -----Original Message----- >> From: [email protected] [mailto:[email protected]] On >> Behalf > Of >> Andreas Dilger >> Sent: Tuesday, February 09, 2010 7:13 PM >> To: Roger Spellman >> Cc: [email protected] >> Subject: Re: [Lustre-discuss] 16T LUNs >> >> On 2010-02-09, at 15:02, Roger Spellman wrote: >>> I see that 1.8.2 supports 16T OSTs for RHEL. >>> >>> Does anyone know when this will be supported for SLES? >> >> No, it will not, because SLES doesn't provide a very uptodate ext4 >> code, and a number of 16TB fixes went into ext4 late in the game. >> RHEL5.4, on the other hand, has very uptodate ext4 code and the RHEL >> ext4 maintainer is one of the ext4 maintainers himself. >> >>> Is anyone currently using a 16T OST, who could share their >>> experiences? Is it stable? >> >> >> I believe a few large customers are already testing/using this. I'll >> let them speak for themselves. >> >> Cheers, Andreas >> -- >> Andreas Dilger >> Sr. Staff Engineer, Lustre Group >> Sun Microsystems of Canada, Inc. > Cheers, Andreas -- Andreas Dilger Sr. Staff Engineer, Lustre Group Sun Microsystems of Canada, Inc. _______________________________________________ Lustre-discuss mailing list [email protected] http://lists.lustre.org/mailman/listinfo/lustre-discuss
