On Tue, Nov 26, 2013 at 7:10 AM, Andrew Holway <[email protected]> wrote: > Taking a holistic approach: What does ZFS mean for HPC? I am excited > about on the fly data compression and snapshotting however for most > scientific datas I suppose dedupe is a bit pointless.
We were looking at Lustre+ZFS last year, and we set up a testbed on a Backblaze Storage Pods 2.0 (for experimenting with different FS layouts only, Storage Pods are not designed for performance). We found the LLNL Sequoia ZFS/Lustre talk presented at Lustre User Group 2012 the most interesting: http://www.opensfs.org/wp-content/uploads/2011/11/LUG12_ZFS_Lustre_for_Sequoia.pdf (The benchmark results are pretty old, and you can find more recent results by googling: LLNL+Sequoia+ZFS+Lustre ) > Are raid controllers a thing of the past? If you believe that Cloud Computing is the future, then RAID controllers are a thing of the past because Amazon EC2 and Google Compute Engine only give you JBOD. We tried to run Lustre in Amazon EC2 on a hs1.8xlarge instance (24 * 2TB HDDs), and we will repeat the tests again on the new i2.8xlarge instances (8 * 800 GB SSD) soon - note that we did the test with our own patched kernel and installed a custom Amazon Kernel Image, and we plan to redo a test on production software before we publish results. Luckily Intel released "Cloud Edition for Lustre" a few months ago, but we are haven't checked to see if supports ZFS or if it is still ldiskfs based... https://aws.amazon.com/marketplace/pp/B00GBFUO9A Rayson ================================================== Open Grid Scheduler - The Official Open Source Grid Engine http://gridscheduler.sourceforge.net/ http://gridscheduler.sourceforge.net/GridEngine/GridEngineCloud.html > > how often to bits flip anyway? What impact does this have? Does > checksumming save our data? > > Ta > > Andrew > _______________________________________________ > Beowulf mailing list, [email protected] sponsored by Penguin Computing > To change your subscription (digest mode or unsubscribe) visit > http://www.beowulf.org/mailman/listinfo/beowulf _______________________________________________ Beowulf mailing list, [email protected] sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf
