Thank you very much for this feedback Balagopal, it's extremely useful. I will look into the MD1000 and revise my plan. -Nick
Balagopal Pillai wrote: > MD3000 series doesn't seem to have raid 6 support, which could be very > useful with lots of sata drives. > Also MD3000i doesn't specify LACP support for the dual or quad Ethernet > ports on the enclosure. But > a pe1950 + perc 6 with MD1000 has raid 6 support and the OSS can benefit > from good ethernet bonding support in Linux. > I have a setup with eight MD1000s on two perc 5's on two OSS. > > > Balagopal > > Nick Jennings wrote: >> Hi Brian! Thanks for the reply, comments below >> >> Brian J. Murrell wrote: >> >>>> Instead of just adding another 1TB server, I need to plan for a more >>>> scalable solution. Immediately Lustre came to mind, but I'm wondering >>>> about the performance. Basically our company does niche web-hosting for >>>> "Creative Professionals" so we need fast access to the data in order to >>>> have snappy web services for our clients. Typically these are smaller >>>> files (2MB pictures, 50MB videos, .swf files, etc.). >>>> >>> Well, I'm not sure those files would fall within our general >>> classification of "small files" (wherein we know we don't perform very >>> well). Our small-file issues are usually characterized by "kernel >>> builds" and ~ use, where files are usually much smaller than 1MB. >>> >> Aha, OK well then that's good to know. There's also some kind of >> read-ahead and client side caching right? So files which are accessed a >> lot will be faster to access. >> >> >> >>>> Also I'm wondering about the best way set this up in terms of speed >>>> and ease of growth. I want the web-servers and the storage pool to be >>>> independent of each other. So I can add web-servers as the web traffic >>>> increases, and add more storage ass our storage needs grow. >>>> >>> Well, your web-servers would be Lustre clients. There is no >>> relationship, or rather requirements in terms of the number of clients >>> and servers being used. You use as many servers as your client load >>> demands. So you could imagine both ends of the spectrum where only a >>> relatively few clients could be used to tax quite a few servers or the >>> opposite where a lot of clients with modest demand requires only a few >>> servers. >>> >>> >>>> I was thinking initially we could start with 2 servers, both attached >>>> to the storage array. setup as OSS' and functioning as (load balanced) >>>> web-servers as well. >>>> >>> Sounds like you are describing 2 storage servers, which would require at >>> least 3 servers total. Don't forget about the MDS. Also don't forget >>> about HA if that's a concern for you. You could make the 2 OSSes >>> failover partners for each other if you are willing to accept a possibly >>> lower performance impact when one of the OSSes failing. >>> >>> If HA is important to you however, you need to address an MDS failover >>> with a second server to pick up the MDT should the active MDS fail. >>> >> HA is definitely critical, if the storage pool becomes inaccessible we >> loose clients (and all fingers point at me!). However, I need to find a >> reasonable balance between cost / scalability / performance. The idea >> would be to start small, with the simplest configuration, but allow for >> a lot of growth. In a years time, if we are using 5TB of data, we will >> be in a very good position financially and can afford a systems expansion. >> >> So for starters, what can I get away with here? 1 OSS, 1MDS & 1 Client >> node? Is it a smart thing to do to have the MDS and OSS share the same >> storage target (just a separate partition for the MDS)? What kind of >> system specs are advisable for each type (MDS, OSS & Client node) as far >> as RAM, CPU, disk configuration etc? Also, is it possible to add more >> OSS' to take over existing OSTs that another OSS was previously >> managing? ie. if I have the MD3000i split into 5x1TB volumes (5xOSTs), >> and the OSS is getting hammered, I set another OSS up and hand off 2 or >> 3 OSTs from the old OSS to the new one, and set it up as failover for >> the remaining OSTs. Do-able? >> >> >> >> >>> As for OSSes being web-servers, that would require the OSS/Webservers >>> also be clients and that is an unsupported configuration due to the risk >>> of deadlock due to memory pressure. The recommended architecture would >>> be to make the webservers Lustre clients. >>> >> I see, so from the get-go I'm going to need an internal gigE network for >> OSS/Client communication. >> >> >> >>>> performance can I expect, am I out of touch to expect something similar >>>> to a directly attached RAID array? >>>> >>> I think our generally talked about numbers are something on the order of >>> achieving 80% of the raw storage bandwidth (assuming a capable network >>> and so on). Maybe somebody who is closer to the benchmarking that we >>> are constantly doing can comment further on how close-to-raw-disk we are >>> achieving lately. >>> >> Is it safe to say my bottleneck is going to be the OSS & not the >> network? Is there some documentation I can read about typical setups, >> usage cases & methods for optimal performance? >> >> Thanks! >> -Nick >> _______________________________________________ >> Lustre-discuss mailing list >> [email protected] >> http://lists.lustre.org/mailman/listinfo/lustre-discuss >> > _______________________________________________ > Lustre-discuss mailing list > [email protected] > http://lists.lustre.org/mailman/listinfo/lustre-discuss _______________________________________________ Lustre-discuss mailing list [email protected] http://lists.lustre.org/mailman/listinfo/lustre-discuss
