Re: Extremely large (70TB) File system/server planning
Mike Smith([EMAIL PROTECTED])@Mon, Feb 05, 2001 at 12:52:24PM -0800: You can't do this with a NetApp either; they max out at about 6TB now (going up to around 12 or so soon). You might want to talk to EMC and/or IBM, both of whom have *extremely* large filers. from my experiences with filers (we have both, country and western here - eg. netapp f740/760 and emc^2 symmetrix/connectrix) i can only say that emc is a pile of sh** - no pun intended. actually the boxes work okay, but you need a shitload of datamover boxes from emc to achieve performance similar to netapp's 760 series (up to 12 data movers with 2gig of ram each). emc goes brute force, netapp use their brains. when it comes to ibm, as far as i understand you have to hook up their filers to rs/6000(aix) or s/370 or s/390 systems since they are "only" fibrechannel or ficon attached raid subsystems, so the client platform is responsible for handling all the filesystem stuff. you might also check out lsi logic's filer products, i think they support 12tb via nas. Your friend may also want to look at Traakan, who have a novel product in this space. i checked out their website which says "under construction" strange... /k -- ... every activity meets with opposition, everyone who acts has his rivals and unfortunately opponents also. But not because people want to be opponents, rather because the tasks and relationships force people to take different points of view. [Dr. Fritz Todt] V I C T O R Y N O T V E N G E A N C E To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-fs" in the body of the message -- Hackers know all the right MOVs. KR433/KR11-RIPE -- http://www.webmonster.de -- ftp://ftp.webmonster.de To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Extremely large (70TB) File system/server planning
when it comes to ibm, as far as i understand you have to hook up their filers to rs/6000(aix) or s/370 or s/390 systems since they are "only" fibrechannel or ficon attached raid subsystems, so the client platform is responsible for handling all the filesystem stuff. Hrrm. The last box I looked at included a pair of RS6000's in the cabinet, and they were touting it as a NAS, but I wasn't paying so much attention then. Your friend may also want to look at Traakan, who have a novel product in this space. i checked out their website which says "under construction" strange... Definitely; they had some neat stuff up there a week or two ago... -- ... every activity meets with opposition, everyone who acts has his rivals and unfortunately opponents also. But not because people want to be opponents, rather because the tasks and relationships force people to take different points of view. [Dr. Fritz Todt] V I C T O R Y N O T V E N G E A N C E To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Extremely large (70TB) File system/server planning
Another company to look at is Yottayotta (www.yottayotta.com). They just announced their first products last November, and there isn't much hard product info online yet. For the arena they're targeting, though, 70TB would be an entry level system. --lyndon To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Extremely large (70TB) File system/server planning
Unless the network is lying to me again, Lyndon Nerenberg said: Another company to look at is Yottayotta (www.yottayotta.com). Yeah, and they have a theme song... http://www.yottayotta.com/images/YottaYotta_Song.mp3 Or is that a reason *NOT* to look at their product? AlanC To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Extremely large (70TB) File system/server planning
"Alan" == Alan Clegg [EMAIL PROTECTED] writes: Alan Or is that a reason *NOT* to look at their product? I'd buy the storage gear, but I think I'll pass on the album ;-) --lyndon To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Extremely large (70TB) File system/server planning
Unless the network is lying to me again, Lyndon Nerenberg said: Another company to look at is Yottayotta (www.yottayotta.com). Yeah, and they have a theme song... http://www.yottayotta.com/images/YottaYotta_Song.mp3 Or is that a reason *NOT* to look at their product? Heh. The theme song was a worry. As was the fact that they are well-staffed at the management level, but they are still advertising for some *very* key development and architectural positions. What do we know about companies with lots of VP-level employees now? 8( -- ... every activity meets with opposition, everyone who acts has his rivals and unfortunately opponents also. But not because people want to be opponents, rather because the tasks and relationships force people to take different points of view. [Dr. Fritz Todt] V I C T O R Y N O T V E N G E A N C E To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Extremely large (70TB) File system/server planning
"Mike" == Mike Smith [EMAIL PROTECTED] writes: Mike As was the fact that they are well-staffed at the management Mike level, but they are still advertising for some *very* key Mike development and architectural positions. Actually, they have a bunch of *very* bright engineers working there. Much of the crew comes from Myrias Research, who were doing some very bleeding edge parallel computing work in the mid to late '80s. (Too bleeding edge for the marketplace, unfortunately ...) --lyndon To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Extremely large (70TB) File system/server planning
Hi Mens, it's exact that filers can't exceed 6TB but we can have eaysyly performance (pretty so good) with their. If you try to have EMC box or IBM, you will have to manage anything that it's not your job (IO for example). I think that netapp can be a very simple solution (where other man sells complexity) Thanks Fab. - Original Message - From: Mike Smith [EMAIL PROTECTED] To: Matt Dillon [EMAIL PROTECTED] Cc: Michael C . Wu [EMAIL PROTECTED]; Mitch Collinsworth [EMAIL PROTECTED]; [EMAIL PROTECTED]; [EMAIL PROTECTED] Sent: Monday, February 05, 2001 9:52 PM Subject: Re: Extremely large (70TB) File system/server planning :| The files are accessed approximately 3 or 4 times a day on average. :| Older files are archived for reference purpose and may never :| be accessed after a week. :| :| Ok, this is a start. Now is the 70 TB the size of the active files? :| Or does that also include the older archived files that may never be :| accessed again? :70TB is the size of the sum of all files, access or no access. :(They still want to maintain accessibility even though the chances are slim.) ... This doesn't sound like something you can just throw together with off-the-shelf PCs and still have something reliable to show for it. You need a big honking RAID system - maybe a NetApp, maybe something else. You have to look at the filesystem and file size limitations of the unit and the client(s). You can't do this with a NetApp either; they max out at about 6TB now (going up to around 12 or so soon). You might want to talk to EMC and/or IBM, both of whom have *extremely* large filers. Your friend may also want to look at Traakan, who have a novel product in this space. -- ... every activity meets with opposition, everyone who acts has his rivals and unfortunately opponents also. But not because people want to be opponents, rather because the tasks and relationships force people to take different points of view. [Dr. Fritz Todt] V I C T O R Y N O T V E N G E A N C E To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message _ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Extremely large (70TB) File system/server planning
While talking to a friend about what his company is planning to do, I found out that he is planning a 70TB filesystem/servers/cluster/db. (Yes, seventy t-e-r-a-b-y-t-e...) We could do this using about 44 of the not-yet-announced TSR-3100 fibre channel RAID storage systems. These are 1.8TB (1.62TB usable) capacity units in a 3U cabinet. It would take around 200A @ 120VAC (about 18KW) to power all of them and should fit in about 5 rack cabinets. Total cost would be about $3 million. -DG David Greenman Co-founder, The FreeBSD Project - http://www.freebsd.org President, TeraSolutions, Inc. - http://www.terasolutions.com Pave the road of life with opportunities. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Extremely large (70TB) File system/server planning
Hello Everyone, While talking to a friend about what his company is planning to do, I found out that he is planning a 70TB filesystem/servers/cluster/db. (Yes, seventy t-e-r-a-b-y-t-e...) Apparently, he has files that go up to 2gb each, and actually require such a horribly sized cluster. If he wanted a PC cluster, and having 5TB on each PC, he would have 350 machines to maintain. From past experience maintaining clusters, I guarantee that he will have at least 1 box failing every other day. And I really do not think his idea of using NFS is that good. ;-) Now if we were to go to the high-end route (and probably more cost effective), we can pick SAN's, large Sun fileservers, or somesuch. I still cannot picture him being able to maintain file integrity. I say that he should attempt to split his filesystems into much smaller chunks, say 1TB each. And attempt some way of having a RAID5 array. Mirroring or other RAID configurations would prove too costly. What would you guys do in this case? :) -- +--+ | [EMAIL PROTECTED] | [EMAIL PROTECTED] | | http://peorth.iteration.net/~keichii | Yes, BSD is a conspiracy. | +--+ To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Extremely large (70TB) File system/server planning
You didn't say what applications this thing is going to support. That does matter. A lot. One thing worth looking at is AFS, or maybe MR-AFS. And now OpenAFS. -Mitch On Mon, 5 Feb 2001, Michael C . Wu wrote: Hello Everyone, While talking to a friend about what his company is planning to do, I found out that he is planning a 70TB filesystem/servers/cluster/db. (Yes, seventy t-e-r-a-b-y-t-e...) Apparently, he has files that go up to 2gb each, and actually require such a horribly sized cluster. If he wanted a PC cluster, and having 5TB on each PC, he would have 350 machines to maintain. From past experience maintaining clusters, I guarantee that he will have at least 1 box failing every other day. And I really do not think his idea of using NFS is that good. ;-) Now if we were to go to the high-end route (and probably more cost effective), we can pick SAN's, large Sun fileservers, or somesuch. I still cannot picture him being able to maintain file integrity. I say that he should attempt to split his filesystems into much smaller chunks, say 1TB each. And attempt some way of having a RAID5 array. Mirroring or other RAID configurations would prove too costly. What would you guys do in this case? :) -- +--+ | [EMAIL PROTECTED] | [EMAIL PROTECTED] | | http://peorth.iteration.net/~keichii | Yes, BSD is a conspiracy. | +--+ To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-fs" in the body of the message To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Extremely large (70TB) File system/server planning
NetApp filers? And what exactly is too costly? He's got enormous costs just in doing backups of this thing, and the savings in using NetApp filers for doing "snapshots" instead of standard backups will buy you some disk in the end... What is this data used for? Archival? How oft is it accessed? How much of the data is "live"? Has he looked at something other than plain disk? Broaden his horizens and get specifics of his needs. On 02/05, Michael C . Wu rearranged the electrons to read: Hello Everyone, While talking to a friend about what his company is planning to do, I found out that he is planning a 70TB filesystem/servers/cluster/db. (Yes, seventy t-e-r-a-b-y-t-e...) Apparently, he has files that go up to 2gb each, and actually require such a horribly sized cluster. If he wanted a PC cluster, and having 5TB on each PC, he would have 350 machines to maintain. From past experience maintaining clusters, I guarantee that he will have at least 1 box failing every other day. And I really do not think his idea of using NFS is that good. ;-) Now if we were to go to the high-end route (and probably more cost effective), we can pick SAN's, large Sun fileservers, or somesuch. I still cannot picture him being able to maintain file integrity. I say that he should attempt to split his filesystems into much smaller chunks, say 1TB each. And attempt some way of having a RAID5 array. Mirroring or other RAID configurations would prove too costly. What would you guys do in this case? :) -- +--+ | [EMAIL PROTECTED] | [EMAIL PROTECTED] | | http://peorth.iteration.net/~keichii | Yes, BSD is a conspiracy. | +--+ To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-fs" in the body of the message Your eyes are weary from staring at the CRT. You feel sleepy. Notice how restful it is to watch the cursor blink. Close your eyes. The opinions stated above are yours. You cannot imagine why you ever felt otherwise. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Extremely large (70TB) File system/server planning
What would you guys do in this case? :) I'd call up my friendly regional SGI, Sun, IBM, and Compaq reps and have them put together proposals. I'm a former SGI guy and know that we've had a bunch of installations of this size and larger (much larger). It's not that big a deal any more. I don't know if that's true for the other vendors, having xfs and 128 processor machines tends to warp one's definiton of what is "hard". :-) -JohnG To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Extremely large (70TB) File system/server planning
On Mon, Feb 05, 2001 at 10:39:02AM -0500, Mitch Collinsworth scribbled: | You didn't say what applications this thing is going to support. | That does matter. A lot. One thing worth looking at is AFS, | or maybe MR-AFS. And now OpenAFS. He has database(s) of graphics simulation results. i.e. large files that are largely unrelated to each other. Compression is not an option. The files are accessed approximately 3 or 4 times a day on average. Older files are archived for reference purpose and may never be accessed after a week. -- +--+ | [EMAIL PROTECTED] | [EMAIL PROTECTED] | | http://peorth.iteration.net/~keichii | Yes, BSD is a conspiracy. | +--+ To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Extremely large (70TB) File system/server planning
On Mon, 5 Feb 2001, Michael C . Wu wrote: Hello Everyone, While talking to a friend about what his company is planning to do, I found out that he is planning a 70TB filesystem/servers/cluster/db. (Yes, seventy t-e-r-a-b-y-t-e...) Apparently, he has files that go up to 2gb each, and actually require such a horribly sized cluster. You later say some files may never be accessed after a week. How about a multi-level storage system, where the files eventually gets written onto dvd's. And either a robot or student :-) to put the requested disks online? Leif To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Extremely large (70TB) File system/server planning
On Mon, 5 Feb 2001, Michael C . Wu wrote: On Mon, Feb 05, 2001 at 10:39:02AM -0500, Mitch Collinsworth scribbled: | You didn't say what applications this thing is going to support. | That does matter. A lot. One thing worth looking at is AFS, | or maybe MR-AFS. And now OpenAFS. He has database(s) of graphics simulation results. i.e. large files that are largely unrelated to each other. Compression is not an option. The files are accessed approximately 3 or 4 times a day on average. Older files are archived for reference purpose and may never be accessed after a week. Ok, this is a start. Now is the 70 TB the size of the active files? Or does that also include the older archived files that may never be accessed again? -Mitch To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Extremely large (70TB) File system/server planning
On Mon, Feb 05, 2001 at 11:47:58AM -0500, Mitch Collinsworth scribbled: | On Mon, 5 Feb 2001, Michael C . Wu wrote: | On Mon, Feb 05, 2001 at 10:39:02AM -0500, Mitch Collinsworth scribbled: | | You didn't say what applications this thing is going to support. | | That does matter. A lot. One thing worth looking at is AFS, | | or maybe MR-AFS. And now OpenAFS. | | He has database(s) of graphics simulation results. i.e. large files that | are largely unrelated to each other. Compression is not an option. | | The files are accessed approximately 3 or 4 times a day on average. | Older files are archived for reference purpose and may never | be accessed after a week. | | Ok, this is a start. Now is the 70 TB the size of the active files? | Or does that also include the older archived files that may never be | accessed again? 70TB is the size of the sum of all files, access or no access. (They still want to maintain accessibility even though the chances are slim.) -- +--+ | [EMAIL PROTECTED] | [EMAIL PROTECTED] | | http://peorth.iteration.net/~keichii | Yes, BSD is a conspiracy. | +--+ To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Extremely large (70TB) File system/server planning
:| The files are accessed approximately 3 or 4 times a day on average. :| Older files are archived for reference purpose and may never :| be accessed after a week. :| :| Ok, this is a start. Now is the 70 TB the size of the active files? :| Or does that also include the older archived files that may never be :| accessed again? :70TB is the size of the sum of all files, access or no access. :(They still want to maintain accessibility even though the chances are slim.) :-- :+--+ :| [EMAIL PROTECTED] | [EMAIL PROTECTED] | :| http://peorth.iteration.net/~keichii | Yes, BSD is a conspiracy. | :+--+ This doesn't sound like something you can just throw together with off-the-shelf PCs and still have something reliable to show for it. You need a big honking RAID system - maybe a NetApp, maybe something else. You have to look at the filesystem and file size limitations of the unit and the client(s). FreeBSD can only support 1 TB sized filesystems. Our device layer converts everything to DEV_BSIZE'd (512) blocks, so to be safe: 2^31 x 512 bytes = 1 TB on Intel boxes. Our NFS implementation has the same per-filesystem limitation. Theoretically UFS/FFS are limited to 2^31 x blocksize, where blocksize can be larger (e.g. 16384 bytes, 65536 bytes), but I have grave doubts that that actually works.. I'm fairly certain that we still convert things to 512 byte block numbers at the device level, and we only use a 32 bit int to store the block number. So FreeBSD could be used as an NFS client, but probably not a server for your application. Considering the number of disks you need to manage, something like a NetApp or other completely self contained RAID-5-capable system for handling the disks is mandatory. -Matt To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Extremely large (70TB) File system/server planning
On Mon, 5 Feb 2001, Michael C . Wu wrote: On Mon, Feb 05, 2001 at 11:47:58AM -0500, Mitch Collinsworth scribbled: | On Mon, 5 Feb 2001, Michael C . Wu wrote: | On Mon, Feb 05, 2001 at 10:39:02AM -0500, Mitch Collinsworth scribbled: | | You didn't say what applications this thing is going to support. | | That does matter. A lot. One thing worth looking at is AFS, | | or maybe MR-AFS. And now OpenAFS. | | He has database(s) of graphics simulation results. i.e. large files that | are largely unrelated to each other. Compression is not an option. | | The files are accessed approximately 3 or 4 times a day on average. | Older files are archived for reference purpose and may never | be accessed after a week. | | Ok, this is a start. Now is the 70 TB the size of the active files? | Or does that also include the older archived files that may never be | accessed again? 70TB is the size of the sum of all files, access or no access. (They still want to maintain accessibility even though the chances are slim.) Ok, well the next question to look at is how do they define "maintain accessibility". In other words what do they consider acceptable? Accessible in 5 seconds, accessible in 1 minute, accessible in 10 minutes, accessible in 1 hour, accessible overnight? 70 TB, as you have already noticed, is no simple feat to accomplish. No matter how you slice it it's going to cost $$. Different levels of accessibility requirement for the archived data can be accomplished with differing technologies and at differing costs. You could rough out a plan for keeping the whole thing online and spinning for instant access and then compare the costs of that with various options that keep the hot data online and archive the rest in varying ways that allow for differing speed of access. Maybe you can archive old data on CDs or tapes. Perhaps keep more recent archives "online" in a jukebox where they are fairly quickly accessible, while older archives are on a rack where someone has to retrieve them as needed. The real question here is: are they really willing to spend what it would take to keep an archive of this size spinning, including systems programmers and administrators? Or are they willing to spend less and have it take a bit longer to get access to the older data? -Mitch To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Extremely large (70TB) File system/server planning
On Mon, Feb 05, 2001 at 09:50:35AM -0800, Matt Dillon wrote: :70TB is the size of the sum of all files, access or no access. :(They still want to maintain accessibility even though the chances are slim.) This doesn't sound like something you can just throw together with off-the-shelf PCs and still have something reliable to show for it. You need a big honking RAID system - maybe a NetApp, maybe something else. You have to look at the filesystem and file size limitations of the unit and the client(s). NetApp's biggest box can "only" handle 6TB of data, currently, using the latest and greatest software. They claim (and I believe them) that 12TB will be the limit later this year. So FreeBSD could be used as an NFS client, but probably not a server for your application. Considering the number of disks you need to manage, something like a NetApp or other completely self contained RAID-5-capable system for handling the disks is mandatory. Netapps are actually RAID-4 (dedicated parity disk), not RAID-5 (parity data is recorded across all drives). -Kurt To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Extremely large (70TB) File system/server planning
:2^31 x 512 bytes = 1 TB on Intel boxes. Our NFS implementation has the :same per-filesystem limitation. Theoretically UFS/FFS are limited Oops. I meant, per-file limitation for NFS clients, not per-filesystem. 1TB per file. -Matt To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Extremely large (70TB) File system/server planning
:| The files are accessed approximately 3 or 4 times a day on average. :| Older files are archived for reference purpose and may never :| be accessed after a week. :| :| Ok, this is a start. Now is the 70 TB the size of the active files? :| Or does that also include the older archived files that may never be :| accessed again? :70TB is the size of the sum of all files, access or no access. :(They still want to maintain accessibility even though the chances are slim.) ... This doesn't sound like something you can just throw together with off-the-shelf PCs and still have something reliable to show for it. You need a big honking RAID system - maybe a NetApp, maybe something else. You have to look at the filesystem and file size limitations of the unit and the client(s). You can't do this with a NetApp either; they max out at about 6TB now (going up to around 12 or so soon). You might want to talk to EMC and/or IBM, both of whom have *extremely* large filers. Your friend may also want to look at Traakan, who have a novel product in this space. -- ... every activity meets with opposition, everyone who acts has his rivals and unfortunately opponents also. But not because people want to be opponents, rather because the tasks and relationships force people to take different points of view. [Dr. Fritz Todt] V I C T O R Y N O T V E N G E A N C E To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
RE: Extremely large (70TB) File system/server planning
Does this have to be a single filesystem? If not, just provide a database front-end that maps some kind of resource identifier to the filesystem name. With that, you can span filers and/or filesystems. Seems like the only thing that would be reasonable. Charles -Original Message- From: Mike Smith [mailto:[EMAIL PROTECTED]] Sent: Monday, February 05, 2001 1:52 PM To: Matt Dillon Cc: Michael C . Wu; Mitch Collinsworth; [EMAIL PROTECTED]; [EMAIL PROTECTED] Subject: Re: Extremely large (70TB) File system/server planning :| The files are accessed approximately 3 or 4 times a day on average. :| Older files are archived for reference purpose and may never :| be accessed after a week. :| :| Ok, this is a start. Now is the 70 TB the size of the active files? :| Or does that also include the older archived files that may never be :| accessed again? :70TB is the size of the sum of all files, access or no access. :(They still want to maintain accessibility even though the chances are slim.) ... This doesn't sound like something you can just throw together with off-the-shelf PCs and still have something reliable to show for it. You need a big honking RAID system - maybe a NetApp, maybe something else. You have to look at the filesystem and file size limitations of the unit and the client(s). You can't do this with a NetApp either; they max out at about 6TB now (going up to around 12 or so soon). You might want to talk to EMC and/or IBM, both of whom have *extremely* large filers. Your friend may also want to look at Traakan, who have a novel product in this space. -- ... every activity meets with opposition, everyone who acts has his rivals and unfortunately opponents also. But not because people want to be opponents, rather because the tasks and relationships force people to take different points of view. [Dr. Fritz Todt] V I C T O R Y N O T V E N G E A N C E To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
RE: Extremely large (70TB) File system/server planning
Besides what platform you decide to run this on, remember that 70TB will put off a surprising amount of heat. Plan your HVAC carefully. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
