Re: Recommended filesystem for 6TB storage
Last time I've checked the developer seems preoccupied with his 'paying job' and the project seems to be stuck. On Feb 3, 2008 5:26 AM, Amos Shapira <[EMAIL PROTECTED]> wrote: > On Feb 2, 2008 9:01 PM, Tom Rosenfeld <[EMAIL PROTECTED]> wrote: > > > I hear ZFS is available on Linux. Anyone has any experience with it > > there? > > > Last time I heard it was implemented as a FUSE module (User-level file > system), both because this is ideal for development and because its license > doesn't allow it to be merged with the Linux kernel (GPL incompatibility). > It is claimed to work well but the developer is still in the "make it rock > solid stage" and not in the "make it bloody fast" stage yet. > > Didn't give an impression to be production-ready. > > --Amos > > -- Not gonna be king of the world if you're slave to the grind - Skid Row
Re: Recommended filesystem for 6TB storage
On Feb 2, 2008 9:01 PM, Tom Rosenfeld <[EMAIL PROTECTED]> wrote: > I hear ZFS is available on Linux. Anyone has any experience with it there? Last time I heard it was implemented as a FUSE module (User-level file system), both because this is ideal for development and because its license doesn't allow it to be merged with the Linux kernel (GPL incompatibility). It is claimed to work well but the developer is still in the "make it rock solid stage" and not in the "make it bloody fast" stage yet. Didn't give an impression to be production-ready. --Amos
Re: Recommended filesystem for 6TB storage
I hear ZFS is available on Linux. Anyone has any experience with it there? -tom On Jan 22, 2008 3:28 PM, Jacob Broido <[EMAIL PROTECTED]> wrote: > I agree with Hetz, it really depends on the use scenario of the FS. > > On a side note, if its a viable platform(solaris), I'd consider seriously > ZFS. ZFS IMHO is one of the best FS's today. > I can say from a personal experience that ZFS works well with large > volumes(10TB+) (both size wise and inode count wise). > > > > > > On Jan 22, 2008 3:05 PM, Hetz Ben Hamo <[EMAIL PROTECTED]> wrote: > > > Hi Oren, > > > > As far as I know, all the popular file-systems do support the sizes > > EXT3 supports. > > > > However, it really depends what are you going to store in your > > storage. if it's tiny files (few K's each file), and many other > > parameters do need to be considered before deciding what FS to use. > > > > Also, make sure your tools support the FS you choose. > > > > Good luck, > > Hetz > > > > On Jan 22, 2008 2:49 PM, Oren Held < [EMAIL PROTECTED]> wrote: > > > Hi, > > > > > > As far as I understand, ext3 on RHEL5 should support a single > > filesystem of > > > 8TB or 16TB.. ( http://www.centos.org/product.html) > > > > > > Still, I wonder if it's smart to create a 6TB ext3 file system. > > > In theory, filesystem size won't affect stability or performance > > (except for > > > fsck speed I guess). > > > But in practice? Anybody got experience with it? > > > > > > I like ext3 for its stability, nativity and popularity; do you think > > that I > > > should still use something else for such huge disks? > > > > > > Thanks! > > > > > > - Oren > > > > > > = > > > To unsubscribe, send mail to [EMAIL PROTECTED] with > > > the word "unsubscribe" in the message body, e.g., run the command > > > echo unsubscribe | mail [EMAIL PROTECTED] > > > > > > > > > > > > > > -- > > Skepticism is the lazy person's default position. > > my blog (hebrew): http://benhamo.org > > > > = > > To unsubscribe, send mail to [EMAIL PROTECTED] with > > the word "unsubscribe" in the message body, e.g., run the command > > echo unsubscribe | mail [EMAIL PROTECTED] > > > > > > > -- > Not gonna be king of the world if you're slave to the grind > - Skid Row -- -tom 054-244-8025
[SOLVED] Recommended filesystem for 6TB storage
[... Once upon a time I asked about Linux support of >2TB filesystems ...]
Okay, task is over and we have a 2.5TB ext3 filesystem (eventually not 6TB),
it looks stable also.
Here are the findings:
1. For >2TB, one must use a "gpt" type partition table and not the default,
msdos one. Same goes for MS-Windows, if I got it right.
1.1 fdisk doesn't support gpt, we used parted.
1.2 grub doesn't seem to either support gpt or such huge disk/partition. We
could afford booting from another, small disk (/boot comes back to
live? :) ), so that did it.. So we didn't research the grub thing too deep.
2. At the beginning we used an msdos partition for the big disk. It didn't
give errors immediately! only when accessing the >2TB blocks problem occurred
(i.e. when running fsck, or after few days of writing..).
3. CentOS5 (and thus RHEL5) partitioning config tool ("disk druid", I think?)
doesn't handle big disks/partition automatically: it created msdos partition
table.
Thanks again for your replies.
- Oren
=
To unsubscribe, send mail to [EMAIL PROTECTED] with
the word "unsubscribe" in the message body, e.g., run the command
echo unsubscribe | mail [EMAIL PROTECTED]
Re: Recommended filesystem for 6TB storage
On Tuesday 22 January 2008 22:18, Gilad Ben-Yossef wrote: > > Hi, > > > > As far as I understand, ext3 on RHEL5 should support a single filesystem > > of 8TB or 16TB.. (http://www.centos.org/product.html) > > > > Still, I wonder if it's smart to create a 6TB ext3 file system. > > In theory, filesystem size won't affect stability or performance (except > > for fsck speed I guess). > > But in practice? Anybody got experience with it? > > > > I like ext3 for its stability, nativity and popularity; do you think that > > I should still use something else for such huge disks? > > We have at least one client which makes use of ext3 for similarly sized > file systems with mostly random access fil eaccess for very critical > information. From what I can judge it works quite well. > > You will have to provide more details to get more specific advice > though. Specifically, what Hetz and other have asked is very very > relevant indeed: how do you plan to use this file system: lots of small > file, couple of big files? How does the typical work load looks like? > random access? streaming? Mostly read, mostly write? etc. I'm talking about a system which puts many many 1MB files, spreaded over quite a deep directory hierarchy - so directories don't have a huge number of files. It's a proprietary software (yes yes too bad =) ) written in the company I work for (videocells.com), the product does heavy writing-new-files (in parallel) and deleting-old-files, mostly. I guess it's not really random access but not really sequential access either. Quite a tough question to answer in theory, no? I'd better do benchmarks when we get the hardware.. Thanks for all the replies..! - Oren = To unsubscribe, send mail to [EMAIL PROTECTED] with the word "unsubscribe" in the message body, e.g., run the command echo unsubscribe | mail [EMAIL PROTECTED]
Re: Recommended filesystem for 6TB storage
Oren Held wrote: Hi, As far as I understand, ext3 on RHEL5 should support a single filesystem of 8TB or 16TB.. (http://www.centos.org/product.html) Still, I wonder if it's smart to create a 6TB ext3 file system. In theory, filesystem size won't affect stability or performance (except for fsck speed I guess). But in practice? Anybody got experience with it? I like ext3 for its stability, nativity and popularity; do you think that I should still use something else for such huge disks? We have at least one client which makes use of ext3 for similarly sized file systems with mostly random access fil eaccess for very critical information. From what I can judge it works quite well. You will have to provide more details to get more specific advice though. Specifically, what Hetz and other have asked is very very relevant indeed: how do you plan to use this file system: lots of small file, couple of big files? How does the typical work load looks like? random access? streaming? Mostly read, mostly write? etc. Gilad = To unsubscribe, send mail to [EMAIL PROTECTED] with the word "unsubscribe" in the message body, e.g., run the command echo unsubscribe | mail [EMAIL PROTECTED]
Re: Recommended filesystem for 6TB storage
I agree with Hetz, it really depends on the use scenario of the FS. On a side note, if its a viable platform(solaris), I'd consider seriously ZFS. ZFS IMHO is one of the best FS's today. I can say from a personal experience that ZFS works well with large volumes(10TB+) (both size wise and inode count wise). On Jan 22, 2008 3:05 PM, Hetz Ben Hamo <[EMAIL PROTECTED]> wrote: > Hi Oren, > > As far as I know, all the popular file-systems do support the sizes > EXT3 supports. > > However, it really depends what are you going to store in your > storage. if it's tiny files (few K's each file), and many other > parameters do need to be considered before deciding what FS to use. > > Also, make sure your tools support the FS you choose. > > Good luck, > Hetz > > On Jan 22, 2008 2:49 PM, Oren Held <[EMAIL PROTECTED]> wrote: > > Hi, > > > > As far as I understand, ext3 on RHEL5 should support a single filesystem > of > > 8TB or 16TB.. (http://www.centos.org/product.html) > > > > Still, I wonder if it's smart to create a 6TB ext3 file system. > > In theory, filesystem size won't affect stability or performance (except > for > > fsck speed I guess). > > But in practice? Anybody got experience with it? > > > > I like ext3 for its stability, nativity and popularity; do you think > that I > > should still use something else for such huge disks? > > > > Thanks! > > > > - Oren > > > > = > > To unsubscribe, send mail to [EMAIL PROTECTED] with > > the word "unsubscribe" in the message body, e.g., run the command > > echo unsubscribe | mail [EMAIL PROTECTED] > > > > > > > > -- > Skepticism is the lazy person's default position. > my blog (hebrew): http://benhamo.org > > = > To unsubscribe, send mail to [EMAIL PROTECTED] with > the word "unsubscribe" in the message body, e.g., run the command > echo unsubscribe | mail [EMAIL PROTECTED] > > -- Not gonna be king of the world if you're slave to the grind - Skid Row
Re: Recommended filesystem for 6TB storage
Hi Oren, As far as I know, all the popular file-systems do support the sizes EXT3 supports. However, it really depends what are you going to store in your storage. if it's tiny files (few K's each file), and many other parameters do need to be considered before deciding what FS to use. Also, make sure your tools support the FS you choose. Good luck, Hetz On Jan 22, 2008 2:49 PM, Oren Held <[EMAIL PROTECTED]> wrote: > Hi, > > As far as I understand, ext3 on RHEL5 should support a single filesystem of > 8TB or 16TB.. (http://www.centos.org/product.html) > > Still, I wonder if it's smart to create a 6TB ext3 file system. > In theory, filesystem size won't affect stability or performance (except for > fsck speed I guess). > But in practice? Anybody got experience with it? > > I like ext3 for its stability, nativity and popularity; do you think that I > should still use something else for such huge disks? > > Thanks! > > - Oren > > = > To unsubscribe, send mail to [EMAIL PROTECTED] with > the word "unsubscribe" in the message body, e.g., run the command > echo unsubscribe | mail [EMAIL PROTECTED] > > -- Skepticism is the lazy person's default position. my blog (hebrew): http://benhamo.org = To unsubscribe, send mail to [EMAIL PROTECTED] with the word "unsubscribe" in the message body, e.g., run the command echo unsubscribe | mail [EMAIL PROTECTED]
Recommended filesystem for 6TB storage
Hi, As far as I understand, ext3 on RHEL5 should support a single filesystem of 8TB or 16TB.. (http://www.centos.org/product.html) Still, I wonder if it's smart to create a 6TB ext3 file system. In theory, filesystem size won't affect stability or performance (except for fsck speed I guess). But in practice? Anybody got experience with it? I like ext3 for its stability, nativity and popularity; do you think that I should still use something else for such huge disks? Thanks! - Oren = To unsubscribe, send mail to [EMAIL PROTECTED] with the word "unsubscribe" in the message body, e.g., run the command echo unsubscribe | mail [EMAIL PROTECTED]
