Re: UFS2 or ZFS for 3TB disk?

2008-10-22 Thread Wojciech Puchar

lots of explanations how to make ZFS stable and usable deleted after 


IMHO, you should be using ZFS for what you want to accomplish.  If you
do go the UFS2 route, I would recommend you set background_fsck=no in

definitely yes.

but UFS works fine on 10TB volume :) and works FAST.

just use larger blocks and fragments (like 4-8K fragment) and reduce inode 
numbers - without this fsck will take REALLY long.

___ mailing list
To unsubscribe, send any mail to [EMAIL PROTECTED]

Re: UFS2 or ZFS for 3TB disk?

2008-10-21 Thread Jeremy Chadwick
On Tue, Oct 21, 2008 at 11:06:07PM -0600, Anthony Chavez wrote:
 The subject says it all.
 I will soon be installing an Areca ARC-1110 and 3x 1.5TB Seagate
 Barracuda SATAs into a 3.2GHz Northwood P4 with 1GB of RAM, and I'm
 wondering which would be the most stable filesystem to use.
 I've read the bigdisk page [1] and the various information about ZFS on
 the FreeBSD Wiki [2].  I'm aware of the tuning requirements that ZFS
 requires, and upgrading to 4GB of RAM would be quite possible as it was
 understood beforehand that ZFS requires a large quantity of it.

The official Solaris ZFS Wiki answers this question.  The I really
don't want to this to get pedantic answer is: caching.

 1. I'm aware of the fact that ZFS works better on 64-bit platforms, and
  that alone has me thinking that it's not a good fit for this particular
 machine.  But apart from that, it seems that ZFS is not yet stable
 enough for my environment (only about 25 users but in production
 nonetheless).  To me, [3] paints all sorts of ugly pictures, which can
 be summarized as count on ZFS-related panics and deadlocks happening
 fairly regularly and disabling ZIL in the interest of stability will
 put your data at risk.  Comments about live systems using ZFS (on
 7.0-RELEASE or 7-STABLE) would be appreciated.

You should review the past 3-4 months of freebsd-fs and freebsd-stable
mailing list posts.  Stabilising ZFS has been thoroughly discussed there.

To date I've only heard of one person having to disable the ZIL to get
stability on RELENG_7.  Everyone else has followed the standard tuning
procedures, which is to tune kmem_size, kmem_size_max, arc_min, and
arc_max.  prefetch_disable being set can also help performance-wise
(many of us find that disabling prefetching provides less stalls or
interruptions during extreme disk I/O).

Remember that kmem_size and kmem_size_max cannot be increased past 2GB
(and you should NOT set them to 2048M!) on RELENG_7 (i386 and amd64).
If you want more memory made available to the kernel, you'll need to run
CURRENT.  None of this affects userland.

What you need to keep in mind is that you need to tune things to get ZFS
stable on RELENG_7, and you will need to keep tune it if you experience
panics relating to ZFS or kmem exhaustion.  The way to avoid panics is
to start with something very restrictive (I advocate 1536M for kmem_size
and kmem_size_max, 16M for arc_min, and 64M of arc_max, on a machine
with 2GB of RAM), and then gradually increase arc_max by ~16M
increments.  You cannot tune this, reboot, let it run for 24 hours and
go IT WORKS.  You will need to seriously test this over a
period of many days, if not weeks.  Each person's environment is
slightly different, and it depends greatly upon the type of data being
accessed, how often, and what sort of workload.  There is no standard.

I use ZFS to manage a raidz1 zpool across 4 500GB disks (all SATA) on my
home machine, which gets thrashed pretty hard due to the content sizes
(500-600MB files).  This machine has 2GB of RAM, and runs amd64.

I also use it in my co-lo as a replacement for /home and on another
machine to be used for backups (rsnapshot+rsync across ssh, results
stored on a ZFS filesystem).  This machine has 2GB of RAM, and runs

The it works better on amd64 platforms is true, but there is
absolutely nothing that stops you from using it on i386.  You should
feel confident using it on both.  If you really want to know of the
performance difference of ZFS on amd64 vs. i386, assuming Google has
nothing, you might ask pjd@ directly.  I think the general deal is that
ZFS was developed with 64-bit pointers and 64-bit values in mind, and
while those are not entirely available on x86 (sans MMX/SSE/SSE2/SSE3
registers) without a bit more work, long long on i386 works fine.

IMHO, you should be using ZFS for what you want to accomplish.  If you
do go the UFS2 route, I would recommend you set background_fsck=no in
your rc.conf, especially since it's obvious data integrity matters a lot
to you (but then again, if it does matter that much, ZFS's checksum
ability should trump your interest in UFS2 entirely).

That's really all I'm going to say on the matter.  (Sorry if that sounds
rude, I'm just incredibly scared that my above comments will induce some
pedantic flame war or battle of sort, which is not my intention.)


| Jeremy Chadwickjdc at |
| Parodius Networking |
| UNIX Systems Administrator  Mountain View, CA, USA |
| Making life hard for others since 1977.  PGP: 4BD6C0CB |

___ mailing list