ZFS works really stable on FreeBSD, but I'm biggest problem is how to
control ZFS memory usage. I've no idea how to leash that beast.
FreeBSD has a backpresure mechanism. I can register my function so it
will be called when there are memory problems, which I do. I using it
for ARC layer.
Even with this in place under heavy load the kernel panics, because
memory with KM_SLEEP cannot be allocated.
Here are some statistics of memory usage when the panic occurs:
zfs_znode_cache: 356 * 11547 = 4110732 bytes
zil_lwb_cache: 176 * 43 = 7568 bytes
arc_buf_t: 20 * 7060 = 141200 bytes
arc_buf_hdr_t: 188 * 7060 = 1327280 bytes
dnode_t: 756 * 162311 = 122707116 bytes !!
dmu_buf_impl_t: 332 * 18649 = 6191468
other: 14432256 bytes (regular kmem_alloc())
There is 1GB of RAM, 320MB is for the kernel. 1/3 if kernel memory is
configured as ARC's maximum.
When it panics, debugger statistics show that there is around 2/3 of
this actually allocated, but probably due memory fragmentation.
The most important part is dnode_t probably as it looks it doesn't obey
any limits. Maybe it is a bug in my port and I'm leaking them somehow?
On the other hand when I unload ZFS kernel module, FreeBSD's kernel
reports any memory leaks - they exist, but are much, much smaller.
There is also quite a lot of znodes, which I'd also like to be able to
free and not sure how. In Solaris vnode's life end in VOP_INACTIVE()
routine, but znode if kept around. In FreeBSD VOP_INACTIVE() means "puts
the vnode onto free vnodes list" and when we want to use this vnode for
different file system VOP_RECLAIM() is called and VOP_RECLAIM() will be
a good place to free znode as well, if possible.
Any ideas how to fix it?
--
Pawel Jakub Dawidek http://www.wheel.pl
pjd at FreeBSD.org http://www.FreeBSD.org
FreeBSD committer Am I Evil? Yes, I Am!
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 187 bytes
Desc: not available
URL:
<http://mail.opensolaris.org/pipermail/zfs-code/attachments/20061102/60aa0fd9/attachment.bin>
