Re: [zfs-discuss] Issue with simultaneous IO to lots of ZFS pools

2008-05-05 Thread Chris Siebenmann
[Jeff Bonwick:] | That said, I suspect I know the reason for the particular problem | you're seeing: we currently do a bit too much vdev-level caching. | Each vdev can have up to 10MB of cache. With 132 pools, even if | each pool is just a single iSCSI device, that's 1.32GB of cache. | | We need

Re: [zfs-discuss] Issue with simultaneous IO to lots of ZFS pools

2008-05-01 Thread Darren J Moffat
Chris Siebenmann wrote: | Still, I'm curious -- why lots of pools? Administration would be | simpler with a single pool containing many filesystems. The short answer is that it is politically and administratively easier to use (at least) one pool per storage-buying group in our

Re: [zfs-discuss] Issue with simultaneous IO to lots of ZFS pools

2008-05-01 Thread David Collier-Brown
Darren J Moffat [EMAIL PROTECTED] wrote: Chris Siebenmann wrote: | Still, I'm curious -- why lots of pools? Administration would be | simpler with a single pool containing many filesystems. The short answer is that it is politically and administratively easier to use (at least) one pool per

Re: [zfs-discuss] Issue with simultaneous IO to lots of ZFS pools

2008-05-01 Thread Chris Siebenmann
| I think the root cause of the issue is that multiple groups are buying | physical rather than virtual storage yet it is all being attached to a | single system. They're actually buying constant-sized chunks of virtual storage, which is provided through a pool of SAN-based disk space. This

Re: [zfs-discuss] Issue with simultaneous IO to lots of ZFS pools

2008-05-01 Thread Richard Elling
David Collier-Brown wrote: Darren J Moffat [EMAIL PROTECTED] wrote: Chris Siebenmann wrote: | Still, I'm curious -- why lots of pools? Administration would be | simpler with a single pool containing many filesystems. The short answer is that it is politically and administratively

Re: [zfs-discuss] Issue with simultaneous IO to lots of ZFS pools

2008-05-01 Thread Chris Siebenmann
| There are two issues here. One is the number of pools, but the other | is the small amount of RAM in the server. To be honest, most laptops | today come with 2 GBytes, and most servers are in the 8-16 GByte range | (hmmm... I suppose I could look up the average size we sell...) Speaking as a

Re: [zfs-discuss] Issue with simultaneous IO to lots of ZFS pools

2008-05-01 Thread Bart Smaalders
Chris Siebenmann wrote: | There are two issues here. One is the number of pools, but the other | is the small amount of RAM in the server. To be honest, most laptops | today come with 2 GBytes, and most servers are in the 8-16 GByte range | (hmmm... I suppose I could look up the average size

Re: [zfs-discuss] Issue with simultaneous IO to lots of ZFS pools

2008-05-01 Thread David Collier-Brown
Chris Siebenmann [EMAIL PROTECTED] wrote: | Speaking as a sysadmin (and a Sun customer), why on earth would I have | to provision 8 GB+ of RAM on my NFS fileservers? I would much rather | have that memory in the NFS client machines, where it can actually be | put to work by user programs. | | (If

Re: [zfs-discuss] Issue with simultaneous IO to lots of ZFS pools

2008-05-01 Thread Richard Elling
Bart Smaalders wrote: Chris Siebenmann wrote: | There are two issues here. One is the number of pools, but the other | is the small amount of RAM in the server. To be honest, most laptops | today come with 2 GBytes, and most servers are in the 8-16 GByte range | (hmmm... I suppose I

Re: [zfs-discuss] Issue with simultaneous IO to lots of ZFS pools

2008-04-30 Thread Bill Moore
A silly question: Why are you using 132 ZFS pools as opposed to a single ZFS pool with 132 ZFS filesystems? --Bill On Wed, Apr 30, 2008 at 01:53:32PM -0400, Chris Siebenmann wrote: I have a test system with 132 (small) ZFS pools[*], as part of our work to validate a new ZFS-based fileserver

Re: [zfs-discuss] Issue with simultaneous IO to lots of ZFS pools

2008-04-30 Thread Jeff Bonwick
Indeed, things should be simpler with fewer (generally one) pool. That said, I suspect I know the reason for the particular problem you're seeing: we currently do a bit too much vdev-level caching. Each vdev can have up to 10MB of cache. With 132 pools, even if each pool is just a single iSCSI

Re: [zfs-discuss] Issue with simultaneous IO to lots of ZFS pools

2008-04-30 Thread Chris Siebenmann
| Still, I'm curious -- why lots of pools? Administration would be | simpler with a single pool containing many filesystems. The short answer is that it is politically and administratively easier to use (at least) one pool per storage-buying group in our environment. This got discussed in more