Re: [zfs-discuss] ZFS QoS and priorities

2012-12-06 Thread Richard Elling
On Dec 6, 2012, at 5:30 AM, Matt Van Mater wrote: > > > I'm unclear on the best way to warm data... do you mean to simply `dd > if=/volumes/myvol/data of=/dev/null`? I have always been under the > impression that ARC/L2ARC has rate limiting how much data can be added to the > cache per inte

Re: [zfs-discuss] ZFS QoS and priorities

2012-12-06 Thread Matt Van Mater
> > > > I'm unclear on the best way to warm data... do you mean to simply `dd > if=/volumes/myvol/data of=/dev/null`? I have always been under the > impression that ARC/L2ARC has rate limiting how much data can be added to > the cache per interval (i can't remember the interval). Is this not the

Re: [zfs-discuss] ZFS QoS and priorities

2012-12-06 Thread Matt Van Mater
> > > At present, I do not see async write QoS as being interesting. That leaves > sync writes and reads > as the managed I/O. Unfortunately, with HDDs, the variance in response > time >> queue management > time, so the results are less useful than the case with SSDs. Control > theory works, once a

Re: [zfs-discuss] ZFS QoS and priorities

2012-12-05 Thread Richard Elling
bug fix below... On Dec 5, 2012, at 1:10 PM, Richard Elling wrote: > On Dec 5, 2012, at 7:46 AM, Matt Van Mater wrote: > >> I don't have anything significant to add to this conversation, but wanted to >> chime in that I also find the concept of a QOS-like capability very >> appealing and tha

Re: [zfs-discuss] ZFS QoS and priorities

2012-12-05 Thread Richard Elling
On Dec 5, 2012, at 7:46 AM, Matt Van Mater wrote: > I don't have anything significant to add to this conversation, but wanted to > chime in that I also find the concept of a QOS-like capability very appealing > and that Jim's recent emails resonate with me. You're not alone! I believe > ther

Re: [zfs-discuss] ZFS QoS and priorities

2012-12-05 Thread Richard Elling
On Dec 5, 2012, at 5:41 AM, Jim Klimov wrote: > On 2012-12-05 04:11, Richard Elling wrote: >> On Nov 29, 2012, at 1:56 AM, Jim Klimov > > wrote: >> >>> I've heard a claim that ZFS relies too much on RAM caching, but >>> implements no sort of priorities (indeed, I've seen

Re: [zfs-discuss] ZFS QoS and priorities

2012-12-05 Thread Matt Van Mater
I don't have anything significant to add to this conversation, but wanted to chime in that I also find the concept of a QOS-like capability very appealing and that Jim's recent emails resonate with me. You're not alone! I believe there are many use cases where a granular prioritization that contr

Re: [zfs-discuss] ZFS QoS and priorities

2012-12-05 Thread Jim Klimov
On 2012-11-29 10:56, Jim Klimov wrote: For example, I might want to have corporate webshop-related databases and appservers to be the fastest storage citizens, then some corporate CRM and email, then various lower priority zones and VMs, and at the bottom of the list - backups. On a side note,

Re: [zfs-discuss] ZFS QoS and priorities

2012-12-05 Thread Jim Klimov
On 2012-12-05 04:11, Richard Elling wrote: On Nov 29, 2012, at 1:56 AM, Jim Klimov mailto:jimkli...@cos.ru>> wrote: I've heard a claim that ZFS relies too much on RAM caching, but implements no sort of priorities (indeed, I've seen no knobs to tune those) - so that if the storage box receives m

Re: [zfs-discuss] ZFS QoS and priorities

2012-12-04 Thread Richard Elling
On Nov 29, 2012, at 1:56 AM, Jim Klimov wrote: > I've heard a claim that ZFS relies too much on RAM caching, but > implements no sort of priorities (indeed, I've seen no knobs to > tune those) - so that if the storage box receives many different > types of IO requests with different "administrati

Re: [zfs-discuss] ZFS QoS and priorities

2012-12-01 Thread Nikola M.
On 12/ 2/12 05:19 AM, Richard Elling wrote: On Dec 1, 2012, at 6:54 PM, "Nikola M." wrote: On 12/ 2/12 03:24 AM, Nikola M. wrote: It is using Solaris Zones and throttling their disk usage on that level, so you separate workload processes on separate zones. Or even put KVM machines under the z

Re: [zfs-discuss] ZFS QoS and priorities

2012-12-01 Thread Richard Elling
On Dec 1, 2012, at 6:54 PM, "Nikola M." wrote: > On 12/ 2/12 03:24 AM, Nikola M. wrote: >> It is using Solaris Zones and throttling their disk usage on that level, >> so you separate workload processes on separate zones. >> Or even put KVM machines under the zones (Joyent and OI support >> Joyen

Re: [zfs-discuss] ZFS QoS and priorities

2012-12-01 Thread Nikola M.
On 12/ 2/12 03:24 AM, Nikola M. wrote: It is using Solaris Zones and throttling their disk usage on that level, so you separate workload processes on separate zones. Or even put KVM machines under the zones (Joyent and OI support Joyent-written KVM/Intel implementation in Illumos) for the same

Re: [zfs-discuss] ZFS QoS and priorities

2012-12-01 Thread Nikola M.
On 11/29/12 10:56 AM, Jim Klimov wrote: For example, I might want to have corporate webshop-related databases and appservers to be the fastest storage citizens, then some corporate CRM and email, then various lower priority zones and VMs, and at the bottom of the list - backups. AFAIK, now such

Re: [zfs-discuss] ZFS QoS and priorities

2012-11-29 Thread Bob Friesenhahn
On Thu, 29 Nov 2012, Jim Klimov wrote: I've heard a claim that ZFS relies too much on RAM caching, but implements no sort of priorities (indeed, I've seen no knobs to tune those) - so that if the storage box receives many different types of IO requests with different "administrative weights" in