Re: [zfs-discuss] [developer] Setting default user/group quotas[usage accounting]?
The time is the creation time of the snapshots. Yes. That is true. Thanks. Fred ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] [developer] Setting default user/group quotas[usage accounting]?
The size accounted for by the userused@ and groupused@ properties is the referenced space, which is used as the basis for many other space accounting values in ZFS (e.g. du / ls -s / stat(2), and the zfs accounting properties referenced, refquota, refreservation, refcompressratio, written). It includes changes local to the dataset (compression, the copies property, file-specific metadata such as indirect blocks), but ignores pool-wide or cross-dataset changes (space shared between a clone and its origin, mirroring, raid-z, dedup[*]). --matt [*] Although dedup can be turned on and off per-dataset, the data is deduplicated against all dedup-enabled data in the pool. Ie, identical data in different datasets will be stored only once, if dedup is enabled for both datasets. Can we also get the *ignored* space accounted? Thanks. Fred ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] [developer] Setting default user/group quotas[usage accounting]?
On Wed, May 2, 2012 at 3:28 PM, Fred Liu fred_...@issi.com wrote: The size accounted for by the userused@ and groupused@ properties is the referenced space, which is used as the basis for many other space accounting values in ZFS (e.g. du / ls -s / stat(2), and the zfs accounting properties referenced, refquota, refreservation, refcompressratio, written). It includes changes local to the dataset (compression, the copies property, file-specific metadata such as indirect blocks), but ignores pool-wide or cross-dataset changes (space shared between a clone and its origin, mirroring, raid-z, dedup[*]). Can we also get the *ignored* space accounted? What problem are you trying to solve? How would you want referenced or userused@... to work? To be more clear: space shared between a clone and its origin is referenced by both the clone and the origin, so it is charged to both the clone's and origin's userused@... properties. The additional space used by mirroring and raid-z applies to all blocks in the pool[*], and is not charged anywhere (except by /sbin/zpool). --matt [*] Assuming you are using the recommended configuration of all the same type of top-level vdevs; if you are not then there's no control over which blocks go to which vdevs. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] [developer] Setting default user/group quotas[usage accounting]?
What problem are you trying to solve? How would you want referenced or userused@... to work? To be more clear: space shared between a clone and its origin is referenced by both the clone and the origin, so it is charged to both the clone's and origin's userused@... properties. The additional space used by mirroring and raid-z applies to all blocks in the pool[*], and is not charged anywhere (except by /sbin/zpool). --matt [*] Assuming you are using the recommended configuration of all the same type of top-level vdevs; if you are not then there's no control over which blocks go to which vdevs. There is no specific problem to resolve. Just want to get sort of accurate equation between the raw storage size and the usable storage size although the *meta file* size is trivial. If you do mass storage budget, this equation is meaningful. Thanks. Fred ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] [developer] Setting default user/group quotas[usage accounting]?
2012-05-03 3:07, Fred Liu wrote: There is no specific problem to resolve. Just want to get sort of accurate equation between the raw storage size and the usable storage size although the *meta file* size is trivial. If you do mass storage budget, this equation is meaningful. I don't think accurate equations are applicable in this case. You can have estimates like no more/no less than X based on, basically, level of redundancy and its overhead. ZFS metadata overhead can also be smaller or bigger, depending on your data's typical block size (fixed for zvols at creation time, variable for files); i.e. if your data is expected to be in very small pieces (comparable to single sector size), you'd have big overhead due to required redundancy and metadata. For data in large chunks overheads would be smaller. This gives you something like available space won't be smaller than M disks from my M+N redundant raidzN arrays minus O percent for metadata. You can also constrain these estimates' range by other assumptions like expected dedup or compression ratios, and hope that your end-users would be able to stuff even more of their addressable data into the pool (because it would be sparse, compressable, and/or not unique), but in the end that's unpredictable from the start. HTH, //Jim ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] [developer] Setting default user/group quotas[usage accounting]?
I don't think accurate equations are applicable in this case. You can have estimates like no more/no less than X based on, basically, level of redundancy and its overhead. ZFS metadata overhead can also be smaller or bigger, depending on your data's typical block size (fixed for zvols at creation time, variable for files); i.e. if your data is expected to be in very small pieces (comparable to single sector size), you'd have big overhead due to required redundancy and metadata. For data in large chunks overheads would be smaller. This gives you something like available space won't be smaller than M disks from my M+N redundant raidzN arrays minus O percent for metadata. You can also constrain these estimates' range by other assumptions like expected dedup or compression ratios, and hope that your end-users would be able to stuff even more of their addressable data into the pool (because it would be sparse, compressable, and/or not unique), but in the end that's unpredictable from the start. Totally agree. We have the similar experimental practice. It varies case by case. Thanks. Fred ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] [developer] Setting default user/group quotas[usage accounting]?
On Apr 26, 2012, at 12:27 AM, Fred Liu wrote: zfs 'userused@' properties and 'zfs userspace' command are good enough to gather usage statistics. I think I mix that with NetApp. If my memory is correct, we have to set quotas to get usage statistics under DataOnTAP. Further, if we can add an ILM-like feature to poll the time-related info(atime,mtime,ctime,etc) with that statistic from ZFS, that will be really cool. In general, file-based ILM has limitations that cause all sorts of issues for things like operating systems, where files might only be needed infrequently, but when they are needed, they are needed right now Have you looked at zfs diff for changed files? Here ILM-like feature, I mean we know how the data is distributed by time per pool/filesystem like how many data are modified/accessed before mm/dd/. And we don't need to do the actual storage-tying operations immediately(moving the infrequently-used data to tie-2 storage). The time-related usage statistics are very useful reference for us. zfs diff will show the delta but not come with the time info. Since no one is focusing on enabling default user/group quota now, the temporarily remedy could be a script which traverse all the users/groups in the directory tree. Tough it is not so decent. The largest market for user/group quotas is .edu. But they represent only a small market when measured by $. There are also many corner cases in this problem space. One might pine for the days of VMS and its file resource management features, those features don't scale well to company- wide LDAP and thousands of file systems. My understanding is the quota management is needed as long as zfs storage is used in NAS way(shared by multi-users). So, for now, the fastest method to solve the problem might be to script some walkers. Yes. That is ture. Thanks. Fred ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] [developer] Setting default user/group quotas[usage accounting]?
On Apr 29, 2012, at 7:59 PM, Fred Liu wrote: On Apr 26, 2012, at 12:27 AM, Fred Liu wrote: “zfs 'userused@' properties” and “'zfs userspace' command” are good enough to gather usage statistics. I think I mix that with NetApp. If my memory is correct, we have to set quotas to get usage statistics under DataOnTAP. Further, if we can add an ILM-like feature to poll the time-related info(atime,mtime,ctime,etc) with that statistic from ZFS, that will be really cool. In general, file-based ILM has limitations that cause all sorts of issues for things like operating systems, where files might only be needed infrequently, but when they are needed, they are needed right now Have you looked at zfs diff for changed files? Here “ILM-like feature”, I mean we know how the data is distributed by time per pool/filesystem like how many data are modified/accessed before mm/dd/. And we don’t need to do the actual storage-tying operations immediately(moving the infrequently-used data to tie-2 storage). The time-related usage statistics are very useful reference for us. “zfs diff” will show the delta but not come with the time info. The time is the creation time of the snapshots. -- richard -- ZFS Performance and Training richard.ell...@richardelling.com +1-760-896-4422 ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] [developer] Setting default user/group quotas[usage accounting]?
2012/4/25 Richard Elling richard.ell...@gmail.com: On Apr 25, 2012, at 8:14 AM, Eric Schrock wrote: ZFS will always track per-user usage information even in the absence of quotas. See the the zfs 'userused@' properties and 'zfs userspace' command. tip: zfs get -H -o value -p userused@username filesystem Yes, and this is the logical size, not physical size. Some ZFS features increase logical size (copies) while others decrease physical size (compression, dedup) The size accounted for by the userused@ and groupused@ properties is the referenced space, which is used as the basis for many other space accounting values in ZFS (e.g. du / ls -s / stat(2), and the zfs accounting properties referenced, refquota, refreservation, refcompressratio, written). It includes changes local to the dataset (compression, the copies property, file-specific metadata such as indirect blocks), but ignores pool-wide or cross-dataset changes (space shared between a clone and its origin, mirroring, raid-z, dedup[*]). --matt [*] Although dedup can be turned on and off per-dataset, the data is deduplicated against all dedup-enabled data in the pool. Ie, identical data in different datasets will be stored only once, if dedup is enabled for both datasets. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] [developer] Setting default user/group quotas[usage accounting]?
“zfs 'userused@' properties” and “'zfs userspace' command” are good enough to gather usage statistics. I think I mix that with NetApp. If my memory is correct, we have to set quotas to get usage statistics under DataOnTAP. Further, if we can add an ILM-like feature to poll the time-related info(atime,mtime,ctime,etc) with that statistic from ZFS, that will be really cool. Since no one is focusing on enabling default user/group quota now, the temporarily remedy could be a script which traverse all the users/groups in the directory tree. Tough it is not so decent. Currently, dedup/compression is pool-based right now, they don’t have the granularity on file system or user or group level. There is also a lot of improving space in this aspect. Thanks. Fred From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss-boun...@opensolaris.org] On Behalf Of Richard Elling Sent: 星期四, 四月 26, 2012 0:48 To: Eric Schrock Cc: zfs-discuss@opensolaris.org; develo...@lists.illumos.org Subject: Re: [zfs-discuss] [developer] Setting default user/group quotas[usage accounting]? On Apr 25, 2012, at 8:14 AM, Eric Schrock wrote: ZFS will always track per-user usage information even in the absence of quotas. See the the zfs 'userused@' properties and 'zfs userspace' command. tip: zfs get -H -o value -p userused@username filesystem Yes, and this is the logical size, not physical size. Some ZFS features increase logical size (copies) while others decrease physical size (compression, dedup) -- richard -- ZFS Performance and Training richard.ell...@richardelling.commailto:richard.ell...@richardelling.com +1-760-896-4422 ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] [developer] Setting default user/group quotas[usage accounting]?
On 2012-04-26 11:27, Fred Liu wrote:
“zfs 'userused@' properties” and “'zfs userspace' command” are good
enough to gather usage statistics.
...
Since no one is focusing on enabling default user/group quota now, the
temporarily remedy could be a script which traverse all the users/groups
in the directory tree. Tough it is not so decent.
find /export/home -type f -uid 12345 -exec du -ks '{}' \; | summing-script
I think you could use some prefetch of dirtree traversal,
like a slocate database, or roll your own (perl script).
But yes, it does seem like stone age compared to ZFS ;)
Currently, dedup/compression is pool-based right now,
Dedup is pool-wide, compression is dataset-wide, applied to
individual blocks. Even deeper, both settings apply to new
writes after the corresponding dataset's property was set
(i.e. a dataset can have files with mixed compression levels,
as well as both deduped and unique files).
they don’t have
the granularity on file system or user or group level.
There is also a lot of improving space in this aspect.
This particular problem was discussed a number of times back
on OpenSolaris forum. It boiled down to what you actually
want to have accounted and perhaps billed - the raw resources
spent by storage system, or the logical resources accessed
and used by its users?
Say, you provide VMs with 100Gb of disk space, but your dedup
is lucky enough to use 1TB overall for say 100 VMs. You can
bill 100 users for full 100Gb each, but your operations budget
(and further planning, etc.) has only been hit for 1Tb.
HTH,
//Jim
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] [developer] Setting default user/group quotas[usage accounting]?
2012/4/26 Fred Liu fred_...@issi.com ** ** Currently, dedup/compression is pool-based right now, they don’t have the granularity on file system or user or group level. There is also a lot of improving space in this aspect. Compression is not pool-based, you can control it with the 'compression' property on a per-filesystem level, and is fundamentally per-block. Dedup is also controlled per-filesystem, though the DDT is global to the pool. If you think there are compelling features lurking here, then by all means grab the code and run with it :-) - Eric -- Eric Schrock Delphix http://blog.delphix.com/eschrock 275 Middlefield Road, Suite 50 Menlo Park, CA 94025 http://www.delphix.com ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] [developer] Setting default user/group quotas[usage accounting]?
2012/4/26 Fred Liu fred_...@issi.com Currently, dedup/compression is pool-based right now, they don't have the granularity on file system or user or group level. There is also a lot of improving space in this aspect. Compression is not pool-based, you can control it with the 'compression' property on a per-filesystem level, and is fundamentally per-block. Dedup is also controlled per-filesystem, though the DDT is global to the pool. If you think there are compelling features lurking here, then by all means grab the code and run with it :-) - Eric -- Thanks for correcting me. I will have a try and see how far I can go. Thanks. Fred ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] [developer] Setting default user/group quotas[usage accounting]?
On 2012-04-26 11:27, Fred Liu wrote:
zfs 'userused@' properties and 'zfs userspace' command are good
enough to gather usage statistics.
...
Since no one is focusing on enabling default user/group quota now, the
temporarily remedy could be a script which traverse all the
users/groups in the directory tree. Tough it is not so decent.
find /export/home -type f -uid 12345 -exec du -ks '{}' \; | summing-script
I think you could use some prefetch of dirtree traversal, like a slocate
database, or roll your own (perl script).
But yes, it does seem like stone age compared to ZFS ;)
Thanks for the hint. I mean traverse all the users/groups in the directory
tree
as getting all user/group info from naming service like nis/ldap for a
specific file system.
And for each found item, we can use zfs set userquota@/groupquota@ to set the
default value.
As for usage accounting, zfs 'userused@' properties and 'zfs userspace'
command are good enough.
We can also use a script to do the summing jobs via traversing all the
pools/filesystems.
Currently, dedup/compression is pool-based right now,
Dedup is pool-wide, compression is dataset-wide, applied to individual blocks.
Even deeper, both settings apply to new writes after the corresponding
dataset's property was set (i.e. a dataset can have files with mixed
compression levels, as well as both deduped and unique files).
they don't have
the granularity on file system or user or group level.
There is also a lot of improving space in this aspect.
This particular problem was discussed a number of times back on OpenSolaris
forum. It boiled down to what you actually want to have accounted and
perhaps billed - the raw resources spent by storage system, or the logical
resources accessed and used by its users?
Say, you provide VMs with 100Gb of disk space, but your dedup is lucky
enough to use 1TB overall for say 100 VMs. You can bill 100 users for full
100Gb each, but your operations budget (and further planning, etc.) has only
been hit for 1Tb.
The ideal situation is we know exactly both the logical usage and the physical
usage per user/group.
But that is not applicable for now. And assuming even we know it, we still
cannot estimate the physical
usage for dedup/compression varies by the using pattern.
Yes. We do get bonus from dedup/compression. But there is no good way to make
it fit into budget plan from my side.
HTH,
//Jim
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] [developer] Setting default user/group quotas[usage accounting]?
On Apr 26, 2012, at 12:27 AM, Fred Liu wrote: “zfs 'userused@' properties” and “'zfs userspace' command” are good enough to gather usage statistics. I think I mix that with NetApp. If my memory is correct, we have to set quotas to get usage statistics under DataOnTAP. Further, if we can add an ILM-like feature to poll the time-related info(atime,mtime,ctime,etc) with that statistic from ZFS, that will be really cool. In general, file-based ILM has limitations that cause all sorts of issues for things like operating systems, where files might only be needed infrequently, but when they are needed, they are needed right now Have you looked at zfs diff for changed files? Since no one is focusing on enabling default user/group quota now, the temporarily remedy could be a script which traverse all the users/groups in the directory tree. Tough it is not so decent. The largest market for user/group quotas is .edu. But they represent only a small market when measured by $. There are also many corner cases in this problem space. One might pine for the days of VMS and its file resource management features, those features don't scale well to company- wide LDAP and thousands of file systems. So, for now, the fastest method to solve the problem might be to script some walkers. -- richard -- ZFS Performance and Training richard.ell...@richardelling.com +1-760-896-4422 ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] [developer] Setting default user/group quotas[usage accounting]?
On Apr 24, 2012, at 2:50 PM, Fred Liu wrote: Yes. Thanks. I am not aware of anyone looking into this. I don't think it is very hard, per se. But such quotas don't fit well with the notion of many file systems. There might be some restricted use cases where it makes good sense, but I'm not convinced it will scale well -- user quotas never scale well. -- richard OK. I see. And I agree such quotas will scale well. From users' side, they always ask for more space or even no quotas at all. One of the main purposes behind such quotas is that we can account usage and get the statistics. Is it possible to do it without setting such quotas? Thanks. Fred ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] [developer] Setting default user/group quotas[usage accounting]?
Missing an important ‘NOT’: OK. I see. And I agree such quotas will **NOT** scale well. From users' side, they always ask for more space or even no quotas at all. One of the main purposes behind such quotas is that we can account usage and get the statistics. Is it possible to do it without setting such quotas? Thanks. Fred _ From: Fred Liu Sent: 星期三, 四月 25, 2012 20:05 To: develo...@lists.illumos.org Cc: 'zfs-discuss@opensolaris.org' Subject: RE: [developer] Setting default user/group quotas[usage accounting]? On Apr 24, 2012, at 2:50 PM, Fred Liu wrote: Yes. Thanks. I am not aware of anyone looking into this. I don't think it is very hard, per se. But such quotas don't fit well with the notion of many file systems. There might be some restricted use cases where it makes good sense, but I'm not convinced it will scale well -- user quotas never scale well. -- richard OK. I see. And I agree such quotas will scale well. From users' side, they always ask for more space or even no quotas at all. One of the main purposes behind such quotas is that we can account usage and get the statistics. Is it possible to do it without setting such quotas? Thanks. Fred ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] [developer] Setting default user/group quotas[usage accounting]?
ZFS will always track per-user usage information even in the absence of quotas. See the the zfs 'userused@' properties and 'zfs userspace' command. - Eric 2012/4/25 Fred Liu fred_...@issi.com Missing an important ‘NOT’: OK. I see. And I agree such quotas will **NOT** scale well. From users' side, they always ask for more space or even no quotas at all. One of the main purposes behind such quotas is that we can account usage and get the statistics. Is it possible to do it without setting such quotas? Thanks. Fred _ *From:* Fred Liu *Sent:* 星期三, 四月 25, 2012 20:05 *To:* develo...@lists.illumos.org *Cc:* 'zfs-discuss@opensolaris.org' *Subject:* RE: [developer] Setting default user/group quotas[usage accounting]? On Apr 24, 2012, at 2:50 PM, Fred Liu wrote: Yes. Thanks. I am not aware of anyone looking into this. I don't think it is very hard, per se. But such quotas don't fit well with the notion of many file systems. There might be some restricted use cases where it makes good sense, but I'm not convinced it will scale well -- user quotas never scale well. -- richard OK. I see. And I agree such quotas will scale well. From users' side, they always ask for more space or even no quotas at all. One of the main purposes behind such quotas is that we can account usage and get the statistics. Is it possible to do it without setting such quotas? Thanks. Fred *illumos-developer* | Archiveshttps://www.listbox.com/member/archive/182179/=now https://www.listbox.com/member/archive/rss/182179/21175057-f8151d0d | Modifyhttps://www.listbox.com/member/?member_id=21175057id_secret=21175057-02786781Your Subscription http://www.listbox.com -- Eric Schrock Delphix http://blog.delphix.com/eschrock 275 Middlefield Road, Suite 50 Menlo Park, CA 94025 http://www.delphix.com ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] [developer] Setting default user/group quotas[usage accounting]?
On Apr 25, 2012, at 8:14 AM, Eric Schrock wrote: ZFS will always track per-user usage information even in the absence of quotas. See the the zfs 'userused@' properties and 'zfs userspace' command. tip: zfs get -H -o value -p userused@username filesystem Yes, and this is the logical size, not physical size. Some ZFS features increase logical size (copies) while others decrease physical size (compression, dedup) -- richard -- ZFS Performance and Training richard.ell...@richardelling.com +1-760-896-4422 ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
