Re: [zfs-discuss] arc_no_grow is set to 1 and never set back to 0
On 19 January, 2012 - Clemens Kalb sent me these 1,1K bytes: > Peter, > > I experience the same issue w/ 2 systems upgraded from Solaris 11 > Express to Solaris 11 11/11. Has your SR with Oracle made any > progress since your last post? Target solaris_11 Customer Status 3-Accepted Severity2-High Last Updated2012-01-07 00:00:00 GMT+00:00 I've filed an SR pointing at the same bug too, to get momentum. /Tomas -- Tomas Forsman, st...@acc.umu.se, http://www.acc.umu.se/~stric/ |- Student at Computing Science, University of Umeå `- Sysadmin at {cs,acc}.umu.se ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] arc_no_grow is set to 1 and never set back to 0
Peter, I experience the same issue w/ 2 systems upgraded from Solaris 11 Express to Solaris 11 11/11. Has your SR with Oracle made any progress since your last post? Thanks > It's supposed to be > > 7111576: arc shrinks in the absence of memory pressure > > currently in status "accepted" and an RPE escalation pending. Clemens Kalb Leiter netlogix Media Telefon: +49 (911) 539909 - 0 E-Mail: clemens.k...@netlogix.de Website: media.netlogix.de -- netlogix GmbH & Co. KG IT-Services | IT-Training | Media Andernacher Stra?e 53 | 90411 N?rnberg Telefon: +49 (911) 539909 - 0 | Fax: +49 (911) 539909 - 99 E-Mail: i...@netlogix.de | Internet: http://www.netlogix.de netlogix GmbH & Co. KG ist eingetragen am Amtsgericht N?rnberg (HRA 13338) Pers?nlich haftende Gesellschafterin: netlogix Verwaltungs GmbH (HRB 20634) Umsatzsteuer-Identifikationsnummer: DE 233472254 Gesch?ftsf?hrer: Stefan Buchta, Matthias Schmidt ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] arc_no_grow is set to 1 and never set back to 0
It's supposed to be 7111576: arc shrinks in the absence of memory pressure currently in status "accepted" and an RPE escalation pending. -Original Message- From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss-boun...@opensolaris.org] On Behalf Of Tomas Forsman Sent: Donnerstag, 5. Januar 2012 10:35 To: zfs-discuss@opensolaris.org Subject: Re: [zfs-discuss] arc_no_grow is set to 1 and never set back to 0 On 04 January, 2012 - Steve Gonczi sent me these 2,5K bytes: > The interesting bit is what happens inside arc_reclaim_needed(), that > is, how it arrives at the conclusion that there is memory pressure. > > Maybe we could trace arg0, which gives the location where we have left > the function. This would finger which return path > arc_reclaim_needed() took. It's new code, basically comparing the inuse/total/free from kstat -n zfs_file_data, which seems buggered. > Steve > > > - Original Message - > > > > Well it looks like the only place this get's changed is in the > arc_reclaim_thread for opensolaris. I suppose you could dtrace it to see what > is going on and investigate what is happening to the return code of the > arc_reclaim_needed is. > > > > http://src.opensolaris.org/source/xref/onnv/onnv-gate/usr/src/uts/comm > on/fs/zfs/arc.c#2089 > > > maybe > > > dtrace -n 'fbt:zfs:arc_reclaim_needed:return { trace(arg1) }' > > > Dave > > > > > /Tomas -- Tomas Forsman, st...@acc.umu.se, http://www.acc.umu.se/~stric/ |- Student at Computing Science, University of Umeå `- Sysadmin at {cs,acc}.umu.se ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] arc_no_grow is set to 1 and never set back to 0
On 04 January, 2012 - Steve Gonczi sent me these 2,5K bytes: > The interesting bit is what happens inside arc_reclaim_needed(), > that is, how it arrives at the conclusion that there is memory pressure. > > Maybe we could trace arg0, which gives the location where > we have left the function. This would finger which return path > arc_reclaim_needed() took. It's new code, basically comparing the inuse/total/free from kstat -n zfs_file_data, which seems buggered. > Steve > > > - Original Message - > > > > Well it looks like the only place this get's changed is in the > arc_reclaim_thread for opensolaris. I suppose you could dtrace it to see what > is going on and investigate what is happening to the return code of the > arc_reclaim_needed is. > > > > http://src.opensolaris.org/source/xref/onnv/onnv-gate/usr/src/uts/common/fs/zfs/arc.c#2089 > > > > maybe > > > dtrace -n 'fbt:zfs:arc_reclaim_needed:return { trace(arg1) }' > > > Dave > > > > > /Tomas -- Tomas Forsman, st...@acc.umu.se, http://www.acc.umu.se/~stric/ |- Student at Computing Science, University of Umeå `- Sysadmin at {cs,acc}.umu.se ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] arc_no_grow is set to 1 and never set back to 0
The interesting bit is what happens inside arc_reclaim_needed(), that is, how it arrives at the conclusion that there is memory pressure. Maybe we could trace arg0, which gives the location where we have left the function. This would finger which return path arc_reclaim_needed() took. Steve - Original Message - Well it looks like the only place this get's changed is in the arc_reclaim_thread for opensolaris. I suppose you could dtrace it to see what is going on and investigate what is happening to the return code of the arc_reclaim_needed is. http://src.opensolaris.org/source/xref/onnv/onnv-gate/usr/src/uts/common/fs/zfs/arc.c#2089 maybe dtrace -n 'fbt:zfs:arc_reclaim_needed:return { trace(arg1) }' Dave ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] arc_no_grow is set to 1 and never set back to 0
On Jan 4, 2012, at 8:49 AM, Peter Radig wrote: > Thanks. The guys from Oracle are currently looking at some new code that was > introduced in arc_reclaim_thread() between b151a and b175. Closed source strategy loses again! -- richard > > Peter Radig, Ahornstrasse 34, 85774 Unterföhring, Germany > tel: +49 89 99536751 - fax: +49 89 99536754 - mobile: +49 171 2652977 > email: pe...@radig.de > > From: David Blasingame [mailto:dbla...@yahoo.com] > Sent: Mittwoch, 4. Januar 2012 17:35 > To: Peter Radig; st...@acc.umu.se > Cc: zfs-discuss@opensolaris.org > Subject: Re: [zfs-discuss] arc_no_grow is set to 1 and never set back to 0 > > > Well it looks like the only place this get's changed is in the > arc_reclaim_thread for opensolaris. I suppose you could dtrace it to see > what is going on and investigate what is happening to the return code of the > arc_reclaim_needed is. > > http://src.opensolaris.org/source/xref/onnv/onnv-gate/usr/src/uts/common/fs/zfs/arc.c#2089 > > maybe > > dtrace -n 'fbt:zfs:arc_reclaim_needed:return { trace(arg1) }' > > Dave > > > ---- Original Message > Subject: > Re: [zfs-discuss] arc_no_grow is set to 1 and never set back to 0 > Date: > Tue, 3 Jan 2012 23:50:04 + > From: > Peter Radig > To: > Tomas Forsman > CC: > zfs-discuss@opensolaris.org > > > Tomas, > > Yup, same here. > > free0 > mem_inuse 16162750464 > mem_total 17171480576 > > Page SummaryPagesMB %Tot > > Kernel 842305 3290 20% > ZFS File Data2930110% > Anon44038 1721% > Exec and libs3731140% > Page cache 8580330% > Free (cachelist) 5504210% > Free (freelist) 3284924 12831 78% > > Total 4192012 16375 > Physical 4192011 16375 > > > I will create an SR with Oracle. > > Thanks, > Peter > > -----Original Message- > From: Tomas Forsman [mailto:st...@acc.umu.se] > Sent: Mittwoch, 4. Januar 2012 00:39 > To: Peter Radig > Cc: zfs-discuss@opensolaris.org > Subject: Re: [zfs-discuss] arc_no_grow is set to 1 and never set back to 0 > > On 03 January, 2012 - Peter Radig sent me these 3,5K bytes: > > > Hello. > > > > I have a Solaris 11/11 x86 box (which I migrated from SolEx 11/10 a couple > > of weeks ago). > > > > Without no obvious reason (at least for me), after an uptime of 1 to 2 days > > (observed 3 times now) Solaris sets arc_no_grow to 1 and then never sets it > > back to 0. ARC is being shrunk to less than 1 GB -- needless to say that > > performance is terrible. There is not much load on this system. > > > > Memory seems to be not an issue (see below). > > > > I looked at the old Nevada code base of onnv_147 and can't find a reason > > for this happening. > > > > How can I find out what's causing this? > > New code that seems to be counting wrong.. I was planning on filing a bug, > but am currently struggling to convince oracle that we bought support.. > > Try this: > kstat -n zfs_file_data > > In my case, I get: > free15322 > mem_inuse 24324866048 > mem_total 25753026560 > .. where ::memstat says: > Kernel2638984 10308 42% > ZFS File Data 39260 1531% > Anon 873549 3412 14% > Exec and libs5199200% > Page cache 20019780% > Free (cachelist) 6608250% > Free (freelist) 2703509 10560 43% > > On another reboot, it refused to go over 130MB on a 24GB system.. > > > > BTW: I was not seeing this on SolEx 11/10. > > Dito. > > > Thanks, > > Peter > > > > > > > > *** ::memstat *** > > Page SummaryPagesMB %Tot > > > > Kernel 860254 3360 21% > > ZFS File Data3047110% > &g
Re: [zfs-discuss] arc_no_grow is set to 1 and never set back to 0
Thanks. The guys from Oracle are currently looking at some new code that was introduced in arc_reclaim_thread() between b151a and b175. Peter Radig, Ahornstrasse 34, 85774 Unterföhring, Germany tel: +49 89 99536751 - fax: +49 89 99536754 - mobile: +49 171 2652977 email: pe...@radig.de<mailto:pe...@radig.de> From: David Blasingame [mailto:dbla...@yahoo.com] Sent: Mittwoch, 4. Januar 2012 17:35 To: Peter Radig; st...@acc.umu.se Cc: zfs-discuss@opensolaris.org Subject: Re: [zfs-discuss] arc_no_grow is set to 1 and never set back to 0 Well it looks like the only place this get's changed is in the arc_reclaim_thread for opensolaris. I suppose you could dtrace it to see what is going on and investigate what is happening to the return code of the arc_reclaim_needed is. http://src.opensolaris.org/source/xref/onnv/onnv-gate/usr/src/uts/common/fs/zfs/arc.c#2089 maybe dtrace -n 'fbt:zfs:arc_reclaim_needed:return { trace(arg1) }' Dave Original Message -------- Subject: Re: [zfs-discuss] arc_no_grow is set to 1 and never set back to 0 Date: Tue, 3 Jan 2012 23:50:04 + From: Peter Radig <mailto:pe...@radig.de> To: Tomas Forsman <mailto:st...@acc.umu.se> CC: zfs-discuss@opensolaris.org<mailto:zfs-discuss@opensolaris.org> <mailto:zfs-discuss@opensolaris.org> Tomas, Yup, same here. free0 mem_inuse 16162750464 mem_total 17171480576 Page SummaryPagesMB %Tot Kernel 842305 3290 20% ZFS File Data2930110% Anon44038 1721% Exec and libs3731140% Page cache 8580330% Free (cachelist) 5504210% Free (freelist) 3284924 12831 78% Total 4192012 16375 Physical 4192011 16375 I will create an SR with Oracle. Thanks, Peter -Original Message- From: Tomas Forsman [mailto:st...@acc.umu.se] Sent: Mittwoch, 4. Januar 2012 00:39 To: Peter Radig Cc: zfs-discuss@opensolaris.org<mailto:zfs-discuss@opensolaris.org> Subject: Re: [zfs-discuss] arc_no_grow is set to 1 and never set back to 0 On 03 January, 2012 - Peter Radig sent me these 3,5K bytes: > Hello. > > I have a Solaris 11/11 x86 box (which I migrated from SolEx 11/10 a couple of > weeks ago). > > Without no obvious reason (at least for me), after an uptime of 1 to 2 days > (observed 3 times now) Solaris sets arc_no_grow to 1 and then never sets it > back to 0. ARC is being shrunk to less than 1 GB -- needless to say that > performance is terrible. There is not much load on this system. > > Memory seems to be not an issue (see below). > > I looked at the old Nevada code base of onnv_147 and can't find a reason for > this happening. > > How can I find out what's causing this? New code that seems to be counting wrong.. I was planning on filing a bug, but am currently struggling to convince oracle that we bought support.. Try this: kstat -n zfs_file_data In my case, I get: free15322 mem_inuse 24324866048 mem_total 25753026560 .. where ::memstat says: Kernel2638984 10308 42% ZFS File Data 39260 1531% Anon 873549 3412 14% Exec and libs5199200% Page cache 20019780% Free (cachelist) 6608250% Free (freelist) 2703509 10560 43% On another reboot, it refused to go over 130MB on a 24GB system.. > BTW: I was not seeing this on SolEx 11/10. Dito. > Thanks, > Peter > > > > *** ::memstat *** > Page SummaryPagesMB %Tot > > Kernel 860254 3360 21% > ZFS File Data3047110% > Anon38246 1491% > Exec and libs3765140% > Page cache 8517330% > Free (cachelist) 5866220% > Free (freelist) 3272317 12782 78% > Total 4192012 16375 > Physical 4192011 16375 > > mem_inuse
Re: [zfs-discuss] arc_no_grow is set to 1 and never set back to 0
Well it looks like the only place this get's changed is in the arc_reclaim_thread for opensolaris. I suppose you could dtrace it to see what is going on and investigate what is happening to the return code of the arc_reclaim_needed is. http://src.opensolaris.org/source/xref/onnv/onnv-gate/usr/src/uts/common/fs/zfs/arc.c#2089 maybe dtrace -n 'fbt:zfs:arc_reclaim_needed:return { trace(arg1) }' Dave Original Message ---- Subject: Re: [zfs-discuss] arc_no_grow is set to 1 and never set back to 0 Date: Tue, 3 Jan 2012 23:50:04 + From: Peter Radig To: Tomas Forsman CC: zfs-discuss@opensolaris.org Tomas, Yup, same here. free0 mem_inuse 16162750464 mem_total 17171480576 Page Summary PagesMB %Tot Kernel 842305 3290 20% ZFS File Data2930110% Anon44038 1721% Exec and libs3731140% Page cache 8580330% Free (cachelist) 5504210% Free (freelist) 3284924 12831 78% Total 4192012 16375 Physical 4192011 16375 I will create an SR with Oracle. Thanks, Peter -Original Message- From: Tomas Forsman [mailto:st...@acc.umu.se] Sent: Mittwoch, 4. Januar 2012 00:39 To: Peter Radig Cc: zfs-discuss@opensolaris.org Subject: Re: [zfs-discuss] arc_no_grow is set to 1 and never set back to 0 On 03 January, 2012 - Peter Radig sent me these 3,5K bytes: > Hello. > > I have a Solaris 11/11 x86 box (which I migrated from SolEx 11/10 a couple of > weeks ago). > > Without no obvious reason (at least for me), after an uptime of 1 to 2 days > (observed 3 times now) Solaris sets arc_no_grow to 1 and then never sets it > back to 0. ARC is being shrunk to less than 1 GB -- needless to say that > performance is terrible. There is not much load on this system. > > Memory seems to be not an issue (see below). > > I looked at the old Nevada code base of onnv_147 and can't find a reason for > this happening. > > How can I find out what's causing this? New code that seems to be counting > wrong.. I was planning on filing a bug, but am currently struggling to > convince oracle that we bought support.. Try this: kstat -n zfs_file_data In my case, I get: free15322 mem_inuse 24324866048 mem_total 25753026560 .. where ::memstat says: Kernel2638984 10308 42% ZFS File Data 39260 1531% Anon 873549 3412 14% Exec and libs5199200% Page cache 20019780% Free (cachelist) 6608250% Free (freelist) 2703509 10560 43% On another reboot, it refused to go over 130MB on a 24GB system.. > BTW: I was not seeing this on SolEx 11/10. Dito. > Thanks, > Peter > > > > *** ::memstat *** > Page SummaryPagesMB %Tot > > Kernel 860254 3360 21% > ZFS File Data3047110% > Anon38246 1491% > Exec and libs3765140% > Page cache 8517330% > Free (cachelist) 5866220% > Free (freelist) 3272317 12782 78% > Total 4192012 16375 > Physical 4192011 16375 > > mem_inuse 4145901568 > mem_total 1077466365952 > > *** ::arc *** > hits = 186279921 > misses= 14366462 > demand_data_hits = 4648464 > demand_data_misses= 8605873 > demand_metadata_hits = 171803126 > demand_metadata_misses= 3805675 > prefetch_data_hits=772678 > prefetch_data_misses = 1464457 > prefetch_metadata_hits= 9055653 > prefetch_metadata_misses =490457 > mru_hits = 12295087 > mru_ghost_hits= 0 > mfu_hits = 175281066 > mfu_ghost_hits= 0 > deleted = 14462192 > mutex_miss=30 > hash_elements = 3752768 > hash_elements_max = 3752770 >
Re: [zfs-discuss] arc_no_grow is set to 1 and never set back to 0
Tomas, Yup, same here. free0 mem_inuse 16162750464 mem_total 17171480576 Page SummaryPagesMB %Tot Kernel 842305 3290 20% ZFS File Data2930110% Anon44038 1721% Exec and libs3731140% Page cache 8580330% Free (cachelist) 5504210% Free (freelist) 3284924 12831 78% Total 4192012 16375 Physical 4192011 16375 I will create an SR with Oracle. Thanks, Peter -Original Message- From: Tomas Forsman [mailto:st...@acc.umu.se] Sent: Mittwoch, 4. Januar 2012 00:39 To: Peter Radig Cc: zfs-discuss@opensolaris.org Subject: Re: [zfs-discuss] arc_no_grow is set to 1 and never set back to 0 On 03 January, 2012 - Peter Radig sent me these 3,5K bytes: > Hello. > > I have a Solaris 11/11 x86 box (which I migrated from SolEx 11/10 a couple of > weeks ago). > > Without no obvious reason (at least for me), after an uptime of 1 to 2 days > (observed 3 times now) Solaris sets arc_no_grow to 1 and then never sets it > back to 0. ARC is being shrunk to less than 1 GB -- needless to say that > performance is terrible. There is not much load on this system. > > Memory seems to be not an issue (see below). > > I looked at the old Nevada code base of onnv_147 and can't find a reason for > this happening. > > How can I find out what's causing this? New code that seems to be counting wrong.. I was planning on filing a bug, but am currently struggling to convince oracle that we bought support.. Try this: kstat -n zfs_file_data In my case, I get: free15322 mem_inuse 24324866048 mem_total 25753026560 .. where ::memstat says: Kernel2638984 10308 42% ZFS File Data 39260 1531% Anon 873549 3412 14% Exec and libs5199200% Page cache 20019780% Free (cachelist) 6608250% Free (freelist) 2703509 10560 43% On another reboot, it refused to go over 130MB on a 24GB system.. > BTW: I was not seeing this on SolEx 11/10. Dito. > Thanks, > Peter > > > > *** ::memstat *** > Page SummaryPagesMB %Tot > > Kernel 860254 3360 21% > ZFS File Data3047110% > Anon38246 1491% > Exec and libs3765140% > Page cache 8517330% > Free (cachelist) 5866220% > Free (freelist) 3272317 12782 78% > Total 4192012 16375 > Physical 4192011 16375 > > mem_inuse 4145901568 > mem_total 1077466365952 > > *** ::arc *** > hits = 186279921 > misses= 14366462 > demand_data_hits = 4648464 > demand_data_misses= 8605873 > demand_metadata_hits = 171803126 > demand_metadata_misses= 3805675 > prefetch_data_hits=772678 > prefetch_data_misses = 1464457 > prefetch_metadata_hits= 9055653 > prefetch_metadata_misses =490457 > mru_hits = 12295087 > mru_ghost_hits= 0 > mfu_hits = 175281066 > mfu_ghost_hits= 0 > deleted = 14462192 > mutex_miss=30 > hash_elements = 3752768 > hash_elements_max = 3752770 > hash_collisions = 11409790 > hash_chains = 8256 > hash_chain_max=20 > p =48 MB > c = 781 MB > c_min =64 MB > c_max = 15351 MB > size = 788 MB > buf_size = 185 MB > data_size = 289 MB > other_size= 313 MB > l2_hits = 0 > l2_misses = 14366462 > l2_feeds
Re: [zfs-discuss] arc_no_grow is set to 1 and never set back to 0
On 03 January, 2012 - Peter Radig sent me these 3,5K bytes: > Hello. > > I have a Solaris 11/11 x86 box (which I migrated from SolEx 11/10 a couple of > weeks ago). > > Without no obvious reason (at least for me), after an uptime of 1 to 2 days > (observed 3 times now) Solaris sets arc_no_grow to 1 and then never sets it > back to 0. ARC is being shrunk to less than 1 GB -- needless to say that > performance is terrible. There is not much load on this system. > > Memory seems to be not an issue (see below). > > I looked at the old Nevada code base of onnv_147 and can't find a reason for > this happening. > > How can I find out what's causing this? New code that seems to be counting wrong.. I was planning on filing a bug, but am currently struggling to convince oracle that we bought support.. Try this: kstat -n zfs_file_data In my case, I get: free15322 mem_inuse 24324866048 mem_total 25753026560 .. where ::memstat says: Kernel2638984 10308 42% ZFS File Data 39260 1531% Anon 873549 3412 14% Exec and libs5199200% Page cache 20019780% Free (cachelist) 6608250% Free (freelist) 2703509 10560 43% On another reboot, it refused to go over 130MB on a 24GB system.. > BTW: I was not seeing this on SolEx 11/10. Dito. > Thanks, > Peter > > > > *** ::memstat *** > Page SummaryPagesMB %Tot > > Kernel 860254 3360 21% > ZFS File Data3047110% > Anon38246 1491% > Exec and libs3765140% > Page cache 8517330% > Free (cachelist) 5866220% > Free (freelist) 3272317 12782 78% > Total 4192012 16375 > Physical 4192011 16375 > > mem_inuse 4145901568 > mem_total 1077466365952 > > *** ::arc *** > hits = 186279921 > misses= 14366462 > demand_data_hits = 4648464 > demand_data_misses= 8605873 > demand_metadata_hits = 171803126 > demand_metadata_misses= 3805675 > prefetch_data_hits=772678 > prefetch_data_misses = 1464457 > prefetch_metadata_hits= 9055653 > prefetch_metadata_misses =490457 > mru_hits = 12295087 > mru_ghost_hits= 0 > mfu_hits = 175281066 > mfu_ghost_hits= 0 > deleted = 14462192 > mutex_miss=30 > hash_elements = 3752768 > hash_elements_max = 3752770 > hash_collisions = 11409790 > hash_chains = 8256 > hash_chain_max=20 > p =48 MB > c = 781 MB > c_min =64 MB > c_max = 15351 MB > size = 788 MB > buf_size = 185 MB > data_size = 289 MB > other_size= 313 MB > l2_hits = 0 > l2_misses = 14366462 > l2_feeds = 0 > l2_rw_clash = 0 > l2_read_bytes = 0 MB > l2_write_bytes= 0 MB > l2_writes_sent= 0 > l2_writes_done= 0 > l2_writes_error = 0 > l2_writes_hdr_miss= 0 > l2_evict_lock_retry = 0 > l2_evict_reading = 0 > l2_abort_lowmem = 490 > l2_cksum_bad = 0 > l2_io_error = 0 > l2_hdr_size = 0 MB > memory_throttle_count = 0 > meta_used = 499 MB > meta_max = 1154 MB > meta_limit= 0 MB > arc_no_grow = 1 > arc_tempreserve = 0 MB > ___ > zfs-discuss mailing list > zfs-discuss@opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss /Tomas -- Tomas Forsman, st...@acc.umu.se, http://www.acc.umu.se/~stric/ |- Student at Computing Science, University of Umeå `- Sysadmin at {cs,acc}.umu.se ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss