[zfs-discuss] swap weirdness
Hi, We have a host, Solaris 10 10/08 s10s_u6wos_07b on SPARC. SWAP is on ZFS. We allocated two swap devices of 64G each, for a total of around 130G. Over the time, swap space usage is increasing until 96% of used: # zpool list NAMESIZE USEDAVAILCAP HEALTH ALTROOT swap_space 130G 125G 4.94G 96% ONLINE - # zfs list NAME USED AVAIL REFER MOUNTPOINT swap_space 127G 487M 1Knone swap_space/swapvol01 64G2.03G 62.4G- swap_space/swapvol02 63G1.02G 62.5G- ZFS parameters for swap space volume: # zfs get all swap_space NAME PROPERTY VALUE SOURCE swap_space used 127G - swap_space available 487M - swap_space referenced1K - swap_space quotanone default swap_space reservationnone default swap_space compressratio1.00x - [...] In swap_space, I have two volumes, swapvol01 and swapvol02: # zfs get all swap_space/swap_space_01 NAMEPROPERTY VALUE SOURCE swap_space/swapvol01 typevolume - swap_space/swapvol01 creation Thu Mar 11 9:22 2010 - swap_space/swapvol01 used64G- swap_space/swapvol01 available 2.03G - swap_space/swapvol01 referenced 62.4G - swap_space/swapvol01 compressratio 1.00x - swap_space/swapvol01 reservation none default swap_space/swapvol01 volsize 64G- swap_space/swapvol01 volblocksize 8K - swap_space/swapvol01 checksum offinherited from swap_space swap_space/swapvol01 compression offdefault swap_space/swapvol01 readonly offdefault swap_space/swapvol01 shareiscsi offdefault swap_space/swapvol01 copies 1 default swap_space/swapvol01 refreservation 64Glocal # zfs get all swap_space/swap_space_02 NAMEPROPERTY VALUE SOURCE swap_space/swapvol02 typevolume - swap_space/swapvol02 creation Tue Jan 18 9:14 2011 - swap_space/swapvol02 used63G- swap_space/swapvol02 available 1.02G - swap_space/swapvol02 referenced 62.5G - swap_space/swapvol02 compressratio 1.00x - swap_space/swapvol02 reservation none default swap_space/swapvol02 volsize 63G- swap_space/swapvol02 volblocksize 8K - swap_space/swapvol02 checksum offinherited from swap_space swap_space/swapvol02 compression offdefault swap_space/swapvol02 readonly offdefault swap_space/swapvol02 shareiscsi offdefault swap_space/swapvol02 copies 1 default swap_space/swapvol02 refreservation 63Glocal However, if we do swap -s, we have more than 54GB of free space (swap file name is changed for confidentiality): # swap -s total: 51745880k bytes allocated + 39077200k reserved = 90823080k used, 56984152k available # swap -l swapfile dev swaplo blocksfree /dev/swapvol01 256,1 16134217712 120021888 /dev/swapvol02 256,2 16132120560 117956320 I understand that swap -l only show space used on those devices and swap -s include physical memory AND backing store. So logically, we should see a low usage of both swap devices ... ? But, /tmp, /etc/svc/volatile and /var/run directories aren't used: # df -h /tmp /var/run /etc/svc/volatile Filesystem size used avail capacity Mounted on swap54G 8.8M54G 1%/tmp swap54G 232K54G 1%/var/run swap54G 1.6M54G 1%/etc/svc/volatile We have 20 zones but no one is taking many space in those directories (cut many lines for better reading): # for i in `zoneadm list |grep -v global`; do zlogin $i df -h /tmp; done Filesystem size used avail capacity Mounted on swap55G 275M54G 1%/tmp swap55G 270M54G 1%/tmp swap55G
Re: [zfs-discuss] swap weirdness
What does echo ::memstat | mdb -k show? max On Aug 2, 2011, at 4:10 PM, Mister Anonyme wrote: Hi, We have a host, Solaris 10 10/08 s10s_u6wos_07b on SPARC. SWAP is on ZFS. We allocated two swap devices of 64G each, for a total of around 130G. Over the time, swap space usage is increasing until 96% of used: # zpool list NAMESIZE USEDAVAILCAP HEALTH ALTROOT swap_space 130G 125G 4.94G 96% ONLINE - # zfs list NAME USED AVAIL REFER MOUNTPOINT swap_space 127G 487M 1Knone swap_space/swapvol01 64G2.03G 62.4G- swap_space/swapvol02 63G1.02G 62.5G- ZFS parameters for swap space volume: # zfs get all swap_space NAME PROPERTY VALUE SOURCE swap_space used 127G - swap_space available 487M - swap_space referenced1K - swap_space quotanone default swap_space reservationnone default swap_space compressratio1.00x - [...] In swap_space, I have two volumes, swapvol01 and swapvol02: # zfs get all swap_space/swap_space_01 NAMEPROPERTY VALUE SOURCE swap_space/swapvol01 typevolume - swap_space/swapvol01 creation Thu Mar 11 9:22 2010 - swap_space/swapvol01 used64G- swap_space/swapvol01 available 2.03G - swap_space/swapvol01 referenced 62.4G - swap_space/swapvol01 compressratio 1.00x - swap_space/swapvol01 reservation none default swap_space/swapvol01 volsize 64G- swap_space/swapvol01 volblocksize 8K - swap_space/swapvol01 checksum offinherited from swap_space swap_space/swapvol01 compression offdefault swap_space/swapvol01 readonly offdefault swap_space/swapvol01 shareiscsi offdefault swap_space/swapvol01 copies 1 default swap_space/swapvol01 refreservation 64Glocal # zfs get all swap_space/swap_space_02 NAMEPROPERTY VALUE SOURCE swap_space/swapvol02 typevolume - swap_space/swapvol02 creation Tue Jan 18 9:14 2011 - swap_space/swapvol02 used63G- swap_space/swapvol02 available 1.02G - swap_space/swapvol02 referenced 62.5G - swap_space/swapvol02 compressratio 1.00x - swap_space/swapvol02 reservation none default swap_space/swapvol02 volsize 63G- swap_space/swapvol02 volblocksize 8K - swap_space/swapvol02 checksum offinherited from swap_space swap_space/swapvol02 compression offdefault swap_space/swapvol02 readonly offdefault swap_space/swapvol02 shareiscsi offdefault swap_space/swapvol02 copies 1 default swap_space/swapvol02 refreservation 63Glocal However, if we do swap -s, we have more than 54GB of free space (swap file name is changed for confidentiality): # swap -s total: 51745880k bytes allocated + 39077200k reserved = 90823080k used, 56984152k available # swap -l swapfile dev swaplo blocksfree /dev/swapvol01 256,1 16134217712 120021888 /dev/swapvol02 256,2 16132120560 117956320 I understand that swap -l only show space used on those devices and swap -s include physical memory AND backing store. So logically, we should see a low usage of both swap devices ... ? But, /tmp, /etc/svc/volatile and /var/run directories aren't used: # df -h /tmp /var/run /etc/svc/volatile Filesystem size used avail capacity Mounted on swap54G 8.8M54G 1%/tmp swap54G 232K54G 1%/var/run swap54G 1.6M54G 1%/etc/svc/volatile We have 20 zones but no one is taking many space in those directories (cut many lines for better reading): # for i in `zoneadm list |grep -v global`; do zlogin $i df -h /tmp; done
Re: [zfs-discuss] swap weirdness
# echo ::memstat | mdb -k
Page SummaryPagesMB %Tot
Kernel3258470 25456 40%
Anon 4427859 34592 54%
Exec and libs 215008 16793%
Page cache 89353 6981%
Free (cachelist)72263 5641%
Free (freelist)172772 13492%
Total 8235725 64341
Physical 8214142 64172
Subject: Re: [zfs-discuss] swap weirdness
From: m...@bruningsystems.com
Date: Tue, 2 Aug 2011 17:20:49 +0200
CC: zfs-discuss@opensolaris.org
To: benet...@hotmail.com
What does
echo ::memstat | mdb -k
show?
max
On Aug 2, 2011, at 4:10 PM, Mister Anonyme wrote:Hi,
We have a host, Solaris 10 10/08 s10s_u6wos_07b on SPARC.
SWAP is on ZFS.
We allocated two swap devices of 64G each, for a total of around 130G.
Over the time, swap space usage is increasing until 96% of used:
# zpool list
NAMESIZE USEDAVAILCAP HEALTH ALTROOT
swap_space 130G 125G 4.94G 96% ONLINE -
# zfs list
NAME USED
AVAIL REFER MOUNTPOINT
swap_space 127G
487M 1Knone
swap_space/swapvol01 64G2.03G
62.4G-
swap_space/swapvol02 63G1.02G
62.5G-
ZFS parameters for swap space volume:
# zfs get all swap_space
NAME PROPERTY VALUE SOURCE
swap_space used 127G -
swap_space available 487M -
swap_space referenced1K -
swap_space quotanone default
swap_space reservationnone default
swap_space compressratio1.00x -
[...]
In swap_space, I have two volumes, swapvol01 and swapvol02:
# zfs get all swap_space/swap_space_01
NAMEPROPERTY VALUE SOURCE
swap_space/swapvol01 typevolume -
swap_space/swapvol01 creation Thu Mar 11 9:22 2010 -
swap_space/swapvol01 used64G-
swap_space/swapvol01 available 2.03G -
swap_space/swapvol01 referenced 62.4G -
swap_space/swapvol01 compressratio 1.00x -
swap_space/swapvol01 reservation none default
swap_space/swapvol01 volsize 64G-
swap_space/swapvol01 volblocksize 8K -
swap_space/swapvol01 checksum offinherited from
swap_space
swap_space/swapvol01 compression offdefault
swap_space/swapvol01 readonly offdefault
swap_space/swapvol01 shareiscsi offdefault
swap_space/swapvol01 copies 1 default
swap_space/swapvol01 refreservation 64Glocal
# zfs get all swap_space/swap_space_02
NAMEPROPERTY VALUE SOURCE
swap_space/swapvol02 typevolume -
swap_space/swapvol02 creation Tue Jan 18 9:14 2011 -
swap_space/swapvol02 used63G-
swap_space/swapvol02 available 1.02G -
swap_space/swapvol02 referenced 62.5G -
swap_space/swapvol02 compressratio 1.00x -
swap_space/swapvol02 reservation none default
swap_space/swapvol02 volsize 63G-
swap_space/swapvol02 volblocksize 8K -
swap_space/swapvol02 checksum offinherited from
swap_space
swap_space/swapvol02 compression offdefault
swap_space/swapvol02 readonly offdefault
swap_space/swapvol02 shareiscsi offdefault
swap_space/swapvol02 copies 1 default
swap_space/swapvol02 refreservation 63Glocal
However, if we do swap -s, we have more than 54GB of free space (swap file name
is changed for confidentiality):
# swap -s
total: 51745880k bytes allocated + 39077200k reserved = 90823080k used,
56984152k available
# swap -l
swapfile dev swaplo blocksfree
/dev/swapvol01 256,1 16134217712 120021888
/dev/swapvol02 256,2 16132120560 117956320
I understand that swap -l only show space used on those devices and swap -s
include
[zfs-discuss] Wrong rpool used after reinstall!
I am having a problem after a new install of Solaris 10. The installed rpool works fine when I have only those disks connected. When I connect disks from an rpool I created during a previous installation, my newly installed rpool is ignored even though the BIOS (x86) is set to boot only from the new rpool drives. When the system starts it uses the old rpool! How can I get around this? I want to use my new install and then import the old rpool under a new name, and clean things up. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Wrong rpool used after reinstall!
I am having a problem after a new install of Solaris 10. The installed rpool works fine when I have only those disks connected. When I connect disks from an rpool I created during a previous installation, my newly installed rpool is ignored even though the BIOS (x86) is set to boot only from the new rpool drives. When the system starts it uses the old rpool! How can I get around this? I want to use my new install and then import the old rpool under a new name, and clean things up. Boot on the install CD, import the old rpool (leave the new rpool drives unplugged) and export it again. Once you have rebooted on the new rpool, you can reimport the old rpool with a new name (rpool import someid newname). Keep in mind that reimporting an rpool may render it useless for booting, at least that's not optimal. Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 r...@karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Wrong rpool used after reinstall!
Hi Roy, things got alot worse since my first email. I don't know what happened but I can't import the old pool at all. It shows no errors but when I import it I get a kernel panic from assertion failed: zvol_get_stats(os, nv) which looks like is fixed by patch 6801926 which is applied in Solaris 10u9. But I cannot boot update 9 on this box! I tried Solaris Express, none of those will run right either. They all go into maintenance mode. The last thing I can boot is update 8 and that is the one with the ZFS bug. I have 200G of files I deliberately copied to this new mirror and now I can't get at them! Any ideas? Thanks. Jim ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Wrong rpool used after reinstall!
On Wed, Aug 3, 2011 at 8:38 AM, Anonymous Remailer (austria) mixmas...@remailer.privacy.at wrote: Hi Roy, things got alot worse since my first email. I don't know what happened but I can't import the old pool at all. It shows no errors but when I import it I get a kernel panic from assertion failed: zvol_get_stats(os, nv) which looks like is fixed by patch 6801926 which is applied in Solaris 10u9. But I cannot boot update 9 on this box! I tried Solaris Express, none of those will run right either. They all go into maintenance mode. The last thing I can boot is update 8 and that is the one with the ZFS bug. If they go into maintenance mode but could recognize the disk, you should still be able to do zfs stuff (zpool import, etc). If you're lucky you'd only miss the GUI I have 200G of files I deliberately copied to this new mirror and now I can't get at them! Any ideas? Another thing you can try (albeit more complex) is use another OS (install, or even use a linux Live CD), install virtualbox or similar on it, pass the disk as raw vmdk, and use solaris express on the VM. You should be able to at least import the pool and recover the data. -- Fajar ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
