Re: kmem_map too small: 3832475648 total allocated
On Friday 30 April 2010 09:11:49 Gary Jennejohn wrote: A number of variables go into calculating vm.kmem_size (see kmeminit() in kern_malloc.c). In the end, the kernel won't allocate more than twice the physical memory size _which it has discovered_. The question is, how much of your physical memory does the kernel actually see? It would appear that the maximum value for vm.kmem_size is twice the amount of avail memory - as opposed to the real memory. I have 6GB installed; 6144MB gets detected but only 5873MB is available. Setting vm.kmem_size to 12G fails but 10G works. -- Bruce Cran ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: kmem_map too small: 3832475648 total allocated
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 10-5-2010 20:09, Bruce Cran wrote: On Friday 30 April 2010 09:11:49 Gary Jennejohn wrote: A number of variables go into calculating vm.kmem_size (see kmeminit() in kern_malloc.c). In the end, the kernel won't allocate more than twice the physical memory size _which it has discovered_. The question is, how much of your physical memory does the kernel actually see? It would appear that the maximum value for vm.kmem_size is twice the amount of avail memory - as opposed to the real memory. I have 6GB installed; 6144MB gets detected but only 5873MB is available. Setting vm.kmem_size to 12G fails but 10G works. real memory = 8589934592 (8192 MB) avail memory = 8257212416 (7874 MB) So I'm setting it to 14G and reboot. (Which is a pain on my Areca card, since it keeps losing disks bu thatt is for another topic) Ans now it gives me: vm.kmem_size: 15032385536 So perhaps somebody should anotate /boot/defaults/loader.conf and/or list this op more positions.. - --WjW -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.10 (MingW32) iQEcBAEBAgAGBQJL6GM8AAoJEP4k4K6R6rBhUEkH/30RkWpqcNnxK3rc0TTl1TWw vNDxNRx+daq3lTjq7H21aM4mblmqhyTDTE+8+JtZA8Imy2hDpgK5dna4YVz5fLX+ Z5bFod2nVFlY1JfsCVpJTP0w2mdlH+DqYFh1C0bqeVLczc9W8BwPPrs8ucatmy2y Fhtgh3T4wKDkb43jneWVxOBpzjXUR3AekvFyL1Dh52WS65KjfgyRPsBnISYs6+iO /ysB2DGXjBwEuPxlEKXWItW/ghA3bnLRGsCqzu16rULIDRaDr2G85sRkMcR0CmSn NNodAW1qOQetmV7RUkwejZqPsxl/pumIK4CbCVjz9hDoXSrApPJE8O4spZyvZiA= =c4Bz -END PGP SIGNATURE- ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: kmem_map too small: 3832475648 total allocated
On Fri, 30 Apr 2010 00:27:00 +0200 Willem Jan Withagen w...@digiware.nl wrote: On 29-4-2010 6:17, Artem Belevich wrote: Do you have vm.kmem_size set in /boot/loader.conf? If not, do set it to about double of your physical RAM size. Defaults are way too conservative for use with large amounts of memory and ZFS. As per this suggestion I set this value to 2*8G: vm.kmem_size=17179869184 And bfore the kernel boots this value is set in the loader. Tested it by goinginto the loader and show the value. But once booted I still get: [/boot] w...@zfs.digiware.nl sysctl vm.kmem_size vm.kmem_size: 3718209536 So who is resetting this value??? A number of variables go into calculating vm.kmem_size (see kmeminit() in kern_malloc.c). In the end, the kernel won't allocate more than twice the physical memory size _which it has discovered_. The question is, how much of your physical memory does the kernel actually see? -- Gary Jennejohn ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re[2]: kmem_map too small: 3832475648 total allocated
On Wed, Apr 28, 2010 at 8:07 PM, James R. Van Artsdalen james-freebsd-curr...@jrv.org wrote: FreeBSD bigback.housenet.jrv 9.0-CURRENT FreeBSD 9.0-CURRENT #1 r206111: Mon Apr 26 01:13:00 CDT 2010 r...@bigback.housenet.jrv:/usr/obj/usr/src/sys/GENERIC ?amd64 svn 206111 is April 2 system is a Core i7 975 (3.33 GHz x 4 cores 3x threads per core) with 12 GB of RAM, a 2x2TB ZFS boot pool and a second (idle) pool of 16x2TB. The system was running this in the clang build area: while true do ?make -j7 buildworld done The panic happened during the 27th iteration. panic: kmem_malloc(131072): kmem_map too small: 3832475648 total allocated I open PR amd64/145654 with problem like it. Have you increasing Wired memory at buildworld ? ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: kmem_map too small: 3832475648 total allocated
According to James R. Van Artsdalen: system is a Core i7 975 (3.33 GHz x 4 cores 3x threads per core) with 12 GB of RAM, a 2x2TB ZFS boot pool and a second (idle) pool of 16x2TB. panic: kmem_malloc(131072): kmem_map too small: 3832475648 total allocated Apart from the fact that you must at least set vm.kmem_size to something like 2x your RAM, one rule of thumb I've seen discussed for ZFS is that you will need approximatively 1 GB of RAM per TB of data so you may be a bit short here to get optimal perfs. -- Ollivier ROBERT -=- FreeBSD: The Power to Serve! -=- robe...@keltia.freenix.fr In memoriam to Ondine : http://ondine.keltia.net/ ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: kmem_map too small: 3832475648 total allocated
On Thu, Apr 29, 2010 at 3:53 PM, Ollivier Robert robe...@keltia.freenix.fr wrote: According to James R. Van Artsdalen: system is a Core i7 975 (3.33 GHz x 4 cores 3x threads per core) with 12 GB of RAM, a 2x2TB ZFS boot pool and a second (idle) pool of 16x2TB. panic: kmem_malloc(131072): kmem_map too small: 3832475648 total allocated Apart from the fact that you must at least set vm.kmem_size to something like 2x your RAM, one rule of thumb I've seen discussed for ZFS is that you will need approximatively 1 GB of RAM per TB of data so you may be a bit short here to get optimal perfs. Citation needed? I have a file server running amd64 8-STABLE with 4GB of RAM, 6 x 1.5 TB drives in raidz, and have never had any problems with memory usage. Are you saying that after my next update, adding another 6 x 1.5 TB drives, it will start being flaky and/or panicing with kmem_map too small errors? Cheers Tom ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: kmem_map too small: 3832475648 total allocated
On Apr 29, 2010, at 9:44 AM, Tom Evans wrote: On Thu, Apr 29, 2010 at 3:53 PM, Ollivier Robert robe...@keltia.freenix.fr wrote: According to James R. Van Artsdalen: system is a Core i7 975 (3.33 GHz x 4 cores 3x threads per core) with 12 GB of RAM, a 2x2TB ZFS boot pool and a second (idle) pool of 16x2TB. panic: kmem_malloc(131072): kmem_map too small: 3832475648 total allocated Apart from the fact that you must at least set vm.kmem_size to something like 2x your RAM, one rule of thumb I've seen discussed for ZFS is that you will need approximatively 1 GB of RAM per TB of data so you may be a bit short here to get optimal perfs. Citation needed? I have a file server running amd64 8-STABLE with 4GB of RAM, 6 x 1.5 TB drives in raidz, and have never had any problems with memory usage. Are you saying that after my next update, adding another 6 x 1.5 TB drives, it will start being flaky and/or panicing with kmem_map too small errors? I'm sorry, but I find it absolutely absurd that any filesystem has to wire down 2GB of RAM, and that the solution to panics is buy more RAM. Scott ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: kmem_map too small: 3832475648 total allocated
If I understand it correctly, the problem in this case is that even when you do have more than enough RAM, kernel just does not provide enough space in kmem_map to map that physical memory into. Unless you want to have dedupe turned on large filesystem (and FreeBSD does not have this feature yet), you don't really *need* all that much RAM. kmem_map too small comes up as an issue way more often than lack of physical memory. --Artem On Thu, Apr 29, 2010 at 9:56 AM, Scott Long sco...@samsco.org wrote: On Apr 29, 2010, at 9:44 AM, Tom Evans wrote: On Thu, Apr 29, 2010 at 3:53 PM, Ollivier Robert robe...@keltia.freenix.fr wrote: According to James R. Van Artsdalen: system is a Core i7 975 (3.33 GHz x 4 cores 3x threads per core) with 12 GB of RAM, a 2x2TB ZFS boot pool and a second (idle) pool of 16x2TB. panic: kmem_malloc(131072): kmem_map too small: 3832475648 total allocated Apart from the fact that you must at least set vm.kmem_size to something like 2x your RAM, one rule of thumb I've seen discussed for ZFS is that you will need approximatively 1 GB of RAM per TB of data so you may be a bit short here to get optimal perfs. Citation needed? I have a file server running amd64 8-STABLE with 4GB of RAM, 6 x 1.5 TB drives in raidz, and have never had any problems with memory usage. Are you saying that after my next update, adding another 6 x 1.5 TB drives, it will start being flaky and/or panicing with kmem_map too small errors? I'm sorry, but I find it absolutely absurd that any filesystem has to wire down 2GB of RAM, and that the solution to panics is buy more RAM. Scott ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: kmem_map too small: 3832475648 total allocated
Андрей Смагин wrote: panic: kmem_malloc(131072): kmem_map too small: 3832475648 total allocated I open PR amd64/145654 with problem like it. Have you increasing Wired memory at buildworld ? No. This # while true do sysctl -A | grep wired make -j8 buildworld done is getting: vm.max_wired: 997427 vm.max_wired: 997427 vm.max_wired: 997427 vm.max_wired: 997427 vm.max_wired: 997427 vm.max_wired: 997427 vm.max_wired: 997427 vm.max_wired: 997427 vm.max_wired: 997427 vm.max_wired: 997427 vm.max_wired: 997427 vm.max_wired: 997427 Is that the right variable to look at? A friend of mine can duplicate your PR too. ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: kmem_map too small: 3832475648 total allocated
Scott Long wrote: I'm sorry, but I find it absolutely absurd that any filesystem has to wire down 2GB of RAM, and that the solution to panics is buy more The only thing buying more RAM is likely to do is make the memory leak behind this harder to reproduce. Likewise increasing kernel tunables. It may only be a short-term leak, i.e. something that would be recovered normally. Does FreeBSD have a kernel debug facility for this? Is there a way to look at a kernel dump file and identify by caller who allocated space? ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: kmem_map too small: 3832475648 total allocated
According to Tom Evans: Citation needed? I have a file server running amd64 8-STABLE with 4GB of RAM, 6 x 1.5 TB drives in raidz, and have never had any problems with memory usage. Are you saying that after my next update, adding another 6 x 1.5 TB drives, it will start being flaky and/or panicing with kmem_map too small errors? I don't have the citation handy but it was on the opensolaris forum in the zfs community. I understand that this is FreeBSD we're speaking about but the figure comes from the fact that ZFS (and particularly the ARC cache) does take a lot of memory. -- Ollivier ROBERT -=- FreeBSD: The Power to Serve! -=- robe...@keltia.freenix.fr In memoriam to Ondine : http://ondine.keltia.net/ ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re[2]: kmem_map too small: 3832475648 total allocated
Thu, 29 Apr 2010 16:20:32 -0500 письмо от James R. Van Artsdalen james-freebsd-curr...@jrv.org: Андрей Смагин wrote: panic: kmem_malloc(131072): kmem_map too small: 3832475648 total allocated I open PR amd64/145654 with problem like it. Have you increasing Wired memory at buildworld ? No. This # while true do sysctl -A | grep wired make -j8 buildworld done is getting: vm.max_wired: 997427 vm.max_wired: 997427 vm.max_wired: 997427 vm.max_wired: 997427 vm.max_wired: 997427 vm.max_wired: 997427 vm.max_wired: 997427 vm.max_wired: 997427 vm.max_wired: 997427 vm.max_wired: 997427 vm.max_wired: 997427 vm.max_wired: 997427 Is that the right variable to look at? strange - by top Wired memory now about 1.2Gbyte, but vm.max_wired: 330338 Also my system crashed after 3-5 buildworld's and kernels. usr/obj+usr/src - ~2GB size, 4GB of RAM - twice more than need to have all data for compilation in memory. A friend of mine can duplicate your PR too. ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: kmem_map too small: 3832475648 total allocated
On 29-4-2010 6:17, Artem Belevich wrote: Do you have vm.kmem_size set in /boot/loader.conf? If not, do set it to about double of your physical RAM size. Defaults are way too conservative for use with large amounts of memory and ZFS. As per this suggestion I set this value to 2*8G: vm.kmem_size=17179869184 And bfore the kernel boots this value is set in the loader. Tested it by goinginto the loader and show the value. But once booted I still get: [/boot] w...@zfs.digiware.nl sysctl vm.kmem_size vm.kmem_size: 3718209536 So who is resetting this value??? Thanx, -WjW ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
kmem_map too small: 3832475648 total allocated
FreeBSD bigback.housenet.jrv 9.0-CURRENT FreeBSD 9.0-CURRENT #1 r206111: Mon Apr 26 01:13:00 CDT 2010 r...@bigback.housenet.jrv:/usr/obj/usr/src/sys/GENERIC amd64 svn 206111 is April 2 system is a Core i7 975 (3.33 GHz x 4 cores 3x threads per core) with 12 GB of RAM, a 2x2TB ZFS boot pool and a second (idle) pool of 16x2TB. The system was running this in the clang build area: while true do make -j7 buildworld done The panic happened during the 27th iteration. panic: kmem_malloc(131072): kmem_map too small: 3832475648 total allocated ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: kmem_map too small: 3832475648 total allocated
Do you have vm.kmem_size set in /boot/loader.conf? If not, do set it to about double of your physical RAM size. Defaults are way too conservative for use with large amounts of memory and ZFS. --Artem On Wed, Apr 28, 2010 at 8:07 PM, James R. Van Artsdalen james-freebsd-curr...@jrv.org wrote: FreeBSD bigback.housenet.jrv 9.0-CURRENT FreeBSD 9.0-CURRENT #1 r206111: Mon Apr 26 01:13:00 CDT 2010 r...@bigback.housenet.jrv:/usr/obj/usr/src/sys/GENERIC amd64 svn 206111 is April 2 system is a Core i7 975 (3.33 GHz x 4 cores 3x threads per core) with 12 GB of RAM, a 2x2TB ZFS boot pool and a second (idle) pool of 16x2TB. The system was running this in the clang build area: while true do make -j7 buildworld done The panic happened during the 27th iteration. panic: kmem_malloc(131072): kmem_map too small: 3832475648 total allocated ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
