detailed map of WIRED memory under FreeBSD 9
what tool and how can be used to display detailed map what exactly wired memory on my system as it is far way too much (1.5GB out of 4GB RAM). i do run 4 virtualboxes but one have 256MB RAM, the others 192 and when i turn them off wired memory goes down right amount but still it is too much used. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
SuperPages utilization survey
hello, I was wondering how much usage superpages get in real-world systems, and made a small script to parse the output of procstat -va: http://people.freebsd.org/~ivoras/stuff/spsurvey.py The results from three systems (with the script being run as root) are here: http://people.freebsd.org/~ivoras/stuff/spsurvey_desktop.txt http://people.freebsd.org/~ivoras/stuff/spsurvey_mixserver.txt http://people.freebsd.org/~ivoras/stuff/spsurvey_webserver.txt What I get from it is that they are really under-utilized, probably because it's a rare occasion that every single page in a 2 MB region is touched to enable its promotion. The only good case seems to be the third one, with the database accessing the whole memory range a lot, but the statistics which procstat reports is inaccurate: there could be only a single superpage in the whole region and procstat will make the region with the S flag. If there's anyone else wishing to run the script and post the results, it could be useful to see. signature.asc Description: OpenPGP digital signature
Re: SuperPages utilization survey
http://people.freebsd.org/~ivoras/stuff/spsurvey.py The results from three systems (with the script being run as root) are here: http://people.freebsd.org/~ivoras/stuff/spsurvey_desktop.txt http://people.freebsd.org/~ivoras/stuff/spsurvey_mixserver.txt http://people.freebsd.org/~ivoras/stuff/spsurvey_webserver.txt your webserver is actually database serwer with addons. mixserver - mix of what? My mixserver have certainly far different mix of your mixserver, as i don't use python heavily for example, while use squid, clamav (both fits well in superpages), and don't run large memory postgres process. What is desktop. A computer sitting on the desk? May run a lot of different programs. Your desktop as i can see use KDE bloatware and postgres. As for me, such namings are completely imprecise and such statistics say NOTHING. What REAL knowledge i acquired from your work is that postgres are very well fit for superpage mapping, if large DB buffer is used. Actually - Whenever there are long running processes in the system that allocate and use large memory chunks then superpage promotion will work. My idea - lets do a survey based on PROCESS NAME. this will give a really meaningful information. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: SuperPages utilization survey
On 1 June 2012 14:35, Wojciech Puchar woj...@wojtek.tensor.gdynia.pl wrote: http://people.freebsd.org/~ivoras/stuff/spsurvey.py The results from three systems (with the script being run as root) are here: http://people.freebsd.org/~ivoras/stuff/spsurvey_desktop.txt http://people.freebsd.org/~ivoras/stuff/spsurvey_mixserver.txt http://people.freebsd.org/~ivoras/stuff/spsurvey_webserver.txt your webserver is actually database serwer with addons. Database + apache + a lot of fairly large FastCGI php processes - which surprisingly don't use superpages. mixserver - mix of what? PostgreSQL, apache, php, dovecot, spamassasin, python web apps, and a lot of other things - I'm sure you can conclude from the list of processes. My mixserver have certainly far different mix of your mixserver, as i don't use python heavily for example, while use squid, clamav (both fits well in superpages), and don't run large memory postgres process. I'd like to learn more - can you post the results from your own server? What is desktop. A computer sitting on the desk? May run a lot of different programs. Your desktop as i can see use KDE bloatware and postgres. Yes, except for postgres which is used rarely, it's a fairly typical KDE desktop. As for me, such namings are completely imprecise and such statistics say NOTHING. Yes, a survey of three machines means nothing. I'm looking for more data. My idea - lets do a survey based on PROCESS NAME. this will give a really meaningful information. If anyone posts more data, I'll analyse it. I'm more worried about the granularity of procstat, where it marks the entire region if a single superpage exists in it - it means any such analysis is only approximate. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: detailed map of WIRED memory under FreeBSD 9
On 01/06/2012 10:19, Wojciech Puchar wrote: what tool and how can be used to display detailed map what exactly wired memory on my system as it is far way too much (1.5GB out of 4GB RAM). Do you use ZFS? signature.asc Description: OpenPGP digital signature
Re: detailed map of WIRED memory under FreeBSD 9
On Jun 1, 2012, at 1:19 AM, Wojciech Puchar woj...@wojtek.tensor.gdynia.pl wrote: what tool and how can be used to display detailed map what exactly wired memory on my system as it is far way too much (1.5GB out of 4GB RAM). dmidecode? -- Devin i do run 4 virtualboxes but one have 256MB RAM, the others 192 and when i turn them off wired memory goes down right amount but still it is too much used. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org _ The information contained in this message is proprietary and/or confidential. If you are not the intended recipient, please: (i) delete the message and all copies; (ii) do not disclose, distribute or use the message in any manner; and (iii) notify the sender immediately. In addition, please be aware that any message addressed to our domain is subject to archiving and review by persons other than the intended recipient. Thank you. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: detailed map of WIRED memory under FreeBSD 9
no it isn't. Problem solved - virtualbox is THAT bad allocates more memory than needed. On Fri, 1 Jun 2012, Teske, Devin wrote: On Jun 1, 2012, at 1:19 AM, Wojciech Puchar woj...@wojtek.tensor.gdynia.pl wrote: what tool and how can be used to display detailed map what exactly wired memory on my system as it is far way too much (1.5GB out of 4GB RAM). dmidecode? -- Devin i do run 4 virtualboxes but one have 256MB RAM, the others 192 and when i turn them off wired memory goes down right amount but still it is too much used. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org _ The information contained in this message is proprietary and/or confidential. If you are not the intended recipient, please: (i) delete the message and all copies; (ii) do not disclose, distribute or use the message in any manner; and (iii) notify the sender immediately. In addition, please be aware that any message addressed to our domain is subject to archiving and review by persons other than the intended recipient. Thank you. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
dtrace filename lookups from fd
As a first foray into dtrace I wanted to create a little script which shows the amount of disk read / write activity. Now the DtraceToolkit includes rwsnoop but this uses Solaris specific requests and on looking around it seems like using rwsnoop vn_fullpath may be the way to go. Has anyone done anything similar to this before or has any tips on going about this? Regards Steve This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmas...@multiplay.co.uk. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: SuperPages utilization survey
On Fri, 2012-06-01 at 14:23:42 +0200, Ivan Voras wrote: hello, I was wondering how much usage superpages get in real-world systems, and made a small script to parse the output of procstat -va: http://people.freebsd.org/~ivoras/stuff/spsurvey.py The results from three systems (with the script being run as root) are here: http://people.freebsd.org/~ivoras/stuff/spsurvey_desktop.txt http://people.freebsd.org/~ivoras/stuff/spsurvey_mixserver.txt http://people.freebsd.org/~ivoras/stuff/spsurvey_webserver.txt What I get from it is that they are really under-utilized, probably because it's a rare occasion that every single page in a 2 MB region is touched to enable its promotion. The only good case seems to be the third one, with the database accessing the whole memory range a lot, but the statistics which procstat reports is inaccurate: there could be only a single superpage in the whole region and procstat will make the region with the S flag. If there's anyone else wishing to run the script and post the results, it could be useful to see. Here's output of a machine doing basically nothing all day: % fetch -o- http://people.freebsd.org/\~ivoras/stuff/spsurvey.py | sudo python - - 100% of 2035 B 664 kBps last pid: 20460; load averages: 0.04, 0.01, 0.00 up 2+01:35:3721:01:08 49 processes: 1 running, 48 sleeping Mem: 104M Active, 2079M Inact, 1593M Wired, 34M Cache, 418M Buf, 133M Free Swap: 4096M Total, 1376K Used, 4095M Free Total accounted memory mappings: 1669 MB (427314 pages) Memory in superpages: 12 MB (2 mappings) + pid: 864 (named) start: 80280 stop: 80300 (8 MB) tp: df path: + pid: 1002 (slapd) start: 80540 stop: 80580 (4 MB) tp: df path: Eligible mappings not promoted: 66 ... Also, what about kernel mappings? With ZFS and stuff there should be more superpages in kernel memory, no? Uli ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: SuperPages utilization survey
[/usr/home/feld]# python spsurvey.py last pid: 54743; load averages: 0.28, 0.26, 0.24 up 18+07:41:02 16:22:45 145 processes: 1 running, 144 sleeping Mem: 828M Active, 845M Inact, 8517M Wired, 174M Cache, 725M Buf, 265M Free Swap: 4096M Total, 88M Used, 4008M Free, 2% Inuse Total accounted memory mappings: 23968 MB (6136043 pages) Memory in superpages: 12941 MB (14 mappings) + pid: 26349 (postgres) start: 80280 stop: 8452c6000 (1066 MB) tp: ph path: + pid: 26351 (postgres) start: 80280 stop: 8452c6000 (1066 MB) tp: ph path: + pid: 26352 (postgres) start: 80280 stop: 8452c6000 (1066 MB) tp: ph path: + pid: 26353 (postgres) start: 80280 stop: 8452c6000 (1066 MB) tp: ph path: + pid: 26374 (postgres) start: 80280 stop: 8452c6000 (1066 MB) tp: ph path: + pid: 26382 (postgres) start: 80280 stop: 8452c6000 (1066 MB) tp: ph path: + pid: 26387 (postgres) start: 80280 stop: 8452c6000 (1066 MB) tp: ph path: + pid: 26388 (postgres) start: 80280 stop: 8452c6000 (1066 MB) tp: ph path: + pid: 26398 (postgres) start: 80280 stop: 8452c6000 (1066 MB) tp: ph path: + pid: 44306 (odasrv) start: 80180 stop: 80600 (72 MB) tp: df path: + pid: 44318 (odasrv) start: 80180 stop: 805c0 (68 MB) tp: df path: + pid: 53549 (postgres) start: 80280 stop: 8452c6000 (1066 MB) tp: ph path: + pid: 53932 (postgres) start: 80280 stop: 8452c6000 (1066 MB) tp: ph path: + pid: 54515 (postgres) start: 80280 stop: 8452c6000 (1066 MB) tp: ph path: Eligible mappings not promoted: 413 Yup, seems like Postgres does a good job of using superpages ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
