superpages and kmem on amd64
Hi all, I'm playing with an algorithm which makes use of large contiguous blocks of kernel memory (ranging from 1M to 1G in size), so it would be nice if those could be somehow forcibly mapped to superpages. I was hoping that the VM system would automagically map (merge) contiguous 4k pages to superpages, but apparently it doesn't: vm.pmap.pdpe.demotions: 2 vm.pmap.pde.promotions: 543 vm.pmap.pde.p_failures: 266253 vm.pmap.pde.mappings: 0 vm.pmap.pde.demotions: 31 I.e. I have 1G of kmem allocated using via malloc(1024 * 1024 * 1024, M_TEMP, M_NOWAIT); but vm.pmap.pde.mappings: 0 suggests that no superpages are in use. Is there an alternative kernel memory allocation method which might force superpages to be used for contiguous memory blocks? And how do I find more details about page mappings for a given kmem virtual address? I'm running 8.3-STABLE on amd64. Thanks, Marko ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: superpages and kmem on amd64
On Sun, May 20, 2012 at 2:01 AM, Marko Zec z...@fer.hr wrote: Hi all, I'm playing with an algorithm which makes use of large contiguous blocks of kernel memory (ranging from 1M to 1G in size), so it would be nice if those could be somehow forcibly mapped to superpages. I was hoping that the VM system would automagically map (merge) contiguous 4k pages to superpages, but apparently it doesn't: vm.pmap.pdpe.demotions: 2 vm.pmap.pde.promotions: 543 vm.pmap.pde.p_failures: 266253 vm.pmap.pde.mappings: 0 vm.pmap.pde.demotions: 31 No, your conclusion is incorrect. These counts show that 543 superpage mappings were created by promotion. Alan ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: Radeon, DRM and crash on 9.0
On Sat, May 19, 2012 at 9:40 PM, Andriy Gapon a...@freebsd.org wrote: on 19/05/2012 17:52 Fernando Apesteguía said the following: Hi, I'm having some system crashes from time to time. I had this before but until recently I couldn't set my system so I could get crash dumps. My video card is a ATI Mobility Radeon 9700. I'm running FreeBSD 9.0-RELEASE for amd64. These are excerpts from two crash dumps text files: core.txt.3: Fatal trap 28: machine check trap while in kernel mode cpuid = 0; apic id = 00 instruction pointer = 0x20:0x816480a3 stack pointer = 0x28:0xff804a5eb970 frame pointer = 0x28:0xff804a5eb990 code segment = base 0x0, limit 0xf, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags = interrupt enabled, IOPL = 3 current process = 2254 (Xorg) trap number = 28 panic: machine check trap cpuid = 0 KDB: stack backtrace: #0 0x80869abe at kdb_backtrace+0x5e #1 0x80833fb7 at panic+0x187 #2 0x80b18b80 at trap_fatal+0x290 #3 0x80b190c0 at trap+0x110 #4 0x80b0396f at calltrap+0x8 #5 0x816a305b at drm_ioctl+0x31b #6 0x8075597b at devfs_ioctl_f+0x7b #7 0x8087afb1 at kern_ioctl+0x111 #8 0x8087b1df at sys_ioctl+0xef #9 0x80b18480 at amd64_syscall+0x450 #10 0x80b03c57 at Xfast_syscall+0xf7 Unread portion of the kernel message buffer: MCA: Bank 4, Status 0xb2070f0f MCA: Global Cap 0x0105, Status 0x0004 MCA: Vendor AuthenticAMD, ID 0xf4a, APIC ID 0 MCA: CPU 0 UNCOR PCC BUSLG ??? ERR Other timed out Did you notice that you were getting the machine check exceptions? You might want to google for this term. Anyway, there is sysutils/mcelog port and this is how mcelog utility decodes the above report: HARDWARE ERROR. This is *NOT* a software problem! Please contact your hardware vendor CPU 0 4 northbridge Northbridge Watchdog error bit57 = processor context corrupt bit61 = error uncorrected bus error 'generic participation, request timed out generic error mem transaction generic access, level generic' STATUS b2070f0f MCGSTATUS 4 MCGCAP 105 APICID 0 SOCKETID 0 CPUID Vendor AMD Family 15 Model 4 Thanks for the reply. I found some information about this specific error. It seems it could be caused by overheating (among other things), and it's true this laptop can become really hot at times. I'll have a look at it. Thanks! core.txt.4 Fatal trap 28: machine check trap while in kernel mode cpuid = 0; apic id = 00 instruction pointer = 0x20:0x816462b6 stack pointer = 0x28:0xff804a5eb930 frame pointer = 0x28:0xff804a5eb940 code segment = base 0x0, limit 0xf, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags = interrupt enabled, IOPL = 3 current process = 2254 (Xorg) trap number = 28 panic: machine check trap cpuid = 0 KDB: stack backtrace: #0 0x80869abe at kdb_backtrace+0x5e #1 0x80833fb7 at panic+0x187 #2 0x80b18b80 at trap_fatal+0x290 #3 0x80b190c0 at trap+0x110 #4 0x80b0396f at calltrap+0x8 #5 0x8164f3cc at radeon_cp_indirect+0x24c #6 0x816a305b at drm_ioctl+0x31b #7 0x8075597b at devfs_ioctl_f+0x7b #8 0x8087afb1 at kern_ioctl+0x111 #9 0x8087b1df at sys_ioctl+0xef #10 0x80b18480 at amd64_syscall+0x450 #11 0x80b03c57 at Xfast_syscall+0xf7 dmesg | grep agp agp0: VIA 8385 host to PCI bridge on hostb0 drm.ko is loaded and agp is included in kernel. AGP for the card seems to be properly detected: dmesg | grep drm drm0: ATI Radeon RV350 Mobility 9600 M10 NP on vgapci0 info: [drm] AGP at 0xe000 256MB info: [drm] Initialized radeon 1.31.0 20080613 info: [drm] Setting GART location based on new memory map info: [drm] Loading R300 Microcode info: [drm] Num pipes: 1 grep -i Direct rendering /var/log/Xorg.0.log (II) RADEON(0): Direct rendering enabled The crash is not easily reproducible but seems to be more likely to occur the more activity there is in the screen (like when scrolling a window quite fast). Any help is appreciated. Thanks in advance. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org -- Andriy Gapon ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: Booting Ubuntu and freebsd side by side
Hi, On 2012-05-19 20:59, سید احمد حسینی wrote: I used boot0 with boot9cfg : boot0cfg -B -b /boot/boot0 ada0 And I can see boot0 menu ,but Ubuntu can't boot! On May 19, 2012 11:26 PM, سید احمد حسینیahmad...@gmail.com wrote: You should make sure that GRUB is installed in the PBR (partition boot record) of your ubuntu partition and not in the MBR since boot0 lives there. I've done that with Linux Mint Debian Edition - and it works. /Uffe I used boot0 with boot9cfg : boot0cfg -B -b /boot/boot0 And I can see boot0 menu ,but Ubuntu can't boot! On May 19, 2012 11:17 PM, User Wojtekwoj...@wojtek.tensor.gdynia.pl wrote: How can I boot Ubuntu12.4 do with FreeBSD9 ? when I using boot0 , Linux was shown, but would not start without a message! Where is the problem? can launch freebsd with Ubuntu GRUB? How? install ubuntu (or whatever) on one MBR partition, FreeBSD on another then install partition selector with boot0cfg -B /dev/yourdisk and select F1...F4 at boot. that simple ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: superpages and kmem on amd64
On Sunday 20 May 2012 09:25:59 Alan Cox wrote: On Sun, May 20, 2012 at 2:01 AM, Marko Zec z...@fer.hr wrote: Hi all, I'm playing with an algorithm which makes use of large contiguous blocks of kernel memory (ranging from 1M to 1G in size), so it would be nice if those could be somehow forcibly mapped to superpages. I was hoping that the VM system would automagically map (merge) contiguous 4k pages to superpages, but apparently it doesn't: vm.pmap.pdpe.demotions: 2 vm.pmap.pde.promotions: 543 vm.pmap.pde.p_failures: 266253 vm.pmap.pde.mappings: 0 vm.pmap.pde.demotions: 31 No, your conclusion is incorrect. These counts show that 543 superpage mappings were created by promotion. OK, that sounds promising. Does created by promotion count reflect historic / cumulative stats, or is vm.pmap.pde.promotions the actual number of superpages active? Or should we subtract vm.pmap.pde.demotions from it to get the current value? In any case, I wish to be certain that a particular kmem virtual address range is mapped to superpages - how can I enforce that at malloc time, and / or find out later if I really got my kmem mapped to superpages? Perhaps vm_map_lookup() could provide more info, but I'm wondering if someone already wrote a wrapper function for that, which takes only the base virtual address as a single argument? BTW, apparently malloc(size, M_TEMP, M_NOWAIT) requests fail for size 1G, even at boot time. Any ideas how to circumvent that (8.3-STABLE, amd64, 4G physical RAM)? Thanks, Marko ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: Booting Ubuntu and freebsd side by side
On 20 May 2012 15:08, Uffe Jakobsen u...@uffe.org wrote: Hi, On 2012-05-19 20:59, سید احمد حسینی wrote: I used boot0 with boot9cfg : boot0cfg -B -b /boot/boot0 ada0 And I can see boot0 menu ,but Ubuntu can't boot! On May 19, 2012 11:26 PM, سید احمد حسینیahmad...@gmail.com wrote: You should make sure that GRUB is installed in the PBR (partition boot record) of your ubuntu partition and not in the MBR since boot0 lives there. I've done that with Linux Mint Debian Edition - and it works. Yes, and grub2 is a particularly horrifying thing to work with. I'd ask on ubuntu-us...@ubuntu.com; there were some clued-up people there when I used to be on that list-- however don't let them talk you into using grub2 to boot FreeBSD ;) Chris ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: superpages and kmem on amd64
On 05/20/2012 09:43, Marko Zec wrote: On Sunday 20 May 2012 09:25:59 Alan Cox wrote: On Sun, May 20, 2012 at 2:01 AM, Marko Zecz...@fer.hr wrote: Hi all, I'm playing with an algorithm which makes use of large contiguous blocks of kernel memory (ranging from 1M to 1G in size), so it would be nice if those could be somehow forcibly mapped to superpages. I was hoping that the VM system would automagically map (merge) contiguous 4k pages to superpages, but apparently it doesn't: vm.pmap.pdpe.demotions: 2 vm.pmap.pde.promotions: 543 vm.pmap.pde.p_failures: 266253 vm.pmap.pde.mappings: 0 vm.pmap.pde.demotions: 31 No, your conclusion is incorrect. These counts show that 543 superpage mappings were created by promotion. OK, that sounds promising. Does created by promotion count reflect historic / cumulative stats, or is vm.pmap.pde.promotions the actual number of superpages active? Or should we subtract vm.pmap.pde.demotions from it to get the current value? The count is cumulative. There is no instantaneous count. Subtracting demotions from promotions plus mappings is not a reliable way to get the instantaneous total because a superpage mapping can be destroyed without first being demoted. In any case, I wish to be certain that a particular kmem virtual address range is mapped to superpages - how can I enforce that at malloc time, and / or find out later if I really got my kmem mapped to superpages? Perhaps vm_map_lookup() could provide more info, but I'm wondering if someone already wrote a wrapper function for that, which takes only the base virtual address as a single argument? Try using pmap_mincore() to verify that the mappings are superpages. BTW, apparently malloc(size, M_TEMP, M_NOWAIT) requests fail for size 1G, even at boot time. Any ideas how to circumvent that (8.3-STABLE, amd64, 4G physical RAM)? I suspect that you need to increase the size of your kmem map. Alan ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: GSoC Project: Automated Kernel Crash Reporting System - Discussion
On 19.05.12 22:02, Mel Flynn wrote:
As I read the original intent is to post crashdumps at a specified
remote location through rc(8) using an sh(1) script on the next
reboot. tar seemed appropriate. I'm only mentioning extending
libfetch(3), because it will be easy for fetch(1) to pick it up, it
benefits more than just this project and once integrated into fetch(1)
can be used in said script above. Other than openssh we don't really
have a good tool in the base system to put local files elsewhere
securely. Also, if the BUGS section of fetch(3) is out of date, I'm
happy to be corrected :)
I'm not going to make you happy...
From lib/libfetch/http.c:
/*
* Store a file by HTTP
*/
FILE *
fetchPutHTTP(struct url *URL __unused, const char *flags __unused)
{
warnx(fetchPutHTTP(): not implemented);
return (NULL);
}
Adding HTTP PUT support to libfetch would be cool, but I doubt that it's
worth wasting GSoC time for that. Most people use curl for that just
because Google tells them to. On the other hand, SSH is available in
FreeBSD system in 99% of use cases, and it would be quite easy to setup
secure file transfer.
The final decision should however be made by TS.
--
Regards,
Ilya Bakulin
http://kibab.com
xmpp://kibab...@jabber.ru
signature.asc
Description: OpenPGP digital signature
Geom_mbr vs. SSD disks (was: proper newts options for SSD disks)
On May 19, 2012, at 11:36 AM, rozhuk...@gmail.com wrote:
Do not use MBR (or manually do all to align).
63 - not 4k aligned.
Right now, the -a alignment option for gpart add is broken when
used with MBR partitions. It looks like the gpart command uses
it to correctly align the start/end, but then the actual MBR geom code
does another alignment pass that rounds the start/size to
a multiple of gpt_sectors, which defaults to 63.
This seems problematic.
It's tempting to change sys/geom/part/g_part_mbr.c so that
it skips this additional alignment when the geometry has
defaulted. Something like this:
Index: sys/geom/part/g_part_mbr.c
===
--- part/g_part_mbr.c (revision 235597)
+++ part/g_part_mbr.c (working copy)
@@ -211,6 +211,7 @@
start = gpp-gpp_start;
size = gpp-gpp_size;
+ if (sectors != 63 || basetable-gpt_heads != 255) {
if (size sectors)
return (EINVAL);
if (start % sectors) {
@@ -221,6 +222,7 @@
size = size - (size % sectors);
if (size sectors)
return (EINVAL);
+ }
if (baseentry-gpe_deleted)
bzero(entry-ent, sizeof(entry-ent));
I'm not really certain I understand all of the implications
of this change, though.
Tim
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: superpages and kmem on amd64
historic / cumulative stats, or is vm.pmap.pde.promotions the actual number of superpages active? Or should we subtract vm.pmap.pde.demotions from it to get the current value? yes substracting gives current value. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: superpages and kmem on amd64
vm.pmap.pde.demotions: 31 No, your conclusion is incorrect. These counts show that 543 superpage mappings were created by promotion. OK, that sounds promising. Does created by promotion count reflect historic / cumulative stats, or is vm.pmap.pde.promotions the actual number of superpages active? Or should we subtract vm.pmap.pde.demotions from it to get the current value? correction to my last answer. something is just wrong IMHO on my 2GB laptop: [root@wojtek ~]# sysctl -ad vm.pmap.pde vm.pmap.pde: 2MB page mapping counters vm.pmap.pde.promotions: 2MB page promotions vm.pmap.pde.p_failures: 2MB page promotion failures vm.pmap.pde.mappings: 2MB page mappings vm.pmap.pde.demotions: 2MB page demotions [root@wojtek ~]# sysctl -a vm.pmap.pde vm.pmap.pde.promotions: 61196 vm.pmap.pde.p_failures: 4796 vm.pmap.pde.mappings: 3051 vm.pmap.pde.demotions: 2306 from description seems like mappings could be current amount , but both it's value as well as promotions-demotions gives nonsense. with 2GB RAM there can not be 3051 or more 2MB pages mapped! and i - at present - doesn't run many large programs actually only some xterms and one compiling task. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: superpages and kmem on amd64
On Sunday 20 May 2012 19:34:26 Alan Cox wrote: ... In any case, I wish to be certain that a particular kmem virtual address range is mapped to superpages - how can I enforce that at malloc time, and / or find out later if I really got my kmem mapped to superpages? Perhaps vm_map_lookup() could provide more info, but I'm wondering if someone already wrote a wrapper function for that, which takes only the base virtual address as a single argument? Try using pmap_mincore() to verify that the mappings are superpages. flags = pmap_mincore(vmspace_pmap(curthread-td_proc-p_vmspace), (vm_offset_t) addr)); OK, that works, and now I know my kmem chunk is on a superpage, horray!!! Thanks! BTW, apparently malloc(size, M_TEMP, M_NOWAIT) requests fail for size 1G, even at boot time. Any ideas how to circumvent that (8.3-STABLE, amd64, 4G physical RAM)? I suspect that you need to increase the size of your kmem map. Huh any hints how should I achieve that? In desperation I placed vm.kmem_size=8G in /boot/loader.conf and got this: vm.kmem_map_free: 8123924480 vm.kmem_map_size: 8364032 vm.kmem_size_scale: 1 vm.kmem_size_max: 329853485875 vm.kmem_size_min: 0 vm.kmem_size: 8132288512 but malloc(2G) still fails... Thanks, Marko ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: superpages and kmem on amd64
On 05/20/2012 17:48, Marko Zec wrote: On Sunday 20 May 2012 19:34:26 Alan Cox wrote: ... In any case, I wish to be certain that a particular kmem virtual address range is mapped to superpages - how can I enforce that at malloc time, and / or find out later if I really got my kmem mapped to superpages? Perhaps vm_map_lookup() could provide more info, but I'm wondering if someone already wrote a wrapper function for that, which takes only the base virtual address as a single argument? Try using pmap_mincore() to verify that the mappings are superpages. flags = pmap_mincore(vmspace_pmap(curthread-td_proc-p_vmspace), (vm_offset_t) addr)); OK, that works, and now I know my kmem chunk is on a superpage, horray!!! Thanks! BTW, apparently malloc(size, M_TEMP, M_NOWAIT) requests fail for size 1G, even at boot time. Any ideas how to circumvent that (8.3-STABLE, amd64, 4G physical RAM)? I suspect that you need to increase the size of your kmem map. Huh any hints how should I achieve that? In desperation I placed vm.kmem_size=8G in /boot/loader.conf and got this: vm.kmem_map_free: 8123924480 vm.kmem_map_size: 8364032 vm.kmem_size_scale: 1 vm.kmem_size_max: 329853485875 vm.kmem_size_min: 0 vm.kmem_size: 8132288512 but malloc(2G) still fails... Here is at least one reason why it fails: void * uma_large_malloc(int size, int wait) Note the type of size. Can you malloc 1GB? ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: superpages and kmem on amd64
On Monday 21 May 2012 01:12:01 Alan Cox wrote: ... BTW, apparently malloc(size, M_TEMP, M_NOWAIT) requests fail for size 1G, even at boot time. Any ideas how to circumvent that (8.3-STABLE, amd64, 4G physical RAM)? I suspect that you need to increase the size of your kmem map. Huh any hints how should I achieve that? In desperation I placed vm.kmem_size=8G in /boot/loader.conf and got this: vm.kmem_map_free: 8123924480 vm.kmem_map_size: 8364032 vm.kmem_size_scale: 1 vm.kmem_size_max: 329853485875 vm.kmem_size_min: 0 vm.kmem_size: 8132288512 but malloc(2G) still fails... Here is at least one reason why it fails: void * uma_large_malloc(int size, int wait) Note the type of size. Can you malloc 1GB? Uff, good catch... malloc(1G) works, malloc(1.99G) works, malloc(2G) doesn't! Anyhow, malloc(1G) is big enough for what I want to do ATM, I was just curious why it breaks with bigger requests. Thanks, Marko ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
