Re: vmmap replacement -- please test
Hi, David Coppa wrote: still rocking hard My old Athlon 1600+ blow dries something like I'm gruvm up the country, where the memory tastes like wine. Is this a regression, then?? --steffen
Re: vmmap replacement -- please test
On Fri, Mar 2, 2012 at 14:37, Steffen Daode Nurpmeso sdao...@googlemail.com wrote: Hi, David Coppa wrote: still rocking hard My old Athlon 1600+ blow dries something like B I'm gruvm up the country, where the memory tastes like wine. Is this a regression, then?? --steffen System is really responsive, nothing to report here. In other words, no regression and runs flawlessly on an i7 amd64-current + vmmap_sys.diff.65 + vmmap_userland.diff.30. Yours, Vlad
Re: vmmap replacement -- please test
On Sat, Feb 18, 2012 at 05:09:02PM +0100, Ariane van der Steldt wrote: Hi, Just to prove my point that vmmap testing is still relevant: today a bug got found and fixed, where grepping a 1.7GB file on i386 tripped an assertion (I love assertions). This brings the latest versions to: http://www.stack.nl/~ariane/vmmap_sys.diff.65 (apply against /usr/src/sys) http://www.stack.nl/~ariane/vmmap_userland.diff.30 (apply against /usr/src) Testing is still welcome. :) -- Ariane I use this diff on i386, amd64 and macppc without problem. Thanks !
Re: vmmap replacement -- please test
On Sat, Feb 18, 2012 at 05:09:02PM +0100, Ariane van der Steldt wrote: Hi, Just to prove my point that vmmap testing is still relevant: today a bug got found and fixed, where grepping a 1.7GB file on i386 tripped an assertion (I love assertions). This brings the latest versions to: http://www.stack.nl/~ariane/vmmap_sys.diff.65 (apply against /usr/src/sys) http://www.stack.nl/~ariane/vmmap_userland.diff.30 (apply against /usr/src) Testing is still welcome. :) applied these to my i386-current checkout of today, system very responsive. so far very good! -ryan
Re: vmmap replacement -- please test
On Sat, Feb 18, 2012 at 5:09 PM, Ariane van der Steldt ari...@stack.nl wrote: Hi, Just to prove my point that vmmap testing is still relevant: today a bug got found and fixed, where grepping a 1.7GB file on i386 tripped an assertion (I love assertions). This brings the latest versions to: http://www.stack.nl/~ariane/vmmap_sys.diff.65 (apply against /usr/src/sys) http://www.stack.nl/~ariane/vmmap_userland.diff.30 (apply against /usr/src) Testing is still welcome. :) No major problems encountered: still rocking hard on my laptop (amd64-current + vmmap_sys.diff.65 + vmmap_userland.diff.30). ciao, David
Re: vmmap replacement -- please test
Hi, Just to prove my point that vmmap testing is still relevant: today a bug got found and fixed, where grepping a 1.7GB file on i386 tripped an assertion (I love assertions). This brings the latest versions to: http://www.stack.nl/~ariane/vmmap_sys.diff.65 (apply against /usr/src/sys) http://www.stack.nl/~ariane/vmmap_userland.diff.30 (apply against /usr/src) Testing is still welcome. :) -- Ariane
Re: vmmap replacement -- please test
Hello. I want to ask you about vmmap changes introduced here and in your presentation http://openbsd.org/papers/tdose_memalloc/presentation.html 1. http://openbsd.org/papers/tdose_memalloc/presentation.html#%2816%29 Browsers, java mono breakage You pointed to some buggy software. If I understand correctly, you try to workaround these bugs by using of VM map address selectors. If it is correct then I don't see reasons for this code and it's workarounds overhead in production (web/db server, firewall, router... ) where these buggy apps don't used at all. Secondly vmmap becomes more complex and harder to understand/debug/develop. Maybe these workarounds must be #ifdef'ed and enabled only by some kernel option? 2. http://openbsd.org/papers/tdose_memalloc/presentation.html#%2828%29 Pivot algorithm For what reason this code used for? Why we can't select some random entry from freespace and some random offset (as you say in http://openbsd.org/papers/tdose_memalloc/presentation.html#%2827%29)? -- antonvm
Re: vmmap replacement -- please test
I want to ask you about vmmap changes introduced here and in your presentation http://openbsd.org/papers/tdose_memalloc/presentation.html 1. http://openbsd.org/papers/tdose_memalloc/presentation.html#%2816%29 Browsers, java mono breakage You pointed to some buggy software. If I understand correctly, you try to workaround these bugs by using of VM map address selectors. If it is correct then I don't see reasons for this code and it's workarounds overhead in production (web/db server, firewall, router... ) where these buggy apps don't used at all. Secondly vmmap becomes more complex and harder to understand/debug/develop. Maybe these workarounds must be #ifdef'ed and enabled only by some kernel option? Ah, so we'd have a very complex subsystem of the kernel which is on for some people, and off for other people. Kind of like shipping two seperate operating systems. With different failure conditions and more effort required to maintain the code quality. That's completely retarded. 2. http://openbsd.org/papers/tdose_memalloc/presentation.html#%2828%29 Pivot algorithm For what reason this code used for? Why we can't select some random entry from freespace and some random offset (as you say in http://openbsd.org/papers/tdose_memalloc/presentation.html#%2827%29)? No, you can't pick a completely random location. A lot of software people want to run breaks.
Re: vmmap replacement -- please test
On 2012/02/17 22:01, Anton Maksimenkov wrote: Hello. I want to ask you about vmmap changes introduced here and in your presentation http://openbsd.org/papers/tdose_memalloc/presentation.html 1. http://openbsd.org/papers/tdose_memalloc/presentation.html#%2816%29 Browsers, java mono breakage You pointed to some buggy software. If I understand correctly, you try to workaround these bugs by using of VM map address selectors. If it is correct then I don't see reasons for this code and it's workarounds overhead in production (web/db server, firewall, router... ) where these buggy apps don't used at all. Secondly vmmap becomes more complex and harder to understand/debug/develop. Maybe these workarounds must be #ifdef'ed and enabled only by some kernel option? Surely that makes the workarounds *even more complex*, and by having different people running different code, harder to debug.
Re: vmmap replacement -- please test
On Sat, Jan 14, 2012 at 8:42 AM, Ariane van der Steldt ari...@stack.nl wrote: Everybody thanks for testing this, it's been a tremendous help to me. As far as I'm concerned, this diff will be commited once we unlock after release (in a coordinated manner ofcourse, since this is uvm we're talking about). Great news. I'm using your diff on my amd64 laptop since 15 days and no problems. Thanks a lot for your work, David
Re: vmmap replacement -- please test
Ariane van der Steldt wrote [2012-01-14 08:42+0100]: As far as I'm concerned, this diff will be commited once we unlock after release (in a coordinated manner ofcourse, since this is uvm we're talking about). It's about time too, ofcourse: 64 revisions of the same diff is alot. :D -- Ariane I too cannot report anything to bring you forward (amd64). --steffen
Re: vmmap replacement -- please test
On Sat, Jan 07, 2012 at 11:47:32PM +0100, Juan Francisco Cantero Hurtado wrote: On Fri, 06 Jan 2012 23:51:28 +0100, Ariane van der Steldt ari...@stack.nl wrote: I found and fixed the i386 bug. Please test this, to confirm that it fixes the problem (and doesn't introduce anything else, ofcourse). (vmmap_sys relative to /usr/src/sys, vmmap_userland relative to /usr/src) New version of the diff: http://www.stack.nl/~ariane/vmmap_sys.diff.64 with corresponding userland: http://www.stack.nl/~ariane/vmmap_userland.diff.30 (the userland portion is unchanged). SHA256 (vmmap_sys.diff.64) = aa87c7a6d1c485b6acb228f238a54bee8c28eb5cc6cfc65da6e6b3b047d94e93 MD5 (vmmap_sys.diff.64) = 2d7587702ce2bf7ce50bf58c5b24a6cc vmmap_sys.diff.64 size: 305807 bytes SHA256 (vmmap_userland.diff.30) = 85d2cb4d85a6a9cffd16940f17498321243ef9731a687161b3b38caadaf6dc06 MD5 (vmmap_userland.diff.30) = f5add708c2d1f84dc05f2ceb1749c9cf vmmap_userland.diff.30 size: 2011 Hmm, did I write 2011 there? That was ofcourse not correct... Must have been confused with the year end or something :P Your patch fix the bug. I've been using the computer for 3 hours without apparent problems. Everybody thanks for testing this, it's been a tremendous help to me. As far as I'm concerned, this diff will be commited once we unlock after release (in a coordinated manner ofcourse, since this is uvm we're talking about). It's about time too, ofcourse: 64 revisions of the same diff is alot. :D -- Ariane
Re: vmmap replacement -- please test
On Fri, 06 Jan 2012 23:51:28 +0100, Ariane van der Steldt ari...@stack.nl wrote: Hi, I found and fixed the i386 bug. Please test this, to confirm that it fixes the problem (and doesn't introduce anything else, ofcourse). (vmmap_sys relative to /usr/src/sys, vmmap_userland relative to /usr/src) New version of the diff: http://www.stack.nl/~ariane/vmmap_sys.diff.64 with corresponding userland: http://www.stack.nl/~ariane/vmmap_userland.diff.30 (the userland portion is unchanged). SHA256 (vmmap_sys.diff.64) = aa87c7a6d1c485b6acb228f238a54bee8c28eb5cc6cfc65da6e6b3b047d94e93 MD5 (vmmap_sys.diff.64) = 2d7587702ce2bf7ce50bf58c5b24a6cc vmmap_sys.diff.64 size: 305807 bytes SHA256 (vmmap_userland.diff.30) = 85d2cb4d85a6a9cffd16940f17498321243ef9731a687161b3b38caadaf6dc06 MD5 (vmmap_userland.diff.30) = f5add708c2d1f84dc05f2ceb1749c9cf vmmap_userland.diff.30 size: 2011 Your patch fix the bug. I've been using the computer for 3 hours without apparent problems. Thanks. -- Juan Francisco Cantero Hurtado http://juanfra.info
Re: vmmap replacement -- please test
Hi, I found and fixed the i386 bug. Please test this, to confirm that it fixes the problem (and doesn't introduce anything else, ofcourse). (vmmap_sys relative to /usr/src/sys, vmmap_userland relative to /usr/src) New version of the diff: http://www.stack.nl/~ariane/vmmap_sys.diff.64 with corresponding userland: http://www.stack.nl/~ariane/vmmap_userland.diff.30 (the userland portion is unchanged). SHA256 (vmmap_sys.diff.64) = aa87c7a6d1c485b6acb228f238a54bee8c28eb5cc6cfc65da6e6b3b047d94e93 MD5 (vmmap_sys.diff.64) = 2d7587702ce2bf7ce50bf58c5b24a6cc vmmap_sys.diff.64 size: 305807 bytes SHA256 (vmmap_userland.diff.30) = 85d2cb4d85a6a9cffd16940f17498321243ef9731a687161b3b38caadaf6dc06 MD5 (vmmap_userland.diff.30) = f5add708c2d1f84dc05f2ceb1749c9cf vmmap_userland.diff.30 size: 2011 Thanks, -- Ariane
Re: vmmap replacement -- please test
On Fri, 30 Dec 2011 17:55:27 +0100, Ariane van der Steldt ari...@stack.nl wrote: On Fri, Dec 30, 2011 at 03:04:30PM +0100, Ariane van der Steldt wrote: On Fri, Dec 30, 2011 at 02:42:18PM +0100, Ariane van der Steldt wrote: The huge diff included below is to replace vmmap for something that works a lot better. This is the second version I plan to commit. I have been running this on my laptop for months and have had no problems. The current incarnation is: vmmap_sys.diff.63 Which never made it to the list apparently. I'll just blame spam filtering and provide it an alternative way: http://www.stack.nl/~ariane/vmmap_sys.diff.63 SHA256 (vmmap_sys.diff.63) = 5c3e71360795f3899baa652f6ba6caa999b137b01a778dca12c253af1dbcff00 MD5 (vmmap_sys.diff.63) = 11736a0bacb0b8b72079935ba01c62c4 size: 305720 bytes Highlights: - the mapping and allocation (address selection) logic is split - the allocator is in compatibility mode, meaning it ought to perform the same strategies (and hence contain the same bugs, yay) as the version that you are running in your kernel at the moment (this means java and various browsers won't die horribly, at least not more than they used to) - despite using the same algorithm, it is a bit^W alot faster - it's only a 3^W 10494 line diff :P I'll need testing on any and all architectures. Thanks, Hi, I've tried both patches on i386. The system crashed. The code in src is a clean checkout from 3 January. I applied the patches in src and src/sys without errors. I compiled the kernel and userland with this steps: export PARALLEL_BUILD=YES export MAKE_JOBS=2 cd /usr/src/sys/arch/i386/conf config GENERIC.MP cd ../compile/GENERIC.MP make clean make make install rm -rf /usr/obj/* cd /usr/src make obj cd /usr/src make build dmesg with trace and ps: http://juanfra.info/bugs-y-listas/obsd-vmmap-201201/dmesg.txt.bz2 When I boot the system with /obsd (from snapshots) and the userland with your patch applied, all works without issues. I will save the dump for a few weeks. If you need more info, ask me. Cheers. -- Juan Francisco Cantero Hurtado http://juanfra.info
Re: vmmap replacement -- please test
On Fri, Dec 30, 2011 at 05:55:27PM +0100, Ariane van der Steldt wrote: - it's only a 3^W 10494 line diff :P Wow, you weren't kidding. I'll need testing on any and all architectures. I'm now running this on my sparc64, macppc and amd64 boxes, the amd64 is the only one i currently actually use, the others i'm just testing with make build. I'll sent an e-mail if any of the machine does anything unexpected.
Re: vmmap replacement -- please test
On Sat, Dec 31, 2011 at 11:24:54AM +0100, Alexander Schrijver wrote: On Fri, Dec 30, 2011 at 05:55:27PM +0100, Ariane van der Steldt wrote: I'll need testing on any and all architectures. Is this in snapshots too? No, it's not in snapshots. -- Ariane
Re: vmmap replacement -- please test
On Fri, Dec 30, 2011 at 05:55:27PM +0100, Ariane van der Steldt wrote: I'll need testing on any and all architectures. Is this in snapshots too?
Re: vmmap replacement -- please test
On Fri, Dec 30, 2011 at 02:42:18PM +0100, Ariane van der Steldt wrote:
The huge diff included below is to replace vmmap for something that
works a lot better. This is the second version I plan to commit.
I have been running this on my laptop for months and have had no
problems.
The current incarnation is:
vmmap_sys.diff.63
And this is the corresponding userspace counterpart of the diff, which
you'll need to compile userspace.
It's version is:
vmmap_userland.diff.30
--
Ariane
Index: lib/libkvm/kvm_proc.c
===
RCS file: /cvs/src/lib/libkvm/kvm_proc.c,v
retrieving revision 1.42
diff -u -d -p -r1.42 kvm_proc.c
--- lib/libkvm/kvm_proc.c 12 Mar 2011 04:54:28 - 1.42
+++ lib/libkvm/kvm_proc.c 21 May 2011 13:38:58 -
@@ -131,7 +131,7 @@ static void ps_str_e(struct ps_strings *
static char *
_kvm_ureadm(kvm_t *kd, const struct miniproc *p, u_long va, u_long *cnt)
{
- u_long addr, head, offset, slot;
+ u_long addr, offset, slot;
struct vm_anon *anonp, anon;
struct vm_map_entry vme;
struct vm_amap amap;
@@ -140,27 +140,28 @@ _kvm_ureadm(kvm_t *kd, const struct mini
if (kd-swapspc == 0) {
kd-swapspc = _kvm_malloc(kd, kd-nbpg);
if (kd-swapspc == 0)
- return (0);
+ return (NULL);
}
/*
* Look through the address map for the memory object
* that corresponds to the given virtual address.
-* The header just has the entire valid range.
*/
- head = (u_long)p-p_vmspace-vm_map.header;
- addr = head;
+ addr = (u_long)RB_ROOT(p-p_vmspace-vm_map.addr);
while (1) {
+ if (addr == 0)
+ return (NULL);
if (KREAD(kd, addr, vme))
- return (0);
+ return (NULL);
- if (va = vme.start va vme.end
- vme.aref.ar_amap != NULL)
+ if (va vme.start)
+ addr = (u_long)RB_LEFT(vme, daddrs.addr_entry);
+ else if (va = vme.end + vme.guard + vme.fspace)
+ addr = (u_long)RB_RIGHT(vme, daddrs.addr_entry);
+ else if (va = vme.end)
+ return (NULL);
+ else
break;
-
- addr = (u_long)vme.next;
- if (addr == head)
- return (0);
}
/*
Index: usr.sbin/procmap/procmap.c
===
RCS file: /cvs/src/usr.sbin/procmap/procmap.c,v
retrieving revision 1.38
diff -u -d -p -r1.38 procmap.c
--- usr.sbin/procmap/procmap.c 23 Apr 2011 01:01:34 - 1.38
+++ usr.sbin/procmap/procmap.c 21 May 2011 13:38:58 -
@@ -169,10 +169,13 @@ struct nlist nl[] = {
void load_symbols(kvm_t *);
void process_map(kvm_t *, pid_t, struct kinfo_proc *, struct sum *);
-size_t dump_vm_map_entry(kvm_t *, struct kbit *, struct kbit *, int,
+struct vm_map_entry *load_vm_map_entries(kvm_t *, struct vm_map_entry *,
+struct vm_map_entry *);
+void unload_vm_map_entries(struct vm_map_entry *);
+size_t dump_vm_map_entry(kvm_t *, struct kbit *, struct vm_map_entry *,
struct sum *);
-char *findname(kvm_t *, struct kbit *, struct kbit *, struct kbit *,
- struct kbit *, struct kbit *);
+char *findname(kvm_t *, struct kbit *, struct vm_map_entry *, struct kbit *,
+struct kbit *, struct kbit *);
int search_cache(kvm_t *, struct kbit *, char **, char *, size_t);
#if 0
void load_name_cache(kvm_t *);
@@ -182,6 +185,19 @@ static void __dead usage(void);
static pid_t strtopid(const char *);
void print_sum(struct sum *, struct sum *);
+/*
+ * uvm_map address tree implementation.
+ */
+static int no_impl(void *, void *);
+static int
+no_impl(void *p, void *q)
+{
+ errx(1, uvm_map address comparison not implemented);
+ return 0;
+}
+
+RB_GENERATE(uvm_map_addr, vm_map_entry, daddrs.addr_entry, no_impl);
+
int
main(int argc, char *argv[])
{
@@ -347,12 +363,12 @@ print_sum(struct sum *sum, struct sum *t
void
process_map(kvm_t *kd, pid_t pid, struct kinfo_proc *proc, struct sum *sum)
{
- struct kbit kbit[4], *vmspace, *vm_map, *header, *vm_map_entry;
- struct vm_map_entry *last;
- u_long addr, next;
+ struct kbit kbit[3], *vmspace, *vm_map;
+ struct vm_map_entry *vm_map_entry;
size_t total = 0;
char *thing;
uid_t uid;
+ int vmmap_flags;
if ((uid = getuid())) {
if (pid == 0) {
@@ -367,13 +383,9 @@ process_map(kvm_t *kd, pid_t pid, struct
vmspace = kbit[0];
vm_map = kbit[1];
- header = kbit[2];
- vm_map_entry = kbit[3];
A(vmspace) = 0;
A(vm_map) = 0;
- A(header) = 0;
- A(vm_map_entry) = 0;
if (pid 0) {
Re: vmmap replacement -- please test
On Fri, Dec 30, 2011 at 03:04:30PM +0100, Ariane van der Steldt wrote: On Fri, Dec 30, 2011 at 02:42:18PM +0100, Ariane van der Steldt wrote: The huge diff included below is to replace vmmap for something that works a lot better. This is the second version I plan to commit. I have been running this on my laptop for months and have had no problems. The current incarnation is: vmmap_sys.diff.63 Which never made it to the list apparently. I'll just blame spam filtering and provide it an alternative way: http://www.stack.nl/~ariane/vmmap_sys.diff.63 SHA256 (vmmap_sys.diff.63) = 5c3e71360795f3899baa652f6ba6caa999b137b01a778dca12c253af1dbcff00 MD5 (vmmap_sys.diff.63) = 11736a0bacb0b8b72079935ba01c62c4 size: 305720 bytes Highlights: - the mapping and allocation (address selection) logic is split - the allocator is in compatibility mode, meaning it ought to perform the same strategies (and hence contain the same bugs, yay) as the version that you are running in your kernel at the moment (this means java and various browsers won't die horribly, at least not more than they used to) - despite using the same algorithm, it is a bit^W alot faster - it's only a 3^W 10494 line diff :P I'll need testing on any and all architectures. Thanks, -- Ariane
Re: vmmap replacement, please test
On Wed, May 18, 2011 at 02:07:28AM +0200, Ariane van der Steldt wrote: On Wed, May 11, 2011 at 05:05:32PM +0200, Ariane van der Steldt wrote: On Wed, May 11, 2011 at 03:44:45AM +0200, Ariane van der Steldt wrote: On Wed, May 11, 2011 at 03:43:19AM +0200, Ariane van der Steldt wrote: The newest version of vmmap (as of now) is vmmap_sys.diff.26 Since the diff is scheduled to go in may 20 and has a lot of changes and fixes, please test this diff and report any failures and successes. I use this separate e-mail to publish the userspace part of the diff. I missed a change in the userspace part of the diff, causing libkvm not to compile. The below diff contains the fixed code. Diff version 28. A small bit of code reworked that I thought broke emacs-22 (but turns out that breakage comes from somewhere else). I want, no _need_ tests on sparc64 hardware for this. Does it break or work? I would like the dmesg and in the case of crash the ddb output of: - trace - show map /f kernel_map -- still panics my U10 while loading init. And the show map command itself panics again. Output below. An interesting data point is that this machine has only 384MB of RAM. On my Blade-100, with 1GB of RAM, init starts, but many (all?) of the shared objects fail with the same kind of errors as emacs on the i386. Since ktrace is itself failing I can't get any trace, without first building static ktrace/kdump. Will do that later if it can help. root on sd0a swap on sd0b dump on sd0b panic: kernel data fault: pc=10131ac addr=4000c82e000 kdb breakpoint at 1465740 Stopped at Debugger+0x4: nop RUN AT LEAST 'trace' AND 'ps' AND INCLUDE OUTPUT WHEN REPORTING THIS PANIC! DO NOT EVEN BOTHER REPORTING THIS WITHOUT INCLUDING THAT INFORMATION! ddb trace data_access_fault(4000aa2d5b0, 30, 10131ac, 4000c82e000, 4000c82e001, 80080d) a t data_access_fault+0x2c8 trapbase_sun4v(4000aa2d760, 0, 502, 4000c82e001, 4000c830001, 1303) at trapbase _sun4v+0x8790 uvm_map(101275c, 4000c82e001, 104, 4000b81b8a5, 0, 40006c900a0) at uvm_map+0x16 c exec_sigcode_map(40006c8e000, 180d2d8, fffd9510, 4000aa2ddd8, 74, 40006 c8e1a2) at exec_sigcode_map+0xe0 sys_execve(0, 0, 0, 40006ccc000, 40006c8c000, 40006c8e000) at sys_execve+0x9c0 start_init(40006c8e000, 4000aa2ddf8, 4000aa2ddf0, 153cb60, 4000aa2ddc0, 4000aa2 ddd8) at start_init+0x17c proc_trampoline(0, 0, 0, 0, 0, 0) at proc_trampoline+0x4 ddb show map /f kernel_map MAP 0x1814258: [0x18c5b18-0x0] auto allocate range: 0x155d008-0x1 0 segments brk() allocate range: 0x0-0x1 1 segments stack allocate range: 0x0-0x12fdc00 1 segments sz=25248144, ref=0, version=0, flags=0x1814210 pmap=0x1856040(resident=25681016) panic: kernel data fault: pc=12e6a00 addr=1 kdb breakpoint at 1465740 Stopped at Debugger+0x4: nop RUN AT LEAST 'trace' AND 'ps' AND INCLUDE OUTPUT WHEN REPORTING THIS PANIC! DO NOT EVEN BOTHER REPORTING THIS WITHOUT INCLUDING THAT INFORMATION! ddb ddb trace data_access_fault(4000aa2cac0, 30, 12e6a00, 1, 1, 800809) at da ta_access_fault+0x2c8 trapbase_sun4v(1, , 187dc78, 0, 1814210, 4000aa2ceb8) a t trapbase_sun4v+0x8790 uvm_map_printit(1814258, 1, 11a4780, 4000aa2cda0, 0, 1886180) at uvm_map_printi t+0xdc db_command(180c168, 0, 0, 0, 0, 180b000) at db_command+0x134 db_command_loop(46, 0, 1895000, 1400, 400185c, 400185c) at db_command_l oop+0xf0 db_trap(1885000, 0, 0, 0, 74, 7fd8) at db_trap+0x148 kdb_trap(101, 4000aa2d240, 1, 0, 0, 4000aa2d3f8) at kdb_trap+0x168 trap(4000aa2d240, 101, 1465740, 86, 0, b) at trap+0x2f4 slowtrap(1, 4000aa2d4c8, 180f000, 4000aa2d4c8, 0, 1) at slowtrap+0x1d8 panic(1611818, 4000aa2d4c8, 1895000, 100, 400185c, 400185c) at panic+0x 8c data_access_fault(4000aa2d5b0, 30, 10131ac, 4000c82e000, 4000c82e001, 80080d) a t data_access_fault+0x2c8 trapbase_sun4v(4000aa2d760, 0, 502, 4000c82e001, 4000c830001, 1303) at trapbase _sun4v+0x8790 uvm_map(101275c, 4000c82e001, 104, 4000b81b8a5, 0, 40006c900a0) at uvm_map+0x16 c exec_sigcode_map(40006c8e000, 180d2d8, fffd9510, 4000aa2ddd8, 74, 40006 c8e1a2) at exec_sigcode_map+0xe0 sys_execve(0, 0, 0, 40006ccc000, 40006c8c000, 40006c8e000) at sys_execve+0x9c0 start_init(40006c8e000, 4000aa2ddf8, 4000aa2ddf0, 153cb60, 4000aa2ddc0, 4000aa2 ddd8) at start_init+0x17c proc_trampoline(0, 0, 0, 0, 0, 0) at proc_trampoline+0x4 ddb -- Matthieu Herrb
Re: vmmap replacement, please test
In fact kernel_map itself is 0 when my machine stops. So the command above also fails. [...] ddb x kernel_map kernel_map: 0 Use x/qx, you're on a 64-bit platform and this is a 64 bit pointer (of which the upper 32 bits are zero).
Re: vmmap replacement, please test
On Wed, 11 May 2011, Ariane van der Steldt wrote: Hi, The newest version of vmmap (as of now) is vmmap_sys.diff.26 Since the diff is scheduled to go in may 20 and has a lot of changes and fixes, please test this diff and report any failures and successes. If I start a program with this patch (usually I just use top), then gdb top `pgrep top`, then bt, the machine hangs. If running the the gdb commands on first console, I get ddb : panic: kernel diagnostic assertion start = srcmap-min_offset end = srcmap -max_offset failed: file ../../../../uvm/uvm_map.c, line 3947 Stopped at Debugger+0x4: popl %ebp RUN AT LEAST 'trace' AND 'ps' AND INCLUDE OUTPUT WHEN REPORTING THIS PANIC! DO NOT EVEN BOTHER REPORTING THIS WITHOUT INCLUDING THAT INFORMATION! ddb Apparently the machine messes with memory so I don't get contents of trace and ps in dmesg on next boot as hinted in man crash, and I don't have serial. If this is important, I can manually type some of it. -- _ _ // \\// Eivind Evensen \/ OpenBSD 4.9-current (GENERIC) #0: Thu May 12 18:04:33 CEST 2011 rumrunner@rev.hjerdalen.lokalnett:/usr/src/sys/arch/i386/compile/GENERIC cpu0: Intel(R) Celeron(R) M processor 1400MHz (GenuineIntel 686-class) 1.40 GHz cpu0: FPU,V86,DE,PSE,TSC,MSR,MCE,CX8,SEP,MTRR,PGE,MCA,CMOV,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,TM,SBF real mem = 233238528 (222MB) avail mem = 218836992 (208MB) mainbus0 at root bios0 at mainbus0: AT/286+ BIOS, date 08/18/04, BIOS32 rev. 0 @ 0xfd740, SMBIOS rev. 2.31 @ 0xdf010 (28 entries) bios0: vendor Hewlett-Packard version F.10 date 08/18/2004 bios0: Hewlett-Packard hp pavilion ze4900 (PJ943EA#ABN) apm0 at bios0: Power Management spec V1.2 apm0: AC on, battery charge unknown acpi at bios0 function 0x0 not configured pcibios0 at bios0: rev 2.1 @ 0xfd740/0x8c0 pcibios0: PCI IRQ Routing Table rev 1.0 @ 0xfdf20/192 (10 entries) pcibios0: PCI Interrupt Router at 000:31:0 (Intel 82371FB ISA rev 0x00) pcibios0: PCI bus #2 is the last bus bios0: ROM list: 0xc/0xcc00! 0xcd000/0x1000 0xdf000/0x1000! 0xe/0x4000! cpu0 at mainbus0: (uniprocessor) pci0 at mainbus0 bus 0: configuration mode 1 (bios) mem address conflict 0xdf0/0x400 pchb0 at pci0 dev 0 function 0 Intel 82855GM Host rev 0x02 Intel 82855GM Memory rev 0x02 at pci0 dev 0 function 1 not configured Intel 82855GM Config rev 0x02 at pci0 dev 0 function 3 not configured vga1 at pci0 dev 2 function 0 Intel 82855GM Video rev 0x02 wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation) wsdisplay0: screen 1-5 added (80x25, vt100 emulation) intagp0 at vga1 agp0 at intagp0: aperture at 0xe800, size 0x800 inteldrm0 at vga1: irq 10 drm0 at inteldrm0 Intel 82855GM Video rev 0x02 at pci0 dev 2 function 1 not configured uhci0 at pci0 dev 29 function 0 Intel 82801DB USB rev 0x03: irq 10 uhci1 at pci0 dev 29 function 1 Intel 82801DB USB rev 0x03: irq 11 uhci2 at pci0 dev 29 function 2 Intel 82801DB USB rev 0x03: couldn't map interrupt ehci0 at pci0 dev 29 function 7 Intel 82801DB USB rev 0x03: irq 11 usb0 at ehci0: USB revision 2.0 uhub0 at usb0 Intel EHCI root hub rev 2.00/1.00 addr 1 ppb0 at pci0 dev 30 function 0 Intel 82801BAM Hub-to-PCI rev 0x83 pci1 at ppb0 bus 1 mem address conflict 0xdf01000/0x1000 rl0 at pci1 dev 0 function 0 Realtek 8139 rev 0x10: irq 10, address 00:c0:9f:50:32:6a rlphy0 at rl0 phy 0: RTL internal PHY cbb0 at pci1 dev 5 function 0 TI PCI1410 CardBus rev 0x02: couldn't map interrupt ichpcib0 at pci0 dev 31 function 0 Intel 82801DBM LPC rev 0x03: 24-bit timer at 3579545Hz pciide0 at pci0 dev 31 function 1 Intel 82801DBM IDE rev 0x03: DMA, channel 0 configured to compatibility, channel 1 configured to compatibility wd0 at pciide0 channel 0 drive 0: ST94019A wd0: 16-sector PIO, LBA48, 38154MB, 78140160 sectors wd0(pciide0:0:0): using PIO mode 4, Ultra-DMA mode 5 atapiscsi0 at pciide0 channel 1 drive 0 scsibus0 at atapiscsi0: 2 targets cd0 at scsibus0 targ 0 lun 0: PHILIPS, CDRW/DVD CDD5263, UH89 ATAPI 5/cdrom removable cd0(pciide0:1:0): using PIO mode 4, DMA mode 2 ichiic0 at pci0 dev 31 function 3 Intel 82801DB SMBus rev 0x03: irq 5 iic0 at ichiic0 spdmem0 at iic0 addr 0x51: 256MB DDR SDRAM non-parity PC2700CL2.5 auich0 at pci0 dev 31 function 5 Intel 82801DB AC97 rev 0x03: irq 5, ICH4 AC97 ac97: codec id 0x43585430 (Conexant CXT48) ac97: codec features reserved, headphone, 18 bit DAC, 18 bit ADC, No 3D Stereo audio0 at auich0 Intel 82801DB Modem rev 0x03 at pci0 dev 31 function 6 not configured usb1 at uhci0: USB revision 1.0 uhub1 at usb1 Intel UHCI root hub rev 1.00/1.00 addr 1 usb2 at uhci1: USB revision 1.0 uhub2 at usb2 Intel UHCI root hub rev 1.00/1.00 addr 1 isa0 at ichpcib0 isadma0 at isa0 pckbc0 at isa0 port 0x60/5 pckbd0 at pckbc0 (kbd slot) pckbc0: using irq 1 for kbd slot wskbd0 at pckbd0: console keyboard, using wsdisplay0 pms0 at pckbc0 (aux slot) pckbc0: using irq 12 for aux slot wsmouse0 at pms0 mux 0 pcppi0 at isa0 port 0x61 spkr0 at pcppi0 npx0 at isa0 port 0xf0/16: reported by
Re: vmmap replacement, please test
there is a newer diff; I won't post the whole thing but in uvm_map.c around line 5954, replace this: KASSERT(start = srcmap-min_offset end = srcmap-max_offset); with this: if ((start PAGE_MASK) != 0 || (end PAGE_MASK) != 0 || end start) return EINVAL; if (start srcmap-min_offset || end srcmap-max_offset) return EINVAL; you are off by about 2000 lines. it is on line 3947 after applying ariane@ original vmmap diff :-)
Re: vmmap replacement, please test
On 2011/05/16 13:04, Amit Kulkarni wrote: there is a newer diff; I won't post the whole thing but in uvm_map.c around line 5954, replace this: KASSERT(start = srcmap-min_offset end = srcmap-max_offset); with this: if ((start PAGE_MASK) != 0 || (end PAGE_MASK) != 0 || end start) return EINVAL; if (start srcmap-min_offset || end srcmap-max_offset) return EINVAL; you are off by about 2000 lines. it is on line 3947 after applying ariane@ original vmmap diff :-) oh right, I was looking at a diff of the diffs...
Re: vmmap replacement, please test
On Mon, 16 May 2011, Stuart Henderson wrote: On 2011/05/16 11:37, Eivind E wrote: If I start a program with this patch (usually I just use top), then gdb top `pgrep top`, then bt, the machine hangs. there is a newer diff; I won't post the whole thing but in uvm_map.c around line 5954, replace this: KASSERT(start = srcmap-min_offset end = srcmap-max_offset); with this: if ((start PAGE_MASK) != 0 || (end PAGE_MASK) != 0 || end start) return EINVAL; if (start srcmap-min_offset || end srcmap-max_offset) return EINVAL; Yes, that works, thanks. -- _ _ // \\// Eivind Evensen \/
Re: vmmap replacement, please test
On Wed, May 11, 2011 at 03:44:45AM +0200, Ariane van der Steldt wrote:
On Wed, May 11, 2011 at 03:43:19AM +0200, Ariane van der Steldt wrote:
The newest version of vmmap (as of now) is vmmap_sys.diff.26
Since the diff is scheduled to go in may 20 and has a lot of changes and
fixes, please test this diff and report any failures and successes.
I use this separate e-mail to publish the userspace part of the diff.
I missed a change in the userspace part of the diff, causing libkvm not
to compile. The below diff contains the fixed code.
--
Ariane
Index: lib/libkvm/kvm_proc.c
===
RCS file: /cvs/src/lib/libkvm/kvm_proc.c,v
retrieving revision 1.42
diff -u -d -p -r1.42 kvm_proc.c
--- lib/libkvm/kvm_proc.c 12 Mar 2011 04:54:28 - 1.42
+++ lib/libkvm/kvm_proc.c 11 May 2011 12:09:20 -
@@ -131,7 +131,7 @@ static void ps_str_e(struct ps_strings *
static char *
_kvm_ureadm(kvm_t *kd, const struct miniproc *p, u_long va, u_long *cnt)
{
- u_long addr, head, offset, slot;
+ u_long addr, offset, slot;
struct vm_anon *anonp, anon;
struct vm_map_entry vme;
struct vm_amap amap;
@@ -140,27 +140,28 @@ _kvm_ureadm(kvm_t *kd, const struct mini
if (kd-swapspc == 0) {
kd-swapspc = _kvm_malloc(kd, kd-nbpg);
if (kd-swapspc == 0)
- return (0);
+ return (NULL);
}
/*
* Look through the address map for the memory object
* that corresponds to the given virtual address.
-* The header just has the entire valid range.
*/
- head = (u_long)p-p_vmspace-vm_map.header;
- addr = head;
+ addr = (u_long)RB_ROOT(p-p_vmspace-vm_map.addr);
while (1) {
+ if (addr == 0)
+ return (NULL);
if (KREAD(kd, addr, vme))
- return (0);
+ return (NULL);
- if (va = vme.start va vme.end
- vme.aref.ar_amap != NULL)
+ if (va vme.start)
+ addr = (u_long)RB_LEFT(vme, daddrs.addr_entry);
+ else if (va = vme.end + vme.fspace)
+ addr = (u_long)RB_RIGHT(vme, daddrs.addr_entry);
+ else if (va = vme.end)
+ return (NULL);
+ else
break;
-
- addr = (u_long)vme.next;
- if (addr == head)
- return (0);
}
/*
Index: usr.sbin/procmap/procmap.c
===
RCS file: /cvs/src/usr.sbin/procmap/procmap.c,v
retrieving revision 1.38
diff -u -d -p -r1.38 procmap.c
--- usr.sbin/procmap/procmap.c 23 Apr 2011 01:01:34 - 1.38
+++ usr.sbin/procmap/procmap.c 11 May 2011 12:09:24 -
@@ -169,10 +169,13 @@ struct nlist nl[] = {
void load_symbols(kvm_t *);
void process_map(kvm_t *, pid_t, struct kinfo_proc *, struct sum *);
-size_t dump_vm_map_entry(kvm_t *, struct kbit *, struct kbit *, int,
+struct vm_map_entry *load_vm_map_entries(kvm_t *, struct vm_map_entry *,
+struct vm_map_entry *);
+void unload_vm_map_entries(struct vm_map_entry *);
+size_t dump_vm_map_entry(kvm_t *, struct kbit *, struct vm_map_entry *,
struct sum *);
-char *findname(kvm_t *, struct kbit *, struct kbit *, struct kbit *,
- struct kbit *, struct kbit *);
+char *findname(kvm_t *, struct kbit *, struct vm_map_entry *, struct kbit *,
+struct kbit *, struct kbit *);
int search_cache(kvm_t *, struct kbit *, char **, char *, size_t);
#if 0
void load_name_cache(kvm_t *);
@@ -182,6 +185,19 @@ static void __dead usage(void);
static pid_t strtopid(const char *);
void print_sum(struct sum *, struct sum *);
+/*
+ * uvm_map address tree implementation.
+ */
+static int no_impl(void *, void *);
+static int
+no_impl(void *p, void *q)
+{
+ errx(1, uvm_map address comparison not implemented);
+ return 0;
+}
+
+RB_GENERATE(uvm_map_addr, vm_map_entry, daddrs.addr_entry, no_impl);
+
int
main(int argc, char *argv[])
{
@@ -347,12 +363,12 @@ print_sum(struct sum *sum, struct sum *t
void
process_map(kvm_t *kd, pid_t pid, struct kinfo_proc *proc, struct sum *sum)
{
- struct kbit kbit[4], *vmspace, *vm_map, *header, *vm_map_entry;
- struct vm_map_entry *last;
- u_long addr, next;
+ struct kbit kbit[3], *vmspace, *vm_map;
+ struct vm_map_entry *vm_map_entry;
size_t total = 0;
char *thing;
uid_t uid;
+ int vmmap_flags;
if ((uid = getuid())) {
if (pid == 0) {
@@ -367,13 +383,9 @@ process_map(kvm_t *kd, pid_t pid, struct
vmspace = kbit[0];
vm_map = kbit[1];
- header = kbit[2];
- vm_map_entry = kbit[3];
A(vmspace) = 0;
A(vm_map) = 0;
- A(header) = 0;
-
Re: vmmap replacement, please test
On Wed, May 11, 2011 at 03:43:19AM +0200, Ariane van der Steldt wrote:
The newest version of vmmap (as of now) is vmmap_sys.diff.26
Since the diff is scheduled to go in may 20 and has a lot of changes and
fixes, please test this diff and report any failures and successes.
I use this separate e-mail to publish the userspace part of the diff.
--
Ariane
Index: lib/libkvm/kvm_proc.c
===
RCS file: /cvs/src/lib/libkvm/kvm_proc.c,v
retrieving revision 1.42
diff -u -d -p -r1.42 kvm_proc.c
--- lib/libkvm/kvm_proc.c 12 Mar 2011 04:54:28 - 1.42
+++ lib/libkvm/kvm_proc.c 10 May 2011 22:23:15 -
@@ -131,7 +131,7 @@ static void ps_str_e(struct ps_strings *
static char *
_kvm_ureadm(kvm_t *kd, const struct miniproc *p, u_long va, u_long *cnt)
{
- u_long addr, head, offset, slot;
+ u_long addr, offset, slot;
struct vm_anon *anonp, anon;
struct vm_map_entry vme;
struct vm_amap amap;
@@ -140,27 +140,28 @@ _kvm_ureadm(kvm_t *kd, const struct mini
if (kd-swapspc == 0) {
kd-swapspc = _kvm_malloc(kd, kd-nbpg);
if (kd-swapspc == 0)
- return (0);
+ return (NULL);
}
/*
* Look through the address map for the memory object
* that corresponds to the given virtual address.
-* The header just has the entire valid range.
*/
- head = (u_long)p-p_vmspace-vm_map.header;
- addr = head;
+ addr = (u_long)RB_ROOT(p-p_vmspace-vm_map.addr);
while (1) {
+ if (addr == 0)
+ return (NULL);
if (KREAD(kd, addr, vme))
- return (0);
+ return (NULL);
- if (va = vme.start va vme.end
- vme.aref.ar_amap != NULL)
+ if (va vme.start)
+ addr = (u_long)RB_LEFT(vme, addr_entry);
+ else if (va = vme.end + vme.fspace)
+ addr = (u_long)RB_RIGHT(vme, addr_entry);
+ else if (va = vme.end)
+ return (NULL);
+ else
break;
-
- addr = (u_long)vme.next;
- if (addr == head)
- return (0);
}
/*
Index: usr.sbin/procmap/procmap.c
===
RCS file: /cvs/src/usr.sbin/procmap/procmap.c,v
retrieving revision 1.38
diff -u -d -p -r1.38 procmap.c
--- usr.sbin/procmap/procmap.c 23 Apr 2011 01:01:34 - 1.38
+++ usr.sbin/procmap/procmap.c 10 May 2011 22:23:23 -
@@ -169,10 +169,13 @@ struct nlist nl[] = {
void load_symbols(kvm_t *);
void process_map(kvm_t *, pid_t, struct kinfo_proc *, struct sum *);
-size_t dump_vm_map_entry(kvm_t *, struct kbit *, struct kbit *, int,
+struct vm_map_entry *load_vm_map_entries(kvm_t *, struct vm_map_entry *,
+struct vm_map_entry *);
+void unload_vm_map_entries(struct vm_map_entry *);
+size_t dump_vm_map_entry(kvm_t *, struct kbit *, struct vm_map_entry *,
struct sum *);
-char *findname(kvm_t *, struct kbit *, struct kbit *, struct kbit *,
- struct kbit *, struct kbit *);
+char *findname(kvm_t *, struct kbit *, struct vm_map_entry *, struct kbit *,
+struct kbit *, struct kbit *);
int search_cache(kvm_t *, struct kbit *, char **, char *, size_t);
#if 0
void load_name_cache(kvm_t *);
@@ -182,6 +185,19 @@ static void __dead usage(void);
static pid_t strtopid(const char *);
void print_sum(struct sum *, struct sum *);
+/*
+ * uvm_map address tree implementation.
+ */
+static int no_impl(void *, void *);
+static int
+no_impl(void *p, void *q)
+{
+ errx(1, uvm_map address comparison not implemented);
+ return 0;
+}
+
+RB_GENERATE(uvm_map_addr, vm_map_entry, daddrs.addr_entry, no_impl);
+
int
main(int argc, char *argv[])
{
@@ -347,12 +363,12 @@ print_sum(struct sum *sum, struct sum *t
void
process_map(kvm_t *kd, pid_t pid, struct kinfo_proc *proc, struct sum *sum)
{
- struct kbit kbit[4], *vmspace, *vm_map, *header, *vm_map_entry;
- struct vm_map_entry *last;
- u_long addr, next;
+ struct kbit kbit[3], *vmspace, *vm_map;
+ struct vm_map_entry *vm_map_entry;
size_t total = 0;
char *thing;
uid_t uid;
+ int vmmap_flags;
if ((uid = getuid())) {
if (pid == 0) {
@@ -367,13 +383,9 @@ process_map(kvm_t *kd, pid_t pid, struct
vmspace = kbit[0];
vm_map = kbit[1];
- header = kbit[2];
- vm_map_entry = kbit[3];
A(vmspace) = 0;
A(vm_map) = 0;
- A(header) = 0;
- A(vm_map_entry) = 0;
if (pid 0) {
A(vmspace) = (u_long)proc-p_vmspace;
@@ -416,34 +428,41 @@ process_map(kvm_t *kd, pid_t pid, struct
printf(%s %p = {, thing, P(vm_map));
