Re: panic with panic: kmem_malloc(4096): kmem_map too small...
[EMAIL PROTECTED] wrote in [EMAIL PROTECTED]: phk I only have 2G ram and that's what I have tested (extensively). If we're phk still broken for 2G ram, somebody needs to revist this. phk phk One thing you can try is reduce the value of the phksysctl kern.maxvnodes phk phk If you set it to the same value as used for 2G (appros 13), I phk think your machine should survive with 3G RAM. Thank you for the clarification. I notice that the panic happens when numvnodes becomes more than about 185000. When kern.maxvnodes=13 (198799 by default) is specified, the system works fine under heavy loads during this week. Is anyone else suffering from it? If it happens on all of 2GB systems, I think it should be solved (or described in relnotes) before the release. -- | Hiroki SATO [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: panic with panic: kmem_malloc(4096): kmem_map too small...
I was able to copy the full 100+Gb. Next I'm going to try and fill the disk to the max as user, but i guess it'll not trigger this bug. And to that fact I have a question: At the moment 8% of the disk is reserved. It being a 170Gb raid, that wastes a good 13,6Gb, which I find at lot. tunefs lets me bring that down to 5% = 8,5Gb without speed penalty. But is for this size of spare space such a large threshold really required?? And IF i would like to experiment, where do I look for the knop to turn?? Thanx for the support, --WjW In message 03a701c2b38c$8e3ad990$471b3dd4@dual, Willem Jan Withagen writes: Which seems a problem sticking up it's head once so often. I had it happen to me now 3 times over the last day. It just drops into the debugger. And I've foun little extra info in the archive. What dows this actually mean? Is something leaking in the kernel. IF so how do I help it go away. I'm copy 100G from a W2K system to my vinum file server with a 170G raid5. Current is as of 28 dec... Please try to move up to current as of today. On Dec 29th I commited code to make the desiredvnodes a limit rather than a vague suggestion and that should solve your problem I hope. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 [EMAIL PROTECTED] | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. ¡Iì¹» ®Þ±éݨ¥¶Ý¢jçH:+éì¹» ®Þ~·nÇ\ººÞا¶¡Ü¨~Ø^ë,j
Re: panic with panic: kmem_malloc(4096): kmem_map too small...
In message 01e701c2b62b$db07ddd0$471b3dd4@dual, Willem Jan Withagen writes: I was able to copy the full 100+Gb. Next I'm going to try and fill the disk to the max as user, but i guess it'll not trigger this bug. And to that fact I have a question: At the moment 8% of the disk is reserved. It being a 170Gb raid, that wastes a good 13,6Gb, which I find at lot. tunefs lets me bring that down to 5% = 8,5Gb without speed penalty. But is for this size of spare space such a large threshold really required?? And IF i would like to experiment, where do I look for the knop to turn?? Basically you don't need to reserve anything, but as you get closer to filling the disk the time to find a free space increases rapidly and your files get very fragmented. Trouble is: they never get defragmented unless you copy them (or do a full dump/restore). I'm not sure how to nail the right number of % down, and I'm sure that both Kirk and I would like to hear your numbers :-) -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 [EMAIL PROTECTED] | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: panic with panic: kmem_malloc(4096): kmem_map too small...
On Tue, 7 Jan 2003 [EMAIL PROTECTED] wrote: In message 01e701c2b62b$db07ddd0$471b3dd4@dual, Willem Jan Withagen writes: I was able to copy the full 100+Gb. Next I'm going to try and fill the disk to the max as user, but i guess it'll not trigger this bug. And to that fact I have a question: At the moment 8% of the disk is reserved. It being a 170Gb raid, that wastes a good 13,6Gb, which I find at lot. tunefs lets me bring that down to 5% = 8,5Gb without speed penalty. But is for this size of spare space such a large threshold really required?? And IF i would like to experiment, where do I look for the knop to turn?? Basically you don't need to reserve anything, but as you get closer to filling the disk the time to find a free space increases rapidly and your files get very fragmented. Trouble is: they never get defragmented unless you copy them (or do a full dump/restore). File get quite fragmented (enough to lose a factor of 2 or so of the disk's bandwidth) even when the disk is almost empty. Then they don't get defragmented unless you copy them, etc. The Real Fragmentation that occurs when a disk is nearly full loses a much larger factor of the disk's bandwidth. I'm not sure how to nail the right number of % down, and I'm sure that both Kirk and I would like to hear your numbers :-) I postprocess output from Kirk's block number printing program to produce fragmentation estimates. All (non-null) seeks are considered to be fragmentation. A (logical) non-null seek seems to be normal for about 10% of (logical) i/o's even in the favourable case of files created by copying to an initially empty filesystem (this is for small files like the ones in FreeBSD's src tree). Bruce To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: panic with panic: kmem_malloc(4096): kmem_map too small...
File get quite fragmented (enough to lose a factor of 2 or so of the
disk's bandwidth) even when the disk is almost empty. Then they don't
get defragmented unless you copy them, etc. The Real Fragmentation
that occurs when a disk is nearly full loses a much larger factor of
the disk's bandwidth.
I'm not sure how to nail the right number of % down, and I'm sure
that both Kirk and I would like to hear your numbers :-)
I postprocess output from Kirk's block number printing program to
produce fragmentation estimates. All (non-null) seeks are considered
to be fragmentation. A (logical) non-null seek seems to be normal for
about 10% of (logical) i/o's even in the favourable case of files
created by copying to an initially empty filesystem (this is for
small files like the ones in FreeBSD's src tree).
I'd be interested in that block number printing program too
After some discussion offline with Poul-Henning that was exactly what is one of the
basic needs to do any analysis in this area:
A way to determing a fragmentation degree/index.
Poul suggested then bean-counting in fsck, so I started looking there. But the 'block
number printing' would generate even better data.
I'm building a system just for running some of these test on it.
Feel free to suggest or discuss the approach.
--WjW
èR{.nÇ+·¬zwfj)m¢f£¢·hkyàRàÂ+aº{.nÇ+·ç±×.®·§¶)í
æèw* ¶¦zË
Re: panic with panic: kmem_malloc(4096): kmem_map too small...
Thus spake [EMAIL PROTECTED] [EMAIL PROTECTED]: And to that fact I have a question: At the moment 8% of the disk is reserved. It being a 170Gb raid, that wastes a good 13,6Gb, which I find at lot. tunefs lets me bring that down to 5% = 8,5Gb without speed penalty. But is for this size of spare space such a large threshold really required?? And IF i would like to experiment, where do I look for the knop to turn?? Basically you don't need to reserve anything, but as you get closer to filling the disk the time to find a free space increases rapidly and your files get very fragmented. Trouble is: they never get defragmented unless you copy them (or do a full dump/restore). I'm not sure how to nail the right number of % down, and I'm sure that both Kirk and I would like to hear your numbers :-) If someone is interested in investigating this in detail, Margo Setzer et alii did a study a few years ago on aging filesystems in order to measure fragmentation. Their intent was not to measure the effect of free space on fragmentation, but their methodology could be applied for that purpose. They happen to have a graph that suggests that large file throughput drops about 20% for a filesystem that is 75% full versus a nearly empty filesystem. http://www.eecs.harvard.edu/~vino/fs-perf/papers/sigmetrics.ps.gz I don't think you'll find that it's a good idea to have a filesystem more than 90% full. At that load factor, even a uniform hash will take on average 2.5 probes to locate free space, and performance can only exponentially decrease from there. No filesystem can avoid fragmentation and perform well without a bit of breathing room. To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: panic with panic: kmem_malloc(4096): kmem_map too small...
In message 03a701c2b38c$8e3ad990$471b3dd4@dual, Willem Jan Withagen writes: Which seems a problem sticking up it's head once so often. I had it happen to me now 3 times over the last day. It just drops into the debugger. And I've foun little extra info in the archive. What dows this actually mean? Is something leaking in the kernel. IF so how do I help it go away. I'm copy 100G from a W2K system to my vinum file server with a 170G raid5. Current is as of 28 dec... Please try to move up to current as of today. On Dec 29th I commited code to make the desiredvnodes a limit rather than a vague suggestion and that should solve your problem I hope. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 [EMAIL PROTECTED] | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: panic with panic: kmem_malloc(4096): kmem_map too small...
Hi, [EMAIL PROTECTED] wrote in [EMAIL PROTECTED]: phk In message 03a701c2b38c$8e3ad990$471b3dd4@dual, Willem Jan Withagen writes: phk Which seems a problem sticking up it's head once so often. phk I had it happen to me now 3 times over the last day. It just drops into the debugger. phk And I've foun little extra info in the archive. phk phk What dows this actually mean? Is something leaking in the kernel. phk IF so how do I help it go away. phk phk I'm copy 100G from a W2K system to my vinum file server with a 170G raid5. phk Current is as of 28 dec... phk phk Please try to move up to current as of today. On Dec 29th I commited phk code to make the desiredvnodes a limit rather than a vague suggestion phk and that should solve your problem I hope. I also had kmem_malloc(4096): kmem_map too small: 275378176 total allocated several times on -current as of Jan 4th. My -current box has 3GB memory, but when the memory size is explicitly specified as 2GB via MAXMEM option, the panic disappears (but I don't know why...). -- | Hiroki SATO [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: panic with panic: kmem_malloc(4096): kmem_map too small...
In message [EMAIL PROTECTED], Hiroki Sato writes: I also had kmem_malloc(4096): kmem_map too small: 275378176 total allocated several times on -current as of Jan 4th. My -current box has 3GB memory, but when the memory size is explicitly specified as 2GB via MAXMEM option, the panic disappears (but I don't know why...). Now that's different from the previous poster. The problem in this case is that some of the system constants are sized based on the amount of RAM and appearantly we do this wrong for large RAM configurations. I only have 2G ram and that's what I have tested (extensively). If we're still broken for 2G ram, somebody needs to revist this. One thing you can try is reduce the value of the sysctl kern.maxvnodes If you set it to the same value as used for 2G (appros 13), I think your machine should survive with 3G RAM. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 [EMAIL PROTECTED] | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
VM page queue mutex not locked panic (WAS:Re: panic with panic: kmem_malloc(4096): kmem_map too small... )
In message 03a701c2b38c$8e3ad990$471b3dd4@dual, Willem Jan Withagen writes:
Which seems a problem sticking up it's head once so often.
I had it happen to me now 3 times over the last day. It just drops into the
debugger.
And I've foun little extra info in the archive.
What dows this actually mean? Is something leaking in the kernel.
IF so how do I help it go away.
I'm copy 100G from a W2K system to my vinum file server with a 170G raid5.
Current is as of 28 dec...
Please try to move up to current as of today. On Dec 29th I commited
code to make the desiredvnodes a limit rather than a vague suggestion
and that should solve your problem I hope.
It seams easy to repeat, AND I haven't copied the whole disk yet.
So there plenty test material. I'll keep you posted.
Today means European saturday during the day? (I'm auto-build everything, cvsup-ing at
6:00)
But the following question is alrady there.
When I woke up this morning I found my box with a double panic:
lock (sleep mutex) VM page queue mutex not locked @
/usr/src/sys/kern/vf
[the remainder was not on the screen]
Is this related, our should this start a new thread?
--WjW
èR{.nÇ+·¬zwfj)m¢f£¢·hkyàRàÂ+aº{.nÇ+·ç±×.®·§¶)í
æèw* ¶¦zË
Re: VM page queue mutex not locked panic (WAS:Re: panic with panic: kmem_malloc(4096): kmem_map too small... )
In message 051f01c2b42e$e4651400$471b3dd4@dual, Willem Jan Withagen writes: But the following question is alrady there. When I woke up this morning I found my box with a double panic: lock (sleep mutex) VM page queue mutex not locked @ /usr/src/sys/kern/vf [the remainder was not on the screen] Is this related, our should this start a new thread? This is unrelated I think, and sounds like some of the stuff alc@ is working on. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 [EMAIL PROTECTED] | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: panic with panic: kmem_malloc(4096): kmem_map too small...
Hiroki Sato wrote: I also had kmem_malloc(4096): kmem_map too small: 275378176 total allocated several times on -current as of Jan 4th. My -current box has 3GB memory, but when the memory size is explicitly specified as 2GB via MAXMEM option, the panic disappears (but I don't know why...). Because the kmem map is a fixed segment of virtual space for which page mappings are established, and adding more memory increases the amount of page mappings required, without increasing the space available for the mappings. It is not something which can be easily tuned automatically. -- Terry To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: panic with panic: kmem_malloc(4096): kmem_map too small...
[EMAIL PROTECTED] wrote: In message [EMAIL PROTECTED], Hiroki Sato writes: I also had kmem_malloc(4096): kmem_map too small: 275378176 total allocated several times on -current as of Jan 4th. My -current box has 3GB memory, but when the memory size is explicitly specified as 2GB via MAXMEM option, the panic disappears (but I don't know why...). Now that's different from the previous poster. The problem in this case is that some of the system constants are sized based on the amount of RAM and appearantly we do this wrong for large RAM configurations. I only have 2G ram and that's what I have tested (extensively). If we're still broken for 2G ram, somebody needs to revist this. One thing you can try is reduce the value of the sysctl kern.maxvnodes If you set it to the same value as used for 2G (appros 13), I think your machine should survive with 3G RAM. You can also increase the maximum number of open files, which will cause the number to be scaled larger. You can do this in the boot loader (e.g. loader.rc). -- Terry To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
panic with panic: kmem_malloc(4096): kmem_map too small...
Which seems a problem sticking up it's head once so often.
I had it happen to me now 3 times over the last day. It just drops into the debugger.
And I've foun little extra info in the archive.
What dows this actually mean? Is something leaking in the kernel.
IF so how do I help it go away.
I'm copy 100G from a W2K system to my vinum file server with a 170G raid5.
Current is as of 28 dec...
System has 96Mb and runs on a P-II 200Mhz/realtek ethernet.
It just runs: kernel/nfs/ssh/samba/sendmail
The other problem the system suffers from is running a backgrounded fsck on
the vinum raid. That is a shure way to freeze the system.
Running it in the forground always works fine. So that problem could be the one that
was discussed on the list before??
Regards,
--WjW
N
'²æìr¸zǧvf¢Új:+v¨· è® ¶§²æìr¸yúÞy» rêëz{bØ^nr¡ûazg¬±¨
Re: Kernel panic with panic: kmem_malloc(4096): kmem_map too small...
On Tue, 15 Oct 2002, Makoto Matsushita wrote: I'm now trying Terry's patch (just rebuilding a kernel). jroberson You are using 100mb of KVA for malloc(9)? Are you certain jroberson that you don't have a memory leak? Maybe there's a chance of a memory leakage by GLOBAL, but I don't sure. jroberson How much memory is in this machine? What are you using it jroberson for? It has 256MB memory, and is used for 5-current release buildbox. I suspect that there is some other bug then. 1/2 of your memory should not be consumed by kernel malloc. Do you have an abnormally large MD or something? Jeff To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Kernel panic with panic: kmem_malloc(4096): kmem_map too small...
After upgrading my 5-current box (as of late September 2002), the kernel panics periodically with following message: panic: kmem_malloc(4096): kmem_map too small: 107651072 total allocated The number '4096' and '107651072' is always the same. What am I missing something? -- - Makoto `MAR' Matsushita To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: Kernel panic with panic: kmem_malloc(4096): kmem_map too small...
Makoto Matsushita wrote: After upgrading my 5-current box (as of late September 2002), the kernel panics periodically with following message: panic: kmem_malloc(4096): kmem_map too small: 107651072 total allocated The number '4096' and '107651072' is always the same. What am I missing something? This was recently discussed on -current. I posted a dumb patch that fixes the problem. Jeff Roberson is looking at fixing the problem the right way right now. Until then, though, you can still use my patch (it makes UMA use kmem_alloc_wait, instead of kmem_malloc). See the archive of the posting, for more details: http://docs.freebsd.org/cgi/getmsg.cgi?fetch=1612532+0+archive/2002/freebsd-current/20021013.freebsd-current -- Terry To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: Kernel panic with panic: kmem_malloc(4096): kmem_map too small...
On Mon, 14 Oct 2002, Makoto Matsushita wrote: After upgrading my 5-current box (as of late September 2002), the kernel panics periodically with following message: panic: kmem_malloc(4096): kmem_map too small: 107651072 total allocated The number '4096' and '107651072' is always the same. What am I missing something? You are using 100mb of KVA for malloc(9)? Are you certain that you don't have a memory leak? How much memory is in this machine? What are you using it for? Thanks! Jeff To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
