Re: panic with panic: kmem_malloc(4096): kmem_map too small...

[Hiroki Sato]

[EMAIL PROTECTED] wrote
  in <[EMAIL PROTECTED]>:

phk> I only have 2G ram and that's what I have tested (extensively).  If we're
phk> still broken for >2G ram, somebody needs to revist this.
phk> 
phk> One thing you can try is reduce the value of the
phk>sysctl kern.maxvnodes
phk> 
phk> If you set it to the same value as used for 2G (appros 13), I
phk> think your machine should survive with 3G RAM.

 Thank you for the clarification.  I notice that the panic
 happens when numvnodes becomes more than about 185000.
 When kern.maxvnodes=13 (198799 by default) is specified,
 the system works fine under heavy loads during this week.

 Is anyone else suffering from it?  If it happens on all of >2GB
 systems, I think it should be solved (or described in relnotes)
 before the release.

-- 
| Hiroki SATO  <[EMAIL PROTECTED]>

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: panic with panic: kmem_malloc(4096): kmem_map too small...

[David Schultz]

Thus spake [EMAIL PROTECTED] <[EMAIL PROTECTED]>:
> >And to that fact I have a question:
> >At the moment 8% of the disk is reserved. 
> >It being a 170Gb raid, that wastes a good 13,6Gb, which I find at lot.
> >tunefs lets me bring that down to 5% = 8,5Gb without speed penalty.
> >But is for this size of spare space such a large threshold really required??
> >And IF i would like to experiment, where do I look for the knop to turn??
> 
> Basically you don't need to reserve anything, but as you get closer to
> filling the disk the time to find a free space increases rapidly and
> your files get very fragmented.  Trouble is: they never get defragmented
> unless you copy them (or do a full dump/restore).
> 
> I'm not sure how to nail the "right" number of % down, and I'm sure
> that both Kirk and I would like to hear your numbers :-)

If someone is interested in investigating this in detail, Margo
Setzer et alii did a study a few years ago on aging filesystems in
order to measure fragmentation.  Their intent was not to measure
the effect of free space on fragmentation, but their methodology
could be applied for that purpose.  They happen to have a graph
that suggests that large file throughput drops about 20% for a
filesystem that is 75% full versus a nearly empty filesystem.

http://www.eecs.harvard.edu/~vino/fs-perf/papers/sigmetrics.ps.gz

I don't think you'll find that it's a good idea to have a
filesystem more than 90% full.  At that load factor, even a
uniform hash will take on average 2.5 probes to locate free
space, and performance can only exponentially decrease from
there.  No filesystem can avoid fragmentation and perform well
without a bit of breathing room.

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: panic with panic: kmem_malloc(4096): kmem_map too small...

[Willem Jan Withagen]

> File get quite fragmented (enough to lose a factor of 2 or so of the
> disk's bandwidth) even when the disk is almost empty.  Then they don't
> get defragmented unless you copy them, etc.  The Real Fragmentation
> that occurs when a disk is nearly full loses a much larger factor of
> the disk's bandwidth.
> 
> > I'm not sure how to nail the "right" number of % down, and I'm sure
> > that both Kirk and I would like to hear your numbers :-)
> 
> I postprocess output from Kirk's block number printing program to
> produce fragmentation estimates.  All (non-null) seeks are considered
> to be fragmentation.  A (logical) non-null seek seems to be normal for
> about 10% of (logical) i/o's even in the favourable case of files
> created by copying to an initially empty filesystem (this is for
> small files like the ones in FreeBSD's src tree).

I'd be interested in that block number printing program too
After some discussion offline with Poul-Henning that was exactly what is one of the 
basic needs to do any analysis in this area: 
A way to determing a fragmentation degree/index.
Poul suggested then bean-counting in fsck, so I started looking there. But the 'block 
number printing' would generate even better data.

I'm building a system just for running some of these test on it.
Feel free to suggest or discuss the approach.

--WjW

èR{.nÇ+‰·¬zwfj)m¢f£¢·hškyàRŠàÂ+aº{.nÇ+‰·Ÿ­ç›±×.®·§¶)í…æèw*¶¦zˁ

Re: panic with panic: kmem_malloc(4096): kmem_map too small...

[Bruce Evans]

On Tue, 7 Jan 2003 [EMAIL PROTECTED] wrote:

> In message <01e701c2b62b$db07ddd0$471b3dd4@dual>, "Willem Jan Withagen" writes:
> >I was able to copy the full 100+Gb.
> >Next I'm going to try and fill the disk to the max as user, but i guess it'll not 
>trigger this bug.
> >
> >And to that fact I have a question:
> >At the moment 8% of the disk is reserved.
> >It being a 170Gb raid, that wastes a good 13,6Gb, which I find at lot.
> >tunefs lets me bring that down to 5% = 8,5Gb without speed penalty.
> >But is for this size of spare space such a large threshold really required??
> >And IF i would like to experiment, where do I look for the knop to turn??
>
> Basically you don't need to reserve anything, but as you get closer to
> filling the disk the time to find a free space increases rapidly and
> your files get very fragmented.  Trouble is: they never get defragmented
> unless you copy them (or do a full dump/restore).

File get quite fragmented (enough to lose a factor of 2 or so of the
disk's bandwidth) even when the disk is almost empty.  Then they don't
get defragmented unless you copy them, etc.  The Real Fragmentation
that occurs when a disk is nearly full loses a much larger factor of
the disk's bandwidth.

> I'm not sure how to nail the "right" number of % down, and I'm sure
> that both Kirk and I would like to hear your numbers :-)

I postprocess output from Kirk's block number printing program to
produce fragmentation estimates.  All (non-null) seeks are considered
to be fragmentation.  A (logical) non-null seek seems to be normal for
about 10% of (logical) i/o's even in the favourable case of files
created by copying to an initially empty filesystem (this is for
small files like the ones in FreeBSD's src tree).

Bruce


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: panic with panic: kmem_malloc(4096): kmem_map too small...

[phk]

In message <01e701c2b62b$db07ddd0$471b3dd4@dual>, "Willem Jan Withagen" writes:
>I was able to copy the full 100+Gb.
>Next I'm going to try and fill the disk to the max as user, but i guess it'll not 
>trigger this bug.
>
>And to that fact I have a question:
>At the moment 8% of the disk is reserved. 
>It being a 170Gb raid, that wastes a good 13,6Gb, which I find at lot.
>tunefs lets me bring that down to 5% = 8,5Gb without speed penalty.
>But is for this size of spare space such a large threshold really required??
>And IF i would like to experiment, where do I look for the knop to turn??

Basically you don't need to reserve anything, but as you get closer to
filling the disk the time to find a free space increases rapidly and
your files get very fragmented.  Trouble is: they never get defragmented
unless you copy them (or do a full dump/restore).

I'm not sure how to nail the "right" number of % down, and I'm sure
that both Kirk and I would like to hear your numbers :-)

-- 
Poul-Henning Kamp   | UNIX since Zilog Zeus 3.20
[EMAIL PROTECTED] | TCP/IP since RFC 956
FreeBSD committer   | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: panic with panic: kmem_malloc(4096): kmem_map too small...

[Willem Jan Withagen]

I was able to copy the full 100+Gb.
Next I'm going to try and fill the disk to the max as user, but i guess it'll not 
trigger this bug.

And to that fact I have a question:
At the moment 8% of the disk is reserved. 
It being a 170Gb raid, that wastes a good 13,6Gb, which I find at lot.
tunefs lets me bring that down to 5% = 8,5Gb without speed penalty.
But is for this size of spare space such a large threshold really required??
And IF i would like to experiment, where do I look for the knop to turn??

Thanx for the support,
--WjW

> In message <03a701c2b38c$8e3ad990$471b3dd4@dual>, "Willem Jan Withagen" writes:
> >Which seems a problem sticking up it's head once so often.
> >I had it happen to me now 3 times over the last day. It just drops into the 
>debugger.
> >And I've foun little extra info in the archive.
> >
> >What dows this actually mean? Is something leaking in the kernel.
> >IF so how do I help it go away.
> >
> >I'm copy 100G from a W2K system to my vinum file server with a 170G raid5.
> >Current is as of 28 dec...
> 
> Please try to move up to current as of today.  On Dec 29th I commited
> code to make the desiredvnodes a limit rather than a vague suggestion
> and that should solve your problem I hope.
> 
> -- 
> Poul-Henning Kamp   | UNIX since Zilog Zeus 3.20
> [EMAIL PROTECTED] | TCP/IP since RFC 956
> FreeBSD committer   | BSD since 4.3-tahoe
> Never attribute to malice what can adequately be explained by incompetence.
> 
> ¡Iì¹»®&Þ±éݙ¨¥¶‰šŽŠÝ¢j­çH:+ƒ­†éì¹»®&Þ~·žnÇ\ººÞžØ§¶›¡Ü¨~Ø^™ë,j

Re: panic with panic: kmem_malloc(4096): kmem_map too small...

[Terry Lambert]

[EMAIL PROTECTED] wrote:
> In message <[EMAIL PROTECTED]>, Hiroki Sato writes:
> > I also had "kmem_malloc(4096): kmem_map too small: 275378176 total allocated"
> > several times on -current as of Jan 4th.  My -current box has 3GB memory,
> > but when the memory size is explicitly specified as 2GB via MAXMEM option,
> > the panic disappears (but I don't know why...).
> 
> Now that's different from the previous poster.
> 
> The problem in this case is that some of the system constants are sized
> based on the amount of RAM and appearantly we do this wrong for large
> RAM configurations.
> 
> I only have 2G ram and that's what I have tested (extensively).  If we're
> still broken for >2G ram, somebody needs to revist this.
> 
> One thing you can try is reduce the value of the
> sysctl kern.maxvnodes
> 
> If you set it to the same value as used for 2G (appros 13), I
> think your machine should survive with 3G RAM.

You can also increase the maximum number of open files, which will
cause the number to be scaled larger.  You can do this in the boot
loader (e.g. loader.rc).

-- Terry

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: panic with panic: kmem_malloc(4096): kmem_map too small...

[Terry Lambert]

Hiroki Sato wrote:
>  I also had "kmem_malloc(4096): kmem_map too small: 275378176 total allocated"
>  several times on -current as of Jan 4th.  My -current box has 3GB memory,
>  but when the memory size is explicitly specified as 2GB via MAXMEM option,
>  the panic disappears (but I don't know why...).

Because the kmem map is a fixed segment of virtual space for
which page mappings are established, and adding more memory
increases the amount of page mappings required, without
increasing the space available for the mappings.  It is not
something which can be easily tuned automatically.

-- Terry

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: VM page queue mutex not locked panic (WAS:Re: panic with panic: kmem_malloc(4096): kmem_map too small... )

[phk]

In message <051f01c2b42e$e4651400$471b3dd4@dual>, "Willem Jan Withagen" writes:

>But the following question is alrady there.
>When I woke up this morning I found my box with a double panic:
>lock (sleep mutex) VM page queue mutex not locked @
>/usr/src/sys/kern/vf
>[the remainder was not on the screen]
>
>Is this related, our should this start a new thread?

This is unrelated I think, and sounds like some of the stuff alc@ is
working on.

-- 
Poul-Henning Kamp   | UNIX since Zilog Zeus 3.20
[EMAIL PROTECTED] | TCP/IP since RFC 956
FreeBSD committer   | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

VM page queue mutex not locked panic (WAS:Re: panic with panic: kmem_malloc(4096): kmem_map too small... )

[Willem Jan Withagen]

> In message <03a701c2b38c$8e3ad990$471b3dd4@dual>, "Willem Jan Withagen" writes:
> >Which seems a problem sticking up it's head once so often.
> >I had it happen to me now 3 times over the last day. It just drops into the 
>debugger.
> >And I've foun little extra info in the archive.
> >
> >What dows this actually mean? Is something leaking in the kernel.
> >IF so how do I help it go away.
> >
> >I'm copy 100G from a W2K system to my vinum file server with a 170G raid5.
> >Current is as of 28 dec...
> 
> Please try to move up to current as of today.  On Dec 29th I commited
> code to make the desiredvnodes a limit rather than a vague suggestion
> and that should solve your problem I hope.

It seams easy to repeat, AND I haven't copied the whole disk yet.
So there plenty test material. I'll keep you posted.

Today means European saturday during the day? (I'm auto-build everything, cvsup-ing at 
6:00)

But the following question is alrady there.
When I woke up this morning I found my box with a double panic:
lock (sleep mutex) VM page queue mutex not locked @
/usr/src/sys/kern/vf
[the remainder was not on the screen]

Is this related, our should this start a new thread?

--WjW
èR{.nÇ+‰·¬zwfj)m¢f£¢·hškyàRŠàÂ+aº{.nÇ+‰·Ÿ­ç›±×.®·§¶)í…æèw*¶¦zˁ

Re: panic with panic: kmem_malloc(4096): kmem_map too small...

[phk]

In message <[EMAIL PROTECTED]>, Hiroki Sato writes:

> I also had "kmem_malloc(4096): kmem_map too small: 275378176 total allocated"
> several times on -current as of Jan 4th.  My -current box has 3GB memory,
> but when the memory size is explicitly specified as 2GB via MAXMEM option,
> the panic disappears (but I don't know why...).

Now that's different from the previous poster.

The problem in this case is that some of the system constants are sized
based on the amount of RAM and appearantly we do this wrong for large
RAM configurations.

I only have 2G ram and that's what I have tested (extensively).  If we're
still broken for >2G ram, somebody needs to revist this.

One thing you can try is reduce the value of the
sysctl kern.maxvnodes

If you set it to the same value as used for 2G (appros 13), I
think your machine should survive with 3G RAM.

-- 
Poul-Henning Kamp   | UNIX since Zilog Zeus 3.20
[EMAIL PROTECTED] | TCP/IP since RFC 956
FreeBSD committer   | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: panic with panic: kmem_malloc(4096): kmem_map too small...

[Hiroki Sato]

Hi,

[EMAIL PROTECTED] wrote
  in <[EMAIL PROTECTED]>:

phk> In message <03a701c2b38c$8e3ad990$471b3dd4@dual>, "Willem Jan Withagen" writes:
phk> >Which seems a problem sticking up it's head once so often.
phk> >I had it happen to me now 3 times over the last day. It just drops into the 
debugger.
phk> >And I've foun little extra info in the archive.
phk> >
phk> >What dows this actually mean? Is something leaking in the kernel.
phk> >IF so how do I help it go away.
phk> >
phk> >I'm copy 100G from a W2K system to my vinum file server with a 170G raid5.
phk> >Current is as of 28 dec...
phk> 
phk> Please try to move up to current as of today.  On Dec 29th I commited
phk> code to make the desiredvnodes a limit rather than a vague suggestion
phk> and that should solve your problem I hope.

 I also had "kmem_malloc(4096): kmem_map too small: 275378176 total allocated"
 several times on -current as of Jan 4th.  My -current box has 3GB memory,
 but when the memory size is explicitly specified as 2GB via MAXMEM option,
 the panic disappears (but I don't know why...).

-- 
| Hiroki SATO  <[EMAIL PROTECTED]>

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: panic with panic: kmem_malloc(4096): kmem_map too small...

[phk]

In message <03a701c2b38c$8e3ad990$471b3dd4@dual>, "Willem Jan Withagen" writes:
>Which seems a problem sticking up it's head once so often.
>I had it happen to me now 3 times over the last day. It just drops into the debugger.
>And I've foun little extra info in the archive.
>
>What dows this actually mean? Is something leaking in the kernel.
>IF so how do I help it go away.
>
>I'm copy 100G from a W2K system to my vinum file server with a 170G raid5.
>Current is as of 28 dec...

Please try to move up to current as of today.  On Dec 29th I commited
code to make the desiredvnodes a limit rather than a vague suggestion
and that should solve your problem I hope.

-- 
Poul-Henning Kamp   | UNIX since Zilog Zeus 3.20
[EMAIL PROTECTED] | TCP/IP since RFC 956
FreeBSD committer   | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

panic with panic: kmem_malloc(4096): kmem_map too small...

[Willem Jan Withagen]

Which seems a problem sticking up it's head once so often.
I had it happen to me now 3 times over the last day. It just drops into the debugger.
And I've foun little extra info in the archive.

What dows this actually mean? Is something leaking in the kernel.
IF so how do I help it go away.

I'm copy 100G from a W2K system to my vinum file server with a 170G raid5.
Current is as of 28 dec...

System has 96Mb and runs on a P-II 200Mhz/realtek ethernet.
It just runs: kernel/nfs/ssh/samba/sendmail

The other problem the system suffers from is running a backgrounded fsck on
the vinum raid. That is a shure way to freeze the system.
Running it in the forground always works fine. So that problem could be the one that 
was discussed on the list before??

Regards,
--WjW




N…'²æìr¸›zǧvf¢–Ú&j:+v‰¨·ž è®"¶§²æìr¸›yúÞy»rêëz{bžØ^n‡r¡ûazg¬±¨

Re: Kernel panic with panic: kmem_malloc(4096): kmem_map too small...

[Jeff Roberson]

On Tue, 15 Oct 2002, Makoto Matsushita wrote:

>
> I'm now trying Terry's patch (just rebuilding a kernel).
>
> jroberson> You are using 100mb of KVA for malloc(9)?  Are you certain
> jroberson> that you don't have a memory leak?
>
> Maybe there's a chance of a memory leakage by GLOBAL, but I don't sure.
>
> jroberson> How much memory is in this machine?  What are you using it
> jroberson> for?
>
> It has 256MB memory, and is used for 5-current release buildbox.
>

I suspect that there is some other bug then.  1/2 of your memory should
not be consumed by kernel malloc.  Do you have an abnormally large MD or
something?

Jeff


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: Kernel panic with panic: kmem_malloc(4096): kmem_map too small...

[Jeff Roberson]

On Mon, 14 Oct 2002, Makoto Matsushita wrote:

>
> After upgrading my 5-current box (as of late September 2002), the
> kernel panics periodically with following message:
>
> panic: kmem_malloc(4096): kmem_map too small: 107651072 total allocated
>
> The number '4096' and '107651072' is always the same.  What am I
> missing something?
>

You are using 100mb of KVA for malloc(9)?  Are  you certain that you don't
have a memory leak?  How much memory is in this machine?  What are you
using it for?

Thanks!
Jeff


To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Re: Kernel panic with panic: kmem_malloc(4096): kmem_map too small...

[Terry Lambert]

Makoto Matsushita wrote:
> After upgrading my 5-current box (as of late September 2002), the
> kernel panics periodically with following message:
> 
> panic: kmem_malloc(4096): kmem_map too small: 107651072 total allocated
> 
> The number '4096' and '107651072' is always the same.  What am I
> missing something?

This was recently discussed on -current.  I posted a dumb patch
that "fixes" the problem.

Jeff Roberson is looking at fixing the problem "the right way"
right now.

Until then, though, you can still use my patch (it makes UMA use
kmem_alloc_wait, instead of kmem_malloc).

See the archive of the posting, for more details:



-- Terry

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Kernel panic with panic: kmem_malloc(4096): kmem_map too small...

[Makoto Matsushita]

After upgrading my 5-current box (as of late September 2002), the
kernel panics periodically with following message:

panic: kmem_malloc(4096): kmem_map too small: 107651072 total allocated

The number '4096' and '107651072' is always the same.  What am I
missing something?

-- -
Makoto `MAR' Matsushita

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message