RE: Possible memory leak in the kernel (contigmalloc)

2018-10-31 Thread Vanco, Juraj
Hi Konstatin,

what we can directly see, is that the number of inactive pages increases and 
this happens after running single loop of contigmalloc -> contigfree even with 
the repeating size of allocated blocks.
The only source of inactive page could come from kernel memory management 
control structures.
Even if such structure becomes once inactive (cached pages), I do not see any 
reason that kernel cannot reclaim these inactive cached pages when again the 
contigmalloc asks for the same space of memory.

I think this cycle should not take all the amount of physical memory without 
being able to reclaim back when it's out of memory:

int array0[10] = {2097152, 1024, 2101248, 1024, 2097152, 2101248, 2101248, 
2097152, 2101248, 1024};
int array1[10] = {1024, 2101248, 2097152, 2101248, 2101248, 2097152, 1024, 
2101248, 1024, 2097152};
void *mem_blocks[10];

for (i = 0; i < 10; i++)
mem_blocks[i] = contigmalloc(array0[i], TEST_MEM, 0, 0, ~0, PAGE_SIZE, 
0);
for (i = 0; i < 10; i++)
contigfree(mem_blocks[i], array0[i], TEST_MEM);
for (i = 0; i < 10; i++)
mem_blocks[i] = contigmalloc(array1[i], TEST_MEM, 0, 0, ~0, PAGE_SIZE, 
0);
for (i = 0; i < 10; i++)
contigfree(mem_blocks[i], array1[i], TEST_MEM);

jv

-Original Message-
From: owner-freebsd-sta...@freebsd.org 
[mailto:owner-freebsd-sta...@freebsd.org] On Behalf Of Konstantin Belousov
Sent: Tuesday, October 30, 2018 6:36 PM
To: Bennett, Ciunas 
Cc: freebsd-stable@freebsd.org
Subject: Re: Possible memory leak in the kernel (contigmalloc)

On Tue, Oct 30, 2018 at 11:15:47AM +, Bennett, Ciunas wrote:
> Hi,
> The only other activity that is running is the csh script  that is inserting 
> and removing the kernel module.
> I am not using the VM for any other purpose but to run this test.
> In my tests the correlation between memory allocation and moving to inactive 
> list can be seen.
> I don't think any other process is creating the inactive memory.
But it is created.  More, since anon private memory is freed (not deactivated) 
on the process exit, something is accumulating the memory.

The other possibility is that the memory is the caching pages from vnodes, but 
for this buffers must be created and then reclaimed, which would suggest even 
more activity on the system.

> Thanks.
> 
> -Original Message-
> From: Konstantin Belousov [mailto:kostik...@gmail.com]
> Sent: Tuesday, October 30, 2018 10:12 AM
> To: Bennett, Ciunas 
> Cc: freebsd-stable@freebsd.org
> Subject: Re: Possible memory leak in the kernel (contigmalloc)
> 
> On Tue, Oct 30, 2018 at 09:46:59AM +, Bennett, Ciunas wrote:
> > Hi,
> > 
> > I was debugging the issue by viewing the free ques "sysctl 
> > vm.phys_free" and also using "show page" in ddb. The inactive memory 
> > is never being released back into the free que. I thought that when 
> > inactive memory reaches a certain threshold that the kernel will 
> > start reclaiming and move it to the free list? In my program this is 
> > not happening, the program uses free memory (contigmalloc), and then 
> > it is put into the inactive que (contiigfree) when the program frees it.
> Contigfree() does not release memory into inactive queue.  By definition, 
> inactive pages have valid content, which is not possible for the pages freed 
> by contigfree().
> contigfree()->kmem_free()->kmem_unback()->vm_page_free().
> 
> 
> > This inactive memory is never released by the kernel, and the 
> > inactive que grows until all the memory is in this que. I have 
> > attached a xml sheet that shows the memory usage in the system.
> Inactive memory is not freed, it makes no sense as far as there is 
> valid content, which is either not recoverable (anon memory or dirty 
> file
> pages) or high-cost to restore (clean file pages, need to re-read from disk). 
>  Inactive is processed by pagedaemon when system has memory deficit, and 
> either inactive pages are written to swap, or written to their file backing 
> storage.
> 
> I suspect that you have other activities on your system going on, which cause 
> creation of the inactive memory and unrecoverable fragmentation.
> Note that contigmalloc() tries to do defragmentation to satisfy requests, but 
> this is not always possible.
> 
> 
> > Ciunas.
> > 
> > -Original Message-
> > From: Konstantin Belousov [mailto:kostik...@gmail.com]
> > Sent: Friday, October 26, 2018 9:13 PM
> > To: Bennett, Ciunas 
> > Cc: freebsd-stable@freebsd.org
> > Subject: Re: Possible memory leak in the kernel (contigmalloc)
> > 
> > On Wed, Oct 24, 2018 at 04:27:52PM +, Bennett, Ciunas wrote:
> > > Hello,
> > > 
> > > 

Re: Possible memory leak in the kernel (contigmalloc)

2018-10-30 Thread Konstantin Belousov
On Tue, Oct 30, 2018 at 11:15:47AM +, Bennett, Ciunas wrote:
> Hi,
> The only other activity that is running is the csh script  that is inserting 
> and removing the kernel module.
> I am not using the VM for any other purpose but to run this test.
> In my tests the correlation between memory allocation and moving to inactive 
> list can be seen.
> I don't think any other process is creating the inactive memory.
But it is created.  More, since anon private memory is freed (not deactivated)
on the process exit, something is accumulating the memory.

The other possibility is that the memory is the caching pages from vnodes,
but for this buffers must be created and then reclaimed, which would suggest
even more activity on the system.

> Thanks.
> 
> -Original Message-
> From: Konstantin Belousov [mailto:kostik...@gmail.com] 
> Sent: Tuesday, October 30, 2018 10:12 AM
> To: Bennett, Ciunas 
> Cc: freebsd-stable@freebsd.org
> Subject: Re: Possible memory leak in the kernel (contigmalloc)
> 
> On Tue, Oct 30, 2018 at 09:46:59AM +, Bennett, Ciunas wrote:
> > Hi,
> > 
> > I was debugging the issue by viewing the free ques "sysctl 
> > vm.phys_free" and also using "show page" in ddb. The inactive memory 
> > is never being released back into the free que. I thought that when 
> > inactive memory reaches a certain threshold that the kernel will start 
> > reclaiming and move it to the free list? In my program this is not 
> > happening, the program uses free memory (contigmalloc), and then it is 
> > put into the inactive que (contiigfree) when the program frees it.
> Contigfree() does not release memory into inactive queue.  By definition, 
> inactive pages have valid content, which is not possible for the pages freed 
> by contigfree().
> contigfree()->kmem_free()->kmem_unback()->vm_page_free().
> 
> 
> > This inactive memory is never released by the kernel, and the inactive 
> > que grows until all the memory is in this que. I have attached a xml 
> > sheet that shows the memory usage in the system.
> Inactive memory is not freed, it makes no sense as far as there is valid 
> content, which is either not recoverable (anon memory or dirty file
> pages) or high-cost to restore (clean file pages, need to re-read from disk). 
>  Inactive is processed by pagedaemon when system has memory deficit, and 
> either inactive pages are written to swap, or written to their file backing 
> storage.
> 
> I suspect that you have other activities on your system going on, which cause 
> creation of the inactive memory and unrecoverable fragmentation.
> Note that contigmalloc() tries to do defragmentation to satisfy requests, but 
> this is not always possible.
> 
> 
> > Ciunas.
> > 
> > -Original Message-
> > From: Konstantin Belousov [mailto:kostik...@gmail.com]
> > Sent: Friday, October 26, 2018 9:13 PM
> > To: Bennett, Ciunas 
> > Cc: freebsd-stable@freebsd.org
> > Subject: Re: Possible memory leak in the kernel (contigmalloc)
> > 
> > On Wed, Oct 24, 2018 at 04:27:52PM +, Bennett, Ciunas wrote:
> > > Hello,
> > > 
> > > I have encountered an issue with a kernel application that I have 
> > > written, the issue might be caused by a memory leak in the kernel.
> > > The application allocates and deallocates contiguous memory using
> > > contigmalloc() and contigfree(). The application will fail after a 
> > > period of time because there is not enough free contiguous memory 
> > > left. There could be an issue with the freeing of memory when using 
> > > the contigfree() function.
> > >
> > 
> > It is unlikely that there is an issue with a leak, but I would be not 
> > surprised if your allocation/free pattern would cause fragmentation on free 
> > lists that results in contigmalloc(9) failures after.
> > 
> > Look at the vmstat -z/vmstat -m output to see uma and malloc stats.
> > More interesting for your case can be the output from
> > sysctl vm.phys_free
> > which provides information about the free queues and order of free pages on 
> > them.
> > --
> > Intel Research and Development Ireland Limited Registered in Ireland 
> > Registered Office: Collinstown Industrial Park, Leixlip, County 
> > Kildare Registered Number: 308263
> > 
> > 
> > This e-mail and any attachments may contain confidential material for 
> > the sole use of the intended recipient(s). Any review or distribution 
> > by others is strictly prohibited. If you are not the intended 
> > recipient, 

RE: Possible memory leak in the kernel (contigmalloc)

2018-10-30 Thread Bennett, Ciunas
Hi,
The only other activity that is running is the csh script  that is inserting 
and removing the kernel module.
I am not using the VM for any other purpose but to run this test.
In my tests the correlation between memory allocation and moving to inactive 
list can be seen.
I don't think any other process is creating the inactive memory.
Thanks.

-Original Message-
From: Konstantin Belousov [mailto:kostik...@gmail.com] 
Sent: Tuesday, October 30, 2018 10:12 AM
To: Bennett, Ciunas 
Cc: freebsd-stable@freebsd.org
Subject: Re: Possible memory leak in the kernel (contigmalloc)

On Tue, Oct 30, 2018 at 09:46:59AM +, Bennett, Ciunas wrote:
> Hi,
> 
> I was debugging the issue by viewing the free ques "sysctl 
> vm.phys_free" and also using "show page" in ddb. The inactive memory 
> is never being released back into the free que. I thought that when 
> inactive memory reaches a certain threshold that the kernel will start 
> reclaiming and move it to the free list? In my program this is not 
> happening, the program uses free memory (contigmalloc), and then it is 
> put into the inactive que (contiigfree) when the program frees it.
Contigfree() does not release memory into inactive queue.  By definition, 
inactive pages have valid content, which is not possible for the pages freed by 
contigfree().
contigfree()->kmem_free()->kmem_unback()->vm_page_free().


> This inactive memory is never released by the kernel, and the inactive 
> que grows until all the memory is in this que. I have attached a xml 
> sheet that shows the memory usage in the system.
Inactive memory is not freed, it makes no sense as far as there is valid 
content, which is either not recoverable (anon memory or dirty file
pages) or high-cost to restore (clean file pages, need to re-read from disk).  
Inactive is processed by pagedaemon when system has memory deficit, and either 
inactive pages are written to swap, or written to their file backing storage.

I suspect that you have other activities on your system going on, which cause 
creation of the inactive memory and unrecoverable fragmentation.
Note that contigmalloc() tries to do defragmentation to satisfy requests, but 
this is not always possible.


> Ciunas.
> 
> -Original Message-
> From: Konstantin Belousov [mailto:kostik...@gmail.com]
> Sent: Friday, October 26, 2018 9:13 PM
> To: Bennett, Ciunas 
> Cc: freebsd-stable@freebsd.org
> Subject: Re: Possible memory leak in the kernel (contigmalloc)
> 
> On Wed, Oct 24, 2018 at 04:27:52PM +, Bennett, Ciunas wrote:
> > Hello,
> > 
> > I have encountered an issue with a kernel application that I have 
> > written, the issue might be caused by a memory leak in the kernel.
> > The application allocates and deallocates contiguous memory using
> > contigmalloc() and contigfree(). The application will fail after a 
> > period of time because there is not enough free contiguous memory 
> > left. There could be an issue with the freeing of memory when using 
> > the contigfree() function.
> >
> 
> It is unlikely that there is an issue with a leak, but I would be not 
> surprised if your allocation/free pattern would cause fragmentation on free 
> lists that results in contigmalloc(9) failures after.
> 
> Look at the vmstat -z/vmstat -m output to see uma and malloc stats.
> More interesting for your case can be the output from
>   sysctl vm.phys_free
> which provides information about the free queues and order of free pages on 
> them.
> --
> Intel Research and Development Ireland Limited Registered in Ireland 
> Registered Office: Collinstown Industrial Park, Leixlip, County 
> Kildare Registered Number: 308263
> 
> 
> This e-mail and any attachments may contain confidential material for 
> the sole use of the intended recipient(s). Any review or distribution 
> by others is strictly prohibited. If you are not the intended 
> recipient, please contact the sender and delete all copies.


--
Intel Research and Development Ireland Limited
Registered in Ireland
Registered Office: Collinstown Industrial Park, Leixlip, County Kildare
Registered Number: 308263


This e-mail and any attachments may contain confidential material for the sole
use of the intended recipient(s). Any review or distribution by others is
strictly prohibited. If you are not the intended recipient, please contact the
sender and delete all copies.

___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"


Re: Possible memory leak in the kernel (contigmalloc)

2018-10-30 Thread Konstantin Belousov
On Tue, Oct 30, 2018 at 09:46:59AM +, Bennett, Ciunas wrote:
> Hi,
> 
> I was debugging the issue by viewing the free ques "sysctl
> vm.phys_free" and also using "show page" in ddb. The inactive memory
> is never being released back into the free que. I thought that when
> inactive memory reaches a certain threshold that the kernel will start
> reclaiming and move it to the free list? In my program this is not
> happening, the program uses free memory (contigmalloc), and then it
> is put into the inactive que (contiigfree) when the program frees it.
Contigfree() does not release memory into inactive queue.  By definition,
inactive pages have valid content, which is not possible for the pages
freed by contigfree().
contigfree()->kmem_free()->kmem_unback()->vm_page_free().


> This inactive memory is never released by the kernel, and the inactive
> que grows until all the memory is in this que. I have attached a xml
> sheet that shows the memory usage in the system.
Inactive memory is not freed, it makes no sense as far as there is valid
content, which is either not recoverable (anon memory or dirty file
pages) or high-cost to restore (clean file pages, need to re-read from
disk).  Inactive is processed by pagedaemon when system has memory deficit,
and either inactive pages are written to swap, or written to their file
backing storage.

I suspect that you have other activities on your system going on, which
cause creation of the inactive memory and unrecoverable fragmentation.
Note that contigmalloc() tries to do defragmentation to satisfy requests,
but this is not always possible.


> Ciunas.
> 
> -Original Message-
> From: Konstantin Belousov [mailto:kostik...@gmail.com] 
> Sent: Friday, October 26, 2018 9:13 PM
> To: Bennett, Ciunas 
> Cc: freebsd-stable@freebsd.org
> Subject: Re: Possible memory leak in the kernel (contigmalloc)
> 
> On Wed, Oct 24, 2018 at 04:27:52PM +, Bennett, Ciunas wrote:
> > Hello,
> > 
> > I have encountered an issue with a kernel application that I have 
> > written, the issue might be caused by a memory leak in the kernel.
> > The application allocates and deallocates contiguous memory using
> > contigmalloc() and contigfree(). The application will fail after a 
> > period of time because there is not enough free contiguous memory 
> > left. There could be an issue with the freeing of memory when using 
> > the contigfree() function.
> >
> 
> It is unlikely that there is an issue with a leak, but I would be not 
> surprised if your allocation/free pattern would cause fragmentation on free 
> lists that results in contigmalloc(9) failures after.
> 
> Look at the vmstat -z/vmstat -m output to see uma and malloc stats.
> More interesting for your case can be the output from
>   sysctl vm.phys_free
> which provides information about the free queues and order of free pages on 
> them.
> --
> Intel Research and Development Ireland Limited
> Registered in Ireland
> Registered Office: Collinstown Industrial Park, Leixlip, County Kildare
> Registered Number: 308263
> 
> 
> This e-mail and any attachments may contain confidential material for the sole
> use of the intended recipient(s). Any review or distribution by others is
> strictly prohibited. If you are not the intended recipient, please contact the
> sender and delete all copies.


___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"


RE: Possible memory leak in the kernel (contigmalloc)

2018-10-30 Thread Bennett, Ciunas
Hi,

I was debugging the issue by viewing the free ques "sysctl vm.phys_free" and 
also using "show page" in ddb.
The inactive memory is never being released back into the free que. 
I thought that when inactive memory reaches a certain threshold that the kernel 
will start reclaiming and move it to the free list?
In my program this is not happening, the program uses free memory 
(contigmalloc), and then it is put into the inactive que (contiigfree) when the 
program frees it.
This inactive memory is never released by the kernel, and the inactive que 
grows until all the memory is in this que.
I  have attached a xml sheet that shows the memory usage in the system.

Ciunas.

-Original Message-
From: Konstantin Belousov [mailto:kostik...@gmail.com] 
Sent: Friday, October 26, 2018 9:13 PM
To: Bennett, Ciunas 
Cc: freebsd-stable@freebsd.org
Subject: Re: Possible memory leak in the kernel (contigmalloc)

On Wed, Oct 24, 2018 at 04:27:52PM +, Bennett, Ciunas wrote:
> Hello,
> 
> I have encountered an issue with a kernel application that I have 
> written, the issue might be caused by a memory leak in the kernel.
> The application allocates and deallocates contiguous memory using
> contigmalloc() and contigfree(). The application will fail after a 
> period of time because there is not enough free contiguous memory 
> left. There could be an issue with the freeing of memory when using 
> the contigfree() function.
>

It is unlikely that there is an issue with a leak, but I would be not surprised 
if your allocation/free pattern would cause fragmentation on free lists that 
results in contigmalloc(9) failures after.

Look at the vmstat -z/vmstat -m output to see uma and malloc stats.
More interesting for your case can be the output from
sysctl vm.phys_free
which provides information about the free queues and order of free pages on 
them.
--
Intel Research and Development Ireland Limited
Registered in Ireland
Registered Office: Collinstown Industrial Park, Leixlip, County Kildare
Registered Number: 308263


This e-mail and any attachments may contain confidential material for the sole
use of the intended recipient(s). Any review or distribution by others is
strictly prohibited. If you are not the intended recipient, please contact the
sender and delete all copies.
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"


Re: Possible memory leak in the kernel (contigmalloc)

2018-10-26 Thread Konstantin Belousov
On Wed, Oct 24, 2018 at 04:27:52PM +, Bennett, Ciunas wrote:
> Hello,
> 
> I have encountered an issue with a kernel application that I have
> written, the issue might be caused by a memory leak in the kernel.
> The application allocates and deallocates contiguous memory using
> contigmalloc() and contigfree(). The application will fail after a
> period of time because there is not enough free contiguous memory
> left. There could be an issue with the freeing of memory when using
> the contigfree() function.
>

It is unlikely that there is an issue with a leak, but I would be not
surprised if your allocation/free pattern would cause fragmentation
on free lists that results in contigmalloc(9) failures after.

Look at the vmstat -z/vmstat -m output to see uma and malloc stats.
More interesting for your case can be the output from
sysctl vm.phys_free
which provides information about the free queues and order of free pages
on them.
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"


Possible memory leak in the kernel (contigmalloc)

2018-10-24 Thread Bennett, Ciunas
Hello,

I have encountered an issue with a kernel application that I have written, the 
issue might be caused by a memory leak in the kernel.
The application allocates  and deallocates contiguous memory using 
contigmalloc() and contigfree().
The application will fail after a period of time because there is not enough 
free contiguous memory left.
There could be an issue with the freeing of memory when using the contigfree() 
function.

I have attached a simplified version of the application.
The resulting kernel module just allocates contiguous memory and then frees the 
memory using contigfree();
This allocation is done in a loop and the attached  src code triggers the issue.

After a period of time when running the application multiple times, the kernel 
reports
that there is no available memory and fails with allocation.

I can see that the amount of free memory is decreasing in correlation to how 
many

times I run the application by using DDB and printing out freepages using "show 
freepages"


I run the application in a loop using a shell script where I kldload then 
kldunload multiple times (script runs up to 100)

The application can take 20 to over 60 minutes to trigger the issue and run out 
of memory but can take longer also.

I am running the application on ->
FreeBSD on 11.2
VM with 2 Gb of ram.
Allocating one cpu core.
Running on an Intel(R) Xeon(R) CPU E5-2658 v2 @ 2.40GH using Ubuntu 16.04 host.

I have attached the test kernel application and a Makefile.

Thanks,
Ciunas.
--
Intel Research and Development Ireland Limited
Registered in Ireland
Registered Office: Collinstown Industrial Park, Leixlip, County Kildare
Registered Number: 308263


This e-mail and any attachments may contain confidential material for the sole
use of the intended recipient(s). Any review or distribution by others is
strictly prohibited. If you are not the intended recipient, please contact the
sender and delete all copies.
#include 
#include 
#include 
#include 
#include 
#include 

#define DEV_MEM "test_mem"

MALLOC_DECLARE(TEST_MEM);
MALLOC_DEFINE(TEST_MEM, "test_mem", "test_mem driver");

/* Test application to allocate contiguous memory then free it */
static int test_application()
{
int i = 0;
int array[10] = {2097152, 1024, 2101248, 1024, 2097152, 2101248,
 2101248, 2097152, 2101248, 1024};
int array1[10] = {1024, 2101248, 2097152, 2101248, 2101248, 2097152,
  1024, 2101248, 1024, 2097152};
void *mem_blocks[10];

for (i = 0; i < 10; i++)
{
mem_blocks[i] = contigmalloc(array[i], TEST_MEM, 0, 0, ~0, PAGE_SIZE, 
0);
if (!mem_blocks[i])
{
printf("%s:%d Unable to allocate contiguous memory slab \n", 
__func__,
   __LINE__);
return -1;
}
}

for (i = 0; i < 10; i++)
contigfree(mem_blocks[i], array[i], TEST_MEM);

for (i = 0; i < 10; i++)
{
mem_blocks[i] = contigmalloc(array1[i], TEST_MEM, 0, 0, ~0, PAGE_SIZE, 
0);
if (!mem_blocks[i])
{
printf("%s:%d Unable to allocate contiguous memory slab \n", 
__func__,
   __LINE__);
return -1;
}
}

for (i = 0; i < 10; i++)
contigfree(mem_blocks[i], array1[i], TEST_MEM);

return 0;
}

static int mem_modevent(module_t mod __unused,
int type,
void *data __unused)
{

switch (type)
{
case MOD_LOAD:
test_application();
return (0);
case MOD_UNLOAD:
return (0);
default:
return (EOPNOTSUPP);
}
}

DEV_MODULE(test_mem, mem_modevent, NULL);
MODULE_VERSION(test_mem, 1);


Makefile
Description: Makefile
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"


Re: Possible memory leak?

2007-03-21 Thread Stefan Ehmann
On Tuesday 20 March 2007 10:44:07 Oliver Fromme wrote:
 Stefan Ehmann wrote:
   Sometimes I'm noticing very high memory usage. Nearly my whole memory
   (1GB) is used although I'm running my usual set of processes - normally
   memory usage is much lower.

 That's normal.  FreeBSD uses nearly all free memory for
 buffer cache and other kinds of caches.

   I killed most processes but memory usage remains high.

 How do you measure memory usage?  The numbers from top(1)
 are mostly meaningless.  Personally I think top should be
 removed from FreeBSD, because it confuses many people (in
 fact I think _most_ people don't interpret the numbers
 correctly), but some people seem to be in love with it. :-)
Obviously I'm one of those people that got it wrong.

My mistake was to think that a memory page is moved to the inactive/cache list 
automatically if it's not used for a longer time. But if I got it correctly 
now, this only happens when there's some kind of memory shortage and the 
pageout daemon is not sleeping.

Now that I think I understand it better, those numbers also seem okay to 
me. :)

What got me suspicious at first was that the systems sometimes started to swap 
although there should have been more than enough memory to fit all processes 
in physical memory. But I see this is also in the FAQ.

Thanks for all your help and hints.

Stefan
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to [EMAIL PROTECTED]


Re: Possible memory leak?

2007-03-20 Thread Oliver Fromme
Stefan Ehmann wrote:
  Sometimes I'm noticing very high memory usage. Nearly my whole memory (1GB) 
  is 
  used although I'm running my usual set of processes - normally memory usage 
  is much lower.

That's normal.  FreeBSD uses nearly all free memory for
buffer cache and other kinds of caches.

  I killed most processes but memory usage remains high.

How do you measure memory usage?  The numbers from top(1)
are mostly meaningless.  Personally I think top should be
removed from FreeBSD, because it confuses many people (in
fact I think _most_ people don't interpret the numbers
correctly), but some people seem to be in love with it. :-)

  Summing the VSZ values of the ps aux output gives about 34MB. top reports 
  316MB active memory.

While summing the VSZ values doesn't make any sense, that
number doesn't sound worrying.

  dmesg/ps/top output can be found here:
  http://stud4.tuwien.ac.at/~e0125637/fbsd/

It all looks perfectly normal to me.

If you need to find out whether you're running short of
RAM, use vmstat 5 and watch it for a while (ignore the
first line because it contains only averages since reboot).
If the po (page-out) and sr (scan rate, which is an
indication of memory pressure) values are constantly very
high, then you either need more RAM, or you have a memory
leak somewhere.  In all other cases (i.e. po and sr are
zero most of the time), there's nothing to worry about.
Note that the pi value is not important, because page-
ins happen normally when reading executables, libraries
or memory-mapped files.

Best regards
   Oliver

-- 
Oliver Fromme, secnetix GmbH  Co. KG, Marktplatz 29, 85567 Grafing b. M.
Handelsregister: Registergericht Muenchen, HRA 74606,  Geschäftsfuehrung:
secnetix Verwaltungsgesellsch. mbH, Handelsregister: Registergericht Mün-
chen, HRB 125758,  Geschäftsführer: Maik Bachmann, Olaf Erb, Ralf Gebhart

FreeBSD-Dienstleistungen, -Produkte und mehr:  http://www.secnetix.de/bsd

... there are two ways of constructing a software design:  One way
is to make it so simple that there are _obviously_ no deficiencies and
the other way is to make it so complicated that there are no _obvious_
deficiencies.-- C.A.R. Hoare, ACM Turing Award Lecture, 1980
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to [EMAIL PROTECTED]


Re: Possible memory leak?

2007-03-20 Thread Wojciech A. Koszek
On Tue, Mar 20, 2007 at 10:44:07AM +0100, Oliver Fromme wrote:
 Stefan Ehmann wrote:
   Sometimes I'm noticing very high memory usage. Nearly my whole memory 
 (1GB) is 
   used although I'm running my usual set of processes - normally memory 
 usage 

[..]

 How do you measure memory usage?  The numbers from top(1)
 are mostly meaningless.  Personally I think top should be
 removed from FreeBSD, because it confuses many people (in
 fact I think _most_ people don't interpret the numbers
 correctly), but some people seem to be in love with it. :-)

You can obtain reliable output from vmstat(8):

vmstat -m
vmstat -z

-- 
Wojciech A. Koszek
[EMAIL PROTECTED]
http://FreeBSD.czest.pl/dunstan/
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to [EMAIL PROTECTED]


Possible memory leak?

2007-03-18 Thread Stefan Ehmann
Sometimes I'm noticing very high memory usage. Nearly my whole memory (1GB) is 
used although I'm running my usual set of processes - normally memory usage 
is much lower.

I killed most processes but memory usage remains high.

Summing the VSZ values of the ps aux output gives about 34MB. top reports 
316MB active memory.

Where did my memory go? Are there any tools for debugging this?

I don't know what's causing this. I'm using the snd_envy24 driver from 
current, but I think I've seen the problem also when not using sound.

dmesg/ps/top output can be found here:
http://stud4.tuwien.ac.at/~e0125637/fbsd/

If needed, I can provide more details.

Stefan
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to [EMAIL PROTECTED]


Re: Possible memory leak?

2007-03-18 Thread Kris Kennaway
On Sun, Mar 18, 2007 at 04:19:52PM +0100, Stefan Ehmann wrote:
 Sometimes I'm noticing very high memory usage. Nearly my whole memory (1GB) 
 is 
 used although I'm running my usual set of processes - normally memory usage 
 is much lower.
 
 I killed most processes but memory usage remains high.
 
 Summing the VSZ values of the ps aux output gives about 34MB. top reports 
 316MB active memory.
 
 Where did my memory go? Are there any tools for debugging this?

This is a FAQ.  free memory is wasted memory.

Kris
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to [EMAIL PROTECTED]


Re: Possible memory leak?

2007-03-18 Thread Stefan Ehmann
On Sunday 18 March 2007 20:25:19 Kris Kennaway wrote:
 On Sun, Mar 18, 2007 at 04:19:52PM +0100, Stefan Ehmann wrote:
  Sometimes I'm noticing very high memory usage. Nearly my whole memory
  (1GB) is used although I'm running my usual set of processes - normally
  memory usage is much lower.
 
  I killed most processes but memory usage remains high.
 
  Summing the VSZ values of the ps aux output gives about 34MB. top reports
  316MB active memory.
 
  Where did my memory go? Are there any tools for debugging this?

 This is a FAQ.  free memory is wasted memory.

Sorry for the noise. I somehow thought that's only true for inactive/cache 
memory and active is different - but obviously I was wrong.
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to [EMAIL PROTECTED]