[HACKERS] Re: [GENERAL] postgres memory management

[Alexander Jerusalem]

At 21:40 22.01.01, Peter Mount wrote:
>At 13:18 21/01/01 +0100, Alexander Jerusalem wrote:
>>Hi all,
>>
>>I'm experiencing some strange behaviour with postgresql 7.0.3 on Red Hat 
>>Linux 7. I'm sending lots of insert statements to the postgresql server
>>>from another machine via JDBC. During that process postgresql continues to
>>take up more and more memory and seemingly never returns it to the 
>>system. Oddly if I watch the postmaster and it's sub processes in ktop, I 
>>can't see which process takes up this memory. ktop shows that the 
>>postgresql related processes have a constant memory usage but the overall 
>>memory usage always increases as long as I continue to send insert statements.
>>
>>When the database connection is closed, no memory is reclaimed, the 
>>overall memory usage stays the same. And when I close down all postgresql 
>>processes including postmaster, it's the same.
>>I'm rather new to Linux and postgresql so I'm not sure if I should call 
>>this a memory leak :-)
>>Has anybody experienced a similar thing?
>
>I'm not sure myself. You can rule out JDBC (or Java) here as you say you 
>are connecting from another machine.
>
>When your JDBC app closes, does it call the connection's close() method? 
>Does any messages like "Unexpected EOF from client" appear on the server side?
>
>The only other thing that comes to mine is possibly something weird is 
>happening with IPC. After you closed down postgres, does ipcclean free up 
>any memory?
>
>I'm cc'in the hackers list and the new jdbc list.
>
>Peter

Thanks for your answer!

Yes I'm calling Connection.close(). I don't get any error messages but 
maybe I just don't see them because postgresql is started automatically at 
run level 3. I'm not sure where the output goes. (pg_log contains only 
garbage or maybe it's a binary file) I tried ipcclean right now and it 
doesn't free the memory but it gives me some messages that I cannot interpret:

Shared memory 0 ... skipped. Process still exists (pid ).
Shared memory 1 ... skipped. Process still exists (pid ).
Shared memory 2 ... skipped. Process still exists (pid ).
Shared memory 3 ... skipped. Process still exists (pid ).
Semaphore 0 ... resource(s) deleted
Semaphore 1 ... resource(s) deleted

Oddly, when I try to run ipcclean a second time, it says: ipcclean: You 
still have a postmaster running. Which is not the case as ps -e proves.

Alexander Jerusalem
[EMAIL PROTECTED]
vknn

Re: [HACKERS] Re: postgres memory management

[Alexander Jerusalem]

At 22:29 22.01.01, Alfred Perlstein wrote:
>* Peter Mount <[EMAIL PROTECTED]> [010122 13:21] wrote:
> > At 13:18 21/01/01 +0100, Alexander Jerusalem wrote:
> > >Hi all,
> > >
> > >I'm experiencing some strange behaviour with postgresql 7.0.3 on Red Hat
> > >Linux 7. I'm sending lots of insert statements to the postgresql server
> > >from another machine via JDBC. During that process postgresql 
> continues to
> > >take up more and more memory and seemingly never returns it to the 
> system.
> > >Oddly if I watch the postmaster and it's sub processes in ktop, I can't
> > >see which process takes up this memory. ktop shows that the postgresql
> > >related processes have a constant memory usage but the overall memory
> > >usage always increases as long as I continue to send insert statements.
> > >
> > >When the database connection is closed, no memory is reclaimed, the
> > >overall memory usage stays the same. And when I close down all postgresql
> > >processes including postmaster, it's the same.
> > >I'm rather new to Linux and postgresql so I'm not sure if I should call
> > >this a memory leak :-)
> > >Has anybody experienced a similar thing?
> >
> > I'm not sure myself. You can rule out JDBC (or Java) here as you say you
> > are connecting from another machine.
> >
> > When your JDBC app closes, does it call the connection's close() method?
> > Does any messages like "Unexpected EOF from client" appear on the 
> server side?
> >
> > The only other thing that comes to mine is possibly something weird is
> > happening with IPC. After you closed down postgres, does ipcclean free up
> > any memory?
>
>I don't know if this is valid for Linux, but it is how FreeBSD
>works, for the most part used memory is never free'd, it is only
>marked as reclaimable.  This is so the system can cache more data.
>On a freshly booted FreeBSD box you'll have a lot of 'free' memory,
>after the box has been running for a long time the 'free' memory
>will probably never go higher that 10megs, the rest is being used
>as cache.
>
>The main things you have to worry about is:
>a) really running out of memory (are you useing a lot of swap?)
>b) not cleaning up IPC as Peter suggested.

Thanks for your answer!

I'm rather new to Linux, so I can't tell if it's that way on Linux. But I 
noticed that other programs free some memory when I quit them. But it's 
true that I'm not running out of memory. I have 300 MB of free RAM and no 
swap space is used. As I wrote in reply to Peters mail, ipcclean doesn't 
change anything.

Alexander Jerusalem
[EMAIL PROTECTED]
vknn