Re: [HACKERS] Memory leaks in record_out and record_send

2012-11-13 Thread Dimitri Fontaine
Tom Lane t...@sss.pgh.pa.us writes: OTOH I can't see trying to back-patch a solution like that. If we want to fix this in the back branches (and note the complaint linked above is against 8.3), I think we have to do it as attached. Thoughts? I've been using textin(record_out(NEW)) in

Re: [HACKERS] Memory leaks in record_out and record_send

2012-11-13 Thread Tom Lane
Dimitri Fontaine dimi...@2ndquadrant.fr writes: Tom Lane t...@sss.pgh.pa.us writes: Thoughts? I've been using textin(record_out(NEW)) in generic partitioning triggers, and you can find examples of that trick in the wiki, so I think we have users of that in the field. I think explicit calls

Re: [HACKERS] Memory leaks in record_out and record_send

2012-11-13 Thread Dimitri Fontaine
Tom Lane t...@sss.pgh.pa.us writes: I think explicit calls like that actually wouldn't be a problem, since they'd be run in a per-tuple context anyway. The cases that are problematic are hard-coded I/O function calls. I'm worried about the ones like, say, plpgsql's built-in conversion

Re: [HACKERS] Memory leaks in record_out and record_send

2012-11-13 Thread Tom Lane
Dimitri Fontaine dimi...@2ndquadrant.fr writes: Tom Lane t...@sss.pgh.pa.us writes: I think explicit calls like that actually wouldn't be a problem, since they'd be run in a per-tuple context anyway. The cases that are problematic are hard-coded I/O function calls. I'm worried about the

Re: [HACKERS] Memory leaks in record_out and record_send

2012-11-13 Thread Robert Haas
On Tue, Nov 13, 2012 at 12:18 AM, Tom Lane t...@sss.pgh.pa.us wrote: I wonder though if we ought to think about running output functions in a short-lived memory context instead of the executor's main context. We've considered that before, I think, and it's always been the path of least

Re: [HACKERS] Memory leaks in record_out and record_send

2012-11-13 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Tue, Nov 13, 2012 at 12:18 AM, Tom Lane t...@sss.pgh.pa.us wrote: I wonder though if we ought to think about running output functions in a short-lived memory context instead of the executor's main context. We've considered that before, I think, and

Re: [HACKERS] Memory leaks in record_out and record_send

2012-11-13 Thread Robert Haas
On Tue, Nov 13, 2012 at 3:21 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Tue, Nov 13, 2012 at 12:18 AM, Tom Lane t...@sss.pgh.pa.us wrote: I wonder though if we ought to think about running output functions in a short-lived memory context instead of the

Re: [HACKERS] Memory leaks in record_out and record_send

2012-11-13 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: Yeah. The thing that concerns me is that I think we have a pretty decent number of memory contexts where the expected number of allocations is very small ... and we have the context *just in case* we do more than that in certain instances. I've

Re: [HACKERS] Memory leaks in record_out and record_send

2012-11-13 Thread Martijn van Oosterhout
On Tue, Nov 13, 2012 at 05:50:08PM -0500, Stephen Frost wrote: * Robert Haas (robertmh...@gmail.com) wrote: Yeah. The thing that concerns me is that I think we have a pretty decent number of memory contexts where the expected number of allocations is very small ... and we have the context

Re: [HACKERS] Memory leaks on SRF rescan

2008-02-21 Thread Tom Lane
Neil Conway [EMAIL PROTECTED] writes: Two different classes of allocations in the EState's per-query context are leaked: (1) In FunctionNext(), we call ExecMakeTableFunctionResult() to compute the result set of the SRF, which allocates the TupleDesc out-parameter in the per-query memory

Re: [HACKERS] Memory leaks on SRF rescan

2008-02-21 Thread Tom Lane
Neil Conway [EMAIL PROTECTED] writes: On Thu, 2008-02-21 at 21:42 -0500, Tom Lane wrote: Yeah. I think it's hopeless to expect these functions to all hew to the straight and narrow path. It seems to me that the right way is for the sub-select to somehow run in its own per-query context.

Re: [HACKERS] Memory leaks on SRF rescan

2008-02-21 Thread Neil Conway
On Thu, 2008-02-21 at 21:42 -0500, Tom Lane wrote: Given your point (2), is this worth fixing by itself? Right, probably not. Yeah. I think it's hopeless to expect these functions to all hew to the straight and narrow path. It seems to me that the right way is for the sub-select to somehow

Re: [HACKERS] Memory leaks

2002-10-23 Thread Nigel J. Andrews
On Tue, 22 Oct 2002, Tom Lane wrote: Greg Copeland [EMAIL PROTECTED] writes: Interesting. Having not looked at memory management schemes used in the pl implementations, can you enlighten me by what you mean by integrate the memory-context notion? Does that mean they are not using

Re: [HACKERS] Memory leaks

2002-10-23 Thread Greg Copeland
On Tue, 2002-10-22 at 22:28, Tom Lane wrote: Greg Copeland [EMAIL PROTECTED] writes: So again, I'm not really sure it they are meaningful at this point. psql might well have some internal leaks; the backend memory-context design doesn't apply to it. Okay. Thanks. I'll probably take

Re: [HACKERS] Memory leaks

2002-10-23 Thread Tom Lane
Nigel J. Andrews [EMAIL PROTECTED] writes: On Tue, 22 Oct 2002, Tom Lane wrote: Not everywhere. plpgsql is full of malloc's and I think the other PL modules are too --- and that's not to mention the allocation policies of the perl, tcl, etc, language interpreters... I was going to make the

Re: [HACKERS] Memory leaks

2002-10-23 Thread Tom Lane
Greg Copeland [EMAIL PROTECTED] writes: Okay. I've started looking at plpython to better understand it's memory needs. I'm seeing a mix of mallocs and PLy_malloc. The PLy version is basically malloc which also checks and reports on memory allocation errors. Anyone know if the cases where

Re: [HACKERS] Memory leaks

2002-10-23 Thread Greg Copeland
On Wed, 2002-10-23 at 08:48, Tom Lane wrote: Greg Copeland [EMAIL PROTECTED] writes: Okay. I've started looking at plpython to better understand it's memory needs. I'm seeing a mix of mallocs and PLy_malloc. The PLy version is basically malloc which also checks and reports on memory

Re: [HACKERS] Memory leaks

2002-10-23 Thread Tom Lane
Greg Copeland [EMAIL PROTECTED] writes: Ya, I'm currently looking to see how the memory is being used and why. I'm trying to better understand it's life cycle. You implying that even the short term memory should be using the palloc stuff? What about long term? Blanket statement that pretty

Re: [HACKERS] Memory leaks

2002-10-22 Thread Petru Paler
On Tue, Oct 22, 2002 at 04:16:16PM -0500, Greg Copeland wrote: I've started playing a little with Postgres to determine if there were memory leaks running around. After some very brief checking, I'm starting[1] to think that the answer is yes. Has anyone already gone through a significant

Re: [HACKERS] Memory leaks

2002-10-22 Thread Tom Lane
Greg Copeland [EMAIL PROTECTED] writes: I've started playing a little with Postgres to determine if there were memory leaks running around. After some very brief checking, I'm starting[1] to think that the answer is yes. Has anyone already gone through a significant effort to locate and

Re: [HACKERS] Memory leaks

2002-10-22 Thread Neil Conway
Petru Paler [EMAIL PROTECTED] writes: valgrind is a great tool I used -- didn't get the time to try it out on Postgres yet, though. Besides leaks, it also catches uninitialized variable access and stuff like that. I've used Valgrind with PostgreSQL a little bit, and it's been fairly useful (I

Re: [HACKERS] Memory leaks

2002-10-22 Thread Greg Copeland
On Tue, 2002-10-22 at 17:09, Tom Lane wrote: Greg Copeland [EMAIL PROTECTED] writes: I've started playing a little with Postgres to determine if there were memory leaks running around. After some very brief checking, I'm starting[1] to think that the answer is yes. Has anyone already gone

Re: [HACKERS] Memory leaks

2002-10-22 Thread Nigel J. Andrews
On 22 Oct 2002, Greg Copeland wrote: On Tue, 2002-10-22 at 17:09, Tom Lane wrote: plpgsql has some issues too, I suspect, but not as bad as pltcl etc. Possibly the best answer is to integrate the memory-context notion into those modules; if they did most of their work in a temp context

Re: [HACKERS] Memory leaks

2002-10-22 Thread Tom Lane
Nigel J. Andrews [EMAIL PROTECTED] writes: I saw use of a couple of malloc (or Python specific malloc) calls the other day but I also seem to recall that, after consideration, I decided the memory needed to survive for the duration of the backend. Should I have created a new child of the top

Re: [HACKERS] Memory leaks

2002-10-22 Thread Tom Lane
Greg Copeland [EMAIL PROTECTED] writes: On Tue, 2002-10-22 at 17:09, Tom Lane wrote: Yes, this has been dealt with before. What tools, aside from noggin v1.0, did they use? Do we know? s/they/me/ ... none. But I don't know of any that I think would be useful. I then moved on to psql,

Re: [HACKERS] Memory leaks

2002-10-22 Thread Karel Zak
On Tue, Oct 22, 2002 at 11:28:23PM -0400, Tom Lane wrote: I then moved on to psql, again, just for fun. Here, I'm thinking that I started to find some other leaks...but again, I've not spent any real time on it. So again, I'm not really sure it they are meaningful at this point. psql

Re: [HACKERS] Memory leaks

2002-10-22 Thread Hannu Krosing
On Wed, 2002-10-23 at 03:09, Tom Lane wrote: It's fairly difficult to get anywhere with standard leak-tracking tools, since they don't know anything about palloc. What's worse, it is *not* a bug for a routine to palloc space it never pfrees, if it knows that it's palloc'ing in a short-lived