Re: [HACKERS] Safe memory allocation functions

2015-01-31 Thread Michael Paquier
On Sat, Jan 31, 2015 at 2:58 AM, Robert Haas wrote: > Committed. I didn't think we really need to expose two separate flags > for the aligned and unaligned cases, so I ripped that out. I also > removed the duplicate documentation of the new constants in the > function header; having two copies o

Re: [HACKERS] Safe memory allocation functions

2015-01-30 Thread Robert Haas
On Fri, Jan 30, 2015 at 1:10 AM, Michael Paquier wrote: > I wrote: >> Yes, this refactoring was good for testing actually... > Oops, I have been too hasty when sending previous patch, there was a > bug related to huge allocations. Patch correcting this bug is > attached. Committed. I didn't thin

Re: [HACKERS] Safe memory allocation functions

2015-01-29 Thread Michael Paquier
I wrote: > Yes, this refactoring was good for testing actually... Oops, I have been too hasty when sending previous patch, there was a bug related to huge allocations. Patch correcting this bug is attached. Attached are as well two things I have used to test the new API: - A hack refactoring the e

Re: [HACKERS] Safe memory allocation functions

2015-01-29 Thread Michael Paquier
On Fri, Jan 30, 2015 at 12:29 AM, Robert Haas wrote: > On Wed, Jan 28, 2015 at 9:34 AM, Michael Paquier > wrote: >> As a result of all the comments on this thread, here are 3 patches >> implementing incrementally the different ideas from everybody: >> 1) 0001 modifies aset.c to return uncondition

Re: [HACKERS] Safe memory allocation functions

2015-01-29 Thread Robert Haas
On Wed, Jan 28, 2015 at 9:34 AM, Michael Paquier wrote: > As a result of all the comments on this thread, here are 3 patches > implementing incrementally the different ideas from everybody: > 1) 0001 modifies aset.c to return unconditionally NULL in case of an > OOM instead of reporting an error.

Re: [HACKERS] Safe memory allocation functions

2015-01-28 Thread Michael Paquier
On Tue, Jan 27, 2015 at 5:34 PM, Andres Freund wrote: > On 2015-01-27 17:27:53 +0900, Michael Paquier wrote: >> Alvaro Herrera wrote: >> >> So how about something like >> >> >> >> #define ALLOCFLAG_HUGE 0x01 >> >> #define ALLOCFLAG_NO_ERROR_ON_OOM 0x02 >> >> void * >> >> Mem

Re: [HACKERS] Safe memory allocation functions

2015-01-27 Thread Andres Freund
On 2015-01-27 17:27:53 +0900, Michael Paquier wrote: > Alvaro Herrera wrote: > >> So how about something like > >> > >> #define ALLOCFLAG_HUGE 0x01 > >> #define ALLOCFLAG_NO_ERROR_ON_OOM 0x02 > >> void * > >> MemoryContextAllocFlags(MemoryContext context, Size size, int flags

Re: [HACKERS] Safe memory allocation functions

2015-01-27 Thread Michael Paquier
Alvaro Herrera wrote: >> So how about something like >> >> #define ALLOCFLAG_HUGE 0x01 >> #define ALLOCFLAG_NO_ERROR_ON_OOM 0x02 >> void * >> MemoryContextAllocFlags(MemoryContext context, Size size, int flags); The flag for huge allocations may be useful, but I don't actuall

Re: [HACKERS] Safe memory allocation functions

2015-01-27 Thread Michael Paquier
On Sat, Jan 17, 2015 at 11:06 PM, Robert Haas wrote: > On Fri, Jan 16, 2015 at 10:56 AM, Alvaro Herrera > wrote: >> So how about something like >> >> #define ALLOCFLAG_HUGE 0x01 >> #define ALLOCFLAG_NO_ERROR_ON_OOM 0x02 >> void * >> MemoryContextAllocFlags(MemoryContext con

Re: [HACKERS] Safe memory allocation functions

2015-01-17 Thread Robert Haas
On Fri, Jan 16, 2015 at 10:56 AM, Alvaro Herrera wrote: > So how about something like > > #define ALLOCFLAG_HUGE 0x01 > #define ALLOCFLAG_NO_ERROR_ON_OOM 0x02 > void * > MemoryContextAllocFlags(MemoryContext context, Size size, int flags); That sounds good, although persona

Re: [HACKERS] Safe memory allocation functions

2015-01-16 Thread Alvaro Herrera
Andres Freund wrote: > On 2015-01-16 12:56:18 -0300, Alvaro Herrera wrote: > > So how about something like > > > > #define ALLOCFLAG_HUGE 0x01 > > #define ALLOCFLAG_NO_ERROR_ON_OOM 0x02 > > void * > > MemoryContextAllocFlags(MemoryContext context, Size size, int flags); >

Re: [HACKERS] Safe memory allocation functions

2015-01-16 Thread Andres Freund
On 2015-01-16 12:56:18 -0300, Alvaro Herrera wrote: > Andres Freund wrote: > > We rely on palloc erroring out on large allocations in a couple places > > as a crosscheck. I don't think this argument holds much water. > > I don't understand what that has to do with it. Surely we're not going > to

Re: [HACKERS] Safe memory allocation functions

2015-01-16 Thread Alvaro Herrera
Andres Freund wrote: > On 2015-01-16 12:09:25 -0300, Alvaro Herrera wrote: > > Robert Haas wrote: > > > On Thu, Jan 15, 2015 at 10:57 AM, Alvaro Herrera > > > wrote: > > > > > >> I do think that "safe" is the wrong suffix. Maybe palloc_soft_fail() > > > >> or palloc_null() or palloc_no_oom() or

Re: [HACKERS] Safe memory allocation functions

2015-01-16 Thread Andres Freund
On 2015-01-16 12:09:25 -0300, Alvaro Herrera wrote: > Robert Haas wrote: > > On Thu, Jan 15, 2015 at 10:57 AM, Alvaro Herrera > > wrote: > > > >> I do think that "safe" is the wrong suffix. Maybe palloc_soft_fail() > > >> or palloc_null() or palloc_no_oom() or palloc_unsafe(). > > > > > > I like

Re: [HACKERS] Safe memory allocation functions

2015-01-16 Thread Alvaro Herrera
Robert Haas wrote: > On Thu, Jan 15, 2015 at 10:57 AM, Alvaro Herrera > wrote: > >> I do think that "safe" is the wrong suffix. Maybe palloc_soft_fail() > >> or palloc_null() or palloc_no_oom() or palloc_unsafe(). > > > > I liked palloc_noerror() better myself FWIW. > > I don't care for noerror

Re: [HACKERS] Safe memory allocation functions

2015-01-16 Thread Robert Haas
On Thu, Jan 15, 2015 at 10:57 AM, Alvaro Herrera wrote: >> Hmm, I understood Tom to be opposing the idea of a palloc variant that >> returns NULL on failure, and I understand you to be supporting it. >> But maybe I'm confused. > > Your understanding seems correct to me. I was just saying that you

Re: [HACKERS] Safe memory allocation functions

2015-01-16 Thread Andres Freund
On 2015-01-16 23:06:12 +0900, Michael Paquier wrote: > /* > + * Wrappers for allocation functions. > + */ > +static void *set_alloc_internal(MemoryContext context, > + Size size, bool > noerror); > +static void *set_realloc_internal(Memo

Re: [HACKERS] Safe memory allocation functions

2015-01-16 Thread Andres Freund
On 2015-01-16 08:47:10 +0900, Michael Paquier wrote: > On Fri, Jan 16, 2015 at 12:57 AM, Alvaro Herrera > wrote: > >> I do think that "safe" is the wrong suffix. Maybe palloc_soft_fail() > >> or palloc_null() or palloc_no_oom() or palloc_unsafe(). > > > > I liked palloc_noerror() better myself FW

Re: [HACKERS] Safe memory allocation functions

2015-01-16 Thread Michael Paquier
On Fri, Jan 16, 2015 at 8:47 AM, Michael Paquier wrote: > Voting for palloc_noerror() as well. And here is an updated patch using this naming, added to the next CF as well. -- Michael From b636c809c2f2cb4177bedc2e5a4883a79b61fbc6 Mon Sep 17 00:00:00 2001 From: Michael Paquier Date: Tue, 13 Jan 2

Re: [HACKERS] Safe memory allocation functions

2015-01-15 Thread Michael Paquier
On Fri, Jan 16, 2015 at 12:57 AM, Alvaro Herrera wrote: >> I do think that "safe" is the wrong suffix. Maybe palloc_soft_fail() >> or palloc_null() or palloc_no_oom() or palloc_unsafe(). > > I liked palloc_noerror() better myself FWIW. Voting for palloc_noerror() as well. -- Michael -- Sent v

Re: [HACKERS] Safe memory allocation functions

2015-01-15 Thread Alvaro Herrera
Robert Haas wrote: > Hmm, I understood Tom to be opposing the idea of a palloc variant that > returns NULL on failure, and I understand you to be supporting it. > But maybe I'm confused. Your understanding seems correct to me. I was just saying that your description of Tom's argument to dislike

Re: [HACKERS] Safe memory allocation functions

2015-01-15 Thread Robert Haas
On Thu, Jan 15, 2015 at 8:42 AM, Andres Freund wrote: > On 2015-01-15 08:40:34 -0500, Robert Haas wrote: >> I do think that "safe" is the wrong suffix. Maybe palloc_soft_fail() >> or palloc_null() or palloc_no_oom() or palloc_unsafe(). > > palloc_or_null()? That'd work for me, too. -- Robert H

Re: [HACKERS] Safe memory allocation functions

2015-01-15 Thread Andres Freund
On 2015-01-15 08:40:34 -0500, Robert Haas wrote: > I do think that "safe" is the wrong suffix. Maybe palloc_soft_fail() > or palloc_null() or palloc_no_oom() or palloc_unsafe(). palloc_or_null()? Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/ Postg

Re: [HACKERS] Safe memory allocation functions

2015-01-15 Thread Robert Haas
On Wed, Jan 14, 2015 at 9:42 PM, Alvaro Herrera wrote: > Robert Haas wrote: >> On Tue, Jan 13, 2015 at 10:10 AM, Tom Lane wrote: >> > However, there is a larger practical problem with this whole concept, >> > which is that experience should teach us to be very wary of the assumption >> > that ask

Re: [HACKERS] Safe memory allocation functions

2015-01-14 Thread Alvaro Herrera
Robert Haas wrote: > On Tue, Jan 13, 2015 at 10:10 AM, Tom Lane wrote: > > However, there is a larger practical problem with this whole concept, > > which is that experience should teach us to be very wary of the assumption > > that asking for memory the system can't give us will just lead to nice

Re: [HACKERS] Safe memory allocation functions

2015-01-14 Thread Robert Haas
On Tue, Jan 13, 2015 at 10:10 AM, Tom Lane wrote: > However, there is a larger practical problem with this whole concept, > which is that experience should teach us to be very wary of the assumption > that asking for memory the system can't give us will just lead to nice > neat malloc-returns-NULL

Re: [HACKERS] Safe memory allocation functions

2015-01-13 Thread Michael Paquier
Tom Lane writes: > [blah] > (This is another reason for "_safe" not being the mot juste :-() My wording was definitely incorrect but I sure you got it: I should have said "safe on error". noerror or error_safe would are definitely more correct. > In that light, I'm not really convinced that there

Re: [HACKERS] Safe memory allocation functions

2015-01-13 Thread Tom Lane
I wrote: > Michael Paquier writes: >> Attached is a patch adding the following set of functions for frontend >> and backends returning NULL instead of reporting ERROR when allocation >> fails: >> - palloc_safe >> - palloc0_safe >> - repalloc_safe > Unimpressed with this naming convention. "_unsa

Re: [HACKERS] Safe memory allocation functions

2015-01-12 Thread David G Johnston
Michael Paquier wrote > Attached is a patch adding the following set of functions for frontend > and backends returning NULL instead of reporting ERROR when allocation > fails: > - palloc_safe > - palloc0_safe > - repalloc_safe The only thing I can contribute is paint...I'm not fond of the word "_

Re: [HACKERS] Safe memory allocation functions

2015-01-12 Thread Tom Lane
Michael Paquier writes: > Attached is a patch adding the following set of functions for frontend > and backends returning NULL instead of reporting ERROR when allocation > fails: > - palloc_safe > - palloc0_safe > - repalloc_safe Unimpressed with this naming convention. "_unsafe" would be nearer

[HACKERS] Safe memory allocation functions

2015-01-12 Thread Michael Paquier
Hi all, For the last couple of weeks it has been mentioned a couple of times that it would be useful to have a set of palloc APIs able to return NULL on OOM to allow certain code paths to not ERROR and to take another route when memory is under pressure. This has been for example mentioned on the