Re: [HACKERS] Re: patch: fix performance problems with repated decomprimation of varlena values in plpgsql

2011-02-08 Thread Robert Haas
On Tue, Feb 8, 2011 at 11:38 AM, Noah Misch wrote: > It's not as if this patch raised complex questions that folks need more time > to > digest.  For a patch this small and simple, we minimally owe Pavel a direct > answer about its rejection. Well, I don't see how we can give a totally straightf

Re: [HACKERS] Re: patch: fix performance problems with repated decomprimation of varlena values in plpgsql

2011-02-08 Thread Pavel Stehule
2011/2/8 Noah Misch : > On Tue, Feb 08, 2011 at 10:24:03AM -0500, Robert Haas wrote: >> Well, Pavel's subsequent reply suggested that he didn't test exactly >> this thing, so maybe there's hope. > > No hope on that basis, no. > >> Or maybe not.  If Tom thought one branch inside exec_eval_datum() wa

Re: [HACKERS] Re: patch: fix performance problems with repated decomprimation of varlena values in plpgsql

2011-02-08 Thread Noah Misch
On Tue, Feb 08, 2011 at 10:24:03AM -0500, Robert Haas wrote: > Well, Pavel's subsequent reply suggested that he didn't test exactly > this thing, so maybe there's hope. No hope on that basis, no. > Or maybe not. If Tom thought one branch inside exec_eval_datum() was > going to be too expensive,

Re: [HACKERS] Re: patch: fix performance problems with repated decomprimation of varlena values in plpgsql

2011-02-08 Thread Pavel Stehule
2011/2/8 Robert Haas : > On Mon, Feb 7, 2011 at 11:52 PM, Noah Misch wrote: >> On Mon, Feb 07, 2011 at 11:16:18PM -0500, Robert Haas wrote: >>> So >>> can we just get rid of should_be_detoasted, and have exec_eval_datum() >>> or its callers instead test: >>> >>> !var->isnull && var->datatype->typb

Re: [HACKERS] Re: patch: fix performance problems with repated decomprimation of varlena values in plpgsql

2011-02-08 Thread Pavel Stehule
2011/2/8 Noah Misch : > On Tue, Feb 08, 2011 at 08:00:42AM +0100, Pavel Stehule wrote: >> 2011/2/8 Noah Misch : >> > On Mon, Feb 07, 2011 at 11:16:18PM -0500, Robert Haas wrote: >> >> So >> >> can we just get rid of should_be_detoasted, and have exec_eval_datum() >> >> or its callers instead test:

Re: [HACKERS] Re: patch: fix performance problems with repated decomprimation of varlena values in plpgsql

2011-02-08 Thread Noah Misch
On Tue, Feb 08, 2011 at 08:00:42AM +0100, Pavel Stehule wrote: > 2011/2/8 Noah Misch : > > On Mon, Feb 07, 2011 at 11:16:18PM -0500, Robert Haas wrote: > >> So > >> can we just get rid of should_be_detoasted, and have exec_eval_datum() > >> or its callers instead test: > >> > >> !var->isnull && var

Re: [HACKERS] Re: patch: fix performance problems with repated decomprimation of varlena values in plpgsql

2011-02-08 Thread Robert Haas
On Mon, Feb 7, 2011 at 11:52 PM, Noah Misch wrote: > On Mon, Feb 07, 2011 at 11:16:18PM -0500, Robert Haas wrote: >> So >> can we just get rid of should_be_detoasted, and have exec_eval_datum() >> or its callers instead test: >> >> !var->isnull && var->datatype->typbyval && var->datatype->typlen =

Re: [HACKERS] Re: patch: fix performance problems with repated decomprimation of varlena values in plpgsql

2011-02-07 Thread Pavel Stehule
2011/2/8 Noah Misch : > On Mon, Feb 07, 2011 at 11:16:18PM -0500, Robert Haas wrote: >> So >> can we just get rid of should_be_detoasted, and have exec_eval_datum() >> or its callers instead test: >> >> !var->isnull && var->datatype->typbyval && var->datatype->typlen == -1 >> && VARATT_IS_EXTENDED(

Re: [HACKERS] Re: patch: fix performance problems with repated decomprimation of varlena values in plpgsql

2011-02-07 Thread Pavel Stehule
2011/2/8 Robert Haas : > On Sun, Feb 6, 2011 at 9:10 AM, Robert Haas wrote: >>> Suppose "foo" is toasted.  As the code stands in master, it gets detoasted >>> in >>> text_lt().  Certainly we won't overwrite the source back in PL/pgSQL from >>> the >>> detoast point in text_lt(). >> >> Right, tha

Re: [HACKERS] Re: patch: fix performance problems with repated decomprimation of varlena values in plpgsql

2011-02-07 Thread Noah Misch
On Mon, Feb 07, 2011 at 11:16:18PM -0500, Robert Haas wrote: > So > can we just get rid of should_be_detoasted, and have exec_eval_datum() > or its callers instead test: > > !var->isnull && var->datatype->typbyval && var->datatype->typlen == -1 > && VARATT_IS_EXTENDED(var->value) FWIW, this is wh

Re: [HACKERS] Re: patch: fix performance problems with repated decomprimation of varlena values in plpgsql

2011-02-07 Thread Robert Haas
On Sun, Feb 6, 2011 at 9:10 AM, Robert Haas wrote: >> Suppose "foo" is toasted.  As the code stands in master, it gets detoasted in >> text_lt().  Certainly we won't overwrite the source back in PL/pgSQL from the >> detoast point in text_lt(). > > Right, that much seems obvious... > >> Pavel's opt

Re: [HACKERS] Re: patch: fix performance problems with repated decomprimation of varlena values in plpgsql

2011-02-06 Thread Robert Haas
On Sun, Feb 6, 2011 at 8:47 AM, Noah Misch wrote: > On Sun, Feb 06, 2011 at 08:15:30AM -0500, Robert Haas wrote: >> On Sun, Feb 6, 2011 at 5:52 AM, Noah Misch wrote: >> > 1. Add PLpgSQL_var.should_be_detoasted; check it in plpgsql_param_fetch(). >> > Essentially Pavel's original patch, only with

Re: [HACKERS] Re: patch: fix performance problems with repated decomprimation of varlena values in plpgsql

2011-02-06 Thread Noah Misch
On Sun, Feb 06, 2011 at 08:15:30AM -0500, Robert Haas wrote: > On Sun, Feb 6, 2011 at 5:52 AM, Noah Misch wrote: > > 1. Add PLpgSQL_var.should_be_detoasted; check it in plpgsql_param_fetch(). > > Essentially Pavel's original patch, only with the check logic moved up from > > exec_eval_datum() to p

Re: [HACKERS] Re: patch: fix performance problems with repated decomprimation of varlena values in plpgsql

2011-02-06 Thread Robert Haas
On Sun, Feb 6, 2011 at 5:52 AM, Noah Misch wrote: > 1. Add PLpgSQL_var.should_be_detoasted; check it in plpgsql_param_fetch(). > Essentially Pavel's original patch, only with the check logic moved up from > exec_eval_datum() to plpgsql_param_fetch() to avoid bothering a couple other > callers that

Re: [HACKERS] Re: patch: fix performance problems with repated decomprimation of varlena values in plpgsql

2011-02-06 Thread Noah Misch
Let's see if I can summarize the facts we've collected so far. I see four options based on the discussion: 1. Add PLpgSQL_var.should_be_detoasted; check it in plpgsql_param_fetch(). Essentially Pavel's original patch, only with the check logic moved up from exec_eval_datum() to plpgsql_param_fetc

Re: [HACKERS] Re: patch: fix performance problems with repated decomprimation of varlena values in plpgsql

2011-02-04 Thread Pavel Stehule
2011/2/4 Robert Haas : > On Fri, Jan 28, 2011 at 2:19 PM, Pavel Stehule > wrote: >> 2011/1/28 Robert Haas : >>> On Tue, Jan 25, 2011 at 11:29 AM, Pavel Stehule >>> wrote: variant a) my first patch - detoast on first usage with avoiding to useless detoast checking variant b) my fi

Re: [HACKERS] Re: patch: fix performance problems with repated decomprimation of varlena values in plpgsql

2011-02-04 Thread Robert Haas
On Fri, Jan 28, 2011 at 2:19 PM, Pavel Stehule wrote: > 2011/1/28 Robert Haas : >> On Tue, Jan 25, 2011 at 11:29 AM, Pavel Stehule >> wrote: >>> variant a) my first patch - detoast on first usage with avoiding to >>> useless detoast checking >>> variant b) my first patch - detoast on first usage

Re: [HACKERS] Re: patch: fix performance problems with repated decomprimation of varlena values in plpgsql

2011-01-28 Thread Pavel Stehule
2011/1/28 Robert Haas : > On Tue, Jan 25, 2011 at 11:29 AM, Pavel Stehule > wrote: >> variant a) my first patch - detoast on first usage with avoiding to >> useless detoast checking >> variant b) my first patch - detoast on first usage without avoiding to >> useless detoast checking >> >> time fo

Re: [HACKERS] Re: patch: fix performance problems with repated decomprimation of varlena values in plpgsql

2011-01-28 Thread Robert Haas
On Tue, Jan 25, 2011 at 11:29 AM, Pavel Stehule wrote: > variant a) my first patch - detoast on first usage with avoiding to > useless detoast checking > variant b) my first patch - detoast on first usage without avoiding to > useless detoast checking > > time for 1 - about 300 ms, a is bout 1.5%

Re: [HACKERS] Re: patch: fix performance problems with repated decomprimation of varlena values in plpgsql

2011-01-25 Thread Pavel Stehule
2011/1/25 Tom Lane : > Robert Haas writes: >> On Tue, Jan 25, 2011 at 10:03 AM, Tom Lane wrote: >>> The arguments that were made against this were about maintenance costs >>> and code footprint.  Claims about performance are not really relevant, >>> especially when they're entirely unsupported by

Re: [HACKERS] Re: patch: fix performance problems with repated decomprimation of varlena values in plpgsql

2011-01-25 Thread Robert Haas
On Tue, Jan 25, 2011 at 10:47 AM, Tom Lane wrote: > Robert Haas writes: >> On Tue, Jan 25, 2011 at 10:03 AM, Tom Lane wrote: >>> The arguments that were made against this were about maintenance costs >>> and code footprint.  Claims about performance are not really relevant, >>> especially when t

Re: [HACKERS] Re: patch: fix performance problems with repated decomprimation of varlena values in plpgsql

2011-01-25 Thread Tom Lane
Robert Haas writes: > On Tue, Jan 25, 2011 at 10:03 AM, Tom Lane wrote: >> The arguments that were made against this were about maintenance costs >> and code footprint.  Claims about performance are not really relevant, >> especially when they're entirely unsupported by evidence. > How much evid

Re: [HACKERS] Re: patch: fix performance problems with repated decomprimation of varlena values in plpgsql

2011-01-25 Thread Robert Haas
On Tue, Jan 25, 2011 at 10:03 AM, Tom Lane wrote: > Heikki Linnakangas writes: >> Detoasting on first usage, ie. exec_eval_datum(), seems the best to me. >> Compared to detoasting on assignment, it avoids the performance >> regression if the value is never used, and I don't think checking if the

Re: [HACKERS] Re: patch: fix performance problems with repated decomprimation of varlena values in plpgsql

2011-01-25 Thread Tom Lane
Heikki Linnakangas writes: > Detoasting on first usage, ie. exec_eval_datum(), seems the best to me. > Compared to detoasting on assignment, it avoids the performance > regression if the value is never used, and I don't think checking if the > value is toasted at every exec_eval_datum() call ad

Re: [HACKERS] Re: patch: fix performance problems with repated decomprimation of varlena values in plpgsql

2011-01-24 Thread Heikki Linnakangas
On 25.01.2011 06:29, Pavel Stehule wrote: 2011/1/25 Noah Misch: On Sat, Jan 22, 2011 at 11:32:02AM +0100, Pavel Stehule wrote: because I am not sure so any complex solution can be done to deadline for 9.1, I created a patch that is based on Tom ideas - just explicitly detoast function parameter

Re: [HACKERS] Re: patch: fix performance problems with repated decomprimation of varlena values in plpgsql

2011-01-24 Thread Pavel Stehule
Hello 2011/1/25 Noah Misch : > On Sat, Jan 22, 2011 at 11:32:02AM +0100, Pavel Stehule wrote: >> because I am not sure so any complex solution can be done to deadline >> for 9.1, I created a patch that is based on Tom ideas - just >> explicitly detoast function parameters. > > I can confirm that,

Re: [HACKERS] Re: patch: fix performance problems with repated decomprimation of varlena values in plpgsql

2011-01-24 Thread Noah Misch
On Sat, Jan 22, 2011 at 11:32:02AM +0100, Pavel Stehule wrote: > because I am not sure so any complex solution can be done to deadline > for 9.1, I created a patch that is based on Tom ideas - just > explicitly detoast function parameters. I can confirm that, for your original test case, this yiel

Re: [HACKERS] Re: patch: fix performance problems with repated decomprimation of varlena values in plpgsql

2011-01-22 Thread Pavel Stehule
Hello, because I am not sure so any complex solution can be done to deadline for 9.1, I created a patch that is based on Tom ideas - just explicitly detoast function parameters. Regards Pavel 2011/1/19 Robert Haas : > On Wed, Jan 19, 2011 at 4:18 PM, Tom Lane wrote: >> One idea that I think we

Re: [HACKERS] Re: patch: fix performance problems with repated decomprimation of varlena values in plpgsql

2011-01-19 Thread Robert Haas
On Wed, Jan 19, 2011 at 4:18 PM, Tom Lane wrote: > One idea that I think we discussed was to tie cache entries to the > memory context they were demanded in, and mark them unused at the next > context reset/delete.  That way they'd be considered unused at the same > points where the current implem

Re: [HACKERS] Re: patch: fix performance problems with repated decomprimation of varlena values in plpgsql

2011-01-19 Thread Tom Lane
"Kevin Grittner" writes: > Tom Lane wrote: >> If we could solve the refcounting problem I think this would be a >> very significant win. > Instead of trying to keep a refcount, how about just evicting from > the buffer as needed based on LRU? Well, unless you know for certain that an item is n

Re: [HACKERS] Re: patch: fix performance problems with repated decomprimation of varlena values in plpgsql

2011-01-19 Thread Kevin Grittner
Tom Lane wrote: > If we could solve the refcounting problem I think this would be a > very significant win. Instead of trying to keep a refcount, how about just evicting from the buffer as needed based on LRU? -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To

Re: [HACKERS] Re: patch: fix performance problems with repated decomprimation of varlena values in plpgsql

2011-01-19 Thread Tom Lane
Robert Haas writes: > I do remember that discussion. Aside from the problem you mention, it > also seems that maintaining the hash table and doing lookups into it > would have some intrinsic cost. Well, sure, but it's still far cheaper than going out to the toast table (which will require multip

Re: [HACKERS] Re: patch: fix performance problems with repated decomprimation of varlena values in plpgsql

2011-01-19 Thread Robert Haas
On Wed, Jan 19, 2011 at 2:58 PM, Tom Lane wrote: > Robert Haas writes: >> On Wed, Jan 19, 2011 at 12:10 PM, Tom Lane wrote: >>> Robert Haas writes: Yeah.  Many-times-repeated detoasting is really bad, and this is not the only place in the backend where we have this problem.  :-( > >>>

Re: [HACKERS] Re: patch: fix performance problems with repated decomprimation of varlena values in plpgsql

2011-01-19 Thread Tom Lane
Robert Haas writes: > On Wed, Jan 19, 2011 at 12:10 PM, Tom Lane wrote: >> Robert Haas writes: >>> Yeah. Many-times-repeated detoasting is really bad, and this is not >>> the only place in the backend where we have this problem. :-( >> Yeah, there's been some discussion of a more general solu

Re: [HACKERS] Re: patch: fix performance problems with repated decomprimation of varlena values in plpgsql

2011-01-19 Thread Tom Lane
Noah Misch writes: > On Wed, Jan 19, 2011 at 12:10:16PM -0500, Tom Lane wrote: >> In the meantime, the proposal at hand seems like a bit of a stop-gap, >> which is why I'd prefer to see something with a very minimal code >> footprint. Detoast at assignment would likely need only a few lines >> of

Re: [HACKERS] Re: patch: fix performance problems with repated decomprimation of varlena values in plpgsql

2011-01-19 Thread Noah Misch
On Wed, Jan 19, 2011 at 12:10:16PM -0500, Tom Lane wrote: > Robert Haas writes: > > On Tue, Jan 18, 2011 at 5:22 PM, Pavel Stehule > > wrote: > >> opinion isn't strong in this topic. One or twenty useless detoasting > >> isn't really significant in almost use cases (problem is thousands > >> det

Re: [HACKERS] Re: patch: fix performance problems with repated decomprimation of varlena values in plpgsql

2011-01-19 Thread Robert Haas
On Wed, Jan 19, 2011 at 12:10 PM, Tom Lane wrote: > Robert Haas writes: >> On Tue, Jan 18, 2011 at 5:22 PM, Pavel Stehule >> wrote: >>> opinion isn't strong in this topic. One or twenty useless detoasting >>> isn't really significant in almost use cases (problem is thousands >>> detoasting). >

Re: [HACKERS] Re: patch: fix performance problems with repated decomprimation of varlena values in plpgsql

2011-01-19 Thread Tom Lane
Robert Haas writes: > On Tue, Jan 18, 2011 at 5:22 PM, Pavel Stehule > wrote: >> opinion isn't strong in this topic. One or twenty useless detoasting >> isn't really significant in almost use cases (problem is thousands >> detoasting). > Yeah. Many-times-repeated detoasting is really bad, and

Re: [HACKERS] Re: patch: fix performance problems with repated decomprimation of varlena values in plpgsql

2011-01-19 Thread Pavel Stehule
2011/1/19 Robert Haas : > On Tue, Jan 18, 2011 at 5:22 PM, Pavel Stehule > wrote: >> opinion isn't strong in this topic. One or twenty useless detoasting >> isn't really significant in almost use cases (problem is thousands >> detoasting). > > Yeah.  Many-times-repeated detoasting is really bad,

Re: [HACKERS] Re: patch: fix performance problems with repated decomprimation of varlena values in plpgsql

2011-01-19 Thread Robert Haas
On Tue, Jan 18, 2011 at 5:22 PM, Pavel Stehule wrote: > opinion isn't strong in this topic. One or twenty useless detoasting > isn't really significant in almost use cases (problem is thousands > detoasting). Yeah. Many-times-repeated detoasting is really bad, and this is not the only place in t

Re: [HACKERS] Re: patch: fix performance problems with repated decomprimation of varlena values in plpgsql

2011-01-18 Thread Pavel Stehule
2011/1/18 Tom Lane : > Pavel Stehule writes: >> 2011/1/18 Tom Lane : >>> I looked at this patch and found it fairly awkward.  What is the point >>> of adding an additional flag to every variable, as opposed to just >>> forcibly detoasting during assignment? > >> But detoasting on assignment isn't

Re: [HACKERS] Re: patch: fix performance problems with repated decomprimation of varlena values in plpgsql

2011-01-18 Thread Tom Lane
Pavel Stehule writes: > 2011/1/18 Tom Lane : >> I looked at this patch and found it fairly awkward.  What is the point >> of adding an additional flag to every variable, as opposed to just >> forcibly detoasting during assignment? > But detoasting on assignment isn't enought: > for i in array_l

Re: [HACKERS] Re: patch: fix performance problems with repated decomprimation of varlena values in plpgsql

2011-01-18 Thread Pavel Stehule
2011/1/18 Tom Lane : > Noah Misch writes: >> On Sun, Jan 16, 2011 at 06:49:27PM +0100, Pavel Stehule wrote: >>> I am sending a updated version with little bit more comments. But I am >>> sure, so somebody with good English have to edit my comments. >>> Minimally I hope, so your questions will be a

Re: [HACKERS] Re: patch: fix performance problems with repated decomprimation of varlena values in plpgsql

2011-01-18 Thread Tom Lane
Noah Misch writes: > On Sun, Jan 16, 2011 at 06:49:27PM +0100, Pavel Stehule wrote: >> I am sending a updated version with little bit more comments. But I am >> sure, so somebody with good English have to edit my comments. >> Minimally I hope, so your questions will be answered. > Thanks. I edit

[HACKERS] Re: patch: fix performance problems with repated decomprimation of varlena values in plpgsql

2011-01-16 Thread Noah Misch
On Sun, Jan 16, 2011 at 06:49:27PM +0100, Pavel Stehule wrote: > I am sending a updated version with little bit more comments. But I am > sure, so somebody with good English have to edit my comments. > Minimally I hope, so your questions will be answered. Thanks. I edited the comments and white s

[HACKERS] Re: patch: fix performance problems with repated decomprimation of varlena values in plpgsql

2011-01-16 Thread Pavel Stehule
Hello 2011/1/15 Noah Misch : > Hello Pavel, > > I'm reviewing this patch for CommitFest 2011-01. > Thank you very much, I am sending a updated version with little bit more comments. But I am sure, so somebody with good English have to edit my comments. Minimally I hope, so your questions will be

[HACKERS] Re: patch: fix performance problems with repated decomprimation of varlena values in plpgsql

2011-01-15 Thread Noah Misch
Hello Pavel, I'm reviewing this patch for CommitFest 2011-01. The patch seems fully desirable. It appropriately contains no documentation updates. It contains no new tests, and that's probably fine, too; I can't think of any corner cases where this would do something other than work correctly o