Re: [HACKERS] Re: patch: fix performance problems with repated decomprimation of varlena values in plpgsql

2011-02-08 Thread Robert Haas
On Mon, Feb 7, 2011 at 11:52 PM, Noah Misch n...@leadboat.com wrote: On Mon, Feb 07, 2011 at 11:16:18PM -0500, Robert Haas wrote: So can we just get rid of should_be_detoasted, and have exec_eval_datum() or its callers instead test: !var-isnull var-datatype-typbyval var-datatype-typlen ==

Re: [HACKERS] Re: patch: fix performance problems with repated decomprimation of varlena values in plpgsql

2011-02-08 Thread Noah Misch
On Tue, Feb 08, 2011 at 08:00:42AM +0100, Pavel Stehule wrote: 2011/2/8 Noah Misch n...@leadboat.com: On Mon, Feb 07, 2011 at 11:16:18PM -0500, Robert Haas wrote: So can we just get rid of should_be_detoasted, and have exec_eval_datum() or its callers instead test: !var-isnull

Re: [HACKERS] Re: patch: fix performance problems with repated decomprimation of varlena values in plpgsql

2011-02-08 Thread Pavel Stehule
2011/2/8 Noah Misch n...@leadboat.com: On Tue, Feb 08, 2011 at 08:00:42AM +0100, Pavel Stehule wrote: 2011/2/8 Noah Misch n...@leadboat.com: On Mon, Feb 07, 2011 at 11:16:18PM -0500, Robert Haas wrote: So can we just get rid of should_be_detoasted, and have exec_eval_datum() or its

Re: [HACKERS] Re: patch: fix performance problems with repated decomprimation of varlena values in plpgsql

2011-02-08 Thread Pavel Stehule
2011/2/8 Robert Haas robertmh...@gmail.com: On Mon, Feb 7, 2011 at 11:52 PM, Noah Misch n...@leadboat.com wrote: On Mon, Feb 07, 2011 at 11:16:18PM -0500, Robert Haas wrote: So can we just get rid of should_be_detoasted, and have exec_eval_datum() or its callers instead test: !var-isnull

Re: [HACKERS] Re: patch: fix performance problems with repated decomprimation of varlena values in plpgsql

2011-02-08 Thread Noah Misch
On Tue, Feb 08, 2011 at 10:24:03AM -0500, Robert Haas wrote: Well, Pavel's subsequent reply suggested that he didn't test exactly this thing, so maybe there's hope. No hope on that basis, no. Or maybe not. If Tom thought one branch inside exec_eval_datum() was going to be too expensive,

Re: [HACKERS] Re: patch: fix performance problems with repated decomprimation of varlena values in plpgsql

2011-02-08 Thread Pavel Stehule
2011/2/8 Noah Misch n...@leadboat.com: On Tue, Feb 08, 2011 at 10:24:03AM -0500, Robert Haas wrote: Well, Pavel's subsequent reply suggested that he didn't test exactly this thing, so maybe there's hope. No hope on that basis, no. Or maybe not.  If Tom thought one branch inside

Re: [HACKERS] Re: patch: fix performance problems with repated decomprimation of varlena values in plpgsql

2011-02-08 Thread Robert Haas
On Tue, Feb 8, 2011 at 11:38 AM, Noah Misch n...@leadboat.com wrote: It's not as if this patch raised complex questions that folks need more time to digest.  For a patch this small and simple, we minimally owe Pavel a direct answer about its rejection. Well, I don't see how we can give a

Re: [HACKERS] Re: patch: fix performance problems with repated decomprimation of varlena values in plpgsql

2011-02-07 Thread Robert Haas
On Sun, Feb 6, 2011 at 9:10 AM, Robert Haas robertmh...@gmail.com wrote: Suppose foo is toasted.  As the code stands in master, it gets detoasted in text_lt().  Certainly we won't overwrite the source back in PL/pgSQL from the detoast point in text_lt(). Right, that much seems obvious...

Re: [HACKERS] Re: patch: fix performance problems with repated decomprimation of varlena values in plpgsql

2011-02-07 Thread Noah Misch
On Mon, Feb 07, 2011 at 11:16:18PM -0500, Robert Haas wrote: So can we just get rid of should_be_detoasted, and have exec_eval_datum() or its callers instead test: !var-isnull var-datatype-typbyval var-datatype-typlen == -1 VARATT_IS_EXTENDED(var-value) FWIW, this is what I meant by

Re: [HACKERS] Re: patch: fix performance problems with repated decomprimation of varlena values in plpgsql

2011-02-07 Thread Pavel Stehule
2011/2/8 Robert Haas robertmh...@gmail.com: On Sun, Feb 6, 2011 at 9:10 AM, Robert Haas robertmh...@gmail.com wrote: Suppose foo is toasted.  As the code stands in master, it gets detoasted in text_lt().  Certainly we won't overwrite the source back in PL/pgSQL from the detoast point in

Re: [HACKERS] Re: patch: fix performance problems with repated decomprimation of varlena values in plpgsql

2011-02-07 Thread Pavel Stehule
2011/2/8 Noah Misch n...@leadboat.com: On Mon, Feb 07, 2011 at 11:16:18PM -0500, Robert Haas wrote: So can we just get rid of should_be_detoasted, and have exec_eval_datum() or its callers instead test: !var-isnull var-datatype-typbyval var-datatype-typlen == -1

Re: [HACKERS] Re: patch: fix performance problems with repated decomprimation of varlena values in plpgsql

2011-02-06 Thread Noah Misch
Let's see if I can summarize the facts we've collected so far. I see four options based on the discussion: 1. Add PLpgSQL_var.should_be_detoasted; check it in plpgsql_param_fetch(). Essentially Pavel's original patch, only with the check logic moved up from exec_eval_datum() to

Re: [HACKERS] Re: patch: fix performance problems with repated decomprimation of varlena values in plpgsql

2011-02-06 Thread Robert Haas
On Sun, Feb 6, 2011 at 5:52 AM, Noah Misch n...@leadboat.com wrote: 1. Add PLpgSQL_var.should_be_detoasted; check it in plpgsql_param_fetch(). Essentially Pavel's original patch, only with the check logic moved up from exec_eval_datum() to plpgsql_param_fetch() to avoid bothering a couple other

Re: [HACKERS] Re: patch: fix performance problems with repated decomprimation of varlena values in plpgsql

2011-02-06 Thread Noah Misch
On Sun, Feb 06, 2011 at 08:15:30AM -0500, Robert Haas wrote: On Sun, Feb 6, 2011 at 5:52 AM, Noah Misch n...@leadboat.com wrote: 1. Add PLpgSQL_var.should_be_detoasted; check it in plpgsql_param_fetch(). Essentially Pavel's original patch, only with the check logic moved up from

Re: [HACKERS] Re: patch: fix performance problems with repated decomprimation of varlena values in plpgsql

2011-02-06 Thread Robert Haas
On Sun, Feb 6, 2011 at 8:47 AM, Noah Misch n...@leadboat.com wrote: On Sun, Feb 06, 2011 at 08:15:30AM -0500, Robert Haas wrote: On Sun, Feb 6, 2011 at 5:52 AM, Noah Misch n...@leadboat.com wrote: 1. Add PLpgSQL_var.should_be_detoasted; check it in plpgsql_param_fetch(). Essentially Pavel's

Re: [HACKERS] Re: patch: fix performance problems with repated decomprimation of varlena values in plpgsql

2011-02-04 Thread Robert Haas
On Fri, Jan 28, 2011 at 2:19 PM, Pavel Stehule pavel.steh...@gmail.com wrote: 2011/1/28 Robert Haas robertmh...@gmail.com: On Tue, Jan 25, 2011 at 11:29 AM, Pavel Stehule pavel.steh...@gmail.com wrote: variant a) my first patch - detoast on first usage with avoiding to useless detoast

Re: [HACKERS] Re: patch: fix performance problems with repated decomprimation of varlena values in plpgsql

2011-02-04 Thread Pavel Stehule
2011/2/4 Robert Haas robertmh...@gmail.com: On Fri, Jan 28, 2011 at 2:19 PM, Pavel Stehule pavel.steh...@gmail.com wrote: 2011/1/28 Robert Haas robertmh...@gmail.com: On Tue, Jan 25, 2011 at 11:29 AM, Pavel Stehule pavel.steh...@gmail.com wrote: variant a) my first patch - detoast on first

Re: [HACKERS] Re: patch: fix performance problems with repated decomprimation of varlena values in plpgsql

2011-01-28 Thread Robert Haas
On Tue, Jan 25, 2011 at 11:29 AM, Pavel Stehule pavel.steh...@gmail.com wrote: variant a) my first patch - detoast on first usage with avoiding to useless detoast checking variant b) my first patch - detoast on first usage without avoiding to useless detoast checking time for 1 - about 300

Re: [HACKERS] Re: patch: fix performance problems with repated decomprimation of varlena values in plpgsql

2011-01-28 Thread Pavel Stehule
2011/1/28 Robert Haas robertmh...@gmail.com: On Tue, Jan 25, 2011 at 11:29 AM, Pavel Stehule pavel.steh...@gmail.com wrote: variant a) my first patch - detoast on first usage with avoiding to useless detoast checking variant b) my first patch - detoast on first usage without avoiding to

Re: [HACKERS] Re: patch: fix performance problems with repated decomprimation of varlena values in plpgsql

2011-01-25 Thread Tom Lane
Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes: Detoasting on first usage, ie. exec_eval_datum(), seems the best to me. Compared to detoasting on assignment, it avoids the performance regression if the value is never used, and I don't think checking if the value is toasted at

Re: [HACKERS] Re: patch: fix performance problems with repated decomprimation of varlena values in plpgsql

2011-01-25 Thread Robert Haas
On Tue, Jan 25, 2011 at 10:03 AM, Tom Lane t...@sss.pgh.pa.us wrote: Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes: Detoasting on first usage, ie. exec_eval_datum(), seems the best to me. Compared to detoasting on assignment, it avoids the performance regression if the value is

Re: [HACKERS] Re: patch: fix performance problems with repated decomprimation of varlena values in plpgsql

2011-01-25 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Tue, Jan 25, 2011 at 10:03 AM, Tom Lane t...@sss.pgh.pa.us wrote: The arguments that were made against this were about maintenance costs and code footprint.  Claims about performance are not really relevant, especially when they're entirely

Re: [HACKERS] Re: patch: fix performance problems with repated decomprimation of varlena values in plpgsql

2011-01-25 Thread Robert Haas
On Tue, Jan 25, 2011 at 10:47 AM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Tue, Jan 25, 2011 at 10:03 AM, Tom Lane t...@sss.pgh.pa.us wrote: The arguments that were made against this were about maintenance costs and code footprint.  Claims about

Re: [HACKERS] Re: patch: fix performance problems with repated decomprimation of varlena values in plpgsql

2011-01-25 Thread Pavel Stehule
2011/1/25 Tom Lane t...@sss.pgh.pa.us: Robert Haas robertmh...@gmail.com writes: On Tue, Jan 25, 2011 at 10:03 AM, Tom Lane t...@sss.pgh.pa.us wrote: The arguments that were made against this were about maintenance costs and code footprint.  Claims about performance are not really relevant,

Re: [HACKERS] Re: patch: fix performance problems with repated decomprimation of varlena values in plpgsql

2011-01-24 Thread Noah Misch
On Sat, Jan 22, 2011 at 11:32:02AM +0100, Pavel Stehule wrote: because I am not sure so any complex solution can be done to deadline for 9.1, I created a patch that is based on Tom ideas - just explicitly detoast function parameters. I can confirm that, for your original test case, this yields

Re: [HACKERS] Re: patch: fix performance problems with repated decomprimation of varlena values in plpgsql

2011-01-24 Thread Pavel Stehule
Hello 2011/1/25 Noah Misch n...@leadboat.com: On Sat, Jan 22, 2011 at 11:32:02AM +0100, Pavel Stehule wrote: because I am not sure so any complex solution can be done to deadline for 9.1, I created a patch that is based on Tom ideas - just explicitly detoast function parameters. I can

Re: [HACKERS] Re: patch: fix performance problems with repated decomprimation of varlena values in plpgsql

2011-01-24 Thread Heikki Linnakangas
On 25.01.2011 06:29, Pavel Stehule wrote: 2011/1/25 Noah Mischn...@leadboat.com: On Sat, Jan 22, 2011 at 11:32:02AM +0100, Pavel Stehule wrote: because I am not sure so any complex solution can be done to deadline for 9.1, I created a patch that is based on Tom ideas - just explicitly detoast

Re: [HACKERS] Re: patch: fix performance problems with repated decomprimation of varlena values in plpgsql

2011-01-22 Thread Pavel Stehule
Hello, because I am not sure so any complex solution can be done to deadline for 9.1, I created a patch that is based on Tom ideas - just explicitly detoast function parameters. Regards Pavel 2011/1/19 Robert Haas robertmh...@gmail.com: On Wed, Jan 19, 2011 at 4:18 PM, Tom Lane

Re: [HACKERS] Re: patch: fix performance problems with repated decomprimation of varlena values in plpgsql

2011-01-19 Thread Robert Haas
On Tue, Jan 18, 2011 at 5:22 PM, Pavel Stehule pavel.steh...@gmail.com wrote: opinion isn't strong in this topic. One or twenty useless detoasting isn't really significant in almost use cases (problem is thousands detoasting). Yeah. Many-times-repeated detoasting is really bad, and this is

Re: [HACKERS] Re: patch: fix performance problems with repated decomprimation of varlena values in plpgsql

2011-01-19 Thread Pavel Stehule
2011/1/19 Robert Haas robertmh...@gmail.com: On Tue, Jan 18, 2011 at 5:22 PM, Pavel Stehule pavel.steh...@gmail.com wrote: opinion isn't strong in this topic. One or twenty useless detoasting isn't really significant in almost use cases (problem is thousands detoasting). Yeah.  

Re: [HACKERS] Re: patch: fix performance problems with repated decomprimation of varlena values in plpgsql

2011-01-19 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Tue, Jan 18, 2011 at 5:22 PM, Pavel Stehule pavel.steh...@gmail.com wrote: opinion isn't strong in this topic. One or twenty useless detoasting isn't really significant in almost use cases (problem is thousands detoasting). Yeah.

Re: [HACKERS] Re: patch: fix performance problems with repated decomprimation of varlena values in plpgsql

2011-01-19 Thread Robert Haas
On Wed, Jan 19, 2011 at 12:10 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Tue, Jan 18, 2011 at 5:22 PM, Pavel Stehule pavel.steh...@gmail.com wrote: opinion isn't strong in this topic. One or twenty useless detoasting isn't really significant in almost

Re: [HACKERS] Re: patch: fix performance problems with repated decomprimation of varlena values in plpgsql

2011-01-19 Thread Noah Misch
On Wed, Jan 19, 2011 at 12:10:16PM -0500, Tom Lane wrote: Robert Haas robertmh...@gmail.com writes: On Tue, Jan 18, 2011 at 5:22 PM, Pavel Stehule pavel.steh...@gmail.com wrote: opinion isn't strong in this topic. One or twenty useless detoasting isn't really significant in almost use

Re: [HACKERS] Re: patch: fix performance problems with repated decomprimation of varlena values in plpgsql

2011-01-19 Thread Tom Lane
Noah Misch n...@leadboat.com writes: On Wed, Jan 19, 2011 at 12:10:16PM -0500, Tom Lane wrote: In the meantime, the proposal at hand seems like a bit of a stop-gap, which is why I'd prefer to see something with a very minimal code footprint. Detoast at assignment would likely need only a few

Re: [HACKERS] Re: patch: fix performance problems with repated decomprimation of varlena values in plpgsql

2011-01-19 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Wed, Jan 19, 2011 at 12:10 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: Yeah. Many-times-repeated detoasting is really bad, and this is not the only place in the backend where we have this problem. :-( Yeah,

Re: [HACKERS] Re: patch: fix performance problems with repated decomprimation of varlena values in plpgsql

2011-01-19 Thread Robert Haas
On Wed, Jan 19, 2011 at 2:58 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Wed, Jan 19, 2011 at 12:10 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: Yeah.  Many-times-repeated detoasting is really bad, and this is not the

Re: [HACKERS] Re: patch: fix performance problems with repated decomprimation of varlena values in plpgsql

2011-01-19 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: I do remember that discussion. Aside from the problem you mention, it also seems that maintaining the hash table and doing lookups into it would have some intrinsic cost. Well, sure, but it's still far cheaper than going out to the toast table (which

Re: [HACKERS] Re: patch: fix performance problems with repated decomprimation of varlena values in plpgsql

2011-01-19 Thread Kevin Grittner
Tom Lane t...@sss.pgh.pa.us wrote: If we could solve the refcounting problem I think this would be a very significant win. Instead of trying to keep a refcount, how about just evicting from the buffer as needed based on LRU? -Kevin -- Sent via pgsql-hackers mailing list

Re: [HACKERS] Re: patch: fix performance problems with repated decomprimation of varlena values in plpgsql

2011-01-19 Thread Tom Lane
Kevin Grittner kevin.gritt...@wicourts.gov writes: Tom Lane t...@sss.pgh.pa.us wrote: If we could solve the refcounting problem I think this would be a very significant win. Instead of trying to keep a refcount, how about just evicting from the buffer as needed based on LRU? Well, unless

Re: [HACKERS] Re: patch: fix performance problems with repated decomprimation of varlena values in plpgsql

2011-01-19 Thread Robert Haas
On Wed, Jan 19, 2011 at 4:18 PM, Tom Lane t...@sss.pgh.pa.us wrote: One idea that I think we discussed was to tie cache entries to the memory context they were demanded in, and mark them unused at the next context reset/delete.  That way they'd be considered unused at the same points where the

Re: [HACKERS] Re: patch: fix performance problems with repated decomprimation of varlena values in plpgsql

2011-01-18 Thread Tom Lane
Noah Misch n...@leadboat.com writes: On Sun, Jan 16, 2011 at 06:49:27PM +0100, Pavel Stehule wrote: I am sending a updated version with little bit more comments. But I am sure, so somebody with good English have to edit my comments. Minimally I hope, so your questions will be answered.

Re: [HACKERS] Re: patch: fix performance problems with repated decomprimation of varlena values in plpgsql

2011-01-18 Thread Pavel Stehule
2011/1/18 Tom Lane t...@sss.pgh.pa.us: Noah Misch n...@leadboat.com writes: On Sun, Jan 16, 2011 at 06:49:27PM +0100, Pavel Stehule wrote: I am sending a updated version with little bit more comments. But I am sure, so somebody with good English have to edit my comments. Minimally I hope, so

Re: [HACKERS] Re: patch: fix performance problems with repated decomprimation of varlena values in plpgsql

2011-01-18 Thread Tom Lane
Pavel Stehule pavel.steh...@gmail.com writes: 2011/1/18 Tom Lane t...@sss.pgh.pa.us: I looked at this patch and found it fairly awkward.  What is the point of adding an additional flag to every variable, as opposed to just forcibly detoasting during assignment? But detoasting on assignment

Re: [HACKERS] Re: patch: fix performance problems with repated decomprimation of varlena values in plpgsql

2011-01-18 Thread Pavel Stehule
2011/1/18 Tom Lane t...@sss.pgh.pa.us: Pavel Stehule pavel.steh...@gmail.com writes: 2011/1/18 Tom Lane t...@sss.pgh.pa.us: I looked at this patch and found it fairly awkward.  What is the point of adding an additional flag to every variable, as opposed to just forcibly detoasting during

[HACKERS] Re: patch: fix performance problems with repated decomprimation of varlena values in plpgsql

2011-01-16 Thread Pavel Stehule
Hello 2011/1/15 Noah Misch n...@leadboat.com: Hello Pavel, I'm reviewing this patch for CommitFest 2011-01. Thank you very much, I am sending a updated version with little bit more comments. But I am sure, so somebody with good English have to edit my comments. Minimally I hope, so your

[HACKERS] Re: patch: fix performance problems with repated decomprimation of varlena values in plpgsql

2011-01-16 Thread Noah Misch
On Sun, Jan 16, 2011 at 06:49:27PM +0100, Pavel Stehule wrote: I am sending a updated version with little bit more comments. But I am sure, so somebody with good English have to edit my comments. Minimally I hope, so your questions will be answered. Thanks. I edited the comments and white

[HACKERS] Re: patch: fix performance problems with repated decomprimation of varlena values in plpgsql

2011-01-15 Thread Noah Misch
Hello Pavel, I'm reviewing this patch for CommitFest 2011-01. The patch seems fully desirable. It appropriately contains no documentation updates. It contains no new tests, and that's probably fine, too; I can't think of any corner cases where this would do something other than work correctly