[HACKERS] SSL renegotiation

2015-02-21 Thread Emil Lenngren
Hi. I noticed your latest mail thread about ssl renegotiations and this commit by hlinnaka: https://github.com/postgres/postgres/commit/272923a0a6956187471df4f032eee06559520390 and how renegotiation is done since that. Before this commit, after sending a Hello Request, the backend waits until

Re: [HACKERS] Abbreviated keys for Numeric

2015-02-21 Thread Peter Geoghegan
On Sat, Feb 21, 2015 at 12:15 PM, Tomas Vondra tomas.von...@2ndquadrant.com wrote: Although I cannot easily explain the disparity in performance between 1M and 5M sized sets for this query: select count(distinct randtxt) from stuff_text You did make sure that the queries didn't spill to

Re: [HACKERS] Abbreviated keys for Numeric

2015-02-21 Thread Tomas Vondra
On 21.2.2015 23:09, Peter Geoghegan wrote: On Sat, Feb 21, 2015 at 12:15 PM, Tomas Vondra tomas.von...@2ndquadrant.com wrote: Although I cannot easily explain the disparity in performance between 1M and 5M sized sets for this query: select count(distinct randtxt) from stuff_text You did

[HACKERS] Variable renaming in AllocSetContextCreate (will commit soon, no functional impact)

2015-02-21 Thread Jeff Davis
This is just to make the variable names in this function consistent with the rest of the file (and less confusing). No functional impact, so I'll commit soon unless someone objects. Previously, this was part of the memory accounting patch, but that hasn't made it in yet. Might as well commit the

Re: [HACKERS] Variable renaming in AllocSetContextCreate (will commit soon, no functional impact)

2015-02-21 Thread Jeff Davis
On Sun, 2015-02-22 at 00:07 -0500, Tom Lane wrote: If you want to have just *one* variable but change its name and type, I'd be ok with that. Thank you for taking a quick look. Committed as a simple rename from context to set. Regards, Jeff Davis -- Sent via pgsql-hackers mailing

Re: [HACKERS] Bootstrap DATA is a pita

2015-02-21 Thread Andres Freund
On 2015-02-21 17:43:09 +0100, Andres Freund wrote: One thing I was considering was to do the regtype and regproc lookups directly in the tool. That'd have two advantages: 1) it'd make it possible to refer to typenames in pg_proc, 2) It'd be much faster. Right now most of initdb's time is doing

Re: [HACKERS] Add min and max execute statement time in pg_stat_statement

2015-02-21 Thread Petr Jelinek
On 21/02/15 22:09, Andrew Dunstan wrote: On 02/16/2015 09:05 PM, Petr Jelinek wrote: I found one more issue with the 1.2--1.3 upgrade script, the DROP FUNCTION pg_stat_statements(); should be DROP FUNCTION pg_stat_statements(bool); since in 1.2 the function identity has changed. I think

Re: [HACKERS] Redesigning checkpoint_segments

2015-02-21 Thread Petr Jelinek
On 13/02/15 18:43, Heikki Linnakangas wrote: Ok, I don't hear any loud objections to min_wal_size and max_wal_size, so let's go with that then. Attached is a new version of this. It now comes in four patches. The first three are just GUC-related preliminary work, the first of which I posted on

Re: [HACKERS] Expanding the use of FLEXIBLE_ARRAY_MEMBER for declarations like foo[1]

2015-02-21 Thread Tom Lane
After some more hacking, the only remaining uses of foo[1] in struct declarations are: 1. A couple of places where the array is actually the only struct member; for some unexplainable reason gcc won't let you use flexible array syntax in that case. 2. struct sqlda_struct in ecpg's

Re: [HACKERS] Abbreviated keys for Numeric

2015-02-21 Thread Peter Geoghegan
On Sat, Feb 21, 2015 at 2:09 PM, Peter Geoghegan p...@heroku.com wrote: I can recreate the problem you see with text sort regressions. Abbreviation is aborted for the case in question, unsurprisingly, and fairly far in. With that many tuples, the idea of taking abbreviated cardinality as a

Re: [HACKERS] Parallel Seq Scan

2015-02-21 Thread Robert Haas
On Tue, Feb 17, 2015 at 11:22 AM, Andres Freund and...@2ndquadrant.com wrote: I still think this belongs in heapam.c somehow or other. If the logic is all in the executor, then it becomes impossible for any code that doensn't use the executor to do a parallel heap scan, and that's probably

Re: [HACKERS] PATCH: decreasing memory needlessly consumed by array_agg

2015-02-21 Thread Tomas Vondra
On 22.2.2015 02:38, Jeff Davis wrote: On Wed, 2015-01-28 at 23:25 +0100, Tomas Vondra wrote: 3) moves the assert into the 'if (release)' branch You missed one, but I got it. Oh, thanks! 4) includes the comments proposed by Ali Akbar in his reviews Warnings at

Re: [HACKERS] PATCH: decreasing memory needlessly consumed by array_agg

2015-02-21 Thread Jeff Davis
On Sun, 2015-02-22 at 03:14 +0100, Tomas Vondra wrote: SELECT COUNT(x) FROM ( SELECT a, array_agg(i) AS x FRM test GROUP BY 1 ) foo; That's actually a bogus test -- array_agg is never executed. Really? How could that happen when the result of array_agg() is passed to the

Re: [HACKERS] Precedence of standard comparison operators

2015-02-21 Thread Tom Lane
Attached is a draft patch to bring the precedence of comparison operators and IS tests into line with the SQL standard. I have not yet looked into producing warnings for changes in parsing decisions; but I was gratified to discover that this patch results in none, nada, zero changes in any of our

[HACKERS] Abbreviated keys for text cost model fix

2015-02-21 Thread Peter Geoghegan
Over in the abbreviated keys for numeric thread, Tomas Vondra reports a case where the ad-hoc cost model of abbreviated keys/sortsupport for text is far too conservative, and aborts a case where abbreviated keys can still greatly help. Previously, I proposed additions to the cost model that dealt

Re: [HACKERS] proposal: searching in array function - array_position

2015-02-21 Thread Petr Jelinek
On 28/01/15 08:15, Pavel Stehule wrote: 2015-01-28 0:01 GMT+01:00 Jim Nasby jim.na...@bluetreble.com mailto:jim.na...@bluetreble.com: On 1/27/15 4:36 AM, Pavel Stehule wrote: It is only partially identical - I would to use cache for array_offset, but it is not necessary

Re: [HACKERS] Variable renaming in AllocSetContextCreate (will commit soon, no functional impact)

2015-02-21 Thread Tom Lane
Jeff Davis pg...@j-davis.com writes: This is just to make the variable names in this function consistent with the rest of the file (and less confusing). No functional impact, so I'll commit soon unless someone objects. If I'm reading this right, this creates two different variables pointing at

Re: [HACKERS] PATCH: decreasing memory needlessly consumed by array_agg

2015-02-21 Thread Jeff Davis
On Wed, 2015-01-28 at 23:25 +0100, Tomas Vondra wrote: 3) moves the assert into the 'if (release)' branch You missed one, but I got it. 4) includes the comments proposed by Ali Akbar in his reviews Warnings at makeArrayResult/makeMdArrayResult about freeing memory with private

Re: [HACKERS] Replication identifiers, take 4

2015-02-21 Thread Petr Jelinek
Now that the issue with padding seems to no longer exists since the patch works both with and without padding, I went through the code and here are some comments I have (in no particular order). In CheckPointReplicationIdentifier: + * FIXME: Add a CRC32 to the end. The function already does

Re: [HACKERS] Bootstrap DATA is a pita

2015-02-21 Thread Andres Freund
On 2015-02-20 22:19:54 -0500, Peter Eisentraut wrote: On 2/20/15 8:46 PM, Josh Berkus wrote: What about YAML? That might have been added somewhat earlier. YAML isn't included in Perl, but there is Module::Build::YAML - Provides just enough YAML support so that Module::Build works even if

Re: [HACKERS] Bootstrap DATA is a pita

2015-02-21 Thread Andrew Dunstan
On 02/21/2015 09:39 AM, Andrew Dunstan wrote: On 02/21/2015 05:04 AM, Andres Freund wrote: Yes, that's a good point. I have zero desire to open-code a format though, I think that's a bad idea. We could say we just include Yaml::Tiny, that's what it's made for. Personally, I think I would

Re: [HACKERS] Abbreviated keys for Numeric

2015-02-21 Thread Tomas Vondra
Hi Gavin, On 21.2.2015 06:35, Gavin Flower wrote: On 21/02/15 18:18, Tomas Vondra wrote: OK, so I've repeated the benchmarks with both patches applied, and I think the results are interesting. I extended the benchmark a bit - see the SQL script attached. 1) multiple queries

Re: [HACKERS] deparsing utility commands

2015-02-21 Thread Andres Freund
Hi, On 2015-02-15 01:48:15 -0300, Alvaro Herrera wrote: One line of defense which I just tought about is that instead of sprinkling EventTriggerStashCommand() calls all over ProcessUtilitySlow, we should add only one at the bottom. Doesn't sound like a bad idea, but I'm not sure whether it's

Re: [HACKERS] Bootstrap DATA is a pita

2015-02-21 Thread Andres Freund
On 2015-02-21 11:34:09 -0500, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: On 2015-02-20 22:19:54 -0500, Peter Eisentraut wrote: On 2/20/15 8:46 PM, Josh Berkus wrote: Or what about just doing CSV? I don't think that would actually address the problems. It would just

Re: [HACKERS] Bootstrap DATA is a pita

2015-02-21 Thread Tom Lane
Andrew Dunstan and...@dunslane.net writes: On 02/21/2015 09:39 AM, Andrew Dunstan wrote: Personally, I think I would prefer that we use JSON (and yes, there's a JSON::Tiny module, which definitely lives up to its name). For one thing, we've made a feature of supporting JSON, so arguably we

Re: [HACKERS] Query Rewrite with Postgres' materialized views

2015-02-21 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: I'm not really sure what Josh was talking about in that answer. In terms of doing this automatically, I doubt that's likely to happen until we have a way to automatically update a materialized view when the underlying data changes --- and Kevin

Re: [HACKERS] Bootstrap DATA is a pita

2015-02-21 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: On 2015-02-20 22:19:54 -0500, Peter Eisentraut wrote: On 2/20/15 8:46 PM, Josh Berkus wrote: Or what about just doing CSV? I don't think that would actually address the problems. It would just be the same format as now with different delimiters.

Re: [HACKERS] Query Rewrite with Postgres' materialized views

2015-02-21 Thread Robert Haas
On Sun, Feb 15, 2015 at 10:05 PM, Eric Grinstein ericgrinst...@gmail.com wrote: Are there any plans for implementing query rewriting (i.e. allowing the optimizer to substitute materialized views for queries) in upcoming versions? I have recently seen this answer, saying that query rewriting

Re: [HACKERS] Query Rewrite with Postgres' materialized views

2015-02-21 Thread Robert Haas
On Sat, Feb 21, 2015 at 1:04 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: I'm not really sure what Josh was talking about in that answer. In terms of doing this automatically, I doubt that's likely to happen until we have a way to automatically update a

Re: [HACKERS] Bootstrap DATA is a pita

2015-02-21 Thread Andrew Dunstan
On 02/21/2015 11:43 AM, Tom Lane wrote: Andrew Dunstan and...@dunslane.net writes: On 02/21/2015 09:39 AM, Andrew Dunstan wrote: Personally, I think I would prefer that we use JSON (and yes, there's a JSON::Tiny module, which definitely lives up to its name). For one thing, we've made a

Re: [HACKERS] Query Rewrite with Postgres' materialized views

2015-02-21 Thread Kevin Grittner
Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: I'm not really sure what Josh was talking about in that answer. In terms of doing this automatically, I doubt that's likely to happen until we have a way to automatically update a materialized view when the

Re: [HACKERS] deparsing utility commands

2015-02-21 Thread Andres Freund
On 2015-02-19 14:39:27 -0300, Alvaro Herrera wrote: diff --git a/src/backend/catalog/objectaddress.c b/src/backend/catalog/objectaddress.c index d899dd7..2bbc15d 100644 --- a/src/backend/catalog/objectaddress.c +++ b/src/backend/catalog/objectaddress.c @@ -531,6 +531,12 @@ ObjectTypeMap[] =

Re: [HACKERS] deparsing utility commands

2015-02-21 Thread Andres Freund
On 2015-02-19 17:14:57 -0300, Alvaro Herrera wrote: The ddl_command_start event occurs just before the execution of a CREATE, ALTER, DROP, SECURITY LABEL, COMMENT, GRANT or REVOKE command. No check whether the affected object exists or doesn't exist is performed before

Re: [HACKERS] Expanding the use of FLEXIBLE_ARRAY_MEMBER for declarations like foo[1]

2015-02-21 Thread Michael Paquier
On Fri, Feb 20, 2015 at 8:57 PM, Michael Paquier michael.paqu...@gmail.com wrote: Attached is a new series. 0001 and 0002 are the same, 0003 and 0004 the backend structures listed previously. I noticed as well that indexed_tlist in setrefs.c meritates some attention. And after all those

Re: [HACKERS] Bootstrap DATA is a pita

2015-02-21 Thread Andrew Dunstan
On 02/21/2015 05:04 AM, Andres Freund wrote: Yes, that's a good point. I have zero desire to open-code a format though, I think that's a bad idea. We could say we just include Yaml::Tiny, that's what it's made for. Personally, I think I would prefer that we use JSON (and yes, there's a

Re: [HACKERS] Add min and max execute statement time in pg_stat_statement

2015-02-21 Thread Andrew Dunstan
On 02/16/2015 09:05 PM, Petr Jelinek wrote: On 17/02/15 02:57, Andrew Dunstan wrote: On 02/16/2015 08:48 PM, Petr Jelinek wrote: On 17/02/15 01:57, Peter Geoghegan wrote: On Mon, Feb 16, 2015 at 4:44 PM, Petr Jelinek p...@2ndquadrant.com wrote: We definitely want this feature, I wished to

Re: [HACKERS] Expanding the use of FLEXIBLE_ARRAY_MEMBER for declarations like foo[1]

2015-02-21 Thread Andres Freund
On 2015-02-21 15:16:55 -0500, Tom Lane wrote: Andres, would you double-check the changes in reorderbuffer.c? There were some weird calculations with offsetof(ReorderBufferTupleBuf, data) - offsetof(HeapTupleHeaderData, t_bits) which Michael simplified in a way that's not 100% equivalent. I

Re: [HACKERS] NOT NULL markings for BKI columns

2015-02-21 Thread Andres Freund
On 2015-02-20 19:06:50 -0300, Alvaro Herrera wrote: Andres Freund wrote: On 2015-02-15 12:54:45 -0500, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: BTW, the precedent of PG_USED_FOR_ASSERTS_ONLY would suggest calling this one PG_FORCE_NOT_NULL, or at least using

Re: [HACKERS] NOT NULL markings for BKI columns

2015-02-21 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: On 2015-02-20 19:06:50 -0300, Alvaro Herrera wrote: I think we should just save this part of the patch until some use turns up. I pondered this for a while and I don't agree. If the flag had been available a couple column that now use 0 instead of

Re: [HACKERS] deparsing utility commands

2015-02-21 Thread Andres Freund
On 2015-02-21 14:51:32 -0500, Stephen Frost wrote: It'd be *really* nice to be able to pass an object identifier to some function and get back the CREATE (in particular, though perhaps DROP, or whatever) command for it. This gets us *awful* close to that without actually giving it to us and

Re: [HACKERS] Abbreviated keys for Numeric

2015-02-21 Thread Peter Geoghegan
On Fri, Feb 20, 2015 at 9:18 PM, Tomas Vondra tomas.von...@2ndquadrant.com wrote: The gains for text are also very nice, although in this case that only happens for the smallest scale (1M rows), and for larger scales it's actually slower than current master :-( That's odd. I have a hard time

Re: [HACKERS] Abbreviated keys for Numeric

2015-02-21 Thread Peter Geoghegan
On Sat, Feb 21, 2015 at 10:57 AM, Peter Geoghegan p...@heroku.com wrote: On Fri, Feb 20, 2015 at 9:18 PM, Tomas Vondra tomas.von...@2ndquadrant.com wrote: The gains for text are also very nice, although in this case that only happens for the smallest scale (1M rows), and for larger scales it's

Re: [HACKERS] deparsing utility commands

2015-02-21 Thread Stephen Frost
Alvaro, all, * Alvaro Herrera (alvhe...@2ndquadrant.com) wrote: This is a repost of the patch to add CREATE command deparsing support to event triggers. It now supports not only CREATE but also ALTER and other commands such as GRANT/REVOKE, COMMENT ON and SECURITY LABEL. This patch series is

Re: [HACKERS] Abbreviated keys for Numeric

2015-02-21 Thread Tomas Vondra
On 21.2.2015 19:57, Peter Geoghegan wrote: On Fri, Feb 20, 2015 at 9:18 PM, Tomas Vondra tomas.von...@2ndquadrant.com wrote: The gains for text are also very nice, although in this case that only happens for the smallest scale (1M rows), and for larger scales it's actually slower than current

Re: [HACKERS] Abbreviated keys for Numeric

2015-02-21 Thread Tomas Vondra
On 21.2.2015 20:33, Peter Geoghegan wrote: On Sat, Feb 21, 2015 at 10:57 AM, Peter Geoghegan p...@heroku.com That's odd. I have a hard time thinking of why the datum sort patch could be at fault, though. Oh, wait. For queries like this, which I now see in your spreadsheet: select *

Re: [HACKERS] Expanding the use of FLEXIBLE_ARRAY_MEMBER for declarations like foo[1]

2015-02-21 Thread Tom Lane
Michael Paquier michael.paqu...@gmail.com writes: And after all those commits attached is a patch changing HeapTupleHeaderData, using the following macro to track the size of the structure: #define SizeofHeapTupleHeader offsetof(HeapTupleHeaderData, t_bits) I've pushed this with a few minor

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE} 2.0

2015-02-21 Thread Heikki Linnakangas
On 02/21/2015 12:15 AM, Peter Geoghegan wrote: On Fri, Feb 20, 2015 at 1:07 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: Then I refuse to believe that the livelock hazard exists, without the pre-check. If you have a livelock scenario in mind, it really shouldn't be that difficult to

Re: [HACKERS] Bootstrap DATA is a pita

2015-02-21 Thread Andres Freund
On February 21, 2015 7:20:04 PM CET, Andrew Dunstan and...@dunslane.net wrote: On 02/21/2015 11:43 AM, Tom Lane wrote: { oid = 2249, oiddefine = 'CSTRINGOID', typname = 'cstring', typlen = -2, typbyval = 1, ... } which is pure

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE} 2.0

2015-02-21 Thread Peter Geoghegan
On Sat, Feb 21, 2015 at 11:15 AM, Heikki Linnakangas hlinnakan...@vmware.com wrote: Ah, ok, I can see the confusion now. Cool. Do we not wait on anything, and just declare that we're done? Then I think that breaks exclusion constraint enforcement, because we need to rescan the index to do