Re: [HACKERS] Abbreviated keys for Numeric

2015-02-22 Thread Andrew Gierth
Gavin == Gavin Flower gavinflo...@archidevsys.co.nz writes: Gavin What are the standard deviations? Gavin Do the arithmetic means change much if you exclude the 2 fastest Gavin 2 slowest? Gavin How do the arithmetic means compare to their respective medians? Gavin Essentially, how

Re: [HACKERS] proposal: searching in array function - array_position

2015-02-22 Thread Pavel Stehule
2015-02-22 3:00 GMT+01:00 Petr Jelinek p...@2ndquadrant.com: On 28/01/15 08:15, Pavel Stehule wrote: 2015-01-28 0:01 GMT+01:00 Jim Nasby jim.na...@bluetreble.com mailto:jim.na...@bluetreble.com: On 1/27/15 4:36 AM, Pavel Stehule wrote: It is only partially identical - I

Re: [HACKERS] SSL renegotiation

2015-02-22 Thread Andres Freund
On 2015-02-22 01:27:54 +0100, Emil Lenngren wrote: I honestly wonder why postgres uses renegotiation at all. The motivation that cryptoanalysis is easier as more data is sent seems quite far-fetched. I don't think so. There's a fair number of algorithms that can/could be much easier be

Re: [HACKERS] Replication identifiers, take 4

2015-02-22 Thread Petr Jelinek
On 22/02/15 09:57, Andres Freund wrote: On 2015-02-19 00:49:50 +0100, Petr Jelinek wrote: On 16/02/15 10:46, Andres Freund wrote: On 2015-02-16 11:34:10 +0200, Heikki Linnakangas wrote: At a quick glance, this basic design seems workable. I would suggest expanding the replication IDs to

Re: [HACKERS] Redesigning checkpoint_segments

2015-02-22 Thread Venkata Balaji N
I am wondering a bit about interaction with wal_keep_segments. One thing is that wal_keep_segments is still specified in number of segments and not size units, maybe it would be worth to change it also? And the other thing is that, if set, the wal_keep_segments is the real max_wal_size from

Re: [HACKERS] hash agg is slower on wide tables?

2015-02-22 Thread Andrew Gierth
Pavel == Pavel Stehule pavel.steh...@gmail.com writes: Pavel why we read all columns from t1? [...] Pavel so it looks so hashagg doesn't eliminate source columns well I don't think it's supposed to eliminate them. This is, if I'm understanding the planner logic right, physical-tlist

Re: [HACKERS] hash agg is slower on wide tables?

2015-02-22 Thread Andres Freund
On 2015-02-22 10:33:16 +, Andrew Gierth wrote: This is, if I'm understanding the planner logic right, physical-tlist optimization; it's faster for a table scan to simply return the whole row (copying nothing, just pointing to the on-disk tuple) and let hashagg pick out the columns it

Re: [HACKERS] Enforce creation of destination folders for source files in pg_regress (Was: pg_regress writes into source tree)

2015-02-22 Thread Michael Paquier
On Sat, Feb 21, 2015 at 6:51 AM, Peter Eisentraut pete...@gmx.net wrote: On 2/20/15 1:56 AM, Michael Paquier wrote: We'd still need the .gitignore files somewhere. Do you want to move them one directory up? I am not sure I am getting what you are pointing to... For extensions that already

Re: [HACKERS] hash agg is slower on wide tables?

2015-02-22 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: I've wondered before if we shouldn't use the caching via slot-tts_values so freely - if you only use a couple values from a wide tuple the current implementation really sucks if those few aren't at the beginning of the tuple. Don't see how you

Re: [HACKERS] Allow snapshot too old error, to prevent bloat

2015-02-22 Thread Amit Kapila
On Mon, Feb 16, 2015 at 10:49 AM, Kevin Grittner kgri...@ymail.com wrote: What this discussion has made me reconsider is the metric for considering a transaction too old. The number of transaction IDs consumed seems inferior as the unit of measure for that to LSN or time. It looks to me to

Re: [HACKERS] Replication identifiers, take 4

2015-02-22 Thread Peter Eisentraut
On 2/15/15 7:24 PM, Andres Freund wrote: On 2015-02-16 01:21:55 +0100, Andres Freund wrote: Here's my next attept attempt at producing something we can agree upon. The major change that might achieve that is that I've now provided a separate method to store the origin_id of a node. I've made

Re: [HACKERS] Allow snapshot too old error, to prevent bloat

2015-02-22 Thread Kevin Grittner
Amit Kapila amit.kapil...@gmail.com wrote: It seems to me that SQL Server also uses similar mechanism to avoid the bloat in version store (place to store previous versions or record). I think if other leading databases provide a way to control the bloat, it indicates that most of the

Re: [HACKERS] 9.5: Better memory accounting, towards memory-bounded HashAgg

2015-02-22 Thread Tomas Vondra
On 22.2.2015 09:14, Jeff Davis wrote: On Wed, 2015-01-07 at 20:07 +0100, Tomas Vondra wrote: So I started digging in the code and I noticed this: hash_mem = MemoryContextMemAllocated(aggstate-hashcontext, true); which is IMHO obviously wrong, because that accounts only for the

Re: [HACKERS] hash agg is slower on wide tables?

2015-02-22 Thread Andres Freund
On 2015-02-22 09:58:31 -0500, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: I've wondered before if we shouldn't use the caching via slot-tts_values so freely - if you only use a couple values from a wide tuple the current implementation really sucks if those few aren't at

Re: [HACKERS] Replication identifiers, take 4

2015-02-22 Thread Andres Freund
On 2015-02-22 10:03:59 -0500, Peter Eisentraut wrote: On 2/15/15 7:24 PM, Andres Freund wrote: On 2015-02-16 01:21:55 +0100, Andres Freund wrote: Here's my next attept attempt at producing something we can agree upon. The major change that might achieve that is that I've now provided a

Re: [HACKERS] hash agg is slower on wide tables?

2015-02-22 Thread Pavel Stehule
2015-02-22 13:22 GMT+01:00 Andres Freund and...@2ndquadrant.com: On 2015-02-22 10:33:16 +, Andrew Gierth wrote: This is, if I'm understanding the planner logic right, physical-tlist optimization; it's faster for a table scan to simply return the whole row (copying nothing, just

Re: [HACKERS] Precedence of standard comparison operators

2015-02-22 Thread Tom Lane
I wrote: Attached is a draft patch to bring the precedence of comparison operators and IS tests into line with the SQL standard. I have not yet looked into producing warnings for changes in parsing decisions ... I've made some progress on getting parse_expr.c to produce warnings by

Re: [HACKERS] Abbreviated keys for Numeric

2015-02-22 Thread Gavin Flower
On 22/02/15 22:59, Andrew Gierth wrote: Gavin == Gavin Flower gavinflo...@archidevsys.co.nz writes: Gavin What are the standard deviations? Gavin Do the arithmetic means change much if you exclude the 2 fastest Gavin 2 slowest? Gavin How do the arithmetic means compare to their

Re: [HACKERS] deparsing utility commands

2015-02-22 Thread Andres Freund
On 2015-02-21 17:30:24 +0100, Andres Freund wrote: /* + * deparse_CreateFunctionStmt + * Deparse a CreateFunctionStmt (CREATE FUNCTION) + * + * Given a function OID and the parsetree that created it, return the JSON + * blob representing the creation command. + * + * XXX

Re: [HACKERS] Abbreviated keys for Numeric

2015-02-22 Thread Tomas Vondra
On 22.2.2015 10:59, Andrew Gierth wrote: Gavin == Gavin Flower gavinflo...@archidevsys.co.nz writes: Gavin Essentially, how consistent are the results, or how great is the Gavin noise? There may be better indicators than the ones I've Gavin suggested above. This is all rather missing

Re: [HACKERS] Enforce creation of destination folders for source files in pg_regress (Was: pg_regress writes into source tree)

2015-02-22 Thread Peter Eisentraut
On 2/22/15 5:41 AM, Michael Paquier wrote: You could argue that these .gitignore files don't actually belong there, but your patch doesn't change or move those files, and even modules that have non-empty sql/ or expected/ directories have .gitignore files there, so it is considered the

Re: [HACKERS] Replication identifiers, take 4

2015-02-22 Thread Andres Freund
On 2015-02-22 04:59:30 +0100, Petr Jelinek wrote: Now that the issue with padding seems to no longer exists since the patch works both with and without padding, I went through the code and here are some comments I have (in no particular order). In CheckPointReplicationIdentifier: + * FIXME:

[HACKERS] hash agg is slower on wide tables?

2015-02-22 Thread Pavel Stehule
Hi I did some benchmarks and I found some strange numbers. do $$ begin drop table if exists t1; execute 'create table t1(' || array_to_string(array(select 'a' || i || ' smallint' from generate_series(1,30) g(i)), ',') || ')'; -- special column a2, a11 insert into t1 select 2008,

Re: [HACKERS] Replication identifiers, take 4

2015-02-22 Thread Andres Freund
On 2015-02-19 00:49:50 +0100, Petr Jelinek wrote: On 16/02/15 10:46, Andres Freund wrote: On 2015-02-16 11:34:10 +0200, Heikki Linnakangas wrote: At a quick glance, this basic design seems workable. I would suggest expanding the replication IDs to regular 4 byte oids. Two extra bytes is a

Re: [HACKERS] 9.5: Better memory accounting, towards memory-bounded HashAgg

2015-02-22 Thread Jeff Davis
On Wed, 2015-01-07 at 20:07 +0100, Tomas Vondra wrote: So I started digging in the code and I noticed this: hash_mem = MemoryContextMemAllocated(aggstate-hashcontext, true); which is IMHO obviously wrong, because that accounts only for the hashtable itself. It might be correct for

Re: [HACKERS] hash agg is slower on wide tables?

2015-02-22 Thread Pavel Stehule
2015-02-22 9:28 GMT+01:00 Pavel Stehule pavel.steh...@gmail.com: Hi I did some benchmarks and I found some strange numbers. do $$ begin drop table if exists t1; execute 'create table t1(' || array_to_string(array(select 'a' || i || ' smallint' from generate_series(1,30) g(i)),

Re: [HACKERS] Abbreviated keys for text cost model fix

2015-02-22 Thread Peter Geoghegan
On Sun, Feb 22, 2015 at 1:30 PM, Peter Geoghegan p...@heroku.com wrote: You should try it with the data fully sorted like this, but with one tiny difference: The very last tuple is out of order. How does that look? Another thing that may be of particular interest to you as a Czech person is

Re: [HACKERS] New CF app deployment

2015-02-22 Thread Tom Lane
Magnus Hagander mag...@hagander.net writes: On Sun, Feb 15, 2015 at 9:46 PM, Peter Geoghegan p...@heroku.com wrote: also think that it's a waste of screen space to show who within the annotation view. Granted, the old app supported this, but I tend to think that if I actually cared who added a

Re: [HACKERS] Abbreviated keys for text cost model fix

2015-02-22 Thread Peter Geoghegan
On Sun, Feb 22, 2015 at 1:19 PM, Tomas Vondra tomas.von...@2ndquadrant.com wrote: In short, this fixes all the cases except for the ASC sorted data. I haven't done any code review, but I think we want this. I'll use data from the i5-2500k, but it applies to the Xeon too, except that the Xeon

Re: [HACKERS] New CF app deployment

2015-02-22 Thread Magnus Hagander
On Sun, Feb 15, 2015 at 7:59 PM, Peter Eisentraut pete...@gmx.net wrote: On 2/14/15 7:30 AM, Magnus Hagander wrote: On Mon, Feb 9, 2015 at 4:56 PM, Robert Haas robertmh...@gmail.com Can we make it smarter, so that the kinds of things people produce intending for them to be patches

Re: [HACKERS] New CF app deployment

2015-02-22 Thread Magnus Hagander
On Sun, Feb 15, 2015 at 9:46 PM, Peter Geoghegan p...@heroku.com wrote: On Sun, Feb 15, 2015 at 4:59 PM, Peter Eisentraut pete...@gmx.net wrote: I think the old system where the patch submitter declared, this message contains my patch, is the only one that will work. I tend to agree. That

Re: [HACKERS] Abbreviated keys for text cost model fix

2015-02-22 Thread Tomas Vondra
On 23.2.2015 00:16, Peter Geoghegan wrote: On Sun, Feb 22, 2015 at 1:30 PM, Peter Geoghegan p...@heroku.com wrote: You should try it with the data fully sorted like this, but with one tiny difference: The very last tuple is out of order. How does that look? I'm running that test now, I'll

Re: [HACKERS] Redesigning checkpoint_segments

2015-02-22 Thread Venkata Balaji N
On Sat, Feb 14, 2015 at 4:43 AM, Heikki Linnakangas hlinnakan...@vmware.com wrote: On 02/04/2015 11:41 PM, Josh Berkus wrote: On 02/04/2015 12:06 PM, Robert Haas wrote: On Wed, Feb 4, 2015 at 1:05 PM, Josh Berkus j...@agliodbs.com wrote: Let me push max_wal_size and min_wal_size again as

Re: [HACKERS] Enforce creation of destination folders for source files in pg_regress (Was: pg_regress writes into source tree)

2015-02-22 Thread Michael Paquier
On Mon, Feb 23, 2015 at 12:00 AM, Peter Eisentraut pete...@gmx.net wrote: On 2/22/15 5:41 AM, Michael Paquier wrote: You could argue that these .gitignore files don't actually belong there, but your patch doesn't change or move those files, and even modules that have non-empty sql/ or

Re: [HACKERS] multiple backends attempting to wait for pincount 1

2015-02-22 Thread Stefan Kaltenbrunner
On 02/13/2015 06:27 AM, Tom Lane wrote: Two different CLOBBER_CACHE_ALWAYS critters recently reported exactly the same failure pattern on HEAD: http://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=markhordt=2015-02-06%2011%3A59%3A59

Re: [HACKERS] dblink: add polymorphic functions.

2015-02-22 Thread Corey Huinker
Changes in this patch: - added polymorphic versions of dblink_fetch() - upped dblink version # to 1.2 because of new functions - migration 1.1 - 1.2 - DocBook changes for dblink(), dblink_get_result(), dblink_fetch() On Sun, Feb 22, 2015 at 11:38 PM, Corey Huinker corey.huin...@gmail.com wrote:

Re: [HACKERS] Expanding the use of FLEXIBLE_ARRAY_MEMBER for declarations like foo[1]

2015-02-22 Thread Michael Paquier
On Sun, Feb 22, 2015 at 6:57 AM, Tom Lane t...@sss.pgh.pa.us wrote: After some more hacking, the only remaining uses of foo[1] in struct declarations are: 1. A couple of places where the array is actually the only struct member; for some unexplainable reason gcc won't let you use flexible

Re: [HACKERS] pg_check_dir comments and implementation mismatch

2015-02-22 Thread Robert Haas
On Fri, Feb 20, 2015 at 12:59 AM, Noah Misch n...@leadboat.com wrote: On Mon, Feb 02, 2015 at 03:48:33PM -0500, Robert Haas wrote: On Sat, Jan 31, 2015 at 8:28 AM, Marco Nenciarini marco.nenciar...@2ndquadrant.it wrote: I've attached a new version of the patch fixing the missing closedir on

Re: [HACKERS] pg_check_dir comments and implementation mismatch

2015-02-22 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Fri, Feb 20, 2015 at 12:59 AM, Noah Misch n...@leadboat.com wrote: On Mon, Feb 02, 2015 at 03:48:33PM -0500, Robert Haas wrote: If readir() fails and closedir() succeeds, the return will be -1 but errno will be 0. Out of curiosity, have you seen a

Re: [HACKERS] restrict global access to be readonly

2015-02-22 Thread Robert Haas
On Tue, Feb 17, 2015 at 4:40 AM, happy times guangzhouzh...@qq.com wrote: The first choice Tom pointed makes sense to me: adding this as eqivalent to setting all subsequent transactions as read only. It is useful enough in the scenarios where disk limit for the instance is reached, we want to

Re: [HACKERS] dblink: add polymorphic functions.

2015-02-22 Thread Corey Huinker
Yes, that was it, I discovered it myself and should have posted a nevermind. Now I'm slogging through figuring out where to find elog() messages from the temporary server. It's slow, but it's progress. On Sun, Feb 22, 2015 at 10:39 PM, Michael Paquier michael.paqu...@gmail.com wrote: On Mon,

Re: [HACKERS] Precedence of standard comparison operators

2015-02-22 Thread Tom Lane
Attached is an improved patch that includes optional warnings for constructs that changed parsing. It's not quite 100% but I think it's about 90% correct; the difference in size between this and the previous patch should be a pretty fair indication of what it's going to cost us to have a warning

Re: [HACKERS] a fast bloat measurement tool (was Re: Measuring relation free space)

2015-02-22 Thread Tomas Vondra
Hi! On 28.1.2015 05:03, Abhijit Menon-Sen wrote: At 2015-01-27 17:00:27 -0600, jim.na...@bluetreble.com wrote: It would be best to get this into a commit fest so it's not lost. It's there already. -- Abhijit I looked at this patch today, so a few comments from me: 1) I believe the

Re: [HACKERS] Redesigning checkpoint_segments

2015-02-22 Thread Robert Haas
On Sat, Feb 21, 2015 at 11:29 PM, Petr Jelinek p...@2ndquadrant.com wrote: I am wondering a bit about interaction with wal_keep_segments. One thing is that wal_keep_segments is still specified in number of segments and not size units, maybe it would be worth to change it also? And the other

Re: [HACKERS] dblink: add polymorphic functions.

2015-02-22 Thread Corey Huinker
I seem to be getting tripped up in the regression test. This line was found in regression.diff + ERROR: could not stat file /home/ubuntu/src/postgres/contrib/dblink/tmp_check/install/usr/local/pgsql/share/extension/dblink--1.2.sql: No such file or directory The file dblink--1.2.sql does exist

Re: [HACKERS] dblink: add polymorphic functions.

2015-02-22 Thread Corey Huinker
nevermind. Found it. On Sun, Feb 22, 2015 at 11:18 PM, Corey Huinker corey.huin...@gmail.com wrote: Yes, that was it, I discovered it myself and should have posted a nevermind. Now I'm slogging through figuring out where to find elog() messages from the temporary server. It's slow, but it's

Re: [HACKERS] Allow snapshot too old error, to prevent bloat

2015-02-22 Thread Andrew Dunstan
On 02/22/2015 11:48 AM, Kevin Grittner wrote: (2) Use a course enough granularity on time and a short enough maximum for the GUC to just keep a circular buffer of the mappings in memory. We might be able to make this dense enough that one minute resolution for up to 60 days could fit in

Re: [HACKERS] TABLESAMPLE patch

2015-02-22 Thread Tomas Vondra
Hi, On 22.2.2015 18:57, Petr Jelinek wrote: Tomas noticed that the patch is missing error check when TABLESAMPLE is used on view, so here is a new version that checks it's only used against table or matview. No other changes. Curious question - could/should this use page prefetch, similar

Re: [HACKERS] a fast bloat measurement tool (was Re: Measuring relation free space)

2015-02-22 Thread Tomas Vondra
On 23.2.2015 03:20, Jim Nasby wrote: On 2/22/15 5:41 PM, Tomas Vondra wrote: Otherwise, the code looks OK to me. Now, there are a few features I'd like to have for production use (to minimize the impact): 1) no index support:-( I'd like to see support for more relation types (at least

Re: [HACKERS] __attribute__ for non-gcc compilers

2015-02-22 Thread Robert Haas
On Tue, Feb 17, 2015 at 8:41 AM, Oskari Saarenmaa o...@ohmu.fi wrote: 15.01.2015, 21:58, Robert Haas kirjoitti: On Wed, Jan 14, 2015 at 5:54 PM, Andres Freund and...@2ndquadrant.com wrote: I think I'd for now simply not define pg_attribute_aligned() on platforms where it's not supported,

Re: [HACKERS] Redesigning checkpoint_segments

2015-02-22 Thread Petr Jelinek
On 23/02/15 03:24, Robert Haas wrote: On Sat, Feb 21, 2015 at 11:29 PM, Petr Jelinek p...@2ndquadrant.com wrote: I am wondering a bit about interaction with wal_keep_segments. One thing is that wal_keep_segments is still specified in number of segments and not size units, maybe it would be

Re: [HACKERS] dblink: add polymorphic functions.

2015-02-22 Thread Michael Paquier
On Mon, Feb 23, 2015 at 12:03 PM, Corey Huinker corey.huin...@gmail.com wrote: + ERROR: could not stat file /home/ubuntu/src/postgres/contrib/dblink/tmp_check/install/usr/local/pgsql/share/extension/dblink--1.2.sql: No such file or directory Didn't you forget to add dblink--1.2.sql to DATA in

Re: [HACKERS] a fast bloat measurement tool (was Re: Measuring relation free space)

2015-02-22 Thread Jim Nasby
On 2/22/15 5:41 PM, Tomas Vondra wrote: Otherwise, the code looks OK to me. Now, there are a few features I'd like to have for production use (to minimize the impact): 1) no index support:-( I'd like to see support for more relation types (at least btree indexes). Are there any plans