[HACKERS] Allowing join removals for more join types

2014-05-17 Thread David Rowley
I'm currently in the early stages of looking into expanding join removals. Currently left outer joins can be removed if none of the columns of the table are required for anything and the table being joined is a base table that contains a unique index on all columns in the join clause. The case I

Re: [HACKERS] Allowing join removals for more join types

2014-05-17 Thread David Rowley
On Sat, May 17, 2014 at 8:57 PM, David Rowley dgrowle...@gmail.com wrote: I'm currently in the early stages of looking into expanding join removals. As I said above, I'm in the early stages of looking at this and I'm currently a bit confused. Basically I've put a breakpoint at the top of the

Re: [HACKERS] 9.4 checksum error in recovery with btree index

2014-05-17 Thread Heikki Linnakangas
On 05/17/2014 12:28 AM, Jeff Janes wrote: More fun with my torn page injection test program on 9.4. 24171 2014-05-16 14:00:44.934 PDT:WARNING: 01000: page verification failed, calculated checksum 21100 but expected 3356 24171 2014-05-16 14:00:44.934 PDT:CONTEXT: xlog redo split_l: rel

Re: [HACKERS] Allowing join removals for more join types

2014-05-17 Thread Tom Lane
David Rowley dgrowle...@gmail.com writes: It looks like the existing join removals are done quite early in the planning and redundant joins are removed before any subqueries from that query are planned. So this innerrel-subroot-parse has not been done yet. It seems to be done later in

Re: [HACKERS] wrapping in extended mode doesn't work well with default pager

2014-05-17 Thread Greg Stark
I'm trying to review all the combinations of the options exhaustively but in the process I noticed a few pre-existing psql oddities. Both of these are present in 9.3: Can anyone explain this? It's linestyle=old-style, border=1, expanded=off, format=aligned. It looks like it's using new-style

Re: [HACKERS] %d in log_line_prefix doesn't work for bg/autovacuum workers

2014-05-17 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: On 2014-05-17 00:17:46 +0200, Andres Freund wrote: On 2014-05-16 17:48:28 -0400, Tom Lane wrote: This is basically reintroducing a variable that used to exist and was intentionally gotten rid of, on the grounds that it'd fail to track any renaming

Re: [HACKERS] wrapping in extended mode doesn't work well with default pager

2014-05-17 Thread Greg Stark
Sorry, a couple things still look to not be quite right. 1) The width of the table when linestyle=old-ascii and border=0 or border=1 (and expanded=on and format=wrapped) seems to off by one. 2) The hyphens following the RECORD NN are short by one I'm surprised the last patch was so big since it

[HACKERS] buildfarm: strange OOM failures on markhor (running CLOBBER_CACHE_RECURSIVELY)

2014-05-17 Thread Tomas Vondra
Hi all, a few days ago I setup an buildfarm animal markhor, running the tests with CLOBBER_CACHE_RECURSIVELY. As the tests are running very long, reporting the results back to the server fails because of a safeguard limit in the buildfarm server. Anyway, that's being discussed in a different

Re: [HACKERS] buildfarm: strange OOM failures on markhor (running CLOBBER_CACHE_RECURSIVELY)

2014-05-17 Thread Tom Lane
Tomas Vondra t...@fuzzy.cz writes: ... then of course the usual 'terminating connection because of crash of another server process' warning. Apparently, it's getting killed by the OOM killer, because it exhausts all the memory assigned to that VM (2GB). Can you fix things so it runs into its

Re: [HACKERS] buildfarm animals and 'snapshot too old'

2014-05-17 Thread Andrew Dunstan
On 05/15/2014 07:47 PM, Tomas Vondra wrote: On 15.5.2014 22:07, Andrew Dunstan wrote: Yes, I've seen that. Frankly, a test that takes something like 500 hours is a bit crazy. Maybe. It certainly is not a test people will use during development. But if it can detect some hard-to-find errors in

Re: [HACKERS] buildfarm: strange OOM failures on markhor (running CLOBBER_CACHE_RECURSIVELY)

2014-05-17 Thread Tomas Vondra
On 17.5.2014 19:55, Tom Lane wrote: Tomas Vondra t...@fuzzy.cz writes: ... then of course the usual 'terminating connection because of crash of another server process' warning. Apparently, it's getting killed by the OOM killer, because it exhausts all the memory assigned to that VM (2GB).

Re: [HACKERS] %d in log_line_prefix doesn't work for bg/autovacuum workers

2014-05-17 Thread Andres Freund
Hi, On 2014-05-17 12:23:51 -0400, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: On 2014-05-17 00:17:46 +0200, Andres Freund wrote: On 2014-05-16 17:48:28 -0400, Tom Lane wrote: This is basically reintroducing a variable that used to exist and was intentionally gotten rid

Re: [HACKERS] buildfarm: strange OOM failures on markhor (running CLOBBER_CACHE_RECURSIVELY)

2014-05-17 Thread Andres Freund
On 2014-05-17 20:41:37 +0200, Tomas Vondra wrote: On 17.5.2014 19:55, Tom Lane wrote: Tomas Vondra t...@fuzzy.cz writes: The tests are already running, and there are a few postgres processes: PID VIRT RES %CPUTIME+ COMMAND 11478 449m 240m 100.0 112:53.57 postgres: pgbuild

[HACKERS] [9.4] Minor SSL/ECDH related doc fixes

2014-05-17 Thread Marko Kreen
- Clarify ECDH decription in release notes. - Fix default value - it's 'prime256v1'. - List curves with good cross-platform support explicitly (NIST P-256 / P-384 / P-521). The -list_curves output is full of garbage, it's hard to know which ones make sense to use. Only those three curves are

Re: [HACKERS] btree_gist macaddr valgrind woes

2014-05-17 Thread Tom Lane
I wrote: BTW, the *real* problem with all this stuff is that the gbtreekeyNN types are declared as having int alignment, even though some of the opclasses store double-aligned types in them. I imagine it's possible to provoke bus errors on machines that are picky about alignment. The first

Re: [HACKERS] %d in log_line_prefix doesn't work for bg/autovacuum workers

2014-05-17 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: On 2014-05-17 12:23:51 -0400, Tom Lane wrote: I think the key issue comes down to this comment in RenameDatabase: * XXX Client applications probably store the current database somewhere, * so renaming it could cause confusion. On the other hand,

Re: [HACKERS] %d in log_line_prefix doesn't work for bg/autovacuum workers

2014-05-17 Thread Andres Freund
On 2014-05-17 16:23:26 -0400, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: On 2014-05-17 12:23:51 -0400, Tom Lane wrote: Given that there haven't been complaints in the past ten years about how you can't rename an active database, I'm OK personally with locking this down

Re: [HACKERS] buildfarm animals and 'snapshot too old'

2014-05-17 Thread Tomas Vondra
On 17.5.2014 19:58, Andrew Dunstan wrote: On 05/15/2014 07:47 PM, Tomas Vondra wrote: On 15.5.2014 22:07, Andrew Dunstan wrote: Yes, I've seen that. Frankly, a test that takes something like 500 hours is a bit crazy. Maybe. It certainly is not a test people will use during development. But

Re: [HACKERS] btree_gist macaddr valgrind woes

2014-05-17 Thread Heikki Linnakangas
On 05/17/2014 11:12 PM, Tom Lane wrote: I wrote: BTW, the *real* problem with all this stuff is that the gbtreekeyNN types are declared as having int alignment, even though some of the opclasses store double-aligned types in them. I imagine it's possible to provoke bus errors on machines that

Re: [HACKERS] buildfarm: strange OOM failures on markhor (running CLOBBER_CACHE_RECURSIVELY)

2014-05-17 Thread Tomas Vondra
On 17.5.2014 22:33, Tomas Vondra wrote: On 17.5.2014 21:31, Andres Freund wrote: On 2014-05-17 20:41:37 +0200, Tomas Vondra wrote: On 17.5.2014 19:55, Tom Lane wrote: Tomas Vondra t...@fuzzy.cz writes: The tests are already running, and there are a few postgres processes: PID VIRT RES

Re: [HACKERS] btree_gist macaddr valgrind woes

2014-05-17 Thread Andres Freund
On 2014-05-17 23:46:39 +0300, Heikki Linnakangas wrote: On 05/17/2014 11:12 PM, Tom Lane wrote: I wrote: BTW, the *real* problem with all this stuff is that the gbtreekeyNN types are declared as having int alignment, even though some of the opclasses store double-aligned types in them. I

Re: [HACKERS] buildfarm: strange OOM failures on markhor (running CLOBBER_CACHE_RECURSIVELY)

2014-05-17 Thread Andres Freund
Hi, On 2014-05-17 22:33:31 +0200, Tomas Vondra wrote: Anyway, the main difference between the analyze snapshot seems to be this: init: CacheMemoryContext: 67100672 total in 17 blocks; ... 350MB: CacheMemoryContext: 134209536 total in 25 blocks; ... 400MB: CacheMemoryContext:

Re: [HACKERS] %d in log_line_prefix doesn't work for bg/autovacuum workers

2014-05-17 Thread Christoph Berg
Re: Tom Lane 2014-05-17 22961.1400343...@sss.pgh.pa.us I think the key issue comes down to this comment in RenameDatabase: * XXX Client applications probably store the current database somewhere, * so renaming it could cause confusion. On the other hand, there may not * be an

Re: [HACKERS] %d in log_line_prefix doesn't work for bg/autovacuum workers

2014-05-17 Thread Christoph Berg
Re: Andres Freund 2014-05-17 20140517203404.gb4...@awork2.anarazel.de For example, if we allow renaming active databases then the subprocesses in a parallel pg_dump or pg_restore could connect to the wrong database, ie not the one the leader process is connected to. The very best-case

Re: [HACKERS] %d in log_line_prefix doesn't work for bg/autovacuum workers

2014-05-17 Thread Andres Freund
On 2014-05-17 23:10:42 +0200, Christoph Berg wrote: Given that there haven't been complaints in the past ten years about how you can't rename an active database, I'm OK personally with locking this down forever. But I wonder if anyone wants to argue the contrary. Fwiw, we ran into

Re: [HACKERS] btree_gist macaddr valgrind woes

2014-05-17 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: On 2014-05-17 23:46:39 +0300, Heikki Linnakangas wrote: I doubt there are many people using btree_gist on int8 or float8 columns. I don't think it's that unlikely. It can make a fair amount of sense to have multicolumn indexes where one columns is

[HACKERS] uuid-ossp (Re: [pgsql-packagers] Postgresapp 9.4 beta build ready)

2014-05-17 Thread Christoph Berg
[redirecting to -hackers] Re: Tom Lane 2014-05-15 31008.1400180...@sss.pgh.pa.us Sandeep Thakkar sandeep.thak...@enterprisedb.com writes: Yes, Jakob is right. On 9.4, we had to patch configure script along with uuid-ossp.c to resolve the uuid issue. I think we need to discuss this on the

Re: [HACKERS] %d in log_line_prefix doesn't work for bg/autovacuum workers

2014-05-17 Thread Christoph Berg
Re: Andres Freund 2014-05-17 20140517211930.ga10...@awork2.anarazel.de On 2014-05-17 23:10:42 +0200, Christoph Berg wrote: Given that there haven't been complaints in the past ten years about how you can't rename an active database, I'm OK personally with locking this down forever. But

Re: [HACKERS] buildfarm: strange OOM failures on markhor (running CLOBBER_CACHE_RECURSIVELY)

2014-05-17 Thread Andres Freund
On 2014-05-17 22:55:14 +0200, Tomas Vondra wrote: And another memory context stats for a session executing CREATE INDEX, while having allocated The interesting thing is there are ~11k lines that look exactly like this: pg_namespace_oid_index: 1024 total in 1 blocks; 88 free (0 chunks); 936

Re: [HACKERS] %d in log_line_prefix doesn't work for bg/autovacuum workers

2014-05-17 Thread Andres Freund
On 2014-05-17 23:35:43 +0200, Christoph Berg wrote: They want to pg_terminate_backend() the sessions on db_old anyway, but to get that point, you need to be able to rename the db. The alternative would be to disallow connections to the db (for which there is no real nice way), kill existing

Re: [HACKERS] Allowing join removals for more join types

2014-05-17 Thread David Rowley
On Sun, May 18, 2014 at 2:55 AM, Tom Lane t...@sss.pgh.pa.us wrote: David Rowley dgrowle...@gmail.com writes: It looks like the existing join removals are done quite early in the planning and redundant joins are removed before any subqueries from that query are planned. So this

Re: [HACKERS] 9.4 beta1 crash on Debian sid/i386

2014-05-17 Thread Christoph Berg
Re: Tom Lane 2014-05-14 1357.1400028...@sss.pgh.pa.us Christoph Berg c...@df7cb.de writes: Building 9.4 beta1 on Debian sid/i386 fails during the regression tests. amd64 works fine, as does i386 on the released distributions. It would appear that something is wrong with

[HACKERS] pgbench is broken on strict-C89 compilers

2014-05-17 Thread Tom Lane
I got around to trying to build PG with the HP-supplied compiler on my ancient HPUX box, something I do about once a release cycle to see if we've finally broken that trailing-edge toolchain. Things still seem to work except for this: cc: pgbench.c, line 1579: error 1521: Incorrect

Re: [HACKERS] pgbench is broken on strict-C89 compilers

2014-05-17 Thread Andres Freund
Hi, On 2014-05-17 19:15:15 -0400, Tom Lane wrote: I got around to trying to build PG with the HP-supplied compiler on my ancient HPUX box, something I do about once a release cycle to see if we've finally broken that trailing-edge toolchain. Things still seem to work except for this: cc:

Re: [HACKERS] 9.4 checksum error in recovery with btree index

2014-05-17 Thread Jeff Janes
On Saturday, May 17, 2014, Heikki Linnakangas hlinnakan...@vmware.com wrote: On 05/17/2014 12:28 AM, Jeff Janes wrote: More fun with my torn page injection test program on 9.4. 24171 2014-05-16 14:00:44.934 PDT:WARNING: 01000: page verification failed, calculated checksum 21100 but

Re: [HACKERS] 9.4 beta1 crash on Debian sid/i386

2014-05-17 Thread Tom Lane
Christoph Berg c...@df7cb.de writes: Re: Tom Lane 2014-05-14 1357.1400028...@sss.pgh.pa.us It would appear that something is wrong with check_stack_depth(), and/or getrlimit(RLIMIT_STACK) is lying to us about the available stack. ulimit -s is 8192 (kB); max_stack_depth is 2MB.

Re: [HACKERS] uuid-ossp (Re: [pgsql-packagers] Postgresapp 9.4 beta build ready)

2014-05-17 Thread Tom Lane
Christoph Berg c...@df7cb.de writes: [redirecting to -hackers] Re: Tom Lane 2014-05-15 31008.1400180...@sss.pgh.pa.us Sandeep Thakkar sandeep.thak...@enterprisedb.com writes: Yes, Jakob is right. On 9.4, we had to patch configure script along with uuid-ossp.c to resolve the uuid issue. I

Re: [HACKERS] pgbench is broken on strict-C89 compilers

2014-05-17 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: On 2014-05-17 19:15:15 -0400, Tom Lane wrote: ... It appears to me that the compiler is within its rights to refuse a nonconstant expression for an inner initializer according to C89, though I don't see any such restriction in C99. Yea, I've

Re: [HACKERS] Possible fix for occasional failures on castoroides etc

2014-05-17 Thread Tom Lane
Dave Page dp...@pgadmin.org writes: On Sat, May 3, 2014 at 8:29 PM, Andres Freund and...@2ndquadrant.com wrote: On 2012-09-17 08:23:01 -0400, Dave Page wrote: I've added MAX_CONNECTIONS=5 to both Castoroides and Protosciurus. I've just noticed (while checking whether backporting 4c8aa8b5aea

Re: [HACKERS] Possible fix for occasional failures on castoroides etc

2014-05-17 Thread Andres Freund
On 2014-05-18 01:35:04 -0400, Tom Lane wrote: Dave Page dp...@pgadmin.org writes: On Sat, May 3, 2014 at 8:29 PM, Andres Freund and...@2ndquadrant.com wrote: On 2012-09-17 08:23:01 -0400, Dave Page wrote: I've added MAX_CONNECTIONS=5 to both Castoroides and Protosciurus. I've just