[HACKERS] Allowing join removals for more join types

2014-05-17 Thread David Rowley
I'm currently in the early stages of looking into expanding join removals. Currently left outer joins can be removed if none of the columns of the table are required for anything and the table being joined is a base table that contains a unique index on all columns in the join clause. The case I

Re: [HACKERS] Allowing join removals for more join types

2014-05-17 Thread David Rowley
On Sat, May 17, 2014 at 8:57 PM, David Rowley wrote: > I'm currently in the early stages of looking into expanding join removals. > > As I said above, I'm in the early stages of looking at this and I'm > currently a bit confused. Basically I've put a breakpoint at the top of > the join_is_removab

Re: [HACKERS] 9.4 checksum error in recovery with btree index

2014-05-17 Thread Heikki Linnakangas
On 05/17/2014 12:28 AM, Jeff Janes wrote: More fun with my torn page injection test program on 9.4. 24171 2014-05-16 14:00:44.934 PDT:WARNING: 01000: page verification failed, calculated checksum 21100 but expected 3356 24171 2014-05-16 14:00:44.934 PDT:CONTEXT: xlog redo split_l: rel 1663/1

Re: [HACKERS] Allowing join removals for more join types

2014-05-17 Thread Tom Lane
David Rowley writes: > It looks like the existing join removals are done quite early in the > planning and redundant joins are removed before any subqueries from that > query are planned. So this innerrel->subroot->parse has not been done yet. > It seems to be done later in query_planner() when ma

Re: [HACKERS] wrapping in extended mode doesn't work well with default pager

2014-05-17 Thread Greg Stark
I'm trying to review all the combinations of the options exhaustively but in the process I noticed a few pre-existing psql oddities. Both of these are present in 9.3: Can anyone explain this? It's linestyle=old-style, border=1, expanded=off, format=aligned. It looks like it's using new-style ascii

Re: [HACKERS] %d in log_line_prefix doesn't work for bg/autovacuum workers

2014-05-17 Thread Tom Lane
Andres Freund writes: > On 2014-05-17 00:17:46 +0200, Andres Freund wrote: >> On 2014-05-16 17:48:28 -0400, Tom Lane wrote: >>> This is basically reintroducing a variable that used to exist and was >>> intentionally gotten rid of, on the grounds that it'd fail to track >>> any renaming of the sess

Re: [HACKERS] wrapping in extended mode doesn't work well with default pager

2014-05-17 Thread Greg Stark
Sorry, a couple things still look to not be quite right. 1) The width of the table when linestyle=old-ascii and border=0 or border=1 (and expanded=on and format=wrapped) seems to off by one. 2) The hyphens following the RECORD NN are short by one I'm surprised the last patch was so big since it

[HACKERS] buildfarm: strange OOM failures on markhor (running CLOBBER_CACHE_RECURSIVELY)

2014-05-17 Thread Tomas Vondra
Hi all, a few days ago I setup an buildfarm animal markhor, running the tests with CLOBBER_CACHE_RECURSIVELY. As the tests are running very long, reporting the results back to the server fails because of a safeguard limit in the buildfarm server. Anyway, that's being discussed in a different threa

Re: [HACKERS] buildfarm: strange OOM failures on markhor (running CLOBBER_CACHE_RECURSIVELY)

2014-05-17 Thread Tom Lane
Tomas Vondra writes: > ... then of course the usual 'terminating connection because of crash of > another server process' warning. Apparently, it's getting killed by the > OOM killer, because it exhausts all the memory assigned to that VM (2GB). Can you fix things so it runs into its process ulim

Re: [HACKERS] buildfarm animals and 'snapshot too old'

2014-05-17 Thread Andrew Dunstan
On 05/15/2014 07:47 PM, Tomas Vondra wrote: On 15.5.2014 22:07, Andrew Dunstan wrote: Yes, I've seen that. Frankly, a test that takes something like 500 hours is a bit crazy. Maybe. It certainly is not a test people will use during development. But if it can detect some hard-to-find errors in

Re: [HACKERS] buildfarm: strange OOM failures on markhor (running CLOBBER_CACHE_RECURSIVELY)

2014-05-17 Thread Tomas Vondra
On 17.5.2014 19:55, Tom Lane wrote: > Tomas Vondra writes: >> ... then of course the usual 'terminating connection because of crash of >> another server process' warning. Apparently, it's getting killed by the >> OOM killer, because it exhausts all the memory assigned to that VM (2GB). > > Can yo

Re: [HACKERS] %d in log_line_prefix doesn't work for bg/autovacuum workers

2014-05-17 Thread Andres Freund
Hi, On 2014-05-17 12:23:51 -0400, Tom Lane wrote: > Andres Freund writes: > > On 2014-05-17 00:17:46 +0200, Andres Freund wrote: > >> On 2014-05-16 17:48:28 -0400, Tom Lane wrote: > >>> This is basically reintroducing a variable that used to exist and was > >>> intentionally gotten rid of, on the

Re: [HACKERS] buildfarm: strange OOM failures on markhor (running CLOBBER_CACHE_RECURSIVELY)

2014-05-17 Thread Andres Freund
On 2014-05-17 20:41:37 +0200, Tomas Vondra wrote: > On 17.5.2014 19:55, Tom Lane wrote: > > Tomas Vondra writes: > The tests are already running, and there are a few postgres processes: > > PID VIRT RES %CPUTIME+ COMMAND > 11478 449m 240m 100.0 112:53.57 postgres: pgbuild regression [

[HACKERS] [9.4] Minor SSL/ECDH related doc fixes

2014-05-17 Thread Marko Kreen
- Clarify ECDH decription in release notes. - Fix default value - it's 'prime256v1'. - List curves with good cross-platform support explicitly (NIST P-256 / P-384 / P-521). The -list_curves output is full of garbage, it's hard to know which ones make sense to use. Only those three curves are su

Re: [HACKERS] btree_gist macaddr valgrind woes

2014-05-17 Thread Tom Lane
I wrote: > BTW, the *real* problem with all this stuff is that the gbtreekeyNN types > are declared as having int alignment, even though some of the opclasses > store double-aligned types in them. I imagine it's possible to provoke > bus errors on machines that are picky about alignment. The firs

Re: [HACKERS] %d in log_line_prefix doesn't work for bg/autovacuum workers

2014-05-17 Thread Tom Lane
Andres Freund writes: > On 2014-05-17 12:23:51 -0400, Tom Lane wrote: >> I think the key issue comes down to this comment in RenameDatabase: >> >> * XXX Client applications probably store the current database somewhere, >> * so renaming it could cause confusion. On the other hand, there may not

Re: [HACKERS] %d in log_line_prefix doesn't work for bg/autovacuum workers

2014-05-17 Thread Andres Freund
On 2014-05-17 16:23:26 -0400, Tom Lane wrote: > Andres Freund writes: > > On 2014-05-17 12:23:51 -0400, Tom Lane wrote: > >> Given that there haven't been complaints in the past ten years about how > >> you can't rename an active database, I'm OK personally with locking this > >> down forever. Bu

Re: [HACKERS] buildfarm animals and 'snapshot too old'

2014-05-17 Thread Tomas Vondra
On 17.5.2014 19:58, Andrew Dunstan wrote: > > On 05/15/2014 07:47 PM, Tomas Vondra wrote: >> On 15.5.2014 22:07, Andrew Dunstan wrote: >>> Yes, I've seen that. Frankly, a test that takes something like 500 >>> hours is a bit crazy. >> Maybe. It certainly is not a test people will use during develo

Re: [HACKERS] btree_gist macaddr valgrind woes

2014-05-17 Thread Heikki Linnakangas
On 05/17/2014 11:12 PM, Tom Lane wrote: I wrote: BTW, the *real* problem with all this stuff is that the gbtreekeyNN types are declared as having int alignment, even though some of the opclasses store double-aligned types in them. I imagine it's possible to provoke bus errors on machines that a

Re: [HACKERS] buildfarm: strange OOM failures on markhor (running CLOBBER_CACHE_RECURSIVELY)

2014-05-17 Thread Tomas Vondra
On 17.5.2014 22:33, Tomas Vondra wrote: > On 17.5.2014 21:31, Andres Freund wrote: >> On 2014-05-17 20:41:37 +0200, Tomas Vondra wrote: >>> On 17.5.2014 19:55, Tom Lane wrote: Tomas Vondra writes: >>> The tests are already running, and there are a few postgres processes: >>> >>> PID VIRT

Re: [HACKERS] btree_gist macaddr valgrind woes

2014-05-17 Thread Andres Freund
On 2014-05-17 23:46:39 +0300, Heikki Linnakangas wrote: > On 05/17/2014 11:12 PM, Tom Lane wrote: > >I wrote: > >>BTW, the *real* problem with all this stuff is that the gbtreekeyNN types > >>are declared as having int alignment, even though some of the opclasses > >>store double-aligned types in t

Re: [HACKERS] buildfarm: strange OOM failures on markhor (running CLOBBER_CACHE_RECURSIVELY)

2014-05-17 Thread Andres Freund
Hi, On 2014-05-17 22:33:31 +0200, Tomas Vondra wrote: > Anyway, the main difference between the analyze snapshot seems to be this: > > init: CacheMemoryContext: 67100672 total in 17 blocks; ... > 350MB: CacheMemoryContext: 134209536 total in 25 blocks; ... > 400MB: CacheMemoryContext: 1929

Re: [HACKERS] %d in log_line_prefix doesn't work for bg/autovacuum workers

2014-05-17 Thread Christoph Berg
Re: Tom Lane 2014-05-17 <22961.1400343...@sss.pgh.pa.us> > I think the key issue comes down to this comment in RenameDatabase: > > * XXX Client applications probably store the current database somewhere, > * so renaming it could cause confusion. On the other hand, there may not > *

Re: [HACKERS] %d in log_line_prefix doesn't work for bg/autovacuum workers

2014-05-17 Thread Christoph Berg
Re: Andres Freund 2014-05-17 <20140517203404.gb4...@awork2.anarazel.de> > > For example, if we allow > > renaming active databases then the subprocesses in a parallel pg_dump or > > pg_restore could connect to the wrong database, ie not the one the leader > > process is connected to. The very best

Re: [HACKERS] %d in log_line_prefix doesn't work for bg/autovacuum workers

2014-05-17 Thread Andres Freund
On 2014-05-17 23:10:42 +0200, Christoph Berg wrote: > > Given that there haven't been complaints in the past ten years about how > > you can't rename an active database, I'm OK personally with locking this > > down forever. But I wonder if anyone wants to argue the contrary. > > Fwiw, we ran into

Re: [HACKERS] btree_gist macaddr valgrind woes

2014-05-17 Thread Tom Lane
Andres Freund writes: > On 2014-05-17 23:46:39 +0300, Heikki Linnakangas wrote: >> I doubt there are many people using btree_gist on int8 or float8 >> columns. > I don't think it's that unlikely. It can make a fair amount of sense to > have multicolumn indexes where one columns is over an int8 an

[HACKERS] uuid-ossp (Re: [pgsql-packagers] Postgresapp 9.4 beta build ready)

2014-05-17 Thread Christoph Berg
[redirecting to -hackers] Re: Tom Lane 2014-05-15 <31008.1400180...@sss.pgh.pa.us> > Sandeep Thakkar writes: > > Yes, Jakob is right. On 9.4, we had to patch configure script along with > > uuid-ossp.c to resolve the uuid issue. > > I think we need to discuss this on the open pgsql-hackers list.

Re: [HACKERS] %d in log_line_prefix doesn't work for bg/autovacuum workers

2014-05-17 Thread Christoph Berg
Re: Andres Freund 2014-05-17 <20140517211930.ga10...@awork2.anarazel.de> > On 2014-05-17 23:10:42 +0200, Christoph Berg wrote: > > > Given that there haven't been complaints in the past ten years about how > > > you can't rename an active database, I'm OK personally with locking this > > > down for

Re: [HACKERS] buildfarm: strange OOM failures on markhor (running CLOBBER_CACHE_RECURSIVELY)

2014-05-17 Thread Andres Freund
On 2014-05-17 22:55:14 +0200, Tomas Vondra wrote: > And another memory context stats for a session executing CREATE INDEX, > while having allocated The interesting thing is there are ~11k lines > that look exactly like this: > > pg_namespace_oid_index: 1024 total in 1 blocks; 88 free (0 chunks);

Re: [HACKERS] %d in log_line_prefix doesn't work for bg/autovacuum workers

2014-05-17 Thread Andres Freund
On 2014-05-17 23:35:43 +0200, Christoph Berg wrote: > They want to pg_terminate_backend() the sessions on db_old anyway, but > to get that point, you need to be able to rename the db. The > alternative would be to disallow connections to the db (for which > there is no real nice way), kill existing

Re: [HACKERS] Allowing join removals for more join types

2014-05-17 Thread David Rowley
On Sun, May 18, 2014 at 2:55 AM, Tom Lane wrote: > David Rowley writes: > > It looks like the existing join removals are done quite early in the > > planning and redundant joins are removed before any subqueries from that > > query are planned. So this innerrel->subroot->parse has not been done

Re: [HACKERS] 9.4 beta1 crash on Debian sid/i386

2014-05-17 Thread Christoph Berg
Re: Tom Lane 2014-05-14 <1357.1400028...@sss.pgh.pa.us> > Christoph Berg writes: > > Building 9.4 beta1 on Debian sid/i386 fails during the regression > > tests. amd64 works fine, as does i386 on the released distributions. > > It would appear that something is wrong with check_stack_depth(), > a

[HACKERS] pgbench is broken on strict-C89 compilers

2014-05-17 Thread Tom Lane
I got around to trying to build PG with the HP-supplied compiler on my ancient HPUX box, something I do about once a release cycle to see if we've finally broken that trailing-edge toolchain. Things still seem to work except for this: cc: "pgbench.c", line 1579: error 1521: Incorrect initializati

Re: [HACKERS] pgbench is broken on strict-C89 compilers

2014-05-17 Thread Andres Freund
Hi, On 2014-05-17 19:15:15 -0400, Tom Lane wrote: > I got around to trying to build PG with the HP-supplied compiler on my > ancient HPUX box, something I do about once a release cycle to see if > we've finally broken that trailing-edge toolchain. Things still seem > to work except for this: > >

Re: [HACKERS] 9.4 checksum error in recovery with btree index

2014-05-17 Thread Jeff Janes
On Saturday, May 17, 2014, Heikki Linnakangas wrote: > On 05/17/2014 12:28 AM, Jeff Janes wrote: > >> More fun with my torn page injection test program on 9.4. >> >> 24171 2014-05-16 14:00:44.934 PDT:WARNING: 01000: page verification >> failed, calculated checksum 21100 but expected 3356 >> 241

Re: [HACKERS] 9.4 beta1 crash on Debian sid/i386

2014-05-17 Thread Tom Lane
Christoph Berg writes: > Re: Tom Lane 2014-05-14 <1357.1400028...@sss.pgh.pa.us> >> It would appear that something is wrong with check_stack_depth(), >> and/or getrlimit(RLIMIT_STACK) is lying to us about the available stack. > ulimit -s is 8192 (kB); max_stack_depth is 2MB. > check_stack_depth

Re: [HACKERS] uuid-ossp (Re: [pgsql-packagers] Postgresapp 9.4 beta build ready)

2014-05-17 Thread Tom Lane
Christoph Berg writes: > [redirecting to -hackers] > Re: Tom Lane 2014-05-15 <31008.1400180...@sss.pgh.pa.us> >> Sandeep Thakkar writes: >>> Yes, Jakob is right. On 9.4, we had to patch configure script along with >>> uuid-ossp.c to resolve the uuid issue. >> I think we need to discuss this on t

Re: [HACKERS] pgbench is broken on strict-C89 compilers

2014-05-17 Thread Tom Lane
Andres Freund writes: > On 2014-05-17 19:15:15 -0400, Tom Lane wrote: >> ... It appears to me that >> the compiler is within its rights to refuse a nonconstant expression >> for an inner initializer according to C89, though I don't see any such >> restriction in C99. > Yea, I've complained about

Re: [HACKERS] Possible fix for occasional failures on castoroides etc

2014-05-17 Thread Tom Lane
Dave Page writes: > On Sat, May 3, 2014 at 8:29 PM, Andres Freund wrote: >> On 2012-09-17 08:23:01 -0400, Dave Page wrote: >>> I've added MAX_CONNECTIONS=5 to both Castoroides and Protosciurus. >> I've just noticed (while checking whether backporting 4c8aa8b5aea caused >> problems) that this doe

Re: [HACKERS] Possible fix for occasional failures on castoroides etc

2014-05-17 Thread Andres Freund
On 2014-05-18 01:35:04 -0400, Tom Lane wrote: > Dave Page writes: > > On Sat, May 3, 2014 at 8:29 PM, Andres Freund > > wrote: > >> On 2012-09-17 08:23:01 -0400, Dave Page wrote: > >>> I've added MAX_CONNECTIONS=5 to both Castoroides and Protosciurus. > > >> I've just noticed (while checking wh

Re: [HACKERS] pgbench is broken on strict-C89 compilers

2014-05-17 Thread Andres Freund
On 2014-05-18 00:36:34 -0400, Tom Lane wrote: > Andres Freund writes: > > On 2014-05-17 19:15:15 -0400, Tom Lane wrote: > >> ... It appears to me that > >> the compiler is within its rights to refuse a nonconstant expression > >> for an inner initializer according to C89, though I don't see any su

Re: [HACKERS] btree_gist macaddr valgrind woes

2014-05-17 Thread Heikki Linnakangas
On 05/18/2014 12:23 AM, Tom Lane wrote: A larger issue is that we evidently have no buildfarm animals that are picky about alignment, or at least none that are running a modern-enough buildfarm script to be running the contrib/logical_decoding test. That seems like a significant gap. I don't wan