pgsql: Postpone calculating total_table_pages until after pruning/exclu

2018-11-07 Thread Tom Lane
Postpone calculating total_table_pages until after pruning/exclusion.

The planner doesn't make any use of root->total_table_pages until it
estimates costs of indexscans, so we don't need to compute it as
early as that's currently done.  By doing the calculation between
set_base_rel_sizes and set_base_rel_pathlists, we can omit relations
that get removed from the query by partition pruning or constraint
exclusion, which seems like a more accurate basis for costing.

(Historical note: I think at the time this code was written, there
was not a separation between the "set sizes" and "set pathlists"
steps, so that this approach would have been impossible at the time.
But now that we have that separation, this is clearly the better way
to do things.)

David Rowley, reviewed by Edmund Horner

Discussion: 
https://postgr.es/m/CAKJS1f-NG1mRM0VOtkAG7=zlqwihoqees9r4ki3ckbb0-fr...@mail.gmail.com

Branch
--
master

Details
---
https://git.postgresql.org/pg/commitdiff/c6e4133fae1fde93769197379ffcc2b379845113

Modified Files
--
src/backend/optimizer/path/allpaths.c | 45 +--
src/backend/optimizer/plan/planmain.c | 30 ---
src/backend/optimizer/util/plancat.c  |  5 ++--
src/include/nodes/relation.h  |  8 +++
4 files changed, 49 insertions(+), 39 deletions(-)



pgsql: Use installcheck-parallel in pg_upgrade's testsuite.

2018-11-07 Thread Andres Freund
Use installcheck-parallel in pg_upgrade's testsuite.

The installcheck run takes a sizable fraction of test.sh to run. Using
a parallel schedule reduces that noticably.

It's possible that we want to backpatch this at some point, to reduce
buildfarm times, but for now lets just see if this upsets the
buildfarm.

Author: Andres Freund
Discussion: 
https://postgr.es/m/20181105210039.hh4vvi4vwoq5b...@alap3.anarazel.de

Branch
--
master

Details
---
https://git.postgresql.org/pg/commitdiff/da906766cd3bef8290db25bd5c8776c03689f3d5

Modified Files
--
src/bin/pg_upgrade/test.sh | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)



pgsql: Build HashState's hashkeys expression with the correct parent.

2018-11-07 Thread Andres Freund
Build HashState's hashkeys expression with the correct parent.

Previously the expressions were built with the HashJoinState as a
parent. That's incorrect.

Currently this does not appear to be harmful, but for the upcoming
'slot abstraction' work this proves to be problematic, as the
underlying slot types can differ between Hash and HashJoin.  It's
possible that this already causes a problem, but I've not been able to
come up with a scenario.  Therefore don't backpatch at this point.

Author: Andres Freund
Discussion: 
https://postgr.es/m/20180220224318.gw4oe5jadhpmc...@alap3.anarazel.de

Branch
--
master

Details
---
https://git.postgresql.org/pg/commitdiff/5f32b29c18195299e90c1fb6c8945e9a46d772d2

Modified Files
--
src/backend/executor/nodeHashjoin.c | 6 +-
1 file changed, 5 insertions(+), 1 deletion(-)



pgsql: pg_upgrade: Allow use of file cloning

2018-11-07 Thread Peter Eisentraut
pg_upgrade: Allow use of file cloning

Add another transfer mode --clone to pg_upgrade (besides the existing
--link and the default copy), using special file cloning calls.  This
makes the file transfer faster and more space efficient, achieving
speed similar to --link mode without the associated drawbacks.

On Linux, file cloning is supported on Btrfs and XFS (if formatted with
reflink support).  On macOS, file cloning is supported on APFS.

Reviewed-by: Michael Paquier 

Branch
--
master

Details
---
https://git.postgresql.org/pg/commitdiff/3a769d8239afdc003c91a56d2d8d5adfadacda5d

Modified Files
--
configure|  2 +-
configure.in |  1 +
doc/src/sgml/ref/pgupgrade.sgml  | 40 +++---
src/bin/pg_upgrade/check.c   | 13 +-
src/bin/pg_upgrade/file.c| 90 
src/bin/pg_upgrade/option.c  |  7 
src/bin/pg_upgrade/pg_upgrade.h  |  6 ++-
src/bin/pg_upgrade/relfilenode.c | 44 +---
src/include/pg_config.h.in   |  3 ++
9 files changed, 181 insertions(+), 25 deletions(-)



pgsql: Quiet valgrind complaints following pread/pwrite changes

2018-11-07 Thread Andrew Dunstan
Quiet valgrind complaints following pread/pwrite changes

Per complaints from buildfarm and elsewhere
Patch from Jasper Pedersen
Discussion: https://postgr.es/m/0f419c91-49ab-2399-0143-13063bd97...@redhat.com

Branch
--
master

Details
---
https://git.postgresql.org/pg/commitdiff/517b0d0b5f38c92c1807a40d71ca4bf6428ca7d5

Modified Files
--
src/tools/valgrind.supp | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)



pgsql: Move EEOP_*_SYSVAR evaluation out of line.

2018-11-07 Thread Andres Freund
Move EEOP_*_SYSVAR evaluation out of line.

This mainly de-duplicates code. As evaluating a system variable isn't
the hottest path and the current inline implementation ends up calling
out to an external function anyway, this is OK from a performance POV.

The main motivation for de-duplicating is the upcoming slot
abstraction work, after which there's not guaranteed to be a HeapTuple
backing the slot.

Author: Andres Freund, Amit Khandekar
Discussion: 
https://postgr.es/m/20181105210039.hh4vvi4vwoq5b...@alap3.anarazel.de

Branch
--
master

Details
---
https://git.postgresql.org/pg/commitdiff/b84a6dafbf2bb921baee53c0c1aba7719ee38817

Modified Files
--
src/backend/executor/execExprInterp.c | 58 ---
src/backend/jit/llvm/llvmjit.c|  4 +--
src/backend/jit/llvm/llvmjit_expr.c   | 38 +--
src/backend/jit/llvm/llvmjit_types.c  |  2 +-
src/include/executor/execExpr.h   |  2 ++
src/include/jit/llvmjit.h |  2 +-
6 files changed, 33 insertions(+), 73 deletions(-)



pgsql: Use parallel installcheck in vcregress.pl's upgrade test

2018-11-07 Thread Andrew Dunstan
Use parallel installcheck in vcregress.pl's upgrade test

This is to keep the test in sync with what's done in test.sh, which
acquired this change in commit da906766c.

Branch
--
master

Details
---
https://git.postgresql.org/pg/commitdiff/54ad7282fe7ad80a15533546d3b0a2be2e19e3f6

Modified Files
--
src/tools/msvc/vcregress.pl | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)



Re: pgsql: Move EEOP_*_SYSVAR evaluation out of line.

2018-11-07 Thread Tom Lane
Andres Freund  writes:
> Move EEOP_*_SYSVAR evaluation out of line.

Seems like many of your JIT-enabled animals don't like this.

regards, tom lane



pgsql: Fix inadequate autoconfiscation of copyfile() usage.

2018-11-07 Thread Tom Lane
Fix inadequate autoconfiscation of copyfile() usage.

Per buildfarm, HAVE_COPYFILE is not the same thing as HAVE_COPYFILE_H.
Add the extra configure test.

Branch
--
master

Details
---
https://git.postgresql.org/pg/commitdiff/c3e6d5d38699812e047270a711be1926b7f6f396

Modified Files
--
configure  | 2 +-
configure.in   | 1 +
src/bin/pg_upgrade/file.c  | 2 +-
src/include/pg_config.h.in | 3 +++
4 files changed, 6 insertions(+), 2 deletions(-)



Re: pgsql: Fix copy-paste error in errhint() introduced in 691d79a07933.

2018-11-07 Thread Christoph Berg
Re: Andres Freund 2018-11-05 
> Fix copy-paste error in errhint() introduced in 691d79a07933.
> 
> Reported-By: Petr Jelinek
> Discussion: 
> https://postgr.es/m/c95a620b-34f0-7930-aeb5-f7ab804f2...@2ndquadrant.com
> Backpatch: 9.4-, like the previous commit

-errhint("Change wal_level to be replica or higher.")));
+errhint("Change wal_level to be logical or higher.")));

Are there wal_levels higher than logical?

Christoph



pgsql: Fixup for b84a6dafbf triggering assert failure in LLVM debug bui

2018-11-07 Thread Andres Freund
Fixup for b84a6dafbf triggering assert failure in LLVM debug builds.

Author: Andres Freund

Branch
--
master

Details
---
https://git.postgresql.org/pg/commitdiff/c14a8ff27e26155345e554a9636bab8236500bb9

Modified Files
--
src/backend/jit/llvm/llvmjit_expr.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)



Re: pgsql: Fix copy-paste error in errhint() introduced in 691d79a07933.

2018-11-07 Thread Andres Freund
Hi,

On 2018-11-07 22:32:22 +0100, Christoph Berg wrote:
> Re: Andres Freund 2018-11-05 
> > Fix copy-paste error in errhint() introduced in 691d79a07933.
> > 
> > Reported-By: Petr Jelinek
> > Discussion: 
> > https://postgr.es/m/c95a620b-34f0-7930-aeb5-f7ab804f2...@2ndquadrant.com
> > Backpatch: 9.4-, like the previous commit
> 
> -errhint("Change wal_level to be replica or higher.")));
> +errhint("Change wal_level to be logical or higher.")));
> 
> Are there wal_levels higher than logical?

Not right now, but it seems quite likely that there will be. IIRC we've
phrased a few other messages like that. It's quite likely that otherwise
the message would be forgotten to be updated. And it's not like it's
incorrect.

Greetings,

Andres Freund



Re: pgsql: Move EEOP_*_SYSVAR evaluation out of line.

2018-11-07 Thread Andres Freund
Hi,

On 2018-11-07 16:23:13 -0500, Tom Lane wrote:
> Andres Freund  writes:
> > Move EEOP_*_SYSVAR evaluation out of line.
> 
> Seems like many of your JIT-enabled animals don't like this.

Thanks. Yea, it's the animals that have the (pretty expensive) LLVM
assertions enabled. I've pushed the trivial oversight fix. Sorry for not
attributing this message/thread - I only saw it afterwards.

Greetings,

Andres Freund