pgsql: Silence MSVC warnings about redefinition of isnan

2018-11-18 Thread Andrew Dunstan
Silence MSVC warnings about redefinition of isnan

Some versions of perl.h define isnan when the compiler is MSVC. To avoid
warnings about this, undefine the symbol before including perl.h and
re-add the definition afterwards if it wasn't recreated.

Discussion: 
https://postgr.es/m/caf0568e-3c1f-07fd-6914-d903f2256...@2ndquadrant.com

Branch
--
master

Details
---
https://git.postgresql.org/pg/commitdiff/d5d7f7f3b78e9ee87ad01d1fb87b28968f65

Modified Files
--
src/pl/plperl/plperl.h | 14 +-
1 file changed, 13 insertions(+), 1 deletion(-)



pgsql: fix typo

2018-11-18 Thread Andrew Dunstan
fix typo

Branch
--
master

Details
---
https://git.postgresql.org/pg/commitdiff/79376e07128fbf3c92f4e1fb457be435afa2e6a6

Modified Files
--
src/pl/plperl/plperl.h | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)



pgsql: Remove unnecessary memcpy when reading WAL record fitting on pag

2018-11-18 Thread Michael Paquier
Remove unnecessary memcpy when reading WAL record fitting on page

When reading a WAL record, its contents are copied into an intermediate
buffer.  However, doing so is not necessary if the record fits fully
into the current page, saving one memcpy for each such record.  The
allocation handling of the intermediate buffer is also now done only
when a record crosses a page boundary, shaving some extra cycles when
reading a WAL record.

Author: Andrey Lepikhov
Reviewed-by: Kyotaro Horiguchi, Heikki Linnakangas
Discussion: 
https://postgr.es/m/c2ea54dd-a1d3-80eb-ddbf-7e6f258e6...@postgrespro.ru

Branch
--
master

Details
---
https://git.postgresql.org/pg/commitdiff/285bd0ac4a7c0538d544c40aa725682e11cb71a9

Modified Files
--
src/backend/access/transam/xlog.c   |  3 ---
src/backend/access/transam/xlogreader.c | 27 +--
2 files changed, 13 insertions(+), 17 deletions(-)



pgsql: Rework documentation of pg_promote

2018-11-18 Thread Michael Paquier
Rework documentation of pg_promote

This clarifies the behavior of how the "wait" flag works, which is
something that the previous version of the documentation failed to do.

Author: Ian Barwick
Discussion: 
https://postgr.es/m/cbd38450-2295-10a1-1f73-591a692ae...@2ndquadrant.com

Branch
--
master

Details
---
https://git.postgresql.org/pg/commitdiff/74171f8c12e68408f1fa895fdee36fdc8b0a9f79

Modified Files
--
doc/src/sgml/func.sgml | 18 ++
1 file changed, 10 insertions(+), 8 deletions(-)



pgsql: PANIC on fsync() failure.

2018-11-18 Thread Thomas Munro
PANIC on fsync() failure.

On some operating systems, it doesn't make sense to retry fsync(),
because dirty data cached by the kernel may have been dropped on
write-back failure.  In that case the only remaining copy of the
data is in the WAL.  A subsequent fsync() could appear to succeed,
but not have flushed the data.  That means that a future checkpoint
could apparently complete successfully but have lost data.

Therefore, violently prevent any future checkpoint attempts by
panicking on the first fsync() failure.  Note that we already
did the same for WAL data; this change extends that behavior to
non-temporary data files.

Provide a GUC data_sync_retry to control this new behavior, for
users of operating systems that don't eject dirty data, and possibly
forensic/testing uses.  If it is set to on and the write-back error
was transient, a later checkpoint might genuinely succeed (on a
system that does not throw away buffers on failure); if the error is
permanent, later checkpoints will continue to fail.  The GUC defaults
to off, meaning that we panic.

Back-patch to all supported releases.

There is still a narrow window for error-loss on some operating
systems: if the file is closed and later reopened and a write-back
error occurs in the intervening time, but the inode has the bad
luck to be evicted due to memory pressure before we reopen, we could
miss the error.  A later patch will address that with a scheme
for keeping files with dirty data open at all times, but we judge
that to be too complicated to back-patch.

Author: Craig Ringer, with some adjustments by Thomas Munro
Reported-by: Craig Ringer
Reviewed-by: Robert Haas, Thomas Munro, Andres Freund
Discussion: 
https://postgr.es/m/20180427222842.in2e4mibx45zdth5%40alap3.anarazel.de

Branch
--
master

Details
---
https://git.postgresql.org/pg/commitdiff/9ccdd7f66e3324d2b6d3dec282cfa9ff084083f1

Modified Files
--
doc/src/sgml/config.sgml  | 32 ++
src/backend/access/heap/rewriteheap.c |  6 ++--
src/backend/access/transam/slru.c |  2 +-
src/backend/access/transam/timeline.c |  4 +--
src/backend/access/transam/xlog.c |  2 +-
src/backend/replication/logical/snapbuild.c   |  3 ++
src/backend/storage/file/fd.c | 48 +++
src/backend/storage/smgr/md.c |  6 ++--
src/backend/utils/cache/relmapper.c   |  2 +-
src/backend/utils/misc/guc.c  |  9 +
src/backend/utils/misc/postgresql.conf.sample |  1 +
src/include/storage/fd.h  |  2 ++
12 files changed, 99 insertions(+), 18 deletions(-)



pgsql: Don't forget about failed fsync() requests.

2018-11-18 Thread Thomas Munro
Don't forget about failed fsync() requests.

If fsync() fails, md.c must keep the request in its bitmap, so that
future attempts will try again.

Back-patch to all supported releases.

Author: Thomas Munro
Reviewed-by: Amit Kapila
Reported-by: Andrew Gierth
Discussion: https://postgr.es/m/87y3i1ia4w.fsf%40news-spur.riddles.org.uk

Branch
--
master

Details
---
https://git.postgresql.org/pg/commitdiff/1556cb2fc5c774c3f7390dd6fb19190ee0c73f8b

Modified Files
--
src/backend/storage/smgr/md.c | 23 ++-
1 file changed, 18 insertions(+), 5 deletions(-)



pgsql: Don't forget about failed fsync() requests.

2018-11-18 Thread Thomas Munro
Don't forget about failed fsync() requests.

If fsync() fails, md.c must keep the request in its bitmap, so that
future attempts will try again.

Back-patch to all supported releases.

Author: Thomas Munro
Reviewed-by: Amit Kapila
Reported-by: Andrew Gierth
Discussion: https://postgr.es/m/87y3i1ia4w.fsf%40news-spur.riddles.org.uk

Branch
--
REL_11_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/542e6f3861cee1a5bd2974c318d2e05cc95d8a63

Modified Files
--
src/backend/storage/smgr/md.c | 23 ++-
1 file changed, 18 insertions(+), 5 deletions(-)



pgsql: PANIC on fsync() failure.

2018-11-18 Thread Thomas Munro
PANIC on fsync() failure.

On some operating systems, it doesn't make sense to retry fsync(),
because dirty data cached by the kernel may have been dropped on
write-back failure.  In that case the only remaining copy of the
data is in the WAL.  A subsequent fsync() could appear to succeed,
but not have flushed the data.  That means that a future checkpoint
could apparently complete successfully but have lost data.

Therefore, violently prevent any future checkpoint attempts by
panicking on the first fsync() failure.  Note that we already
did the same for WAL data; this change extends that behavior to
non-temporary data files.

Provide a GUC data_sync_retry to control this new behavior, for
users of operating systems that don't eject dirty data, and possibly
forensic/testing uses.  If it is set to on and the write-back error
was transient, a later checkpoint might genuinely succeed (on a
system that does not throw away buffers on failure); if the error is
permanent, later checkpoints will continue to fail.  The GUC defaults
to off, meaning that we panic.

Back-patch to all supported releases.

There is still a narrow window for error-loss on some operating
systems: if the file is closed and later reopened and a write-back
error occurs in the intervening time, but the inode has the bad
luck to be evicted due to memory pressure before we reopen, we could
miss the error.  A later patch will address that with a scheme
for keeping files with dirty data open at all times, but we judge
that to be too complicated to back-patch.

Author: Craig Ringer, with some adjustments by Thomas Munro
Reported-by: Craig Ringer
Reviewed-by: Robert Haas, Thomas Munro, Andres Freund
Discussion: 
https://postgr.es/m/20180427222842.in2e4mibx45zdth5%40alap3.anarazel.de

Branch
--
REL9_4_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/f1ff5f51d2490817c2b3bba202f4e2c73a846389

Modified Files
--
doc/src/sgml/config.sgml  | 32 
src/backend/access/heap/rewriteheap.c |  6 ++--
src/backend/access/transam/slru.c |  2 +-
src/backend/access/transam/timeline.c |  4 +--
src/backend/access/transam/xlog.c |  2 +-
src/backend/replication/logical/snapbuild.c   |  3 ++
src/backend/storage/file/fd.c | 42 +--
src/backend/storage/smgr/md.c |  6 ++--
src/backend/utils/cache/relmapper.c   |  2 +-
src/backend/utils/misc/guc.c  |  9 ++
src/backend/utils/misc/postgresql.conf.sample |  1 +
src/include/storage/fd.h  |  2 ++
12 files changed, 97 insertions(+), 14 deletions(-)



pgsql: Don't forget about failed fsync() requests.

2018-11-18 Thread Thomas Munro
Don't forget about failed fsync() requests.

If fsync() fails, md.c must keep the request in its bitmap, so that
future attempts will try again.

Back-patch to all supported releases.

Author: Thomas Munro
Reviewed-by: Amit Kapila
Reported-by: Andrew Gierth
Discussion: https://postgr.es/m/87y3i1ia4w.fsf%40news-spur.riddles.org.uk

Branch
--
REL_10_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/28117764db7ed0c9fc589dd9323c439235040b05

Modified Files
--
src/backend/storage/smgr/md.c | 23 ++-
1 file changed, 18 insertions(+), 5 deletions(-)



pgsql: PANIC on fsync() failure.

2018-11-18 Thread Thomas Munro
PANIC on fsync() failure.

On some operating systems, it doesn't make sense to retry fsync(),
because dirty data cached by the kernel may have been dropped on
write-back failure.  In that case the only remaining copy of the
data is in the WAL.  A subsequent fsync() could appear to succeed,
but not have flushed the data.  That means that a future checkpoint
could apparently complete successfully but have lost data.

Therefore, violently prevent any future checkpoint attempts by
panicking on the first fsync() failure.  Note that we already
did the same for WAL data; this change extends that behavior to
non-temporary data files.

Provide a GUC data_sync_retry to control this new behavior, for
users of operating systems that don't eject dirty data, and possibly
forensic/testing uses.  If it is set to on and the write-back error
was transient, a later checkpoint might genuinely succeed (on a
system that does not throw away buffers on failure); if the error is
permanent, later checkpoints will continue to fail.  The GUC defaults
to off, meaning that we panic.

Back-patch to all supported releases.

There is still a narrow window for error-loss on some operating
systems: if the file is closed and later reopened and a write-back
error occurs in the intervening time, but the inode has the bad
luck to be evicted due to memory pressure before we reopen, we could
miss the error.  A later patch will address that with a scheme
for keeping files with dirty data open at all times, but we judge
that to be too complicated to back-patch.

Author: Craig Ringer, with some adjustments by Thomas Munro
Reported-by: Craig Ringer
Reviewed-by: Robert Haas, Thomas Munro, Andres Freund
Discussion: 
https://postgr.es/m/20180427222842.in2e4mibx45zdth5%40alap3.anarazel.de

Branch
--
REL9_6_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/b9cce9ddfa1748c76d1d35c62aa35fe70ddc2dbe

Modified Files
--
doc/src/sgml/config.sgml  | 32 ++
src/backend/access/heap/rewriteheap.c |  6 ++--
src/backend/access/transam/slru.c |  2 +-
src/backend/access/transam/timeline.c |  4 +--
src/backend/access/transam/xlog.c |  2 +-
src/backend/replication/logical/snapbuild.c   |  3 ++
src/backend/storage/file/fd.c | 48 +++
src/backend/storage/smgr/md.c |  6 ++--
src/backend/utils/cache/relmapper.c   |  2 +-
src/backend/utils/misc/guc.c  |  9 +
src/backend/utils/misc/postgresql.conf.sample |  1 +
src/include/storage/fd.h  |  2 ++
12 files changed, 99 insertions(+), 18 deletions(-)



pgsql: PANIC on fsync() failure.

2018-11-18 Thread Thomas Munro
PANIC on fsync() failure.

On some operating systems, it doesn't make sense to retry fsync(),
because dirty data cached by the kernel may have been dropped on
write-back failure.  In that case the only remaining copy of the
data is in the WAL.  A subsequent fsync() could appear to succeed,
but not have flushed the data.  That means that a future checkpoint
could apparently complete successfully but have lost data.

Therefore, violently prevent any future checkpoint attempts by
panicking on the first fsync() failure.  Note that we already
did the same for WAL data; this change extends that behavior to
non-temporary data files.

Provide a GUC data_sync_retry to control this new behavior, for
users of operating systems that don't eject dirty data, and possibly
forensic/testing uses.  If it is set to on and the write-back error
was transient, a later checkpoint might genuinely succeed (on a
system that does not throw away buffers on failure); if the error is
permanent, later checkpoints will continue to fail.  The GUC defaults
to off, meaning that we panic.

Back-patch to all supported releases.

There is still a narrow window for error-loss on some operating
systems: if the file is closed and later reopened and a write-back
error occurs in the intervening time, but the inode has the bad
luck to be evicted due to memory pressure before we reopen, we could
miss the error.  A later patch will address that with a scheme
for keeping files with dirty data open at all times, but we judge
that to be too complicated to back-patch.

Author: Craig Ringer, with some adjustments by Thomas Munro
Reported-by: Craig Ringer
Reviewed-by: Robert Haas, Thomas Munro, Andres Freund
Discussion: 
https://postgr.es/m/20180427222842.in2e4mibx45zdth5%40alap3.anarazel.de

Branch
--
REL_11_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/6534d544cd77e14a93f5c779fa8addee8d916a66

Modified Files
--
doc/src/sgml/config.sgml  | 32 ++
src/backend/access/heap/rewriteheap.c |  6 ++--
src/backend/access/transam/slru.c |  2 +-
src/backend/access/transam/timeline.c |  4 +--
src/backend/access/transam/xlog.c |  2 +-
src/backend/replication/logical/snapbuild.c   |  3 ++
src/backend/storage/file/fd.c | 48 +++
src/backend/storage/smgr/md.c |  6 ++--
src/backend/utils/cache/relmapper.c   |  2 +-
src/backend/utils/misc/guc.c  |  9 +
src/backend/utils/misc/postgresql.conf.sample |  1 +
src/include/storage/fd.h  |  2 ++
12 files changed, 99 insertions(+), 18 deletions(-)



pgsql: Don't forget about failed fsync() requests.

2018-11-18 Thread Thomas Munro
Don't forget about failed fsync() requests.

If fsync() fails, md.c must keep the request in its bitmap, so that
future attempts will try again.

Back-patch to all supported releases.

Author: Thomas Munro
Reviewed-by: Amit Kapila
Reported-by: Andrew Gierth
Discussion: https://postgr.es/m/87y3i1ia4w.fsf%40news-spur.riddles.org.uk

Branch
--
REL9_5_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/a4a4aede5fabe6d94bd9b6445725db0c57eecd9b

Modified Files
--
src/backend/storage/smgr/md.c | 23 ++-
1 file changed, 18 insertions(+), 5 deletions(-)



pgsql: PANIC on fsync() failure.

2018-11-18 Thread Thomas Munro
PANIC on fsync() failure.

On some operating systems, it doesn't make sense to retry fsync(),
because dirty data cached by the kernel may have been dropped on
write-back failure.  In that case the only remaining copy of the
data is in the WAL.  A subsequent fsync() could appear to succeed,
but not have flushed the data.  That means that a future checkpoint
could apparently complete successfully but have lost data.

Therefore, violently prevent any future checkpoint attempts by
panicking on the first fsync() failure.  Note that we already
did the same for WAL data; this change extends that behavior to
non-temporary data files.

Provide a GUC data_sync_retry to control this new behavior, for
users of operating systems that don't eject dirty data, and possibly
forensic/testing uses.  If it is set to on and the write-back error
was transient, a later checkpoint might genuinely succeed (on a
system that does not throw away buffers on failure); if the error is
permanent, later checkpoints will continue to fail.  The GUC defaults
to off, meaning that we panic.

Back-patch to all supported releases.

There is still a narrow window for error-loss on some operating
systems: if the file is closed and later reopened and a write-back
error occurs in the intervening time, but the inode has the bad
luck to be evicted due to memory pressure before we reopen, we could
miss the error.  A later patch will address that with a scheme
for keeping files with dirty data open at all times, but we judge
that to be too complicated to back-patch.

Author: Craig Ringer, with some adjustments by Thomas Munro
Reported-by: Craig Ringer
Reviewed-by: Robert Haas, Thomas Munro, Andres Freund
Discussion: 
https://postgr.es/m/20180427222842.in2e4mibx45zdth5%40alap3.anarazel.de

Branch
--
REL_10_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/afbe03f6547015790d13d21d5eb5083cc604eeec

Modified Files
--
doc/src/sgml/config.sgml  | 32 ++
src/backend/access/heap/rewriteheap.c |  6 ++--
src/backend/access/transam/slru.c |  2 +-
src/backend/access/transam/timeline.c |  4 +--
src/backend/access/transam/xlog.c |  2 +-
src/backend/replication/logical/snapbuild.c   |  3 ++
src/backend/storage/file/fd.c | 48 +++
src/backend/storage/smgr/md.c |  6 ++--
src/backend/utils/cache/relmapper.c   |  2 +-
src/backend/utils/misc/guc.c  |  9 +
src/backend/utils/misc/postgresql.conf.sample |  1 +
src/include/storage/fd.h  |  2 ++
12 files changed, 99 insertions(+), 18 deletions(-)



pgsql: Don't forget about failed fsync() requests.

2018-11-18 Thread Thomas Munro
Don't forget about failed fsync() requests.

If fsync() fails, md.c must keep the request in its bitmap, so that
future attempts will try again.

Back-patch to all supported releases.

Author: Thomas Munro
Reviewed-by: Amit Kapila
Reported-by: Andrew Gierth
Discussion: https://postgr.es/m/87y3i1ia4w.fsf%40news-spur.riddles.org.uk

Branch
--
REL9_6_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/a4d6d6a25c12ca9a19a5e7ec6f76115bced69521

Modified Files
--
src/backend/storage/smgr/md.c | 23 ++-
1 file changed, 18 insertions(+), 5 deletions(-)



pgsql: PANIC on fsync() failure.

2018-11-18 Thread Thomas Munro
PANIC on fsync() failure.

On some operating systems, it doesn't make sense to retry fsync(),
because dirty data cached by the kernel may have been dropped on
write-back failure.  In that case the only remaining copy of the
data is in the WAL.  A subsequent fsync() could appear to succeed,
but not have flushed the data.  That means that a future checkpoint
could apparently complete successfully but have lost data.

Therefore, violently prevent any future checkpoint attempts by
panicking on the first fsync() failure.  Note that we already
did the same for WAL data; this change extends that behavior to
non-temporary data files.

Provide a GUC data_sync_retry to control this new behavior, for
users of operating systems that don't eject dirty data, and possibly
forensic/testing uses.  If it is set to on and the write-back error
was transient, a later checkpoint might genuinely succeed (on a
system that does not throw away buffers on failure); if the error is
permanent, later checkpoints will continue to fail.  The GUC defaults
to off, meaning that we panic.

Back-patch to all supported releases.

There is still a narrow window for error-loss on some operating
systems: if the file is closed and later reopened and a write-back
error occurs in the intervening time, but the inode has the bad
luck to be evicted due to memory pressure before we reopen, we could
miss the error.  A later patch will address that with a scheme
for keeping files with dirty data open at all times, but we judge
that to be too complicated to back-patch.

Author: Craig Ringer, with some adjustments by Thomas Munro
Reported-by: Craig Ringer
Reviewed-by: Robert Haas, Thomas Munro, Andres Freund
Discussion: 
https://postgr.es/m/20180427222842.in2e4mibx45zdth5%40alap3.anarazel.de

Branch
--
REL9_5_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/3124352322179a4b3bdc47f98caf2ef8e0e3de65

Modified Files
--
doc/src/sgml/config.sgml  | 32 
src/backend/access/heap/rewriteheap.c |  6 ++--
src/backend/access/transam/slru.c |  2 +-
src/backend/access/transam/timeline.c |  4 +--
src/backend/access/transam/xlog.c |  2 +-
src/backend/replication/logical/snapbuild.c   |  3 ++
src/backend/storage/file/fd.c | 42 +--
src/backend/storage/smgr/md.c |  6 ++--
src/backend/utils/cache/relmapper.c   |  2 +-
src/backend/utils/misc/guc.c  |  9 ++
src/backend/utils/misc/postgresql.conf.sample |  1 +
src/include/storage/fd.h  |  2 ++
12 files changed, 97 insertions(+), 14 deletions(-)



pgsql: Don't forget about failed fsync() requests.

2018-11-18 Thread Thomas Munro
Don't forget about failed fsync() requests.

If fsync() fails, md.c must keep the request in its bitmap, so that
future attempts will try again.

Back-patch to all supported releases.

Author: Thomas Munro
Reviewed-by: Amit Kapila
Reported-by: Andrew Gierth
Discussion: https://postgr.es/m/87y3i1ia4w.fsf%40news-spur.riddles.org.uk

Branch
--
REL9_4_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/2b2010d12a62c4b77fdffbd988f5cbbcdcddb98e

Modified Files
--
src/backend/storage/smgr/md.c | 23 ++-
1 file changed, 18 insertions(+), 5 deletions(-)



pgsql: pg_archivecleanup: Update file header comment a bit

2018-11-18 Thread Peter Eisentraut
pg_archivecleanup: Update file header comment a bit

Branch
--
master

Details
---
https://git.postgresql.org/pg/commitdiff/fc47e99a157a56a92dea679a74d546e28b253bb7

Modified Files
--
src/bin/pg_archivecleanup/pg_archivecleanup.c | 10 +++---
1 file changed, 3 insertions(+), 7 deletions(-)