pgsql: Fix multiple problems with satisfies_hash_partition.

2017-11-21 Thread Robert Haas
Fix multiple problems with satisfies_hash_partition.

Fix the function header comment to describe the actual behavior.
Check that table OID, modulus, and remainder arguments are not NULL
before accessing them.  Check that the modulus and remainder are
sensible.  If the table OID doesn't exist, return NULL instead of
emitting an internal error, similar to what we do elsewhere.  Check
that the actual argument types match, or at least are binary coercible
to, the expected argument types.  Correctly handle invocation of this
function using the VARIADIC syntax.  Add regression tests.

Robert Haas and Amul Sul, per a report by Andreas Seltenreich and
subsequent followup investigation.

Discussion: http://postgr.es/m/[email protected]

Branch
--
master

Details
---
https://git.postgresql.org/pg/commitdiff/f3b0897a1213f46b4d3a99a7f8ef3a4b32e03572

Modified Files
--
src/backend/catalog/partition.c | 202 +++-
src/test/regress/expected/hash_part.out | 113 ++
src/test/regress/parallel_schedule  |   2 +-
src/test/regress/serial_schedule|   1 +
src/test/regress/sql/hash_part.sql  |  90 ++
5 files changed, 377 insertions(+), 31 deletions(-)



pgsql: Provide for forward compatibility with future minor protocol ver

2017-11-21 Thread Robert Haas
Provide for forward compatibility with future minor protocol versions.

Previously, any attempt to request a 3.x protocol version other than
3.0 would lead to a hard connection failure, which made the minor
protocol version really no different from the major protocol version
and precluded gentle protocol version breaks.  Instead, when the
client requests a 3.x protocol version where x is greater than 0, send
the new NegotiateProtocolVersion message to convey that we support
only 3.0.  This makes it possible to introduce new minor protocol
versions without requiring a connection retry when the server is
older.

In addition, if the startup packet includes name/value pairs where
the name starts with "_pq_.", assume that those are protocol options,
not GUCs.  Include those we don't support (i.e. all of them, at
present) in the NegotiateProtocolVersion message so that the client
knows they were not understood.  This makes it possible for the
client to request previously-unsupported features without bumping
the protocol version at all; the client can tell from the server's
response whether the option was understood.

It will take some time before servers that support these new
facilities become common in the wild; to speed things up and make
things easier for a future 3.1 protocol version, back-patch to all
supported releases.

Robert Haas and Badrul Chowdhury

Discussion: 
http://postgr.es/m/bn6pr21mb0772ffa0cbd298b76017744cd1...@bn6pr21mb0772.namprd21.prod.outlook.com
Discussion: http://postgr.es/m/[email protected]

Branch
--
master

Details
---
https://git.postgresql.org/pg/commitdiff/ae65f6066dc3d19a55f4fdcd3b30003c5ad8dbed

Modified Files
--
doc/src/sgml/protocol.sgml  | 117 
src/backend/postmaster/postmaster.c |  58 --
2 files changed, 159 insertions(+), 16 deletions(-)



pgsql: Provide for forward compatibility with future minor protocol ver

2017-11-21 Thread Robert Haas
Provide for forward compatibility with future minor protocol versions.

Previously, any attempt to request a 3.x protocol version other than
3.0 would lead to a hard connection failure, which made the minor
protocol version really no different from the major protocol version
and precluded gentle protocol version breaks.  Instead, when the
client requests a 3.x protocol version where x is greater than 0, send
the new NegotiateProtocolVersion message to convey that we support
only 3.0.  This makes it possible to introduce new minor protocol
versions without requiring a connection retry when the server is
older.

In addition, if the startup packet includes name/value pairs where
the name starts with "_pq_.", assume that those are protocol options,
not GUCs.  Include those we don't support (i.e. all of them, at
present) in the NegotiateProtocolVersion message so that the client
knows they were not understood.  This makes it possible for the
client to request previously-unsupported features without bumping
the protocol version at all; the client can tell from the server's
response whether the option was understood.

It will take some time before servers that support these new
facilities become common in the wild; to speed things up and make
things easier for a future 3.1 protocol version, back-patch to all
supported releases.

Robert Haas and Badrul Chowdhury

Discussion: 
http://postgr.es/m/bn6pr21mb0772ffa0cbd298b76017744cd1...@bn6pr21mb0772.namprd21.prod.outlook.com
Discussion: http://postgr.es/m/[email protected]

Branch
--
REL9_4_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/294136d42230467b1f5496693aac847ce2199e64

Modified Files
--
doc/src/sgml/protocol.sgml  | 116 
src/backend/postmaster/postmaster.c |  58 --
2 files changed, 158 insertions(+), 16 deletions(-)



pgsql: Provide for forward compatibility with future minor protocol ver

2017-11-21 Thread Robert Haas
Provide for forward compatibility with future minor protocol versions.

Previously, any attempt to request a 3.x protocol version other than
3.0 would lead to a hard connection failure, which made the minor
protocol version really no different from the major protocol version
and precluded gentle protocol version breaks.  Instead, when the
client requests a 3.x protocol version where x is greater than 0, send
the new NegotiateProtocolVersion message to convey that we support
only 3.0.  This makes it possible to introduce new minor protocol
versions without requiring a connection retry when the server is
older.

In addition, if the startup packet includes name/value pairs where
the name starts with "_pq_.", assume that those are protocol options,
not GUCs.  Include those we don't support (i.e. all of them, at
present) in the NegotiateProtocolVersion message so that the client
knows they were not understood.  This makes it possible for the
client to request previously-unsupported features without bumping
the protocol version at all; the client can tell from the server's
response whether the option was understood.

It will take some time before servers that support these new
facilities become common in the wild; to speed things up and make
things easier for a future 3.1 protocol version, back-patch to all
supported releases.

Robert Haas and Badrul Chowdhury

Discussion: 
http://postgr.es/m/bn6pr21mb0772ffa0cbd298b76017744cd1...@bn6pr21mb0772.namprd21.prod.outlook.com
Discussion: http://postgr.es/m/[email protected]

Branch
--
REL9_6_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/7c84bc0b3d709f59720fa913b0d5997754d2691f

Modified Files
--
doc/src/sgml/protocol.sgml  | 116 
src/backend/postmaster/postmaster.c |  58 --
2 files changed, 158 insertions(+), 16 deletions(-)



pgsql: Provide for forward compatibility with future minor protocol ver

2017-11-21 Thread Robert Haas
Provide for forward compatibility with future minor protocol versions.

Previously, any attempt to request a 3.x protocol version other than
3.0 would lead to a hard connection failure, which made the minor
protocol version really no different from the major protocol version
and precluded gentle protocol version breaks.  Instead, when the
client requests a 3.x protocol version where x is greater than 0, send
the new NegotiateProtocolVersion message to convey that we support
only 3.0.  This makes it possible to introduce new minor protocol
versions without requiring a connection retry when the server is
older.

In addition, if the startup packet includes name/value pairs where
the name starts with "_pq_.", assume that those are protocol options,
not GUCs.  Include those we don't support (i.e. all of them, at
present) in the NegotiateProtocolVersion message so that the client
knows they were not understood.  This makes it possible for the
client to request previously-unsupported features without bumping
the protocol version at all; the client can tell from the server's
response whether the option was understood.

It will take some time before servers that support these new
facilities become common in the wild; to speed things up and make
things easier for a future 3.1 protocol version, back-patch to all
supported releases.

Robert Haas and Badrul Chowdhury

Discussion: 
http://postgr.es/m/bn6pr21mb0772ffa0cbd298b76017744cd1...@bn6pr21mb0772.namprd21.prod.outlook.com
Discussion: http://postgr.es/m/[email protected]

Branch
--
REL9_5_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/c703aa6258997d22ca7338bc221c31ff83feefb7

Modified Files
--
doc/src/sgml/protocol.sgml  | 116 
src/backend/postmaster/postmaster.c |  58 --
2 files changed, 158 insertions(+), 16 deletions(-)



pgsql: Provide for forward compatibility with future minor protocol ver

2017-11-21 Thread Robert Haas
Provide for forward compatibility with future minor protocol versions.

Previously, any attempt to request a 3.x protocol version other than
3.0 would lead to a hard connection failure, which made the minor
protocol version really no different from the major protocol version
and precluded gentle protocol version breaks.  Instead, when the
client requests a 3.x protocol version where x is greater than 0, send
the new NegotiateProtocolVersion message to convey that we support
only 3.0.  This makes it possible to introduce new minor protocol
versions without requiring a connection retry when the server is
older.

In addition, if the startup packet includes name/value pairs where
the name starts with "_pq_.", assume that those are protocol options,
not GUCs.  Include those we don't support (i.e. all of them, at
present) in the NegotiateProtocolVersion message so that the client
knows they were not understood.  This makes it possible for the
client to request previously-unsupported features without bumping
the protocol version at all; the client can tell from the server's
response whether the option was understood.

It will take some time before servers that support these new
facilities become common in the wild; to speed things up and make
things easier for a future 3.1 protocol version, back-patch to all
supported releases.

Robert Haas and Badrul Chowdhury

Discussion: 
http://postgr.es/m/bn6pr21mb0772ffa0cbd298b76017744cd1...@bn6pr21mb0772.namprd21.prod.outlook.com
Discussion: http://postgr.es/m/[email protected]

Branch
--
REL_10_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/5b2a877070913ee4e7088f3593804595309b3938

Modified Files
--
doc/src/sgml/protocol.sgml  | 117 
src/backend/postmaster/postmaster.c |  58 --
2 files changed, 159 insertions(+), 16 deletions(-)



pgsql: Provide for forward compatibility with future minor protocol ver

2017-11-21 Thread Robert Haas
Provide for forward compatibility with future minor protocol versions.

Previously, any attempt to request a 3.x protocol version other than
3.0 would lead to a hard connection failure, which made the minor
protocol version really no different from the major protocol version
and precluded gentle protocol version breaks.  Instead, when the
client requests a 3.x protocol version where x is greater than 0, send
the new NegotiateProtocolVersion message to convey that we support
only 3.0.  This makes it possible to introduce new minor protocol
versions without requiring a connection retry when the server is
older.

In addition, if the startup packet includes name/value pairs where
the name starts with "_pq_.", assume that those are protocol options,
not GUCs.  Include those we don't support (i.e. all of them, at
present) in the NegotiateProtocolVersion message so that the client
knows they were not understood.  This makes it possible for the
client to request previously-unsupported features without bumping
the protocol version at all; the client can tell from the server's
response whether the option was understood.

It will take some time before servers that support these new
facilities become common in the wild; to speed things up and make
things easier for a future 3.1 protocol version, back-patch to all
supported releases.

Robert Haas and Badrul Chowdhury

Discussion: 
http://postgr.es/m/bn6pr21mb0772ffa0cbd298b76017744cd1...@bn6pr21mb0772.namprd21.prod.outlook.com
Discussion: http://postgr.es/m/[email protected]

Branch
--
REL9_3_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/aa3a78c3ddaf380355dc5264acd2eb8e128ee422

Modified Files
--
doc/src/sgml/protocol.sgml  | 116 
src/backend/postmaster/postmaster.c |  58 --
2 files changed, 158 insertions(+), 16 deletions(-)



pgsql: pgbench: fix stats reporting when some transactions are skipped.

2017-11-21 Thread Tom Lane
pgbench: fix stats reporting when some transactions are skipped.

pgbench can skip some transactions when both -R and -L options are used.
Previously, this resulted in slightly silly statistics both in progress
reports and final output, because the skipped transactions were counted
as executed for TPS and related stats.  Discount skipped xacts in TPS
numbers, and also when figuring the percentage of xacts exceeding the
latency limit.

Also, don't print per-script skipped-transaction counts when there is
only one script.  That's redundant with the overall count, and it's
inconsistent with the fact that we don't print other per-script stats
when there's only one script.  Clean up some unnecessary interactions
between what should be independent options that were due to that
decision.

While at it, avoid division-by-zero in cases where no transactions were
executed.  While on modern platforms this would generally result in
printing "NaN" rather than a crash, that isn't spelled consistently
across platforms and it would confuse many people.  Skip the relevant
output entirely when practical, else print zeroes.

Fabien Coelho, reviewed by Steve Singer, additional hacking by me

Discussion: https://postgr.es/m/[email protected]

Branch
--
master

Details
---
https://git.postgresql.org/pg/commitdiff/16827d442448d1935ed644e944a4cb8213345351

Modified Files
--
doc/src/sgml/ref/pgbench.sgml|   3 +-
src/bin/pgbench/pgbench.c| 104 ---
src/bin/pgbench/t/001_pgbench_with_server.pl |   2 +-
3 files changed, 65 insertions(+), 44 deletions(-)



pgsql: Doc: fix broken markup.

2017-11-21 Thread Tom Lane
Doc: fix broken markup.

Branch
--
master

Details
---
https://git.postgresql.org/pg/commitdiff/41761265e88f09fba4028352b8e2be82d049cedc

Modified Files
--
doc/src/sgml/ref/create_table.sgml | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)



Re: [HACKERS] [COMMITTERS] pgsql: Fix freezing of a dead HOT-updated tuple

2017-11-21 Thread Michael Paquier
On Tue, Nov 21, 2017 at 4:18 AM, Andres Freund  wrote:
> On 2017-11-13 19:03:41 -0800, Andres Freund wrote:
>> On 2017-11-03 07:53:30 -0700, Andres Freund wrote:
>> > Here's that patch.  I've stared at this some, and Robert did too. Robert
>> > mentioned that the commit message might need some polish and I'm not
>> > 100% sure about the error message texts yet.
>> >
>> > I'm not yet convinced that the new elog in vacuumlazy can never trigger
>> > - but I also don't think we want to actually freeze the tuple in that
>> > case.
>>
>> I'm fairly sure it could be triggered, therefore I've rewritten that.
>>
>> I've played around quite some with the attached patch. So far, after
>> applying the second patch, neither VACUUM nor VACUUM FULL / CLUSTER make
>> the situation worse for already existing corruption. HOT pruning can
>> change the exact appearance of existing corruption a bit, but I don't
>> think it can make the corruption meaningfully worse.  It's a bit
>> annoying and scary to add so many checks to backbranches but it kinda
>> seems required.  The error message texts aren't perfect, but these are
>> "should never be hit" type elog()s so I'm not too worried about that.
>>
>> Please review!
>
> Ping? Alvaro, it'd be good to get some input here.

Note that I will be able to jump on the ship after being released from
commit fest duties. This is likely a multi-day task for testing and
looking at it, and I am not the most knowledgeable human being with
this code.
-- 
Michael



pgsql: Set es_output_cid in replication worker

2017-11-21 Thread Simon Riggs
Set es_output_cid in replication worker

Allows triggers to operate correctly

Author: Petr Jelinek 
Reported-by: Konstantin Knizhnik 

Branch
--
master

Details
---
https://git.postgresql.org/pg/commitdiff/7e17a6889a4441c2cebca2dd47f4170ff8dc5de2

Modified Files
--
src/backend/replication/logical/worker.c | 2 ++
1 file changed, 2 insertions(+)