Re: [HACKERS] backward incompatible pg_basebackup and pg_receivexlog
On 19.03.2013 13:49, Magnus Hagander wrote:
On Tue, Mar 19, 2013 at 11:39 AM, Heikki Linnakangas
hlinnakan...@vmware.com wrote:
On 19.03.2013 04:42, Peter Eisentraut wrote:
Also, using the old tools against new server versions either behaves
funny or silently appears to work, both of which might be a problem.
Hmm, it would be good to fix that. I wonder how, though. The most
straightforward way would be to add an explicit version check in the
clients, in backbranches. That would give a nice error message, but that
would only help with new minor versions.
Still better to do it in a backbranch, than not at all. At least we
are then nicer to the ones that do keep up with upgrades, which we
recommend they do...
Ok, here are patches for 9.1, 9.2 and master, to add explicit version
checks. Each branch has its own quirks. A 9.1 client should still work
with a 9.2 server, because we don't want to break things in a minor
version that used to accidentally work, even if it was never explicitly
supported. In master, I tweaked pg_basebackup so that it still works
with older servers if you don't use the -X stream option. The changes
to the streaming protocol only affected -X stream.
This doesn't yet add the streaming protocol version number that was
discussed.
- Heikki
commit aa5d7d58ba40187bd8c6a2216bfd24514da78003
Author: Heikki Linnakangas heikki.linnakan...@iki.fi
Date: Mon Mar 25 11:03:20 2013 +0200
Add a server version check to pg_basebackup and pg_receivexlog.
These programs don't work against 9.0 or earlier servers, so check that when
the connection is made. That's better than a cryptic error message you got
before.
Also, these programs won't work with a 9.3 server, because the WAL streaming
protocol was changed in a non-backwards-compatible way. As a general rule,
we don't make any guarantee that an old client will work with a new server,
so check that. However, allow a 9.1 client to connect to a 9.2 server, to
avoid breaking environments that currently work; a 9.1 client happens to
work with a 9.2 server, even though we didn't make any great effort to
ensure that.
This patch is for the 9.1 and 9.2 branches, I'll commit a similar patch to
master later. Although this isn't a critical bug fix, it seems safe enough
to back-patch. The error message you got when connecting to a 9.3devel
server without this patch was cryptic enough to warrant backpatching.
diff --git a/src/bin/pg_basebackup/pg_basebackup.c b/src/bin/pg_basebackup/pg_basebackup.c
index 472df3a..d68e742 100644
--- a/src/bin/pg_basebackup/pg_basebackup.c
+++ b/src/bin/pg_basebackup/pg_basebackup.c
@@ -816,6 +816,9 @@ BaseBackup(void)
int i;
char xlogstart[64];
char xlogend[64];
+ int minServerMajor,
+maxServerMajor;
+ int serverMajor;
/*
* Connect in replication mode to the server
@@ -823,6 +826,24 @@ BaseBackup(void)
conn = GetConnection();
/*
+ * Check server version. BASE_BACKUP command was introduced in 9.1, so
+ * we can't work with servers older than 9.1. We don't officially support
+ * servers newer than the client, but the 9.1 version happens to work with
+ * a 9.2 server. This version check was added to 9.1 branch in a minor
+ * release, so allow connecting to a 9.2 server, to avoid breaking
+ * environments that worked before this version check was added.
+ */
+ minServerMajor = 901;
+ maxServerMajor = 902;
+ serverMajor = PQserverVersion(conn) / 100;
+ if (serverMajor minServerMajor || serverMajor maxServerMajor)
+ {
+ fprintf(stderr, _(%s: unsupported server version %s\n),
+progname, PQparameterStatus(conn, server_version));
+ disconnect_and_exit(1);
+ }
+
+ /*
* Start the actual backup
*/
PQescapeStringConn(conn, escaped_label, label, sizeof(escaped_label), i);
commit 6980497f7d7f4d17b918a7a433aa904943a4bb97
Author: Heikki Linnakangas heikki.linnakan...@iki.fi
Date: Mon Mar 25 11:02:55 2013 +0200
Add a server version check to pg_basebackup and pg_receivexlog.
These programs don't work against 9.0 or earlier servers, so check that when
the connection is made. That's better than a cryptic error message you got
before.
Also, these programs won't work with a 9.3 server, because the WAL streaming
protocol was changed in a non-backwards-compatible way. As a general rule,
we don't make any guarantee that an old client will work with a new server,
so check that. However, allow a 9.1 client to connect to a 9.2 server, to
avoid breaking environments that currently work; a 9.1 client happens to
work with a 9.2 server, even though we didn't make any great effort to
ensure that.
This patch is for the 9.1 and 9.2 branches, I'll commit a similar patch to
master later. Although this isn't a critical bug fix, it seems safe enough
to back-patch. The error message you got when connecting to a 9.3devel
server without this patch
Re: [HACKERS] backward incompatible pg_basebackup and pg_receivexlog
On 25.03.2013 11:23, Heikki Linnakangas wrote: On 19.03.2013 13:49, Magnus Hagander wrote: On Tue, Mar 19, 2013 at 11:39 AM, Heikki Linnakangas hlinnakan...@vmware.com wrote: On 19.03.2013 04:42, Peter Eisentraut wrote: Also, using the old tools against new server versions either behaves funny or silently appears to work, both of which might be a problem. Hmm, it would be good to fix that. I wonder how, though. The most straightforward way would be to add an explicit version check in the clients, in backbranches. That would give a nice error message, but that would only help with new minor versions. Still better to do it in a backbranch, than not at all. At least we are then nicer to the ones that do keep up with upgrades, which we recommend they do... Ok, here are patches for 9.1, 9.2 and master, to add explicit version checks. Each branch has its own quirks. A 9.1 client should still work with a 9.2 server, because we don't want to break things in a minor version that used to accidentally work, even if it was never explicitly supported. In master, I tweaked pg_basebackup so that it still works with older servers if you don't use the -X stream option. The changes to the streaming protocol only affected -X stream. This doesn't yet add the streaming protocol version number that was discussed. Committed this.. Will work on the additional version number. - Heikki -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] backward incompatible pg_basebackup and pg_receivexlog
On 19.03.2013 04:42, Peter Eisentraut wrote:
pg_basebackup and pg_receivexlog from 9.3 won't work with earlier
servers anymore. I wonder if this has been fully thought through. We
have put in a lot of effort to make client programs compatible with many
server versions as well as keeping the client/server protocol compatible
across many versions. Both of these assumptions are now being broken,
which will result in all kinds of annoyances.
It seems to me that these tools could probably be enhanced to understand
both old and new formats.
Yes, this was discussed, and the consensus was to break
backwards-compatibility in 9.3, so that we can clean up the protocol to
be architecture-independent. That makes it easier to write portable
clients, from 9.3 onwards. See the thread ending at
http://www.postgresql.org/message-id/4fe2279c.2070...@enterprisedb.com.
Also, using the old tools against new server versions either behaves
funny or silently appears to work, both of which might be a problem.
Hmm, it would be good to fix that. I wonder how, though. The most
straightforward way would be to add an explicit version check in the
clients, in backbranches. That would give a nice error message, but that
would only help with new minor versions.
I think if we are documenting the replication protocol as part of the
frontend/backend protocol and are exposing client tools that use it,
changes need to be done with the same rigor as other protocol changes.
Agreed. The plan is that we're going to be more careful with it from now on.
As far as I can tell, there is no separate version number for the
replication part of the protocol, so either there needs to be one or the
protocol as a whole needs to be updated.
Good point.
I propose that we add a version number, and call the 9.3 version version
2. Let's add a new field to the result set of the IDENTIFY_SYSTEM
command for the replication protocol version number. The version number
should be bumped if the replication protocol is changed in a
non-backwards-compatible way. That includes changes to the messages sent
in the COPY-both mode, after the START_REPLICATION command. If we just
add new commands, there's no need to bump the version; a client can
still check the server version number to determine if a command exists
or not.
We could also try to support old client + new server combination to some
extent by future-proofing the protocol a bit. We could specify that the
client should ignore any message types that it does not understand, and
also add a header length field to the WalData message ('w'), so that we
can add new header fields to it that old clients can just ignore. That
way we can keep the protocol version unchanged if we just add some
optional stuff to it. I'm not sure how useful that is in practice
though; it's not unreasonable that you must upgrade to the latest
client, as long as the new client works with old server versions.
- Heikki
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] backward incompatible pg_basebackup and pg_receivexlog
On Tue, Mar 19, 2013 at 11:39 AM, Heikki Linnakangas
hlinnakan...@vmware.com wrote:
On 19.03.2013 04:42, Peter Eisentraut wrote:
pg_basebackup and pg_receivexlog from 9.3 won't work with earlier
servers anymore. I wonder if this has been fully thought through. We
have put in a lot of effort to make client programs compatible with many
server versions as well as keeping the client/server protocol compatible
across many versions. Both of these assumptions are now being broken,
which will result in all kinds of annoyances.
It seems to me that these tools could probably be enhanced to understand
both old and new formats.
Yes, this was discussed, and the consensus was to break
backwards-compatibility in 9.3, so that we can clean up the protocol to be
architecture-independent. That makes it easier to write portable clients,
from 9.3 onwards. See the thread ending at
http://www.postgresql.org/message-id/4fe2279c.2070...@enterprisedb.com.
Also, using the old tools against new server versions either behaves
funny or silently appears to work, both of which might be a problem.
Hmm, it would be good to fix that. I wonder how, though. The most
straightforward way would be to add an explicit version check in the
clients, in backbranches. That would give a nice error message, but that
would only help with new minor versions.
Still better to do it in a backbranch, than not at all. At least we
are then nicer to the ones that do keep up with upgrades, which we
recommend they do...
I think if we are documenting the replication protocol as part of the
frontend/backend protocol and are exposing client tools that use it,
changes need to be done with the same rigor as other protocol changes.
Agreed. The plan is that we're going to be more careful with it from now on.
As far as I can tell, there is no separate version number for the
replication part of the protocol, so either there needs to be one or the
protocol as a whole needs to be updated.
Good point.
I propose that we add a version number, and call the 9.3 version version 2.
Let's add a new field to the result set of the IDENTIFY_SYSTEM command for
the replication protocol version number. The version number should be bumped
if the replication protocol is changed in a non-backwards-compatible way.
+1.
That includes changes to the messages sent in the COPY-both mode, after the
START_REPLICATION command. If we just add new commands, there's no need to
bump the version; a client can still check the server version number to
determine if a command exists or not.
Sounds good.
We could also try to support old client + new server combination to some
extent by future-proofing the protocol a bit. We could specify that the
client should ignore any message types that it does not understand, and also
add a header length field to the WalData message ('w'), so that we can add
new header fields to it that old clients can just ignore. That way we can
keep the protocol version unchanged if we just add some optional stuff to
it. I'm not sure how useful that is in practice though; it's not
unreasonable that you must upgrade to the latest client, as long as the new
client works with old server versions.
I think that's quite reasonable, as long as we detect it, and can give
a nice error message telling the user how to deal with it.
--
Magnus Hagander
Me: http://www.hagander.net/
Work: http://www.redpill-linpro.com/
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] backward incompatible pg_basebackup and pg_receivexlog
pg_basebackup and pg_receivexlog from 9.3 won't work with earlier servers anymore. I wonder if this has been fully thought through. We have put in a lot of effort to make client programs compatible with many server versions as well as keeping the client/server protocol compatible across many versions. Both of these assumptions are now being broken, which will result in all kinds of annoyances. It seems to me that these tools could probably be enhanced to understand both old and new formats. Also, using the old tools against new server versions either behaves funny or silently appears to work, both of which might be a problem. I think if we are documenting the replication protocol as part of the frontend/backend protocol and are exposing client tools that use it, changes need to be done with the same rigor as other protocol changes. As far as I can tell, there is no separate version number for the replication part of the protocol, so either there needs to be one or the protocol as a whole needs to be updated. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
