Re: register/unregister standby Re: [HACKERS] Synchronous replication
Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes: Hmm, that's clever. I was thinking that you'd initialize the standby from an existing backup, and in that context the standby would not need to connect to the master except via the replication connection. To take a base backup, you'll need not only that but also access to the filesystem in the master, ie. shell access. In fact you don't need shell access here, it's rather easy to stream the base backup from the libpq connection, as implemented here : http://github.com/dimitri/pg_basebackup There's been some talk of being able to stream a base backup over the replication connection too, which would be extremely handy. Yes please ! :) -- dim -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: register/unregister standby Re: [HACKERS] Synchronous replication
On Thu, Sep 2, 2010 at 6:41 PM, Dimitri Fontaine dfonta...@hi-media.com wrote: In fact you don't need shell access here, it's rather easy to stream the base backup from the libpq connection, as implemented here : http://github.com/dimitri/pg_basebackup There's been some talk of being able to stream a base backup over the replication connection too, which would be extremely handy. Yes please ! :) One issue of the base backup function is that the operation will be a long transaction. So, non-transactional special commands, as like as VACUUM, would be better in terms of performance. For example, CREATE or ALTER REPLICATION. Of course, function-based approach is more flexible and less invasive to the SQL parser. There are trade-offs. -- Itagaki Takahiro -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: register/unregister standby Re: [HACKERS] Synchronous replication
On 30 August 2010 13:14, Fujii Masao masao.fu...@gmail.com wrote:
I think that the advantage of registering standbys is that we can
specify which WAL files the master has to keep for the upcoming
standby. IMO, it's usually called together with pg_start_backup
as follows:
SELECT register_standby('foo', pg_start_backup())
This requests the master keep to all the WAL files following the
backup starting location which pg_start_backup returns. Now we
can do that by using wal_keep_segments, but it's not easy to set
because it's difficult to predict how many WAL files the standby
will require.
+1 I don't like the idea of having to guess how many WAL files you
think you'll need to keep around.
And if these standby instances have to register, could there be a view
to list subscriber information?
--
Thom Brown
Twitter: @darkixion
IRC (freenode): dark_ixion
Registered Linux user: #516935
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: register/unregister standby Re: [HACKERS] Synchronous replication
Itagaki Takahiro itagaki.takah...@gmail.com writes: http://github.com/dimitri/pg_basebackup There's been some talk of being able to stream a base backup over the replication connection too, which would be extremely handy. Yes please ! :) One issue of the base backup function is that the operation will be a long transaction. So, non-transactional special commands, as like as VACUUM, would be better in terms of performance. For example, CREATE or ALTER REPLICATION. Well, you still need to stream the data to the client in a format it will understand. Would that be the plan of your command proposal? Of course, function-based approach is more flexible and less invasive to the SQL parser. There are trade-offs. Well that was easier for a proof-of-concept at least. -- Dimitri Fontaine PostgreSQL DBA, Architecte -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: register/unregister standby Re: [HACKERS] Synchronous replication
On Thu, Sep 2, 2010 at 7:54 PM, Dimitri Fontaine dfonta...@hi-media.com wrote: One issue of the base backup function is that the operation will be a long transaction. So, non-transactional special commands, as like as VACUUM, would be better in terms of performance. For example, CREATE or ALTER REPLICATION. Well, you still need to stream the data to the client in a format it will understand. True, but using libpq connection might be not the most important thing. The most simplest proof-of-concept might be system(rsync) in the function ;-) Would that be the plan of your command proposal? What I meant was function-based maintenance does not work well in some cases. I heard before pg_start_backup( no-fast-checkpoint ) caused table bloating problem because it was a long transaction for 20+ minutes. The backup function would have the similar issue. -- Itagaki Takahiro -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: register/unregister standby Re: [HACKERS] Synchronous replication
On 30/08/10 15:14, Fujii Masao wrote:
I think that the advantage of registering standbys is that we can
specify which WAL files the master has to keep for the upcoming
standby. IMO, it's usually called together with pg_start_backup
as follows:
SELECT register_standby('foo', pg_start_backup())
This requests the master keep to all the WAL files following the
backup starting location which pg_start_backup returns.
Hmm, that's clever. I was thinking that you'd initialize the standby
from an existing backup, and in that context the standby would not need
to connect to the master except via the replication connection. To take
a base backup, you'll need not only that but also access to the
filesystem in the master, ie. shell access.
There's been some talk of being able to stream a base backup over the
replication connection too, which would be extremely handy. And that
makes my point even stronger that registering a standby should be
possible via the replication connection.
Of course, you could well expose the functionality as both a built-in
function and a command in replication mode, so this detail isn't really
that important right now.
--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
register/unregister standby Re: [HACKERS] Synchronous replication
On Tue, Aug 10, 2010 at 5:58 AM, Heikki Linnakangas
heikki.linnakan...@enterprisedb.com wrote:
On 05/08/10 17:14, Fujii Masao wrote:
I'm thinking to make users register and unregister each standbys via SQL
functions like register_standby() and unregister_standby():
The register/unregister facility should be accessible from the streaming
replication connection, so that you don't need to connect to any particular
database in addition to the streaming connection.
Probably I've not understood your point correctly yet.
I think that the advantage of registering standbys is that we can
specify which WAL files the master has to keep for the upcoming
standby. IMO, it's usually called together with pg_start_backup
as follows:
SELECT register_standby('foo', pg_start_backup())
This requests the master keep to all the WAL files following the
backup starting location which pg_start_backup returns. Now we
can do that by using wal_keep_segments, but it's not easy to set
because it's difficult to predict how many WAL files the standby
will require.
So I've thought that the register/unregister facility should be
used from the normal client connection. Why do you think it should
be accessible from the SR connection?
Regards,
--
Fujii Masao
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
