Robert Haas wrote:
On Thu, Nov 11, 2010 at 10:13 PM, Bruce Momjian br...@momjian.us wrote:
Robert Haas wrote:
On Thu, Oct 28, 2010 at 1:13 AM, Josh Berkus j...@agliodbs.com wrote:
I sort of agree with you that the current checkpoint_segments
parameter is a bit hard to tune, at least
Robert Haas wrote:
On Thu, Oct 28, 2010 at 1:13 AM, Josh Berkus j...@agliodbs.com wrote:
I sort of agree with you that the current checkpoint_segments
parameter is a bit hard to tune, at least if your goal is to control
the amount of disk space that will be used by WAL files. ?But I'm not
Robert Haas wrote:
On Wed, Oct 27, 2010 at 3:53 AM, Simon Riggs si...@2ndquadrant.com wrote:
On Tue, 2010-10-26 at 22:03 -0400, Robert Haas wrote:
On Tue, Oct 26, 2010 at 9:59 PM, Josh Berkus j...@agliodbs.com wrote:
If you set wal_keep_segments=0, archive_mode=on, and
On Thu, Nov 11, 2010 at 10:13 PM, Bruce Momjian br...@momjian.us wrote:
Robert Haas wrote:
On Thu, Oct 28, 2010 at 1:13 AM, Josh Berkus j...@agliodbs.com wrote:
I sort of agree with you that the current checkpoint_segments
parameter is a bit hard to tune, at least if your goal is to
Hannu Krosing ha...@2ndquadrant.com writes:
To make pg_basebackup.py self-sufficient it should also open 2nd
connection to the same master and make sure that all WAL files are
copied for the duration of base copy.
Excellent idea, will make that happen soon'ish.
Regards,
--
Dimitri Fontaine
On Sat, 2010-11-06 at 18:02 +0100, Dimitri Fontaine wrote:
Hannu Krosing ha...@2ndquadrant.com writes:
To make pg_basebackup.py self-sufficient it should also open 2nd
connection to the same master and make sure that all WAL files are
copied for the duration of base copy.
Excellent idea,
Hannu Krosing ha...@2ndquadrant.com writes:
To make pg_basebackup.py self-sufficient it should also open 2nd
connection to the same master and make sure that all WAL files are
copied for the duration of base copy.
Done now, please have a look and try it if possible:
On Tue, 2010-10-19 at 10:31 +0200, Dimitri Fontaine wrote:
4. I can start a new replica off the master by running a single command-line
utility on the standby and giving it connection information to the master.
Using this connection, it should be able to start a backup snapshot, copy
the
Josh Berkus j...@agliodbs.com writes:
You have to put the WAL files *somewhere* while you do the base backup.
Agreed. That's why I like the idea of having a
max_wal_size/min_wal_time instead of keep_wal_segments or
checkpoint_segments. It's relatively simple for a DBA to know how much
disk
On Thu, Oct 28, 2010 at 1:13 AM, Josh Berkus j...@agliodbs.com wrote:
I sort of agree with you that the current checkpoint_segments
parameter is a bit hard to tune, at least if your goal is to control
the amount of disk space that will be used by WAL files. But I'm not
sure your proposal is
On Tue, 2010-10-26 at 22:03 -0400, Robert Haas wrote:
On Tue, Oct 26, 2010 at 9:59 PM, Josh Berkus j...@agliodbs.com wrote:
If you set wal_keep_segments=0, archive_mode=on, and
archive_command=something, you might run out of disk space.
If you set wal_keep_segments=-1, you might run out
It is the same to the user either way. In either case you have to
change some settings and restart the master.
Except that changing wal_keep_segments doesn't require restarting the master.
Our docs say that it does:
This parameter can only be set in the postgresql.conf file or on the
Josh Berkus j...@agliodbs.com wrote:
Except that changing wal_keep_segments doesn't require restarting
the master.
Our docs say that it does:
This parameter can only be set in the postgresql.conf file or on
the server command line.
That sounds as though a reload would do it; I don't see
Kevin Grittner kevin.gritt...@wicourts.gov writes:
Josh Berkus j...@agliodbs.com wrote:
Except that changing wal_keep_segments doesn't require restarting
the master.
Our docs say that it does:
This parameter can only be set in the postgresql.conf file or on
the server command line.
That
You have to put the WAL files *somewhere* while you do the base backup.
PostgreSQL can't itself work out where that is, nor can it work out
ahead of time how big it will need to be, since it is up to you how you
do your base backup. Setting a parameter to -1 doesn't make the problem
go away,
On Wed, Oct 27, 2010 at 3:53 AM, Simon Riggs si...@2ndquadrant.com wrote:
On Tue, 2010-10-26 at 22:03 -0400, Robert Haas wrote:
On Tue, Oct 26, 2010 at 9:59 PM, Josh Berkus j...@agliodbs.com wrote:
If you set wal_keep_segments=0, archive_mode=on, and
archive_command=something, you might
On Wed, Oct 27, 2010 at 5:01 PM, Josh Berkus j...@agliodbs.com wrote:
You have to put the WAL files *somewhere* while you do the base backup.
PostgreSQL can't itself work out where that is, nor can it work out
ahead of time how big it will need to be, since it is up to you how you
do your
I sort of agree with you that the current checkpoint_segments
parameter is a bit hard to tune, at least if your goal is to control
the amount of disk space that will be used by WAL files. But I'm not
sure your proposal is better. Instead of having a complicated formula
for predicting how much
What happens if max_wal_size is less than checkpoint_segments?
Currently a checkpoint tries to leave WAL files which were generated
from the prior ckpt start to current ckpt end. Because those WAL files
are required for crash recovery. But we should delete some of them
according to
On Thu, 2010-10-21 at 20:57 -0400, Robert Haas wrote:
Very true. But the lack of a -1 setting for wal_keep_segments means
that if you would like to take a backup without archiving, you must
set wal_keep_segments to a value greater than or equal to the rate at
which you generate WAL segments
On Tue, Oct 26, 2010 at 8:27 AM, Simon Riggs si...@2ndquadrant.com wrote:
On Thu, 2010-10-21 at 20:57 -0400, Robert Haas wrote:
Very true. But the lack of a -1 setting for wal_keep_segments means
that if you would like to take a backup without archiving, you must
set wal_keep_segments to a
If you set wal_keep_segments=0, archive_mode=on, and
archive_command=something, you might run out of disk space.
If you set wal_keep_segments=-1, you might run out of disk space.
Are you any more screwed in the second case than you are in the first
case?
It is the same to the user
On Tue, Oct 26, 2010 at 9:59 PM, Josh Berkus j...@agliodbs.com wrote:
If you set wal_keep_segments=0, archive_mode=on, and
archive_command=something, you might run out of disk space.
If you set wal_keep_segments=-1, you might run out of disk space.
Are you any more screwed in the second
On Sat, Oct 23, 2010 at 2:34 AM, Josh Berkus j...@agliodbs.com wrote:
When we get close enough to max_wal_size (we'll need a couple segments of
leeway, I think), we start recycling WAL segments even if they are less that
min_wal_time old.
What happens if max_wal_size is less than
Please see
http://archives.postgresql.org/pgsql-docs/2010-10/msg00038.php
Ye gods and little fishes!
You really want to talk arcane formulas. I've re-read that
three times, and am still not sure that I could tell someone
definitively how much disk space WAL needs for a given group of
Josh Berkus j...@agliodbs.com writes:
What would be sensible for DBAs is to have two settings:
max_wal_size
min_wal_size
[ scratches head... ] What's the functional effect of min_wal_size, exactly?
Even better would be to replace min_wal_size with min_wal_time, which
would set a time
max_wal_size
min_wal_size
[ scratches head... ] What's the functional effect of min_wal_size, exactly?
Replaces wal_keep_segments. The rename is to make the GUCs obviously
symmetrical, and to make it clear that the *meaning* of the variable has
changed.
Even better would be to replace
On 19/10/10 13:16, Josh Berkus wrote:
Robert asked me to write this up, so here it is.
It is critical that we make replication easier to set up, administrate
and monitor than it currently is. In my conversations with people,
this is more important to our users and the adoption of PostgreSQL
Josh Berkus wrote:
Greg,
The way things stand you *always* need archived logs. Even if you have
streaming set up it might try to use archived logs if it falls too far
behind.
Actually, you don't. If you're willing to accept possible
desynchronization and recloning of the standbys,
On Thu, Oct 21, 2010 at 8:22 PM, Bruce Momjian br...@momjian.us wrote:
Josh Berkus wrote:
Greg,
The way things stand you *always* need archived logs. Even if you have
streaming set up it might try to use archived logs if it falls too far
behind.
Actually, you don't. If you're willing
Agreed, but as a reality check: when I proposed that wal_keep_segments
= -1 would keep all WAL segments (for use while the file system was
being backed up), I was told administrators shoud compute how much free
disk space they had. Obviously easy of use is not our #1 priority.
Depends.
On Thu, Oct 21, 2010 at 5:46 PM, Josh Berkus j...@agliodbs.com wrote:
Agreed, but as a reality check: when I proposed that wal_keep_segments
= -1 would keep all WAL segments (for use while the file system was
being backed up), I was told administrators shoud compute how much free
disk space
On Thu, Oct 21, 2010 at 8:52 PM, Greg Stark gsst...@mit.edu wrote:
On Thu, Oct 21, 2010 at 5:46 PM, Josh Berkus j...@agliodbs.com wrote:
Agreed, but as a reality check: when I proposed that wal_keep_segments
= -1 would keep all WAL segments (for use while the file system was
being backed
Robert Haas wrote:
However the user-unfriendliness isn't the fact that administrators
need to determine how much disk they're willing to dedicate to
Postgres. The user-unfriendliness is that they then have to specify
this in terms of WAL log files and also have to know that we sometimes
Very true. But the lack of a -1 setting for wal_keep_segments means
that if you would like to take a backup without archiving, you must
set wal_keep_segments to a value greater than or equal to the rate at
which you generate WAL segments multiplied by the time it takes you to
run a backup.
On Thu, Oct 21, 2010 at 9:09 PM, Josh Berkus j...@agliodbs.com wrote:
Very true. But the lack of a -1 setting for wal_keep_segments means
that if you would like to take a backup without archiving, you must
set wal_keep_segments to a value greater than or equal to the rate at
which you
I think it's pretty well explained in the fine manual.
http://www.postgresql.org/docs/current/static/runtime-config-wal.html#GUC-WAL-KEEP-SEGMENTS
Nope. No relationship to checkpoint_segments is explained there. Try
again?
If checkpoint_segments were a hard limit, then we could let
On Thu, Oct 21, 2010 at 10:03 PM, Josh Berkus j...@agliodbs.com wrote:
I think it's pretty well explained in the fine manual.
http://www.postgresql.org/docs/current/static/runtime-config-wal.html#GUC-WAL-KEEP-SEGMENTS
Nope. No relationship to checkpoint_segments is explained there. Try
On Fri, Oct 22, 2010 at 11:03 AM, Josh Berkus j...@agliodbs.com wrote:
I think it's pretty well explained in the fine manual.
http://www.postgresql.org/docs/current/static/runtime-config-wal.html#GUC-WAL-KEEP-SEGMENTS
Nope. No relationship to checkpoint_segments is explained there. Try
Josh Berkus j...@agliodbs.com writes:
Well, one thing to be addressed is separating the PITR functionality from
replication. PITR needs a lot of features -- timelines, recovery stop
points, etc. -- which replication doesn't need or want. I think that
focussing on streaming replication
On Tue, Oct 19, 2010 at 11:16 AM, Greg Smith g...@2ndquadrant.com wrote:
Josh Berkus wrote:
It is critical that we make replication easier to set up, administrate and
monitor than it currently is. In my conversations with people, this is more
important to our users and the adoption of
Hi,
Josh Berkus j...@agliodbs.com writes:
It is critical that we make replication easier to set up, administrate and
monitor than it currently is. In my conversations with people, this is more
important to our users and the adoption of PostgreSQL than synchronous
replication is.
I want to
Josh Berkus wrote:
It is critical that we make replication easier to set up, administrate
and monitor than it currently is. In my conversations with people,
this is more important to our users and the adoption of PostgreSQL
than synchronous replication is.
You should enjoy one of the
You should enjoy one of the patches we're furiously working on then,
which is aiming at some of the administration and monitoring pieces
here.
Great, glad to hear it! Would you be willing to go into detail?
I have my own grand vision of how easy replication should be to
setup too.
So,
Josh Berkus wrote:
*shrug*. Robert asked me to write it up for the list based on the
discussions around synch rep. Now you're going to bash me for doing so?
Sorry, next time I'll make sure to bash Robert too. I don't have any
problems with the basic ideas you're proposing, just concerns
Dimitri, Greg,
I want to say a big big +1 here. The way replication and PITR setup are
implemented now are a very good prototype, it's time to consolidate and
get to something usable by normal people, as opposed to PostgreSQL full
time geeks.
Well, one thing to be addressed is separating the
On Tue, Oct 19, 2010 at 9:16 AM, Josh Berkus j...@agliodbs.com wrote:
Well, one thing to be addressed is separating the PITR functionality from
replication. PITR needs a lot of features -- timelines, recovery stop
points, etc. -- which replication doesn't need or want. I think that
focussing
Greg,
The way things stand you *always* need archived logs. Even if you have
streaming set up it might try to use archived logs if it falls too far
behind.
Actually, you don't. If you're willing to accept possible
desynchronization and recloning of the standbys, then you can skip the
archive
Robert asked me to write this up, so here it is.
It is critical that we make replication easier to set up, administrate
and monitor than it currently is. In my conversations with people, this
is more important to our users and the adoption of PostgreSQL than
synchronous replication is.
On 19 October 2010 11:16, Josh Berkus j...@agliodbs.com wrote:
4. I can start a new replica off the master by running a single command-line
utility on the standby and giving it connection information to the master.
Using this connection, it should be able to start a backup snapshot, copy
the
50 matches
Mail list logo
