Is there any real use-case for not setting wal_buffers to -1 these days?
Or should we just remove it and use the -1 behaviour at all times?
IIRC we discussed not keeping it at all when the autotune behavior was
introduced, but said we wanted to keep it just in case. If we're not
ready to remove
On 17 January 2014 13:01, Magnus Hagander mag...@hagander.net wrote:
Is there any real use-case for not setting wal_buffers to -1 these days?
Or should we just remove it and use the -1 behaviour at all times?
IIRC we discussed not keeping it at all when the autotune behavior was
introduced,
Hi,
On 2014-01-17 14:01:56 +0100, Magnus Hagander wrote:
Is there any real use-case for not setting wal_buffers to -1 these days?
Or should we just remove it and use the -1 behaviour at all times?
I have seen improvements by setting it larger than the max -1 one
value. Also, for some
On Fri, Jan 17, 2014 at 2:07 PM, Thom Brown t...@linux.com wrote:
On 17 January 2014 13:01, Magnus Hagander mag...@hagander.net wrote:
Is there any real use-case for not setting wal_buffers to -1 these days?
Or should we just remove it and use the -1 behaviour at all times?
IIRC we
On 17 January 2014 13:20, Magnus Hagander mag...@hagander.net wrote:
On Fri, Jan 17, 2014 at 2:07 PM, Thom Brown t...@linux.com wrote:
On 17 January 2014 13:01, Magnus Hagander mag...@hagander.net wrote:
Is there any real use-case for not setting wal_buffers to -1 these days?
Or should we
On Fri, Jan 17, 2014 at 8:20 AM, Magnus Hagander mag...@hagander.net wrote:
Robert Haas reported that setting it to 32MB can yield a considerable
performance benefit:
http://www.postgresql.org/message-id/CA+TgmobgMv_aaakLoasBt=5iYfi=kdcOUz0X9TdYe0c2SZ=2...@mail.gmail.com
In that case,
On Wed, Aug 29, 2012 at 10:25 PM, Peter Geoghegan pe...@2ndquadrant.com wrote:
On 19 February 2012 05:24, Robert Haas robertmh...@gmail.com wrote:
I have attached tps scatterplots. The obvious conclusion appears to
be that, with only 16MB of wal_buffers, the buffer wraps around with
some
On Thursday, August 30, 2012 7:14 PM Robert Haas wrote:
On Wed, Aug 29, 2012 at 10:25 PM, Peter Geoghegan pe...@2ndquadrant.com
wrote:
On 19 February 2012 05:24, Robert Haas robertmh...@gmail.com wrote:
I have attached tps scatterplots. The obvious conclusion appears to
be that, with only 16MB
On Tuesday, August 28, 2012 9:33 PM Bruce Momjian wrote:
On Tue, Aug 28, 2012 at 09:40:33AM +0530, Amit Kapila wrote:
From: pgsql-hackers-ow...@postgresql.org
[mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Bruce Momjian
Added to TODO:
Allow reporting of stalls due to
On 19 February 2012 05:24, Robert Haas robertmh...@gmail.com wrote:
I have attached tps scatterplots. The obvious conclusion appears to
be that, with only 16MB of wal_buffers, the buffer wraps around with
some regularity: we can't insert more WAL because the buffer we need
to use still
On Tue, Aug 28, 2012 at 09:40:33AM +0530, Amit Kapila wrote:
From: pgsql-hackers-ow...@postgresql.org
[mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Bruce Momjian
Added to TODO:
Allow reporting of stalls due to wal_buffer wrap-around
Added to TODO:
Allow reporting of stalls due to wal_buffer wrap-around
http://archives.postgresql.org/pgsql-hackers/2012-02/msg00826.php
---
On Sun, Feb 19, 2012 at 12:24:12AM -0500, Robert
From: pgsql-hackers-ow...@postgresql.org
[mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Bruce Momjian
Added to TODO:
Allow reporting of stalls due to wal_buffer wrap-around
http://archives.postgresql.org/pgsql-hackers/2012-02/msg00826.php
Isn't this indicates that
On Mon, Mar 12, 2012 at 7:16 AM, Robert Haas robertmh...@gmail.com wrote:
On Sun, Mar 11, 2012 at 11:51 PM, Fujii Masao masao.fu...@gmail.com wrote:
On Sun, Mar 11, 2012 at 12:55 PM, Robert Haas robertmh...@gmail.com wrote:
I've finally been able to run some more tests of the effect of
On Tue, Mar 13, 2012 at 11:18 PM, Robert Haas robertmh...@gmail.com wrote:
On Tue, Mar 13, 2012 at 6:44 PM, Josh Berkus j...@agliodbs.com wrote:
That's a speedup of nearly a factor of two, so clearly fsync-related
stalls are a big problem here, even with wal_buffers cranked up
through the
On Wed, Mar 14, 2012 at 3:29 PM, Jeff Janes jeff.ja...@gmail.com wrote:
I think my analysis is pretty much a re-wording of yours, but I'd
emphasize that getting the WALWriteLock is bad not just because they
fight over the lock, but because someone else (probably background wal
writer) is
On Tuesday, March 13, 2012 03:26:34 AM Robert Haas wrote:
Meanwhile, here are some TPS graphs at 16MB and 32MB on the IBM POWER7
machine. 32 clients, 1800 seconds, scale factor 300, synchronous
commit off.
That graph makes me cringe because its pretty representative of what I have
seen in
On Mon, Mar 12, 2012 at 4:45 PM, Jeff Janes jeff.ja...@gmail.com wrote:
Rerunning all 4 benchmarks (both 16MB and 32MB wal_buffers on both
machines) with fsync=off (as well as synchronous_commit=off still)
might help clarify things.
If it increases the TPS of Nate@16MB, but doesn't change the
On Mon, Mar 12, 2012 at 4:45 PM, Jeff Janes jeff.ja...@gmail.com wrote:
Rerunning all 4 benchmarks (both 16MB and 32MB wal_buffers on both
machines) with fsync=off (as well as synchronous_commit=off still)
might help clarify things.
I reran the 32-client benchmark on the IBM machine with
On Tue, Mar 13, 2012 at 3:48 PM, Robert Haas robertmh...@gmail.com wrote:
On Mon, Mar 12, 2012 at 4:45 PM, Jeff Janes jeff.ja...@gmail.com wrote:
Rerunning all 4 benchmarks (both 16MB and 32MB wal_buffers on both
machines) with fsync=off (as well as synchronous_commit=off still)
might help
On Tue, Mar 13, 2012 at 4:55 AM, Andres Freund and...@anarazel.de wrote:
On Tuesday, March 13, 2012 03:26:34 AM Robert Haas wrote:
Meanwhile, here are some TPS graphs at 16MB and 32MB on the IBM POWER7
machine. 32 clients, 1800 seconds, scale factor 300, synchronous
commit off.
That graph
That's a speedup of nearly a factor of two, so clearly fsync-related
stalls are a big problem here, even with wal_buffers cranked up
through the ceiling.
H. Do you have any ability to test on XFS?
--
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com
--
Sent via pgsql-hackers
On Wed, Mar 14, 2012 at 7:20 AM, Robert Haas robertmh...@gmail.com wrote:
On Tue, Mar 13, 2012 at 3:48 PM, Robert Haas robertmh...@gmail.com wrote:
On Mon, Mar 12, 2012 at 4:45 PM, Jeff Janes jeff.ja...@gmail.com wrote:
Rerunning all 4 benchmarks (both 16MB and 32MB wal_buffers on both
On Tue, Mar 13, 2012 at 10:02 PM, Fujii Masao masao.fu...@gmail.com wrote:
On Wed, Mar 14, 2012 at 7:20 AM, Robert Haas robertmh...@gmail.com wrote:
On Tue, Mar 13, 2012 at 3:48 PM, Robert Haas robertmh...@gmail.com wrote:
On Mon, Mar 12, 2012 at 4:45 PM, Jeff Janes jeff.ja...@gmail.com wrote:
On Tue, Mar 13, 2012 at 6:44 PM, Josh Berkus j...@agliodbs.com wrote:
That's a speedup of nearly a factor of two, so clearly fsync-related
stalls are a big problem here, even with wal_buffers cranked up
through the ceiling.
H. Do you have any ability to test on XFS?
It seems I do.
On Sun, Mar 11, 2012 at 11:51 PM, Fujii Masao masao.fu...@gmail.com wrote:
On Sun, Mar 11, 2012 at 12:55 PM, Robert Haas robertmh...@gmail.com wrote:
I've finally been able to run some more tests of the effect of
adjusting wal_buffers to values higher than 16MB. I ran the test on
the 16 core
On Sat, Mar 10, 2012 at 7:55 PM, Robert Haas robertmh...@gmail.com wrote:
I've finally been able to run some more tests of the effect of
adjusting wal_buffers to values higher than 16MB. I ran the test on
the 16 core (x 4 hw threads/core) IBM POWER7 machine, with my usual
configuration
On Mon, Mar 12, 2012 at 12:32 PM, Jeff Janes jeff.ja...@gmail.com wrote:
On Nate Boley's machine, the difference was ~100% increase rather than
~10%.
Oh, right. I had forgotten how dramatic the changes were in those
test runs. I guess I should be happy that the absolute numbers on
this
On Mon, Mar 12, 2012 at 10:55 AM, Robert Haas robertmh...@gmail.com wrote:
On Mon, Mar 12, 2012 at 12:32 PM, Jeff Janes jeff.ja...@gmail.com wrote:
On Nate Boley's machine, the difference was ~100% increase rather than
~10%.
Oh, right. I had forgotten how dramatic the changes were in those
On Mon, Mar 12, 2012 at 4:45 PM, Jeff Janes jeff.ja...@gmail.com wrote:
Do you think the difference is in the CPU architecture, or the
IO subsystem?
That is an excellent question. I tried looking at vmstat output, but
a funny thing kept happening: periodically, the iowait column would
show
On Sun, Mar 11, 2012 at 12:55 PM, Robert Haas robertmh...@gmail.com wrote:
I've finally been able to run some more tests of the effect of
adjusting wal_buffers to values higher than 16MB. I ran the test on
the 16 core (x 4 hw threads/core) IBM POWER7 machine, with my usual
configuration
I've finally been able to run some more tests of the effect of
adjusting wal_buffers to values higher than 16MB. I ran the test on
the 16 core (x 4 hw threads/core) IBM POWER7 machine, with my usual
configuration settings:
shared_buffers = 8GB
maintenance_work_mem = 1GB
synchronous_commit = off
On Mon, Feb 20, 2012 at 4:10 AM, Fujii Masao masao.fu...@gmail.com wrote:
On Mon, Feb 20, 2012 at 3:08 AM, Robert Haas robertmh...@gmail.com wrote:
On Sun, Feb 19, 2012 at 9:46 AM, Euler Taveira de Oliveira
eu...@timbira.com wrote:
On 19-02-2012 02:24, Robert Haas wrote:
I have attached tps
On Mon, Feb 20, 2012 at 3:59 AM, Simon Riggs si...@2ndquadrant.com wrote:
There is no existing statistics view suitable to include such information.
What about defining pg_stat_xlog or something?
Perhaps pg_stat_perf so we don't need to find a new home every time.
Thinking about it, I think
On 19-02-2012 02:24, Robert Haas wrote:
I have attached tps scatterplots. The obvious conclusion appears to
be that, with only 16MB of wal_buffers, the buffer wraps around with
some regularity
Isn't it useful to print some messages on the log when we have wrap around?
In this case, we have an
On Sun, Feb 19, 2012 at 9:46 AM, Euler Taveira de Oliveira
eu...@timbira.com wrote:
On 19-02-2012 02:24, Robert Haas wrote:
I have attached tps scatterplots. The obvious conclusion appears to
be that, with only 16MB of wal_buffers, the buffer wraps around with
some regularity
Isn't it
Robert Haas robertmh...@gmail.com writes:
On Sun, Feb 19, 2012 at 9:46 AM, Euler Taveira de Oliveira
eu...@timbira.com wrote:
Isn't it useful to print some messages on the log when we have wrap around?
In this case, we have an idea that wal_buffers needs to be increased.
I was thinking about
On Sun, Feb 19, 2012 at 6:33 PM, Tom Lane t...@sss.pgh.pa.us wrote:
Robert Haas robertmh...@gmail.com writes:
On Sun, Feb 19, 2012 at 9:46 AM, Euler Taveira de Oliveira
eu...@timbira.com wrote:
Isn't it useful to print some messages on the log when we have wrap
around?
In this case, we have
On Mon, Feb 20, 2012 at 3:08 AM, Robert Haas robertmh...@gmail.com wrote:
On Sun, Feb 19, 2012 at 9:46 AM, Euler Taveira de Oliveira
eu...@timbira.com wrote:
On 19-02-2012 02:24, Robert Haas wrote:
I have attached tps scatterplots. The obvious conclusion appears to
be that, with only 16MB of
On 02/19/2012 12:24 AM, Robert Haas wrote:
I think we might want to consider
adjusting our auto-tuning formula for wal_buffers to allow for a
higher cap, although this is obviously not enough data to draw any
firm conclusions.
That's an easy enough idea to throw into my testing queue. The
Just for kicks, I ran two 30-minute pgbench tests at scale factor 300
tonight on Nate Boley's machine, with -n -l -c 32 -j 32. The
configurations were identical, except that on one of them, I set
wal_buffers=64MB. It seemed to make quite a lot of difference:
wal_buffers not set (thus, 16MB):
Robert Haas wrote:
On Sun, Apr 3, 2011 at 1:26 PM, Tom Lane wrote:
It would probably take less than a day to flesh out this idea and
make it happen, but it does seem like a rather large change for
late alpha.
what we're trying to avoid is committing new stuff that may require
I wrote:
IMO the real problem is essentially that GUC assign hooks have two
functions, checking and canonicalization of the value-to-be-stored
versus executing secondary actions when an assignment is made; and
there's no way to get at just the first one. So we cannot canonicalize
the value
On Mon, Apr 4, 2011 at 2:41 PM, Tom Lane t...@sss.pgh.pa.us wrote:
I wrote:
IMO the real problem is essentially that GUC assign hooks have two
functions, checking and canonicalization of the value-to-be-stored
versus executing secondary actions when an assignment is made; and
there's no way
Robert Haas robertmh...@gmail.com writes:
On Mon, Apr 4, 2011 at 2:41 PM, Tom Lane t...@sss.pgh.pa.us wrote:
Given these rules, a check_hook and assign_hook could cooperate to store
additional data in what guc.c thinks is just a pointer to a string
value, ie, there can be more data after the
On Mon, Apr 4, 2011 at 2:58 PM, Tom Lane t...@sss.pgh.pa.us wrote:
Robert Haas robertmh...@gmail.com writes:
On Mon, Apr 4, 2011 at 2:41 PM, Tom Lane t...@sss.pgh.pa.us wrote:
Given these rules, a check_hook and assign_hook could cooperate to store
additional data in what guc.c thinks is just
Robert Haas robertmh...@gmail.com writes:
On Mon, Apr 4, 2011 at 2:58 PM, Tom Lane t...@sss.pgh.pa.us wrote:
Another variant would be to allow the check_hook to pass back a separate
void * value that could be passed on to the assign_hook, containing
any necessary derived data. This is
On Mon, Apr 4, 2011 at 3:14 PM, Tom Lane t...@sss.pgh.pa.us wrote:
Robert Haas robertmh...@gmail.com writes:
On Mon, Apr 4, 2011 at 2:58 PM, Tom Lane t...@sss.pgh.pa.us wrote:
Another variant would be to allow the check_hook to pass back a separate
void * value that could be passed on to the
Robert Haas robertmh...@gmail.com writes:
OK. Please comment the crap out of whatever you do, or maybe even add
a README. This stuff is just a bit arcane, and guideposts help a lot.
We already have a README for that ;-). PFA, a patch to
src/backend/utils/misc/README describing the proposed
On Mon, Apr 4, 2011 at 4:57 PM, Tom Lane t...@sss.pgh.pa.us wrote:
Robert Haas robertmh...@gmail.com writes:
OK. Please comment the crap out of whatever you do, or maybe even add
a README. This stuff is just a bit arcane, and guideposts help a lot.
We already have a README for that ;-).
I wrote:
Robert Haas robertmh...@gmail.com writes:
I had intended to commit Greg's patch with a show hook and an assign
hook and the calculated value stored separately from the GUC variable,
which I believe would avoid this is a problem, but Tom thought this
way was better. Unfortunately, my
On Sun, Apr 3, 2011 at 1:26 PM, Tom Lane t...@sss.pgh.pa.us wrote:
IMO the real problem is essentially that GUC assign hooks have two
functions, checking and canonicalization of the value-to-be-stored
versus executing secondary actions when an assignment is made; and
there's no way to get at
Robert Haas robertmh...@gmail.com writes:
On Sun, Apr 3, 2011 at 1:26 PM, Tom Lane t...@sss.pgh.pa.us wrote:
IMO the real problem is essentially that GUC assign hooks have two
functions, checking and canonicalization of the value-to-be-stored
versus executing secondary actions when an
Robert Haas robertmh...@gmail.com writes:
On Thu, Mar 31, 2011 at 8:38 AM, Bernd Helmle maili...@oopsware.de wrote:
This might be nitpicking (or i'm currently missing something), but i
recognized that setting wal_buffers = -1 always triggers the following on
reload, even if nothing to
This might be nitpicking (or i'm currently missing something), but i recognized
that setting wal_buffers = -1 always triggers the following on reload, even if
nothing to wal_buffers had changed:
$ pg_ctl reload
LOG: received SIGHUP, reloading configuration files
LOG: parameter wal_buffers
On Thu, Mar 31, 2011 at 8:38 AM, Bernd Helmle maili...@oopsware.de wrote:
This might be nitpicking (or i'm currently missing something), but i
recognized that setting wal_buffers = -1 always triggers the following on
reload, even if nothing to wal_buffers had changed:
$ pg_ctl reload
LOG:
56 matches
Mail list logo
