On Wed, Jan 4, 2012 at 7:24 AM, Heikki Linnakangas
heikki.linnakan...@enterprisedb.com wrote:
Setting a latch that's already set is fast,
but surely it's even faster to not even try.
Agreed. I think we should SetLatch() at the first point a backend
writes a dirty buffer because the bgwriter
On Tue, Jan 3, 2012 at 11:00 PM, Jim Nasby j...@nasby.net wrote:
On Jan 3, 2012, at 4:21 PM, Kevin Grittner wrote:
(2) I'm not sure about doing this in three parts, to skip the
checksum itself and the hole in the middle of the page. Is this
because the hole might not have predictable data?
On Tue, Jan 3, 2012 at 10:21 PM, Kevin Grittner
kevin.gritt...@wicourts.gov wrote:
I'm happy with how this looks, except (as noted in a code comment)
that there seems to be room for optimization of the calculation
itself. Details below:
...
(3) Rather than having PageSetVerificationInfo()
On 01/04/2012 12:47 AM, David E. Wheeler wrote:
Is this perhaps by design?
Oy, this doesn’t look good:
$ do LANGUAGE plperl $$ elog(NOTICE, $^V) $$;
ERROR: server conn crashed?
ERROR: server conn crashed?
The connection to the server was lost. Attempting reset: Succeeded.
On Tuesday, January 03, 2012 11:21:42 PM Kevin Grittner wrote:
(1) I like the choice of Fletcher-16. It should be very good at
detecting problems while being a lot less expensive that an official
CRC calculation.
I wonder if CRC32c wouldn't be a good alternative given more and more cpus
On Wed, Jan 4, 2012 at 9:20 AM, Andres Freund and...@anarazel.de wrote:
On Tuesday, January 03, 2012 11:21:42 PM Kevin Grittner wrote:
(1) I like the choice of Fletcher-16. It should be very good at
detecting problems while being a lot less expensive that an official
CRC calculation.
I
On Tue, Jan 3, 2012 at 11:28 PM, Tom Lane t...@sss.pgh.pa.us wrote:
Jim Nasby j...@nasby.net writes:
On Jan 3, 2012, at 12:11 PM, Simon Riggs wrote:
This could well be related to the fact that DropRelFileNodeBuffers()
does a scan of shared_buffers, which is an O(N) approach no matter the
size
On Wed, Jan 4, 2012 at 3:49 AM, Robert Haas robertmh...@gmail.com wrote:
On Fri, Dec 30, 2011 at 11:58 AM, Jeff Janes jeff.ja...@gmail.com wrote:
On 12/29/11, Ants Aasma ants.aa...@eesti.ee wrote:
Unless I'm missing something, double-writes are needed for all writes,
not only the first page
2012/1/4 Simon Riggs si...@2ndquadrant.com:
On Wed, Jan 4, 2012 at 9:20 AM, Andres Freund and...@anarazel.de wrote:
I wonder if CRC32c wouldn't be a good alternative given more and more cpus
(its in SSE 4.2) support calculating it in silicon.
We're trying to get something that fits in
I have installed RAMdisk and pointed the parameter:
#stats_temp_directory = 'B:\pg_stat_tmp'
I also tried #stats_temp_directory = 'B:/pg_stat_tmp'
But, still there is no file created in the RAM disk.
The previous stat file is touched even after the change is made. (I have
restarted the service
On 4 Leden 2012, 13:17, pratikchirania wrote:
I have installed RAMdisk and pointed the parameter:
#stats_temp_directory = 'B:\pg_stat_tmp'
I also tried #stats_temp_directory = 'B:/pg_stat_tmp'
But, still there is no file created in the RAM disk.
The previous stat file is touched even after
Simon Riggs wrote:
We can either
(1) report all errors on a page, including errors that don't change
PostgreSQL data. This involves checksumming long strings of zeroes,
which the checksum algorithm can't tell apart from long strings of
ones.
What do you mean? Each byte which goes into
Excerpts from Kevin Grittner's message of mié ene 04 04:12:43 -0300 2012:
Kevin Grittner wrote:
if we define sum1 and sum2 as uint I don't see how we can get an
overflow with 8k byes
I feel the need to amend that opinion.
While sum1 only needs to hold a maximum of (BLCKSZ * 255),
Robert Haas wrote:
Jeff Janes wrote:
But it doesn't seem safe to me replace a page from the DW buffer
and then apply WAL to that replaced page which preceded the age of
the page in the buffer.
That's what LSNs are for.
Agreed.
If we write the page to the checkpoint buffer just once
Simon, all,
* Simon Riggs (si...@2ndquadrant.com) wrote:
(1) report all errors on a page, including errors that don't change
PostgreSQL data. This involves checksumming long strings of zeroes,
which the checksum algorithm can't tell apart from long strings of
ones.
Do we actually know
Simon Riggs wrote:
My focus was on getting something working first, then tuning. If
we're agreed that we have everything apart from the tuning then we
can proceed with tests to see which works better.
Sure. I just think you are there already except for what I got into.
FWIW, moving the
Alvaro Herrera wrote:
We don't support BLCKSZ higher than 32k anyway
Thanks for pointing that out. Then I think we should declare sum1 to
be uint and sum2 to be uint64. We can take out the % 255 out from
where it sits in the v2 patch, and just add something like this after
the sums are
Pavel Stehule wrote:
here is new version of CHECK FUNCTION patch
I changed implementation of interface:
* checked functions returns table instead raising exceptions - it
necessary for describing more issues inside one function - and it
allow to use better structured data then
On Tue, Jan 3, 2012 at 9:22 PM, Robert Haas robertmh...@gmail.com wrote:
It seems to me that if you are happy with #2, you don't really need to
enable sync rep in the first place.
At any rate, even without multiple component failures, this
configuration makes it pretty easy to lose
On 4 January 2012 07:24, Heikki Linnakangas
heikki.linnakan...@enterprisedb.com wrote:
I think SetBufferCommitInfoNeedsSave() needs the same treatment as
MarkBufferDirty(). And it would probably be good to only set the latch if
the buffer wasn't dirty already. Setting a latch that's already set
On Wed, Jan 4, 2012 at 1:31 PM, Stephen Frost sfr...@snowman.net wrote:
Simon, all,
* Simon Riggs (si...@2ndquadrant.com) wrote:
(1) report all errors on a page, including errors that don't change
PostgreSQL data. This involves checksumming long strings of zeroes,
which the checksum
Why PostgreSQL needs to write WAL record for Checkpoint when it maintains
same information in pg_control file?
This may be required in case we need information about more than one
checkpoint as pg_control can hold information of only recent checkpoint. But
I could not think of a case where more
On 04.01.2012 08:42, Amit Kapila wrote:
Why PostgreSQL needs to write WAL record for Checkpoint when it maintains
same information in pg_control file?
I guess it wouldn't be strictly necessary...
This may be required in case we need information about more than one
checkpoint as pg_control
On Wed, Jan 4, 2012 at 3:56 PM, Heikki Linnakangas
heikki.linnakan...@enterprisedb.com wrote:
On 04.01.2012 08:42, Amit Kapila wrote:
Why PostgreSQL needs to write WAL record for Checkpoint when it maintains
same information in pg_control file?
I guess it wouldn't be strictly necessary...
Andrew Dunstan and...@dunslane.net writes:
On 01/04/2012 12:47 AM, David E. Wheeler wrote:
Oy, this doesnt look good:
$ do LANGUAGE plperl $$ elog(NOTICE, $^V) $$;
The connection to the server was lost. Attempting reset: Succeeded.
Try
elog(NOTICE, $^V)
Isn't this a Perl bug? It
On Jan 4, 2012, at 12:44 AM, Andrew Dunstan wrote:
Try
elog(NOTICE, $^V)
Yeah, I used
elog(NOTICE, version-new($^V))
Which was fine. But still, it should’t segfault.
David
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
On Jan 4, 2012, at 8:15 AM, Tom Lane wrote:
Isn't this a Perl bug? It seems to be crashing in SvPVutf8, which
means that either Perl passed something that's not an SV to a function
declared to accept SVs, or that SvPVutf8 fails on some SVs. Either
way, Perl is failing to satisfy the POLA if
On 01/04/2012 11:15 AM, Tom Lane wrote:
Andrew Dunstanand...@dunslane.net writes:
On 01/04/2012 12:47 AM, David E. Wheeler wrote:
Oy, this doesn’t look good:
$ do LANGUAGE plperl $$ elog(NOTICE, $^V) $$;
The connection to the server was lost. Attempting reset: Succeeded.
Try
Andrew Dunstan and...@dunslane.net writes:
The docs (perldoc perlvar) seem to suggest $^V isn't an SV (i.e. a
scalar) but some other sort of animal:
Yeah, it's a version object, but I'd have thought that SvPV and friends
would automatically stringify such an object. Otherwise, practically
any
On Wed, Jan 4, 2012 at 11:02 AM, Simon Riggs si...@2ndquadrant.com wrote:
On Wed, Jan 4, 2012 at 3:56 PM, Heikki Linnakangas
heikki.linnakan...@enterprisedb.com wrote:
On 04.01.2012 08:42, Amit Kapila wrote:
Why PostgreSQL needs to write WAL record for Checkpoint when it maintains
same
On Wed, Jan 4, 2012 at 9:28 AM, Aidan Van Dyk ai...@highrise.ca wrote:
I'ld love a hook script that was run if sync-rep state ever changed
(heck, I'ld even like it if it just choose a new sync standby).
That seems useful. I don't think the current code quite knows its own
state; we seem to
In http://archives.postgresql.org/pgsql-admin/2012-01/msg8.php
it's pointed out that recent versions of pg_restore fall over on
archives made with -Fc --inserts (or --column-inserts), but only when
restoring direct to database; if you ask for text output it's perfectly
fine. Investigation
On Wed, Jan 4, 2012 at 8:31 AM, Kevin Grittner
kevin.gritt...@wicourts.gov wrote:
When we reach a restartpoint, we fsync everything down to disk and
then nuke the double-write buffer.
I think we add to the double-write buffer as we write pages from the
buffer to disk. I don't think it makes
On Fri, Dec 30, 2011 at 4:25 AM, Marko Kreen mark...@gmail.com wrote:
I have the (hopefully wrong) impression that you're missing the fact
that it already exists, at least in 9.0.
You are right, I missed it. For quite obvious reason:
$ grep -ri aclexplode doc/
$
Is there a good reason
Robert Haas robertmh...@gmail.com wrote:
we only fsync() at end-of-checkpoint. So we'd have to think about
what to fsync, and how often, to keep the double-write buffer to a
manageable size.
I think this is the big tuning challenge with this technology.
I can't help thinking that any
On Tue, Jan 3, 2012 at 9:23 PM, Tom Lane t...@sss.pgh.pa.us wrote:
Robert Haas robertmh...@gmail.com writes:
On Tue, Jan 3, 2012 at 7:39 PM, Peter Geoghegan pe...@2ndquadrant.com
wrote:
Yes, I know that these only appeared in GCC 4.6+ and as such are a
relatively recent phenomenon, but there
On 4 January 2012 18:44, Robert Haas robertmh...@gmail.com wrote:
On Tue, Jan 3, 2012 at 9:23 PM, Tom Lane t...@sss.pgh.pa.us wrote:
I'm also less than thrilled with the idea that whatever the gcc boys
decide to make a warning tomorrow will automatically become a MUST FIX
NOW for us.
I'm not
On Dec 23, 2011, at 2:23 PM, Kevin Grittner wrote:
Jeff Janes jeff.ja...@gmail.com wrote:
Could we get some major OLTP users to post their CLOG for
analysis? I wouldn't think there would be much
security/propietary issues with CLOG data.
FWIW, I got the raw numbers to do my quick check
On 04.01.2012 20:44, Robert Haas wrote:
On Tue, Jan 3, 2012 at 9:23 PM, Tom Lanet...@sss.pgh.pa.us wrote:
Robert Haasrobertmh...@gmail.com writes:
On Tue, Jan 3, 2012 at 7:39 PM, Peter Geogheganpe...@2ndquadrant.com wrote:
Yes, I know that these only appeared in GCC 4.6+ and as such are a
On 01/04/2012 02:35 PM, Heikki Linnakangas wrote:
On 04.01.2012 20:44, Robert Haas wrote:
On Tue, Jan 3, 2012 at 9:23 PM, Tom Lanet...@sss.pgh.pa.us wrote:
Robert Haasrobertmh...@gmail.com writes:
On Tue, Jan 3, 2012 at 7:39 PM, Peter
Geogheganpe...@2ndquadrant.com wrote:
Yes, I know
Jim Nasby j...@nasby.net wrote:
Here's output from our largest OLTP system... not sure exactly how
to interpret it, so I'm just providing the raw data. This spans
almost exactly 1 month.
Those number wind up meaning that 18% of the 256-byte blocks (1024
transactions each) were all commits.
On Wed, Jan 4, 2012 at 1:32 PM, Kevin Grittner
kevin.gritt...@wicourts.gov wrote:
Robert Haas robertmh...@gmail.com wrote:
we only fsync() at end-of-checkpoint. So we'd have to think about
what to fsync, and how often, to keep the double-write buffer to a
manageable size.
I think this is
On 01/04/2012 12:56 PM, Tom Lane wrote:
Andrew Dunstanand...@dunslane.net writes:
The docs (perldoc perlvar) seem to suggest $^V isn't an SV (i.e. a
scalar) but some other sort of animal:
Yeah, it's a version object, but I'd have thought that SvPV and friends
would automatically stringify
Git master can already avoid rewriting the table for column type changes like
varchar(10) - varchar(20). However, if the column in question is on either
side of a FK relationship, we always revalidate the foreign key. Concretely,
I wanted these no-rewrite type changes to also assume FK validity:
On Wed, Jan 4, 2012 at 13:13, Andrew Dunstan and...@dunslane.net wrote:
On 01/04/2012 12:56 PM, Tom Lane wrote:
I looked at that last night but it appeared that SvOK would be perfectly
happy. (Didn't actually try it, though, I was just eyeballing the flags
in gdb.)
I tested it and
On Wed, Jan 4, 2012 at 3:02 PM, Kevin Grittner
kevin.gritt...@wicourts.gov wrote:
Jim Nasby j...@nasby.net wrote:
Here's output from our largest OLTP system... not sure exactly how
to interpret it, so I'm just providing the raw data. This spans
almost exactly 1 month.
Those number wind up
On Tue, Jan 3, 2012 at 9:38 PM, Tom Lane t...@sss.pgh.pa.us wrote:
Robert Haas robertmh...@gmail.com writes:
On Tue, Jan 3, 2012 at 6:25 PM, Peter Eisentraut pete...@gmx.net wrote:
[ reasons ]
I agree with these reasons. We don't get charged $0.50 per GUC, so
there's no particular reason to
Further notes, from Andrew (RhodiumToad) on IRC about the cause of this crasher:
[12:03pm] RhodiumToad: what happens is this
[12:04pm] RhodiumToad: postquel_start know this statement doesn't
return the result, so it supplies None_Receiver as the dest-receiver
for the query
[12:04pm] RhodiumToad:
I neglected to commit after revising the text of a few comments; use this
version instead. Thanks.
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 3b52415..9eba8e8 100644
*** a/src/backend/commands/tablecmds.c
--- b/src/backend/commands/tablecmds.c
Robert Haas robertmh...@gmail.com wrote:
I think it may also be tricky to make sure that a backend that
needs to write a dirty buffer doesn't end up having to wait for a
double-write to be fsync'd.
This and other parts of your post seem to ignore the BBU write-back
cache. Multiple fsyncs
Andrew Dunstan and...@dunslane.net writes:
On 01/04/2012 12:56 PM, Tom Lane wrote:
I looked at that last night but it appeared that SvOK would be perfectly
happy. (Didn't actually try it, though, I was just eyeballing the flags
in gdb.)
I tested it and you're right, it doesn't help. I don't
Robert Haas robertmh...@gmail.com wrote:
2. The CLOG code isn't designed to manage a large number of
buffers, so adding more might cause a performance regression on
small systems.
On Nate Boley's 32-core system, running pgbench at scale factor
100, the optimal number of buffers seems to
Robert Haas robertmh...@gmail.com writes:
But, the OP makes me wonder: why can a standby only perform a
restartpoint where the master performed a checkpoint? It seems like a
standby ought to be able to create a restartpoint anywhere, just by
writing everything, flushing it to disk, and update
Paul Ramsey pram...@cleverelephant.ca writes:
Further notes, from Andrew (RhodiumToad) on IRC about the cause of this
crasher:
[12:31pm] RhodiumToad: there's no trivial fix
IMO the main bug here is that functions.c isn't expecting qd-dest to be
overwritten, so we could work around it by
On Wed, Jan 4, 2012 at 3:51 PM, Kevin Grittner
kevin.gritt...@wicourts.gov wrote:
If double writes aren't going to give us anything for free,
maybe that's not the right place to be focusing our
efforts...
I'm not sure why it's not enough that they improve performance over
the alternative.
On Wed, Jan 4, 2012 at 4:06 PM, Tom Lane t...@sss.pgh.pa.us wrote:
Robert Haas robertmh...@gmail.com writes:
But, the OP makes me wonder: why can a standby only perform a
restartpoint where the master performed a checkpoint? It seems like a
standby ought to be able to create a restartpoint
On Jan 4, 2012, at 2:02 PM, Kevin Grittner wrote:
Jim Nasby j...@nasby.net wrote:
Here's output from our largest OLTP system... not sure exactly how
to interpret it, so I'm just providing the raw data. This spans
almost exactly 1 month.
Those number wind up meaning that 18% of the 256-byte
On Wed, Jan 4, 2012 at 4:02 PM, Kevin Grittner
kevin.gritt...@wicourts.gov wrote:
Robert Haas robertmh...@gmail.com wrote:
2. The CLOG code isn't designed to manage a large number of
buffers, so adding more might cause a performance regression on
small systems.
On Nate Boley's 32-core
On 01/04/2012 03:56 PM, Tom Lane wrote:
Andrew Dunstanand...@dunslane.net writes:
On 01/04/2012 12:56 PM, Tom Lane wrote:
I looked at that last night but it appeared that SvOK would be perfectly
happy. (Didn't actually try it, though, I was just eyeballing the flags
in gdb.)
I tested it
Andrew Dunstan and...@dunslane.net writes:
On 01/04/2012 03:56 PM, Tom Lane wrote:
I think what's being passed *is* an SV --- at least, the contents look
reasonable in gdb --- but for some reason SvPVutf8 isn't coping with
this particular kind of SV. Googling suggests that SvPVutf8 used to
On 01/04/2012 01:13 PM, Tom Lane wrote:
In http://archives.postgresql.org/pgsql-admin/2012-01/msg8.php
it's pointed out that recent versions of pg_restore fall over on
archives made with -Fc --inserts (or --column-inserts), but only when
restoring direct to database; if you ask for text
On 01/04/2012 05:05 PM, Tom Lane wrote:
Well, the crash is apparently solved by the following, which your
investigation suggested to me:
- cmsg = sv2cstr(msg);
+ cmsg = sv2cstr(newSVsv(msg));
That's kinda grotty ... and leaky ...
Of course it is. It wasn't meant as a solution
On Dec 20, 2011, at 11:29 PM, Tom Lane wrote:
Robert Haas robertmh...@gmail.com writes:
So, what do we do about this? The obvious answer is increase
NUM_CLOG_BUFFERS, and I'm not sure that's a bad idea.
As you say, that's likely to hurt people running in small shared
memory. I too have
On Dec 19, 2011, at 12:31 PM, Simon Riggs wrote:
On Sun, Dec 18, 2011 at 4:22 PM, Jim Nasby j...@nasby.net wrote:
On Dec 18, 2011, at 2:28 AM, Gianni Ciolli wrote:
I have written some notes about autonomous subtransactions, which have
already been touched (at least) in two separate threads;
On Jan 4, 2012, at 2:48 PM, Andrew Dunstan wrote:
That's kinda grotty ... and leaky ...
Of course it is. It wasn't meant as a solution but as validation of your
suspicions about the nature of the problem. (The leakiness could be solved,
though.)
From #p5p on irc.perl.org:
Andrew Dunstan and...@dunslane.net writes:
On 01/04/2012 01:13 PM, Tom Lane wrote:
Not entirely sure what to do about this. We could consider reverting
the aforesaid patch and trying to find another way of fixing that code's
failure to cope with standard-conforming strings, but I'm not sure
I wrote:
Perhaps the intoRel stuff should be
saving/restoring the original destreceiver instead of just blindly
overwriting it.
I concluded that was the best fix, and have committed it.
regards, tom lane
--
Sent via pgsql-hackers mailing list
I agree with Pavel also. Putting these constants in the pg_catalog isn't
the cleanest solution.
Though one can make its own little lib in python, perl, whatever, to store
these constants, it would be better if through the compilation, these C
constants were copied in a way for PL/*.
I can't
On 01/04/2012 06:20 PM, Tom Lane wrote:
But we'd have to deal with
standard-conforming strings some way. The idea I had about that, since
we have an open database connection at hand, is to check the connected
backend's standard_conforming_strings state via PQparameterStatus. If
it doesn't
On Wed, Jan 04, 2012 at 04:58:08PM -0600, Jim Nasby wrote:
Except AFAIR Oracle uses the term to indicate something that is
happening *outside* of your current transaction, which is definitely
not what the proposal is talking about.
That feature is commonly translated in PostgreSQL to a
Andrew Dunstan and...@dunslane.net writes:
On 01/04/2012 06:20 PM, Tom Lane wrote:
But we'd have to deal with
standard-conforming strings some way. The idea I had about that, since
we have an open database connection at hand, is to check the connected
backend's standard_conforming_strings
On 01/04/2012 06:15 PM, David E. Wheeler wrote:
[11:12pm]TonyC:theory: using sv_mortalcopy() instead of newSVsv()
should prevent the leak in that workaround, assuming there's no
FREETMPS between the call and use of the return value
That's the solution to leakiness I had in mind.
Tom said:
Andrew Dunstan and...@dunslane.net writes:
Tom said:
2. A slightly cleaner fix for this should be to duplicate the SV and
then release the copy around the SvPVutf8 call, only if it's readonly.
Fixing it in do_util_elog is entirely the wrong thing.
How do we tell if it's readonly?
Thanks, i missed that.
After doing these changes, following is the observation:
1. The size of the pgstat file is 86KB. Last edited was when i moved the
file location to RAMdisk.
2. The issue persists. I am seeing continuous logs:
2012-01-05 00:00:06 JST WARNING: pgstat wait timeout
2012-01-05
74 matches
Mail list logo
