Thanks, i missed that.
After doing these changes, following is the observation:
1. The size of the pgstat file is 86KB. Last edited was when i moved the
file location to RAMdisk.
2. The issue persists. I am seeing continuous logs:
2012-01-05 00:00:06 JST WARNING: pgstat wait timeout
2012-01-05
Andrew Dunstan writes:
> Tom said:
>> 2. A slightly cleaner fix for this should be to duplicate the SV and
>> then release the copy around the SvPVutf8 call, only if it's readonly.
>> "Fixing" it in do_util_elog is entirely the wrong thing.
> How do we tell if it's readonly?
SvREADONLY(sv) macro
On 01/04/2012 06:15 PM, David E. Wheeler wrote:
[11:12pm]TonyC:theory: using sv_mortalcopy() instead of newSVsv()
should prevent the leak in that workaround, assuming there's no
FREETMPS between the call and use of the return value
That's the solution to leakiness I had in mind.
Tom said:
Andrew Dunstan writes:
> On 01/04/2012 06:20 PM, Tom Lane wrote:
>> But we'd have to deal with
>> standard-conforming strings some way. The idea I had about that, since
>> we have an open database connection at hand, is to check the connected
>> backend's standard_conforming_strings state via PQp
On Wed, Jan 04, 2012 at 04:58:08PM -0600, Jim Nasby wrote:
> Except AFAIR Oracle uses the term to indicate something that is
> happening *outside* of your current transaction, which is definitely
> not what the proposal is talking about.
That feature is commonly translated in PostgreSQL to a dblin
On 01/04/2012 06:20 PM, Tom Lane wrote:
But we'd have to deal with
standard-conforming strings some way. The idea I had about that, since
we have an open database connection at hand, is to check the connected
backend's standard_conforming_strings state via PQparameterStatus. If
it doesn't hav
I agree with Pavel also. Putting these constants in the pg_catalog isn't
the cleanest solution.
Though one can make its own little lib in python, perl, whatever, to store
these constants, it would be better if through the compilation, these C
constants were copied in a way for PL/*.
I can't reall
I wrote:
> Perhaps the intoRel stuff should be
> saving/restoring the original destreceiver instead of just blindly
> overwriting it.
I concluded that was the best fix, and have committed it.
regards, tom lane
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgre
Andrew Dunstan writes:
> On 01/04/2012 01:13 PM, Tom Lane wrote:
>> Not entirely sure what to do about this. We could consider reverting
>> the aforesaid patch and trying to find another way of fixing that code's
>> failure to cope with standard-conforming strings, but I'm not sure that
>> there'
On Jan 4, 2012, at 2:48 PM, Andrew Dunstan wrote:
>> That's kinda grotty ... and leaky ...
>>
>
> Of course it is. It wasn't meant as a solution but as validation of your
> suspicions about the nature of the problem. (The leakiness could be solved,
> though.)
From #p5p on irc.perl.org:
[10:5
On Dec 19, 2011, at 12:31 PM, Simon Riggs wrote:
> On Sun, Dec 18, 2011 at 4:22 PM, Jim Nasby wrote:
>> On Dec 18, 2011, at 2:28 AM, Gianni Ciolli wrote:
>>> I have written some notes about autonomous subtransactions, which have
>>> already been touched (at least) in two separate threads; please f
On Dec 20, 2011, at 11:29 PM, Tom Lane wrote:
> Robert Haas writes:
>> So, what do we do about this? The obvious answer is "increase
>> NUM_CLOG_BUFFERS", and I'm not sure that's a bad idea.
>
> As you say, that's likely to hurt people running in small shared
> memory. I too have thought about
On 01/04/2012 05:05 PM, Tom Lane wrote:
Well, the crash is apparently solved by the following, which your
investigation suggested to me:
- cmsg = sv2cstr(msg);
+ cmsg = sv2cstr(newSVsv(msg));
That's kinda grotty ... and leaky ...
Of course it is. It wasn't meant as a solution
On 01/04/2012 01:13 PM, Tom Lane wrote:
In http://archives.postgresql.org/pgsql-admin/2012-01/msg8.php
it's pointed out that recent versions of pg_restore fall over on
archives made with -Fc --inserts (or --column-inserts), but only when
restoring direct to database; if you ask for text out
Andrew Dunstan writes:
> On 01/04/2012 03:56 PM, Tom Lane wrote:
>> I think what's being passed *is* an SV --- at least, the contents look
>> reasonable in gdb --- but for some reason SvPVutf8 isn't coping with
>> this particular kind of SV. Googling suggests that SvPVutf8 used to
>> fail on READ
On 01/04/2012 03:56 PM, Tom Lane wrote:
Andrew Dunstan writes:
On 01/04/2012 12:56 PM, Tom Lane wrote:
I looked at that last night but it appeared that SvOK would be perfectly
happy. (Didn't actually try it, though, I was just eyeballing the flags
in gdb.)
I tested it and you're right, it
On Wed, Jan 4, 2012 at 4:02 PM, Kevin Grittner
wrote:
> Robert Haas wrote:
>
>> 2. The CLOG code isn't designed to manage a large number of
>> buffers, so adding more might cause a performance regression on
>> small systems.
>>
>> On Nate Boley's 32-core system, running pgbench at scale factor
>>
On Jan 4, 2012, at 2:02 PM, Kevin Grittner wrote:
> Jim Nasby wrote:
>> Here's output from our largest OLTP system... not sure exactly how
>> to interpret it, so I'm just providing the raw data. This spans
>> almost exactly 1 month.
>
> Those number wind up meaning that 18% of the 256-byte blocks
On Wed, Jan 4, 2012 at 4:06 PM, Tom Lane wrote:
> Robert Haas writes:
>> But, the OP makes me wonder: why can a standby only perform a
>> restartpoint where the master performed a checkpoint? It seems like a
>> standby ought to be able to create a restartpoint anywhere, just by
>> writing everyt
On Wed, Jan 4, 2012 at 3:51 PM, Kevin Grittner
wrote:
>> If double writes aren't going to give us anything "for free",
>> maybe that's not the right place to be focusing our
>> efforts...
>
> I'm not sure why it's not enough that they improve performance over
> the alternative. Making some other
Paul Ramsey writes:
> Further notes, from Andrew (RhodiumToad) on IRC about the cause of this
> crasher:
> [12:31pm] RhodiumToad: there's no trivial fix
IMO the main bug here is that functions.c isn't expecting qd->dest to be
overwritten, so we could work around it by keeping a separate private
Robert Haas writes:
> But, the OP makes me wonder: why can a standby only perform a
> restartpoint where the master performed a checkpoint? It seems like a
> standby ought to be able to create a restartpoint anywhere, just by
> writing everything, flushing it to disk, and update pg_control.
Perh
Robert Haas wrote:
> 2. The CLOG code isn't designed to manage a large number of
> buffers, so adding more might cause a performance regression on
> small systems.
>
> On Nate Boley's 32-core system, running pgbench at scale factor
> 100, the optimal number of buffers seems to be around 32. I'
Andrew Dunstan writes:
> On 01/04/2012 12:56 PM, Tom Lane wrote:
>> I looked at that last night but it appeared that SvOK would be perfectly
>> happy. (Didn't actually try it, though, I was just eyeballing the flags
>> in gdb.)
> I tested it and you're right, it doesn't help. I don't see what el
Robert Haas wrote:
> I think it may also be tricky to make sure that a backend that
> needs to write a dirty buffer doesn't end up having to wait for a
> double-write to be fsync'd.
This and other parts of your post seem to ignore the BBU write-back
cache. Multiple fsyncs of a single page can
I neglected to commit after revising the text of a few comments; use this
version instead. Thanks.
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 3b52415..9eba8e8 100644
*** a/src/backend/commands/tablecmds.c
--- b/src/backend/commands/tablecmds.c
**
Further notes, from Andrew (RhodiumToad) on IRC about the cause of this crasher:
[12:03pm] RhodiumToad: what happens is this
[12:04pm] RhodiumToad: postquel_start know this statement doesn't
return the result, so it supplies None_Receiver as the dest-receiver
for the query
[12:04pm] RhodiumToad: h
On Tue, Jan 3, 2012 at 9:38 PM, Tom Lane wrote:
> Robert Haas writes:
>> On Tue, Jan 3, 2012 at 6:25 PM, Peter Eisentraut wrote:
>>> [ reasons ]
>
>> I agree with these reasons. We don't get charged $0.50 per GUC, so
>> there's no particular reason to contort things to have fewer of them.
>
> W
On Wed, Jan 4, 2012 at 3:02 PM, Kevin Grittner
wrote:
> Jim Nasby wrote:
>> Here's output from our largest OLTP system... not sure exactly how
>> to interpret it, so I'm just providing the raw data. This spans
>> almost exactly 1 month.
>
> Those number wind up meaning that 18% of the 256-byte bl
On Wed, Jan 4, 2012 at 13:13, Andrew Dunstan wrote:
>
>
> On 01/04/2012 12:56 PM, Tom Lane wrote:
>> I looked at that last night but it appeared that SvOK would be perfectly
>> happy. (Didn't actually try it, though, I was just eyeballing the flags
>> in gdb.)
>
> I tested it and you're right,
Git master can already avoid rewriting the table for column type changes like
varchar(10) -> varchar(20). However, if the column in question is on either
side of a FK relationship, we always revalidate the foreign key. Concretely,
I wanted these no-rewrite type changes to also assume FK validity:
On 01/04/2012 12:56 PM, Tom Lane wrote:
Andrew Dunstan writes:
The docs (perldoc perlvar) seem to suggest $^V isn't an SV (i.e. a
scalar) but some other sort of animal:
Yeah, it's a version object, but I'd have thought that SvPV and friends
would automatically stringify such an object. Othe
On Wed, Jan 4, 2012 at 1:32 PM, Kevin Grittner
wrote:
> Robert Haas wrote:
>> we only fsync() at end-of-checkpoint. So we'd have to think about
>> what to fsync, and how often, to keep the double-write buffer to a
>> manageable size.
>
> I think this is the big tuning challenge with this technol
Jim Nasby wrote:
> Here's output from our largest OLTP system... not sure exactly how
> to interpret it, so I'm just providing the raw data. This spans
> almost exactly 1 month.
Those number wind up meaning that 18% of the 256-byte blocks (1024
transactions each) were all commits. Yikes. Tha
On 01/04/2012 02:35 PM, Heikki Linnakangas wrote:
On 04.01.2012 20:44, Robert Haas wrote:
On Tue, Jan 3, 2012 at 9:23 PM, Tom Lane wrote:
Robert Haas writes:
On Tue, Jan 3, 2012 at 7:39 PM, Peter
Geoghegan wrote:
Yes, I know that these only appeared in GCC 4.6+ and as such are a
relative
On 04.01.2012 20:44, Robert Haas wrote:
On Tue, Jan 3, 2012 at 9:23 PM, Tom Lane wrote:
Robert Haas writes:
On Tue, Jan 3, 2012 at 7:39 PM, Peter Geoghegan wrote:
Yes, I know that these only appeared in GCC 4.6+ and as such are a
relatively recent phenomenon, but there has been some effort
On Dec 23, 2011, at 2:23 PM, Kevin Grittner wrote:
> Jeff Janes wrote:
>
>> Could we get some major OLTP users to post their CLOG for
>> analysis? I wouldn't think there would be much
>> security/propietary issues with CLOG data.
>
> FWIW, I got the raw numbers to do my quick check using this
On 4 January 2012 18:44, Robert Haas wrote:
> On Tue, Jan 3, 2012 at 9:23 PM, Tom Lane wrote:
>> I'm also less than thrilled with the idea that whatever the gcc boys
>> decide to make a warning tomorrow will automatically become a MUST FIX
>> NOW for us.
>
> I'm not thrilled about that either. E
On Tue, Jan 3, 2012 at 9:23 PM, Tom Lane wrote:
> Robert Haas writes:
>> On Tue, Jan 3, 2012 at 7:39 PM, Peter Geoghegan
>> wrote:
>>> Yes, I know that these only appeared in GCC 4.6+ and as such are a
>>> relatively recent phenomenon, but there has been some effort to
>>> eliminate them, and i
Robert Haas wrote:
> we only fsync() at end-of-checkpoint. So we'd have to think about
> what to fsync, and how often, to keep the double-write buffer to a
> manageable size.
I think this is the big tuning challenge with this technology.
> I can't help thinking that any extra fsyncs are pre
On Fri, Dec 30, 2011 at 4:25 AM, Marko Kreen wrote:
>> I have the (hopefully wrong) impression that you're missing the fact
>> that it already exists, at least in 9.0.
>
> You are right, I missed it. For quite obvious reason:
>
> $ grep -ri aclexplode doc/
> $
>
> Is there a good reason why it'
On Wed, Jan 4, 2012 at 8:31 AM, Kevin Grittner
wrote:
>> When we reach a restartpoint, we fsync everything down to disk and
>> then nuke the double-write buffer.
>
> I think we add to the double-write buffer as we write pages from the
> buffer to disk. I don't think it makes sense to do potential
In http://archives.postgresql.org/pgsql-admin/2012-01/msg8.php
it's pointed out that recent versions of pg_restore fall over on
archives made with -Fc --inserts (or --column-inserts), but only when
restoring direct to database; if you ask for text output it's perfectly
fine. Investigation show
On Wed, Jan 4, 2012 at 9:28 AM, Aidan Van Dyk wrote:
> I'ld love a "hook" script that was run if sync-rep state ever changed
> (heck, I'ld even like it if it just choose a new sync standby).
That seems useful. I don't think the current code quite knows its own
state; we seem to have each walsend
On Wed, Jan 4, 2012 at 11:02 AM, Simon Riggs wrote:
> On Wed, Jan 4, 2012 at 3:56 PM, Heikki Linnakangas
> wrote:
>> On 04.01.2012 08:42, Amit Kapila wrote:
>>>
>>> Why PostgreSQL needs to write WAL record for Checkpoint when it maintains
>>> same information in pg_control file?
>>
>>
>> I guess
Andrew Dunstan writes:
> The docs (perldoc perlvar) seem to suggest $^V isn't an SV (i.e. a
> scalar) but some other sort of animal:
Yeah, it's a version object, but I'd have thought that SvPV and friends
would automatically stringify such an object. Otherwise, practically
any kind of perl exte
On 01/04/2012 11:15 AM, Tom Lane wrote:
Andrew Dunstan writes:
On 01/04/2012 12:47 AM, David E. Wheeler wrote:
Oy, this doesn’t look good:
$ do LANGUAGE plperl $$ elog(NOTICE, $^V) $$;
The connection to the server was lost. Attempting reset: Succeeded.
Try
elog(NOTICE, "$^V")
Isn't th
On Jan 4, 2012, at 8:15 AM, Tom Lane wrote:
> Isn't this a Perl bug? It seems to be crashing in SvPVutf8, which
> means that either Perl passed something that's not an SV to a function
> declared to accept SVs, or that SvPVutf8 fails on some SVs. Either
> way, Perl is failing to satisfy the POLA
On Jan 4, 2012, at 12:44 AM, Andrew Dunstan wrote:
> Try
>
> elog(NOTICE, "$^V")
Yeah, I used
elog(NOTICE, version->new($^V))
Which was fine. But still, it should’t segfault.
David
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscript
Andrew Dunstan writes:
> On 01/04/2012 12:47 AM, David E. Wheeler wrote:
>> Oy, this doesnt look good:
>> $ do LANGUAGE plperl $$ elog(NOTICE, $^V) $$;
>> The connection to the server was lost. Attempting reset: Succeeded.
> Try
> elog(NOTICE, "$^V")
Isn't this a Perl bug? It seems to be c
On Wed, Jan 4, 2012 at 3:56 PM, Heikki Linnakangas
wrote:
> On 04.01.2012 08:42, Amit Kapila wrote:
>>
>> Why PostgreSQL needs to write WAL record for Checkpoint when it maintains
>> same information in pg_control file?
>
>
> I guess it wouldn't be strictly necessary...
Apart from replicated stan
On 04.01.2012 08:42, Amit Kapila wrote:
Why PostgreSQL needs to write WAL record for Checkpoint when it maintains
same information in pg_control file?
I guess it wouldn't be strictly necessary...
This may be required in case we need information about more than one
checkpoint as pg_control can
Why PostgreSQL needs to write WAL record for Checkpoint when it maintains
same information in pg_control file?
This may be required in case we need information about more than one
checkpoint as pg_control can hold information of only recent checkpoint. But
I could not think of a case where more th
On Wed, Jan 4, 2012 at 1:31 PM, Stephen Frost wrote:
> Simon, all,
>
> * Simon Riggs (si...@2ndquadrant.com) wrote:
>> (1) report all errors on a page, including errors that don't change
>> PostgreSQL data. This involves checksumming long strings of zeroes,
>> which the checksum algorithm can't te
On 4 January 2012 07:24, Heikki Linnakangas
wrote:
> I think SetBufferCommitInfoNeedsSave() needs the same treatment as
> MarkBufferDirty(). And it would probably be good to only set the latch if
> the buffer wasn't dirty already. Setting a latch that's already set is fast,
> but surely it's even
On Tue, Jan 3, 2012 at 9:22 PM, Robert Haas wrote:
> It seems to me that if you are happy with #2, you don't really need to
> enable sync rep in the first place.
>
> At any rate, even without multiple component failures, this
> configuration makes it pretty easy to lose durability (which is the
>
Pavel Stehule wrote:
> here is new version of CHECK FUNCTION patch
>
> I changed implementation of interface:
>
> * checked functions returns table instead raising exceptions - it
> necessary for describing more issues inside one function - and it
> allow to use better structured data then Except
Alvaro Herrera wrote:
> We don't support BLCKSZ higher than 32k anyway
Thanks for pointing that out. Then I think we should declare sum1 to
be uint and sum2 to be uint64. We can take out the "% 255" out from
where it sits in the v2 patch, and just add something like this after
the sums are g
Simon Riggs wrote:
> My focus was on getting something working first, then tuning. If
> we're agreed that we have everything apart from the tuning then we
> can proceed with tests to see which works better.
Sure. I just think you are there already except for what I got into.
FWIW, moving th
Simon, all,
* Simon Riggs (si...@2ndquadrant.com) wrote:
> (1) report all errors on a page, including errors that don't change
> PostgreSQL data. This involves checksumming long strings of zeroes,
> which the checksum algorithm can't tell apart from long strings of
> ones.
Do we actually know whe
Robert Haas wrote:
> Jeff Janes wrote:
>> But it doesn't seem safe to me replace a page from the DW buffer
>> and then apply WAL to that replaced page which preceded the age of
>> the page in the buffer.
>
> That's what LSNs are for.
Agreed.
> If we write the page to the checkpoint buffer j
Excerpts from Kevin Grittner's message of mié ene 04 04:12:43 -0300 2012:
> "Kevin Grittner" wrote:
>
> > if we define sum1 and sum2 as uint I don't see how we can get an
> > overflow with 8k byes
>
> I feel the need to amend that opinion.
>
> While sum1 only needs to hold a maximum of (BLC
Simon Riggs wrote:
> We can either
>
> (1) report all errors on a page, including errors that don't change
> PostgreSQL data. This involves checksumming long strings of zeroes,
> which the checksum algorithm can't tell apart from long strings of
> ones.
What do you mean? Each byte which goes
On 4 Leden 2012, 13:17, pratikchirania wrote:
> I have installed RAMdisk and pointed the parameter:
>
> #stats_temp_directory = 'B:\pg_stat_tmp'
> I also tried #stats_temp_directory = 'B:/pg_stat_tmp'
>
> But, still there is no file created in the RAM disk.
> The previous stat file is touched even
I have installed RAMdisk and pointed the parameter:
#stats_temp_directory = 'B:\pg_stat_tmp'
I also tried #stats_temp_directory = 'B:/pg_stat_tmp'
But, still there is no file created in the RAM disk.
The previous stat file is touched even after the change is made. (I have
restarted the service af
2012/1/4 Simon Riggs :
> On Wed, Jan 4, 2012 at 9:20 AM, Andres Freund wrote:
>
>> I wonder if CRC32c wouldn't be a good alternative given more and more cpus
>> (its in SSE 4.2) support calculating it in silicon.
>
> We're trying to get something that fits in 16bits for this release.
> I'm guessi
On Wed, Jan 4, 2012 at 3:49 AM, Robert Haas wrote:
> On Fri, Dec 30, 2011 at 11:58 AM, Jeff Janes wrote:
>> On 12/29/11, Ants Aasma wrote:
>>> Unless I'm missing something, double-writes are needed for all writes,
>>> not only the first page after a checkpoint. Consider this sequence of
>>> even
On Tue, Jan 3, 2012 at 11:28 PM, Tom Lane wrote:
> Jim Nasby writes:
>> On Jan 3, 2012, at 12:11 PM, Simon Riggs wrote:
>>> This could well be related to the fact that DropRelFileNodeBuffers()
>>> does a scan of shared_buffers, which is an O(N) approach no matter the
>>> size of the index.
>
>> C
On Wed, Jan 4, 2012 at 9:20 AM, Andres Freund wrote:
> On Tuesday, January 03, 2012 11:21:42 PM Kevin Grittner wrote:
>> (1) I like the choice of Fletcher-16. It should be very good at
>> detecting problems while being a lot less expensive that an official
>> CRC calculation.
> I wonder if CRC32
On Tuesday, January 03, 2012 11:21:42 PM Kevin Grittner wrote:
> (1) I like the choice of Fletcher-16. It should be very good at
> detecting problems while being a lot less expensive that an official
> CRC calculation.
I wonder if CRC32c wouldn't be a good alternative given more and more cpus
(
On 01/04/2012 12:47 AM, David E. Wheeler wrote:
Is this perhaps by design?
Oy, this doesn’t look good:
$ do LANGUAGE plperl $$ elog(NOTICE, $^V) $$;
ERROR: server conn crashed?
ERROR: server conn crashed?
The connection to the server was lost. Attempting reset: Succeeded.
(pgxn@localhost:59
On Tue, Jan 3, 2012 at 10:21 PM, Kevin Grittner
wrote:
> I'm happy with how this looks, except (as noted in a code comment)
> that there seems to be room for optimization of the calculation
> itself. Details below:
...
> (3) Rather than having PageSetVerificationInfo() use memcpy,
> followed
On Tue, Jan 3, 2012 at 11:00 PM, Jim Nasby wrote:
> On Jan 3, 2012, at 4:21 PM, Kevin Grittner wrote:
>> (2) I'm not sure about doing this in three parts, to skip the
>> checksum itself and the hole in the middle of the page. Is this
>> because the hole might not have predictable data? Why woul
On Wed, Jan 4, 2012 at 7:24 AM, Heikki Linnakangas
wrote:
> Setting a latch that's already set is fast,
> but surely it's even faster to not even try.
Agreed. I think we should SetLatch() at the first point a backend
writes a dirty buffer because the bgwriter has been inactive.
Continually waki
74 matches
Mail list logo
