Simon Riggs wrote:
You have posted patches that I have said I don't agree with. My name is
going to be on this when it goes in, so I don't think it makes any sense
to force that commit to include changes I don't agree with. I cannot
prevent you making changes afterwards, nor would I wish to.
On Oct 9, 2009, at 1:21 PM, Bruce Momjian br...@momjian.us wrote:
Simon Riggs wrote:
You have posted patches that I have said I don't agree with. My
name is
going to be on this when it goes in, so I don't think it makes any
sense
to force that commit to include changes I don't agree with.
Robert Haas wrote:
But at least for simple features I think that there would be a value
in separating the patch author's work from the committer's adjustments.
That is just going to make life harder for committers.
There are plenty of things with my name on them that are not exactly
On Fri, 2009-10-09 at 14:05 -0400, Andrew Dunstan wrote:
Robert Haas wrote:
But at least for simple features I think that there would be a value
in separating the patch author's work from the committer's adjustments.
That is just going to make life harder for committers.
There
Andrew Dunstan wrote:
Robert Haas wrote:
But at least for simple features I think that there would be a value
in separating the patch author's work from the committer's adjustments.
That is just going to make life harder for committers.
There are plenty of things with my name
On Wed, 2009-10-07 at 23:19 -0400, Heikki Linnakangas wrote:
Heikki Linnakangas wrote:
I've made public the version I'm working on. That's the version I'm
ultimately going to commit. It would be a lot more helpful if you
provided these patches over that version. Otherwise I have to
Simon Riggs wrote:
I would still like you to make a clear statement that the contents of
that repository are BSD licenced open source contributions.
Ok. All the content in the repository at
git://git.postgresql.org/git/users/heikki/postgres.git is released under
the same BSD license as
On Thu, 2009-10-08 at 07:33 -0400, Heikki Linnakangas wrote:
Simon Riggs wrote:
I would still like you to make a clear statement that the contents of
that repository are BSD licenced open source contributions.
Ok. All the content in the repository at
On Thu, 2009-10-08 at 12:49 +0100, Simon Riggs wrote:
I'll get cracking on some changes.
This will probably be next week now, just in case you're wondering when
I'll start adding patches.
--
Simon Riggs www.2ndQuadrant.com
--
Sent via pgsql-hackers mailing list
On Tue, 2009-09-22 at 12:53 +0300, Heikki Linnakangas wrote:
It looks like the standby tries to remove XID 4323 from the
known-assigned hash table, but it's not there because it was removed
and set in pg_subtrans by an XLOG_XACT_ASSIGNMENT record earlier. I
guess we should just not throw an
On Wed, 2009-09-23 at 12:07 +0300, Heikki Linnakangas wrote:
we need be careful to avoid putting any extra work into the normal
recovery path. Otherwise bugs in hot standby related code can cause
crash recovery to fail.
Re-checked code and found a couple of additional places that needed
tests
Simon Riggs wrote:
On Tue, 2009-09-22 at 12:53 +0300, Heikki Linnakangas wrote:
It looks like the standby tries to remove XID 4323 from the
known-assigned hash table, but it's not there because it was removed
and set in pg_subtrans by an XLOG_XACT_ASSIGNMENT record earlier. I
guess we should
Heikki Linnakangas wrote:
I've made public the version I'm working on. That's the version I'm
ultimately going to commit. It would be a lot more helpful if you
provided these patches over that version. Otherwise I have to refactor
them over that codebase, possibly introducing new bugs.
On Sun, 2009-09-27 at 14:59 +0300, Heikki Linnakangas wrote:
The locking in smgr_redo_commit and smgr_redo_abort doesn't look right.
First of all, smgr_redo_abort is not holding XidGenLock and
ProcArrayLock while modifying ShmemVariableCache-nextXid and
ShmemVariableCache-latestCompletedXid,
On Sun, 2009-09-27 at 15:35 +0300, Heikki Linnakangas wrote:
TransactionIdIsInProgress() doesn't consult the known-assigned-xids
structure. That's a problem: in the standby, TransactionIdIsInProgress()
can return false for a transaction that is still running in the master.
HeapTupleSatisfies*
The locking in smgr_redo_commit and smgr_redo_abort doesn't look right.
First of all, smgr_redo_abort is not holding XidGenLock and
ProcArrayLock while modifying ShmemVariableCache-nextXid and
ShmemVariableCache-latestCompletedXid, respectively, like
smgr_redo_commit is. Attached patch fixes that.
TransactionIdIsInProgress() doesn't consult the known-assigned-xids
structure. That's a problem: in the standby, TransactionIdIsInProgress()
can return false for a transaction that is still running in the master.
HeapTupleSatisfies* functions can incorrectly set HEAP_XMIN/XMAX_INVALID
hint bits
Looking at the startup sequence now.
I see that you modified ExtendSUBTRANS so that it doesn't wipe out
previously set values if it's called with out-of-order xids. I guess
that works, although I think it can leave pages unzeroed if it's called
with a large enough gap between xids, so that the
On Thu, 2009-09-24 at 13:33 +0300, Heikki Linnakangas wrote:
Heikki Linnakangas wrote:
The problem becomes a lot easier if we accept that it's OK to have a
lock included in the running-xacts snapshot and also appear in a
XLOG_RELATION_LOCK record later. The standby should handle that
On Wed, 2009-09-23 at 19:07 +0300, Heikki Linnakangas wrote:
Rather than keep the numHeldLocks counters per-proc in proc array, I
think it would be simpler to have a single (or one per lock partition)
counter in shared memory in lock.c. It's just an optimization to make it
faster to find out
On Wed, 2009-09-23 at 17:45 +0300, Heikki Linnakangas wrote:
Heikki Linnakangas wrote:
Simon Riggs wrote:
On Wed, 2009-09-23 at 11:13 +0300, Heikki Linnakangas wrote:
I note that we don't emit RunningXacts after a shutdown checkpoint. So
if recovery starts at a shutdown checkpoint, we
Simon Riggs wrote:
On Wed, 2009-09-23 at 19:07 +0300, Heikki Linnakangas wrote:
Rather than keep the numHeldLocks counters per-proc in proc array, I
think it would be simpler to have a single (or one per lock partition)
counter in shared memory in lock.c. It's just an optimization to make it
On Fri, 2009-09-25 at 11:49 +0300, Heikki Linnakangas wrote:
Looking at the startup sequence now.
I see that you modified ExtendSUBTRANS so that it doesn't wipe out
previously set values if it's called with out-of-order xids. I guess
that works, although I think it can leave pages unzeroed
On Fri, 2009-09-25 at 13:23 +0300, Heikki Linnakangas wrote:
Simon Riggs wrote:
On Wed, 2009-09-23 at 17:45 +0300, Heikki Linnakangas wrote:
XactClearRecoveryTransactions() when we see a shutdown checkpoint, which
clears all recovery locks. But doesn't that prematurely clear all locks
Simon Riggs wrote:
Definitely need to cope with them for Hot Standby. My point was general
one to say that behaviour is very non-useful for users with prepared
transactions. It just causes manual effort by a DBA each time the system
is shutdown.
The transaction manager is supposed to
Simon Riggs wrote:
On Wed, 2009-09-23 at 17:45 +0300, Heikki Linnakangas wrote:
XactClearRecoveryTransactions() when we see a shutdown checkpoint, which
clears all recovery locks. But doesn't that prematurely clear all locks
belonging to prepared transactions as well?
Much better to read
On Fri, 2009-09-25 at 13:14 +0300, Heikki Linnakangas wrote:
Simon Riggs wrote:
On Wed, 2009-09-23 at 19:07 +0300, Heikki Linnakangas wrote:
Rather than keep the numHeldLocks counters per-proc in proc array, I
think it would be simpler to have a single (or one per lock partition)
In XidInMVCCSnapshot:
if (snapshot-takenDuringRecovery)
{
/*
* If the snapshot contains full subxact data, the fastest way
to check
* things is just to compare the given XID against both subxact
XIDs and
* top-level
On Fri, 2009-09-25 at 15:50 +0300, Heikki Linnakangas wrote:
Hang on, isn't this 180 degrees backwards?
A witty riposte escapes me; yes, the if test is !correct.
What I find amazing is that it passed the test where I put it doing make
installcheck in an infinite loop for a long time. I guess
Simon Riggs wrote:
What I find amazing is that it passed the test where I put it doing make
installcheck in an infinite loop for a long time. I guess that means the
regression tests hardly touch the concurrency code at all, which now I
think about it makes sense but I still find that very
On Fri, 2009-09-25 at 16:30 -0400, Alvaro Herrera wrote:
Simon Riggs wrote:
What I find amazing is that it passed the test where I put it doing make
installcheck in an infinite loop for a long time. I guess that means the
regression tests hardly touch the concurrency code at all, which
Alvaro Herrera wrote:
Simon Riggs wrote:
What I find amazing is that it passed the test where I put it doing make
installcheck in an infinite loop for a long time. I guess that means the
regression tests hardly touch the concurrency code at all, which now I
think about it makes sense but
Andrew Dunstan wrote:
Alvaro Herrera wrote:
Try installcheck-parallel, which should be a bit better. It's probably
not yet good enough though because it always runs the same tests
concurrently.
It is also quite easy to set up your own schedule.
... except you have to be careful with
Heikki Linnakangas wrote:
The problem becomes a lot easier if we accept that it's OK to have a
lock included in the running-xacts snapshot and also appear in a
XLOG_RELATION_LOCK record later. The standby should handle that
gracefully already. If we just remove RecoveryInfoLock, that can
Looking at the way cache invalidations are handled in two-phase
transactions, it would be simpler if we store the shared cache
invalidation messages in the twophase state file header, like we store
deleted relations and subxids. This allows them to be copied to the
COMMIT_PREPARED WAL record, so
The logic in the lock manager to track the number of held
AccessExclusiveLocks (with ProcArrayIncrementNumHeldLocks and
ProcArrayDecrementNumHeldLocks) seems to be broken. I added an Assertion
into ProcArrayDecrementNumHeldLocks:
--- a/src/backend/storage/ipc/procarray.c
+++
On Wed, 2009-09-23 at 11:13 +0300, Heikki Linnakangas wrote:
Looking at the way cache invalidations are handled in two-phase
transactions, it would be simpler if we store the shared cache
invalidation messages in the twophase state file header, like we store
deleted relations and subxids.
On Wed, 2009-09-23 at 12:07 +0300, Heikki Linnakangas wrote:
it highlights that we
need be careful to avoid putting any extra work into the normal
recovery
path. Otherwise bugs in hot standby related code can cause crash
recovery to fail.
Excellent point. I will put in additional protective
On Wed, 2009-09-23 at 12:07 +0300, Heikki Linnakangas wrote:
seems to be broken
Agreed.
Patch withdrawn for correction and rework. Nothing serious, but not much
point doing further testing to all current issues resolved.
Tracking of issues raised and later solved via Wiki.
--
Simon Riggs
Simon Riggs wrote:
On Wed, 2009-09-23 at 11:13 +0300, Heikki Linnakangas wrote:
I note that we don't emit RunningXacts after a shutdown checkpoint. So
if recovery starts at a shutdown checkpoint, we don't let read-only
backends in until the first online checkpoint. Could we treat a shutdown
Heikki Linnakangas wrote:
Simon Riggs wrote:
On Wed, 2009-09-23 at 11:13 +0300, Heikki Linnakangas wrote:
I note that we don't emit RunningXacts after a shutdown checkpoint. So
if recovery starts at a shutdown checkpoint, we don't let read-only
backends in until the first online checkpoint.
On Tue, Sep 22, 2009 at 10:02 AM, Alvaro Herrera
alvhe...@commandprompt.com wrote:
Heikki Linnakangas escribió:
Simon Riggs wrote:
On Mon, 2009-09-21 at 19:42 -0700, Jeff Janes wrote:
jjanes=# begin;
BEGIN
jjanes=# lock table pgbench_history in access exclusive mode;
LOCK TABLE
Jeff Janes jeff.ja...@gmail.com writes:
Unfortunately, isolation level serializable is not truly
serializable. Usually it is good enough, but when it isn't good
enough and you need an explicit table lock (a very rare but not
nonexistent situation), I think it should either lock the table in
Simon Riggs wrote:
On Wed, 2009-09-23 at 12:07 +0300, Heikki Linnakangas wrote:
seems to be broken
Agreed.
Looking at the relation lock stuff a bit more...
When someone tries to acquire an AccessExclusiveLock, but can't get it
immediately, we sleep while holding RecoveryInfoLock. That
Simon,
Patch withdrawn for correction and rework. Nothing serious, but not much
point doing further testing to all current issues resolved.
:-(
Good thing we went for 4 CFs.
Is there a GIT branch of Simon's current working version up somewhere?
--
Josh Berkus
PostgreSQL Experts Inc.
Josh Berkus wrote:
Patch withdrawn for correction and rework. Nothing serious, but not much
point doing further testing to all current issues resolved.
:-(
Good thing we went for 4 CFs.
I think we should try to hammer this in in this commitfest. None of the
issues found this far are too
Heikki,
I think we should try to hammer this in in this commitfest. None of the
issues found this far are too serious, nothing that requires major rewrites.
It would certainly be valuable to get users testing it starting with
Alpha2 instead of waiting 2 months.
--
Josh Berkus
PostgreSQL
Simon Riggs wrote:
On Mon, 2009-09-21 at 14:01 +0100, Simon Riggs wrote:
On Mon, 2009-09-21 at 13:50 +0300, Heikki Linnakangas wrote:
is this that we seem to be missing conflict
resolution for GiST index tuples deleted by the kill_prior_tuples
mechanism. Unless I'm missing something, we
On Mon, 2009-09-21 at 19:42 -0700, Jeff Janes wrote:
On Tue, Sep 15, 2009 at 2:41 PM, Simon Riggs si...@2ndquadrant.com wrote:
OK, here is the latest version of the Hot Standby patchset. This is
about version 30+ by now, but we should regard this as 0.2.1
Patch against CVS HEAD (now):
Simon Riggs wrote:
On Mon, 2009-09-21 at 19:42 -0700, Jeff Janes wrote:
jjanes=# begin;
BEGIN
jjanes=# lock table pgbench_history in access exclusive mode;
LOCK TABLE
jjanes=# select count(*) from pgbench_history;
count
519104
(1 row)
jjanes=# select count(*) from
On Tue, 2009-09-22 at 11:04 +0300, Heikki Linnakangas wrote:
By me, yes. WAL replay does not require a table lock to progress. Any
changes are protected with block-level locks. It does acquire a table
lock and cancel conflicting queries when it is about to replay something
that would
In testing, it looks like there's still something wrong with the
subtransaction handling. I created a test function to create a large
number of subtransactions:
CREATE LANGUAGE plpgsql;
CREATE TABLE bar (id int4);
CREATE OR REPLACE FUNCTION subxids (n integer) RETURNS void LANGUAGE
plpgsql AS $$
On Tue, 2009-09-22 at 12:53 +0300, Heikki Linnakangas wrote:
In testing, it looks like there's still something wrong with the
subtransaction handling. I created a test function to create a large
number of subtransactions:
OK, looking at this now. Thanks for the report.
--
Simon Riggs
On Tue, 2009-09-22 at 12:53 +0300, Heikki Linnakangas wrote:
It looks like the standby tries to remove XID 4323 from the
known-assigned hash table, but it's not there because it was removed
and set in pg_subtrans by an XLOG_XACT_ASSIGNMENT record earlier. I
guess we should just not throw an
On Tue, 2009-09-22 at 12:53 +0300, Heikki Linnakangas wrote:
In testing, it looks like there's still something wrong with the
subtransaction handling. I created a test function to create a large
number of subtransactions:
CREATE LANGUAGE plpgsql;
CREATE TABLE bar (id int4);
CREATE OR
Heikki Linnakangas escribió:
Simon Riggs wrote:
On Mon, 2009-09-21 at 19:42 -0700, Jeff Janes wrote:
jjanes=# begin;
BEGIN
jjanes=# lock table pgbench_history in access exclusive mode;
LOCK TABLE
jjanes=# select count(*) from pgbench_history;
count
519104
(1 row)
Simon Riggs wrote:
OK, here is the latest version of the Hot Standby patchset. This is
about version 30+ by now, but we should regard this as 0.2.1
Patch against CVS HEAD (now): clean apply, compile, no known bugs.
OVERVIEW
Anyone who is interested in how the hot standby behaves should
Simon Riggs wrote:
OK, here is the latest version of the Hot Standby patchset. This is
about version 30+ by now, but we should regard this as 0.2.1
Patch against CVS HEAD (now): clean apply, compile, no known bugs.
Thanks! Attached is some minor comment and fixes, and some dead code
removal.
On Mon, 2009-09-21 at 13:50 +0300, Heikki Linnakangas wrote:
The only bug I've found
!
is this that we seem to be missing conflict
resolution for GiST index tuples deleted by the kill_prior_tuples
mechanism. Unless I'm missing something, we need similar handling there
that we have in
On Mon, 2009-09-21 at 13:50 +0300, Heikki Linnakangas wrote:
The documentation talks about setting and checking
default_transaction_read_only, but I think it doesn't say anything
about
transaction_read_only, which I find odd. This in particular:
Users will be able to tell whether their
On Mon, 2009-09-21 at 14:01 +0100, Simon Riggs wrote:
On Mon, 2009-09-21 at 13:50 +0300, Heikki Linnakangas wrote:
is this that we seem to be missing conflict
resolution for GiST index tuples deleted by the kill_prior_tuples
mechanism. Unless I'm missing something, we need similar
On Mon, Sep 21, 2009 at 9:01 AM, Simon Riggs si...@2ndquadrant.com wrote:
On Mon, 2009-09-21 at 13:50 +0300, Heikki Linnakangas wrote:
The only bug I've found
!
Yeah, wow.
...Robert
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your
Simon Riggs wrote:
OK, here is the latest version of the Hot Standby patchset. This is
about version 30+ by now, but we should regard this as 0.2.1
Patch against CVS HEAD (now): clean apply, compile, no known bugs.
Wow, great! Simon has allowed us to pass a great milestone in Postgres
On Tue, Sep 15, 2009 at 2:41 PM, Simon Riggs si...@2ndquadrant.com wrote:
OK, here is the latest version of the Hot Standby patchset. This is
about version 30+ by now, but we should regard this as 0.2.1
Patch against CVS HEAD (now): clean apply, compile, no known bugs.
OVERVIEW
You can
On Thu, 2009-09-17 at 19:01 -0400, Robert Haas wrote:
I'm going to put the index-only scans aside for now to focus on hot
standby and streaming replication. Both are big patches, so there's
plenty of work in those two alone, and not only for me.
What is the best way to attack this?
Robert Haas robertmh...@gmail.com writes:
Also, stepping back from me personally, should we try to assign some
additional reviewers to these patches? Is there some way we can
divide up review tasks among multiple people so that we're not
repeating each others work?
Thoughts appreciated,
On Tue, Sep 15, 2009 at 2:41 PM, Simon Riggs si...@2ndquadrant.com wrote:
OK, here is the latest version of the Hot Standby patchset. This is
about version 30+ by now, but we should regard this as 0.2.1
Patch against CVS HEAD (now): clean apply, compile, no known bugs.
Hi Simon,
Is there
On Fri, 2009-09-18 at 07:23 -0700, Jeff Janes wrote:
On Tue, Sep 15, 2009 at 2:41 PM, Simon Riggs si...@2ndquadrant.com
wrote:
OK, here is the latest version of the Hot Standby patchset.
This is
about version 30+ by now, but we should regard this as 0.2.1
Simon Riggs si...@2ndquadrant.com writes:
On Fri, 2009-09-18 at 07:23 -0700, Jeff Janes wrote:
Is there a reason that you remove the WAL_DEBUG shown below?
WAL_DEBUG is not removed by the patch, though that section of code is
removed, as you observe. I recall an earlier bug report by
On Fri, 2009-09-18 at 11:14 -0400, Tom Lane wrote:
Simon Riggs si...@2ndquadrant.com writes:
On Fri, 2009-09-18 at 07:23 -0700, Jeff Janes wrote:
Is there a reason that you remove the WAL_DEBUG shown below?
WAL_DEBUG is not removed by the patch, though that section of code is
removed,
Enviados: Jueves, 17 de Septiembre 2009 20:53:24 GMT -10:00 Hawai
Asunto: Re: [HACKERS] Hot Standby 0.2.1
On Thu, 2009-09-17 at 19:01 -0400, Robert Haas wrote:
I'm going to put the index-only scans aside for now to focus on hot
standby and streaming replication. Both are big patches, so there's
Robert Haas wrote:
On Wed, Sep 16, 2009 at 6:05 PM, Josh Berkus j...@agliodbs.com wrote:
Now that Simon has submitted this, can some of the heavy-hitters here
review it? Heikki?
Nobody's name is next to it.
I don't think anyone is planning to ignore this patch, but it wasn't
included in
On Thu, 2009-09-17 at 09:54 +0300, Heikki Linnakangas wrote:
This is a pretty small CommitFest, so there
shouldn't be any shortage of reviewers, though Heikki's time may be
stretched a little thin, since Streaming Replication is also in the
queue, and he is working on index-only scans.
On Thu, Sep 17, 2009 at 2:54 AM, Heikki Linnakangas
heikki.linnakan...@enterprisedb.com wrote:
Robert Haas wrote:
On Wed, Sep 16, 2009 at 6:05 PM, Josh Berkus j...@agliodbs.com wrote:
Now that Simon has submitted this, can some of the heavy-hitters here
review it? Heikki?
Nobody's name is
--On 15. September 2009 22:41:59 +0100 Simon Riggs si...@2ndquadrant.com
wrote:
http://wiki.postgresql.org/images/0/01/Hot_Standby_Recovery_Functions.pdf
This doesn't work for me, it seems the correct link is
http://wiki.postgresql.org/images/1/10/Hot_Standby_Recovery_Functions.pdf
?
All,
Now that Simon has submitted this, can some of the heavy-hitters here
review it? Heikki?
Nobody's name is next to it.
--
Josh Berkus
PostgreSQL Experts Inc.
www.pgexperts.com
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
On Wed, Sep 16, 2009 at 6:05 PM, Josh Berkus j...@agliodbs.com wrote:
Now that Simon has submitted this, can some of the heavy-hitters here
review it? Heikki?
Nobody's name is next to it.
I don't think anyone is planning to ignore this patch, but it wasn't
included in the first round of
On Tue, Sep 15, 2009 at 10:41:59PM +0100, Simon Riggs wrote:
OK, here is the latest version of the Hot Standby patchset. This is
about version 30+ by now, but we should regard this as 0.2.1 Patch
against CVS HEAD (now): clean apply, compile, no known bugs.
Kudos
Cheers,
David.
--
78 matches
Mail list logo
