Re: [HACKERS] Block level concurrency during recovery

2008-11-03 Thread Simon Riggs
On Mon, 2008-11-03 at 10:07 -0500, Tom Lane wrote: Simon Riggs [EMAIL PROTECTED] writes: VACUUM with a btree index proceeds like this: 1. Scan table 2. Remove rows from btree identified in (1) 3. Remove rows from heap identified in (1) The purpose of the additional locking

Re: [HACKERS] Hot standby v5 patch assertion failure

2008-11-03 Thread Simon Riggs
On Mon, 2008-11-03 at 06:41 +, Simon Riggs wrote: On Mon, 2008-11-03 at 12:16 +1300, Mark Kirkwood wrote: Trying out a few different scenarios I ran across this: CONTEXT: xlog redo update: rel 1663/16384/16397; tid 9614/62; new 158828/59 DEBUG: start recovery xid = 7002 lsn = 0

Re: [HACKERS] Bitmap Indexes patch (was Re: Bitmap Indexes: request for feedback)

2008-11-03 Thread Simon Riggs
it, complete. I think the objective was minimal change away from Gavin's original. But it sounds like you found some out of date comments. Extensive docs have been added as a README, mainly because it was pretty hard to understand without them. -- Simon Riggs www.2ndQuadrant.com

Re: [HACKERS] Bitmap Indexes patch (was Re: Bitmap Indexes: request for feedback)

2008-11-03 Thread Simon Riggs
On Mon, 2008-11-03 at 23:28 +, Simon Riggs wrote: On Mon, 2008-11-03 at 17:37 -0500, Greg Stark wrote: Secondly the locking seems to be a bit overoptimistic. I'm pretty sure you have to take an exclusive lock on an index page any time you make any data modifications in index pages

Re: [HACKERS] Bitmap Indexes patch (was Re: Bitmap Indexes: request for feedback)

2008-11-03 Thread Simon Riggs
specifically because of the possible effects on Hot Standby. I haven't seen any problems that would be caused by locking. In general, all aspects of code needs to be checked, especially if there are bugs, else how would you resolve them? -- Simon Riggs www.2ndQuadrant.com PostgreSQL

[HACKERS] Re: Hot standby v5 patch - Databases created post backup remain inaccessible + replica SIGSEGV when coming out of standby

2008-11-03 Thread Simon Riggs
archived) 3/ Attempt to connect to 'bench' on the replica Head from 2nd Nov with v5 patch applied on Freebsd 7.1-Prerelease as usual Case acknowledged. -- Simon Riggs www.2ndQuadrant.com PostgreSQL Training, Services and Support -- Sent via pgsql-hackers mailing list (pgsql

[HACKERS] Re: Hot standby v5 patch - restarted replica changes to warm standby mode

2008-11-03 Thread Simon Riggs
. Not being able to connect after a restart is a design feature, to protect you from running potentially invalid queries. -- Simon Riggs www.2ndQuadrant.com PostgreSQL Training, Services and Support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes

Re: [HACKERS] Signal handling patch (v2) for Synch Rep

2008-11-04 Thread Simon Riggs
On Tue, 2008-11-04 at 21:04 +0900, Fujii Masao wrote: To be reviewed easily, I'm splitting Synch Rep patch into some pieces. Great idea. I'll be doing that also. -- Simon Riggs www.2ndQuadrant.com PostgreSQL Training, Services and Support -- Sent via pgsql-hackers mailing list

Re: [HACKERS] Bitmap Indexes patch (was Re: Bitmap Indexes: request for feedback)

2008-11-03 Thread Simon Riggs
On Mon, 2008-11-03 at 23:28 +, Simon Riggs wrote: On Mon, 2008-11-03 at 17:37 -0500, Greg Stark wrote: There are a lot of comments in the code which imply that vacuuming is not implemented but in fact from what I can see it is -- sort of. It does rewrite the bitmap in bmbulkdelete

[HACKERS] Re: Hot standby v5 patch - Databases created post backup remain inaccessible + replica SIGSEGV when coming out of standby

2008-11-04 Thread Simon Riggs
, but I'll go looking for other issues there. -- Simon Riggs www.2ndQuadrant.com PostgreSQL Training, Services and Support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Re: Hot standby v5 patch - Databases created post backup remain inaccessible + replica SIGSEGV when coming out of standby

2008-11-05 Thread Simon Riggs
On Tue, 2008-11-04 at 09:52 +, Simon Riggs wrote: postgres=# \c bench FATAL: database bench does not exist Previous connection kept CREATE DATABASE didn't trigger the db flat file update, code for which existed and was triggered in the cases when a transaction would normally

Re: [HACKERS] Re: Hot standby v5 patch - restarted replica changes to warm standby mode

2008-11-05 Thread Simon Riggs
On Wed, 2008-11-05 at 21:05 +1300, Mark Kirkwood wrote: So this one is entirely user error! No worries. I'd rather have false positives, since can often reveal some usability problem. I think we're OK here for now though. Thanks very much for testing. -- Simon Riggs www

Re: [HACKERS] Synchronous replication patch v1

2008-11-05 Thread Simon Riggs
/months, so this is just a few short initial comments. Well done for getting this together so quickly, especially with your visit to hospital taking away time. -- Simon Riggs www.2ndQuadrant.com PostgreSQL Training, Services and Support -- Sent via pgsql-hackers mailing list (pgsql

Re: [HACKERS] Synchronous replication patch v1

2008-11-05 Thread Simon Riggs
data: COPY. If you implemented the send as a new command, similar to COPY, it would all work very easily. SENDFILE? -- Simon Riggs www.2ndQuadrant.com PostgreSQL Training, Services and Support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes

Re: [HACKERS] broken URL in commitfest page

2008-11-05 Thread Simon Riggs
. -- Simon Riggs www.2ndQuadrant.com PostgreSQL Training, Services and Support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] broken URL in commitfest page

2008-11-05 Thread Simon Riggs
item fails to point to a patch, also. I've removed that entry now. The patch was being worked on by Jonah but it looks like we didn't make the deadline. -- Simon Riggs www.2ndQuadrant.com PostgreSQL Training, Services and Support -- Sent via pgsql-hackers mailing list (pgsql-hackers

[HACKERS] Timing problem in DROP TABLESPACE?

2008-11-06 Thread Simon Riggs
working on patches I don't think I'm causing it myself. This error happens on the *master*, not on the standby server. Is there a timing problem? I see that we attempt to do a checkpoint to clean up deleted files. Is that working? Is it ignoring certain messages? -- Simon Riggs www

Re: [HACKERS] broken URL in commitfest page

2008-11-06 Thread Simon Riggs
On Wed, 2008-11-05 at 13:23 -0500, Jonah H. Harris wrote: On Wed, Nov 5, 2008 at 12:35 PM, Simon Riggs [EMAIL PROTECTED] wrote: The Join Removal item fails to point to a patch, also. I've removed that entry now. The patch was being worked on by Jonah but it looks like we didn't make

Re: [HACKERS] Proposed Patch to Improve Performance of Multi-Batch Hash Join for Skewed Data Sets

2008-11-06 Thread Simon Riggs
to test whether they overlap? Perhaps you already did that and I missed it because I'm not very tuned in on this thread. -- Simon Riggs www.2ndQuadrant.com PostgreSQL Training, Services and Support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes

Re: [HACKERS] Walsender process patch v1 for Synch Rep

2008-11-07 Thread Simon Riggs
, for later discussion. Shutdown I arranged the shutdown timing of walsender. For example, in smart shutdown case, walsender should exit after bgwriter at least in order to replicate a shutdown checkpoint xlog. Agreed. -- Simon Riggs www.2ndQuadrant.com PostgreSQL

Re: [HACKERS] Updates of SE-PostgreSQL 8.4devel patches (r1197)

2008-11-07 Thread Simon Riggs
, I'd say diff -c. Good, thorough work. -- Simon Riggs www.2ndQuadrant.com PostgreSQL Training, Services and Support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Timing problem in DROP TABLESPACE?

2008-11-07 Thread Simon Riggs
On Thu, 2008-11-06 at 10:20 -0500, Tom Lane wrote: Alvaro Herrera [EMAIL PROTECTED] writes: Simon Riggs wrote: On a recent test the last command of the last test has failed: DROP TABLESPACE testspace; ERROR: tablespace testspace is not empty Maybe it is failing due to files

Re: [HACKERS] Updates of SE-PostgreSQL 8.4devel patches (r1197)

2008-11-07 Thread Simon Riggs
to be deleted by the lower privileged user, whereas in fact it was merely updated. There need not be any ordering to the labels for this scheme to work. -- Simon Riggs www.2ndQuadrant.com PostgreSQL Training, Services and Support -- Sent via pgsql-hackers mailing list (pgsql-hackers

Re: [HACKERS] Updates of SE-PostgreSQL 8.4devel patches (r1197)

2008-11-07 Thread Simon Riggs
not; I am unsure. Maybe. We already handle such complexity for comboids and multixacts, so I suggest we do the same thing here. Any system with more than 32,000 security contexts is going to be unmanageable and probably therefore insecure... -- Simon Riggs www.2ndQuadrant.com

Re: [HACKERS] Updates of SE-PostgreSQL 8.4devel patches (r1197)

2008-11-08 Thread Simon Riggs
On Fri, 2008-11-07 at 16:52 -0500, Bruce Momjian wrote: Simon Riggs wrote: So if somebody with context x tries to delete value1 from TableB, they will be refused because of a row they cannot see. In this case the correct action is to update the tuple in TableB so it now has

Re: [HACKERS] [PATCHES] Infrastructure changes for recovery (v8)

2008-11-08 Thread Simon Riggs
On Fri, 2008-11-07 at 15:44 -0800, Jeff Davis wrote: Is there a way to avoid wiping A and making a new base backup? rsync -- Simon Riggs www.2ndQuadrant.com PostgreSQL Training, Services and Support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make

Re: [HACKERS] Updates of SE-PostgreSQL 8.4devel patches (r1197)

2008-11-08 Thread Simon Riggs
that they would want to give Top Secret level access to DBAs. Nor should any level of DBAs be able to inspect who is running which SQL by simple manipulation of parameters such as log_min_duration_statement. -- Simon Riggs www.2ndQuadrant.com PostgreSQL Training, Services and Support -- Sent via

Re: [HACKERS] Updates of SE-PostgreSQL 8.4devel patches (r1197)

2008-11-08 Thread Simon Riggs
will be questioned again in the future, so it will be good to have them clear. Thanks. -- Simon Riggs www.2ndQuadrant.com PostgreSQL Training, Services and Support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org

Re: [HACKERS] pg_stop_backup wait bug fix

2008-11-08 Thread Simon Riggs
On Fri, 2008-11-07 at 18:32 -0300, Alvaro Herrera wrote: Simon Riggs wrote: Minor bug fix for pg_stop_backup() to prevent it waiting longer than necessary in certain circumstances. As far as I can tell, this patch needs to be applied to HEAD, 8.3 and 8.2 (when xlog switch was implemented

Re: [HACKERS] Reducing some DDL Locks to ShareLock

2008-11-09 Thread Simon Riggs
some more and report back. -- Simon Riggs www.2ndQuadrant.com PostgreSQL Training, Services and Support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Reducing some DDL Locks to ShareLock

2008-11-09 Thread Simon Riggs
On Sun, 2008-11-09 at 13:58 -0500, Tom Lane wrote: Simon Riggs [EMAIL PROTECTED] writes: On Tue, 2008-10-07 at 11:15 -0400, Tom Lane wrote: Simon Riggs [EMAIL PROTECTED] writes: 2. Also need to decide whether we want pg_class.reltriggers as int2 (as implemented here) or switch

Re: [HACKERS] Reducing some DDL Locks to ShareLock

2008-11-10 Thread Simon Riggs
On Sun, 2008-11-09 at 20:18 -0500, Tom Lane wrote: Simon Riggs [EMAIL PROTECTED] writes: Will think some more and report back. If you want to do some more development, here's the portion of the patch as yet unapplied --- will save you extracting it for yourself. Thanks. More thought

Re: [HACKERS] Walsender process patch v1 for Synch Rep

2008-11-10 Thread Simon Riggs
to run a long running transaction to initiate wal sending feature, if we do it just with user function. I'd like to see a longer README explaining these design aspects though. -- Simon Riggs www.2ndQuadrant.com PostgreSQL Training, Services and Support -- Sent via pgsql-hackers mailing

Re: [HACKERS] Reducing some DDL Locks to ShareLock

2008-11-11 Thread Simon Riggs
On Mon, 2008-11-10 at 19:15 -0500, Tom Lane wrote: The reason I was thinking about heap_lock_tuple is that it might provide a suitable defense against that case. OK. Lock tuple works OK, but its the unlock that I'm worried about. How would non-transactional un-lock tuple work? -- Simon

Re: [HACKERS] Reducing some DDL Locks to ShareLock

2008-11-12 Thread Simon Riggs
On Tue, 2008-11-11 at 21:57 -0500, Tom Lane wrote: Simon Riggs [EMAIL PROTECTED] writes: On Mon, 2008-11-10 at 19:15 -0500, Tom Lane wrote: The reason I was thinking about heap_lock_tuple is that it might provide a suitable defense against that case. OK. Lock tuple works OK, but its

Re: [HACKERS] Updates of SE-PostgreSQL 8.4devel patches (r1197)

2008-11-12 Thread Simon Riggs
large high security apps - which is catastrophic because many of them are very large these days. -- Simon Riggs www.2ndQuadrant.com PostgreSQL Training, Services and Support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http

Re: [HACKERS] Reducing some DDL Locks to ShareLock

2008-11-12 Thread Simon Riggs
On Wed, 2008-11-12 at 16:25 -0500, Tom Lane wrote: Simon Riggs [EMAIL PROTECTED] writes: On Tue, 2008-11-11 at 21:57 -0500, Tom Lane wrote: I was imagining that the heap_inplace_update operation would release the lock. Is there some problem with the concept? Not the concept, just

Re: [HACKERS] Enabling archive_mode without restart

2008-11-12 Thread Simon Riggs
working on this. We need to look at the full set of options we'll need for replication and hot standby. -- Simon Riggs www.2ndQuadrant.com PostgreSQL Training, Services and Support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription

Re: [HACKERS] Simple postgresql.conf wizard

2008-11-13 Thread Simon Riggs
. In any case, saying that somebody is certifiably insane in a public forum is at best questionable. I would like to see the comment withdrawn. -- Simon Riggs www.2ndQuadrant.com PostgreSQL Training, Services and Support -- Sent via pgsql-hackers mailing list (pgsql-hackers

Re: [HACKERS] Simple postgresql.conf wizard

2008-11-14 Thread Simon Riggs
mentioned to me privately that you disliked on-list bullies. -- Simon Riggs www.2ndQuadrant.com PostgreSQL Training, Services and Support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref

Re: [HACKERS] Simple postgresql.conf wizard

2008-11-14 Thread Simon Riggs
On Fri, 2008-11-14 at 08:57 +, Dave Page wrote: On Fri, Nov 14, 2008 at 8:10 AM, Simon Riggs [EMAIL PROTECTED] wrote: On Fri, 2008-11-14 at 02:21 +, Gregory Stark wrote: On the other hand what does occur to me in retrospect is that I regret that I didn't think about how I

Re: [HACKERS] Updates of SE-PostgreSQL 8.4devel patches (r1197)

2008-11-14 Thread Simon Riggs
. -- Simon Riggs www.2ndQuadrant.com PostgreSQL Training, Services and Support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Simple postgresql.conf wizard

2008-11-14 Thread Simon Riggs
On Fri, 2008-11-14 at 02:21 +, Gregory Stark wrote: Simon Riggs [EMAIL PROTECTED] writes: Your factual comments are accurate, but for Josh's stated target of Data Warehousing, a stats target of 400 is not unreasonable in some cases. What you forget to mention is that sample size

Re: [HACKERS] Updates of SE-PostgreSQL 8.4devel patches (r1197)

2008-11-14 Thread Simon Riggs
for existing data. -- Simon Riggs www.2ndQuadrant.com PostgreSQL Training, Services and Support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Synchronous replication patch v2

2008-11-14 Thread Simon Riggs
synch rep and we're committing. What happens when we're doing async rep or running something like a large load. I wouldn't want to presume that the network packet size and the disk write size are always identical. -- Simon Riggs www.2ndQuadrant.com PostgreSQL Training, Services

Re: [HACKERS] Synchronous replication patch v2

2008-11-14 Thread Simon Riggs
On Fri, 2008-11-14 at 19:23 +0200, Heikki Linnakangas wrote: Simon Riggs wrote: On Fri, 2008-11-14 at 19:00 +0200, Heikki Linnakangas wrote: Why do we need a separate XLogsndRqst variable in shared memory? Don't we always want to send the WAL up to the same point as we flush

Re: [HACKERS] Synchronous replication patch v2

2008-11-17 Thread Simon Riggs
, including Tom, review it too. Yes please to multiple reviewers. The Hot Standby patch can be considered v9 of the infrastructure patch, as mentioned on the wiki. I'll look to separate them so we review the correct code. -- Simon Riggs www.2ndQuadrant.com PostgreSQL Training

Re: [HACKERS] [PATCHES] Infrastructure changes for recovery (v8)

2008-11-17 Thread Simon Riggs
On Mon, 2008-11-17 at 15:51 +0200, Heikki Linnakangas wrote: Simon Riggs wrote: diff --git a/src/backend/access/transam/subtrans.c b/src/backend/access/transam/ index 063b366..5e64cb4 100644 --- a/src/backend/access/transam/subtrans.c +++ b/src/backend/access/transam/subtrans.c

Re: [HACKERS] [PATCHES] Infrastructure changes for recovery (v8)

2008-11-17 Thread Simon Riggs
something else that mitigates the problem? The flat file is not race condition proof. When the file is read it is just a guide and the real data is re-accessed from catalog. So the problem you see does exist, but is handled elsewhere - not in this patch. -- Simon Riggs www.2ndQuadrant.com

Re: [HACKERS] [PATCHES] Infrastructure changes for recovery (v8)

2008-11-17 Thread Simon Riggs
On Mon, 2008-11-17 at 16:18 +0200, Heikki Linnakangas wrote: Simon Riggs wrote: @@ -3845,6 +3850,52 @@ sigusr1_handler(SIGNAL_ARGS) PG_SETMASK(BlockSig); + if (CheckPostmasterSignal(PMSIGNAL_RECOVERY_START)) + { + Assert(pmState == PM_STARTUP

Re: [HACKERS] Synchronous replication patch v2

2008-11-17 Thread Simon Riggs
On Mon, 2008-11-17 at 18:56 +0200, Heikki Linnakangas wrote: Simon Riggs wrote: The Hot Standby patch can be considered v9 of the infrastructure patch, as mentioned on the wiki. I'll look to separate them so we review the correct code. Ok, I started reviewing the other v9 (http

Re: [HACKERS] Updates of SE-PostgreSQL 8.4devel patches (r1197)

2008-11-17 Thread Simon Riggs
on Red Hat. That's nice, but I want to see the patch open things up more for other distros/OS, plus options for people who don't want MAC, but do want row level security. -- Simon Riggs www.2ndQuadrant.com PostgreSQL Training, Services and Support -- Sent via pgsql-hackers mailing

Re: [HACKERS] Updates of SE-PostgreSQL 8.4devel patches (r1197)

2008-11-17 Thread Simon Riggs
On Tue, 2008-11-18 at 11:51 +0900, KaiGai Kohei wrote: Simon Riggs wrote: The other is an issue on implementation. Even if row-level security is disabled, tuples within pg_class, pg_attribute, pg_proc and pg_largeobject has to hold its security context, because it means the security

Re: [HACKERS] Updates of SE-PostgreSQL 8.4devel patches (r1197)

2008-11-17 Thread Simon Riggs
On Tue, 2008-11-18 at 13:10 +0900, KaiGai Kohei wrote: KaiGai Kohei wrote: Simon Riggs wrote: The other is an issue on implementation. Even if row-level security is disabled, tuples within pg_class, pg_attribute, pg_proc and pg_largeobject has to hold its security context, because

Re: [HACKERS] Updates of SE-PostgreSQL 8.4devel patches (r1197)

2008-11-17 Thread Simon Riggs
SELinux now? I have not heard that. -- Simon Riggs www.2ndQuadrant.com PostgreSQL Training, Services and Support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Updates of SE-PostgreSQL 8.4devel patches (r1197)

2008-11-18 Thread Simon Riggs
AppArmor. However, I don't have enough information for the roadmap of SUSE Enterprise Server which is a production version of Novell. OK, I heard that slightly differently. Thanks for the link. The report I read emphasised the lack of support. -- Simon Riggs www.2ndQuadrant.com

Re: [HACKERS] Updates of SE-PostgreSQL 8.4devel patches (r1197)

2008-11-18 Thread Simon Riggs
things right takes time and it really is worth it in the end. -- Simon Riggs www.2ndQuadrant.com PostgreSQL Training, Services and Support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref

Re: [HACKERS] Updates of SE-PostgreSQL 8.4devel patches (r1197)

2008-11-18 Thread Simon Riggs
On Wed, 2008-11-19 at 01:42 +0900, KaiGai Kohei wrote: Simon, I'm sorry, if my expression felt you uncomfortable. No worries at all. I know exactly how you feel. Good Luck. -- Simon Riggs www.2ndQuadrant.com PostgreSQL Training, Services and Support -- Sent via pgsql-hackers

Re: [HACKERS] Reducing some DDL Locks to ShareLock

2008-11-18 Thread Simon Riggs
the buffer, locks it, then releases the original tuple. It then returns a copy of the on-block tuple. So all other code the same as before when we were working on a copy produced from the syscache. Is that roughly what you intended? -- Simon Riggs www.2ndQuadrant.com PostgreSQL Training

Re: [HACKERS] [PATCHES] Infrastructure changes for recovery (v8)

2008-11-20 Thread Simon Riggs
On Thu, 2008-11-20 at 11:06 +0530, Pavan Deolasee wrote: On Mon, Nov 17, 2008 at 9:01 PM, Simon Riggs [EMAIL PROTECTED] wrote: It is, in a later version. Apologies if you're reviewing the wrong one. The most recent version I can

Re: [HACKERS] Hot Standby (commit fest version - v5)

2008-11-20 Thread Simon Riggs
with that change and I'll make it so. -- Simon Riggs www.2ndQuadrant.com PostgreSQL Training, Services and Support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] [PATCHES] Infrastructure changes for recovery (v8)

2008-11-20 Thread Simon Riggs
On Thu, 2008-11-20 at 15:19 +0530, Pavan Deolasee wrote: Do you intend to split the patch into smaller pieces ? The latest hot standby patch is almost 10K+ lines. Splitting that would definitely help the review process. If it helps you, then I'll do it. Hang on an hour or so. -- Simon

Re: [HACKERS] Hot Standby (commit fest version - v5)

2008-11-20 Thread Simon Riggs
On Thu, 2008-11-20 at 12:03 +0530, Pavan Deolasee wrote: On Sat, Nov 1, 2008 at 10:02 PM, Simon Riggs [EMAIL PROTECTED] wrote: Hot Standby patch, including all major planned features. While experimenting with the patch, I noticed that sometimes the archiver process

Re: [HACKERS] Hot Standby (commit fest version - v5)

2008-11-20 Thread Simon Riggs
and decided it was more difficult than it first appeared. So I've left it for now. -- Simon Riggs www.2ndQuadrant.com PostgreSQL Training, Services and Support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http

Re: [HACKERS] Hot Standby (commit fest version - v5)

2008-11-21 Thread Simon Riggs
reached the point at which we pmsignal or not. -- Simon Riggs www.2ndQuadrant.com PostgreSQL Training, Services and Support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Review: Hot standby

2008-11-21 Thread Simon Riggs
a #define because same value is used elsewhere in code. The use of malloc should also be avoided (unless the memory subsystem is not up yet; I haven't checked). The malloc was part of the existing code, explained by comments. -- Simon Riggs www.2ndQuadrant.com PostgreSQL Training

Re: [HACKERS] Review: Hot standby

2008-11-21 Thread Simon Riggs
wait for the lock. This won't happen because the lock is held by the startup process on behalf of slot 1. Explained in comments in inval.c code. -- Simon Riggs www.2ndQuadrant.com PostgreSQL Training, Services and Support -- Sent via pgsql-hackers mailing list (pgsql-hackers

Re: [HACKERS] Opening a recovering DB in for read-only access?

2008-11-21 Thread Simon Riggs
data with this approach. -- Simon Riggs www.2ndQuadrant.com PostgreSQL Training, Services and Support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] How should pg_standby get over the gap of timeline?

2008-11-21 Thread Simon Riggs
, hacking pg_standby is not the way to do it. But I'm not convinced there is a problem worth solving. -- Simon Riggs www.2ndQuadrant.com PostgreSQL Training, Services and Support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your

Re: [HACKERS] How should pg_standby get over the gap of timeline?

2008-11-22 Thread Simon Riggs
On Sat, 2008-11-22 at 03:39 +0900, Fujii Masao wrote: Hi, Simon. Thanks for the comment!! On Sat, Nov 22, 2008 at 2:09 AM, Simon Riggs [EMAIL PROTECTED] wrote: On Thu, 2008-11-20 at 22:41 +0900, Fujii Masao wrote: In the current Synch Rep patch, the standby cannot catch up

Re: [HACKERS] Updates of SE-PostgreSQL 8.4devel patches (r1197)

2008-11-22 Thread Simon Riggs
. This allows us to include the feature in the main release package, which was my main objective. -- Simon Riggs www.2ndQuadrant.com PostgreSQL Training, Services and Support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription

Re: [HACKERS] Review: Hot standby

2008-11-22 Thread Simon Riggs
in many ways. It only ever runs within bgwriter, and again, once allocated the memory is never released. So malloc is appropriate there also. I will change the #define as you suggested earlier. -- Simon Riggs www.2ndQuadrant.com PostgreSQL Training, Services and Support -- Sent

Re: [HACKERS] Review: Hot standby

2008-11-23 Thread Simon Riggs
it didn't seem worth trying to plug the gap. -- Simon Riggs www.2ndQuadrant.com PostgreSQL Training, Services and Support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Review: Hot standby

2008-11-25 Thread Simon Riggs
with protective changes. You've identified a way of breaking part the first line of defence, but the command was caught by the second line of defence in the patch. Problem, yes. Major issue, no. Will fix. -- Simon Riggs www.2ndQuadrant.com PostgreSQL Training, Services and Support

Re: [HACKERS] Updates of SE-PostgreSQL 8.4devel patches (r1197)

2008-11-25 Thread Simon Riggs
to be labeled as 'lib_t' type which means shared library files installed correctly. We definitely want to include add-in modules with high security systems, e.g. GIS and oracle compatibility functions. -- Simon Riggs www.2ndQuadrant.com PostgreSQL Training, Services and Support -- Sent

Re: [HACKERS] Review: Hot standby

2008-11-27 Thread Simon Riggs
been tied up with a few things over last few days, but I'm free of that now). -- Simon Riggs www.2ndQuadrant.com PostgreSQL Training, Services and Support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http

Re: [HACKERS] New trigger file in pg_standby to promote the standby to the primary

2008-11-27 Thread Simon Riggs
, sometimes we want complete catchup. So my solution is to put a message in the trigger file to say which one we want. Happy for the default to change to complete catchup. -- Simon Riggs www.2ndQuadrant.com PostgreSQL Training, Services and Support -- Sent via pgsql-hackers

Re: [HACKERS] blatantly a bug in the documentation

2008-11-27 Thread Simon Riggs
better if we have table and column names in the local language. One more thing to translate maybe, but lets leave the door open for localisation. -- Simon Riggs www.2ndQuadrant.com PostgreSQL Training, Services and Support -- Sent via pgsql-hackers mailing list (pgsql-hackers

Re: [HACKERS] Distinct types

2008-11-27 Thread Simon Riggs
by Jeff Davis. Shame, this was sorely needed. -- Simon Riggs www.2ndQuadrant.com PostgreSQL Training, Services and Support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Review: Hot standby

2008-11-28 Thread Simon Riggs
On Thu, 2008-11-27 at 17:14 +, Simon Riggs wrote: On Wed, 2008-11-26 at 18:02 +0530, Pavan Deolasee wrote: I think whats happening is that ResolveRecoveryConflictWithVirtualXIDs() is failing to abort the open transaction Btw, ISTM that SIGINT works

Re: [HACKERS] Immediate shutdown during recovery

2008-11-28 Thread Simon Riggs
file \%s\ from archive: return code %d, xlogfname, rc))); Agree there is an existing problem. Suggest we fix it after the main patches are committed. -- Simon Riggs www.2ndQuadrant.com PostgreSQL Training, Services and Support -- Sent via pgsql-hackers mailing list

Re: [HACKERS] Review: Hot standby

2008-11-28 Thread Simon Riggs
On Fri, 2008-11-28 at 11:14 -0500, Tom Lane wrote: Simon Riggs [EMAIL PROTECTED] writes: After some thought, the way I would handle this is by sending a slightly different kind of signal. We can send a shared invalidation message which means end the transaction, whether or not you

Re: [HACKERS] Distinct types

2008-11-28 Thread Simon Riggs
On Fri, 2008-11-28 at 11:20 -0500, Tom Lane wrote: Peter Eisentraut [EMAIL PROTECTED] writes: Simon Riggs wrote: Shame, this was sorely needed. I understand, but the work required to make it work properly is too much under the commit fest spirit right now. Personally I

Re: [HACKERS] Review: Hot standby

2008-11-28 Thread Simon Riggs
On Fri, 2008-11-28 at 11:44 -0500, Tom Lane wrote: Simon Riggs [EMAIL PROTECTED] writes: On Fri, 2008-11-28 at 11:14 -0500, Tom Lane wrote: The sinval queue is an *utterly* inappropriate mechanism for such a thing. To be honest, it did seem quite a neat solution. Any particular

Re: [HACKERS] Review: Hot standby

2008-11-28 Thread Simon Riggs
On Fri, 2008-11-28 at 12:45 -0500, Tom Lane wrote: Simon Riggs [EMAIL PROTECTED] writes: On Fri, 2008-11-28 at 11:44 -0500, Tom Lane wrote: I hadn't been following the discussion closely enough to know what the problem is. When we replay an AccessExclusiveLock on the standby we need

[HACKERS] Sync Rep: First Thoughts on Code

2008-12-01 Thread Simon Riggs
. -- Simon Riggs www.2ndQuadrant.com PostgreSQL Training, Services and Support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] V2 of PITR performance improvement for 8.4

2008-12-01 Thread Simon Riggs
replication would we be able to forget about doing the prefetch altogether? -- Simon Riggs www.2ndQuadrant.com PostgreSQL Training, Services and Support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org

Re: [HACKERS] Sync Rep: First Thoughts on Code

2008-12-02 Thread Simon Riggs
On Tue, 2008-12-02 at 21:37 +0900, Fujii Masao wrote: Thanks for taking many hours to review the code!! On Mon, Dec 1, 2008 at 8:42 PM, Simon Riggs [EMAIL PROTECTED] wrote: Can you confirm that all the docs on the Wiki page are up to date? There are a few minor discrepancies that make me

Re: [HACKERS] PiTR and other architectures....

2008-12-02 Thread Simon Riggs
. Is the risk worth it? -- Simon Riggs www.2ndQuadrant.com PostgreSQL Training, Services and Support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Sync Rep: First Thoughts on Code

2008-12-02 Thread Simon Riggs
On Tue, 2008-12-02 at 11:08 -0800, Jeff Davis wrote: On Tue, 2008-12-02 at 13:09 +, Simon Riggs wrote: Is it dangerous to abort the transaction with replication continued when the timeout occurs? I think that the WAL consistency between two servers might be broken. Because the WAL

Re: [HACKERS] Sync Rep: First Thoughts on Code

2008-12-03 Thread Simon Riggs
definitely need the archiver to move the files written by walreceiver to archive and then move them back out again? Seems like we can streamline that part in many (all?) cases. -- Simon Riggs www.2ndQuadrant.com PostgreSQL Training, Services and Support -- Sent via pgsql-hackers mailing

Re: [HACKERS] V2 of PITR performance improvement for 8.4

2008-12-04 Thread Simon Riggs
makes sense, but not until then. So far many people have argued in favour of using FPW=on, which was the whole point of pg_lesslog. Are we now saying that we would run the primary with FPW=off? -- Simon Riggs www.2ndQuadrant.com PostgreSQL Training, Services and Support -- Sent via

Re: [HACKERS] Sync Rep: First Thoughts on Code

2008-12-04 Thread Simon Riggs
need to worry about the obsolescence of the key. Understood. Is the periodic renegotiation of keys something that would interfere with the performance or robustness of replication? Is the delay likely to effect sync rep? I'm just checking we've thought about it. -- Simon Riggs www

Re: [HACKERS] Sync Rep: First Thoughts on Code

2008-12-04 Thread Simon Riggs
that point. -- Simon Riggs www.2ndQuadrant.com PostgreSQL Training, Services and Support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Sync Rep: First Thoughts on Code

2008-12-04 Thread Simon Riggs
it - or refuse to use - not disable renegotiation. I didn't mean to imply renegotiation might optional. I just wanted to check whether there is anything to worry about as a result of it, there may not be. *If* it took a long time, I would not want sync commits to wait for it. -- Simon Riggs

Re: [HACKERS] Sync Rep: First Thoughts on Code

2008-12-05 Thread Simon Riggs
* standby restarts to change shmmem settings etc -- Simon Riggs www.2ndQuadrant.com PostgreSQL Training, Services and Support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Sync Rep: First Thoughts on Code

2008-12-05 Thread Simon Riggs
On Fri, 2008-12-05 at 12:09 +0900, Fujii Masao wrote: On Thu, Dec 4, 2008 at 6:29 PM, Simon Riggs [EMAIL PROTECTED] wrote: The only sensible settings are synchronous_commit = on, synchronous_replication = on synchronous_commit = on, synchronous_replication = off synchronous_commit = off

Re: [HACKERS] Sync Rep: First Thoughts on Code

2008-12-08 Thread Simon Riggs
complex than the original. Forgive me, but I don't understand that. Can you explain? What is the procedure if the standby shuts down, for example if we wish to restart server to change a parameter? Or to reboot the system it is on. Does the primary switch back to writing files to archive? -- Simon

Re: [HACKERS] Multiplexing SUGUSR1

2008-12-08 Thread Simon Riggs
On Mon, 2008-12-08 at 10:04 +0200, Heikki Linnakangas wrote: And is the performance impact of that acceptable? No, but I think we already agreed to change that to a separate lwlock. -- Simon Riggs www.2ndQuadrant.com PostgreSQL Training, Services and Support -- Sent via pgsql

Re: [HACKERS] Synchronous replication: reading recovery.conf

2008-12-08 Thread Simon Riggs
also already been made on the existing review thread. We're discussing some architectural aspects, so it would be a good if you were there also. It's important we all agree on those things. -- Simon Riggs www.2ndQuadrant.com PostgreSQL Training, Services and Support -- Sent via

<    9   10   11   12   13   14   15   16   17   18   >