Re: [HACKERS] Synchronous Log Shipping Replication

2008-09-09 Thread Pavan Deolasee
On Wed, Sep 10, 2008 at 12:05 PM, Hannu Krosing <[EMAIL PROTECTED]> wrote: > > >> Because, at the time, current WAL position might be in the middle of >> WAL file. Even if the master sends only current WAL data, the slave >> which don't have the corresponding WAL file can not handle it. > > I agree

Re: [HACKERS] Synchronous Log Shipping Replication

2008-09-09 Thread Hannu Krosing
On Wed, 2008-09-10 at 15:15 +0900, Fujii Masao wrote: > On Wed, Sep 10, 2008 at 12:26 AM, Heikki Linnakangas > <[EMAIL PROTECTED]> wrote: > > If a slave falls behind, how does it catch up? I guess you're saying that it > > can't fall behind, because the master will block before that happens. Also >

Re: [HACKERS] Keeping creation time of objects

2008-09-09 Thread Volkan YAZICI
On Tue, 9 Sep 2008, David Fetter <[EMAIL PROTECTED]> writes: >> AFAICS, PostgreSQL is not keeping info about when a table, database, >> sequence, etc was created. We cannot get that info even from OS, >> since CLUSTER or VACUUM FULL may change the metadata of >> corresponding relfilenode. > > When

Re: [HACKERS] Synchronous Log Shipping Replication

2008-09-09 Thread Fujii Masao
On Wed, Sep 10, 2008 at 12:26 AM, Heikki Linnakangas <[EMAIL PROTECTED]> wrote: > If a slave falls behind, how does it catch up? I guess you're saying that it > can't fall behind, because the master will block before that happens. Also > in asynchronous replication? And what about when the slave is

Re: [HACKERS] Synchronous Log Shipping Replication

2008-09-09 Thread Fujii Masao
On Tue, Sep 9, 2008 at 8:42 PM, Markus Wanner <[EMAIL PROTECTED]> wrote: >> In the viewpoint of detection of a network failure, this feature is >> necessary. >> When the network goes down, WAL sender can be blocked until it detects >> the network failure, i.e. WAL sender keeps waiting for the respo

Re: [HACKERS] Synchronous Log Shipping Replication

2008-09-09 Thread Fujii Masao
On Tue, Sep 9, 2008 at 8:38 PM, Heikki Linnakangas <[EMAIL PROTECTED]> wrote: > There's one thing I haven't figured out in this discussion. Does the write > to the disk happen before or after the write to the slave? Can you guarantee > that if a transaction is committed in the master, it's also com

Re: [HACKERS] Keeping creation time of objects

2008-09-09 Thread Andrew Dunstan
Robert Haas wrote: I try pretty hard not to shoot myself in the foot. But if someone comes up to me and offers me some shoes that are have the same cost, appearance, comfort-level, and durability as regular shoes but are slightly more bullet resistant, should I refuse them on principle? Why?

Re: [HACKERS] Keeping creation time of objects

2008-09-09 Thread Robert Haas
> When people aren't keeping track of their DDL, that is very strictly a > process problem on their end. When people are shooting themselves in > the foot, it's a great disservice to market Kevlar shoes to them. I can't believe anyone is going to stop tracking their DDL because, ooh goody, now we

Re: [HACKERS] [PATCH] Cleanup of GUC units code

2008-09-09 Thread Robert Haas
>> A good start might be to always OUTPUT memory parameters using the >> same base unit. > SHOW gives output that matches what you input. Not for me it doesn't. portal=# show temp_buffers; temp_buffers -- 1024 (1 row) portal=# set temp_buffers = '16MB'; SET portal=# show temp_buffe

Re: [HACKERS] [PATCH] Cleanup of GUC units code

2008-09-09 Thread Greg Smith
On Tue, 9 Sep 2008, Tom Lane wrote: How do you come to that conclusion? Leaving off the unit entirely certainly doesn't make the user's intent clearer. Same way I do all my conclusions in this area--talking to people in the field regularly who've never configured a postgresql.conf before. I

Re: [HACKERS] Potential Join Performance Issue

2008-09-09 Thread Tom Lane
"Lawrence, Ramon" <[EMAIL PROTECTED]> writes: > Our research group has been using the PostgreSQL code base to test new > join algorithms. During testing, we noticed that the planner is not > pushing down projections to the outer relation in a hash join. Although > this makes sense for in-memory (

Re: [HACKERS] [PATCHES] to_date() validation

2008-09-09 Thread Alex Hunsaker
On Tue, Sep 9, 2008 at 6:46 AM, Brendan Jurd <[EMAIL PROTECTED]> wrote: > On Tue, Sep 9, 2008 at 9:04 PM, Brendan Jurd <[EMAIL PROTECTED]> wrote: >> On Tue, Sep 9, 2008 at 7:29 PM, Martijn van Oosterhout >> <[EMAIL PROTECTED]> wrote: >>> The use of palloc/pfree in this routine seems excessive. Does

Re: [HACKERS] pg_regress inputdir

2008-09-09 Thread Tom Lane
Alvaro Herrera <[EMAIL PROTECTED]> writes: > Peter Eisentraut wrote: >> There is interest among packagers to run the regression tests or other >> tests after the build. The Red Hat RPMs have shipped a postgresql-test >> package for years with a hacked-up makefile that will probably overwrite

Re: [HACKERS] [PATCHES] to_date() validation

2008-09-09 Thread Alex Hunsaker
On Mon, Sep 8, 2008 at 2:24 AM, Brendan Jurd <[EMAIL PROTECTED]> wrote: > HEAD actually gets this one wrong; in defiance of the documentation it > returns 2000-09-07. So, it seems to me that the patch shifts the > behaviour in the right direction. > > Barring actually teaching the code that some n

Re: [HACKERS] [PATCH] Cleanup of GUC units code

2008-09-09 Thread Tom Lane
Greg Smith <[EMAIL PROTECTED]> writes: > What I would like to see (but don't > have nearly enough time to argue in support of considering the resistance > to change here) is that this syntax: > shared_buffers=1024 > Would assume the user meant 1024 *bytes*, with the server silently > rounding

Re: [HACKERS] Keeping creation time of objects

2008-09-09 Thread Gregory Williamson
Andrew Dunstan wrote: <...> > > Can someone please give a good, concrete use case for this stuff? "Might > be nice to have" doesn't cut it, I'm afraid. In particular, I'd like to > know why logging statements won't do the trick here. > Please pardon the kibbitzer intrusion ... Informix has t

[HACKERS] Proposed patch: make SQL interval-literal syntax work per spec

2008-09-09 Thread Tom Lane
Over in that TPC-H thread, I was bemoaning once again the never-finished support for SQL-spec interval literals. I decided to go look at exactly how unfinished it was, and it turns out that it's actually pretty close. Hence the attached proposed patch ;-) The main gating factor is that coerce_typ

Re: [HACKERS] Keeping creation time of objects

2008-09-09 Thread David Fetter
On Tue, Sep 09, 2008 at 10:20:00PM +0300, Devrim GUNDUZ wrote: > Hi, > > AFAICS, PostgreSQL is not keeping info about when a table, database, > sequence, etc was created. We cannot get that info even from OS, > since CLUSTER or VACUUM FULL may change the metadata of > corresponding relfilenode.

Re: [HACKERS] [PATCH] Cleanup of GUC units code

2008-09-09 Thread Greg Smith
On Tue, 9 Sep 2008, Robert Haas wrote: A good start might be to always OUTPUT memory parameters using the same base unit. SHOW gives output that matches what you input. If you want to see things with consistant units, look at pg_settings: # select name,unit,setting,current_setting(name) fr

Re: [HACKERS] [PATCH] Cleanup of GUC units code

2008-09-09 Thread Hannu Krosing
On Thu, 2008-09-04 at 11:51 -0400, Andrew Sullivan wrote: > On Thu, Sep 04, 2008 at 07:01:18AM -0700, Steve Atkins wrote: > > Settings in postgresql.conf are currently case-insensitive. Except > > for the units. > > And, of course, filenames when you are using a case-sensitive > filesystem. Becau

Re: [HACKERS] Keeping creation time of objects

2008-09-09 Thread Devrim GÜNDÜZ
Hi Andrew, On Tue, 2008-09-09 at 16:22 -0400, Andrew Dunstan wrote: > I'd like to know why logging statements won't do the trick here. It is not on by default, logs are rotated, and may be lost, etc. Regards, -- Devrim GÜNDÜZ, RHCE devrim~gunduz.org, devrim~PostgreSQL.org, devrim.gunduz~linux.o

Re: [HACKERS] Keeping creation time of objects

2008-09-09 Thread Andrew Dunstan
daveg wrote: On Tue, Sep 09, 2008 at 11:03:56PM +0300, Hannu Krosing wrote: On Tue, 2008-09-09 at 12:40 -0700, daveg wrote: I'd be very interested in seeing a last schema modification time for pg_class objects. I don't care about it being preserved over dump and restore as my use cas

Re: [HACKERS] Keeping creation time of objects

2008-09-09 Thread daveg
On Tue, Sep 09, 2008 at 11:03:56PM +0300, Hannu Krosing wrote: > On Tue, 2008-09-09 at 12:40 -0700, daveg wrote: > > > I'd be very interested in seeing a last schema modification time for > > pg_class > > objects. I don't care about it being preserved over dump and restore as my > > use case is m

Re: [HACKERS] Common Table Expressions (WITH RECURSIVE) patch

2008-09-09 Thread Tom Lane
"Robert Haas" <[EMAIL PROTECTED]> writes: >> I am blind, I didn't find any reason, why materialisation isn't useable. > I believe it's because of these two (closely related) problems: > # The basic > # implementation clearly ought to be to dump the result of the subquery > # into a tuplestore an

Re: [HACKERS] Keeping creation time of objects

2008-09-09 Thread Hannu Krosing
On Tue, 2008-09-09 at 12:40 -0700, daveg wrote: > I'd be very interested in seeing a last schema modification time for pg_class > objects. I don't care about it being preserved over dump and restore as my > use case is more to find out when a table was created with a view to finding > out if it is

Re: [HACKERS] Keeping creation time of objects

2008-09-09 Thread Robert Haas
> There isn't sufficient support for such a "feature". It sounds like a useful feature to me. > In any case, why > would creation time (as opposed to any other time, eg last schema > modification, last data modification, yadda yadda) be especially > significant? Those would be nice to have too,

Re: [HACKERS] Keeping creation time of objects

2008-09-09 Thread Devrim GÜNDÜZ
On Tue, 2008-09-09 at 15:36 -0400, Tom Lane wrote: > why would creation time (as opposed to any other time, eg last schema > modification, last data modification, yadda yadda) be especially > significant? Hmm, those would be cool, too. Seriously, I believe we can get last data modification from

Re: [HACKERS] Keeping creation time of objects

2008-09-09 Thread daveg
On Tue, Sep 09, 2008 at 03:36:19PM -0400, Tom Lane wrote: > Devrim =?ISO-8859-1?Q?G=DCND=DCZ?= <[EMAIL PROTECTED]> writes: > > AFAICS, PostgreSQL is not keeping info about when a table, database, > > sequence, etc was created. We cannot get that info even from OS, since > > CLUSTER or VACUUM FULL m

Re: [HACKERS] Keeping creation time of objects

2008-09-09 Thread Tom Lane
Devrim =?ISO-8859-1?Q?G=DCND=DCZ?= <[EMAIL PROTECTED]> writes: > AFAICS, PostgreSQL is not keeping info about when a table, database, > sequence, etc was created. We cannot get that info even from OS, since > CLUSTER or VACUUM FULL may change the metadata of corresponding > relfilenode. > Does any

Re: [HACKERS] [PATCH] Cleanup of GUC units code

2008-09-09 Thread Robert Haas
> This would have definitional problems of its own, however. If you interpret > K, M, and G strictly as unit-less multipliers, then > > SET shared_buffers = 2 G I don't think it would be a good idea to make them unit-less, for exactly the reasons you mention. > We could possibly settle some of t

[HACKERS] Keeping creation time of objects

2008-09-09 Thread Devrim GÜNDÜZ
Hi, AFAICS, PostgreSQL is not keeping info about when a table, database, sequence, etc was created. We cannot get that info even from OS, since CLUSTER or VACUUM FULL may change the metadata of corresponding relfilenode. Does anyone think that adding a timestamp column to pg_class would bring an

Re: [HACKERS] Patch: plan invalidation vs stored procedures

2008-09-09 Thread Tom Lane
Martin Pihlak <[EMAIL PROTECTED]> writes: > Yes, creating a new message type was a bit short sighted -- attached is a > patch > that uses syscache invalidation messages instead. This also adds additional > tupleId field to SharedInvalCatcacheMsg. This is used to identify the > invalidated tuple in

Re: [HACKERS] Synchronous Log Shipping Replication

2008-09-09 Thread Zeugswetter Andreas OSB sIT
> > Don't understand. I am referring to the logic at the top of > > AdvanceXLInsertBuffer(). We would need to wait for all people reading > > the contents of wal_buffers. > > Oh, I see. > > If a slave falls behind, how does it catch up? I guess you're saying > that it can't fall behind, because th

[HACKERS] Potential Join Performance Issue

2008-09-09 Thread Lawrence, Ramon
PostgreSQL development community: Our research group has been using the PostgreSQL code base to test new join algorithms. During testing, we noticed that the planner is not pushing down projections to the outer relation in a hash join. Although this makes sense for in-memory (1 batch) joins,

Re: [HACKERS] pg_regress inputdir

2008-09-09 Thread Alvaro Herrera
Peter Eisentraut wrote: > There is interest among packagers to run the regression tests or other > tests after the build. The Red Hat RPMs have shipped a postgresql-test > package for years with a hacked-up makefile that will probably overwrite > random files that it shouldn't in /usr/lib.

Re: [HACKERS] Common Table Expressions (WITH RECURSIVE) patch

2008-09-09 Thread Pavel Stehule
2008/9/9 Jeff Davis <[EMAIL PROTECTED]>: > On Tue, 2008-09-09 at 18:51 +0200, Pavel Stehule wrote: >> hmm. I solve similar problem in grouping sets :( etc >> I have special executor node - feeder, it hold one tuple and others nodes read from this node. It's usable for hash aggregates. Pavel plan

Re: [HACKERS] Common Table Expressions (WITH RECURSIVE) patch

2008-09-09 Thread Jeff Davis
On Tue, 2008-09-09 at 18:51 +0200, Pavel Stehule wrote: > hmm. I solve similar problem in grouping sets :( etc > How did you solve it? Regards, Jeff Davis -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.

Re: [HACKERS] [PATCH] Cleanup of GUC units code

2008-09-09 Thread Peter Eisentraut
Tom Lane wrote: "Greg Stark" <[EMAIL PROTECTED]> writes: On Mon, Sep 8, 2008 at 2:11 PM, Tom Lane <[EMAIL PROTECTED]> wrote: But of course case insensitivity isn't going to fix that example for you. So we're right back at the question of where we should draw the line in trying to accept variant

Re: [HACKERS] Common Table Expressions (WITH RECURSIVE) patch

2008-09-09 Thread Pavel Stehule
2008/9/9 Robert Haas <[EMAIL PROTECTED]>: >> My interpretation of 7.13: General Rules: 2.b is that it should be >> single evaluation, even if RECURSIVE is present. >> >> The previous discussion was here: >> >> http://archives.postgresql.org/pgsql-hackers/2008-07/msg012

Re: [HACKERS] Window functions patch v04 for the September commit fest

2008-09-09 Thread Hitoshi Harada
>> Also, current implementation has only a type of plan which uses sort >> operation. It should be optimized by re-position the windows and/or >> using hashtable. > > I would like to see some performance test results also. It would be good > to know whether they are fast/slow etc.. It will definite

Re: [HACKERS] Common Table Expressions (WITH RECURSIVE) patch

2008-09-09 Thread Robert Haas
>>> >> My interpretation of 7.13: General Rules: 2.b is that it should be >>> >> single evaluation, even if RECURSIVE is present. >>> >> >>> >> The previous discussion was here: >>> >> >>> >> http://archives.postgresql.org/pgsql-hackers/2008-07/msg01292.php > > I am blind, I didn't find any reason

Re: [HACKERS] Common Table Expressions (WITH RECURSIVE) patch

2008-09-09 Thread Pavel Stehule
2008/9/9 Tatsuo Ishii <[EMAIL PROTECTED]>: >> Hello >> >> 2008/9/9 Tatsuo Ishii <[EMAIL PROTECTED]>: >> >> On Tue, 2008-09-09 at 13:45 +0900, Tatsuo Ishii wrote: >> >> > Thanks for the review. >> >> > >> >> > > The standard specifies that non-recursive WITH should be evaluated >> >> > > once. >

Re: [HACKERS] Synchronous Log Shipping Replication

2008-09-09 Thread Simon Riggs
On Tue, 2008-09-09 at 18:26 +0300, Heikki Linnakangas wrote: > Simon Riggs wrote: > > Don't understand. I am referring to the logic at the top of > > AdvanceXLInsertBuffer(). We would need to wait for all people reading > > the contents of wal_buffers. > > Oh, I see. > > If a slave falls behind

Re: [HACKERS] Synchronous Log Shipping Replication

2008-09-09 Thread Heikki Linnakangas
Simon Riggs wrote: Don't understand. I am referring to the logic at the top of AdvanceXLInsertBuffer(). We would need to wait for all people reading the contents of wal_buffers. Oh, I see. If a slave falls behind, how does it catch up? I guess you're saying that it can't fall behind, because

Re: [HACKERS] Verbosity of Function Return Type Checks

2008-09-09 Thread Alvaro Herrera
Volkan YAZICI wrote: > On Mon, 8 Sep 2008, Alvaro Herrera <[EMAIL PROTECTED]> writes: > >> Modified as you suggested. BTW, will there be a similar i18n scenario > >> for "dropped column" you mentioned below? > > > > Yes, you need _() around those too. > > For this purpose, I introduced a dropped_c

Re: [HACKERS] Common Table Expressions (WITH RECURSIVE) patch

2008-09-09 Thread Tatsuo Ishii
> > * Aggregates allowed: > > > > with recursive foo(i) as > > (values(1) > > union all > > select max(i)+1 from foo where i < 10) > > select * from foo; > > > > Aggregates should be blocked according to the standard. > > Also, causes an infinite loop. This should be fixed for

Re: [HACKERS] Synchronous Log Shipping Replication

2008-09-09 Thread Simon Riggs
On Tue, 2008-09-09 at 17:17 +0300, Heikki Linnakangas wrote: > Tom Lane wrote: > > Simon Riggs <[EMAIL PROTECTED]> writes: > >> On Tue, 2008-09-09 at 08:24 -0400, Tom Lane wrote: > >>> "Agreed"? That last restriction is a deal-breaker. > > > >> OK, I should have said *if wal_buffers are full* XL

Re: [HACKERS] Common Table Expressions (WITH RECURSIVE) patch

2008-09-09 Thread Tatsuo Ishii
> Hello > > 2008/9/9 Tatsuo Ishii <[EMAIL PROTECTED]>: > >> On Tue, 2008-09-09 at 13:45 +0900, Tatsuo Ishii wrote: > >> > Thanks for the review. > >> > > >> > > The standard specifies that non-recursive WITH should be evaluated > >> > > once. > >> > > >> > What shall we do? I don't think there

Re: [HACKERS] Common Table Expressions (WITH RECURSIVE) patch

2008-09-09 Thread Tatsuo Ishii
> Hello > > 2008/9/9 Tatsuo Ishii <[EMAIL PROTECTED]>: > >> On Tue, 2008-09-09 at 13:45 +0900, Tatsuo Ishii wrote: > >> > Thanks for the review. > >> > > >> > > The standard specifies that non-recursive WITH should be evaluated > >> > > once. > >> > > >> > What shall we do? I don't think there

Re: [HACKERS] Common Table Expressions (WITH RECURSIVE) patch

2008-09-09 Thread Pavel Stehule
Hello 2008/9/9 Tatsuo Ishii <[EMAIL PROTECTED]>: >> On Tue, 2008-09-09 at 13:45 +0900, Tatsuo Ishii wrote: >> > Thanks for the review. >> > >> > > The standard specifies that non-recursive WITH should be evaluated >> > > once. >> > >> > What shall we do? I don't think there's a easy way to fix

Re: [HACKERS] Common Table Expressions (WITH RECURSIVE) patch

2008-09-09 Thread Tatsuo Ishii
> On Tue, 2008-09-09 at 13:45 +0900, Tatsuo Ishii wrote: > > Thanks for the review. > > > > > The standard specifies that non-recursive WITH should be evaluated > > > once. > > > > What shall we do? I don't think there's a easy way to fix this. Maybe > > we should not allow WITH clause withou

Re: [HACKERS] Synchronous Log Shipping Replication

2008-09-09 Thread Heikki Linnakangas
Tom Lane wrote: Simon Riggs <[EMAIL PROTECTED]> writes: On Tue, 2008-09-09 at 08:24 -0400, Tom Lane wrote: "Agreed"? That last restriction is a deal-breaker. OK, I should have said *if wal_buffers are full* XLogInsert() cannot advance to a new page while we are waiting to send or write. So

Re: [HACKERS] Synchronous Log Shipping Replication

2008-09-09 Thread Simon Riggs
On Tue, 2008-09-09 at 16:05 +0200, Dimitri Fontaine wrote: > Le mardi 09 septembre 2008, Simon Riggs a écrit : > > If the WALWriter|Sender is available, it can begin the task immediately. > > There is no need for it to wait if you want synchronous behaviour. > > Ok. Now I'm as lost as anyone with

Re: [HACKERS] Synchronous Log Shipping Replication

2008-09-09 Thread Dimitri Fontaine
Le mardi 09 septembre 2008, Simon Riggs a écrit : > If the WALWriter|Sender is available, it can begin the task immediately. > There is no need for it to wait if you want synchronous behaviour. Ok. Now I'm as lost as anyone with respect to how you get Group Commit :) -- dim signature.asc Descri

Re: [HACKERS] Synchronous Log Shipping Replication

2008-09-09 Thread Markus Wanner
Hi, Dimitri Fontaine wrote: Exactly the point. The process is now already waiting in all cases, so maybe we could just force waiting some WALSender signal before sending the fsync() order, so we now have Group Commit. A single process can only wait on either fsync() or on select(), but not o

Re: [HACKERS] Synchronous Log Shipping Replication

2008-09-09 Thread Markus Wanner
Hi, ITAGAKI Takahiro wrote: Signals and locking, borrewed from Postgres-R, are now studied for the purpose in the log shipping, Cool. Let me know if you have any questions WRT this imessages stuff. Regards Markus Wanner -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)

Re: [HACKERS] Common Table Expressions (WITH RECURSIVE) patch

2008-09-09 Thread Robert Haas
> 3. "I think this is a "must fix" because of the point about volatile > functions --- changing it later will result in user-visible semantics > changes, so we have to get it right the first time." > > I don't entirely agree with #3. It is user-visible, but only in the > sense that someone is depen

Re: [HACKERS] Synchronous Log Shipping Replication

2008-09-09 Thread Simon Riggs
On Tue, 2008-09-09 at 15:32 +0200, Dimitri Fontaine wrote: > The process is now already waiting in all cases If the WALWriter|Sender is available, it can begin the task immediately. There is no need for it to wait if you want synchronous behaviour. -- Simon Riggs www.2ndQuadrant.com

Re: [HACKERS] Synchronous Log Shipping Replication

2008-09-09 Thread Dimitri Fontaine
Le mardi 09 septembre 2008, Markus Wanner a écrit : > ..and it will still has to wait until WAL is written to disk on the > local node, as we do now. These are two different things to wait for. > One is a network socket operation, the other is an fsync(). As these > don't work together too well (bl

Re: [HACKERS] Synchronous Log Shipping Replication

2008-09-09 Thread Markus Wanner
Hi, Dimitri Fontaine wrote: It might just be I'm not understanding what it's all about, but it seems to me with WALSender process A will wait, whatever happens, either until the WAL is sent to slave or written to disk on the slave. ..and it will still has to wait until WAL is written to disk

Re: [HACKERS] reducing statistics write overhead

2008-09-09 Thread Martin Pihlak
Alvaro Herrera wrote: >> Attached is a patch that implements the described signalling. > > Are we keeping the idea that the reader sends a stat message when it > needs to read stats? What about the lossiness of the transport? > As the message is resent in the wait loop, the collector should rec

Re: [EMAIL PROTECTED]: Re: [HACKERS] [patch] GUC source file and line number]

2008-09-09 Thread Alvaro Herrera
Greg Smith wrote: > On Tue, 9 Sep 2008, Magnus Hagander wrote: > >>> (I dropped the "default" stuff for now, as it doesn't seem that a >>> consensus has been reached on that topic.) >> >> This is one of the reasons I suggested keeping that one as a separate >> patch in the first place. The other ma

Re: [HACKERS] Synchronous Log Shipping Replication

2008-09-09 Thread Dimitri Fontaine
Hi, Le mardi 09 septembre 2008, Heikki Linnakangas a écrit : > The tricky part is, how does A know if it should wait, and for how long? > commit_delay sure isn't ideal, but AFAICS the log shipping proposal > doesn't provide any solution to that. It might just be I'm not understanding what it's al

Re: [HACKERS] Synchronous Log Shipping Replication

2008-09-09 Thread Tom Lane
Simon Riggs <[EMAIL PROTECTED]> writes: > On Tue, 2008-09-09 at 08:24 -0400, Tom Lane wrote: >> "Agreed"? That last restriction is a deal-breaker. > OK, I should have said *if wal_buffers are full* XLogInsert() cannot > advance to a new page while we are waiting to send or write. So I don't > thi

Re: [HACKERS] [PATCHES] to_date() validation

2008-09-09 Thread Brendan Jurd
On Tue, Sep 9, 2008 at 9:04 PM, Brendan Jurd <[EMAIL PROTECTED]> wrote: > On Tue, Sep 9, 2008 at 7:29 PM, Martijn van Oosterhout > <[EMAIL PROTECTED]> wrote: >> The use of palloc/pfree in this routine seems excessive. Does len have >> upper bound? If so a simple array will do it. >> > > I suppose I

Re: [HACKERS] Synchronous Log Shipping Replication

2008-09-09 Thread Simon Riggs
On Tue, 2008-09-09 at 08:24 -0400, Tom Lane wrote: > "Fujii Masao" <[EMAIL PROTECTED]> writes: > > On Tue, Sep 9, 2008 at 5:11 PM, Simon Riggs <[EMAIL PROTECTED]> wrote: > >> > >> Yes. We should have a LogwrtRqst pointer and LogwrtResult pointer for > >> the send operation. The Write and Send ope

Re: [HACKERS] Synchronous Log Shipping Replication

2008-09-09 Thread Tom Lane
"Fujii Masao" <[EMAIL PROTECTED]> writes: > On Tue, Sep 9, 2008 at 5:11 PM, Simon Riggs <[EMAIL PROTECTED]> wrote: >> >> Yes. We should have a LogwrtRqst pointer and LogwrtResult pointer for >> the send operation. The Write and Send operations can then continue >> independently of one another. XLo

Re: [HACKERS] Our CLUSTER implementation is pessimal

2008-09-09 Thread Tom Lane
Gregory Stark <[EMAIL PROTECTED]> writes: > In particular I'm thinking of people clustering on a covering index (which > isn't as uncommon as it sounds, if you have a covering index you probably do > want to cluster it -- consider many-to-many join tables). We should be able to > do an index-only s

Re: [HACKERS] Synchronous Log Shipping Replication

2008-09-09 Thread Simon Riggs
On Tue, 2008-09-09 at 13:42 +0200, Markus Wanner wrote: > How about multiple standby servers? There are various ways for getting things to work with multiple servers. I hope we can make this work with just a single standby before we try to make it work on more. There are various options for sync

Re: [HACKERS] Synchronous Log Shipping Replication

2008-09-09 Thread Simon Riggs
On Tue, 2008-09-09 at 20:12 +0900, Fujii Masao wrote: > I'd like to introduce new parameter "synchronous_replication" which specifies > whether backends waits for the response from WAL sender process. By > combining synchronous_commit and synchronous_replication, users can > choose various option

Re: [HACKERS] Synchronous Log Shipping Replication

2008-09-09 Thread Simon Riggs
On Tue, 2008-09-09 at 12:54 +0300, Heikki Linnakangas wrote: > Note that we already have the logic to flush all pending commit > records at once. But only when you can grab WALInsertLock when flushing. If you look at the way I suggested, it does not rely upon that lock being available. So it is

Re: [HACKERS] Synchronous Log Shipping Replication

2008-09-09 Thread Heikki Linnakangas
Fujii Masao wrote: What makes the sender process bottleneck? The keyword here is "might". There's many possibilities, like: - Slow network. - Ridiculously fast disk. Like a RAM disk. If you have a synchronous slave you can fail over to, putting WAL on a RAM disk isn't that crazy. - slower WAL

Re: [HACKERS] Synchronous Log Shipping Replication

2008-09-09 Thread Simon Riggs
On Tue, 2008-09-09 at 20:12 +0900, Fujii Masao wrote: > What makes the sender process bottleneck? In my experience, the Atlantic. But I guess the Pacific does it too. :-) -- Simon Riggs www.2ndQuadrant.com PostgreSQL Training, Services and Support -- Sent via pgsql-hackers maili

Re: [HACKERS] Synchronous Log Shipping Replication

2008-09-09 Thread Markus Wanner
Hi, Fujii Masao wrote: Really? In the benchmark result of my prototype, the bottleneck is still disk I/O. The communication (between the master and the slave) latency is smaller than WAL writing (fsyncing) one. Of course, I assume that we use not-poor network like 1000BASE-T. Sure. If you do W

Re: [HACKERS] Synchronous Log Shipping Replication

2008-09-09 Thread Markus Wanner
Hi, ITAGAKI Takahiro wrote: Signals and locking, borrewed from Postgres-R, are now studied for the purpose in the log shipping, but I'm not sure it can be also used in the group commit. Yeah. As Heikki points out, there is a completely orthogonal question WRT group commit: how does transactio

Re: [HACKERS] Synchronous Log Shipping Replication

2008-09-09 Thread Fujii Masao
On Tue, Sep 9, 2008 at 5:11 PM, Simon Riggs <[EMAIL PROTECTED]> wrote: > > Yes. We should have a LogwrtRqst pointer and LogwrtResult pointer for > the send operation. The Write and Send operations can then continue > independently of one another. XLogInsert() cannot advance to a new page > while we

Re: [HACKERS] [PATCHES] to_date() validation

2008-09-09 Thread Brendan Jurd
On Tue, Sep 9, 2008 at 7:29 PM, Martijn van Oosterhout <[EMAIL PROTECTED]> wrote: > I actually had a look at this patch also, though not as thoroughly as > Alex. There was one part that I had some thoughts about in > from_char_parse_int_len: > Hi Martijn. Thanks for your comments. > The use of

Re: [HACKERS] Synchronous Log Shipping Replication

2008-09-09 Thread ITAGAKI Takahiro
Heikki Linnakangas <[EMAIL PROTECTED]> wrote: > The tricky part is, how does A know if it should wait, and for how long? > commit_delay sure isn't ideal, but AFAICS the log shipping proposal > doesn't provide any solution to that. They have no relation each other directly, but they need simila

Re: [HACKERS] Our CLUSTER implementation is pessimal

2008-09-09 Thread Gregory Stark
Tom Lane <[EMAIL PROTECTED]> writes: > Gregory Stark <[EMAIL PROTECTED]> writes: >> Yeah, I've been thinking about how to use the planner to do this. > > I thought the answer to that was going to be more or less "call > cost_sort() and cost_index() and compare the answers". That was the way I was

Re: [HACKERS] Synchronous Log Shipping Replication

2008-09-09 Thread Heikki Linnakangas
Simon Riggs wrote: Multiple backends waiting while we perform a write. Commits then happen as a group (to WAL at least), hence Group Commit. The problem with our current commit protocol is this: 1. Backend A inserts commit record A 2. Backend A starts to flush commit record A 3. Backend B inse

Re: [HACKERS] Synchronous Log Shipping Replication

2008-09-09 Thread Simon Riggs
On Tue, 2008-09-09 at 12:24 +0300, Heikki Linnakangas wrote: > Simon Riggs wrote: > > This gives us the Group Commit feature also, even if we are not using > > replication. So we can drop the commit_delay stuff. > > Huh? How does that give us group commit? Multiple backends waiting while we perf

Re: [HACKERS] [PATCHES] to_date() validation

2008-09-09 Thread Martijn van Oosterhout
On Mon, Sep 08, 2008 at 06:24:14PM +1000, Brendan Jurd wrote: > On Sun, Sep 7, 2008 at 5:58 AM, Alex Hunsaker <[EMAIL PROTECTED]> wrote: > > Im just following this: > > http://wiki.postgresql.org/wiki/Reviewing_a_Patch so lets get started. > > > > Hi Alex. Thanks for taking the time to review my

Re: [EMAIL PROTECTED]: Re: [HACKERS] [patch] GUC source file and line number]

2008-09-09 Thread Greg Smith
On Tue, 9 Sep 2008, Magnus Hagander wrote: (I dropped the "default" stuff for now, as it doesn't seem that a consensus has been reached on that topic.) This is one of the reasons I suggested keeping that one as a separate patch in the first place. The other main reason being that once it gets

Re: [HACKERS] Synchronous Log Shipping Replication

2008-09-09 Thread Heikki Linnakangas
Simon Riggs wrote: This gives us the Group Commit feature also, even if we are not using replication. So we can drop the commit_delay stuff. Huh? How does that give us group commit? -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pg

Re: [HACKERS] Some newbie questions

2008-09-09 Thread Greg Smith
On Sun, 7 Sep 2008, M2Y wrote: Why does a replication solution need log shipping and why cant we just ship the transaction statements to a standby node? Here's one of the classic examples of why that doesn't work: create table x (d decimal); insert into x values (random()); If you execute th

Re: [EMAIL PROTECTED]: Re: [HACKERS] [patch] GUC source file and line number]

2008-09-09 Thread Magnus Hagander
Alvaro Herrera wrote: > Tom Lane wrote: >> Alvaro Herrera <[EMAIL PROTECTED]> writes: >>> Tom Lane wrote: > >>> Hmm, I didn't recheck after Greg's patch, but in mine, it doesn't, >>> because the location is saved as "reset location" and restored when the >>> variable is reset. It worked fine in a

Re: [HACKERS] Move src/tools/backend/ to wiki

2008-09-09 Thread Greg Smith
On Mon, 8 Sep 2008, Bruce Momjian wrote: I don't think much changes at the flow chart level from release to release so it would fine if it was just CVS HEAD. I also don't think many people do back-branch development. It's easy with Mediawiki to both 1) see the old version for those situation

Re: [HACKERS] Synchronous Log Shipping Replication

2008-09-09 Thread Simon Riggs
On Mon, 2008-09-08 at 17:40 -0400, Bruce Momjian wrote: > Fujii Masao wrote: > > On Mon, Sep 8, 2008 at 8:44 PM, Markus Wanner <[EMAIL PROTECTED]> wrote: > > >>Merge into WAL writer? > > > > > > Uh.. that would mean you'd loose parallelism between WAL writing to disk > > > and > > > WAL s

[HACKERS] PLUGINS Functionlity in Win32 build scripts

2008-09-09 Thread MUHAMMAD ASIF
Hi, I am currently working in EnterpriseDB (www.enterprisedb.com). During the integration of pldebugger ( http://pgfoundry.org/projects/edb-debugger ) with postgres on windows I faced a problem that plugins are not being copied to the lib/plugins directory. Plugins should be copied in (