Re: [HACKERS] Advice on MyXactMade* flags, MyLastRecPtr, pendingDeletes and lazy XID assignment

2007-08-30 Thread Tom Lane
Alvaro Herrera <[EMAIL PROTECTED]> writes: > Why don't we create temp tables in a separate directory, anyway? That > would make it trivially easy to get rid of them during recovery: just > zap all the files in there. Hmm ... doesn't do anything to fix the problem of file leakage for crashes when

Re: [HACKERS] Advice on MyXactMade* flags, MyLastRecPtr, pendingDeletes and lazy XID assignment

2007-08-30 Thread Florian G. Pflug
Tom Lane wrote: Whether that's common enough to be worth special optimization, I'm not sure. Your point that the transaction may have been heavyweight enough that it'd hardly matter is still on-target, you just got there by the wrong reasoning ;-) My point is that there is hardly any effort ne

Re: [HACKERS] Advice on MyXactMade* flags, MyLastRecPtr, pendingDeletes and lazy XID assignment

2007-08-30 Thread Alvaro Herrera
Tom Lane wrote: > Simon Riggs <[EMAIL PROTECTED]> writes: > > -1 to second: If there are any temporary deleted files then it is > > because we've done a large sort, hash join etc, so avoiding the sync > > makes no difference to the overall response time. > > I think you're confused, actually: thi

Re: [HACKERS] Advice on MyXactMade* flags, MyLastRecPtr, pendingDeletes and lazy XID assignment

2007-08-30 Thread Tom Lane
Simon Riggs <[EMAIL PROTECTED]> writes: > -1 to second: If there are any temporary deleted files then it is > because we've done a large sort, hash join etc, so avoiding the sync > makes no difference to the overall response time. I think you're confused, actually: this is not about temporary sor

Re: [HACKERS] Advice on MyXactMade* flags, MyLastRecPtr, pendingDeletes and lazy XID assignment

2007-08-30 Thread Simon Riggs
On Thu, 2007-08-30 at 14:14 -0400, Tom Lane wrote: > "Florian G. Pflug" <[EMAIL PROTECTED]> writes: > > So should we not force synchronous commit if all to-be-deleted files > > are temporary? Or is that pushing our luck too much, because it opens > > the window of possible file leakage considerably

Re: [HACKERS] Advice on MyXactMade* flags, MyLastRecPtr, pendingDeletes and lazy XID assignment

2007-08-30 Thread Simon Riggs
On Thu, 2007-08-30 at 00:33 -0400, Tom Lane wrote: > "Florian G. Pflug" <[EMAIL PROTECTED]> writes: > > And I'm quite tempted to not flush the XLOG at all during ABORT, and to > > only force synchronous commits if one of the to-be-deleted files is > > non-temporary. > > +1 on the first, but -1 on

Re: [HACKERS] Advice on MyXactMade* flags, MyLastRecPtr, pendingDeletes and lazy XID assignment

2007-08-30 Thread Tom Lane
"Florian G. Pflug" <[EMAIL PROTECTED]> writes: > So should we not force synchronous commit if all to-be-deleted files > are temporary? Or is that pushing our luck too much, because it opens > the window of possible file leakage considerably wider? I think this area is something to be left for late

Re: [HACKERS] Advice on MyXactMade* flags, MyLastRecPtr, pendingDeletes and lazy XID assignment

2007-08-30 Thread Florian G. Pflug
Tom Lane wrote: "Florian G. Pflug" <[EMAIL PROTECTED]> writes: Tom Lane wrote: +1 on the first, but -1 on the second, because we'd have to track whether deleted files are temp or not ... it's very unclear that it'd be worth the trouble. I'm confused. smgrcreate already tracks if it created a

Re: [HACKERS] Advice on MyXactMade* flags, MyLastRecPtr, pendingDeletes and lazy XID assignment

2007-08-30 Thread Tom Lane
"Florian G. Pflug" <[EMAIL PROTECTED]> writes: > Tom Lane wrote: >> +1 on the first, but -1 on the second, because we'd have to track >> whether deleted files are temp or not ... it's very unclear that it'd >> be worth the trouble. > I'm confused. smgrcreate already tracks if it created a file for

Re: [HACKERS] Advice on MyXactMade* flags, MyLastRecPtr, pendingDeletes and lazy XID assignment

2007-08-30 Thread Gregory Stark
"Florian G. Pflug" <[EMAIL PROTECTED]> writes: >>> It seems doable, but it's not pretty. One possible scheme would be to >>> emit a record *after* chosing a name but *before* creating the file, >> >> No, because the way you know the name is good is a successful >> open(O_CREAT). > > The idea was t

Re: [HACKERS] Advice on MyXactMade* flags, MyLastRecPtr, pendingDeletes and lazy XID assignment

2007-08-30 Thread Florian G. Pflug
Tom Lane wrote: "Florian G. Pflug" <[EMAIL PROTECTED]> writes: And I'm quite tempted to not flush the XLOG at all during ABORT, and to only force synchronous commits if one of the to-be-deleted files is non-temporary. +1 on the first, but -1 on the second, because we'd have to track whether de

Re: [HACKERS] Advice on MyXactMade* flags, MyLastRecPtr, pendingDeletes and lazy XID assignment

2007-08-29 Thread Tom Lane
"Florian G. Pflug" <[EMAIL PROTECTED]> writes: > And I'm quite tempted to not flush the XLOG at all during ABORT, and to > only force synchronous commits if one of the to-be-deleted files is > non-temporary. +1 on the first, but -1 on the second, because we'd have to track whether deleted files ar

Re: [HACKERS] Advice on MyXactMade* flags, MyLastRecPtr, pendingDeletes and lazy XID assignment

2007-08-29 Thread Florian G. Pflug
Tom Lane wrote: "Florian G. Pflug" <[EMAIL PROTECTED]> writes: One comment is that at the time we make an entry into smgr's pending-deletes list, I think we might not have acquired an XID yet Hm.. I was just going to implement this, but I'm now wondering if thats really worth it. Basically

Re: [HACKERS] Advice on MyXactMade* flags, MyLastRecPtr, pendingDeletes and lazy XID assignment

2007-08-29 Thread Tom Lane
"Florian G. Pflug" <[EMAIL PROTECTED]> writes: >> One comment is that at the time we make an entry into smgr's >> pending-deletes list, I think we might not have acquired an XID yet > Hm.. I was just going to implement this, but I'm now wondering if > thats really worth it. Basically what you'd g

Re: [HACKERS] Advice on MyXactMade* flags, MyLastRecPtr, pendingDeletes and lazy XID assignment

2007-08-29 Thread Florian G. Pflug
Tom Lane wrote: One comment is that at the time we make an entry into smgr's pending-deletes list, I think we might not have acquired an XID yet --- if I understand your patch correctly, a CREATE TABLE would acquire an XID when it makes its first catalog insertion, and that happens after creating

Re: [HACKERS] Advice on MyXactMade* flags, MyLastRecPtr, pendingDeletes and lazy XID assignment

2007-08-29 Thread Florian G. Pflug
Tom Lane wrote: "Florian G. Pflug" <[EMAIL PROTECTED]> writes: Tom Lane wrote: One thought here is that it's not clear that we really need a concept of transaction-controlled vs not-transaction-controlled xlog records anymore. I've thinking about keeping XLOG_NO_TRAN, and doing if (!no_tran)

Re: [HACKERS] Advice on MyXactMade* flags, MyLastRecPtr, pendingDeletes and lazy XID assignment

2007-08-29 Thread Tom Lane
"Florian G. Pflug" <[EMAIL PROTECTED]> writes: > Tom Lane wrote: >> One thought here is that it's not clear that we really need a concept of >> transaction-controlled vs not-transaction-controlled xlog records >> anymore. > I've thinking about keeping XLOG_NO_TRAN, and doing > if (!no_tran) >A

Re: [HACKERS] Advice on MyXactMade* flags, MyLastRecPtr, pendingDeletes and lazy XID assignment

2007-08-29 Thread Florian G. Pflug
Tom Lane wrote: "Florian G. Pflug" <[EMAIL PROTECTED]> writes: I propose to do the following in my lazy XID assignment patch - can anyone see a hole in that? > One comment is that at the time we make an entry into smgr's pending-deletes list, I think we might not have acquired an XID yet --- i

Re: [HACKERS] Advice on MyXactMade* flags, MyLastRecPtr, pendingDeletes and lazy XID assignment

2007-08-29 Thread Tom Lane
"Florian G. Pflug" <[EMAIL PROTECTED]> writes: > I propose to do the following in my lazy XID assignment patch - can > anyone see a hole in that? Cleaning up this area seems like a good idea. Just FYI, one reason why there are so many LastRec pointer variables is that the WAL record format used t