August Zajonc wrote:
The thing is, the leak occurs in situation where a COMMIT hasn't
returned to the user, so we are trying to guarantee no data-loss even
when the user doesn't see a successful commit? That's a tall order
obviously and hopefully people design their apps to attend to
Tom Lane wrote:
There's also the plan B of scanning pg_class to decide which relfilenode
values are legit. IIRC Bruce did up a patch for this about a year ago,
which I vetoed because I was afraid of the consequences if it removed
data that someone really needed.
I posted a patch like that,
Heikki Linnakangas [EMAIL PROTECTED] writes:
Tom Lane wrote:
Someone just mentioned doing the same
thing but pushing the unreferenced files into a trash directory
instead of actually deleting them.
That sounds like a good idea to me.
It suddenly strikes me that there's lots of precedent for
Tom Lane wrote:
There's also the plan B of scanning pg_class to decide which relfilenode
values are legit. IIRC Bruce did up a patch for this about a year ago,
which I vetoed because I was afraid of the consequences if it removed
data that someone really needed. Someone just mentioned doing
August Zajonc [EMAIL PROTECTED] writes:
I do have a question about jamming though. Will the system work if the
file ended up stuck in this folder? Let's say the move destination has a
duplicate file that conflicts, or permissions change under you, or disks
fill.
Yeah, the move-the-file
Tom Lane wrote:
I had an idea this morning that might be useful: back off the strength
of what we try to guarantee. Specifically, does it matter if we leak a
file on crash, as long as it isn't occupying a lot of disk space?
(I suppose if you had enough crashes to accumulate many thousands of
Heikki Linnakangas wrote:
Tom Lane wrote:
I had an idea this morning that might be useful: back off the strength
of what we try to guarantee. Specifically, does it matter if we leak a
file on crash, as long as it isn't occupying a lot of disk space?
(I suppose if you had enough crashes to
August Zajonc wrote:
Yes, checkpoints would need to include a list of
created-but-yet-uncommitted
files. I think the hardest part is figuring out a way to get that
information
to the backend doing the checkpoint - my idea was to track them in shared
memory, but that would impose a hard limit
Florian G. Pflug [EMAIL PROTECTED] writes:
It might be even worse - I'm not sure that a rename is an atomic operation
on most filesystems.
rename(2) is specified to be atomic by POSIX, but relinking a file into
a different directory can hardly be --- it's not even provided as a
single kernel
Florian G. Pflug wrote:
August Zajonc wrote:
I'm confused about this.
As long as we assert the rule that the file name can't change on the
move, then after commit the file can be in only one of two places.
The name of the file is known (ie, pg_class). The directories are
known. What needs
Tom Lane wrote:
What I was thinking about was a flag file separate from the data file
itself, a bit like what we use for archiver signaling. If is the
new data file, then touch .new to mark the file as needing to be
deleted on restart. Remove these files just *before* commit. This
Tom Lane wrote:
Florian G. Pflug [EMAIL PROTECTED] writes:
It might be even worse - I'm not sure that a rename is an atomic operation
on most filesystems.
rename(2) is specified to be atomic by POSIX, but relinking a file into
a different directory can hardly be --- it's not even provided as
Florian G. Pflug [EMAIL PROTECTED] writes:
Tom Lane wrote:
rename(2) is specified to be atomic by POSIX, but relinking a file into
a different directory can hardly be --- it's not even provided as a
single kernel call, is it?
I'd have thought that they only guarantee that if the new name
Florian G. Pflug [EMAIL PROTECTED] writes:
The patch can be found at: http://soc.phlo.org/lazyxidassign.v2.patch
(Seems that large attachments don't get through on this list - or was that
just bad luck when I tried posting the first version of that patch?)
No, the message size limit on
Tom Lane wrote:
Florian G. Pflug [EMAIL PROTECTED] writes:
A few comments ...
GetOldestXmin() is changed slightly - it used to use
GetTopTransactionId() as a starting point for it's xmin
calculation, falling back to ReadNewTransactionId() if called
from outside a transaction. Now, it always
Florian G. Pflug [EMAIL PROTECTED] writes:
Tom Lane wrote:
I don't much like this, since as I mentioned before I don't think
MyProc-xmin is going to be constant over a whole transaction for
long. I don't think xid_age is performance-critical in any way,
so I'd vote for letting it force XID
Tom Lane [EMAIL PROTECTED] writes:
Florian G. Pflug [EMAIL PROTECTED] writes:
Tom Lane wrote:
I don't much like this, since as I mentioned before I don't think
MyProc-xmin is going to be constant over a whole transaction for
long. I don't think xid_age is performance-critical in any way,
Gregory Stark [EMAIL PROTECTED] writes:
It seems both this and some of the other cases of having to call
ReadNewTransactionId would be eliminated if we invented a new xid treated
similarly to current_time() and others which are calculated once per
transaction and then cached for subsequent
August Zajonc wrote:
I assume that you meant that mail to go to pgsql-hackers too, and just
clicked the wrong button by accident. If not, please forgive that
I post the reply to the list.
Are you going to look at putting temp relations into a special folder
for 8.4? I really like that idea for
Gregory Stark wrote:
Tom Lane [EMAIL PROTECTED] writes:
Florian G. Pflug [EMAIL PROTECTED] writes:
Tom Lane wrote:
I don't much like this, since as I mentioned before I don't think
MyProc-xmin is going to be constant over a whole transaction for
long. I don't think xid_age is
Florian G. Pflug wrote:
August Zajonc wrote:
I assume that you meant that mail to go to pgsql-hackers too, and just
clicked the wrong button by accident. If not, please forgive that
I post the reply to the list.
Are you going to look at putting temp relations into a special folder for
8.4?
Tom Lane wrote:
Florian G. Pflug [EMAIL PROTECTED] writes:
Tom Lane wrote:
I don't much like this, since as I mentioned before I don't think
MyProc-xmin is going to be constant over a whole transaction for
long. I don't think xid_age is performance-critical in any way,
so I'd vote for letting
Alvaro Herrera wrote:
Florian G. Pflug wrote:
August Zajonc wrote:
I assume that you meant that mail to go to pgsql-hackers too, and just
clicked the wrong button by accident. If not, please forgive that
I post the reply to the list.
Are you going to look at putting temp relations into a
Florian G. Pflug [EMAIL PROTECTED] writes:
Hm.. but xid_age already exposes an implementation detail (that xid
comparison is circular).
But the only reason anyone cares about it at all, really, is that
xid comparison is circular ... the function was only invented in
the first place to help
Tom Lane wrote:
Florian G. Pflug [EMAIL PROTECTED] writes:
Actually, after more thought on this I'd
argue that the only really correct reference for computing a xid's
age is GetOldestXmin(). It's what VACUUM uses to calculate the
freezeLimit, after all. Now, using GetOldestXmin() directly is
Tom Lane wrote:
I note that pg_description defines xid_age as age of a transaction ID,
in transactions before current transaction, and I'm content to stick
with that definition. Even if it does force consuming an XID. I'm not
sure that we could devise a comparably succinct description of
Florian G. Pflug wrote:
You could still leak them (ie, you move to final location and bail
before commit) but it seems to narrow the window down significantly.
That leak could be prevented I think if the COMMIT record indicated
which files are to be moved, and the actual move happens after the
27 matches
Mail list logo
