Re: [HACKERS] could not create directory "...": File exists

2013-01-19 Thread Stephen Frost
* Tom Lane (t...@sss.pgh.pa.us) wrote: > I committed this with a couple of changes: Great, thanks! > * I used GetLatestSnapshot rather than GetTransactionSnapshot. Since > we don't allow these operations inside transaction blocks, there > shouldn't be much difference, but in principle GetTransac

Re: [HACKERS] could not create directory "...": File exists

2013-01-18 Thread Tom Lane
Stephen Frost writes: > Tom, > * Tom Lane (t...@sss.pgh.pa.us) wrote: >> Don't see what. The main reason we've not yet attempted a global fix is >> that the most straightforward way (take a new snapshot each time we >> start a new SnapshotNow scan) seems too expensive. But CREATE DATABASE >> is

Re: [HACKERS] could not create directory "...": File exists

2013-01-18 Thread Tom Lane
Stephen Frost writes: > Patch attached. Passes the regression tests and our internal testing > shows that it eliminates the error we were seeing (and doesn't cause > blocking, which is even better). > We have a workaround in place for our build system (more-or-less > "don't do that" ap

Re: [HACKERS] could not create directory "...": File exists

2013-01-18 Thread Stephen Frost
Tom, * Tom Lane (t...@sss.pgh.pa.us) wrote: > Don't see what. The main reason we've not yet attempted a global fix is > that the most straightforward way (take a new snapshot each time we > start a new SnapshotNow scan) seems too expensive. But CREATE DATABASE > is so expensive that the cost of

Re: [HACKERS] could not create directory "...": File exists

2013-01-17 Thread Tom Lane
Stephen Frost writes: > * Tom Lane (t...@sss.pgh.pa.us) wrote: >> This seems to provide a reasonably principled >> argument why we might want to fix this case with a localized use of an >> MVCC scan before we have such a fix globally. > I had discussed that idea a bit with Andres on IRC and my on

Re: [HACKERS] could not create directory "...": File exists

2013-01-17 Thread Stephen Frost
* Tom Lane (t...@sss.pgh.pa.us) wrote: > This seems to provide a reasonably principled > argument why we might want to fix this case with a localized use of an > MVCC scan before we have such a fix globally. I had discussed that idea a bit with Andres on IRC and my only concern was if there's some

Re: [HACKERS] could not create directory "...": File exists

2013-01-17 Thread Tom Lane
I wrote: > Stephen Frost writes: >> I feel like we should be able to do better than what we have now, at >> least. Using ShareLock prevented the specific case that we were >> experiencing and is therefore MUCH better (for us, anyway) than >> released versions where we ran into the error on a regu

Re: [HACKERS] could not create directory "...": File exists

2013-01-17 Thread Tom Lane
Stephen Frost writes: > * Tom Lane (t...@sss.pgh.pa.us) wrote: >> Or maybe we should just write this off as a case we can't realistically >> fix before we have MVCC catalog scans. It seems that any other fix is >> going to be hopelessly ugly. > I feel like we should be able to do better than wha

Re: [HACKERS] could not create directory "...": File exists

2013-01-17 Thread Stephen Frost
* Tom Lane (t...@sss.pgh.pa.us) wrote: > The case where you see a tuple twice is if an update drops a new version > of a row beyond your seqscan, and then commits before you get to the new > version. But if it drops the new version of the row *behind* your > seqscan, and then commits before you ge

Re: [HACKERS] could not create directory "...": File exists

2013-01-17 Thread Andres Freund
On 2013-01-17 13:46:44 -0500, Tom Lane wrote: > Or maybe we should just write this off as a case we can't realistically > fix before we have MVCC catalog scans. It seems that any other fix is > going to be hopelessly ugly. ISTM we could just use a MVCC snapshot in this specific case? Andres --

Re: [HACKERS] could not create directory "...": File exists

2013-01-17 Thread Tom Lane
Stephen Frost writes: > * Tom Lane (t...@sss.pgh.pa.us) wrote: >> [ thinks for a bit... ] Ugh, no, because the *other* risk you've got >> here is not seeing a row at all, which would be really bad. > I'm not sure that I see how that could happen..? I agree that it'd be > really bad if it did th

Re: [HACKERS] could not create directory "...": File exists

2013-01-17 Thread Stephen Frost
* Tom Lane (t...@sss.pgh.pa.us) wrote: > Ugh. Still another problem with non-MVCC catalog scans. Indeed. > It seems that the only thing we actually use from each tuple is the OID. Yes, that's true. > So there are other ways to fix it, of which probably the minimum-change > one is to keep a lis

Re: [HACKERS] could not create directory "...": File exists

2013-01-17 Thread Tom Lane
Stephen Frost writes: > It turns out that createdb() currently only takes an AccessShareLock > on pg_tablespace when scanning it with SnapshotNow, making it possible > for a concurrent process to make some uninteresting modification to a > tablespace (such as an ACL change) which will caus

[HACKERS] could not create directory "...": File exists

2013-01-17 Thread Stephen Frost
Greetings, We've come across this rather annoying error happening during our builds: ERROR: could not create directory "pg_tblspc/25120/PG_9.3_201212081/231253": File exists It turns out that this is coming from copydir() when called by createdb() during a CREATE DATABASE .. FROM TEM