Re: [HACKERS] pg_restore and create FK without verification check

2003-11-28 Thread Oli Sennhauser
People might be more interested in debating this topic with you if we hadn't discussed it at length just a couple months back. There wasn't consensus then that we had to offer an escape hatch, and you've not offered any argument that wasn't made before. I'm simply presenting a problem for

Re: [HACKERS] pg_restore and create FK without verification check

2003-11-28 Thread Jan Wieck
Oli Sennhauser wrote: People might be more interested in debating this topic with you if we hadn't discussed it at length just a couple months back. There wasn't consensus then that we had to offer an escape hatch, and you've not offered any argument that wasn't made before. I'm simply

Re: [HACKERS] pg_restore and create FK without verification check

2003-11-27 Thread Alvaro Herrera
On Wed, Nov 26, 2003 at 10:11:20PM -0800, ow wrote: --- Alvaro Herrera [EMAIL PROTECTED] wrote: On Thu, Nov 27, 2003 at 12:40:28AM +0100, Andreas Pflug wrote: A common mistake, can't count how often I created this one... And not easy to find, because EXPLAIN won't explain triggers.

Re: [HACKERS] pg_restore and create FK without verification check

2003-11-27 Thread ow
--- Alvaro Herrera [EMAIL PROTECTED] wrote: In what scenarios? I'd easily buy this if you are talking about small tables. Read the message again. __ Do you Yahoo!? Free Pop-Up Blocker - Get it now http://companion.yahoo.com/

Re: [HACKERS] pg_restore and create FK without verification check

2003-11-27 Thread Andreas Pflug
Kevin Brown wrote: WAL is not the bottleneck ... as I already mentioned today, pg_clog (and more specifically the meaning of transaction IDs) is what really makes a cluster an indivisible whole at the physical level. The ability to restore a single large database quickly is, I think, a

Re: [HACKERS] pg_restore and create FK without verification check

2003-11-27 Thread Andreas Pflug
ow wrote: I'd like to emphasize again that NOT having an index on the FK column is a perfectly valid approach, despite some opinions to the contrary. OW, you might insist that there are several cases when an index is not needed, but I didn't propose to create the index automatically (this

Re: [HACKERS] pg_restore and create FK without verification check

2003-11-27 Thread Jan Wieck
ow wrote: --- Tom Lane [EMAIL PROTECTED] wrote: People might be more interested in debating this topic with you if we hadn't discussed it at length just a couple months back. There wasn't consensus then that we had to offer an escape hatch, and you've not offered any argument that wasn't made

[HACKERS] pg_restore and create FK without verification check

2003-11-26 Thread ow
--- ow [EMAIL PROTECTED] wrote: IMHO, not only data need to loaded before FK constraints are created but also there has got to be a feature to allow creation of an FK constraint WITHOUT doing the verification that all loaded/existing records satisfy the FK constraint. The ability to create a

Re: [HACKERS] pg_restore and create FK without verification check

2003-11-26 Thread Andreas Pflug
ow wrote: It appears there's not a lot of interest in discussing the possibility of FK constraint creation WITHOUT the verification check. How then should one handle the situation with pg_restore and large dbs where creation of FK constraint(s) may take hours? I'd prefer a backup/restore

Re: [HACKERS] pg_restore and create FK without verification check

2003-11-26 Thread Hannu Krosing
Andreas Pflug kirjutas K, 26.11.2003 kell 12:09: ow wrote: It appears there's not a lot of interest in discussing the possibility of FK constraint creation WITHOUT the verification check. How then should one handle the situation with pg_restore and large dbs where creation of FK

Re: [HACKERS] pg_restore and create FK without verification check

2003-11-26 Thread Andreas Pflug
Hannu Krosing wrote: Andreas Pflug kirjutas K, 26.11.2003 kell 12:09: ow wrote: It appears there's not a lot of interest in discussing the possibility of FK constraint creation WITHOUT the verification check. How then should one handle the situation with pg_restore and large dbs where

Re: [HACKERS] pg_restore and create FK without verification check

2003-11-26 Thread ow
--- Andreas Pflug [EMAIL PROTECTED] wrote: Yes, I mentioned it just a few days when discussing dependency in pg_dump. This is somewhat complementary to WAL and PITR. I'm seeking for a fast way to dump and restore a complete database, like physical file copy, without shutting down the

Re: [HACKERS] pg_restore and create FK without verification check

2003-11-26 Thread Andreas Pflug
ow wrote: --- Andreas Pflug [EMAIL PROTECTED] wrote: Yes, I mentioned it just a few days when discussing dependency in pg_dump. This is somewhat complementary to WAL and PITR. I'm seeking for a fast way to dump and restore a complete database, like physical file copy, without shutting down

Re: [HACKERS] pg_restore and create FK without verification check

2003-11-26 Thread Oli Sennhauser
Hello I was asking about this too, one or two weeks ago. It appears there's not a lot of interest in discussing the possibility of FK constraint creation WITHOUT the verification check. How then should one handle the situation with pg_restore and large dbs where creation of FK constraint(s)

Re: [HACKERS] pg_restore and create FK without verification check

2003-11-26 Thread Zeugswetter Andreas SB SD
Q2: New situation: Why is it not a good idea to backup the database files of a cluster incl. all c_log and x_log (log files last) to get a physicaly hot backup. In principle it is the same situation like a server which is crashing (not a once but during some time). After restoring, it

Re: [HACKERS] pg_restore and create FK without verification check

2003-11-26 Thread Tom Lane
Andreas Pflug [EMAIL PROTECTED] writes: This is somewhat complementary to WAL and PITR. I'm seeking for a fast way to dump and restore a complete database, like physical file copy, without shutting down the backend. I was thinking of a BACKUP command that streams out the files including any

Re: [HACKERS] pg_restore and create FK without verification check

2003-11-26 Thread ow
--- Tom Lane [EMAIL PROTECTED] wrote: This doesn't really replace pg_dump/pg_restore, because it probably wouldn't be able to upgrade a cluster. Right, any such physical dump would be limited to restoring a whole cluster as-is: no imports into other clusters, no selectivity, no fancy

Re: [HACKERS] pg_restore and create FK without verification check

2003-11-26 Thread Tom Lane
ow [EMAIL PROTECTED] writes: --- Tom Lane [EMAIL PROTECTED] wrote: Right, any such physical dump would be limited to restoring a whole cluster as-is: no imports into other clusters, no selectivity, no fancy games. But that would not help people who would HAVE to use pg_dump/pg_restore (e.g.

Re: [HACKERS] pg_restore and create FK without verification check

2003-11-26 Thread ow
--- Tom Lane [EMAIL PROTECTED] wrote: Quite honestly, I think they should check their foreign keys. Generally speaking, I agree. The problem is that verification of FK constraint(s) may take too long, depending on the size of the db and other conditions. In my case, on test data, it takes about

Re: [HACKERS] pg_restore and create FK without verification check

2003-11-26 Thread Greg Stark
ow [EMAIL PROTECTED] writes: --- Tom Lane [EMAIL PROTECTED] wrote: Quite honestly, I think they should check their foreign keys. What should I do if I *know* there will be a FK failure but I want to correct it manually. Perhaps by creating all the necessary target records, perhaps by

Re: [HACKERS] pg_restore and create FK without verification check

2003-11-26 Thread Andreas Pflug
Tom Lane wrote: Andreas Pflug [EMAIL PROTECTED] writes: This is somewhat complementary to WAL and PITR. I'm seeking for a fast way to dump and restore a complete database, like physical file copy, without shutting down the backend. I was thinking of a BACKUP command that streams out the

Re: [HACKERS] pg_restore and create FK without verification check

2003-11-26 Thread Andreas Pflug
Greg Stark wrote: If I could disable and reenable the constraint the danger that I would get the definition wrong would be eliminated. And if I had already done the work to ensure there were no broken relationships I would optionally be able to skip the redundant automatic check. I could even

Re: [HACKERS] pg_restore and create FK without verification check

2003-11-26 Thread Tom Lane
Andreas Pflug [EMAIL PROTECTED] writes: In principle you could do this today, but we don't have enough support code in place to make it work smoothly, eg WAL segment files aren't labeled with enough identifying information to let you manage an archive full of 'em. Still it doesn't seem that

Re: [HACKERS] pg_restore and create FK without verification check

2003-11-26 Thread Andreas Pflug
Tom Lane wrote: - how to restore a single database You don't. As I said, any physical backup is going to be all-or-nothing. These techniques are not a replacement for pg_dump. That's sad. I've been backing up and restoring single databases from a cluster frequently, so I'd really like

Re: [HACKERS] pg_restore and create FK without verification check

2003-11-26 Thread Stephan Szabo
On Wed, 26 Nov 2003, ow wrote: People might be more interested in debating this topic with you if we hadn't discussed it at length just a couple months back. There wasn't consensus then that we had to offer an escape hatch, and you've not offered any argument that wasn't made before.

Re: [HACKERS] pg_restore and create FK without verification check

2003-11-26 Thread Stephan Szabo
On Wed, 26 Nov 2003, Tom Lane wrote: Quite honestly, I think they should check their foreign keys. In a partial restore situation there is no guarantee that the referenced table and the referencing table are being restored at the same time from the same dump. An override in that situation

Re: [HACKERS] pg_restore and create FK without verification check

2003-11-26 Thread Robert Treat
On Wed, 2003-11-26 at 12:43, Andreas Pflug wrote: Greg Stark wrote: If I could disable and reenable the constraint the danger that I would get the definition wrong would be eliminated. And if I had already done the work to ensure there were no broken relationships I would optionally be able

Re: [HACKERS] pg_restore and create FK without verification check

2003-11-26 Thread Tom Lane
ow [EMAIL PROTECTED] writes: --- Tom Lane [EMAIL PROTECTED] wrote: Quite honestly, I think they should check their foreign keys. Generally speaking, I agree. The problem is that verification of FK constraint(s) may take too long, depending on the size of the db and other conditions. In my

Re: [HACKERS] pg_restore and create FK without verification check

2003-11-26 Thread Stephan Szabo
On Wed, 26 Nov 2003, Tom Lane wrote: ow [EMAIL PROTECTED] writes: --- Tom Lane [EMAIL PROTECTED] wrote: Quite honestly, I think they should check their foreign keys. Generally speaking, I agree. The problem is that verification of FK constraint(s) may take too long, depending on the

Re: [HACKERS] pg_restore and create FK without verification check

2003-11-26 Thread Andreas Pflug
Stephan Szabo wrote: IIRC, he was. I think the thing causing the difference between his times and the ones we saw typically when doing the tests was that he didn't have an index on the fktable's referencing column. A common mistake, can't count how often I created this one... And not easy

Re: [HACKERS] pg_restore and create FK without verification check

2003-11-26 Thread Tom Lane
Stephan Szabo [EMAIL PROTECTED] writes: On Wed, 26 Nov 2003, Tom Lane wrote: If you're seeing this on 7.4, I'd like to see the details of the exact commands being issued. If it's not 7.4, it's not a relevant IIRC, he was. I think the thing causing the difference between his times and the

Re: [HACKERS] pg_restore and create FK without verification check

2003-11-26 Thread ow
--- Andreas Pflug [EMAIL PROTECTED] wrote: Stephan Szabo wrote: IIRC, he was. I think the thing causing the difference between his times and the ones we saw typically when doing the tests was that he didn't have an index on the fktable's referencing column. A common mistake,

Re: [HACKERS] pg_restore and create FK without verification check

2003-11-26 Thread Christopher Kings-Lynne
A common mistake, can't count how often I created this one... And not easy to find, because EXPLAIN won't explain triggers. I'm planning to create some kind of fk index wizard in pgAdmin3, which finds out about fks using columns that aren't covered by an appropriate index. Maybe this check

Re: [HACKERS] pg_restore and create FK without verification check

2003-11-26 Thread Kevin Brown
Tom Lane wrote: Andreas Pflug [EMAIL PROTECTED] writes: In principle you could do this today, but we don't have enough support code in place to make it work smoothly, eg WAL segment files aren't labeled with enough identifying information to let you manage an archive full of 'em. Still

Re: [HACKERS] pg_restore and create FK without verification check

2003-11-26 Thread Alvaro Herrera
On Thu, Nov 27, 2003 at 12:40:28AM +0100, Andreas Pflug wrote: A common mistake, can't count how often I created this one... And not easy to find, because EXPLAIN won't explain triggers. That's a pity. And the lack of EXPLAINing function execution, too. Maybe it's not that hard to do? --

Re: [HACKERS] pg_restore and create FK without verification check

2003-11-26 Thread Tom Lane
Kevin Brown [EMAIL PROTECTED] writes: Tom Lane wrote: You don't. As I said, any physical backup is going to be all-or-nothing. These techniques are not a replacement for pg_dump. But this is just an artifact of the fact that the WAL is a single instance-wide entity, rather than a

Re: [HACKERS] pg_restore and create FK without verification check

2003-11-26 Thread Tom Lane
Alvaro Herrera [EMAIL PROTECTED] writes: That's a pity. And the lack of EXPLAINing function execution, too. Maybe it's not that hard to do? Not sure if it's hard or not, but it'd sure be a nice thing to have. regards, tom lane ---(end of

Re: [HACKERS] pg_restore and create FK without verification check

2003-11-26 Thread Kevin Brown
Tom Lane wrote: Kevin Brown [EMAIL PROTECTED] writes: Tom Lane wrote: You don't. As I said, any physical backup is going to be all-or-nothing. These techniques are not a replacement for pg_dump. But this is just an artifact of the fact that the WAL is a single instance-wide entity,

Re: [HACKERS] pg_restore and create FK without verification check

2003-11-26 Thread ow
--- Alvaro Herrera [EMAIL PROTECTED] wrote: On Thu, Nov 27, 2003 at 12:40:28AM +0100, Andreas Pflug wrote: A common mistake, can't count how often I created this one... And not easy to find, because EXPLAIN won't explain triggers. That's a pity. And the lack of EXPLAINing function