Re: [HACKERS] [PATCHES] Continue transactions after errors in
At 12:28 AM 27/04/2005, Tom Lane wrote: Can you show a plausible use-case for such a thing? A not-uncommon case in other DBs is to handle insert/update code where insert is the most likely result. Not sure if this is relevant to scripts: Begin; ...do stuff... insert into update... ...more stuff... commit; Also, the blunder-on-regardless approach is popular in pg_dump, or so I'm told ;-). Philip Warner| __---_ Albatross Consulting Pty. Ltd. |/ - \ (A.B.N. 75 008 659 498) | /(@) __---_ Tel: (+61) 0500 83 82 81 | _ \ Fax: (+61) 03 5330 3172 | ___ | Http://www.rhyme.com.au |/ \| |---- PGP key available upon request, | / and from pgp.mit.edu:11371 |/ ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [PATCHES] pg_restore recognizing $-quotes
At 01:22 AM 19/08/2004, Tom Lane wrote: Philip Warner <[EMAIL PROTECTED]> writes: > If the patch is not kosher, then I'd vote for adding a "do not parse" flag > on the TOC entries when dumping them. Or a statement count. Unless you plan to abandon compatibility with existing dump files, this doesn't seem like much of a solution Not really; pg_restore is able to read all old formats; in the case of old dump files it would behave as though statement count > 1. In the case of new dump files, it would use the statement count. It's only new dump files that exhibit the problem, so this will work. -------- Philip Warner| __---_ Albatross Consulting Pty. Ltd. |/ - \ (A.B.N. 75 008 659 498) | /(@) __---_ Tel: (+61) 0500 83 82 81 | _ \ Fax: (+61) 03 5330 3172 | ___ | Http://www.rhyme.com.au |/ \| |---- PGP key available upon request, | / and from pgp.mit.edu:11371 |/ ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send "unregister YourEmailAddressHere" to [EMAIL PROTECTED])
Re: [PATCHES] pg_restore recognizing $-quotes
At 01:33 AM 19/08/2004, Tom Lane wrote: "pg_dump -Fc | pg_restore" generate exactly the same script as pg_dump. Hard to believe this was not always true. Not sure I like the requirement that pg_restore to a database behave just like 'pg_restore | psql', though 8-}. -------- Philip Warner| __---_ Albatross Consulting Pty. Ltd. |/ - \ (A.B.N. 75 008 659 498) | /(@) __---_ Tel: (+61) 0500 83 82 81 | _ \ Fax: (+61) 03 5330 3172 | ___ | Http://www.rhyme.com.au |/ \| |---- PGP key available upon request, | / and from pgp.mit.edu:11371 |/ ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
Re: [PATCHES] pg_restore recognizing $-quotes
At 12:47 AM 19/08/2004, Tom Lane wrote: I don't want to do that, but I did think that a simpler alternative would be to inhibit pg_restore from attempting to parse FUNCTION entries. I can't see any strong need for it to do so. I don't like hard-coding stuff based on the TOC tags; but we *might* be able to get away with a more general rule: do not parse if it's an object definition (as opposed to data). In the longer term I think we will need to continue to parse TOC entries. In playing around with pg_dump(all), I put user definitions in one TOC entry. For those, we will need to add as many users as possible and ignore individual failures (something we can't to if a single multi-statement string is sent to the backend). Other TOC entries may need to be atomic. Not sure. If the patch is not kosher, then I'd vote for adding a "do not parse" flag on the TOC entries when dumping them. Or a statement count. -------- Philip Warner| __---_ Albatross Consulting Pty. Ltd. |/ - \ (A.B.N. 75 008 659 498) | /(@) __---_ Tel: (+61) 0500 83 82 81 | _ \ Fax: (+61) 03 5330 3172 | ___ | Http://www.rhyme.com.au |/ \| |---- PGP key available upon request, | / and from pgp.mit.edu:11371 |/ ---(end of broadcast)--- TIP 2: you can get off all lists at once with the unregister command (send "unregister YourEmailAddressHere" to [EMAIL PROTECTED])
Re: [PATCHES] pg_restore recognizing $-quotes
At 01:44 PM 18/08/2004, Bruce Momjian wrote: This is quite a large patch, but we do need a solution to this problem. Should it be applied? It's not as large as you might think; I had to indent a large chunk of code, and that shows up in the diff. Try applying it, and looking at a 'diff -b'. There are two issues that I can see: - I think the logic used to check for $-quotes is valid, but someone else should look at it. - Adding a full parser would be too big a change for now; and should be avoided if possible. -------- Philip Warner| __---_ Albatross Consulting Pty. Ltd. |/ - \ (A.B.N. 75 008 659 498) | /(@) __---_ Tel: (+61) 0500 83 82 81 | _ \ Fax: (+61) 03 5330 3172 | ___ | Http://www.rhyme.com.au |/ \| |---- PGP key available upon request, | / and from pgp.mit.edu:11371 |/ ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [PATCHES] [HACKERS] pg_dump 'die_on_errors'
At 01:32 AM 16/08/2004, Tom Lane wrote: It'd be substantially *more* helpful if it reported the failing command. They are two different problems; the TOC entry is important for any multiline command or to rerun the command easily later. Whereas displaying the failed SQL command is a matter of fixing the error messages. The latter is complicated by failed COPY commands which, with die-on-errors off, results in the data being processed as a command, so dumping the command will dump all of the data. In the case of long commands, should the whole command be dumped? eg. (eg. several pages of function definition). In the case of the COPY command, I'm not sure what to do. Obviously, it would be best to avoid sending the data, but the data and command are combined (from memory). Also, the 'data' may be in the form of INSERT statements. Attached patch produces the first 125 chars of the command: pg_restore: [archiver (db)] Error while PROCESSING TOC: pg_restore: [archiver (db)] Error from TOC Entry 26; 1255 16449270 FUNCTION plpgsql_call_handler() pjw pg_restore: [archiver (db)] could not execute query: ERROR: function "plpgsql_call_handler" already exists with same argument types Command was: CREATE FUNCTION plpgsql_call_handler() RETURNS language_handler AS '/var/lib/pgsql-8.0b1/lib/plpgsql', 'plpgsql_call_han... pg_restore: [archiver (db)] Error from TOC Entry 27; 1255 16449271 FUNCTION plpgsql_validator(oid) pjw pg_restore: [archiver (db)] could not execute query: ERROR: function "plpgsql_validator" already exists with same argument types Command was: CREATE FUNCTION plpgsql_validator(oid) RETURNS void AS '/var/lib/pgsql-8.0b1/lib/plpgsql', 'plpgsql_validator' LANGU... ---- Philip Warner| __---_ Albatross Consulting Pty. Ltd. |/ - \ (A.B.N. 75 008 659 498) | /(@) __---_ Tel: (+61) 0500 83 82 81 | _ \ Fax: (+61) 03 5330 3172 | ___ | Http://www.rhyme.com.au |/ \| |---- PGP key available upon request, | / and from pgp.mit.edu:11371 |/ pg_dump_040815-1.patch.gz Description: Binary data ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [PATCHES] [HACKERS] pg_dump 'die_on_errors'
At 02:32 PM 12/08/2004, Philip Warner wrote: >At 01:27 PM 12/08/2004, Bruce Momjian wrote: >Set client_min_messages to WARNING? > >Sounds like a plan. Attached patch sets client_min_messages as above and gives some context to errors messages, eg: pg_restore: [archiver (db)] Error from TOC Entry 19; 1255 16438403 FUNCTION foo() pjw pg_restore: [archiver (db)] could not execute query: ERROR: no schema has been selected to create in pg_restore: [archiver (db)] could not execute query: ERROR: schema "public" does not exist pg_restore: [archiver (db)] Error from TOC Entry 1475; 0 0 ACL public postgres pg_restore: [archiver (db)] could not execute query: ERROR: schema "public" does not exist pg_restore: [archiver (db)] could not execute query: ERROR: schema "public" does not exist pg_restore: [archiver (db)] could not execute query: ERROR: schema "public" does not exist pg_restore: [archiver (db)] could not execute query: ERROR: schema "public" does not exist -------- Philip Warner| __---_ Albatross Consulting Pty. Ltd. |/ - \ (A.B.N. 75 008 659 498) | /(@) __---_ Tel: (+61) 0500 83 82 81 | _ \ Fax: (+61) 03 5330 3172 | ___ | Http://www.rhyme.com.au |/ \| |---- PGP key available upon request, | / and from pgp.mit.edu:11371 |/ pg_dump_040815.patch.gz Description: Binary data ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
[PATCHES] pg_restore recognizing $-quotes
Not sure if this is the desired approach, but since it works, I thought I'd send it. This patch allows pg_restore to recognize $-quotes in SQL queries. It will treat any unquoted string that starts with a $ and has no preceding identifier chars as a potential $-quote tag, it then makes sure that the tag chars are valid. If so, it processes the $-quote. Tested against local DBs and regression DB. Philip Warner| __---_ Albatross Consulting Pty. Ltd. |/ - \ (A.B.N. 75 008 659 498) | /(@) __---_ Tel: (+61) 0500 83 82 81 | _ \ Fax: (+61) 03 5330 3172 | ___ | Http://www.rhyme.com.au |/ \| |---- PGP key available upon request, | / and from pgp.mit.edu:11371 |/ pg_dump_040812.patch.gz Description: Binary data ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [PATCHES] Patch for pg_dump: Multiple -t options and new
At 12:59 PM 21/07/2004, Bruce Momjian wrote: Yes, no need for it. We discourage that. Might be polite, not to mention legally required, to check with the author of the patch first. Philip Warner| __---_ Albatross Consulting Pty. Ltd. |/ - \ (A.B.N. 75 008 659 498) | /(@) __---_ Tel: (+61) 0500 83 82 81 | _ \ Fax: (+61) 03 5330 3172 | ___ | Http://www.rhyme.com.au |/ \| |---- PGP key available upon request, | / and from pgp.mit.edu:11371 |/ ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
