Re: [HACKERS] small patch to crypt.c

2013-06-09 Thread Stephen Frost
* Joshua D. Drake (j...@commandprompt.com) wrote: Well I was more referring to the default is: check if null, if true return ok check if valuntil today, if true return error else return ok To me we don't need the null check. However, when I tested it, without the null check you can't

Re: [HACKERS] Hard limit on WAL space used (because PANIC sucks)

2013-06-09 Thread Craig Ringer
On 06/09/2013 08:32 AM, MauMau wrote: - Failure of a disk containing data directory or tablespace If checkpoint can't write buffers to disk because of disk failure, checkpoint cannot complete, thus WAL files accumulate in pg_xlog/. This means that one disk failure will lead to postgres

Re: [HACKERS] Hard limit on WAL space used (because PANIC sucks)

2013-06-09 Thread Craig Ringer
On 06/09/2013 03:02 AM, Jeff Janes wrote: It would be nice to have the ability to specify multiple log destinations with different log_min_messages for each one. I'm sure syslog already must implement some kind of method for doing that, but I've been happy enough with the text logs that I've

Re: [HACKERS] Bad error message on valuntil

2013-06-09 Thread Craig Ringer
On 06/08/2013 04:07 AM, Joshua D. Drake wrote: FATAL: Authentication failed: Check server log for specifics And then we make sure we log proper info? FATAL: Authentication using method 'password' failed, possible reasons are no/wrong password sent, account expired; see server log for details ?

Re: [HACKERS] Proposed patch: remove hard-coded limit MAX_ALLOCATED_DESCS

2013-06-09 Thread Cédric Villemain
Le samedi 8 juin 2013 23:27:16, Tom Lane a écrit : =?iso-8859-1?q?C=E9dric_Villemain?= ced...@2ndquadrant.com writes: I'm not sure of expected value of max_safe_fds. Your patch now initialize with 5 slots instead of 10, if max_safe_fds is large maybe it is better to double the size each

Re: [HACKERS] pgbench: introduce a new automatic variable 'client_number'

2013-06-09 Thread Heikki Linnakangas
On 06.06.2013 06:53, Gurjeet Singh wrote: Please find attached a patch for pgbench that introduces a new auto-variable 'client_number'. Following in the footsteps of 'scale' auto-variable, this is not declared if the user has specified this variable using -D switch. Since 'clientid' is a very

Re: [HACKERS] Batch API for After Triggers

2013-06-09 Thread Simon Riggs
On 8 June 2013 22:25, Kevin Grittner kgri...@ymail.com wrote: Simon Riggs si...@2ndquadrant.com wrote: Comments please. How much of this problem space do you think could be addressed by providing OLD and NEW *relations* to AFTER EACH STATEMENT triggers? It's a reasonable question because

Re: [HACKERS] Batch API for After Triggers

2013-06-09 Thread Simon Riggs
On 9 June 2013 05:08, Stephen Frost sfr...@snowman.net wrote: * Simon Riggs (si...@2ndquadrant.com) wrote: While fiddling with FK tuning, Noah suggested batching trigger executions together to avoid execution overhead. I like the general idea, but I'd prefer a way to avoid having to queue up

Re: [HACKERS] how to find out whether a view is updatable

2013-06-09 Thread Dean Rasheed
On 6 June 2013 08:09, Dean Rasheed dean.a.rash...@gmail.com wrote: On 5 June 2013 08:59, Dean Rasheed dean.a.rash...@gmail.com wrote: I'm still not happy with pg_view_is_updatable() et al. and the information_schema views. I accept that the information_schema views have to be the way they are

Re: [HACKERS] [PATCH] add --throttle to pgbench (submission 3)

2013-06-09 Thread Craig Ringer
On 05/31/2013 03:41 PM, Fabien COELHO wrote: However I'm not sure that pg_stat_replication currently has the necessary information on either side to measure the lag (in time transactions, but how do I know when a transaction was committed? or number of transactions?). The BDR codebase now

Re: [HACKERS] Optimising Foreign Key checks

2013-06-09 Thread Simon Riggs
On 9 June 2013 02:12, Noah Misch n...@leadboat.com wrote: On Sat, Jun 08, 2013 at 08:20:42PM -0400, Robert Haas wrote: On Sat, Jun 8, 2013 at 5:41 PM, Noah Misch n...@leadboat.com wrote: Likewise; I don't see why we couldn't perform an optimistic check ASAP and schedule a final

Re: [HACKERS] Vacuum, Freeze and Analyze: the big picture

2013-06-09 Thread Craig Ringer
On 06/07/2013 04:38 AM, Jeff Janes wrote: On Mon, Jun 3, 2013 at 5:03 AM, Craig Ringer cr...@2ndquadrant.com wrote: My database is slow - This autovacuum thing is using up lots of I/O and CPU, I'll increase this delay setting here Do you think this was the correct diagnosis but with the

Re: [HACKERS] Batch API for After Triggers

2013-06-09 Thread Martijn van Oosterhout
On Sun, Jun 09, 2013 at 10:15:09AM +0100, Simon Riggs wrote: As I mentioned in my post, I did consider that and then chose not to do that. However, having a final func is a major modification in the way that we specify trigger functions. We'd also need to cope with recursive trigger execution,

Re: [HACKERS] Batch API for After Triggers

2013-06-09 Thread Craig Ringer
On 06/09/2013 04:58 PM, Simon Riggs wrote: There are also difficulties in semantics, since when we have OLD and NEW at row level we know we are discussing the same row. With sets of OLD and NEW we'd need to be able to link the relations back together somehow, which couldn't be done by PK since

Re: [HACKERS] Optimising Foreign Key checks

2013-06-09 Thread Andres Freund
On 2013-06-01 09:41:13 +0100, Simon Riggs wrote: FK checks can be expensive, especially when loading large volumes of data into an existing table or partition. A couple of ideas for improving performance are discussed here: Another idea would be to optimize away the row level locks if we have

Re: [HACKERS] ALTER TABLE ... ALTER CONSTRAINT

2013-06-09 Thread Andres Freund
On 2013-06-08 21:45:24 +0100, Simon Riggs wrote: While fiddling with FK tuning, it was useful to be able to enable and disable the DEFERRED mode of constraints. That is not currently possible in SQL, so I wrote this patch. Without this you have to drop and then re-add a constraint, which is

Re: [HACKERS] Hard limit on WAL space used (because PANIC sucks)

2013-06-09 Thread Andres Freund
On 2013-06-08 13:26:56 -0700, Joshua D. Drake wrote: At the points where the XLogInsert()s happens we're in critical sections out of which we *cannot* ERROR out because we already may have made modifications that cannot be allowed to be performed partially/unlogged. That's why we're throwing a

Re: [HACKERS] on existing update construct

2013-06-09 Thread Craig Ringer
On 05/16/2013 02:44 AM, Dev Kumkar wrote: Hello, Is there an alternative of Sybase on existing update construct in pgsql. No. See: http://www.depesz.com/2012/06/10/why-is-upsert-so-complicated/ -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7

Re: [HACKERS] MVCC catalog access

2013-06-09 Thread Andres Freund
On 2013-06-06 12:49:14 -0400, Robert Haas wrote: On Thu, Jun 6, 2013 at 5:30 AM, Andres Freund and...@2ndquadrant.com wrote: + * XXX: Now that we have MVCC catalog access, the reasoning above is no longer + * true. Are there other good reasons to hard-code this, or should we revisit

Re: [HACKERS] [PATCH] add --throttle to pgbench (submission 3)

2013-06-09 Thread Andres Freund
On 2013-06-09 17:50:13 +0800, Craig Ringer wrote: On 05/31/2013 03:41 PM, Fabien COELHO wrote: However I'm not sure that pg_stat_replication currently has the necessary information on either side to measure the lag (in time transactions, but how do I know when a transaction was committed?

[HACKERS] Re: [COMMITTERS] pgsql: Don't downcase non-ascii identifier chars in multi-byte encoding

2013-06-09 Thread Andrew Dunstan
On 06/09/2013 12:38 AM, Noah Misch wrote: On Sat, Jun 08, 2013 at 11:50:53PM -0400, Andrew Dunstan wrote: On 06/08/2013 10:52 PM, Noah Misch wrote: Let's return to the drawing board on this one. I would be inclined to keep the current bad behavior until we implement the i18n-aware case

Re: [HACKERS] Batch API for After Triggers

2013-06-09 Thread Kevin Grittner
Simon Riggs si...@2ndquadrant.com wrote: On 8 June 2013 22:25, Kevin Grittner kgri...@ymail.com wrote: Simon Riggs si...@2ndquadrant.com wrote: There are also difficulties in semantics, since when we have OLD and NEW at row level we know we are discussing the same row. With sets of OLD and

Re: [HACKERS] Optimising Foreign Key checks

2013-06-09 Thread Greg Stark
On Sun, Jun 9, 2013 at 10:51 AM, Simon Riggs si...@2ndquadrant.com wrote: AFAICS there are weird cases where changing the way FKs execute will change the way complex trigger applications will execute. I don't see a way to avoid that other than do nothing. Currently, we execute the checks

Re: [HACKERS] Vacuum, Freeze and Analyze: the big picture

2013-06-09 Thread Kevin Grittner
Craig Ringer cr...@2ndquadrant.com wrote: On 06/07/2013 04:38 AM, Jeff Janes wrote: Craig Ringer cr...@2ndquadrant.com The problem is that vacuum running too slow tends to result in table and index bloat. Which results in less efficient cache use, slower scans, and generally worsening

Re: [HACKERS] ALTER TABLE ... ALTER CONSTRAINT

2013-06-09 Thread Kevin Grittner
Andres Freund and...@2ndquadrant.com wrote: On 2013-06-08 21:45:24 +0100, Simon Riggs wrote: ALTER TABLE foo ALTER CONSTRAINT fktable_fk_fkey DEFERRED INITIALLY IMMEDIATE; I haven't looked at the patch in detail, but I am very, very much in favor of the feature in general… I have wished

Re: [HACKERS] Re: [COMMITTERS] pgsql: Don't downcase non-ascii identifier chars in multi-byte encoding

2013-06-09 Thread Tom Lane
Andrew Dunstan and...@dunslane.net writes: On 06/09/2013 12:38 AM, Noah Misch wrote: PostgreSQL has lived with this wrong behavior since ... the beginning? It's a problem, certainly, but a bandage fix brings its own trouble. I don't see this as particularly bandage-y. It's a subset of the

Re: [HACKERS] small patch to crypt.c

2013-06-09 Thread Tom Lane
Stephen Frost sfr...@snowman.net writes: Regardless, setting vuntil to some magic value that really means it's actually NULL, which is what you'd need to do in order to get rid of that explicit check for null, doesn't strike me as a good idea. When a value is null, we shouldn't be looking at

Re: [HACKERS] small patch to crypt.c

2013-06-09 Thread Joshua D. Drake
On 06/09/2013 09:28 AM, Tom Lane wrote: Even aside from that, the proposed change seems like a bad idea because it introduces an unnecessary call of GetCurrentTimestamp() in the common case where there's no valuntil limit. On some platforms that call is pretty slow. And that would explain

Re: [HACKERS] Optimising Foreign Key checks

2013-06-09 Thread Simon Riggs
On 9 June 2013 14:59, Greg Stark st...@mit.edu wrote: On Sun, Jun 9, 2013 at 10:51 AM, Simon Riggs si...@2ndquadrant.com wrote: AFAICS there are weird cases where changing the way FKs execute will change the way complex trigger applications will execute. I don't see a way to avoid that other

[HACKERS] postgres_fdw regression tests order dependency

2013-06-09 Thread Andrew Dunstan
It looks like the postgres_fdw's regression tests expect data back from the following statement in a given order, which presumably isn't guaranteed: UPDATE ft2 SET c2 = c2 + 600 WHERE c1 % 10 = 8 RETURNING *; See

[HACKERS] Valgrind Memcheck support

2013-06-09 Thread Noah Misch
Valgrind's Memcheck tool[1] is handy for finding bugs, but our use of a custom allocator limits its ability to detect problems in unmodified PostgreSQL. During the 9.1 beta cycle, I found some bugs[2] with a rough patch adding instrumentation to aset.c and mcxt.c such that Memcheck understood our

[HACKERS] 9.3 crop of memory errors

2013-06-09 Thread Noah Misch
My make installcheck runs while completing the just-posted Valgrind Memcheck patch revealed seven new and newly-detected (due to tighter checking) memory errors. Proposed patches attached. * SP-GiST moveLeafs() and doPickSplit() read past the end of a palloc These functions construct arrays of

Re: [HACKERS] Valgrind Memcheck support

2013-06-09 Thread Andres Freund
On 2013-06-09 17:25:59 -0400, Noah Misch wrote: Valgrind's Memcheck tool[1] is handy for finding bugs, but our use of a custom allocator limits its ability to detect problems in unmodified PostgreSQL. During the 9.1 beta cycle, I found some bugs[2] with a rough patch adding instrumentation to

Re: [HACKERS] Hard limit on WAL space used (because PANIC sucks)

2013-06-09 Thread MauMau
From: Craig Ringer cr...@2ndquadrant.com On 06/09/2013 08:32 AM, MauMau wrote: - Failure of a disk containing data directory or tablespace If checkpoint can't write buffers to disk because of disk failure, checkpoint cannot complete, thus WAL files accumulate in pg_xlog/. This means that one

Re: [HACKERS] JSON and unicode surrogate pairs

2013-06-09 Thread Andrew Dunstan
On 06/06/2013 12:53 PM, Robert Haas wrote: On Wed, Jun 5, 2013 at 10:46 AM, Andrew Dunstan and...@dunslane.net wrote: In 9.2, the JSON parser didn't check the validity of the use of unicode escapes other than that it required 4 hex digits to follow '\u'. In 9.3, that is still the case.

Re: [HACKERS] postgres_fdw regression tests order dependency

2013-06-09 Thread Tom Lane
Andrew Dunstan and...@dunslane.net writes: It looks like the postgres_fdw's regression tests expect data back from the following statement in a given order, which presumably isn't guaranteed: UPDATE ft2 SET c2 = c2 + 600 WHERE c1 % 10 = 8 RETURNING *; See

Re: [HACKERS] Hard limit on WAL space used (because PANIC sucks)

2013-06-09 Thread Craig Ringer
On 06/10/2013 06:39 AM, MauMau wrote: The problem is that the reliability of the database system decreases with more disks, because failure of any one of those disks would result in a database PANIC shutdown More specifically, with more independent sets of disks / file systems. I'd rather

[HACKERS] Revisit items marked 'NO' in sql_features.txt

2013-06-09 Thread Robins Tharakan
Hi, While reviewing sql_features.txt, found a few items marked NO ('Not supported') whereas, at the outset, they seemed to be supported. Apologies, if this is already considered and / or still marked 'NO' for a reason, but a list of such items mentioned below: F202TRUNCATE TABLE: identity

Re: [HACKERS] JSON and unicode surrogate pairs

2013-06-09 Thread Tom Lane
Andrew Dunstan and...@dunslane.net writes: I did that, but it's evident from the buildfarm that there's more work to do. The problem is that we do the de-escaping as we lex the json to construct the look ahead token, and at that stage we don't know whether or not it's really going to be

[HACKERS] Server side lo-funcs name

2013-06-09 Thread Tatsuo Ishii
Recently we got a complain about server side large object function names described in the doc: http://www.postgresql.org/message-id/51b2413f.8010...@gmail.com In the doc: http://www.postgresql.org/docs/9.3/static/lo-funcs.html There are server-side functions callable from SQL that correspond to

Re: [HACKERS] pg_dump with postgis extension dumps rules separately

2013-06-09 Thread Joe Conway
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 06/03/2013 07:57 PM, Tom Lane wrote: I'd have put the getRules call where getEventTriggers is now, or at least adjacent to getTriggers in one direction or the other. I'm not sure there is anything functionally wrong with what you have here; but

Re: [HACKERS] Placing hints in line pointers

2013-06-09 Thread Jeff Davis
On Sat, 2013-06-01 at 15:45 +0100, Simon Riggs wrote: Doing this would have two purposes: * We wouldn't need to follow the pointer if the row is marked aborted. This would save a random memory access for that tuple That's quite similar to LP_DEAD, right? You could potentially set this new

[HACKERS] pg_filedump 9.3: checksums (and a few other fixes)

2013-06-09 Thread Jeff Davis
Attached is a first draft of an update to pg_filedump for 9.3. I know pg_filedump is a pgfoundry project, but that seems like it's just there to host the download; so please excuse the slightly off-topic post here on -hackers. I made a few changes to support 9.3, which were mostly fixes related

Re: [HACKERS] pg_filedump 9.3: checksums (and a few other fixes)

2013-06-09 Thread Alvaro Herrera
Jeff Davis wrote: --- 1000,1015 strcat (flagString, HASEXTERNAL|); if (infoMask HEAP_HASOID) strcat (flagString, HASOID|); + if (infoMask HEAP_XMAX_KEYSHR_LOCK) + strcat (flagString, XMAX_KEYSHR_LOCK|); if (infoMask HEAP_COMBOCID)