Re: [HACKERS] how to find out whether a view is updatable

2013-06-05 Thread Dean Rasheed
On 4 June 2013 23:35, Michael Paquier michael.paqu...@gmail.com wrote: On Wed, Jun 5, 2013 at 12:59 AM, Peter Eisentraut pete...@gmx.net wrote: I was looking for a way in which the average psql user could learn whether a view is updatable. I was expecting something in \d, \d+, \dv, \dv+,

Re: [HACKERS] pg_rewind, a tool for resynchronizing an old master after failover

2013-06-05 Thread Greg Stark
On Thu, May 23, 2013 at 1:03 PM, Simon Riggs si...@2ndquadrant.com wrote: The COPYRIGHT file shows that VMware is claiming copyright on unstated parts of the code for this. As such, its not a normal submission to the PostgreSQL project, which involves placing copyright with the PGDG. Fwiw I

Re: [HACKERS] Optimising Foreign Key checks

2013-06-05 Thread Greg Stark
On Sat, Jun 1, 2013 at 9:41 AM, Simon Riggs si...@2ndquadrant.com wrote: COMMIT; The inserts into order_line repeatedly execute checks against the same ordid. Deferring and then de-duplicating the checks would optimise the transaction. Proposal: De-duplicate multiple checks against same

Re: [HACKERS] Optimising Foreign Key checks

2013-06-05 Thread Hannu Krosing
On 06/05/2013 11:37 AM, Greg Stark wrote: On Sat, Jun 1, 2013 at 9:41 AM, Simon Riggs si...@2ndquadrant.com wrote: COMMIT; The inserts into order_line repeatedly execute checks against the same ordid. Deferring and then de-duplicating the checks would optimise the transaction. Proposal:

Re: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL [review]

2013-06-05 Thread Amit Kapila
On Monday, May 27, 2013 4:17 PM Amit Kapila wrote: On Wednesday, April 03, 2013 11:55 AM Amit Kapila wote: On Tuesday, April 02, 2013 9:49 PM Peter Eisentraut wrote: There are 2 options to proceed for this patch for 9.4 1. Upload the SET PERSISTENT syntax patch for coming CF by fixing

[HACKERS] Redesigning checkpoint_segments

2013-06-05 Thread Heikki Linnakangas
checkpoint_segments is awkward. From an admin's point of view, it controls two things: 1. it limits the amount of disk space needed for pg_xlog. (it's a soft limit, but still) 2. it limits the time required to recover after a crash. For limiting the disk space needed for pg_xlog,

Re: [HACKERS] Time for beta2 ?

2013-06-05 Thread Tom Lane
Heikki Linnakangas hlinnakan...@vmware.com writes: We've almost cleared the open items list, and I'm not aware of any other unfixed issues in 9.3beta1. Could we make a beta2 release soon? There have been a bunch of recovery-related fixes since beta1, it would be nice to get those fixes in

Re: [HACKERS] Time for beta2 ?

2013-06-05 Thread Andres Freund
On 2013-06-05 09:50:07 -0400, Tom Lane wrote: Heikki Linnakangas hlinnakan...@vmware.com writes: We've almost cleared the open items list, and I'm not aware of any other unfixed issues in 9.3beta1. Could we make a beta2 release soon? There have been a bunch of recovery-related fixes since

Re: [HACKERS] MVCC catalog access

2013-06-05 Thread Greg Stark
On Wed, May 22, 2013 at 3:18 AM, Robert Haas robertmh...@gmail.com wrote: We've had a number of discussions about the evils of SnapshotNow. As far as I can tell, nobody likes it and everybody wants it gone, but there is concern about the performance impact. I was always under the impression

Re: [HACKERS] Vacuum, Freeze and Analyze: the big picture

2013-06-05 Thread Greg Stark
On Thu, May 30, 2013 at 7:48 PM, Josh Berkus j...@agliodbs.com wrote: The big, big picture is this: 90% of our users need to think about VACUUM/ANALYZE at least 10% of the time and 10% of our users need to think about it almost 90% of the time. That's considerably better than

Re: [HACKERS] MVCC catalog access

2013-06-05 Thread Hannu Krosing
On 06/05/2013 04:28 PM, Greg Stark wrote: On Wed, May 22, 2013 at 3:18 AM, Robert Haas robertmh...@gmail.com wrote: We've had a number of discussions about the evils of SnapshotNow. As far as I can tell, nobody likes it and everybody wants it gone, but there is concern about the performance

[HACKERS] JSON and unicode surrogate pairs

2013-06-05 Thread Andrew Dunstan
In 9.2, the JSON parser didn't check the validity of the use of unicode escapes other than that it required 4 hex digits to follow '\u'. In 9.3, that is still the case. However, the JSON accessor functions and operators also try to turn JSON strings into text in the server encoding, and this

Re: [HACKERS] MVCC catalog access

2013-06-05 Thread Andres Freund
On 2013-06-05 15:28:09 +0100, Greg Stark wrote: On Wed, May 22, 2013 at 3:18 AM, Robert Haas robertmh...@gmail.com wrote: We've had a number of discussions about the evils of SnapshotNow. As far as I can tell, nobody likes it and everybody wants it gone, but there is concern about the

[HACKERS] EXPLAIN (ANALYZE) broken

2013-06-05 Thread Kevin Grittner
Commit 2c92edad48796119c83d7dbe6c33425d1924626d has broken EXPLAIN's ANALYZE option in parentheses, which also makes some other options unusable. test=# EXPLAIN (ANALYZE) SELECT 1; ERROR:  syntax error at or near ANALYZE LINE 1: EXPLAIN (ANALYZE) SELECT 1;   ^ -- Kevin Grittner

Re: [HACKERS] EXPLAIN (ANALYZE) broken

2013-06-05 Thread Tom Lane
Kevin Grittner kgri...@ymail.com writes: Commit 2c92edad48796119c83d7dbe6c33425d1924626d has broken EXPLAIN's ANALYZE option in parentheses, which also makes some other options unusable. test=# EXPLAIN (ANALYZE) SELECT 1; ERROR:  syntax error at or near ANALYZE LINE 1: EXPLAIN (ANALYZE)

Re: [HACKERS] pg_rewind, a tool for resynchronizing an old master after failover

2013-06-05 Thread Tom Lane
Bruce Momjian br...@momjian.us writes: On Wed, Jun 5, 2013 at 10:12:17AM +0100, Greg Stark wrote: I'm not a lawyer and I make no judgement on how solid a practice this is but that's VMware doesn't seem to be doing anything special here. They can retain copyright ownership of their

Re: [HACKERS] pg_rewind, a tool for resynchronizing an old master after failover

2013-06-05 Thread Bruce Momjian
On Wed, Jun 5, 2013 at 10:12:17AM +0100, Greg Stark wrote: On Thu, May 23, 2013 at 1:03 PM, Simon Riggs si...@2ndquadrant.com wrote: The COPYRIGHT file shows that VMware is claiming copyright on unstated parts of the code for this. As such, its not a normal submission to the PostgreSQL

[HACKERS] Possible bug in cascaded standby

2013-06-05 Thread Pavan Deolasee
Hello, I am experimenting with the cascade standby and hit a problem which is reproducible with the current HEAD. I haven't tried other branches, but not sure if the test setup I am trying even works for older releases because of the timeline ID issue. Anyways, I set up a cascaded standby such

Re: [HACKERS] MVCC catalog access

2013-06-05 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: On 2013-06-05 15:28:09 +0100, Greg Stark wrote: I thought there were many call sites that were specifically depending on seeing dirty reads to avoid race conditions with other backends -- which probably just narrowed the race condition or created

Re: [HACKERS] erroneous restore into pg_catalog schema

2013-06-05 Thread Greg Stark
On Tue, May 14, 2013 at 11:59 AM, Stephen Frost sfr...@snowman.net wrote: * Dimitri Fontaine (dimi...@2ndquadrant.fr) wrote: I'm not sure I agree with that view about pg_catalog. Sometimes we talk about moving some parts of core in pre-installed extensions instead, and if we do that we will

Re: [HACKERS] pg_rewind, a tool for resynchronizing an old master after failover

2013-06-05 Thread Josh Berkus
I'm not a lawyer and I make no judgement on how solid a practice this is but that's VMware doesn't seem to be doing anything special here. They can retain copyright ownership of their contributions as long as they're happy releasing it under the Postgres copyright. Ideally they wold also be

Re: [HACKERS] MVCC catalog access

2013-06-05 Thread Andres Freund
On 2013-06-05 11:35:58 -0400, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: On 2013-06-05 15:28:09 +0100, Greg Stark wrote: I thought there were many call sites that were specifically depending on seeing dirty reads to avoid race conditions with other backends -- which

Re: [HACKERS] Possible bug in cascaded standby

2013-06-05 Thread Fujii Masao
On Thu, Jun 6, 2013 at 1:03 AM, Pavan Deolasee pavan.deola...@gmail.com wrote: Hello, I am experimenting with the cascade standby and hit a problem which is reproducible with the current HEAD. I haven't tried other branches, but not sure if the test setup I am trying even works for older

Re: [HACKERS] pg_rewind, a tool for resynchronizing an old master after failover

2013-06-05 Thread Dave Page
On Wed, Jun 5, 2013 at 6:10 PM, Josh Berkus j...@agliodbs.com wrote: I'm not a lawyer and I make no judgement on how solid a practice this is but that's VMware doesn't seem to be doing anything special here. They can retain copyright ownership of their contributions as long as they're happy

Re: [HACKERS] Redesigning checkpoint_segments

2013-06-05 Thread Fujii Masao
On Wed, Jun 5, 2013 at 9:16 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: checkpoint_segments is awkward. From an admin's point of view, it controls two things: 1. it limits the amount of disk space needed for pg_xlog. (it's a soft limit, but still) 2. it limits the time required to

Re: [HACKERS] Redesigning checkpoint_segments

2013-06-05 Thread Heikki Linnakangas
On 05.06.2013 21:16, Fujii Masao wrote: On Wed, Jun 5, 2013 at 9:16 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: I propose that we do something similar, but not exactly the same. Let's have a setting, max_wal_size, to control the max. disk space reserved for WAL. Once that's reached

Re: [HACKERS] Configurable location for extension .control files

2013-06-05 Thread Josh Berkus
Tom, Yeah, if the config option were to be superuser-only, the security issue would be ameliorated --- not removed entirely, IMO, but at least weakened. However, this seems to me to be missing the point, which is that the extensions feature is designed to let the DBA have control over which

Re: [HACKERS] Redesigning checkpoint_segments

2013-06-05 Thread Kevin Grittner
Heikki Linnakangas hlinnakan...@vmware.com wrote: I was not thinking of making it a hard limit. It would be just like checkpoint_segments from that point of view - if a checkpoint takes a long time, max_wal_size might still be exceeded. Then I suggest we not use exactly that name.  I feel

Re: [HACKERS] Redesigning checkpoint_segments

2013-06-05 Thread Fujii Masao
On Thu, Jun 6, 2013 at 3:35 AM, Heikki Linnakangas hlinnakan...@vmware.com wrote: On 05.06.2013 21:16, Fujii Masao wrote: On Wed, Jun 5, 2013 at 9:16 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: I propose that we do something similar, but not exactly the same. Let's have a

[HACKERS] About large objects asynchronous and non-blocking support

2013-06-05 Thread Giovanni Mascellani
Hi. At the moment libpq doesn't seem to support asynchronous and non-blocking support for large objects, in the style of PQsendQuery/PQgetResult. This makes large objects hardly suited for single-threaded programs based on some variant of select(). I would like to know whether this is a

Re: [HACKERS] Redesigning checkpoint_segments

2013-06-05 Thread Josh Berkus
Heikki, We shouldn't impose that calculation on the user. It should be possible to just specify checkpoint_segments=512MB, and the system would initiate checkpoints so that the total size of WAL in pg_xlog stays below 512MB. Agreed. For limiting the time required to recover after crash,

Re: [HACKERS] MVCC catalog access

2013-06-05 Thread Robert Haas
On Wed, Jun 5, 2013 at 10:28 AM, Greg Stark st...@mit.edu wrote: On Wed, May 22, 2013 at 3:18 AM, Robert Haas robertmh...@gmail.com wrote: We've had a number of discussions about the evils of SnapshotNow. As far as I can tell, nobody likes it and everybody wants it gone, but there is concern

Re: [HACKERS] About large objects asynchronous and non-blocking support

2013-06-05 Thread Dmitriy Igrishin
2013/6/5 Giovanni Mascellani g.mascell...@gmail.com Hi. At the moment libpq doesn't seem to support asynchronous and non-blocking support for large objects, in the style of PQsendQuery/PQgetResult. This makes large objects hardly suited for single-threaded programs based on some variant of

Re: [HACKERS] MVCC catalog access

2013-06-05 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: Now, I did find a couple that I thought should probably stick with SnapshotNow, specifically pgrowlocks and pgstattuple. Those are just gathering statistical information, so there's no harm in having the snapshot change part-way through the scan, and

Re: [HACKERS] About large objects asynchronous and non-blocking support

2013-06-05 Thread Tatsuo Ishii
Hi. At the moment libpq doesn't seem to support asynchronous and non-blocking support for large objects, in the style of PQsendQuery/PQgetResult. This makes large objects hardly suited for single-threaded programs based on some variant of select(). I would like to know whether this is a

Re: [HACKERS] Redesigning checkpoint_segments

2013-06-05 Thread Robert Haas
On Wed, Jun 5, 2013 at 3:24 PM, Fujii Masao masao.fu...@gmail.com wrote: OTOH, if we use max_wal_size as a hard limit, we can avoid such PANIC error and long down time. Of course, in this case, once max_wal_size is reached, we cannot complete any query writing WAL until the checkpoint has

Re: [HACKERS] MVCC catalog access

2013-06-05 Thread Robert Haas
On Wed, Jun 5, 2013 at 6:56 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: Now, I did find a couple that I thought should probably stick with SnapshotNow, specifically pgrowlocks and pgstattuple. Those are just gathering statistical information, so there's no

Re: [HACKERS] Redesigning checkpoint_segments

2013-06-05 Thread Joshua D. Drake
On 06/05/2013 05:37 PM, Robert Haas wrote: On Wed, Jun 5, 2013 at 3:24 PM, Fujii Masao masao.fu...@gmail.com wrote: OTOH, if we use max_wal_size as a hard limit, we can avoid such PANIC error and long down time. Of course, in this case, once max_wal_size is reached, we cannot complete any

Re: [HACKERS] Redesigning checkpoint_segments

2013-06-05 Thread Michael Paquier
On Thu, Jun 6, 2013 at 10:00 AM, Joshua D. Drake j...@commandprompt.comwrote: On 06/05/2013 05:37 PM, Robert Haas wrote: On Wed, Jun 5, 2013 at 3:24 PM, Fujii Masao masao.fu...@gmail.com wrote: OTOH, if we use max_wal_size as a hard limit, we can avoid such PANIC error and long down

Re: [HACKERS] Redesigning checkpoint_segments

2013-06-05 Thread Daniel Farina
On Wed, Jun 5, 2013 at 6:00 PM, Joshua D. Drake j...@commandprompt.com wrote: I didn't see that proposal, link? Because the idea of slowing down wal-writing sounds insane. It's not as insane as introducing an archiving gap, PANICing and crashing, or running this hunk o junk I wrote

[HACKERS] Regarding GIN Fast Update Technique

2013-06-05 Thread Amit Langote
Hello, At what point do the entries in the pending list are moved to the main GIN data structure? From documentation, I read that overflowing work_mem and vacuum are two such causes; what about when the concerned backend is to exit and autovacuum has not yet kicked in? -- Amit Langote --

Re: [HACKERS] Make targets of doc links used by phpPgAdmin static

2013-06-05 Thread Peter Eisentraut
On Tue, 2013-06-04 at 22:27 -0500, Karl O. Pinc wrote: On 06/04/2013 10:16:20 PM, Peter Eisentraut wrote: On Tue, 2013-05-07 at 23:18 -0400, Alvaro Herrera wrote: Peter Eisentraut wrote: On Tue, 2013-05-07 at 00:32 -0500, Karl O. Pinc wrote: Attached is a documentation patch against

Re: [HACKERS] Redesigning checkpoint_segments

2013-06-05 Thread Joshua D. Drake
On 06/05/2013 05:37 PM, Robert Haas wrote: - If it looks like we're going to exceed limit #3 before the checkpoint completes, we start exerting back-pressure on writers by making them wait every time they write WAL, probably in proportion to the number of bytes written. We keep ratcheting up

Re: [HACKERS] Redesigning checkpoint_segments

2013-06-05 Thread Joshua D. Drake
On 06/05/2013 06:23 PM, Daniel Farina wrote: On Wed, Jun 5, 2013 at 6:00 PM, Joshua D. Drake j...@commandprompt.com wrote: I didn't see that proposal, link? Because the idea of slowing down wal-writing sounds insane. It's not as insane as introducing an archiving gap, PANICing and crashing,

[HACKERS] pgbench: introduce a new automatic variable 'client_number'

2013-06-05 Thread Gurjeet Singh
Please find attached a patch for pgbench that introduces a new auto-variable 'client_number'. Following in the footsteps of 'scale' auto-variable, this is not declared if the user has specified this variable using -D switch. Since 'clientid' is a very common name a user can use for their own

Re: [HACKERS] Possible bug in cascaded standby

2013-06-05 Thread Pavan Deolasee
On Wed, Jun 5, 2013 at 10:57 PM, Fujii Masao masao.fu...@gmail.com wrote: I was not able to reproduce the problem. Maybe this is the timing problem. Hmm. I can't reproduce this on my Ubuntu box either. I will retry on the Mac machine in the evening. Surprisingly, I could reproduce it very

Re: [HACKERS] Redesigning checkpoint_segments

2013-06-05 Thread Daniel Farina
On Wed, Jun 5, 2013 at 8:23 PM, Joshua D. Drake j...@commandprompt.com wrote: It's not as insane as introducing an archiving gap, PANICing and crashing, or running this hunk o junk I wrote http://github.com/fdr/ratchet Well certainly we shouldn't PANIC and crash but that is a simple fix. You

Re: [HACKERS] Redesigning checkpoint_segments

2013-06-05 Thread Joshua D. Drake
On 6/5/2013 10:07 PM, Daniel Farina wrote: If I told you there were some of us who would prefer to attenuate the rate that things get written rather than cancel or delay archiving for a long period of time, would that explain the framing of the problem? I understand that based on what you

Re: [HACKERS] Redesigning checkpoint_segments

2013-06-05 Thread Peter Geoghegan
On Wed, Jun 5, 2013 at 10:27 PM, Joshua D. Drake j...@commandprompt.com wrote: I just wonder if we are looking in the right place (outside of some obvious badness like the PANIC running out of disk space). So you don't think we should PANIC on running out of disk space? If you don't think we