Re: [HACKERS] SSI and Hot Standby

2011-01-20 Thread Dan Ports
On Fri, Jan 21, 2011 at 08:44:59AM +0200, Heikki Linnakangas wrote: > We have enough information in the standby to reconstruct all writes done > in the master. I gather that's not enough, in order to roll back > read-only transaction T3 on the standby which would see an anomaly, we'd > also need

Re: [HACKERS] SSI and Hot Standby

2011-01-20 Thread Heikki Linnakangas
On 21.01.2011 03:19, Dan Ports wrote: What I'm still not clear on is why that HS is different. Whatever rules apply on the master must also apply on the standby, immutably. Why is it we need to pass explicit snapshot information from master to standby? We don't do that, except at startup for norm

Re: [HACKERS] pg_basebackup for streaming base backups

2011-01-20 Thread Fujii Masao
On Fri, Jan 21, 2011 at 1:00 AM, Tom Lane wrote: > Fujii Masao writes: >> On Thu, Jan 20, 2011 at 10:53 AM, Tom Lane wrote: I'm not sure why that's the right solution. Why do you think that we should not create the tablespace under the $PGDATA directory? I'm not surprised that peo

Re: [HACKERS] Transaction-scope advisory locks

2011-01-20 Thread Marko Tiikkaja
On 1/20/2011 7:35 AM, Tom Lane wrote: Marko Tiikkaja writes: This seems useful, since the xact lock would be automatically released if an error happens during "-- do something here" so you wouldn't need to worry about releasing the lock elsewhere. But I'm not sure this is safe. Can anyone see

Re: [HACKERS] SSI and Hot Standby

2011-01-20 Thread Kevin Grittner
Robert Haas wrote: > Kevin Grittner wrote: >> As I mentioned in another email, we might want to throttle this. >> My thinking was that we could start a timer on capturing a >> snapshot, and continue to gather new ones as they become >> available. When you hit the timer limit (maybe 100ms?) you se

Re: [HACKERS] How to know killed by pg_terminate_backend

2011-01-20 Thread Tatsuo Ishii
> Here is the patch to implement the feature. > > 1) pg_terminate_backend() sends SIGUSR1 signal rather than SIGTERM to >the target backend. > 2) The infrastructure used for message passing is >storage/ipc/procsignal.c The new message type for ProcSignalReason >is "PROCSIG_TERMNINATE_B

Re: [HACKERS] Error code for "terminating connection due to conflict with recovery"

2011-01-20 Thread Tatsuo Ishii
> On Fri, Jan 14, 2011 at 1:51 PM, Robert Haas wrote: >> On Fri, Jan 14, 2011 at 12:29 PM, Simon Riggs wrote: >>> This whole thing is confused. No change is appropriate here at all. >>> >>> We issue ERRCODE_T_R_SERIALIZATION_FAILURE almost all of the time for >>> recovery conflicts. >>> >>> We is

Re: [HACKERS] bug in SignalSomeChildren

2011-01-20 Thread Fujii Masao
On Thu, Jan 20, 2011 at 9:53 PM, Bernd Helmle wrote: > I had a look at this for the current CF and the patch looks reasonable to > me. Some testing shows that the changes are working as intended (at least, > the wal sender actually receives now signals from SignalSomeChildren() as > far as the DEB

Re: [HACKERS] READ ONLY fixes

2011-01-20 Thread Kevin Grittner
> Robert Haas wrote: > Jeff Janes wrote: >> I found the following message somewhat confusing: >> ERROR: read-only property must be set before any query > > I think what we need here is two messages, this one and a similar one > that starts with "read-write property...". > >> When a subtransacti

Re: [HACKERS] READ ONLY fixes

2011-01-20 Thread Kevin Grittner
Robert Haas wrote: > Kevin Grittner wrote: >> Attached is a rebased roll-up of the 3 and 3a patches from last >> month. > do you have a link to previous discussion? http://archives.postgresql.org/pgsql-hackers/2010-12/msg00582.php That thread seems to break, but if you look at the reference

Re: [HACKERS] Error code for "terminating connection due to conflict with recovery"

2011-01-20 Thread Robert Haas
On Fri, Jan 14, 2011 at 1:51 PM, Robert Haas wrote: > On Fri, Jan 14, 2011 at 12:29 PM, Simon Riggs wrote: >> This whole thing is confused. No change is appropriate here at all. >> >> We issue ERRCODE_T_R_SERIALIZATION_FAILURE almost all of the time for >> recovery conflicts. >> >> We issue ERRCO

Re: [HACKERS] READ ONLY fixes

2011-01-20 Thread Robert Haas
On Sun, Jan 16, 2011 at 6:58 PM, Jeff Janes wrote: > I found the following message somewhat confusing: > ERROR:  read-only property must be set before any query I think what we need here is two messages, this one and a similar one that starts with "read-write property...". > When a subtransactio

Re: [HACKERS] READ ONLY fixes

2011-01-20 Thread Robert Haas
On Mon, Jan 10, 2011 at 11:27 AM, Kevin Grittner wrote: > Attached is a rebased roll-up of the 3 and 3a patches from last month. Sorry to be a dweeb, but do you have a link to previous discussion? -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Se

Re: [HACKERS] ALTER TYPE 1: recheck index-based constraints

2011-01-20 Thread Robert Haas
On Thu, Jan 20, 2011 at 2:22 PM, Noah Misch wrote: > Okay.  I've attached a new patch version based on that strategy. Thanks. Committed and back-patched to 9.0 (but I didn't use your regression test). -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -

Re: [HACKERS] Is there a way to build PostgreSQL client libraries with MinGW

2011-01-20 Thread Andrew Dunstan
On 01/20/2011 09:52 PM, Robert Haas wrote: On Thu, Jan 20, 2011 at 10:17 AM, XiaoboGu wrote: Hi, We are using R to work with 64bit PostgreSQL client libraries, and to avoid compiler compatibility issues the R development community suggest using the same compiler for both the main appl

Re: [HACKERS] Is there a way to build PostgreSQL client libraries with MinGW

2011-01-20 Thread Robert Haas
On Thu, Jan 20, 2011 at 10:17 AM, XiaoboGu wrote: > Hi, >        We are using R to work with 64bit PostgreSQL client libraries, and > to avoid compiler compatibility issues the R development community suggest > using the same compiler for both the main application and dlls. So do you > have any ex

Re: [HACKERS] REVIEW: EXPLAIN and nfiltered

2011-01-20 Thread Robert Haas
On Thu, Jan 20, 2011 at 9:35 PM, Mark Kirkwood wrote: > On 21/01/11 15:24, Robert Haas wrote: >> >> On Thu, Jan 20, 2011 at 9:17 PM, Kevin Grittner >>  wrote: >>> >>> Right -- God only knows the number of things were filtered out to >>> leave me with filtered water.  What's "filtered" in this cas

Re: [HACKERS] ALTER TABLE ... REPLACE WITH

2011-01-20 Thread Robert Haas
On Thu, Jan 20, 2011 at 6:19 PM, Noah Misch wrote: > On Thu, Jan 20, 2011 at 09:36:11PM +, Simon Riggs wrote: >> I agree that the DDL behaviour is wrong and should be fixed. Thank you >> for championing that alternative view. >> >> Swapping based upon names only works and is very flexible, muc

Re: [HACKERS] ALTER TABLE ... REPLACE WITH

2011-01-20 Thread Robert Haas
On Thu, Jan 20, 2011 at 11:17 AM, Simon Riggs wrote: > On Wed, 2011-01-19 at 22:16 -0500, Robert Haas wrote: > >> That's another way of saying "the patch is not anywhere close to being done". > > My patch is materially incomplete. Certainly we may see that as grounds > for rejection, which I would

Re: [HACKERS] REVIEW: EXPLAIN and nfiltered

2011-01-20 Thread Mark Kirkwood
On 21/01/11 15:24, Robert Haas wrote: On Thu, Jan 20, 2011 at 9:17 PM, Kevin Grittner wrote: Right -- God only knows the number of things were filtered out to leave me with filtered water. What's "filtered" in this case is what was passed through, not what was removed. Hmm, I guess I see you

Re: [HACKERS] SSI and Hot Standby

2011-01-20 Thread Robert Haas
On Thu, Jan 20, 2011 at 8:54 PM, Kevin Grittner wrote: > As I mentioned in another email, we might want to throttle this.  My > thinking was that we could start a timer on capturing a snapshot, and > continue to gather new ones as they become available.  When you hit > the timer limit (maybe 100ms

Re: [HACKERS] REVIEW: EXPLAIN and nfiltered

2011-01-20 Thread Tom Lane
"Kevin Grittner" writes: >> Robert Haas wrote: >> Oh, you mean water that had some things you didn't want taken out >> of it? > Right -- God only knows the number of things were filtered out to > leave me with filtered water. What's "filtered" in this case is what > was passed through, not wha

Re: [HACKERS] REVIEW: EXPLAIN and nfiltered

2011-01-20 Thread Robert Haas
On Thu, Jan 20, 2011 at 9:17 PM, Kevin Grittner wrote: > Right -- God only knows the number of things were filtered out to > leave me with filtered water.  What's "filtered" in this case is what > was passed through, not what was removed. Hmm, I guess I see your point now. Well, I'm not wedded t

Re: [HACKERS] One Role, Two Passwords

2011-01-20 Thread Stephen Frost
* Daniel Farina (drfar...@acm.org) wrote: > On Thu, Jan 20, 2011 at 6:12 PM, Robert Haas wrote: > > I'm not likely to write a patch for it, but if someone else writes one > > I would be willing to (a) support it and (b) subject to consensus, > > commit it. > > Wouldn't this require a client appli

Re: [HACKERS] REVIEW: EXPLAIN and nfiltered

2011-01-20 Thread Kevin Grittner
> Robert Haas wrote: > On Thu, Jan 20, 2011 at 3:57 PM, Kevin Grittner > wrote: >> Robert Haas wrote: >> >>> I think filtered is pretty clear and like it... >> >> I find it ambiguous. [Takes sip of filtered water.] > > Oh, you mean water that had some things you didn't want taken out > of it?

Re: [HACKERS] One Role, Two Passwords

2011-01-20 Thread Florian Pflug
On Jan21, 2011, at 03:14 , Daniel Farina wrote: > On Thu, Jan 20, 2011 at 6:12 PM, Robert Haas wrote: >> On Thu, Jan 20, 2011 at 9:07 PM, Stephen Frost wrote: >>> * Robert Haas (robertmh...@gmail.com) wrote: It strikes me that it would be useful to have a GUC that sets the owner of any

Re: [HACKERS] ToDo List Item - System Table Index Clustering

2011-01-20 Thread Robert Haas
On Thu, Jan 20, 2011 at 4:40 PM, Simone Aiken wrote: > After playing with this in benchmarks and researching the weird results I > got I'm going to advise dropping the todo for now unless something happens > to change how postgres handles clustering. I agree, let's remove it. That having been sa

Re: [HACKERS] One Role, Two Passwords

2011-01-20 Thread Florian Pflug
On Jan21, 2011, at 03:03 , Robert Haas wrote: > It strikes me that it would be useful to have a GUC that sets the > owner of any new objects you create (much as you can control their > default schemas using search_path). Obviously, you'd need to restrict > it so that it wouldn't allow you to creat

Re: [HACKERS] One Role, Two Passwords

2011-01-20 Thread Daniel Farina
On Thu, Jan 20, 2011 at 6:12 PM, Robert Haas wrote: > On Thu, Jan 20, 2011 at 9:07 PM, Stephen Frost wrote: >> * Robert Haas (robertmh...@gmail.com) wrote: >>> It strikes me that it would be useful to have a GUC that sets the >>> owner of any new objects you create (much as you can control their

Re: [HACKERS] One Role, Two Passwords

2011-01-20 Thread Robert Haas
On Thu, Jan 20, 2011 at 9:07 PM, Stephen Frost wrote: > * Robert Haas (robertmh...@gmail.com) wrote: >> It strikes me that it would be useful to have a GUC that sets the >> owner of any new objects you create (much as you can control their >> default schemas using search_path). > > There was a gre

Re: [HACKERS] REVIEW: EXPLAIN and nfiltered

2011-01-20 Thread Robert Haas
On Thu, Jan 20, 2011 at 4:32 PM, Tom Lane wrote: > Robert Haas writes: >> On Thu, Jan 20, 2011 at 3:47 PM, Tom Lane wrote: >>> BTW, is it just me, or is the terminology "number filtered" pretty >>> confusing/ambiguous in itself?  It doesn't seem at all clear to me >>> whether that's the number o

Re: [HACKERS] One Role, Two Passwords

2011-01-20 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: > It strikes me that it would be useful to have a GUC that sets the > owner of any new objects you create (much as you can control their > default schemas using search_path). There was a great deal of discussion along these lines over the summer of '09

Re: [HACKERS] REVIEW: EXPLAIN and nfiltered

2011-01-20 Thread Robert Haas
On Thu, Jan 20, 2011 at 3:57 PM, Kevin Grittner wrote: > Robert Haas wrote: > >> I think filtered is pretty clear and like it... > > I find it ambiguous.  [Takes sip of filtered water.] Oh, you mean water that had some things you didn't want taken out of it? -- Robert Haas EnterpriseDB: http:/

Re: [HACKERS] One Role, Two Passwords

2011-01-20 Thread Robert Haas
On Thu, Jan 20, 2011 at 8:32 PM, Josh Berkus wrote: > >> * Eventual Retirement of old credentials without having to issue ALTER >> statements (or really statements of any kind...) against application >> schema objects. > > OK, that's a different goal.  You want to be able to expire passwords > wit

Re: [HACKERS] One Role, Two Passwords

2011-01-20 Thread Stephen Frost
* Daniel Farina (drfar...@acm.org) wrote: > I have thought about that, although LDAP is the only one that came to > mind (I don't know a whole lot of systems in detail, only by name...so > suggestions welcome for low-administrative-overhead variants). My preference is Kerberos and I find that it w

Re: [HACKERS] One Role, Two Passwords

2011-01-20 Thread Daniel Farina
On Thu, Jan 20, 2011 at 5:32 PM, Josh Berkus wrote: > >> * Eventual Retirement of old credentials without having to issue ALTER >> statements (or really statements of any kind...) against application >> schema objects. > > OK, that's a different goal.  You want to be able to expire passwords > wit

Re: [HACKERS] SSI and Hot Standby

2011-01-20 Thread Kevin Grittner
Tom Lane wrote: > Simon Riggs writes: >> On Wed, 2011-01-19 at 19:05 -0600, Kevin Grittner wrote: >>> The idea is that whenever we see a valid snapshot which would >>> yield a truly serializable view of the data for a READ ONLY >>> transaction, we add a WAL record with that snapshot information.

Re: [HACKERS] One Role, Two Passwords

2011-01-20 Thread Josh Berkus
> * Eventual Retirement of old credentials without having to issue ALTER > statements (or really statements of any kind...) against application > schema objects. OK, that's a different goal. You want to be able to expire passwords with an overlap period. That's quite different from wanting an i

Re: [HACKERS] SSI and Hot Standby

2011-01-20 Thread Florian Pflug
On Jan21, 2011, at 01:28 , Simon Riggs wrote: > What I'm still not clear on is why that HS is different. Whatever rules > apply on the master must also apply on the standby, immutably. Why is it > we need to pass explicit snapshot information from master to standby? We > don't do that, except at st

Re: [HACKERS] SSI and Hot Standby

2011-01-20 Thread Dan Ports
> What I'm still not clear on is why that HS is different. Whatever rules > apply on the master must also apply on the standby, immutably. Why is it > we need to pass explicit snapshot information from master to standby? We > don't do that, except at startup for normal HS. Why do we need that? > >

Re: [HACKERS] JSON data type status?

2011-01-20 Thread Bruce Momjian
Itagaki Takahiro wrote: > On Fri, Jan 21, 2011 at 09:11, Bruce Momjian wrote: > > What happened to our work to add a JSON data type for PG 9.1? > > Nothing will happen in 9.1. > I assume we are in "competition" status: > > http://archives.postgresql.org/pgsql-hackers/2010-11/msg00481.php > > Al

Re: [HACKERS] JSON data type status?

2011-01-20 Thread Itagaki Takahiro
On Fri, Jan 21, 2011 at 09:11, Bruce Momjian wrote: > What happened to our work to add a JSON data type for PG 9.1? Nothing will happen in 9.1. I assume we are in "competition" status: http://archives.postgresql.org/pgsql-hackers/2010-11/msg00481.php Also, if PGXN will work well, we might not h

Re: [HACKERS] One Role, Two Passwords

2011-01-20 Thread Daniel Farina
On Thu, Jan 20, 2011 at 4:35 PM, Josh Berkus wrote: > >> How does this work with newly created objects? Is there a way to have >> them default objects to a different owner, the parent of the two >> roles? > > No, but you could easily assign default permissions. > >> In the case of password rotatio

Re: [HACKERS] One Role, Two Passwords

2011-01-20 Thread Josh Berkus
> How does this work with newly created objects? Is there a way to have > them default objects to a different owner, the parent of the two > roles? No, but you could easily assign default permissions. > In the case of password rotation, the goal would be to > drop the old password after all clie

Re: [HACKERS] SSI and Hot Standby

2011-01-20 Thread Kevin Grittner
Tom Lane wrote: > I'm pretty concerned about the performance implications, too. In > particular that sounds like you could get an unbounded amount of > WAL emitted from a *purely read only* transaction flow. No. Read only transactions wouldn't create any flow at all. And I suggested that we

Re: [HACKERS] SSI and Hot Standby

2011-01-20 Thread Simon Riggs
On Fri, 2011-01-21 at 00:26 +0100, Florian Pflug wrote: > On Jan21, 2011, at 00:11 , Simon Riggs wrote: > > It's not clear to me what the reason is that this doesn't just work on > > HS already. If you started there it might help. > > > The problem is that snapshots taken on the master sometimes

Re: [HACKERS] One Role, Two Passwords

2011-01-20 Thread Daniel Farina
On Thu, Jan 20, 2011 at 3:34 PM, Tom Lane wrote: > Daniel Farina writes: >> I wanted to test the waters on how receptive people might be to an >> extension that would allow Postgres to support two passwords for a >> given role. > > Not very.  Why don't you just put two roles in the same group? H

[HACKERS] JSON data type status?

2011-01-20 Thread Bruce Momjian
What happened to our work to add a JSON data type for PG 9.1? -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)

Re: [HACKERS] SSI and Hot Standby

2011-01-20 Thread Tom Lane
Simon Riggs writes: > On Wed, 2011-01-19 at 19:05 -0600, Kevin Grittner wrote: >> The idea is that whenever we see a valid snapshot which would yield >> a truly serializable view of the data for a READ ONLY transaction, >> we add a WAL record with that snapshot information. > You haven't explain

Re: [HACKERS] One Role, Two Passwords

2011-01-20 Thread Tom Lane
Daniel Farina writes: > I wanted to test the waters on how receptive people might be to an > extension that would allow Postgres to support two passwords for a > given role. Not very. Why don't you just put two roles in the same group? regards, tom lane -- Sent via pgs

Re: [HACKERS] Orphaned statements issue

2011-01-20 Thread Josh Berkus
On 1/20/11 2:26 PM, Tom Lane wrote: > Josh Berkus writes: >>> I would take that to mean that it's waiting on the client. > >> You mean that the client timed out and isn't accepting data from the >> query anymore? > > No, if the backend is in RECV state, it's waiting for the client to > *send* it

Re: [HACKERS] SSI and Hot Standby

2011-01-20 Thread Kevin Grittner
I wrote: > Why not? We already generate appropriate snapshots for this in > SSI, so is the problem in getting the appropriate information into > the WAL stream or in having a request for a snapshot within a > serializable transaction while running in hot standby the problem? I dropped few word

Re: [HACKERS] SSI and Hot Standby

2011-01-20 Thread Florian Pflug
On Jan21, 2011, at 00:11 , Simon Riggs wrote: > It's not clear to me what the reason is that this doesn't just work on > HS already. If you started there it might help. The problem is that snapshots taken on the master sometimes represent a state of the database which cannot occur under any (vali

Re: [HACKERS] REVIEW: EXPLAIN and nfiltered

2011-01-20 Thread Marko Tiikkaja
On 1/20/2011 12:47 PM, Tom Lane wrote: So the line I'm thinking we should pursue is to visually associate the new counter with the filter condition, either like Filter Cond: (x> 42) (nfiltered = 123) or Filter Cond: (x> 42) Rows Filtered: 123 I'd prefer the latter.

Re: [HACKERS] SSI and Hot Standby

2011-01-20 Thread Kevin Grittner
Simon Riggs wrote: > On Wed, 2011-01-19 at 19:05 -0600, Kevin Grittner wrote: > >> The idea is that whenever we see a valid snapshot which would >> yield a truly serializable view of the data for a READ ONLY >> transaction, we add a WAL record with that snapshot information. > > You haven't exp

Re: [HACKERS] ALTER TABLE ... REPLACE WITH

2011-01-20 Thread Tom Lane
Robert Haas writes: > On Thu, Jan 20, 2011 at 4:24 PM, Tom Lane wrote: >> If you're willing to substitute an incompatible table, it's not clear >> why you don't just do >> >>begin; >>drop table t; >>alter table t_new rename to t; >>

Re: [HACKERS] ALTER TABLE ... REPLACE WITH

2011-01-20 Thread Noah Misch
On Thu, Jan 20, 2011 at 09:36:11PM +, Simon Riggs wrote: > I agree that the DDL behaviour is wrong and should be fixed. Thank you > for championing that alternative view. > > Swapping based upon names only works and is very flexible, much more so > than EXCHANGE could be. > > A separate utili

Re: [HACKERS] [GENERAL] Large object corruption during 'piped' pg_restore

2011-01-20 Thread Tom Lane
Bosco Rama writes: >>> If 'standard_conforming_strings = on' is set in our DB (which is required >>> for >>> our app) then the piped restore method (e.g. pg_restore -O backup.dat | >>> psql) >>> results in the large objects being corrupted. > All servers and client tools involved are PG 8.4.6 o

Re: [HACKERS] SSI and Hot Standby

2011-01-20 Thread Simon Riggs
On Wed, 2011-01-19 at 19:05 -0600, Kevin Grittner wrote: > The idea is that whenever we see a valid snapshot which would yield > a truly serializable view of the data for a READ ONLY transaction, > we add a WAL record with that snapshot information. You haven't explained why this approach is the

Re: [HACKERS] One Role, Two Passwords

2011-01-20 Thread Daniel Farina
On Thu, Jan 20, 2011 at 2:45 PM, Andrew Dunstan wrote: > Have you thought of trying to use an external auth source like LDAP for such > a scheme? I have thought about that, although LDAP is the only one that came to mind (I don't know a whole lot of systems in detail, only by name...so suggestion

Re: [HACKERS] ALTER TABLE ... REPLACE WITH

2011-01-20 Thread Robert Haas
On Thu, Jan 20, 2011 at 4:24 PM, Tom Lane wrote: > Noah Misch writes: >> Heikki's suggestion seemed straightforward, so much so that I couldn't figure >> why nobody had done it.  That would usually mean I'm missing something. > > If you're willing to substitute an incompatible table, it's not cle

Re: [HACKERS] One Role, Two Passwords

2011-01-20 Thread Andrew Dunstan
On 01/20/2011 05:28 PM, Daniel Farina wrote: Hello list, I wanted to test the waters on how receptive people might be to an extension that would allow Postgres to support two passwords for a given role. I have recently encountered a case where this would be highly useful when performing rollin

[HACKERS] One Role, Two Passwords

2011-01-20 Thread Daniel Farina
Hello list, I wanted to test the waters on how receptive people might be to an extension that would allow Postgres to support two passwords for a given role. I have recently encountered a case where this would be highly useful when performing rolling password upgrades across many client applicatio

Re: [HACKERS] Orphaned statements issue

2011-01-20 Thread Tom Lane
Josh Berkus writes: >> I would take that to mean that it's waiting on the client. > You mean that the client timed out and isn't accepting data from the > query anymore? No, if the backend is in RECV state, it's waiting for the client to *send* it something. (Although if this is an SSL connecti

Re: [HACKERS] SSI and Hot Standby

2011-01-20 Thread Josh Berkus
Kevin, > So, based on a more complete description of the issues, any more > opinions on whether to generate the error, as suggested by Heikki? If it's a choice between generating an error and letting users see inconsistent data, I'll take the former. > Does anyone think this justifies the compa

Re: [HACKERS] estimating # of distinct values

2011-01-20 Thread Tomas Vondra
Dne 20.1.2011 11:05, Csaba Nagy napsal(a): > Hi Tomas, > > On Wed, 2011-01-19 at 23:13 +0100, Tomas Vondra wrote: >> No, the multi-column statistics do not require constant updating. There >> are cases where a sampling is perfectly fine, although you may need a >> bit larger sample. Generally if y

Re: [HACKERS] SSI and Hot Standby

2011-01-20 Thread Kevin Grittner
Heikki Linnakangas wrote: > On 20.01.2011 03:05, Kevin Grittner wrote: >> If we don't do something like this, do we just provide REPEATABLE >> READ on the standby as the strictest level of transaction >> isolation? If so, do we generate an error on a request for >> SERIALIZABLE, warn and provide

Re: [HACKERS] estimating # of distinct values

2011-01-20 Thread Tomas Vondra
Dne 20.1.2011 09:10, Heikki Linnakangas napsal(a): > It seems that the suggested multi-column selectivity estimator would be > more sensitive to ndistinct of the individual columns. Is that correct? > How is it biased? If we routinely under-estimate ndistinct of individual > columns, for example, d

Re: [HACKERS] Orphaned statements issue

2011-01-20 Thread Josh Berkus
> I would take that to mean that it's waiting on the client. You mean that the client timed out and isn't accepting data from the query anymore? Shouldn't Postgres time out on that after a while? In one case, the orphaned statement was 16 hours old before we killed it. If it's relevant, the cl

Re: [HACKERS] estimating # of distinct values

2011-01-20 Thread Tomas Vondra
Dne 20.1.2011 03:36, Robert Haas napsal(a): > On Wed, Jan 19, 2011 at 5:13 PM, Tomas Vondra wrote: Regarding the crash scenario - if the commit fails, just throw away the local estimator copy, it's not needed. I'm not sure how to take care of the case when commit succeeds and the wr

Re: [HACKERS] ToDo List Item - System Table Index Clustering

2011-01-20 Thread Simone Aiken
After playing with this in benchmarks and researching the weird results I got I'm going to advise dropping the todo for now unless something happens to change how postgres handles clustering. You guys probably already grokked this so I am just recording it for the list archives. The primary

Re: [HACKERS] Orphaned statements issue

2011-01-20 Thread Tom Lane
Josh Berkus writes: > One of our clients is seeing an unusual issue with statements which are > waiting going into sleep forever and never completing. This first > e-mail is for a "has anyone else seen this problem before?" while we try > to collect additional information for a diagnosis. > An

Re: [HACKERS] ALTER TABLE ... REPLACE WITH

2011-01-20 Thread Simon Riggs
On Thu, 2011-01-20 at 13:14 -0500, Noah Misch wrote: > When DDL has taken AccessExclusiveLock and a query waits for it, it's the > Right > Thing for that query to wake up and proceed based on the complete, final state > of that committed DDL. Aside from the waiting itself, the query should behav

Re: [HACKERS] Orphaned statements issue

2011-01-20 Thread Josh Berkus
> I have seen it -- on 8.1 too. On our case it was caused by an insert > that was doing lots of toast insertions, so it needed to grab the > extension lock frequently for the toast table; and this was slowed > down by a largish shared_buffers setting, somehow (8.1 doesn't have lock > partitioning

Re: [HACKERS] REVIEW: EXPLAIN and nfiltered

2011-01-20 Thread Tom Lane
Robert Haas writes: > On Thu, Jan 20, 2011 at 3:47 PM, Tom Lane wrote: >> BTW, is it just me, or is the terminology "number filtered" pretty >> confusing/ambiguous in itself?  It doesn't seem at all clear to me >> whether that's the number of rows passed by the filter condition or >> the number o

Re: [HACKERS] estimating # of distinct values

2011-01-20 Thread Tomas Vondra
Dne 20.1.2011 03:06, Nathan Boley napsal(a): >> And actually it does not depend on ndistinct for the columns only, it >> depends on ndistinct estimates for the combination of columns. So >> improving the ndistinct estimates for columns is just a necessary first >> step (and only if it works reasona

Re: [HACKERS] ALTER TABLE ... REPLACE WITH

2011-01-20 Thread Tom Lane
Noah Misch writes: > Heikki's suggestion seemed straightforward, so much so that I couldn't figure > why nobody had done it. That would usually mean I'm missing something. If you're willing to substitute an incompatible table, it's not clear why you don't just do begin;

Re: [HACKERS] Orphaned statements issue

2011-01-20 Thread Alvaro Herrera
Excerpts from Josh Berkus's message of jue ene 20 18:05:15 -0300 2011: > > One of our clients is seeing an unusual issue with statements which are > waiting going into sleep forever and never completing. This first > e-mail is for a "has anyone else seen this problem before?" while we try > to

Re: [HACKERS] pg_basebackup for streaming base backups

2011-01-20 Thread Tom Lane
Robert Haas writes: > On Thu, Jan 20, 2011 at 2:10 PM, Dimitri Fontaine > wrote: >> Robert Haas writes: >>> I think that the basic problem with wal_level is that to increase it >>> you need to somehow ensure that all the backends have the new setting, >>> and then checkpoint. >> >> Well, you ju

[HACKERS] Orphaned statements issue

2011-01-20 Thread Josh Berkus
Hackers, One of our clients is seeing an unusual issue with statements which are waiting going into sleep forever and never completing. This first e-mail is for a "has anyone else seen this problem before?" while we try to collect additional information for a diagnosis. This problem only happen

Re: [HACKERS] REVIEW: EXPLAIN and nfiltered

2011-01-20 Thread Kevin Grittner
Robert Haas wrote: > I think filtered is pretty clear and like it... I find it ambiguous. [Takes sip of filtered water.] How about excluded? -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpr

Re: [HACKERS] REVIEW: EXPLAIN and nfiltered

2011-01-20 Thread Robert Haas
On Thu, Jan 20, 2011 at 3:47 PM, Tom Lane wrote: > The main functional problem I see with this format is that it assumes > there is one and only one filter step associated with every plan node. > That is just plain wrong.  Many don't have any, and there are important > cases where there are two.  

Re: [HACKERS] REVIEW: EXPLAIN and nfiltered

2011-01-20 Thread Tom Lane
hubert depesz lubaczewski writes: > On Thu, Jan 20, 2011 at 02:48:59PM -0500, Stephen Frost wrote: >> He also mentioned that he didn't feel it was terribly complicated or >> that it'd be difficult to update for this. Looking over the code, it's >> got a simple regex for matching that line which w

Re: [HACKERS] pl/python refactoring

2011-01-20 Thread Peter Eisentraut
On ons, 2011-01-19 at 10:06 +0900, Hitoshi Harada wrote: > - This is not in the patch, but around line 184 "vis versa" in comment > seems like typo. Fixed. > - A line break should be added before PLy_add_exception() after "static void" I'll add that when I get to the patch. > - This is also not

Re: [HACKERS] exceptions not present in plpy with Python 3

2011-01-20 Thread Peter Eisentraut
On lör, 2010-12-18 at 18:56 +0100, Jan Urbański wrote: > there seems to be a problem in the way we add exceptions to the plpy > module in PL/Python compiled with Python 3k. > > Try this: DO $$ plpy.SPIError $$ language plpython3u; > > I'm not a Python 3 expert, but I nicked some code from the Int

Re: [HACKERS] pg_basebackup for streaming base backups

2011-01-20 Thread Heikki Linnakangas
On 20.01.2011 22:15, Dimitri Fontaine wrote: Robert Haas writes: Sending them a signal seems like a promising approach, but the trick is guaranteeing that they've actually acted on it before you start the checkpoint. How much using a latch here would help? Or be overkill? A latch doesn't g

Re: [HACKERS] pg_basebackup for streaming base backups

2011-01-20 Thread Dimitri Fontaine
Robert Haas writes: > Sending them a signal seems like a promising approach, but the trick > is guaranteeing that they've actually acted on it before you start the > checkpoint. How much using a latch here would help? Or be overkill? -- Dimitri Fontaine http://2ndQuadrant.fr PostgreSQL : E

Re: [HACKERS] REVIEW: EXPLAIN and nfiltered

2011-01-20 Thread hubert depesz lubaczewski
On Thu, Jan 20, 2011 at 02:48:59PM -0500, Stephen Frost wrote: > * Robert Haas (robertmh...@gmail.com) wrote: > > How much has that code been updated from one release to the next? > > Just an FYI, I talked to depesz on IRC (please chime in if you disagree > with any of this) and he indicated that

Re: [HACKERS] pg_basebackup for streaming base backups

2011-01-20 Thread Robert Haas
On Thu, Jan 20, 2011 at 2:10 PM, Dimitri Fontaine wrote: > Robert Haas writes: >> I think that the basic problem with wal_level is that to increase it >> you need to somehow ensure that all the backends have the new setting, >> and then checkpoint.  Right now, the backends get the value through >

Re: [HACKERS] REVIEW: EXPLAIN and nfiltered

2011-01-20 Thread Robert Haas
On Thu, Jan 20, 2011 at 1:47 PM, Stephen Frost wrote: > * Tom Lane (t...@sss.pgh.pa.us) wrote: >> Robert Haas writes: >> > I agree; we make bigger changes than this all the time. >> >> No, we don't. > > Alright, do we want to go down the road of adding new things to the > XML/JSON/YAML/Whatever-e

Re: [HACKERS] REVIEW: EXPLAIN and nfiltered

2011-01-20 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: > How much has that code been updated from one release to the next? Just an FYI, I talked to depesz on IRC (please chime in if you disagree with any of this) and he indicated that he's had to update the code from time to time, mostly because the parser

Re: [HACKERS] ALTER TYPE 1: recheck index-based constraints

2011-01-20 Thread Noah Misch
On Thu, Jan 20, 2011 at 09:26:29AM -0500, Robert Haas wrote: > My main beef with the Boolean flags is that this kind of thing is not too > clear: > >reindex_relation(myrel, false, false, true, true, false, true, > false, false, true); > > Unless you have an excellent memory, you can't tell w

Re: [HACKERS] pg_basebackup for streaming base backups

2011-01-20 Thread Dimitri Fontaine
Robert Haas writes: > I think that the basic problem with wal_level is that to increase it > you need to somehow ensure that all the backends have the new setting, > and then checkpoint. Right now, the backends get the value through > the GUC machinery, and so there's no particular bound on how l

Re: [HACKERS] Moving test_fsync to /contrib?

2011-01-20 Thread Josh Berkus
On 1/20/11 6:15 AM, Robert Haas wrote: > On Thu, Jan 20, 2011 at 9:13 AM, Bruce Momjian wrote: >> OK, I am ready to move test_fsync to /contrib. Is pg_test_fsync the >> best name? pg_check_fsync? pg_fsync_performance? pg_verify_fsync? > > I don't see too much reason to rename it more than nec

Re: [HACKERS] REVIEW: EXPLAIN and nfiltered

2011-01-20 Thread Stephen Frost
* Tom Lane (t...@sss.pgh.pa.us) wrote: > Robert Haas writes: > > I agree; we make bigger changes than this all the time. > > No, we don't. Alright, do we want to go down the road of adding new things to the XML/JSON/YAML/Whatever-else format that isn't displayed in the TEXT version, to avoid thi

Re: [HACKERS] ALTER TABLE ... REPLACE WITH

2011-01-20 Thread Noah Misch
On Thu, Jan 20, 2011 at 10:07:23AM +, Simon Riggs wrote: > On Wed, 2011-01-19 at 17:46 -0500, Noah Misch wrote: > > > First, I'd like to note that the thread for this patch had *four* "me-too" > > responses to the use case. That's extremely unusual; the subject is > > definitely > > compelli

Re: [HACKERS] REVIEW: EXPLAIN and nfiltered

2011-01-20 Thread Magnus Hagander
On Jan 20, 2011 6:43 PM, "Tom Lane" wrote: > > Robert Haas writes: > > On Thu, Jan 20, 2011 at 11:55 AM, Stephen Frost wrote: > >> While I agree completely about the general "if you're going to break, > >> break it big" approach, but I don't particularly care for holding output > >> strings from

Re: [HACKERS] REVIEW: EXPLAIN and nfiltered

2011-01-20 Thread Robert Haas
On Thu, Jan 20, 2011 at 12:57 PM, Magnus Hagander wrote: > > On Jan 20, 2011 6:43 PM, "Tom Lane" wrote: >> >> Robert Haas writes: >> > On Thu, Jan 20, 2011 at 11:55 AM, Stephen Frost >> > wrote: >> >> While I agree completely about the general "if you're going to break, >> >> break it big" appr

Re: [HACKERS] pg_basebackup for streaming base backups

2011-01-20 Thread Robert Haas
On Thu, Jan 20, 2011 at 11:59 AM, Dimitri Fontaine wrote: > Robert Haas writes: >> Also, it won't actually work unless the server has replication >> configured (wal_level!=minimal, max_wal_senders>0, and possibly some >> setting for wal_keep_segments), which has been the main point of the >> nami

Re: [HACKERS] CommitFest wrap-up

2011-01-20 Thread Tom Lane
Robert Haas writes: > On Tue, Dec 21, 2010 at 10:49 PM, Robert Haas wrote: >> On Tue, Dec 21, 2010 at 11:12 AM, Tom Lane wrote: > Robert Haas writes: > On Wed, Dec 15, 2010 at 11:29 AM, Tom Lane wrote: - Writeable CTEs - I think we need Tom to pick this one up. - Fix snapshot taking

  1   2   >