Re: [HACKERS] Latch for the WAL writer - further reducing idle wake-ups.

2012-05-03 Thread Heikki Linnakangas
On 03.05.2012 03:41, Robert Haas wrote: On Wed, May 2, 2012 at 7:21 PM, Tom Lanet...@sss.pgh.pa.us wrote: Adding any contention at all to XLogInsert doesn't seem like a smart idea, even if you failed to measure any problem in the specific tests you made. I wonder whether we could not improve

Re: [HACKERS] Advisory locks seem rather broken

2012-05-03 Thread Simon Riggs
On Thu, May 3, 2012 at 1:19 AM, Tom Lane t...@sss.pgh.pa.us wrote: According to http://archives.postgresql.org/pgsql-general/2012-04/msg00374.php advisory locks now cause problems for prepared transactions, which ought to ignore them.  It appears to me that this got broken by commit

Re: [HACKERS] PL/Python result set slicing broken in Python 3

2012-05-03 Thread Jan Urbański
On 02/05/12 20:18, Peter Eisentraut wrote: This doesn't work anymore with Python 3: rv = plpy.execute(...) do_something(rv[0:1]) Apparently, they changed the C API for doing slicing, or rather made one of the two APIs for it silently do nothing. Details are difficult to find, but this email

Re: [HACKERS] Temporary tables under hot standby

2012-05-03 Thread Simon Riggs
On Thu, May 3, 2012 at 1:57 AM, Josh Berkus j...@agliodbs.com wrote: Michael, What is the use case for temporary tables on a hot standby server? Perhaps this is a noobie question, but it seems to me that a hot standby server's use by* applications* or *users* should be limited to

Re: [HACKERS] Modeling consumed shmem sizes, and some thorns

2012-05-03 Thread Simon Riggs
On Wed, May 2, 2012 at 9:38 PM, Daniel Farina dan...@heroku.com wrote: Besides accuracy, there is a thornier problem here that has to do with hot standby (although the use case is replication more generally) when one has heterogeneously sized database resources. As-is, it is required that

Re: [HACKERS] Modeling consumed shmem sizes, and some thorns

2012-05-03 Thread Daniel Farina
On Thu, May 3, 2012 at 2:23 AM, Simon Riggs si...@2ndquadrant.com wrote: On Wed, May 2, 2012 at 9:38 PM, Daniel Farina dan...@heroku.com wrote: Besides accuracy, there is a thornier problem here that has to do with hot standby (although the use case is replication more generally) when one has

Re: [HACKERS] Latch for the WAL writer - further reducing idle wake-ups.

2012-05-03 Thread Magnus Hagander
On Thu, May 3, 2012 at 2:41 AM, Robert Haas robertmh...@gmail.com wrote: On Wed, May 2, 2012 at 7:21 PM, Tom Lane t...@sss.pgh.pa.us wrote: It is getting a bit late to be considering such changes for 9.2, but I'm willing to review and commit this if there's not anybody who feels strongly that

Re: [HACKERS] How hard would it be to support LIKE in return declaration of generic record function calls ?

2012-05-03 Thread Hannu Krosing
On Wed, 2012-05-02 at 14:32 -0500, Merlin Moncure wrote: On Wed, May 2, 2012 at 12:06 PM, Peter Eisentraut pete...@gmx.net wrote: On ons, 2012-05-02 at 13:40 +0200, Hannu Krosing wrote: How hard would it be to add support for LIKE syntax, similar to table def in field list declaration for

Re: [HACKERS] How hard would it be to support LIKE in return declaration of generic record function calls ?

2012-05-03 Thread Pavel Stehule
Hello (1 row) This works the same indeed, just seems to be a hack, though a cool one :) hannu=# insert into test hannu-# SELECT * FROM populate_record(null::test,                              'id=456, data=zzz'); INSERT 0 1 few years back I proposed anytypename type with this feature,

[HACKERS] unexpected EOF messages

2012-05-03 Thread Magnus Hagander
I had a request from a customer asking if we could make a switch to specifically disable the unexpected EOF message that fills lots of peoples logs. Along the same way that we have a flag to turn off the nonstandard use of string escapes message that is another culprit (that's actually a much

Re: [HACKERS] unexpected EOF messages

2012-05-03 Thread Simon Riggs
On Thu, May 3, 2012 at 1:26 PM, Magnus Hagander mag...@hagander.net wrote: I had a request from a customer asking if we could make a switch to specifically disable the unexpected EOF message that fills lots of peoples logs. Along the same way that we have a flag to turn off the nonstandard use

[HACKERS] ALTER DATABASE and datallowconn

2012-05-03 Thread Magnus Hagander
Is there a particular reason we don't have an ALTER DATABASE switch that controls the datallowconn, or is it just something missed out? --  Magnus Hagander  Me: http://www.hagander.net/  Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)

Re: [HACKERS] unexpected EOF messages

2012-05-03 Thread Magnus Hagander
On Thu, May 3, 2012 at 2:31 PM, Simon Riggs si...@2ndquadrant.com wrote: On Thu, May 3, 2012 at 1:26 PM, Magnus Hagander mag...@hagander.net wrote: I had a request from a customer asking if we could make a switch to specifically disable the unexpected EOF message that fills lots of peoples

Re: [HACKERS] unexpected EOF messages

2012-05-03 Thread Vik Reykja
On Thu, May 3, 2012 at 2:31 PM, Simon Riggs si...@2ndquadrant.com wrote: Would we consider adding such a switch (it should be easy enough to do), or do we want to push this off to the mythical let's improve the logging subsystem project that might eventually materialize if we're lucky?

Re: [HACKERS] unexpected EOF messages

2012-05-03 Thread Magnus Hagander
On Thu, May 3, 2012 at 2:34 PM, Vik Reykja vikrey...@gmail.com wrote: On Thu, May 3, 2012 at 2:31 PM, Simon Riggs si...@2ndquadrant.com wrote: Would we consider adding such a switch (it should be easy enough to do), or do we want to push this off to the mythical let's improve the logging

Re: [HACKERS] unexpected EOF messages

2012-05-03 Thread Kevin Grittner
Magnus Hagander wrote: Also, AFAIK we don't *have* a message type at this point (one of the things said mythical project wanted to look at), so the only thing we could really filter on would be the whole text of the message, no? We have SQLSTATE, but this seems to be one of those

Re: [HACKERS] ALTER DATABASE and datallowconn

2012-05-03 Thread Simon Riggs
On Thu, May 3, 2012 at 1:31 PM, Magnus Hagander mag...@hagander.net wrote: Is there a particular reason we don't have an ALTER DATABASE switch that controls the datallowconn, or is it just something missed out? I think it can be removed, or rather deprecated. datconnlimit can be set to 0 If

Re: [HACKERS] Torn page hazard in ginRedoUpdateMetapage()

2012-05-03 Thread Robert Haas
On Thu, May 3, 2012 at 12:16 AM, Tom Lane t...@sss.pgh.pa.us wrote: Having said all that, I wasn't really arguing that this was a guaranteed safe thing for us to rely on; just pointing out that it's quite likely that the issue hasn't been seen in the field because of this type of

Re: [HACKERS] ALTER DATABASE and datallowconn

2012-05-03 Thread Magnus Hagander
On Thu, May 3, 2012 at 2:48 PM, Simon Riggs si...@2ndquadrant.com wrote: On Thu, May 3, 2012 at 1:31 PM, Magnus Hagander mag...@hagander.net wrote: Is there a particular reason we don't have an ALTER DATABASE switch that controls the datallowconn, or is it just something missed out? I think

[HACKERS] outdated comment in heapam.c

2012-05-03 Thread Andres Freund
Hi, It seems that when Heikki added the multi_insert code the following comment in htup.h wasn't updated: /* * We ran out of opcodes, so heapam.c now has a second RmgrId. These opcodes * are associated with RM_HEAP2_ID, but are not logically different from * the ones above associated with

Re: [HACKERS] Temporary tables under hot standby

2012-05-03 Thread Merlin Moncure
On Thu, May 3, 2012 at 4:11 AM, Simon Riggs si...@2ndquadrant.com wrote: which seems to me to be actually harder than just rewriting as derived table and isn't an option on Microstrategy etc, hence my observation that GTTs don't help HS much. What I would like to see, one day, is for temp

Re: [HACKERS] How hard would it be to support LIKE in return declaration of generic record function calls ?

2012-05-03 Thread Merlin Moncure
On Thu, May 3, 2012 at 7:13 AM, Pavel Stehule pavel.steh...@gmail.com wrote: Hello (1 row) This works the same indeed, just seems to be a hack, though a cool one :) Yeah -- the syntax isn't great, but IMO it's more generally usable than what you're proposing because it's a scalar returning

Re: [HACKERS] How hard would it be to support LIKE in return declaration of generic record function calls ?

2012-05-03 Thread Pavel Stehule
2012/5/3 Merlin Moncure mmonc...@gmail.com: On Thu, May 3, 2012 at 7:13 AM, Pavel Stehule pavel.steh...@gmail.com wrote: Hello (1 row) This works the same indeed, just seems to be a hack, though a cool one :) Yeah -- the syntax isn't great, but IMO it's more generally usable than what

Re: [HACKERS] How hard would it be to support LIKE in return declaration of generic record function calls ?

2012-05-03 Thread Andrew Dunstan
On 05/03/2012 09:43 AM, Pavel Stehule wrote: 2012/5/3 Merlin Moncuremmonc...@gmail.com: On Thu, May 3, 2012 at 7:13 AM, Pavel Stehulepavel.steh...@gmail.com wrote: Hello (1 row) This works the same indeed, just seems to be a hack, though a cool one :) Yeah -- the syntax isn't great, but

Re: [HACKERS] ALTER DATABASE and datallowconn

2012-05-03 Thread Tom Lane
Magnus Hagander mag...@hagander.net writes: Is there a particular reason we don't have an ALTER DATABASE switch that controls the datallowconn, or is it just something missed out? It was never intended to be a user-accessible switch, just something to protect template0. I don't agree with

Re: [HACKERS] unexpected EOF messages

2012-05-03 Thread Tom Lane
Simon Riggs si...@2ndquadrant.com writes: On Thu, May 3, 2012 at 1:26 PM, Magnus Hagander mag...@hagander.net wrote: I had a request from a customer asking if we could make a switch to specifically disable the unexpected EOF message that fills lots of peoples logs. Yes, if the new parameter

Re: [HACKERS] How hard would it be to support LIKE in return declaration of generic record function calls ?

2012-05-03 Thread Merlin Moncure
On Thu, May 3, 2012 at 9:01 AM, Andrew Dunstan and...@dunslane.net wrote: On 05/03/2012 09:43 AM, Pavel Stehule wrote: 2012/5/3 Merlin Moncuremmonc...@gmail.com: On Thu, May 3, 2012 at 7:13 AM, Pavel Stehulepavel.steh...@gmail.com  wrote: Hello (1 row) This works the same indeed, just

Re: [HACKERS] ALTER DATABASE and datallowconn

2012-05-03 Thread Magnus Hagander
On Thu, May 3, 2012 at 4:13 PM, Tom Lane t...@sss.pgh.pa.us wrote: Magnus Hagander mag...@hagander.net writes: Is there a particular reason we don't have an ALTER DATABASE switch that controls the datallowconn, or is it just something missed out? It was never intended to be a user-accessible

Re: [HACKERS] ALTER DATABASE and datallowconn

2012-05-03 Thread Tom Lane
Magnus Hagander mag...@hagander.net writes: On Thu, May 3, 2012 at 4:13 PM, Tom Lane t...@sss.pgh.pa.us wrote: Magnus Hagander mag...@hagander.net writes: Is there a particular reason we don't have an ALTER DATABASE switch that controls the datallowconn, or is it just something missed out?

Re: [HACKERS] ALTER DATABASE and datallowconn

2012-05-03 Thread Magnus Hagander
On Thu, May 3, 2012 at 4:26 PM, Tom Lane t...@sss.pgh.pa.us wrote: Magnus Hagander mag...@hagander.net writes: On Thu, May 3, 2012 at 4:13 PM, Tom Lane t...@sss.pgh.pa.us wrote: Magnus Hagander mag...@hagander.net writes: Is there a particular reason we don't have an ALTER DATABASE switch

Re: [HACKERS] How hard would it be to support LIKE in return declaration of generic record function calls ?

2012-05-03 Thread Tom Lane
Andrew Dunstan and...@dunslane.net writes: You guys seem to be taking the original proposal off into the weeds. I have often wanted to be able to use LIKE in type expressions, and I'd like to see exactly that implemented. This notion of anytypename is utterly unworkable anyway; there's no

Re: [HACKERS] How hard would it be to support LIKE in return declaration of generic record function calls ?

2012-05-03 Thread Pavel Stehule
2012/5/3 Tom Lane t...@sss.pgh.pa.us: Andrew Dunstan and...@dunslane.net writes: You guys seem to be taking the original proposal off into the weeds. I have often wanted to be able to use LIKE in type expressions, and I'd like to see exactly that implemented. This notion of anytypename is

Re: [HACKERS] How hard would it be to support LIKE in return declaration of generic record function calls ?

2012-05-03 Thread Andrew Dunstan
On 05/03/2012 10:18 AM, Merlin Moncure wrote: On Thu, May 3, 2012 at 9:01 AM, Andrew Dunstanand...@dunslane.net wrote: On 05/03/2012 09:43 AM, Pavel Stehule wrote: 2012/5/3 Merlin Moncuremmonc...@gmail.com: On Thu, May 3, 2012 at 7:13 AM, Pavel Stehulepavel.steh...@gmail.com wrote:

Re: [HACKERS] unexpected EOF messages

2012-05-03 Thread Magnus Hagander
On Thu, May 3, 2012 at 2:46 PM, Kevin Grittner kevin.gritt...@wicourts.gov wrote: Magnus Hagander  wrote: Also, AFAIK we don't *have* a message type at this point (one of the things said mythical project wanted to look at), so the only thing we could really filter on would be the whole text

Re: [HACKERS] unexpected EOF messages

2012-05-03 Thread Magnus Hagander
On Thu, May 3, 2012 at 4:17 PM, Tom Lane t...@sss.pgh.pa.us wrote: Simon Riggs si...@2ndquadrant.com writes: On Thu, May 3, 2012 at 1:26 PM, Magnus Hagander mag...@hagander.net wrote: I had a request from a customer asking if we could make a switch to specifically disable the unexpected EOF

Re: [HACKERS] How hard would it be to support LIKE in return declaration of generic record function calls ?

2012-05-03 Thread Tom Lane
Pavel Stehule pavel.steh...@gmail.com writes: 2012/5/3 Tom Lane t...@sss.pgh.pa.us: This notion of anytypename is utterly unworkable anyway; there's no way for the parser to know soon enough that a given argument position needs to be read as a type name rather than a normal expression. type

Re: [HACKERS] unexpected EOF messages

2012-05-03 Thread Tom Lane
Magnus Hagander mag...@hagander.net writes: On Thu, May 3, 2012 at 4:17 PM, Tom Lane t...@sss.pgh.pa.us wrote: I agree with Simon --- a disable for that specific message seems like a kluge, and an ugly one at that. (The right solution for this customer is to fix their broken application.)

Re: [HACKERS] How hard would it be to support LIKE in return declaration of generic record function calls ?

2012-05-03 Thread Merlin Moncure
On Thu, May 3, 2012 at 9:44 AM, Andrew Dunstan and...@dunslane.net wrote: Why would you always need FROM? that was coming from Hannu's original example: insert into test2 select * from json_to_record(jrec json) as (like test2); how do you work it so you can call: select json_to_record(jrec

Re: [HACKERS] unexpected EOF messages

2012-05-03 Thread Magnus Hagander
On Thu, May 3, 2012 at 4:53 PM, Tom Lane t...@sss.pgh.pa.us wrote: Magnus Hagander mag...@hagander.net writes: On Thu, May 3, 2012 at 4:17 PM, Tom Lane t...@sss.pgh.pa.us wrote: I agree with Simon --- a disable for that specific message seems like a kluge, and an ugly one at that.  (The right

Re: [HACKERS] remove dead ports?

2012-05-03 Thread Bruce Momjian
On Tue, May 01, 2012 at 04:39:32PM -0400, Bruce Momjian wrote: On Tue, Apr 24, 2012 at 09:29:39PM +0300, Peter Eisentraut wrote: I propose that we remove support for the following OS ports from our source tree. They are totally dead, definitely don't work, and/or probably no one remembers

Re: [HACKERS] How hard would it be to support LIKE in return declaration of generic record function calls ?

2012-05-03 Thread Pavel Stehule
2012/5/3 Tom Lane t...@sss.pgh.pa.us: Pavel Stehule pavel.steh...@gmail.com writes: 2012/5/3 Tom Lane t...@sss.pgh.pa.us: This notion of anytypename is utterly unworkable anyway; there's no way for the parser to know soon enough that a given argument position needs to be read as a type name

Re: [HACKERS] unexpected EOF messages

2012-05-03 Thread Tom Lane
Magnus Hagander mag...@hagander.net writes: On Thu, May 3, 2012 at 4:53 PM, Tom Lane t...@sss.pgh.pa.us wrote: Magnus Hagander mag...@hagander.net writes: Are you thinking basically regexp against the main text, or something else, when you say generic filter capacity? In the context of

Re: [HACKERS] How hard would it be to support LIKE in return declaration of generic record function calls ?

2012-05-03 Thread Tom Lane
Pavel Stehule pavel.steh...@gmail.com writes: 2012/5/3 Tom Lane t...@sss.pgh.pa.us: No, it isn't, at least not if you have any ambition to support array types for instance; to say nothing of types whose standard names are keywords, multiple words, etc. we can identify a position anytypename

Re: [HACKERS] unexpected EOF messages

2012-05-03 Thread Alvaro Herrera
Excerpts from Magnus Hagander's message of jue may 03 10:58:12 -0400 2012: On Thu, May 3, 2012 at 4:53 PM, Tom Lane t...@sss.pgh.pa.us wrote: In the context of yesterday's discussions, I wonder whether a filter by SQLSTATE would be appropriate. I'm worried it's not really granular enough.

Re: [HACKERS] unexpected EOF messages

2012-05-03 Thread Robert Haas
On Thu, May 3, 2012 at 11:20 AM, Alvaro Herrera alvhe...@commandprompt.com wrote: Hey, maybe we could add a UUID to each ereport() call site ;-) I can't help but feel we're designing a $10.00 solution to a $0.25 problem. I think I'd actually support adding something like a UUID to every ereport

Re: [HACKERS] Torn page hazard in ginRedoUpdateMetapage()

2012-05-03 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: Are you planning to commit Noah's patch? I wasn't intending to do so personally in the near future; I've got other things on my to-do list. I won't object if somebody else commits it though. regards, tom lane -- Sent via

Re: [HACKERS] unexpected EOF messages

2012-05-03 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Thu, May 3, 2012 at 11:20 AM, Alvaro Herrera alvhe...@commandprompt.com wrote: Hey, maybe we could add a UUID to each ereport() call site ;-) I can't help but feel we're designing a $10.00 solution to a $0.25 problem. I think I'd actually support

Re: [HACKERS] unexpected EOF messages

2012-05-03 Thread Robert Haas
On Thu, May 3, 2012 at 11:39 AM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Thu, May 3, 2012 at 11:20 AM, Alvaro Herrera alvhe...@commandprompt.com wrote: Hey, maybe we could add a UUID to each ereport() call site ;-) I can't help but feel we're designing

Re: [HACKERS] unexpected EOF messages

2012-05-03 Thread Kevin Grittner
Alvaro Herrera alvhe...@commandprompt.com wrote: Excerpts from Magnus Hagander's message: Tom Lane t...@sss.pgh.pa.us wrote: In the context of yesterday's discussions, I wonder whether a filter by SQLSTATE would be appropriate. I'm worried it's not really granular enough. Yeah. Just to

Re: [HACKERS] Advisory locks seem rather broken

2012-05-03 Thread Tom Lane
Simon Riggs si...@2ndquadrant.com writes: On Thu, May 3, 2012 at 1:19 AM, Tom Lane t...@sss.pgh.pa.us wrote: If this patch weren't already in a released branch I would be arguing for reverting it.  As is, I think we're going to have to clean it up. I don't have time to look at it in detail

Re: [HACKERS] How hard would it be to support LIKE in return declaration of generic record function calls ?

2012-05-03 Thread Merlin Moncure
On Thu, May 3, 2012 at 10:12 AM, Tom Lane t...@sss.pgh.pa.us wrote: Maybe some keyword can help to us. What do you think about new operator TYPE that can returns regtype value and can be used together with polymorphic functions. Doesn't have any more attraction for me than the proposed LIKE

Re: [HACKERS] remove dead ports?

2012-05-03 Thread Peter Eisentraut
On tor, 2012-05-03 at 10:59 -0400, Bruce Momjian wrote: Having received no replies on general from bsdi users considering upgrading to 9.2, I have removed the port. I think that was quite premature. There is no requirement that bsdi users need to read pgsql-general, especially if you give them

Re: [HACKERS] Advisory locks seem rather broken

2012-05-03 Thread Merlin Moncure
On Thu, May 3, 2012 at 11:04 AM, Tom Lane t...@sss.pgh.pa.us wrote: I'm inclined to think that a saner implementation would involve splitting the userlock lockmethod into two, one transactional and one not.  That gets rid of the when-to-release kluges, but instead we have to think of a way for

Re: [HACKERS] Advisory locks seem rather broken

2012-05-03 Thread Simon Riggs
On Thu, May 3, 2012 at 5:04 PM, Tom Lane t...@sss.pgh.pa.us wrote: I'm inclined to think that a saner implementation would involve splitting the userlock lockmethod into two, one transactional and one not. Agreed  That gets rid of the when-to-release kluges, but instead we have to think of

Re: [HACKERS] remove dead ports?

2012-05-03 Thread Bruce Momjian
On Thu, May 03, 2012 at 07:11:47PM +0300, Peter Eisentraut wrote: On tor, 2012-05-03 at 10:59 -0400, Bruce Momjian wrote: Having received no replies on general from bsdi users considering upgrading to 9.2, I have removed the port. I think that was quite premature. There is no requirement

Re: [HACKERS] Advisory locks seem rather broken

2012-05-03 Thread Tom Lane
Simon Riggs si...@2ndquadrant.com writes: On Thu, May 3, 2012 at 5:04 PM, Tom Lane t...@sss.pgh.pa.us wrote: That gets rid of the when-to-release kluges, but instead we have to think of a way for two different lockmethods to share the same lock keyspace. If we don't split it then we

Re: [HACKERS] Advisory locks seem rather broken

2012-05-03 Thread Tom Lane
Merlin Moncure mmonc...@gmail.com writes: On Thu, May 3, 2012 at 11:04 AM, Tom Lane t...@sss.pgh.pa.us wrote: I'm inclined to think that a saner implementation would involve splitting the userlock lockmethod into two, one transactional and one not. hm, would that be exposed through the

[HACKERS] CLOG extension

2012-05-03 Thread Robert Haas
Currently, the following can happen: 1. A backend needs a new transaction, so it calls GetNewTransactionId(). It acquires XidGenLock and then calls ExtendCLOG(). 2. ExtendCLOG() decides that a new CLOG page is needed, so it acquires CLogControlLock and then calls ZeroCLOGPage(). 3.

Re: [HACKERS] remove dead ports?

2012-05-03 Thread Peter Geoghegan
On 3 May 2012 17:21, Bruce Momjian br...@momjian.us wrote: I think I was the only user left;  I have never heard from a BSD/OS user in the past 5-7 years. I'm inclined to agree with Bruce. While it's not reasonable to assume that the lack of a BSD/OS user complaining on -general indicates that

Re: [HACKERS] unexpected EOF messages

2012-05-03 Thread Robert Haas
On Thu, May 3, 2012 at 11:46 AM, Kevin Grittner kevin.gritt...@wicourts.gov wrote: Alvaro Herrera alvhe...@commandprompt.com wrote: Excerpts from Magnus Hagander's message: Tom Lane t...@sss.pgh.pa.us wrote: In the context of yesterday's discussions, I wonder whether a filter by SQLSTATE

Re: [HACKERS] unexpected EOF messages

2012-05-03 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: Well, nearby Tom and I discussed demoting the message to DEBUG1 when no transaction is in progress. Presumably the two messages would share the same SQL state, unless we're going to create separate SQL states for connection-closed-not-in-a-txn and

Re: [HACKERS] outdated comment in heapam.c

2012-05-03 Thread Heikki Linnakangas
On 03.05.2012 16:08, Andres Freund wrote: Hi, It seems that when Heikki added the multi_insert code the following comment in htup.h wasn't updated: /* * We ran out of opcodes, so heapam.c now has a second RmgrId. These opcodes * are associated with RM_HEAP2_ID, but are not logically

Re: [HACKERS] Future In-Core Replication

2012-05-03 Thread Josh Berkus
On 5/2/12 10:58 PM, Jim Nasby wrote: On 4/29/12 6:03 AM, Simon Riggs wrote: The DML-WITH-LIMIT-1 is required to do single logical updates on tables with non-unique rows. And as for any logical updates we will have huge performance problem when doing UPDATE or DELETE on large table with

Re: [HACKERS] CLOG extension

2012-05-03 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: [ CLOG extension is horrid for concurrency ] Yeah. When that code was designed, a page's worth of transactions seemed like a lot so we didn't worry too much about performance glitches when we crossed a page boundary. It's time to do something about it

Re: [HACKERS] Future In-Core Replication

2012-05-03 Thread Josh Berkus
Something that a in-core method might be able to do that an external one can't would be to support a method of uniquely identifying rows in tables with no PK's. A gross example (that undoubtedly wouldn't work in the real world) would be using TID's. A real-world implementation might be based

Re: [HACKERS] unexpected EOF messages

2012-05-03 Thread Magnus Hagander
On Thu, May 3, 2012 at 5:39 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Thu, May 3, 2012 at 11:20 AM, Alvaro Herrera alvhe...@commandprompt.com wrote: Hey, maybe we could add a UUID to each ereport() call site ;-) I can't help but feel we're designing

Re: [HACKERS] unexpected EOF messages

2012-05-03 Thread Tom Lane
Magnus Hagander mag...@hagander.net writes: On Thu, May 3, 2012 at 5:39 PM, Tom Lane t...@sss.pgh.pa.us wrote: I could support that with one tweak: it's only DEBUG1 if you don't have an open transaction.  Dropping the connection while in a transaction *is* an application bug; I don't care how

Re: [HACKERS] extending relations more efficiently

2012-05-03 Thread Stephen Frost
Robert, * Stephen Frost (sfr...@snowman.net) wrote: In all seriousness, this is not a great test case unless you can provide some scripts to make it easy to run it in a reproducible fashion. Can you? Yeah, sure, I'll do that. The PostGIS folks have scripts, but they're kind of ugly,

Re: [HACKERS] Advisory locks seem rather broken

2012-05-03 Thread Josh Berkus
AFAICS you'd either use transactional or session level, but to use both seems bizarre. And if you really did need both, you can put a wrapper around the function to check whether a session level exists before you grant the transaction level lock, or vice versa. You wouldn't want to

Re: [HACKERS] unexpected EOF messages

2012-05-03 Thread Kevin Grittner
Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: Well, nearby Tom and I discussed demoting the message to DEBUG1 when no transaction is in progress. Presumably the two messages would share the same SQL state, unless we're going to create separate SQL states for

Re: [HACKERS] unexpected EOF messages

2012-05-03 Thread Tom Lane
Kevin Grittner kevin.gritt...@wicourts.gov writes: Would it make sense to use 08003 (connection_does_not_exist) when a broken connection for an idle process is discovered, and 08006 (connection_failure) for the in transaction failure? What about a failure just after COMMIT and before

Re: [HACKERS] unexpected EOF messages

2012-05-03 Thread Magnus Hagander
On Thu, May 3, 2012 at 7:09 PM, Tom Lane t...@sss.pgh.pa.us wrote: Magnus Hagander mag...@hagander.net writes: On Thu, May 3, 2012 at 5:39 PM, Tom Lane t...@sss.pgh.pa.us wrote: I could support that with one tweak: it's only DEBUG1 if you don't have an open transaction.  Dropping the

Re: [HACKERS] Future In-Core Replication

2012-05-03 Thread Simon Riggs
On Thu, May 3, 2012 at 6:03 PM, Josh Berkus j...@agliodbs.com wrote: I do agree that depending on user-defined PKs raises a whole host of issues which we'd rather just sidestep, though. What do you have in mind instead? --  Simon Riggs   http://www.2ndQuadrant.com/  

Re: [HACKERS] Advisory locks seem rather broken

2012-05-03 Thread Robert Haas
On Thu, May 3, 2012 at 12:12 PM, Simon Riggs si...@2ndquadrant.com wrote: AFAICS you'd either use transactional or session level, but to use both seems bizarre. I'm a bit confused by all this, because we use both transaction and session level locks internally - on the same lock tags - so I

Re: [HACKERS] CLOG extension

2012-05-03 Thread Simon Riggs
On Thu, May 3, 2012 at 5:59 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: [ CLOG extension is horrid for concurrency ] Yeah.  When that code was designed, a page's worth of transactions seemed like a lot so we didn't worry too much about performance glitches

Re: [HACKERS] Advisory locks seem rather broken

2012-05-03 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Thu, May 3, 2012 at 12:12 PM, Simon Riggs si...@2ndquadrant.com wrote: AFAICS you'd either use transactional or session level, but to use both seems bizarre. I'm a bit confused by all this, because we use both transaction and session level locks

Re: [HACKERS] unexpected EOF messages

2012-05-03 Thread Magnus Hagander
On Thu, May 3, 2012 at 7:21 PM, Magnus Hagander mag...@hagander.net wrote: On Thu, May 3, 2012 at 7:09 PM, Tom Lane t...@sss.pgh.pa.us wrote: Magnus Hagander mag...@hagander.net writes: On Thu, May 3, 2012 at 5:39 PM, Tom Lane t...@sss.pgh.pa.us wrote: I could support that with one tweak: it's

Re: [HACKERS] unexpected EOF messages

2012-05-03 Thread Kevin Grittner
Tom Lane t...@sss.pgh.pa.us wrote: AFAICS, all the 08 class is meant to be issued by client-side code, not the server. I think we probably have to use nonstandard SQLSTATEs for these messages. OK, if we're going that route, how about using Class 2D * Invalid Transaction Termination? I

Re: [HACKERS] unexpected EOF messages

2012-05-03 Thread Tom Lane
Magnus Hagander mag...@hagander.net writes: Heh - we already used ERRCODE_CONNECTION_FAILURE on the errors in copy.c. Since COPY can only happen when there is a transaction (right?), I just changed those error messages for consistency. Agreed on changing the message texts to match, but I

Re: [HACKERS] Advisory locks seem rather broken

2012-05-03 Thread Andres Freund
On Thursday, May 03, 2012 06:12:04 PM Simon Riggs wrote: AFAICS you'd either use transactional or session level, but to use both seems bizarre. And if you really did need both, you can put a wrapper around the function to check whether a session level exists before you grant the transaction

Re: [HACKERS] unexpected EOF messages

2012-05-03 Thread Tom Lane
Kevin Grittner kevin.gritt...@wicourts.gov writes: I still think it might be useful to differentiate in our server log between the case where the transaction failed and the case where the transaction committed but we don't know that the client got the news of that. How about something like:

Re: [HACKERS] CLOG extension

2012-05-03 Thread Robert Haas
On Thu, May 3, 2012 at 1:27 PM, Simon Riggs si...@2ndquadrant.com wrote: Why not switch to 1 WAL record per file, rather than 1 per page. (32 pages, IIRC). We can then have the whole new file written as zeroes by a background process, which needn't do that while holding the XidGenLock. I

Re: [HACKERS] patch submission: truncate trailing nulls from heap rows to reduce the size of the null bitmap

2012-05-03 Thread Josh Berkus
Tom, So that I can test this properly, what is the specific use-case we'd expect to be slow with this patch? -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription:

Re: [HACKERS] Re: xReader, double-effort (was: Temporary tables under hot standby)

2012-05-03 Thread Josh Berkus
One thing I wanted to mention is that non-binary replication has an added advantage over binary from a DR standpoint: if corruption occurs on a master it is more likely to make it into your replicas thanks to full page writes. You might want to consider that depending on how sensitive your

Re: [HACKERS] Temporary tables under hot standby

2012-05-03 Thread Josh Berkus
(on standby) INSERT INTO s1 SELECT1; SELECT ... FROM s1 WHERE ... which seems to me to be actually harder than just rewriting as derived table and isn't an option on Microstrategy etc, hence my observation that GTTs don't help HS much. What I would like to see, one day, is for temp tables

Re: [HACKERS] CLOG extension

2012-05-03 Thread Robert Haas
On Thu, May 3, 2012 at 2:50 PM, Robert Haas robertmh...@gmail.com wrote: Doing it a background process, though, may make sense.  What I'm a little worried about is that - on a busy system - we've only got about 2 seconds to complete each CLOG extension, and we must do an fsync in order to get

Re: [HACKERS] CLOG extension

2012-05-03 Thread Simon Riggs
On Thu, May 3, 2012 at 7:50 PM, Robert Haas robertmh...@gmail.com wrote: On Thu, May 3, 2012 at 1:27 PM, Simon Riggs si...@2ndquadrant.com wrote: Why not switch to 1 WAL record per file, rather than 1 per page. (32 pages, IIRC). We can then have the whole new file written as zeroes by a

Re: [HACKERS] unexpected EOF messages

2012-05-03 Thread Magnus Hagander
On Thu, May 3, 2012 at 7:48 PM, Tom Lane t...@sss.pgh.pa.us wrote: Magnus Hagander mag...@hagander.net writes: Heh - we already used ERRCODE_CONNECTION_FAILURE on the errors in copy.c. Since COPY can only happen when there is a transaction (right?), I just changed those error messages for

Re: [HACKERS] Re: xReader, double-effort (was: Temporary tables under hot standby)

2012-05-03 Thread Kevin Grittner
Josh Berkus j...@agliodbs.com wrote: One thing I wanted to mention is that non-binary replication has an added advantage over binary from a DR standpoint: if corruption occurs on a master it is more likely to make it into your replicas thanks to full page writes. You might want to consider

Re: [HACKERS] Uppercase tab completion keywords in psql?

2012-05-03 Thread Bruce Momjian
Peter, where are we on this? --- On Fri, Mar 30, 2012 at 08:16:59PM +0300, Peter Eisentraut wrote: On fre, 2012-03-23 at 07:52 -0700, David Fetter wrote: On Thu, Mar 22, 2012 at 06:05:30PM -0400, Andrew Dunstan wrote:

Re: [HACKERS] Have we out-grown Flex?

2012-05-03 Thread james
I haven't tried quex, but I have tried lemon (which can be broken out of SQLite) and re2c and ragel. I like ragel and lemon, but the combination supports a push-parser style from memory, and many tools are inconvenient unless you are prepared to suck in a whole message before parsing, or let

Re: [HACKERS] Advisory locks seem rather broken

2012-05-03 Thread Tom Lane
I wrote: Robert Haas robertmh...@gmail.com writes: In fact I'm a bit confused by the original complaint for the same reason - if LockRelationOid and LockRelationIdForSession can coexist, why doesn't the same thing work for advisory locks? The problem (or problems) is bad implementation, not

Re: [HACKERS] Have we out-grown Flex?

2012-05-03 Thread james
Doesn't that imply that a plan cache might be worthwhile? But no matter: didn't the OP really have issue with packaging and Windows support - and there are a lot of Windows users, and in general there are many Windows devs: making it easier for them to contribute has to be good doesn't it?

Re: [HACKERS] Re: xReader, double-effort (was: Temporary tables under hot standby)

2012-05-03 Thread Josh Berkus
That's an interesting point. Out of curiosity, how did the corruption originate? We're still not sure. It appears to be in the system catalogs, though. Note that the original master developed memory issues. It suggests a couple questions: (1) Was Slony running before the corruption

Re: [HACKERS] Have we out-grown Flex?

2012-05-03 Thread Daniel Farina
On Thu, May 3, 2012 at 12:51 PM, james ja...@mansionfamily.plus.com wrote: I haven't tried quex, but I have tried lemon (which can be broken out of SQLite) and re2c and ragel. I like ragel and lemon, but the combination supports a push-parser style from memory, and many tools are inconvenient

Re: [HACKERS] Have we out-grown Flex?

2012-05-03 Thread james
I believe there are tools that are significantly faster than flex. I believe re2c generates code that is faster. But the key thing is to test, probably, or perhaps ask around. I'm out of touch, but from memory flex wasn't the be-all and end-all. Lemon is definitely easy to maintain/port

Re: [HACKERS] CLOG extension

2012-05-03 Thread Robert Haas
On Thu, May 3, 2012 at 3:20 PM, Simon Riggs si...@2ndquadrant.com wrote: Your two paragraphs have roughly opposite arguments... Doing it every 32 pages would give you 30 seconds to complete the fsync, if you kicked it off when half way through the previous file - at current maximum rates. So

Re: [HACKERS] Advisory locks seem rather broken

2012-05-03 Thread Robert Haas
On Thu, May 3, 2012 at 3:52 PM, Tom Lane t...@sss.pgh.pa.us wrote: I'm inclined to say that you can PREPARE if your session holds a given advisory lock at either session or transaction level, but not both. This is a bit annoying but doesn't seem likely to be a real problem in practice, so

Re: [HACKERS] CLOG extension

2012-05-03 Thread Daniel Farina
On Thu, May 3, 2012 at 1:56 PM, Robert Haas robertmh...@gmail.com wrote: Possibly.  I have some fear of ending up with too many background processes, but we may need them. I sort of care about this, but only on systems that are not very busy and could otherwise get by with fewer resources --

Re: [HACKERS] Advisory locks seem rather broken

2012-05-03 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Thu, May 3, 2012 at 3:52 PM, Tom Lane t...@sss.pgh.pa.us wrote: I'm inclined to say that you can PREPARE if your session holds a given advisory lock at either session or transaction level, but not both. This is a bit annoying but doesn't seem likely

  1   2   >