Re: [HACKERS] smart shutdown at end of transaction (was: Default mode for shutdown)

2012-04-29 Thread Peter Eisentraut
On lör, 2012-04-28 at 11:12 -0400, Tom Lane wrote: Peter Eisentraut pete...@gmx.net writes: On fre, 2012-04-27 at 22:30 +0200, Andres Freund wrote: In the few cases where I investigated it TMs don't use transactions themselves (which I think is correct, they don't need them), so

Re: [HACKERS] smart shutdown at end of transaction (was: Default mode for shutdown)

2012-04-29 Thread Peter Eisentraut
On fre, 2012-04-27 at 14:57 -0400, Robert Haas wrote: I think there is no point at all in having a discussion about this unless we can first agree that the overwhelming majority of people who have commented on this issue on this list are unhappy with the current default behavior. If we are

Re: [HACKERS] smart shutdown at end of transaction (was: Default mode for shutdown)

2012-04-29 Thread Simon Riggs
On Sun, Apr 29, 2012 at 12:41 AM, Robert Haas robertmh...@gmail.com wrote: On Sat, Apr 28, 2012 at 7:04 AM, Simon Riggs si...@2ndquadrant.com wrote: So lets implement the new shutdown mode and work out a transition path to a new default. Changing rapidly screws up the people we love the most.

Re: [HACKERS] Future In-Core Replication

2012-04-29 Thread Simon Riggs
On Sat, Apr 28, 2012 at 8:40 PM, Hannu Krosing ha...@krosing.net wrote: As to what LCRs should contain, it will probably be locical equivalents of INSERT, UPDATE ... LIMIT 1, DELETE ... LIMIT 1, TRUNCATE and all DDL. Yeh I would even go as far as propose a variant for DML-WITH-LIMIT-1 to be

Re: [HACKERS] Re: patch submission: truncate trailing nulls from heap rows to reduce the size of the null bitmap

2012-04-29 Thread Simon Riggs
On Sun, Apr 29, 2012 at 12:24 AM, Robert Haas robertmh...@gmail.com wrote: Let's not go looking for reasons to reject the approach just because we didn't expect it to work as well as it does. Who here, in your opinion, is looking for reasons to reject anything? --  Simon Riggs   

Re: [HACKERS] Re: patch submission: truncate trailing nulls from heap rows to reduce the size of the null bitmap

2012-04-29 Thread Gokulakannan Somasundaram
There might be a patch available for this already. In the worst case articulated above (less than 64 columns), if all the nulls are trailing nulls, the bitmap need not be saved. Actually it is not 64(actually 72), as postgres heaptupleheader is only 23 bytes and one byte is left for the start of

Re: [HACKERS] Patch: add conversion from pg_wchar to multibyte

2012-04-29 Thread Erik Rijkers
Hi Alexander, Perhaps I'm too early with these tests, but FWIW I reran my earlier test program against three instances. (the patches compiled fine, and make check was without problem). -- 3 instances: HEAD port 6542 trgm_regex port 6547 HEAD + trgm-regexp patch (22

Re: [HACKERS] default_transaction_isolation = serializable causes crash under Hot Standby

2012-04-29 Thread Simon Riggs
On Sat, Apr 28, 2012 at 5:56 AM, Kevin Grittner kevin.gritt...@wicourts.gov wrote: But if you set it in the postgresql.conf file, it's not pretty: kevin@kevin-desktop:~$ psql -p 5433 test psql: FATAL:  can not create a serializable snapshot during recovery Ideas? The patch as submitted

Re: [HACKERS] 9.2 release notes, beta time?

2012-04-29 Thread Simon Riggs
On Sat, Apr 28, 2012 at 12:15 AM, Devrim GÜNDÜZ dev...@gunduz.org wrote: (As a side note, RPMs *may not* be ready, because I (and Magnus) will be at PGDay Turkey on 12th, and will be busy over the whole weekend). Is that a closed meeting? I hadn't seen any mention of that anywhere. --  Simon

Re: [HACKERS] default_transaction_isolation = serializable causes crash under Hot Standby

2012-04-29 Thread Kevin Grittner
Simon Riggs wrote: Kevin Grittner wrote: But if you set it in the postgresql.conf file, it's not pretty: kevin@kevin-desktop:~$ psql -p 5433 test psql: FATAL: can not create a serializable snapshot during recovery Ideas? The patch as submitted doesn't do anything useful for the case

Re: [HACKERS] Re: xReader, double-effort (was: Temporary tables under hot standby)

2012-04-29 Thread Kevin Grittner
Tom Lane wrote: Kevin Grittner writes: Well, the xReader approach (Aakash's GSoC project) is to serve as a proxy for a WAL stream going to a hot standby, to interpret each incoming WAL record by cheating and querying the HS before passing the WAL along, and then using sync rep to the HS

Re: [HACKERS] smart shutdown at end of transaction (was: Default mode for shutdown)

2012-04-29 Thread Tom Lane
Peter Eisentraut pete...@gmx.net writes: In any case, if either the existing session of the TM is cut or it cannot create a new connection, it will, after some time, have to give up roll back the prepared transactions on the other servers. So some kind of setting to not shut down if there are

Re: [HACKERS] smart shutdown at end of transaction (was: Default mode for shutdown)

2012-04-29 Thread Tom Lane
Peter Eisentraut pete...@gmx.net writes: On fre, 2012-04-27 at 14:57 -0400, Robert Haas wrote: I think there is no point at all in having a discussion about this unless we can first agree that the overwhelming majority of people who have commented on this issue on this list are unhappy with

Re: [HACKERS] smart shutdown at end of transaction (was: Default mode for shutdown)

2012-04-29 Thread Simon Riggs
On Sun, Apr 29, 2012 at 4:06 PM, Tom Lane t...@sss.pgh.pa.us wrote: Peter Eisentraut pete...@gmx.net writes: In any case, if either the existing session of the TM is cut or it cannot create a new connection, it will, after some time, have to give up roll back the prepared transactions on the

Re: [HACKERS] 9.2 release notes, beta time?

2012-04-29 Thread Magnus Hagander
On Sun, Apr 29, 2012 at 14:23, Simon Riggs si...@2ndquadrant.com wrote: On Sat, Apr 28, 2012 at 12:15 AM, Devrim GÜNDÜZ dev...@gunduz.org wrote: (As a side note, RPMs *may not* be ready, because I (and Magnus) will be at PGDay Turkey on 12th, and will be busy over the whole weekend). Is that

Re: [HACKERS] smart shutdown at end of transaction (was: Default mode for shutdown)

2012-04-29 Thread Tom Lane
Simon Riggs si...@2ndquadrant.com writes: I think we only need one new mode, shutdown when transactions are finished should only shutdown when all types of transaction are complete. For people that don't use prepared transactions the difference is irrelevant. For people that do use prepared

Re: [HACKERS] default_transaction_isolation = serializable causes crash under Hot Standby

2012-04-29 Thread Simon Riggs
On Sun, Apr 29, 2012 at 1:40 PM, Kevin Grittner kevin.gritt...@wicourts.gov wrote: IMHO the desired behaviour would be * prevent default_transaction_isolation = serializable as a default setting when we enter Hot Standby by throwing a FATAL error from the startup process. I can help

Re: [HACKERS] default_transaction_isolation = serializable causes crash under Hot Standby

2012-04-29 Thread Kevin Grittner
Simon Riggs wrote: The only way default_transaction_isolation = serializable would be acceptable when hot_standby = on is if we silently downgrade the isolation level to read committed. That way everything just works, albeit not quite as requested. So I think that's the best way forwards.

Re: [HACKERS] Re: xReader, double-effort (was: Temporary tables under hot standby)

2012-04-29 Thread Simon Riggs
On Sun, Apr 29, 2012 at 3:27 PM, Kevin Grittner kevin.gritt...@wicourts.gov wrote: It would be enormously more performant for the master to be emitting logical replication records to start with, since it already has all the right names etc at hand at basically no cost. Not when the consumers

Re: [HACKERS] Future In-Core Replication

2012-04-29 Thread Hannu Krosing
On Sun, 2012-04-29 at 12:03 +0100, Simon Riggs wrote: On Sat, Apr 28, 2012 at 8:40 PM, Hannu Krosing ha...@krosing.net wrote: As to what LCRs should contain, it will probably be locical equivalents of INSERT, UPDATE ... LIMIT 1, DELETE ... LIMIT 1, TRUNCATE and all DDL. Yeh I would

Re: [HACKERS] smart shutdown at end of transaction (was: Default mode for shutdown)

2012-04-29 Thread Simon Riggs
On Sun, Apr 29, 2012 at 5:41 PM, Tom Lane t...@sss.pgh.pa.us wrote: Simon Riggs si...@2ndquadrant.com writes: I think we only need one new mode, shutdown when transactions are finished should only shutdown when all types of transaction are complete. For people that don't use prepared

Re: [HACKERS] Future In-Core Replication

2012-04-29 Thread Simon Riggs
On Sun, Apr 29, 2012 at 6:20 PM, Hannu Krosing ha...@2ndquadrant.com wrote: On Sun, 2012-04-29 at 12:03 +0100, Simon Riggs wrote: On Sat, Apr 28, 2012 at 8:40 PM, Hannu Krosing ha...@krosing.net wrote: As to what LCRs should contain, it will probably be locical equivalents of INSERT, UPDATE

Re: [HACKERS] default_transaction_isolation = serializable causes crash under Hot Standby

2012-04-29 Thread Simon Riggs
On Sun, Apr 29, 2012 at 5:54 PM, Kevin Grittner kevin.gritt...@wicourts.gov wrote: Simon Riggs  wrote: The only way default_transaction_isolation = serializable would be acceptable when hot_standby = on is if we silently downgrade the isolation level to read committed. That way everything

Re: [HACKERS] smart shutdown at end of transaction (was: Default mode for shutdown)

2012-04-29 Thread Peter Eisentraut
On sön, 2012-04-29 at 10:19 +0100, Simon Riggs wrote: Maybe we don't need to do this over multiple releases, but we do need to give warning of possible incompatibilities. It would be good to see a specific post on hackers called Planned Incompatibilities in 9.2, or collect such things on the

[HACKERS] precision and scale functions for numeric

2012-04-29 Thread Peter Eisentraut
I didn't find a good way to find out how many digits a numeric value has or things like whether a numeric value is an integer. (I had to go through bc(1) for the latter.) Functions like precision() and scale() would have been quite handy. Are there other ways to do this, or would this make a

Re: [HACKERS] 9.2 release notes, beta time?

2012-04-29 Thread Jay Levitt
Magnus Hagander wrote: 2012/4/28 Josh Berkusj...@agliodbs.com: Ugh. Maybe the whole idea of getting a beta out before PGCon is doomed. Still, if we don't try for this schedule, we're looking at at least two more weeks' slip, because we're surely not going to wrap during PGCon. We could do it

Re: [HACKERS] Request to add options to tools/git_changelog

2012-04-29 Thread Jay Levitt
Bruce Momjian wrote: I am again requesting the addition of options to tools/git_changelog so I can more easily produce the release notes. I asked for this during 9.1 development and it was rejected. I am currently using my own custom version of the tool, but have to merge community

Re: [HACKERS] Re: xReader, double-effort (was: Temporary tables under hot standby)

2012-04-29 Thread Kevin Grittner
Simon Riggs wrote: Though it isn't a given that logical change records (LCRs) will require more bandwidth than physical WAL. It would help if you could share a little more of what you have in mind. It's not yet clear to me whether you're talking about adding logical records to the WAL

Re: [HACKERS] default_transaction_isolation = serializable causes crash under Hot Standby

2012-04-29 Thread Kevin Grittner
Simon Riggs wrote: On Sun, Apr 29, 2012 at 5:54 PM, Kevin Grittner wrote: Simon Riggs wrote: The only way default_transaction_isolation = serializable would be acceptable when hot_standby = on is if we silently downgrade the isolation level to read committed. That way everything just

Re: [HACKERS] default_transaction_isolation = serializable causes crash under Hot Standby

2012-04-29 Thread Robert Haas
On Sun, Apr 29, 2012 at 8:20 AM, Simon Riggs si...@2ndquadrant.com wrote: * prevent default_transaction_isolation = serializable as a default setting when we enter Hot Standby by throwing a FATAL error from the startup process. I can help implement that if we agree. I am strongly disinclined

Re: [HACKERS] Temporary tables under hot standby

2012-04-29 Thread Noah Misch
On Tue, Apr 24, 2012 at 11:55:15PM -0400, Noah Misch wrote: Concerning everyone's favorite topic, how to name the new type of table, I liked Tom's proposal[1] to make CREATE TEMP TABLE retain current behavior and have CREATE GLOBAL TEMP TABLE and/or CREATE LOCAL TEMP TABLE request the new

Re: [HACKERS] Re: xReader, double-effort (was: Temporary tables under hot standby)

2012-04-29 Thread Simon Riggs
On Sun, Apr 29, 2012 at 7:36 PM, Kevin Grittner kevin.gritt...@wicourts.gov wrote: Simon Riggs  wrote: Though it isn't a given that logical change records (LCRs) will require more bandwidth than physical WAL. It would help if you could share a little more of what you have in mind.  It's not

Re: [HACKERS] Re: xReader, double-effort (was: Temporary tables under hot standby)

2012-04-29 Thread Robert Haas
On Sat, Apr 28, 2012 at 11:06 AM, Tom Lane t...@sss.pgh.pa.us wrote: Simon Riggs si...@2ndquadrant.com writes: Translating WAL is a very hard task. No kidding.  I would think it's impossible on its face.  Just for starters, where will you get table and column names from?  (Looking at the

Re: [HACKERS] smart shutdown at end of transaction (was: Default mode for shutdown)

2012-04-29 Thread Robert Haas
On Sun, Apr 29, 2012 at 1:48 PM, Peter Eisentraut pete...@gmx.net wrote: On sön, 2012-04-29 at 10:19 +0100, Simon Riggs wrote: Maybe we don't need to do this over multiple releases, but we do need to give warning of possible incompatibilities. It would be good to see a specific post on hackers

Re: [HACKERS] Re: patch submission: truncate trailing nulls from heap rows to reduce the size of the null bitmap

2012-04-29 Thread Robert Haas
On Sun, Apr 29, 2012 at 7:19 AM, Simon Riggs si...@2ndquadrant.com wrote: On Sun, Apr 29, 2012 at 12:24 AM, Robert Haas robertmh...@gmail.com wrote: Let's not go looking for reasons to reject the approach just because we didn't expect it to work as well as it does. Who here, in your opinion,

Re: [HACKERS] smart shutdown at end of transaction (was: Default mode for shutdown)

2012-04-29 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: Erred on the side of progress might even be a little strong, because I think for the most part we have been extremely judicious about backward incompatibilities in the last few releases (which is a good thing). Obviously, 8.3 was a flag day of the

Re: [HACKERS] Re: xReader, double-effort (was: Temporary tables under hot standby)

2012-04-29 Thread Hannu Krosing
On Sun, 2012-04-29 at 16:33 -0400, Robert Haas wrote: On Sat, Apr 28, 2012 at 11:06 AM, Tom Lane t...@sss.pgh.pa.us wrote: Simon Riggs si...@2ndquadrant.com writes: Translating WAL is a very hard task. No kidding. I would think it's impossible on its face. Just for starters, where

Re: [HACKERS] Future In-Core Replication

2012-04-29 Thread Alvaro Herrera
Excerpts from Simon Riggs's message of jue abr 26 11:10:09 -0300 2012: On Thu, Apr 26, 2012 at 1:41 PM, Simon Riggs si...@2ndquadrant.com wrote: I will also be organising a small-medium sized Future of In-Core Replication meeting in Ottawa on Wed 16 May, 6-10pm. Thanks for such rapid

Re: [HACKERS] Patch: add timing of buffer I/O requests

2012-04-29 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: I like the idea of including the word block in there. I don't think it was probably a terribly good idea to abbreviate that to blk everywhere, but at this point it's probably better to be consistent, sigh. As for track_iotiming - track_io_timing,

Re: [HACKERS] 9.2 release notes, beta time?

2012-04-29 Thread Devrim GÜNDÜZ
On Sun, 2012-04-29 at 13:23 +0100, Simon Riggs wrote: (As a side note, RPMs *may not* be ready, because I (and Magnus) will be at PGDay Turkey on 12th, and will be busy over the whole weekend). Is that a closed meeting? I hadn't seen any mention of that anywhere. Not that much. I've been

Re: [HACKERS] [PATCH] Allow breaking out of hung connection attempts

2012-04-29 Thread Alvaro Herrera
Excerpts from Ryan Kelly's message of sáb ene 14 16:22:21 -0300 2012: I have attached a new patch which handles the connect_timeout option by adding a PQconnectTimeout(conn) function to access the connect_timeout which I then use to retrieve the existing value from the old connection. Was

Re: [HACKERS] smart shutdown at end of transaction (was: Default mode for shutdown)

2012-04-29 Thread Robert Haas
On Sun, Apr 29, 2012 at 5:27 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: Erred on the side of progress might even be a little strong, because I think for the most part we have been extremely judicious about backward incompatibilities in the last few

Re: [HACKERS] Re: xReader, double-effort (was: Temporary tables under hot standby)

2012-04-29 Thread Robert Haas
On Sun, Apr 29, 2012 at 6:00 PM, Hannu Krosing ha...@2ndquadrant.com wrote: I think the question we should be asking ourselves is not whether WAL as it currently exists is adequate for logical replication, but rather or not it could be made adequate. Agreed. And of course I meant but rather

Re: [HACKERS] Re: xReader, double-effort (was: Temporary tables under hot standby)

2012-04-29 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Sun, Apr 29, 2012 at 6:00 PM, Hannu Krosing ha...@2ndquadrant.com wrote: So you basically need a large part of postgres for reliably making sense of WAL. Agreed, but I think that's a problem we need to fix and not a tolerable situation at all. If

Re: [HACKERS] Re: xReader, double-effort (was: Temporary tables under hot standby)

2012-04-29 Thread Robert Haas
On Sun, Apr 29, 2012 at 11:29 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Sun, Apr 29, 2012 at 6:00 PM, Hannu Krosing ha...@2ndquadrant.com wrote: So you basically need a large part of postgres for reliably making sense of WAL. Agreed, but I think