Re: [HACKERS] crash-safe visibility map, take four

2011-03-23 Thread Jesper Krogh
On 2011-03-22 21:43, Robert Haas wrote: I took a crack at implementing the first approach described above, which seems to be by far the simplest idea we've come up with to date. Patch attached. It doesn't seem to be that complicated, which could mean either that it's not that complicated or

Re: [HACKERS] crash-safe visibility map, take four

2011-03-23 Thread Gokulakannan Somasundaram
All operations that clear the bit area are already WAL-logged. Is it the case with visibility map also? Thanks.

Re: [HACKERS] Re: [COMMITTERS] pgsql: Efficient transaction-controlled synchronous replication.

2011-03-23 Thread Markus Wanner
On 03/22/2011 09:33 PM, Robert Haas wrote: We might have a version of synchronous replication that works this way some day, but it's not the version were shipping with 9.1. The slave acknowledges the WAL records when they hit the disk (i.e. fsync) not when they are applied; WAL apply can lag

Re: [HACKERS] Re: [COMMITTERS] pgsql: Basic Recovery Control functions for use in Hot Standby. Pause,

2011-03-23 Thread Fujii Masao
On Fri, Mar 18, 2011 at 9:31 PM, Robert Haas robertmh...@gmail.com wrote: On Fri, Mar 18, 2011 at 8:27 AM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: You could also argue for log a warning, continue until we can open for Hot standby, then pause. I don't like that one much.

Re: [HACKERS] Sync Rep v19

2011-03-23 Thread Fujii Masao
On Sat, Mar 19, 2011 at 11:28 AM, Robert Haas robertmh...@gmail.com wrote: On Fri, Mar 18, 2011 at 10:25 PM, Robert Haas robertmh...@gmail.com wrote: On Tue, Mar 8, 2011 at 7:05 AM, Fujii Masao masao.fu...@gmail.com wrote: * Smart shutdown Smart shutdown should wait for all the waiting

Re: [HACKERS] Planner regression in 9.1: min(x) cannot use partial index with NOT NULL

2011-03-23 Thread Marti Raudsepp
On Tue, Mar 22, 2011 at 01:02, Tom Lane t...@sss.pgh.pa.us wrote: I studied the code some more, and I think this probably can be made to work.  The basic idea is to have preprocess_minmax_aggregates build simplified queries like the above (working by modifying the query tree that exists at the

Re: [HACKERS] Replication server timeout patch

2011-03-23 Thread Heikki Linnakangas
On 16.03.2011 11:11, Fujii Masao wrote: On Wed, Mar 16, 2011 at 4:49 PM, Fujii Masaomasao.fu...@gmail.com wrote: Agreed. I'll change the patch. Done. I attached the updated patch. I don't much like the API for this. Walsender shouldn't need to know about the details of the FE/BE protocol,

Re: [HACKERS] writing a script to examine dead tuples

2011-03-23 Thread Heikki Linnakangas
On 23.03.2011 01:36, aaronenabs wrote: Can anyone help me, i am trying to carry out an investigation which involves accessing dead tuples within the postgresql. I have been advised i could write a script that allows me to examine dead tuples and am seeking advise on how to achieve this, or

Re: [HACKERS] Re: [COMMITTERS] pgsql: Basic Recovery Control functions for use in Hot Standby. Pause,

2011-03-23 Thread Robert Haas
On Wed, Mar 23, 2011 at 4:51 AM, Fujii Masao masao.fu...@gmail.com wrote: On Fri, Mar 18, 2011 at 9:31 PM, Robert Haas robertmh...@gmail.com wrote: On Fri, Mar 18, 2011 at 8:27 AM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: You could also argue for log a warning, continue

Re: [HACKERS] Re: [COMMITTERS] pgsql: Efficient transaction-controlled synchronous replication.

2011-03-23 Thread Robert Haas
On Wed, Mar 23, 2011 at 3:27 AM, Markus Wanner mar...@bluegap.ch wrote: On 03/22/2011 09:33 PM, Robert Haas wrote: We might have a version of synchronous replication that works this way some day, but it's not the version were shipping with 9.1.  The slave acknowledges the WAL records when they

Re: [HACKERS] pg_ctl restart - behaviour based on wrong instance

2011-03-23 Thread Robert Haas
On Wed, Mar 23, 2011 at 1:48 AM, Fujii Masao masao.fu...@gmail.com wrote: On Sat, Mar 19, 2011 at 10:20 AM, Robert Haas robertmh...@gmail.com wrote: On Fri, Mar 18, 2011 at 1:19 PM, Erik Rijkers e...@xs4all.nl wrote: This is OK and expected.  But then it continues (in the logfile) with:

Re: [HACKERS] crash-safe visibility map, take four

2011-03-23 Thread Robert Haas
On Wed, Mar 23, 2011 at 2:29 AM, Gokulakannan Somasundaram gokul...@gmail.com wrote: All operations that clear the bit area are already WAL-logged. Is it the case with visibility map also? Thanks. Yes. Look at the comment that the patch removes. That describes the problem being fixed. --

Re: [HACKERS] crash-safe visibility map, take four

2011-03-23 Thread Robert Haas
On Wed, Mar 23, 2011 at 2:16 AM, Jesper Krogh jes...@krogh.cc wrote: On 2011-03-22 21:43, Robert Haas wrote: I took a crack at implementing the first approach described above, which seems to be by far the simplest idea we've come up with to date.  Patch attached.  It doesn't seem to be that

Re: [HACKERS] Re: [COMMITTERS] pgsql: Efficient transaction-controlled synchronous replication.

2011-03-23 Thread Markus Wanner
On 03/23/2011 12:52 PM, Robert Haas wrote: Yes. What this won't do is let you build a big load-balancing network (at least not without great caution about what you assume). This sounds too strong to me. Session-aware load balancing is pretty common these days. It's the default mode of

[HACKERS] copy / paste object - preliminary part

2011-03-23 Thread Vladimir Kokovic
Hi, I did a preliminary part of the functions for copy / paste object and now I expect confirmation that the implementation is correct. Changed files: frm/frmMain.cpp, nclude/frm/frmMain.h New files: frm/frmPasteObject.cpp, include/frm/frmPasteObject.h Best regards, Vladimir Kokovic, DP

Re: [HACKERS] Re: [COMMITTERS] pgsql: Basic Recovery Control functions for use in Hot Standby. Pause,

2011-03-23 Thread Simon Riggs
On Wed, Mar 23, 2011 at 11:50 AM, Robert Haas robertmh...@gmail.com wrote: On Wed, Mar 23, 2011 at 4:51 AM, Fujii Masao masao.fu...@gmail.com wrote: On Fri, Mar 18, 2011 at 9:31 PM, Robert Haas robertmh...@gmail.com wrote: On Fri, Mar 18, 2011 at 8:27 AM, Heikki Linnakangas

[HACKERS] corner case about replication and shutdown

2011-03-23 Thread Fujii Masao
Hi, When I read the shutdown code to create the smart shutdown patch for sync rep, I found the corner case where shutdown can get stuck infinitely. This happens when postmaster reaches PM_WAIT_BACKENDS state before walsender marks itself as WAL sender process for streaming WAL (i.e., before

Re: [HACKERS] crash-safe visibility map, take four

2011-03-23 Thread Merlin Moncure
On Wed, Mar 23, 2011 at 1:16 AM, Jesper Krogh jes...@krogh.cc wrote: On 2011-03-22 21:43, Robert Haas wrote: I took a crack at implementing the first approach described above, which seems to be by far the simplest idea we've come up with to date.  Patch attached.  It doesn't seem to be that

Re: [HACKERS] Re: [COMMITTERS] pgsql: Basic Recovery Control functions for use in Hot Standby. Pause,

2011-03-23 Thread Robert Haas
On Wed, Mar 23, 2011 at 9:38 AM, Simon Riggs si...@2ndquadrant.com wrote: Actually, my previous email was all nonsense, wasn't it?  If we don't reach the consistency point, we can't enter normal running anyway - shut down is the only option no matter what. Presumably you mean that the way its

Re: [HACKERS] Re: [COMMITTERS] pgsql: Efficient transaction-controlled synchronous replication.

2011-03-23 Thread Robert Haas
On Wed, Mar 23, 2011 at 8:16 AM, Markus Wanner mar...@bluegap.ch wrote: On 03/23/2011 12:52 PM, Robert Haas wrote: Yes.  What this won't do is let you build a big load-balancing network (at least not without great caution about what you assume). This sounds too strong to me.  Session-aware

[HACKERS] making write location work (was: Efficient transaction-controlled synchronous replication)

2011-03-23 Thread Robert Haas
On Fri, Mar 18, 2011 at 8:16 AM, Robert Haas robertmh...@gmail.com wrote: On Fri, Mar 18, 2011 at 3:52 AM, Simon Riggs si...@2ndquadrant.com wrote: I agree to get rid of write_location. No, don't remove it. We seem to be just looking for things to tweak without any purpose. Removing this

Re: [HACKERS] 2nd Level Buffer Cache

2011-03-23 Thread Radosław Smogura
Merlin Moncure mmonc...@gmail.com Monday 21 March 2011 20:58:16 On Mon, Mar 21, 2011 at 2:08 PM, Greg Stark gsst...@mit.edu wrote: On Mon, Mar 21, 2011 at 3:54 PM, Merlin Moncure mmonc...@gmail.com wrote: Can't you make just one large mapping and lock it in 8k regions? I thought the

Re: [HACKERS] 2nd Level Buffer Cache

2011-03-23 Thread Radosław Smogura
Merlin Moncure mmonc...@gmail.com Tuesday 22 March 2011 23:06:02 On Tue, Mar 22, 2011 at 4:28 PM, Radosław Smogura rsmog...@softperience.eu wrote: Merlin Moncure mmonc...@gmail.com Monday 21 March 2011 20:58:16 On Mon, Mar 21, 2011 at 2:08 PM, Greg Stark gsst...@mit.edu wrote: On

Re: [HACKERS] [COMMITTERS] pgsql: Efficient transaction-controlled synchronous replication.

2011-03-23 Thread Robert Haas
On Fri, Mar 18, 2011 at 10:10 AM, Robert Haas robertmh...@gmail.com wrote: On Mon, Mar 7, 2011 at 3:44 AM, Fujii Masao masao.fu...@gmail.com wrote: On Mon, Mar 7, 2011 at 5:27 PM, Fujii Masao masao.fu...@gmail.com wrote: On Mon, Mar 7, 2011 at 7:51 AM, Simon Riggs si...@2ndquadrant.com wrote:

Re: [HACKERS] Comments on SQL/Med objects

2011-03-23 Thread Robert Haas
On Tue, Mar 22, 2011 at 6:23 PM, Guillaume Lelarge guilla...@lelarge.info wrote: While working on adding support for SQL/Med objects to pgAdmin, I'm quite surprised to see there is no way to add comments to SQL/Med objects. Is this on purpose or is it just something that was simply missed? I

[HACKERS] Re: making write location work (was: Efficient transaction-controlled synchronous replication)

2011-03-23 Thread Simon Riggs
On Wed, Mar 23, 2011 at 3:35 PM, Robert Haas robertmh...@gmail.com wrote: On Fri, Mar 18, 2011 at 8:16 AM, Robert Haas robertmh...@gmail.com wrote: On Fri, Mar 18, 2011 at 3:52 AM, Simon Riggs si...@2ndquadrant.com wrote: I agree to get rid of write_location. No, don't remove it. We seem to

Re: [HACKERS] making write location work (was: Efficient transaction-controlled synchronous replication)

2011-03-23 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: Specifically, if we're not going to remove write location, then I think we need to apply something like the attached. while (walrcv_receive(0, type, buf, len)) XLogWalRcvProcessMsg(type, buf, len);

Re: [HACKERS] Comments on SQL/Med objects

2011-03-23 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Tue, Mar 22, 2011 at 6:23 PM, Guillaume Lelarge guilla...@lelarge.info wrote: While working on adding support for SQL/Med objects to pgAdmin, I'm quite surprised to see there is no way to add comments to SQL/Med objects. Is this on purpose or is it

Re: [HACKERS] Comments on SQL/Med objects

2011-03-23 Thread Guillaume Lelarge
Le 23/03/2011 17:53, Tom Lane a écrit : Robert Haas robertmh...@gmail.com writes: On Tue, Mar 22, 2011 at 6:23 PM, Guillaume Lelarge guilla...@lelarge.info wrote: While working on adding support for SQL/Med objects to pgAdmin, I'm quite surprised to see there is no way to add comments to

Re: [HACKERS] Re: making write location work (was: Efficient transaction-controlled synchronous replication)

2011-03-23 Thread Robert Haas
On Wed, Mar 23, 2011 at 12:10 PM, Simon Riggs si...@2ndquadrant.com wrote: Specifically, if we're not going to remove write location, then I think we need to apply something like the attached. The protocol supports different write/fsync values, so the view should display them. That's exactly

Re: [HACKERS] making write location work (was: Efficient transaction-controlled synchronous replication)

2011-03-23 Thread Robert Haas
On Wed, Mar 23, 2011 at 12:44 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: Specifically, if we're not going to remove write location, then I think we need to apply something like the attached.                       while (walrcv_receive(0, type, buf, len))

Re: [HACKERS] 2nd Level Buffer Cache

2011-03-23 Thread Jim Nasby
On Mar 22, 2011, at 2:53 PM, Robert Haas wrote: On Tue, Mar 22, 2011 at 11:24 AM, Jeff Janes jeff.ja...@gmail.com wrote: On Fri, Mar 18, 2011 at 9:19 AM, Robert Haas robertmh...@gmail.com wrote: On Fri, Mar 18, 2011 at 11:14 AM, Kevin Grittner kevin.gritt...@wicourts.gov wrote: Maybe the

Re: [HACKERS] psql \dt and table size

2011-03-23 Thread Susanne Ebrecht
Hello Bernd, On 21.03.2011 18:44, Bernd Helmle wrote: Attached minor patch extends \dt to use pg_table_size() starting with PostgreSQL 9.0, not sure if we backport such changes though. It would be interesting for 9.1, however. As I already told you: I tested and it worked. The code looks

Re: [HACKERS] Re: making write location work (was: Efficient transaction-controlled synchronous replication)

2011-03-23 Thread Robert Haas
On Wed, Mar 23, 2011 at 2:43 PM, Simon Riggs si...@2ndquadrant.com wrote: On Wed, Mar 23, 2011 at 6:22 PM, Robert Haas robertmh...@gmail.com wrote: On Wed, Mar 23, 2011 at 12:10 PM, Simon Riggs si...@2ndquadrant.com wrote: Specifically, if we're not going to remove write location, then I think

Re: [HACKERS] Re: making write location work (was: Efficient transaction-controlled synchronous replication)

2011-03-23 Thread Simon Riggs
On Wed, Mar 23, 2011 at 7:29 PM, Robert Haas robertmh...@gmail.com wrote: On Wed, Mar 23, 2011 at 2:43 PM, Simon Riggs si...@2ndquadrant.com wrote: On Wed, Mar 23, 2011 at 6:22 PM, Robert Haas robertmh...@gmail.com wrote: On Wed, Mar 23, 2011 at 12:10 PM, Simon Riggs si...@2ndquadrant.com

Re: [HACKERS] 2nd Level Buffer Cache

2011-03-23 Thread Robert Haas
On Wed, Mar 23, 2011 at 1:53 PM, Jim Nasby j...@nasby.net wrote: When we started using 192G servers we tried switching our largest OLTP database (would have been about 1.2TB at the time) from 8GB shared buffers to 28GB. Performance went down enough to notice; I don't have any solid metrics,

Re: [HACKERS] Re: making write location work (was: Efficient transaction-controlled synchronous replication)

2011-03-23 Thread Robert Haas
On Wed, Mar 23, 2011 at 3:33 PM, Simon Riggs si...@2ndquadrant.com wrote: In any case, that's not the only argument for keeping it. We introduce the view in this release and I would like it to stay the same from now, since we know we will need that info later. At least as I understand it, it's

Re: [HACKERS] psql \dt and table size

2011-03-23 Thread Robert Haas
On Mon, Mar 21, 2011 at 1:44 PM, Bernd Helmle maili...@oopsware.de wrote: It stroke me today again, that \dt+ isn't displaying the acurate table size for tables, since it uses pg_relation_size() till now. With having pg_table_size() since PostgreSQL 9.0 available, i believe it would be more

Re: [HACKERS] 2nd Level Buffer Cache

2011-03-23 Thread Greg Stark
On Wed, Mar 23, 2011 at 8:00 PM, Robert Haas robertmh...@gmail.com wrote: It looks like the only way anything can ever get put on the free list right now is if a relation or database is dropped.  That doesn't seem too good.  I wonder if the background writer shouldn't be trying to maintain the

Re: [HACKERS] Re: making write location work (was: Efficient transaction-controlled synchronous replication)

2011-03-23 Thread Simon Riggs
On Wed, Mar 23, 2011 at 8:20 PM, Robert Haas robertmh...@gmail.com wrote: On Wed, Mar 23, 2011 at 3:33 PM, Simon Riggs si...@2ndquadrant.com wrote: In any case, that's not the only argument for keeping it. We introduce the view in this release and I would like it to stay the same from now,

Re: [HACKERS] psql \dt and table size

2011-03-23 Thread Alvaro Herrera
Excerpts from Robert Haas's message of mié mar 23 17:24:59 -0300 2011: On Mon, Mar 21, 2011 at 1:44 PM, Bernd Helmle maili...@oopsware.de wrote: It stroke me today again, that \dt+ isn't displaying the acurate table size for tables, since it uses pg_relation_size() till now. With having

Re: [HACKERS] 2nd Level Buffer Cache

2011-03-23 Thread Radosław Smogura
Greg Stark gsst...@mit.edu Wednesday 23 March 2011 21:30:04 On Wed, Mar 23, 2011 at 8:00 PM, Robert Haas robertmh...@gmail.com wrote: It looks like the only way anything can ever get put on the free list right now is if a relation or database is dropped. That doesn't seem too good. I

Re: [HACKERS] psql \dt and table size

2011-03-23 Thread Pavel Stehule
2011/3/23 Alvaro Herrera alvhe...@commandprompt.com: Excerpts from Robert Haas's message of mié mar 23 17:24:59 -0300 2011: On Mon, Mar 21, 2011 at 1:44 PM, Bernd Helmle maili...@oopsware.de wrote: It stroke me today again, that \dt+ isn't displaying the acurate table size for tables, since

[HACKERS] PG Session #2 : Call For Papers

2011-03-23 Thread damien clochard
Hi, The first PostgreSQL Session organized by Dalibo last february was a big success. More than 80 public and private participants gathered in Paris. Dalibo and Oslandia want to carry on this success and set up a new conference day dedicated to PostGIS, on June 23d in Paris.

[HACKERS] Lack of post creation hook on extension

2011-03-23 Thread Kohei KaiGai
I found a problem that extension.c does not invoke post-creation hook on its creation time, although no module supports to assign security label on extension objects right now. The attached patch tries to fix it. Thanks, -- KaiGai Kohei kai...@kaigai.gr.jp pgsql-extension-hook.patch

Re: [HACKERS] 2nd Level Buffer Cache

2011-03-23 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: It looks like the only way anything can ever get put on the free list right now is if a relation or database is dropped. That doesn't seem too good. Why not? AIUI the free list is only for buffers that are totally dead, ie contain no info that's

[HACKERS] missing history for pg_exec_query_string()

2011-03-23 Thread Jaime Casanova
Hi, I'm looking for the history of pg_exec_query_string() and found that it dissapear in 7.4, so i tried to look at git log to find out if it was renamed or removed completely and found only this 3 commits:

Re: [HACKERS] missing history for pg_exec_query_string()

2011-03-23 Thread Cédric Villemain
2011/3/23 Jaime Casanova ja...@2ndquadrant.com: Hi, I'm looking for the history of pg_exec_query_string() and found that it dissapear in 7.4, so i tried to look at git log to find out if it was renamed or removed completely and found only this 3 commits:

[HACKERS] Query to generate CREATE INDEX statement from

2011-03-23 Thread Andrew Hammond
I'm building some partitioning support functions. I'm working on writing one called clone_indexes_to_partition right now. The idea is to take all the indexes applied to the parent and create a matching index on the child. Is there existing code that generates a CREATE INDEX statement given an

Re: [HACKERS] missing history for pg_exec_query_string()

2011-03-23 Thread Jaime Casanova
On Wed, Mar 23, 2011 at 6:50 PM, Cédric Villemain cedric.villemain.deb...@gmail.com wrote: This : git log --pretty=oneline -S'pg_exec_query_string' origin/REL7_4_STABLE git show  de28dc9a04c4df5d711815b7a518501b43535a26 give me the answer : ah! ok, so the problema was the way i was asking

Re: [HACKERS] Query to generate CREATE INDEX statement from

2011-03-23 Thread Cédric Villemain
2011/3/24 Andrew Hammond andrew.george.hamm...@gmail.com: I'm building some partitioning support functions. I'm working on writing one called clone_indexes_to_partition right now. The idea is to take all the indexes applied to the parent and create a matching index on the child. Is there

Re: [HACKERS] Query to generate CREATE INDEX statement from

2011-03-23 Thread Tom Lane
Andrew Hammond andrew.george.hamm...@gmail.com writes: I'm building some partitioning support functions. I'm working on writing one called clone_indexes_to_partition right now. The idea is to take all the indexes applied to the parent and create a matching index on the child. Is there existing

Re: [HACKERS] crash-safe visibility map, take four

2011-03-23 Thread Gokulakannan Somasundaram
Yeah. i looked at it. I don't think it addresses the problem raised here. http://archives.postgresql.org/pgsql-hackers/2010-02/msg02097.php http://archives.postgresql.org/pgsql-hackers/2010-02/msg02097.phpOr may be i am missing something. Thanks. On Wed, Mar 23, 2011 at 7:54 PM, Robert Haas