Re: [HACKERS] Condition to become the standby mode.

2013-07-26 Thread Andres Freund
Hi, On 2013-07-26 13:19:34 +0900, Sawada Masahiko wrote: When the slave server starts, the slave server perform the following steps in StartupXLOG(): 1. Read latest CheckPoint record LSN from pg_control file. 2. Try to fetch CheckPoint record from pg_xlog directory at first. ( The server

Re: [HACKERS] inconsistent state after crash recovery

2013-07-26 Thread Andres Freund
Hi, On 2013-07-26 13:33:13 +0900, Satoshi Nagayasu wrote: I received a question about inconsistent state after crash recovery. When a table file is broken (or just lost), PostgreSQL can not recover a whole table, and does not show any notice while recoverying. I think it means inconsistent

Re: [HACKERS] comment for fast promote

2013-07-26 Thread Fujii Masao
On Fri, Jul 26, 2013 at 11:19 AM, Tomonari Katsumata katsumata.tomon...@po.ntts.co.jp wrote: Hi Fujii-san, Thank you for response. (2013/07/25 21:15), Fujii Masao wrote: On Thu, Jul 25, 2013 at 5:33 PM, Tomonari Katsumata katsumata.tomon...@po.ntts.co.jp wrote: Hi, Now I'm seeing xlog.c

Re: [HACKERS] Patch to add support of IF NOT EXISTS to others CREATE statements

2013-07-26 Thread Karol Trzcionka
W dniu 26.07.2013 02:44, Fabrízio de Royes Mello pisze: Should be... I fix that in attached patch. Hello, as I can see there are more inconsistent places. First style: OperatorCreate --- Second style: ProcedureCreate TypeCreate DefineTSParser DefineType DefineEnum --- Third style: CreateCast

Re: [HACKERS] dynamic background workers, round two

2013-07-26 Thread Andres Freund
On 2013-07-25 12:35:30 -0400, Robert Haas wrote: On Wed, Jul 24, 2013 at 5:34 PM, Andres Freund and...@2ndquadrant.com wrote: This seems like a sensible idea to me. But, in the context of dynamic query, don't we also need the reverse infrastructure of notifying a bgworker that the client,

Re: [HACKERS] Design proposal: fsync absorb linear slider

2013-07-26 Thread Greg Smith
On 7/25/13 6:02 PM, didier wrote: It was surely already discussed but why isn't postresql writing sequentially its cache in a temporary file? If you do that, reads of the data will have to traverse that temporary file to assemble their data. You'll make every later reader pay the random

Re: [HACKERS] Design proposal: fsync absorb linear slider

2013-07-26 Thread Hannu Krosing
On 07/26/2013 11:42 AM, Greg Smith wrote: On 7/25/13 6:02 PM, didier wrote: It was surely already discussed but why isn't postresql writing sequentially its cache in a temporary file? If you do that, reads of the data will have to traverse that temporary file to assemble their data. You'll

Re: [HACKERS] Design proposal: fsync absorb linear slider

2013-07-26 Thread Hannu Krosing
On 07/26/2013 11:42 AM, Greg Smith wrote: On 7/25/13 6:02 PM, didier wrote: It was surely already discussed but why isn't postresql writing sequentially its cache in a temporary file? If you do that, reads of the data will have to traverse that temporary file to assemble their data. In

Re: [HACKERS] install libpq.dll in bin directory on Windows / Cygwin

2013-07-26 Thread marco atzeri
Il 7/25/2013 11:02 PM, Alvaro Herrera ha scritto: Andrew Dunstan wrote: Jeff Janes asked me about this, and Bruce just tripped up on it. Usually on Windows it's necessary to have libpq.dll/cygpq.dll either in the PATH or in the same directory as client .exe files. The buildfarm client has for

Re: [HACKERS] Design proposal: fsync absorb linear slider

2013-07-26 Thread Greg Smith
On 7/26/13 5:59 AM, Hannu Krosing wrote: Well, SSD disks do it in the way proposed by didier (AFAIK), by putting random fs pages on one large disk page and having an extra index layer for resolving random-to-sequential ordering. If your solution to avoiding random writes now is to do

Re: ALTER SYSTEM SET command to change postgresql.conf parameters (RE: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL [review])

2013-07-26 Thread Amit Kapila
On Friday, July 26, 2013 10:35 AM Alvaro Herrera wrote: Josh Berkus escribió: On 07/25/2013 02:02 PM, Tom Lane wrote: Robert Haas robertmh...@gmail.com writes: My thought is that people might put postgresql.conf in a directory that only contains configuration files and isn't

Re: [HACKERS] inconsistent state after crash recovery

2013-07-26 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: On 2013-07-26 13:33:13 +0900, Satoshi Nagayasu wrote: Is this expected or acceptable? I'd say it's both. Postgres is built on the assumption that the underlying filesystem is reliable, ie, once you've successfully fsync'd some data that data won't

Re: [HACKERS] Design proposal: fsync absorb linear slider

2013-07-26 Thread Tom Lane
Greg Smith g...@2ndquadrant.com writes: On 7/26/13 5:59 AM, Hannu Krosing wrote: Well, SSD disks do it in the way proposed by didier (AFAIK), by putting random fs pages on one large disk page and having an extra index layer for resolving random-to-sequential ordering. If your solution to

Re: ALTER SYSTEM SET command to change postgresql.conf parameters (RE: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL [review])

2013-07-26 Thread Tom Lane
Alvaro Herrera alvhe...@2ndquadrant.com writes: The main contention point I see is where conf.d lives; the two options are in $PGDATA or together with postgresql.conf. Tom and Robert, above, say it should be in $PGDATA; but this goes against Debian packaging and the Linux FHS (or whatever

Re: [HACKERS] getting rid of SnapshotNow

2013-07-26 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: OK. I've taken care of all remaining uses of SnapshotNow in the code base. I think we can go ahead and remove it, now. Patch attached. (And there was, hopefully, much rejoicing.) What about SnapshotSelf? regards, tom lane

Re: [HACKERS] dynamic background workers, round two

2013-07-26 Thread Robert Haas
On Fri, Jul 26, 2013 at 5:34 AM, Andres Freund and...@2ndquadrant.com wrote: It doesn't need to be the postmaster, but I think we need to provide central infrastructure for that. I don't want this to end up being redone poorly in multiple places. I just wanted to mention it, it obviously

Re: [HACKERS] getting rid of SnapshotNow

2013-07-26 Thread Andres Freund
On 2013-07-26 08:49:38 -0400, Tom Lane wrote: Robert Haas robertmh...@gmail.com writes: OK. I've taken care of all remaining uses of SnapshotNow in the code base. I think we can go ahead and remove it, now. Patch attached. (And there was, hopefully, much rejoicing.) What about

Re: [HACKERS] getting rid of SnapshotNow

2013-07-26 Thread Andres Freund
On 2013-07-25 19:24:53 -0400, Robert Haas wrote: (And there was, hopefully, much rejoicing.) Definitely! Thanks. Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers

Re: [HACKERS] getting rid of SnapshotNow

2013-07-26 Thread Robert Haas
On Fri, Jul 26, 2013 at 8:49 AM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: OK. I've taken care of all remaining uses of SnapshotNow in the code base. I think we can go ahead and remove it, now. Patch attached. (And there was, hopefully, much rejoicing.)

Re: [HACKERS] Extensions makefiles - coverage

2013-07-26 Thread Robert Haas
On Thu, Jul 25, 2013 at 11:07 AM, Ronan Dunklau rdunk...@gmail.com wrote: Hello. I was having trouble figuring how to use the coverage targets when using an extension. I am using approximatively the layout that was proposed here:

Re: [HACKERS] Design proposal: fsync absorb linear slider

2013-07-26 Thread didier
Hi, On Fri, Jul 26, 2013 at 11:42 AM, Greg Smith g...@2ndquadrant.com wrote: On 7/25/13 6:02 PM, didier wrote: It was surely already discussed but why isn't postresql writing sequentially its cache in a temporary file? If you do that, reads of the data will have to traverse that

Re: [HACKERS] getting rid of SnapshotNow

2013-07-26 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Fri, Jul 26, 2013 at 8:49 AM, Tom Lane t...@sss.pgh.pa.us wrote: What about SnapshotSelf? Well, that's still used in _bt_check_unique, unique_key_recheck (trigger function to do a deferred uniqueness check), RI_FKey_check, and rather extensively

Re: [HACKERS] Extensions makefiles - coverage

2013-07-26 Thread Ronan Dunklau
Thank you for the tip, its done. 2013/7/26 Robert Haas robertmh...@gmail.com: On Thu, Jul 25, 2013 at 11:07 AM, Ronan Dunklau rdunk...@gmail.com wrote: Hello. I was having trouble figuring how to use the coverage targets when using an extension. I am using approximatively the layout that

Re: [HACKERS] Design proposal: fsync absorb linear slider

2013-07-26 Thread Greg Smith
On 7/26/13 9:14 AM, didier wrote: During recovery you have to load the log in cache first before applying WAL. Checkpoints exist to bound recovery time after a crash. That is their only purpose. What you're suggesting moves a lot of work into the recovery path, which will slow down how

Re: [HACKERS] getting rid of SnapshotNow

2013-07-26 Thread Robert Haas
On Fri, Jul 26, 2013 at 9:18 AM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Fri, Jul 26, 2013 at 8:49 AM, Tom Lane t...@sss.pgh.pa.us wrote: What about SnapshotSelf? Well, that's still used in _bt_check_unique, unique_key_recheck (trigger function to do a

Re: [HACKERS] Design proposal: fsync absorb linear slider

2013-07-26 Thread Greg Smith
On 7/26/13 8:32 AM, Tom Lane wrote: What I'd point out is that that is exactly what WAL does for us, ie convert a bunch of random writes into sequential writes. But sooner or later you have to put the data where it belongs. Hannu was observing that SSD often doesn't do that at all. They can

Re: [HACKERS] getting rid of SnapshotNow

2013-07-26 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Fri, Jul 26, 2013 at 9:18 AM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: Well, that's still used in _bt_check_unique, unique_key_recheck (trigger function to do a deferred uniqueness check), RI_FKey_check, and

Re: [HACKERS] getting rid of SnapshotNow

2013-07-26 Thread Andres Freund
On 2013-07-26 09:50:32 -0400, Robert Haas wrote: sepgsql is using SnapshotSelf to find the old version of a tuple that was updated by the core code just before. That should be safe in the sense that there can't be a currently-committing transaction somewhere else that's updated that tuple, if

Re: [HACKERS] Condition to become the standby mode.

2013-07-26 Thread Fujii Masao
On Fri, Jul 26, 2013 at 3:08 PM, Andres Freund and...@2ndquadrant.com wrote: Hi, On 2013-07-26 13:19:34 +0900, Sawada Masahiko wrote: When the slave server starts, the slave server perform the following steps in StartupXLOG(): 1. Read latest CheckPoint record LSN from pg_control file. 2.

Re: [HACKERS] Condition to become the standby mode.

2013-07-26 Thread Andres Freund
On 2013-07-26 23:47:59 +0900, Fujii Masao wrote: If this problem is solved, there is possible of that we can failback by removing the all WAL record which is in pg_xlog before server starts as the slave server. ( And we also use synchronous_transfer which I'm proposing, I think we can

Re: [HACKERS] Design proposal: fsync absorb linear slider

2013-07-26 Thread didier
Hi, On Fri, Jul 26, 2013 at 3:41 PM, Greg Smith greg@2ndquadg...@2ndquadrant.com(needrant.comg...@2ndquadrant.com wrote: On 7/26/13 9:14 AM, didier wrote: During recovery you have to load the log in cache first before applying WAL. Checkpoints exist to bound recovery time after a crash.

Re: [HACKERS] Condition to become the standby mode.

2013-07-26 Thread Fujii Masao
On Fri, Jul 26, 2013 at 11:55 PM, Andres Freund and...@2ndquadrant.com wrote: On 2013-07-26 23:47:59 +0900, Fujii Masao wrote: If this problem is solved, there is possible of that we can failback by removing the all WAL record which is in pg_xlog before server starts as the slave server.

Re: [HACKERS] pg_upgrade -j broken on Windows

2013-07-26 Thread Bruce Momjian
On Thu, Jul 25, 2013 at 10:57:28AM -0400, Bruce Momjian wrote: Everyone should be aware that the 9.3 pg_upgrade -j/--jobs option on Windows is currently broken, and hopefully will be fixed by the next beta. Someone at PGDay UK told me they were getting pg_upgrade -j crashes on Windows.

Re: [HACKERS] pg_upgrade -j broken on Windows

2013-07-26 Thread Bruce Momjian
On Fri, Jul 26, 2013 at 01:27:34PM -0400, Bruce Momjian wrote: On Thu, Jul 25, 2013 at 10:57:28AM -0400, Bruce Momjian wrote: Everyone should be aware that the 9.3 pg_upgrade -j/--jobs option on Windows is currently broken, and hopefully will be fixed by the next beta. Someone at PGDay

Re: [HACKERS] getting rid of SnapshotNow

2013-07-26 Thread Robert Haas
On Fri, Jul 26, 2013 at 10:16 AM, Andres Freund and...@2ndquadrant.com wrote: I am more concerned about the care needed when placing CommandCounterIncrement()'s somewhere though. It seems more than likely that this will get repeatedly broken, even if it's not atm (which I doubt). E.g.

Re: ALTER SYSTEM SET command to change postgresql.conf parameters (RE: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL [review])

2013-07-26 Thread Stephen Frost
* Tom Lane (t...@sss.pgh.pa.us) wrote: Come to think of it, maybe part of the reason we're having such a hard time getting to consensus is that people are conflating the snippet part with the writable part? I mean, if you are thinking you want system-management tools to be able to drop in

Re: [HACKERS] [GENERAL] postgres FDW cost estimation options unrecognized in 9.3-beta1

2013-07-26 Thread Tom Lane
Lonni J Friedman netll...@gmail.com writes: nightly=# ALTER SERVER cuda_db10 OPTIONS (SET use_remote_estimate 'true') ; ERROR: option use_remote_estimate not found Am I doing something wrong, or is this a bug? [ experiments... ] You need to say ADD, not SET, to add a new option to the list.

Re: [HACKERS] [GENERAL] postgres FDW cost estimation options unrecognized in 9.3-beta1

2013-07-26 Thread Lonni J Friedman
On Fri, Jul 26, 2013 at 3:28 PM, Tom Lane t...@sss.pgh.pa.us wrote: Lonni J Friedman netll...@gmail.com writes: nightly=# ALTER SERVER cuda_db10 OPTIONS (SET use_remote_estimate 'true') ; ERROR: option use_remote_estimate not found Am I doing something wrong, or is this a bug? [

[HACKERS] Re: Patch to add support of IF NOT EXISTS to others CREATE statements

2013-07-26 Thread Abhijit Menon-Sen
At 2013-07-26 10:39:00 +0200, karl...@gmail.com wrote: Hello, as I can see there are more inconsistent places. Right. This is what I was referring to in my original review. All of the relevant sites (pre-patch) that currently do: if (already exists) ereport(ERROR …) should instead

Re: [HACKERS] [GENERAL] currval and DISCARD ALL

2013-07-26 Thread Fabrízio de Royes Mello
On 25-07-2013 05:32, suresh.balasubra wrote: Disclaimer: I am no hacker, just a PostGreSQL user, trying to provide a user scenario where DISCARD SEQUENCES functionality is required. We have designed a developed a small Application Development platform for which the backend is PostGreSQL. There