Re: [HACKERS] postgresql.conf archive_command example

2011-09-08 Thread Fujii Masao
On Wed, Sep 7, 2011 at 11:53 PM, Robert Treat r...@xzilla.net wrote: On Tue, Sep 6, 2011 at 10:11 PM, Fujii Masao masao.fu...@gmail.com wrote: I agree that basically archive_command should not overwrite an existing file. But if the size of existing file is less than 16MB, it should do that.

Re: [HACKERS] Large C files

2011-09-08 Thread Simon Riggs
On Wed, Sep 7, 2011 at 9:02 PM, Tom Lane t...@sss.pgh.pa.us wrote: Simon Riggs si...@2ndquadrant.com writes: Please lets not waste effort on refactoring efforts in mid dev cycle. Say what?  When else would you have us do it? When else would you have us develop? Major changes happen at start

Re: [HACKERS] postgresql.conf archive_command example

2011-09-08 Thread Simon Riggs
On Thu, Sep 8, 2011 at 7:05 AM, Fujii Masao masao.fu...@gmail.com wrote: On Wed, Sep 7, 2011 at 11:53 PM, Robert Treat r...@xzilla.net wrote: On Tue, Sep 6, 2011 at 10:11 PM, Fujii Masao masao.fu...@gmail.com wrote: I agree that basically archive_command should not overwrite an existing file.

Re: [HACKERS] FATAL: lock AccessShareLock on object 0/1260/0 is already held

2011-09-08 Thread daveg
On Wed, Sep 07, 2011 at 09:02:04PM -0400, Tom Lane wrote: daveg da...@sonic.net writes: On Wed, Sep 07, 2011 at 07:39:15PM -0400, Tom Lane wrote: BTW ... what were the last versions you were running on which you had *not* seen the problem? (Just wondering about the possibility that we

Re: [HACKERS] postgresql.conf archive_command example

2011-09-08 Thread Fujii Masao
On Thu, Sep 8, 2011 at 3:26 PM, Simon Riggs si...@2ndquadrant.com wrote: On Thu, Sep 8, 2011 at 7:05 AM, Fujii Masao masao.fu...@gmail.com wrote: That's an option. But I don't think that finding an existing file is so serious problem. The recommendation should be that the archived files are

[HACKERS] pg_last_xact_insert_timestamp

2011-09-08 Thread Fujii Masao
On Thu, Sep 8, 2011 at 7:06 AM, Chris Redekop ch...@replicon.com wrote: Is there anything available to get the last time a transaction occurred?like say pg_last_xact_timestamp?  In order to accurately calculate how far behind my slave is I need to do something like

Re: [HACKERS] pg_last_xact_insert_timestamp

2011-09-08 Thread Simon Riggs
On Thu, Sep 8, 2011 at 9:36 AM, Fujii Masao masao.fu...@gmail.com wrote: On Thu, Sep 8, 2011 at 7:06 AM, Chris Redekop ch...@replicon.com wrote: Is there anything available to get the last time a transaction occurred?like say pg_last_xact_timestamp?  In order to accurately calculate how

Re: [HACKERS] pg_last_xact_insert_timestamp

2011-09-08 Thread Fujii Masao
On Thu, Sep 8, 2011 at 5:55 PM, Simon Riggs si...@2ndquadrant.com wrote: On Thu, Sep 8, 2011 at 9:36 AM, Fujii Masao masao.fu...@gmail.com wrote: The above has been posted in pgsql-general. The reason why Chris thinks a counterpart of pg_last_xact_replay_timestamp() is required sounds

Re: [HACKERS] Back branch update releases this week; beta postponed

2011-09-08 Thread Simon Riggs
On Wed, Sep 7, 2011 at 2:43 AM, Bruce Momjian br...@momjian.us wrote: Tom Lane wrote: Bernd Helmle maili...@oopsware.de writes: --On 10. April 2011 13:53:52 -0400 Tom Lane t...@sss.pgh.pa.us wrote: The core team has therefore decided to wrap back-branch update releases this Thursday for

[HACKERS] EXPLAIN and nfiltered, take two

2011-09-08 Thread Marko Tiikkaja
Hi, Here's a patch for $SUBJECT based on the feedback here: http://archives.postgresql.org/message-id/9053.1295888...@sss.pgh.pa.us I intentionally decided to omit the information for Join Filter, since the information can already be deduced from EXPLAIN ANALYZE output, and for Left Joins and

Re: [HACKERS] [PATCH] Don't truncate integer part in to_char for 'FM99.'

2011-09-08 Thread Marti Raudsepp
On Wed, Sep 7, 2011 at 23:48, Tom Lane t...@sss.pgh.pa.us wrote:  Also, the way yours is set up, I'm dubious that it does the right thing when the last '0' specifier is to the left of the decimal point. When the last '0' is left of the decimal point, Num-zero_end is set to 0, so the branch

Re: [HACKERS] pg_last_xact_insert_timestamp

2011-09-08 Thread Robert Haas
On Thu, Sep 8, 2011 at 6:14 AM, Fujii Masao masao.fu...@gmail.com wrote: OTOH, new function enables users to monitor the delay as a timestamp. For users, a timestamp is obviously easier to handle than LSN, and the delay as a timestamp is more intuitive. So, I think that it's worth adding

Re: [HACKERS] postgresql.conf archive_command example

2011-09-08 Thread Aidan Van Dyk
On Thu, Sep 8, 2011 at 2:05 AM, Fujii Masao masao.fu...@gmail.com wrote: That's an option. But I don't think that finding an existing file is so serious problem. The most common cases which cause a partially-filled archived file are; 1. The server crashes while WAL file is being archived,

[HACKERS] concurrent snapshots

2011-09-08 Thread Ants Aasma
Hi, I have been thinking about how to handle long running transactions with Robert’s commit sequence number (CSN) idea. http://archives.postgresql.org/message-id/CA%2BTgmoaAjiq%3Dd%3DkYt3qNj%2BUvi%2BMB-aRovCwr75Ca9egx-Ks9Ag%40mail.gmail.com I just started to go through transaction management

Re: [HACKERS] postgresql.conf archive_command example

2011-09-08 Thread Kevin Grittner
Aidan Van Dyk ai...@highrise.ca wrote: If you're copying a file into the archive, and making it appear non-atomically in your archive, your doing something wrong. Period. No excuses. +1 -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to

Re: [HACKERS] cheaper snapshots redux

2011-09-08 Thread Robert Haas
On Tue, Sep 6, 2011 at 11:06 PM, Amit Kapila amit.kap...@huawei.com wrote: 1. With the above, you want to reduce/remove the concurrency issue between the GetSnapshotData() [used at begining of sql command execution] and ProcArrayEndTransaction() [used at end transaction]. The concurrency issue

Re: [HACKERS] Large C files

2011-09-08 Thread Tom Lane
Simon Riggs si...@2ndquadrant.com writes: On Wed, Sep 7, 2011 at 9:02 PM, Tom Lane t...@sss.pgh.pa.us wrote: Simon Riggs si...@2ndquadrant.com writes: Please lets not waste effort on refactoring efforts in mid dev cycle. Say what?  When else would you have us do it? When else would you have

Re: [HACKERS] concurrent snapshots

2011-09-08 Thread Robert Haas
On Thu, Sep 8, 2011 at 9:26 AM, Ants Aasma ants.aa...@eesti.ee wrote: When go try to find the new csnmin and discover that a backend has a csnmin that is too old, we go through the snapshots of that backend and convert every snapshot under the desired csnmin to a traditional snapshot. I

Re: [HACKERS] Large C files

2011-09-08 Thread Bruce Momjian
Tom Lane wrote: Simon Riggs si...@2ndquadrant.com writes: On Wed, Sep 7, 2011 at 9:02 PM, Tom Lane t...@sss.pgh.pa.us wrote: Simon Riggs si...@2ndquadrant.com writes: Please lets not waste effort on refactoring efforts in mid dev cycle. Say what? ?When else would you have us do it?

Re: [HACKERS] Large C files

2011-09-08 Thread Robert Haas
On Thu, Sep 8, 2011 at 10:29 AM, Bruce Momjian br...@momjian.us wrote: Tom Lane wrote: Simon Riggs si...@2ndquadrant.com writes: On Wed, Sep 7, 2011 at 9:02 PM, Tom Lane t...@sss.pgh.pa.us wrote: Simon Riggs si...@2ndquadrant.com writes: Please lets not waste effort on refactoring efforts

Re: [HACKERS] WIP: Fast GiST index build

2011-09-08 Thread Heikki Linnakangas
On 06.09.2011 01:18, Alexander Korotkov wrote: Small bugfix: in gistBufferingFindCorrectParent check that downlinkoffnum doesn't exceed maximal offset number. I've committed the patch now, including that fix. Thanks for a great GSoC project! -- Heikki Linnakangas EnterpriseDB

Re: [HACKERS] WIP: Fast GiST index build

2011-09-08 Thread Robert Haas
On Thu, Sep 8, 2011 at 10:59 AM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: On 06.09.2011 01:18, Alexander Korotkov wrote: Small bugfix: in gistBufferingFindCorrectParent check that downlinkoffnum doesn't exceed maximal offset number. I've committed the patch now,

Re: [HACKERS] FATAL: lock AccessShareLock on object 0/1260/0 is already held

2011-09-08 Thread Tom Lane
daveg da...@sonic.net writes: On Wed, Sep 07, 2011 at 09:02:04PM -0400, Tom Lane wrote: daveg da...@sonic.net writes: The first version we saw it on was 8.4.7. Yeah, you said that. I was wondering what you'd last run before 8.4.7. Sorry, misunderstood. We were previously running 8.4.4, but

Re: [HACKERS] concurrent snapshots

2011-09-08 Thread Ants Aasma
On Thu, Sep 8, 2011 at 5:28 PM, Robert Haas robertmh...@gmail.com wrote: On Thu, Sep 8, 2011 at 9:26 AM, Ants Aasma ants.aa...@eesti.ee wrote: When go try to find the new csnmin and discover that a backend has a csnmin that is too old, we go through the snapshots of that backend and convert

Re: [HACKERS] concurrent snapshots

2011-09-08 Thread Tom Lane
Ants Aasma ants.aa...@eesti.ee writes: On Thu, Sep 8, 2011 at 5:28 PM, Robert Haas robertmh...@gmail.com wrote: 1. A backend can have lots of snapshots, potentially requiring an unbounded amount of shared memory.  We can't accommodate that. If PostgreSQL gets POSIX shared memory capability at

Re: [HACKERS] concurrent snapshots

2011-09-08 Thread Robert Haas
On Thu, Sep 8, 2011 at 11:33 AM, Ants Aasma ants.aa...@eesti.ee wrote: On Thu, Sep 8, 2011 at 5:28 PM, Robert Haas robertmh...@gmail.com wrote: On Thu, Sep 8, 2011 at 9:26 AM, Ants Aasma ants.aa...@eesti.ee wrote: When go try to find the new csnmin and discover that a backend has a csnmin that

Re: [HACKERS] force_not_null option support for file_fdw

2011-09-08 Thread Kohei Kaigai
Hi Hanada-san. ISTM that your results are reasonable for each collation setting. Former ordering is same as C locale, and in latter case alphabetical order has priority over case distinctions. Do you mean that ordering used in file_fdw is affected from something unexpected, without

[HACKERS] Fast GiST index build - further improvements

2011-09-08 Thread Heikki Linnakangas
Now that the main GiST index build patch has been committed, there's a few further improvements that could make it much faster still: Better management of the buffer pages on disk. At the moment, the temporary file is used as a heap of pages belonging to all the buffers in random order. I

Re: [HACKERS] Large C files

2011-09-08 Thread Simon Riggs
On Thu, Sep 8, 2011 at 3:25 PM, Tom Lane t...@sss.pgh.pa.us wrote: Simon Riggs si...@2ndquadrant.com writes: On Wed, Sep 7, 2011 at 9:02 PM, Tom Lane t...@sss.pgh.pa.us wrote: Simon Riggs si...@2ndquadrant.com writes: Please lets not waste effort on refactoring efforts in mid dev cycle. Say

Re: [HACKERS] Large C files

2011-09-08 Thread Bruce Momjian
Simon Riggs wrote: On Thu, Sep 8, 2011 at 3:25 PM, Tom Lane t...@sss.pgh.pa.us wrote: Simon Riggs si...@2ndquadrant.com writes: On Wed, Sep 7, 2011 at 9:02 PM, Tom Lane t...@sss.pgh.pa.us wrote: Simon Riggs si...@2ndquadrant.com writes: Please lets not waste effort on refactoring efforts

Re: [HACKERS] WIP: Fast GiST index build

2011-09-08 Thread Oleg Bartunov
My congratulations too, Alexander ! Hope to work on SP-GiST together ! Oleg On Thu, 8 Sep 2011, Heikki Linnakangas wrote: On 06.09.2011 01:18, Alexander Korotkov wrote: Small bugfix: in gistBufferingFindCorrectParent check that downlinkoffnum doesn't exceed maximal offset number. I've

[HACKERS] Protecting against multiple instances per cluster

2011-09-08 Thread Thom Brown
Hi all, I've come across a PostgreSQL set up where there are 2 servers, each with the same version of PostgreSQL on, both mounting the same SAN onto their respective file systems. It was intended that only 1 of the servers would be running an instance of PostgreSQL at a time as they both point

Re: [HACKERS] Protecting against multiple instances per cluster

2011-09-08 Thread Tom Lane
Thom Brown t...@linux.com writes: I've come across a PostgreSQL set up where there are 2 servers, each with the same version of PostgreSQL on, both mounting the same SAN onto their respective file systems. It was intended that only 1 of the servers would be running an instance of PostgreSQL

Re: [HACKERS] Large C files

2011-09-08 Thread Tom Lane
Simon Riggs si...@2ndquadrant.com writes: You clearly have the bit between your teeth on this. Personally, I'm neither intending to break up xlog.c right now, nor asking you to do it. I'm just objecting to your claim that there should be some project-policy restriction on when refactoring gets

[HACKERS] pg_dump.c

2011-09-08 Thread Andrew Dunstan
In the refactoring Large C files discussion one of the biggest files Bruce mentioned is pg_dump.c. There has been discussion in the past of turning lots of the knowledge currently embedded in this file into a library, which would make it available to other clients (e.g. psql). I'm not sure

Re: [HACKERS] WIP: Fast GiST index build

2011-09-08 Thread Alexander Korotkov
Thanks for congratulations! Tnanks to Heikki for mentoring and his work on patch! -- With best regards, Alexander Korotkov.

[HACKERS] Re: [COMMITTERS] pgsql: Add missing format argument to ecpg_log() call

2011-09-08 Thread Peter Eisentraut
On tor, 2011-09-08 at 15:32 -0400, Tom Lane wrote: Peter Eisentraut pete...@gmx.net writes: Add missing format argument to ecpg_log() call Oh, fun. So why isn't there an __attribute__((format...)) decoration on ecpg_log()? I have a larger patch for that. I just wanted to get the one

Re: [HACKERS] memory-related bugs

2011-09-08 Thread Daniel Farina
On Tue, Sep 6, 2011 at 12:00 PM, Tom Lane t...@sss.pgh.pa.us wrote: [ Sorry for letting this slip through the cracks ... I think I got  distracted by collation bugs :-( ] Noah Misch n...@leadboat.com writes: On Sat, Mar 12, 2011 at 12:44:29PM -0500, Tom Lane wrote: Noah Misch

Re: [HACKERS] Large C files

2011-09-08 Thread Peter Geoghegan
On 8 September 2011 15:43, Robert Haas robertmh...@gmail.com wrote: I wouldn't be too enthusiastic about starting a project like this in January, but now seems fine.  A bigger problem is that I don't hear anyone volunteering to do the work. You seem to have a fairly strong opinion on the

Re: [HACKERS] memory-related bugs

2011-09-08 Thread Tom Lane
Daniel Farina dan...@heroku.com writes: On Tue, Sep 6, 2011 at 12:00 PM, Tom Lane t...@sss.pgh.pa.us wrote: I'm still of the opinion that there's no real need to avoid memcpy with identical source and destination, so I didn't apply this first patch. I am leery of memcpy with overlapping

Re: [HACKERS] Protecting against multiple instances per cluster

2011-09-08 Thread Magnus Hagander
On Thu, Sep 8, 2011 at 20:40, Thom Brown t...@linux.com wrote: Hi all, I've come across a PostgreSQL set up where there are 2 servers, each with the same version of PostgreSQL on, both mounting the same SAN onto their respective file systems.  It was intended that only 1 of the servers would

Re: [HACKERS] concurrent snapshots

2011-09-08 Thread Ants Aasma
On Thu, Sep 8, 2011 at 6:46 PM, Robert Haas robertmh...@gmail.com wrote: I'm not convinced it's anywhere near that easy.  For one thing, on at least one big server I'm playing with, memory latency on shared memory is vastly higher (like 10x!) than on backend-local memory due to NUMA effects.

Re: [HACKERS] Large C files

2011-09-08 Thread Josh Berkus
Simon, Robert, Bruce, Tom, Say what? When else would you have us do it? When else would you have us develop? In my eyes that sort of activity *is* development. I find the distinction you are drawing entirely artificial, and more calculated to make sure refactoring never happens

Re: [HACKERS] pg_last_xact_insert_timestamp

2011-09-08 Thread Chris Redekop
Thanks for all the feedback guys. Just to throw another monkey wrench in here - I've been playing with Simon's proposed solution of returning 0 when the WAL positions match, and I've come to the realizatiion that even if using pg_last_xact_insert_timestamp, although it would help, we still

[HACKERS] Patch to improve reliability of postgresql on linux nfs

2011-09-08 Thread George Barnett
Hi Hackers, postgresql-writeall.patch Description: Binary data I run a number of postgresql installations on NFS and on the whole I find this to be very reliable. I have however run into a few issues when there is concurrent writes from multiple processes. I see errors such as the

Re: [HACKERS] Patch to improve reliability of postgresql on linux nfs

2011-09-08 Thread Josh Berkus
George, I'm quoting you here because in the version of your email which got posted to the list your whole explanation got put below the patch text, making it hard to find the justification for the patch. Follows: I run a number of postgresql installations on NFS and on the whole I find this

Re: [HACKERS] Large C files

2011-09-08 Thread Robert Haas
On Thu, Sep 8, 2011 at 4:45 PM, Peter Geoghegan pe...@2ndquadrant.com wrote: On 8 September 2011 15:43, Robert Haas robertmh...@gmail.com wrote: I wouldn't be too enthusiastic about starting a project like this in January, but now seems fine.  A bigger problem is that I don't hear anyone

Re: [HACKERS] pg_last_xact_insert_timestamp

2011-09-08 Thread Fujii Masao
On Thu, Sep 8, 2011 at 10:03 PM, Robert Haas robertmh...@gmail.com wrote: On Thu, Sep 8, 2011 at 6:14 AM, Fujii Masao masao.fu...@gmail.com wrote: OTOH, new function enables users to monitor the delay as a timestamp. For users, a timestamp is obviously easier to handle than LSN, and the delay

Re: [HACKERS] force_not_null option support for file_fdw

2011-09-08 Thread Shigeru Hanada
Thanks for the review, Kaigai-san. (2011/09/09 0:47), Kohei Kaigai wrote: I found one other point to be fixed: On get_force_not_null(), it makes a list of attribute names with force_not_null option. + foreach (cell, options) + { + DefElem*def = (DefElem *)

Re: [HACKERS] pg_dump.c

2011-09-08 Thread Pavel Golub
Hello, Andrew. You wrote: AD In the refactoring Large C files discussion one of the biggest files AD Bruce mentioned is pg_dump.c. There has been discussion in the past of AD turning lots of the knowledge currently embedded in this file into a AD library, which would make it available to other

Re: [HACKERS] Large C files

2011-09-08 Thread Heikki Linnakangas
On 08.09.2011 23:45, Peter Geoghegan wrote: On 8 September 2011 15:43, Robert Haasrobertmh...@gmail.com wrote: I wouldn't be too enthusiastic about starting a project like this in January, but now seems fine. A bigger problem is that I don't hear anyone volunteering to do the work. You seem