[HACKERS] Re: [COMMITTERS] pgsql: Hot Standby feedback for avoidance of cleanup conflicts on stand

2011-02-18 Thread Simon Riggs
On Fri, 2011-02-18 at 11:44 +0900, Fujii Masao wrote: > > This begs the questions "what is the xmin of all the normal > backends?" > > and "Whats is the xmin of prepared transactions?" as well. I wasn't > sure > that we should expose that information for walsenders when we > don't do > it for every

Re: [HACKERS] Transaction-scope advisory locks

2011-02-18 Thread Marko Tiikkaja
On 2011-02-18 7:16 AM +0200, Itagaki Takahiro wrote: Committed with a few typo fixes. Thanks, Marko! Thanks a lot! Regards, Marko Tiikkaja -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hack

Re: [HACKERS] Sync Rep for 2011CF1

2011-02-18 Thread Simon Riggs
On Fri, 2011-02-18 at 00:48 +, Simon Riggs wrote: > Complete but rough hack, for comments, but nothing surprising. This is an implicit requirement from our earlier agreed API, so its blocking further work on Sync Rep. I'm looking to commit this in about 3-4 hours unless I get comments. --

[HACKERS] pg_basebackup and wal streaming

2011-02-18 Thread Magnus Hagander
Better late than never (or?), here's the final cleanup of pg_streamrecv for moving into the main distribution, per discussion back in late dec or early jan. It also includes the "stream logs in parallel to backup" part that was not completed on pg_basebackup. Other than that, the only changes to p

Re: [HACKERS] ECPG dynamic cursor fix for UPDATE/DELETE ... WHERE CURRENT OF :curname

2011-02-18 Thread Michael Meskes
> Did this ever get applied? If so, I can't find it. No, my bad, I simply forgot about it, sorry. Will work on this now. Michael -- Michael Meskes Michael at Fam-Meskes dot De, Michael at Meskes dot (De|Com|Net|Org) Michael at BorussiaFan dot De, Meskes at (Debian|Postgresql) dot Org Jabber: m

Re: [HACKERS] Re: ECPG dynamic cursor fix for UPDATE/DELETE ... WHERE CURRENT OF :curname

2011-02-18 Thread Michael Meskes
> sorry for the late answer. Here is a minimal patch against the My answer is way later, so I have to apologize. > current GIT tree, so the WHERE CURRENT OF accepts > dynamic cursornames, plus the test case that shows the problem. It doesn't on my test system. I just committed the minimal patc

Re: [HACKERS] Debian readline/libedit breakage

2011-02-18 Thread Magnus Hagander
On Fri, Feb 18, 2011 at 02:01, wrote: >> On 02/17/2011 12:34 PM, Bruce Momjian wrote: >> > Andrew Dunstan wrote: >> >> >> >> On 02/17/2011 12:13 PM, Bruce Momjian wrote: >> FWIW, the only interactively usable version of psql for windows I know >> of is the one that runs under Cygwin. It

Re: [HACKERS] SSI bug?

2011-02-18 Thread Magnus Hagander
On Thu, Feb 17, 2011 at 23:11, Kevin Grittner wrote: > Dan Ports wrote: > >> Oops. Those are both definitely bugs (and my fault). Your patch >> looks correct. Thanks for catching that! > > Could a committer please apply the slightly modified version here?: > > http://archives.postgresql.org/messa

Re: [HACKERS] COPY ENCODING revisited

2011-02-18 Thread Itagaki Takahiro
On Fri, Feb 18, 2011 at 03:57, Robert Haas wrote: > On Wed, Feb 16, 2011 at 10:45 PM, Itagaki Takahiro > I am not qualified to fully review this patch because I'm not all that > familiar with the encoding stuff, but it looks reasonably sensible on > a quick read-through.  I am supportive of making

Re: [HACKERS] ALTER TYPE COLLATABLE?

2011-02-18 Thread Peter Eisentraut
On tor, 2011-02-17 at 17:50 -0500, Tom Lane wrote: > What are we going to do to allow the citext update script to fix this? > I see no sign that ALTER TYPE can fix it (and am unsure that we'd want > to add such a feature, particularly not right now). How would this normally be handled if a type ch

[HACKERS] Assertion failure on UNLOGGED VIEW and SEQUENCE

2011-02-18 Thread Itagaki Takahiro
UNLOGGED is defigned in OptTemp in gram.y, so the parser accepts CREATE UNLOGGED VIEW and CREATE UNLOGGED SEQUENCE, but they are actually not supported. =# CREATE UNLOGGED VIEW uv AS SELECT 1; TRAP: FailedAssertion("!(relkind == 'r' || relkind == 't')", File: "heap.c", Line: 1246) =# CREATE UNLOG

[HACKERS] Server Name

2011-02-18 Thread Simon Riggs
We agreed to add a parameter called sync_standbys http://archives.postgresql.org/message-id/4d1dcf5a.7070...@enterprisedb.com. that required the concept of a server name. I've written this patch to add a server name parameter and to send an info message which returns the server name, attached. F

[HACKERS] Re: [COMMITTERS] pgsql: Separate messages for standby replies and hot standby feedback.

2011-02-18 Thread Robert Haas
On Fri, Feb 18, 2011 at 6:34 AM, Simon Riggs wrote: > Separate messages for standby replies and hot standby feedback. > Allow messages to be sent at different times, and greatly reduce > the frequency of hot standby feedback. Refactor to allow additional > message types. You could refactor this s

Re: [HACKERS] About the performance of startup after dropping many tables

2011-02-18 Thread Robert Haas
On Thu, Feb 17, 2011 at 10:37 PM, Tom Lane wrote: > Gan Jiadong writes: >> we have PG 8.3.13 in our system. When running performance cases, we find the >> startup recovery cost about 3 minutes. It is too long in our system. > > Maybe you should rethink the assumption that dropping 4 tables is

Re: [HACKERS] SQL/MED - file_fdw

2011-02-18 Thread Shigeru HANADA
I prefer FileFdwExecutionState to FileFdwPrivate, because there are > two different 'fdw_private' variables in FdwPlan and FdwExecutionState. > > * The comment in fileIterate seems wrong. It should be > /* Store the next tuple as a virtual tuple. */ , right? > >

Re: [HACKERS] contrib loose ends: 9.0 to 9.1 incompatibilities

2011-02-18 Thread Robert Haas
On Thu, Feb 17, 2011 at 9:13 PM, Tom Lane wrote: > I wrote: >> Robert Haas writes: >>> I think we should try to make the state match as closely as possible, >>> no matter how you got there.  Otherwise, I think we're storing up a >>> host of future pain for ourselves. > >> Well, if you're willing

Re: [HACKERS] Add support for logging the current role

2011-02-18 Thread Robert Haas
On Thu, Feb 17, 2011 at 10:40 PM, Tom Lane wrote: > Robert Haas writes: >> On Thu, Feb 17, 2011 at 4:53 PM, Tom Lane wrote: >>> In short, add a bit of overhead at SetUserId time in order to make this >>> cheap (and accurate) in elog.c. > >> As Stephen says, I think this is utterly impractical; t

Re: [HACKERS] WIP - Add ability to constrain backend temporary file space

2011-02-18 Thread Robert Haas
On Thu, Feb 17, 2011 at 10:17 PM, Mark Kirkwood wrote: > This is WIP, it does seem to work ok, but some areas/choices I'm not > entirely clear about are mentioned in the patch itself. Mainly: > > - name of the guc... better suggestions welcome > - datatype for the guc - real would be good, but at

Re: [HACKERS] Assertion failure on UNLOGGED VIEW and SEQUENCE

2011-02-18 Thread Robert Haas
On Fri, Feb 18, 2011 at 6:42 AM, Itagaki Takahiro wrote: > UNLOGGED is defigned in OptTemp in gram.y, so the parser accepts > CREATE UNLOGGED VIEW and CREATE UNLOGGED SEQUENCE, but they are > actually not supported. > > =# CREATE UNLOGGED VIEW uv AS SELECT 1; > TRAP: FailedAssertion("!(relkind ==

[HACKERS] Re: [COMMITTERS] pgsql: Separate messages for standby replies and hot standby feedback.

2011-02-18 Thread Simon Riggs
On Fri, 2011-02-18 at 07:11 -0500, Robert Haas wrote: > On Fri, Feb 18, 2011 at 6:34 AM, Simon Riggs wrote: > > Separate messages for standby replies and hot standby feedback. > > Allow messages to be sent at different times, and greatly reduce > > the frequency of hot standby feedback. Refactor t

Re: [HACKERS] About the performance of startup after dropping many tables

2011-02-18 Thread Tom Lane
Robert Haas writes: > Possibly, but it's not necessarily a bad idea to improve performance > for people with crazy schemas. It is if it introduces unmaintainable code. I see no way to collapse multiple drop operations into one that's not going to be a Rube Goldberg device. I'm especially unwill

Re: [HACKERS] Assertion failure on UNLOGGED VIEW and SEQUENCE

2011-02-18 Thread Tom Lane
Robert Haas writes: > On Fri, Feb 18, 2011 at 6:42 AM, Itagaki Takahiro >> The most easiest fix would be preventing them in parser level. > Well, that sucks. I had intended for that to be disallowed at the > parser level, but obviously I fail. It seems that disallowing this in > the parser will

Re: [HACKERS] Fix for Index Advisor related hooks

2011-02-18 Thread Gurjeet Singh
On Fri, Feb 18, 2011 at 2:27 AM, Heikki Linnakangas < heikki.linnakan...@enterprisedb.com> wrote: > On 17.02.2011 14:30, Gurjeet Singh wrote: > >> On Wed, Feb 16, 2011 at 6:37 PM, Tom Lane wrote: >> >> Fetch the values you need and stuff 'em in the struct. Don't expect >>> relcache to do it for

[HACKERS] Re: [COMMITTERS] pgsql: Make a hard state change from catchup to streaming mode.

2011-02-18 Thread Robert Haas
On Fri, Feb 18, 2011 at 10:09 AM, Simon Riggs wrote: > Make a hard state change from catchup to streaming mode. > More useful state change for monitoring purposes, plus a > required change for synchronous replication patch. As far as I can see, this patch was not posted or discussed before commit

[HACKERS] Re: [COMMITTERS] pgsql: Make a hard state change from catchup to streaming mode.

2011-02-18 Thread Kevin Grittner
Robert Haas wrote: > Simon Riggs wrote: >> Make a hard state change from catchup to streaming mode. >> More useful state change for monitoring purposes, plus a >> required change for synchronous replication patch. > > As far as I can see, this patch was not posted or discussed before > commit,

Re: [HACKERS] Re: [COMMITTERS] pgsql: Make a hard state change from catchup to streaming mode.

2011-02-18 Thread Tom Lane
Robert Haas writes: > On Fri, Feb 18, 2011 at 10:09 AM, Simon Riggs wrote: >> Make a hard state change from catchup to streaming mode. >> More useful state change for monitoring purposes, plus a >> required change for synchronous replication patch. > As far as I can see, this patch was not poste

Re: [HACKERS] Debian readline/libedit breakage

2011-02-18 Thread Andrew Dunstan
On 02/17/2011 04:09 PM, Martijn van Oosterhout wrote: On Wed, Feb 16, 2011 at 04:33:19PM -0800, Joshua D. Drake wrote: Maybe we really should consider moving to NSS insread? http://www.mozilla.org/projects/security/pki/nss/ If it solves the license problem, it is well supported etc.. For th

Re: [HACKERS] review: FDW API

2011-02-18 Thread Tom Lane
Heikki Linnakangas writes: > On 15.02.2011 23:00, Tom Lane wrote: >> I think moving the error check downstream would be a good thing. > Ok, I tried moving the error checks to preprocess_rowmarks(). > Unfortunately RelOptInfos haven't been built at that stage yet, so you > still have to do the c

Re: [HACKERS] Debian readline/libedit breakage

2011-02-18 Thread Tom Lane
Andrew Dunstan writes: > On 02/17/2011 04:09 PM, Martijn van Oosterhout wrote: >> This is supported. Where it goes wonky is that this also has to work >> when the connection is via SSL. So libpq provides a function to return >> (via a void*) a pointer to the OpenSSL structure so that can be used t

Re: [HACKERS] Debian readline/libedit breakage

2011-02-18 Thread Magnus Hagander
On Fri, Feb 18, 2011 at 16:51, Tom Lane wrote: > Andrew Dunstan writes: >> On 02/17/2011 04:09 PM, Martijn van Oosterhout wrote: >>> This is supported. Where it goes wonky is that this also has to work >>> when the connection is via SSL. So libpq provides a function to return >>> (via a void*) a

Re: [HACKERS] Re: [COMMITTERS] pgsql: Make a hard state change from catchup to streaming mode.

2011-02-18 Thread Simon Riggs
On Fri, 2011-02-18 at 10:41 -0500, Tom Lane wrote: > Robert Haas writes: > > On Fri, Feb 18, 2011 at 10:09 AM, Simon Riggs wrote: > >> Make a hard state change from catchup to streaming mode. > >> More useful state change for monitoring purposes, plus a > >> required change for synchronous replic

Re: [HACKERS] Snapshot synchronization, again...

2011-02-18 Thread Alvaro Herrera
Looking into this patch. -- Álvaro Herrera The PostgreSQL Company - Command Prompt, Inc. PostgreSQL Replication, Consulting, Custom Development, 24x7 support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/m

Re: [HACKERS] Students enrollment

2011-02-18 Thread Kevin Grittner
Roman Prykhodchenko wrote: > My students have to make their science projects during this > half-year. I would like to involve some of them in PostgreSQL to > give them an opportunity to work on a real project instead of > writing their own equation solvers or any other useless apps. > > Here is

Re: [HACKERS] ALTER TYPE COLLATABLE?

2011-02-18 Thread Tom Lane
Peter Eisentraut writes: > On tor, 2011-02-17 at 17:50 -0500, Tom Lane wrote: >> Is it time for a direct UPDATE on the pg_type row? If so, to what? I see >> pg_type.typcollation is supposed to be an OID, so how the heck does >> one map a bool CREATE TYPE parameter into the catalog entry? > It's

Re: [HACKERS] Re: [COMMITTERS] pgsql: Make a hard state change from catchup to streaming mode.

2011-02-18 Thread Robert Haas
On Fri, Feb 18, 2011 at 10:41 AM, Tom Lane wrote: > Robert Haas writes: >> On Fri, Feb 18, 2011 at 10:09 AM, Simon Riggs wrote: >>> Make a hard state change from catchup to streaming mode. >>> More useful state change for monitoring purposes, plus a >>> required change for synchronous replicatio

Re: [HACKERS] contrib loose ends: 9.0 to 9.1 incompatibilities

2011-02-18 Thread Tom Lane
Robert Haas writes: > On Thu, Feb 17, 2011 at 9:13 PM, Tom Lane wrote: >> 2. intarray and tsearch2 use some core support functions in their >> GIN opclasses, and those support functions changed signatures in 9.1. >> The current solution to this involves having stub functions in core >> with the o

Re: [HACKERS] Re: [COMMITTERS] pgsql: Make a hard state change from catchup to streaming mode.

2011-02-18 Thread Robert Haas
On Fri, Feb 18, 2011 at 10:56 AM, Simon Riggs wrote: > We need a hard state change at one particular point to avoid sync rep > requestors from hanging for hours when the standby is connected but a > long way from being caught up. That's a matter of opinion. I think there are a number of people h

Re: [HACKERS] contrib loose ends: 9.0 to 9.1 incompatibilities

2011-02-18 Thread Robert Haas
On Fri, Feb 18, 2011 at 12:01 PM, Tom Lane wrote: > Robert Haas writes: >> On Thu, Feb 17, 2011 at 9:13 PM, Tom Lane wrote: >>> 2. intarray and tsearch2 use some core support functions in their >>> GIN opclasses, and those support functions changed signatures in 9.1. >>> The current solution to

Re: [HACKERS] contrib loose ends: 9.0 to 9.1 incompatibilities

2011-02-18 Thread Tom Lane
Robert Haas writes: > On Fri, Feb 18, 2011 at 12:01 PM, Tom Lane wrote: >> OK, I held my nose and inserted UPDATE commands to make the opclasses >> match.  AFAICT the only remaining discrepancy between contrib modules >> made fresh in 9.1 and those updated from 9.0 is the question of citext's >>

Re: [HACKERS] Fix for Index Advisor related hooks

2011-02-18 Thread Heikki Linnakangas
On 18.02.2011 17:02, Gurjeet Singh wrote: On Fri, Feb 18, 2011 at 2:27 AM, Heikki Linnakangas< heikki.linnakan...@enterprisedb.com> wrote: On 17.02.2011 14:30, Gurjeet Singh wrote: On Wed, Feb 16, 2011 at 6:37 PM, Tom Lane wrote: Fetch the values you need and stuff 'em in the struct. Don

Re: [HACKERS] Fix for Index Advisor related hooks

2011-02-18 Thread Tom Lane
Heikki Linnakangas writes: > On 18.02.2011 17:02, Gurjeet Singh wrote: >>> On Wed, Feb 16, 2011 at 6:37 PM, Tom Lane wrote: Fetch the values you need and stuff 'em in the struct. Don't expect relcache to do it for you. >>> I also wish to make Index Advisor faster by not having to loo

[HACKERS] Proposal: collect frequency statistics for arrays

2011-02-18 Thread Alexander Korotkov
Hackers, I have following proposal. Currently the ts_typanalyze function accumulates frequency statistics for ts_vector using lossy counting technique. But no frequency statistics is collecting over arrays. I'm going to generalize ts_typanalyze to make it collecting statistics for arrays too. ts_t

Re: [HACKERS] Proposal: collect frequency statistics for arrays

2011-02-18 Thread Tom Lane
Alexander Korotkov writes: > I have following proposal. Currently the ts_typanalyze function accumulates > frequency statistics for ts_vector using lossy counting technique. But no > frequency statistics is collecting over arrays. I'm going to generalize > ts_typanalyze to make it collecting stati

Re: [HACKERS] Assertion failure on UNLOGGED VIEW and SEQUENCE

2011-02-18 Thread Robert Haas
On Fri, Feb 18, 2011 at 10:02 AM, Tom Lane wrote: > If by the first option you mean causing the failure report to be "syntax > error" rather than anything more specific, then I agree that option > sucks.  I'd actually vote for putting the error test somewhere in > statement execution code, well do

[HACKERS] SR standby hangs

2011-02-18 Thread Andrew Dunstan
[this time from the right address - apologies if we get a duplicate] PostgreSQL Experts Inc has a client with a 9.0.2 streaming replication server that somehow becomes wedged after running for some time. The server is running as a warm standby, and the client's application tries to connect to

Re: [HACKERS] Assertion failure on UNLOGGED VIEW and SEQUENCE

2011-02-18 Thread Tom Lane
Robert Haas writes: > OK. Proposed patch attached. It looks to me like an unlogged view is > inherently nonsensical, whereas an unlogged sequence is sensible but > we don't implement it (and may never implement it, absent some > evidence that the overhead of WAL logging sequence changes is worth

Re: [HACKERS] review: FDW API

2011-02-18 Thread Robert Haas
On Fri, Feb 18, 2011 at 10:47 AM, Tom Lane wrote: > Heikki Linnakangas writes: >> On 15.02.2011 23:00, Tom Lane wrote: >>> I think moving the error check downstream would be a good thing. > >> Ok, I tried moving the error checks to preprocess_rowmarks(). >> Unfortunately RelOptInfos haven't been

Re: [HACKERS] SR standby hangs

2011-02-18 Thread Tom Lane
Andrew Dunstan writes: > The symptom is that the recovery process blocks forever on a semaphore. > We've crashed it and got the following backtrace: > #0 0x003493ed5337 in semop () from /lib64/libc.so.6 > #1 0x005bd103 in PGSemaphoreLock (sema=0x2b14986aec38, > interruptOK=

Re: [HACKERS] Assertion failure on UNLOGGED VIEW and SEQUENCE

2011-02-18 Thread Robert Haas
On Fri, Feb 18, 2011 at 2:17 PM, Tom Lane wrote: > Robert Haas writes: >> OK.  Proposed patch attached.  It looks to me like an unlogged view is >> inherently nonsensical, whereas an unlogged sequence is sensible but >> we don't implement it (and may never implement it, absent some >> evidence th

Re: [HACKERS] using a lot of maintenance_work_mem

2011-02-18 Thread Frederik Ramm
Tom & Kevin, thank you for your replies. Kevin, I had already employed all the tricks you mention, except using temporary tables which would be hard for me due to the structure of my application (but I could try using something like pgbouncer or so), but thanks a lot for sharing the ideas.

Re: [HACKERS] pg_basebackup and wal streaming

2011-02-18 Thread Bruce Momjian
Magnus Hagander wrote: > Better late than never (or?), here's the final cleanup of > pg_streamrecv for moving into the main distribution, per discussion > back in late dec or early jan. It also includes the "stream logs in > parallel to backup" part that was not completed on pg_basebackup. > > Oth

Re: [HACKERS] SR standby hangs

2011-02-18 Thread Robert Haas
On Fri, Feb 18, 2011 at 2:16 PM, Andrew Dunstan wrote: > I'm not quite sure where to start digging. Has anyone else seen > something similar? Our consultant reports having seen a similar problem > elsewhere, at a client who was running hot standby on 9.0.1, but the > problem did not recur, as it d

Re: [HACKERS] review: FDW API

2011-02-18 Thread Tom Lane
Robert Haas writes: > On Fri, Feb 18, 2011 at 10:47 AM, Tom Lane wrote: >> The main downside of that is that relation relkinds would have >> to become fixed (because there would be no practical way of updating >> RTEs in stored rules), which means the "convert relation to view" hack >> would have

Re: [HACKERS] SR standby hangs

2011-02-18 Thread Andrew Dunstan
On 02/18/2011 02:23 PM, Tom Lane wrote: Andrew Dunstan writes: The symptom is that the recovery process blocks forever on a semaphore. We've crashed it and got the following backtrace: #0 0x003493ed5337 in semop () from /lib64/libc.so.6 #1 0x005bd103 in PGSemaphoreLock

Re: [HACKERS] Debian readline/libedit breakage

2011-02-18 Thread Bruce Momjian
Tom Lane wrote: > Andrew Dunstan writes: > > On 02/17/2011 04:09 PM, Martijn van Oosterhout wrote: > >> This is supported. Where it goes wonky is that this also has to work > >> when the connection is via SSL. So libpq provides a function to return > >> (via a void*) a pointer to the OpenSSL struc

Re: [HACKERS] Assertion failure on UNLOGGED VIEW and SEQUENCE

2011-02-18 Thread Tom Lane
Robert Haas writes: > On Fri, Feb 18, 2011 at 2:17 PM, Tom Lane wrote: >> +1 for where you put the tests, but I don't think >> ERRCODE_SYNTAX_ERROR is an appropriate errcode. I'd go with >> FEATURE_NOT_SUPPORTED for both, I think. > I hesitate to use FEATURE_NOT_SUPPORTED for something that's >

Re: [HACKERS] using a lot of maintenance_work_mem

2011-02-18 Thread Tom Lane
Frederik Ramm writes: > The single query where pg9.0 beat pg8.3 by a country mile was a CREATE > INDEX statement on a BIGINT column to a table with about 500 million > records - this cost 2679 seconds on normal 8.3, 2443 seconds on > large-memory 8.3, and aroung 1650 seconds on 9.0, large memor

Re: [HACKERS] WIP - Add ability to constrain backend temporary file space

2011-02-18 Thread Josh Berkus
Mark, > I got to wonder how hard this would be to do in Postgres, and attached > is my (WIP) attempt. It provides a guc (max_temp_files_size) to limit > the size of all temp files for a backend and amends fd.c cancel > execution if the total size of temporary files exceeds this. First, are we jus

Re: [HACKERS] SR standby hangs

2011-02-18 Thread Robert Haas
On Fri, Feb 18, 2011 at 2:35 PM, Andrew Dunstan wrote: > It's not running HS, so there's no query to wait on. That seems to imply that recovery has leaked a buffer pin. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing

Re: [HACKERS] WIP - Add ability to constrain backend temporary file space

2011-02-18 Thread Robert Haas
On Fri, Feb 18, 2011 at 2:41 PM, Josh Berkus wrote: > Second, the main issue with these sorts of macro-counters has generally > been their locking effect on concurrent activity.  Have you been able to > run any tests which try to run lots of small externally-sorted queries > at once on a multi-cor

Re: [HACKERS] WIP - Add ability to constrain backend temporary file space

2011-02-18 Thread Josh Berkus
On 2/18/11 11:44 AM, Robert Haas wrote: > On Fri, Feb 18, 2011 at 2:41 PM, Josh Berkus wrote: >> Second, the main issue with these sorts of macro-counters has generally >> been their locking effect on concurrent activity. Have you been able to >> run any tests which try to run lots of small exter

Re: [HACKERS] SR standby hangs

2011-02-18 Thread Tom Lane
Robert Haas writes: > On Fri, Feb 18, 2011 at 2:35 PM, Andrew Dunstan wrote: >> It's not running HS, so there's no query to wait on. > That seems to imply that recovery has leaked a buffer pin. No, because then the sanity check in LockBufferForCleanup would have fired: /* There should

Re: [HACKERS] Snapshot synchronization, again...

2011-02-18 Thread Alvaro Herrera
I have two questions: 1. why are you using the expansible char array stuff instead of using the StringInfo facility? 2. is md5 the most appropriate digest for this? If you need a cryptographically secure hash, do we need something stronger? If not, why not just use hash_any? -- Álvaro Herr

Re: [HACKERS] review: FDW API

2011-02-18 Thread Tom Lane
Heikki Linnakangas writes: > Another version, rebased against master branch and with a bunch of small > cosmetic fixes. > I guess this is as good as this is going to get for 9.1. This is *badly* in need of another cleanup pass; it's full of typos, contradictory comments, #ifdef NOT_USED stuff,

Re: [HACKERS] contrib loose ends: 9.0 to 9.1 incompatibilities

2011-02-18 Thread Dimitri Fontaine
Tom Lane writes: >> OK. Thanks for nailing all of this down - that's got to have been a >> heck of a job. +1 > Yeah, it was a bit of a pain, and took longer than I would've hoped. Well, with some luck (and effort) 9.2 will have the missing DDL pieces. I think the extension features means we no

Re: [HACKERS] review: FDW API

2011-02-18 Thread Heikki Linnakangas
On 18.02.2011 22:16, Tom Lane wrote: Heikki Linnakangas writes: Another version, rebased against master branch and with a bunch of small cosmetic fixes. I guess this is as good as this is going to get for 9.1. This is *badly* in need of another cleanup pass; it's full of typos, contradicto

[HACKERS] SR standby hangs

2011-02-18 Thread Andrew Dunstan
PostgreSQL Experts Inc has a client with a 9.0.2 streaming replication server that somehow becomes wedged after running for some time. The server is running as a warm standby, and the client's application tries to connect to both the master and the slave, accepting whichever lets it connect

Re: [HACKERS] WIP - Add ability to constrain backend temporary file space

2011-02-18 Thread Robert Haas
On Fri, Feb 18, 2011 at 2:48 PM, Josh Berkus wrote: > On 2/18/11 11:44 AM, Robert Haas wrote: >> On Fri, Feb 18, 2011 at 2:41 PM, Josh Berkus wrote: >>> Second, the main issue with these sorts of macro-counters has generally >>> been their locking effect on concurrent activity.  Have you been abl

Re: [HACKERS] SR standby hangs

2011-02-18 Thread Robert Haas
On Fri, Feb 18, 2011 at 2:50 PM, Tom Lane wrote: > Robert Haas writes: >> On Fri, Feb 18, 2011 at 2:35 PM, Andrew Dunstan wrote: >>> It's not running HS, so there's no query to wait on. > >> That seems to imply that recovery has leaked a buffer pin. > > No, because then the sanity check in LockB

Re: [HACKERS] review: FDW API

2011-02-18 Thread Tom Lane
Heikki Linnakangas writes: > On 18.02.2011 22:16, Tom Lane wrote: >> Question after first look: what is the motivation for passing >> estate->es_param_list_info to BeginScan? AFAICS, even if there is a >> reason for that function to need that, it isn't receiving any info that >> would be sufficie

Re: [HACKERS] WIP - Add ability to constrain backend temporary file space

2011-02-18 Thread Mark Kirkwood
On 19/02/11 02:34, Robert Haas wrote: Please add this to the next CommitFest: https://commitfest.postgresql.org/action/commitfest_view/open With respect to the datatype of the GUC, int seems clearly correct. Why would you want to use a float? Added. With respect to the datatype, using int w

Re: [HACKERS] WIP - Add ability to constrain backend temporary file space

2011-02-18 Thread Mark Kirkwood
On 19/02/11 08:48, Josh Berkus wrote: On 2/18/11 11:44 AM, Robert Haas wrote: On Fri, Feb 18, 2011 at 2:41 PM, Josh Berkus wrote: Second, the main issue with these sorts of macro-counters has generally been their locking effect on concurrent activity. Have you been able to run any tests which

Re: [HACKERS] WIP - Add ability to constrain backend temporary file space

2011-02-18 Thread Josh Berkus
> Obviously you need to do the same sort of arithmetic as you do with > work_mem to decide on a reasonable limit to cope with multiple users > creating temp files. Conservative dbas might want to set it to (free > disk)/max_connections etc. Obviously for ad-hoc systems it is a bit more > challengi

Re: [HACKERS] WIP - Add ability to constrain backend temporary file space

2011-02-18 Thread Tom Lane
Mark Kirkwood writes: > Added. With respect to the datatype, using int with KB units means the > largest temp size is approx 2047GB - I know that seems like a lot now... > but maybe someone out there wants (say) their temp files limited to > 4096GB :-) [ shrug... ] Sorry, I can't imagine a us

Re: [HACKERS] DropRelFileNodeBuffers API change (was Re: [BUGS] BUG #5599: Vacuum fails due to index corruption issues)

2011-02-18 Thread Bruce Momjian
Did we make any progress on this? Is it a TODO? --- Tom Lane wrote: > Robert Haas writes: > > On Sun, Aug 15, 2010 at 9:25 PM, Tom Lane wrote: > >> ... and PANIC is absolutely, entirely, 100% unacceptable here. ?I don't >

Re: [HACKERS] DropRelFileNodeBuffers API change (was Re: [BUGS] BUG #5599: Vacuum fails due to index corruption issues)

2011-02-18 Thread Tom Lane
Bruce Momjian writes: > Did we make any progress on this? Is it a TODO? AFAIR nothing's been done about it, so it's a TODO. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgres

Re: [HACKERS] DropRelFileNodeBuffers API change (was Re: [BUGS] BUG #5599: Vacuum fails due to index corruption issues)

2011-02-18 Thread Robert Haas
On Fri, Feb 18, 2011 at 4:46 PM, Tom Lane wrote: > Bruce Momjian writes: >> Did we make any progress on this?  Is it a TODO? > > AFAIR nothing's been done about it, so it's a TODO. I was thinking of adding it to the 9.1 open items list, but the wiki's been down every time I've tried to go there.

Re: [HACKERS] DropRelFileNodeBuffers API change (was Re: [BUGS] BUG #5599: Vacuum fails due to index corruption issues)

2011-02-18 Thread Bruce Momjian
Robert Haas wrote: > On Fri, Feb 18, 2011 at 4:46 PM, Tom Lane wrote: > > Bruce Momjian writes: > >> Did we make any progress on this? ?Is it a TODO? > > > > AFAIR nothing's been done about it, so it's a TODO. > > I was thinking of adding it to the 9.1 open items list, but the wiki's > been down

Re: [HACKERS] WIP - Add ability to constrain backend temporary file space

2011-02-18 Thread Mark Kirkwood
On 19/02/11 10:38, Josh Berkus wrote: To answer the other question, what happens when the limit is exceeded is modeled on statement timeout, i.e query is canceled and a message says why (exceeded temp files size). When does this happen? When you try to allocate the file, or when it does the o

Re: [HACKERS] DropRelFileNodeBuffers API change (was Re: [BUGS] BUG #5599: Vacuum fails due to index corruption issues)

2011-02-18 Thread Tom Lane
Robert Haas writes: > On Fri, Feb 18, 2011 at 4:46 PM, Tom Lane wrote: >> AFAIR nothing's been done about it, so it's a TODO. > I was thinking of adding it to the 9.1 open items list, but the wiki's > been down every time I've tried to go there. Since the problem's been there since forever, I d

Re: [HACKERS] DropRelFileNodeBuffers API change (was Re: [BUGS] BUG #5599: Vacuum fails due to index corruption issues)

2011-02-18 Thread Robert Haas
On Fri, Feb 18, 2011 at 4:59 PM, Tom Lane wrote: > Robert Haas writes: >> On Fri, Feb 18, 2011 at 4:46 PM, Tom Lane wrote: >>> AFAIR nothing's been done about it, so it's a TODO. > >> I was thinking of adding it to the 9.1 open items list, but the wiki's >> been down every time I've tried to go

Re: [HACKERS] DropRelFileNodeBuffers API change (was Re: [BUGS] BUG #5599: Vacuum fails due to index corruption issues)

2011-02-18 Thread Bruce Momjian
Robert Haas wrote: > On Fri, Feb 18, 2011 at 4:59 PM, Tom Lane wrote: > > Robert Haas writes: > >> On Fri, Feb 18, 2011 at 4:46 PM, Tom Lane wrote: > >>> AFAIR nothing's been done about it, so it's a TODO. > > > >> I was thinking of adding it to the 9.1 open items list, but the wiki's > >> been

Re: [HACKERS] SR standby hangs

2011-02-18 Thread Andrew Dunstan
On 02/18/2011 03:42 PM, Robert Haas wrote: On Fri, Feb 18, 2011 at 2:50 PM, Tom Lane wrote: Robert Haas writes: On Fri, Feb 18, 2011 at 2:35 PM, Andrew Dunstan wrote: It's not running HS, so there's no query to wait on. That seems to imply that recovery has leaked a buffer pin. No, beca

Re: [HACKERS] WIP - Add ability to constrain backend temporary file space

2011-02-18 Thread Josh Berkus
> Yeah, the disadvantage is that (like statement timeout) it is a 'bottom > of the cliff' type of protection. The advantage is there are no false > positives... Yeah, just trying to get a handle on the proposed feature. I have no objections; it seems like a harmless limit for most people, and us

Re: [HACKERS] WIP - Add ability to constrain backend temporary file space

2011-02-18 Thread Mark Kirkwood
On 19/02/11 11:30, Josh Berkus wrote: Yeah, the disadvantage is that (like statement timeout) it is a 'bottom of the cliff' type of protection. The advantage is there are no false positives... Yeah, just trying to get a handle on the proposed feature. I have no objections; it seems like a harml

[HACKERS] disposition of remaining patches

2011-02-18 Thread Robert Haas
The CommitFest application currently reflects 17 remaining patches for CommitFest 2011-01. 1. Change pg_last_xlog_receive_location not to move backwards. We don't have complete consensus on what to do here. If we can agree on a way forward, I think we can finish this one up pretty quickly. It's

Re: [HACKERS] DropRelFileNodeBuffers API change (was Re: [BUGS] BUG #5599: Vacuum fails due to index corruption issues)

2011-02-18 Thread Robert Haas
On Fri, Feb 18, 2011 at 5:10 PM, Bruce Momjian wrote: > Robert Haas wrote: >> On Fri, Feb 18, 2011 at 4:59 PM, Tom Lane wrote: >> > Robert Haas writes: >> >> On Fri, Feb 18, 2011 at 4:46 PM, Tom Lane wrote: >> >>> AFAIR nothing's been done about it, so it's a TODO. >> > >> >> I was thinking of

Re: [HACKERS] Initial review of xslt with no limits patch

2011-02-18 Thread Bruce Momjian
Tom Lane wrote: > Bruce Momjian writes: > > Tom Lane wrote: > >> I think we have a few TODO items here: > >> > >> * Invent ... and document ... an API that permits safe assembly of a > >> parameter list from non-constant (and perhaps untrustworthy) values. > >> > >> * Fix xslt_process' failure t

Re: [HACKERS] disposition of remaining patches

2011-02-18 Thread Tom Lane
Robert Haas writes: > 3, 4, 5. SQL/MED. Tom has picked up the main FDW API patch, which I > expect means it'll go in. I am not so sure about the FDW patches, > though: in particular, based on Heikki's comments, the postgresql_fdw > patch seems to be badly in need of some more work. The file_fdw

Re: [HACKERS] disposition of remaining patches

2011-02-18 Thread Josh Berkus
On 2/18/11 2:47 PM, Robert Haas wrote: > The CommitFest application currently reflects 17 remaining patches for > CommitFest 2011-01. I'm impressed, actually. This is way further along than I expected us to be. -- -- Josh Berkus

Re: [HACKERS] disposition of remaining patches

2011-02-18 Thread Josh Berkus
On 2/18/11 3:04 PM, Tom Lane wrote: > postgresql_fdw may have to live as an external project for the 9.1 > cycle, unless it's in much better shape than you suggest above. > I won't feel too bad about that as long as the core support exists. > More than likely, people would want to improve it on a f

Re: [HACKERS] disposition of remaining patches

2011-02-18 Thread Andrew Dunstan
On 02/18/2011 05:47 PM, Robert Haas wrote: 3, 4, 5. SQL/MED. Tom has picked up the main FDW API patch, which I expect means it'll go in. I am not so sure about the FDW patches, though: in particular, based on Heikki's comments, the postgresql_fdw patch seems to be badly in need of some more w

Re: [HACKERS] disposition of remaining patches

2011-02-18 Thread Tom Lane
Josh Berkus writes: > On 2/18/11 3:04 PM, Tom Lane wrote: >> postgresql_fdw may have to live as an external project for the 9.1 >> cycle, unless it's in much better shape than you suggest above. >> I won't feel too bad about that as long as the core support exists. >> More than likely, people woul

Re: [HACKERS] DropRelFileNodeBuffers API change (was Re: [BUGS] BUG #5599: Vacuum fails due to index corruption issues)

2011-02-18 Thread Bruce Momjian
Robert Haas wrote: > On Fri, Feb 18, 2011 at 5:10 PM, Bruce Momjian wrote: > > Robert Haas wrote: > >> On Fri, Feb 18, 2011 at 4:59 PM, Tom Lane wrote: > >> > Robert Haas writes: > >> >> On Fri, Feb 18, 2011 at 4:46 PM, Tom Lane wrote: > >> >>> AFAIR nothing's been done about it, so it's a TODO

Re: [HACKERS] review: FDW API

2011-02-18 Thread Tom Lane
I wrote: > ... My feeling is it'd be best to pass down > all the information the executor node has got --- probably we should > just pass the ForeignScanState node itself, and leave a void * in that > for FDW-private data, and be done with it. Otherwise we're going to be > adding missed stuff back

[HACKERS] Sync Rep v17

2011-02-18 Thread Simon Riggs
Well, good news all round. v17 implements what I believe to be the final set of features for sync rep. This one I'm actually fairly happy with. It can be enjoyed best at DEBUG3. The patch is very lite touch on a few areas of code, plus a chunk of specific code, all on master-side. Pretty straigh

Re: [HACKERS] Sync Rep v17

2011-02-18 Thread Thom Brown
On 19 February 2011 00:06, Simon Riggs wrote: > > Well, good news all round. > > v17 implements what I believe to be the final set of features for sync > rep. This one I'm actually fairly happy with. It can be enjoyed best at > DEBUG3. > > The patch is very lite touch on a few areas of code, plus

Re: [HACKERS] Re: [COMMITTERS] pgsql: Make a hard state change from catchup to streaming mode.

2011-02-18 Thread Simon Riggs
On Fri, 2011-02-18 at 09:41 -0600, Kevin Grittner wrote: > Robert Haas wrote: > > Simon Riggs wrote: > > >> Make a hard state change from catchup to streaming mode. > >> More useful state change for monitoring purposes, plus a > >> required change for synchronous replication patch. > > > > As

Re: [HACKERS] Sync Rep v17

2011-02-18 Thread Simon Riggs
On Sat, 2011-02-19 at 00:32 +, Thom Brown wrote: > I see the updated patch I provided last time to fix various errata and > spacing issues got lost in the last round of conversations. Maybe > it's safer to provide a patch for the patch. I'm sorry about that Thom, your changes were and are we

Re: [HACKERS] Snapshot synchronization, again...

2011-02-18 Thread Joachim Wieland
On Fri, Feb 18, 2011 at 8:57 PM, Alvaro Herrera wrote: > 1. why are you using the expansible char array stuff instead of using > the StringInfo facility? > > 2. is md5 the most appropriate digest for this?  If you need a > cryptographically secure hash, do we need something stronger?  If not, > wh

  1   2   >