Re: [HACKERS] Logging of PAM Authentication Failure

2013-05-28 Thread Jeff Janes
On 5/27/13, Craig Ringer cr...@2ndquadrant.com wrote: We were just talking about things we'd like to do in wire protocol 4. Allowing multi-stage authentication has come up repeatedly and should perhaps go on that list. The most obvious case being ident auth failed, demand md5. I'd like to

Re: [HACKERS] Logging of PAM Authentication Failure

2013-05-28 Thread David Fetter
On Tue, May 28, 2013 at 01:32:53PM +0800, Craig Ringer wrote: On 05/11/2013 03:25 AM, Robert Haas wrote: Not really. We could potentially fix it by extending the wire protocol to allow the server to respond to the client's startup packet with a further challenge, and extend libpq to report

[HACKERS] plpgsql redesign (related to plpgsql check function)

2013-05-28 Thread Pavel Stehule
Hello all I am searching way how to push our plpgsql_check_function to upstream. One possibility is redesign of plpgsql architecture. Now, we have two stages - compilation and execution, and almost all compilation logic is in gram file. If I understand to this design well, then a reason for it

Re: [HACKERS] Logging of PAM Authentication Failure

2013-05-28 Thread Amit Langote
On Tue, May 28, 2013 at 2:32 PM, Craig Ringer cr...@2ndquadrant.com wrote: On 05/11/2013 03:25 AM, Robert Haas wrote: Not really. We could potentially fix it by extending the wire protocol to allow the server to respond to the client's startup packet with a further challenge, and extend libpq

Re: [HACKERS] Logging of PAM Authentication Failure

2013-05-28 Thread Amit Langote
On Tue, May 28, 2013 at 5:04 PM, Amit Langote amitlangot...@gmail.com wrote: On Tue, May 28, 2013 at 2:32 PM, Craig Ringer cr...@2ndquadrant.com wrote: On 05/11/2013 03:25 AM, Robert Haas wrote: Not really. We could potentially fix it by extending the wire protocol to allow the server to

Re: [HACKERS] [PATCH] add --throttle to pgbench (submission 3)

2013-05-28 Thread Fabien COELHO
The use case of the option is to be able to generate a continuous gentle load for functional tests, eg in a practice session with students or for testing features on a laptop. If you add this to https://commitfest.postgresql.org/action/commitfest_view?id=18 I'll review it next month. I have

Re: [HACKERS] [PATCH] add --throttle to pgbench (submission 3)

2013-05-28 Thread Craig Ringer
On 05/28/2013 04:13 PM, Fabien COELHO wrote: You can try to use and improve the --progress option in another patch submission which shows how things are going. That'll certainly be useful, but won't solve this issue. The thing is that with asynchronous replication you need to know how long it

Re: [HACKERS] Unsigned integer types

2013-05-28 Thread Maciej Gajewski
The reasons are: performance, storage and frustration. I think the frustration comes from the fact that unsigned integers are universally available, except in PostgreSQL. I work with a really complex system, with many moving parts, and Postgres really is one of the components that causes the

[HACKERS] converting numeric to string in postgres code

2013-05-28 Thread Szymon Guz
Hi, while hacking on some Postgres code I've found a problem. I need to convert numeric to string. I've got datum with numeric inside, so I'm getting it like: Numeric *numeric = DatumGetNumeric(d); but later I need to have string (most probably: const char *). I've found a couple of different

Re: [HACKERS] converting numeric to string in postgres code

2013-05-28 Thread Pavel Stehule
Hello 2013/5/28 Szymon Guz mabew...@gmail.com: Hi, while hacking on some Postgres code I've found a problem. I need to convert numeric to string. I've got datum with numeric inside, so I'm getting it like: Numeric *numeric = DatumGetNumeric(d); but later I need to have string (most

Re: [HACKERS] converting numeric to string in postgres code

2013-05-28 Thread Szymon Guz
On 28 May 2013 12:07, Pavel Stehule pavel.steh...@gmail.com wrote: Hello 2013/5/28 Szymon Guz mabew...@gmail.com: Hi, while hacking on some Postgres code I've found a problem. I need to convert numeric to string. I've got datum with numeric inside, so I'm getting it like:

Re: [HACKERS] commit fest schedule for 9.4

2013-05-28 Thread Heikki Linnakangas
On 28.05.2013 01:12, Craig Ringer wrote: On 05/16/2013 01:44 AM, Josh Berkus wrote: I'll also say: * we need to assign CF managers at least 2 weeks in advance of each CF * we need to replace them if they get too busy to follow-through, * and the last CF needs two managers. Strong +1 on both of

Re: [HACKERS] plpgsql redesign (related to plpgsql check function)

2013-05-28 Thread Heikki Linnakangas
On 28.05.2013 11:00, Pavel Stehule wrote: Hello all I am searching way how to push our plpgsql_check_function to upstream. One possibility is redesign of plpgsql architecture. Now, we have two stages - compilation and execution, and almost all compilation logic is in gram file. If I

Re: [HACKERS] plpgsql redesign (related to plpgsql check function)

2013-05-28 Thread Pavel Stehule
2013/5/28 Heikki Linnakangas hlinnakan...@vmware.com: On 28.05.2013 11:00, Pavel Stehule wrote: Hello all I am searching way how to push our plpgsql_check_function to upstream. One possibility is redesign of plpgsql architecture. Now, we have two stages - compilation and execution, and

Re: [HACKERS] [PATCH] add --throttle to pgbench (submission 3)

2013-05-28 Thread Fabien COELHO
You can try to use and improve the --progress option in another patch submission which shows how things are going. That'll certainly be useful, but won't solve this issue. The thing is that with asynchronous replication you need to know how long it takes until all nodes are back in sync,

[HACKERS] storing plpython global pointer

2013-05-28 Thread Szymon Guz
Hi, I need to store a global pointer for plpython usage. This is a PyObject* which can be initialized per session I think, as we have to deal with Python 2 and Python 3. This pointer points to a Python constructor of Python's Decimal type, taken from python stdlib. I've got working code, however

Re: [HACKERS] PostgreSQL 9.3 beta breaks some extensions make install

2013-05-28 Thread Cédric Villemain
I just took time to inspect our contribs, USE_PGXS is not supported by all of them atm because of SHLIB_PREREQS (it used submake) I have a patch pending here to fix that. Attached patch fix SHLIB_PREREQS when building with USE_PGXS commit 19e231b introduced SHLIB_PREREQS but failed to port

Re: [HACKERS] storing plpython global pointer

2013-05-28 Thread Jan Urbański
On 28/05/13 14:04, Szymon Guz wrote: Hi, I need to store a global pointer for plpython usage. This is a PyObject* which can be initialized per session I think Where should I keep such a pointer? Hi, you probably could use a global variable, similar to PLy_interp_globals that's defined in

Re: [HACKERS] PostgreSQL 9.3 beta breaks some extensions make install

2013-05-28 Thread Cédric Villemain
Once all our contribs can build with USE_PGXS I fix the VPATH. The last step is interesting: installcheck/REGRESS. For this one, if I can know exactly what's required (for debian build for example), then I can also fix this target. There is a hack to link the regression data files from the

Re: [HACKERS] Unsigned integer types

2013-05-28 Thread Andrew Dunstan
On 05/28/2013 05:17 AM, Maciej Gajewski wrote: I'm afraid that implementing uints as and extension would introduce some performance penalty (I may be wrong). You are. I'm also afraid that with the extension I'd be left on my own maintaining it forever. While if this could go into the core

Re: [HACKERS] storing plpython global pointer

2013-05-28 Thread Szymon Guz
On 28 May 2013 14:15, Jan Urbański wulc...@wulczer.org wrote: On 28/05/13 14:04, Szymon Guz wrote: Hi, I need to store a global pointer for plpython usage. This is a PyObject* which can be initialized per session I think Where should I keep such a pointer? Hi, you probably could use a

Re: [HACKERS] Extent Locks

2013-05-28 Thread Stephen Frost
* Craig Ringer (cr...@2ndquadrant.com) wrote: On 05/17/2013 11:38 AM, Robert Haas wrote: maybe with a bit of modest pre-extension. When it comes to pre-extension, is it realistic to get a count of backends waiting on the lock and extend the relation by (say) 2x the number of waiting

Re: [HACKERS] PostgreSQL 9.3 beta breaks some extensions make install

2013-05-28 Thread Cédric Villemain
Once all our contribs can build with USE_PGXS I fix the VPATH. Attached patch set VPATH for out-of-tree extension builds If the makefile is not in the current directory (where we launch 'make') then assume we are building out-of-src tree and set the VPATH to the directory of the *first*

Re: [HACKERS] PostgreSQL 9.3 beta breaks some extensions make install

2013-05-28 Thread Cédric Villemain
Le mardi 28 mai 2013 14:16:38, Cédric Villemain a écrit : Once all our contribs can build with USE_PGXS I fix the VPATH. The last step is interesting: installcheck/REGRESS. For this one, if I can know exactly what's required (for debian build for example), then I can also fix this

Re: [HACKERS] PostgreSQL 9.3 beta breaks some extensions make install

2013-05-28 Thread Cédric Villemain
Le samedi 25 mai 2013 16:41:24, Cédric Villemain a écrit : If it seems to be on the right way, I'll keep fixing EXTENSION building with VPATH. I haven't tried the patch, but let me just say that Debian (and apt.postgresql.org) would very much like the VPATH situation getting

[HACKERS] preserving forensic information when we freeze

2013-05-28 Thread Robert Haas
Various people, including at least Heikki, Andres, and myself, have proposed various schemes for avoiding freezing that amount to doing it sooner, when we're already writing WAL anyway, or at least when the buffer is already dirty anyway, or at least while the buffer is already in shared_buffers

Re: [HACKERS] Move unused buffers to freelist

2013-05-28 Thread Robert Haas
Instead, I suggest modifying BgBufferSync, specifically this part right here: else if (buffer_state BUF_REUSABLE) reusable_buffers++; What I would suggest is that if the BUF_REUSABLE flag is set here, use that as the trigger to do StrategyMoveBufferToFreeListEnd(). I

Re: [HACKERS] ASYNC Privileges proposal

2013-05-28 Thread Bruce Momjian
On Mon, May 20, 2013 at 02:44:58AM +0100, Chris Farmiloe wrote: Hey all, I find the current LISTEN / NOTIFY rather limited in the context of databases with multiple roles. As it stands it is not possible to restrict the use of LISTEN or NOTIFY to specific roles, and therefore notifications

Re: [HACKERS] Extent Locks

2013-05-28 Thread Jaime Casanova
On Tue, May 28, 2013 at 7:36 AM, Stephen Frost sfr...@snowman.net wrote: On the other hand, I do feel like people are worried about over-extending a relation and wasting disk space- but with the way that vacuum can clean up pages at the end, that would only be a temporary situation anyway.

Re: [HACKERS] Running pgindent

2013-05-28 Thread Bruce Momjian
On Wed, May 22, 2013 at 01:52:28PM -0400, Bruce Momjian wrote: Do we want to run pgindent soon? OK, should I run it this week? Wednesday, 1800 GMT? -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com + It's

Re: [HACKERS] MVCC catalog access

2013-05-28 Thread Robert Haas
On Sun, May 26, 2013 at 9:10 PM, Michael Paquier michael.paqu...@gmail.com wrote: Perhaps we see little difference in performance because PGPROC has been separated into PGPROC and PGXACT, reducing lock contention with getting snapshot data? By the way, I grabbed a 32-core machine and did some

Re: [HACKERS] Extent Locks

2013-05-28 Thread Jaime Casanova
On Tue, May 28, 2013 at 8:38 AM, Jaime Casanova ja...@2ndquadrant.com wrote: We can also think in GUC/reloption for next_extend_blocks so formula is needed, or of course the automated calculation that has been proposed s/so formula is needed/so *no* formula is needed btw, we can also use a

Re: [HACKERS] Running pgindent

2013-05-28 Thread Magnus Hagander
On Tue, May 28, 2013 at 9:48 AM, Robert Haas robertmh...@gmail.com wrote: On Tue, May 28, 2013 at 9:40 AM, Bruce Momjian br...@momjian.us wrote: On Wed, May 22, 2013 at 01:52:28PM -0400, Bruce Momjian wrote: Do we want to run pgindent soon? OK, should I run it this week? Wednesday, 1800 GMT?

Re: [HACKERS] Running pgindent

2013-05-28 Thread Bruce Momjian
On Tue, May 28, 2013 at 09:49:32AM -0400, Magnus Hagander wrote: On Tue, May 28, 2013 at 9:48 AM, Robert Haas robertmh...@gmail.com wrote: On Tue, May 28, 2013 at 9:40 AM, Bruce Momjian br...@momjian.us wrote: On Wed, May 22, 2013 at 01:52:28PM -0400, Bruce Momjian wrote: Do we want to run

Re: [HACKERS] Running pgindent

2013-05-28 Thread Robert Haas
On Tue, May 28, 2013 at 9:40 AM, Bruce Momjian br...@momjian.us wrote: On Wed, May 22, 2013 at 01:52:28PM -0400, Bruce Momjian wrote: Do we want to run pgindent soon? OK, should I run it this week? Wednesday, 1800 GMT? wfm. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The

Re: [HACKERS] fallocate / posix_fallocate for new WAL file creation (etc...)

2013-05-28 Thread Robert Haas
On Sat, May 25, 2013 at 2:55 PM, Jon Nelson jnelson+pg...@jamponi.net wrote: The biggest thing missing from this submission is information about what performance testing you did. Ideally performance patches are submitted with enough information for a reviewer to duplicate the same test the

Re: [HACKERS] [BUGS] COPY .... (FORMAT binary) syntax doesn't work

2013-05-28 Thread Robert Haas
On Mon, May 27, 2013 at 10:31 AM, Tom Lane t...@sss.pgh.pa.us wrote: Simon Riggs si...@2ndquadrant.com writes: On 26 May 2013 17:10, Tom Lane t...@sss.pgh.pa.us wrote: More readable would be to invent an intermediate nonterminal falling between ColId and ColLabel, whose expansion would be

Re: [HACKERS] Extent Locks

2013-05-28 Thread Stephen Frost
* Jaime Casanova (ja...@2ndquadrant.com) wrote: btw, we can also use a next_extend_blocks GUC/reloption as a limit for autovacuum so it will allow that empty pages at the end of the table I'm really not, at all, excited about adding in GUCs for this. We just need to realize when the only

Re: [HACKERS] getting rid of freezing

2013-05-28 Thread Robert Haas
On Sat, May 25, 2013 at 6:14 AM, Simon Riggs si...@2ndquadrant.com wrote: One thought I had is that it might be beneficial to freeze when a page ceases to be all-visible, rather than when it becomes all-visible. Any operation that makes the page not-all-visible is going to emit an FPI anyway,

Re: [HACKERS] fallocate / posix_fallocate for new WAL file creation (etc...)

2013-05-28 Thread Andres Freund
On 2013-05-28 10:03:58 -0400, Robert Haas wrote: On Sat, May 25, 2013 at 2:55 PM, Jon Nelson jnelson+pg...@jamponi.net wrote: The biggest thing missing from this submission is information about what performance testing you did. Ideally performance patches are submitted with enough

Re: [HACKERS] fallocate / posix_fallocate for new WAL file creation (etc...)

2013-05-28 Thread Robert Haas
On Tue, May 28, 2013 at 10:15 AM, Andres Freund and...@2ndquadrant.com wrote: On 2013-05-28 10:03:58 -0400, Robert Haas wrote: On Sat, May 25, 2013 at 2:55 PM, Jon Nelson jnelson+pg...@jamponi.net wrote: The biggest thing missing from this submission is information about what performance

Re: [HACKERS] background worker and normal exit

2013-05-28 Thread Robert Haas
On Sun, May 26, 2013 at 6:48 PM, Michael Paquier michael.paqu...@gmail.com wrote: Hmm so you can't have workers just doing something once and exit? I have to admit, i didn't follow bgworkers closely in the past, but could you give a short insight on why this is currently not possible?

Re: [HACKERS] getting rid of freezing

2013-05-28 Thread Andres Freund
On 2013-05-26 16:58:58 -0700, Josh Berkus wrote: I was talking this over with Jeff on the plane, and we wanted to be clear on your goals here: are you looking to eliminate the *write* cost of freezing, or just the *read* cost of re-reading already frozen pages? Both. The latter is what I have

Re: [HACKERS] background worker and normal exit

2013-05-28 Thread Andres Freund
On 2013-05-28 10:23:46 -0400, Robert Haas wrote: On Sun, May 26, 2013 at 6:48 PM, Michael Paquier - set bgw_restart_time to BGW_NEVER_RESTART. and have the bgworler exit with non-0 status code. That might be good enough, though. I suggested that to Fujii at pgcon, and it seems to work for

Re: [HACKERS] background worker and normal exit

2013-05-28 Thread Robert Haas
On Tue, May 28, 2013 at 10:31 AM, Andres Freund and...@2ndquadrant.com wrote: On 2013-05-28 10:23:46 -0400, Robert Haas wrote: On Sun, May 26, 2013 at 6:48 PM, Michael Paquier - set bgw_restart_time to BGW_NEVER_RESTART. and have the bgworler exit with non-0 status code. That might be

Re: [HACKERS] background worker and normal exit

2013-05-28 Thread Andres Freund
On 2013-05-28 10:33:47 -0400, Robert Haas wrote: On Tue, May 28, 2013 at 10:31 AM, Andres Freund and...@2ndquadrant.com wrote: On 2013-05-28 10:23:46 -0400, Robert Haas wrote: On Sun, May 26, 2013 at 6:48 PM, Michael Paquier - set bgw_restart_time to BGW_NEVER_RESTART. and have the

Re: [HACKERS] PostgreSQL Process memory architecture

2013-05-28 Thread Robert Haas
On Mon, May 27, 2013 at 10:23 AM, Atri Sharma atri.j...@gmail.com wrote: We may still be able to do better than what we're doing today, but I'm still suspicious that you're going to run into other issues with having 500 indexes on a table anyway. +1. I am suspicious that the large number of

Re: [HACKERS] streaming replication, frozen snapshot backup on it and missing relfile (postgres 9.2.3 on xfs + LVM)

2013-05-28 Thread Benedikt Grundmann
Today we have seen 2013-05-28 04:11:12.300 EDT,,,30600,,51a41946.7788,1,,2013-05-27 22:41:10 EDT,,0,ERROR,XX000,xlog flush request 1E95/AFB2DB10 is not satisfied --- flushed only to 1E7E/21CB79A0,writing block 9 of relation base/16416/293974676 2013-05-28 04:11:13.316

Re: [HACKERS] Planning incompatibilities for Postgres 10.0

2013-05-28 Thread Merlin Moncure
On Sat, May 25, 2013 at 11:27 AM, Merlin Moncure mmonc...@gmail.com wrote: On Sat, May 25, 2013 at 4:39 AM, Simon Riggs si...@2ndquadrant.com wrote: There are a number of changes we'd probably like to make to the way things work in Postgres. This thread is not about discussing what those are,

Re: [HACKERS] fallocate / posix_fallocate for new WAL file creation (etc...)

2013-05-28 Thread Jon Nelson
On Tue, May 28, 2013 at 9:19 AM, Robert Haas robertmh...@gmail.com wrote: On Tue, May 28, 2013 at 10:15 AM, Andres Freund and...@2ndquadrant.com wrote: On 2013-05-28 10:03:58 -0400, Robert Haas wrote: On Sat, May 25, 2013 at 2:55 PM, Jon Nelson jnelson+pg...@jamponi.net wrote: The biggest

Re: [HACKERS] PostgreSQL Process memory architecture

2013-05-28 Thread Merlin Moncure
On Mon, May 27, 2013 at 7:29 AM, Stephen Frost sfr...@snowman.net wrote: * Atri Sharma (atri.j...@gmail.com) wrote: Yes, too many indexes wont hurt much.BTW,wont making too many indexes on columns that probably dont have as many values as to deserve them(so,essentially,indiscriminately making

Re: [HACKERS] fallocate / posix_fallocate for new WAL file creation (etc...)

2013-05-28 Thread Andres Freund
On 2013-05-28 10:12:05 -0500, Jon Nelson wrote: On Tue, May 28, 2013 at 9:19 AM, Robert Haas robertmh...@gmail.com wrote: On Tue, May 28, 2013 at 10:15 AM, Andres Freund and...@2ndquadrant.com wrote: On 2013-05-28 10:03:58 -0400, Robert Haas wrote: On Sat, May 25, 2013 at 2:55 PM, Jon

Re: [HACKERS] fallocate / posix_fallocate for new WAL file creation (etc...)

2013-05-28 Thread Greg Smith
On 5/28/13 11:12 AM, Jon Nelson wrote: It opens a new file, fallocates 16MB, calls fdatasync. Outside of the run for performance testing, I think it would be good at this point to validate that there is really a 16MB file full of zeroes resulting from these operations. I am not really

Re: [HACKERS] Planning incompatibilities for Postgres 10.0

2013-05-28 Thread Hannu Krosing
On 05/28/2013 06:13 AM, Joshua D. Drake wrote: On 05/27/2013 06:53 PM, Craig Ringer wrote: On 05/28/2013 09:39 AM, Gavin Flower wrote: Yes, I hate the Firefox style number inflation. I was arguing *for* it ;-) I don't like it much either, but (a) we do about one release a year, not one

[HACKERS] potential bug in JSON

2013-05-28 Thread Szymon Guz
I've found a potential bug. Why the - operator returns JSON instead of TEXT? It doesn't make sens for me, and the documentation doesn't inform about that. postgres=# SELECT ('{id: 1}'::json - 'id')::int; ERROR: cannot cast type json to integer LINE 1: SELECT ('{id: 1}'::json - 'id')::int;

Re: [HACKERS] Extent Locks

2013-05-28 Thread Merlin Moncure
On Tue, May 28, 2013 at 9:07 AM, Stephen Frost sfr...@snowman.net wrote: * Jaime Casanova (ja...@2ndquadrant.com) wrote: btw, we can also use a next_extend_blocks GUC/reloption as a limit for autovacuum so it will allow that empty pages at the end of the table I'm really not, at all, excited

Re: [HACKERS] potential bug in JSON

2013-05-28 Thread Andrew Dunstan
On 05/28/2013 11:38 AM, Szymon Guz wrote: I've found a potential bug. Why the - operator returns JSON instead of TEXT? It doesn't make sens for me, and the documentation doesn't inform about that. postgres=# SELECT ('{id: 1}'::json - 'id')::int; ERROR: cannot cast type json to integer LINE

Re: [HACKERS] Extent Locks

2013-05-28 Thread Andres Freund
On 2013-05-28 10:07:06 -0400, Stephen Frost wrote: * Jaime Casanova (ja...@2ndquadrant.com) wrote: btw, we can also use a next_extend_blocks GUC/reloption as a limit for autovacuum so it will allow that empty pages at the end of the table I'm really not, at all, excited about adding in

Re: [HACKERS] potential bug in JSON

2013-05-28 Thread Josh Berkus
On 05/28/2013 08:38 AM, Szymon Guz wrote: I've found a potential bug. Why the - operator returns JSON instead of TEXT? It doesn't make sens for me, and the documentation doesn't inform about that. Yes, it most certainly does: http://www.postgresql.org/docs/9.3/static/functions-json.html If

Re: [HACKERS] Planning incompatibilities for Postgres 10.0

2013-05-28 Thread Josh Berkus
This argument comes up every couple of years and the people that are trying to solve the problem by changing the versioning are ignoring the fact that there is no problem to solve. We just had this discussion on -advocacy (where it belongs, frankly) a couple months ago:

Re: [HACKERS] Extent Locks

2013-05-28 Thread Stephen Frost
* Andres Freund (and...@2ndquadrant.com) wrote: On 2013-05-28 10:07:06 -0400, Stephen Frost wrote: I'm really not, at all, excited about adding in GUCs for this. But I thought you were in favor of doing complex stuff like mapping segments filled somewhere else into place :P That wouldn't

Re: [HACKERS] Planning incompatibilities for Postgres 10.0

2013-05-28 Thread Robert Haas
On Tue, May 28, 2013 at 11:56 AM, Josh Berkus j...@agliodbs.com wrote: This argument comes up every couple of years and the people that are trying to solve the problem by changing the versioning are ignoring the fact that there is no problem to solve. We just had this discussion on -advocacy

Re: [HACKERS] potential bug in JSON

2013-05-28 Thread Szymon Guz
On 28 May 2013 17:53, Josh Berkus j...@agliodbs.com wrote: On 05/28/2013 08:38 AM, Szymon Guz wrote: I've found a potential bug. Why the - operator returns JSON instead of TEXT? It doesn't make sens for me, and the documentation doesn't inform about that. Yes, it most certainly does:

Re: [HACKERS] Extent Locks

2013-05-28 Thread Jaime Casanova
On Tue, May 28, 2013 at 10:53 AM, Andres Freund and...@2ndquadrant.com wrote: But I agree. This needs to work without much manual intervention. I think we just need to make autovacuum truncate only if it finds more free space than whatever amount we might have added at that relation size plus

Re: [HACKERS] Extent Locks

2013-05-28 Thread Stephen Frost
* Jaime Casanova (ja...@2ndquadrant.com) wrote: On Tue, May 28, 2013 at 10:53 AM, Andres Freund and...@2ndquadrant.com wrote: But I agree. This needs to work without much manual intervention. I think we just need to make autovacuum truncate only if it finds more free space than whatever

[HACKERS] GRANT role_name TO role_name ON database_name

2013-05-28 Thread Clark C. Evans
I'd really love the ability to grant a *user* role-based privileges database by database. For background, I have several databases running in a single cluster, one database per business unit. Each database has the same core schema with the same basic role permissions, but with significant

Re: [HACKERS] Planning incompatibilities for Postgres 10.0

2013-05-28 Thread Joshua D. Drake
On 05/28/2013 08:36 AM, Hannu Krosing wrote: The conversation does not change. Further, we are not Firefox. We are not user software. We are developer software. At least some of the real-world problems with PostgreSQL comes from We are developer software mentality. Yes, We are developer

Re: [HACKERS] XLogInsert scaling, revisited

2013-05-28 Thread Alvaro Herrera
Heikki Linnakangas wrote: I've been slowly continuing to work that I started last winder to make XLogInsert scale better. I have tried quite a few different approaches since then, and have settled on the attached. This is similar but not exactly the same as what I did in the patches I posted

[HACKERS] all_visible replay aborting due to uninitialized pages

2013-05-28 Thread Andres Freund
Hi, A customer of ours reporting a standby loosing sync with the primary due to the following error: CONTEXT: xlog redo visible: rel 1663/XXX/XXX; blk 173717 WARNING: page 173717 of relation base/XXX/XXX is uninitialized ... PANIC: WAL contains references to invalid pages Guessing around I

Re: [HACKERS] pg_rewind, a tool for resynchronizing an old master after failover

2013-05-28 Thread Bruce Momjian
On Thu, May 23, 2013 at 01:48:24PM -0400, Heikki Linnakangas wrote: On 23.05.2013 08:03, Simon Riggs wrote: On 23 May 2013 12:10, Heikki Linnakangashlinnakan...@vmware.com wrote: Please take a look: https://github.com/vmware/pg_rewind The COPYRIGHT file shows that VMware is claiming

Re: [HACKERS] pg_rewind, a tool for resynchronizing an old master after failover

2013-05-28 Thread Andres Freund
On 2013-05-28 14:32:07 -0400, Bruce Momjian wrote: We have a lot of code in PostgreSQL source tree with different copyright notices, and there's no problem with that as long as the coe is licensed under the PostgreSQL license. For patches that add Really? Where? I think we have removed

Re: [HACKERS] pg_rewind, a tool for resynchronizing an old master after failover

2013-05-28 Thread Bruce Momjian
On Tue, May 28, 2013 at 08:37:44PM +0200, Andres Freund wrote: On 2013-05-28 14:32:07 -0400, Bruce Momjian wrote: We have a lot of code in PostgreSQL source tree with different copyright notices, and there's no problem with that as long as the coe is licensed under the PostgreSQL

Re: [HACKERS] pg_rewind, a tool for resynchronizing an old master after failover

2013-05-28 Thread Andres Freund
On 2013-05-28 14:50:57 -0400, Bruce Momjian wrote: On Tue, May 28, 2013 at 08:37:44PM +0200, Andres Freund wrote: On 2013-05-28 14:32:07 -0400, Bruce Momjian wrote: We have a lot of code in PostgreSQL source tree with different copyright notices, and there's no problem with that as long

Re: [HACKERS] pg_rewind, a tool for resynchronizing an old master after failover

2013-05-28 Thread Alvaro Herrera
Bruce Momjian wrote: Oh, I see. Have we historically been OK with these as long as it is clear it is the PG copyright? I know we had do some cleanups in the past, but I don't remember the details, obviously. We've had request from companies because they wanted to distribute Postgres and

Re: [HACKERS] pg_rewind, a tool for resynchronizing an old master after failover

2013-05-28 Thread Christophe Pettus
On May 28, 2013, at 12:49 PM, Alvaro Herrera wrote: We've had request from companies because they wanted to distribute Postgres and lawyers weren't comfortable with copyright statements in assorted files. In those cases we've asked the people mentioned in such copyright statements, got

Re: [HACKERS] pg_rewind, a tool for resynchronizing an old master after failover

2013-05-28 Thread Bruce Momjian
On Tue, May 28, 2013 at 03:49:14PM -0400, Alvaro Herrera wrote: Bruce Momjian wrote: Oh, I see. Have we historically been OK with these as long as it is clear it is the PG copyright? I know we had do some cleanups in the past, but I don't remember the details, obviously. We've had

[HACKERS] [PATCH] Fix conversion for Decimal arguments in plpython functions

2013-05-28 Thread Szymon Guz
Hi, I've got a patch. This is for a plpython enhancement. There is an item at the TODO list http://wiki.postgresql.org/wiki/Todo#Server-Side_Languages Fix loss of information during conversion of numeric type to Python float This patch uses a decimal.Decimal type from Python standard library

Re: [HACKERS] Unsigned integer types

2013-05-28 Thread Bruce Momjian
On Tue, May 28, 2013 at 11:17:42AM +0200, Maciej Gajewski wrote: 2. INTEGER I had to store a record with several uint32. I had to store an awful lot of them; hundreds GB of data per day. Roughly half of the record consists of uint32 fields. Increasing the data type to bigint would mean that

Re: [HACKERS] Planning incompatibilities for Postgres 10.0

2013-05-28 Thread Bruce Momjian
On Tue, May 28, 2013 at 07:58:33AM +0800, Craig Ringer wrote: On 05/28/2013 12:41 AM, Simon Riggs wrote: I'm happy with that. I was also thinking about collecting changes not related just to disk format, if any exist. Any wire protocol or syntax changes? I can't seem to find a things

Re: [HACKERS] Planning incompatibilities for Postgres 10.0

2013-05-28 Thread Bruce Momjian
On Mon, May 27, 2013 at 05:21:16PM -0700, Joshua D. Drake wrote: On 05/27/2013 04:58 PM, Craig Ringer wrote: On 05/28/2013 12:41 AM, Simon Riggs wrote: I'm happy with that. I was also thinking about collecting changes not related just to disk format, if any exist. Any wire protocol or

Re: [HACKERS] Planning incompatibilities for Postgres 10.0

2013-05-28 Thread Bruce Momjian
On Mon, May 27, 2013 at 02:09:05PM -0400, Tom Lane wrote: Bruce Momjian br...@momjian.us writes: On Mon, May 27, 2013 at 09:17:50AM -0400, Bruce Momjian wrote: Yes, we should be collecting things we want to do for a pg_upgrade break so we can see the list all in one place. OK, I have

Re: [HACKERS] Planning incompatibilities for Postgres 10.0

2013-05-28 Thread Joshua D. Drake
On 05/28/2013 02:18 PM, Bruce Momjian wrote: I would like to see the ability to define if a query is read only at the protocol level, so that load balances that speak libpq can know what to do with the query without parsing it. Sounds nice, but how would we do that? That would require libpq

[HACKERS] FIX: auto_explain docs

2013-05-28 Thread Tomas Vondra
Hi, I've just noticed that this patch in 2012-01 commitfest https://commitfest.postgresql.org/action/patch_view?id=729 added log_timing option to auto_explain, but it never actually made it to the docs. Attached is a patch for current master, but 9.2 should be patched too. regards Tomas diff

Re: [HACKERS] Planning incompatibilities for Postgres 10.0

2013-05-28 Thread Alvaro Herrera
Bruce Momjian wrote: On Mon, May 27, 2013 at 05:21:16PM -0700, Joshua D. Drake wrote: I would like to see the ability to define if a query is read only at the protocol level, so that load balances that speak libpq can know what to do with the query without parsing it. Sounds nice, but

Re: [HACKERS] pg_dump with postgis extension dumps rules separately

2013-05-28 Thread Joe Conway
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 04/06/2013 04:49 PM, Joe Conway wrote: If I create a database and install postgis as an extension, and then run pg_dump I get this: [...] CREATE EXTENSION IF NOT EXISTS postgis WITH SCHEMA public; [...] CREATE RULE geometry_columns_delete AS

Re: [HACKERS] Planning incompatibilities for Postgres 10.0

2013-05-28 Thread Bruce Momjian
On Tue, May 28, 2013 at 02:26:06PM -0700, Joshua D. Drake wrote: Sounds nice, but how would we do that? That would require libpq to know it, right? Do we pass anything back after parsing but before execution? Could it be optional? What about functions that modify the database --- isn't

Re: [HACKERS] Planning incompatibilities for Postgres 10.0

2013-05-28 Thread Bruce Momjian
On Mon, May 27, 2013 at 03:06:13PM -0400, Alvaro Herrera wrote: Bruce Momjian wrote: OK, I have added a section to the TODO list for this: Desired changes that would prevent upgrades with pg_upgrade 32-bit page checksums Are there any others? I would have

Re: [HACKERS] Planning incompatibilities for Postgres 10.0

2013-05-28 Thread Joshua D. Drake
On 05/28/2013 03:36 PM, Bruce Momjian wrote: The other option would be to do it on query execute but that doesn't seem as efficient as it would have to be parsed each time. Although it would still be better than reading the actual SQL. Well, you could do SET TRANSACTION READ ONLY, and that

Re: [HACKERS] Logging of PAM Authentication Failure

2013-05-28 Thread Bruce Momjian
On Tue, May 28, 2013 at 01:32:53PM +0800, Craig Ringer wrote: On 05/11/2013 03:25 AM, Robert Haas wrote: Not really. We could potentially fix it by extending the wire protocol to allow the server to respond to the client's startup packet with a further challenge, and extend libpq to report

Re: [HACKERS] Unsigned integer types

2013-05-28 Thread Jim Nasby
On 5/28/13 4:07 PM, Bruce Momjian wrote: On Tue, May 28, 2013 at 11:17:42AM +0200, Maciej Gajewski wrote: 2. INTEGER I had to store a record with several uint32. I had to store an awful lot of them; hundreds GB of data per day. Roughly half of the record consists of uint32 fields. Increasing

Re: [HACKERS] Unsigned integer types

2013-05-28 Thread David Johnston
Maciej Gajewski wrote I'm also afraid that with the extension I'd be left on my own maintaining it forever. While if this could go into the core product, it would live forever. Clarification from the gallery: are we talking an extension or a custom PostgreSQL build/fork? If it is an extension

Re: [HACKERS] Unsigned integer types

2013-05-28 Thread Bruce Momjian
On Tue, May 28, 2013 at 05:57:41PM -0500, Jim Nasby wrote: On 5/28/13 4:07 PM, Bruce Momjian wrote: On Tue, May 28, 2013 at 11:17:42AM +0200, Maciej Gajewski wrote: 2. INTEGER I had to store a record with several uint32. I had to store an awful lot of them; hundreds GB of data per day.

Re: [HACKERS] Planning incompatibilities for Postgres 10.0

2013-05-28 Thread Bruce Momjian
On Tue, May 28, 2013 at 03:39:10PM -0700, Joshua D. Drake wrote: On 05/28/2013 03:36 PM, Bruce Momjian wrote: The other option would be to do it on query execute but that doesn't seem as efficient as it would have to be parsed each time. Although it would still be better than reading the

Re: [HACKERS] preserving forensic information when we freeze

2013-05-28 Thread Josh Berkus
On 05/28/2013 06:21 AM, Robert Haas wrote: As a general statement, I view this work as something that is likely needed no matter which one of the remove freezing approaches that have been proposed we choose to adopt. It does not fix anything in and of itself, but it (hopefully) removes an

Re: [HACKERS] getting rid of freezing

2013-05-28 Thread Josh Berkus
On 05/28/2013 07:17 AM, Andres Freund wrote: On 2013-05-26 16:58:58 -0700, Josh Berkus wrote: I was talking this over with Jeff on the plane, and we wanted to be clear on your goals here: are you looking to eliminate the *write* cost of freezing, or just the *read* cost of re-reading already

Re: [HACKERS] Unsigned integer types

2013-05-28 Thread Andrew Dunstan
On 05/28/2013 07:00 PM, Bruce Momjian wrote: On Tue, May 28, 2013 at 05:57:41PM -0500, Jim Nasby wrote: Did you try 'oid' as an unsigned int4? Using an internal catalog type for user data seems like a horrible idea to me... Uh, not sure if we can say oid is only an internal catalog type.

Re: [HACKERS] getting rid of freezing

2013-05-28 Thread Andres Freund
On 2013-05-28 09:29:26 -0700, Josh Berkus wrote: On 05/28/2013 07:17 AM, Andres Freund wrote: On 2013-05-26 16:58:58 -0700, Josh Berkus wrote: I was talking this over with Jeff on the plane, and we wanted to be clear on your goals here: are you looking to eliminate the *write* cost of

Re: [HACKERS] preserving forensic information when we freeze

2013-05-28 Thread Andres Freund
On 2013-05-28 09:39:13 -0700, Josh Berkus wrote: On 05/28/2013 06:21 AM, Robert Haas wrote: As a general statement, I view this work as something that is likely needed no matter which one of the remove freezing approaches that have been proposed we choose to adopt. It does not fix anything

Re: [HACKERS] streaming replication, frozen snapshot backup on it and missing relfile (postgres 9.2.3 on xfs + LVM)

2013-05-28 Thread Robert Haas
On Tue, May 28, 2013 at 10:53 AM, Benedikt Grundmann bgrundm...@janestreet.com wrote: Today we have seen 2013-05-28 04:11:12.300 EDT,,,30600,,51a41946.7788,1,,2013-05-27 22:41:10 EDT,,0,ERROR,XX000,xlog flush request 1E95/AFB2DB10 is not satisfied --- flushed only to 1E7E/21CB79A0,writing

Re: [HACKERS] preserving forensic information when we freeze

2013-05-28 Thread Robert Haas
On Tue, May 28, 2013 at 7:27 PM, Andres Freund and...@2ndquadrant.com wrote: On 2013-05-28 09:39:13 -0700, Josh Berkus wrote: On 05/28/2013 06:21 AM, Robert Haas wrote: As a general statement, I view this work as something that is likely needed no matter which one of the remove freezing

  1   2   >