Re: [HACKERS] [GENERAL] pg_upgrade -u

2013-05-28 Thread Joshua D. Drake
On 05/28/2013 07:55 PM, Bruce Momjian wrote: Perhaps just documenting the behavior is all that is needed, but -U is everywhere and I think that's a good thing. [ moved to hacker ] Wow, I never realized other tools used -U for user, instead of -u. Should I change pg_upgrade to use -U for 9.4?

Re: [HACKERS] Planning incompatibilities for Postgres 10.0

2013-05-28 Thread Alvaro Herrera
Bruce Momjian wrote: > On Mon, May 27, 2013 at 03:06:13PM -0400, Alvaro Herrera wrote: > > Bruce Momjian wrote: > > I would have each data segment be self-identifying, i.e. have a magic > > number at the beginning of the file and the relation OID, some fork > > identification and the segment numbe

Re: [HACKERS] [GENERAL] pg_upgrade -u

2013-05-28 Thread Bruce Momjian
On Wed, May 22, 2013 at 03:05:57PM -0400, Ray Stell wrote: > > However, if we pass these items into the scripts, we then force > > these values to be used, even if the user wants to use a different > > value. It is a balance between supplying defaults vs. requiring the > > user to supply or change

Re: [HACKERS] fallocate / posix_fallocate for new WAL file creation (etc...)

2013-05-28 Thread Greg Smith
On 5/28/13 10:00 PM, Jon Nelson wrote: On Tue, May 28, 2013 at 10:36 AM, Greg Smith wrote: On 5/28/13 11:12 AM, Jon Nelson wrote: It opens a new file, fallocates 16MB, calls fdatasync. Outside of the run for performance testing, I think it would be good at this point to validate that there

Re: [HACKERS] preserving forensic information when we freeze

2013-05-28 Thread Andres Freund
On 2013-05-28 21:26:49 -0400, Robert Haas wrote: > On Tue, May 28, 2013 at 8:00 PM, Andres Freund wrote: > > I only suggested MOVED_IN/OFF because I didn't remember > > HEAP_XMIN_COMMITTED|HEAP_XMIN_INVALID was still free ;). So, unless that > > combination could have been produced in the past in

Re: [HACKERS] fallocate / posix_fallocate for new WAL file creation (etc...)

2013-05-28 Thread Jon Nelson
On Tue, May 28, 2013 at 10:36 AM, Greg Smith wrote: > On 5/28/13 11:12 AM, Jon Nelson wrote: >> >> It opens a new file, fallocates 16MB, calls fdatasync. > > > Outside of the run for performance testing, I think it would be good at this > point to validate that there is really a 16MB file full of

Re: [HACKERS] all_visible replay aborting due to uninitialized pages

2013-05-28 Thread Andres Freund
On 2013-05-28 21:36:17 -0400, Robert Haas wrote: > On Tue, May 28, 2013 at 1:58 PM, Andres Freund wrote: > > Guessing around I looked and noticed the following problematic pattern: > > 1) A: wants to do an update, doesn't have enough freespace > > 2) A: extends the relation on the filesystem level

[HACKERS] pg_stat_replication when standby is unreachable

2013-05-28 Thread Abhishek Rai
Hello Postgres gurus, I'm writing a thin clustering layer on top of Postgres using the synchronous replication feature. The goal is to enable HA and survive permanent loss of a single node. Using an external coordinator (Zookeeper), one of the nodes is elected as the primary. The primary node t

Re: [HACKERS] all_visible replay aborting due to uninitialized pages

2013-05-28 Thread Robert Haas
On Tue, May 28, 2013 at 1:58 PM, Andres Freund wrote: > Guessing around I looked and noticed the following problematic pattern: > 1) A: wants to do an update, doesn't have enough freespace > 2) A: extends the relation on the filesystem level (RelationGetBufferForTuple) > 3) A: does PageInit (Relat

Re: [HACKERS] preserving forensic information when we freeze

2013-05-28 Thread Robert Haas
On Tue, May 28, 2013 at 8:00 PM, Andres Freund wrote: > I only suggested MOVED_IN/OFF because I didn't remember > HEAP_XMIN_COMMITTED|HEAP_XMIN_INVALID was still free ;). So, unless that > combination could have been produced in the past in a way I couldn't > find so far, I am all for it. I don't

Re: [HACKERS] Planning incompatibilities for Postgres 10.0

2013-05-28 Thread Tatsuo Ishii
> On 05/28/2013 04:05 PM, Bruce Momjian wrote: >> >> On Tue, May 28, 2013 at 03:39:10PM -0700, Joshua D. Drake wrote: >>> >>> On 05/28/2013 03:36 PM, Bruce Momjian wrote: >>> > The other option would be to do it on query execute but that doesn't > seem as efficient as it would have to be pa

Re: [HACKERS] Planning incompatibilities for Postgres 10.0

2013-05-28 Thread Jaime Casanova
On Tue, May 28, 2013 at 4:26 PM, Joshua D. Drake wrote: > > On 05/28/2013 02:18 PM, Bruce Momjian wrote: > >>> I would like to see the ability to define if a query is read only at >>> the protocol level, so that load balances that speak libpq can know >>> what to do with the query without parsing

Re: [HACKERS] Planning incompatibilities for Postgres 10.0

2013-05-28 Thread Craig Ringer
On 05/29/2013 05:11 AM, Bruce Momjian wrote: > Sure, it is on the TODO list: > > https://wiki.postgresql.org/wiki/Todo#.2Fcontrib.2Fpg_upgrade > > I can only get a link to pg_upgrade from there, so look two sections > below that for "Wire Protocol Changes". Thanks. The direct link is https:

Re: [HACKERS] Logging of PAM Authentication Failure

2013-05-28 Thread Craig Ringer
On 05/28/2013 04:04 PM, Amit Langote wrote: > On Tue, May 28, 2013 at 2:32 PM, Craig Ringer wrote: >> On 05/11/2013 03:25 AM, Robert Haas wrote: >>> Not really. We could potentially fix it by extending the wire >>> protocol to allow the server to respond to the client's startup packet >>> with a

Re: [HACKERS] commit fest schedule for 9.4

2013-05-28 Thread Josh Berkus
> Shared responsibility is no-one's responsibility. If we are to have > multiple CF managers, I think it would be good to have one who's mainly > responsible, and the second one's job is to nag the first manager if > ernothing happens, and quickly take over if necessary. Ie. a hot standby > arrang

Re: [HACKERS] preserving forensic information when we freeze

2013-05-28 Thread Andres Freund
On 2013-05-28 09:21:27 -0400, Robert Haas wrote: > I have attempted to implement this. Trouble is, we're out of infomask > bits. Using an infomask2 bit might work but we don't have many of > them left either, so it's worth casting about a bit for a better > solution. Andres proposed using HEAP_

Re: [HACKERS] getting rid of freezing

2013-05-28 Thread Robert Haas
On Tue, May 28, 2013 at 12:29 PM, Josh Berkus wrote: > On 05/28/2013 07:17 AM, Andres Freund wrote: >> On 2013-05-26 16:58:58 -0700, Josh Berkus wrote: >>> I was talking this over with Jeff on the plane, and we wanted to be >>> clear on your goals here: are you looking to eliminate the *write* co

Re: [HACKERS] Planning incompatibilities for Postgres 10.0

2013-05-28 Thread Joshua D. Drake
On 05/28/2013 04:05 PM, Bruce Momjian wrote: On Tue, May 28, 2013 at 03:39:10PM -0700, Joshua D. Drake wrote: On 05/28/2013 03:36 PM, Bruce Momjian wrote: The other option would be to do it on query execute but that doesn't seem as efficient as it would have to be parsed each time. Although

Re: [HACKERS] preserving forensic information when we freeze

2013-05-28 Thread Robert Haas
On Tue, May 28, 2013 at 7:27 PM, Andres Freund wrote: > On 2013-05-28 09:39:13 -0700, Josh Berkus wrote: >> On 05/28/2013 06:21 AM, Robert Haas wrote: >> > As a general statement, I view this work as something that is likely >> > needed no matter which one of the "remove freezing" approaches that

Re: [HACKERS] streaming replication, "frozen snapshot backup on it" and missing relfile (postgres 9.2.3 on xfs + LVM)

2013-05-28 Thread Robert Haas
On Tue, May 28, 2013 at 10:53 AM, Benedikt Grundmann wrote: > Today we have seen > > 2013-05-28 04:11:12.300 EDT,,,30600,,51a41946.7788,1,,2013-05-27 22:41:10 > EDT,,0,ERROR,XX000,"xlog flush request 1E95/AFB2DB10 is not satisfied --- > flushed only to 1E7E/21CB79A0","writing block 9 of relati

Re: [HACKERS] preserving forensic information when we freeze

2013-05-28 Thread Andres Freund
On 2013-05-28 09:39:13 -0700, Josh Berkus wrote: > On 05/28/2013 06:21 AM, Robert Haas wrote: > > As a general statement, I view this work as something that is likely > > needed no matter which one of the "remove freezing" approaches that > > have been proposed we choose to adopt. It does not fix

Re: [HACKERS] getting rid of freezing

2013-05-28 Thread Andres Freund
On 2013-05-28 09:29:26 -0700, Josh Berkus wrote: > On 05/28/2013 07:17 AM, Andres Freund wrote: > > On 2013-05-26 16:58:58 -0700, Josh Berkus wrote: > >> I was talking this over with Jeff on the plane, and we wanted to be > >> clear on your goals here: are you looking to eliminate the *write* cost

Re: [HACKERS] Unsigned integer types

2013-05-28 Thread Andrew Dunstan
On 05/28/2013 07:00 PM, Bruce Momjian wrote: On Tue, May 28, 2013 at 05:57:41PM -0500, Jim Nasby wrote: Did you try 'oid' as an unsigned int4? Using an internal catalog type for user data seems like a horrible idea to me... Uh, not sure if we can say oid is only an internal catalog type. It

Re: [HACKERS] getting rid of freezing

2013-05-28 Thread Josh Berkus
On 05/28/2013 07:17 AM, Andres Freund wrote: > On 2013-05-26 16:58:58 -0700, Josh Berkus wrote: >> I was talking this over with Jeff on the plane, and we wanted to be >> clear on your goals here: are you looking to eliminate the *write* cost >> of freezing, or just the *read* cost of re-reading al

Re: [HACKERS] preserving forensic information when we freeze

2013-05-28 Thread Josh Berkus
On 05/28/2013 06:21 AM, Robert Haas wrote: > As a general statement, I view this work as something that is likely > needed no matter which one of the "remove freezing" approaches that > have been proposed we choose to adopt. It does not fix anything in > and of itself, but it (hopefully) removes a

Re: [HACKERS] Planning incompatibilities for Postgres 10.0

2013-05-28 Thread Bruce Momjian
On Tue, May 28, 2013 at 03:39:10PM -0700, Joshua D. Drake wrote: > > On 05/28/2013 03:36 PM, Bruce Momjian wrote: > > >>The other option would be to do it on query execute but that doesn't > >>seem as efficient as it would have to be parsed each time. Although > >>it would still be better than re

Re: [HACKERS] Unsigned integer types

2013-05-28 Thread Bruce Momjian
On Tue, May 28, 2013 at 05:57:41PM -0500, Jim Nasby wrote: > On 5/28/13 4:07 PM, Bruce Momjian wrote: > >On Tue, May 28, 2013 at 11:17:42AM +0200, Maciej Gajewski wrote: > >>2. INTEGER > >> > >>I had to store a record with several uint32. I had to store an awful > >>lot of them; hundreds GB of data

Re: [HACKERS] Unsigned integer types

2013-05-28 Thread David Johnston
Maciej Gajewski wrote > I'm also afraid that with > the extension I'd be left on my own maintaining it forever. While if > this could go into the core product, it would live forever. Clarification from the gallery: are we talking an extension or a custom PostgreSQL build/fork? If it is an extensi

Re: [HACKERS] Unsigned integer types

2013-05-28 Thread Jim Nasby
On 5/28/13 4:07 PM, Bruce Momjian wrote: On Tue, May 28, 2013 at 11:17:42AM +0200, Maciej Gajewski wrote: 2. INTEGER I had to store a record with several uint32. I had to store an awful lot of them; hundreds GB of data per day. Roughly half of the record consists of uint32 fields. Increasing th

Re: [HACKERS] Logging of PAM Authentication Failure

2013-05-28 Thread Bruce Momjian
On Tue, May 28, 2013 at 01:32:53PM +0800, Craig Ringer wrote: > On 05/11/2013 03:25 AM, Robert Haas wrote: > > Not really. We could potentially fix it by extending the wire > > protocol to allow the server to respond to the client's startup packet > > with a further challenge, and extend libpq to

Re: [HACKERS] Planning incompatibilities for Postgres 10.0

2013-05-28 Thread Joshua D. Drake
On 05/28/2013 03:36 PM, Bruce Momjian wrote: The other option would be to do it on query execute but that doesn't seem as efficient as it would have to be parsed each time. Although it would still be better than reading the actual SQL. Well, you could do SET TRANSACTION READ ONLY, and that wo

Re: [HACKERS] Planning incompatibilities for Postgres 10.0

2013-05-28 Thread Bruce Momjian
On Mon, May 27, 2013 at 03:06:13PM -0400, Alvaro Herrera wrote: > Bruce Momjian wrote: > > > OK, I have added a section to the TODO list for this: > > > > Desired changes that would prevent upgrades with pg_upgrade > > > > 32-bit page checksums > > > > Are there any others? >

Re: [HACKERS] Planning incompatibilities for Postgres 10.0

2013-05-28 Thread Bruce Momjian
On Tue, May 28, 2013 at 02:26:06PM -0700, Joshua D. Drake wrote: > >Sounds nice, but how would we do that? That would require libpq to know > >it, right? Do we pass anything back after parsing but before execution? > > Could it be optional? What about functions that modify the database > >--- i

Re: [HACKERS] pg_dump with postgis extension dumps rules separately

2013-05-28 Thread Joe Conway
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 04/06/2013 04:49 PM, Joe Conway wrote: > If I create a database and install postgis as an extension, and > then run pg_dump I get this: > > [...] CREATE EXTENSION IF NOT EXISTS postgis WITH SCHEMA public; > [...] CREATE RULE geometry_columns_delet

Re: [HACKERS] Planning incompatibilities for Postgres 10.0

2013-05-28 Thread Alvaro Herrera
Bruce Momjian wrote: > On Mon, May 27, 2013 at 05:21:16PM -0700, Joshua D. Drake wrote: > > I would like to see the ability to define if a query is read only at > > the protocol level, so that load balances that speak libpq can know > > what to do with the query without parsing it. > > Sounds nic

[HACKERS] FIX: auto_explain docs

2013-05-28 Thread Tomas Vondra
Hi, I've just noticed that this patch in 2012-01 commitfest https://commitfest.postgresql.org/action/patch_view?id=729 added log_timing option to auto_explain, but it never actually made it to the docs. Attached is a patch for current master, but 9.2 should be patched too. regards Tomas diff

Re: [HACKERS] Planning incompatibilities for Postgres 10.0

2013-05-28 Thread Joshua D. Drake
On 05/28/2013 02:18 PM, Bruce Momjian wrote: I would like to see the ability to define if a query is read only at the protocol level, so that load balances that speak libpq can know what to do with the query without parsing it. Sounds nice, but how would we do that? That would require libpq

Re: [HACKERS] Planning incompatibilities for Postgres 10.0

2013-05-28 Thread Bruce Momjian
On Mon, May 27, 2013 at 02:09:05PM -0400, Tom Lane wrote: > Bruce Momjian writes: > > On Mon, May 27, 2013 at 09:17:50AM -0400, Bruce Momjian wrote: > >> Yes, we should be collecting things we want to do for a pg_upgrade break > >> so we can see the list all in one place. > > > OK, I have added a

Re: [HACKERS] Planning incompatibilities for Postgres 10.0

2013-05-28 Thread Bruce Momjian
On Mon, May 27, 2013 at 05:21:16PM -0700, Joshua D. Drake wrote: > > On 05/27/2013 04:58 PM, Craig Ringer wrote: > > > >On 05/28/2013 12:41 AM, Simon Riggs wrote: > >>I'm happy with that. > >> > >>I was also thinking about collecting changes not related just to disk > >>format, if any exist. > >An

Re: [HACKERS] Planning incompatibilities for Postgres 10.0

2013-05-28 Thread Bruce Momjian
On Tue, May 28, 2013 at 07:58:33AM +0800, Craig Ringer wrote: > On 05/28/2013 12:41 AM, Simon Riggs wrote: > > I'm happy with that. > > > > I was also thinking about collecting changes not related just to disk > > format, if any exist. > Any wire protocol or syntax changes? > > I can't seem to fin

Re: [HACKERS] Unsigned integer types

2013-05-28 Thread Bruce Momjian
On Tue, May 28, 2013 at 11:17:42AM +0200, Maciej Gajewski wrote: > 2. INTEGER > > I had to store a record with several uint32. I had to store an awful > lot of them; hundreds GB of data per day. Roughly half of the record > consists of uint32 fields. > Increasing the data type to bigint would mean

[HACKERS] [PATCH] Fix conversion for Decimal arguments in plpython functions

2013-05-28 Thread Szymon Guz
Hi, I've got a patch. This is for a plpython enhancement. There is an item at the TODO list http://wiki.postgresql.org/wiki/Todo#Server-Side_Languages "Fix loss of information during conversion of numeric type to Python float" This patch uses a decimal.Decimal type from Python standard library f

Re: [HACKERS] pg_rewind, a tool for resynchronizing an old master after failover

2013-05-28 Thread Bruce Momjian
On Tue, May 28, 2013 at 03:49:14PM -0400, Alvaro Herrera wrote: > Bruce Momjian wrote: > > > Oh, I see. Have we historically been OK with these as long as it is > > clear it is the PG copyright? I know we had do some cleanups in the > > past, but I don't remember the details, obviously. > > We'

Re: [HACKERS] pg_rewind, a tool for resynchronizing an old master after failover

2013-05-28 Thread Christophe Pettus
On May 28, 2013, at 12:49 PM, Alvaro Herrera wrote: > We've had request from companies because they wanted to distribute > Postgres and lawyers weren't comfortable with copyright statements in > assorted files. In those cases we've asked the people mentioned in such > copyright statements, got a

Re: [HACKERS] pg_rewind, a tool for resynchronizing an old master after failover

2013-05-28 Thread Alvaro Herrera
Bruce Momjian wrote: > Oh, I see. Have we historically been OK with these as long as it is > clear it is the PG copyright? I know we had do some cleanups in the > past, but I don't remember the details, obviously. We've had request from companies because they wanted to distribute Postgres and l

Re: [HACKERS] pg_rewind, a tool for resynchronizing an old master after failover

2013-05-28 Thread Andres Freund
On 2013-05-28 14:50:57 -0400, Bruce Momjian wrote: > On Tue, May 28, 2013 at 08:37:44PM +0200, Andres Freund wrote: > > On 2013-05-28 14:32:07 -0400, Bruce Momjian wrote: > > > > We have a lot of code in PostgreSQL source tree with different > > > > copyright notices, and there's no problem with th

Re: [HACKERS] pg_rewind, a tool for resynchronizing an old master after failover

2013-05-28 Thread Bruce Momjian
On Tue, May 28, 2013 at 08:37:44PM +0200, Andres Freund wrote: > On 2013-05-28 14:32:07 -0400, Bruce Momjian wrote: > > > We have a lot of code in PostgreSQL source tree with different > > > copyright notices, and there's no problem with that as long as the > > > coe is licensed under the PostgreSQ

Re: [HACKERS] pg_rewind, a tool for resynchronizing an old master after failover

2013-05-28 Thread Andres Freund
On 2013-05-28 14:32:07 -0400, Bruce Momjian wrote: > > We have a lot of code in PostgreSQL source tree with different > > copyright notices, and there's no problem with that as long as the > > coe is licensed under the PostgreSQL license. For patches that add > > Really? Where? I think we have re

Re: [HACKERS] pg_rewind, a tool for resynchronizing an old master after failover

2013-05-28 Thread Bruce Momjian
On Thu, May 23, 2013 at 01:48:24PM -0400, Heikki Linnakangas wrote: > On 23.05.2013 08:03, Simon Riggs wrote: > >On 23 May 2013 12:10, Heikki Linnakangas wrote: > > > >>Please take a look: https://github.com/vmware/pg_rewind > > > >The COPYRIGHT file shows that VMware is claiming copyright on unst

[HACKERS] all_visible replay aborting due to uninitialized pages

2013-05-28 Thread Andres Freund
Hi, A customer of ours reporting a standby loosing sync with the primary due to the following error: CONTEXT: xlog redo visible: rel 1663/XXX/XXX; blk 173717 WARNING: page 173717 of relation base/XXX/XXX is uninitialized ... PANIC: WAL contains references to invalid pages Guessing around I loo

Re: [HACKERS] XLogInsert scaling, revisited

2013-05-28 Thread Alvaro Herrera
Heikki Linnakangas wrote: > I've been slowly continuing to work that I started last winder to > make XLogInsert scale better. I have tried quite a few different > approaches since then, and have settled on the attached. This is > similar but not exactly the same as what I did in the patches I > pos

Re: [HACKERS] Planning incompatibilities for Postgres 10.0

2013-05-28 Thread Joshua D. Drake
On 05/28/2013 08:36 AM, Hannu Krosing wrote: The conversation does not change. Further, we are not Firefox. We are not user software. We are developer software. At least some of the real-world problems with PostgreSQL comes from "We are developer software" mentality. Yes, We are developer so

[HACKERS] GRANT role_name TO role_name ON database_name

2013-05-28 Thread Clark C. Evans
I'd really love the ability to grant a *user* role-based privileges database by database. For background, I have several databases running in a single cluster, one database per business unit. Each database has the same core schema with the same basic role permissions, but with significant cu

Re: [HACKERS] Extent Locks

2013-05-28 Thread Stephen Frost
* Jaime Casanova (ja...@2ndquadrant.com) wrote: > On Tue, May 28, 2013 at 10:53 AM, Andres Freund > wrote: > > But I agree. This needs to work without much manual intervention. I > > think we just need to make autovacuum truncate only if it finds more > > free space than whatever amount we might

Re: [HACKERS] Extent Locks

2013-05-28 Thread Jaime Casanova
On Tue, May 28, 2013 at 10:53 AM, Andres Freund wrote: > > But I agree. This needs to work without much manual intervention. I > think we just need to make autovacuum truncate only if it finds more > free space than whatever amount we might have added at that relation > size plus some slop. > And

Re: [HACKERS] potential bug in JSON

2013-05-28 Thread Szymon Guz
On 28 May 2013 17:53, Josh Berkus wrote: > On 05/28/2013 08:38 AM, Szymon Guz wrote: > > I've found a potential bug. Why the "->" operator returns JSON instead of > > TEXT? It doesn't make sens for me, and the documentation doesn't inform > > about that. > > Yes, it most certainly does: > http://

Re: [HACKERS] Planning incompatibilities for Postgres 10.0

2013-05-28 Thread Robert Haas
On Tue, May 28, 2013 at 11:56 AM, Josh Berkus wrote: > >>> This argument comes up every couple of years and the people that >>> are trying to solve the problem by changing the versioning are >>> ignoring the fact that there is no problem to solve. > > We just had this discussion on -advocacy (wher

Re: [HACKERS] Extent Locks

2013-05-28 Thread Stephen Frost
* Andres Freund (and...@2ndquadrant.com) wrote: > On 2013-05-28 10:07:06 -0400, Stephen Frost wrote: > > I'm really not, at all, excited about adding in GUCs for this. > > But I thought you were in favor of doing complex stuff like mapping > segments filled somewhere else into place :P That would

Re: [HACKERS] Planning incompatibilities for Postgres 10.0

2013-05-28 Thread Josh Berkus
>> This argument comes up every couple of years and the people that >> are trying to solve the problem by changing the versioning are >> ignoring the fact that there is no problem to solve. We just had this discussion on -advocacy (where it belongs, frankly) a couple months ago: http://www.postg

Re: [HACKERS] potential bug in JSON

2013-05-28 Thread Josh Berkus
On 05/28/2013 08:38 AM, Szymon Guz wrote: > I've found a potential bug. Why the "->" operator returns JSON instead of > TEXT? It doesn't make sens for me, and the documentation doesn't inform > about that. Yes, it most certainly does: http://www.postgresql.org/docs/9.3/static/functions-json.html

Re: [HACKERS] Extent Locks

2013-05-28 Thread Andres Freund
On 2013-05-28 10:07:06 -0400, Stephen Frost wrote: > * Jaime Casanova (ja...@2ndquadrant.com) wrote: > > btw, we can also use a next_extend_blocks GUC/reloption as a limit for > > autovacuum so it will allow that empty pages at the end of the table > > I'm really not, at all, excited about adding

Re: [HACKERS] potential bug in JSON

2013-05-28 Thread Andrew Dunstan
On 05/28/2013 11:38 AM, Szymon Guz wrote: I've found a potential bug. Why the "->" operator returns JSON instead of TEXT? It doesn't make sens for me, and the documentation doesn't inform about that. postgres=# SELECT ('{"id": 1}'::json -> 'id')::int; ERROR: cannot cast type json to integer

Re: [HACKERS] Extent Locks

2013-05-28 Thread Merlin Moncure
On Tue, May 28, 2013 at 9:07 AM, Stephen Frost wrote: > * Jaime Casanova (ja...@2ndquadrant.com) wrote: >> btw, we can also use a next_extend_blocks GUC/reloption as a limit for >> autovacuum so it will allow that empty pages at the end of the table > > I'm really not, at all, excited about adding

[HACKERS] potential bug in JSON

2013-05-28 Thread Szymon Guz
I've found a potential bug. Why the "->" operator returns JSON instead of TEXT? It doesn't make sens for me, and the documentation doesn't inform about that. postgres=# SELECT ('{"id": 1}'::json -> 'id')::int; ERROR: cannot cast type json to integer LINE 1: SELECT ('{"id": 1}'::json -> 'id')::int

Re: [HACKERS] Planning incompatibilities for Postgres 10.0

2013-05-28 Thread Hannu Krosing
On 05/28/2013 06:13 AM, Joshua D. Drake wrote: > > On 05/27/2013 06:53 PM, Craig Ringer wrote: >> >> On 05/28/2013 09:39 AM, Gavin Flower wrote: >>> Yes, I hate the Firefox style number inflation. >> I was arguing *for* it ;-) >> >> I don't like it much either, but (a) we do about one release a yea

Re: [HACKERS] fallocate / posix_fallocate for new WAL file creation (etc...)

2013-05-28 Thread Greg Smith
On 5/28/13 11:12 AM, Jon Nelson wrote: It opens a new file, fallocates 16MB, calls fdatasync. Outside of the run for performance testing, I think it would be good at this point to validate that there is really a 16MB file full of zeroes resulting from these operations. I am not really concer

Re: [HACKERS] fallocate / posix_fallocate for new WAL file creation (etc...)

2013-05-28 Thread Andres Freund
On 2013-05-28 10:12:05 -0500, Jon Nelson wrote: > On Tue, May 28, 2013 at 9:19 AM, Robert Haas wrote: > > On Tue, May 28, 2013 at 10:15 AM, Andres Freund > > wrote: > >> On 2013-05-28 10:03:58 -0400, Robert Haas wrote: > >>> On Sat, May 25, 2013 at 2:55 PM, Jon Nelson > >>> wrote: > >>> >> The

Re: [HACKERS] PostgreSQL Process memory architecture

2013-05-28 Thread Merlin Moncure
On Mon, May 27, 2013 at 7:29 AM, Stephen Frost wrote: > * Atri Sharma (atri.j...@gmail.com) wrote: >> Yes, too many indexes wont hurt much.BTW,wont making too many indexes >> on columns that probably dont have as many values as to deserve >> them(so,essentially,indiscriminately making indexes) hur

Re: [HACKERS] fallocate / posix_fallocate for new WAL file creation (etc...)

2013-05-28 Thread Jon Nelson
On Tue, May 28, 2013 at 9:19 AM, Robert Haas wrote: > On Tue, May 28, 2013 at 10:15 AM, Andres Freund > wrote: >> On 2013-05-28 10:03:58 -0400, Robert Haas wrote: >>> On Sat, May 25, 2013 at 2:55 PM, Jon Nelson >>> wrote: >>> >> The biggest thing missing from this submission is information abo

Re: [HACKERS] Planning incompatibilities for Postgres 10.0

2013-05-28 Thread Merlin Moncure
On Sat, May 25, 2013 at 11:27 AM, Merlin Moncure wrote: > On Sat, May 25, 2013 at 4:39 AM, Simon Riggs wrote: >> There are a number of changes we'd probably like to make to the way >> things work in Postgres. This thread is not about discussing what >> those are, just to say that requirements exi

Re: [HACKERS] streaming replication, "frozen snapshot backup on it" and missing relfile (postgres 9.2.3 on xfs + LVM)

2013-05-28 Thread Benedikt Grundmann
Today we have seen 2013-05-28 04:11:12.300 EDT,,,30600,,51a41946.7788,1,,2013-05-27 22:41:10 EDT,,0,ERROR,XX000,"xlog flush request 1E95/AFB2DB10 is not satisfied --- flushed only to 1E7E/21CB79A0","writing block 9 of relation base/16416/293974676""" 2013-05-28 04:11:13.316 EDT,,,30600,,51

Re: [HACKERS] PostgreSQL Process memory architecture

2013-05-28 Thread Robert Haas
On Mon, May 27, 2013 at 10:23 AM, Atri Sharma wrote: > >We may still be able to do better than what we're doing >> today, but I'm still suspicious that you're going to run into other >> issues with having 500 indexes on a table anyway. > > +1. I am suspicious that the large number of indexes is

Re: [HACKERS] background worker and normal exit

2013-05-28 Thread Andres Freund
On 2013-05-28 10:33:47 -0400, Robert Haas wrote: > On Tue, May 28, 2013 at 10:31 AM, Andres Freund > wrote: > > On 2013-05-28 10:23:46 -0400, Robert Haas wrote: > >> On Sun, May 26, 2013 at 6:48 PM, Michael Paquier > >> > - set bgw_restart_time to BGW_NEVER_RESTART. and have the bgworler exit >

Re: [HACKERS] background worker and normal exit

2013-05-28 Thread Robert Haas
On Tue, May 28, 2013 at 10:31 AM, Andres Freund wrote: > On 2013-05-28 10:23:46 -0400, Robert Haas wrote: >> On Sun, May 26, 2013 at 6:48 PM, Michael Paquier >> > - set bgw_restart_time to BGW_NEVER_RESTART. and have the bgworler exit >> > with >> > non-0 status code. >> >> That might be good eno

Re: [HACKERS] background worker and normal exit

2013-05-28 Thread Andres Freund
On 2013-05-28 10:23:46 -0400, Robert Haas wrote: > On Sun, May 26, 2013 at 6:48 PM, Michael Paquier > > - set bgw_restart_time to BGW_NEVER_RESTART. and have the bgworler exit with > > non-0 status code. > > That might be good enough, though. I suggested that to Fujii at pgcon, and it seems to wo

Re: [HACKERS] getting rid of freezing

2013-05-28 Thread Andres Freund
On 2013-05-26 16:58:58 -0700, Josh Berkus wrote: > I was talking this over with Jeff on the plane, and we wanted to be > clear on your goals here: are you looking to eliminate the *write* cost > of freezing, or just the *read* cost of re-reading already frozen pages? Both. The latter is what I ha

Re: [HACKERS] background worker and normal exit

2013-05-28 Thread Robert Haas
On Sun, May 26, 2013 at 6:48 PM, Michael Paquier wrote: >> Hmm so you can't have workers just "doing something once" and exit? I have >> to admit, i didn't follow bgworkers closely in the past, but could you give >> a short insight on why this is currently not possible? > > Bgworkers are expected

Re: [HACKERS] fallocate / posix_fallocate for new WAL file creation (etc...)

2013-05-28 Thread Robert Haas
On Tue, May 28, 2013 at 10:15 AM, Andres Freund wrote: > On 2013-05-28 10:03:58 -0400, Robert Haas wrote: >> On Sat, May 25, 2013 at 2:55 PM, Jon Nelson >> wrote: >> >> The biggest thing missing from this submission is information about what >> >> performance testing you did. Ideally performanc

Re: [HACKERS] fallocate / posix_fallocate for new WAL file creation (etc...)

2013-05-28 Thread Andres Freund
On 2013-05-28 10:03:58 -0400, Robert Haas wrote: > On Sat, May 25, 2013 at 2:55 PM, Jon Nelson wrote: > >> The biggest thing missing from this submission is information about what > >> performance testing you did. Ideally performance patches are submitted > >> with > >> enough information for a

Re: [HACKERS] getting rid of freezing

2013-05-28 Thread Robert Haas
On Sat, May 25, 2013 at 6:14 AM, Simon Riggs wrote: >> One thought I had is that it might be beneficial to freeze when a page >> ceases to be all-visible, rather than when it becomes all-visible. >> Any operation that makes the page not-all-visible is going to emit an >> FPI anyway, so we don't ha

Re: [HACKERS] Extent Locks

2013-05-28 Thread Stephen Frost
* Jaime Casanova (ja...@2ndquadrant.com) wrote: > btw, we can also use a next_extend_blocks GUC/reloption as a limit for > autovacuum so it will allow that empty pages at the end of the table I'm really not, at all, excited about adding in GUCs for this. We just need to realize when the only avai

Re: [HACKERS] [BUGS] COPY .... (FORMAT binary) syntax doesn't work

2013-05-28 Thread Robert Haas
On Mon, May 27, 2013 at 10:31 AM, Tom Lane wrote: > Simon Riggs writes: >> On 26 May 2013 17:10, Tom Lane wrote: >>> More readable would be to invent an intermediate nonterminal falling >>> between ColId and ColLabel, whose expansion would be "IDENT | >>> unreserved_keyword | col_name_keyword |

Re: [HACKERS] fallocate / posix_fallocate for new WAL file creation (etc...)

2013-05-28 Thread Robert Haas
On Sat, May 25, 2013 at 2:55 PM, Jon Nelson wrote: >> The biggest thing missing from this submission is information about what >> performance testing you did. Ideally performance patches are submitted with >> enough information for a reviewer to duplicate the same test the author did, >> as well

Re: [HACKERS] Running pgindent

2013-05-28 Thread Robert Haas
On Tue, May 28, 2013 at 9:40 AM, Bruce Momjian wrote: > On Wed, May 22, 2013 at 01:52:28PM -0400, Bruce Momjian wrote: >> Do we want to run pgindent soon? > > OK, should I run it this week? Wednesday, 1800 GMT? wfm. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise Postgr

Re: [HACKERS] Running pgindent

2013-05-28 Thread Bruce Momjian
On Tue, May 28, 2013 at 09:49:32AM -0400, Magnus Hagander wrote: > On Tue, May 28, 2013 at 9:48 AM, Robert Haas wrote: > > On Tue, May 28, 2013 at 9:40 AM, Bruce Momjian wrote: > >> On Wed, May 22, 2013 at 01:52:28PM -0400, Bruce Momjian wrote: > >>> Do we want to run pgindent soon? > >> > >> OK,

Re: [HACKERS] Running pgindent

2013-05-28 Thread Magnus Hagander
On Tue, May 28, 2013 at 9:48 AM, Robert Haas wrote: > On Tue, May 28, 2013 at 9:40 AM, Bruce Momjian wrote: >> On Wed, May 22, 2013 at 01:52:28PM -0400, Bruce Momjian wrote: >>> Do we want to run pgindent soon? >> >> OK, should I run it this week? Wednesday, 1800 GMT? > > wfm. +1. -- Magnus H

Re: [HACKERS] Extent Locks

2013-05-28 Thread Jaime Casanova
On Tue, May 28, 2013 at 8:38 AM, Jaime Casanova wrote: > > We can also think in GUC/reloption for next_extend_blocks so formula > is needed, or of course the automated calculation that has been > proposed > s/so formula is needed/so *no* formula is needed btw, we can also use a next_extend_block

Re: [HACKERS] Running pgindent

2013-05-28 Thread Bruce Momjian
On Wed, May 22, 2013 at 01:52:28PM -0400, Bruce Momjian wrote: > Do we want to run pgindent soon? OK, should I run it this week? Wednesday, 1800 GMT? -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everyth

Re: [HACKERS] MVCC catalog access

2013-05-28 Thread Robert Haas
On Sun, May 26, 2013 at 9:10 PM, Michael Paquier wrote: > Perhaps we see little difference in performance because PGPROC has been > separated into PGPROC and PGXACT, reducing lock contention with getting > snapshot data? > > By the way, I grabbed a 32-core machine and did some more performance tes

Re: [HACKERS] Extent Locks

2013-05-28 Thread Jaime Casanova
On Tue, May 28, 2013 at 7:36 AM, Stephen Frost wrote: > > On the other hand, I do feel like people are worried about > over-extending a relation and wasting disk space- but with the way that > vacuum can clean up pages at the end, that would only be a temporary > situation anyway. > Hi, Maybe i'

Re: [HACKERS] ASYNC Privileges proposal

2013-05-28 Thread Bruce Momjian
On Mon, May 20, 2013 at 02:44:58AM +0100, Chris Farmiloe wrote: > Hey all, > > I find the current LISTEN / NOTIFY rather limited in the context of databases > with multiple roles. As it stands it is not possible to restrict the use of > LISTEN or NOTIFY to specific roles, and therefore notificatio

Re: [HACKERS] Move unused buffers to freelist

2013-05-28 Thread Robert Haas
>> Instead, I suggest modifying BgBufferSync, specifically this part right >> here: >> >> else if (buffer_state & BUF_REUSABLE) >> reusable_buffers++; >> >> What I would suggest is that if the BUF_REUSABLE flag is set here, use >> that as the trigger to do StrategyMoveBufferToFr

[HACKERS] preserving forensic information when we freeze

2013-05-28 Thread Robert Haas
Various people, including at least Heikki, Andres, and myself, have proposed various schemes for avoiding freezing that amount to doing it sooner, when we're already writing WAL anyway, or at least when the buffer is already dirty anyway, or at least while the buffer is already in shared_buffers an

Re: [HACKERS] PostgreSQL 9.3 beta breaks some extensions "make install"

2013-05-28 Thread Cédric Villemain
Le samedi 25 mai 2013 16:41:24, Cédric Villemain a écrit : > > > If it seems to be on the right way, I'll keep fixing EXTENSION building > > > with VPATH. > > > > I haven't tried the patch, but let me just say that Debian (and > > apt.postgresql.org) would very much like the VPATH situation gettin

Re: [HACKERS] PostgreSQL 9.3 beta breaks some extensions "make install"

2013-05-28 Thread Cédric Villemain
Le mardi 28 mai 2013 14:16:38, Cédric Villemain a écrit : > > Once all our contribs can build with USE_PGXS I > > fix the VPATH. > > > > The last step is interesting: installcheck/REGRESS. For this one, if I > > can know exactly what's required (for debian build for example), then I > > can also f

Re: [HACKERS] PostgreSQL 9.3 beta breaks some extensions "make install"

2013-05-28 Thread Cédric Villemain
> Once all our contribs can build with USE_PGXS I > fix the VPATH. Attached patch set VPATH for out-of-tree extension builds If the makefile is not in the current directory (where we launch 'make') then assume we are building out-of-src tree and set the VPATH to the directory of the *first* makef

Re: [HACKERS] Extent Locks

2013-05-28 Thread Stephen Frost
* Craig Ringer (cr...@2ndquadrant.com) wrote: > On 05/17/2013 11:38 AM, Robert Haas wrote: > > maybe with a bit of modest pre-extension. > When it comes to pre-extension, is it realistic to get a count of > backends waiting on the lock and extend the relation by (say) 2x the > number of waiting ba

Re: [HACKERS] Unsigned integer types

2013-05-28 Thread Andrew Dunstan
On 05/28/2013 05:17 AM, Maciej Gajewski wrote: I'm afraid that implementing uints as and extension would introduce some performance penalty (I may be wrong). You are. I'm also afraid that with the extension I'd be left on my own maintaining it forever. While if this could go into the core pr

Re: [HACKERS] storing plpython global pointer

2013-05-28 Thread Szymon Guz
On 28 May 2013 14:15, Jan Urbański wrote: > On 28/05/13 14:04, Szymon Guz wrote: > >> Hi, >> I need to store a global pointer for plpython usage. This is a PyObject* >> which can be initialized per session I think >> >> Where should I keep such a pointer? >> > > Hi, > > you probably could use a g

Re: [HACKERS] PostgreSQL 9.3 beta breaks some extensions "make install"

2013-05-28 Thread Cédric Villemain
> Once all our contribs can build with USE_PGXS I > fix the VPATH. > > The last step is interesting: installcheck/REGRESS. For this one, if I can > know exactly what's required (for debian build for example), then I can > also fix this target. There is a hack to link the regression data files fro

  1   2   >