Re: [HACKERS] 9.3: Empty arrays returned by array_remove()

2013-05-31 Thread Brendan Jurd
On 31 May 2013 02:52, Dean Rasheed dean.a.rash...@gmail.com wrote: Testing 9.3beta, it seems that array_remove() may return an empty 1-d array whose upper bound is lower than its lower bound. I know that we discussed allowing this kind of array, but I don't think that discussion reached any

Re: [HACKERS] [PATCH] add --throttle to pgbench (submission 3)

2013-05-31 Thread Fabien COELHO
However I'm not sure that pg_stat_replication currently has the necessary information on either side to measure the lag (in time transactions, but how do I know when a transaction was committed? or number of transactions?). The BDR codebase now has a handy function to report when a

Re: [HACKERS] 9.3: Empty arrays returned by array_remove()

2013-05-31 Thread Dean Rasheed
On 31 May 2013 08:34, Brendan Jurd dire...@gmail.com wrote: On 31 May 2013 02:52, Dean Rasheed dean.a.rash...@gmail.com wrote: Testing 9.3beta, it seems that array_remove() may return an empty 1-d array whose upper bound is lower than its lower bound. I know that we discussed allowing this

Re: [HACKERS] Freezing without write I/O

2013-05-31 Thread Heikki Linnakangas
On 31.05.2013 00:06, Bruce Momjian wrote: On Thu, May 30, 2013 at 04:33:50PM +0300, Heikki Linnakangas wrote: This would also be the first step in allowing the clog to grow larger than 2 billion transactions, eliminating the need for anti-wraparound freezing altogether. You'd still want to

Re: [HACKERS] Behavior of a pg_trgm index for 2 (or 3) character LIKE queries

2013-05-31 Thread Sawada Masahiko
On Fri, May 31, 2013 at 11:16 AM, Amit Langote amitlangot...@gmail.com wrote: On Fri, May 31, 2013 at 4:25 AM, Alexander Korotkov aekorot...@gmail.com wrote: On Thu, May 30, 2013 at 12:49 PM, Sawada Masahiko sawada.m...@gmail.com wrote: following emails are discussed about partial match of

Re: [HACKERS] removing PD_ALL_VISIBLE

2013-05-31 Thread Bruce Momjian
On Thu, May 30, 2013 at 09:47:22AM -0400, Robert Haas wrote: Well, as Heikki points out, I think that's unacceptably dangerous. Loss or corruption of a single visibility map page means possible loss of half a gigabyte of data. Also, if we go that route, looking at the visibility map is no

Re: [HACKERS] Unsigned integer types

2013-05-31 Thread David E. Wheeler
On May 29, 2013, at 10:48 AM, Fabien COELHO coe...@cri.ensmp.fr wrote: If you do it, having uint1 (1 byte) would be nice as well. There is a signed 1byte int on PGXN, FWIW: http://pgxn.org/extension/tinyint Best, David -- Sent via pgsql-hackers mailing list

Re: [HACKERS] removing PD_ALL_VISIBLE

2013-05-31 Thread Andres Freund
On 2013-05-31 13:14:13 -0400, Bruce Momjian wrote: On Thu, May 30, 2013 at 09:47:22AM -0400, Robert Haas wrote: Well, as Heikki points out, I think that's unacceptably dangerous. Loss or corruption of a single visibility map page means possible loss of half a gigabyte of data. Also, if

Re: [HACKERS] Freezing without write I/O

2013-05-31 Thread Bruce Momjian
On Thu, May 30, 2013 at 10:04:23PM -0400, Robert Haas wrote: Hm. Why? If freezing gets notably cheaper I don't really see much need for keeping that much clog around? If we still run into anti-wraparound areas, there has to be some major operational problem. That is true, but we have a

Re: [HACKERS] removing PD_ALL_VISIBLE

2013-05-31 Thread Josh Berkus
Isn't the visibility map already required for proper return results as we use it for index-only scans. I think the optimization-only ship has sailed. At the moment we can remove it without causing corruption. If we were to use it for freezing we couldn't anymore. So there's a difference -

[HACKERS] detecting binary backup in progress

2013-05-31 Thread Joe Conway
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 I can check for the presence of $PGDATA/backup_label in order to detect a backup in progress (i.e. pg_start_backup() has been run and pg_stop_backup() has not yet been run). However there is a period of time after pg_start_backup() is first executed

Re: [HACKERS] Behavior of a pg_trgm index for 2 (or 3) character LIKE queries

2013-05-31 Thread Amit Langote
On Sat, Jun 1, 2013 at 1:48 AM, Sawada Masahiko sawada.m...@gmail.com wrote: On Fri, May 31, 2013 at 11:16 AM, Amit Langote amitlangot...@gmail.com wrote: 2) And if that is so, is there problem in gin_extract_query_trgm(), that is while generating trigrams from a query search term that causes

Re: [HACKERS] detecting binary backup in progress

2013-05-31 Thread Andres Freund
On 2013-05-31 10:38:56 -0700, Joe Conway wrote: I can check for the presence of $PGDATA/backup_label in order to detect a backup in progress (i.e. pg_start_backup() has been run and pg_stop_backup() has not yet been run). However there is a period of time after pg_start_backup() is first

Re: [HACKERS] detecting binary backup in progress

2013-05-31 Thread Thom Brown
On 31 May 2013 18:38, Joe Conway m...@joeconway.com wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 I can check for the presence of $PGDATA/backup_label in order to detect a backup in progress (i.e. pg_start_backup() has been run and pg_stop_backup() has not yet been run). However

Re: [HACKERS] removing PD_ALL_VISIBLE

2013-05-31 Thread Bruce Momjian
On Fri, May 31, 2013 at 10:28:12AM -0700, Josh Berkus wrote: Isn't the visibility map already required for proper return results as we use it for index-only scans. I think the optimization-only ship has sailed. At the moment we can remove it without causing corruption. If we were to

Re: [HACKERS] detecting binary backup in progress

2013-05-31 Thread Heikki Linnakangas
On 31.05.2013 20:38, Joe Conway wrote: I can check for the presence of $PGDATA/backup_label in order to detect a backup in progress (i.e. pg_start_backup() has been run and pg_stop_backup() has not yet been run). However there is a period of time after pg_start_backup() is first executed to

Re: [HACKERS] Behavior of a pg_trgm index for 2 (or 3) character LIKE queries

2013-05-31 Thread Alexander Korotkov
On Fri, May 31, 2013 at 9:41 PM, Amit Langote amitlangot...@gmail.comwrote: On Sat, Jun 1, 2013 at 1:48 AM, Sawada Masahiko sawada.m...@gmail.com wrote: On Fri, May 31, 2013 at 11:16 AM, Amit Langote amitlangot...@gmail.com wrote: 2) And if that is so, is there problem in

Re: [HACKERS] removing PD_ALL_VISIBLE

2013-05-31 Thread Josh Berkus
Bruce, Roberts statement was: Loss or corruption of a single visibility map page means possible loss of half a gigabyte of data. I fail to be alarmed at this; currently losing a single page of the clog causes just as widespread corruption (worse, actually, since it's not confined to one

Re: [HACKERS] removing PD_ALL_VISIBLE

2013-05-31 Thread Bruce Momjian
On Fri, May 31, 2013 at 11:00:19AM -0700, Josh Berkus wrote: Bruce, Roberts statement was: Loss or corruption of a single visibility map page means possible loss of half a gigabyte of data. I fail to be alarmed at this; currently losing a single page of the clog causes just as

Re: [HACKERS] detecting binary backup in progress

2013-05-31 Thread Joe Conway
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 05/31/2013 10:42 AM, Andres Freund wrote: On 2013-05-31 10:38:56 -0700, Joe Conway wrote: I can check for the presence of $PGDATA/backup_label in order to detect a backup in progress (i.e. pg_start_backup() has been run and pg_stop_backup() has

Re: [HACKERS] detecting binary backup in progress

2013-05-31 Thread Joe Conway
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 05/31/2013 10:46 AM, Heikki Linnakangas wrote: On 31.05.2013 20:38, Joe Conway wrote: I can check for the presence of $PGDATA/backup_label in order to detect a backup in progress (i.e. pg_start_backup() has been run and pg_stop_backup() has not

Re: [HACKERS] removing PD_ALL_VISIBLE

2013-05-31 Thread Stephen Frost
* Bruce Momjian (br...@momjian.us) wrote: Right, and it is hard to see that losing and adding are somehow more/less likely, so it seems we already realy on the visibility map being correct. Yes, but there's also a way to get *back* to a valid state if things go south with the visibility map.

Re: [HACKERS] detecting binary backup in progress

2013-05-31 Thread Heikki Linnakangas
On 31.05.2013 21:41, Joe Conway wrote: On 05/31/2013 10:46 AM, Heikki Linnakangas wrote: On 31.05.2013 20:38, Joe Conway wrote: I can check for the presence of $PGDATA/backup_label in order to detect a backup in progress (i.e. pg_start_backup() has been run and pg_stop_backup() has not yet

Re: [HACKERS] detecting binary backup in progress

2013-05-31 Thread Joe Conway
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 05/31/2013 12:29 PM, Heikki Linnakangas wrote: Note that pg_is_in_backup() just checks for presence of $PGDATA/backup_label. Also note that pg_basebackup doesn't create backup_label in the server. It's included in the backup that's sent to

Re: [HACKERS] detecting binary backup in progress

2013-05-31 Thread Andres Freund
On 2013-05-31 22:29:45 +0300, Heikki Linnakangas wrote: On 31.05.2013 21:41, Joe Conway wrote: On 05/31/2013 10:46 AM, Heikki Linnakangas wrote: On 31.05.2013 20:38, Joe Conway wrote: I can check for the presence of $PGDATA/backup_label in order to detect a backup in progress (i.e.

Re: [HACKERS] detecting binary backup in progress

2013-05-31 Thread Joe Conway
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 05/31/2013 12:36 PM, Andres Freund wrote: On 2013-05-31 22:29:45 +0300, Heikki Linnakangas wrote: Note that pg_is_in_backup() just checks for presence of $PGDATA/backup_label. Also note that pg_basebackup doesn't create backup_label in the

Re: [HACKERS] removing PD_ALL_VISIBLE

2013-05-31 Thread Josh Berkus
On 05/31/2013 12:22 PM, Stephen Frost wrote: Where I'm going with this whole thing is simply that I do worry a bit about using a bitmap for freeze, or similar, information and not being able to reconstruct that bitmap from the heap. Perhaps that's overly paranoid, but, well, we also write the

Re: [HACKERS] Running pgindent

2013-05-31 Thread Andrew Dunstan
On 05/29/2013 11:41 PM, Bruce Momjian wrote: On Wed, May 29, 2013 at 10:08:10PM -0400, Stephen Frost wrote: * Bruce Momjian (br...@momjian.us) wrote: Wow, uh, yeah, I guess we could do that. I will await more feedback. Please don't. I'm already rather concerned by this one. It looks like

Re: [HACKERS] detecting binary backup in progress

2013-05-31 Thread Heikki Linnakangas
On 31.05.2013 22:36, Andres Freund wrote: On 2013-05-31 22:29:45 +0300, Heikki Linnakangas wrote: Note that pg_is_in_backup() just checks for presence of $PGDATA/backup_label. Also note that pg_basebackup doesn't create backup_label in the server. It's included in the backup that's sent to the

Re: [HACKERS] detecting binary backup in progress

2013-05-31 Thread Alvaro Herrera
Heikki Linnakangas wrote: The manual says: pg_is_in_backup()boolTrue if an on-line exclusive backup is still in progress. So clearly that is intentional. That could use some rephrasing, though; a layman won't know what an exclusive backup is. Heck, I don't understand what it is

Re: [HACKERS] detecting binary backup in progress

2013-05-31 Thread Andres Freund
On 2013-05-31 22:53:14 +0300, Heikki Linnakangas wrote: On 31.05.2013 22:36, Andres Freund wrote: On 2013-05-31 22:29:45 +0300, Heikki Linnakangas wrote: Note that pg_is_in_backup() just checks for presence of $PGDATA/backup_label. Also note that pg_basebackup doesn't create backup_label in

Re: [HACKERS] Running pgindent

2013-05-31 Thread Bruce Momjian
On Fri, May 31, 2013 at 03:51:30PM -0400, Andrew Dunstan wrote: On 05/29/2013 11:41 PM, Bruce Momjian wrote: On Wed, May 29, 2013 at 10:08:10PM -0400, Stephen Frost wrote: * Bruce Momjian (br...@momjian.us) wrote: Wow, uh, yeah, I guess we could do that. I will await more feedback. Please

Re: [HACKERS] Unsigned integer types

2013-05-31 Thread Fabien COELHO
If you do it, having uint1 (1 byte) would be nice as well. There is a signed 1byte int on PGXN, FWIW: http://pgxn.org/extension/tinyint That's good, thanks for the pointer! However, it is a signed tinyint (-128..127 range), not an unsigned one. -- Fabien. -- Sent via pgsql-hackers

Re: [HACKERS] removing PD_ALL_VISIBLE

2013-05-31 Thread Stephen Frost
* Josh Berkus (j...@agliodbs.com) wrote: On 05/31/2013 12:22 PM, Stephen Frost wrote: Where I'm going with this whole thing is simply that I do worry a bit about using a bitmap for freeze, or similar, information and not being able to reconstruct that bitmap from the heap. Perhaps that's

Re: [HACKERS] Running pgindent

2013-05-31 Thread Stephen Frost
* Bruce Momjian (br...@momjian.us) wrote: On Fri, May 31, 2013 at 03:51:30PM -0400, Andrew Dunstan wrote: I spent quite a lot of time trying to make the tool behave the same as the old script. Yes, and I believe we tested running the Perl version to make sure it was the same, so the

Re: [HACKERS] Running pgindent

2013-05-31 Thread Bruce Momjian
On Fri, May 31, 2013 at 04:57:20PM -0400, Stephen Frost wrote: * Bruce Momjian (br...@momjian.us) wrote: On Fri, May 31, 2013 at 03:51:30PM -0400, Andrew Dunstan wrote: I spent quite a lot of time trying to make the tool behave the same as the old script. Yes, and I believe we tested

Re: [HACKERS] 9.3: Empty arrays returned by array_remove()

2013-05-31 Thread Noah Misch
On Fri, May 31, 2013 at 08:55:49AM +0100, Dean Rasheed wrote: On 31 May 2013 08:34, Brendan Jurd dire...@gmail.com wrote: On 31 May 2013 02:52, Dean Rasheed dean.a.rash...@gmail.com wrote: Testing 9.3beta, it seems that array_remove() may return an empty 1-d array whose upper bound is lower

Re: [HACKERS] fallocate / posix_fallocate for new WAL file creation (etc...)

2013-05-31 Thread Noah Misch
On Thu, May 30, 2013 at 02:58:26PM +0200, Andres Freund wrote: * Andres Freund (and...@2ndquadrant.com) wrote: But really, I am not at all concerned about some obscure values being returned, but about a read() not being successful.. After a bit of standard perusing writing a single byte

Re: [HACKERS] removing PD_ALL_VISIBLE

2013-05-31 Thread Robert Haas
On Fri, May 31, 2013 at 1:44 PM, Bruce Momjian br...@momjian.us wrote: On Fri, May 31, 2013 at 10:28:12AM -0700, Josh Berkus wrote: Isn't the visibility map already required for proper return results as we use it for index-only scans. I think the optimization-only ship has sailed. At

Re: [HACKERS] removing PD_ALL_VISIBLE

2013-05-31 Thread Robert Haas
On Fri, May 31, 2013 at 3:45 PM, Josh Berkus j...@agliodbs.com wrote: On 05/31/2013 12:22 PM, Stephen Frost wrote: Where I'm going with this whole thing is simply that I do worry a bit about using a bitmap for freeze, or similar, information and not being able to reconstruct that bitmap from

Re: [HACKERS] Freezing without write I/O

2013-05-31 Thread Robert Haas
On Fri, May 31, 2013 at 1:26 PM, Bruce Momjian br...@momjian.us wrote: On Thu, May 30, 2013 at 10:04:23PM -0400, Robert Haas wrote: Hm. Why? If freezing gets notably cheaper I don't really see much need for keeping that much clog around? If we still run into anti-wraparound areas, there has

Re: [HACKERS] extensible external toast tuple support

2013-05-31 Thread Robert Haas
On Thu, May 30, 2013 at 7:42 AM, Andres Freund and...@2ndquadrant.com wrote: In http://archives.postgresql.org/message-id/20130216164231.GA15069%40awork2.anarazel.de I presented the need for 'indirect' toast tuples which point into memory instead of a toast table. In the comments to that

Re: [HACKERS] pg_dump with postgis extension dumps rules separately

2013-05-31 Thread Robert Haas
On Wed, May 29, 2013 at 6:55 PM, Joe Conway m...@joeconway.com wrote: OK, simple enough. New patch attached. I still need to do some testing to verify this does not break anything, but other than that, any complaints (including the notion of backpatching this back to 9.1)? Here's a cleaned