Re: [HACKERS] Confusing comment in xlog.c or am I missing something?

2013-05-02 Thread Amit Langote
/* Check for crossing of xlog segment boundary */ if (RecPtr-xrecoff= XLogFileSize) { (RecPtr-xlogid)++; RecPtr-xrecoff = 0; } Is that xlog file boundary or am I missing something? The WAL space is divided into 4GB log files, which are not physical files but

Re: [HACKERS] Confusing comment in xlog.c or am I missing something?

2013-05-02 Thread Heikki Linnakangas
On 02.05.2013 09:11, Amit Langote wrote: In that case, should the comment be /* Check for crossing of xlog file boundary */ instead of /* Check for crossing of xlog segment boundary */, since ( RecPtr-xrecoff= XLogFileSize ) would mean crossing the xlog file (not segment) boundary, right?

Re: [HACKERS] Confusing long option in pg_receivexlog/basebackup/dumpall

2013-05-02 Thread Pavan Deolasee
On Thu, May 2, 2013 at 11:24 AM, Heikki Linnakangas hlinnakan...@vmware.com wrote: In other tools too, like psql, you can pass a connection string with -d/--dbname, which is why I thought it would be best to use the same option for passing a connection string to pg_basebackup/pg_dumpall

Re: [HACKERS] Confusing comment in xlog.c or am I missing something?

2013-05-02 Thread Amit Langote
Yeah, that would be more correct. The phrase we seem to use elsewhere in xlog.c is crossing a logid boundary. Should we change it in 9.2 to clear the confusion? (Attached is a rather small patch to fix that! :) ) -- Amit Langote minor-xlog-comment.patch (914 bytes)

Re: [HACKERS] Recovery target 'immediate'

2013-05-02 Thread Simon Riggs
On 26 April 2013 18:13, Heikki Linnakangas hlinnakan...@vmware.com wrote: On 26.04.2013 19:50, Magnus Hagander wrote: On Fri, Apr 26, 2013 at 6:43 PM, Simon Riggssi...@2ndquadrant.com wrote: On 26 April 2013 17:25, Heikki Linnakangashlinnakan...@vmware.com wrote: Actually, from a

Re: [HACKERS] [COMMITTERS] pgsql: Make fast promotion the default promotion mode.

2013-05-02 Thread Fujii Masao
On Thu, May 2, 2013 at 9:25 AM, Bruce Momjian br...@momjian.us wrote: On Wed, May 1, 2013 at 07:23:33PM -0400, Robert Haas wrote: On Wed, May 1, 2013 at 12:10 PM, Simon Riggs si...@2ndquadrant.com wrote: Heikki said ...remove pg_ctl -m fast/smart option altogether. There is no need to

Re: [HACKERS] Recovery target 'immediate'

2013-05-02 Thread Magnus Hagander
On Thu, May 2, 2013 at 8:55 AM, Simon Riggs si...@2ndquadrant.com wrote: On 26 April 2013 18:13, Heikki Linnakangas hlinnakan...@vmware.com wrote: On 26.04.2013 19:50, Magnus Hagander wrote: On Fri, Apr 26, 2013 at 6:43 PM, Simon Riggssi...@2ndquadrant.com wrote: On 26 April 2013 17:25,

Re: [HACKERS] Documentation epub format

2013-05-02 Thread Andrew Dunstan
On 05/01/2013 11:36 PM, Gavin Flower wrote: On 02/05/13 15:23, Peter Eisentraut wrote: On Wed, 2013-05-01 at 18:27 +0200, Fabien COELHO wrote: I must admit that there is a bit of a disappointement as far as the user experience is concerned: the generated file is barely usable on an iPad2 with

Re: [HACKERS] Recovery target 'immediate'

2013-05-02 Thread Simon Riggs
On 2 May 2013 08:31, Magnus Hagander mag...@hagander.net wrote: That said, there is one property that's very unclear now and that's that you can only set one of recovery_target_time, recovery_target_xid and recovery_target_name. But they can be freely combined with recovery_target_timeline

Re: [HACKERS] [PATCH] add --throttle to pgbench (submission 3)

2013-05-02 Thread Fabien COELHO
Hello Greg, If you add this to https://commitfest.postgresql.org/action/commitfest_view?id=18 I'll review it next month. Ok. Thanks. I just did that. I have a lot of use cases for a pgbench that doesn't just run at 100% all the time. I had tried to simulate something with simple sleep

[HACKERS] GSOC13 proposal - extend RETURNING syntax

2013-05-02 Thread Karol Trzcionka
Hello, I'm student who want to participate in Google Summer of Code. I want to implement feature which allows to get old values directly from update statement. I mean there should be possibility to choose the value immedietly before or after update in RETURNING statement. The syntax may be

[HACKERS] [PATCH] add --progress option to pgbench

2013-05-02 Thread Fabien COELHO
Please find attached a small patch submission, for reference to the next commit fest. Each thread reports its progress about every the number of seconds specified with the option. May be particularly useful for long running pgbench invocations, which should always be the case. shell

[HACKERS] [PATCH] pgbench --throttle (submission 4)

2013-05-02 Thread Fabien COELHO
Minor changes wrt to the previous submission, so as to avoid running some stuff twice under some conditions. This is for reference to the next commit fest. -- Fabien.diff --git a/contrib/pgbench/pgbench.c b/contrib/pgbench/pgbench.c index bc01f07..e692aa1 100644 ---

Re: [HACKERS] Cube extension improvement, GSoC

2013-05-02 Thread Heikki Linnakangas
Hi Stas, and sorry for the late response. On 23.03.2013 01:10, Stas Kelvich wrote: * Adding point data type support to the cube extension in order to avoid storing of coincident upper left and lower right vertices, which may reduce the volume that leaf nodes occupy almost twice. Makes

Re: [HACKERS] high io BUT huge amount of free memory

2013-05-02 Thread Shaun Thomas
On 05/01/2013 06:37 PM, Bruce Momjian wrote: Sorry to be dense here, but what is the problem with that output? That there is a lot of memory marked as free? Why would it mark any memory free? That's kind of my point. :) That 14GB isn't allocated to cache, buffers, any process, or anything

Re: [HACKERS] \watch stuck on execution of commands returning no tuples

2013-05-02 Thread Tom Lane
Michael Paquier michael.paqu...@gmail.com writes: Hi all, When testing \watch, I noticed that process waits indefinitely when executing it with a DDL or a DML. For example: postgres=# CREATE TABLE aa (a int); postgres=# ANALYSE aa \watch 10 -- Process waiting here It's not waiting, it's

Re: [HACKERS] Change to pg_test_fsync output

2013-05-02 Thread Bruce Momjian
On Tue, Apr 16, 2013 at 07:10:38PM -0400, Bruce Momjian wrote: I propose the attached patch to change pg_test_fsync's output from microsecs to usecs, which is the designation we use in other places. Also remove parentheses, e.g. 1 * 16kB open_sync write8012.933 ops/sec

Re: [HACKERS] pg_controldata gobbledygook

2013-05-02 Thread Bruce Momjian
On Fri, Apr 26, 2013 at 08:51:23AM -0400, Robert Haas wrote: On Fri, Apr 26, 2013 at 5:08 AM, Andres Freund and...@2ndquadrant.com wrote: I have to admit I don't see the point. None of those values is particularly interesting to anybody without implementation level knowledge and those will

Re: [HACKERS] GSOC13 proposal - extend RETURNING syntax

2013-05-02 Thread David Fetter
On Thu, May 02, 2013 at 11:04:15AM +0200, Karol Trzcionka wrote: Hello, I'm student who want to participate in Google Summer of Code. I want to implement feature which allows to get old values directly from update statement. I mean there should be possibility to choose the value immedietly

Re: [HACKERS] Confusing comment in xlog.c or am I missing something?

2013-05-02 Thread Amit Langote
Attached patch fixes this. -- Amit Langote minor-xlog-comment-fix.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Confusing comment in xlog.c or am I missing something?

2013-05-02 Thread Heikki Linnakangas
On 02.05.2013 18:05, Amit Langote wrote: Attached patch fixes this. ok, applied. - Heikki -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] GSOC13 proposal - extend RETURNING syntax

2013-05-02 Thread Tom Lane
David Fetter da...@fetter.org writes: 1. As the SQL standard mandates that OLD and NEW be reserved words, we'll re-reserve them. As I mentioned yesterday, I'm not exactly thrilled with re-reserving those, and especially not NEW as it is utterly unnecessary (since the default would already be

Re: [HACKERS] GSOC13 proposal - extend RETURNING syntax

2013-05-02 Thread Karol Trzcionka
W dniu 02.05.2013 17:17, Tom Lane pisze: It should in any case be possible to do this without converting them to reserved words; rather the implementation could be that those table aliases are made available when parsing the UPDATE RETURNING expressions. (This is, in fact, the way that rules

Re: [HACKERS] GSOC13 proposal - extend RETURNING syntax

2013-05-02 Thread Marko Tiikkaja
On 2013-05-02 17:32, Karol Trzcionka wrote: W dniu 02.05.2013 17:17, Tom Lane pisze: It should in any case be possible to do this without converting them to reserved words; rather the implementation could be that those table aliases are made available when parsing the UPDATE RETURNING

Re: [HACKERS] GSOC13 proposal - extend RETURNING syntax

2013-05-02 Thread Tom Lane
Marko Tiikkaja ma...@joh.to writes: What I'm more interested in is: how can we make this feature work in PL/PgSQL where OLD means something different? That's a really good point: if you follow this approach then you're creating fundamental conflicts for use of the feature in trigger functions

Re: [HACKERS] GSOC13 proposal - extend RETURNING syntax

2013-05-02 Thread Andres Freund
On 2013-05-02 12:23:19 -0400, Tom Lane wrote: Marko Tiikkaja ma...@joh.to writes: What I'm more interested in is: how can we make this feature work in PL/PgSQL where OLD means something different? That's a really good point: if you follow this approach then you're creating fundamental

Re: [HACKERS] Remaining beta blockers

2013-05-02 Thread Bruce Momjian
On Tue, Apr 30, 2013 at 12:02:59PM -0400, Robert Haas wrote: On Tue, Apr 30, 2013 at 10:40 AM, Kevin Grittner kgri...@ymail.com wrote: What is a real problem or risk with using this mechanism until we engineer something better? What problems with converting to a later major release does

Re: [HACKERS] GSOC13 proposal - extend RETURNING syntax

2013-05-02 Thread David Fetter
On Thu, May 02, 2013 at 06:28:53PM +0200, Andres Freund wrote: On 2013-05-02 12:23:19 -0400, Tom Lane wrote: Marko Tiikkaja ma...@joh.to writes: What I'm more interested in is: how can we make this feature work in PL/PgSQL where OLD means something different? That's a really good

Re: [HACKERS] high io BUT huge amount of free memory

2013-05-02 Thread Josh Berkus
That's kind of my point. :) That 14GB isn't allocated to cache, buffers, any process, or anything else. It's just... free. In the middle of the day, where 800 PG threads are pulling 7000TPS on average. Based on that scenario, I'd like to think it would cache pretty aggressively, but instead,

Re: [HACKERS] GSOC13 proposal - extend RETURNING syntax

2013-05-02 Thread Pavel Stehule
2013/5/2 Tom Lane t...@sss.pgh.pa.us Marko Tiikkaja ma...@joh.to writes: What I'm more interested in is: how can we make this feature work in PL/PgSQL where OLD means something different? That's a really good point: if you follow this approach then you're creating fundamental conflicts

Re: [HACKERS] GSOC13 proposal - extend RETURNING syntax

2013-05-02 Thread Tom Lane
David Fetter da...@fetter.org writes: On Thu, May 02, 2013 at 06:28:53PM +0200, Andres Freund wrote: prior/after? Both are unreserved keywords atm and it seems far less likely to have conflicts than new/old. BEFORE/AFTER seems more logical to me. Works for me.

Re: [HACKERS] GSOC13 proposal - extend RETURNING syntax

2013-05-02 Thread Karol Trzcionka
W dniu 02.05.2013 19:40, Tom Lane pisze: BEFORE/AFTER seems more logical to me. Works for me. What do you think about function- or cast-like syntax. I mean: RETURNING OLD(foo.bar) or RETURNING OLD(foo).bar or RETURNING (foo::OLD).bar ? I think none of them should conflict with any other

[HACKERS] [PATCH] add long options to pgbench (submission 1)

2013-05-02 Thread Fabien COELHO
This is mostly for reference to the next commitfest. This very minor patch adds a corresponding long option to all short (one letter) options of pgbench. In particular for connection options there is now --host --username --port options similar to the psql client. While I was at developing

Re: [HACKERS] pgbench --startup option

2013-05-02 Thread Fabien COELHO
I've just done a quick review of the source, as I've been hacking in pgbench myself. I think that the feature makes sense. About the details of the patch: (1) Some changes in the patch are unrelated to the purpose of the patch (e.g. spacing changes, error message...), and should be

Re: [HACKERS] GSOC13 proposal - extend RETURNING syntax

2013-05-02 Thread David Fetter
On Thu, May 02, 2013 at 01:40:59PM -0400, Tom Lane wrote: David Fetter da...@fetter.org writes: On Thu, May 02, 2013 at 06:28:53PM +0200, Andres Freund wrote: prior/after? Both are unreserved keywords atm and it seems far less likely to have conflicts than new/old. BEFORE/AFTER seems

Re: [HACKERS] GSOC13 proposal - extend RETURNING syntax

2013-05-02 Thread Atri Sharma
Sent from my iPad On 03-May-2013, at 0:07, David Fetter da...@fetter.org wrote: On Thu, May 02, 2013 at 01:40:59PM -0400, Tom Lane wrote: David Fetter da...@fetter.org writes: On Thu, May 02, 2013 at 06:28:53PM +0200, Andres Freund wrote: prior/after? Both are unreserved keywords atm and

Re: [HACKERS] GSOC13 proposal - extend RETURNING syntax

2013-05-02 Thread Tom Lane
David Fetter da...@fetter.org writes: Maybe we can make BEFORE and AFTER implied aliases rather than keywords. What say? Well, they still have to be unreserved keywords for their use in trigger-related commands, but as far as this feature is concerned I think they're best off being handled as

Re: [HACKERS] GSOC13 proposal - extend RETURNING syntax

2013-05-02 Thread Tom Lane
Karol Trzcionka karl...@gmail.com writes: What do you think about function- or cast-like syntax. I mean: RETURNING OLD(foo.bar) or RETURNING OLD(foo).bar or RETURNING (foo::OLD).bar ? I think none of them should conflict with any other statements. I think you'll find those alternatives are

Re: [HACKERS] GSOC13 proposal - extend RETURNING syntax

2013-05-02 Thread Karol Trzcionka
W dniu 02.05.2013 20:45, Tom Lane pisze: David Fetter da...@fetter.org writes: Maybe we can make BEFORE and AFTER implied aliases rather than keywords. What say? Well, they still have to be unreserved keywords for their use in trigger-related commands, but as far as this feature is concerned

Re: [HACKERS] 9.3 Beta1 status report

2013-05-02 Thread Bruce Momjian
On Sat, Apr 27, 2013 at 05:29:51PM -0700, Josh Berkus wrote: Bruce, So here's my draft list of Major Enhancements for the relase notes: * Writeable Foreign Tables * pgsql_fdw driver for federation of PostgreSQL databases * Automatically updatable VIEWs * MATERIALIZED VIEW declaration *

Re: [HACKERS] 9.3 Beta1 status report

2013-05-02 Thread Josh Berkus
listitem para Add a Postgres foreign data wrapper contrib module (Shigeru Hanada, KaiGai Kohei) /para para This foreign data wrapper allows writes; potentially other foreign data wrappers can now support writes. /para

Re: [HACKERS] high io BUT huge amount of free memory

2013-05-02 Thread Shaun Thomas
On 05/02/2013 12:04 PM, Josh Berkus wrote: There is a good, but sad, reason for this: IBM and Oracle and their partners are the largest employers of people hacking on core Linux memory/IO functionality, and both of those companies use DirectIO extensively in their products. I never thought of

Re: [HACKERS] Remaining beta blockers

2013-05-02 Thread Kevin Grittner
Bruce Momjian br...@momjian.us wrote: Robert Haas wrote: Kevin Grittner kgri...@ymail.com wrote: What is a real problem or risk with using this mechanism until we engineer something better?  What problems with converting to a later major release does anyone see? Well, it's a pg_upgrade

Re: [HACKERS] GSOC13 proposal - extend RETURNING syntax

2013-05-02 Thread Gavin Flower
On 03/05/13 04:52, David Fetter wrote: On Thu, May 02, 2013 at 06:28:53PM +0200, Andres Freund wrote: On 2013-05-02 12:23:19 -0400, Tom Lane wrote: Marko Tiikkaja ma...@joh.to writes: What I'm more interested in is: how can we make this feature work in PL/PgSQL where OLD means something

Re: [HACKERS] GSOC13 proposal - extend RETURNING syntax

2013-05-02 Thread Karol Trzcionka
W dniu 02.05.2013 23:34, Gavin Flower pisze: I prefer 'PRIOR 'AFTER' as the both have the same length - but perhaps that is just me! :-) I think it doesn't matter at now because PRIOR has the same problems as BEFORE ;) Regards, Karol

[HACKERS] matview niceties: pick any two of these three

2013-05-02 Thread Kevin Grittner
Tom has refactored where and how certain parts of the work get done for materialized views, reducing issues with modularity violations. I have been and will continue to review his changes to better understand how the pieces of the code fit together, so hopefully he won't need to do so much with

[HACKERS] 9.3 Beta1 status report

2013-05-02 Thread Jeff Janes
On Sat, Apr 20, 2013 at 10:02 PM, Bruce Momjian br...@momjian.us wrote: I am not sure if Tom shared yet, but we are planning to package 9.3 beta1 on April 29, with a release on May 2. Those dates might change, but that is the current plan. I have completed a draft 9.3 release notes, which you

Re: [HACKERS] matview niceties: pick any two of these three

2013-05-02 Thread Josh Berkus
Tom wants to ditch (2) to allow the others. Robert wants to ditch (1) to allow the others. I want to ditch (3) to allow the others. Andres wants (3) and has not expressed an opinion on which he would prefer to give up to get it. I believe Josh Berkus has mentioned how useful he thinks

Re: [HACKERS] 9.3 Beta1 status report

2013-05-02 Thread Bruce Momjian
On Thu, May 2, 2013 at 02:09:21PM -0700, Josh Berkus wrote: listitem para Add a Postgres foreign data wrapper contrib module (Shigeru Hanada, KaiGai Kohei) /para para This foreign data wrapper allows writes; potentially other

Re: [HACKERS] high io BUT huge amount of free memory

2013-05-02 Thread Andres Freund
On 2013-05-02 16:13:42 -0500, Shaun Thomas wrote: On 05/02/2013 12:04 PM, Josh Berkus wrote: Yeah, this is why I want to go to Linux Plumbers this year. The Kernel.org engineers are increasingly doing things which makes Linux unsuitable for applications which depend on the filesystem. Uh.

Re: [HACKERS] 9.3 Beta1 status report

2013-05-02 Thread Bruce Momjian
On Thu, May 2, 2013 at 03:03:58PM -0700, Jeff Janes wrote: Some suggestions, perhaps just based on my preference for verbosity: para Add cache of local locks (Jeff Janes) /para para This speeds lock release at statement completion in transactions

Re: [HACKERS] Should pg_upgrade use --quote-all-identifiers?

2013-05-02 Thread Bruce Momjian
On Tue, Apr 30, 2013 at 07:55:33PM -0400, Tom Lane wrote: Seems like this might be a good idea to avoid the type of failure exhibited in bug #8128. We don't care too much about the readability of the dump script created during an upgrade, so it's hard to see a downside. Fine with me. --

Re: [HACKERS] \watch stuck on execution of commands returning no tuples

2013-05-02 Thread Bruce Momjian
On Thu, May 2, 2013 at 10:01:28AM -0400, Tom Lane wrote: Michael Paquier michael.paqu...@gmail.com writes: Hi all, When testing \watch, I noticed that process waits indefinitely when executing it with a DDL or a DML. For example:

Re: [HACKERS] Recovery target 'immediate'

2013-05-02 Thread Bruce Momjian
On Thu, May 2, 2013 at 09:04:20AM +0100, Simon Riggs wrote: If we feel strongly about user interface design problems we should treat them the same way we treat performance issues. Profile to identify problem areas, analyze problems in those areas and suggest solutions, then make tests to

Re: [HACKERS] Recovery target 'immediate'

2013-05-02 Thread Bruce Momjian
On Thu, May 2, 2013 at 09:31:03AM +0200, Magnus Hagander wrote: Actually, there is - I hear it quite often from people not so experienced in PostgreSQL. Though in fairness, I'm not entirely sure the new syntax would help - some of those need a tool to do it for them, really (and such tools

Re: [HACKERS] Remaining beta blockers

2013-05-02 Thread Bruce Momjian
On Thu, May 2, 2013 at 02:20:15PM -0700, Kevin Grittner wrote: Yes, pg_upgrade is never going to write to data pages as that would be slow and prevent the ability to roll back to the previous cluster on error. The only person who has suggested anything which would require that is

Re: [HACKERS] Recovery target 'immediate'

2013-05-02 Thread Michael Paquier
On Fri, May 3, 2013 at 8:56 AM, Bruce Momjian br...@momjian.us wrote: On Thu, May 2, 2013 at 09:31:03AM +0200, Magnus Hagander wrote: Actually, there is - I hear it quite often from people not so experienced in PostgreSQL. Though in fairness, I'm not entirely sure the new syntax would

Re: [HACKERS] \watch stuck on execution of commands returning no tuples

2013-05-02 Thread Michael Paquier
On Thu, May 2, 2013 at 11:01 PM, Tom Lane t...@sss.pgh.pa.us wrote: Michael Paquier michael.paqu...@gmail.com writes: Hi all, When testing \watch, I noticed that process waits indefinitely when executing it with a DDL or a DML. For example: postgres=# CREATE TABLE aa (a int);

Re: [HACKERS] [PATCH] pgbench --throttle (submission 4)

2013-05-02 Thread Michael Paquier
Hi, It would be better to submit updated versions of a patch on the email thread it is dedicated to and not create a new thread so as people can easily follow the progress you are doing. Thanks, -- Michael

Re: [HACKERS] GSOC13 proposal - extend RETURNING syntax

2013-05-02 Thread Tom Lane
Karol Trzcionka karl...@gmail.com writes: It will not solve the problems: 1. How to access to old rows if the table is named BEFORE? The user can simply choose to use a different table alias, as Andres explained upthread. If any table's active alias is before (or after), we just don't activate

Re: [HACKERS] Documentation epub format

2013-05-02 Thread Peter Eisentraut
On Wed, 2013-05-01 at 18:27 +0200, Fabien COELHO wrote: The table of contents too much detailed, so it is long and slow to scan, and there is no clear shortcut. Flipping pages in the documentation takes ages (well, close to one second or more if I flip a few pages). Do not try search. EPUB

Re: [HACKERS] Documentation epub format

2013-05-02 Thread Gavin Flower
On 03/05/13 15:16, Peter Eisentraut wrote: On Wed, 2013-05-01 at 18:27 +0200, Fabien COELHO wrote: The table of contents too much detailed, so it is long and slow to scan, and there is no clear shortcut. Flipping pages in the documentation takes ages (well, close to one second or more if I flip

Re: [HACKERS] Documentation epub format

2013-05-02 Thread David Fetter
On Thu, May 02, 2013 at 03:42:33AM -0400, Andrew Dunstan wrote: On 05/01/2013 11:36 PM, Gavin Flower wrote: On 02/05/13 15:23, Peter Eisentraut wrote: On Wed, 2013-05-01 at 18:27 +0200, Fabien COELHO wrote: I must admit that there is a bit of a disappointement as far as the user experience

Re: [HACKERS] Documentation epub format

2013-05-02 Thread Fabien COELHO
This seems to suggest that instead of generating one large ebook, the build should generate a set of ebooks, say one for each part. At the minimum, a less detailed toc could be more usable and help navigate the huge contents. Once upon a time we had multiple books as documentation, then at