Re: [HACKERS] WIP patch for hint bit i/o mitigation

2012-11-07 Thread Amit Kapila
From: pgsql-hackers-ow...@postgresql.org [mailto:pgsql-hackers- ow...@postgresql.org] On Behalf Of Merlin Moncure Sent: Wednesday, November 07, 2012 5:26 AM To: PostgreSQL-development Cc: Atri Sharma Subject: [HACKERS] WIP patch for hint bit i/o mitigation Following the sig is a first cut

Re: [HACKERS] WIP patch for hint bit i/o mitigation

2012-11-07 Thread Atri Sharma
On 07-Nov-2012, at 15:46, Amit Kapila amit.kap...@huawei.com wrote: From: pgsql-hackers-ow...@postgresql.org [mailto:pgsql-hackers- ow...@postgresql.org] On Behalf Of Merlin Moncure Sent: Wednesday, November 07, 2012 5:26 AM To: PostgreSQL-development Cc: Atri Sharma Subject: [HACKERS] WIP

Re: [HACKERS] WIP patch for hint bit i/o mitigation

2012-11-07 Thread Amit Kapila
-Original Message- From: Atri Sharma [mailto:atri.j...@gmail.com] Sent: Wednesday, November 07, 2012 4:02 PM To: Amit Kapila Cc: Merlin Moncure; PostgreSQL-development Subject: Re: [HACKERS] WIP patch for hint bit i/o mitigation On 07-Nov-2012, at 15:46, Amit Kapila

[HACKERS] Extend libpq to support mixed text and binary results

2012-11-07 Thread Albe Laurenz
The Frontend/Backend protocol's bind message allows to specify mixed text and binary result formats, but there is not support in the C API for that. I'd like to add support for that to libpq for these reasons: - I think that every feature of the line protocol should be exposed in the C API. -

Re: [HACKERS] WIP patch for hint bit i/o mitigation

2012-11-07 Thread Atri Sharma
On Wed, Nov 7, 2012 at 5:31 PM, Amit Kapila amit.kap...@huawei.com wrote: -Original Message- From: Atri Sharma [mailto:atri.j...@gmail.com] Sent: Wednesday, November 07, 2012 4:02 PM To: Amit Kapila Cc: Merlin Moncure; PostgreSQL-development Subject: Re: [HACKERS] WIP patch

Re: [HACKERS] Extend libpq to support mixed text and binary results

2012-11-07 Thread Simon Riggs
On 7 November 2012 13:08, Albe Laurenz laurenz.a...@wien.gv.at wrote: - I think that every feature of the line protocol should be exposed in the C API. Exposing every possible bug in ther underlying protocol isn't the best plan though, especially when doing so complicates the API just to

Re: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL

2012-11-07 Thread Magnus Hagander
On Wed, Nov 7, 2012 at 5:19 AM, Amit Kapila amit.kap...@huawei.com wrote: On Tuesday, November 06, 2012 11:30 PM Robert Haas wrote: On Wed, Oct 31, 2012 at 8:17 AM, Magnus Hagander mag...@hagander.net wrote: I'm not convinced we ever *had* a consensus on this. There were proposals, but I'm

[HACKERS] TRUNCATE SERIALIZABLE and frozen COPY

2012-11-07 Thread Simon Riggs
For 9.2 we discussed having COPY setting tuples as frozen. Various details apply. Earlier threads: RFC: Making TRUNCATE more MVCC-safe COPY with hints, rebirth I was unhappy with changing the behaviour of TRUNCATE, and still am. So the proposal here is to have a specific modifier on TRUNCATE

[HACKERS] crash in DROP INDEX CONCURRENTLY

2012-11-07 Thread Alvaro Herrera
I just observed that if I run src/test/regress' make installcheck-parallel and concurrently src/test/isolation's make installcheck, I very reproducibly get a segfault (in an assert-enabled build). Apparently the problem is that somebody is trying to transfer predicate locks to a relation, but the

Re: [HACKERS] WIP patch for hint bit i/o mitigation

2012-11-07 Thread Merlin Moncure
On Wed, Nov 7, 2012 at 6:01 AM, Amit Kapila amit.kap...@huawei.com wrote: Following the sig is a first cut at a patch (written by Atri) that attempts to mitigate hint bit i/o penalty when many pages worth of tuples are sequentially written out with the same transaction id. There have been

Re: [HACKERS] Extend libpq to support mixed text and binary results

2012-11-07 Thread Merlin Moncure
On Wed, Nov 7, 2012 at 7:08 AM, Albe Laurenz laurenz.a...@wien.gv.at wrote: The Frontend/Backend protocol's bind message allows to specify mixed text and binary result formats, but there is not support in the C API for that. I'd like to add support for that to libpq for these reasons: - I

Re: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL

2012-11-07 Thread Tom Lane
Magnus Hagander mag...@hagander.net writes: On Wed, Nov 7, 2012 at 5:19 AM, Amit Kapila amit.kap...@huawei.com wrote: However there is one more point which I am not able to clearly make out is how to write into file that contains all configuration parameters changed by SQL. Perhaps you can

Re: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL

2012-11-07 Thread Magnus Hagander
On Wed, Nov 7, 2012 at 5:53 PM, Tom Lane t...@sss.pgh.pa.us wrote: Magnus Hagander mag...@hagander.net writes: On Wed, Nov 7, 2012 at 5:19 AM, Amit Kapila amit.kap...@huawei.com wrote: However there is one more point which I am not able to clearly make out is how to write into file that

Re: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL

2012-11-07 Thread Tom Lane
Magnus Hagander mag...@hagander.net writes: On Wed, Nov 7, 2012 at 5:53 PM, Tom Lane t...@sss.pgh.pa.us wrote: I'm not sure that the above approach works anyway --- for instance, the current setting might be a SET LOCAL result, in which case you still don't know anything about what the

Re: [HACKERS] [BUGS] BUG #7534: walreceiver takes long time to detect n/w breakdown

2012-11-07 Thread Heikki Linnakangas
On 16.10.2012 15:31, Heikki Linnakangas wrote: On 15.10.2012 19:31, Fujii Masao wrote: On Mon, Oct 15, 2012 at 11:27 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: On 15.10.2012 13:13, Heikki Linnakangas wrote: Oh, I didn't remember that we've documented the specific structs that we

Re: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL

2012-11-07 Thread Magnus Hagander
On Wed, Nov 7, 2012 at 6:19 PM, Tom Lane t...@sss.pgh.pa.us wrote: Magnus Hagander mag...@hagander.net writes: On Wed, Nov 7, 2012 at 5:53 PM, Tom Lane t...@sss.pgh.pa.us wrote: I'm not sure that the above approach works anyway --- for instance, the current setting might be a SET LOCAL result,

Re: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL

2012-11-07 Thread Tom Lane
Magnus Hagander mag...@hagander.net writes: On Wed, Nov 7, 2012 at 6:19 PM, Tom Lane t...@sss.pgh.pa.us wrote: You could dig it out of the stack if it's there, but that doesn't fix the race-condition aspect. Now a race is inevitable if two sessions try to set the *same* variable, but I think

Re: [HACKERS] Extend libpq to support mixed text and binary results

2012-11-07 Thread Tom Lane
Simon Riggs si...@2ndquadrant.com writes: On 7 November 2012 13:08, Albe Laurenz laurenz.a...@wien.gv.at wrote: - I think that every feature of the line protocol should be exposed in the C API. My recollection is that there are several things in the V3 protocol that were put there to satisfy

Re: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL

2012-11-07 Thread Robert Haas
On Wed, Nov 7, 2012 at 12:29 PM, Tom Lane t...@sss.pgh.pa.us wrote: Magnus Hagander mag...@hagander.net writes: On Wed, Nov 7, 2012 at 6:19 PM, Tom Lane t...@sss.pgh.pa.us wrote: You could dig it out of the stack if it's there, but that doesn't fix the race-condition aspect. Now a race is

Re: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL

2012-11-07 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Wed, Nov 7, 2012 at 12:29 PM, Tom Lane t...@sss.pgh.pa.us wrote: ... we don't normally read the config file within-commands, and there are both semantic and implementation problems to overcome if you want to do so. Why would you need to? It seems

Re: [HACKERS] crash in DROP INDEX CONCURRENTLY

2012-11-07 Thread Andres Freund
On Wed, Nov 07, 2012 at 12:54:57PM -0300, Alvaro Herrera wrote: I just observed that if I run src/test/regress' make installcheck-parallel and concurrently src/test/isolation's make installcheck, I very reproducibly get a segfault (in an assert-enabled build). Apparently the problem is that

Re: [HACKERS] crash in DROP INDEX CONCURRENTLY

2012-11-07 Thread Alvaro Herrera
Andres Freund wrote: On Wed, Nov 07, 2012 at 12:54:57PM -0300, Alvaro Herrera wrote: I think the fix is just to move the heap_open calls some lines upwards, as in the attached patch. Looks like the right thing to me. Pushed. -- Álvaro Herrera

Re: [HACKERS] RFC: Timing Events

2012-11-07 Thread Josh Berkus
You're being obtuse, Josh. These parameters can be set the same way any other parameters can, including ALTER ROLE SET or ALTER DATABASE SET. The superuser only restriction is that only a superuser can execute the ALTER, not that the target role has to be superuser. Oh? Ok, that's helpful.

Re: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL

2012-11-07 Thread Josh Berkus
Well, Magnus' proposed implementation supposed that the existing values *have* been loaded into the current session. I agree that with some locking and yet more code you could implement it without that. But this still doesn't seem to offer any detectable benefit over value-per-file. Well,

Re: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL

2012-11-07 Thread Robert Haas
On Wed, Nov 7, 2012 at 2:50 PM, Josh Berkus j...@agliodbs.com wrote: Well, Magnus' proposed implementation supposed that the existing values *have* been loaded into the current session. I agree that with some locking and yet more code you could implement it without that. But this still

Re: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL

2012-11-07 Thread Andres Freund
On Wed, Nov 07, 2012 at 03:15:07PM -0500, Robert Haas wrote: On Wed, Nov 7, 2012 at 2:50 PM, Josh Berkus j...@agliodbs.com wrote: Well, Magnus' proposed implementation supposed that the existing values *have* been loaded into the current session. I agree that with some locking and yet more

Re: [HACKERS] RFC: Timing Events

2012-11-07 Thread Tom Lane
Josh Berkus j...@agliodbs.com writes: It still eliminates the main potential use of auto-explain on production sites, though, which is to turn it on only for specific operations. I've never been able to make use of auto-explain for any real diagnostic purpose on a production site, and I don't

[HACKERS] RFC: New log_destination 'fifo'

2012-11-07 Thread Joshua D. Drake
Hello, So it is possible to do this in other ways but I thought it might be interesting to allow people to define fifo or pipe as a log_desination. This would allow a person to tail the fifo to receive the outputs of the log as it happens but would not take up precious IO or space from the

Re: [HACKERS] RFC: Timing Events

2012-11-07 Thread Jeff Janes
On Wed, Nov 7, 2012 at 11:45 AM, Josh Berkus j...@agliodbs.com wrote: You're being obtuse, Josh. These parameters can be set the same way any other parameters can, including ALTER ROLE SET or ALTER DATABASE SET. The superuser only restriction is that only a superuser can execute the ALTER,

Re: [HACKERS] RFC: New log_destination 'fifo'

2012-11-07 Thread Magnus Hagander
On Wed, Nov 7, 2012 at 9:55 PM, Joshua D. Drake j...@commandprompt.com wrote: Hello, So it is possible to do this in other ways but I thought it might be interesting to allow people to define fifo or pipe as a log_desination. This would allow a person to tail the fifo to receive the outputs

Re: [HACKERS] Proof of concept: auto updatable views [Review of Patch]

2012-11-07 Thread Tom Lane
Dean Rasheed dean.a.rash...@gmail.com writes: Thanks for looking at this. Attached is a rebased patch using new OIDs. I'm starting to look at this patch now. I don't understand the intended interaction with qualified INSTEAD rules. The code looks like + if (!instead

Re: [HACKERS] Proof of concept: auto updatable views [Review of Patch]

2012-11-07 Thread David Fetter
On Wed, Nov 07, 2012 at 05:04:48PM -0500, Tom Lane wrote: Dean Rasheed dean.a.rash...@gmail.com writes: Thanks for looking at this. Attached is a rebased patch using new OIDs. I'm starting to look at this patch now. I don't understand the intended interaction with qualified INSTEAD

Re: [HACKERS] RFC: New log_destination 'fifo'

2012-11-07 Thread David Fetter
On Wed, Nov 07, 2012 at 12:55:03PM -0800, Joshua D. Drake wrote: Hello, So it is possible to do this in other ways but I thought it might be interesting to allow people to define fifo or pipe as a log_desination. This would allow a person to tail the fifo to receive the outputs of the

Re: [HACKERS] Proof of concept: auto updatable views [Review of Patch]

2012-11-07 Thread Tom Lane
David Fetter da...@fetter.org writes: On Wed, Nov 07, 2012 at 05:04:48PM -0500, Tom Lane wrote: Should we be doing something about such cases, or is playing dumb correct? The SQL standard handles deciding the behavior based on whether WITH CHECK OPTION is included in the view DDL. See the

Re: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL

2012-11-07 Thread Greg Smith
On 11/2/12 11:17 AM, Magnus Hagander wrote: -Add a configuration subdirectory to the default installation. Needs to follow the config file location, so things like the Debian relocation of postgresql.conf still work. Maybe it has zero files; maybe it has one that's named for

Re: [HACKERS] Proof of concept: auto updatable views [Review of Patch]

2012-11-07 Thread Dean Rasheed
On 7 November 2012 22:04, Tom Lane t...@sss.pgh.pa.us wrote: Dean Rasheed dean.a.rash...@gmail.com writes: Thanks for looking at this. Attached is a rebased patch using new OIDs. I'm starting to look at this patch now. I don't understand the intended interaction with qualified INSTEAD

Re: [HACKERS] auto_explain WAS: RFC: Timing Events

2012-11-07 Thread Josh Berkus
What prevents you from doing so? The performance impact? The volume of logs generated? Yes and yes. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription:

Re: [HACKERS] auto_explain WAS: RFC: Timing Events

2012-11-07 Thread Jeff Janes
On Wed, Nov 7, 2012 at 4:55 PM, Josh Berkus j...@agliodbs.com wrote: What prevents you from doing so? The performance impact? The volume of logs generated? Yes and yes. I've never noticed a performance impact, unless you are having it analyze, or having it log every query. How do you

[HACKERS] Further pg_upgrade analysis for many tables

2012-11-07 Thread Bruce Momjian
As a followup to Magnus's report that pg_upgrade was slow for many tables, I did some more testing with many tables, e.g.: CREATE TABLE test991 (x SERIAL); I ran it for 0, 1k, 2k, ... 16k tables, and got these results: tablespg_dump restore pg_upgrade(increase)

Re: [HACKERS] Proof of concept: auto updatable views [Review of Patch]

2012-11-07 Thread Tom Lane
Dean Rasheed dean.a.rash...@gmail.com writes: On 7 November 2012 22:04, Tom Lane t...@sss.pgh.pa.us wrote: This seems to me to be dangerous and unintuitive, not to mention underdocumented. I think it would be better to just not do anything if there is any INSTEAD rule, period. Is this any

[HACKERS] commit 1eb1dde049ccfffc42c80c2 broke make -j2

2012-11-07 Thread Jeff Janes
Since commit 1eb1dde049ccfffc42c80c2, Have make never delete intermediate files automatically, I've frequently been getting errors doing make -j2 after a maintainer-clean. parser.c:25:21: fatal error: preproc.h: No such file or directory compilation terminated. make[4]: *** [parser.o] Error 1

Re: [HACKERS] commit 1eb1dde049ccfffc42c80c2 broke make -j2

2012-11-07 Thread Tom Lane
Jeff Janes jeff.ja...@gmail.com writes: Since commit 1eb1dde049ccfffc42c80c2, Have make never delete intermediate files automatically, I've frequently been getting errors doing make -j2 after a maintainer-clean. Worksforme on a Fedora 16 box (make 3.82). What gmake version are you running?

[HACKERS] Fix errcontext() function

2012-11-07 Thread Chen Huajun
Hello, I am sending patch for errcontext() function. I use procedural languages to do some operation, but when error occurs ,the CONTEXT error messages from procedural languages doesn't display in local language. for example: postgres=#

Re: [HACKERS] RFC: New log_destination 'fifo'

2012-11-07 Thread Peter Eisentraut
On Wed, 2012-11-07 at 12:55 -0800, Joshua D. Drake wrote: So it is possible to do this in other ways but I thought it might be interesting to allow people to define fifo or pipe as a log_desination. You could do this with a logging hook as a plugin, so you don't have to make the effort to

Re: [HACKERS] commit 1eb1dde049ccfffc42c80c2 broke make -j2

2012-11-07 Thread Tom Lane
I wrote: Jeff Janes jeff.ja...@gmail.com writes: Since commit 1eb1dde049ccfffc42c80c2, Have make never delete intermediate files automatically, I've frequently been getting errors doing make -j2 after a maintainer-clean. Worksforme on a Fedora 16 box (make 3.82). What gmake version are you