Re: [HACKERS] newline conversion in SQL command strings

2012-09-20 Thread Heikki Linnakangas
On 20.09.2012 05:56, Peter Eisentraut wrote: I have received a number of bug reports about plsh choking on Windows-style line endings. The problem is that the user uses some Windows-based tool or other to execute an SQL command line this: CREATE FUNCTION foo() RETURNS somethingCRLF LANGUAGE

Re: [HACKERS] Fwd: PATCH: psql boolean display

2012-09-20 Thread Heikki Linnakangas
It doesn't look like this patch is going anywhere. I agree with Tom's comments that we need to think how this works for all datatypes, not just booleans. And a simple substitution of values isn't enough; an application might want to output all integers in hex, for example. A custom domain in

[HACKERS] [PATCH] Make pg_basebackup configure and start standby [Review]

2012-09-20 Thread Amit Kapila
On Sun, 01 Jul 2012 13:02:17 +0200 Boszormenyi Zoltan wrote: attached is a patch that does $SUBJECT. It's a usability enhancement, to take a backup, write a minimalistic recovery.conf and start the streaming standby in one go. Comments? [Review of Patch] Basic stuff:

[HACKERS] Doubt Regarding changes to disable keepalives in walsender

2012-09-20 Thread Amit Kapila
In commit da4efa13d801ccc179f1d2c24d8a60c4a2f8ede9, sending of keepalive messages have been disabled in walsender. Commit message is Turn off WalSender keepalives by default, users can enable if desired. But I am not able to see how users can enable it, can you please point me if I am missing

Re: [HACKERS] ToDo: allow to get a number of processed rows by COPY statement [Review of Patch]

2012-09-20 Thread Amit Kapila
On 16.08.2012 14:43, Pavel Stehule wrote: Hello here is updated patch [Review of Patch] Basic stuff: - Patch applies OK. but offset difference in line numbers. - Compiles with errors in contrib [pg_stat_statements, sepgsql] modules - Regression failed; one test-case

Re: [HACKERS] newline conversion in SQL command strings

2012-09-20 Thread Peter Eisentraut
On 9/20/12 2:01 AM, Heikki Linnakangas wrote: Could you strip the CRs? Either at CREATE FUNCTION time, or when the function is executed. It has been proposed that the plsh handler should strip the CRs before execution. But I don't think that is a correct solution, because that is user data

Re: [HACKERS] Invalid optimization of VOLATILE function in WHERE clause?

2012-09-20 Thread Merlin Moncure
On Wed, Sep 19, 2012 at 2:39 PM, Kevin Grittner kevin.gritt...@wicourts.gov wrote: Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: It still seems like awfully weird behavior. Why? The WHERE condition relates only to the output of the _stats subquery, so why

Re: [v9.3] Extra Daemons (Re: [HACKERS] elegant and effective way for running jobs inside a database)

2012-09-20 Thread Alvaro Herrera
Excerpts from Amit Kapila's message of jue sep 20 02:10:23 -0300 2012: Why can't worker tasks be also permanent, which can be controlled through configuration. What I mean to say is that if user has need for parallel operations he can configure max_worker_tasks and those many worker

Re: [v9.3] Extra Daemons (Re: [HACKERS] elegant and effective way for running jobs inside a database)

2012-09-20 Thread Kohei KaiGai
2012/9/20 Amit Kapila amit.kap...@huawei.com: On Thursday, September 20, 2012 1:44 AM Simon Riggs wrote: On 12 September 2012 04:30, Amit Kapila amit.kap...@huawei.com wrote: On Tuesday, September 11, 2012 9:09 PM Alvaro Herrera wrote: Excerpts from Boszormenyi Zoltan's message of vie jun 29

[HACKERS] XLogInsert scaling, revisited

2012-09-20 Thread Heikki Linnakangas
I've been slowly continuing to work that I started last winder to make XLogInsert scale better. I have tried quite a few different approaches since then, and have settled on the attached. This is similar but not exactly the same as what I did in the patches I posted earlier. The basic idea,

Re: [HACKERS] Invalid optimization of VOLATILE function in WHERE clause?

2012-09-20 Thread Merlin Moncure
On Thu, Sep 20, 2012 at 9:24 AM, Kevin Grittner kevin.gritt...@wicourts.gov wrote: Merlin Moncure mmonc...@gmail.com wrote: Hm, I bet it's possible (although probably not easy) to deduce volatility from the function body...maybe through the validator. If you could do that (perhaps warning in

Re: [HACKERS] XLogInsert scaling, revisited

2012-09-20 Thread Andres Freund
On Thursday, September 20, 2012 05:37:42 PM Tom Lane wrote: Heikki Linnakangas hlinnakan...@vmware.com writes: I've been slowly continuing to work that I started last winder to make XLogInsert scale better. I have tried quite a few different approaches since then, and have settled on the

Re: [HACKERS] XLogInsert scaling, revisited

2012-09-20 Thread Heikki Linnakangas
On 20.09.2012 18:37, Tom Lane wrote: I suspect that it's not a coincidence that you're seeing a sweet spot for #slots ~= #CPUs. Yeah, quite possible. I did not test with any different number of slots, so I don't know if that's the sweet spot, but I wouldn't be surprised if it is. If that

Re: [HACKERS] XLogInsert scaling, revisited

2012-09-20 Thread Tom Lane
Heikki Linnakangas hlinnakan...@vmware.com writes: I've been slowly continuing to work that I started last winder to make XLogInsert scale better. I have tried quite a few different approaches since then, and have settled on the attached. This is similar but not exactly the same as what I

[HACKERS] Suggestion for --truncate-tables to pg_restore

2012-09-20 Thread Karl O. Pinc
Hi, I've had problems using pg_restore --data-only when restoring individual schemas (which contain data which has had bad things done to it). --clean does not work well because of dependent objects in other schemas. Patch to the docs attached (before I go and do any real coding.) Karl

Re: [HACKERS] XLogInsert scaling, revisited

2012-09-20 Thread Simon Riggs
On 20 September 2012 16:29, Heikki Linnakangas hlinnakan...@vmware.com wrote: 1. Insert 1000 rows in each transaction. This is exactly the sort of workload where WALInsertLock currently becomes a bottleneck. Without the the patch, the test scales very badly, with about 420 TPS with a single

Re: [HACKERS] Move postgresql_fdw_validator into dblink

2012-09-20 Thread Kohei KaiGai
Hanada-san, I checked your patch. It can be applied to the latest master branch without any conflicts, and regression tests were fine. Unlike the original postgresql_fdw_validator(), the new dblink_fdw_validator() has wise idea; that pulls list of connection options from libpq, instead of

Re: [HACKERS] newline conversion in SQL command strings

2012-09-20 Thread Andrew Dunstan
On 09/20/2012 09:12 AM, Peter Eisentraut wrote: On 9/20/12 2:01 AM, Heikki Linnakangas wrote: Could you strip the CRs? Either at CREATE FUNCTION time, or when the function is executed. It has been proposed that the plsh handler should strip the CRs before execution. But I don't think that is

Re: [HACKERS] newline conversion in SQL command strings

2012-09-20 Thread Tom Lane
Andrew Dunstan and...@dunslane.net writes: On 09/20/2012 09:12 AM, Peter Eisentraut wrote: It has been proposed that the plsh handler should strip the CRs before execution. But I don't think that is a correct solution, because that is user data which could be relevant. It could be the case,

Re: [HACKERS] ToDo: allow to get a number of processed rows by COPY statement [Review of Patch]

2012-09-20 Thread Pavel Stehule
Hello Basic stuff: - Patch applies OK. but offset difference in line numbers. - Compiles with errors in contrib [pg_stat_statements, sepgsql] modules - Regression failed; one test-case in COPY due to incomplete test-case attached patch. – same as reported by Heikki fixed

Re: [HACKERS] newline conversion in SQL command strings

2012-09-20 Thread Andrew Dunstan
On 09/20/2012 03:34 PM, Tom Lane wrote: Andrew Dunstan and...@dunslane.net writes: On 09/20/2012 09:12 AM, Peter Eisentraut wrote: It has been proposed that the plsh handler should strip the CRs before execution. But I don't think that is a correct solution, because that is user data which

Re: [HACKERS] Re: proposal and patch : support INSERT INTO...RETURNING with partitioned table using rule

2012-09-20 Thread Tom Lane
John Lumby johnlu...@hotmail.com writes: On Fri, 22 Jun 2012 09:55:13, Robert Haas wrote: I do notice that the RETURNING clause of the INSERT can't reference NEW, which seems like a restriction that we probably ought to lift, but it doesn't seem to have much to do with your patch. The main

Re: [HACKERS] PATCH: pgbench - aggregation of info written into log

2012-09-20 Thread Tomas Vondra
On 3.9.2012 01:28, Jeff Janes wrote: On Sun, Sep 2, 2012 at 3:46 PM, Tomas Vondra t...@fuzzy.cz wrote: Fixed. I've kept use_log_agg only and I've removed the default. Also I've added one more check (that the total duration is a multiple of the aggregation interval). Hi Tomas, In the

[HACKERS] Assigning NULL to a record variable

2012-09-20 Thread Kevin Grittner
Related to bug #6123, Wisconsin Courts are now using triggers with the workaround to be safe with the patch put forward by Tom, even though they are still running with the earlier patch proposal by me (which tolerates an UPDATE or DELETE of a row being deleted).  The general structure of the

Re: [HACKERS] Configuration include directory

2012-09-20 Thread Selena Deckelmann
Hello! I've spent a little time with this patch and have attached revision 6. Thanks, Noah, for a fantastically detailed review. The only thing I didn't do that Noah suggested was run pgindent on guc-file.l. A cursory search did not reveal source compatible with my operating system for

Re: [HACKERS] Assigning NULL to a record variable

2012-09-20 Thread Tom Lane
Kevin Grittner kgri...@mail.com writes: ... This did not work for cases where the AFTER DELETE trigger performed an action which was not idempotent because, while return_value was NULL enough to enter that last IF clause, it was not NULL enough to prevent the DELETE attempt and fire subsequent

Re: [HACKERS] Assigning NULL to a record variable

2012-09-20 Thread Pavel Stehule
2012/9/20 Tom Lane t...@sss.pgh.pa.us: Kevin Grittner kgri...@mail.com writes: ... This did not work for cases where the AFTER DELETE trigger performed an action which was not idempotent because, while return_value was NULL enough to enter that last IF clause, it was not NULL enough to

Re: [HACKERS] [COMMITTERS] pgsql: Properly set relpersistence for fake relcache entries.

2012-09-20 Thread Andres Freund
On Monday, September 17, 2012 03:58:37 PM Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: Btw, I played with this some more on Saturday and I think, while definitely a bad bug, the actual consequences aren't as bad as at least I initially feared. Fake relcache entries are

Re: [HACKERS] Assigning NULL to a record variable

2012-09-20 Thread Tom Lane
Pavel Stehule pavel.steh...@gmail.com writes: 2012/9/20 Tom Lane t...@sss.pgh.pa.us: I'm not sure what the performance tradeoffs would be --- some things would get faster and others slower, probably, since field access would be more work but conversion to/from HeapTuple would get far cheaper.

Re: [HACKERS] alter enum add value if not exists

2012-09-20 Thread Tom Lane
Andrew Dunstan and...@dunslane.net writes: On 08/23/2012 07:39 AM, Magnus Hagander wrote: It doesn't break, of course ,since it's protected by the unique index. But aren't you at risk of getting the very error message you're trying to avoid? Yeah, looking further this was probably a thinko

Re: [HACKERS] 64-bit API for large object

2012-09-20 Thread Nozomi Anzai
3) Backend inv_api.c functions(Nozomi Anzai) No need to add new functions. Just extend them to handle 64-bit data. BTW , what will happen if older 32-bit libpq accesses large objects over 2GB? lo_read and lo_write: they can read or write lobjs using 32-bit API as long as

[HACKERS] pg_reorg in core?

2012-09-20 Thread Michael Paquier
Hi all, During the last PGCon, I heard that some community members would be interested in having pg_reorg directly in core. Just to recall, pg_reorg is a functionality developped by NTT that allows to redistribute a table without taking locks on it. The technique it uses to reorganize the table

Re: [HACKERS] pg_reorg in core?

2012-09-20 Thread Josh Kupershmidt
On Thu, Sep 20, 2012 at 7:05 PM, Michael Paquier michael.paqu...@gmail.com wrote: Hi all, During the last PGCon, I heard that some community members would be interested in having pg_reorg directly in core. I'm actually not crazy about this idea, at least not given the current state of

Re: [HACKERS] 64-bit API for large object

2012-09-20 Thread Tatsuo Ishii
To pass 64-bit integer to PQfn, PQArgBlock is used like this: int *ptr is a pointer to 64-bit integer and actual data is placed somewhere else. There might be other way: add new member to union u to store 64-bit integer: typedef struct { int len; int

Re: [HACKERS] 64-bit API for large object

2012-09-20 Thread Tom Lane
Tatsuo Ishii is...@postgresql.org writes: To pass 64-bit integer to PQfn, PQArgBlock is used like this: int *ptr is a pointer to 64-bit integer and actual data is placed somewhere else. Yeah, I think we have to do it like that. Changing the size of PQArgBlock would be a libpq ABI break, which

Re: [HACKERS] pg_reorg in core?

2012-09-20 Thread Michael Paquier
On Fri, Sep 21, 2012 at 12:07 PM, Josh Kupershmidt schmi...@gmail.comwrote: On Thu, Sep 20, 2012 at 7:05 PM, Michael Paquier michael.paqu...@gmail.com wrote: Hi all, During the last PGCon, I heard that some community members would be interested in having pg_reorg directly in core. I'm

Re: [HACKERS] pg_reorg in core?

2012-09-20 Thread Hitoshi Harada
On Thu, Sep 20, 2012 at 7:05 PM, Michael Paquier michael.paqu...@gmail.com wrote: Hi all, During the last PGCon, I heard that some community members would be interested in having pg_reorg directly in core. Just to recall, pg_reorg is a functionality developped by NTT that allows to

Re: [HACKERS] pg_reorg in core?

2012-09-20 Thread Josh Kupershmidt
On Thu, Sep 20, 2012 at 8:33 PM, Michael Paquier michael.paqu...@gmail.com wrote: On Fri, Sep 21, 2012 at 12:07 PM, Josh Kupershmidt schmi...@gmail.com wrote: On Thu, Sep 20, 2012 at 7:05 PM, Michael Paquier michael.paqu...@gmail.com wrote: What could be also great is to move the project

Re: [HACKERS] Assigning NULL to a record variable

2012-09-20 Thread Pavel Stehule
2012/9/20 Tom Lane t...@sss.pgh.pa.us: Pavel Stehule pavel.steh...@gmail.com writes: 2012/9/20 Tom Lane t...@sss.pgh.pa.us: I'm not sure what the performance tradeoffs would be --- some things would get faster and others slower, probably, since field access would be more work but conversion

Re: [v9.3] Extra Daemons (Re: [HACKERS] elegant and effective way for running jobs inside a database)

2012-09-20 Thread Amit Kapila
On Thursday, September 20, 2012 7:13 PM Alvaro Herrera wrote: Excerpts from Amit Kapila's message of jue sep 20 02:10:23 -0300 2012: Why can't worker tasks be also permanent, which can be controlled through configuration. What I mean to say is that if user has need for parallel operations

Re: [v9.3] Extra Daemons (Re: [HACKERS] elegant and effective way for running jobs inside a database)

2012-09-20 Thread Amit Kapila
On Thursday, September 20, 2012 7:35 PM Kohei KaiGai wrote: 2012/9/20 Amit Kapila amit.kap...@huawei.com: On Thursday, September 20, 2012 1:44 AM Simon Riggs wrote: On 12 September 2012 04:30, Amit Kapila amit.kap...@huawei.com wrote: On Tuesday, September 11, 2012 9:09 PM Alvaro Herrera wrote: