Re: [HACKERS] Inserting heap tuples in bulk in COPY

2011-11-26 Thread Heikki Linnakangas
On 25.11.2011 23:32, Jeff Janes wrote: On Fri, Nov 25, 2011 at 12:53 PM, Jeff Janesjeff.ja...@gmail.com wrote: Thanks for this patch. Doing bulk copies in parallel for me is now limited by the IO subsystem rather than the CPU. This patch, commit number d326d9e8ea1d69, causes fillfactor to

Re: [HACKERS] vpath builds and verbose error messages

2011-11-26 Thread Peter Eisentraut
On fre, 2011-11-18 at 09:44 -0500, Tom Lane wrote: It wouldn't be that hard for elog.c to do strrchr(fname, '/') or something like that, Here is a patch for that. I would also like to backpatch this. diff --git i/src/backend/utils/error/elog.c w/src/backend/utils/error/elog.c index

Re: [HACKERS] psql \ir filename normalization

2011-11-26 Thread Bruce Momjian
Bruce Momjian wrote: Robert Haas wrote: On Mon, Nov 21, 2011 at 2:30 PM, Bruce Momjian br...@momjian.us wrote: Robert Haas wrote: On Mon, Nov 21, 2011 at 1:05 PM, Bruce Momjian br...@momjian.us wrote: Robert Haas wrote: Argh. ?The root of the problem here seems to be that

Re: [HACKERS] proposal : backend startup hook / after logon trigger

2011-11-26 Thread Tomas Vondra
Dne 25.11.2011 17:48, Tom Lane napsal(a): Tomas Vondra t...@fuzzy.cz writes: On 25 Listopad 2011, 2:44, Robert Haas wrote: I've thought of this before, but I'm not exactly clear on what the use cases are. The startup hook is useful for initializing an extension written in C, when the

Re: [HACKERS] vpath builds and verbose error messages

2011-11-26 Thread Tom Lane
Peter Eisentraut pete...@gmx.net writes: On fre, 2011-11-18 at 09:44 -0500, Tom Lane wrote: It wouldn't be that hard for elog.c to do strrchr(fname, '/') or something like that, Here is a patch for that. I would also like to backpatch this. Hmmm ... is it possible that strrchr could change

[HACKERS] Avoiding repeated snapshot computation

2011-11-26 Thread Pavan Deolasee
On some recent benchmarks and profile data, I saw GetSnapshotData figures at the very top or near top. For lesser number of clients, it can account for 10-20% of time, but more number of clients I have seen it taking up as much as 40% of sample time. Unfortunately, the machine of which I was

Re: [HACKERS] [COMMITTERS] pgsql: Modify pg_dump to use error-free memory allocation macros. This

2011-11-26 Thread Tom Lane
Bruce Momjian br...@momjian.us writes: Bruce Momjian wrote: Tom Lane wrote: object to arbitrarily moving a bunch of longstanding code from one file to another. What that is mainly going to accomplish is creating a big headache whenever we have to back-patch fixes that touch that code ...

Re: [HACKERS] psql setenv command

2011-11-26 Thread Andrew Dunstan
On 11/20/2011 11:07 AM, Josh Kupershmidt wrote: On Wed, Nov 2, 2011 at 5:36 PM, Andrew Dunstanand...@dunslane.net wrote: Updated patch is attached - adding to Nov commitfest. Review of the v2 patch: * Submission Review Patch applies with some fuzz and builds without warnings. I noticed

Re: [HACKERS] plpython SPI cursors

2011-11-26 Thread Steve Singer
On 11-11-23 01:58 PM, Jan Urbański wrote: On 20/11/11 19:14, Steve Singer wrote: On 11-10-15 07:28 PM, Jan Urbański wrote: Hi, attached is a patch implementing the usage of SPI cursors in PL/Python. Currently when trying to process a large table in PL/Python you have slurp it all into memory

Re: [HACKERS] Avoiding repeated snapshot computation

2011-11-26 Thread Robert Haas
On Sat, Nov 26, 2011 at 10:52 AM, Pavan Deolasee pavan.deola...@gmail.com wrote: What we can do is when a transaction comes to compute its snapshot, it checks if some other transaction is already computing a snapshot for itself. If so, it just sleeps on the lock. When the other process

[HACKERS] Re: [COMMITTERS] pgsql: Modify pg_dump to use error-free memory allocation macros. This

2011-11-26 Thread Bruce Momjian
Tom Lane wrote: It is not possible to just link the old common.c into pg_restore and pg_dumpall because it contains calls to pg_dump-only functions. Ideas? The only other idea I have is to keep most functions in the mis-named common.c and move the memory functions into dumpmem.c and

Re: [HACKERS] Re: [COMMITTERS] pgsql: Force strings passed to and from plperl to be in UTF8 encoding.

2011-11-26 Thread Andrew Dunstan
On 10/05/2011 03:58 AM, Amit Khandekar wrote: On 5 October 2011 12:29, Alex Hunsakerbada...@gmail.com wrote: Find it attached. [ Note I didn't put the check inside the if (ret == utf8_str) as it seemed a bit cleaner (indentation wise) to have it outside ] I have no more issues with the

Re: [HACKERS] [COMMITTERS] pgsql: Modify pg_dump to use error-free memory allocation macros. This

2011-11-26 Thread Tom Lane
Bruce Momjian br...@momjian.us writes: Basically, if we keep the existing functions in common.c, we are deciding that it is never worth moving C functions to new C files for source code clarity. I was not sure we had made that decision in the past. I once held hope that git would let us be

Re: [HACKERS] Avoiding repeated snapshot computation

2011-11-26 Thread Pavan Deolasee
On Sat, Nov 26, 2011 at 10:43 PM, Robert Haas robertmh...@gmail.com wrote: On Sat, Nov 26, 2011 at 10:52 AM, Pavan Deolasee pavan.deola...@gmail.com wrote: What we can do is when a transaction comes to compute its snapshot, it checks if some other transaction is already computing a snapshot

Re: [HACKERS] Avoiding repeated snapshot computation

2011-11-26 Thread Tom Lane
Pavan Deolasee pavan.deola...@gmail.com writes: On Sat, Nov 26, 2011 at 10:43 PM, Robert Haas robertmh...@gmail.com wrote: Furthermore, it's hard to understand how this could be a net improvement in the general case, because now both B and F are copying everything twice (once to the shared

Re: [HACKERS] Inlining comparators as a performance optimisation

2011-11-26 Thread Peter Geoghegan
3 items are attached: 1. A spreadsheet, results.ods, which has results for automated multiple runs of the doubled up dellstore orderlines queries (where orderlines is 385 MB). Results are sorted, with the median (or the lower middle value, didn't get the mean of the two middle runs) result for

Re: [HACKERS] Notes on implementing URI syntax for libpq

2011-11-26 Thread Alexander Shulgin
Excerpts from Peter Eisentraut's message of Thu Nov 24 22:05:09 +0200 2011: On tor, 2011-11-24 at 15:43 +0200, Alexander Shulgin wrote: Huh? The service definitions are read from a local pg_service.conf, and are specified by setting PGSERVICE (and PGSERVICEFILE) environment variables,

Re: [HACKERS] Prep object creation hooks, and related sepgsql updates

2011-11-26 Thread Dimitri Fontaine
Kohei KaiGai kai...@kaigai.gr.jp writes: We still don't have clear direction of the way to implement external permission checks on object creation time. So, please consider these patches are on the proof-of-concept stage; using prep-creation-hook to permission checks. I wonder if you could

[HACKERS] GiST range-contained-by searches versus empty ranges

2011-11-26 Thread Tom Lane
I started to wonder why the test in range_gist_consistent_int() for RANGESTRAT_CONTAINED_BY was return true (ie, search the entire index) rather than range_overlaps, which is what is tested in the comparable case in rtree_internal_consistent(). The regression tests showed me how come: an

Re: [HACKERS] Notes on implementing URI syntax for libpq

2011-11-26 Thread Alexander Shulgin
Excerpts from Robert Haas's message of Thu Nov 24 15:59:08 +0200 2011: I think we could do something like: postgresql://user:pw@host:port/database?param1=val1param2=val2param3=val3... I wonder if this should be allowed syntax (i.e. specify a user, but connect locally, so leave 'host' to

Re: [HACKERS] why do we need two snapshots per query?

2011-11-26 Thread Dimitri Fontaine
Hi, Robert Haas robertmh...@gmail.com writes: On Sun, Nov 13, 2011 at 8:57 PM, Robert Haas robertmh...@gmail.com wrote: In the -M extended case, we take a snapshot from exec_parse_message(), and the same two in the exec_bind_message() call that are taken in the -M prepared case.  So reducing

Re: [HACKERS] Notes on implementing URI syntax for libpq

2011-11-26 Thread Alexander Shulgin
Excerpts from Robert Haas's message of Thu Nov 24 13:57:17 +0200 2011: I think it would be really weird not to support user:pw@host:port. You can presumably also support the JDBC style for backward compatibility, but I don't think we should adopt that syntax as project standard. By the

Re: [HACKERS] Patch: Perl xsubpp

2011-11-26 Thread Andrew Dunstan
On 10/12/2011 08:55 PM, Alex Hunsaker wrote: On Wed, Oct 12, 2011 at 17:53, David E. Wheelerda...@kineticode.com wrote: On Sep 15, 2011, at 3:04 PM, Alex Hunsaker wrote: Close, seems I was wrong about the typemap ExtUtils::ParseXS does not install a new one so we still need to point to the

[HACKERS] psql line number reporting from stdin

2011-11-26 Thread Peter Eisentraut
There is a long-standing oddity in psql that running psql -f foo.sql returns error messages with file name and line number, like psql:foo.sql:1: ERROR: syntax error at or near foo but running psql foo.sql does not. I suggest we change the latter to print psql:stdin:1: ERROR: syntax error

Re: [HACKERS] Avoiding repeated snapshot computation

2011-11-26 Thread Andres Freund
Hi, On Saturday, November 26, 2011 04:52:50 PM Pavan Deolasee wrote: I think now that we have reduced the run time of the function itself, we should now try to reduce the number of times the function is called. Robert proposed a way to reduce the number of calls per transaction. I think we

Re: [HACKERS] Avoiding repeated snapshot computation

2011-11-26 Thread Tom Lane
Andres Freund and...@anarazel.de writes: You could also try if it makes a difference reducing SnapshotData to one instead of two cachelines. The data itself fits into one without problems. Trivial patch attached. On what grounds do you argue that this patch gets SnapshotData into one

Re: [HACKERS] [PATCH] PostgreSQL fails to build with 32bit MinGW-w64

2011-11-26 Thread Andrew Dunstan
On 11/24/2011 06:29 AM, Lars Kanis wrote: Isn't it better to check the value of macros itsef rather than checking for system dependent macros that does not directly relate to the issue? specifically for getaddrinfo.c case I think #if EAI_NODATA != EAI_NONAME is a better check than checking for

Re: [HACKERS] Avoiding repeated snapshot computation

2011-11-26 Thread Andres Freund
On Saturday, November 26, 2011 09:52:17 PM Tom Lane wrote: Andres Freund and...@anarazel.de writes: You could also try if it makes a difference reducing SnapshotData to one instead of two cachelines. The data itself fits into one without problems. Trivial patch attached. On what grounds do

Re: [HACKERS] Avoiding repeated snapshot computation

2011-11-26 Thread Andres Freund
On Saturday, November 26, 2011 09:52:17 PM Tom Lane wrote: I'd just as soon keep the fields in a logical order. Btw, I don't think the new order is necessarily worse than the old one. Andres -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your

Re: [HACKERS] Avoiding repeated snapshot computation

2011-11-26 Thread Robert Haas
On Sat, Nov 26, 2011 at 5:28 PM, Andres Freund and...@anarazel.de wrote: On Saturday, November 26, 2011 09:52:17 PM Tom Lane wrote: I'd just as soon keep the fields in a logical order. Btw, I don't think the new order is necessarily worse than the old one. You forget to attach the benchmark

Re: [HACKERS] Avoiding repeated snapshot computation

2011-11-26 Thread Andres Freund
On Saturday, November 26, 2011 11:39:23 PM Robert Haas wrote: On Sat, Nov 26, 2011 at 5:28 PM, Andres Freund and...@anarazel.de wrote: On Saturday, November 26, 2011 09:52:17 PM Tom Lane wrote: I'd just as soon keep the fields in a logical order. Btw, I don't think the new order is

Re: [HACKERS] Avoiding repeated snapshot computation

2011-11-26 Thread Andres Freund
On Saturday, November 26, 2011 11:39:23 PM Robert Haas wrote: On Sat, Nov 26, 2011 at 5:28 PM, Andres Freund and...@anarazel.de wrote: On Saturday, November 26, 2011 09:52:17 PM Tom Lane wrote: I'd just as soon keep the fields in a logical order. Btw, I don't think the new order is

Re: [HACKERS] Avoiding repeated snapshot computation

2011-11-26 Thread Kevin Grittner
Andres Freund wrote: All current x86 cpus use 64bytes. That matches what I found in recent research on the topic. -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] GiST range-contained-by searches versus empty ranges

2011-11-26 Thread Tom Lane
I wrote: I started to wonder why the test in range_gist_consistent_int() for RANGESTRAT_CONTAINED_BY was return true (ie, search the entire index) rather than range_overlaps, which is what is tested in the comparable case in rtree_internal_consistent(). The regression tests showed me how