Re: [HACKERS] SQL/MED estimated time of arrival?

2010-11-24 Thread Shigeru HANADA
On Tue, 23 Nov 2010 10:22:02 -0800 Joshua D. Drake j...@commandprompt.com wrote: On Tue, 2010-11-23 at 20:18 +0200, Heikki Linnakangas wrote: On 23.11.2010 14:22, Shigeru HANADA wrote: OID is supported to get oid from the source table (yes, it works only for postgresql_fdw but it seems

Re: [HACKERS] Hot Standby: too many KnownAssignedXids

2010-11-24 Thread Heikki Linnakangas
On 24.11.2010 06:56, Joachim Wieland wrote: On Tue, Nov 23, 2010 at 8:45 AM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: On 19.11.2010 23:46, Joachim Wieland wrote: FATAL: too many KnownAssignedXids. head: 0, tail: 0, nxids: 9978, pArray-maxKnownAssignedXids: 6890 Hmm,

Re: [HACKERS] Hot Standby: too many KnownAssignedXids

2010-11-24 Thread Heikki Linnakangas
On 24.11.2010 12:48, Heikki Linnakangas wrote: On 24.11.2010 06:56, Joachim Wieland wrote: On Tue, Nov 23, 2010 at 8:45 AM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: On 19.11.2010 23:46, Joachim Wieland wrote: FATAL: too many KnownAssignedXids. head: 0, tail: 0, nxids:

Re: [HACKERS] Hot Standby: too many KnownAssignedXids

2010-11-24 Thread Heikki Linnakangas
On 24.11.2010 13:38, Heikki Linnakangas wrote: It's dangerous to initialize latestObservedXid to anything to an older value. older value than the nextXid-1 from the checkpoint record, I meant to say. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via

Re: [HACKERS] Instrument checkpoint sync calls

2010-11-24 Thread Robert Haas
On Wed, Nov 24, 2010 at 1:02 AM, Greg Smith g...@2ndquadrant.com wrote: Robert Haas wrote: Did this get eaten by the email goblin, or you're still working on it? Fell behind due to an unfortunately timed bit of pneumonia.  Hurray for the health benefits of cross country flights.  Will fix

Re: [HACKERS] profiling connection overhead

2010-11-24 Thread Robert Haas
On Wed, Nov 24, 2010 at 2:10 AM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: Anything we can do about this?  That's a lot of overhead, and it'd be a lot worse on a big machine with 8GB of shared_buffers. Micro-optimizing that search for the non-zero value helps a little bit

Re: [HACKERS] Suggested easy TODO: pg_dump --from-list

2010-11-24 Thread Robert Haas
On Wed, Nov 24, 2010 at 1:15 AM, Tom Lane t...@sss.pgh.pa.us wrote: Josh Berkus j...@agliodbs.com writes: Well, very little about pg_dump is very [E], IMNSHO. The question in my mind here is what format the list file will take I was thinking same format as pg_restore -l, only without the

Re: [HACKERS] final patch - plpgsql: for-in-array

2010-11-24 Thread Robert Haas
On Wed, Nov 24, 2010 at 1:06 AM, Tom Lane t...@sss.pgh.pa.us wrote: =?ISO-8859-1?Q?C=E9dric_Villemain?= cedric.villemain.deb...@gmail.com writes: I think you (Robert) misunderstood dramatically what Pavel try to do. Pavel did an excellent optimization work for a specific point. This specific

Re: [HACKERS] Suggested easy TODO: pg_dump --from-list

2010-11-24 Thread Dmitriy Igrishin
Hey hackers, Completely agree with Robert ! It would be nice to dump functions definitions, e.g. to make it possible keep them in git separately. I also want to propose to make it possible dump function definitions as CREATE OR REPLACE FUNCTION rather than just CREATE FUNCTION (as pg_dump dumps

Re: [HACKERS] Extensions, this time with a patch

2010-11-24 Thread Itagaki Takahiro
On Tue, Nov 23, 2010 at 18:19, Dimitri Fontaine dimi...@2ndquadrant.fr wrote: Please find that done in attached v4 of the cfparser patch. RECOVERY_COMMAND_FILE is opened twice in the patch. The first time is for checking the existence, and the second time is for parsing. Instead of the repeat,

Re: [HACKERS] Suggested easy TODO: pg_dump --from-list

2010-11-24 Thread Andrew Dunstan
On 11/24/2010 07:29 AM, Robert Haas wrote: As a first attempt at syntax, I might suggest something along the lines of object type: object name, where the types and names might look to COMMENT ON for inspiration. pg_dump already uses a list of object types (e.g. as seen in the output from

Re: [HACKERS] Suggested easy TODO: pg_dump --from-list

2010-11-24 Thread Robert Haas
On Wed, Nov 24, 2010 at 8:41 AM, Andrew Dunstan and...@dunslane.net wrote: On 11/24/2010 07:29 AM, Robert Haas wrote:  As a first attempt at syntax, I might suggest something along the lines of object type: object name, where the types and names might look to COMMENT ON for inspiration.

[HACKERS] requested feature: limit for text or bytea parameters in log

2010-11-24 Thread Pavel Stehule
Hello We use a very rich log - for pgFouine processing. But sometime there are logger 1MB parameters. It's absolutely useless. These long values can be replaced by first n chars ... truncated original length: 223636, md5: jhagjkafhskdjfhdsf Regards Pavel Stehule -- Sent via

Re: [HACKERS] Suggested easy TODO: pg_dump --from-list

2010-11-24 Thread Joachim Wieland
On Wed, Nov 24, 2010 at 1:15 AM, Tom Lane t...@sss.pgh.pa.us wrote: Nope ... those strings are just helpful comments, they aren't really guaranteed to be unique identifiers.  In any case, it seems unlikely that a user could expect to get the more complicated cases exactly right other than by

Re: [HACKERS] Suggested easy TODO: pg_dump --from-list

2010-11-24 Thread Andrew Dunstan
On 11/24/2010 09:05 AM, Joachim Wieland wrote: On Wed, Nov 24, 2010 at 1:15 AM, Tom Lanet...@sss.pgh.pa.us wrote: Nope ... those strings are just helpful comments, they aren't really guaranteed to be unique identifiers. In any case, it seems unlikely that a user could expect to get the more

Re: [HACKERS] Suggested easy TODO: pg_dump --from-list

2010-11-24 Thread Tom Lane
Joachim Wieland j...@mcknight.de writes: On Wed, Nov 24, 2010 at 1:15 AM, Tom Lane t...@sss.pgh.pa.us wrote: Nope ... those strings are just helpful comments, they aren't really guaranteed to be unique identifiers.  In any case, it seems unlikely that a user could expect to get the more

Re: [HACKERS] Suggested easy TODO: pg_dump --from-list

2010-11-24 Thread Robert Haas
On Wed, Nov 24, 2010 at 9:52 AM, Tom Lane t...@sss.pgh.pa.us wrote: Actually, what occurs to me to wonder is whether the facility has to be guaranteed unique at all.  If for instance you have a group of overloaded functions, is there really a big use-case for dumping just one and not the whole

Re: [HACKERS] Hot Standby: too many KnownAssignedXids

2010-11-24 Thread Simon Riggs
On Wed, 2010-11-24 at 12:48 +0200, Heikki Linnakangas wrote: When recovery starts, we fetch the oldestActiveXid from the checkpoint record. Let's say that it's 100. We then start replaying WAL records from the Redo pointer, and the first record (heap insert in your case) contains an Xid

Re: [HACKERS] profiling connection overhead

2010-11-24 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Wed, Nov 24, 2010 at 2:10 AM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: Micro-optimizing that search for the non-zero value helps a little bit (attached). Reduces the percentage shown by oprofile from about 16% to 12% on my

Re: [HACKERS] Suggested easy TODO: pg_dump --from-list

2010-11-24 Thread Joachim Wieland
On Wed, Nov 24, 2010 at 9:38 AM, Andrew Dunstan and...@dunslane.net wrote: It would be unique, but a pain in the neck for users to get. Robert's idea will have more traction with users. Whatever approach we use, we need to think about the use case where 1% of the objects should be dumped but

Re: [HACKERS] final patch - plpgsql: for-in-array

2010-11-24 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: Right, that was my impression, too. But, I think this may be partly a case of people talking past each other. My impression of this conversation was a repetition of this sequence: A: This syntax is bad. B: But it's way faster! ...which makes no

Re: [HACKERS] Suggested easy TODO: pg_dump --from-list

2010-11-24 Thread Tom Lane
Dmitriy Igrishin dmit...@gmail.com writes: I also want to propose to make it possible dump function definitions as CREATE OR REPLACE FUNCTION rather than just CREATE FUNCTION (as pg_dump dumps them now). It's intentional that pg_dump doesn't do that. Please don't think that pg_dump is a

Re: [HACKERS] Suggested easy TODO: pg_dump --from-list

2010-11-24 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Wed, Nov 24, 2010 at 9:52 AM, Tom Lane t...@sss.pgh.pa.us wrote: Actually, what occurs to me to wonder is whether the facility has to be guaranteed unique at all.  If for instance you have a group of overloaded functions, is there really a big

Re: [HACKERS] Suggested easy TODO: pg_dump --from-list

2010-11-24 Thread Tom Lane
Joachim Wieland j...@mcknight.de writes: Whatever approach we use, we need to think about the use case where 1% of the objects should be dumped but should also make sure that you can more or less easily dump 99% of the objects. Roberts use case is the 1% use case. Especially for the 99% case

Re: [HACKERS] Suggested easy TODO: pg_dump --from-list

2010-11-24 Thread Robert Haas
On Wed, Nov 24, 2010 at 10:45 AM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Wed, Nov 24, 2010 at 9:52 AM, Tom Lane t...@sss.pgh.pa.us wrote: Actually, what occurs to me to wonder is whether the facility has to be guaranteed unique at all.  If for instance

Re: [HACKERS] profiling connection overhead

2010-11-24 Thread Robert Haas
On Wed, Nov 24, 2010 at 10:25 AM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: The first optimization that occurred to me was remove the loop altogether. Or make it execute only in assert-enabled mode, perhaps. This check had some use back in the bad old

Re: [HACKERS] profiling connection overhead

2010-11-24 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Wed, Nov 24, 2010 at 10:25 AM, Tom Lane t...@sss.pgh.pa.us wrote: Or make it execute only in assert-enabled mode, perhaps. But making the check execute only in assert-enabled more doesn't seem right, since the check actually acts to mask other

Re: [HACKERS] Suggested easy TODO: pg_dump --from-list

2010-11-24 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Wed, Nov 24, 2010 at 10:45 AM, Tom Lane t...@sss.pgh.pa.us wrote: It *will* be manifestly harder to use if users have to spell the argument types just so.  Consider int4 versus integer, varchar versus character varying (and not character

Re: [HACKERS] final patch - plpgsql: for-in-array

2010-11-24 Thread Pavel Stehule
Hello 2010/11/24 Tom Lane t...@sss.pgh.pa.us: Robert Haas robertmh...@gmail.com writes: Right, that was my impression, too.  But, I think this may be partly a case of people talking past each other.  My impression of this conversation was a repetition of this sequence: A: This syntax is

Re: [HACKERS] [JDBC] JDBC and Binary protocol error, for some statements

2010-11-24 Thread Maciek Sakrejda
Result is oid=23, format=(0) T, value = 0x00,0x00,0x00,0x02 What do you mean regarding the format? Are you just inferring that from the data? If memory serves, the format of a particular column is not specified anywhere other than the RowDescription, and according to your JDBC log output above,

Re: [HACKERS] profiling connection overhead

2010-11-24 Thread Robert Haas
On Wed, Nov 24, 2010 at 11:33 AM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Wed, Nov 24, 2010 at 10:25 AM, Tom Lane t...@sss.pgh.pa.us wrote: Or make it execute only in assert-enabled mode, perhaps. But making the check execute only in assert-enabled more

Re: [HACKERS] profiling connection overhead

2010-11-24 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: OK, patch attached. Two comments: 1. A comment would help, something like Assert we released all buffer pins. 2. AtProcExit_LocalBuffers should be redone the same way, for consistency (it likely won't make any performance difference). Note the comment

Re: [HACKERS] final patch - plpgsql: for-in-array

2010-11-24 Thread Robert Haas
On Wed, Nov 24, 2010 at 10:33 AM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: Right, that was my impression, too.  But, I think this may be partly a case of people talking past each other.  My impression of this conversation was a repetition of this sequence:

Re: [HACKERS] 8.4-vintage problem in postmaster.c

2010-11-24 Thread Stefan Kaltenbrunner
On 11/15/2010 03:24 PM, Alvaro Herrera wrote: Excerpts from Tom Lane's message of sáb nov 13 19:07:50 -0300 2010: Stefan Kaltenbrunnerste...@kaltenbrunner.cc writes: On 11/13/2010 06:58 PM, Tom Lane wrote: Just looking at it, I think that the logic in canAcceptConnections got broken by

Re: [HACKERS] [JDBC] JDBC and Binary protocol error, for some statements

2010-11-24 Thread Radosław Smogura
I didn't described log correctly, 1st attached response is normal execution; flags QUERY_SUPPRESS_BEGIN | QUERY_ONESHOT, 2nd is compiled statement QUERY_SUPPRESS_BEGIN only. Text format is marked as 0, binary format is 1. The 1st shown execution (flags=17) is good, it tells that result is

Re: [HACKERS] profiling connection overhead

2010-11-24 Thread Robert Haas
On Wed, Nov 24, 2010 at 1:06 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: OK, patch attached. Two comments: Revised patch attached. I tried configuring oprofile with --callgraph=10 and then running oprofile with -c, but it gives kooky looking output I

Re: [HACKERS] [JDBC] JDBC and Binary protocol error, for some statements

2010-11-24 Thread Maciek Sakrejda
Text format is marked as 0, binary format is 1. Sorry--I misread the docs. This is consistent and something does look fishy. Thanks for the tarball. I can take a look tonight. --- Maciek Sakrejda | System Architect | Truviso 1065 E. Hillsdale Blvd., Suite 215 Foster City, CA 94404 (650)

Re: [HACKERS] profiling connection overhead

2010-11-24 Thread Gerhard Heift
On Wed, Nov 24, 2010 at 01:20:36PM -0500, Robert Haas wrote: On Wed, Nov 24, 2010 at 1:06 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: OK, patch attached. Two comments: Revised patch attached. I tried configuring oprofile with --callgraph=10 and

Re: [HACKERS] profiling connection overhead

2010-11-24 Thread Andres Freund
On Wednesday 24 November 2010 19:01:32 Robert Haas wrote: Somehow I don't think I'm going to get much further with this without figuring out how to get oprofile to cough up a call graph. I think to do that sensibly you need CFLAGS=-O2 -fno-omit-frame-pointer... -- Sent via pgsql-hackers

[HACKERS] function(contants) evaluated for every row

2010-11-24 Thread Bruce Momjian
Someone offlist reported query slowness because we don't convert function calls with all-constant parameters to be a constants before we start a sequential scan: EXPLAIN SELECT * FROM test WHERE x = to_date('2001-01-01', '-MM-DD') AND x =

Re: [HACKERS] function(contants) evaluated for every row

2010-11-24 Thread Tom Lane
Bruce Momjian br...@momjian.us writes: Notice the to_date()'s were not converted to constants in EXPLAIN so they are evaluated for every row. to_date() is marked STABLE. Is this something we should improve? No. This is per expectation. Only IMMUTABLE functions can be folded to constants in

Re: [HACKERS] function(contants) evaluated for every row

2010-11-24 Thread Bruce Momjian
Tom Lane wrote: Bruce Momjian br...@momjian.us writes: Notice the to_date()'s were not converted to constants in EXPLAIN so they are evaluated for every row. to_date() is marked STABLE. Is this something we should improve? No. This is per expectation. Only IMMUTABLE functions can be

Re: [HACKERS] profiling connection overhead

2010-11-24 Thread Tom Lane
Gerhard Heift ml-postgresql-20081012-3...@gheift.de writes: On Wed, Nov 24, 2010 at 01:20:36PM -0500, Robert Haas wrote: I tried configuring oprofile with --callgraph=10 and then running oprofile with -c, but it gives kooky looking output I can't interpret. Have a look at the wiki:

Re: [HACKERS] Extensions, this time with a patch

2010-11-24 Thread Dimitri Fontaine
Itagaki Takahiro itagaki.takah...@gmail.com writes: RECOVERY_COMMAND_FILE is opened twice in the patch. The first time is for checking the existence, and the second time is for parsing. Instead of the repeat, how about adding FILE* version of parser? It will be also called from

Re: [HACKERS] profiling connection overhead

2010-11-24 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: Revised patch attached. The asserts in AtProcExit_LocalBuffers are a bit pointless since you forgot to remove the code that forcibly zeroes LocalRefCount[]... otherwise +1. regards, tom lane -- Sent via pgsql-hackers mailing

Re: [HACKERS] profiling connection overhead

2010-11-24 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: Full results, and call graph, attached. The first obvious fact is that most of the memset overhead appears to be coming from InitCatCache. AFAICT that must be the palloc0 calls that are zeroing out (mostly) the hash bucket headers. I don't see any

Re: [HACKERS] function(contants) evaluated for every row

2010-11-24 Thread Tom Lane
Bruce Momjian br...@momjian.us writes: I realize they can't be converted to constants before the query starts but is there a reason we can't convert those functions to constants in the executor before a table scan? Other than the significant number of cycles that would be wasted (in most

Re: [HACKERS] Per-column collation

2010-11-24 Thread Peter Eisentraut
On mån, 2010-11-22 at 11:58 +0900, Itagaki Takahiro wrote: * Did you see any performance regression by collation? I found a bug in lc_collate_is_c(); result = 0 should be checked before any other checks. SearchSysCache1() here would be a performance regression. That code turned out to be

Re: [HACKERS] function(contants) evaluated for every row

2010-11-24 Thread Marti Raudsepp
On Wed, Nov 24, 2010 at 21:52, Tom Lane t...@sss.pgh.pa.us wrote: Bruce Momjian br...@momjian.us writes: Notice the to_date()'s were not converted to constants in EXPLAIN so they are evaluated for every row.  to_date() is marked STABLE. No.  This is per expectation.  Only IMMUTABLE functions

Re: [HACKERS] Per-column collation, work in progress

2010-11-24 Thread Peter Eisentraut
On tor, 2010-10-14 at 22:54 -0400, Robert Haas wrote: It seems you've falsified the header comment in pathkeys_useful_for_merging(), although I guess it's already false because it doesn't seem to have been updated for the NULLS ASC/DESC stuff, and the interior comment in

Re: [HACKERS] Per-column collation, work in progress

2010-11-24 Thread Peter Eisentraut
On ons, 2010-09-22 at 19:44 +0900, Itagaki Takahiro wrote: * CREATE TABLE (LIKE table_with_collation) doesn't inherit collations. This was fixed in the CF2010-11 patch. * psql \d needs a separator between collate and not null modifiers. And this as well. -- Sent via pgsql-hackers mailing

Re: [HACKERS] profiling connection overhead

2010-11-24 Thread Robert Haas
On Wed, Nov 24, 2010 at 3:14 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: Full results, and call graph, attached.  The first obvious fact is that most of the memset overhead appears to be coming from InitCatCache. AFAICT that must be the palloc0 calls that

Re: [HACKERS] profiling connection overhead

2010-11-24 Thread Andres Freund
On Wednesday 24 November 2010 21:47:32 Robert Haas wrote: On Wed, Nov 24, 2010 at 3:14 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: Full results, and call graph, attached. The first obvious fact is that most of the memset overhead appears to be coming

Re: [HACKERS] profiling connection overhead

2010-11-24 Thread Robert Haas
On Wed, Nov 24, 2010 at 3:53 PM, Andres Freund and...@anarazel.de wrote: On Wednesday 24 November 2010 21:47:32 Robert Haas wrote: On Wed, Nov 24, 2010 at 3:14 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: Full results, and call graph, attached.  The first

[HACKERS] Regex code versus Unicode chars beyond codepoint 255

2010-11-24 Thread Tom Lane
Bug #5766 points out that we're still not there yet in terms of having sane behavior for locale-specific regex operations in Unicode encoding. The reason it's not working is that regc_locale does this to expand the set of characters that are considered to match [[:alnum:]] : /* *

Re: [HACKERS] profiling connection overhead

2010-11-24 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Wed, Nov 24, 2010 at 3:53 PM, Andres Freund and...@anarazel.de wrote: The idea I had was to go the other way and say, hey, if these hash tables can't be expanded anyway, let's put them on the BSS instead of heap-allocating them. Won't this just

Re: [HACKERS] profiling connection overhead

2010-11-24 Thread Andres Freund
On Wednesday 24 November 2010 21:54:53 Robert Haas wrote: On Wed, Nov 24, 2010 at 3:53 PM, Andres Freund and...@anarazel.de wrote: On Wednesday 24 November 2010 21:47:32 Robert Haas wrote: On Wed, Nov 24, 2010 at 3:14 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com

Re: [HACKERS] profiling pgbench

2010-11-24 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: I did some profiling of pgbench -j 36 -c 36 -T 500 banging on my two-core desktop box - with synchronous_commit turned off to keep the fsyncs from dominating the profile - and got these results: 29634 4.7124 postgres base_yyparse

Re: [HACKERS] profiling pgbench

2010-11-24 Thread Andres Freund
On Wednesday 24 November 2010 21:24:43 Robert Haas wrote: I'd like to get access to a box with (a lot) more cores, to see whether the lock stuff moves up in the profile. A big chunk of that hash_search_with_hash_value overhead is coming from LockAcquireExtended. The __strcmp_sse2 is almost

Re: [HACKERS] profiling connection overhead

2010-11-24 Thread Robert Haas
On Nov 24, 2010, at 4:05 PM, Andres Freund and...@anarazel.de wrote: Won't this just cause loads of additional pagefaults after fork() when those pages are used the first time and then a second time when first written to (to copy it)? Aren't we incurring those page faults anyway, for

Re: [HACKERS] profiling pgbench

2010-11-24 Thread Andres Freund
On Wednesday 24 November 2010 22:14:04 Andres Freund wrote: On Wednesday 24 November 2010 21:24:43 Robert Haas wrote: I'd like to get access to a box with (a lot) more cores, to see whether the lock stuff moves up in the profile. A big chunk of that hash_search_with_hash_value overhead is

Re: [HACKERS] profiling connection overhead

2010-11-24 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Nov 24, 2010, at 4:05 PM, Andres Freund and...@anarazel.de wrote: Yes, but only once. Also scrubbing a page is faster than copying it... (and there were patches floating around to do that in advance, not sure if they got integrated into mainline

Re: [HACKERS] profiling connection overhead

2010-11-24 Thread Andres Freund
On Wednesday 24 November 2010 22:18:08 Robert Haas wrote: On Nov 24, 2010, at 4:05 PM, Andres Freund and...@anarazel.de wrote: Won't this just cause loads of additional pagefaults after fork() when those pages are used the first time and then a second time when first written to (to copy

Re: [HACKERS] profiling connection overhead

2010-11-24 Thread Andres Freund
On Wednesday 24 November 2010 22:25:45 Tom Lane wrote: Robert Haas robertmh...@gmail.com writes: On Nov 24, 2010, at 4:05 PM, Andres Freund and...@anarazel.de wrote: Yes, but only once. Also scrubbing a page is faster than copying it... (and there were patches floating around to do that in

Re: [HACKERS] profiling connection overhead

2010-11-24 Thread Robert Haas
On Wed, Nov 24, 2010 at 4:05 PM, Tom Lane t...@sss.pgh.pa.us wrote: (You might be able to confirm or disprove this theory if you ask oprofile to count memory access stalls instead of CPU clock cycles...) I don't see an event for that. # opcontrol --list-events | grep STALL

Re: [HACKERS] profiling connection overhead

2010-11-24 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Wed, Nov 24, 2010 at 4:05 PM, Tom Lane t...@sss.pgh.pa.us wrote: (You might be able to confirm or disprove this theory if you ask oprofile to count memory access stalls instead of CPU clock cycles...) I don't see an event for that. You probably

Re: [HACKERS] duplicate connection failure messages

2010-11-24 Thread Bruce Momjian
Bruce Momjian wrote: Tom Lane wrote: Bruce Momjian br...@momjian.us writes: I assume you are suggesting to use our inet_net_ntop() even if the system has inet_ntop(). If you're going to have code to do the former, it doesn't seem to be worth the trouble to also have code that does

Re: [HACKERS] profiling connection overhead

2010-11-24 Thread Andres Freund
On Wednesday 24 November 2010 23:03:48 Tom Lane wrote: Robert Haas robertmh...@gmail.com writes: On Wed, Nov 24, 2010 at 4:05 PM, Tom Lane t...@sss.pgh.pa.us wrote: (You might be able to confirm or disprove this theory if you ask oprofile to count memory access stalls instead of CPU clock

Re: [HACKERS] profiling connection overhead

2010-11-24 Thread Robert Haas
On Wed, Nov 24, 2010 at 5:15 PM, Andres Freund and...@anarazel.de wrote: On Wednesday 24 November 2010 23:03:48 Tom Lane wrote: Robert Haas robertmh...@gmail.com writes: On Wed, Nov 24, 2010 at 4:05 PM, Tom Lane t...@sss.pgh.pa.us wrote: (You might be able to confirm or disprove this theory

Re: [HACKERS] profiling pgbench

2010-11-24 Thread Jeff Janes
On Wed, Nov 24, 2010 at 1:19 PM, Andres Freund and...@anarazel.de wrote: On Wednesday 24 November 2010 22:14:04 Andres Freund wrote: On Wednesday 24 November 2010 21:24:43 Robert Haas wrote: Recarding LWLockAcquire costs: Yes, its pretty noticeable - on loads of different usages. On a bunch

Re: [HACKERS] profiling pgbench

2010-11-24 Thread Robert Haas
On Wed, Nov 24, 2010 at 5:33 PM, Jeff Janes jeff.ja...@gmail.com wrote: On Wed, Nov 24, 2010 at 1:19 PM, Andres Freund and...@anarazel.de wrote: On Wednesday 24 November 2010 22:14:04 Andres Freund wrote: On Wednesday 24 November 2010 21:24:43 Robert Haas wrote: Recarding LWLockAcquire costs:

Re: [HACKERS] profiling pgbench

2010-11-24 Thread Andres Freund
On Wednesday 24 November 2010 23:34:46 Robert Haas wrote: On Wed, Nov 24, 2010 at 5:33 PM, Jeff Janes jeff.ja...@gmail.com wrote: On Wed, Nov 24, 2010 at 1:19 PM, Andres Freund and...@anarazel.de wrote: On Wednesday 24 November 2010 22:14:04 Andres Freund wrote: On Wednesday 24 November

Re: [HACKERS] profiling connection overhead

2010-11-24 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: I don't see anything for BUS OUTSTANDING. For CACHE and MISS I have some options: DATA_CACHE_MISSES: (counter: all) L3_CACHE_MISSES: (counter: all) Those two look promising, though I can't claim to be an expert. regards, tom

Re: [HACKERS] profiling connection overhead

2010-11-24 Thread Robert Haas
On Wed, Nov 24, 2010 at 5:42 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: I don't see anything for BUS OUTSTANDING.  For CACHE and MISS I have some options: DATA_CACHE_MISSES: (counter: all) L3_CACHE_MISSES: (counter: all) Those two look promising,

Re: [HACKERS] [COMMITTERS] pgsql: Remove useless whitespace at end of lines

2010-11-24 Thread Tom Lane
Alvaro Herrera alvhe...@commandprompt.com writes: Excerpts from Peter Eisentraut's message of mar nov 23 17:52:18 -0300 2010: Remove useless whitespace at end of lines This was stuck in the moderation queue because of message size limit (30 kB). Is it worth increasing that value? Evidently

Re: [HACKERS] [COMMITTERS] pgsql: Remove useless whitespace at end of lines

2010-11-24 Thread Magnus Hagander
On Wed, Nov 24, 2010 at 23:45, Tom Lane t...@sss.pgh.pa.us wrote: Alvaro Herrera alvhe...@commandprompt.com writes: Excerpts from Peter Eisentraut's message of mar nov 23 17:52:18 -0300 2010: Remove useless whitespace at end of lines This was stuck in the moderation queue because of message

Re: [HACKERS] [COMMITTERS] pgsql: Remove useless whitespace at end of lines

2010-11-24 Thread Bruce Momjian
Magnus Hagander wrote: On Wed, Nov 24, 2010 at 23:45, Tom Lane t...@sss.pgh.pa.us wrote: Alvaro Herrera alvhe...@commandprompt.com writes: Excerpts from Peter Eisentraut's message of mar nov 23 17:52:18 -0300 2010: Remove useless whitespace at end of lines This was stuck in the

Re: [HACKERS] profiling pgbench

2010-11-24 Thread Tom Lane
Jeff Janes jeff.ja...@gmail.com writes: I've played a bit with hash_search_with_hash_value and found that most of the time is spent on shared hash tables, not private ones. And the time attributed to it for the shared hash tables mostly seems to be due to the time it takes to fight cache

Re: [HACKERS] [COMMITTERS] pgsql: Remove useless whitespace at end of lines

2010-11-24 Thread Tom Lane
Magnus Hagander mag...@hagander.net writes: On Wed, Nov 24, 2010 at 23:45, Tom Lane t...@sss.pgh.pa.us wrote: Alvaro Herrera alvhe...@commandprompt.com writes: This was stuck in the moderation queue because of message size limit (30 kB).  Is it worth increasing that value? Evidently we

Re: [HACKERS] profiling pgbench

2010-11-24 Thread Jeff Janes
On Wed, Nov 24, 2010 at 2:34 PM, Robert Haas robertmh...@gmail.com wrote: On Wed, Nov 24, 2010 at 5:33 PM, Jeff Janes jeff.ja...@gmail.com wrote: I've played a bit with hash_search_with_hash_value and found that most of the time is spent on shared hash tables, not private ones.  And the time

Re: [HACKERS] function(contants) evaluated for every row

2010-11-24 Thread Michael Glaesemann
On Nov 24, 2010, at 15:28 , Marti Raudsepp wrote: On Wed, Nov 24, 2010 at 21:52, Tom Lane t...@sss.pgh.pa.us wrote: Bruce Momjian br...@momjian.us writes: Notice the to_date()'s were not converted to constants in EXPLAIN so they are evaluated for every row. to_date() is marked STABLE.

Re: [HACKERS] ALTER TABLE ... IF EXISTS feature?

2010-11-24 Thread Bruce Momjian
Robert Haas wrote: With respect to the syntax itself, I have mixed feelings. On the one hand, I'm a big fan of CREATE IF NOT EXISTS and DROP IF EXISTS precisely because I believe they handle many common cases that people want in real life without much hullabaloo. But, there's clearly some

Re: [HACKERS] duplicate connection failure messages

2010-11-24 Thread Alvaro Herrera
Excerpts from Bruce Momjian's message of mié nov 24 19:04:30 -0300 2010: Bruce Momjian wrote: OK, we will not call inet_ntop() at all. I moved the CIDR part of adt/inet_net_ntop.c into adt/inet_cidr_ntop.c, and moved the remaining net part to /port/inet_net_ntop.c. Applied. This broke

Re: [HACKERS] ALTER TABLE ... IF EXISTS feature?

2010-11-24 Thread Daniel Farina
On Wed, Nov 24, 2010 at 4:30 PM, Bruce Momjian br...@momjian.us wrote: Robert Haas wrote: With respect to the syntax itself, I have mixed feelings.  On the one hand, I'm a big fan of CREATE IF NOT EXISTS and DROP IF EXISTS precisely because I believe they handle many common cases that people

[HACKERS] Re: [COMMITTERS] pgsql: Document that a CHECKPOINT before taking a file system snapshot

2010-11-24 Thread Cédric Villemain
2010/11/25 Bruce Momjian br...@momjian.us: Document that a CHECKPOINT before taking a file system snapshot can reduce recovery time. I didn't follow that on -hackers, but : * checkpoint take place in the pg_start_backup process (before it releases the hand, so before you can start snapshoting)

Re: [HACKERS] profiling pgbench

2010-11-24 Thread Robert Haas
On Nov 24, 2010, at 5:49 PM, Tom Lane t...@sss.pgh.pa.us wrote: Jeff Janes jeff.ja...@gmail.com writes: I've played a bit with hash_search_with_hash_value and found that most of the time is spent on shared hash tables, not private ones. And the time attributed to it for the shared hash tables

Re: [HACKERS] duplicate connection failure messages

2010-11-24 Thread Bruce Momjian
Alvaro Herrera wrote: Excerpts from Bruce Momjian's message of nov 24 19:04:30 -0300 2010: Bruce Momjian wrote: OK, we will not call inet_ntop() at all. I moved the CIDR part of adt/inet_net_ntop.c into adt/inet_cidr_ntop.c, and moved the remaining net part to /port/inet_net_ntop.c.

[HACKERS] Re: [COMMITTERS] pgsql: Document that a CHECKPOINT before taking a file system snapshot

2010-11-24 Thread Bruce Momjian
C?dric Villemain wrote: 2010/11/25 Bruce Momjian br...@momjian.us: Document that a CHECKPOINT before taking a file system snapshot can reduce recovery time. I didn't follow that on -hackers, but : * checkpoint take place in the pg_start_backup process (before it releases the hand, so

Re: [HACKERS] ALTER TABLE ... IF EXISTS feature?

2010-11-24 Thread Bruce Momjian
Daniel Farina wrote: On Wed, Nov 24, 2010 at 4:30 PM, Bruce Momjian br...@momjian.us wrote: Robert Haas wrote: With respect to the syntax itself, I have mixed feelings. ?On the one hand, I'm a big fan of CREATE IF NOT EXISTS and DROP IF EXISTS precisely because I believe they handle many

Re: [HACKERS] ALTER TABLE ... IF EXISTS feature?

2010-11-24 Thread Daniel Farina
On Wed, Nov 24, 2010 at 7:03 PM, Bruce Momjian br...@momjian.us wrote: Daniel Farina wrote: On Wed, Nov 24, 2010 at 4:30 PM, Bruce Momjian br...@momjian.us wrote: Robert Haas wrote: With respect to the syntax itself, I have mixed feelings. ?On the one hand, I'm a big fan of CREATE IF NOT

Re: [HACKERS] ALTER TABLE ... IF EXISTS feature?

2010-11-24 Thread Bruce Momjian
Daniel Farina wrote: On Wed, Nov 24, 2010 at 7:03 PM, Bruce Momjian br...@momjian.us wrote: Daniel Farina wrote: On Wed, Nov 24, 2010 at 4:30 PM, Bruce Momjian br...@momjian.us wrote: Robert Haas wrote: With respect to the syntax itself, I have mixed feelings. ?On the one hand, I'm a

Re: [HACKERS] duplicate connection failure messages

2010-11-24 Thread Bruce Momjian
bruce wrote: Alvaro Herrera wrote: Excerpts from Bruce Momjian's message of nov 24 19:04:30 -0300 2010: Bruce Momjian wrote: OK, we will not call inet_ntop() at all. I moved the CIDR part of adt/inet_net_ntop.c into adt/inet_cidr_ntop.c, and moved the remaining net part to

Re: [HACKERS] ALTER TABLE ... IF EXISTS feature?

2010-11-24 Thread Daniel Farina
On Wed, Nov 24, 2010 at 7:21 PM, Bruce Momjian br...@momjian.us wrote: What are we adding a pl/pgsql dependency for?  What is the benefit that will warrant requiring people who disable plpgsql to enable it for restores? There are two use cases I want to cover: 1) It should be possible to

Re: [HACKERS] Per-column collation, work in progress

2010-11-24 Thread Robert Haas
On Wed, Nov 24, 2010 at 3:37 PM, Peter Eisentraut pete...@gmx.net wrote: On tor, 2010-10-14 at 22:54 -0400, Robert Haas wrote: It seems you've falsified the header comment in pathkeys_useful_for_merging(), although I guess it's already false because it doesn't seem to have been updated for the

Re: [HACKERS] Tab completion for view triggers in psql

2010-11-24 Thread Josh Kupershmidt
On Tue, Nov 23, 2010 at 10:21 PM, David Fetter da...@fetter.org wrote: Please find attached a patch changing both this and updateable to updatable, also per the very handy git grep I just learned about :) I looked a little more at this patch today. I didn't find any serious problems, though it

Re: [HACKERS] security hooks on object creation

2010-11-24 Thread KaiGai Kohei
The attached patch is a revised patch. - The utils/hooks.h was renamed to catalog/objectaccess.h - Numeric in the tail of InvokeObjectAccessHook0() has gone. - Fixed bug in ATExecAddColumn; it gave AttributeRelationId to the hook instead of RelationRelationId. In addition, I found that we

Re: [HACKERS] Label switcher function

2010-11-24 Thread KaiGai Kohei
The attached patch is a revised one. It provides two hooks; the one informs core PG whether the supplied function needs to be hooked, or not. the other is an actual hook on prepare, start, end and abort of function invocations. typedef bool (*needs_function_call_type)(Oid fn_oid); typedef

Re: [HACKERS] Extensions, this time with a patch

2010-11-24 Thread Itagaki Takahiro
On Thu, Nov 25, 2010 at 05:02, Dimitri Fontaine dimi...@2ndquadrant.fr wrote: Something like the attached, version 5 of the patch? I've been using the function name ParseConfigFp because the internal parameter was called fp in the previous function body. I suppose that could easily be changed

Re: [HACKERS] contrib: auth_delay module

2010-11-24 Thread KaiGai Kohei
(2010/11/19 16:57), KaiGai Kohei wrote: (2010/11/18 2:17), Robert Haas wrote: On Wed, Nov 17, 2010 at 10:32 AM, Ross J. Reedstromreeds...@rice.edu wrote: On Tue, Nov 16, 2010 at 09:41:37PM -0500, Robert Haas wrote: On Tue, Nov 16, 2010 at 8:15 PM, KaiGai Koheikai...@ak.jp.nec.com wrote: If

[HACKERS] pg_execute_from_file review

2010-11-24 Thread Joshua Tolley
I've just looked at pg_execute_from_file[1]. The idea here is to execute all the SQL commands in a given file. My comments: * It applies well enough, and builds fine * It seems to work, and I've not come up with a way to make it break * It seems useful, and to follow the limited design discussion

  1   2   >