Re: [HACKERS] [PATCH 10/16] Introduce the concept that wal has a 'origin' node

2012-06-19 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: On Monday, June 18, 2012 11:51:27 PM Daniel Farina wrote: What's the cost of going a lot higher? Because if one makes enough numerical space available, one can assign node identities without a coordinator, a massive decrease in complexity. It

Re: [HACKERS] Resource Owner reassign Locks

2012-06-19 Thread Amit Kapila
And it doesn't seem right for ResourceOwnerRemember/ForgetLock to have to accept a NULL owner. I am not sure, if it can ever enter into this flow without resowner as mentioned in jeff comments for session level locks. If it cannot enter then it is okay. Please take a look to see if I broke

Re: [HACKERS] Resource Owner reassign Locks

2012-06-19 Thread Heikki Linnakangas
On 19.06.2012 09:02, Amit Kapila wrote: Please take a look to see if I broke something. In you previous mail you agreed with level as ERROR for elog message in function ResourceOwnerForgetLock(..) function, but in your modification you have used PANIC, is there any specific reason for it.

Re: [HACKERS] Pg default's verbosity?

2012-06-19 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: There might be something to the idea of demoting a few of the things we've traditionally had as NOTICEs, though. IME, the following two messages account for a huge percentage of the chatter: NOTICE: CREATE TABLE will create implicit sequence

Re: [HACKERS] Testing 9.2 in ~production environment

2012-06-19 Thread Peter Eisentraut
On mån, 2012-06-18 at 17:57 -0400, James Cloos wrote: JB == Josh Berkus j...@agliodbs.com writes: JB Can you check the collations of the two databases? I'm wondering if 9.1 JB is in C collation and 9.2 is something else. Thanks! pg_dump -C tells me these two differences: -SET

Re: [HACKERS] Testing 9.2 in ~production environment

2012-06-19 Thread Tom Lane
Peter Eisentraut pete...@gmx.net writes: On mån, 2012-06-18 at 17:57 -0400, James Cloos wrote: I presume that lc_ctype is the significant difference? It certainly makes some difference, but it's a bit shocking that makes things that much slower. If James is testing text-comparison-heavy

Re: [HACKERS] Pg default's verbosity?

2012-06-19 Thread Martijn van Oosterhout
On Mon, Jun 18, 2012 at 09:30:14PM -0400, Robert Haas wrote: There might be something to the idea of demoting a few of the things we've traditionally had as NOTICEs, though. IME, the following two messages account for a huge percentage of the chatter: NOTICE: CREATE TABLE will create

Re: [HACKERS] Testing 9.2 in ~production environment

2012-06-19 Thread Peter Eisentraut
On tis, 2012-06-19 at 02:38 -0400, Tom Lane wrote: Peter Eisentraut pete...@gmx.net writes: On mån, 2012-06-18 at 17:57 -0400, James Cloos wrote: I presume that lc_ctype is the significant difference? It certainly makes some difference, but it's a bit shocking that makes things that

Re: [HACKERS] [PATCH 10/16] Introduce the concept that wal has a 'origin' node

2012-06-19 Thread Andres Freund
On Tuesday, June 19, 2012 12:47:54 AM Christopher Browne wrote: On Mon, Jun 18, 2012 at 11:50 AM, Andres Freund and...@2ndquadrant.com wrote: Hi Simon, On Monday, June 18, 2012 05:35:40 PM Simon Riggs wrote: On 13 June 2012 19:28, Andres Freund and...@2ndquadrant.com wrote: This adds

Re: [HACKERS] psql tab completion for GRANT role

2012-06-19 Thread Peter Eisentraut
On tor, 2012-06-14 at 13:38 -0400, Robert Haas wrote: On Sun, Jan 8, 2012 at 3:48 PM, Peter Eisentraut pete...@gmx.net wrote: psql tab completion currently only supports the form GRANT privilege ON something TO someone (and the analogous REVOKE), but not the form GRANT role TO someone.

Re: [HACKERS] [PATCH 10/16] Introduce the concept that wal has a 'origin' node

2012-06-19 Thread Andres Freund
Hi, On Tuesday, June 19, 2012 08:03:04 AM Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: On Monday, June 18, 2012 11:51:27 PM Daniel Farina wrote: What's the cost of going a lot higher? Because if one makes enough numerical space available, one can assign node identities

Re: [HACKERS] [PATCH 10/16] Introduce the concept that wal has a 'origin' node

2012-06-19 Thread Andres Freund
On Tuesday, June 19, 2012 04:12:47 AM Steve Singer wrote: On 12-06-18 07:30 AM, Andres Freund wrote: Hrmpf #666. I will go through through the series commit-by-commit again to make sure everything compiles again. Reordinging this late definitely wasn't a good idea... I pushed a rebased

Re: [HACKERS] Skip checkpoint on promoting from streaming replication

2012-06-19 Thread Kyotaro HORIGUCHI
Thank you. What happens if the server skips an end-of-recovery checkpoint, is promoted to the master, runs some write transactions, crashes and restarts automatically before it completes checkpoint? In this case, the server needs to do crash recovery from the last checkpoint record with old

Re: [HACKERS] Pg default's verbosity?

2012-06-19 Thread Fabien COELHO
There might be something to the idea of demoting a few of the things we've traditionally had as NOTICEs, though. IME, the following two messages account for a huge percentage of the chatter: NOTICE: CREATE TABLE will create implicit sequence foo_a_seq for serial column foo.a NOTICE: CREATE

Re: [HACKERS] Libxml2 load error on Windows

2012-06-19 Thread Dave Page
On Tue, Jun 19, 2012 at 4:43 AM, Robert Haas robertmh...@gmail.com wrote: Please add this patch here so that it doesn't get lost in the shuffle: https://commitfest.postgresql.org/action/commitfest_view/open Hmm, that raises an interesting question (though maybe I've just missed this

Re: [HACKERS] [PATCH] lock_timeout and common SIGALRM framework

2012-06-19 Thread Boszormenyi Zoltan
2012-06-18 19:46 keltezéssel, Alvaro Herrera írta: Excerpts from Boszormenyi Zoltan's message of vie may 11 03:54:13 -0400 2012: Hi, another rebasing and applied the GIT changes in ada8fa08fc6cf5f199b6df935b4d0a730aaa4fec to the Windows implementation of PGSemaphoreTimedLock. Hi, I gave the

Re: [HACKERS] Allow WAL information to recover corrupted pg_controldata

2012-06-19 Thread Amit Kapila
I'm almost inclined to suggest that we not get next-LSN from WAL, but by scanning all the pages in the main data store and computing the max observed LSN. This is clearly not very attractive from a performance standpoint, but it would avoid the obvious failure mode where you lost some recent

Re: [HACKERS] [PATCH] Lazy hashaggregate when no aggregation is needed

2012-06-19 Thread Etsuro Fujita
Hi, -Original Message- From: Robert Haas [mailto:robertmh...@gmail.com] Sent: Tuesday, June 19, 2012 3:12 AM To: Ants Aasma Cc: Etsuro Fujita; Jay Levitt; Tom Lane; PostgreSQL-development; Francois Deliege Subject: Re: [HACKERS] [PATCH] Lazy hashaggregate when no aggregation is

Re: [HACKERS] Libxml2 load error on Windows

2012-06-19 Thread Alex Shulgin
Dave Page dp...@pgadmin.org writes: On Tue, Jun 19, 2012 at 4:43 AM, Robert Haas robertmh...@gmail.com wrote: Please add this patch here so that it doesn't get lost in the shuffle: https://commitfest.postgresql.org/action/commitfest_view/open Hmm, that raises an interesting question

Re: [HACKERS] Libxml2 load error on Windows

2012-06-19 Thread Dave Page
On Tue, Jun 19, 2012 at 11:04 AM, Alex Shulgin a...@commandprompt.com wrote: Dave Page dp...@pgadmin.org writes: On Tue, Jun 19, 2012 at 4:43 AM, Robert Haas robertmh...@gmail.com wrote: Please add this patch here so that it doesn't get lost in the shuffle:

Re: [HACKERS] Libxml2 load error on Windows

2012-06-19 Thread Alex Shulgin
Dave Page dp...@pgadmin.org writes: On Tue, Jun 19, 2012 at 11:04 AM, Alex Shulgin a...@commandprompt.com wrote: In a real bug-tracking system we would create a new bug/ticket and set it's target version to 'candidate for next minor release' or something like that.  This way, if we don't

Re: [HACKERS] return values of backend sub-main functions

2012-06-19 Thread Peter Eisentraut
On ons, 2012-01-18 at 21:21 +0200, Peter Eisentraut wrote: On lör, 2012-01-07 at 16:41 -0500, Tom Lane wrote: Peter Eisentraut pete...@gmx.net writes: I suggest that we change PostgresMain(), PostmasterMain(), BackendRun(), WalSenderMain(), and WalSndLoop() to return void as well. I

Re: [HACKERS] pgsql_fdw in contrib

2012-06-19 Thread Dickson S. Guedes
2012/6/18 Merlin Moncure mmonc...@gmail.com: I can't help but wonder (having been down the contrib/core/extension road myself) if it isn't better to improve the facilities to register and search for qualified extensions (like Perl CPAN) so that people looking for code to improve their backends

[HACKERS] use of int4/int32 in C code

2012-06-19 Thread Peter Eisentraut
What is the latest theory on using int4 vs. int32 in C code? (equivalently int2, int16) I had the idea that using int4 was sort of deprecated, and most code uses int32, but I've come across several uses of int4 lately that looked odd to me. I think the main reason that we define int4 in C is

Re: [HACKERS] Testing 9.2 in ~production environment

2012-06-19 Thread Tom Lane
Peter Eisentraut pete...@gmx.net writes: On tis, 2012-06-19 at 02:38 -0400, Tom Lane wrote: If James is testing text-comparison-heavy operations, it doesn't seem shocking in the least. strcoll() in most non-C locales is a pig. Ah yes, of course, having lc_ctype != C also selects strcoll

Re: [HACKERS] use of int4/int32 in C code

2012-06-19 Thread Tom Lane
Peter Eisentraut pete...@gmx.net writes: What is the latest theory on using int4 vs. int32 in C code? (equivalently int2, int16) I thought the general idea was to use int32 most places, but int4 in catalog declarations. I don't think it's tremendously important if somebody uses the other

Re: [HACKERS] pgsql_fdw in contrib

2012-06-19 Thread Merlin Moncure
On Mon, Jun 18, 2012 at 11:01 PM, Robert Haas robertmh...@gmail.com wrote: On Mon, Jun 18, 2012 at 12:24 PM, Merlin Moncure mmonc...@gmail.com wrote: I can't help but wonder (having been down the contrib/core/extension road myself) if it isn't better to improve the facilities to register and

Re: [HACKERS] Allow WAL information to recover corrupted pg_controldata

2012-06-19 Thread Fujii Masao
On Tue, Jun 19, 2012 at 2:44 AM, Amit Kapila amit.kap...@huawei.com wrote: AFAIR you can create pg_control from scratch already with pg_resetxlog. The hard part is coming up with values for the counters, such as the next WAL location.  Some of them such as next OID are pretty harmless if you

Re: [HACKERS] [PATCH 10/16] Introduce the concept that wal has a 'origin' node

2012-06-19 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: On Tuesday, June 19, 2012 08:03:04 AM Tom Lane wrote: Every WAL record? Why in heck would you attach it to every record? Surely putting it in WAL page headers would be sufficient. The idea is that you can have cascading, circular and whatever

[HACKERS] Do we want a xmalloc or similar function in the Backend?

2012-06-19 Thread Andres Freund
Hi, There are 70+ calls of malloc in the backend in the form of type* foo = malloc(sizeof(...)); if(!foo) elog(ERROR, could not allocate memory); which is a bit annoying to write at times. Would somebody argue against introducing a function that does the above named xmalloc() or

Re: [HACKERS] [PATCH 10/16] Introduce the concept that wal has a 'origin' node

2012-06-19 Thread Andres Freund
On Tuesday, June 19, 2012 04:17:01 PM Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: On Tuesday, June 19, 2012 08:03:04 AM Tom Lane wrote: Every WAL record? Why in heck would you attach it to every record? Surely putting it in WAL page headers would be sufficient. The

Re: [HACKERS] [PATCH 10/16] Introduce the concept that wal has a 'origin' node

2012-06-19 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: On Tuesday, June 19, 2012 04:17:01 PM Tom Lane wrote: ... (If you are thinking of something sufficiently high-level that merging could possibly work, then it's not WAL, and we shouldn't be trying to make the WAL representation cater for it.) The

Re: [HACKERS] Do we want a xmalloc or similar function in the Backend?

2012-06-19 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: There are 70+ calls of malloc in the backend in the form of type* foo = malloc(sizeof(...)); if(!foo) elog(ERROR, could not allocate memory); which is a bit annoying to write at times. Would somebody argue against introducing a function that

Re: [HACKERS] sortsupport for text

2012-06-19 Thread Peter Geoghegan
So, just to give a bit more weight to my argument that we should recognise that equivalent strings ought to be treated identically, I direct your attention to conformance requirement C9 of Unicode 3.0: http://www.unicode.org/unicode/standard/versions/enumeratedversions.html#Unicode_3_0_0 This

Re: [HACKERS] [PATCH 10/16] Introduce the concept that wal has a 'origin' node

2012-06-19 Thread Andres Freund
On Tuesday, June 19, 2012 04:30:59 PM Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: On Tuesday, June 19, 2012 04:17:01 PM Tom Lane wrote: ... (If you are thinking of something sufficiently high-level that merging could possibly work, then it's not WAL, and we shouldn't be

Re: [HACKERS] Do we want a xmalloc or similar function in the Backend?

2012-06-19 Thread Andres Freund
On Tuesday, June 19, 2012 04:38:56 PM Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: There are 70+ calls of malloc in the backend in the form of type* foo = malloc(sizeof(...)); if(!foo) elog(ERROR, could not allocate memory); which is a bit annoying to write

Re: [HACKERS] [PATCH] lock_timeout and common SIGALRM framework

2012-06-19 Thread Alvaro Herrera
Excerpts from Boszormenyi Zoltan's message of mar jun 19 04:44:35 -0400 2012: 2012-06-18 19:46 keltezéssel, Alvaro Herrera írta: Excerpts from Boszormenyi Zoltan's message of vie may 11 03:54:13 -0400 2012: Hi, another rebasing and applied the GIT changes in

Re: [HACKERS] pgsql_fdw in contrib

2012-06-19 Thread Kohei KaiGai
2012/6/19 Merlin Moncure mmonc...@gmail.com: On Mon, Jun 18, 2012 at 11:01 PM, Robert Haas robertmh...@gmail.com wrote: On Mon, Jun 18, 2012 at 12:24 PM, Merlin Moncure mmonc...@gmail.com wrote: I can't help but wonder (having been down the contrib/core/extension road myself) if it isn't

Re: [HACKERS] sortsupport for text

2012-06-19 Thread Kevin Grittner
Peter Geoghegan pe...@2ndquadrant.com wrote: So, just to give a bit more weight to my argument that we should recognise that equivalent strings ought to be treated identically Since we appear to be questioning everything in this area, I'll raise something which has been bugging me for a

Re: [HACKERS] initdb and fsync

2012-06-19 Thread David Fetter
On Mon, Jun 18, 2012 at 09:34:30PM +0300, Peter Eisentraut wrote: On mån, 2012-06-18 at 18:05 +0200, Andres Freund wrote: - defaulting to initdb -N in the regression suite is not a good imo, because that way the buildfarm won't catch problems in that area... The regression test suite also

Re: [HACKERS] libpq compression

2012-06-19 Thread Robert Haas
On Mon, Jun 18, 2012 at 1:42 PM, Martijn van Oosterhout klep...@svana.org wrote: On Sun, Jun 17, 2012 at 12:29:53PM -0400, Tom Lane wrote: The fly in the ointment with any of these ideas is that the configure list is not a list of exact cipher names, as per Magnus' comment that the current

Re: [HACKERS] pl/perl and utf-8 in sql_ascii databases

2012-06-19 Thread Robert Haas
On Mon, Jun 18, 2012 at 3:30 PM, Alvaro Herrera alvhe...@commandprompt.com wrote: Excerpts from Alex Hunsaker's message of vie feb 10 16:53:05 -0300 2012: Seems like we missed the fact that we still did SvUTF8_on() in sv2cstr and SvPVUTF8() when turning a perl string into a cstring. Hmm,

Re: [HACKERS] WAL format changes

2012-06-19 Thread Andres Freund
On Tuesday, June 19, 2012 10:14:08 AM Heikki Linnakangas wrote: On 18.06.2012 21:08, Heikki Linnakangas wrote: On 18.06.2012 21:00, Robert Haas wrote: On Thu, Jun 14, 2012 at 5:58 PM, Andres Freundand...@2ndquadrant.com wrote: 1. Use a 64-bit segment number, instead of the log/seg

Re: [HACKERS] Pg default's verbosity?

2012-06-19 Thread Robert Haas
On Tue, Jun 19, 2012 at 2:15 AM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: There might be something to the idea of demoting a few of the things we've traditionally had as NOTICEs, though.  IME, the following two messages account for a huge percentage of the

Re: [HACKERS] WAL format changes

2012-06-19 Thread Robert Haas
On Tue, Jun 19, 2012 at 4:14 AM, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: Well, that was easier than I thought. Attached is a patch to make XLogRecPtr a uint64, on top of my other WAL format patches. I think we should go ahead with this. +1. The LSNs on pages are still

Re: [HACKERS] Skip checkpoint on promoting from streaming replication

2012-06-19 Thread Fujii Masao
On Tue, Jun 19, 2012 at 5:30 PM, Kyotaro HORIGUCHI horiguchi.kyot...@lab.ntt.co.jp wrote: Thank you. What happens if the server skips an end-of-recovery checkpoint, is promoted to the master, runs some write transactions, crashes and restarts automatically before it completes checkpoint? In

Re: [HACKERS] Transactions over pathological TCP connections

2012-06-19 Thread Robert Haas
On Tue, Jun 19, 2012 at 1:56 AM, Tom Lane t...@sss.pgh.pa.us wrote: The transaction would be committed before a command success report is delivered to the client, so I don't think delivered-and-not-marked is possible. ...unless you have configured synchronous_commit=off, or fsync=off. Or

Re: [HACKERS] [PATCH 10/16] Introduce the concept that wal has a 'origin' node

2012-06-19 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: On Tuesday, June 19, 2012 04:30:59 PM Tom Lane wrote: ... (If you are thinking of something sufficiently high-level that merging could possibly work, then it's not WAL, and we shouldn't be trying to make the WAL representation cater for it.) Do

Re: [HACKERS] pgsql_fdw in contrib

2012-06-19 Thread Merlin Moncure
On Tue, Jun 19, 2012 at 10:15 AM, Kohei KaiGai kai...@kaigai.gr.jp wrote: Let me push the pgsql_fdw in core from different perspective. Right now, FDW is a feature that will take many enhancement in the near future like join-pushdown, writable APIs and so on. If we would not have a FDW

Re: [HACKERS] sortsupport for text

2012-06-19 Thread Peter Geoghegan
On 19 June 2012 16:17, Kevin Grittner kevin.gritt...@wicourts.gov wrote: Peter Geoghegan pe...@2ndquadrant.com wrote: So, just to give a bit more weight to my argument that we should recognise that equivalent strings ought to be treated identically Since we appear to be questioning

Re: [HACKERS] WIP Patch: Selective binary conversion of CSV file foreign tables

2012-06-19 Thread Kohei KaiGai
Hi Fujita-san, Could you rebase this patch towards the latest tree? It was unavailable to apply the patch cleanly. I looked over the patch, then noticed a few points. At ProcessCopyOptions(), defel-arg can be NIL, isn't it? If so, cstate-convert_binary is not a suitable flag to check redundant

Re: [HACKERS] [PATCH 10/16] Introduce the concept that wal has a 'origin' node

2012-06-19 Thread Andres Freund
Hi, On Tuesday, June 19, 2012 06:11:20 PM Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: On Tuesday, June 19, 2012 04:30:59 PM Tom Lane wrote: ... (If you are thinking of something sufficiently high-level that merging could possibly work, then it's not WAL, and we

Re: [HACKERS] sortsupport for text

2012-06-19 Thread Kevin Grittner
Peter Geoghegan pe...@2ndquadrant.com wrote: Kevin Grittner kevin.gritt...@wicourts.gov wrote: Since we appear to be questioning everything in this area, I'll raise something which has been bugging me for a while: in some other systems I've used, the tie-breaker comparison for equivalent

[HACKERS] Near-duplicate RI NO ACTION and RESTRICT triggers

2012-06-19 Thread Tom Lane
Our current interpretation of the difference between foreign keys with ON UPDATE/DELETE NO ACTION and those with ON UPDATE/DELETE RESTRICT is that they mean the same thing but RESTRICT checks are not deferrable. It follows from this that the trigger code ought to be the same for NO ACTION and

Re: [HACKERS] [PATCH] lock_timeout and common SIGALRM framework

2012-06-19 Thread Alvaro Herrera
Excerpts from Boszormenyi Zoltan's message of mar jun 19 12:44:04 -0400 2012: OK, all 4 Check* functions are now moved back into proc.c, nothing outside of timeout.c touches anything in it. New patches are attached. Yeah, I like this one better, thanks. It seems to me that the check

Re: [HACKERS] sortsupport for text

2012-06-19 Thread Peter Geoghegan
On 19 June 2012 17:45, Kevin Grittner kevin.gritt...@wicourts.gov wrote: Peter Geoghegan pe...@2ndquadrant.com wrote: Are you sure that they actually have a tie-breaker, and don't just make the distinction between equality and equivalence (if only internally)? I'm pretty sure that when I was

Re: [HACKERS] initdb and fsync

2012-06-19 Thread Jeff Davis
On Mon, 2012-06-18 at 21:41 +0200, Andres Freund wrote: It calls pg_flush_data inside of copy_file which does the posix_fadvise... So maybe just put the sync_file_range in pg_flush_data? The functions in fd.c aren't linked to initdb, so it's a challenge to share that code (I remember now:

Re: [HACKERS] [PATCH 10/16] Introduce the concept that wal has a 'origin' node

2012-06-19 Thread Robert Haas
On Tue, Jun 19, 2012 at 12:11 PM, Tom Lane t...@sss.pgh.pa.us wrote: Andres Freund and...@2ndquadrant.com writes: On Tuesday, June 19, 2012 04:30:59 PM Tom Lane wrote: ...  (If you are thinking of something sufficiently high-level that merging could possibly work, then it's not WAL, and we

Re: [HACKERS] initdb and fsync

2012-06-19 Thread Andres Freund
On Tuesday, June 19, 2012 07:22:02 PM Jeff Davis wrote: On Mon, 2012-06-18 at 21:41 +0200, Andres Freund wrote: It calls pg_flush_data inside of copy_file which does the posix_fadvise... So maybe just put the sync_file_range in pg_flush_data? The functions in fd.c aren't linked to initdb,

Re: [HACKERS] pl/perl and utf-8 in sql_ascii databases

2012-06-19 Thread Alex Hunsaker
On Mon, Jun 18, 2012 at 1:30 PM, Alvaro Herrera alvhe...@commandprompt.com wrote: Excerpts from Alex Hunsaker's message of vie feb 10 16:53:05 -0300 2012: Seems like we missed the fact that we still did SvUTF8_on() in sv2cstr and SvPVUTF8() when turning a perl string into a cstring. Hmm,

Re: [HACKERS] Do we want a xmalloc or similar function in the Backend?

2012-06-19 Thread Robert Haas
On Tue, Jun 19, 2012 at 10:17 AM, Andres Freund and...@2ndquadrant.com wrote: There are 70+ calls of malloc in the backend in the form of type* foo = malloc(sizeof(...)); if(!foo)   elog(ERROR, could not allocate memory); which is a bit annoying to write at times. Would somebody argue

Re: [HACKERS] Do we want a xmalloc or similar function in the Backend?

2012-06-19 Thread Andres Freund
On Tuesday, June 19, 2012 07:35:53 PM Robert Haas wrote: On Tue, Jun 19, 2012 at 10:17 AM, Andres Freund and...@2ndquadrant.com wrote: There are 70+ calls of malloc in the backend in the form of type* foo = malloc(sizeof(...)); if(!foo) elog(ERROR, could not allocate memory);

Re: [HACKERS] sortsupport for text

2012-06-19 Thread Kevin Grittner
Peter Geoghegan pe...@2ndquadrant.com wrote: Kevin Grittner kevin.gritt...@wicourts.gov wrote: I'm pretty sure that when I was using Sybase ASE the order for non-equal values was always predictable, and it behaved in the manner I describe below. I'm less sure about any other product.

Re: [HACKERS] [RFC][PATCH] Logical Replication/BDR prototype and architecture

2012-06-19 Thread Andres Freund
Hi, The most important part, even for people not following my discussion with Robert is at the bottom where the possible wal decoding strategies are laid out. On Tuesday, June 19, 2012 03:20:58 AM Robert Haas wrote: On Sat, Jun 16, 2012 at 7:43 AM, Andres Freund and...@2ndquadrant.com wrote:

Re: [HACKERS] sortsupport for text

2012-06-19 Thread Peter Geoghegan
On 19 June 2012 18:57, Kevin Grittner kevin.gritt...@wicourts.gov wrote: We weren't using en_US.UTF-8 collation (or any other proper collation) on Sybase -- I'm not sure whether they even supported proper collation sequences on the versions we used.  I'm thinking of when we were using their

Re: [HACKERS] Event Triggers reduced, v1

2012-06-19 Thread Robert Haas
On Fri, Jun 15, 2012 at 4:27 PM, Dimitri Fontaine dfonta...@hi-media.com wrote: Allow me to open the new season of the DML trigger series, named pg_event_trigger. This first episode is all about setting up the drama, so that next ones make perfect sense. Comments: 1. I still think we ought to

Re: [HACKERS] sortsupport for text

2012-06-19 Thread Tom Lane
Kevin Grittner kevin.gritt...@wicourts.gov writes: I wasn't aware that en_US.UTF-8 doesn't have equivalence without equality. I guess that surprising result in my last post is just plain inevitable with that collation then. Bummer. Is there actually anyone who finds that to be a useful

Re: [HACKERS] sortsupport for text

2012-06-19 Thread Peter Geoghegan
On 19 June 2012 19:44, Peter Geoghegan pe...@2ndquadrant.com wrote: You could do that, and some people do use custom collations for various reasons. That's obviously very much of minority interest though. Most people will just use citext or something. However, since citext is itself a client

Re: [HACKERS] pl/perl and utf-8 in sql_ascii databases

2012-06-19 Thread Alvaro Herrera
Excerpts from Robert Haas's message of mar jun 19 11:36:41 -0400 2012: On Mon, Jun 18, 2012 at 3:30 PM, Alvaro Herrera alvhe...@commandprompt.com wrote: Excerpts from Alex Hunsaker's message of vie feb 10 16:53:05 -0300 2012: Seems like we missed the fact that we still did SvUTF8_on()

Re: [HACKERS] use of int4/int32 in C code

2012-06-19 Thread Robert Haas
On Tue, Jun 19, 2012 at 9:47 AM, Tom Lane t...@sss.pgh.pa.us wrote: Peter Eisentraut pete...@gmx.net writes: What is the latest theory on using int4 vs. int32 in C code? (equivalently int2, int16) I thought the general idea was to use int32 most places, but int4 in catalog declarations.  I

Re: [HACKERS] [PATCH 04/16] Add embedded list interface (header only)

2012-06-19 Thread Robert Haas
On Wed, Jun 13, 2012 at 7:28 AM, Andres Freund and...@2ndquadrant.com wrote: Adds a single and a double linked list which can easily embedded into other datastructures and can be used without any additional allocations. dllist.h advertises that it's embeddable. Can you use that instead, or

Re: [HACKERS] [PATCH 10/16] Introduce the concept that wal has a 'origin' node

2012-06-19 Thread Andres Freund
On Tuesday, June 19, 2012 07:24:13 PM Robert Haas wrote: On Tue, Jun 19, 2012 at 12:11 PM, Tom Lane t...@sss.pgh.pa.us wrote: Andres Freund and...@2ndquadrant.com writes: On Tuesday, June 19, 2012 04:30:59 PM Tom Lane wrote: ... (If you are thinking of something sufficiently high-level

Re: [HACKERS] use of int4/int32 in C code

2012-06-19 Thread Peter Geoghegan
On 19 June 2012 20:11, Robert Haas robertmh...@gmail.com wrote: On Tue, Jun 19, 2012 at 9:47 AM, Tom Lane t...@sss.pgh.pa.us wrote: Peter Eisentraut pete...@gmx.net writes: What is the latest theory on using int4 vs. int32 in C code? (equivalently int2, int16) I thought the general idea was

Re: [HACKERS] Allow WAL information to recover corrupted pg_controldata

2012-06-19 Thread Robert Haas
On Tue, Jun 19, 2012 at 1:43 AM, Tom Lane t...@sss.pgh.pa.us wrote: Amit Kapila amit.kap...@huawei.com writes: AFAIR you can create pg_control from scratch already with pg_resetxlog. The hard part is coming up with values for the counters, such as the next WAL location.  Some of them such as

Re: [HACKERS] pl/perl and utf-8 in sql_ascii databases

2012-06-19 Thread Robert Haas
On Tue, Jun 19, 2012 at 3:03 PM, Alvaro Herrera alvhe...@commandprompt.com wrote: That's what I thought.  I will commit to both branches soon, then. I think there might be three branches involved. Mind you, this should have been an open item, not a commitfest item. (Actually not even an open

Re: [HACKERS] return values of backend sub-main functions

2012-06-19 Thread Robert Haas
On Tue, Jun 19, 2012 at 7:31 AM, Peter Eisentraut pete...@gmx.net wrote: On ons, 2012-01-18 at 21:21 +0200, Peter Eisentraut wrote: On lör, 2012-01-07 at 16:41 -0500, Tom Lane wrote: Peter Eisentraut pete...@gmx.net writes: I suggest that we change PostgresMain(), PostmasterMain(),

Re: [HACKERS] [PATCH 04/16] Add embedded list interface (header only)

2012-06-19 Thread Andres Freund
On Tuesday, June 19, 2012 09:16:41 PM Robert Haas wrote: On Wed, Jun 13, 2012 at 7:28 AM, Andres Freund and...@2ndquadrant.com wrote: Adds a single and a double linked list which can easily embedded into other datastructures and can be used without any additional allocations. dllist.h

Re: [HACKERS] [PATCH 01/16] Overhaul walsender wakeup handling

2012-06-19 Thread Robert Haas
On Wed, Jun 13, 2012 at 7:28 AM, Andres Freund and...@2ndquadrant.com wrote: From: Andres Freund and...@anarazel.de The previous coding could miss xlog writeouts at several places. E.g. when wal was written out by the background writer or even after a commit if synchronous_commit=off. This

Re: [HACKERS] [PATCH 04/16] Add embedded list interface (header only)

2012-06-19 Thread Robert Haas
On Tue, Jun 19, 2012 at 3:48 PM, Andres Freund and...@2ndquadrant.com wrote: Why is that code not used more widely? Quite a bit of our list usage should be replaced embedding list element in larger structs imo. There are also open- coded inline list manipulations around (check aset.c for

Re: [HACKERS] [PATCH 04/16] Add embedded list interface (header only)

2012-06-19 Thread Marko Kreen
On Wed, Jun 13, 2012 at 2:28 PM, Andres Freund and...@2ndquadrant.com wrote: +/* + * removes a node from a list + * Attention: O(n) + */ +static inline void ilist_s_remove(ilist_s_head *head, +                                  ilist_s_node *node) +{ +       ilist_s_node *last = head-head;

Re: [HACKERS] [PATCH 04/16] Add embedded list interface (header only)

2012-06-19 Thread Andres Freund
Hi, On Tuesday, June 19, 2012 09:59:48 PM Marko Kreen wrote: On Wed, Jun 13, 2012 at 2:28 PM, Andres Freund and...@2ndquadrant.com wrote: +/* + * removes a node from a list + * Attention: O(n) + */ +static inline void ilist_s_remove(ilist_s_head *head, +

Re: [HACKERS] [PATCH 01/16] Overhaul walsender wakeup handling

2012-06-19 Thread Andres Freund
On Tuesday, June 19, 2012 09:55:30 PM Robert Haas wrote: On Wed, Jun 13, 2012 at 7:28 AM, Andres Freund and...@2ndquadrant.com wrote: From: Andres Freund and...@anarazel.de The previous coding could miss xlog writeouts at several places. E.g. when wal was written out by the background

Re: [HACKERS] [PATCH 04/16] Add embedded list interface (header only)

2012-06-19 Thread Marko Kreen
On Tue, Jun 19, 2012 at 11:02 PM, Andres Freund and...@2ndquadrant.com wrote: On Tuesday, June 19, 2012 09:59:48 PM Marko Kreen wrote: On Wed, Jun 13, 2012 at 2:28 PM, Andres Freund and...@2ndquadrant.com wrote: +/* + * removes a node from a list + * Attention: O(n) + */ +static

Re: [HACKERS] [PATCH 04/16] Add embedded list interface (header only)

2012-06-19 Thread Andres Freund
On Tuesday, June 19, 2012 09:58:43 PM Robert Haas wrote: On Tue, Jun 19, 2012 at 3:48 PM, Andres Freund and...@2ndquadrant.com wrote: Why is that code not used more widely? Quite a bit of our list usage should be replaced embedding list element in larger structs imo. There are also open-

Re: [HACKERS] [PATCH 10/16] Introduce the concept that wal has a 'origin' node

2012-06-19 Thread Kevin Grittner
Andres Freund and...@2ndquadrant.com wrote: Robert Haas wrote: Tom Lane t...@sss.pgh.pa.us wrote: However, if we're dead set on doing it that way, let us put information that is only relevant to logical replication records into only the logical replication records. Right. If we decide we

Re: [HACKERS] performance regression in 9.2 when loading lots of small tables

2012-06-19 Thread Robert Haas
On Mon, Jun 18, 2012 at 8:42 PM, Jeff Janes jeff.ja...@gmail.com wrote: There was a regression introduced in 9.2 that effects the creation and loading of lots of small tables in a single transaction. It affects the loading of a pg_dump file which has a large number of small tables (10,000

Re: [HACKERS] [PATCH 10/16] Introduce the concept that wal has a 'origin' node

2012-06-19 Thread Marko Kreen
On Mon, Jun 18, 2012 at 6:35 PM, Simon Riggs si...@2ndquadrant.com wrote: On 13 June 2012 19:28, Andres Freund and...@2ndquadrant.com wrote: This adds a new configuration parameter multimaster_node_id which determines the id used for wal originating in one cluster. Looks good and it seems

Re: [HACKERS] Near-duplicate RI NO ACTION and RESTRICT triggers

2012-06-19 Thread Dean Rasheed
On 19 June 2012 17:48, Tom Lane t...@sss.pgh.pa.us wrote: I think that the argument for having the RESTRICT triggers behave like this is that the SQL spec envisions the RESTRICT check occurring immediately when the individual PK row is updated/deleted, and so there would be no opportunity for

Re: [HACKERS] Testing 9.2 in ~production environment

2012-06-19 Thread Peter Eisentraut
On tis, 2012-06-19 at 09:33 -0400, Tom Lane wrote: Come to think of it, another possible factor is that LIKE can't use ordinary indexes on text if the locale isn't C. But he reported that the plans are the same. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make

Re: [HACKERS] [RFC][PATCH] Logical Replication/BDR prototype and architecture

2012-06-19 Thread Kevin Grittner
Andres Freund and...@2ndquadrant.com wrote: The problem is just that to support basically arbitrary decoding requirements you need to provide at least those pieces of information in a transactionally consistent manner: * the data * table names * column names * type information *

Re: [HACKERS] performance regression in 9.2 when loading lots of small tables

2012-06-19 Thread Robert Haas
On Tue, Jun 19, 2012 at 4:33 PM, Robert Haas robertmh...@gmail.com wrote: On Mon, Jun 18, 2012 at 8:42 PM, Jeff Janes jeff.ja...@gmail.com wrote: There was a regression introduced in 9.2 that effects the creation and loading of lots of small tables in a single transaction. It affects the

Re: [HACKERS] [PATCH 10/16] Introduce the concept that wal has a 'origin' node

2012-06-19 Thread Simon Riggs
On 20 June 2012 04:58, Marko Kreen mark...@gmail.com wrote: On Mon, Jun 18, 2012 at 6:35 PM, Simon Riggs si...@2ndquadrant.com wrote: On 13 June 2012 19:28, Andres Freund and...@2ndquadrant.com wrote: This adds a new configuration parameter multimaster_node_id which determines the id used for

Re: [HACKERS] Backport of fsync queue compaction

2012-06-19 Thread Robert Haas
On Tue, Jun 19, 2012 at 5:33 PM, Greg Smith g...@2ndquadrant.com wrote: In January of 2011 Robert committed 7f242d880b5b5d9642675517466d31373961cf98 to try and compact the fsync queue when clients find it full.  There's no visible behavior change, just a substantial performance boost possible

Re: [HACKERS] use of int4/int32 in C code

2012-06-19 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Tue, Jun 19, 2012 at 9:47 AM, Tom Lane t...@sss.pgh.pa.us wrote: I thought the general idea was to use int32 most places, but int4 in catalog declarations. I don't think it's tremendously important if somebody uses the other though. I concur with

Re: [HACKERS] [PATCH 10/16] Introduce the concept that wal has a 'origin' node

2012-06-19 Thread Robert Haas
On Tue, Jun 19, 2012 at 3:18 PM, Andres Freund and...@2ndquadrant.com wrote: More seriously: Even if we don't put MM in core I think putting the basis for it in core so that somebody can build such a solution reusing the existing infrastructure is a sensible idea. Imo the only thing that

Re: [HACKERS] Backport of fsync queue compaction

2012-06-19 Thread Christopher Browne
On Tue, Jun 19, 2012 at 5:39 PM, Robert Haas robertmh...@gmail.com wrote: On Tue, Jun 19, 2012 at 5:33 PM, Greg Smith g...@2ndquadrant.com wrote: In January of 2011 Robert committed 7f242d880b5b5d9642675517466d31373961cf98 to try and compact the fsync queue when clients find it full.  There's

Re: [HACKERS] Backport of fsync queue compaction

2012-06-19 Thread Alvaro Herrera
Excerpts from Robert Haas's message of mar jun 19 17:39:46 -0400 2012: On Tue, Jun 19, 2012 at 5:33 PM, Greg Smith g...@2ndquadrant.com wrote: In January of 2011 Robert committed 7f242d880b5b5d9642675517466d31373961cf98 to try and compact the fsync queue when clients find it full.  There's

Re: [HACKERS] Transactions over pathological TCP connections

2012-06-19 Thread Leon Smith
On Tue, Jun 19, 2012 at 11:59 AM, Robert Haas robertmh...@gmail.com wrote: On Tue, Jun 19, 2012 at 1:56 AM, Tom Lane t...@sss.pgh.pa.us wrote: The transaction would be committed before a command success report is delivered to the client, so I don't think delivered-and-not-marked is

Re: [HACKERS] [PATCH 10/16] Introduce the concept that wal has a 'origin' node

2012-06-19 Thread Simon Riggs
On 19 June 2012 14:03, Tom Lane t...@sss.pgh.pa.us wrote: Every WAL record?  Why in heck would you attach it to every record? Surely putting it in WAL page headers would be sufficient.  We could easily afford to burn a page switch (if not a whole segment switch) when changing masters. This

Re: [HACKERS] use of int4/int32 in C code

2012-06-19 Thread Kevin Grittner
Tom Lane t...@sss.pgh.pa.us wrote: I have not looked to see how many places do that. If it's a reasonably small number of places, I'm OK with getting rid of int4 at the C level. (int2/int8 the same of course.) $ find -name '*.h' -or -name '*.c' | egrep -v '/tmp_check/' | xargs cat \ |

  1   2   >