Re: [HACKERS] Logical to physical page mapping

2012-10-30 Thread Markus Wanner
On 10/29/2012 12:05 PM, Robert Haas wrote: OK, I'll stop babbling now... Not perceived as babbling here. Thanks for that nice round-up of options and ideas around the torn page problem. Regards Markus Wanner -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make

Re: [HACKERS] WIP checksums patch

2012-10-30 Thread jesper
On 10/1/12 12:22 PM, Josh Berkus wrote: Perhaps we don't allow this to be turned per page, but rather per cluster, and per-cluster would require the entire cluster to be rewritten. We dicussed this last year, and options which require a total rewrite of the database in order to turn on the

Re: [HACKERS] [PERFORM] out of memory

2012-10-30 Thread Tatsuo Ishii
i have sql file (it's size are 1GB ) when i execute it then the String is 987098801 bytr too long for encoding conversion error occured . pls give me solution about You hit the upper limit of internal memory allocation limit in PostgreSQL. IMO, there's no way to avoid the error except you

[HACKERS] What are the advantages of not being able to access multiple databases with one connection?

2012-10-30 Thread crocket
MySQL permits a connection to access multiple databases. But Postgresql restricts a connection to one database. I think postgresql database connection is somewhat limited. Is it an old and decrepit design? or does it deserve some appreciations? -- Sent via pgsql-hackers mailing list

Re: [HACKERS] What are the advantages of not being able to access multiple databases with one connection?

2012-10-30 Thread Will Crawford
On 30 October 2012 12:37, crocket crockabisc...@gmail.com wrote: MySQL permits a connection to access multiple databases. But Postgresql restricts a connection to one database. I think postgresql database connection is somewhat limited. Is it an old and decrepit design? or does it deserve

Re: [HACKERS] What are the advantages of not being able to access multiple databases with one connection?

2012-10-30 Thread Hannu Krosing
On 10/30/2012 01:37 PM, crocket wrote: MySQL permits a connection to access multiple databases. But Postgresql restricts a connection to one database. I think postgresql database connection is somewhat limited. Is it an old and decrepit design? or does it deserve some appreciations? It's an

Re: [HACKERS] [PATCH 3/8] Add support for a generic wal reading facility dubbed XLogReader

2012-10-30 Thread Andres Freund
On Monday, October 29, 2012 08:58:53 PM Alvaro Herrera wrote: Heikki Linnakangas escribió: Hmm. I was thinking that making this work in a non-backend context would be too hard, so I didn't give that much thought, but I guess there isn't many dependencies to backend functions after all.

[HACKERS] Re: [PATCH 3/8] Add support for a generic wal reading facility dubbed XLogReader

2012-10-30 Thread Alvaro Herrera
Andres Freund escribió: On Monday, October 29, 2012 08:58:53 PM Alvaro Herrera wrote: Heikki Linnakangas escribió: Andres commented elsewhere about reading xlog records, processing them as they came in, and do a running CRC while we're still reading it. I think this is a mistake; we

Re: [HACKERS] [PATCH 3/8] Add support for a generic wal reading facility dubbed XLogReader

2012-10-30 Thread Andres Freund
On Tuesday, October 30, 2012 03:20:03 PM Alvaro Herrera wrote: Andres Freund escribió: On Monday, October 29, 2012 08:58:53 PM Alvaro Herrera wrote: Heikki Linnakangas escribió: Andres commented elsewhere about reading xlog records, processing them as they came in, and do a running

Re: [HACKERS] September 2012 commitfest

2012-10-30 Thread Greg Stark
On Mon, Oct 29, 2012 at 3:27 PM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: * tuplesort memory usage: grow_memtuples Greg Stark signed up for this I'll commit this later this week. I looked at it briefly at the conference but I think it actually does need some minor tweaks. * Trim

Re: [HACKERS] [PATCH 3/8] Add support for a generic wal reading facility dubbed XLogReader

2012-10-30 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: On Tuesday, October 30, 2012 03:20:03 PM Alvaro Herrera wrote: Am I the only one who finds this rather bizarre? Maybe this was okay when xlog data would only come from WAL files stored in the data directory at recovery, but if we're now receiving

Re: [HACKERS] [PATCH 3/8] Add support for a generic wal reading facility dubbed XLogReader

2012-10-30 Thread Alvaro Herrera
Tom Lane escribió: Andres Freund and...@2ndquadrant.com writes: On Tuesday, October 30, 2012 03:20:03 PM Alvaro Herrera wrote: Am I the only one who finds this rather bizarre? Maybe this was okay when xlog data would only come from WAL files stored in the data directory at recovery, but

Re: [HACKERS] [PATCH 3/8] Add support for a generic wal reading facility dubbed XLogReader

2012-10-30 Thread Tom Lane
Alvaro Herrera alvhe...@2ndquadrant.com writes: And yeah, I was thinking in one sum for the header and another one for the data. I don't think it's worth the space. If we're using CRC to detect end of WAL, what sense does it make to have to read the whole record if we can detect the end by

Re: [HACKERS] [PATCH 3/8] Add support for a generic wal reading facility dubbed XLogReader

2012-10-30 Thread Andres Freund
On Tuesday, October 30, 2012 04:24:21 PM Alvaro Herrera wrote: Tom Lane escribió: Andres Freund and...@2ndquadrant.com writes: On Tuesday, October 30, 2012 03:20:03 PM Alvaro Herrera wrote: Am I the only one who finds this rather bizarre? Maybe this was okay when xlog data would only

Re: [HACKERS] Re: [HACKERS] Patch für MAP_HUGETLB for mmap() shared memory

2012-10-30 Thread Christian Kruse
Hi, On 29/10/12 21:14, Peter Geoghegan wrote: I have a few initial observations on this. Thanks for your feedback. * I think you should be making the new GUC PGC_INTERNAL on platforms where MAP_HUGETLB is not defined or available. See also, effective_io_concurrency. This gives sane error

Re: [HACKERS] Patch für MAP_HUGETLB for mmap() shared memory

2012-10-30 Thread Christian Kruse
Hi, On 29/10/12 16:33, Tom Lane wrote: I created a patch which implements MAP_HUGETLB for sysv shared memory segments (PGSharedMemoryCreate). It is based on tests of Tom Lane and Andres Freund, I added error handling, huge page size detection and a GUC variable. My recollection is

Re: [HACKERS] Re: [HACKERS] Patch für MAP_HUGETLB for mmap() shared memory

2012-10-30 Thread Christian Kruse
Hey, Oh man, first I didn't sent the email to the list and now I forgot the attachment. I should really get some sleep, sorry for any inconveniences :( Greetings, CK diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml index b4fcbaf..66ed10f 100644 --- a/doc/src/sgml/config.sgml +++

Re: [HACKERS] Re: [HACKERS] Patch für MAP_HUGETLB for mmap() shared memory

2012-10-30 Thread Andres Freund
On Tuesday, October 30, 2012 08:20:33 PM Christian Kruse wrote: Hey, Oh man, first I didn't sent the email to the list and now I forgot the attachment. I should really get some sleep, sorry for any inconveniences :( +#ifdef MAP_HUGETLB +# ifdef __ia64__ +#define PG_HUGETLB_BASE_ADDR

Re: [HACKERS] Re: [HACKERS] Patch für MAP_HUGETLB for mmap() shared memory

2012-10-30 Thread Christian Kruse
Hey, On 30/10/12 20:33, Andres Freund wrote: +#ifdef MAP_HUGETLB +# ifdef __ia64__ +#define PG_HUGETLB_BASE_ADDR (void *)(0x8000UL) +#define PG_MAP_HUGETLB (MAP_HUGETLB|MAP_FIXED) +# else Not your fault, but that looks rather strange to me. The level of

Re: [HACKERS] Re: [HACKERS] Patch für MAP_HUGETLB for mmap() shared memory

2012-10-30 Thread Christian Kruse
Hey, ok, I think I implemented all of the changes you requested. All but the ia64 dependent, I have to do more research for this one. Greetings, CK diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml index b4fcbaf..66ed10f 100644 --- a/doc/src/sgml/config.sgml +++

[HACKERS] [PATCH] PL/Python: Add spidata to all spiexceptions

2012-10-30 Thread Oskari Saarenmaa
PL/Python maps Python SPIError exceptions with 'spidata' attribute into SQL errors. PL/Python also creates classes in plpy.spiexceptions for all known errors but does not initialize their spidata, so when a PL/Python function raises such an exception it is not recognized properly and is always

Re: [HACKERS] Extensions Documentation

2012-10-30 Thread Peter Eisentraut
On Fri, 2012-10-26 at 09:09 -0700, David E. Wheeler wrote: On Oct 26, 2012, at 5:27 AM, Peter Eisentraut pete...@gmx.net wrote: The advantage that these programming language ecosystems have is that they can implement the processors for the documentation format in the language itself, so

[HACKERS] Re: [COMMITTERS] pgsql: Preserve intermediate .c files in coverage mode

2012-10-30 Thread Peter Eisentraut
On Sun, 2012-10-28 at 11:10 -0400, Tom Lane wrote: Peter Eisentraut pete...@gmx.net writes: Preserve intermediate .c files in coverage mode The introduction of the .y - .c pattern rule causes some .c files such as bootparse.c to be considered intermediate files in the .y - .c - .o rule

Re: [HACKERS] Extensions Documentation

2012-10-30 Thread David E. Wheeler
On Oct 30, 2012, at 2:08 PM, Peter Eisentraut pete...@gmx.net wrote: True, which is why I was thinking of something relatively light-weight and self-contained like sundown. That's a markdown library, which transforms markdown to HTML, right? So what would we do with the HTML? Put it into

Re: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL

2012-10-30 Thread Josh Berkus
On 10/29/12 6:40 AM, Chris Corbyn wrote: What's the use case of this? It sounds like it will just create a maintenance nightmare where some stuff you expect to lookup in in postgresql.conf is actually hiding in the .auto file. Assuming only super users/sysadmins would have the ability to

Re: [HACKERS] [COMMITTERS] pgsql: Preserve intermediate .c files in coverage mode

2012-10-30 Thread Tom Lane
Peter Eisentraut pete...@gmx.net writes: On Sun, 2012-10-28 at 11:10 -0400, Tom Lane wrote: [ blink... ] I'd vote for making them precious all the time. No such behavioral change was discussed or agreed to, This is standard, default make behavior. It only showed up here because the

Re: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL

2012-10-30 Thread Christopher Browne
On Tue, Oct 30, 2012 at 5:25 PM, Josh Berkus j...@agliodbs.com wrote: On 10/29/12 6:40 AM, Chris Corbyn wrote: What's the use case of this? It sounds like it will just create a maintenance nightmare where some stuff you expect to lookup in in postgresql.conf is actually hiding in the .auto

Re: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL

2012-10-30 Thread Josh Berkus
I should think that doing this requires heading back towards there being a single unique configuration stream, and over the course of several versions, deprecating the INCLUDE directive. Oh, maybe I should take a closer look at Amit's proposal then. I thought we planned to make use of the

[HACKERS] Limiting the number of parameterized indexpaths created

2012-10-30 Thread Tom Lane
I looked into the complaint of unreasonable planner runtime in bug #7626, http://archives.postgresql.org/pgsql-bugs/2012-10/msg00232.php In the given example, the indexed relation foo has join clauses with 30 other relations. The code that I added in commit

Re: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL

2012-10-30 Thread Hannu Krosing
On 10/29/2012 03:14 PM, Amit Kapila wrote: On Monday, October 29, 2012 7:11 PM Chris Corbyn What's the use case of this? It sounds like it will just create a maintenance nightmare where some stuff you expect to lookup in in postgresql.conf is actually hiding in the .auto file. Assuming only

Re: [HACKERS] Re: [HACKERS] Patch für MAP_HUGETLB for mmap() shared memory

2012-10-30 Thread Christian Kruse
Hey, On Tuesday 30 October 2012 20:33:18 Andres Freund wrote: +#ifdef MAP_HUGETLB +# ifdef __ia64__ +#define PG_HUGETLB_BASE_ADDR (void *)(0x8000UL) +#define PG_MAP_HUGETLB (MAP_HUGETLB|MAP_FIXED) +# else Not your fault, but that looks rather strange to me. The level

Re: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL

2012-10-30 Thread Andres Freund
On Tuesday, October 30, 2012 11:24:20 PM Tom Lane wrote: The fact that this isn't being done by a large number of people (is anybody at all actually doing it?) suggests to me that maybe the demand isn't all that great. It might also be that the idea of implementing that yourself is quite

Re: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL

2012-10-30 Thread Josh Berkus
Tom, I'm not convinced we ever *had* a consensus on this. There were proposals, but I'm not sure a majority ever bought into any one of 'em. The whole problem of intermixing manual editing and programmatic editing is just a big can of worms, and not everybody is prepared to give up the

Re: [HACKERS] Move postgresql_fdw_validator into dblink

2012-10-30 Thread Shigeru Hanada
Sorry for long absence. On Sat, Oct 20, 2012 at 4:24 PM, Kohei KaiGai kai...@kaigai.gr.jp wrote: IIRC, the reason why postgresql_fdw instead of pgsql_fdw was no other fdw module has shorten naming such as ora_fdw for Oracle. However, I doubt whether it is enough strong reason to force to

Re: [HACKERS] Multiple --table options for other commands

2012-10-30 Thread Josh Kupershmidt
On Sun, Oct 28, 2012 at 4:30 PM, Josh Kupershmidt schmi...@gmail.com wrote: I see there's already a TODO for allowing pg_restore to accept multiple --table arguments[1], but would folks support adding this capability to various other commands which currently accept only a single --table

Re: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL

2012-10-30 Thread Amit Kapila
On Wednesday, October 31, 2012 4:14 AM Josh Berkus wrote: Tom, I'm not convinced we ever *had* a consensus on this. There were proposals, but I'm not sure a majority ever bought into any one of 'em. The whole problem of intermixing manual editing and programmatic editing is just a

Re: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL

2012-10-30 Thread Amit Kapila
On Wednesday, October 31, 2012 3:58 AM Andres Freund wrote: On Tuesday, October 30, 2012 11:24:20 PM Tom Lane wrote: The fact that this isn't being done by a large number of people (is anybody at all actually doing it?) suggests to me that maybe the demand isn't all that great. It might

Re: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL

2012-10-30 Thread Amit Kapila
On Wednesday, October 31, 2012 3:32 AM Hannu Krosing wrote: On 10/29/2012 03:14 PM, Amit Kapila wrote: On Monday, October 29, 2012 7:11 PM Chris Corbyn What's the use case of this? It sounds like it will just create a maintenance nightmare where some stuff you expect to lookup in in