Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-11-13 Thread Andres Freund
On 2012-11-12 19:21:28 +, Simon Riggs wrote: On 10 September 2012 17:50, Tom Lane t...@sss.pgh.pa.us wrote: The point of the proposal that I am making is to have a simple, low-maintenance solution for people who need a single-application database. A compromise somewhere in the middle

Re: [HACKERS] Inadequate thought about buffer locking during hot standby replay

2012-11-13 Thread Heikki Linnakangas
On 12.11.2012 22:53, Tom Lane wrote: Here's an updated patch that fixes the GIST replay functions as well as the other minor issues that were mentioned. Barring objections, I'll set about back-patching this as far as 9.0. Ok. It won't help all that much on 9.0, though. One thing that could

Re: [HACKERS] Re: [HACKERS] Patch für MAP_HUGETLB for mmap() shared memory

2012-11-13 Thread Andres Freund
Hi CK, On 2012-10-30 21:16:07 +0100, Christian Kruse wrote: index b4fcbaf..66ed10f 100644 --- a/doc/src/sgml/config.sgml +++ b/doc/src/sgml/config.sgml I think a short introduction or at least a reference on how to configure hugepages would be a good thing. varlistentry

Re: [HACKERS] Patch für MAP_HUGETLB for mmap() shared memory

2012-11-13 Thread Andres Freund
Oh, one more thing... On 2012-10-30 21:16:07 +0100, Christian Kruse wrote: ok, I think I implemented all of the changes you requested. All but the ia64 dependent, I have to do more research for this one. I vote for simply not caring about ia64. This is: +#ifdef MAP_HUGETLB +# ifdef

Re: [HACKERS] Enabling Checksums

2012-11-13 Thread Markus Wanner
On 11/13/2012 01:22 AM, Greg Smith wrote: Once you accept that eventually there need to be online conversion tools, there needs to be some easy way to distinguish which pages have been processed for several potential implementations. Agreed. What I'm saying is that this identification doesn't

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-11-13 Thread Simon Riggs
On 13 November 2012 06:14, Amit kapila amit.kap...@huawei.com wrote: I get the installability thang, very very much, I just don't see the single process thing as the only solution. At very least an open minded analysis of the actual problem and ways of solving it is called for, not just reach for

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-11-13 Thread Alvaro Herrera
Simon Riggs escribió: So even if this solution doesn't meet all requirements of single process solution (and neither I think it is written to address all) but can't we think of it as first version and then based on requirements extend it to have other capabilities: a. to have a mechnism

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-11-13 Thread Simon Riggs
On 13 November 2012 13:05, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Simon Riggs escribió: So even if this solution doesn't meet all requirements of single process solution (and neither I think it is written to address all) but can't we think of it as first version and then based on

Re: [HACKERS] Inadequate thought about buffer locking during hot standby replay

2012-11-13 Thread Tom Lane
Heikki Linnakangas hlinnakan...@vmware.com writes: On 12.11.2012 22:53, Tom Lane wrote: Here's an updated patch that fixes the GIST replay functions as well as the other minor issues that were mentioned. Barring objections, I'll set about back-patching this as far as 9.0. Ok. It won't help

Re: [HACKERS] Inadequate thought about buffer locking during hot standby replay

2012-11-13 Thread Merlin Moncure
On Tue, Nov 13, 2012 at 9:03 AM, Tom Lane t...@sss.pgh.pa.us wrote: Ok. It won't help all that much on 9.0, though. Well, it won't help GIST much, but the actually-reported-from-the-field case is in btree, and it does fix that. It occurs to me that if we're sufficiently scared of this case,

Re: [HACKERS] [BUGS] BUG #7534: walreceiver takes long time to detect n/w breakdown

2012-11-13 Thread Fujii Masao
On Tue, Nov 13, 2012 at 1:06 PM, Amit kapila amit.kap...@huawei.com wrote: On Monday, November 12, 2012 8:23 PM Fujii Masao wrote: On Fri, Nov 9, 2012 at 3:03 PM, Amit Kapila amit.kap...@huawei.com wrote: On Thursday, November 08, 2012 10:42 PM Fujii Masao wrote: On Thu, Nov 8, 2012 at 5:53

Re: [HACKERS] Memory leaks in record_out and record_send

2012-11-13 Thread Dimitri Fontaine
Tom Lane t...@sss.pgh.pa.us writes: OTOH I can't see trying to back-patch a solution like that. If we want to fix this in the back branches (and note the complaint linked above is against 8.3), I think we have to do it as attached. Thoughts? I've been using textin(record_out(NEW)) in

Re: [HACKERS] Memory leaks in record_out and record_send

2012-11-13 Thread Tom Lane
Dimitri Fontaine dimi...@2ndquadrant.fr writes: Tom Lane t...@sss.pgh.pa.us writes: Thoughts? I've been using textin(record_out(NEW)) in generic partitioning triggers, and you can find examples of that trick in the wiki, so I think we have users of that in the field. I think explicit calls

Re: [HACKERS] Index only scans wiki page

2012-11-13 Thread Robert Haas
On Mon, Nov 12, 2012 at 8:25 PM, Peter Geoghegan pe...@2ndquadrant.com wrote: On 13 November 2012 01:03, Jeff Janes jeff.ja...@gmail.com wrote: https://wiki.postgresql.org/wiki/Index-only_scans This page is seriously out of date. It suggests they are not yet implemented, but only being

Re: [HACKERS] Memory leaks in record_out and record_send

2012-11-13 Thread Dimitri Fontaine
Tom Lane t...@sss.pgh.pa.us writes: I think explicit calls like that actually wouldn't be a problem, since they'd be run in a per-tuple context anyway. The cases that are problematic are hard-coded I/O function calls. I'm worried about the ones like, say, plpgsql's built-in conversion

Re: [HACKERS] [PATCH] Patch to compute Max LSN of Data Pages

2012-11-13 Thread Fujii Masao
On Tue, Nov 13, 2012 at 1:23 PM, Amit kapila amit.kap...@huawei.com wrote: On Monday, November 12, 2012 9:56 PM Alvaro Herrera wrote: Robert Haas escribió: On Tue, Jul 31, 2012 at 8:09 AM, Amit kapila amit.kap...@huawei.com wrote: I think I can see all of those things being potentially

Re: [HACKERS] Memory leaks in record_out and record_send

2012-11-13 Thread Tom Lane
Dimitri Fontaine dimi...@2ndquadrant.fr writes: Tom Lane t...@sss.pgh.pa.us writes: I think explicit calls like that actually wouldn't be a problem, since they'd be run in a per-tuple context anyway. The cases that are problematic are hard-coded I/O function calls. I'm worried about the

Re: [HACKERS] [BUGS] BUG #7656: PL/Perl SPI_freetuptable() segfault

2012-11-13 Thread Tom Lane
pgm...@joh.to writes: I have a reproducible segmentation fault in PL/Perl. I have yet to narrow down the test case to something sensible, but I do have a backtrace: 219 while (context-firstchild != NULL) (gdb) bt #0 0x000104e90782 in MemoryContextDeleteChildren

Re: [HACKERS] [BUGS] BUG #7656: PL/Perl SPI_freetuptable() segfault

2012-11-13 Thread Andrew Dunstan
On 11/13/2012 12:17 PM, Tom Lane wrote: pgm...@joh.to writes: I have a reproducible segmentation fault in PL/Perl. I have yet to narrow down the test case to something sensible, but I do have a backtrace: 219 while (context-firstchild != NULL) (gdb) bt #0 0x000104e90782 in

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-11-13 Thread Tom Lane
Simon Riggs si...@2ndquadrant.com writes: The most popular relational database in the world is Microsoft Access, not MySQL. Access appears desirable because it allows a single user to create and use a database (which is very good). But all business databases have a requirement for at least one

Re: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL

2012-11-13 Thread Josh Berkus
On 11/12/12 7:59 PM, Amit kapila wrote: On Monday, November 12, 2012 12:07 PM Greg Smith wrote: On 11/9/12 11:59 PM, Amit kapila wrote: Please let me know if there are any objections or problems in above method of implementation, else I can go ahead to prepare the patch for the coming CF.

Re: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL

2012-11-13 Thread Tom Lane
Josh Berkus j...@agliodbs.com writes: I was imagining writing single, specific settings, which inevitably leads to one-setting-per-file, e.g.: SET PERSISTENT work_mem = 256MB; What Amit seems to be talking about is more EXPORT SETTINGS, where you dump all current settings in the session to

Re: [HACKERS] Inadequate thought about buffer locking during hot standby replay

2012-11-13 Thread Gavin Flower
On 14/11/12 04:32, Merlin Moncure wrote: On Tue, Nov 13, 2012 at 9:03 AM, Tom Lane t...@sss.pgh.pa.us wrote: Ok. It won't help all that much on 9.0, though. Well, it won't help GIST much, but the actually-reported-from-the-field case is in btree, and it does fix that. It occurs to me that if

Re: [HACKERS] Inadequate thought about buffer locking during hot standby replay

2012-11-13 Thread Robert Haas
On Tue, Nov 13, 2012 at 10:32 AM, Merlin Moncure mmonc...@gmail.com wrote: On Tue, Nov 13, 2012 at 9:03 AM, Tom Lane t...@sss.pgh.pa.us wrote: Ok. It won't help all that much on 9.0, though. Well, it won't help GIST much, but the actually-reported-from-the-field case is in btree, and it does

Re: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL

2012-11-13 Thread Robert Haas
On Tue, Nov 13, 2012 at 1:16 PM, Tom Lane t...@sss.pgh.pa.us wrote: Josh Berkus j...@agliodbs.com writes: I was imagining writing single, specific settings, which inevitably leads to one-setting-per-file, e.g.: SET PERSISTENT work_mem = 256MB; What Amit seems to be talking about is more

Re: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL

2012-11-13 Thread Robert Haas
On Mon, Nov 12, 2012 at 10:59 PM, Amit kapila amit.kap...@huawei.com wrote: Is the above opinion about only locking or even on a way to write the changed things in a file as mentioned in point-1 in mail chain upthread. (Point-1: 1. While writing .auto file, it will always assume that .auto

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-11-13 Thread Robert Haas
On Tue, Nov 13, 2012 at 12:38 PM, Tom Lane t...@sss.pgh.pa.us wrote: Simon Riggs si...@2ndquadrant.com writes: The most popular relational database in the world is Microsoft Access, not MySQL. Access appears desirable because it allows a single user to create and use a database (which is very

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-11-13 Thread Simon Riggs
On 13 November 2012 17:38, Tom Lane t...@sss.pgh.pa.us wrote: Simon Riggs si...@2ndquadrant.com writes: The most popular relational database in the world is Microsoft Access, not MySQL. Access appears desirable because it allows a single user to create and use a database (which is very good).

[HACKERS] Variable length array element encoding…

2012-11-13 Thread Sean Chittenden
[ Not subscribed, please keep me in the CC list ] Is there a standard idiom for encoding small variable length data in an array? I wrote the varint extension[1] that encodes data using a variable width encoding scheme[2] for signed and unsigned integers[3]. Right now the extension is mostly of

Re: [HACKERS] Memory leaks in record_out and record_send

2012-11-13 Thread Robert Haas
On Tue, Nov 13, 2012 at 12:18 AM, Tom Lane t...@sss.pgh.pa.us wrote: I wonder though if we ought to think about running output functions in a short-lived memory context instead of the executor's main context. We've considered that before, I think, and it's always been the path of least

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-11-13 Thread Tom Lane
Simon Riggs si...@2ndquadrant.com writes: On 13 November 2012 17:38, Tom Lane t...@sss.pgh.pa.us wrote: ... The fact of the matter is that there is *lots* of demand for simple single-user databases, and what I'm proposing is at least a first step towards getting there. I agree there is lots

Re: [HACKERS] Memory leaks in record_out and record_send

2012-11-13 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Tue, Nov 13, 2012 at 12:18 AM, Tom Lane t...@sss.pgh.pa.us wrote: I wonder though if we ought to think about running output functions in a short-lived memory context instead of the executor's main context. We've considered that before, I think, and

Re: [HACKERS] Enabling Checksums

2012-11-13 Thread Robert Haas
On Sun, Nov 11, 2012 at 5:52 PM, Jeff Davis pg...@j-davis.com wrote: Per-database does sound easier than per-table. I'd have to think about how that would affect shared catalogs though. For now, I'm leaning toward an offline utility to turn checksums on or off, called pg_checksums. It could

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-11-13 Thread Christopher Browne
Preface: I think there's some great commentary here, and find myself agreeing pretty whole-heartedly. On Tue, Nov 13, 2012 at 2:45 PM, Simon Riggs si...@2ndquadrant.com wrote: On 13 November 2012 17:38, Tom Lane t...@sss.pgh.pa.us wrote: Simon Riggs si...@2ndquadrant.com writes: The most

Re: [HACKERS] Enabling Checksums

2012-11-13 Thread Robert Haas
On Mon, Nov 12, 2012 at 4:44 AM, Craig Ringer cr...@2ndquadrant.com wrote: That'll make it hard for VACUUM, hint-bit setting, etc to opportunistically checksum pages whenever they're doing a page write anyway. Is it absurd to suggest using another bitmap, like the FSM or visibility map, to

Re: [HACKERS] Memory leaks in record_out and record_send

2012-11-13 Thread Robert Haas
On Tue, Nov 13, 2012 at 3:21 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Tue, Nov 13, 2012 at 12:18 AM, Tom Lane t...@sss.pgh.pa.us wrote: I wonder though if we ought to think about running output functions in a short-lived memory context instead of the

Re: [HACKERS] [PATCH] Patch to compute Max LSN of Data Pages

2012-11-13 Thread Robert Haas
On Tue, Nov 13, 2012 at 11:46 AM, Fujii Masao masao.fu...@gmail.com wrote: Without this utility, it's difficult to calculate the maximum LSN of data page, so basically we needed to take a backup when starting the standby. In the future, thanks to this utility, we can calculate the maximum LSN,

Re: [HACKERS] Enabling Checksums

2012-11-13 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: Hmm... what if we took this a step further and actually stored the checksums in a separate relation fork? That would make it pretty simple to support enabling/disabling checksums for particular relations. It would also allow us to have a wider

Re: [HACKERS] foreign key locks

2012-11-13 Thread Alvaro Herrera
Noah Misch wrote: On Wed, Oct 31, 2012 at 05:22:10PM -0300, Alvaro Herrera wrote: Not really sure about the proposed syntax, but yes clearly we need some other syntax to mean FOR NON KEY UPDATE. I would rather keep FOR UPDATE to mean what I currently call FOR KEY UPDATE. More proposals

[HACKERS] Process waiting for ExclusiveLock on INSERT

2012-11-13 Thread Jehan-Guillaume de Rorthais
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hello, I had a surprising question today from a customer about exclusive locks on INSERT. His log file shows something like: == LOG: process 1881 still waiting for ExclusiveLock on extension of relation 168318652 of database

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-11-13 Thread Dimitri Fontaine
Tom Lane t...@sss.pgh.pa.us writes: I agree there is lots of demand for simple single-user databases and I wish that too. What I don't agree with is something that casts that requirement in stone by architecturally/permanently disallowing secondary connections. If you want secondary

Re: [HACKERS] Index only scans wiki page

2012-11-13 Thread Peter Geoghegan
On 13 November 2012 16:37, Robert Haas robertmh...@gmail.com wrote: I found this an interesting read. As one of the people who worked on the feature, I'm sort of curious whether people have any experience yet with how this actually shakes out in the field. Are you (or is anyone) aware of

Re: [HACKERS] Memory leaks in record_out and record_send

2012-11-13 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: Yeah. The thing that concerns me is that I think we have a pretty decent number of memory contexts where the expected number of allocations is very small ... and we have the context *just in case* we do more than that in certain instances. I've

Re: [HACKERS] Index only scans wiki page

2012-11-13 Thread Peter Geoghegan
On 13 November 2012 16:37, Robert Haas robertmh...@gmail.com wrote: I found this an interesting read. As one of the people who worked on the feature, I'm sort of curious whether people have any experience yet with how this actually shakes out in the field. Are you (or is anyone) aware of

[HACKERS] Add contrib module functions to docs' function index

2012-11-13 Thread Craig Ringer
Hi all Andreas Heiduk on -bugs suggested that we add the functions provided by contrib modules to the function index in the docs, so it's easier to go from, say, what the heck is idx(...) to finding it in the intarray contrib module. This seems like a good idea and I'd like to pop it in the

Re: [HACKERS] Add contrib module functions to docs' function index

2012-11-13 Thread David Johnston
-Original Message- From: pgsql-hackers-ow...@postgresql.org [mailto:pgsql-hackers- ow...@postgresql.org] On Behalf Of Craig Ringer Sent: Tuesday, November 13, 2012 6:15 PM To: PostgreSQL Hackers Subject: [HACKERS] Add contrib module functions to docs' function index Hi all

Re: [HACKERS] Add contrib module functions to docs' function index

2012-11-13 Thread Craig Ringer
On 11/14/2012 07:56 AM, David Johnston wrote: For clarity does this proposal refer to Chapter 9 of the documentation, the Index, or both. If modifying Chapter 9 the function and operator tables should be extended to include a source column with values of base or contrib: module name or

Re: [HACKERS] Further pg_upgrade analysis for many tables

2012-11-13 Thread Bruce Momjian
On Tue, Nov 13, 2012 at 07:03:51PM -0500, Bruce Momjian wrote: I am attaching an updated pg_upgrade patch, which I believe is ready for application for 9.3. Correction, here is the proper patch. The previous posted version was had pending merges from the master branch. -- Bruce Momjian

Re: [HACKERS] Doc patch, index search_path where it's used to secure functions

2012-11-13 Thread Peter Eisentraut
On Fri, 2012-09-28 at 12:17 -0500, Karl O. Pinc wrote: On 09/28/2012 11:28:39 AM, Karl O. Pinc wrote: Doc patch, index search_path where it's used to secure functions. search_path-securing.patch Second version. Should be indexing the concept, not the run-time setting. Well, I'm not

Re: [HACKERS] Doc patch, further describe and-mask nature of the permission system

2012-11-13 Thread Peter Eisentraut
On Sat, 2012-09-29 at 01:16 -0500, Karl O. Pinc wrote: This patch makes some sweeping statements. Unfortunately, they are wrong. What you term the additive nature is really only a special case in the relationship between table and column privileges. Schema and database privileges are

Re: [HACKERS] Further pg_upgrade analysis for many tables

2012-11-13 Thread Ants Aasma
On Wed, Nov 14, 2012 at 2:03 AM, Bruce Momjian br...@momjian.us wrote: At 64k I see pg_upgrade taking 12% of the duration time, if I subtract out the dump/restore times. My percentage numbers only included CPU time and I used SSD storage. For the most part there was no IO wait to speak of, but

[HACKERS] recursive view syntax

2012-11-13 Thread Peter Eisentraut
I noticed we don't implement the recursive view syntax, even though it's part of the standard SQL feature set for recursive queries. Here is a patch to add that. It basically converts CREATE RECURSIVE VIEW name (columns) AS SELECT ...; to CREATE VIEW name AS WITH RECURSIVE name (columns) AS

Re: [HACKERS] recursive view syntax

2012-11-13 Thread Tom Lane
Peter Eisentraut pete...@gmx.net writes: I noticed we don't implement the recursive view syntax, even though it's part of the standard SQL feature set for recursive queries. Here is a patch to add that. Can't you simplify that by using SELECT * FROM name? regards, tom

Re: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL

2012-11-13 Thread Amit Kapila
On Wednesday, November 14, 2012 12:25 AM Robert Haas wrote: On Mon, Nov 12, 2012 at 10:59 PM, Amit kapila amit.kap...@huawei.com wrote: Is the above opinion about only locking or even on a way to write the changed things in a file as mentioned in point-1 in mail chain upthread. (Point-1:

Re: [HACKERS] Doc patch, index search_path where it's used to secure functions

2012-11-13 Thread Karl O. Pinc
On 11/13/2012 08:46:19 PM, Peter Eisentraut wrote: Well, I'm not sure. We currently have three index entries on the topic: search path search_path search_path configuration parameter I think I'd put them all under search_path. Ok. I think you are right that they need to all be under

Re: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL

2012-11-13 Thread Amit Kapila
On Tuesday, November 13, 2012 11:43 PM Josh Berkus wrote: On 11/12/12 7:59 PM, Amit kapila wrote: On Monday, November 12, 2012 12:07 PM Greg Smith wrote: On 11/9/12 11:59 PM, Amit kapila wrote: Please let me know if there are any objections or problems in above method of implementation,

Re: [HACKERS] Memory leaks in record_out and record_send

2012-11-13 Thread Martijn van Oosterhout
On Tue, Nov 13, 2012 at 05:50:08PM -0500, Stephen Frost wrote: * Robert Haas (robertmh...@gmail.com) wrote: Yeah. The thing that concerns me is that I think we have a pretty decent number of memory contexts where the expected number of allocations is very small ... and we have the context