Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-11-13 Thread Andres Freund
On 2012-11-12 19:21:28 +, Simon Riggs wrote: > On 10 September 2012 17:50, Tom Lane wrote: > > > The point of the proposal that I am making is to have a simple, > > low-maintenance solution for people who need a single-application > > database. A compromise somewhere in the middle isn't like

Re: [HACKERS] Inadequate thought about buffer locking during hot standby replay

2012-11-13 Thread Heikki Linnakangas
On 12.11.2012 22:53, Tom Lane wrote: Here's an updated patch that fixes the GIST replay functions as well as the other minor issues that were mentioned. Barring objections, I'll set about back-patching this as far as 9.0. Ok. It won't help all that much on 9.0, though. One thing that could u

Re: [HACKERS] Re: [HACKERS] Patch für MAP_HUGETLB for mmap() shared memory

2012-11-13 Thread Andres Freund
Hi CK, On 2012-10-30 21:16:07 +0100, Christian Kruse wrote: > index b4fcbaf..66ed10f 100644 > --- a/doc/src/sgml/config.sgml > +++ b/doc/src/sgml/config.sgml I think a short introduction or at least a reference on how to configure hugepages would be a good thing. > >temp_buffers (

Re: [HACKERS] Patch für MAP_HUGETLB for mmap() shared memory

2012-11-13 Thread Andres Freund
Oh, one more thing... On 2012-10-30 21:16:07 +0100, Christian Kruse wrote: > ok, I think I implemented all of the changes you requested. All but > the ia64 dependent, I have to do more research for this one. I vote for simply not caring about ia64. This is: > +#ifdef MAP_HUGETLB > +# ifdef __i

Re: [HACKERS] Enabling Checksums

2012-11-13 Thread Markus Wanner
On 11/13/2012 01:22 AM, Greg Smith wrote: > Once you accept that eventually there need to be online conversion > tools, there needs to be some easy way to distinguish which pages have > been processed for several potential implementations. Agreed. What I'm saying is that this identification doesn'

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-11-13 Thread Simon Riggs
On 13 November 2012 06:14, Amit kapila wrote: >>I get the installability thang, very very much, I just don't see the >>single process thing as the only solution. At very least an open >>minded analysis of the actual problem and ways of solving it is called >>for, not just reach for a close to han

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-11-13 Thread Alvaro Herrera
Simon Riggs escribió: > > So even if this solution doesn't meet all requirements of single > > process solution (and neither I think it is written to address all) > > but can't we think of it as first version and then based on > > requirements extend it to have other capabilities: > > a. to have a

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-11-13 Thread Simon Riggs
On 13 November 2012 13:05, Alvaro Herrera wrote: > Simon Riggs escribió: > >> > So even if this solution doesn't meet all requirements of single >> > process solution (and neither I think it is written to address all) >> > but can't we think of it as first version and then based on >> > requiremen

Re: [HACKERS] Inadequate thought about buffer locking during hot standby replay

2012-11-13 Thread Tom Lane
Heikki Linnakangas writes: > On 12.11.2012 22:53, Tom Lane wrote: >> Here's an updated patch that fixes the GIST replay functions as well as >> the other minor issues that were mentioned. Barring objections, I'll >> set about back-patching this as far as 9.0. > Ok. It won't help all that much on

Re: [HACKERS] Inadequate thought about buffer locking during hot standby replay

2012-11-13 Thread Merlin Moncure
On Tue, Nov 13, 2012 at 9:03 AM, Tom Lane wrote: >> Ok. It won't help all that much on 9.0, though. > > Well, it won't help GIST much, but the actually-reported-from-the-field > case is in btree, and it does fix that. > > It occurs to me that if we're sufficiently scared of this case, we could > p

Re: [HACKERS] [BUGS] BUG #7534: walreceiver takes long time to detect n/w breakdown

2012-11-13 Thread Fujii Masao
On Tue, Nov 13, 2012 at 1:06 PM, Amit kapila wrote: > On Monday, November 12, 2012 8:23 PM Fujii Masao wrote: > On Fri, Nov 9, 2012 at 3:03 PM, Amit Kapila wrote: >> On Thursday, November 08, 2012 10:42 PM Fujii Masao wrote: >>> On Thu, Nov 8, 2012 at 5:53 PM, Amit Kapila >>> wrote: >>> > On Thu

Re: [HACKERS] Memory leaks in record_out and record_send

2012-11-13 Thread Dimitri Fontaine
Tom Lane writes: > OTOH I can't see trying to back-patch a solution like that. If we want > to fix this in the back branches (and note the complaint linked above is > against 8.3), I think we have to do it as attached. > > Thoughts? I've been using textin(record_out(NEW)) in generic partitionin

Re: [HACKERS] Memory leaks in record_out and record_send

2012-11-13 Thread Tom Lane
Dimitri Fontaine writes: > Tom Lane writes: >> Thoughts? > I've been using textin(record_out(NEW)) in generic partitioning > triggers, and you can find examples of that trick in the wiki, so I > think we have users of that in the field. I think explicit calls like that actually wouldn't be a pr

Re: [HACKERS] Index only scans wiki page

2012-11-13 Thread Robert Haas
On Mon, Nov 12, 2012 at 8:25 PM, Peter Geoghegan wrote: > On 13 November 2012 01:03, Jeff Janes wrote: >> https://wiki.postgresql.org/wiki/Index-only_scans >> >> This page is seriously out of date. It suggests they are not yet >> implemented, but only being talked about. > > Attached is a piece

Re: [HACKERS] Memory leaks in record_out and record_send

2012-11-13 Thread Dimitri Fontaine
Tom Lane writes: > I think explicit calls like that actually wouldn't be a problem, > since they'd be run in a per-tuple context anyway. The cases that > are problematic are hard-coded I/O function calls. I'm worried > about the ones like, say, plpgsql's built-in conversion operations. > We coul

Re: [HACKERS] [PATCH] Patch to compute Max LSN of Data Pages

2012-11-13 Thread Fujii Masao
On Tue, Nov 13, 2012 at 1:23 PM, Amit kapila wrote: > On Monday, November 12, 2012 9:56 PM Alvaro Herrera wrote: > Robert Haas escribió: >> On Tue, Jul 31, 2012 at 8:09 AM, Amit kapila wrote: > >>> >> I think I can see all of those things being potentially useful. There >>> >> are a couple of pe

Re: [HACKERS] Memory leaks in record_out and record_send

2012-11-13 Thread Tom Lane
Dimitri Fontaine writes: > Tom Lane writes: >> I think explicit calls like that actually wouldn't be a problem, >> since they'd be run in a per-tuple context anyway. The cases that >> are problematic are hard-coded I/O function calls. I'm worried >> about the ones like, say, plpgsql's built-in

Re: [HACKERS] [BUGS] BUG #7656: PL/Perl SPI_freetuptable() segfault

2012-11-13 Thread Tom Lane
pgm...@joh.to writes: > I have a reproducible segmentation fault in PL/Perl. I have yet to narrow > down the test case to something sensible, but I do have a backtrace: > 219 while (context->firstchild != NULL) > (gdb) bt > #0 0x000104e90782 in MemoryContextDeleteChildren (context=

Re: [HACKERS] [BUGS] BUG #7656: PL/Perl SPI_freetuptable() segfault

2012-11-13 Thread Andrew Dunstan
On 11/13/2012 12:17 PM, Tom Lane wrote: pgm...@joh.to writes: I have a reproducible segmentation fault in PL/Perl. I have yet to narrow down the test case to something sensible, but I do have a backtrace: 219 while (context->firstchild != NULL) (gdb) bt #0 0x000104e90782 in Me

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-11-13 Thread Tom Lane
Simon Riggs writes: > The most popular relational database in the world is Microsoft Access, > not MySQL. Access appears desirable because it allows a single user to > create and use a database (which is very good). But all business > databases have a requirement for at least one of: high availabi

Re: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL

2012-11-13 Thread Josh Berkus
On 11/12/12 7:59 PM, Amit kapila wrote: > On Monday, November 12, 2012 12:07 PM Greg Smith wrote: > On 11/9/12 11:59 PM, Amit kapila wrote: > >>> Please let me know if there are any objections or problems in above method >>> of implementation, >>> else I can go ahead to prepare the patch for the

Re: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL

2012-11-13 Thread Tom Lane
Josh Berkus writes: > I was imagining writing single, specific settings, which inevitably > leads to one-setting-per-file, e.g.: > SET PERSISTENT work_mem = 256MB; > What Amit seems to be talking about is more EXPORT SETTINGS, where you > dump all current settings in the session to a file. This

Re: [HACKERS] Inadequate thought about buffer locking during hot standby replay

2012-11-13 Thread Gavin Flower
On 14/11/12 04:32, Merlin Moncure wrote: On Tue, Nov 13, 2012 at 9:03 AM, Tom Lane wrote: Ok. It won't help all that much on 9.0, though. Well, it won't help GIST much, but the actually-reported-from-the-field case is in btree, and it does fix that. It occurs to me that if we're sufficiently

Re: [HACKERS] Inadequate thought about buffer locking during hot standby replay

2012-11-13 Thread Robert Haas
On Tue, Nov 13, 2012 at 10:32 AM, Merlin Moncure wrote: > On Tue, Nov 13, 2012 at 9:03 AM, Tom Lane wrote: >>> Ok. It won't help all that much on 9.0, though. >> >> Well, it won't help GIST much, but the actually-reported-from-the-field >> case is in btree, and it does fix that. >> >> It occurs t

Re: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL

2012-11-13 Thread Robert Haas
On Tue, Nov 13, 2012 at 1:16 PM, Tom Lane wrote: > Josh Berkus writes: >> I was imagining writing single, specific settings, which inevitably >> leads to one-setting-per-file, e.g.: > >> SET PERSISTENT work_mem = 256MB; > >> What Amit seems to be talking about is more EXPORT SETTINGS, where you >

Re: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL

2012-11-13 Thread Robert Haas
On Mon, Nov 12, 2012 at 10:59 PM, Amit kapila wrote: > Is the above opinion about only locking or even on a way to write the changed > things in a file as mentioned in point-1 in mail chain upthread. > (Point-1: > 1. While writing .auto file, it will always assume that .auto > file contain >> al

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-11-13 Thread Robert Haas
On Tue, Nov 13, 2012 at 12:38 PM, Tom Lane wrote: > Simon Riggs writes: >> The most popular relational database in the world is Microsoft Access, >> not MySQL. Access appears desirable because it allows a single user to >> create and use a database (which is very good). But all business >> databa

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-11-13 Thread Simon Riggs
On 13 November 2012 17:38, Tom Lane wrote: > Simon Riggs writes: >> The most popular relational database in the world is Microsoft Access, >> not MySQL. Access appears desirable because it allows a single user to >> create and use a database (which is very good). But all business >> databases hav

[HACKERS] Variable length array element encoding…

2012-11-13 Thread Sean Chittenden
[ Not subscribed, please keep me in the CC list ] Is there a standard idiom for encoding small variable length data in an array? I wrote the varint extension[1] that encodes data using a variable width encoding scheme[2] for signed and unsigned integers[3]. Right now the extension is mostly of

Re: [HACKERS] Memory leaks in record_out and record_send

2012-11-13 Thread Robert Haas
On Tue, Nov 13, 2012 at 12:18 AM, Tom Lane wrote: > I wonder though if we ought to think about running output functions in > a short-lived memory context instead of the executor's main context. > We've considered that before, I think, and it's always been the path > of least resistance to fix the

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-11-13 Thread Tom Lane
Simon Riggs writes: > On 13 November 2012 17:38, Tom Lane wrote: >> ... The fact of the matter is that there is *lots* of demand >> for simple single-user databases, and what I'm proposing is at least a >> first step towards getting there. > I agree there is lots of demand for simple single-use

Re: [HACKERS] Memory leaks in record_out and record_send

2012-11-13 Thread Tom Lane
Robert Haas writes: > On Tue, Nov 13, 2012 at 12:18 AM, Tom Lane wrote: >> I wonder though if we ought to think about running output functions in >> a short-lived memory context instead of the executor's main context. >> We've considered that before, I think, and it's always been the path >> of l

Re: [HACKERS] Enabling Checksums

2012-11-13 Thread Robert Haas
On Sun, Nov 11, 2012 at 5:52 PM, Jeff Davis wrote: > Per-database does sound easier than per-table. I'd have to think about > how that would affect shared catalogs though. > > For now, I'm leaning toward an offline utility to turn checksums on or > off, called pg_checksums. It could do so lazily (

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-11-13 Thread Christopher Browne
Preface: I think there's some great commentary here, and find myself agreeing pretty whole-heartedly. On Tue, Nov 13, 2012 at 2:45 PM, Simon Riggs wrote: > On 13 November 2012 17:38, Tom Lane wrote: > > Simon Riggs writes: > >> The most popular relational database in the world is Microsoft Acc

Re: [HACKERS] Enabling Checksums

2012-11-13 Thread Robert Haas
On Mon, Nov 12, 2012 at 4:44 AM, Craig Ringer wrote: > That'll make it hard for VACUUM, hint-bit setting, etc to > opportunistically checksum pages whenever they're doing a page write anyway. > > Is it absurd to suggest using another bitmap, like the FSM or visibility > map, to store information o

Re: [HACKERS] Memory leaks in record_out and record_send

2012-11-13 Thread Robert Haas
On Tue, Nov 13, 2012 at 3:21 PM, Tom Lane wrote: > Robert Haas writes: >> On Tue, Nov 13, 2012 at 12:18 AM, Tom Lane wrote: >>> I wonder though if we ought to think about running output functions in >>> a short-lived memory context instead of the executor's main context. >>> We've considered tha

Re: [HACKERS] [PATCH] Patch to compute Max LSN of Data Pages

2012-11-13 Thread Robert Haas
On Tue, Nov 13, 2012 at 11:46 AM, Fujii Masao wrote: > Without this utility, it's difficult to calculate the maximum LSN of > data page, so > basically we needed to take a backup when starting the standby. In the future, > thanks to this utility, we can calculate the maximum LSN, and can skip a >

Re: [HACKERS] Enabling Checksums

2012-11-13 Thread Tom Lane
Robert Haas writes: > Hmm... what if we took this a step further and actually stored the > checksums in a separate relation fork? That would make it pretty > simple to support enabling/disabling checksums for particular > relations. It would also allow us to have a wider checksum, like 32 > or 6

Re: [HACKERS] foreign key locks

2012-11-13 Thread Alvaro Herrera
Noah Misch wrote: > On Wed, Oct 31, 2012 at 05:22:10PM -0300, Alvaro Herrera wrote: > > Not really sure about the proposed syntax, but yes clearly we need some > > other syntax to mean "FOR NON KEY UPDATE". I would rather keep FOR > > UPDATE to mean what I currently call FOR KEY UPDATE. More pro

[HACKERS] Process waiting for ExclusiveLock on INSERT

2012-11-13 Thread Jehan-Guillaume de Rorthais
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hello, I had a surprising question today from a customer about exclusive locks on INSERT. His log file shows something like: ==> LOG: process 1881 still waiting for ExclusiveLock on extension of relation 168318652 of database

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2012-11-13 Thread Dimitri Fontaine
Tom Lane writes: >> I agree there is lots of demand for simple single-user databases and I >> wish that too. What I don't agree with is something that casts that >> requirement in stone by architecturally/permanently disallowing >> secondary connections. > > If you want secondary connections, then

Re: [HACKERS] Index only scans wiki page

2012-11-13 Thread Peter Geoghegan
On 13 November 2012 16:37, Robert Haas wrote: > I found this an interesting read. As one of the people who worked on > the feature, I'm sort of curious whether people have any experience > yet with how this actually shakes out in the field. Are you (or is > anyone) aware of positive/negative fie

Re: [HACKERS] Memory leaks in record_out and record_send

2012-11-13 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: > Yeah. The thing that concerns me is that I think we have a pretty > decent number of memory contexts where the expected number of > allocations is very small ... and we have the context *just in case* > we do more than that in certain instances. I've

Re: [HACKERS] Index only scans wiki page

2012-11-13 Thread Peter Geoghegan
On 13 November 2012 16:37, Robert Haas wrote: > I found this an interesting read. As one of the people who worked on > the feature, I'm sort of curious whether people have any experience > yet with how this actually shakes out in the field. Are you (or is > anyone) aware of positive/negative fie

[HACKERS] Add contrib module functions to docs' function index

2012-11-13 Thread Craig Ringer
Hi all Andreas Heiduk on -bugs suggested that we add the functions provided by contrib modules to the function index in the docs, so it's easier to go from, say, "what the heck is idx(...)" to finding it in the intarray contrib module. This seems like a good idea and I'd like to pop it in the TO

Re: [HACKERS] Add contrib module functions to docs' function index

2012-11-13 Thread David Johnston
> -Original Message- > From: pgsql-hackers-ow...@postgresql.org [mailto:pgsql-hackers- > ow...@postgresql.org] On Behalf Of Craig Ringer > Sent: Tuesday, November 13, 2012 6:15 PM > To: PostgreSQL Hackers > Subject: [HACKERS] Add contrib module functions to docs' function index > > Hi all

Re: [HACKERS] Add contrib module functions to docs' function index

2012-11-13 Thread Craig Ringer
On 11/14/2012 07:56 AM, David Johnston wrote: > > For clarity does this proposal refer to Chapter 9 of the documentation, the > "Index", or both. > > If modifying Chapter 9 the function and operator tables should be extended > to include a "source" column with values of "base" or "contrib: name>"

Re: [HACKERS] Further pg_upgrade analysis for many tables

2012-11-13 Thread Bruce Momjian
On Tue, Nov 13, 2012 at 07:03:51PM -0500, Bruce Momjian wrote: > I am attaching an updated pg_upgrade patch, which I believe is ready for > application for 9.3. Correction, here is the proper patch. The previous posted version was had pending merges from the master branch. -- Bruce Momjian

Re: [HACKERS] Doc patch, index search_path where it's used to secure functions

2012-11-13 Thread Peter Eisentraut
On Fri, 2012-09-28 at 12:17 -0500, Karl O. Pinc wrote: > On 09/28/2012 11:28:39 AM, Karl O. Pinc wrote: > > > Doc patch, index search_path where it's used to secure functions. > > search_path-securing.patch > > Second version. Should be indexing the concept, not the run-time > setting. Well, I

Re: [HACKERS] Doc patch, further describe and-mask nature of the permission system

2012-11-13 Thread Peter Eisentraut
On Sat, 2012-09-29 at 01:16 -0500, Karl O. Pinc wrote: > This patch makes some sweeping statements. Unfortunately, they are wrong. What you term the additive nature is really only a special case in the relationship between table and column privileges. Schema and database privileges are completel

Re: [HACKERS] Further pg_upgrade analysis for many tables

2012-11-13 Thread Ants Aasma
On Wed, Nov 14, 2012 at 2:03 AM, Bruce Momjian wrote: > At 64k I see pg_upgrade taking 12% of the duration time, if I subtract > out the dump/restore times. My percentage numbers only included CPU time and I used SSD storage. For the most part there was no IO wait to speak of, but it's completely

[HACKERS] recursive view syntax

2012-11-13 Thread Peter Eisentraut
I noticed we don't implement the recursive view syntax, even though it's part of the standard SQL feature set for recursive queries. Here is a patch to add that. It basically converts CREATE RECURSIVE VIEW name (columns) AS SELECT ...; to CREATE VIEW name AS WITH RECURSIVE name (columns) AS (S

Re: [HACKERS] recursive view syntax

2012-11-13 Thread Tom Lane
Peter Eisentraut writes: > I noticed we don't implement the recursive view syntax, even though it's > part of the standard SQL feature set for recursive queries. Here is a > patch to add that. Can't you simplify that by using "SELECT * FROM name"? regards, tom lane --

Re: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL

2012-11-13 Thread Amit Kapila
On Wednesday, November 14, 2012 12:25 AM Robert Haas wrote: > On Mon, Nov 12, 2012 at 10:59 PM, Amit kapila > wrote: > > Is the above opinion about only locking or even on a way to write the > changed things in a file as mentioned in point-1 in mail chain upthread. > > (Point-1: > 1. While writing

Re: [HACKERS] Doc patch, index search_path where it's used to secure functions

2012-11-13 Thread Karl O. Pinc
On 11/13/2012 08:46:19 PM, Peter Eisentraut wrote: > > Well, I'm not sure. We currently have three index entries on the > topic: > > search path > search_path > search_path configuration parameter > > I think I'd put them all under search_path. Ok. I think you are right that they need to all

Re: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL

2012-11-13 Thread Amit Kapila
On Tuesday, November 13, 2012 11:43 PM Josh Berkus wrote: > On 11/12/12 7:59 PM, Amit kapila wrote: > > On Monday, November 12, 2012 12:07 PM Greg Smith wrote: > > On 11/9/12 11:59 PM, Amit kapila wrote: > > > >>> Please let me know if there are any objections or problems in above > method of imple

Re: [HACKERS] Memory leaks in record_out and record_send

2012-11-13 Thread Martijn van Oosterhout
On Tue, Nov 13, 2012 at 05:50:08PM -0500, Stephen Frost wrote: > * Robert Haas (robertmh...@gmail.com) wrote: > > Yeah. The thing that concerns me is that I think we have a pretty > > decent number of memory contexts where the expected number of > > allocations is very small ... and we have the co