Re: [HACKERS] Hot standby and synchronous replication status
On Tuesday, August 11, 2009, Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: 2009/8/11 Robert Haas robertmh...@gmail.com We should probably have a separate discussion about what the least committable unit would be for this patch. I wonder if it might be sufficient to provide a facility for streaming WAL, plus a standalone tool for receving it and storing it to a file. This might be designed as an improvement on our existing concept of an archive; the advantage would be that you could have all but perhaps the last few seconds of WAL if the primary kicked the bucket, rather than being behind by up to checkpoint_timeout. Allowing the WAL to be received directly by PostgreSQL could be a future enhancement. That's an interesting idea. That would essentially be another method to set up a WAL archive. I'm not sure it's worthwhile on its own, but once we have the wal-sender infrastructure in place it should be easy to write such a tool. It most definitely would be useful on it's own. I have several installations where we'd love such a capability. /Magnus -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] WIP: to_char, support for EEEE format
2009/8/11 Pavel Stehule pavel.steh...@gmail.com: It's nice. I am playing with it, and now I found some potential issue. The parser is maybe too tolerant: postgres=# select to_char(3.14323,'9.9(a'); to_char -- 3.1e+00 (1 row) I guess we *could* add code to throw an error where the 9's aren't placed immediately before the E's in the format pattern, but to be honest I can't work up any enthusiasm about it. Cheers, BJ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Filtering dictionaries support and unaccent dictionary
On Tuesday 11 August 2009 08:28:24 Jaime Casanova wrote: try to build the docs to see how to properly test this and seems like you have to teach contrib.sgml and bookindex.sgml about dict-unaccent... and when i did that i got this: openjade -wall -wno-unused-param -wno-empty -wfully-tagged -D . -c /usr/share/sgml/docbook/stylesheet/dsssl/modular/catalog -d stylesheet.dsl -t sgml -i output-html -V html-index postgres.sgml openjade:dict-unaccent.sgml:48:1:E: non SGML character number 128 openjade:dict-unaccent.sgml:49:1:E: non SGML character number 129 openjade:dict-unaccent.sgml:50:1:E: non SGML character number 130 openjade:dict-unaccent.sgml:51:1:E: non SGML character number 131 openjade:dict-unaccent.sgml:52:1:E: non SGML character number 132 openjade:dict-unaccent.sgml:53:1:E: non SGML character number 133 openjade:dict-unaccent.sgml:54:1:E: non SGML character number 134 openjade:dict-unaccent.sgml:116:4:E: element B undefined make: *** [HTML.index] Error 1 make: *** Se borra el archivo `HTML.index' You should escape the special characters as well as the b that appears as part of the example output using character entitities (amp; etc.). -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Hot standby?
What is hot and standby about the proposed hot standby feature? The way I understand these terms in a replication/cluster scenario are: cold - If the first node dies, you need to start the replacement node from a standing start. warm - If the first node dies, the replacement node needs to do some work to get ready, but it's a lot quicker than cold. hot - If the first node dies, the replacement node can take over immediately. standby - While the master node is running, the standby node instance cannot be used for anything (useful). slave - While the master node is running, the slave node can be used in limited capacity (typically read-only). master - Both/all nodes have equivalent capabilities all the time while the cluster is up. For example, I'd say that a DRBD-based solution would be a cold standby. Among WAL-based solutions, what we have now with pg_standby (nomen est omen), is a warmish standby. From what I understand, Simon's patch set does not change the warm property of this arrangement at all. It only changes the standby to a slave. Am I off? What other definition of terms justifies the description of hot standby? -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] WIP: to_char, support for EEEE format
2009/8/11 Tom Lane t...@sss.pgh.pa.us: Brendan Jurd dire...@gmail.com writes: Here's version 7. Applied with a couple of corrections: the numeric case wasn't dealing with NaNs in the same way as the float cases, Thanks for that. I do think that the whole business of printing #.# is highly bogus, and I might look at getting it to throw errors instead as a separate patch. I've added a TODO for same. Cheers, BJ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Shipping documentation untarred
On Monday 10 August 2009 21:53:57 Alvaro Herrera wrote: I understand that the placement of the generated docs in the sourcedir instead of the builddir is so that it is included in the tarball, correct? I admit I was surprised by that change. I did point that out upthread, with you in particular in mind. ;-) I've been thinking that we could actually get rid of that build-in-srcdir behavior, which also occasionally puzzles vpath users with respect to gram.c and so on. The new behavior would be to build targets in the local directory. The only requirement would be that distribution tarballs be built in-tree, so that the stuff that is distprep'ed actually ends up in the tarball, but I think that should not be a problem. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Table and Index compression
Well, here is the patch. I've included a README, which I paste here. If someone wants to play with it (after the CommitFest...) feel free to do so. While it was an interesting thing to try, I don't think it has enough potential to justify more effort... * How to test - apply the patch - copy minilzo.c and minilzo.h to src/backend/storage/smgr - configure make - enjoy * How it works - pg block size set to 32K - an extra field is added in the header telling the compressed length THIS IS BAD, this information should be stored in a separate fork of the relation, because - it would then be backwards compatible - the number of bytes to read from a compressed page would be known in advance - the table file is sparse - the page header is not compressed - pages are written at their normal positions, but only the compressed bytes are written - if compression gains nothing, un-compressed page is stored - the filesystem doesn't store the un-written blocks * Benefits - Sparse file holes are not cached, so OS disk cache efficiency is at least x2 - Random access is faster, having a better probability to hit cache (sometimes a bit faster, sometimes it's spectatular) - Yes, it does save space ( 50%) * Problems - Biggest problem : any write to a table that writes data that compresses less than whatever was there before can fail on a disk full error. - ext3 sparse file handling isn't as fast as I wish it would be : on seq scans, even if it reads 2x less data, and decompresses very fast, it's still slower... - many seq scans (especially with aggregates) are CPU bound anyway - therefore, some kind of background-reader-decompressor would be needed - pre-allocation has to be done to avoid extreme fragmentation of the file, which kind of defeats the purpose - it still causes fragmentation * Conclusion (for now) It was a nice thing to try, but I believe it would be better if this was implemented directly in the filesystem, on the condition that it should be implemented well (ie not like NTFS compression). pg_8.4.0_compression_patch_v001.tar.gz Description: GNU Zip compressed data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Patch for 8.5, transformationHook
On Mon, Aug 10, 2009 at 03:43:45PM -0400, Tom Lane wrote:
Kevin Grittner kevin.gritt...@wicourts.gov writes:
Tom Lane t...@sss.pgh.pa.us wrote:
Kevin Grittner kevin.gritt...@wicourts.gov writes:
Still, it rates pretty high on my astonishment scale that a
COALESCE of two untyped NULLs (or for that matter, any two values
of unknown type) returns a text value.
What would you have it do instead, throw an error?
Return a value of unknown type.
That would require doing actual computation on values of unknown type.
A better way would be to say it's of polymorphic type. PG's support of
polymorphism is currently a bit ad-hoc, but this would be something I'd
love to change. It would be quite a big change and I've not thought
through all the details yet.
In the specific case of COALESCE, we could theoretically do that,
since the only computation it needs is IS NULL which is
datatype-independent.
Yes, this would be the only valid operator I can see working. COUNT
would work as an aggregate.
In most situations, however, you can't evaluate
the function without knowledge of the datatype semantics. As an
example, consider NULLIF('0', '00'). This gives different answers if
you suppose the literals are text than if you suppose they are integers.
Yup, which is when it gets fun and I think would mean we'd end up
throwing out a few more queries as ambiguous if I had my way!
As long as there was *one* type in the above expression then it would
be OK, for example it would be unambiguous in either of the following
cases:
SELECT NULLIF(INT '0', '00');
SELECT NULLIF('0', INT '00');
and I'd also like the following to be OK:
SELECT NULLIF('0', '00') + 5;
SELECT n+5 FROM (SELECT NULLIF('0', '00')) x(n);
But PG currently throws these out as it's type resolution (also known
as type unification) is too eager. The same arguments would obviously
apply to any polymorphic function. For example, I'd expect to be able
to do:
SELECT ('{1,2}')[1] + 5;
and have PG figure out that the literal is of type INT[]. Not sure what
ambiguity is being prevented that causes PG to need the brackets, but
that's a side issue.
It also raises the issue of the fact that there's no general way
to ascribe types in PG. You can cast (using a couple of different
syntaxes) but this isn't the same as type ascription. For example, I'd
like to be able to do things like:
SELECT NULLIF('0', '00')::INT + 5;
But I'm doing a cast here, I'm not saying that the NULLIF function
evaluates to a value of type INT which is what I want to be doing. So
this currently results in 5 being returned and not NULL as I really
want. The above obviously isn't the syntax to use as it would break
code, but the functionality would be useful.
--
Sam http://samason.me.uk/
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Patch for 8.5, transformationHook
On Tue, Aug 11, 2009 at 6:35 AM, Sam Masons...@samason.me.uk wrote:
On Mon, Aug 10, 2009 at 03:43:45PM -0400, Tom Lane wrote:
Kevin Grittner kevin.gritt...@wicourts.gov writes:
Tom Lane t...@sss.pgh.pa.us wrote:
Kevin Grittner kevin.gritt...@wicourts.gov writes:
Still, it rates pretty high on my astonishment scale that a
COALESCE of two untyped NULLs (or for that matter, any two values
of unknown type) returns a text value.
What would you have it do instead, throw an error?
Return a value of unknown type.
That would require doing actual computation on values of unknown type.
A better way would be to say it's of polymorphic type. PG's support of
polymorphism is currently a bit ad-hoc, but this would be something I'd
love to change. It would be quite a big change and I've not thought
through all the details yet.
In the specific case of COALESCE, we could theoretically do that,
since the only computation it needs is IS NULL which is
datatype-independent.
Yes, this would be the only valid operator I can see working. COUNT
would work as an aggregate.
In most situations, however, you can't evaluate
the function without knowledge of the datatype semantics. As an
example, consider NULLIF('0', '00'). This gives different answers if
you suppose the literals are text than if you suppose they are integers.
Yup, which is when it gets fun and I think would mean we'd end up
throwing out a few more queries as ambiguous if I had my way!
As long as there was *one* type in the above expression then it would
be OK, for example it would be unambiguous in either of the following
cases:
SELECT NULLIF(INT '0', '00');
SELECT NULLIF('0', INT '00');
and I'd also like the following to be OK:
SELECT NULLIF('0', '00') + 5;
SELECT n+5 FROM (SELECT NULLIF('0', '00')) x(n);
But PG currently throws these out as it's type resolution (also known
as type unification) is too eager. The same arguments would obviously
apply to any polymorphic function. For example, I'd expect to be able
to do:
SELECT ('{1,2}')[1] + 5;
and have PG figure out that the literal is of type INT[]. Not sure what
ambiguity is being prevented that causes PG to need the brackets, but
that's a side issue.
It also raises the issue of the fact that there's no general way
to ascribe types in PG. You can cast (using a couple of different
syntaxes) but this isn't the same as type ascription. For example, I'd
like to be able to do things like:
SELECT NULLIF('0', '00')::INT + 5;
But I'm doing a cast here, I'm not saying that the NULLIF function
evaluates to a value of type INT which is what I want to be doing. So
this currently results in 5 being returned and not NULL as I really
want. The above obviously isn't the syntax to use as it would break
code, but the functionality would be useful.
What you're talking about here is called type inference.
http://en.wikipedia.org/wiki/Type_inference
...Robert
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Hot standby?
On Tue, Aug 11, 2009 at 5:30 AM, Peter Eisentrautpete...@gmx.net wrote: What is hot and standby about the proposed hot standby feature? Absolutely nothing. It's horribly misnamed. I have also long argued that Synchronous Replication should really be called Streaming Replication. Perhaps it may eventually support a synchronous mode of operation, but that will probably never be the only one. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Quick pointer required re indexing geometry
Witting a box@point function easy. Having a spot of trouble trying to figure out where and how to graft this into the GiST stuff. Could someone please point me in the general direction? Thanks. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCH] could not reattach to shared memory on Windows
On Mon, Aug 10, 2009 at 13:41, Magnus Hagandermag...@hagander.net wrote: On Wed, Jul 22, 2009 at 17:05, Magnus Hagandermag...@hagander.net wrote: Dave has built binaries for 8.3.7 and 8.4.0 for this, available at: http://developer.pgadmin.org/~dpage/postgres_exe_virtualalloc-8_3.zip http://developer.pgadmin.org/~dpage/postgres_exe_virtualalloc-8_4.zip We would like as many people as possible to test this both on systems that currently experience the problem and systems that don't, and let us know the status. To test, just replace your current postgres.exe binary with the one in the appropriate ZIP file above. Obviously, take a backup before you do it! These binaries contain just this one patch - the rest of what's been applied to the 8.3 and 8.4 branches for the next minor version is *not* included. It's been a couple of weeks now, and I've had a number of reports both on-list, on-blog and in private, from people using this. I have not yet had a single report of a problem caused by this patch (not counting the case where there was a version mismatch - can't fault the patch for that). Given that, I say we apply this for 8.3 and 8.4 now. Comments? Backpatched to 8.3 and 8.4 for now. -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Hot standby?
Am I off? What other definition of terms justifies the description of hot standby? I think that Hot Standby is associated with the high WAL recovery capacity. In my opinion, is a good term to symbolizes the superiority compared with Warm Standby. -- Matheus Ricardo Espanhol --- Dextra Sistemas http://www.dextra.com.br/postgres/
TODO list, was Re: [HACKERS] WIP: to_char, support for EEEE format
Brendan Jurd wrote: 2009/8/11 Tom Lane t...@sss.pgh.pa.us: Brendan Jurd dire...@gmail.com writes: Here's version 7. Applied with a couple of corrections: the numeric case wasn't dealing with NaNs in the same way as the float cases, Thanks for that. I do think that the whole business of printing #.# is highly bogus, and I might look at getting it to throw errors instead as a separate patch. I've added a TODO for same. Everyone, please, when you update the TODO list, put some text in the Summary box at the bottom before you commit the change so people know what has been changed. I usually just copy the first sentence of my new text and add a tag at the beginning like Added or Done. -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Hot standby?
Robert Haas wrote: On Tue, Aug 11, 2009 at 5:30 AM, Peter Eisentrautpete...@gmx.net wrote: What is hot and standby about the proposed hot standby feature? OK, so it is warm slave. Absolutely nothing. It's horribly misnamed. I have also long argued that Synchronous Replication should really be called Streaming Replication. Perhaps it may eventually support a synchronous mode of operation, but that will probably never be the only one. Yes, I like streaming replication because no matter how we implement this feature, there are going to be synchronous and asynchronous modes eventually. -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] HEAD docs
have the latest docs at http://developer.postgresql.org/pgdocs/postgres/index.html stopped being built? It sure looks like it somehow. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] HEAD docs
On Tue, Aug 11, 2009 at 14:37, Andrew Dunstanand...@dunslane.net wrote: have the latest docs at http://developer.postgresql.org/pgdocs/postgres/index.html stopped being built? It sure looks like it somehow. My guess it's designed to just uncompress the tarball, and is thus failing. I don't know where those scripts are supposed to live.. Marc? -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] HEAD docs
I believe out of petere's directory / crontab .. Sent from my iPhone On 2009-08-11, at 9:43, Magnus Hagander mag...@hagander.net wrote: On Tue, Aug 11, 2009 at 14:37, Andrew Dunstanand...@dunslane.net wrote: have the latest docs at http://developer.postgresql.org/pgdocs/postgres/index.html stopped being built? It sure looks like it somehow. My guess it's designed to just uncompress the tarball, and is thus failing. I don't know where those scripts are supposed to live.. Marc? -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] HEAD docs
Magnus Hagander wrote: On Tue, Aug 11, 2009 at 14:37, Andrew Dunstanand...@dunslane.net wrote: have the latest docs at http://developer.postgresql.org/pgdocs/postgres/index.html stopped being built? It sure looks like it somehow. My guess it's designed to just uncompress the tarball, and is thus failing. I don't know where those scripts are supposed to live.. Marc? yeah pretty sure that the docs changes also broke (-HEAD) snapshot generation in a similiar way - but I have not yet had time to look into that. Stefan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] HEAD docs
Stefan Kaltenbrunner ste...@kaltenbrunner.cc writes: Magnus Hagander wrote: My guess it's designed to just uncompress the tarball, and is thus failing. I don't know where those scripts are supposed to live.. Marc? yeah pretty sure that the docs changes also broke (-HEAD) snapshot generation in a similiar way - but I have not yet had time to look into that. You may just need to re-point where it looks for the HTML files. They are now one level down in the tree from where they were built before, ie, doc/src/sgml/html/. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Hot standby?
Bruce Momjian br...@momjian.us wrote: OK, so it is warm slave. That is technically accurate, given the preceding definitions, but it has disturbing connotations. Enough so, in my view, to merit getting a little more creative in the naming. How about warm replica? Other ideas? I agree that the present moniker misleads. -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Quick pointer required re indexing geometry
Hi, Paul Matthews p...@netspace.net.au writes: Witting a box@point function easy. Having a spot of trouble trying to figure out where and how to graft this into the GiST stuff. Could someone please point me in the general direction? You want index support for it, I suppose? Without index support (but needed anyway), you implement your code in a C module then make it visible from SQL. I'm not sure you're asking help for the C part so don't expand, please ask more if needed: CREATE OR REPLACE FUNCTION box_contains_point(box, point) RETURNS bool AS 'MODULE_PATHNAME' LANGUAGE 'C' IMMUTABLE STRICT; CREATE OPERATOR @ ( LEFTARG= box, RIGHTARG = point, PROCEDURE = box_contains_point, COMMUTATOR = '@', RESTRICT = contsel, JOIN = contjoinsel ); COMMENT ON OPERATOR @(box, point) IS 'box contains point?'; Now for adding support for index lookups, you have to see documentation about OPERATOR CLASS and OPERATOR FAMILY. I guess you need to make a family out of point and box if none already exists, add the @ operator there and point to your function. I'm unclear if you need to add it to an OPERATOR CLASS too and which, so I won't risk into writing the commands. Cross datatype indexing support, as far as I've understood, is tied together with operator families. Hope this helps, regards, -- dim -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Shipping documentation untarred
Peter Eisentraut pete...@gmx.net writes: I've been thinking that we could actually get rid of that build-in-srcdir behavior, which also occasionally puzzles vpath users with respect to gram.c and so on. The new behavior would be to build targets in the local directory. The only requirement would be that distribution tarballs be built in-tree, so that the stuff that is distprep'ed actually ends up in the tarball, but I think that should not be a problem. I think one of the arguments for the current behavior is that the derived files are platform-independent, so this way saves work if you are building several sets of binaries from the same source tree. However, I don't know of anyone actively making use of that behavior --- and if they did want to, they'd likely use a tarball not a CVS pull anyway. Having all the derived files in the build directory definitely seems to me to reduce the complexity and surprise factor, so +1 for changing. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCH] could not reattach to shared memory on Windows
On Mon, Aug 10, 2009 at 19:33, Magnus Hagandermag...@hagander.net wrote:
On Mon, Aug 10, 2009 at 16:58, Tom Lanet...@sss.pgh.pa.us wrote:
Magnus Hagander mag...@hagander.net writes:
On Mon, Aug 10, 2009 at 16:10, Tom Lanet...@sss.pgh.pa.us wrote:
8.2 as well, no?
8.2 has a different shmem implementation - the one that emulates sysv
shmem. The patch will need to be changed around for that, and I
haven't looked at that. It may be worthwhile to do that, but it's a
separate patch, so let's get it out in 8.3 and 8.4 first.
If it's at all hard to do, I could see deprecating 8.2 for Windows
instead.
I haven't looked at how much work it would be at all yet. So let's do
that before we decide to deprecate anything. As mentioned downthread,
8.2 is a very widespread release, and we really want to avoid
deprecating it.
Here's an attempt at a backport to 8.2. I haven't examined it in
detail, but it passes make check on mingw.
Comments?
--
Magnus Hagander
Me: http://www.hagander.net/
Work: http://www.redpill-linpro.com/
*** a/src/backend/port/sysv_shmem.c
--- b/src/backend/port/sysv_shmem.c
***
*** 49,54 typedef int IpcMemoryId; /* shared memory ID returned by shmget(2) */
--- 49,57
unsigned long UsedShmemSegID = 0;
void *UsedShmemSegAddr = NULL;
+ #ifdef WIN32
+ Size UsedShmemSegSize = 0;
+ #endif
static void *InternalIpcMemoryCreate(IpcMemoryKey memKey, Size size);
static void IpcMemoryDetach(int status, Datum shmaddr);
***
*** 403,408 PGSharedMemoryCreate(Size size, bool makePrivate, int port)
--- 406,412
/* Save info for possible future use */
UsedShmemSegAddr = memAddress;
+ UsedShmemSegSize = size;
UsedShmemSegID = (unsigned long) NextShmemSegID;
return hdr;
*** a/src/backend/port/win32/shmem.c
--- b/src/backend/port/win32/shmem.c
***
*** 12,19
--- 12,22
*/
#include postgres.h
+ #include miscadmin.h
static DWORD s_segsize = 0;
+ extern void *UsedShmemSegAddr;
+ extern Size UsedShmemSegSize;
/* Detach from a shared mem area based on its address */
int
***
*** 29,34 shmdt(const void *shmaddr)
--- 32,44
void *
shmat(int memId, void *shmaddr, int flag)
{
+ /* Release the memory region reserved in the postmaster */
+ if (IsUnderPostmaster)
+ {
+ if (VirtualFree(shmaddr, 0, MEM_RELEASE) == 0)
+ elog(FATAL, failed to release reserved memory region (addr=%p): %lu,
+ shmaddr, GetLastError());
+ }
/* TODO -- shmat needs to count # attached to shared mem */
void *lpmem = MapViewOfFileEx((HANDLE) memId,
FILE_MAP_WRITE | FILE_MAP_READ,
***
*** 128,130 shmget(int memKey, int size, int flag)
--- 138,190
return (int) hmap;
}
+
+ /*
+ * pgwin32_ReserveSharedMemoryRegion(hChild)
+ *
+ * Reserve the memory region that will be used for shared memory in a child
+ * process. It is called before the child process starts, to make sure the
+ * memory is available.
+ *
+ * Once the child starts, DLLs loading in different order or threads getting
+ * scheduled differently may allocate memory which can conflict with the
+ * address space we need for our shared memory. By reserving the shared
+ * memory region before the child starts, and freeing it only just before we
+ * attempt to get access to the shared memory forces these allocations to
+ * be given different address ranges that don't conflict.
+ *
+ * NOTE! This function executes in the postmaster, and should for this
+ * reason not use elog(FATAL) since that would take down the postmaster.
+ */
+ int
+ pgwin32_ReserveSharedMemoryRegion(HANDLE hChild)
+ {
+ void *address;
+
+ Assert(UsedShmemSegAddr != NULL);
+ Assert(UsedShmemSegSize != 0);
+
+ address = VirtualAllocEx(hChild, UsedShmemSegAddr, UsedShmemSegSize,
+ MEM_RESERVE, PAGE_READWRITE);
+ if (address == NULL) {
+ /* Don't use FATAL since we're running in the postmaster */
+ elog(LOG, could not reserve shared memory region (addr=%p) for child %lu: %lu,
+ UsedShmemSegAddr, hChild, GetLastError());
+ return false;
+ }
+ if (address != UsedShmemSegAddr)
+ {
+ /*
+ * Should never happen - in theory if allocation granularity causes strange
+ * effects it could, so check just in case.
+ *
+ * Don't use FATAL since we're running in the postmaster.
+ */
+ elog(LOG, reserved shared memory region got incorrect address %p, expected %p,
+ address, UsedShmemSegAddr);
+ VirtualFreeEx(hChild, address, 0, MEM_RELEASE);
+ return false;
+ }
+
+ return true;
+ }
*** a/src/backend/postmaster/postmaster.c
--- b/src/backend/postmaster/postmaster.c
***
*** 3184,3190 internal_forkexec(int argc, char *argv[], Port *port)
return -1;/* log made by save_backend_variables */
}
! /* Drop the shared memory that is now inherited to the backend */
if (!UnmapViewOfFile(param))
elog(LOG, could not unmap view of
Re: [HACKERS] Hot standby and synchronous replication status
Hi, On Tue, Aug 11, 2009 at 3:33 PM, Magnus Hagandermag...@hagander.net wrote: We should probably have a separate discussion about what the least committable unit would be for this patch. I wonder if it might be sufficient to provide a facility for streaming WAL, plus a standalone tool for receving it and storing it to a file. This might be designed as an improvement on our existing concept of an archive; the advantage would be that you could have all but perhaps the last few seconds of WAL if the primary kicked the bucket, rather than being behind by up to checkpoint_timeout. Allowing the WAL to be received directly by PostgreSQL could be a future enhancement. That's an interesting idea. That would essentially be another method to set up a WAL archive. I'm not sure it's worthwhile on its own, but once we have the wal-sender infrastructure in place it should be easy to write such a tool. It most definitely would be useful on it's own. I have several installations where we'd love such a capability. Yeah, this is my initial proposal for WAL receiving side. I think that it's useful to provide such tool as a contrib (or pgfoundry) program. http://archives.postgresql.org/pgsql-hackers/2008-10/msg01639.php Regards, -- Fujii Masao NIPPON TELEGRAPH AND TELEPHONE CORPORATION NTT Open Source Software Center -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Hot standby?
On Aug 11, 2009, at 5:32 AM, Bruce Momjian wrote: OK, so it is warm slave. I suggest steaming servant. Or tepid assistant. David -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCH] could not reattach to shared memory on Windows
On Tue, Aug 11, 2009 at 16:30, Magnus Hagandermag...@hagander.net wrote: On Mon, Aug 10, 2009 at 19:33, Magnus Hagandermag...@hagander.net wrote: On Mon, Aug 10, 2009 at 16:58, Tom Lanet...@sss.pgh.pa.us wrote: Magnus Hagander mag...@hagander.net writes: On Mon, Aug 10, 2009 at 16:10, Tom Lanet...@sss.pgh.pa.us wrote: 8.2 as well, no? 8.2 has a different shmem implementation - the one that emulates sysv shmem. The patch will need to be changed around for that, and I haven't looked at that. It may be worthwhile to do that, but it's a separate patch, so let's get it out in 8.3 and 8.4 first. If it's at all hard to do, I could see deprecating 8.2 for Windows instead. I haven't looked at how much work it would be at all yet. So let's do that before we decide to deprecate anything. As mentioned downthread, 8.2 is a very widespread release, and we really want to avoid deprecating it. Here's an attempt at a backport to 8.2. I haven't examined it in detail, but it passes make check on mingw. Comments? I've also built a binary that should be copy:able on top of an 8.2.13 installation made from the standard installer, to test this feature. Anybody on 8.2 on Windows, please give it a shot and let us know how it works. http://www.hagander.net/pgsql/postgres_exe_virtualalloc_8_2.zip -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Hot standby and synchronous replication status
Hi, On Tue, Aug 11, 2009 at 1:25 PM, Robert Haasrobertmh...@gmail.com wrote: But just to kick off the discussion, here is Heikki's review of Synch Rep on 7/15: http://archives.postgresql.org/pgsql-hackers/2009-07/msg00913.php I think the key phrases in this review are I believe we have consensus on four major changes and As a hint, I think you'll find it a lot easier if you implement only asynchronous replication at first. That reduces the amount of inter-process communication a lot. I think this points to a need to try to reduce the scope of this patch to something more manageable. Heikki also points out that major change #4 was raised back in Decemeber, and I actually think #1 and #3 were as well. Thanks for clarifying the status. According to Heikki's advice, I'm working at asynchronous replication at first. I'll submit the patch by the next CommitFest at least. Regards, -- Fujii Masao NIPPON TELEGRAPH AND TELEPHONE CORPORATION NTT Open Source Software Center -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Hot standby?
On Tue, Aug 11, 2009 at 12:30:58PM +0300, Peter Eisentraut wrote: warm - If the first node dies, the replacement node needs to do some work to get ready, but it's a lot quicker than cold. hot - If the first node dies, the replacement node can take over immediately. For example, I'd say that a DRBD-based solution would be a cold standby. Among WAL-based solutions, what we have now with pg_standby (nomen est omen), is a warmish standby. From what I understand, Simon's patch set does not change the warm property of this arrangement at all. It only changes the standby to a slave. Am I off? What other definition of terms justifies the description of hot standby? As for warm/hot, it depends on what you exactly mean with get ready: (A) If you mean it is possible to connect to the second node, then Simon's patch is hot. IIRC this meaning agrees with the terminology used by other databases, such as Oracle, Sybase, etc. (B) If you mean the contents of the second node are equal to the contents of the first node, then Simon's patch is warm. But then we have by definition hot == synchronous and either warm or cold == asynchronous. IMHO the other case is preferrable, as the word sets synchronous/asynchronous and cold/warm/hot are more expressive because they measure two independent properties. Best regards, Dr. Gianni Ciolli - 2ndQuadrant Italia PostgreSQL Training, Services and Support gianni.cio...@2ndquadrant.it | www.2ndquadrant.it -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Any tutorial or FAQ on building an extension?
Hello Group, I'd like to build an extension to PostgreSQL. It will intercept queries and perform some transformations on the query and on the data returned, given some business rules that the users have specified. What's the best way to do this? It seems like if I model the pgpool- II architecture that would be a good start. That way existing clients don't really know that they are talking to my application instead of a real PostgreSQL postmaster. My app won't be written in C but I suppose I could start with this approach. Is there an easier way of going about this other than replacing the postmaster / postgres components? Thanks for any pointers, Matt -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] pgindent timing (was Re: [COMMITTERS] pgsql: Refactor NUM_cache_remove calls in error report path to a PG_TRY)
Robert Haas robertmh...@gmail.com writes: On Mon, Aug 10, 2009 at 6:52 PM, Tom Lanet...@sss.pgh.pa.us wrote: Only if they aren't applied by then. One reason that we normally only run pgindent at the end of the devel cycle is that that's when (presumably) the smallest amount of patches remain outstanding. OK, I get it. Thanks for bearing with me. The theory that the smallest amount of patches remain outstanding at that point is probably only true if the pgindent run is done relatively soon after the last CommitFest. In the 8.4 cycle, the pgindent run was done something like 7 months after the start of the last CommitFest, by which time a fair number of patches had accumulated. Yeah, that's a fair point. Maybe we should institute a new policy that pgindent should happen immediately after close of the last commitfest in a cycle, instead of delaying until almost release time. A more aggressive approach would be to run pgindent immediately after the close of *each* commitfest, but that would tend to break patches that had gotten punted to the next fest. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Hot standby?
On Tue, 2009-08-11 at 08:12 -0700, David E. Wheeler wrote: On Aug 11, 2009, at 5:32 AM, Bruce Momjian wrote: OK, so it is warm slave. I suggest steaming servant. Or tepid assistant. We can't use those, I think they are on the list for Ubuntu. Joshua D. Drake David -- PostgreSQL - XMPP: jdr...@jabber.postgresql.org Consulting, Development, Support, Training 503-667-4564 - http://www.commandprompt.com/ The PostgreSQL Company, serving since 1997 -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Hot standby?
On Tue, Aug 11, 2009 at 08:56:38AM -0500, Kevin Grittner wrote: Bruce Momjian br...@momjian.us wrote: OK, so it is warm slave. That is technically accurate, given the preceding definitions, but it has disturbing connotations. Enough so, in my view, to merit getting a little more creative in the naming. How about warm replica? Other ideas? Warm Read Streamed Copy Cheers, David. -- David Fetter da...@fetter.org http://fetter.org/ Phone: +1 415 235 3778 AIM: dfetter666 Yahoo!: dfetter Skype: davidfetter XMPP: david.fet...@gmail.com Remember to vote! Consider donating to Postgres: http://www.postgresql.org/about/donate -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] machine-readable explain output v4
Robert Haas robertmh...@gmail.com writes:
I think I might be starting to understand what you're getting at here.
Let me check: I think what you're saying is that the Expr node is
potentially useful to clients for identifying where in the tree the
Exprs are, even without specific knowledge about the different types
of filter nodes, much as the Plans node is useful for finding all of
the subplan objects.
Well, my point is that there are a bunch of different places that
expressions can appear in a plan tree --- filters, join filters, sort
keys, hash clauses, merge clauses, index clauses, output expressions,
and I'm not sure I remembered them all, and I fully expect that some
more might appear in future. I don't think it's appropriate to require
clients to keep track of exactly which tree properties are expressions,
especially not when we can easily label them.
If that is correct, then let me make two comments. First, I don't
think it should be a goal of this format to be self-documenting.
It isn't. But I think it should be a goal that a client can, say,
find all the references to a given variable without a huge amount of
hard-wired knowledge about specific properties of specific node types.
So it should be able to find all the expressions with a relatively
dumb search of the tree (eg, XPath?).
But, second, there might be a way that we can have our cake and eat it
too. Instead of decorating each node with properties Filter,
Join-Filter, One-Time-Filter, and so on, what we could do is decorate
each node with a single property called Filters, which would be a list
of hashes:
'Filters' : [ { 'Filter-Type' = 'Join', 'Text-Expr' = 'a.foo b.bar' }]
I still say that's a bad idea for child Plans; and it's a worse one for
expressions. The point about retaining knowledge of child order is
absolutely critical for many of the places where exprs appear, such as
sort keys and output columns. You'd have to seriously uglify the
notation in order to do it like this.
One way that we could handle this, I guess, is to embed more information
in the property names --- eg use Expr Text where I just had Text.
But it's not apparent to me why that should be considered better style
than the other way.
regards, tom lane
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Re: pgindent timing (was Re: [COMMITTERS] pgsql: Refactor NUM_cache_remove calls in error report path to a PG_TRY)
On Tue, Aug 11, 2009 at 11:56 AM, Tom Lanet...@sss.pgh.pa.us wrote: OK, I get it. Thanks for bearing with me. The theory that the smallest amount of patches remain outstanding at that point is probably only true if the pgindent run is done relatively soon after the last CommitFest. In the 8.4 cycle, the pgindent run was done something like 7 months after the start of the last CommitFest, by which time a fair number of patches had accumulated. Yeah, that's a fair point. Maybe we should institute a new policy that pgindent should happen immediately after close of the last commitfest in a cycle, instead of delaying until almost release time. A more aggressive approach would be to run pgindent immediately after the close of *each* commitfest, but that would tend to break patches that had gotten punted to the next fest. I'm not sure whether that would work out to a net positive or not. Presumably, it would mostly only break patches that had touched that part of the code, and therefore were likely to be broken anyway. But by the same token, since they're under active development, they're also more likely to have already been fixed. I'm not sure there's a good solution to this problem short of making pgindent easy enough that we can make it a requirement for patch submission, and I'm not sure that's practical. But in any case, I think running pgindent immediately after the last CommitFest rather than after a longish delay would be a good idea. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Re: pgindent timing (was Re: [COMMITTERS] pgsql: Refactor NUM_cache_remove calls in error report path to a PG_TRY)
Robert Haas wrote: I'm not sure there's a good solution to this problem short of making pgindent easy enough that we can make it a requirement for patch submission, and I'm not sure that's practical. But in any case, I think running pgindent immediately after the last CommitFest rather than after a longish delay would be a good idea. Frankly, fixing up patch bitrot caused by pgindent is not terribly difficult in my experience - bitrot caused by code drift is a much harder problem (and yes, git fans, I know git can help with that). cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Hot standby?
Peter, I believe we're just copying Oracle's terminology. While that terminology is not consistent, it is understood by the industry. Oracle defined their Hot Standby to have both asynchronous and synchronous modes: http://www.oracle.com/technology/products/rdb/htdocs/dbms/hotstandby.html The other industry term which would be understood would be log-based replication. Terms we invent would be less likely to be understood, and users would not get excited about them. Otherwise, I say we go with simmering follower. ;-) -- Josh Berkus PostgreSQL Experts Inc. www.pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] machine-readable explain output v4
On Tue, Aug 11, 2009 at 12:11 PM, Tom Lanet...@sss.pgh.pa.us wrote:
Robert Haas robertmh...@gmail.com writes:
I think I might be starting to understand what you're getting at here.
Let me check: I think what you're saying is that the Expr node is
potentially useful to clients for identifying where in the tree the
Exprs are, even without specific knowledge about the different types
of filter nodes, much as the Plans node is useful for finding all of
the subplan objects.
Well, my point is that there are a bunch of different places that
expressions can appear in a plan tree --- filters, join filters, sort
keys, hash clauses, merge clauses, index clauses, output expressions,
and I'm not sure I remembered them all, and I fully expect that some
more might appear in future. I don't think it's appropriate to require
clients to keep track of exactly which tree properties are expressions,
especially not when we can easily label them.
If that is correct, then let me make two comments. First, I don't
think it should be a goal of this format to be self-documenting.
It isn't. But I think it should be a goal that a client can, say,
find all the references to a given variable without a huge amount of
hard-wired knowledge about specific properties of specific node types.
So it should be able to find all the expressions with a relatively
dumb search of the tree (eg, XPath?).
But, second, there might be a way that we can have our cake and eat it
too. Instead of decorating each node with properties Filter,
Join-Filter, One-Time-Filter, and so on, what we could do is decorate
each node with a single property called Filters, which would be a list
of hashes:
'Filters' : [ { 'Filter-Type' = 'Join', 'Text-Expr' = 'a.foo b.bar' }]
I still say that's a bad idea for child Plans; and it's a worse one for
expressions. The point about retaining knowledge of child order is
absolutely critical for many of the places where exprs appear, such as
sort keys and output columns. You'd have to seriously uglify the
notation in order to do it like this.
One way that we could handle this, I guess, is to embed more information
in the property names --- eg use Expr Text where I just had Text.
But it's not apparent to me why that should be considered better style
than the other way.
Unfortunately, I have to admit to total confusion. The idea in the
last paragraph seems reasonable to me, but since I don't understand
the other alternative, I can't say whether it's better or worse. I
wonder if we would be better off waiting for feedback from actual tool
authors. I have no illusions that the format is perfect...
...Robert
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Any tutorial or FAQ on building an extension?
Is there an easier way of going about this other than replacing the postmaster / postgres components? I'd start with creating my own extended version to psql (the client library), I suppose. But since I don't really know what kind of transformations you have in mind, any advice is going to be purely speculative. -- Josh Berkus PostgreSQL Experts Inc. www.pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Hot standby?
On Tue, Aug 11, 2009 at 1:08 PM, Josh Berkusj...@agliodbs.com wrote: I believe we're just copying Oracle's terminology. While that terminology is not consistent, it is understood by the industry. Oracle defined their Hot Standby to have both asynchronous and synchronous modes: http://www.oracle.com/technology/products/rdb/htdocs/dbms/hotstandby.html I think that you are wrong. Hot Standby is described in that document as a way of replication the database and providing failover in the event that the master dies. This feature has to do with allowing queries to be executed on the slave, which is not discussed in that document. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Re: pgindent timing (was Re: [COMMITTERS] pgsql: Refactor NUM_cache_remove calls in error report path to a PG_TRY)
On Tue, Aug 11, 2009 at 12:52 PM, Andrew Dunstanand...@dunslane.net wrote: Robert Haas wrote: I'm not sure there's a good solution to this problem short of making pgindent easy enough that we can make it a requirement for patch submission, and I'm not sure that's practical. But in any case, I think running pgindent immediately after the last CommitFest rather than after a longish delay would be a good idea. Frankly, fixing up patch bitrot caused by pgindent is not terribly difficult in my experience - bitrot caused by code drift is a much harder problem (and yes, git fans, I know git can help with that). Where it really bit me as when it reindented the DATA() statements that were touched by ALTER TABLE ... SET STATISTICS DISTINCT. It's not so hard to compare code, but comparing DATA() lines is the pits. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Hot standby?
David Fetter wrote: On Tue, Aug 11, 2009 at 08:56:38AM -0500, Kevin Grittner wrote: Bruce Momjian br...@momjian.us wrote: OK, so it is warm slave. Why isn't it just a read only slave. Do some systems have read-only slave databases that can't serve as a warm standby system as well as this one could? That is technically accurate, given the preceding definitions, but it has disturbing connotations. Enough so, in my view, to merit getting a little more creative in the naming. How about warm replica? Other ideas? Warm Read Streamed Copy Master/Slave Replication and Warm Standby systems are common enough terms that I can google them or look them up in many computer science books. While coming up with creative politically correct euphemisms might be fun, I hope we stick near terms that other DBAs could already be familiar with. ISTM the best way to refer to it formally would be a Read Only Slave / Warm Standby system, even if informally we might have informal discussions of just how hot our slaves are when hot-standby features get added down the road. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] machine-readable explain output v4
On Tue, 11 Aug 2009 13:11:47 -0400 Robert Haas robertmh...@gmail.com wrote: Unfortunately, I have to admit to total confusion. The idea in the last paragraph seems reasonable to me, but since I don't understand the other alternative, I can't say whether it's better or worse. I wonder if we would be better off waiting for feedback from actual tool authors. I have no illusions that the format is perfect... Have any tool authors stepped up and committed resources to utilizing this feature in the near term? This is one of the features I have been most interested in, and I believe it has the potential to greatly improve PostgreSQL's reputation of being hard to optimize if used correctly. I envision a web site where users can paste the XML explain output and have it return some pretty specific information and fancy graphs describing exactly whats going on with their query and possible ideas on how to further optimize the query itself and even PostgreSQL configuration settings. The reason I would like to provide this tool in a web-based form is that no additional software installation would be necessary for the user, reducing any hurdles to using it to zero. I'm guessing that my vision likely exceeds the scope of this feature in its initial form at least, but assuming no one else has stepped up, I'm more than willing to start committing resources as early as this weekend with the understanding that this feature is still in development and likely will change several times before or if its finally committed for 8.5. -- Mike -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] machine-readable explain output v4
Mike i...@snappymail.ca writes: Have any tool authors stepped up and committed resources to utilizing this feature in the near term? I don't think anyone's promised much. If you want to have a go at using it, we'd be very happy. I'm guessing that my vision likely exceeds the scope of this feature in its initial form at least, but assuming no one else has stepped up, I'm more than willing to start committing resources as early as this weekend with the understanding that this feature is still in development and likely will change several times before or if its finally committed for 8.5. It's definitely committed for 8.5, but the exact format of the output is (obviously) still subject to change. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Re: pgindent timing (was Re: [COMMITTERS] pgsql: Refactor NUM_cache_remove calls in error report path to a PG_TRY)
Robert Haas wrote: On Tue, Aug 11, 2009 at 12:52 PM, Andrew Dunstanand...@dunslane.net wrote: Robert Haas wrote: I'm not sure there's a good solution to this problem short of making pgindent easy enough that we can make it a requirement for patch submission, and I'm not sure that's practical. But in any case, I think running pgindent immediately after the last CommitFest rather than after a longish delay would be a good idea. Frankly, fixing up patch bitrot caused by pgindent is not terribly difficult in my experience - bitrot caused by code drift is a much harder problem (and yes, git fans, I know git can help with that). Where it really bit me as when it reindented the DATA() statements that were touched by ALTER TABLE ... SET STATISTICS DISTINCT. It's not so hard to compare code, but comparing DATA() lines is the pits. Oh? Maybe that's a problem we need to address more directly. I just looked at what it did to the DATA lines - it seems to have changed 501 of them, and all the changes seem to be to do with tabbing. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Hot standby?
On 08/11/2009 09:56 AM, Kevin Grittner wrote: Bruce Momjianbr...@momjian.us wrote: OK, so it is warm slave. That is technically accurate, given the preceding definitions, but it has disturbing connotations. Enough so, in my view, to merit getting a little more creative in the naming. How about warm replica? Other ideas? I agree that the present moniker misleads. I remember this debate from 6 months ago. :-) I prefer not to try and fit square pegs into round holes. Streaming replication sounds like the best description. It may not be the keywords that newbies are looking for, but too bad for them. Calling it something different than what it is, just so that people who don't understand why it is wrong will have something that approximates the right understanding, is not a just cause. :-) Cheers, mark -- Mark Mielkem...@mielke.cc
Re: [HACKERS] Hot standby?
On Tue, Aug 11, 2009 at 2:48 PM, Mark Mielkem...@mark.mielke.cc wrote: On 08/11/2009 09:56 AM, Kevin Grittner wrote: Bruce Momjian br...@momjian.us wrote: OK, so it is warm slave. That is technically accurate, given the preceding definitions, but it has disturbing connotations. Enough so, in my view, to merit getting a little more creative in the naming. How about warm replica? Other ideas? I agree that the present moniker misleads. I remember this debate from 6 months ago. :-) I prefer not to try and fit square pegs into round holes. Streaming replication sounds like the best description. It may not be the keywords that newbies are looking for, but too bad for them. Calling it something different than what it is, just so that people who don't understand why it is wrong will have something that approximates the right understanding, is not a just cause. :-) Uhm, I think you are confused. Hot Standby = Allow read-only queries on a PostgreSQL server during archive recovery Synchronous (or Streaming) Replication = Allow WAL to be streamed on a byte level rather than copied file-by-file Hot Standby is not any sort of replication. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Re: pgindent timing (was Re: [COMMITTERS] pgsql: Refactor NUM_cache_remove calls in error report path to a PG_TRY)
Andrew Dunstan and...@dunslane.net writes: Robert Haas wrote: Where it really bit me as when it reindented the DATA() statements that were touched by ALTER TABLE ... SET STATISTICS DISTINCT. It's not so hard to compare code, but comparing DATA() lines is the pits. Oh? Maybe that's a problem we need to address more directly. I just looked at what it did to the DATA lines - it seems to have changed 501 of them, and all the changes seem to be to do with tabbing. That's interesting --- the whitespace in those macros has always been wildly inconsistent, so I assumed pgindent wasn't touching them at all. I wonder what it thinks it's doing... regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Hot standby?
On Tue, Aug 11, 2009 at 2:10 PM, Ron Mayerrm...@cheapcomplexdevices.com wrote: David Fetter wrote: On Tue, Aug 11, 2009 at 08:56:38AM -0500, Kevin Grittner wrote: Bruce Momjian br...@momjian.us wrote: OK, so it is warm slave. Why isn't it just a read only slave. Do some systems have read-only slave databases that can't serve as a warm standby system as well as this one could? I think that's about right. What we have now via pg_standby or similar tools is a warm standby. What this tool does is makes the warm-standby also serve as a read-only slave. It doesn't make failover any simpler so it's not making the standby any hotter - it instead makes the standby be able to do more useful work when no failover has occurred. The technical description for the commit message is probably something like this: Allow read-only queries to be processed during archive recovery. The P/R version is probably something like this: Warm standby servers now function as read-only slaves. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Re: pgindent timing (was Re: [COMMITTERS] pgsql: Refactor NUM_cache_remove calls in error report path to a PG_TRY)
Andrew Dunstan escribió: Tom Lane wrote: Andrew Dunstan and...@dunslane.net writes: Robert Haas wrote: Where it really bit me as when it reindented the DATA() statements that were touched by ALTER TABLE ... SET STATISTICS DISTINCT. It's not so hard to compare code, but comparing DATA() lines is the pits. Oh? Maybe that's a problem we need to address more directly. I just looked at what it did to the DATA lines - it seems to have changed 501 of them, and all the changes seem to be to do with tabbing. That's interesting --- the whitespace in those macros has always been wildly inconsistent, so I assumed pgindent wasn't touching them at all. I wonder what it thinks it's doing... Here's the extract attached. I replace tabs with a literal '\t' so I could see what it was doing. I can't make much head or tail of it either. pgindent uses entab/detab, which counts spaces and replaces them with tabs. It is wildly undocumented. See src/tools/entab -- Alvaro Herrerahttp://www.CommandPrompt.com/ PostgreSQL Replication, Consulting, Custom Development, 24x7 support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Any tutorial or FAQ on building an extension?
On Aug 11, 1:11 pm, j...@agliodbs.com (Josh Berkus) wrote: Is there an easier way of going about this other than replacing the postmaster / postgres components? I'd start with creating my own extended version to psql (the client library), I suppose. But since I don't really know what kind of transformations you have in mind, any advice is going to be purely speculative. Thanks for the response Josh. I'm not sure that psql is the right thing for me to do though, since I want to build a back-end component that takes the place of the existing postmaster. Very possible I misunderstood you though. To clarify, essentially what I want to do is this: Client [ psql | JDBC driver | pgAdmin | etc. ] issues a Query [ Select * from sales ] | | \/ My new component intercepts this, and decides if it wants to do something | | \/ If it does not, it simply passes this on to the real PostgreSQL server running somewhere | | \/ If it does, it passes the request over to my new server (via sockets), does its work, and pass back the results | | \/ The client gets the results back, either from PostgreSQL or from my new server, and goes about its way. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Hot standby?
On 08/11/2009 02:52 PM, Robert Haas wrote: On Tue, Aug 11, 2009 at 2:48 PM, Mark Mielkem...@mark.mielke.cc wrote: I remember this debate from 6 months ago. :-) I prefer not to try and fit square pegs into round holes. Streaming replication sounds like the best description. It may not be the keywords that newbies are looking for, but too bad for them. Calling it something different than what it is, just so that people who don't understand why it is wrong will have something that approximates the right understanding, is not a just cause. :-) Uhm, I think you are confused. Hot Standby = Allow read-only queries on a PostgreSQL server during archive recovery Synchronous (or Streaming) Replication = Allow WAL to be streamed on a byte level rather than copied file-by-file Hot Standby is not any sort of replication. I don't think I was confused before - but I am confused now. :-) This patch does not provide streaming replication? Having the standby allow for read-only queries is a would be nice for me, but it's not very useful. I've been monitoring these threads (and wishing they were at a level I could participate and contribute on), because I want the ability to have near real time updates such that the standby can become live. Hot standby to me means the slave is as close to up-to-date as possible and can potentially take over at any time in a near instance. This *implies* some sort of streaming replication (byte level rather than file-by-file) rather than waiting for WAL logs to become full and shipped. If this patch doesn't give me near real time replication, then I am confused about why I would want it at all. pg_standby already gives the ability to do replication on a per completed WAL log file basis. Cheers, mark -- Mark Mielkem...@mielke.cc
Re: [HACKERS] Hot standby?
Mark Mielke escribió: I don't think I was confused before - but I am confused now. :-) This patch does not provide streaming replication? No. What it does is allow you to query the slave while it's still replaying transactions. There's another patch allowing you to do stream replication. Hopefully both things could be used simultaneously. -- Alvaro Herrerahttp://www.CommandPrompt.com/ PostgreSQL Replication, Consulting, Custom Development, 24x7 support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Hot standby?
Mark Mielke m...@mark.mielke.cc wrote: This patch does not provide streaming replication? There's a separate effort to provide asynchronous and synchronous streaming replication. Different patch. Hot standby to me means the slave is as close to up-to-date as possible and can potentially take over at any time in a near instance. This *implies* some sort of streaming replication (byte level rather than file-by-file) rather than waiting for WAL logs to become full and shipped. Most of use would expect that from something called hot standby. That's why so many of us have been saying that the name is misleading. This patch, as I understand it, would allow you to use the warm standby to run read-only queries -- for reports and such, to take some load off the primary database. No more; no less. So it's still warm, not hot, and it's still usable as a warm standby. It just lets you squeeze a little extra benefit from the copy while it sits there periodically updating itself. -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Re: pgindent timing (was Re: [COMMITTERS] pgsql: Refactor NUM_cache_remove calls in error report path to a PG_TRY)
Alvaro Herrera alvhe...@commandprompt.com writes: Andrew Dunstan escribió: Here's the extract attached. I replace tabs with a literal '\t' so I could see what it was doing. I can't make much head or tail of it either. pgindent uses entab/detab, which counts spaces and replaces them with tabs. It is wildly undocumented. See src/tools/entab What surprises me is that it seems to be changing lines that have been there for quite some time. Bruce, have you been changing pgindent underneath us? regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] machine-readable explain output v4
Tom Lane wrote: Mike i...@snappymail.ca writes: Have any tool authors stepped up and committed resources to utilizing this feature in the near term? I don't think anyone's promised much. If you want to have a go at using it, we'd be very happy. I'm guessing that my vision likely exceeds the scope of this feature in its initial form at least, but assuming no one else has stepped up, I'm more than willing to start committing resources as early as this weekend with the understanding that this feature is still in development and likely will change several times before or if its finally committed for 8.5. It's definitely committed for 8.5, but the exact format of the output is (obviously) still subject to change. Good. I had a look at this for a little while yesterday. I built it, did an install, loaded auto_explain and then ran the regression tests. I didn't like the output much. It looks like the XML has been dumbed down to fit a data model that works for JSON as well, particularly in the lack of use of attributes. An XML processor won't care that much, but humans will certainly find it more tiresome to read. In effect we are swapping horizontal expansion for vertical expansion. It would be nicer to be able to fit a plan into a screen. I also took the last relaxng spec I could find and used a nice little tool called Trang to translate it into an XML Schema spec. The good news is that that worked. The bad news is that the spec almost certainly needs some tightening, especially around those elements that probably should be XML attributes. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Any tutorial or FAQ on building an extension?
Matt Culbreth mattculbr...@gmail.com wrote: My new component intercepts this, and decides if it wants to do something If it does, it passes the request over to my new server (via sockets), does its work, and pass back the results That's still too vague to allow people to give very specific advice. For example, I don't have a clue yet whether the query rewrite rules might satisfy your needs: http://www.postgresql.org/docs/8.4/interactive/sql-createrule.html -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Hot standby?
All, So really, the streaming replication patch should be called hot standby, and the hot standby patch should be called read only slaves? And *why* can't we call it log-based replication? -- Josh Berkus PostgreSQL Experts Inc. www.pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Hot standby?
On Tue, Aug 11, 2009 at 4:07 PM, Josh Berkusj...@agliodbs.com wrote: All, So really, the streaming replication patch should be called hot standby, and the hot standby patch should be called read only slaves? And *why* can't we call it log-based replication? +1 *) it _is_ used to replicate a database (replicate means make a copy!) *) our target market will perceive it that way *) sounds cool 'synchronous log-based replication' 'asynchronous log-based replication' or, 'log-based replication', in both synchronous and asynchronous modes merlin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] dependencies for generated header files
On Sunday 28 June 2009 21:21:35 Robert Haas wrote: I think that our dependencies for generated header files (gram.h, fmgroids.h, probes.h) are not as good as they could be. What we do right now is make src/backend/Makefile rebuild these before recursing through its subdirectories. This works OK for a top-level make, but if you run make further down in the tree (like under src/backend/commands) it won't necessarily rebuild everything that it should. The attached patch moves some of this logic from src/backend/Makefile to src/Makefile.global.in. That way, if you --enable-depend and then do something like touch src/include/catalog/pg_proc.h and then cd src/backend/commands; make vacuum.o, it rebuilds fmgroids.h and then recompiles vacuum.c. Under HEAD, it just tells you that vacuum.o is up to date. I'm not convinced by this use case. Why do you want to rebuild vacuum.o anyway? If you change something, then you probably want to rebuild postgres, and that works fine without this change. I'm concerned how this change moves random make rules into a global makefile. This is exacerbated by your next patch, which among other things moves the entire catalog list to build the various output files into Makefile.global, far away from its home. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] machine-readable explain output v4
On Tuesday 11 August 2009 21:59:48 Andrew Dunstan wrote: Tom Lane wrote: Mike i...@snappymail.ca writes: Have any tool authors stepped up and committed resources to utilizing this feature in the near term? I don't think anyone's promised much. If you want to have a go at using it, we'd be very happy. I'm guessing that my vision likely exceeds the scope of this feature in its initial form at least, but assuming no one else has stepped up, I'm more than willing to start committing resources as early as this weekend with the understanding that this feature is still in development and likely will change several times before or if its finally committed for 8.5. It's definitely committed for 8.5, but the exact format of the output is (obviously) still subject to change. Good. I had a look at this for a little while yesterday. I built it, did an install, loaded auto_explain and then ran the regression tests. I didn't like the output much. It looks like the XML has been dumbed down to fit a data model that works for JSON as well, particularly in the lack of use of attributes. An XML processor won't care that much, but humans will certainly find it more tiresome to read. In effect we are swapping horizontal expansion for vertical expansion. It would be nicer to be able to fit a plan into a screen. The problem is that nobody yet came up with one that is easy to generate and liked by many people... Aesthetically I do not like the current schema as well, but personally I don't think it matters that much. Everything fancy has the problem of requiring relatively much code... I also took the last relaxng spec I could find and used a nice little tool called Trang to translate it into an XML Schema spec. The good news is that that worked. The bad news is that the spec almost certainly needs some tightening, especially around those elements that probably should be XML attributes. Unrelated to the other issues the relaxng schema has some missing pieces if I remember it correctly (constraint = constraint-name, trigger = trigger-name) I think). It is also by far not strict enough yet... Andres -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] autogenerating headers bki stuff
On Tuesday 30 June 2009 06:59:51 Robert Haas wrote:
The attached patch merges all of the logic currently in genbki.sh and
Gen_fmgrtab.{sh,pl} into a single script called gen_catalog.pl
I can't really convince myself to like this change. I think there is some
merit that these scripts are separate. I'm not sure what the combined script
buys us except that it is a lot bigger and does everything at once instead of
in two or three steps.
That together with the big makefile moving around makes me think that this
would cause more confusion and not much advantage.
Btw., is this stamp-h business really necessary? I guess you copied that from
pg_config.h, but the reason there is that everything depends on pg_config.h,
and rerunning configure would normally cause everything to be rebuilt. The
files we are looking at here should only change when something is edited.
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Shipping documentation untarred
On Tuesday 11 August 2009 17:02:01 Tom Lane wrote: Having all the derived files in the build directory definitely seems to me to reduce the complexity and surprise factor, so +1 for changing. I've looked at that briefly, and it's a bit more complicated than it would appear. I will figure this out later, but it's not going to happen this week. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Hot standby?
On Tue, Aug 11, 2009 at 01:14:56PM -0400, Robert Haas wrote: On Tue, Aug 11, 2009 at 1:08 PM, Josh Berkusj...@agliodbs.com wrote: I believe we're just copying Oracle's terminology. While that terminology is not consistent, it is understood by the industry. Oracle defined their Hot Standby to have both asynchronous and synchronous modes: http://www.oracle.com/technology/products/rdb/htdocs/dbms/hotstandby.html I think that you are wrong. Hot Standby is described in that document as a way of replication the database and providing failover in the event that the master dies. This feature has to do with allowing queries to be executed on the slave, which is not discussed in that document. That web page contains only a brief summary of Oracle's Hot Standby; by looking at the attached PDF guide you can see that the capability of allowing read-only queries on the standby node is actually part of their architecture. Since we are comparing the features with Oracle's Hot Standby, it is perhaps useful to mention two of the major differences: * Oracle's Hot Standby allows only one Standby node, while on PostgreSQL we can have as many as we want, since the Primary is totally unaware of them. Allowing N clones instead of only one may be a crucial advantage in these use cases where read-only queries represent the vast majority of the load. * Oracle's Hot Standby allows cloning hash indexes, which on PostgreSQL are not WAL safe (anyway, they are currently discouraged by the documentation). Best regards, Dr. Gianni Ciolli - 2ndQuadrant Italia PostgreSQL Training, Services and Support gianni.cio...@2ndquadrant.it | www.2ndquadrant.it -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Hot standby?
On Tuesday 11 August 2009 18:16:04 Gianni Ciolli wrote: As for warm/hot, it depends on what you exactly mean with get ready: (A) If you mean it is possible to connect to the second node, then Simon's patch is hot. Yeah, but by that definiton doing a pg_dump/pg_restore every hour is also hot. ;-) Incidentally, we billed pg_dump as hot backup at some point. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Hot standby?
On Tue, Aug 11, 2009 at 4:07 PM, Josh Berkusj...@agliodbs.com wrote: So really, the streaming replication patch should be called hot standby, No. AIUI, hot standby means that when your primary falls over, the secondary automatically promotes itself and takes over. It requires things like heartbeat monitoring and STONITH and is unrelated to anything we currently have under consideration. and the hot standby patch should be called read only slaves? Yes. And *why* can't we call it log-based replication? Well, we can call it anything we want. For example, up until now we've been calling it hot standby, even though that's clearly wrong. :-) But on the merits: log-based replication is, I think, what we already have. Both of these patches do things that make it better. Streaming replication (fka synch rep) makes it more nearly real-time, and read-only slaves (fka hot standby) makes the standby server more useful. But neither is ADDING log-based replication, they're both IMPROVING it, in different ways. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] autogenerating headers bki stuff
Peter Eisentraut escribió:
On Tuesday 30 June 2009 06:59:51 Robert Haas wrote:
The attached patch merges all of the logic currently in genbki.sh and
Gen_fmgrtab.{sh,pl} into a single script called gen_catalog.pl
I can't really convince myself to like this change. I think there is some
merit that these scripts are separate. I'm not sure what the combined script
buys us except that it is a lot bigger and does everything at once instead of
in two or three steps.
Maybe we can move forward in little steps. For example it would be
excellent to have the schemapg.h file autogenerated instead of having to
edit two copies of those definitions.
I'm not sure that it buys us a lot to have all those things in a single
script. Why not have a script to generate schemapg.h, another one to
generate the bki stuff, another one to generate the fmgrtab header?
They don't seem to share a lot of code anyway (and even if they do,
surely we can add a .pm module containing common code).
--
Alvaro Herrerahttp://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] autogenerating headers bki stuff
On Tue, Aug 11, 2009 at 4:52 PM, Peter Eisentrautpete...@gmx.net wrote:
On Tuesday 30 June 2009 06:59:51 Robert Haas wrote:
The attached patch merges all of the logic currently in genbki.sh and
Gen_fmgrtab.{sh,pl} into a single script called gen_catalog.pl
I can't really convince myself to like this change. I think there is some
merit that these scripts are separate. I'm not sure what the combined script
buys us except that it is a lot bigger and does everything at once instead of
in two or three steps.
That together with the big makefile moving around makes me think that this
would cause more confusion and not much advantage.
Btw., is this stamp-h business really necessary? I guess you copied that from
pg_config.h, but the reason there is that everything depends on pg_config.h,
and rerunning configure would normally cause everything to be rebuilt. The
files we are looking at here should only change when something is edited.
It definitely has less appeal without the anum.h stuff.
I think there is some benefit in having a single script because it
means that if we want to add additional syntax in the future (like
BKI_EXECUTE or DATA_DEFAULTS or some kind of more human-readable
notation for functions or opclasses, all of which were discussed
upthread) there is only one place to change. But is that sufficient
reason to commit it at this point, given that we don't have a
fully-fleshed out design for any of those things? Not sure. Building
schemapg.h automatically seems definitely nice to me.
The stamp-h stuff is quite important for minimizing unnecessary
rebuilds. Without that, any change to any include/catalog/*.h file
will trigger a massive rebuild. With it, it only triggers a rebuild
if one of the outputs actually changes, and only for those portions
for which the output actually changed.
...Robert
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Hot standby and synchronous replication status
Hi, Le 11 août 09 à 07:50, Heikki Linnakangas a écrit : 2009/8/11 Robert Haas robertmh...@gmail.com We should probably have a separate discussion about what the least committable unit would be for this patch. I wonder if it might be sufficient to provide a facility for streaming WAL, plus a standalone tool for receving it and storing it to a file. That's an interesting idea. That would essentially be another method to set up a WAL archive. I'm not sure it's worthwhile on its own, but once we have the wal-sender infrastructure in place it should be easy to write such a tool. Well it might be over engineering time *again* but here's what it makes me think about: We should somehow provide a default archive and restore command integrated into the main product, so that it's as easy as turning it 'on' in the configuration for users to have something trustworthy: PostgreSQL will keep past logs into a pg_xlog/archives subdir or some other default place, and will know about the setup at startup time when/if needed. Now, the archive and restore commands would make a independent subsystem, the only one (for modularisation sake) allowed to work with the archives. So we extend it to support sending and receiving archives to/from a remote PostgreSQL server, using libpq and the already proposed protocol in the current patch form. It could be that for integration purpose we'd need something like the autovacuum launcher, an archive manager daemon child of postmaster accepting signals in order to spawn a specific tasks. Sender part could be launched more than once at any time, e.g. Of course the included automatic and easy to setup daemon wouldn't care about setting up a remote archiving policy, but on the other hand a remote PostgreSQL instance could easily be set up as a wal receiver from the master's archive. The archive retention policy and how it applies to known list of receivers is to discuss :) As far as the (a)Sync Rep patch is concerned, this could solve the setup part of it, this step where starting from a filesystem level backup you realize you need archived WALs before being able to apply currently received entries (LSN granularity). Regards, -- dim -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Hot standby?
Incidentally, we billed pg_dump as hot backup at some point. mysql calls mysqlhotcopy a script that locks and flushes all tables, then makes a copy of the database directory (all queries being locked out while this is in progress, of course). -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] dependencies for generated header files
On Tue, Aug 11, 2009 at 4:38 PM, Peter Eisentrautpete...@gmx.net wrote: On Sunday 28 June 2009 21:21:35 Robert Haas wrote: I think that our dependencies for generated header files (gram.h, fmgroids.h, probes.h) are not as good as they could be. What we do right now is make src/backend/Makefile rebuild these before recursing through its subdirectories. This works OK for a top-level make, but if you run make further down in the tree (like under src/backend/commands) it won't necessarily rebuild everything that it should. The attached patch moves some of this logic from src/backend/Makefile to src/Makefile.global.in. That way, if you --enable-depend and then do something like touch src/include/catalog/pg_proc.h and then cd src/backend/commands; make vacuum.o, it rebuilds fmgroids.h and then recompiles vacuum.c. Under HEAD, it just tells you that vacuum.o is up to date. I'm not convinced by this use case. Why do you want to rebuild vacuum.o anyway? If you change something, then you probably want to rebuild postgres, and that works fine without this change. Well, my original motivation for developing this patch was that I often go into a subdirectory and start hacking on a .c file and then type make to rebuild just that subdirectory (so it's easier to see warnings and errors). But sometimes I would edit a relevant header file and then the rebuild wouldn't happen. That bugged me. Your mileage may vary. I'm concerned how this change moves random make rules into a global makefile. This is exacerbated by your next patch, which among other things moves the entire catalog list to build the various output files into Makefile.global, far away from its home. *shrug* You don't have to accept the patch, but I'm unclear as to how make from a subdirectory will ever work reliably without something like this. The problem occurs when .c files have dependencies on files that are generated by a Makefile in some other subdirectory. So it's not random stuff: it's the stuff where that particular thing happens. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Hot standby and synchronous replication status
On Tue, Aug 11, 2009 at 5:20 PM, Dimitri Fontainedfonta...@hi-media.com wrote: We should somehow provide a default archive and restore command integrated into the main product, so that it's as easy as turning it 'on' in the configuration for users to have something trustworthy: PostgreSQL will keep past logs into a pg_xlog/archives subdir or some other default place, and will know about the setup at startup time when/if needed. I might be missing something, but isn't this completely silly? If you archive your logs to the same partition where you keep your database cluster, it seems to me that you might as well delete them. Even better, turn off XLogArchiving altogether and save yourself the overhead of not using WAL-bypass. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Any tutorial or FAQ on building an extension?
On Tue, Aug 11, 2009 at 4:00 PM, Kevin Grittnerkevin.gritt...@wicourts.gov wrote: Matt Culbreth mattculbr...@gmail.com wrote: My new component intercepts this, and decides if it wants to do something If it does, it passes the request over to my new server (via sockets), does its work, and pass back the results That's still too vague to allow people to give very specific advice. For example, I don't have a clue yet whether the query rewrite rules might satisfy your needs: http://www.postgresql.org/docs/8.4/interactive/sql-createrule.html Yeah. I suspect you are going to better off using some combination of views, triggers, set-returning functions, and SQL-level permissions to do whatever it is that you are trying to do here rather than inventing a whole middleware layer. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] machine-readable explain output v4
On Tue, Aug 11, 2009 at 3:59 PM, Andrew Dunstanand...@dunslane.net wrote: Good. I had a look at this for a little while yesterday. I built it, did an install, loaded auto_explain and then ran the regression tests. I didn't like the output much. It looks like the XML has been dumbed down to fit a data model that works for JSON as well, particularly in the lack of use of attributes. An XML processor won't care that much, but humans will certainly find it more tiresome to read. In effect we are swapping horizontal expansion for vertical expansion. It would be nicer to be able to fit a plan into a screen. Isn't that what text format is for? Not that the XML format is perfect; I haven't heard one person say that they like anything about it except that it is already implemented. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Hot standby and synchronous replication status
Le 11 août 09 à 23:30, Robert Haas a écrit : On Tue, Aug 11, 2009 at 5:20 PM, Dimitri Fontainedfonta...@hi-media.com wrote: We should somehow provide a default archive and restore command integrated into the main product, so that it's as easy as turning it 'on' in the configuration for users to have something trustworthy: PostgreSQL will keep past logs into a pg_xlog/archives subdir or some other default place, and will know about the setup at startup time when/if needed. I might be missing something, but isn't this completely silly? If you archive your logs to the same partition where you keep your database cluster, it seems to me that you might as well delete them. Even better, turn off XLogArchiving altogether and save yourself the overhead of not using WAL-bypass. Nice, the pushback is about the default location, thanks for supporting the idea :) Seriously, debian package will install pg_xlog in $PGDATA which is often not what I want. So first thing after install, I stop the cluster, move the pg_xlog, setup a ln -s and restart. I figured having to do the same for setting up archiving would make my day, when compared to current documentation setup. Any better idea for a safe enough default location is welcome, of course. Oh, and I hope you didn't read that the archive mode be 'on' by default in my proposal, because that's not what I meant. Regards, -- dim -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Re: pgsql: Ship documentation without intermediate tarballs Documentation
On Monday 10 August 2009 23:03:12 Alvaro Herrera wrote: Hmm, I notice that this rule to install manpages is pretty slow: for file in /pgsql/source/00head/doc/src/sgml/man1/*.1 /pgsql/source/00head/doc/src/sgml/man3/*.3 /pgsql/source/00head/doc/src/sgml/man7/*.7; do /bin/sh /pgsql/source/00head/config/install-sh -c -m 644 $file /pgsql/install/00head/share/man/`echo $file | sed 's,^/pgsql/source/00head/doc/src/sgml/,,'` || exit; done Can we use basename here instead of the `echo | sed` hack? Hmm, oh, I see it's stripping everything except the last directory level. I guess I'd go for doing a simple cp inside each man directory. Yeah, that was really freakishly complicated. Fixed now. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] machine-readable explain output v4
Robert Haas robertmh...@gmail.com wrote: Andrew Dunstanand...@dunslane.net wrote: find it more tiresome to read. In effect we are swapping horizontal expansion for vertical expansion. It would be nicer to be able to fit a plan into a screen. Isn't that what text format is for? In my experience XML which represents anything of any real complexity is tiresome to read, period. I would worry more about it accurately representing the data and being easy to massage with software. That's not an opinion on any particular choice here, just on what should matter in making a choice. -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Hot standby and synchronous replication status
On Tue, 2009-08-11 at 17:30 -0400, Robert Haas wrote: On Tue, Aug 11, 2009 at 5:20 PM, Dimitri Fontainedfonta...@hi-media.com wrote: We should somehow provide a default archive and restore command integrated into the main product, so that it's as easy as turning it 'on' in the configuration for users to have something trustworthy: PostgreSQL will keep past logs into a pg_xlog/archives subdir or some other default place, and will know about the setup at startup time when/if needed. I might be missing something, but isn't this completely silly? If you archive your logs to the same partition where you keep your database cluster, it seems to me that you might as well delete them. Even better, turn off XLogArchiving altogether and save yourself the overhead of not using WAL-bypass. Depends on all kinds of factors. For example, PITRTools will keep a copy local until it knows that the remote has received it. Joshua D. Drake ...Robert -- PostgreSQL - XMPP: jdr...@jabber.postgresql.org Consulting, Development, Support, Training 503-667-4564 - http://www.commandprompt.com/ The PostgreSQL Company, serving since 1997 -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] dependencies for generated header files
Robert Haas escribió: *shrug* You don't have to accept the patch, but I'm unclear as to how make from a subdirectory will ever work reliably without something like this. The problem occurs when .c files have dependencies on files that are generated by a Makefile in some other subdirectory. So it's not random stuff: it's the stuff where that particular thing happens. But what use would have to build that particular .o file, and not the whole postgres binary? -- Alvaro Herrerahttp://www.CommandPrompt.com/ PostgreSQL Replication, Consulting, Custom Development, 24x7 support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Hot standby and synchronous replication status
On Tue, Aug 11, 2009 at 5:40 PM, Dimitri Fontainedfonta...@hi-media.com wrote: Le 11 août 09 à 23:30, Robert Haas a écrit : On Tue, Aug 11, 2009 at 5:20 PM, Dimitri Fontainedfonta...@hi-media.com wrote: We should somehow provide a default archive and restore command integrated into the main product, so that it's as easy as turning it 'on' in the configuration for users to have something trustworthy: PostgreSQL will keep past logs into a pg_xlog/archives subdir or some other default place, and will know about the setup at startup time when/if needed. I might be missing something, but isn't this completely silly? If you archive your logs to the same partition where you keep your database cluster, it seems to me that you might as well delete them. Even better, turn off XLogArchiving altogether and save yourself the overhead of not using WAL-bypass. Nice, the pushback is about the default location, thanks for supporting the idea :) Seriously, debian package will install pg_xlog in $PGDATA which is often not what I want. So first thing after install, I stop the cluster, move the pg_xlog, setup a ln -s and restart. I figured having to do the same for setting up archiving would make my day, when compared to current documentation setup. Any better idea for a safe enough default location is welcome, of course. *scratches head* I don't really know how you COULD pick a safe default location. Presumably any location that's in the default postgresql.conf file would be under $PGDATA, which kind of defeats the purpose of the whole thing. In other words, you're always going to have to move it anyway, so why bother with a default that is bound to be wrong? Maybe I'm all wet? ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] machine-readable explain output v4
Robert Haas wrote: On Tue, Aug 11, 2009 at 3:59 PM, Andrew Dunstanand...@dunslane.net wrote: Good. I had a look at this for a little while yesterday. I built it, did an install, loaded auto_explain and then ran the regression tests. I didn't like the output much. It looks like the XML has been dumbed down to fit a data model that works for JSON as well, particularly in the lack of use of attributes. An XML processor won't care that much, but humans will certainly find it more tiresome to read. In effect we are swapping horizontal expansion for vertical expansion. It would be nicer to be able to fit a plan into a screen. Isn't that what text format is for? Not that the XML format is perfect; I haven't heard one person say that they like anything about it except that it is already implemented. Well, I don't think that the fact that we are producing machine-readable output means we can just ignore the human side of it. It is more than likely that such output will be read by both machines and humans. Obviously, we need to make sure that it meets machine processing needs first, but once we have done that we should not ignore the human requirements. BTW, I think it's great that we have machine-readable formats. This opens the door to a lot of tools. Well done. No doubt there will be disagreements about the formats. Our community certainly has a tendency to argue over matters of appearance, sometimes to the point of silliness. My understanding was that the idea of getting this in now was to let people have a look, and play to some extent. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Hot standby?
Pierre Frédéric Caillaud wrote: Incidentally, we billed pg_dump as hot backup at some point. mysql calls mysqlhotcopy a script that locks and flushes all tables, then makes a copy of the database directory (all queries being locked out while this is in progress, of course). Doesn't sound very hot to me. Tepid at best. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Re: pgsql: Ship documentation without intermediate tarballs Documentation
Peter Eisentraut wrote: On Monday 10 August 2009 23:03:12 Alvaro Herrera wrote: Hmm, I notice that this rule to install manpages is pretty slow: Yeah, that was really freakishly complicated. Fixed now. It is much better now, thanks. -- Alvaro Herrerahttp://www.CommandPrompt.com/ PostgreSQL Replication, Consulting, Custom Development, 24x7 support -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Hot standby?
Incidentally, we billed pg_dump as hot backup at some point. It *is* hot backup as in taken while the database is running and fully accessible. mysql calls mysqlhotcopy a script that locks and flushes all tables, then makes a copy of the database directory (all queries being locked out while this is in progress, of course). Which is actually 'cold backup'; it's only hot compared to earlier MySQLs where you had to shut the db down. So it's sort of a lukewarm backup. -- Josh Berkus PostgreSQL Experts Inc. www.pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] TODO item: Allow more complex user/database default GUC settings
Hi, There's a longstanding TODO item, in subject. Previous discussion was here: http://archives.postgresql.org/pgsql-hackers/2006-09/msg02341.php In looking what it would take to implement it, I find that it is trivial. The only part that looks complex is the UI for it. Is anyone interested in giving this patch a shot? Implementation-side, it requires a new catalog (pg_settings), with the following columns: setdatabase oid setrole oid setconfig text[] datconfig and rolconfig are removed. ALTER DATABASE / SET sets setdatabase to the OID of the database in command, and setrole to 0 (InvalidOid); ALTER ROLE / SET sets setrole and leaves setdatabase 0. A new command allows one to set a config that applies to both database and role. At startup, the settings are applied in the following order: database=value role=0 database=0 role=value database=value role=value This way, current behavior is maintained (ALTER ROLE trumps ALTER DATABASE). The only real work in this is figuring out what the grammar for the new command looks like. Maybe we could have some like ALTER ROLE foo ALTER DATABASE bar SET config There are of course many possible variations but this looks the most reasonable one. Any better ideas? -- Alvaro Herrerahttp://www.CommandPrompt.com/ The PostgreSQL Company - Command Prompt, Inc. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [COMMITTERS] pgsql: Ship documentation without intermediate tarballs Documentation
On Monday 10 August 2009 18:59:51 Tom Lane wrote: After doing make then make distclean in doc/src/sgml, I see the following undesirable files left behind: -rw-rw-r-- 1 tgl tgl 58 Aug 10 11:51 version.sgml -rw-rw-r-- 1 tgl tgl 38548 Aug 10 11:51 features-unsupported.sgml -rw-rw-r-- 1 tgl tgl 42014 Aug 10 11:51 features-supported.sgml -rw-rw-r-- 1 tgl tgl 345398 Aug 10 11:52 HTML.index -rw-rw-r-- 1 tgl tgl 298859 Aug 10 11:52 bookindex.sgml -rw-rw-r-- 1 tgl tgl 0 Aug 10 11:53 html-stamp I would argue that both make clean and make distclean should remove these. OK, I fixed that, but html-stamp has to stay to keep the dependencies satisfied. (Well, there are alternatives. We could make the whole thing depend on html/index.html or whatever.) Also, we seem to need .cvsignore entries for the html/ and manN/ subdirectories. IMO the policy for .cvsignore is that anything intentionally left behind by make distclean is to be cvsignore'd. Fixed. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Alpha 1 release notes
OK, since there was no clear consensus or volunteer for preparing release notes for alpha 1, I have started something. Let me know what you think. (reStructuredText, if you want to play around) .. -*- mode: rst -*- = Release 8.5alpha1 = .. last commit: Simplify and speed up man page installation Overview PostgreSQL alpha releases are snapshots of development code. They are intended to preview and test upcoming features and to provide the possibility for early feedback. They should not be used in production installations or active development projects. While the PostgreSQL code is continuously subject to a number of automated and manual tests, alpha releases might have serious bugs. Also features may be changed incompatibly or removed at any time during the development cycle. The development cycle of a PostgreSQL major release alternates between periods of development and periods of integration work, called commit fests, normally one month each. Alpha releases are planned to be produced at the end of every commit fest, thus every two months. Since the first commit fest starts within a month from the beginning of development altogether, early alpha releases are not indicative of the likely feature set of the final release. The release notes below highlight user visible changes and new features. There are normally numerous bug fixes and performance improvements in every new snapshot of PostgreSQL, and it would be too bulky to attempt to list them all. Note that many bug fixes are also backported to stable releases of PostgreSQL, and you should be using those if you are looking for bug-fix-only upgrades for your current installations. Migration - To upgrade from any release to an alpha release or from an alpha release to any other release will most likely require a dump/restore upgrade procedure. It may happen that this is not necessary in particular cases, but that is not verified beforehand. (The server will warn you in any case when a dump/restore is necessary if you attempt to use it with an old data directory.) Note, however, that the dump/restore upgrade procedure is expected to work for alpha releases, and problems in this area should be reported. Changes --- Server ~~ - Added log_line_prefix placeholder %e to contain the current SQL state - Allow parentheses around the query expression that follows a WITH clause. - Fixed ancient bug in handling of to_char modifier 'TH', when used with HH. - Use floor() not rint() when reducing precision of fractional seconds in timestamp_trunc, timestamptz_trunc, and interval_trunc() for the float-datetime case. - More sensible values for character_octet_length column in information schema. - Information schema updated to SQL:2008 - Backend header files are now safe to use with C++. - Make GEQO's planning deterministic by having it start from a predictable random number seed each time. - Rewrite GEQO's gimme_tree function so that it always finds a legal join sequence. Previously, it could have failed to produce a plan in some cases. - Simplify the forms foo true and foo false to foo = true and foo = false. - DROP IF EXISTS for columns and constraints - Tweak TOAST code so that columns marked with MAIN storage strategy are not forced out-of-line unless that is necessary to make the row fit on a page. Previously, they were forced out-of-line if needed to get the row down to the default target size (1/4th page). - EXPLAIN can now take generic options - EXPLAIN allows output of plans in XML or JSON format. - Support (scientific notation) in to_char(). - Allow * as parameter for FORCE QUOTE for COPY CSV - Support deferrable uniqueness constraints. - Added ALTER TABLE ... ALTER COLUMN ... SET STATISTICS DISTINCT - has_sequence_privilege() - New hex-string input and output for type BYTEA. - Fast shutdown stop should forcibly disconnect any active backends, even if a smart shutdown is already in progress. Backpatched to 8.3. pg_dump ~~~ - Fixed up --binary-upgrade option so that it behaves properly with inherited columns and check constraints. - Properly restore pg_largeobject.relfozenxid in binary upgrade mode. - Make pg_dump/pg_restore --clean options drop large objects too. - Modify parallel pg_restore logic to avoid potential O(N^2) slowdown in extreme cases. psql - Have \\d show child tables that inherit from the specified parent - Show definition of index columns in \\d on index ecpg - Added STRING datatype for Informix compatibility mode. Procedural Languages - Improve plpgsql's ability to cope with rowtypes containing dropped columns, Contrib ~~~ - Multi-threaded version of pgbench - Adds the ability to retrieve async notifications using dblink, via the addition of the function dblink_get_notify(). - Add matchorig, matchsynonyms, and keepsynonyms options to
[HACKERS] Collation
Folks, While trying unsuccessfully to help someone in IRC, they pointed me to this: http://www.flexiguided.de/publications.pgcollkey.en.html Is anybody working with the kind people of FlexiGuided GmbH to see about integrating this feature more tightly with PostgreSQL? If not, how would we make such an approach? Cheers, David. -- David Fetter da...@fetter.org http://fetter.org/ Phone: +1 415 235 3778 AIM: dfetter666 Yahoo!: dfetter Skype: davidfetter XMPP: david.fet...@gmail.com Remember to vote! Consider donating to Postgres: http://www.postgresql.org/about/donate -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Hot standby?
On Wed, Aug 12, 2009 at 12:11:28AM +0300, Peter Eisentraut wrote: On Tuesday 11 August 2009 18:16:04 Gianni Ciolli wrote: As for warm/hot, it depends on what you exactly mean with get ready: (A) If you mean it is possible to connect to the second node, then Simon's patch is hot. Yeah, but by that definiton doing a pg_dump/pg_restore every hour is also hot. ;-) OK, but only if (a) the data is so small that the restore takes less than one hour, and if (b) the workload is far from 100% ;-) Since there seem to be multiple views about terminology, it may be useful to recall the proposed wording from http://wiki.postgresql.org/wiki/Hot_Standby : * the first node is called either Primary or Master * the second node is called Standby * the Standby is referred to as a Clone instead of a Slave, to mean that it is an exact copy, which, instead of being built by repeating the actions of the master, is constructed just by implementing their effects. Best regards, Dr. Gianni Ciolli - 2ndQuadrant Italia PostgreSQL Training, Services and Support gianni.cio...@2ndquadrant.it | www.2ndquadrant.it -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Quick pointer required re indexing geometry
Dimitri Fontaine wrote: Paul Matthews p...@netspace.net.au writes: Witting a box@point function easy. Having a spot of trouble trying to figure out where and how to graft this into the GiST stuff. Could someone please point me in the general direction? You want index support for it, I suppose? Yes Without index support (but needed anyway), you implement your code in a C module then make it visible from SQL. ...cut... COMMENT ON OPERATOR @(box, point) IS 'box contains point?'; All done but the comment part :-) Now for adding support for index lookups, you have to see documentation about OPERATOR CLASS and OPERATOR FAMILY. Hope this helps, regards, Thanks overlooked CLASS and FAMILY in my hurry. Hopefully that is where the problem is. What I am hoping to do is provide a "real" patch to make box@point available for all. But we can get to that later. A lot of code in pgsql, but not much in the way of comments.
Re: [HACKERS] Re: pgindent timing (was Re: [COMMITTERS] pgsql: Refactor NUM_cache_remove calls in error report path to a PG_TRY)
Tom Lane wrote: Alvaro Herrera alvhe...@commandprompt.com writes: Andrew Dunstan escribi?: Here's the extract attached. I replace tabs with a literal '\t' so I could see what it was doing. I can't make much head or tail of it either. pgindent uses entab/detab, which counts spaces and replaces them with tabs. It is wildly undocumented. See src/tools/entab What surprises me is that it seems to be changing lines that have been there for quite some time. Bruce, have you been changing pgindent underneath us? Not unless I am CVS committing without showing anyone. ;-) These are the only commits I see recently, which are harmless: revision 1.101 date: 2009/06/11 22:21:44; author: momjian; state: Exp; lines: +7 -2 Document struct/union problem with pgindent. revision 1.100 date: 2008/11/03 15:56:47; author: momjian; state: Exp; lines: +2 -2 Small shell syntax improvement. revision 1.99 date: 2008/04/16 21:03:08; author: momjian; state: Exp; lines: +2 -2 Ignore blank lines in typedef file. revision 1.98 date: 2008/01/16 20:13:44; author: momjian; state: Exp; lines: +2 -2 Improve usage message for pgindent. revision 1.97 date: 2007/12/21 14:20:36; author: momjian; state: Exp; lines: +11 -2008 Modify pgindent to use an external typedefs file rather than included list. and entab is similarly unchanged: revision 1.18 date: 2007/02/08 11:10:27; author: petere; state: Exp; lines: +2 -2 Normalize fgets() calls to use sizeof() for calculating the buffer size where possible, and fix some sites that apparently thought that fgets() will overwrite the buffer by one byte. Also add some strlcpy() to eliminate some weird memory handling. I am stumped, but can keep researching. -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] WIP: getting rid of the pg_database flat file
In the discussion of bug #4919 I wrote: In some sense this is a bootstrap problem: what does it take to get to the point of being able to read pg_database and its indexes? That is necessarily not dependent on the particular database we want to join. Maybe we could solve it by having the relcache write a global cache file containing only entries for the global tables, and load that before we have identified the database we want to join (after which, we'll load another cache file for the local entries). It would doubtless take some rearrangement of the backend startup sequence, but it doesn't seem obviously impossible. Attached is a proof-of-concept patch which shows that this idea makes it possible to start backends without the pg_database flat file, and that the required search of pg_database can be done with an index as long as we have the shared relcache cache file available (which should always be true except for the first backend start after postmaster bootup or crash recovery). There are a few loose ends yet to fix, but on the whole it was easier than I expected. The main costs of doing it this way are: * pg_database has to become a nailed-in-cache relation, as does its index on datname. (Its index on OID will have to be nailed too, unless we can get rid of the kluge that lets autovacuum give InitPostgres a database OID instead of database name. I have not looked at autovacuum yet.) This doesn't really cost anything except a few more bytes in the relcache ... and in reality I suspect pg_database is always in that cache anyway. * We have to have a Schema_pg_database macro in pg_attribute.h. This means a little more hand maintenance (unless we accept Robert Haas' patch to autogenerate all that stuff); but it's still not a big problem. I think this is clearly worth cleaning up and committing, since even without any further progress it eliminates number-of-databases as a significant factor in backend startup time. Does anyone have any objection to the above side-effects? To actually get rid of the pg_database flat file, we'd need to take the further step of teaching the AV launcher to read pg_database for itself, or else refactor things so that the AV workers can do that for it. (Alvaro, any comments about the best way to proceed there?) I'd also like to look into getting rid of the pg_auth flat file. As previously noted, that means postponing client auth to later in the startup sequence. If we were willing to eliminate role membership as an available piece of information for auth method selection, we could still do much of the auth work before initializing the backend proper; in particular we could issue a password challenge and wait for a response, which would be good in terms of reducing our exposure to lightweight DDOS attacks. I'm not sure if anyone will think that's a good tradeoff though, since any attacker who can connect to the postmaster port can probably DDOS the postmaster just fine anyway. Comments? regards, tom lane binLpaO4qRx57.bin Description: look-ma-no-flat-file-1.patch.gz -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Alpha 1 release notes
A Dimecres, 12 d'agost de 2009, Peter Eisentraut va escriure: OK, since there was no clear consensus or volunteer for preparing release notes for alpha 1, I have started something. Let me know what you think. (reStructuredText, if you want to play around) Maybe I'd be interesting to add development docs URL [1] so testers can easily find the syntax of new features, or given that they'll be downloading the alpha tarball they should use that documentation? [1] http://developer.postgresql.org/pgdocs/postgres/index.html -- Albert Cervera i Areny http://www.NaN-tic.com Mòbil: +34 669 40 40 18 -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Re: pgindent timing (was Re: [COMMITTERS] pgsql: Refactor NUM_cache_remove calls in error report path to a PG_TRY)
Alvaro Herrera wrote: Andrew Dunstan escribi?: Tom Lane wrote: Andrew Dunstan and...@dunslane.net writes: Robert Haas wrote: Where it really bit me as when it reindented the DATA() statements that were touched by ALTER TABLE ... SET STATISTICS DISTINCT. It's not so hard to compare code, but comparing DATA() lines is the pits. Oh? Maybe that's a problem we need to address more directly. I just looked at what it did to the DATA lines - it seems to have changed 501 of them, and all the changes seem to be to do with tabbing. That's interesting --- the whitespace in those macros has always been wildly inconsistent, so I assumed pgindent wasn't touching them at all. I wonder what it thinks it's doing... Here's the extract attached. I replace tabs with a literal '\t' so I could see what it was doing. I can't make much head or tail of it either. pgindent uses entab/detab, which counts spaces and replaces them with tabs. It is wildly undocumented. See src/tools/entab I am not sure what documentation you want for it that isn't already there. There is an entab.man, and it is mentioned in the developer's FAQ, and it understands 'entab -h' for help. -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] HEAD docs
On Tuesday 11 August 2009 15:37:42 Andrew Dunstan wrote: have the latest docs at http://developer.postgresql.org/pgdocs/postgres/index.html stopped being built? It sure looks like it somehow. Fixed -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] WIP: getting rid of the pg_database flat file
Tom Lane wrote: I'd also like to look into getting rid of the pg_auth flat file. That would be sad for many users of pgbouncer. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Hot standby?
Robert Haas wrote: On Tue, Aug 11, 2009 at 4:07 PM, Josh Berkusj...@agliodbs.com wrote: So really, the streaming replication patch should be called hot standby, No. AIUI, hot standby means that when your primary falls over, the secondary automatically promotes itself and takes over. It requires things like heartbeat monitoring and STONITH and is unrelated to anything we currently have under consideration. and the hot standby patch should be called read only slaves? Yes. And *why* can't we call it log-based replication? Well, we can call it anything we want. For example, up until now we've been calling it hot standby, even though that's clearly wrong. :-) How about streaming archive logging for synchronous replication, and continuous archive slave for read-only queries on a warm standby. -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com + If your life is a hard drive, Christ can be your backup. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Re: pgindent timing (was Re: [COMMITTERS] pgsql: Refactor NUM_cache_remove calls in error report path to a PG_TRY)
On Tue, Aug 11, 2009 at 4:56 PM, Tom Lanet...@sss.pgh.pa.us wrote: A more aggressive approach would be to run pgindent immediately after the close of *each* commitfest, but that would tend to break patches that had gotten punted to the next fest. What would happen if we ran pgindent immediately after every commit? So nobody would ever see a checkout that wasn't pgindent-clean? The only losers I see would be people working on multi-part patches. If just one patch was committed they would have to resolve the conflicts in their subsequent patches before resubmitting. Of course in all likelihood tom would have rewritten their first patch anyways... -- greg http://mit.edu/~gsstark/resume.pdf -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Alpha 1 release notes
If I didn't read this email I would still be trying to figure out how
to use the explain XML patch. Thanks Albert.
I found the syntax for the explain xml format to be quite difficult to
understand at first, it would be nice to give an example or two, ie:
EXPLAIN (ANALYZE, FORMAT XML) SELECT * FROM foo WHERE i = 4;
The thing that caused me the most trouble was that the , wasn't very
noticeable sitting near the end of this line:
EXPLAIN [ ( { ANALYZE boolean | VERBOSE boolean | COSTS boolean |
FORMAT { TEXT | XML | JSON } } [, ...] ) ] statement
It may just be me, but I read that as the comma being optional, not
mandatory in circumstances where you want to specify more then one
option.
Nonetheless, an additional example with multiple options would sure
help.
Thanks.
On Wed, 12 Aug 2009 01:08:54 +0200
Albert Cervera i Areny alb...@nan-tic.com wrote:
A Dimecres, 12 d'agost de 2009, Peter Eisentraut va escriure:
OK, since there was no clear consensus or volunteer for preparing
release notes for alpha 1, I have started something. Let me know
what you think.
(reStructuredText, if you want to play around)
Maybe I'd be interesting to add development docs URL [1] so testers
can easily find the syntax of new features, or given that they'll be
downloading the alpha tarball they should use that documentation?
[1] http://developer.postgresql.org/pgdocs/postgres/index.html
--
Mike
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Alpha 1 release notes
Mike wrote:
The thing that caused me the most trouble was that the , wasn't very
noticeable sitting near the end of this line:
EXPLAIN [ ( { ANALYZE boolean | VERBOSE boolean | COSTS boolean |
FORMAT { TEXT | XML | JSON } } [, ...] ) ] statement
It may just be me, but I read that as the comma being optional, not
mandatory in circumstances where you want to specify more then one
option.
It is you -- what's optional is having more than one option, but as soon
as you have a second one, the comma is mandatory.
I agree some examples would be useful.
--
Alvaro Herrerahttp://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
