Re: [HACKERS] Keepalive for max_standby_delay
On Wed, 2010-06-16 at 21:56 -0400, Tom Lane wrote: Robert Haas robertmh...@gmail.com writes: On Wed, Jun 9, 2010 at 8:01 PM, Tom Lane t...@sss.pgh.pa.us wrote: Yes, I'll get with it ... Any update on this? Sorry, I've been a bit distracted by other responsibilities (libtiff security issues for Red Hat, if you must know). I'll get on it shortly. I don't think the PostgreSQL project should wait any longer on this. If it does we risk loss of quality in final release, assuming no slippage. From here, I will rework my patch of 31 May to * use arrival time on standby as base for max_standby_delay * make delay apply to both streaming and file cases * min_standby_grace_period - min grace on every query, default 0 Decision time, so thoughts please? -- Simon Riggs www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] beta3 the open items list
On 19 June 2010 14:43, Robert Haas robertmh...@gmail.com wrote: It would be nice if we could make a final push to get these issues resolved and another beta out the door before the end of the month... So should we expect beta3 imminently, or are these issues still outstanding? Thanks Thom -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Why are these modules built without respecting my LDFLAGS?
Tom Lane wrote: Should we try to make that a bit more consistent, and if so how? The shenanigans in Makefile.shlib would get a lot simpler if we said that shlib links always include LDFLAGS *plus* LDFLAGS_SL, but I would think that that would carry substantial risk of breakage. Surely there are cases where linker switches are appropriate for making executables but not shlibs. Perhaps we should set up three variables instead of two, viz LDFLAGS = switches for linking both executables and shlibs LDFLAGS_EX = extra switches for linking executables only LDFLAGS_SL = extra switches for linking shlibs only Then we could get rid of that untrustworthy hack for extracting -L switches ... While we're on the subject... this reminds me of another issue that's come up a few times on the PostGIS mailing lists. AFAICT pg_config doesn't have a method for generating LDFLAGS for libpq client applications, only backend libraries with pg_config --libs. Normally we just go for -lpq but that doesn't always seem to work on platforms where you need to explicitly give all libpq dependencies during link time, e.g. http://postgis.refractions.net/pipermail/postgis-users/2010-April/026349.html. Would LDFLAGS_EX in this case be what we need? If so, could it be exposed via a pg_config --libpq option or similar? ATB, Mark. -- Mark Cave-Ayland - Senior Technical Architect PostgreSQL - PostGIS Sirius Corporation plc - control through freedom http://www.siriusit.co.uk t: +44 870 608 0063 Sirius Labs: http://www.siriusit.co.uk/labs -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] get_whatever_oid, part 1: object types with unqualifed names
(2010/05/27 11:51), Robert Haas wrote:
Link to previous discussion:
http://archives.postgresql.org/pgsql-hackers/2010-05/msg01195.php
Attached, please find a patch which standardizes the following
interface for object types that use unqualifed names.
Oid get_whatever_oid(char *name, bool missing_ok);
It also refactors the existing code to use these functions whenever
possible. I'm going to work up a similar path for the object types
that use qualified names, but I thought it would be a good idea to
send this first before I invest too much time in it.
This patch is not obviously small, but most part of the changeset are
mechanical. So, I could not find any other matters in this patch except
for the following three items.
* Patch format.
This patch uses the unified format, instead of the context format. :D
* ExecAlterOwnerStmt()
The original code uses get_roleid_checked() which does not allow invalid
username, but the new code gives missing_ok = true on the get_role_oid().
It should be fixed.
| --- a/src/backend/commands/alter.c
| +++ b/src/backend/commands/alter.c
| @@ -210,7 +210,7 @@ ExecAlterObjectSchemaStmt(AlterObjectSchemaStmt *stmt)
| void
| ExecAlterOwnerStmt(AlterOwnerStmt *stmt)
| {
| - Oid newowner = get_roleid_checked(stmt-newowner);
| + Oid newowner = get_role_oid(stmt-newowner, true);
|
| switch (stmt-objectType)
| {
* assign_temp_tablespaces()?
| @@ -1116,21 +1116,13 @@ assign_temp_tablespaces(const char *newval, bool
doit, GucSource source)
| continue;
| }
|
| - /* Else verify that name is a valid tablespace name */
| - curoid = get_tablespace_oid(curname);
| + /*
| +* In an interactive SET command, we ereport for bad info.
| +* Otherwise, silently ignore any bad list elements.
| +*/
| + curoid = get_tablespace_oid(curname, source PGC_S_INTERACTIVE);
| if (curoid == InvalidOid)
It seems to me if (!OidIsValid(curoid)) should be here, instead of comparison
with InvalidOid here. However, it may be cleaned up in any other patch instead
of get_whatever_oid() efforts.
| - {
| - /*
| -* In an interactive SET command, we ereport for bad info.
| -* Otherwise, silently ignore any bad list elements.
| -*/
| - if (source = PGC_S_INTERACTIVE)
| - ereport(ERROR,
| - (errcode(ERRCODE_UNDEFINED_OBJECT),
| -errmsg(tablespace \%s\ does not exist,
| - curname)));
| continue;
| - }
|
| /*
| * Allow explicit specification of database's default tablespace
In addition, I could find out the following candidates to be replaced with
the new get_xxx_oid() APIs. Apart from whether these items should be cleaned
up in this patch at once, or not, it seems to me this patch can refactor the
following redundant codes.
* at the CreateRole(CreateRoleStmt *stmt)
| tuple = SearchSysCache1(AUTHNAME, PointerGetDatum(stmt-role));
| if (HeapTupleIsValid(tuple))
| ereport(ERROR,
| (errcode(ERRCODE_DUPLICATE_OBJECT),
| errmsg(role \%s\ already exists,
| stmt-role)));
I saw similar code which was replaced with the new APIs in this patch.
It seems to me if (OidIsValid(get_role_oid(stmt-role, true))) can be used,
and it enables to write the code more clean.
* at the DefineOpFamily()
| /* Get necessary info about access method */
| tup = SearchSysCache1(AMNAME, CStringGetDatum(stmt-amname));
| if (!HeapTupleIsValid(tup))
| ereport(ERROR,
| (errcode(ERRCODE_UNDEFINED_OBJECT),
| errmsg(access method \%s\ does not exist,
| stmt-amname)));
|
| amoid = HeapTupleGetOid(tup);
|
| /* XXX Should we make any privilege check against the AM? */
|
| ReleaseSysCache(tup);
It can be replaced by get_am_oid(stmt-amname, false).
* at the RenameSchema()
| /* make sure the new name doesn't exist */
| if (HeapTupleIsValid(
| SearchSysCache1(NAMESPACENAME,
| CStringGetDatum(newname
| ereport(ERROR,
| (errcode(ERRCODE_DUPLICATE_SCHEMA),
| errmsg(schema \%s\ already exists, newname)));
It is similar to the case of CreateRole(). Does get_namespace_oid()
enables to write the code more clean?
Thanks,
--
KaiGai Kohei kai...@ak.jp.nec.com
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_dump's checkSeek() seems inadequate
Tom Lane wrote: Andrew Dunstan and...@dunslane.net writes: Tom Lane wrote: A somewhat more plausible scenario is that somebody might hope that they could do something like this: echo 'some custom header' pg.dump pg_dump -Fc pg.dump What would anyone hope to achieve by such a manoeuvre, even if it worked, which I am close the dead sure it would not? It looks to me like it probably would actually work, so far as pg_dump is concerned, but _discoverArchiveFormat() would break it because that tries to do an unconditional fseeko(fp, 0, SEEK_SET) (and the position counting is screwed up even if the fseeko fails). That could probably be fixed if anyone thought this scenario was interesting enough to justify work directed specifically at it. IIRC pg_restore expects the archive to begin with the header and TOC produced by pg_dump. Maybe I'm being dense, but I'm not able to see how prefixing that with something else could possibly do something useful or workable. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] suppress automatic recovery after back crash
On Sun, Jun 27, 2010 at 9:02 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Thu, Jun 17, 2010 at 7:25 PM, Tom Lane t...@sss.pgh.pa.us wrote: Then all you need is a tweak to make the postmaster exit(1) after a crash instead of trying to launch recovery. This seems useful to me so here's a patch to implement it. Hm, is it useful in the absence of the other components of the proposed feature? I think so. People are already using PostgreSQL as a feature-rich cache; the point of the other changes Josh mentioned is just to make it more performant. Specifically, he mentioned: (a) Eliminate WAL logging entirely, (b) Eliminate checkpointing, and (c) Turn off the background writer. I'm worked on unlogged tables, which will take us about as far as we're likely to go in the direction of (a), per the discussion on -performance. I haven't thought too much about (b) and (c) so I'm not sure how involved that is, or how far we get just by setting bgwriter_lru_maxpagess=0 as Greg Smith suggested, but, again, it's just a performance optimization of something people are already doing. One stylistic gripe: @@ -80,6 +80,7 @@ enum config_group COMPAT_OPTIONS, COMPAT_OPTIONS_PREVIOUS, COMPAT_OPTIONS_CLIENT, + ERROR_HANDLING, PRESET_OPTIONS, CUSTOM_OPTIONS, DEVELOPER_OPTIONS Please spell that ERROR_HANDLING_OPTIONS, both for consistency with the other enum members and to avoid likely conflicts with other uses of such a generic-looking identifier. I mulled over which of those names was better; updated version, reflecting your proposed naming, attached. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise Postgres Company guc_automatic_restart-v2.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] testing plpython3u on 9.0beta2
On Fri, Jun 25, 2010 at 2:49 PM, Peter Eisentraut pete...@gmx.net wrote: On fre, 2010-06-25 at 10:17 -0400, Tom Lane wrote: Peter Eisentraut pete...@gmx.net writes: The problem is apparently that when CREATE LANGUAGE creates a language from a pg_pltemplate entry, it creates the proname from the tmplhandler name, and if it finds a fitting proname entry already, it used that one. So when you create plpython2 first and plpython3 second, the pg_language entries of the latter point to the pg_proc entries of the former. If you fix that up manually (create additional pg_proc entries and fix the pg_language entries to point there), it works better. The fix ought to be to change the function nmes used by plpython3 ... Right. What shall we do about the catversion? We should go ahead and apply this, either with (my vote) or without (Tom's vote) a catversion bump. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Keepalive for max_standby_delay
On Mon, Jun 28, 2010 at 3:17 AM, Simon Riggs si...@2ndquadrant.com wrote: On Wed, 2010-06-16 at 21:56 -0400, Tom Lane wrote: Robert Haas robertmh...@gmail.com writes: On Wed, Jun 9, 2010 at 8:01 PM, Tom Lane t...@sss.pgh.pa.us wrote: Yes, I'll get with it ... Any update on this? Sorry, I've been a bit distracted by other responsibilities (libtiff security issues for Red Hat, if you must know). I'll get on it shortly. I don't think the PostgreSQL project should wait any longer on this. If it does we risk loss of quality in final release, assuming no slippage. I agree, and had actually been intending to post a similar message in the next day or two. We've been waiting for this for nearly a month. From here, I will rework my patch of 31 May to * use arrival time on standby as base for max_standby_delay Assuming that by this you mean, in the case of SR, the time of receipt of the current WAL chunk, and in the case of the archive, the time of its acquisition from the archive, +1. * make delay apply to both streaming and file cases +1. * min_standby_grace_period - min grace on every query, default 0 I could go either way on this. +0.5, I guess. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_dump's checkSeek() seems inadequate
Andrew Dunstan and...@dunslane.net writes: Tom Lane wrote: echo 'some custom header' pg.dump pg_dump -Fc pg.dump IIRC pg_restore expects the archive to begin with the header and TOC produced by pg_dump. Maybe I'm being dense, but I'm not able to see how prefixing that with something else could possibly do something useful or workable. You'd have to do something like (1) open the file as stdin (2) read the custom header (3) exec pg_restore, telling it to read stdin A possible application for this would be for the header to contain information needed to prepare pg_restore's arguments, like where to contact the server. I still think it's a bit too klugy to justify the effort though. In a more abstract sense, what this would be is basically a custom label for a dump file. I could see that being useful, but if we wanted to support it then it ought to be an actual Feature and not a kluge like this. Something like pg_dump --label 'any string' ... other args ... pg_restore --print-label dump.file regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Keepalive for max_standby_delay
Simon Riggs si...@2ndquadrant.com writes: On Wed, 2010-06-16 at 21:56 -0400, Tom Lane wrote: Sorry, I've been a bit distracted by other responsibilities (libtiff security issues for Red Hat, if you must know). I'll get on it shortly. I don't think the PostgreSQL project should wait any longer on this. If it does we risk loss of quality in final release, assuming no slippage. It will get done. It is not the very first thing on my to-do list. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] get_whatever_oid, part 1: object types with unqualifed names
Robert Haas robertmh...@gmail.com writes: I agree, done. In fact, aren't we leaking a syscache reference here? How so? There's a ReleaseSysCache call. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] get_whatever_oid, part 1: object types with unqualifed names
Robert Haas robertmh...@gmail.com writes: 2010/6/28 KaiGai Kohei kai...@ak.jp.nec.com: * at the RenameSchema() This looks like another syscache reference leak. Actually that one *is* a leak, although it doesn't matter much because the leak occurs only in an error path, so transaction abort will clean up the leaked reference. Still, it's sucky coding style: SearchSysCacheExists should have been used. I'm not sure I agree that replacing SearchSysCacheExists calls (or things that should have been SearchSysCacheExists calls) with OidIsValid(get_whatever_oid()) is an improvement. The Exists call tells what you're actually trying to accomplish. The other way is an overspecification of the required result. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Issue: Deprecation of the XML2 module 'xml_is_well_formed' function
We need to make extensive use of the 'xml_is_well_formed' function provided
by the XML2 module.
Yet the documentation says that the xml2 module will be deprecated since
XML syntax checking and XPath queries
is covered by the XML-related functionality based on the SQL/XML standard in
the core server from PostgreSQL 8.3 onwards.
However, the core function XMLPARSE does not provide equivalent
functionality since when it detects an invalid XML document,
it throws an error rather than returning a truth value (which is what we
need and currently have with the 'xml_is_well_formed' function).
For example:
select xml_is_well_formed('br/br2');
xml_is_well_formed
f
(1 row)
select XMLPARSE( DOCUMENT 'br/br2' );
ERROR: invalid XML document
DETAIL: Entity: line 1: parser error : expected ''
br/br2
^
Entity: line 1: parser error : Extra content at the end of the document
br/br2
^
Is there some way to use the new, core XML functionality to simply return a
truth value
in the way that we need?.
Thanks,
-- Mike Berrow
[HACKERS] Built-in connection pool
The thread on Admission Control caused me to realize that one policy I'd been considering as a possibly useful second tier (execution) admission control policy was actually more appropriate as a first tier (simple) admission control policy. The difference (as defined in the Hellerstein, Stonebraker Hamilton paper) is that second tier policies are enforced after initial query planning, so that they can make use of information acquired during the planning phase; first tier policies are enforced before that. Now, some may not realize it, but PostgreSQL already has a first tier policy, consisting of the max_connections and superuser_reserved_connections GUCs, and maybe others. What I'm looking at could be considered an extension of that first tier or, if you tilt your head right, could be considered a built-in connection pool. The mechanism would be not much more complex than max_connections. My experience with PostgreSQL benchmarks and production use has indicated that performance falls off when you have more active connections than required to saturate available resources. It has also indicated that we get best results by keeping a relatively small number of database transactions busy until they complete, queuing up requests for new transactions so that at saturation, a new transaction begins as soon as a previous one ends. Relatively small, in my experience, is typically somewhere near (2 * core_count) + effective_spindle_count. I think this could be achieved with the following: When preparing to deal with a new statement: - acquire lockX - if not a superuser - and not in an active transaction - and countX = max_active_transactions - place current process at tail of waitX queue, and block - (lockX would be released while blocked) - increment countX - release lockX When preparing to complete (commit or rollback) a transaction: - acquire lockX - decrement countX - if countX max_active_transactions - and waitX queue is not empty - remove process at head of queue and unblock it - release lockX It's possible this code could be place inside of some block of code which is already locked around transaction creation and completion; otherwise, I suspect that a spinlock would suffice. There's not a lot going on inside a block of code protected by lockX. It seems theoretically possible that user locks which span transactions, if used in a certain way, could cause this scheme to deadlock. I'm skeptical that anyone is using them in such a way. If we're worried about that, however, there would need to be some sort of deadlock detection. Another issue would be how to show this to users. If we could show the transaction in pg_locks as having its virtualxid exclusive lock not granted, with waiting showing as true in pg_stat_activity, that would make sense to me. Does this sound at all sane? -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_dump's checkSeek() seems inadequate
Tom Lane wrote: In a more abstract sense, what this would be is basically a custom label for a dump file. I could see that being useful, but if we wanted to support it then it ought to be an actual Feature and not a kluge like this. Something like pg_dump --label 'any string' ... other args ... pg_restore --print-label dump.file Right, or possibly a file that would be added. I don't think we should waste any effort at all on the kluge, or worrying about whether or not it would work. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Built-in connection pool
I wrote: When preparing to deal with a new statement: - acquire lockX - if not a superuser - and not in an active transaction - and countX = max_active_transactions - place current process at tail of waitX queue, and block - (lockX would be released while blocked) - increment countX - release lockX There's a bug there already. This should be better: - if not in an active transaction - acquire lockX - if not a superuser - and countX = max_active_transactions - place current process at tail of waitX queue, and block - (lockX would be released while blocked) - increment countX - release lockX -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Issue: Deprecation of the XML2 module 'xml_is_well_formed' function
On Mon, Jun 28, 2010 at 11:08 AM, Mike Berrow mber...@gmail.com wrote:
We need to make extensive use of the 'xml_is_well_formed' function provided
by the XML2 module.
Yet the documentation says that the xml2 module will be deprecated since
XML syntax checking and XPath queries
is covered by the XML-related functionality based on the SQL/XML standard in
the core server from PostgreSQL 8.3 onwards.
However, the core function XMLPARSE does not provide equivalent
functionality since when it detects an invalid XML document,
it throws an error rather than returning a truth value (which is what we
need and currently have with the 'xml_is_well_formed' function).
For example:
select xml_is_well_formed('br/br2');
xml_is_well_formed
f
(1 row)
select XMLPARSE( DOCUMENT 'br/br2' );
ERROR: invalid XML document
DETAIL: Entity: line 1: parser error : expected ''
br/br2
^
Entity: line 1: parser error : Extra content at the end of the document
br/br2
^
Is there some way to use the new, core XML functionality to simply return a
truth value
in the way that we need?.
You could do something like this (untested):
CREATE OR REPLACE FUNCTION my_xml_is_valid ( x TEXT ) RETURNS BOOL AS $$
BEGIN
PERFORM XMLPARSE( DOCUMENT x::XML );
RETURN TRUE;
EXCEPTION WHEN OTHERS THEN
RETURN FALSE;
END;
$$ LANGUAGE PLPGSQL;
--
Mike Rylander
| VP, Research and Design
| Equinox Software, Inc. / The Evergreen Experts
| phone: 1-877-OPEN-ILS (673-6457)
| email: mi...@esilibrary.com
| web: http://www.esilibrary.com
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] get_whatever_oid, part 1: object types with unqualifed names
On Mon, Jun 28, 2010 at 10:48 AM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: I agree, done. In fact, aren't we leaking a syscache reference here? How so? There's a ReleaseSysCache call. Oops, you're right. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Admission Control
On Sat, Jun 26, 2010 at 01:19:57PM -0400, Robert Haas wrote: I'm not sure. What does seem clear is that it's fundamentally at odds with the admission control approach Kevin is advocating. When you start to run short on a resource (perhaps memory), you have to decide between (a) waiting for memory to become available and (b) switching to a more memory-efficient plan. The danger of (b) is that using less memory probably means using more of some other resource, like CPU or disk, and now you've just switched around which release you're overloading - but on the other hand, if the difference in CPU/disk is small and the memory savings is large, maybe it makes sense. Perhaps in the end we'll find we need both capabilities. I can't help feeling like some good instrumentation would be helpful in answering some of these questions, although I don't know where to put it. One issue with this is that no matter how expensive you make a query, it will run - it just may take a very long time (if the cost is a reasonable estimate) This is also an implied suggestion for a dynamically self-modifying cost param, since the memory cost isn't absolute, but rather relative to free memory. In addition, as Robert points out, the tradeoff between resources is dynamic, as well. Hmm, I'm suddenly struck by the idea of having a max_cost parameter, that refuses to run (or delays?) queries that have too high a cost. That might have some interactive-SQL uses, as well: catch the cases you forgot a join condition, so have an unintended cartesian explosion, etc. Could also be a belt-and-suspenders last defense for DB admins who aren't sure the client software completely stops the users from doing something stupid. Clearly, default to current behavior, -1 (infinity). Ross -- Ross Reedstrom, Ph.D. reeds...@rice.edu Systems Engineer Admin, Research Scientistphone: 713-348-6166 The Connexions Project http://cnx.orgfax: 713-348-3665 Rice University MS-375, Houston, TX 77005 GPG Key fingerprint = F023 82C8 9B0E 2CC6 0D8E F888 D3AE 810E 88F0 BEDE -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Issue: Deprecation of the XML2 module 'xml_is_well_formed' function
On Mon, Jun 28, 2010 at 08:08:53AM -0700, Mike Berrow wrote:
We need to make extensive use of the 'xml_is_well_formed' function provided
by the XML2 module.
Yet the documentation says that the xml2 module will be deprecated since
XML syntax checking and XPath queries
is covered by the XML-related functionality based on the SQL/XML standard in
the core server from PostgreSQL 8.3 onwards.
However, the core function XMLPARSE does not provide equivalent
functionality since when it detects an invalid XML document,
it throws an error rather than returning a truth value (which is what we
need and currently have with the 'xml_is_well_formed' function).
For example:
select xml_is_well_formed('br/br2');
xml_is_well_formed
f
(1 row)
select XMLPARSE( DOCUMENT 'br/br2' );
ERROR: invalid XML document
DETAIL: Entity: line 1: parser error : expected ''
br/br2
^
Entity: line 1: parser error : Extra content at the end of the document
br/br2
^
Is there some way to use the new, core XML functionality to simply
return a truth value in the way that we need?.
Here's a PL/pgsql wrapper for it. You could create a similar wrapper
for other commands.
CREATE OR REPLACE FUNCTION xml_is_well_formed(in_putative_xml TEXT)
STRICT /* Leave this line here if you want RETURNS NULL ON NULL INPUT behavior.
*/
RETURNS BOOLEAN
LANGUAGE plpgsql
AS $$
BEGIN
PERFORM XMLPARSE(DOCUMENT(in_putative_xml));
RETURN true;
EXCEPTION
WHEN invalid_xml_document THEN
RETURN false;
END;
$$;
While tracking this down, I didn't see a way to get SQLSTATE or the
corresponding condition name via psql. Is this an oversight? A bug,
perhaps?
Cheers,
David.
--
David Fetter da...@fetter.org http://fetter.org/
Phone: +1 415 235 3778 AIM: dfetter666 Yahoo!: dfetter
Skype: davidfetter XMPP: david.fet...@gmail.com
iCal: webcal://www.tripit.com/feed/ical/people/david74/tripit.ics
Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] get_whatever_oid, part 1: object types with unqualifed names
On Mon, Jun 28, 2010 at 10:53 AM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: 2010/6/28 KaiGai Kohei kai...@ak.jp.nec.com: * at the RenameSchema() This looks like another syscache reference leak. Actually that one *is* a leak, although it doesn't matter much because the leak occurs only in an error path, so transaction abort will clean up the leaked reference. Still, it's sucky coding style: SearchSysCacheExists should have been used. I'm not sure I agree that replacing SearchSysCacheExists calls (or things that should have been SearchSysCacheExists calls) with OidIsValid(get_whatever_oid()) is an improvement. The Exists call tells what you're actually trying to accomplish. The other way is an overspecification of the required result. It is, but on the other hand it's only good fortune that testing whether a schema exists is a one-liner even without using get_namespace_oid(). Take a look at get_tablespace_oid() in CVS HEAD [which BTW is already used in this style], or at get_trigger_oid() in the get_whatever_oid, part 2 patch, for example. I think the assumption that system objects (with the exception of attributes) are identified by OIDs is embedded pretty deeply in the code, so I don't think it costs us much to rely on it. We're more likely to want to do things like add or remove a syscache, in which case a style that minimizes direct syscache references is apt to make life easier. It's also slightly less wordy, which I like, too. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Issue: Deprecation of the XML2 module 'xml_is_well_formed' function
On Mon, Jun 28, 2010 at 11:42 AM, Mike Rylander mrylan...@gmail.com wrote: You could do something like this (untested): CREATE OR REPLACE FUNCTION my_xml_is_valid ( x TEXT ) RETURNS BOOL AS $$ BEGIN PERFORM XMLPARSE( DOCUMENT x::XML ); RETURN TRUE; EXCEPTION WHEN OTHERS THEN RETURN FALSE; END; $$ LANGUAGE PLPGSQL; This might perform significantly worse, though: exception handling ain't cheap. It's not a bad workaround, but I think the OP has a point. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] get_whatever_oid, part 1: object types with unqualifed names
Robert Haas robertmh...@gmail.com writes: On Mon, Jun 28, 2010 at 10:53 AM, Tom Lane t...@sss.pgh.pa.us wrote: I'm not sure I agree that replacing SearchSysCacheExists calls (or things that should have been SearchSysCacheExists calls) with OidIsValid(get_whatever_oid()) is an improvement. The Exists call tells what you're actually trying to accomplish. The other way is an overspecification of the required result. It is, but on the other hand it's only good fortune that testing whether a schema exists is a one-liner even without using get_namespace_oid(). True. Is it worth providing whatever_exists() macros that wrap get_whatever_oid() like this, just so that callers are a bit clearer as to what they're doing? I'm not certain though how many places it could be used, since many callers probably actually do need the OID, and would have to write separate lines anyway: objoid = get_whatever_oid(...); if (!OidIsValid(objoid)) ereport(...); ... code using objoid here ... But it's been useful to have the SearchSysCacheExists functions, even though they're just wrappers, so maybe whatever_exists() would be worth its keep. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] get_whatever_oid, part 1: object types with unqualifed names
On Mon, Jun 28, 2010 at 12:17 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Mon, Jun 28, 2010 at 10:53 AM, Tom Lane t...@sss.pgh.pa.us wrote: I'm not sure I agree that replacing SearchSysCacheExists calls (or things that should have been SearchSysCacheExists calls) with OidIsValid(get_whatever_oid()) is an improvement. The Exists call tells what you're actually trying to accomplish. The other way is an overspecification of the required result. It is, but on the other hand it's only good fortune that testing whether a schema exists is a one-liner even without using get_namespace_oid(). True. Is it worth providing whatever_exists() macros that wrap get_whatever_oid() like this, just so that callers are a bit clearer as to what they're doing? I'm not certain though how many places it could be used, since many callers probably actually do need the OID, and would have to write separate lines anyway: objoid = get_whatever_oid(...); if (!OidIsValid(objoid)) ereport(...); ... code using objoid here ... But it's been useful to have the SearchSysCacheExists functions, even though they're just wrappers, so maybe whatever_exists() would be worth its keep. I haven't made a detailed study of this issue, so I'm not 100% sure. My gut feeling however is that nearly all of the callers need the OID, and that some of the whatever_exists() functions wouldn't have any callers at all. Which makes me pretty hesitant to add them, especially given our decision not to centralize all the get_whatever_oid() functions in one place. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] get_whatever_oid, part 1: object types with unqualifed names
Robert Haas robertmh...@gmail.com writes: On Mon, Jun 28, 2010 at 12:17 PM, Tom Lane t...@sss.pgh.pa.us wrote: True. Is it worth providing whatever_exists() macros that wrap get_whatever_oid() like this, just so that callers are a bit clearer as to what they're doing? I haven't made a detailed study of this issue, so I'm not 100% sure. My gut feeling however is that nearly all of the callers need the OID, and that some of the whatever_exists() functions wouldn't have any callers at all. Which makes me pretty hesitant to add them, especially given our decision not to centralize all the get_whatever_oid() functions in one place. Well, the whatever_exists() things would just be one-liner macros declared in the same headers that declare the underlying get_whatever_oid() functions. So the cost of carrying ones that happen to not be used would be nil. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Issue: Deprecation of the XML2 module 'xml_is_well_formed' function
Robert Haas wrote: On Mon, Jun 28, 2010 at 11:42 AM, Mike Rylander mrylan...@gmail.com wrote: You could do something like this (untested): CREATE OR REPLACE FUNCTION my_xml_is_valid ( x TEXT ) RETURNS BOOL AS $$ BEGIN PERFORM XMLPARSE( DOCUMENT x::XML ); RETURN TRUE; EXCEPTION WHEN OTHERS THEN RETURN FALSE; END; $$ LANGUAGE PLPGSQL; This might perform significantly worse, though: exception handling ain't cheap. It's not a bad workaround, but I think the OP has a point. Should the IS DOCUMENT predicate support this? At the moment you get the following: template1=# SELECT 'townstownBidford-on-Avon/towntownCwmbran/towntownBristol/town/towns' IS DOCUMENT; ?column? -- t (1 row) template1=# SELECT 'townstownBidford-on-Avon/towntownCwmbran/towntownBristol/town/towns' IS DOCUMENT; ERROR: invalid XML content LINE 1: SELECT 'townstownBidford-on-Avon/towntownCwmbran/to... ^ DETAIL: Entity: line 1: parser error : expected '' ownstownBidford-on-Avon/towntownCwmbran/towntownBristol/town/towns ^ Entity: line 1: parser error : chunk is not well balanced ownstownBidford-on-Avon/towntownCwmbran/towntownBristol/town/towns ^ I would've hoped the second would've returned 'f' rather than failing. I've had a glance at the XML/SQL standard and I don't see anything in the detail of the predicate (8.2) that would specifically prohibit us from changing this behavior, unless the common rule 'Parsing a string as an XML value' (10.16) must always be in force. I'm no standard expert, but IMHO this would be an acceptable change to improve usability. What do others think? Regards, -- Mike Fowler Registered Linux user: 379787 I could be a genius if I just put my mind to it, and I, I could do anything, if only I could get 'round to it -PULP 'Glory Days' -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] get_whatever_oid, part 1: object types with unqualifed names
On Mon, Jun 28, 2010 at 12:31 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Mon, Jun 28, 2010 at 12:17 PM, Tom Lane t...@sss.pgh.pa.us wrote: True. Is it worth providing whatever_exists() macros that wrap get_whatever_oid() like this, just so that callers are a bit clearer as to what they're doing? I haven't made a detailed study of this issue, so I'm not 100% sure. My gut feeling however is that nearly all of the callers need the OID, and that some of the whatever_exists() functions wouldn't have any callers at all. Which makes me pretty hesitant to add them, especially given our decision not to centralize all the get_whatever_oid() functions in one place. Well, the whatever_exists() things would just be one-liner macros declared in the same headers that declare the underlying get_whatever_oid() functions. So the cost of carrying ones that happen to not be used would be nil. True, but I think there's a pretty high chance that they woudn't get used even in places where they ought to, for lack of existing examples, or that new code would fail to add matching macros for new object types. Even so, it's not a terrible idea; I just don't think it should be a requirement for this particular patch. And to be honest, I'd sort of like to see how this shakes out before going too much further with it. Another, and related idea that I had while looking at this is that a lot of object types could benefit from a get_whatever_heaptuple() function with the same calling syntax. get_whatever_oid() could be restructured to use it, and most object types would have other callers, also. But that too seems like opening a larger can of worms than I really want to get into at this point. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Admission Control
Ross J. Reedstrom reeds...@rice.edu writes: Hmm, I'm suddenly struck by the idea of having a max_cost parameter, that refuses to run (or delays?) queries that have too high a cost. That's been suggested before, and shot down on the grounds that the planner's cost estimates are not trustworthy enough to rely on for purposes of outright-failing a query. If you didn't want random unexpected failures, you'd have to set the limit so much higher than your regular queries cost that it'd be pretty much useless. Maybe it'd be all right if it were just used to delay launching the query a bit, but I'm not entirely sure I see the point of that. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] get_whatever_oid, part 1: object types with unqualifed names
Robert Haas robertmh...@gmail.com writes: On Mon, Jun 28, 2010 at 12:31 PM, Tom Lane t...@sss.pgh.pa.us wrote: Well, the whatever_exists() things would just be one-liner macros declared in the same headers that declare the underlying get_whatever_oid() functions. So the cost of carrying ones that happen to not be used would be nil. True, but I think there's a pretty high chance that they woudn't get used even in places where they ought to, for lack of existing examples, or that new code would fail to add matching macros for new object types. Even so, it's not a terrible idea; I just don't think it should be a requirement for this particular patch. And to be honest, I'd sort of like to see how this shakes out before going too much further with it. Well, once you've finished the get_whatever_oid() patch it won't be hard to count how many instances of OidIsValid(get_whatever_oid()) there are. If there's more than a few then I think the macros would be appropriate to provide. Another, and related idea that I had while looking at this is that a lot of object types could benefit from a get_whatever_heaptuple() function with the same calling syntax. get_whatever_oid() could be restructured to use it, and most object types would have other callers, also. But that too seems like opening a larger can of worms than I really want to get into at this point. This is the sort of thing that I think we should get right the first time, rather than have multiple waves of large-scale changes. I'm actually inclined to think we should try to land this stuff in 9.0 if we're going to do it at all. As a new committer, I suspect you do not realize exactly how much pain this sort of thing inflicts on back-patchers. The SearchSysCache call convention changes have already ensured that the 8.4 to 9.0 crossover is going to be a major, major PITA for back-patching, probably nearly as bad as the 8.1 changes in pgindent's comment wrapping rules. (If I had it to do over I'd have vetoed those changes --- I don't even want to think about how many man-days I've lost to that completely useless cosmetic change.) This proposed change will touch many of the same places that were already modified for that. It'd be nice to have only one version boundary where we're manually adjusting back-patched fixes, and not two. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Keepalive for max_standby_delay
It will get done. It is not the very first thing on my to-do list. ??? What is then? If it's not the first thing on your priority list, with 9.0 getting later by the day, maybe we should leave it to Robert and Simon, who *do* seem to have it first on *their* list? I swear, when Simon was keeping his branch to himself in August everyone was on his case. It sure seems like Tom is doing exactly the same thing. -- -- Josh Berkus PostgreSQL Experts Inc. http://www.pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Keepalive for max_standby_delay
On Mon, Jun 28, 2010 at 10:19 AM, Tom Lane t...@sss.pgh.pa.us wrote: Simon Riggs si...@2ndquadrant.com writes: On Wed, 2010-06-16 at 21:56 -0400, Tom Lane wrote: Sorry, I've been a bit distracted by other responsibilities (libtiff security issues for Red Hat, if you must know). I'll get on it shortly. I don't think the PostgreSQL project should wait any longer on this. If it does we risk loss of quality in final release, assuming no slippage. It will get done. It is not the very first thing on my to-do list. That's apparent. On June 9th, you wrote Yes, I'll get with it ...; on June 16th, you wrote I'll get on it shortly. Two weeks later you're backing off from shortly to eventually. It is unfair to the community to assert a vigorous opinion of how something should be handled in the code and then refuse to commit to a time frame for providing a patch. It is even more unreasonable to commit to providing a timely patch (twice) and then fail to do so. We are trying to finalize a release here, and you've made it clear you think this code needs revision before then. I respect your opinion, but not your right to make the project release timetable dependent on your own schedule, and not your right to shut other people out of working on the issues you've raised. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Admission Control
While this does have the advantage of being relatively simple to implement, I think it would be a bitch to tune... Precisely. So, there's a number of issues to solve here: 1) We'd need to add accouting for total memory usage to explain plans (worth doing on its own, really, even without admission control). 2) Our memory usage estimates aren't that accurate, which would still force us to underallocate (as we do with work_mem) if we were letting queries through based on memory usage, or force us to abort queries whose actual memory usage was too far off estimated usage. 3) Due to concurrency, by the time the query executes, there will be a different set of queries executing on the server than were executing at evaluation time. This will also cause underallocation to continue. 4) Many other processes can use major hunks of memory (autovacuum, for example) and would need to be brought under centralized management if this was to be a real fix for the underallocation issue. 5) Running out of CPU is, in my experience, more common than running out of memory on modern systems. So it's questionable how useful managing overall memory usage at all would be. Having tinkered with it, I'll tell you that (2) is actually a very hard problem, so any solution we implement should delay as long as possible in implementing (2). In the case of Greenplum, what Mark did originally IIRC was to check against the global memory pool for each work_mem allocation. This often resulted in 100's of global locking checks per query ... like I said, feasible for DW, not for OLTP. The same is the case with (3). In some ways, (3) is an even harder issue because it allows any kind of admission system to be its own enemy; you can imagine admission storms where the server tries to serve 150 queries which have been waiting for admission at once as soon as current load drops. Given this, I think that actually solving the issue of underallocation, or even materially improving on memory underallocation compared to where we are now, is a much harder issue than a lot of people realize. However, it does point towards two areas for further work: First, (1) would be valuable on its own as part of diagnostics, logging, pg_stat_statements, etc. And seems like a good easy TODO. We can go back to Kevin's originally proposed simple feature: just allowing the DBA to limit the number of concurrently executing queries by role and overall. This would consist of two parameters, max_concurrent_statements and max_statement_wait; the second would say how long the connection would wait for a query slot before giving up and returning an error. This might help the overall underallocation issue, and would be far simpler than schemes involving RAM, I/O and CPU accounting. However, even this solution has two thorny issues to be solved: a) the waiting query storm issue mentioned above, and b) pending queries are sitting on idle connections, which could easily block higher-priority queries, so managing max_connections per role would become much more complex. Overall, it sounds like a lot of work for improving underallocation issues for users who can't make effective use of connection pooling but nevertheless have high-performance needs. Are there enough of these users to make it worthwhile? -- -- Josh Berkus PostgreSQL Experts Inc. http://www.pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Admission Control
Josh Berkus j...@agliodbs.com wrote: We can go back to Kevin's originally proposed simple feature: just allowing the DBA to limit the number of concurrently executing queries by role and overall. Well, that's more sophisticated than what I proposed, but it's an interesting twist on it. This would consist of two parameters, max_concurrent_statements and max_statement_wait; the second would say how long the connection would wait for a query slot before giving up and returning an error. The timeout is also an embellishment to what I proposed, but another interesting idea. even this solution has two thorny issues to be solved: a) the waiting query storm issue mentioned above I fleshed out the idea a bit more on the thread titled Built-in connection pool, since this would effectively function in a very similar way to a connection pool. If you look at that proposal, at most one transaction would be released to execution when one transaction completed. I'm not seeing anything resembling a storm in that, so you must be envisioning something rather different. Care to clarify? b) pending queries are sitting on idle connections, which could easily block higher-priority queries, so managing max_connections per role would become much more complex. That is a good point. The biggest difference between the functionality of the proposal on the other thread and the connection pool built in to our application framework is that the latter has a prioritized FIFO queue, with ten levels of priority. A small query which is run as you tab between controls in a GUI window runs at a much higher priority than a query which fills a list with a large number of rows. This involves both connections reserved for higher priorities and having higher priority transactions jump ahead of lower priority transactions. This helps the user perception of performance in the fat-client GUI applications. I suppose if we had limits by role, we could approach this level of functionality within PostgreSQL. On the other hand, our web apps run everything at the same priority, so there would effectively be *no* performance difference between what I proposed we build in to PostgreSQL and what our shop currently puts in front of PostgreSQL for a connection pool. -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Keepalives win32
Hi! I'm looking at adding win32 support for keepalives in libpq (well, also backend, but libpq for now), per the request from Robert Haas. I've come up with one issue though - in Windows, you can only set the idle and interval parameter together in a single syscall (in Unix, you have one for each). There is no support for setting the counter at all. However, there is no API for *reading* the current value, nor the default value. There is no way to specify set the default. If we set one of them to zero, it really means zero - no interval at all (so it'll flood out the packets - really fun when you enable it for keepalive_idle). The default value for these are available in the registry only. The way I see it, we have two options: 1) Read the default value from the registry. That's some fairly ugly code, imho. 2) Ignore the registry value and use the default value of 2 hours/1 second. That will override any changes the user made in the registry, which seems pretty ugly. 3) Require that these two parameters are always specified together (on windows). Which is annoying. Not sure which one to pick - opinions? The API used is documented at: http://msdn.microsoft.com/en-us/library/dd877220(v=VS.85).aspx Patch as it looks now (libpq only, and with obvious problems with this issue): http://github.com/mhagander/postgres/compare/master...win32keepalive -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Keepalive for max_standby_delay
Robert Haas robertmh...@gmail.com writes: ... It is even more unreasonable to commit to providing a timely patch (twice) and then fail to do so. We are trying to finalize a release here, and you've made it clear you think this code needs revision before then. I respect your opinion, but not your right to make the project release timetable dependent on your own schedule, and not your right to shut other people out of working on the issues you've raised. Since nobody has put forward a proposed beta3 release date, I don't feel that I'm holding anything up. In the meantime, I have many responsibilities and am facing Red Hat internal deadlines. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Propose Beta3 for July
Folks, There have been quite a number of fixes since Beta2. If we're going to make a summer release date at all, we need to get moving. Therefore, I propose that we set a beta3 release date for July 8th. That should give it enough space from the American Holiday. This would imply, of course, that we're going to go ahead an accept Simon's patches for Keepalive, since we don't see to have any others. -- -- Josh Berkus PostgreSQL Experts Inc. http://www.pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Admission Control
On 2010-06-25 22:44, Robert Haas wrote: On Fri, Jun 25, 2010 at 3:52 PM, Kevin Grittner kevin.gritt...@wicourts.gov wrote: Heck, I think an even *more* trivial admission control policy which limits the number of active database transactions released to execution might solve a lot of problems. That wouldn't have any benefit over what you can already do with a connection pooler, though, I think. In fact, it would probably be strictly worse, since enlarging the number of backends slows the system down even if they aren't actually doing anything much. Sorry if I'm asking silly questions, but how does transactions and connection pooler's interact? Say if you have 100 clients all doing fairly inactive database work in transactions lasting a couple of minutes at the same time. If I understand connection poolers they dont help much in those situations where an accounting system on limited resources across all backends definately would help. (yes, its a real-world application here, wether it is clever or not... ) In a fully web environment where all transaction last 0.1s .. a pooler might make fully sense (when traffic goes up). -- Jesper -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Propose Beta3 for July
Josh Berkus j...@agliodbs.com writes: Therefore, I propose that we set a beta3 release date for July 8th. That should give it enough space from the American Holiday. You mean wrap on Thursday the 8th for release on Monday the 12th? That'd be fine with me. Actual release on the 8th would mean asking people to do release prep work when they should be out watching fireworks. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Propose Beta3 for July
On 6/28/10 11:40 AM, Tom Lane wrote: Josh Berkus j...@agliodbs.com writes: Therefore, I propose that we set a beta3 release date for July 8th. That should give it enough space from the American Holiday. You mean wrap on Thursday the 8th for release on Monday the 12th? That'd be fine with me. Actual release on the 8th would mean asking people to do release prep work when they should be out watching fireworks. Yes. I'll need to recruit someone else for the announcement (I'll be in transit on the 12th) but that's easy. -- -- Josh Berkus PostgreSQL Experts Inc. http://www.pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Keepalives win32
Magnus Hagander mag...@hagander.net writes: [ can't read system's keepalive values in windows ] The way I see it, we have two options: 1) Read the default value from the registry. That's some fairly ugly code, imho. 2) Ignore the registry value and use the default value of 2 hours/1 second. That will override any changes the user made in the registry, which seems pretty ugly. 3) Require that these two parameters are always specified together (on windows). Which is annoying. I vote for #2. It's the least inconsistent --- we don't pay attention to the registry for much of anything else, do we? In practice I think people who were setting either would set both, so it's not worth a huge amount of effort to have an unsurprising behavior when only one is set. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] hstore == and deprecate =
All, I don't much like hstore(hstore, text[]) because it's not strictly a constructor. But I could certainly live with something based on the word slice. The existing SQL function backing the operator is called slice_hstore(), whereas I would probably prefer hstore_slice() or just slice(), but I can't talk about it right now because I have to go finish laundering the paint out of my entire wardrobe. Having already written three patches to rename this operator (to three different names), I'm in no hurry to write a fourth unless the degree of consensus is sufficient to convince me I shan't need to write a fifth one. While I would personally prefer to have an operator for the slicing opeeration, I'm not willing to spend time arguing about it. So, +1 to implement the subset operation as the function slice(), and defer having an operator until later. In some ways, it makes more sense to talk about additional operators in the context of also adding them to intarray, and I don't want to go there yet. -- -- Josh Berkus PostgreSQL Experts Inc. http://www.pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Keepalives win32
On Mon, Jun 28, 2010 at 20:45, Tom Lane t...@sss.pgh.pa.us wrote: Magnus Hagander mag...@hagander.net writes: [ can't read system's keepalive values in windows ] The way I see it, we have two options: 1) Read the default value from the registry. That's some fairly ugly code, imho. 2) Ignore the registry value and use the default value of 2 hours/1 second. That will override any changes the user made in the registry, which seems pretty ugly. 3) Require that these two parameters are always specified together (on windows). Which is annoying. I vote for #2. It's the least inconsistent --- we don't pay attention to the registry for much of anything else, do we? Directly, no? Indirectly, we do. For every other TCP parameter (because the registry controls what we'll get as the default when we just use things) In practice I think people who were setting either would set both, so it's not worth a huge amount of effort to have an unsurprising behavior when only one is set. There's unsurprising, and downright hostile (the way we get by default is if you don't set keepalive_time, it'll spew keepalive packages continuously, which is certainly not good). In tha tcase, it's probably better to throw an error (which would be trivial to do, of course) -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] get_whatever_oid, part 1: object types with unqualifed names
On Mon, Jun 28, 2010 at 1:00 PM, Tom Lane t...@sss.pgh.pa.us wrote: Another, and related idea that I had while looking at this is that a lot of object types could benefit from a get_whatever_heaptuple() function with the same calling syntax. get_whatever_oid() could be restructured to use it, and most object types would have other callers, also. But that too seems like opening a larger can of worms than I really want to get into at this point. This is the sort of thing that I think we should get right the first time, rather than have multiple waves of large-scale changes. After taking a walk around the block, I have two further thoughts about this: 1. I wouldn't have submitted these patches (this one and the part two patch) unless it reflected my best judgment about how far it makes sense to proceed with refactoring in a certain direction. I'm willing to be second-guessed, if you or someone else wants to move the goal-posts. But I don't personally feel that there's enough bang for the buck in going further in this direction, or I would have done it already. I'm not planning to commit these patches and then start immediately working on a second set of patches that touches all of these same spots in the code over again. In the particular area that this patch touches (mapping names as gathered by the parser to OIDs), there are many different permutations: the input can be a cstring, or a text datum, or a list of cstrings, etc.; and the output can be a boolean (is it there?), an OID, a heap tuple, etc. If you try to cover every combination, especially for the more obscure object types, you'll drive yourself nuts; on the other hand, trying to regularize the more common cases is, I think, helpful and worthwhile. So it's a trade-off; I took my best crack at it. 2. It might be too optimistic to think that we're going to avoid having large-scale code changes in 9.1 by committing these to 9.0. I think refactoring is a fact of life as we try to move the project forward, and while we want to be careful about how we do it for the reasons you mention, it's also important if we want to have a clean base to build on for future features (which, in fact, is why I proposed these patches in the first place - I discovered that this code wasn't too clean right now while thinking about SE-PostgreSQL security labels at PGCon). I have at least one other patch that's basically just refactoring in the queue for 9.1 already, which is fairly wide-ranging but in an area completely unrelated to these patches, and a couple other less ambitious ones that I plan to work on as time permits; so I think that the need for these kinds of changes is not going to go away. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Propose Beta3 for July
Why not do prep work with a release on the 5th? On Mon, 28 Jun 2010, Tom Lane wrote: Josh Berkus j...@agliodbs.com writes: Therefore, I propose that we set a beta3 release date for July 8th. That should give it enough space from the American Holiday. You mean wrap on Thursday the 8th for release on Monday the 12th? That'd be fine with me. Actual release on the 8th would mean asking people to do release prep work when they should be out watching fireworks. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers Marc G. FournierHub.Org Hosting Solutions S.A. scra...@hub.org http://www.hub.org Yahoo:yscrappySkype: hub.orgICQ:7615664MSN:scra...@hub.org -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Propose Beta3 for July
On 6/28/10 11:52 AM, Marc G. Fournier wrote: Why not do prep work with a release on the 5th? I think that a bunch of the people needed for wraps are Americans. No? -- -- Josh Berkus PostgreSQL Experts Inc. http://www.pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Keepalive for max_standby_delay
On Mon, Jun 28, 2010 at 2:26 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: ... It is even more unreasonable to commit to providing a timely patch (twice) and then fail to do so. We are trying to finalize a release here, and you've made it clear you think this code needs revision before then. I respect your opinion, but not your right to make the project release timetable dependent on your own schedule, and not your right to shut other people out of working on the issues you've raised. Since nobody has put forward a proposed beta3 release date, I don't feel that I'm holding anything up. In the meantime, I have many responsibilities and am facing Red Hat internal deadlines. See here, last paragraph: http://archives.postgresql.org/pgsql-hackers/2010-06/msg01093.php On a related note, this list is getting pretty darn short: http://wiki.postgresql.org/wiki/PostgreSQL_9.0_Open_Items ...partly because, in my desire to get another beta out, I have been devoting a lot of time to clearing it out. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Propose Beta3 for July
Josh Berkus j...@agliodbs.com writes: On 6/28/10 11:52 AM, Marc G. Fournier wrote: Why not do prep work with a release on the 5th? I think that a bunch of the people needed for wraps are Americans. No? In any case, you won't get much attention for an announcement on the 5th. Better to wait a week. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Propose Beta3 for July
On Mon, 28 Jun 2010, Josh Berkus wrote: On 6/28/10 11:52 AM, Marc G. Fournier wrote: Why not do prep work with a release on the 5th? I think that a bunch of the people needed for wraps are Americans. No? I'm not sure of all our nationalities .. I'm in Canada, Dave is in EU ... I *think* the FreeBSD ports maintainer is also in the EU ... Devrim is ... ? Marc G. FournierHub.Org Hosting Solutions S.A. scra...@hub.org http://www.hub.org Yahoo:yscrappySkype: hub.orgICQ:7615664MSN:scra...@hub.org -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Propose Beta3 for July
On Mon, Jun 28, 2010 at 2:56 PM, Tom Lane t...@sss.pgh.pa.us wrote: Josh Berkus j...@agliodbs.com writes: On 6/28/10 11:52 AM, Marc G. Fournier wrote: Why not do prep work with a release on the 5th? I think that a bunch of the people needed for wraps are Americans. No? In any case, you won't get much attention for an announcement on the 5th. Better to wait a week. At PGCon, the plan we discussed was to branch the tree on 7/1 and start the next CommitFest on 7/15. If beta3 is going to happen relatively close to the beginning of July, it might make sense to create the new branch just after we wrap beta3. But if we're not going to wrap beta3 until mid-July, then we need another plan. But I *really* don't want to wait that long. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Keepalives win32
Magnus Hagander mag...@hagander.net writes: On Mon, Jun 28, 2010 at 20:45, Tom Lane t...@sss.pgh.pa.us wrote: I vote for #2. It's the least inconsistent --- we don't pay attention to the registry for much of anything else, do we? Directly, no? Indirectly, we do. For every other TCP parameter (because the registry controls what we'll get as the default when we just use things) Not if we make the code use the RFC values as the defaults. I'm envisioning the GUC assign hooks doing something like #ifdef WIN32 if (newval == 0) newval = RFC-specified-default; #endif so that the main GUC logic can still think that zero means use the default. We're just redefining where the default comes from. This would be a change from previous behavior, but so what? Implementing any functionality at all here is a change from previous behavior on Windows. I don't have the slightest problem with saying as of 9.0, set these values via postgresql.conf, not the registry. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Propose Beta3 for July
Robert Haas robertmh...@gmail.com writes: At PGCon, the plan we discussed was to branch the tree on 7/1 and start the next CommitFest on 7/15. If beta3 is going to happen relatively close to the beginning of July, it might make sense to create the new branch just after we wrap beta3. Seems reasonable. The 7/1 date was never set in stone anyway. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] keepalives on MacOS X
It looks like the recent keepalives patch won't support MacOS X, because MacOS X does not have the TCP_KEEPIDLE, TCP_KEEPINTVL, and TCP_KEEPCNT socket parameters. It does have this: #define TCP_KEEPALIVE 0x10/* idle time used when SO_KEEPALIVE is enabled */ Should we try to support that as a synonym for TCP_KEEPIDLE, if that's what it is? Or not worry about it? Or... what? -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Keepalives win32
On Mon, Jun 28, 2010 at 21:03, Andrew Chernow a...@esilo.com wrote: The way I see it, we have two options: 1) Read the default value from the registry. That's some fairly ugly code, imho. It seems faily simple to yank these values out, no? Even easier if you use the all-in-wonder shell function SHGetValue(). We don't want to use that function, because it brings in a bunch of extra dependencies. This makes libpq.dll more heavyweight and more importantly, decreases the number of parallell connections we can deal with on the server side (on win32 at least, not sure about win64). HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\Tcpip\Parameters Values: KeepAliveTime, KeepAliveInterval Type: DWORD The only annoying thing is that the values may not exist. Well, it is also Right, we'd need an fallback in case they don't exist as well. rather annoying there is no way to set the counter. Yeah, but that's at least well documented how it behaves. In fact, there used to be a way to set that (via registry key), but they removed it in Vista. -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Keepalives win32
On Mon, Jun 28, 2010 at 21:03, Tom Lane t...@sss.pgh.pa.us wrote: Magnus Hagander mag...@hagander.net writes: On Mon, Jun 28, 2010 at 20:45, Tom Lane t...@sss.pgh.pa.us wrote: I vote for #2. It's the least inconsistent --- we don't pay attention to the registry for much of anything else, do we? Directly, no? Indirectly, we do. For every other TCP parameter (because the registry controls what we'll get as the default when we just use things) Not if we make the code use the RFC values as the defaults. I'm envisioning the GUC assign hooks doing something like #ifdef WIN32 if (newval == 0) newval = RFC-specified-default; #endif Right. (I've only looked at the libpq side so far) Also, we could avoid caling it *at all* if neither one of those parameters is set. That'll take a bit more code (using the unix-codepath of setsockopt() to enable keepalives at all), but it shouldn't amount to many lines.. so that the main GUC logic can still think that zero means use the default. We're just redefining where the default comes from. Yeah. This would be a change from previous behavior, but so what? Implementing any functionality at all here is a change from previous behavior on Windows. I don't have the slightest problem with saying as of 9.0, set these values via postgresql.conf, not the registry. Works for me. -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] hstore == and deprecate =
Josh Berkus j...@agliodbs.com writes: While I would personally prefer to have an operator for the slicing opeeration, I'm not willing to spend time arguing about it. So, +1 to implement the subset operation as the function slice(), and defer having an operator until later. Yeah, I think the consensus is to avoid picking an operator name at all. slice() is OK by me. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Keepalives win32
It seems faily simple to yank these values out, no? Even easier if you use the all-in-wonder shell function SHGetValue(). We don't want to use that function, because it brings in a bunch of extra dependencies. This makes libpq.dll more heavyweight and more importantly, decreases the number of parallell connections we can deal with on the server side (on win32 at least, not sure about win64). Oh, didn't know that. Are the standard reg functions, open/query/close really that bad? Can't be any worse than the security api or MAPI hell ;) -- Andrew Chernow eSilo, LLC every bit counts http://www.esilo.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Propose Beta3 for July
At PGCon, the plan we discussed was to branch the tree on 7/1 and start the next CommitFest on 7/15. If beta3 is going to happen relatively close to the beginning of July, it might make sense to create the new branch just after we wrap beta3. But if we're not going to wrap beta3 until mid-July, then we need another plan. But I *really* don't want to wait that long. Well, do you consider 7/8 to be mid-July or close to the beginning? -- -- Josh Berkus PostgreSQL Experts Inc. http://www.pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Admission Control
Jesper Krogh jes...@krogh.cc wrote: Sorry if I'm asking silly questions, but how does transactions and connection pooler's interact? That depends a great deal on the pooler and its configuration, as well as your client architecture. Our shop gathers up the information needed for our database transaction and submits it to a server application which has all the logic needed to use that data to apply the transaction. We determined long ago that it is a Very Bad Idea for us to have an open database transaction which is waiting for a user to do something before it can proceed. Say if you have 100 clients all doing fairly inactive database work in transactions lasting a couple of minutes at the same time. I'm not sure what you mean by that. If you mean that part of a database transaction hits the database, and then it takes a while for the rest of the statements for the transaction to be generated (based on network latency or waits for user input), then it is hard to see how you can effectively use a connection pool. I have yet to see an environment where I think that's a good approach, but I haven't seen everything yet. :-) On the other hand, if the issue is that 100 transactions are fired off at the same time and it takes two minutes for them all to finish, unless you have *a lot* of CPUs and spindles, that's not efficient use of your resources. A connection pool might indeed help with that; you might start getting transactions back in one second, and have them all done in a minute and a half if you used a properly configured connection pool. -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Propose Beta3 for July
On Mon, Jun 28, 2010 at 3:23 PM, Josh Berkus j...@agliodbs.com wrote: At PGCon, the plan we discussed was to branch the tree on 7/1 and start the next CommitFest on 7/15. If beta3 is going to happen relatively close to the beginning of July, it might make sense to create the new branch just after we wrap beta3. But if we're not going to wrap beta3 until mid-July, then we need another plan. But I *really* don't want to wait that long. Well, do you consider 7/8 to be mid-July or close to the beginning? 7/8 would be about the last date I'd consider reasonable, given a CF starting 7/15. I would prefer a few days earlier, but I'm not sure how much of a vote I get considering that I'm not actively involved in the release process or the packaging of installers. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Propose Beta3 for July
7/8 would be about the last date I'd consider reasonable, given a CF starting 7/15. I would prefer a few days earlier, but I'm not sure how much of a vote I get considering that I'm not actively involved in the release process or the packaging of installers. Well, I think realistically we could wrap earlier and release on the 7th or 8th. That just depends on being able to do the wrap with a 100% non-US (as in, no Bruce or Tom) team. I don't see that as impossible, but then I'm not involved in wrapping either. -- -- Josh Berkus PostgreSQL Experts Inc. http://www.pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Propose Beta3 for July
On Mon, Jun 28, 2010 at 3:32 PM, Josh Berkus j...@agliodbs.com wrote: 7/8 would be about the last date I'd consider reasonable, given a CF starting 7/15. I would prefer a few days earlier, but I'm not sure how much of a vote I get considering that I'm not actively involved in the release process or the packaging of installers. Well, I think realistically we could wrap earlier and release on the 7th or 8th. That just depends on being able to do the wrap with a 100% non-US (as in, no Bruce or Tom) team. I don't see that as impossible, but then I'm not involved in wrapping either. I don't know what all the steps are. I can certainly tag and bundle if that's helpful... my first try (alpha5) got the tags right and the bundling wrong, but I think I know how to do it. But before that we have to get all the fixes in, and after that someone needs to build installers and RPMs, distribute things through the mirror network, etc. and I don't know how all that works. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] simplifying emode_for_corrupt_record
On Fri, Jun 25, 2010 at 5:07 PM, Robert Haas robertmh...@gmail.com wrote: I spend a little bit of time analyzing this today and it appears to me that all of the calls to emode_for_corrupt_record() arrive via ReadRecord(), which itself takes an emode argument that is always passed by the caller as either LOG or PANIC. Therefore, the effect of the first if test in emode_for_corrupt_record() is to reduce the logging level of messages coming from SR or the archive from LOG to WARNING. (WARNING would be higher in an interactive session, but not here, per Tom's point.) This seems clearly a bad idea, so I propose to rip it out, which simplifies this function considerably. Proposed patch attached. Since this appears to be non-controversial, I'm going to go ahead and commit it. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Propose Beta3 for July
Robert Haas robertmh...@gmail.com writes: On Mon, Jun 28, 2010 at 3:23 PM, Josh Berkus j...@agliodbs.com wrote: Well, do you consider 7/8 to be mid-July or close to the beginning? 7/8 would be about the last date I'd consider reasonable, given a CF starting 7/15. I would prefer a few days earlier, but I'm not sure how much of a vote I get considering that I'm not actively involved in the release process or the packaging of installers. It would be better not to branch until we're certain we won't need to re-wrap the beta. I'm wondering why you see a need for any daylight at all between the branch date and the start of the CF. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Propose Beta3 for July
On Mon, Jun 28, 2010 at 3:44 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Mon, Jun 28, 2010 at 3:23 PM, Josh Berkus j...@agliodbs.com wrote: Well, do you consider 7/8 to be mid-July or close to the beginning? 7/8 would be about the last date I'd consider reasonable, given a CF starting 7/15. I would prefer a few days earlier, but I'm not sure how much of a vote I get considering that I'm not actively involved in the release process or the packaging of installers. It would be better not to branch until we're certain we won't need to re-wrap the beta. I'm wondering why you see a need for any daylight at all between the branch date and the start of the CF. Well, I guess it isn't technically necessary, but I think it might make things less hectic. Last year we had two weeks, which was more than we needed, but I think a little slack time will make things go more smoothly. BTW, if we're going to do a second pgindent run for this release, we probably want to do that before the branch point. Not sure if it should be done before or after beta3. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Keepalives win32
The way I see it, we have two options: 1) Read the default value from the registry. That's some fairly ugly code, imho. It seems faily simple to yank these values out, no? Even easier if you use the all-in-wonder shell function SHGetValue(). HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\Tcpip\Parameters Values: KeepAliveTime, KeepAliveInterval Type: DWORD The only annoying thing is that the values may not exist. Well, it is also rather annoying there is no way to set the counter. The API used is documented at: http://msdn.microsoft.com/en-us/library/dd877220(v=VS.85).aspx Patch as it looks now (libpq only, and with obvious problems with this issue): http://github.com/mhagander/postgres/compare/master...win32keepalive and here :) http://archives.postgresql.org/pgsql-hackers/2009-05/msg01099.php -- Andrew Chernow eSilo, LLC every bit counts http://www.esilo.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Propose Beta3 for July
Robert Haas robertmh...@gmail.com writes: On Mon, Jun 28, 2010 at 3:44 PM, Tom Lane t...@sss.pgh.pa.us wrote: It would be better not to branch until we're certain we won't need to re-wrap the beta. I'm wondering why you see a need for any daylight at all between the branch date and the start of the CF. Well, I guess it isn't technically necessary, but I think it might make things less hectic. Last year we had two weeks, which was more than we needed, but I think a little slack time will make things go more smoothly. I don't want to do it on the same day or anything like that, but I think a couple days slack is plenty. BTW, if we're going to do a second pgindent run for this release, we probably want to do that before the branch point. Not sure if it should be done before or after beta3. Yes, we must do that before branching. My preference would be to do it before the beta --- I still feel there's a nonzero chance of pgindent breaking things, so testing should happen on post-pgindent code. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Admission Control
On 2010-06-28 21:24, Kevin Grittner wrote: Jesper Kroghjes...@krogh.cc wrote: Sorry if I'm asking silly questions, but how does transactions and connection pooler's interact? That depends a great deal on the pooler and its configuration, as well as your client architecture. Our shop gathers up the information needed for our database transaction and submits it to a server application which has all the logic needed to use that data to apply the transaction. We determined long ago that it is a Very Bad Idea for us to have an open database transaction which is waiting for a user to do something before it can proceed. The situation is more: 1) Grab a bunch of data (using pg_try_advisory_lock() to lock out other processes from grabbing the same). 2) Process the data (in external software). 3) Push results back into the database, including a flag telling that the data has been processed. 4) Release advisory locks. Step 2 takes somewhere between a couple of seconds to a couple of minutes depending on the task to be done. It might not be optimal but it is extremely robust and simple to wrap 1 to 4 within a BEGIN / COMMIT block. On the application side is really nice not having to deal with partly processed data in the database, which I can get around with by just keeping the transaction open. From my POV, a connection pooler doesn't buy anything, and I cannot stop all processes from executing at the same time, allthough it most likely will not happen. There is no wait for user involved. And that means somewhere in the 100+ backends, allthough they are mostly idle, seen from a database perspective. I have not hit any issues with the work_mem being too high, but I'm absolutely sure that I could flood the system if they happened to be working at the same time. Jesper -- Jesper -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Keepalives win32
On Mon, Jun 28, 2010 at 21:10, Magnus Hagander mag...@hagander.net wrote: On Mon, Jun 28, 2010 at 21:03, Tom Lane t...@sss.pgh.pa.us wrote: Magnus Hagander mag...@hagander.net writes: On Mon, Jun 28, 2010 at 20:45, Tom Lane t...@sss.pgh.pa.us wrote: I vote for #2. It's the least inconsistent --- we don't pay attention to the registry for much of anything else, do we? Directly, no? Indirectly, we do. For every other TCP parameter (because the registry controls what we'll get as the default when we just use things) Not if we make the code use the RFC values as the defaults. I'm envisioning the GUC assign hooks doing something like #ifdef WIN32 if (newval == 0) newval = RFC-specified-default; #endif Right. (I've only looked at the libpq side so far) Also, we could avoid caling it *at all* if neither one of those parameters is set. That'll take a bit more code (using the unix-codepath of setsockopt() to enable keepalives at all), but it shouldn't amount to many lines.. Here's what I'm thinking, for the libpq side. Similar change on the server side. Seems ok? (still http://github.com/mhagander/postgres/compare/master...win32keepalive for those that prefer that interface) -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ libpq_keepalives_win32.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Keepalives win32
Magnus Hagander mag...@hagander.net writes: Here's what I'm thinking, for the libpq side. Similar change on the server side. Seems ok? I had in mind just legislating that the defaults are the RFC values, none of this try to use the registry values in one case business. I don't believe that you can make the server side act that way without much more ugliness than is warranted. Also, at least on the libpq side there is no backwards compatibility argument to be made, because we never turned on keepalives at all on that side in previous releases. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Keepalives win32
On Mon, Jun 28, 2010 at 22:39, Tom Lane t...@sss.pgh.pa.us wrote: Magnus Hagander mag...@hagander.net writes: Here's what I'm thinking, for the libpq side. Similar change on the server side. Seems ok? I had in mind just legislating that the defaults are the RFC values, none of this try to use the registry values in one case business. Um, if you look at that patch, it doesn't try to use the registry. It falls back directly to the system default, ignoring the registry. The only special case is where the user doesn't specify any of the parameters. I don't believe that you can make the server side act that way without much more ugliness than is warranted. Also, at least on the libpq side there is no backwards compatibility argument to be made, because we never turned on keepalives at all on that side in previous releases. -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Admission Control
Jesper Krogh jes...@krogh.cc wrote: I have not hit any issues with the work_mem being too high, but I'm absolutely sure that I could flood the system if they happened to be working at the same time. OK, now that I understand your workload, I agree that a connection pool at the transaction level won't do you much good. Something which limited the concurrent *query* count, or an execution admission controller based on resource usage, could save you from occasional random incidents of resource over-usage. -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] testing plpython3u on 9.0beta2
Josh Berkus wrote: You could argue it either way. The number of beta testers with plpython3 installations is probably very small, so I'm kinda leaning to just changing the code without a catversion bump. OTOH, if we want to encourage testing of pg_upgrade ... FWIW, the last bump has led to a lot of testing of pgupgrade. And fixes, that will appear in 9.0 beta3. :-) Most fixes were related to platform compile portability. -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com + None of us is going to be here forever. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Issue: Deprecation of the XML2 module 'xml_is_well_formed' function
We need to make extensive use of the 'xml_is_well_formed' function provided
by the XML2 module.
Yet the documentation says that the xml2 module will be deprecated since
XML syntax checking and XPath queries
is covered by the XML-related functionality based on the SQL/XML standard in
the core server from PostgreSQL 8.3 onwards.
However, the core function XMLPARSE does not provide equivalent
functionality since when it detects an invalid XML document,
it throws an error rather than returning a truth value (which is what we
need and currently have with the 'xml_is_well_formed' function).
For example:
select xml_is_well_formed('br/br2');
xml_is_well_formed
f
(1 row)
select XMLPARSE( DOCUMENT 'br/br2' );
ERROR: invalid XML document
DETAIL: Entity: line 1: parser error : expected ''
br/br2
^
Entity: line 1: parser error : Extra content at the end of the document
br/br2
^
Is there some way to use the new, core XML functionality to simply return a
truth value
in the way that we need?.
Thanks,
-- Mike Berrow
Re: [HACKERS] testing plpython3u on 9.0beta2
Bruce Momjian br...@momjian.us writes: Josh Berkus wrote: You could argue it either way. The number of beta testers with plpython3 installations is probably very small, so I'm kinda leaning to just changing the code without a catversion bump. OTOH, if we want to encourage testing of pg_upgrade ... FWIW, the last bump has led to a lot of testing of pgupgrade. And fixes, that will appear in 9.0 beta3. :-) Most fixes were related to platform compile portability. Well, if you think that pg_upgrade has changed materially since beta2, that would be a good argument for getting some fresh testing for it, which in turn argues for doing the catversion bump here. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] testing plpython3u on 9.0beta2
Tom Lane wrote: Bruce Momjian br...@momjian.us writes: Josh Berkus wrote: You could argue it either way. The number of beta testers with plpython3 installations is probably very small, so I'm kinda leaning to just changing the code without a catversion bump. OTOH, if we want to encourage testing of pg_upgrade ... FWIW, the last bump has led to a lot of testing of pgupgrade. And fixes, that will appear in 9.0 beta3. :-) Most fixes were related to platform compile portability. Well, if you think that pg_upgrade has changed materially since beta2, that would be a good argument for getting some fresh testing for it, which in turn argues for doing the catversion bump here. Attached are the changes since beta2; they are pretty minor. The good news is I think all reporters eventually got it working. I assume using it for beta3 would allow it work even better, and once you have use it once, using it again is simple. -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com + None of us is going to be here forever. + /check.c /dump.c /function.c /option.c /pg_upgrade.c /pg_upgrade.h /version.c /version_old_8_3.c momjian Have pg_upgrade create its output files in the current directory, rather than in a subdirectory of the $HOME directory, or $TMP in Windows. --- /pg_upgrade.h momjian Update pg_upgrade C comment about cwd. --- /pg_upgrade.h /relfilenode.c momjian Properly define pg_upgrade global variable, per bug report from Chris Ruprecht on Mac (64 bit). --- /option.c momjian Fix storage of getopt() return, should be 'int', for pg_upgrade. Steve Singer --- /tablespace.c momjian Fix pg_upgrade to remove malloc(0) call. --- /dump.c /option.c /pg_upgrade.h /server.c momjian Fix pg_upgrade's use of pg_ctl on Win32 to not send command and sever output to the same file, because it is impossible. Also set user name for pg_dumpall in pg_upgrade. --- /check.c /pg_upgrade.c /server.c momjian Add username designations to all pg_upgrade utility calls that support it. --- -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Admission Control
On 29/06/10 04:48, Tom Lane wrote: Ross J. Reedstromreeds...@rice.edu writes: Hmm, I'm suddenly struck by the idea of having a max_cost parameter, that refuses to run (or delays?) queries that have too high a cost. That's been suggested before, and shot down on the grounds that the planner's cost estimates are not trustworthy enough to rely on for purposes of outright-failing a query. If you didn't want random unexpected failures, you'd have to set the limit so much higher than your regular queries cost that it'd be pretty much useless. I wrote something along the lines of this for Greenplum (is probably still available in the Bizgres cvs). Yes, cost is not an ideal metric to use for bounding workload (but was perhaps better than nothing at all in the case it was intended for). One difficulty with looking at things from the statement cost point of view is that all the requisite locks are already taken by the time you have a plan - so if you delay execution, these are still held, so deadlock likelihood is increased (unless you release locks for waiters, and retry for them later - but possibly need to restart executor from scratch to cope with possible table or schema changes). Maybe it'd be all right if it were just used to delay launching the query a bit, but I'm not entirely sure I see the point of that. I recall handling this by having a configurable option to let these queries run if nothing else was. Clearly to have this option on you would have to be confident that no single query could bring the system down. Cheers Mark -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] testing plpython3u on 9.0beta2
On fre, 2010-06-25 at 18:57 -0400, Peter Eisentraut wrote: On fre, 2010-06-25 at 23:44 +0200, Andres Freund wrote: Has anybody actually researched if it is safe to run python2 and python3 in the same address space? You can't run plpython2 and plpython3 in the same session, because the libraries are loaded with dlopen(RTLD_GLOBAL) (with RTLD_LOCAL it would apparently work). But you can use them in different sessions on the same database, for example. By the way, I thought there had been some discussion about this in the past, but I couldn't find it ... Why are we using RTLD_GLOBAL? -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Keepalives win32
Magnus Hagander mag...@hagander.net writes: On Mon, Jun 28, 2010 at 22:39, Tom Lane t...@sss.pgh.pa.us wrote: I had in mind just legislating that the defaults are the RFC values, none of this try to use the registry values in one case business. Um, if you look at that patch, it doesn't try to use the registry. It falls back directly to the system default, ignoring the registry. The only special case is where the user doesn't specify any of the parameters. What I was trying to say is I think we could dispense with the setsockopt() code path, and just always use the WSAIoctl() path anytime keepalives are turned on. I don't know what system default values you're speaking of, if they're not the registry entries; and I definitely don't see the point of consulting such values if they aren't user-settable. We might as well just consult the RFCs and be done. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] testing plpython3u on 9.0beta2
Peter Eisentraut pete...@gmx.net writes: Why are we using RTLD_GLOBAL? Some guy named Eisentraut wanted it for plpython: http://archives.postgresql.org/pgsql-hackers/2001-05/msg00563.php http://archives.postgresql.org/pgsql-committers/2001-05/msg00283.php Possibly that no longer applies, though. In general it seems like our usage of loadable modules treats them all as independent objects, so that not using RTLD_GLOBAL would be a more sensible policy. If it won't break any of the existing PLs then I'm fine with removing it. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] get_whatever_oid, part 1: object types with unqualifed names
(2010/06/29 3:52), Robert Haas wrote: On Mon, Jun 28, 2010 at 1:00 PM, Tom Lanet...@sss.pgh.pa.us wrote: Another, and related idea that I had while looking at this is that a lot of object types could benefit from a get_whatever_heaptuple() function with the same calling syntax. get_whatever_oid() could be restructured to use it, and most object types would have other callers, also. But that too seems like opening a larger can of worms than I really want to get into at this point. This is the sort of thing that I think we should get right the first time, rather than have multiple waves of large-scale changes. After taking a walk around the block, I have two further thoughts about this: 1. I wouldn't have submitted these patches (this one and the part two patch) unless it reflected my best judgment about how far it makes sense to proceed with refactoring in a certain direction. I'm willing to be second-guessed, if you or someone else wants to move the goal-posts. But I don't personally feel that there's enough bang for the buck in going further in this direction, or I would have done it already. I'm not planning to commit these patches and then start immediately working on a second set of patches that touches all of these same spots in the code over again. In the particular area that this patch touches (mapping names as gathered by the parser to OIDs), there are many different permutations: the input can be a cstring, or a text datum, or a list of cstrings, etc.; and the output can be a boolean (is it there?), an OID, a heap tuple, etc. If you try to cover every combination, especially for the more obscure object types, you'll drive yourself nuts; on the other hand, trying to regularize the more common cases is, I think, helpful and worthwhile. So it's a trade-off; I took my best crack at it. Sorry, I couldn't understand some of idiomatic expressions, but you are saying that we have many variations of syscaches, so it is nonsense to wrap up all the usecases of them, but being worthful to focus on major usecases, such as name to oid translation, aren't you? If so, it also seems to me fair enough. It seems to me translations from name into oid, tuple or verifying object existence are major use cases, but we have no reason why replace all the SysCacheSearch(). 2. It might be too optimistic to think that we're going to avoid having large-scale code changes in 9.1 by committing these to 9.0. I think refactoring is a fact of life as we try to move the project forward, and while we want to be careful about how we do it for the reasons you mention, it's also important if we want to have a clean base to build on for future features (which, in fact, is why I proposed these patches in the first place - I discovered that this code wasn't too clean right now while thinking about SE-PostgreSQL security labels at PGCon). I have at least one other patch that's basically just refactoring in the queue for 9.1 already, which is fairly wide-ranging but in an area completely unrelated to these patches, and a couple other less ambitious ones that I plan to work on as time permits; so I think that the need for these kinds of changes is not going to go away. -- KaiGai Kohei kai...@ak.jp.nec.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] get_whatever_oid, part 1: object types with unqualifed names
2010/6/28 KaiGai Kohei kai...@ak.jp.nec.com: Sorry, I couldn't understand some of idiomatic expressions, but you are saying that we have many variations of syscaches, so it is nonsense to wrap up all the usecases of them, but being worthful to focus on major usecases, such as name to oid translation, aren't you? Yes. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Issue: Deprecation of the XML2 module 'xml_is_well_formed' function
On Mon, Jun 28, 2010 at 11:03 AM, Mike Berrow mber...@gmail.com wrote: Is there some way to use the new, core XML functionality to simply return a truth value in the way that we need?. Have you tried using a wrapper function like the one suggested independently by Mike Rylander and David Fetter upthread? If so, how'd it work out for you? -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Issue: Deprecation of the XML2 module 'xml_is_well_formed' function
Yes, I went ahead and tried the original suggestion.
Here is what the added function went in as:
CREATE OR REPLACE FUNCTION xml_is_ok(x text)
RETURNS boolean AS
$BODY$
BEGIN
PERFORM XMLPARSE( DOCUMENT x::XML );
RETURN TRUE;
EXCEPTION WHEN OTHERS THEN
RETURN FALSE;
END;
$BODY$
LANGUAGE 'plpgsql' VOLATILE
COST 100;
ALTER FUNCTION xml_is_ok(text) OWNER TO postgres;
It worked fine. Thanks Mike and David.
The only other issue is that when I benchmarked it on a 5,000 record data
set that I have,
the original XML2 function ('xml_is_well_formed') took about 9.5 seconds
and this (deprecation driven) replacement took about 17.2 seconds.
-- Mike Berrow
On Mon, Jun 28, 2010 at 7:51 PM, Robert Haas robertmh...@gmail.com wrote:
On Mon, Jun 28, 2010 at 11:03 AM, Mike Berrow mber...@gmail.com wrote:
Is there some way to use the new, core XML functionality to simply return
a
truth value
in the way that we need?.
Have you tried using a wrapper function like the one suggested
independently by Mike Rylander and David Fetter upthread? If so,
how'd it work out for you?
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise Postgres Company
Re: [HACKERS] keepalives on MacOS X
On Tue, Jun 29, 2010 at 4:07 AM, Robert Haas robertmh...@gmail.com wrote: It looks like the recent keepalives patch won't support MacOS X, because MacOS X does not have the TCP_KEEPIDLE, TCP_KEEPINTVL, and TCP_KEEPCNT socket parameters. It does have this: #define TCP_KEEPALIVE 0x10 /* idle time used when SO_KEEPALIVE is enabled */ Should we try to support that as a synonym for TCP_KEEPIDLE, if that's what it is? Or not worry about it? Or... what? I'm not sure that can be a synonym for TCP_KEEPIDLE, but if so we should change not only a client-side but also server-side. Regards, -- Fujii Masao NIPPON TELEGRAPH AND TELEPHONE CORPORATION NTT Open Source Software Center -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] keepalives on MacOS X
Robert Haas robertmh...@gmail.com writes: It looks like the recent keepalives patch won't support MacOS X, because MacOS X does not have the TCP_KEEPIDLE, TCP_KEEPINTVL, and TCP_KEEPCNT socket parameters. It does have this: It looks to me like there's support for setting KEEPIDLE and KEEPINTVL via sysctl, but of course that's system-wide and presumably requires root privilege to set. (Apple seems to have inherited that from various BSDen, btw; it's not unique to Darwin.) #define TCP_KEEPALIVE 0x10/* idle time used when SO_KEEPALIVE is enabled */ Should we try to support that as a synonym for TCP_KEEPIDLE, if that's what it is? Or not worry about it? Or... what? Yeah, a bit of rooting around in the Darwin sources shows that this value is used as a per-connection override for tcp_keepidle. So it is a synonym. Not sure if it's worth supporting when the other value can't be set too. Maybe it'd be more useful to document that people can set the system-wide values if they're desperate. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] suppress automatic recovery after back crash
On Mon, Jun 28, 2010 at 9:54 AM, Robert Haas robertmh...@gmail.com wrote: This seems useful to me so here's a patch to implement it. +1 This would be very useful for people who want to give a clusterware control of postgres. Regards, -- Fujii Masao NIPPON TELEGRAPH AND TELEPHONE CORPORATION NTT Open Source Software Center -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
