Re: [HACKERS] Patch: Implement failover on libpq connect level.
I've tried to deal with some of these problems.
My patch have support for following things:
1. Check whether database instance is in the recovery/standby mode and
try to find another one if so.
2. Let cluster management software to have some time to promote one of
the standbys to master. I.e. there can be failover timeout specified to
allow retry after some time if no working master found.
Really there is room for some improvements in handling of connect
timeout (which became much more important thing when ability to try
next host appears). Now it is handled only by blocking-mode connect
functions, not by async state machine. But I decided to publish patch
without these improvements to get feedback from community.
A bit of testing on this turns up a problem.
Consider a connection string that specifies two hosts and a read/write
connection:
postgresql://korry@127.0.0.1:5301,127.0.0.1:5300/edb?readonly=0
If the first host is a healthy standby (meaning that I can connect to it
but pg_is_in_recovery() returns 't'), the state machine will never move
on to the second host.
The problem seems to be in PQconnectPoll() in the case for
CONNECTION_AUTH_OK, specifically this code:
/* We can release the address list now. */
pg_freeaddrinfo_all(conn->addrlist_family, conn->addrlist);
conn->addrlist = NULL;
conn->addr_cur = NULL;
That frees up the list of alternative host addresses. The state machine
then progresses to CONNECTION_CHECK_RO (which invokes
pg_is_in_recovery()), then CONNECTION_CHECK_RW (waiting for the response
from the server). Since we just connected to a standby replica,
pg_is_in_recovery() returns 't' and the state changes to
CONNECTION_NEEDED. The next call to try_next_address() will fail to
find a next address because we freed the list in the case for
CONNECTION_AUTH_OK.
A related issue: when the above sequence occurs, no error message is
returned (because the case for CONNECTION_NEEDED thinks "an appropriate
error message is already set up").
In short, if you successfully connect to standby replica (and specify
readonly=0), the remaining hosts are ignored, even though one of those
hosts is a master.
And one comment about the code itself - in connectDBStart(), you've
added quite a bit of code to parse multiple hosts/ports. I would
recommend adding a comment that shows the expected format, and then
choosing better variable names (better than 'p', 'q', and 'r'); perhaps
the variable names could refer to components of the connection string
that you are parsing (like 'host', 'port', 'delimiter', ...). That
would make the code much easier to read/maintain.
Thanks.
-- Korry
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Patch: Implement failover on libpq connect level.
I've tried to deal with some of these problems. My patch have support for following things: 1. Check whether database instance is in the recovery/standby mode and try to find another one if so. 2. Let cluster management software to have some time to promote one of the standbys to master. I.e. there can be failover timeout specified to allow retry after some time if no working master found. Really there is room for some improvements in handling of connect timeout (which became much more important thing when ability to try next host appears). Now it is handled only by blocking-mode connect functions, not by async state machine. But I decided to publish patch without these improvements to get feedback from community. A bit of testing on this turns up a problem. Consider a connection string that specifies two hosts and a read/write connection: postgresql://korry@127.0.0.1:5301,127.0.0.1:5300/edb?readonly=0 If the first host is a healthy standby (meaning that I can connect to it but pg_is_in_recovery() returns 't'), the state machine will never move on to the second host. The problem seems to be in PQconnectPoll() in the case for CONNECTION_AUTH_OK, specifically this code: /* We can release the address list now. */ pg_freeaddrinfo_all(conn->addrlist_family, conn->addrlist); conn->addrlist = NULL; conn->addr_cur = NULL; That frees up the list of alternative host addresses. The state machine then progresses to CONNECTION_CHECK_RO (which invokes pg_is_in_recovery()), then CONNECTION_CHECK_RW (waiting for the response from the server). Since we just connected to a standby replica, pg_is_in_recovery() returns 't' and the state changes to CONNECTION_NEEDED. The next call to try_next_address() will fail to find a next address because we freed the list in the case for CONNECTION_AUTH_OK. A related issue: when the above sequence occurs, no error message is returned (because the case for CONNECTION_NEEDED thinks "an appropriate error message is already set up"). In short, if you successfully connect to standby replica (and specify readonly=0), the remaining hosts are ignored, even though one of those hosts is a master. A follow-up - the conn->addrlist is also freed when the case for CONNECTION_CHECK_RW decides that conn->status != CONNECTION_OK and calls closePGConn(). -- Korry -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Patch (2): Implement failover on libpq connect level.
On 2015.10.14 at 13:41:51 +0300, Victor Wagner wrote: Attached patch which implements client library failover and loadbalancing as was described in the proposal <20150818041850.ga5...@wagner.pp.ru>. I'm sending imporoved verison of patch. As Olexander Shulgin noted, previous version of patch lacks support for service files. Now support for service files is implemented and multiple host statements in the service file are allowed. A couple of minor nits: When you call pg_is_in_recovery(), you should schema-qualify the function name, just in case some other version of that function exists in the search_path. Also, pg_is_in_recovery() returns a boolean value - PQgetvalue() will not return "true" or "false", it will return "t" or "f". And, you have a bit of garbage in the patch (the patch inserts UNIXSOCK_PATH(portstr, portnum, conn->pgunixsocket); in the header comment at the top of fe-connect.c). -- Korry -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Why does TRIM() expect an expr_list?
In gram.y, the productions for the TRIM() expression expect an
argument of trim_list:
TRIM '(' trim_list ')'
TRIM '(' TRAILING trim_list ')'
TRIM '(' LEADING trim_list ')'
TRIM '(' BOTH trim_list ')'
And trim_list is defined as:
trim_list:a_expr FROM expr_list { $$ = lappend($3, $1); }
| FROM expr_list{ $$ = $2; }
| expr_list { $$ = $1; }
But it seems wrong for trim_list to be defined in terms of
expr_list's. The way it's currently written, we allow expressions
such as:
TRIM( 'foo', now(), 4+2)
or
TRIM( LEADING FROM 'foo', 4+2)
The parser translates the TRIM expression into a call to btrim() (or
ltrim() or rtrim()) and we seem to (accidentally) make up a silly
argument list if the user includes an actual expr_list (with multiple
expressions).
The first example results in function ltrim(unknown, timestamp with
time zone, integer) does not exist.
The second example above is translated to ltrim(4+2, 'foo').
It seems to me that trim_list should defined as:
trim_list:a_expr FROM a_expr { $$ = list_make2($3, $1); }
| FROM a_expr { $$ = list_make1($2); }
| a_expr{ $$ =
list_make1($1); }
Am I missing something?
-- Korry
---
Korry Douglas
Senior Database Dude
EnterpriseDB Corporation
The Enterprise Postgres Company
Phone: (804)241-4301
Mobile: (620) EDB-NERD
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Why does TRIM() expect an expr_list?
It seems to me that trim_list should defined as:
trim_list:a_expr FROM a_expr { $$ = list_make2($3, $1); }
| FROM a_expr { $$ = list_make1($2); }
| a_expr{ $$ =
list_make1($1); }
Am I missing something?
That will break the ability to call trim() with ordinary function
syntax.
We possibly could change that in conjunction with adding a straight
TRIM '(' expr_list ')' production, though.
Hmm... it seems counterintuitive to call TRIM() using ordinary
function syntax anyway. What would the argument list look like?
I think you would have to reverse the order of the arguments (and
there's no way to factor the LEADING/TRAILING/BOTH stuff into the
argument list since those map to calls to different functions).
For example to write:
TRIM( 'foo' FROM 'foobar' )
using function syntax, you would have to write:
TRIM( 'foobar', 'foo' )
As far as I know, that usage is not documented anywhere. And since
trim is not really a function, you can't discover the proper
argument list using \df
On the other hand, someone is surely (ab)using TRIM() that way...
What this looks like to me is somebody was trying to allow for future
extensions in the keyword-ized syntax, but I can't imagine the SQL
committee introducing a mix of keyword-ized and non-keyword-ized
arguments. So I agree that the expr_list cases are pretty silly
except for the bare no-keyword-anywhere path.
I suspect that it was simply easier to write it that way than to code
the make_list1() invocations, but who knows.
Actually, on closer examination I think there's another bug here.
I see this in SQL99:
trim function ::=
TRIM left paren trim operands right paren
trim operands ::=
[ [ trim specification ] [ trim character ] FROM ]
trim source
trim specification ::=
LEADING
| TRAILING
| BOTH
trim character ::= character value expression
trim source ::= character value expression
It looks to me like you're not supposed to be able to omit FROM if
you've written a trim specification. Should we tighten our
syntax to reject that?
That depends on how much code you want to break. Doesn't really
matter to me.
-- Korry
---
Korry Douglas
Senior Database Dude
EnterpriseDB Corporation
The Enterprise Postgres Company
Phone: (804)241-4301
Mobile: (620) EDB-NERD
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [ADMIN] License Issue
I am part of an organization that is beginning to write commercial applications for various Portals (Liferay, etc.). We would like to be able to deploy PostGresQL as the lightweight database for the Portal to connect to. Our request for approval to our Legal Department has stalled due to the following language in one of the files: Seems like the easy solution is to rip out the AIX files in your server deployments ... or are you actually intending to support AIX? AIX itself now offers the functions found in src/backend/port/ dynloader/aix.c so I think that file may be obsolete as of (at least AIX 5.3). aix.c was required long ago because AIX did not offer the popular dlopen(), dlsym(), dlclose(), and dlerror() functions - you had to write them yourself. -- Korry -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [PATCH] Cleanup of GUC units code
Settings in postgresql.conf are currently case-insensitive. Except for the units. And, of course, filenames when you are using a case-sensitive filesystem. Because these are things that are defined by some convention other than the ones the PGDG made up. Since units fall into that category, it seems to me that we're stuck with using external conventions. Just a thought... since there are very few (if any) places where a user would specify a variable in terms of bits (kilobits, megabits, gigabits), we could make the units case-insensitive and assume that kb, gb, and mb (and all case variants) refer to some number of bytes. If a user wants to specify a variable in terms of bits, he would have to spell out the units completely, as in 4 gigabits, 20 kilobits, or 16 megabits. -- Korry -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] code coverage patch
The problem with those files is that the source file contains lines like this: #line 1042 y.tab.c but that source file does not exist, as it is renamed to gram.c. We could fix that in one of two ways: 1) Use bison's -o option to put the output file in the right place directly, if we are dealing with bison (and don't bother to support code coverage analysis with other yaccs), or 2) Run a pattern replacement across the grammar output files as their are renamed. Why not use the %output directive in the grammar file instead; that way you don't need to add any special flags to the Makefile. -- Korry -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Parsing of pg_hba.conf and authenticationinconsistencies
On Aug 5, 2008, at 4:07 PM, Simon Riggs wrote: On Sun, 2008-08-03 at 10:36 +0200, Magnus Hagander wrote: Tom Lane wrote: Magnus Hagander [EMAIL PROTECTED] writes: The good way to solve this would be to have independant command line utilities which check pg_hba.conf, pg_ident.conf and postgresql.conf for errors. Then DBAs could run a check *before* restarting the server. While clearly useful, it'd still leave the fairly large foot-gun that is editing the hba file and HUPing things which can leave you with a completely un-connectable database because of a small typo. That will *always* be possible, just because software is finite and human foolishness is not ;-). Certainly - been bitten by that more than once. But we can make it harder or easier to make the mistakes.. Yeah. I'm sure we've all done it. Would it be possible to have two config files? An old and a new? That way we could specify new file, but if an error is found we revert to the last known-good file? That would encourage the best practice of take-a-copy-then-edit. Perhaps the --check-config option should take an (optional) file name? That would allow you to validate a config file without having to copy it into place first. postgres --check-config=myFilenameGoesHere -D $PGDATA -- Korry -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Debugging PostgreSQL with GDB
I have problems on how to debug PostgreSQL on Linux using GDB. I made some changes to src/backend/utils/sort/tuplesort.c but it hangs while performing run formation (where my changes are). I configured it using ./configure --prefix=/usr/local/pgsql/8.3/ CFLAGS=-O0 -DTRACE_SORT --enable-debug --enable-cassert --enable-depend and trying to debug it using 'gdb postmaster' (which revealed to be not a very good idea) and 'gdb pg_ctl' followed by 'run -D /usr/local/psql/data' This last choice allowed me to set a breackpoint on puttuple_common (contained into tuplesort.c) but then I'm unable to run 'psql'. http://pastebin.com/m6a97b4dd Run psql, find the process ID of the backend (server) process by executing the command SELECT * FROM pg_backend_pid();, then attach to that process with gdb and set a breakpoint in your new code; finally, go back to your psql session and execute a command that exercises your code. -- Korry -- Korry Douglas [EMAIL PROTECTED] EnterpriseDBhttp://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [GENERAL] Connection to PostgreSQL Using Certificate: Wrong Permissions on Private Key File
Now libpq doesn't have any provision for DETAIL or HINT in its
locally-generated messages at the moment, so we can't just duplicate
the backend message, but we could do something like this example
from elsewhere in libpq:
if (stat_buf.st_mode (S_IRWXG | S_IRWXO))
{
fprintf(stderr,
libpq_gettext(WARNING: password file \%s\ has world or group read
access; permission should be u=rw (0600)\n),
pgpassfile);
return NULL;
}
Hmmm... I'm not crazy about libpq printing error messages to stderr.
The client application can't intercept those messages. And those
messages will often get lost - many client applications don't have
useful stderr streams (think GUI application on Win32).
-- Korry
--
Korry Douglas [EMAIL PROTECTED]
EnterpriseDBhttp://www.enterprisedb.com
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Cast as compound type
David Fetter wrote: I'd like to take a whack at making set-returning functions returning SETOF RECORD a little more fun to use. Let's imagine that we have a table foo and a function returning SETOF RECORD that can return foos. The call might look something like: SELECT a, b, c FROM f(ROW OF foo) WHERE ...; This would make it much easier and less error-prone to use SETOF RECORD. David, it sounds like you really want to declare the return type of the function? In your above example, you want to say that, in this particular invocation, function f() returns a SETOF foo's. Is that correct? If you were to create function that returns a RECORD (not a SETOF RECORD), you would call it like this: SELECT * FROM f() AS (column1 type1, column2 type2, column3 type3); In your case, I think you want to declare the return type using an explicitly defined composite type (possibly a table row); which would imply syntax such as: SELECT * FROM f() AS (foo); or SELECT * FROM f() AS (foo.*); So, it seems like you want the syntax to look more like: SELECT a,b,c, FROM f() AS (SETOF foo); Does that make sense to you? Your original syntax implied that the ROW OF foo was somehow related to the function arguments. -- Korry -- Korry Douglas [EMAIL PROTECTED] EnterpriseDBhttp://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] MemoryContextSwitchTo() confusion
Everything seems to work fine, except on the final call it has no more data to return so cleans up all it's internal data structures and returns with SRF_RETURN_DONE(funcctx). If you are doing your development work on a Linux host, you may find it useful to run the postmaster through valgrind (http://en.wikipedia.org/wiki/Valgrind). valgrind can spot many (but not all) memory-related problems and makes it easier to track them to their source. Be aware that valgrind's memory leak features may be pretty useless with PostgreSQL. To run the server (and all backend processes) under valgrind: $ valgrind --trace-children=yes --log-file=/tmp/pg.vg pg_ctl start After that, you'll see one /tmp/pg.vg.processID file for each process spawned by pg_ctl/postmaster. Use SELECT * FROM pg_backend_pid() to find the processID of the backend where you are testing your extensions and then look through the valgrind output file for that process. -- Korry -- Korry Douglas [EMAIL PROTECTED] EnterpriseDBhttp://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Units in postgresql.conf
Time units is easy: 1h = 60min = 3600s = 360ms We don't need anything larger than seconds at the moment. Except for log_rotation_age perhaps? -- Korry
Re: [HACKERS] [PATCHES] Generic Monitoring Framework with DTrace
On Fri, Jul 21, 2006 at 01:42:26PM +0200, Peter Eisentraut wrote: I would prefer to drop the PG_ prefixes on PG_TRACE and pg_trace.h. We know which software we're dealing with. I don't know. "trace" is a fairly generic word, how do you know that none of the dozen other libraries we include don't already have a "trace.h" or a TRACE() macro? On any of our supported platforms? I concur with Martijn. We've already regretted using ERROR as a macro name, let's not make the same mistake with TRACE. PG_TRACE is good, and so is pg_trace.h. (But invoking it as utils/trace.h would be ok.) How about the obvious DTRACE( ) or some similar variant? -- Korry
Re: [HACKERS] Loading the PL/pgSQL debugger (and other plugins)
Sorry to poke - but I'd like to get a patch submitted next week. Any
more comments? Thanks.
-- Korry
Thanks
for the quick feedback.
1) I think the most straightforward way to load an instrumentation
plugin is to define a new custom GUC variable (using the
custom_variable_classes mechanism).
This seems a bit messy and special-purpose.
Agreed, I'm not crazy about using a custom_variable_class variable
either.
I see no good reason to tie
it to plpgsql; we'll just need another one for every other language.
Hmmm... but the plugins themselves would be language-specific. I can't
imagine that a plugin (say a profiler) for PL/python would work for
PL/pgSQL. It seems to me that, even if we come up with a common
mechanism, we'll still need a separate GUC variable *name* for each
PL. Or am I not understanding something? Can you post an example of
what you are thinking (what would such a GUC variable look like)?
IMHO what we want is something with similar properties to preload_libraries,
but processed on a per-backend basis instead of once at postmaster start.
(You could almost just tell people to select the plugin they want by
LOADing it, but that is hard to use if you're trying to debug a
non-interactive application. A GUC variable can be set for an app
without much cooperation from the app.)
Agreed.
When the plugin's shared library gets loaded, one way or the other,
it should construct the function-pointer struct and then pass it to a
function defined by plpgsql (this lets us hide/postpone the decision
about whether there can be more than one active plugin).
But there's a timing issue there. If you ask the plugin to call a
call-handler function, then you can't load the plugin at backend
startup because the PL/pgSQL call-handler isn't loaded until it's
required. Since both the plugin and the call-handler are dynamically
loaded, I think one of them has to load the other. We already have a
mechanism for loading call-handlers on demand - it seems kind of messy
to introduce another mechanism for loading plugins (that in turn load
the call-handlers).
The PL/pgSQL call-handler has a convenient initialization function that
could read the GUC variable and load the referenced plugin (that's what
I'm doing right now).
What I'm thinking is that the plpgsql_init() function would look
something like this (my changes in red);
PLpgSQL_plugin pluginHooks;
typedef void (*plugin_loader_func)(PLpgSQL_plugin *hooks);
void
plpgsql_init(void)
{
static char * pluginName;
plugin_load_func plugin_loader();
/* Do initialization only once */
if (!plpgsql_firstcall)
return;
plpgsql_HashTableInit();
RegisterXactCallback(plpgsql_xact_cb, NULL);
plpgsql_firstcall = false;
/* Load any instrumentation plugins */
DefineCustomStringVariable( "plpgsql.plugin",
"Name of instrumentation plugin to use
when PL/pgSQL function is invoked",
NULL,
pluginName,
PGC_USERSET,
NULL,
NULL );
EmitWarningsOnPlaceholders("plpgsql");
if (pluginName )
{
plugin_loader = (plugin_loader_func
*)load_external_function(pluginName, "plugin_loader", false, NULL );
if (plugin_loader)
(*plugin_loader)(pluginHooks);
}
}
(Ignore the custom variable stuff for now)
Each plugin would export a plugin_loader() function - that function,
given a pointer to a PLpgSQL_plugin structure, would fill in that
structure with the required function pointers.
One issue that needs to be thought about with either this proposal or
your original is what permissions are needed to set the GUC variable.
I don't think we dare allow non-superusers to specify LOADing of
arbitrary shared libraries, so there has to be some filter function.
Perhaps a better way is that the GUC variable specifies a (list of)
initialization functions to call at backend start, and then the
superuserness is involved with installing the init functions into
pg_proc, and the GUC variable itself needs no special permissions.
Again, a plugin's init function would just register its function-pointer
struct with plpgsql.
You're right, privileges are an issue. Is it safe enough if we force
all plugins to reside in $libdir? Each plugin could enforce additional
security as needed that way, but you'd have to hold enough privileges
to get your plugin into $libdir to begin with so you can't write your
own nasty plugin to gain more privileges than you ought to have.
We should also think about a deregistration function. This would allow
you to turn debugging on and off within an interactive session. The
GUC variable is really only for coercing non-interactive applications
into being debuggable --- I don't see it as being important for
interactive debugging, as compared to just "select plugin_init();" ...
Ok.
3) Any comments on the PLpgSQL_plugin st
Re: [HACKERS] Loading the PL/pgSQL debugger (and other plugins)
I'm unqualified to comment on the server side design, but I was wondering if there was consensus on how the client interface to the debugger would work. From previous threads I saw DBGP mentioned (http://xdebug.org/docs-dbgp.php), but I don't recall seeing any final commitment to it. The patch that I'll be submitting for 8.2 will implement a way to instrument PL/pgSQL (and that idea can be extended to other PL languages). 'Instrumentation' can mean different things - it may be a debugger, a profiler, a coverage analyzer, a tracer, ... EnterpriseDB has developed a few plugins that we'll be contributing soon (a debugger, a profiler, and a tracer). The debugger is by far the largest plugin that we've developed and we implemented it before we had the idea to use a modular architecture (we're still in the process of converting the debugger to modular form, at the moment it's pretty heavily integrated into the PL/pgSQL interpreter). As soon as we get a patch in for the plugin architecture, we'll open-source at least one or two of the plugins so others can use them and/or write more (the debugger will take a little longer). That means that we (i.e. the community) haven't made a firm commitment to the debugger client protocol. I can tell you a little about the protocol that we are currently using, but it may change by the time we're ready to open-source the debugger. I gave a presentation at the anniversary summit that described the overall architecture and also showed the client/server protocol - the slides and audio should be available at the conference web site real soon now. The most important part, from your perspective (assuming that you might want to add a debugger to pgEdit), is the method that a debugger client application uses to interact with the debugger server. That's done through a collection of server-side functions that you can call from any libpq application. For example, to set a breakpoint, you would: SELECT * FROM pldbg_set_breakpoint( sessionHandle, functionOID, lineNumber, processID ); to step/over: SELECT * FROM pldbg_step_over( sessionHandle ); to step/into: SELECT * FROM pldbg_step_into( sessionHandle ); to get a copy of all local variables: SELECT * FROM pldbg_get_variables( sessionHandle, stackFrame ); and so on. There are a few functions that you can call to attach your debugger client to a target server and to set global breakpoints. I'll be posting more information as we get closer to releasing this stuff. -- Korry ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Loading the PL/pgSQL debugger (and other plugins)
When the plugin's shared library gets loaded, one way or the other, it should construct the function-pointer struct and then pass it to a function defined by plpgsql (this lets us hide/postpone the decision about whether there can be more than one active plugin). But there's a timing issue there. If you ask the plugin to call a call-handler function, then you can't load the plugin at backend startup because the PL/pgSQL call-handler isn't loaded until it's required. Since both the plugin and the call-handler are dynamically loaded, I think one of them has to load the other. Right, but if you set up the mechanism such that each individual PL is responsible for loading plugins, then we'll have to duplicate all that code each time we instrument another PL. I want to do as much as possible of the work in the core code so that we don't end up with duplicate code to maintain. I think I'm missing something important here. At minimum, you need a way to identify a plugin (or a list of plugins), and, if we generalize the mechanism, a way to identify the language that that plugin is associated with (like, this profiler works with PL/tcl, this debugger works with PL/Java, ...). Once you have that, you've got two choices: 1) The plugin loads the language or 2) The language loads the plugin You are suggesting option 1. That means that we must: a) come up with a way to identify the set of plugins desired (probably some GUC variables?) b) Extend the pg_language structure c) Extend the CREATE LANGUAGE statement d) come up with a way for the backend to load the plugins (the backend already knows how to load a language-handler) e) add loader code to each plugin (there should be more plugins than languages eventually) f) add loader code to each language (at least each language that wants to support a plugin) On the other hand, if the language loads the plugin, we must: a) come up with a way to identify the set of plugins desired (probably some GUC variables?) b) add loader code to each plugin c) add loader code to each language (that wants to support a plugin) In either case, the loader code in the language-handlers and the loader code in the plugins could be simple calls to common functions that are defined in the core, avoiding a lot of duplicate code. For example, each language handler (in it's initialization code) could include a call such as: pl_load_plugins( "pl/pgsql", functionPointers ); or pl_load_plugins( "pl/java", functionPointers ); pl_load_plugins() would reside in the core, it would find the list of plugins, load each one, find the plugin's initialization function, and call that function with functionPointers (the initializer would fill in the functionPointers structure). So what am I missing? What's the advantage to having the plugin load the language? To do it without a pg_language column, we'd need code in each plugin to identify the language shared library (by looking in pg_language), force loading of same (using existing fmgr code), and look up and call a plugin receptor function given an expected C-code name for it (again, most of this already exists in fmgr). It's not a huge amount of code, probably, but again duplicating it in each plugin seems unappealing. I suppose we could make fmgr export a general function to find a plugin receptor function given the PL name and the expected C symbol. That's what I was thinking too. But we could avoid hard-coded names using a syntax similar to preload_libraries (each entry in preload_libraries can contain the name of an optional initialization function). If you specify libraryName:functionName, we would assume that functionName was the loader function, if you just specify libraryName, we could look for a hard-coded default. (Oh, and any more comments on security? Is it enough to require that all plugins live in $libdir?) -- Korry
[HACKERS] Loading the PL/pgSQL debugger (and other plugins)
I'm working on a patch that implements the PL/pgSQL instrumentation
stuff (i.e. the PL/pgSQL debugger) that I discussed at the Anniversary
Summit and I need some opinions (this seems like a good place to look
for opinions :-)
A quick review: the PL/pgSQL debugger is designed as an optional
plugin that loads into the PL/pgSQL interpreter on-demand. You can
use the plugin idea to implement other kinds of instrumentation (I
demo'ed a tracer and a profiler at the conference, along with a
debugger). A plugin architecture greatly reduces the (source code)
footprint that would normally be required to implement a full-featured
debugger.
A plugin is basically a structure that contains a few function
pointers. If those function pointers are NULL, the PL/pgSQL interpreter
works exactly the way it does today. If any of those function pointers
are non-NULL, the PL/pgSQL interpreter calls the target function (which
points to a chunk of code inside of the plugin) and the plugin does
whatever it needs to do.
Right now, the plugin structure looks like this:
typedef struct
{
void (*init)( estate, func, error_callback, assign_expr, expr );
void (*func_beg)( PLpgSQL_execstate * estate, PLpgSQL_function * func );
void (*func_end)( PLpgSQL_execstate * estate, PLpgSQL_function * func );
void (*stmt_beg)( PLpgSQL_execstate * estate, PLpgSQL_stmt * stmt );
void (*stmt_end)( PLpgSQL_execstate * estate, PLpgSQL_stmt * stmt );
} PLpgSQL_plugin;
I've truncated the argument list (in this e-mail) for the (*init)()
function since it's rather long (error_callback and assign_expr are both
function pointers).
When the PL/pgSQL intrepreter loads the plugin, it calls the
plugin-init() function.
When the PL/pgSQL intrepreter starts running a new function, it calls
the plugin-func_beg() function.
When the PL/pgSQL intrepreter completes a function, it calls the
plugin-func_end() function.
When the PL/pgSQL interpreter is about to execute a line of PL/pgSQL
code, it calls plugin-stmt_beg()
When the PL/pgSQL interpreter has finished executing a line of PL/pgSQL
code, it calls plugin-stmt_end()
So here is where I need a few opinions:
1) I think the most straightforward way to load an instrumentation
plugin is to define a new custom GUC variable (using the
custom_variable_classes mechanism). When the PL/pgSQL call-handler
loads, it can check that config. variable (something like plpgsql.plugin
= '$libdir/plugin_profiler' or plpgsql.plugin =
'$libdir/plugin_debugger') and load the plugin if non-NULL. That seems
a little obtuse to me since custom variables don't appear in the
prototype postgresql.conf file. Would it be better to add a real GUC
variable instead of a custom variable?
2) Given that plpgsql.plugin points to the name of a shared-object file
(or DLL or whatever you prefer to call it), we need to find *something*
inside of the file. The most obvious choice would be to look for a
variable (a structure or structure pointer) with a fixed name. That
would mean, for example, that a plugin would define an externally
visible PLpgSQL_plugin structure named plugin_hooks and the PL/pgSQL
interpreter would look for that symbol inside of the plugin.
Alternatively, we could look for a function inside of the plugin
(something like 'plugin_loader') and then call that function with a
pointer to a PLpgSQL_plugin structure. I prefer the function-pointer
approach since we already have a reliable mechanism in place for finding
a function inside of a shared-object (the same mechanism works for
finding a variable instead of a function pointer, but I doubt that that
has been tested in all platforms).
3) Any comments on the PLpgSQL_plugin structure? Should it include (as
it's first member) a structure version number so we can add to/change
the structure as needed?
4) Do we need to support multiple active plugins? Would you ever need
to load the debugger at the same time you've loaded the profiler (no)?
Would you ever need to load the tracer at the same time you need the
debugger (probably not)? If we need to support multiple plugins, should
be just introduce a meta-plugin that knows how to handle a list of other
plugins? (Messy, but certainly gets the job done without worrying about
it right now).
5) I'll also be adding a void pointer to the PLpgSQL_execstate structure
(think of a PLpgSQL_execstate as a stack frame). The new pointer is
reserved for use by the plugin. It may be handy to add a void pointer
to each PLpgSQL_stmt as well - is that acceptable? (That would mean an
extra 4-bytes per-line of compiled PL/pgSQL code, even if you don't have
a plugin loaded).
Any other comments? Obviously, you'll have a chance to critique the
patch when I get it sent in.
Thanks for your help.
-- Korry
---(end of broadcast)---
TIP 4: Have you searched our list archives?
http
Re: [HACKERS] Loading the PL/pgSQL debugger (and other plugins)
or version changes (via magic-block checking), I'm not sure I see a need for this. That makes a lot more sense - I don't like the idea of each plugin managing its own version information. We can always add more function pointers to the end of the plugin structure - if the pointers are non-NULL, you gain more functionality. 4) Do we need to support multiple active plugins? Probably, but let's fix the API to hide this, so we don't have to commit now. Cool. -- Korry
Re: [HACKERS] Possible explanation for Win32 stats regression test
Ah-hah, I see it. pgwin32_select() uses WaitForMultipleObjectsEx() with an event for the socket read-ready plus an event for signal arrival. It returns EINTR if the return code from WaitForMultipleObjectsEx shows the signal-arrival event as fired. However, WaitForMultipleObjectsEx is defined to return the number of the *first* event in the list that is fired. This means that if the socket comes read-ready at the same time the SIGALRM arrives, pgwin32_select() will ignore the signal, and it'll be processed by the subsequent pgwin32_recv(). Now I don't know anything about the Windows scheduler, but I suppose it gives processes time quantums like everybody else does. So at the same time really means within the same scheduler clock tick, which is not so unlikely after all. In short, before the just-committed patch, the Windows stats collector would fail if a stats message arrived during the same clock tick that its SIGALRM timeout expired. I think this explains not only the intermittent stats regression failures, but the reports we've heard from Merlin and others about the stats collector being unstable under load on Windows. The heavier the load of stats messages, the more likely one is to arrive during the tick when the timeout expires. There's a second problem in pgwin32_waitforsinglesocket() that may be getting in your way. Inside of pgwin32_waitforsingleselect(), we create a kernel synchronization object (an Event) and associate that Event with the socket. When the TCP/IP stack detects interesting traffic on the socket, it signals the Event object (interesting in this case is READ, WRITE, CLOSE, or ACCEPT, depending on the caller) and that wakes up the call to WaitForMultipleObjectsEx(). That all works fine, unless you have two or more sockets in the backend (the important part is that src/include/port/win32.h #define's select() and other socket-related function - if you compile a piece of network code that happens to #include port/win32.h, you'll get the pgwin32_xxx() versions). The problem is that, each time you go through pgwin32_waitforsinglesocket(), you tie the *same* kernel object (waitevent is static) to each socket. If you have more than one socket, you'll tie each socket to the same kernel event. The kernel will signal that Event whenever interesting traffic appears on *any* of the sockets. The net effect is that, if you are waiting for activity on socket A, any activity on socket B will also awaken WaitForMultipleObjects(). If you then try to read from socket A, you'll get an operation would block error because nothing happened on socket A. The fix is pretty simple - just call WSAEventSelect( s, waitevent, 0 ) after WaitForMultipleObjectsEx() returns. That disassociates the socket from the Event (it will get re-associated the next time pgwin32_waitforsingleselect() is called. I ran into this problem working on the PL/pgSQL debugger and I haven't gotten around to posting a patch yet, sorry. -- Korry ([EMAIL PROTECTED])
Re: [HACKERS] shall we have a TRACE_MEMORY mode
As I follow Relyea Mike's recent post of possible memory leak, I think that we are lack of a good way of identifing memory usage. Maybe we should also remember __FILE__, __LINE__ etc for better memory usage diagnose when TRACE_MEMORY is on? I find __FILE__ and __LINE__ very helpful when debugging both leaks and corruption. I also add a per-context (AllocSetContext.callCount) call counter. Each time a new piece of memory is allocated from a context, I bump the call counter and record the new value in the header for that chunk of memory (AllocChunkData.callCount). That way, I can look at a chunk of memory and know that it was allocated the 42nd time that I grabbed a hunk of memory from that context. The next time I run my test, I can set a conditional breakpoint (cond callCounter==42) that stops at the correct allocation (and thus grab my stack dump). The numbers aren't always exactly the same, but in most cases they are. Obviously you're now writing 12 extra bytes of overhead to each AllocChunkData (__FILE__, __LINE__, and callCount) and 4 bytes to each AllocSetContext (callCount). -- Korry
Re: [HACKERS] file-locking and postmaster.pid
That's not workable, unless you want to assume that nothing on the system except Postgres uses SysV semaphores. Otherwise something else could randomly gobble up the semid you want to use. I don't care very much for requiring a distinct semid to be hand-specified for each postmaster on a machine, either. Yeah, that does suck. Ok, naming problems seem to make semaphores useless. I'm back to byte-range locking, but if NFS is important and is truly unreliable, then that's out too. I've never had locking problems on NFS (probably because we tell our users not to use NFS), but now that I think about it, SMB locking is very unreliable so Win32 would be an issue too. -- Korry
Re: [HACKERS] file-locking and postmaster.pid
I'm sure there's a good reason for having it the way it is, having so many smart knowledgeable people working on this project. Could someone please explain the rationale of the current solution to me? We've ignored Andreas' original question. Why not use a lock to indicate that the postmaster is still running? At first blush, that seems more reliable than checking for a (possibly recycled) process ID. -- Korry
Re: [HACKERS] file-locking and postmaster.pid
On Wednesday 24 May 2006 21:03, korry wrote: I'm sure there's a good reason for having it the way it is, having so many smart knowledgeable people working on this project. Could someone please explain the rationale of the current solution to me? We've ignored Andreas' original question. Why not use a lock to indicate that the postmaster is still running? At first blush, that seems more reliable than checking for a (possibly recycled) process ID. As Tom replied: Portability. Thanks - I missed that part of Tom's message. The only platform (although certainly not a minor issue) that I can think of that would have a portability issue would be Win32. You can't even read a locked byte in Win32. I usually solve that problem by locking a byte past the end of the file (which is portable). Is there some other portability issue that I'm missing? -- Korry
Re: [HACKERS] file-locking and postmaster.pid
Certainly on all platforms there must be *some* locking primitive. We just need to figure out the appropiate parameters to fcntl() or flock() or lockf() on each. Right. The Win32 API for locking seems mighty strange to me. Linux/Unix byte locking is advisory (meaning that one lock can block another lock, but it can't block a read). Win32 locking is mandatory (at least in the most portable form) so a lock blocks a reader. To avoid that problem, youlock a byte that you never intend to read (that is, you lock a byte past the end of the file). Locking past the end-of-file is portable to all Unix/Linux systems that I've seen (that way, you can lock a region of a file before you grow the file). -- Korry
Re: [HACKERS] file-locking and postmaster.pid
On Wed, 2006-05-24 at 16:34 -0400, Alvaro Herrera wrote: korry wrote: The Win32 API for locking seems mighty strange to me. Linux/Unix byte locking is advisory (meaning that one lock can block another lock, but it can't block a read). No -- it is advisory meaning that a process that does not try to acquire the lock is not locked out. Right, that's why I said can block instead of will block. An advisory lock will only block another locker, not another reader (except in Win32). You can certainly block a file in exclusive mode, using the LOCK_EX flag. (And at least on my Linux system, there is mandatory locking too, using the fcntl() interface). My fault - I'm not really talking about file locking, I'm talking about byte-range locking (via lockf() and family). I don't believe that you can use byte-range locking to block read-access to a file, you can only use byte-range locking to block other locks. A simple exclusive lock on the first byte past the end of the file will do. I think the next question is -- how would the lock interface be used? We could acquire an exclusive lock on postmaster start (to make sure no backend is running), then reduce it to a shared lock. Every backend would inherit the shared lock. But the lock exchange is not guaranteed to be atomic so a new postmaster could start just after we acquire the lock and acquire the shared lock. It'd need to be complemented with another lock. You never need to reduce it to a shared lock. On postmaster startup, try to lock the sentinel byte (one byte past the end-of-file). If you can lock it, you know that no other postmaster has that byte locked. If you can't lock it, another postmaster is running. It is an atomic operation. However, Tom may be correct about NFS locking, but I guess I'm surprised that anyone would care :-) Win32 locking is mandatory (at least in the most portable form) so a lock blocks a reader. There is also shared/exclusive locking of a file on Win32. Yes, but Win32 shared locking only works on NTFS-type file systems. And you don't need shared locking anyway. -- Korry
Re: [HACKERS] file-locking and postmaster.pid
Alvaro Herrera [EMAIL PROTECTED] writes: Certainly on all platforms there must be *some* locking primitive. We just need to figure out the appropiate parameters to fcntl() or flock() or lockf() on each. I use lockf() (not fcntl() or flock()) on every platform other than Win32. Of course, I may not run on every system that PostgreSQL supports. Quite aside from the hassle factor of needing to deal with N variants of the syscalls, I'm not convinced that it's guaranteed to work. ISTR that for instance NFS file locking is pretty much Alice-in-Wonderland :-( Since the entire point here is to have a guaranteed bulletproof check, locks that work most of the time on most platforms/filesystems aren't gonna be an improvement. NFS file locking may certainly be problematic. I don't know about NFS byte-range locking. What we currently have in place is not bulletproof. I think holding a byte-range lock in addition to the is there some process with the right pid? check might be a little more bullet resistant :-) -- Korry
Re: [HACKERS] file-locking and postmaster.pid
You never need to reduce it to a shared lock. On postmaster startup, try to lock the sentinel byte (one byte past the end-of-file). If you can lock it, you know that no other postmaster has that byte locked. If you can't lock it, another postmaster is running. It is an atomic operation. This doesn't work if the postmaster dies but a backend continues to run, which is arguably the most important case we need to protect against. I may be confused here, but I don't see the problem - byte-range locks are not inherited across a fork. A backend would never hold the lock, a backend would never even look for the lock. However, Tom may be correct about NFS locking, but I guess I'm surprised that anyone would care :-) Quite a lot of people run NFS-mounted data directories ... I'm happy to take your word for that, and I agree that if NFS is important and locking is brain-dead on NFS, then relying solely on a lock is unacceptable. -- Korry
Re: [HACKERS] file-locking and postmaster.pid
We already have two platforms that don't use the SysV semaphore interface, and even on ones that have it, I wouldn't want to assume they all support SEM_UNDO. Which platforms, just out of curiousity? I assume that Win32 is one of them. But aside from any portability issues, ISTM this would have its own failure modes. In particular you still have to rely on a pid-file (only now it's holding a semaphore ID not a PID) You've lost me... why would you store the semid and not the pid? I was thinking that the semid might be a postgresql.conf thingie. and there's still a bit of a leap of faith required to get from the observation that somebody is holding a lock on semaphore X to the conclusion that that somebody is a conflicting postmaster. Isn't that sort of like saying that if a postmaster.pid file exists, it must have been written by a postmaster? Pick a semaphore id and dedicate it to postmaster exclusion. It doesn't look to me like this is any better than the PID solution, really, as far as false positives go. As long as the kernel cleans up SEM_UNDO semaphores, I guess I don't see have you would have a false positive. Oh, I guess I should say that is you use a SEM_UNDO semaphore, you don't need the pid check anymore. And, no worry about NFS. As for false negatives: ipcrm. Yes, that's a problem, but I think it's the same as rm postmaster.pid, isn't it?
Re: [HACKERS] pg_hba.conf alternative
If you want the data hidden from system administrators, you need to have the client encrypt it before storing it. Of course, that will have massive implications for your application. Have you considered storing your data on an encrypted filesystem? I have no idea what kind of performance hit you would suffer, but you wouldn't have to change your application at all that way. Perhaps a private mount so that only the postgresql process tree could see the decrypted bits? -- Korry ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [HACKERS] pg_hba.conf alternative
Since what he is worried about is the ability of admins to get at the data by connecting to the postgres server (after changing pg_hba.conf), this will not make the slightest difference - the data would be decrypted before it ever got to the intruder. I was suggesting that pg_hba.conf could be stored in the same encrypting filesystem. -- Korry ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [HACKERS] pg_hba.conf alternative
I was suggesting that pg_hba.conf could be stored in the same encrypting filesystem. Then how can it be changed? What if you need to allow access from, say, another user or another network? Oh, the admins have to change it ... Not all admins are equal... the admin that takes care of the database would obviously have the decrypt password for the encrypting filesystem. That admin (but not other admins) can change the pg_hba.conf file. -- Korry ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [HACKERS] pg_hba.conf alternative
Why would you not simply set this up on a seperate machine to which only the trusted admins had access? Most data centers I am familiar with use single purpose machines anyway. If someone is trusted as root on your box they can screw you no matter what you do. Pretending otherwise is just folly. Agreed - that would be a much better (easier and more secure) solution where practical. -- Korry ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
[HACKERS] Coding standards? Recommendations?
I've noticed a variety of coding styles in the PostgreSQL source code. In particular, I see a mix of naming conventions. Some variables use camelCase (or CamelCase), others use under_score_style. I'm just wondering if there's a preferred naming convention. Is there an official convention? An unofficial recommendation? Or even a majority preference? I'm not necessarily arguing for an official standard, but I'd be happy to follow a standard/recommendation if there is one. I find it just as easy to write conforming code as non-conforming code. (sorry if this has been covered before - if you can point me to a document that covers the information I'm looking for, that would be very helpful). -- Korry ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] Coding standards? Recommendations?
I asked this question about a half year ago or so, and got: http://archives.postgresql.org/pgsql-hackers/2005-05/msg00331.php Thanks James, that's just what I was looking for. -- Korry ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [HACKERS] question on some code.
If I understand the code right, your trying to pass in to appendBinaryStringInfo an address or reference to the n8, n16, or n32 variables and cast them so that a char * pointer can access that address space. Through some testing that I've been doing (outputting the values in the appendBinaryStringInfo function), I never seem to see any data, The variable pointer that references the n8, n16, or n32 value is not holding any data in the appendBinaryStringInfo function. How are you looking at the data? If you're treating the pointer as if it were a null-terminated string, it's likely that the first byte is a null character much of the time. Since this is binary data (not character data), you'll need to look at the individual bytes. If you're using gdb, use 'x' command instead of the 'p' command. If you've thrown a printf() into the code, you can't use a %s format specifier, you'll have to look at the bytes one at a time using something like %02X. -- Korry
Re: [HACKERS] exception handling in plpgsql
I am using the following sytex to handle exceptions in plpgsql (I am using postgres 8 rc1) some code EXCEPTION WHEN NO_DATA THEN RAISE NOTICE 'NO DATA'; WHEN OTHERS THEN RAISE NOTICE 'An exception occurred'; RETURN emp_rec; and i receive the following error ERROR: unrecognized exception condition no_data How can i rectify this error? It seems you cant trap every condition listed in errocodes-appendix.html; in particular, you cant trap SUCCESSFUL_COMPLETION, any of the conditions listed in the WARNING category, or any of the conditions listed in the NO DATA category. (At least through 8.0 rc1 - I haven't checked in later versions). -- Korry
Re: [HACKERS] exception handling in plpgsql
On Sat, 2005-01-01 at 14:10 -0500, Tom Lane wrote: Korry [EMAIL PROTECTED] writes: It seems you cant trap every condition listed in errocodes- appendix.html; in particular, you cant trap SUCCESSFUL_COMPLETION, any of the conditions listed in the WARNING category, or any of the conditions listed in the NO DATA category. (At least through 8.0 rc1 - I haven't checked in later versions). Those aren't errors. Right. Just trying to clarify the issue for the person that asked the question. The pl/pgSQL documentation (37.7.5) says: The condition names can be any of those shown in Appendix A. As you say, not all of the conditions listed in Appendix A are error conditions. Perhaps 37.7.5 should be changed to clarify? Without looking at plerrcodes.h, you can't find a definitive list. -- Korry
Re: [HACKERS] What's a good PostgreSQL guide book?
The thing that's wrong with any of the books that are available is that they have considerable portions about the whole variety of language bindings (e.g. - Perl, Python, C, C++, ...) which bulk up the book when it's really only likely that you'd need a reference on one or two of the languages. I would have loved to see twice or three times as much in the NR book on performance tuning, and at least twice as much discussion about the implications of MVCC. Christopher, of all the topics that we covered, I enjoyed writing the performance chapter the most. In my daytime job, I work on languages and development tools and a lot of what I do is performance and tuning work. I'd love to hear suggestions on what you would like to see in the second edition. I've been trying to talk some of the publishers into letting us do an Advanced PostgreSQL Performance Tuning book. Not much luck yet - if our first book does really well, I think we'll be able to get the publishers a bit more interested in that sort of thing. The language bindings chapters did get a bit tedious to write, but it was fun to work on the PL/pgSQL and ecpg chapters - it's been difficult to find good information on either of those (until now). I hope you enjoy the rest of the book. -- Korry ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [HACKERS] protocol change in 7.4
There has been some previous discussion of changing the FE/BE protocol in 7.4, in order to fix several problems. I think this is worth doing: if we can resolve all these issues in a single release, it will lessen the upgrade difficulties for users. Here are a couple of other changes you might consider (maybe these changes already exist and I just don't know about them): a) Make much of the metadata sent to the client optional. When I execute 20 fetches against the same cursor, I don't need the same metadata 20 times. For narrow result sets, the metadata can easily double or triple the number of bytes sent across the net. It looks like the protocol needs the field count, but everything else seems to be sent for the convenience of the client application. b) Send a decoded version of atttypmod - specifically, decode the precision and scale for numeric types. ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [HACKERS] protocol change in 7.4
b) Send a decoded version of atttypmod - specifically, decode the precision and scale for numeric types. I want decode type,length,precision and scale Type is returned by PQftype(), length is returned by PQfsize(). Precision and scale are encoded in the return value from PQfmod() and you have to have a magic decoder ring to understand them. (Magic decoder rings are available, you just have to read the source code :-) PQftype() is not easy to use because it returns an OID instead of a name (or a standardized symbol), but I can't think of anything better to return to the client. Of course if you really want to make use of PQftype(), you can preload a client-side cache of type definitions. I seem to remember seeing a patch a while back that would build the cache and decode precision and scale too. PQfsize() is entertaining, but not often what you really want (you really want the width of the widest value in the column after conversion to some string format - it seems reasonable to let the client applicatin worry about that, although maybe that would be a useful client-side libpq function). ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [HACKERS] pg_database datistemplate
In the docs it is mentioned for datistemplate that If true then this database can be used in the TEMPLATE clause of CREATE DATABASE to create the new database as a clone of this one. However, one can create a database using as template another DB that has datistemplate set to false. In this situation, what is the point of having datistemplate? If datistemplate is true, any user with CREATEDB privileges can clone the database. If datistemplate is false, only a cluster superuser can clone the database. ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
