Re: [PATCHES] Exec statement logging
Bruce Momjian wrote:
Tom Lane wrote:
Bruce Momjian pgman@candle.pha.pa.us writes:
a null for the prepare string. Also, rather than change the API for
pg_parse_query(), I use a global variable that I check after the
function call.
This is horribly ugly and will doubtless lead to pfree crashes. What
Agreed, it needs work. I modified the patch to use malloc/free so that
you can always free the memory at the top of the function.
was the point again?
Simon said that the EXECUTE is pretty useless for debugging unless we
show the prepare query. His patch shows the prepare query for
client-side prepare, but not for server side when using the
PREPARE/EXECUTE commands --- seems we should display it in both cases.
Here is an updated version of the patch. It pulls post-parse logging
out into a separate function, called log_after_parse(), so we only log
in places we want it. I added code to log client-side parse, and
prepare, were appropriate, and have the logging of client-side and
server-side execute with the PREPARE string.
--
Bruce Momjian| http://candle.pha.pa.us
pgman@candle.pha.pa.us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup.| Newtown Square, Pennsylvania 19073
Index: src/backend/commands/prepare.c
===
RCS file: /cvsroot/pgsql/src/backend/commands/prepare.c,v
retrieving revision 1.36
diff -c -c -r1.36 prepare.c
*** src/backend/commands/prepare.c 1 Jan 2005 05:43:06 - 1.36
--- src/backend/commands/prepare.c 20 May 2005 14:15:12 -
***
*** 104,112
/* Generate plans for queries. Snapshot is already set. */
plan_list = pg_plan_queries(query_list, NULL, false);
! /* Save the results. */
StorePreparedStatement(stmt-name,
! NULL,/* text form
not available */
commandTag,
query_list,
plan_list,
--- 104,115
/* Generate plans for queries. Snapshot is already set. */
plan_list = pg_plan_queries(query_list, NULL, false);
! /*
!* Save the results. We don't have the query string for this
PREPARE,
!* but we do have the string we got from the client, so use that.
!*/
StorePreparedStatement(stmt-name,
! debug_query_string,
commandTag,
query_list,
plan_list,
Index: src/backend/tcop/postgres.c
===
RCS file: /cvsroot/pgsql/src/backend/tcop/postgres.c,v
retrieving revision 1.443
diff -c -c -r1.443 postgres.c
*** src/backend/tcop/postgres.c 21 Apr 2005 19:18:13 - 1.443
--- src/backend/tcop/postgres.c 20 May 2005 14:15:17 -
***
*** 82,90
LogStmtLevel log_statement = LOGSTMT_NONE;
- /* flag indicating if the statement satisfies log_statement */
- bool statement_logged;
-
/* GUC variable for maximum stack depth (measured in kilobytes) */
int max_stack_depth = 2048;
--- 82,87
***
*** 152,157
--- 149,156
static intInteractiveBackend(StringInfo inBuf);
static intSocketBackend(StringInfo inBuf);
static intReadCommand(StringInfo inBuf);
+ static bool log_after_parse(List *raw_parsetree_list,
+ const char *query_string, char
**prepare_string);
static void start_xact_command(void);
static void finish_xact_command(void);
static void SigHupHandler(SIGNAL_ARGS);
***
*** 465,538
pg_parse_query(const char *query_string)
{
List *raw_parsetree_list;
- ListCell *parsetree_item;
-
- statement_logged = false;
- if (log_statement == LOGSTMT_ALL)
- {
- ereport(LOG,
- (errmsg(statement: %s, query_string)));
- statement_logged = true;
- }
if (log_parser_stats)
ResetUsage();
raw_parsetree_list = raw_parser(query_string);
- /* do log_statement tests for mod and ddl */
- if (log_statement == LOGSTMT_MOD ||
- log_statement == LOGSTMT_DDL)
- {
- foreach(parsetree_item, raw_parsetree_list)
- {
- Node *parsetree = (Node *) lfirst(parsetree_item);
- const char *commandTag;
-
- if (IsA(parsetree, ExplainStmt)
- ((ExplainStmt *) parsetree)-analyze)
-
Re: [PATCHES] [HACKERS] New wal_sync_method for Darwin?
Applied. --- Bruce Momjian wrote: Attached is an updated version of your Darwin Fsync-writethrough patch. I did much more restructuring because we do fsync in more places than just WAL (like at checkpoints), and we should follow the WAL write-through in those places too if that method is used for fsync of WAL, rather than the normal fsync without the force-writethrough. I restructured the code so now pg_fsync does writethrough or not based on the wal setting. My feeling is you need WAL writethrough, you need it for other fsyncs as well. Write-through was added in an 8.0.X subrelease, and because it was a subrelease, the patch did a minimal amount of code change. This change is the restructuring that makes sense for 8.1. --- Chris Campbell wrote: Thanks, Bruce! Here's the patch (would you rather patches be attached files, or inline?). Some explanation. In the current code, there is no fsync on Windows, only fsync_writethrough. But, internally, it behaves identically to fsync. The only special case was for the GUC strings. The patch changes it thusly: since fsync and fsync_writethrough cannot be identical on Darwin, I added a new constant for SYNC_METHOD_FSYNC_WRITE_THROUGH (3). A GUC value of fsync_writethrough sets fsync_method to this new constant. On Windows, the fsync_writethrough case falls through to the fsync case (which simply runs fsync). On Darwin, the fsync_writethrough case executes fcntl(openLogFile, F_FULLFSYNC, 0), which does everything that fsync does, and then flushes the disk's write cache. The code does not need to call fsync directly, just fcntl. The performance of a script that executes 11,000 INSERT statements (outside a transaction) is pretty slow, since it requires flushing the disk write cache after EVERY query (transaction commit). I waited about 15 minutes before I just killed it. But if you do all the inserts inside a transaction, then I only pay the cache flush price once at transaction commit, and it's quite usable. I asked around on IRC, and the only place where flushing the disk cache is important is when syncing a WAL file. So I didn't change the behavior of fsync() globally for the code...just for WAL syncs. I added a new #define called HAVE_FSYNC_WRITE_THROUGH, which is defined only by the win32.h and darwin.h port headers. This controls the availability of the fsync_writethrough GUC setting. Thanks to the port headers, no configure test is needed. Thanks! Let me know what you think of the patch, and tell me what else I can do. - Chris -- Bruce Momjian| http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 Index: doc/src/sgml/runtime.sgml === RCS file: /cvsroot/pgsql/doc/src/sgml/runtime.sgml,v retrieving revision 1.319 diff -c -c -r1.319 runtime.sgml *** doc/src/sgml/runtime.sgml 15 May 2005 00:26:18 - 1.319 --- doc/src/sgml/runtime.sgml 17 May 2005 22:03:49 - *** *** 1595,1601 values are literalfsync/ (call functionfsync()/ at each commit), literalfdatasync/ (call functionfdatasync()/ at each commit), ! literalfsync_writethrough/ (call function_commit()/ at each commit on Windows), literalopen_sync/ (write WAL files with functionopen()/ option symbolO_SYNC/), and literalopen_datasync/ (write WAL files with functionopen()/ option symbolO_DSYNC/). Not all of these choices are available on all platforms. --- 1595,1601 values are literalfsync/ (call functionfsync()/ at each commit), literalfdatasync/ (call functionfdatasync()/ at each commit), ! literalfsync_writethrough/ (force write-through of any disk write cache), literalopen_sync/ (write WAL files with functionopen()/ option symbolO_SYNC/), and literalopen_datasync/ (write WAL files with functionopen()/ option symbolO_DSYNC/). Not all of these choices are available on all platforms. Index: src/backend/access/transam/xlog.c === RCS file: /cvsroot/pgsql/src/backend/access/transam/xlog.c,v retrieving revision 1.191 diff -c -c -r1.191 xlog.c *** src/backend/access/transam/xlog.c 10 May 2005 22:27:29 - 1.191 --- src/backend/access/transam/xlog.c 17 May 2005 22:03:53 - *** *** 51,61 * default method. We assume that fsync() is always available, and that * configure
Re: [PATCHES] WIP XLog Switch
Any farther on this? --- Simon Riggs wrote: WIP patch to perform a switch from one log file to next when we do a pg_stop_backup(), including wal replay handling at recovery time. Patch currently crashes server at various points, so don't stare too hard, but patch applies cleanly on cvstip, compiles and make checks. Main issue is the need to poke the xlog record pointer with a new value after the log switch. I'm a little uncertain about that approach and I'm very likely getting it wrong now. Better ideas welcome. Patch is incomplete in that it doesn't handle shutdown checkpoints as log switches in archive mode (yet) Also nothing in here about standby databases (yet) Any comments appreciated before I spend too much time on this. Best Regards, Simon Riggs [ Attachment, skipping... ] ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly -- Bruce Momjian| http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
