* Stephen Frost (sfr...@snowman.net) wrote: > Ugh, sorry about that, I should have realized that needed to be done. > Updated patch attached.
Errr, for real this time. Thanks, Stephen commit 25e94dcb390f56502bc46e683b438c20d2dc74e0 Author: Stephen Frost <sfr...@snowman.net> Date: Tue Feb 15 08:50:17 2011 -0500 assign_csvlog_fields() - reset context on error On error, we need to make sure to reset the memory context back to what it was when we entered.
*** a/doc/src/sgml/config.sgml --- b/doc/src/sgml/config.sgml *************** *** 3542,3548 **** local0.* /var/log/postgresql </row> <row> <entry><literal>%u</literal></entry> ! <entry>User name</entry> <entry>yes</entry> </row> <row> --- 3542,3561 ---- </row> <row> <entry><literal>%u</literal></entry> ! <entry>Session user name, typically the user name which was used ! to authenticate to <productname>PostgreSQL</productname> with, ! but can be changed by a superuser, see <command>SET SESSION ! AUTHORIZATION</></entry> ! <entry>yes</entry> ! </row> ! <row> ! <entry><literal>%U</literal></entry> ! <entry>Current role name, when set with <command>SET ROLE</>; ! the current role identifier is relevant for permission checking; ! Returns 'none' if the current role matches the session user. ! Note: Log messages from inside <literal>SECURITY DEFINER</> ! functions will show the calling role, not the effective role ! inside the <literal>SECURITY DEFINER</> function</entry> <entry>yes</entry> </row> <row> *************** *** 3659,3664 **** FROM pg_stat_activity; --- 3672,3717 ---- </listitem> </varlistentry> + <varlistentry id="guc-csvlog-fields" xreflabel="csvlog_fields"> + <term><varname>csvlog_fields</varname> (<type>string</type>)</term> + <indexterm> + <primary><varname>csvlog_fields</> configuration parameter</primary> + </indexterm> + <listitem> + <para> + Controls the set and order of the fields which are written out in + the CSV-format log file. + + The default is: + <literal>log_time, user_name, database_name, process_id, + connection_from, session_id, session_line_num, command_tag, + session_start_time, virtual_transaction_id, transaction_id, + error_severity, sql_state_code, message, detail, hint, + internal_query, internal_query_pos, context, query, query_pos, + location, application_name</literal> + + For details on what these fields are, refer to the + <varname>log_line_prefix</varname> and + <xref linkend="runtime-config-logging-csvlog"> documentation. + </para> + </listitem> + </varlistentry> + + <varlistentry id="guc-csvlog-header" xreflabel="csvlog_header"> + <term><varname>csvlog_header</varname> (<type>boolean</type>)</term> + <indexterm> + <primary><varname>csvlog_header</> configuration parameter</primary> + </indexterm> + <listitem> + <para> + Controls if a header should be output for each file logged through + the CSV-format logging. + + The default is: <literal>false</literal>, for backwards compatibility. + </para> + </listitem> + </varlistentry> + <varlistentry id="guc-log-lock-waits" xreflabel="log_lock_waits"> <term><varname>log_lock_waits</varname> (<type>boolean</type>)</term> <indexterm> *************** *** 3766,3799 **** FROM pg_stat_activity; Including <literal>csvlog</> in the <varname>log_destination</> list provides a convenient way to import log files into a database table. This option emits log lines in comma-separated-values ! (<acronym>CSV</>) format, ! with these columns: ! timestamp with milliseconds, ! user name, ! database name, ! process ID, ! client host:port number, ! session ID, ! per-session line number, ! command tag, ! session start time, ! virtual transaction ID, ! regular transaction ID, ! error severity, ! SQLSTATE code, ! error message, ! error message detail, ! hint, ! internal query that led to the error (if any), ! character count of the error position therein, ! error context, ! user query that led to the error (if any and enabled by ! <varname>log_min_error_statement</>), ! character count of the error position therein, ! location of the error in the PostgreSQL source code ! (if <varname>log_error_verbosity</> is set to <literal>verbose</>), ! and application name. ! Here is a sample table definition for storing CSV-format log output: <programlisting> CREATE TABLE postgres_log --- 3819,3971 ---- Including <literal>csvlog</> in the <varname>log_destination</> list provides a convenient way to import log files into a database table. This option emits log lines in comma-separated-values ! (<acronym>CSV</>) format. The following table defines the fields ! which can be included in the CSV output, their meanings, and if they ! are included in the default CSV layout (the default ordering matches ! the order of this table). ! ! <informaltable> ! <tgroup cols="3"> ! <thead> ! <row> ! <entry>CSV Field Name</entry> ! <entry>Definition</entry> ! <entry>Included by Default</entry> ! </row> ! </thead> ! <tbody> ! <row> ! <entry><literal>log_time</literal></entry> ! <entry>timestamp with milliseconds</entry> ! <entry>yes</entry> ! </row> ! <row> ! <entry><literal>user_name</literal></entry> ! <entry>session user name</entry> ! <entry>yes</entry> ! </row> ! <row> ! <entry><literal>role_name</literal></entry> ! <entry>current role name</entry> ! <entry>no</entry> ! </row> ! <row> ! <entry><literal>database_name</literal></entry> ! <entry>name of database connected to</entry> ! <entry>yes</entry> ! </row> ! <row> ! <entry><literal>process_id</literal></entry> ! <entry>process ID of the backend PG process</entry> ! <entry>yes</entry> ! </row> ! <row> ! <entry><literal>connection_from</literal></entry> ! <entry>client host/IP and port number</entry> ! <entry>yes</entry> ! </row> ! <row> ! <entry><literal>session_id</literal></entry> ! <entry>ID of the session</entry> ! <entry>yes</entry> ! </row> ! <row> ! <entry><literal>session_line_number</literal></entry> ! <entry>per-session line number</entry> ! <entry>yes</entry> ! </row> ! <row> ! <entry><literal>command_tag</literal></entry> ! <entry>Command tag of the logged command</entry> ! <entry>yes</entry> ! </row> ! <row> ! <entry><literal>session_start_time</literal></entry> ! <entry>Start time of the current session</entry> ! <entry>yes</entry> ! </row> ! <row> ! <entry><literal>virtual_transaction_id</literal></entry> ! <entry>Virtual Transaction ID</entry> ! <entry>yes</entry> ! </row> ! <row> ! <entry><literal>transaction_id</literal></entry> ! <entry>Regular Transaction ID</entry> ! <entry>yes</entry> ! </row> ! <row> ! <entry><literal>error_severity</literal></entry> ! <entry>Error severity code of the log message</entry> ! <entry>yes</entry> ! </row> ! <row> ! <entry><literal>sql_state_code</literal></entry> ! <entry>SQLSTATE code of the command being logged</entry> ! <entry>yes</entry> ! </row> ! <row> ! <entry><literal>message</literal></entry> ! <entry>Error message</entry> ! <entry>yes</entry> ! </row> ! <row> ! <entry><literal>detail</literal></entry> ! <entry>Error message detail</entry> ! <entry>yes</entry> ! </row> ! <row> ! <entry><literal>hint</literal></entry> ! <entry>Error message hint</entry> ! <entry>yes</entry> ! </row> ! <row> ! <entry><literal>internal_query</literal></entry> ! <entry>internal query that led to the error (if any)</entry> ! <entry>yes</entry> ! </row> ! <row> ! <entry><literal>internal_query_pos</literal></entry> ! <entry>character count of the error position of the internal query</entry> ! <entry>yes</entry> ! </row> ! <row> ! <entry><literal>context</literal></entry> ! <entry>error context</entry> ! <entry>yes</entry> ! </row> ! <row> ! <entry><literal>query</literal></entry> ! <entry>user query that led to the error (if any and enabled by <varname>log_min_error_statement</varname>)</entry> ! <entry>yes</entry> ! </row> ! <row> ! <entry><literal>query_pos</literal></entry> ! <entry>character count of the error position of the user query</entry> ! <entry>yes</entry> ! </row> ! <row> ! <entry><literal>location</literal></entry> ! <entry>location of the error in the PostgreSQL source code (if <varname>log_error_verbosity</varname> is set to <literal>verbose</literal>)</entry> ! <entry>yes</entry> ! </row> ! <row> ! <entry><literal>application_name</literal></entry> ! <entry>Name of the connecting application, if provided by the application</entry> ! <entry>yes</entry> ! </row> ! </tbody> ! </tgroup> ! </informaltable> ! ! The set of columns to be included, and their order, in the CSV ! output can be controlled using the <varname>csvlog_fields</varname> option. ! ! For additional details on the definition of the above columns, refer ! to the documentation for <varname>log_line_prefix</varname>. ! ! Here is a sample table definition for storing the default CSV-format ! log output: <programlisting> CREATE TABLE postgres_log *** a/src/backend/commands/variable.c --- b/src/backend/commands/variable.c *************** *** 847,852 **** assign_session_authorization(const char *value, bool doit, GucSource source) --- 847,857 ---- return result; } + /* + * function to return the stored session username, needed because we + * can't do catalog lookups when possibly being called after an error, + * eg: from elog.c or part of GUC handling. + */ const char * show_session_authorization(void) { *************** *** 972,977 **** assign_role(const char *value, bool doit, GucSource source) --- 977,987 ---- return result; } + /* + * function to return the stored role username, needed because we + * can't do catalog lookups when possibly being called after an error, + * eg: from elog.c or part of GUC handling. + */ const char * show_role(void) { *** a/src/backend/postmaster/syslogger.c --- b/src/backend/postmaster/syslogger.c *************** *** 147,152 **** static char *logfile_getname(pg_time_t timestamp, const char *suffix); --- 147,153 ---- static void set_next_rotation_time(void); static void sigHupHandler(SIGNAL_ARGS); static void sigUsr1Handler(SIGNAL_ARGS); + static void write_csvlog_header(FILE *out_fh); /* *************** *** 988,993 **** pipeThread(void *arg) --- 989,1019 ---- #endif /* WIN32 */ /* + * Internal function for writing out the header of a CSV-style log file + * to the passed-in file handle. + */ + static void + write_csvlog_header(FILE *out_fh) + { + int rc; + int header_length = strlen(csvlog_fields); + + /* Write out the csvlog_fields GUC, which matches the CSV log format + * header, at least, if we did everything right. */ + rc = fwrite(csvlog_fields, 1, header_length, out_fh); + + /* can't use ereport here because of possible recursion */ + if (rc != header_length) + write_stderr("could not write to new log file: %s\n", strerror(errno)); + + rc = fputc('\n', out_fh); + if (rc != '\n') + write_stderr("could not write to new log file: %s\n", strerror(errno)); + + return; + } + + /* * open the csv log file - we do this opportunistically, because * we don't know if CSV logging will be wanted. */ *************** *** 995,1004 **** static void open_csvlogfile(void) { char *filename; filename = logfile_getname(time(NULL), ".csv"); ! csvlogFile = logfile_open(filename, "a", false); pfree(filename); } --- 1021,1037 ---- open_csvlogfile(void) { char *filename; + FILE *fh; filename = logfile_getname(time(NULL), ".csv"); ! fh = logfile_open(filename, "a", false); ! ! /* Check if we are asked to write out a header for the CSV file. */ ! if (csvlog_header) ! write_csvlog_header(fh); ! ! csvlogFile = fh; pfree(filename); } *************** *** 1165,1170 **** logfile_rotate(bool time_based_rotation, int size_rotation_for) --- 1198,1207 ---- return; } + /* Check if we are asked to write out a header for the CSV file. */ + if (csvlog_header) + write_csvlog_header(fh); + fclose(csvlogFile); csvlogFile = fh; *** a/src/backend/utils/error/elog.c --- b/src/backend/utils/error/elog.c *************** *** 3,8 **** --- 3,17 ---- * elog.c * error logging and reporting * + * A few comments about situations where error processing is called: + * + * We need to be cautious of both a performance hit when logging, since + * log messages can be generated at a huge rate if every command is being + * logged and we also need to watch out for what can happen when we are + * trying to log from an aborted transaction. Specifically, attempting to + * do SysCache lookups and possibly use other usually available backend + * systems will fail badly when logging from an aborted transaction. + * * Some notes about recursion and errors during error processing: * * We need to be robust about recursive-error scenarios --- for example, *************** *** 59,73 **** --- 68,85 ---- #include "access/transam.h" #include "access/xact.h" + #include "commands/variable.h" #include "libpq/libpq.h" #include "libpq/pqformat.h" #include "mb/pg_wchar.h" #include "miscadmin.h" + #include "nodes/pg_list.h" #include "postmaster/postmaster.h" #include "postmaster/syslogger.h" #include "storage/ipc.h" #include "storage/proc.h" #include "tcop/tcopprot.h" + #include "utils/builtins.h" #include "utils/guc.h" #include "utils/memutils.h" #include "utils/ps_status.h" *************** *** 93,98 **** extern bool redirection_done; --- 105,144 ---- int Log_error_verbosity = PGERROR_VERBOSE; char *Log_line_prefix = NULL; /* format for extra log line info */ int Log_destination = LOG_DESTINATION_STDERR; + char *csvlog_fields = NULL; + bool csvlog_header = false; + + static List *csvlog_field_list = NIL; + + /* To add a CSV field option, you need to update the enum in elog.h, check + * if the last value in the enum changed and if so update MAX_CSVLOG_OPTS, + * add code to handle the option in write_csv(), and add it here. */ + const char *CSVFieldNames[] = { + "log_time", /* CSVLOG_LOG_TIME */ + "user_name", /* CSVLOG_USER_NAME */ + "role_name", /* CSVLOG_ROLE_NAME */ + "database_name", /* CSVLOG_DATABASE_NAME */ + "process_id", /* CSVLOG_PROCESS_ID */ + "connection_from", /* CSVLOG_CONNECTION_FROM */ + "session_id", /* CSVLOG_SESSION_ID */ + "session_line_num", /* CSVLOG_SESSION_LINE_NUM */ + "command_tag", /* CSVLOG_COMMAND_TAG */ + "session_start_time", /* CSVLOG_SESSION_START_TIME */ + "virtual_transaction_id", /* CSVLOG_VIRTUAL_TRANSACTION_ID */ + "transaction_id", /* CSVLOG_TRANSACTION_ID */ + "error_severity", /* CSVLOG_ERROR_SEVERITY */ + "sql_state_code", /* CSVLOG_SQL_STATE_CODE */ + "message", /* CSVLOG_MESSAGE */ + "detail", /* CSVLOG_DETAIL */ + "hint", /* CSVLOG_HINT */ + "internal_query", /* CSVLOG_INTERNAL_QUERY */ + "internal_query_pos", /* CSVLOG_INTERNAL_QUERY_POS */ + "context", /* CSVLOG_CONTEXT */ + "query", /* CSVLOG_QUERY */ + "query_pos", /* CSVLOG_QUERY_POS */ + "location", /* CSVLOG_LOCATION */ + "application_name" /* CSVLOG_APPLICATION_NAME */ + }; #ifdef HAVE_SYSLOG *************** *** 161,166 **** static void write_csvlog(ErrorData *edata); --- 207,217 ---- static void setup_formatted_log_time(void); static void setup_formatted_start_time(void); + /* extern'd and used from guc.c... */ + const char *assign_csvlog_fields(const char *newval, bool doit, + GucSource source); + + /* * in_error_recursion_trouble --- are we at risk of infinite error recursion? *************** *** 1817,1831 **** log_line_prefix(StringInfo buf, ErrorData *edata) } break; case 'u': - if (MyProcPort) { ! const char *username = MyProcPort->user_name; ! ! if (username == NULL || *username == '\0') ! username = _("[unknown]"); ! appendStringInfoString(buf, username); } break; case 'd': if (MyProcPort) { --- 1868,1891 ---- } break; case 'u': { ! const char *session_auth = show_session_authorization(); ! ! if (*session_auth != '\0') ! appendStringInfoString(buf, session_auth); ! else if (MyProcPort) ! { ! const char *username = MyProcPort->user_name; ! ! if (username == NULL || *username == '\0') ! username = _("[unknown]"); ! appendStringInfoString(buf, username); ! } } break; + case 'U': + appendStringInfoString(buf, show_role()); + break; case 'd': if (MyProcPort) { *************** *** 1921,1926 **** log_line_prefix(StringInfo buf, ErrorData *edata) --- 1981,2077 ---- } /* + * Called when the GUC csvlog_fields() option has been set + * (currently only allowed in postmaster.conf, on PG restart). + * + * Processes the list passed in from the GUC system and updates the + * csvlog_field_list variable, which will then be used to generate + * CSV log output. + */ + const char * + assign_csvlog_fields(const char *newval, bool doit, GucSource source) + { + /* Verify the list is valid */ + List *new_csv_fields = NIL; /* List we're building */ + List *column_list = NIL; /* List of columns from user */ + ListCell *l; + char *rawstring; /* Copy of user string */ + MemoryContext oldcontext; + + /* Need a modifyable version to pass to SplitIdentifierString */ + rawstring = pstrdup(newval); + + /* Parse string into list of identifiers */ + if (!SplitIdentifierString(rawstring, ',', &column_list)) + { + list_free(column_list); + pfree(rawstring); + return NULL; + } + + /* Empty isn't a valid option */ + if (column_list == NIL) + { + pfree(rawstring); + return NULL; + } + + /* + * We need the allocations done for the csvlog_field_list to + * be preserved, so allocate them in TopMemoryContext. + */ + oldcontext = MemoryContextSwitchTo(TopMemoryContext); + + /* + * Loop through all of the fields provided by the user and build + * up our new_csv_fields list which will be processed by write_csvlog + */ + foreach(l, column_list) + { + int curr_option; + + /* Loop through all of the valid field options to try and match the + * current entry in the list to one of them. */ + for (curr_option = 0; curr_option < MAX_CSVLOG_OPTS; curr_option++) + if (pg_strcasecmp(lfirst(l),CSVFieldNames[curr_option]) == 0) + { + new_csv_fields = lappend_int(new_csv_fields,curr_option); + break; + } + + /* check if no option matched, and if so, return error */ + if (curr_option == MAX_CSVLOG_OPTS) + { + /* Switch back to the calling context */ + MemoryContextSwitchTo(oldcontext); + + list_free(column_list); + pfree(rawstring); + + return NULL; + } + } + + if (doit) + { + /* put new list in place */ + List *old_list = csvlog_field_list; + + csvlog_field_list = new_csv_fields; + + list_free(old_list); + } + + list_free(column_list); + pfree(rawstring); + + /* Switch back to the calling context */ + MemoryContextSwitchTo(oldcontext); + + return newval; + } + + /* * append a CSV'd version of a string to a StringInfo * We use the PostgreSQL defaults for CSV, i.e. quote = escape = '"' * If it's NULL, append nothing. *************** *** 1946,1957 **** appendCSVLiteral(StringInfo buf, const char *data) } /* ! * Constructs the error message, depending on the Errordata it gets, in a CSV ! * format which is described in doc/src/sgml/config.sgml. */ static void write_csvlog(ErrorData *edata) { StringInfoData buf; bool print_stmt = false; --- 2097,2110 ---- } /* ! * Constructs the error message, depending on the Errordata it gets, in the CSV ! * format requested by the user, based on the csvlog_fields GUC. */ static void write_csvlog(ErrorData *edata) { + int num_fields; + bool first_field = true; StringInfoData buf; bool print_stmt = false; *************** *** 1961,1966 **** write_csvlog(ErrorData *edata) --- 2114,2126 ---- /* has counter been reset in current process? */ static int log_my_pid = 0; + ListCell *l; + + const char *session_auth = show_session_authorization(); + + /* csvlog_field_list should never be empty when we reach here */ + Assert(csvlog_field_list != NIL); + /* * This is one of the few places where we'd rather not inherit a static * variable's value from the postmaster. But since we will, reset it when *************** *** 1977,2134 **** write_csvlog(ErrorData *edata) initStringInfo(&buf); /* ! * timestamp with milliseconds ! * ! * Check if the timestamp is already calculated for the syslog message, ! * and use it if so. Otherwise, get the current timestamp. This is done ! * to put same timestamp in both syslog and csvlog messages. */ ! if (formatted_log_time[0] == '\0') ! setup_formatted_log_time(); ! appendStringInfoString(&buf, formatted_log_time); ! appendStringInfoChar(&buf, ','); ! /* username */ ! if (MyProcPort) ! appendCSVLiteral(&buf, MyProcPort->user_name); ! appendStringInfoChar(&buf, ','); ! /* database name */ ! if (MyProcPort) ! appendCSVLiteral(&buf, MyProcPort->database_name); ! appendStringInfoChar(&buf, ','); ! /* Process id */ ! if (MyProcPid != 0) ! appendStringInfo(&buf, "%d", MyProcPid); ! appendStringInfoChar(&buf, ','); ! /* Remote host and port */ ! if (MyProcPort && MyProcPort->remote_host) ! { ! appendStringInfoChar(&buf, '"'); ! appendStringInfoString(&buf, MyProcPort->remote_host); ! if (MyProcPort->remote_port && MyProcPort->remote_port[0] != '\0') ! { ! appendStringInfoChar(&buf, ':'); ! appendStringInfoString(&buf, MyProcPort->remote_port); ! } ! appendStringInfoChar(&buf, '"'); ! } ! appendStringInfoChar(&buf, ','); ! /* session id */ ! appendStringInfo(&buf, "%lx.%x", (long) MyStartTime, MyProcPid); ! appendStringInfoChar(&buf, ','); ! /* Line number */ ! appendStringInfo(&buf, "%ld", log_line_number); ! appendStringInfoChar(&buf, ','); ! /* PS display */ ! if (MyProcPort) ! { ! StringInfoData msgbuf; ! const char *psdisp; ! int displen; ! initStringInfo(&msgbuf); ! psdisp = get_ps_display(&displen); ! appendBinaryStringInfo(&msgbuf, psdisp, displen); ! appendCSVLiteral(&buf, msgbuf.data); ! pfree(msgbuf.data); ! } ! appendStringInfoChar(&buf, ','); ! /* session start timestamp */ ! if (formatted_start_time[0] == '\0') ! setup_formatted_start_time(); ! appendStringInfoString(&buf, formatted_start_time); ! appendStringInfoChar(&buf, ','); ! /* Virtual transaction id */ ! /* keep VXID format in sync with lockfuncs.c */ ! if (MyProc != NULL && MyProc->backendId != InvalidBackendId) ! appendStringInfo(&buf, "%d/%u", MyProc->backendId, MyProc->lxid); ! appendStringInfoChar(&buf, ','); ! /* Transaction id */ ! appendStringInfo(&buf, "%u", GetTopTransactionIdIfAny()); ! appendStringInfoChar(&buf, ','); ! /* Error severity */ ! appendStringInfoString(&buf, error_severity(edata->elevel)); ! appendStringInfoChar(&buf, ','); ! /* SQL state code */ ! appendStringInfoString(&buf, unpack_sql_state(edata->sqlerrcode)); ! appendStringInfoChar(&buf, ','); ! /* errmessage */ ! appendCSVLiteral(&buf, edata->message); ! appendStringInfoChar(&buf, ','); ! /* errdetail or errdetail_log */ ! if (edata->detail_log) ! appendCSVLiteral(&buf, edata->detail_log); ! else ! appendCSVLiteral(&buf, edata->detail); ! appendStringInfoChar(&buf, ','); ! /* errhint */ ! appendCSVLiteral(&buf, edata->hint); ! appendStringInfoChar(&buf, ','); ! /* internal query */ ! appendCSVLiteral(&buf, edata->internalquery); ! appendStringInfoChar(&buf, ','); ! /* if printed internal query, print internal pos too */ ! if (edata->internalpos > 0 && edata->internalquery != NULL) ! appendStringInfo(&buf, "%d", edata->internalpos); ! appendStringInfoChar(&buf, ','); ! /* errcontext */ ! appendCSVLiteral(&buf, edata->context); ! appendStringInfoChar(&buf, ','); ! /* user query --- only reported if not disabled by the caller */ ! if (is_log_level_output(edata->elevel, log_min_error_statement) && ! debug_query_string != NULL && ! !edata->hide_stmt) ! print_stmt = true; ! if (print_stmt) ! appendCSVLiteral(&buf, debug_query_string); ! appendStringInfoChar(&buf, ','); ! if (print_stmt && edata->cursorpos > 0) ! appendStringInfo(&buf, "%d", edata->cursorpos); ! appendStringInfoChar(&buf, ','); ! ! /* file error location */ ! if (Log_error_verbosity >= PGERROR_VERBOSE) ! { ! StringInfoData msgbuf; ! ! initStringInfo(&msgbuf); ! ! if (edata->funcname && edata->filename) ! appendStringInfo(&msgbuf, "%s, %s:%d", ! edata->funcname, edata->filename, ! edata->lineno); ! else if (edata->filename) ! appendStringInfo(&msgbuf, "%s:%d", ! edata->filename, edata->lineno); ! appendCSVLiteral(&buf, msgbuf.data); ! pfree(msgbuf.data); ! } ! appendStringInfoChar(&buf, ','); ! /* application name */ ! if (application_name) ! appendCSVLiteral(&buf, application_name); appendStringInfoChar(&buf, '\n'); --- 2137,2385 ---- initStringInfo(&buf); /* ! * Get the number of fields, so we make sure to *not* include a comma ! * after the last field. */ ! num_fields = list_length(csvlog_field_list); ! /* ! * Loop through the fields requested by the user, in the order requested, in ! * the csvlog_fields GUC. ! */ ! foreach(l, csvlog_field_list) ! { ! /* If this isn't the first field, prepend a comma to seperate this ! * field from the previous one */ ! if (!first_field) ! appendStringInfoChar(&buf, ','); ! else ! first_field = false; ! switch (lfirst_int(l)) ! { ! case CSVLOG_LOG_TIME: ! { ! /* ! * timestamp with milliseconds ! * ! * Check if the timestamp is already calculated for the syslog message, ! * and use it if so. Otherwise, get the current timestamp. This is done ! * to put same timestamp in both syslog and csvlog messages. ! */ ! if (formatted_log_time[0] == '\0') ! setup_formatted_log_time(); ! ! appendStringInfoString(&buf, formatted_log_time); ! } ! break; ! case CSVLOG_USER_NAME: ! { ! /* session username, as done for %u */ ! if (*session_auth != '\0') ! appendCSVLiteral(&buf, session_auth); ! else ! /* username */ ! if (MyProcPort) ! { ! const char *username = MyProcPort->user_name; ! if (username == NULL || *username == '\0') ! username = _("[unknown]"); ! appendCSVLiteral(&buf, MyProcPort->user_name); ! } ! } ! break; ! case CSVLOG_ROLE_NAME: ! /* current role, not updated if someone renames it in another ! * session, of course */ ! appendCSVLiteral(&buf, show_role()); ! break; ! case CSVLOG_DATABASE_NAME: ! { ! /* database name */ ! if (MyProcPort) ! appendCSVLiteral(&buf, MyProcPort->database_name); ! } ! break; ! ! case CSVLOG_PROCESS_ID: ! { ! /* Process id */ ! if (MyProcPid != 0) ! appendStringInfo(&buf, "%d", MyProcPid); ! } ! break; ! case CSVLOG_CONNECTION_FROM: ! { ! /* Remote host and port */ ! if (MyProcPort && MyProcPort->remote_host) ! { ! appendStringInfoChar(&buf, '"'); ! appendStringInfoString(&buf, MyProcPort->remote_host); ! if (MyProcPort->remote_port && MyProcPort->remote_port[0] != '\0') ! { ! appendStringInfoChar(&buf, ':'); ! appendStringInfoString(&buf, MyProcPort->remote_port); ! } ! appendStringInfoChar(&buf, '"'); ! } ! } ! break; ! case CSVLOG_SESSION_ID: ! /* session id */ ! appendStringInfo(&buf, "%lx.%x", (long) MyStartTime, MyProcPid); ! break; ! case CSVLOG_SESSION_LINE_NUM: ! /* Line number */ ! appendStringInfo(&buf, "%ld", log_line_number); ! break; ! case CSVLOG_COMMAND_TAG: ! { ! /* PS display */ ! if (MyProcPort) ! { ! StringInfoData msgbuf; ! const char *psdisp; ! int displen; ! initStringInfo(&msgbuf); ! psdisp = get_ps_display(&displen); ! appendBinaryStringInfo(&msgbuf, psdisp, displen); ! appendCSVLiteral(&buf, msgbuf.data); ! pfree(msgbuf.data); ! } ! } ! break; ! case CSVLOG_SESSION_START_TIME: ! { ! /* session start timestamp */ ! if (formatted_start_time[0] == '\0') ! setup_formatted_start_time(); ! appendStringInfoString(&buf, formatted_start_time); ! } ! break; ! case CSVLOG_VIRTUAL_TRANSACTION_ID: ! { ! /* Virtual transaction id */ ! /* keep VXID format in sync with lockfuncs.c */ ! if (MyProc != NULL && MyProc->backendId != InvalidBackendId) ! appendStringInfo(&buf, "%d/%u", MyProc->backendId, MyProc->lxid); ! } ! break; ! case CSVLOG_TRANSACTION_ID: ! /* Transaction id */ ! appendStringInfo(&buf, "%u", GetTopTransactionIdIfAny()); ! break; ! case CSVLOG_ERROR_SEVERITY: ! /* Error severity */ ! appendStringInfoString(&buf, error_severity(edata->elevel)); ! break; ! case CSVLOG_SQL_STATE_CODE: ! /* SQL state code */ ! appendStringInfoString(&buf, unpack_sql_state(edata->sqlerrcode)); ! break; ! case CSVLOG_MESSAGE: ! /* errmessage */ ! appendCSVLiteral(&buf, edata->message); ! break; ! case CSVLOG_DETAIL: ! { ! /* errdetail or errdetail_log */ ! if (edata->detail_log) ! appendCSVLiteral(&buf, edata->detail_log); ! else ! appendCSVLiteral(&buf, edata->detail); ! } ! break; ! ! case CSVLOG_HINT: ! /* errhint */ ! appendCSVLiteral(&buf, edata->hint); ! break; ! case CSVLOG_INTERNAL_QUERY: ! /* internal query */ ! appendCSVLiteral(&buf, edata->internalquery); ! break; ! case CSVLOG_INTERNAL_QUERY_POS: ! { ! /* if printed internal query, print internal pos too */ ! if (edata->internalpos > 0 && edata->internalquery != NULL) ! appendStringInfo(&buf, "%d", edata->internalpos); ! } ! break; ! case CSVLOG_CONTEXT: ! /* errcontext */ ! appendCSVLiteral(&buf, edata->context); ! break; ! case CSVLOG_QUERY: ! { ! /* user query --- only reported if not disabled by the caller */ ! if (is_log_level_output(edata->elevel, log_min_error_statement) && ! debug_query_string != NULL && ! !edata->hide_stmt) ! print_stmt = true; ! if (print_stmt) ! appendCSVLiteral(&buf, debug_query_string); ! } ! break; ! ! case CSVLOG_QUERY_POS: ! { ! if (print_stmt && edata->cursorpos > 0) ! appendStringInfo(&buf, "%d", edata->cursorpos); ! } ! break; ! ! case CSVLOG_LOCATION: ! { ! /* file error location */ ! if (Log_error_verbosity >= PGERROR_VERBOSE) ! { ! StringInfoData msgbuf; ! ! initStringInfo(&msgbuf); ! ! if (edata->funcname && edata->filename) ! appendStringInfo(&msgbuf, "%s, %s:%d", ! edata->funcname, edata->filename, ! edata->lineno); ! else if (edata->filename) ! appendStringInfo(&msgbuf, "%s:%d", ! edata->filename, edata->lineno); ! appendCSVLiteral(&buf, msgbuf.data); ! pfree(msgbuf.data); ! } ! } ! break; ! case CSVLOG_APPLICATION_NAME: ! { ! /* application name */ ! if (application_name) ! appendCSVLiteral(&buf, application_name); ! } ! break; ! } ! } appendStringInfoChar(&buf, '\n'); *** a/src/backend/utils/misc/guc.c --- b/src/backend/utils/misc/guc.c *************** *** 65,70 **** --- 65,71 ---- #include "tsearch/ts_cache.h" #include "utils/builtins.h" #include "utils/bytea.h" + #include "utils/elog.h" #include "utils/guc_tables.h" #include "utils/memutils.h" #include "utils/pg_locale.h" *************** *** 191,196 **** static char *config_enum_get_options(struct config_enum * record, --- 192,200 ---- const char *prefix, const char *suffix, const char *separator); + /* Needs to be defined here because elog.h can't #include guc.h */ + extern const char *assign_csvlog_fields(const char *newval, + bool doit, GucSource source); /* * Options for enum values defined in this module. *************** *** 1034,1039 **** static struct config_bool ConfigureNamesBool[] = --- 1038,1052 ---- false, NULL, NULL }, { + {"csvlog_header", PGC_POSTMASTER, LOGGING_WHAT, + gettext_noop("Enables including a header on CSV log files."), + NULL, + }, + &csvlog_header, + false, NULL, NULL + }, + + { {"sql_inheritance", PGC_USERSET, COMPAT_OPTIONS_PREVIOUS, gettext_noop("Causes subtables to be included by default in various commands."), NULL *************** *** 2326,2331 **** static struct config_string ConfigureNamesString[] = --- 2339,2355 ---- }, { + {"csvlog_fields", PGC_POSTMASTER, LOGGING_WHAT, + gettext_noop("Controls fields logged to CSV logfiles."), + gettext_noop("If blank, the default set of fields is used."), + GUC_LIST_INPUT + }, + &csvlog_fields, + "log_time, user_name, database_name, process_id, connection_from, session_id, session_line_num, command_tag, session_start_time, virtual_transaction_id, transaction_id, error_severity, sql_state_code, message, detail, hint, internal_query, internal_query_pos, context, query, query_pos, location, application_name", + assign_csvlog_fields, NULL + }, + + { {"log_timezone", PGC_SIGHUP, LOGGING_WHAT, gettext_noop("Sets the time zone to use in log messages."), NULL *** a/src/backend/utils/misc/postgresql.conf.sample --- b/src/backend/utils/misc/postgresql.conf.sample *************** *** 360,366 **** #log_hostname = off #log_line_prefix = '' # special values: # %a = application name ! # %u = user name # %d = database name # %r = remote host and port # %h = remote host --- 360,367 ---- #log_hostname = off #log_line_prefix = '' # special values: # %a = application name ! # %u = session user name ! # %U = current role name # %d = database name # %r = remote host and port # %h = remote host *************** *** 378,383 **** --- 379,389 ---- # processes # %% = '%' # e.g. '<%u%%%d> ' + + # fields to include in the CSV log output + #csvlog_fields = 'log_time, user_name, database_name, process_id, connection_from, session_id, session_line_num, command_tag, session_start_time, virtual_transaction_id, transaction_id, error_severity, sql_state_code, message, detail, hint, internal_query, internal_query_pos, context, query, query_pos, location, application_name' + #csvlog_header = false # should csvlog files have a header? + #log_lock_waits = off # log lock waits >= deadlock_timeout #log_statement = 'none' # none, ddl, mod, all #log_temp_files = -1 # log temporary files equal or larger *** a/src/include/utils/elog.h --- b/src/include/utils/elog.h *************** *** 330,337 **** typedef enum --- 330,386 ---- extern int Log_error_verbosity; extern char *Log_line_prefix; + extern char *csvlog_fields; /* List of fields to log with CSV logging */ + extern bool csvlog_header; /* Whether to include a header on CSV log files */ extern int Log_destination; + /* + * Enum of the CSV fields we understand for CSV-based logging, + * if an new field is added, the enum has to be updated, the + * definition of field names in elog.c needs to be updated, and the + * new field needs to be handled in write_csv() in elog.c. + * Also be sure to update MAX_CSVLOG_OPTS if you change what the last + * option in the enum list is. + */ + typedef enum LogCSVFields + { + CSVLOG_LOG_TIME, + CSVLOG_USER_NAME, + CSVLOG_ROLE_NAME, + CSVLOG_DATABASE_NAME, + CSVLOG_PROCESS_ID, + CSVLOG_CONNECTION_FROM, + CSVLOG_SESSION_ID, + CSVLOG_SESSION_LINE_NUM, + CSVLOG_COMMAND_TAG, + CSVLOG_SESSION_START_TIME, + CSVLOG_VIRTUAL_TRANSACTION_ID, + CSVLOG_TRANSACTION_ID, + CSVLOG_ERROR_SEVERITY, + CSVLOG_SQL_STATE_CODE, + CSVLOG_MESSAGE, + CSVLOG_DETAIL, + CSVLOG_HINT, + CSVLOG_INTERNAL_QUERY, + CSVLOG_INTERNAL_QUERY_POS, + CSVLOG_CONTEXT, + CSVLOG_QUERY, + CSVLOG_QUERY_POS, + CSVLOG_LOCATION, + CSVLOG_APPLICATION_NAME + } LogCSVFields; + + /* Make sure to update this if you add CSV log options and change + * what the last CSVLOG option is */ + #define MAX_CSVLOG_OPTS CSVLOG_APPLICATION_NAME+1 + + /* + * Array of the names of each of the CSV fields we allow for logging, + * if an new field is added, the enum has to be updated *and* the + * definition of field names in elog.c needs to be updated. + */ + extern const char *CSVFieldNames[]; + /* Log destination bitmap */ #define LOG_DESTINATION_STDERR 1 #define LOG_DESTINATION_SYSLOG 2 *** a/src/tools/pgindent/typedefs.list --- b/src/tools/pgindent/typedefs.list *************** *** 855,860 **** LockTagType --- 855,861 ---- LockTupleMode LockingClause LogStmtLevel + LogCSVFields LogicalTape LogicalTapeSet MAGIC
signature.asc
Description: Digital signature