On Tue, Nov 16, 2021 at 1:12 AM Justin Pryzby <pry...@telsasoft.com> wrote:
>
> On Mon, Nov 15, 2021 at 09:12:49PM +0530, vignesh C wrote:
> > The idea here is to implement & expose pg_print_backtrace function,
> > internally
>
> This patch is closely related to this one
> https://commitfest.postgresql.org/35/3142/
> | Logging plan of the currently running query
>
> I suggest to review that patch and make sure there's nothing you could borrow.
I had a look and felt the attached patch is in similar lines
> My only comment for now is that maybe the function name should be
> pg_log_backtrace() rather than pg_print_backtrace(), since it doesn't actually
> "print" the backtrace, but rather request the other backend to log its
> backtrace.
Modified
> Did you see that the windows build failed ?
> https://ci.appveyor.com/project/postgresql-cfbot/postgresql/build/1.0.153557
> I think you'll need to create an "alternate" output like
> src/test/regress/expected/misc_functions_1.out
>
> It's possible that's best done by creating a totally new .sql and .out files
> added to src/test/regress/parallel_schedule - because otherwise, you'd have to
> duplicate the existing 300 lines of misc_fuctions.out.
Modified
Attached v14 patch has the fixes for the same.
Regards,
Vignesh
From e267fb1772c05ad8f622d9dae0e474bcd8b25da6 Mon Sep 17 00:00:00 2001
From: Vignesh C <vignes...@gmail.com>
Date: Tue, 9 Nov 2021 16:28:03 +0530
Subject: [PATCH v14 1/2] Add function to log the backtrace of the specified
backend process.
This commit adds pg_log_backtrace() function that requests to log
the backtrace of the specified backend process.
Only superusers are allowed to request to log the backtrace
because allowing any users to issue this request at an unbounded rate
would cause lots of log messages and which can lead to denial of service.
On receipt of the request, at the next CHECK_FOR_INTERRUPTS(),
the target backend logs its backtrace at LOG_SERVER_ONLY level,
so that the backtrace will appear in the server log but not
be sent to the client.
Bump catalog version.
---
doc/src/sgml/func.sgml | 85 +++++++++++++++++++++++
src/backend/catalog/system_functions.sql | 2 +
src/backend/postmaster/autovacuum.c | 4 ++
src/backend/storage/ipc/procsignal.c | 41 +++++++++++
src/backend/storage/ipc/signalfuncs.c | 49 +++++++++++++
src/backend/tcop/postgres.c | 4 ++
src/backend/utils/error/elog.c | 20 ++++--
src/backend/utils/init/globals.c | 1 +
src/include/catalog/catversion.h | 2 +-
src/include/catalog/pg_proc.dat | 5 ++
src/include/miscadmin.h | 1 +
src/include/storage/procsignal.h | 3 +-
src/include/utils/elog.h | 2 +
src/test/regress/expected/backtrace.out | 42 +++++++++++
src/test/regress/expected/backtrace_1.out | 46 ++++++++++++
src/test/regress/parallel_schedule | 2 +-
src/test/regress/sql/backtrace.sql | 30 ++++++++
17 files changed, 332 insertions(+), 7 deletions(-)
create mode 100644 src/test/regress/expected/backtrace.out
create mode 100644 src/test/regress/expected/backtrace_1.out
create mode 100644 src/test/regress/sql/backtrace.sql
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 24447c0017..a88583edfd 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25345,6 +25345,32 @@ SELECT collation for ('foo' COLLATE "de_DE");
</para></entry>
</row>
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_log_backtrace</primary>
+ </indexterm>
+ <function>pg_log_backtrace</function> ( <parameter>pid</parameter> <type>integer</type> )
+ <returnvalue>boolean</returnvalue>
+ </para>
+ <para>
+ Requests to log the backtrace of the
+ <glossterm linkend="glossary-backend">backend</glossterm>
+ with the specified process ID. The backtrace will be logged at message
+ level <literal>LOG</literal>. It will appear in the server log based on
+ the log configuration set (See <xref linkend="runtime-config-logging"/>
+ for more information), but will not be sent to the client regardless of
+ <xref linkend="guc-client-min-messages"/>. A backtrace will identify
+ where exactly the backend process is currently executing. This may be
+ useful to developers to diagnose stuck processes and other problems.
+ This feature is not supported for the postmaster, logger, checkpointer,
+ walwriter, background writer or statistics collector process. This
+ feature will be available if PostgreSQL was built with the ability to
+ capture backtrace. If not available, the function will emit a warning
+ and return false.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
@@ -25458,6 +25484,65 @@ LOG: Grand total: 1651920 bytes in 201 blocks; 622360 free (88 chunks); 1029560
because it may generate a large number of log messages.
</para>
+ <para>
+ <function>pg_log_backtrace</function> can be used to log the backtrace of
+ a backend process. For example:
+<programlisting>
+postgres=# select pg_log_backtrace(pg_backend_pid());
+ pg_log_backtrace
+------------------
+ t
+(1 row)
+</programlisting>
+The backtrace will be logged to the log file if logging is enabled, if logging
+is disabled backtrace will be logged to the console where the postmaster was
+started. For example:
+<screen>
+2021-01-27 11:33:50.247 IST [111735] LOG: current backtrace:
+ postgres: postgresdba postgres [local] SELECT(set_backtrace+0x38) [0xae06c5]
+ postgres: postgresdba postgres [local] SELECT(ProcessInterrupts+0x788) [0x950c34]
+ postgres: postgresdba postgres [local] SELECT() [0x761e89]
+ postgres: postgresdba postgres [local] SELECT() [0x71bbda]
+ postgres: postgresdba postgres [local] SELECT() [0x71e380]
+ postgres: postgresdba postgres [local] SELECT(standard_ExecutorRun+0x1d6) [0x71c1fe]
+ postgres: postgresdba postgres [local] SELECT(ExecutorRun+0x55) [0x71c026]
+ postgres: postgresdba postgres [local] SELECT() [0x953fc5]
+ postgres: postgresdba postgres [local] SELECT(PortalRun+0x262) [0x953c7e]
+ postgres: postgresdba postgres [local] SELECT() [0x94db78]
+ postgres: postgresdba postgres [local] SELECT(PostgresMain+0x7d7) [0x951e72]
+ postgres: postgresdba postgres [local] SELECT() [0x896b2f]
+ postgres: postgresdba postgres [local] SELECT() [0x8964b5]
+ postgres: postgresdba postgres [local] SELECT() [0x892a79]
+ postgres: postgresdba postgres [local] SELECT(PostmasterMain+0x116b) [0x892350]
+ postgres: postgresdba postgres [local] SELECT() [0x795f72]
+ /lib64/libc.so.6(__libc_start_main+0xf5) [0x7f2107bbd505]
+ postgres: postgresdba postgres [local] SELECT() [0x4842a9]
+</screen>
+ You can get the file name and line number by using gdb/addr2line in
+ linux platforms, as a prerequisite users must ensure gdb/addr2line is
+ already installed:
+<programlisting>
+1) "info line *address" from gdb on postgres executable. For example:
+gdb ./postgres
+(gdb) info line *0x71c25d
+Line 378 of "execMain.c" starts at address 0x71c25d <literal><</literal>standard_ExecutorRun+470<literal>></literal> and ends at 0x71c263 <literal><</literal>standard_ExecutorRun+476<literal>></literal>.
+OR
+2) Using "addr2line -e postgres address", For example:
+addr2line -e ./postgres 0x71c25d
+/home/postgresdba/src/backend/executor/execMain.c:378
+</programlisting>
+ If PostgreSQL was not built with the ability to capture backtrace. The
+ function will emit a warning and return false, for example:
+<screen>
+WARNING: backtrace generation is not supported by this installation
+HINT: You need to rebuild PostgreSQL using library containing backtrace_symbols.
+ pg_log_backtrace
+------------------
+ f
+(1 row)
+</screen>
+ </para>
+
</sect2>
<sect2 id="functions-admin-backup">
diff --git a/src/backend/catalog/system_functions.sql b/src/backend/catalog/system_functions.sql
index 54c93b16c4..6e598f9bdb 100644
--- a/src/backend/catalog/system_functions.sql
+++ b/src/backend/catalog/system_functions.sql
@@ -701,6 +701,8 @@ REVOKE EXECUTE ON FUNCTION pg_ls_dir(text,boolean,boolean) FROM public;
REVOKE EXECUTE ON FUNCTION pg_log_backend_memory_contexts(integer) FROM PUBLIC;
+REVOKE EXECUTE ON FUNCTION pg_log_backtrace(integer) FROM PUBLIC;
+
--
-- We also set up some things as accessible to standard roles.
--
diff --git a/src/backend/postmaster/autovacuum.c b/src/backend/postmaster/autovacuum.c
index 96332320a7..90111135ff 100644
--- a/src/backend/postmaster/autovacuum.c
+++ b/src/backend/postmaster/autovacuum.c
@@ -840,6 +840,10 @@ HandleAutoVacLauncherInterrupts(void)
if (LogMemoryContextPending)
ProcessLogMemoryContextInterrupt();
+ /* Process logging backtrace */
+ if (LogBacktracePending)
+ ProcessLogBacktraceInterrupt();
+
/* Process sinval catchup interrupts that happened while sleeping */
ProcessCatchupInterrupt();
}
diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index 6e69398cdd..3dac4a8f4b 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -612,6 +612,44 @@ ProcessBarrierPlaceholder(void)
return true;
}
+/*
+ * HandleLogBacktraceInterrupt - Handle receipt of an interrupt indicating
+ * logging of backtrace.
+ *
+ * All the actual work is deferred to ProcessLogBacktraceInterrupt(),
+ * because we cannot safely emit a log message inside the signal handler.
+ */
+static void
+HandleLogBacktraceInterrupt(void)
+{
+ InterruptPending = true;
+ LogBacktracePending = true;
+ /* latch will be set by procsignal_sigusr1_handler */
+}
+
+/*
+ * ProcessLogBacktraceInterrupt - Perform logging of backtrace of this
+ * backend process.
+ *
+ * Any backend that participates in ProcSignal signaling must arrange
+ * to call this function if we see LogBacktracePending set.
+ * It is called from CHECK_FOR_INTERRUPTS(), which is enough because
+ * the target process for logging of backtrace is a backend.
+ */
+void
+ProcessLogBacktraceInterrupt(void)
+{
+ LogBacktracePending = false;
+
+ /*
+ * Use LOG_SERVER_ONLY to prevent this message from being sent to the
+ * connected client.
+ */
+ ereport(LOG_SERVER_ONLY,
+ (errmsg("logging backtrace of PID %d", MyProcPid)));
+ set_backtrace(NULL, 0);
+}
+
/*
* CheckProcSignal - check to see if a particular reason has been
* signaled, and clear the signal flag. Should be called after receiving
@@ -661,6 +699,9 @@ procsignal_sigusr1_handler(SIGNAL_ARGS)
if (CheckProcSignal(PROCSIG_LOG_MEMORY_CONTEXT))
HandleLogMemoryContextInterrupt();
+ if (CheckProcSignal(PROCSIG_LOG_BACKTRACE))
+ HandleLogBacktraceInterrupt();
+
if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_DATABASE))
RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_DATABASE);
diff --git a/src/backend/storage/ipc/signalfuncs.c b/src/backend/storage/ipc/signalfuncs.c
index de69d60e79..d042feefee 100644
--- a/src/backend/storage/ipc/signalfuncs.c
+++ b/src/backend/storage/ipc/signalfuncs.c
@@ -23,6 +23,7 @@
#include "storage/pmsignal.h"
#include "storage/proc.h"
#include "storage/procarray.h"
+#include "tcop/tcopprot.h"
#include "utils/acl.h"
#include "utils/builtins.h"
@@ -298,3 +299,51 @@ pg_rotate_logfile_v2(PG_FUNCTION_ARGS)
SendPostmasterSignal(PMSIGNAL_ROTATE_LOGFILE);
PG_RETURN_BOOL(true);
}
+
+/*
+ * pg_log_backtrace - log backtrace of backend process.
+ *
+ * By default, only superusers can log backtrace. Additional roles can be
+ * permitted with GRANT.
+ */
+Datum
+pg_log_backtrace(PG_FUNCTION_ARGS)
+{
+#ifdef HAVE_BACKTRACE_SYMBOLS
+ int pid = PG_GETARG_INT32(0);
+ PGPROC *proc;
+
+ /*
+ * BackendPidGetProc returns NULL if the pid isn't valid; but by the time
+ * we reach kill(), a process for which we get a valid proc here might have
+ * terminated on its own. There's no way to acquire a lock on an arbitrary
+ * process to prevent that. But since this mechanism is usually used to
+ * debug a backend running and consuming lots of CPU cycles, that it might
+ * end on its own first and its backtrace are not logged is not a problem.
+ */
+ proc = BackendPidGetProc(pid);
+ if (proc == NULL)
+ {
+ ereport(WARNING,
+ (errmsg("PID %d is not a PostgreSQL server process", pid)));
+ PG_RETURN_BOOL(false);
+ }
+
+ /*
+ * Send SIGUSR1 to postgres backend whose pid matches pid by
+ * setting PROCSIG_LOG_BACKTRACE, the backend process will log
+ * the backtrace once the signal is received.
+ */
+ if (!SendProcSignal(pid, PROCSIG_LOG_BACKTRACE, proc->backendId))
+ PG_RETURN_BOOL(true);
+ else
+ ereport(WARNING,
+ (errmsg("could not send signal to process %d: %m", pid))); /* return false below */
+#else
+ ereport(WARNING,
+ errmsg("backtrace generation is not supported by this installation"),
+ errhint("You need to rebuild PostgreSQL using library containing backtrace_symbols."));
+#endif
+
+ PG_RETURN_BOOL(false);
+}
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index 0775abe35d..3a2b670d23 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -3366,6 +3366,10 @@ ProcessInterrupts(void)
if (LogMemoryContextPending)
ProcessLogMemoryContextInterrupt();
+
+ /* Process logging backtrace */
+ if (LogBacktracePending)
+ ProcessLogBacktraceInterrupt();
}
diff --git a/src/backend/utils/error/elog.c b/src/backend/utils/error/elog.c
index f33729513a..6166965a39 100644
--- a/src/backend/utils/error/elog.c
+++ b/src/backend/utils/error/elog.c
@@ -172,7 +172,6 @@ static char formatted_log_time[FORMATTED_TS_LEN];
static const char *err_gettext(const char *str) pg_attribute_format_arg(1);
-static pg_noinline void set_backtrace(ErrorData *edata, int num_skip);
static void set_errdata_field(MemoryContextData *cxt, char **ptr, const char *str);
static void write_console(const char *line, int len);
static void setup_formatted_log_time(void);
@@ -949,9 +948,10 @@ errbacktrace(void)
* Compute backtrace data and add it to the supplied ErrorData. num_skip
* specifies how many inner frames to skip. Use this to avoid showing the
* internal backtrace support functions in the backtrace. This requires that
- * this and related functions are not inlined.
+ * this and related functions are not inlined. If edata pointer is valid
+ * backtrace information will be set in edata.
*/
-static void
+void
set_backtrace(ErrorData *edata, int num_skip)
{
StringInfoData errtrace;
@@ -978,7 +978,19 @@ set_backtrace(ErrorData *edata, int num_skip)
"backtrace generation is not supported by this installation");
#endif
- edata->backtrace = errtrace.data;
+ if (edata)
+ edata->backtrace = errtrace.data;
+ else
+ {
+ /*
+ * LOG_SERVER_ONLY is used intentionally to make sure this information
+ * is not sent to client based on client_min_messages. We don't want
+ * to mess up a different session as pg_log_backtrace will be
+ * sending SIGNAL to a different backend.
+ */
+ elog(LOG_SERVER_ONLY, "current backtrace:%s", errtrace.data);
+ pfree(errtrace.data);
+ }
}
/*
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 381d9e548d..66570a902c 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -36,6 +36,7 @@ volatile sig_atomic_t IdleInTransactionSessionTimeoutPending = false;
volatile sig_atomic_t IdleSessionTimeoutPending = false;
volatile sig_atomic_t ProcSignalBarrierPending = false;
volatile sig_atomic_t LogMemoryContextPending = false;
+volatile sig_atomic_t LogBacktracePending = false;
volatile uint32 InterruptHoldoffCount = 0;
volatile uint32 QueryCancelHoldoffCount = 0;
volatile uint32 CritSectionCount = 0;
diff --git a/src/include/catalog/catversion.h b/src/include/catalog/catversion.h
index 49e8e59129..9f3e7b7503 100644
--- a/src/include/catalog/catversion.h
+++ b/src/include/catalog/catversion.h
@@ -53,6 +53,6 @@
*/
/* yyyymmddN */
-#define CATALOG_VERSION_NO 202111091
+#define CATALOG_VERSION_NO 202111141
#endif
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index d068d6532e..91551bb2ee 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -11689,4 +11689,9 @@
prorettype => 'bytea', proargtypes => 'pg_brin_minmax_multi_summary',
prosrc => 'brin_minmax_multi_summary_send' },
+# function to get the backtrace of server process
+{ oid => '6105', descr => 'log backtrace of process',
+ proname => 'pg_log_backtrace', provolatile => 'v', prorettype => 'bool',
+ proargtypes => 'int4', prosrc => 'pg_log_backtrace' },
+
]
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 90a3016065..8a7f5fba54 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -94,6 +94,7 @@ extern PGDLLIMPORT volatile sig_atomic_t IdleInTransactionSessionTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t IdleSessionTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcSignalBarrierPending;
extern PGDLLIMPORT volatile sig_atomic_t LogMemoryContextPending;
+extern PGDLLIMPORT volatile sig_atomic_t LogBacktracePending;
extern PGDLLIMPORT volatile sig_atomic_t CheckClientConnectionPending;
extern PGDLLIMPORT volatile sig_atomic_t ClientConnectionLost;
diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h
index eec186be2e..e14763820f 100644
--- a/src/include/storage/procsignal.h
+++ b/src/include/storage/procsignal.h
@@ -35,6 +35,7 @@ typedef enum
PROCSIG_WALSND_INIT_STOPPING, /* ask walsenders to prepare for shutdown */
PROCSIG_BARRIER, /* global barrier interrupt */
PROCSIG_LOG_MEMORY_CONTEXT, /* ask backend to log the memory contexts */
+ PROCSIG_LOG_BACKTRACE, /* ask backend to log the current backtrace */
/* Recovery conflict reasons */
PROCSIG_RECOVERY_CONFLICT_DATABASE,
@@ -70,7 +71,7 @@ extern int SendProcSignal(pid_t pid, ProcSignalReason reason,
extern uint64 EmitProcSignalBarrier(ProcSignalBarrierType type);
extern void WaitForProcSignalBarrier(uint64 generation);
extern void ProcessProcSignalBarrier(void);
-
+extern void ProcessLogBacktraceInterrupt(void);
extern void procsignal_sigusr1_handler(SIGNAL_ARGS);
#endif /* PROCSIGNAL_H */
diff --git a/src/include/utils/elog.h b/src/include/utils/elog.h
index f53607e12e..c63d25716a 100644
--- a/src/include/utils/elog.h
+++ b/src/include/utils/elog.h
@@ -453,4 +453,6 @@ extern void set_syslog_parameters(const char *ident, int facility);
*/
extern void write_stderr(const char *fmt,...) pg_attribute_printf(1, 2);
+extern pg_noinline void set_backtrace(ErrorData *edata, int num_skip);
+
#endif /* ELOG_H */
diff --git a/src/test/regress/expected/backtrace.out b/src/test/regress/expected/backtrace.out
new file mode 100644
index 0000000000..88ee383d4d
--- /dev/null
+++ b/src/test/regress/expected/backtrace.out
@@ -0,0 +1,42 @@
+--
+-- pg_log_backtrace()
+--
+-- Backtrace are logged and they are not returned to the function.
+-- Furthermore, their contents can vary depending on the timing. However,
+-- we can at least verify that the code doesn't fail, and that the
+-- permissions are set properly.
+--
+SELECT pg_log_backtrace(pg_backend_pid());
+ pg_log_backtrace
+------------------
+ t
+(1 row)
+
+CREATE ROLE regress_log_backtrace;
+SELECT has_function_privilege('regress_log_backtrace',
+ 'pg_log_backtrace(integer)', 'EXECUTE'); -- no
+ has_function_privilege
+------------------------
+ f
+(1 row)
+
+GRANT EXECUTE ON FUNCTION pg_log_backtrace(integer)
+ TO regress_log_backtrace;
+SELECT has_function_privilege('regress_log_backtrace',
+ 'pg_log_backtrace(integer)', 'EXECUTE'); -- yes
+ has_function_privilege
+------------------------
+ t
+(1 row)
+
+SET ROLE regress_log_backtrace;
+SELECT pg_log_backtrace(pg_backend_pid());
+ pg_log_backtrace
+------------------
+ t
+(1 row)
+
+RESET ROLE;
+REVOKE EXECUTE ON FUNCTION pg_log_backtrace(integer)
+ FROM regress_log_backtrace;
+DROP ROLE regress_log_backtrace;
diff --git a/src/test/regress/expected/backtrace_1.out b/src/test/regress/expected/backtrace_1.out
new file mode 100644
index 0000000000..62beef8f51
--- /dev/null
+++ b/src/test/regress/expected/backtrace_1.out
@@ -0,0 +1,46 @@
+--
+-- pg_log_backtrace()
+--
+-- Backtrace are logged and they are not returned to the function.
+-- Furthermore, their contents can vary depending on the timing. However,
+-- we can at least verify that the code doesn't fail, and that the
+-- permissions are set properly.
+--
+SELECT pg_log_backtrace(pg_backend_pid());
+WARNING: backtrace generation is not supported by this installation
+HINT: You need to rebuild PostgreSQL using library containing backtrace_symbols.
+ pg_log_backtrace
+------------------
+ f
+(1 row)
+
+CREATE ROLE regress_log_backtrace;
+SELECT has_function_privilege('regress_log_backtrace',
+ 'pg_log_backtrace(integer)', 'EXECUTE'); -- no
+ has_function_privilege
+------------------------
+ f
+(1 row)
+
+GRANT EXECUTE ON FUNCTION pg_log_backtrace(integer)
+ TO regress_log_backtrace;
+SELECT has_function_privilege('regress_log_backtrace',
+ 'pg_log_backtrace(integer)', 'EXECUTE'); -- yes
+ has_function_privilege
+------------------------
+ t
+(1 row)
+
+SET ROLE regress_log_backtrace;
+SELECT pg_log_backtrace(pg_backend_pid());
+WARNING: backtrace generation is not supported by this installation
+HINT: You need to rebuild PostgreSQL using library containing backtrace_symbols.
+ pg_log_backtrace
+------------------
+ f
+(1 row)
+
+RESET ROLE;
+REVOKE EXECUTE ON FUNCTION pg_log_backtrace(integer)
+ FROM regress_log_backtrace;
+DROP ROLE regress_log_backtrace;
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index 017e962fed..ca8b2a6d8e 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -86,7 +86,7 @@ test: brin_bloom brin_multi
# ----------
# Another group of parallel tests
# ----------
-test: create_table_like alter_generic alter_operator misc async dbsize misc_functions sysviews tsrf tid tidscan tidrangescan collate.icu.utf8 incremental_sort
+test: create_table_like alter_generic alter_operator misc async dbsize misc_functions sysviews tsrf tid tidscan tidrangescan collate.icu.utf8 incremental_sort backtrace
# rules cannot run concurrently with any test that creates
# a view or rule in the public schema
diff --git a/src/test/regress/sql/backtrace.sql b/src/test/regress/sql/backtrace.sql
new file mode 100644
index 0000000000..2688bc7330
--- /dev/null
+++ b/src/test/regress/sql/backtrace.sql
@@ -0,0 +1,30 @@
+--
+-- pg_log_backtrace()
+--
+-- Backtrace are logged and they are not returned to the function.
+-- Furthermore, their contents can vary depending on the timing. However,
+-- we can at least verify that the code doesn't fail, and that the
+-- permissions are set properly.
+--
+
+SELECT pg_log_backtrace(pg_backend_pid());
+
+CREATE ROLE regress_log_backtrace;
+
+SELECT has_function_privilege('regress_log_backtrace',
+ 'pg_log_backtrace(integer)', 'EXECUTE'); -- no
+
+GRANT EXECUTE ON FUNCTION pg_log_backtrace(integer)
+ TO regress_log_backtrace;
+
+SELECT has_function_privilege('regress_log_backtrace',
+ 'pg_log_backtrace(integer)', 'EXECUTE'); -- yes
+
+SET ROLE regress_log_backtrace;
+SELECT pg_log_backtrace(pg_backend_pid());
+RESET ROLE;
+
+REVOKE EXECUTE ON FUNCTION pg_log_backtrace(integer)
+ FROM regress_log_backtrace;
+
+DROP ROLE regress_log_backtrace;
--
2.30.2
From fea4198a289d20d4ec1cb0523d8ef91c5d298ce7 Mon Sep 17 00:00:00 2001
From: Vignesh C <vignes...@gmail.com>
Date: Wed, 17 Nov 2021 08:40:18 +0530
Subject: [PATCH v14 2/2] pg_log_backtrace support for logging backtrace of aux
procs
Enhanced pg_log_backtrace to support logging backtrace of auxiliary
processes such as bgwriter, checkpointer, wal writer, archiver, startup
process and wal receiver. It will be useful to look at the backtrace of
these processes too, for debugging purposes and to check if the process
is genuinely taking time or if it is a stuck process. Inside the code,
we could use the AuxiliaryPidGetProc() to get the PGPROC of these
processes. Note that, neither AuxiliaryPidGetProc() nor
---
doc/src/sgml/func.sgml | 26 ++++++++++-------
src/backend/postmaster/checkpointer.c | 4 +++
src/backend/postmaster/interrupt.c | 4 +++
src/backend/postmaster/pgarch.c | 4 +++
src/backend/postmaster/startup.c | 4 +++
src/backend/postmaster/walwriter.c | 4 +++
src/backend/storage/ipc/procsignal.c | 3 +-
src/backend/storage/ipc/signalfuncs.c | 35 ++++++++++++++++-------
src/test/regress/expected/backtrace.out | 11 +++++++
src/test/regress/expected/backtrace_1.out | 13 +++++++++
src/test/regress/sql/backtrace.sql | 9 ++++++
11 files changed, 95 insertions(+), 22 deletions(-)
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index a88583edfd..7e245711da 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25355,19 +25355,25 @@ SELECT collation for ('foo' COLLATE "de_DE");
</para>
<para>
Requests to log the backtrace of the
- <glossterm linkend="glossary-backend">backend</glossterm>
- with the specified process ID. The backtrace will be logged at message
- level <literal>LOG</literal>. It will appear in the server log based on
- the log configuration set (See <xref linkend="runtime-config-logging"/>
- for more information), but will not be sent to the client regardless of
+ <glossterm linkend="glossary-backend">backend</glossterm> or the
+ <glossterm linkend="glossary-wal-sender">WAL sender</glossterm> or the
+ <glossterm linkend="glossary-auxiliary-proc">auxiliary process</glossterm>
+ with the specified process ID. This function cannot request
+ <glossterm linkend="glossary-postmaster">postmaster process</glossterm> or
+ <glossterm linkend="glossary-logger">logger</glossterm> or
+ <glossterm linkend="glossary-stats-collector">statistics collector</glossterm>
+ (all other <glossterm linkend="glossary-auxiliary-proc">auxiliary processes</glossterm>
+ it can) for the backtrace. The backtrace will be logged at
+ <literal>LOG</literal> message level. They will appear in the server
+ log based on the log configuration set (See
+ <xref linkend="runtime-config-logging"/> for more information), but
+ will not be sent to the client regardless of
<xref linkend="guc-client-min-messages"/>. A backtrace will identify
where exactly the backend process is currently executing. This may be
useful to developers to diagnose stuck processes and other problems.
- This feature is not supported for the postmaster, logger, checkpointer,
- walwriter, background writer or statistics collector process. This
- feature will be available if PostgreSQL was built with the ability to
- capture backtrace. If not available, the function will emit a warning
- and return false.
+ This feature will be available if PostgreSQL was built with the
+ ability to capture backtrace. If not available, the function will emit
+ a warning and return false.
</para></entry>
</row>
diff --git a/src/backend/postmaster/checkpointer.c b/src/backend/postmaster/checkpointer.c
index be7366379d..7a25557702 100644
--- a/src/backend/postmaster/checkpointer.c
+++ b/src/backend/postmaster/checkpointer.c
@@ -581,6 +581,10 @@ HandleCheckpointerInterrupts(void)
/* Normal exit from the checkpointer is here */
proc_exit(0); /* done */
}
+
+ /* Process logging backtrace */
+ if (LogBacktracePending)
+ ProcessLogBacktraceInterrupt();
}
/*
diff --git a/src/backend/postmaster/interrupt.c b/src/backend/postmaster/interrupt.c
index dd9136a942..6298b48da3 100644
--- a/src/backend/postmaster/interrupt.c
+++ b/src/backend/postmaster/interrupt.c
@@ -43,6 +43,10 @@ HandleMainLoopInterrupts(void)
if (ShutdownRequestPending)
proc_exit(0);
+
+ /* Process logging backtrace */
+ if (LogBacktracePending)
+ ProcessLogBacktraceInterrupt();
}
/*
diff --git a/src/backend/postmaster/pgarch.c b/src/backend/postmaster/pgarch.c
index 3b33e01d95..938c08b0eb 100644
--- a/src/backend/postmaster/pgarch.c
+++ b/src/backend/postmaster/pgarch.c
@@ -856,4 +856,8 @@ HandlePgArchInterrupts(void)
ConfigReloadPending = false;
ProcessConfigFile(PGC_SIGHUP);
}
+
+ /* Process logging backtrace */
+ if (LogBacktracePending)
+ ProcessLogBacktraceInterrupt();
}
diff --git a/src/backend/postmaster/startup.c b/src/backend/postmaster/startup.c
index 47ec737888..41d40358bf 100644
--- a/src/backend/postmaster/startup.c
+++ b/src/backend/postmaster/startup.c
@@ -200,6 +200,10 @@ HandleStartupProcInterrupts(void)
/* Process barrier events */
if (ProcSignalBarrierPending)
ProcessProcSignalBarrier();
+
+ /* Process logging backtrace */
+ if (LogBacktracePending)
+ ProcessLogBacktraceInterrupt();
}
diff --git a/src/backend/postmaster/walwriter.c b/src/backend/postmaster/walwriter.c
index 626fae8454..8bc404d17b 100644
--- a/src/backend/postmaster/walwriter.c
+++ b/src/backend/postmaster/walwriter.c
@@ -306,4 +306,8 @@ HandleWalWriterInterrupts(void)
proc_exit(0);
}
+
+ /* Process logging backtrace */
+ if (LogBacktracePending)
+ ProcessLogBacktraceInterrupt();
}
diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index 3dac4a8f4b..e70bfce0bd 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -633,8 +633,7 @@ HandleLogBacktraceInterrupt(void)
*
* Any backend that participates in ProcSignal signaling must arrange
* to call this function if we see LogBacktracePending set.
- * It is called from CHECK_FOR_INTERRUPTS(), which is enough because
- * the target process for logging of backtrace is a backend.
+ * CHECK_FOR_INTERRUPTS() or from process specific interrupt handlers.
*/
void
ProcessLogBacktraceInterrupt(void)
diff --git a/src/backend/storage/ipc/signalfuncs.c b/src/backend/storage/ipc/signalfuncs.c
index d042feefee..21441c64b8 100644
--- a/src/backend/storage/ipc/signalfuncs.c
+++ b/src/backend/storage/ipc/signalfuncs.c
@@ -312,29 +312,44 @@ pg_log_backtrace(PG_FUNCTION_ARGS)
#ifdef HAVE_BACKTRACE_SYMBOLS
int pid = PG_GETARG_INT32(0);
PGPROC *proc;
+ PGPROC *aux_proc = NULL;
+ BackendId backendId = InvalidBackendId;
/*
- * BackendPidGetProc returns NULL if the pid isn't valid; but by the time
- * we reach kill(), a process for which we get a valid proc here might have
- * terminated on its own. There's no way to acquire a lock on an arbitrary
- * process to prevent that. But since this mechanism is usually used to
- * debug a backend running and consuming lots of CPU cycles, that it might
- * end on its own first and its backtrace are not logged is not a problem.
+ * BackendPidGetProc or AuxiliaryPidGetProc returns NULL if the pid isn't
+ * valid; but by the time we reach kill(), a process for which we get a
+ * valid proc here might have terminated on its own. There's no way to
+ * acquire a lock on an arbitrary process to prevent that. But since this
+ * mechanism is usually used to debug a backend running and consuming lots
+ * of CPU cycles, that it might end on its own first and its backtrace are
+ * not logged is not a problem.
*/
proc = BackendPidGetProc(pid);
if (proc == NULL)
{
- ereport(WARNING,
- (errmsg("PID %d is not a PostgreSQL server process", pid)));
- PG_RETURN_BOOL(false);
+ /* See if the process with given pid is an auxiliary process. */
+ aux_proc = AuxiliaryPidGetProc(pid);
+ if (aux_proc == NULL)
+ {
+ ereport(WARNING,
+ (errmsg("PID %d is not a PostgreSQL server process", pid)));
+ PG_RETURN_BOOL(false);
+ }
}
+ /*
+ * Only regular backends will have valid backend id, auxiliary processes
+ * don't.
+ */
+ if (!aux_proc)
+ backendId = proc->backendId;
+
/*
* Send SIGUSR1 to postgres backend whose pid matches pid by
* setting PROCSIG_LOG_BACKTRACE, the backend process will log
* the backtrace once the signal is received.
*/
- if (!SendProcSignal(pid, PROCSIG_LOG_BACKTRACE, proc->backendId))
+ if (!SendProcSignal(pid, PROCSIG_LOG_BACKTRACE, backendId))
PG_RETURN_BOOL(true);
else
ereport(WARNING,
diff --git a/src/test/regress/expected/backtrace.out b/src/test/regress/expected/backtrace.out
index 88ee383d4d..52aec7088c 100644
--- a/src/test/regress/expected/backtrace.out
+++ b/src/test/regress/expected/backtrace.out
@@ -12,6 +12,17 @@ SELECT pg_log_backtrace(pg_backend_pid());
t
(1 row)
+CREATE FUNCTION get_proc_pid(text)
+RETURNS int
+LANGUAGE SQL
+AS 'SELECT pid FROM pg_stat_activity WHERE backend_type = $1';
+SELECT pg_log_backtrace(get_proc_pid('checkpointer'));
+ pg_log_backtrace
+------------------
+ t
+(1 row)
+
+DROP FUNCTION get_proc_pid(text);
CREATE ROLE regress_log_backtrace;
SELECT has_function_privilege('regress_log_backtrace',
'pg_log_backtrace(integer)', 'EXECUTE'); -- no
diff --git a/src/test/regress/expected/backtrace_1.out b/src/test/regress/expected/backtrace_1.out
index 62beef8f51..cfe51e1329 100644
--- a/src/test/regress/expected/backtrace_1.out
+++ b/src/test/regress/expected/backtrace_1.out
@@ -14,6 +14,19 @@ HINT: You need to rebuild PostgreSQL using library containing backtrace_symbols
f
(1 row)
+CREATE FUNCTION get_proc_pid(text)
+RETURNS int
+LANGUAGE SQL
+AS 'SELECT pid FROM pg_stat_activity WHERE backend_type = $1';
+SELECT pg_log_backtrace(get_proc_pid('checkpointer'));
+WARNING: backtrace generation is not supported by this installation
+HINT: You need to rebuild PostgreSQL using library containing backtrace_symbols.
+ pg_log_backtrace
+------------------
+ f
+(1 row)
+
+DROP FUNCTION get_proc_pid(text);
CREATE ROLE regress_log_backtrace;
SELECT has_function_privilege('regress_log_backtrace',
'pg_log_backtrace(integer)', 'EXECUTE'); -- no
diff --git a/src/test/regress/sql/backtrace.sql b/src/test/regress/sql/backtrace.sql
index 2688bc7330..11f52c7d29 100644
--- a/src/test/regress/sql/backtrace.sql
+++ b/src/test/regress/sql/backtrace.sql
@@ -9,6 +9,15 @@
SELECT pg_log_backtrace(pg_backend_pid());
+CREATE FUNCTION get_proc_pid(text)
+RETURNS int
+LANGUAGE SQL
+AS 'SELECT pid FROM pg_stat_activity WHERE backend_type = $1';
+
+SELECT pg_log_backtrace(get_proc_pid('checkpointer'));
+
+DROP FUNCTION get_proc_pid(text);
+
CREATE ROLE regress_log_backtrace;
SELECT has_function_privilege('regress_log_backtrace',
--
2.30.2
