On 2021-03-23 17:24, Kyotaro Horiguchi wrote:
Thanks for reviewing and suggestions!
At Mon, 22 Mar 2021 15:09:58 +0900, torikoshia
<torikos...@oss.nttdata.com> wrote in
>> If MemoryContextStatsPrint(), i.e. showing 100 children at most is
>> enough, this hard limit may be acceptable.
> Can't this number be passed via shared memory?
The attached patch uses static shared memory to pass the number.
"pg_print_backend_memory_contexts"
That name looks like as if it returns the result as text when used on
command-line. We could have pg_get_backend_memory_context(bool
dump_to_log (or where to dump), int limit). Or couldn't we name it
differently even in the ase we add a separate function?
Redefined pg_get_backend_memory_contexts() as
pg_get_backend_memory_contexts(pid, int max_children).
When pid equals 0, pg_get_backend_memory_contexts() prints local memory
contexts as original pg_get_backend_memory_contexts() does.
In this case, 'max_children' is ignored.
When 'pid' does not equal 0 and it is the PID of the client backend,
memory contexts are logged through elog().
+/*
+ * MaxChildrenPerContext
+ * Max number of children to print per one parent context.
+ */
+int *MaxChildrenPerContext = NULL;
Perhaps it'd be better to have a struct even if it consists only of
one member. (Aligned) C-int values are atomic so we can omit the
McxtPrintLock. (I don't think it's a problem even if it is modifed
while reading^^:)
Fixed them.
+ if(max_children <= 0)
+ {
+ ereport(WARNING,
+ (errmsg("%d is invalid value",
max_children),
+ errhint("second parameter is the number
of context and it must
be set to a value greater than or equal to 1")));
It's annoying to choose a number large enough when I want to dump
children unlimitedly. Couldn't we use 0 to specify "unlimited"?
Modified as you suggested.
+ (errmsg("%d is invalid value",
max_children),
+ errhint("second parameter is the number
of context and it must
be set to a value greater than or equal to 1")));
For the main message, (I think) we usually spell the "%d is invalid
value" as "maximum number of children must be positive" or such. For
the hint, we don't need a copy of the primary section of the
documentation here.
Modified it to "The maximum number of children must be greater than 0".
I think we should ERROR out for invalid parameters, at least for
max_children. I'm not sure about pid since we might call it based on
pg_stat_activity..
Changed to ERROR out when the 'max_children' is less than 0.
Regarding pid, I left it untouched considering the consistency with
other
signal sending functions such as pg_cancel_backend().
+ if(!SendProcSignal(pid, PROCSIG_PRINT_MEMORY_CONTEXT,
InvalidBackendId))
We know the backendid of the process here.
Added it.
+ if (is_dst_stderr)
+ {
+ for (i = 0; i <= level; i++)
+ fprintf(stderr, " ");
The fprintf path is used nowhere in the patch at all. It can be used
while attaching debugger but I'm not sure we need that code. The
footprint of this patch is largely shrinked by removing it.
According to the past discussion[1], people wanted MemoryContextStats
as it was, so I think it's better that MemoryContextStats can be used
as before.
+ strcat(truncated_ident, delimiter);
strcpy is sufficient here. And we don't need the delimiter to be a
variable. (we can copy a string literal into truncate_ident, then
count the length of truncate_ident, instead of the delimiter
variable.)
True.
+ $current_logfiles = slurp_file($node->data_dir .
'/current_logfiles');
...
+my $lfname = $current_logfiles;
+$lfname =~ s/^stderr //;
+chomp $lfname;
$node->logfile is the current log file name.
+ 'target PID is not PostgreSQL server process');
Maybe "check if PID check is working" or such? And, we can do
something like the following to exercise in a more practical way.
select pg_print_backend...(pid,) from pg_stat_activity where
backend_type = 'checkpointer';
It seems better.
As documented, the current implementation allows that when multiple
pg_print_backend_memory_contexts() called in succession or
simultaneously, max_children can be the one of another
pg_print_backend_memory_contexts().
I had tried to avoid this by adding some state information and using
before_shmem_exit() in case of process termination for cleaning up the
state information as in the patch I presented earlier, but since
kill()
returns success before the dumper called signal handler, it seemed
there were times when we couldn't clean up the state.
Since this happens only when multiple
pg_print_backend_memory_contexts()
are called and their specified number of children are different, and
the
effect is just the not intended number of children to print, it might
be
acceptable.
I see it as a non-issue. Even though the behavior looks somewhat
strange, that usage is stranger than the behavior.
Thanks for your comments!
Or it might be better to wait for some seconds if num_chilren on
shared
memory is not the initialized value(meaning some other process is
requesting to print memory contexts).
>> Only superusers can call pg_print_backend_memory_contexts().
> + /* Only allow superusers to signal superuser-owned backends. */
> + if (superuser_arg(proc->roleId) && !superuser())
> The patch seems to allow even non-superuser to request to print the
> memory
> contexts if the target backend is owned by non-superuser. Is this
> intentional?
> I think that only superuser should be allowed to execute
> pg_print_backend_memory_contexts() whoever owns the target backend.
> Because that function can cause lots of log messages.
Thanks, it's not intentional, modified it.
By the way we can send a procsig to other than a client backend. And
most of the postgres processes are using the standard signal handler
and intializes the procsig facility. So some of such kind of processes
can dump the memory context information as-is. Otherwise we can add
CHECK_FOR_INTERRUPT to appropriate place to allow that. I'm not sure
how it is useful for other kind of processes, but it might be useful
for autovacuum workers.
Yeah, I also think it's possible to get memory contexts other than the
client backend.
But I'm not sure whether people want to do it...
[1] https://www.postgresql.org/message-id/906.1513707472%40sss.pgh.pa.us
regards.
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 1d3429fbd9..a4017a0760 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -24821,6 +24821,37 @@ SELECT collation for ('foo' COLLATE "de_DE");
</para></entry>
</row>
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_get_backend_memory_contexts</primary>
+ </indexterm>
+ <function>pg_get_backend_memory_contexts</function> (
+ <parameter>pid</parameter> <type>integer</type>,
+ <parameter>max_children</parameter> <type>integer</type> )
+ <returnvalue>setof record</returnvalue>
+ </para>
+ <para>
+ Get memory contexts whose backend process has the specified process ID.
+ <parameter>max_children</parameter> limits the max number of children
+ to print per one parent context. 0 means unlimited.
+ When <parameter>pid</parameter> equals 0,
+ <function>pg_get_backend_memory_contexts</function> displays all
+ the memory contexts of the local process regardless of
+ <parameter>max_children</parameter>.
+ When <parameter>pid</parameter> does not equal 0,
+ memory contexts will be printed based on the log configuration set.
+ See <xref linkend="runtime-config-logging"/> for more information.
+ Only superusers can call this function even when the specified process
+ is non-superuser backend.
+ Note that when multiple
+ <function>pg_get_backend_memory_contexts</function> called in
+ succession or simultaneously, <parameter>max_children</parameter> can
+ be the one of another
+ <function>pg_get_backend_memory_contexts</function>.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 0dca65dc7b..48a1a0e958 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -555,10 +555,10 @@ REVOKE ALL ON pg_shmem_allocations FROM PUBLIC;
REVOKE EXECUTE ON FUNCTION pg_get_shmem_allocations() FROM PUBLIC;
CREATE VIEW pg_backend_memory_contexts AS
- SELECT * FROM pg_get_backend_memory_contexts();
+ SELECT * FROM pg_get_backend_memory_contexts(0, 0);
REVOKE ALL ON pg_backend_memory_contexts FROM PUBLIC;
-REVOKE EXECUTE ON FUNCTION pg_get_backend_memory_contexts() FROM PUBLIC;
+REVOKE EXECUTE ON FUNCTION pg_get_backend_memory_contexts FROM PUBLIC;
-- Statistics views
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
index 3e4ec53a97..ed5393324a 100644
--- a/src/backend/storage/ipc/ipci.c
+++ b/src/backend/storage/ipc/ipci.c
@@ -46,6 +46,7 @@
#include "storage/sinvaladt.h"
#include "storage/spin.h"
#include "utils/snapmgr.h"
+#include "utils/memutils.h"
/* GUCs */
int shared_memory_type = DEFAULT_SHARED_MEMORY_TYPE;
@@ -269,6 +270,7 @@ CreateSharedMemoryAndSemaphores(void)
BTreeShmemInit();
SyncScanShmemInit();
AsyncShmemInit();
+ McxtLogShmemInit();
#ifdef EXEC_BACKEND
diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c
index c6a8d4611e..c61d5079e2 100644
--- a/src/backend/storage/ipc/procsignal.c
+++ b/src/backend/storage/ipc/procsignal.c
@@ -30,6 +30,7 @@
#include "storage/shmem.h"
#include "storage/sinval.h"
#include "tcop/tcopprot.h"
+#include "utils/memutils.h"
/*
* The SIGUSR1 signal is multiplexed to support signaling multiple event
@@ -440,6 +441,20 @@ HandleProcSignalBarrierInterrupt(void)
/* latch will be set by procsignal_sigusr1_handler */
}
+/*
+ * HandleProcSignalLogMemoryContext
+ *
+ * Handle receipt of an interrupt indicating logging memory context.
+ * Signal handler portion of interrupt handling.
+ */
+static void
+HandleProcSignalLogMemoryContext(void)
+{
+ InterruptPending = true;
+ LogMemoryContextPending = true;
+ /* latch will be set by procsignal_sigusr1_handler */
+}
+
/*
* Perform global barrier related interrupt checking.
*
@@ -580,6 +595,27 @@ ProcessProcSignalBarrier(void)
ConditionVariableBroadcast(&MyProcSignalSlot->pss_barrierCV);
}
+/*
+ * ProcessLogMemoryContextInterrupt
+ * The portion of logging memory context interrupt handling that runs
+ * outside of the signal handler.
+ */
+void
+ProcessLogMemoryContextInterrupt(void)
+{
+ int max_children;
+
+ LogMemoryContextPending = false;
+
+ max_children = mcxtLogData->maxChildrenPerContext;
+
+ ereport(LOG,
+ (errmsg("Logging memory contexts of PID %d. Max number of children per context is %d", MyProcPid, max_children)));
+
+ MemoryContextStatsDetail(TopMemoryContext, max_children, false);
+}
+
+
/*
* If it turns out that we couldn't absorb one or more barrier types, either
* because the barrier-processing functions returned false or due to an error,
@@ -675,6 +711,9 @@ procsignal_sigusr1_handler(SIGNAL_ARGS)
if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_BUFFERPIN))
RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_BUFFERPIN);
+ if (CheckProcSignal(PROCSIG_LOG_MEMORY_CONTEXT))
+ HandleProcSignalLogMemoryContext();
+
SetLatch(MyLatch);
errno = save_errno;
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
index 2b1b68109f..afaf3b1cce 100644
--- a/src/backend/tcop/postgres.c
+++ b/src/backend/tcop/postgres.c
@@ -3295,6 +3295,9 @@ ProcessInterrupts(void)
if (ParallelMessagePending)
HandleParallelMessages();
+
+ if (LogMemoryContextPending)
+ ProcessLogMemoryContextInterrupt();
}
diff --git a/src/backend/utils/adt/mcxtfuncs.c b/src/backend/utils/adt/mcxtfuncs.c
index c02fa47550..de2040640c 100644
--- a/src/backend/utils/adt/mcxtfuncs.c
+++ b/src/backend/utils/adt/mcxtfuncs.c
@@ -19,6 +19,7 @@
#include "miscadmin.h"
#include "mb/pg_wchar.h"
#include "utils/builtins.h"
+#include "utils/mcxtfuncs.h"
/* ----------
* The max bytes for showing identifiers of MemoryContext.
@@ -61,7 +62,7 @@ PutMemoryContextsStatsTupleStore(Tuplestorestate *tupstore,
/* Examine the context itself */
memset(&stat, 0, sizeof(stat));
- (*context->methods->stats) (context, NULL, (void *) &level, &stat);
+ (*context->methods->stats) (context, NULL, (void *) &level, &stat, true);
memset(values, 0, sizeof(values));
memset(nulls, 0, sizeof(nulls));
@@ -117,6 +118,9 @@ PutMemoryContextsStatsTupleStore(Tuplestorestate *tupstore,
Datum
pg_get_backend_memory_contexts(PG_FUNCTION_ARGS)
{
+ int pid = PG_GETARG_INT32(0);
+ int max_children = PG_GETARG_INT32(1);
+
ReturnSetInfo *rsinfo = (ReturnSetInfo *) fcinfo->resultinfo;
TupleDesc tupdesc;
Tuplestorestate *tupstore;
@@ -147,8 +151,11 @@ pg_get_backend_memory_contexts(PG_FUNCTION_ARGS)
MemoryContextSwitchTo(oldcontext);
- PutMemoryContextsStatsTupleStore(tupstore, tupdesc,
- TopMemoryContext, NULL, 0);
+ if( pid == 0)
+ PutMemoryContextsStatsTupleStore(tupstore, tupdesc,
+ TopMemoryContext, NULL, 0);
+ else
+ pg_log_backend_memory_contexts(pid, max_children);
/* clean up and return the tuplestore */
tuplestore_donestoring(tupstore);
diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c
index 73e0a672ae..6c27065f96 100644
--- a/src/backend/utils/init/globals.c
+++ b/src/backend/utils/init/globals.c
@@ -34,6 +34,7 @@ volatile sig_atomic_t ClientConnectionLost = false;
volatile sig_atomic_t IdleInTransactionSessionTimeoutPending = false;
volatile sig_atomic_t IdleSessionTimeoutPending = false;
volatile sig_atomic_t ProcSignalBarrierPending = false;
+volatile sig_atomic_t LogMemoryContextPending = false;
volatile uint32 InterruptHoldoffCount = 0;
volatile uint32 QueryCancelHoldoffCount = 0;
volatile uint32 CritSectionCount = 0;
diff --git a/src/backend/utils/mmgr/aset.c b/src/backend/utils/mmgr/aset.c
index ec6c130d0f..8e7fe09a0f 100644
--- a/src/backend/utils/mmgr/aset.c
+++ b/src/backend/utils/mmgr/aset.c
@@ -272,7 +272,8 @@ static Size AllocSetGetChunkSpace(MemoryContext context, void *pointer);
static bool AllocSetIsEmpty(MemoryContext context);
static void AllocSetStats(MemoryContext context,
MemoryStatsPrintFunc printfunc, void *passthru,
- MemoryContextCounters *totals);
+ MemoryContextCounters *totals,
+ bool is_dst_stderr);
#ifdef MEMORY_CONTEXT_CHECKING
static void AllocSetCheck(MemoryContext context);
@@ -1336,11 +1337,12 @@ AllocSetIsEmpty(MemoryContext context)
* printfunc: if not NULL, pass a human-readable stats string to this.
* passthru: pass this pointer through to printfunc.
* totals: if not NULL, add stats about this context into *totals.
+ * is_dst_stderr: is the output should be stderr or elog.
*/
static void
AllocSetStats(MemoryContext context,
MemoryStatsPrintFunc printfunc, void *passthru,
- MemoryContextCounters *totals)
+ MemoryContextCounters *totals, bool is_dst_stderr)
{
AllocSet set = (AllocSet) context;
Size nblocks = 0;
@@ -1379,7 +1381,7 @@ AllocSetStats(MemoryContext context,
"%zu total in %zd blocks; %zu free (%zd chunks); %zu used",
totalspace, nblocks, freespace, freechunks,
totalspace - freespace);
- printfunc(context, passthru, stats_string);
+ printfunc(context, passthru, stats_string, is_dst_stderr);
}
if (totals)
diff --git a/src/backend/utils/mmgr/generation.c b/src/backend/utils/mmgr/generation.c
index 2b90034764..b286308aa4 100644
--- a/src/backend/utils/mmgr/generation.c
+++ b/src/backend/utils/mmgr/generation.c
@@ -155,7 +155,8 @@ static Size GenerationGetChunkSpace(MemoryContext context, void *pointer);
static bool GenerationIsEmpty(MemoryContext context);
static void GenerationStats(MemoryContext context,
MemoryStatsPrintFunc printfunc, void *passthru,
- MemoryContextCounters *totals);
+ MemoryContextCounters *totals,
+ bool is_dst_stderr);
#ifdef MEMORY_CONTEXT_CHECKING
static void GenerationCheck(MemoryContext context);
@@ -665,6 +666,7 @@ GenerationIsEmpty(MemoryContext context)
* printfunc: if not NULL, pass a human-readable stats string to this.
* passthru: pass this pointer through to printfunc.
* totals: if not NULL, add stats about this context into *totals.
+ * is_dst_stderr: is the output should be stderr or elog.
*
* XXX freespace only accounts for empty space at the end of the block, not
* space of freed chunks (which is unknown).
@@ -672,7 +674,7 @@ GenerationIsEmpty(MemoryContext context)
static void
GenerationStats(MemoryContext context,
MemoryStatsPrintFunc printfunc, void *passthru,
- MemoryContextCounters *totals)
+ MemoryContextCounters *totals, bool is_dst_stderr)
{
GenerationContext *set = (GenerationContext *) context;
Size nblocks = 0;
@@ -704,7 +706,7 @@ GenerationStats(MemoryContext context,
"%zu total in %zd blocks (%zd chunks); %zu free (%zd chunks); %zu used",
totalspace, nblocks, nchunks, freespace,
nfreechunks, totalspace - freespace);
- printfunc(context, passthru, stats_string);
+ printfunc(context, passthru, stats_string, is_dst_stderr);
}
if (totals)
diff --git a/src/backend/utils/mmgr/mcxt.c b/src/backend/utils/mmgr/mcxt.c
index 84472b9158..f02331c36c 100644
--- a/src/backend/utils/mmgr/mcxt.c
+++ b/src/backend/utils/mmgr/mcxt.c
@@ -23,6 +23,10 @@
#include "mb/pg_wchar.h"
#include "miscadmin.h"
+#include "storage/proc.h"
+#include "storage/procarray.h"
+#include "storage/procsignal.h"
+#include "utils/fmgrprotos.h"
#include "utils/memdebug.h"
#include "utils/memutils.h"
@@ -37,6 +41,13 @@
*/
MemoryContext CurrentMemoryContext = NULL;
+/*
+ * McxtLogData
+ * Data for communication between memory context logging
+ * dumper and requestor.
+ */
+McxtLogData *mcxtLogData = NULL;
+
/*
* Standard top-level contexts. For a description of the purpose of each
* of these contexts, refer to src/backend/utils/mmgr/README
@@ -55,9 +66,11 @@ MemoryContext PortalContext = NULL;
static void MemoryContextCallResetCallbacks(MemoryContext context);
static void MemoryContextStatsInternal(MemoryContext context, int level,
bool print, int max_children,
- MemoryContextCounters *totals);
+ MemoryContextCounters *totals,
+ bool is_dst_stderr);
static void MemoryContextStatsPrint(MemoryContext context, void *passthru,
- const char *stats_string);
+ const char *stats_string,
+ bool is_dst_stderr);
/*
* You should not do memory allocations within a critical section, because
@@ -487,6 +500,84 @@ MemoryContextMemAllocated(MemoryContext context, bool recurse)
return total;
}
+/*
+ * McxtLogShmemInit
+ * Initialize shared memory area
+ */
+void
+McxtLogShmemInit(void)
+{
+ bool found;
+
+ mcxtLogData = (McxtLogData *)
+ ShmemInitStruct("Memory Context Logging Data",
+ sizeof(int),
+ &found);
+ if (!found)
+ {
+ mcxtLogData->maxChildrenPerContext = 0;
+ }
+}
+
+/*
+ * pg_log_backend_memory_contexts
+ * Log memory contexts of the specified backend process.
+ */
+Datum
+pg_log_backend_memory_contexts(int pid, int max_children)
+{
+ PGPROC *proc = BackendPidGetProc(pid);
+
+ if(max_children < 0)
+ {
+ ereport(ERROR,
+ (errmsg("The maximum number of children must be greater than or equal to 0")));
+
+ PG_RETURN_BOOL(false);
+ }
+
+ /*
+ * Check whether the target process is PostgreSQL backend process.
+ */
+ if (proc == NULL)
+ {
+ /*
+ * According to the thread below, it might be better to
+ * change this message.
+ *
+ * https://www.postgresql.org/message-id/CALj2ACW7Rr-R7mBcBQiXWPp%3DJV5chajjTdudLiF5YcpW-BmHhg%40mail.gmail.com
+ *
+ * Until the conclusion of the discussion, output the same
+ * message as pg_cancel_backend().
+ */
+ ereport(WARNING,
+ (errmsg("PID %d is not a PostgreSQL server process", pid)));
+
+ PG_RETURN_BOOL(false);
+ }
+
+ /* Only allow superusers to print memory contexts. */
+ if (!superuser())
+ {
+ ereport(WARNING,
+ (errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
+ errmsg("must be a superuser to log memory contexts")));
+ PG_RETURN_BOOL(false);
+ }
+
+ mcxtLogData->maxChildrenPerContext = max_children;
+
+ if(!SendProcSignal(pid, PROCSIG_LOG_MEMORY_CONTEXT, proc->backendId))
+ PG_RETURN_BOOL(true);
+ else
+ {
+ ereport(WARNING,
+ (errmsg("failed to send signal: %m")));
+
+ PG_RETURN_BOOL(false);
+ }
+}
+
/*
* MemoryContextStats
* Print statistics about the named context and all its descendants.
@@ -499,7 +590,7 @@ void
MemoryContextStats(MemoryContext context)
{
/* A hard-wired limit on the number of children is usually good enough */
- MemoryContextStatsDetail(context, 100);
+ MemoryContextStatsDetail(context, 100, true);
}
/*
@@ -508,19 +599,28 @@ MemoryContextStats(MemoryContext context)
* Entry point for use if you want to vary the number of child contexts shown.
*/
void
-MemoryContextStatsDetail(MemoryContext context, int max_children)
+MemoryContextStatsDetail(MemoryContext context, int max_children, bool is_dst_stderr)
{
MemoryContextCounters grand_totals;
memset(&grand_totals, 0, sizeof(grand_totals));
- MemoryContextStatsInternal(context, 0, true, max_children, &grand_totals);
+ MemoryContextStatsInternal(context, 0, true, max_children, &grand_totals, is_dst_stderr);
- fprintf(stderr,
- "Grand total: %zu bytes in %zd blocks; %zu free (%zd chunks); %zu used\n",
- grand_totals.totalspace, grand_totals.nblocks,
- grand_totals.freespace, grand_totals.freechunks,
- grand_totals.totalspace - grand_totals.freespace);
+ if (is_dst_stderr)
+ fprintf(stderr,
+ "Grand total: %zu bytes in %zd blocks; %zu free (%zd chunks); %zu used\n",
+ grand_totals.totalspace, grand_totals.nblocks,
+ grand_totals.freespace, grand_totals.freechunks,
+ grand_totals.totalspace - grand_totals.freespace);
+ else
+ ereport(LOG_SERVER_ONLY,
+ (errhidestmt(true),
+ errhidecontext(true),
+ errmsg_internal("Grand total: %zu bytes in %zd blocks; %zu free (%zd chunks); %zu used",
+ grand_totals.totalspace, grand_totals.nblocks,
+ grand_totals.freespace, grand_totals.freechunks,
+ grand_totals.totalspace - grand_totals.freespace)));
}
/*
@@ -533,7 +633,8 @@ MemoryContextStatsDetail(MemoryContext context, int max_children)
static void
MemoryContextStatsInternal(MemoryContext context, int level,
bool print, int max_children,
- MemoryContextCounters *totals)
+ MemoryContextCounters *totals,
+ bool is_dst_stderr)
{
MemoryContextCounters local_totals;
MemoryContext child;
@@ -545,11 +646,13 @@ MemoryContextStatsInternal(MemoryContext context, int level,
context->methods->stats(context,
print ? MemoryContextStatsPrint : NULL,
(void *) &level,
- totals);
+ totals, is_dst_stderr);
/*
* Examine children. If there are more than max_children of them, we do
* not print the rest explicitly, but just summarize them.
+ *
+ * Note that specifying 0 to max_children means umlimited.
*/
memset(&local_totals, 0, sizeof(local_totals));
@@ -557,33 +660,50 @@ MemoryContextStatsInternal(MemoryContext context, int level,
child != NULL;
child = child->nextchild, ichild++)
{
- if (ichild < max_children)
+ if (ichild < max_children || max_children == 0)
MemoryContextStatsInternal(child, level + 1,
print, max_children,
- totals);
+ totals,
+ is_dst_stderr);
else
MemoryContextStatsInternal(child, level + 1,
false, max_children,
- &local_totals);
+ &local_totals,
+ is_dst_stderr);
}
- /* Deal with excess children */
- if (ichild > max_children)
+ /*Deal with excess children */
+ if (ichild > max_children && max_children != 0)
{
if (print)
{
int i;
- for (i = 0; i <= level; i++)
- fprintf(stderr, " ");
- fprintf(stderr,
- "%d more child contexts containing %zu total in %zd blocks; %zu free (%zd chunks); %zu used\n",
- ichild - max_children,
- local_totals.totalspace,
- local_totals.nblocks,
- local_totals.freespace,
- local_totals.freechunks,
- local_totals.totalspace - local_totals.freespace);
+ if (is_dst_stderr)
+ {
+ for (i = 0; i <= level; i++)
+ fprintf(stderr, " ");
+ fprintf(stderr,
+ "%d more child contexts containing %zu total in %zd blocks; %zu free (%zd chunks); %zu used\n",
+ ichild - max_children,
+ local_totals.totalspace,
+ local_totals.nblocks,
+ local_totals.freespace,
+ local_totals.freechunks,
+ local_totals.totalspace - local_totals.freespace);
+ }
+ else
+ ereport(LOG_SERVER_ONLY,
+ (errhidestmt(true),
+ errhidecontext(true),
+ errmsg_internal("level: %d %d more child contexts containing %zu total in %zd blocks; %zu free (%zd chunks); %zu used",
+ level,
+ ichild - max_children,
+ local_totals.totalspace,
+ local_totals.nblocks,
+ local_totals.freespace,
+ local_totals.freechunks,
+ local_totals.totalspace - local_totals.freespace)));
}
if (totals)
@@ -605,12 +725,14 @@ MemoryContextStatsInternal(MemoryContext context, int level,
*/
static void
MemoryContextStatsPrint(MemoryContext context, void *passthru,
- const char *stats_string)
+ const char *stats_string,
+ bool is_dst_stderr)
{
int level = *(int *) passthru;
const char *name = context->name;
const char *ident = context->ident;
int i;
+ char truncated_ident[110];
/*
* It seems preferable to label dynahash contexts with just the hash table
@@ -623,9 +745,8 @@ MemoryContextStatsPrint(MemoryContext context, void *passthru,
ident = NULL;
}
- for (i = 0; i < level; i++)
- fprintf(stderr, " ");
- fprintf(stderr, "%s: %s", name, stats_string);
+ truncated_ident[0] = '\0';
+
if (ident)
{
/*
@@ -637,24 +758,42 @@ MemoryContextStatsPrint(MemoryContext context, void *passthru,
int idlen = strlen(ident);
bool truncated = false;
+ strcpy(truncated_ident, ": ");
+
if (idlen > 100)
{
idlen = pg_mbcliplen(ident, idlen, 100);
truncated = true;
}
- fprintf(stderr, ": ");
+
+ i = strlen(truncated_ident);
+
while (idlen-- > 0)
{
unsigned char c = *ident++;
if (c < ' ')
c = ' ';
- fputc(c, stderr);
+ truncated_ident[i++] = c;
}
+ truncated_ident[i] = '\0';
+
if (truncated)
- fprintf(stderr, "...");
+ strcat(truncated_ident, "...");
+ }
+
+ if (is_dst_stderr)
+ {
+ for (i = 0; i < level; i++)
+ fprintf(stderr, " ");
+ fprintf(stderr, "%s: %s%s\n", name, stats_string, truncated_ident);
}
- fputc('\n', stderr);
+ else
+ ereport(LOG_SERVER_ONLY,
+ (errhidestmt(true),
+ errhidecontext(true),
+ errmsg_internal("level: %d %s: %s%s",
+ level, name, stats_string, truncated_ident)));
}
/*
diff --git a/src/backend/utils/mmgr/slab.c b/src/backend/utils/mmgr/slab.c
index 9213be7c95..479b08bedd 100644
--- a/src/backend/utils/mmgr/slab.c
+++ b/src/backend/utils/mmgr/slab.c
@@ -135,7 +135,8 @@ static Size SlabGetChunkSpace(MemoryContext context, void *pointer);
static bool SlabIsEmpty(MemoryContext context);
static void SlabStats(MemoryContext context,
MemoryStatsPrintFunc printfunc, void *passthru,
- MemoryContextCounters *totals);
+ MemoryContextCounters *totals,
+ bool is_dst_stderr);
#ifdef MEMORY_CONTEXT_CHECKING
static void SlabCheck(MemoryContext context);
#endif
@@ -632,11 +633,13 @@ SlabIsEmpty(MemoryContext context)
* printfunc: if not NULL, pass a human-readable stats string to this.
* passthru: pass this pointer through to printfunc.
* totals: if not NULL, add stats about this context into *totals.
+ * is_dst_stderr: is the output should be stderr or elog.
*/
static void
SlabStats(MemoryContext context,
MemoryStatsPrintFunc printfunc, void *passthru,
- MemoryContextCounters *totals)
+ MemoryContextCounters *totals,
+ bool is_dst_stderr)
{
SlabContext *slab = castNode(SlabContext, context);
Size nblocks = 0;
@@ -671,7 +674,7 @@ SlabStats(MemoryContext context,
"%zu total in %zd blocks; %zu free (%zd chunks); %zu used",
totalspace, nblocks, freespace, freechunks,
totalspace - freespace);
- printfunc(context, passthru, stats_string);
+ printfunc(context, passthru, stats_string, is_dst_stderr);
}
if (totals)
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 464fa8d614..fa20889105 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -7921,12 +7921,14 @@
descr => 'information about all memory contexts of local backend',
proname => 'pg_get_backend_memory_contexts', prorows => '100',
proretset => 't', provolatile => 'v', proparallel => 'r',
- prorettype => 'record', proargtypes => '',
- proallargtypes => '{text,text,text,int4,int8,int8,int8,int8,int8}',
- proargmodes => '{o,o,o,o,o,o,o,o,o}',
- proargnames => '{name, ident, parent, level, total_bytes, total_nblocks, free_bytes, free_chunks, used_bytes}',
+ prorettype => 'record', proargtypes => 'int4 int4',
+ proallargtypes => '{int4,int4,text,text,text,int4,int8,int8,int8,int8,int8}',
+ proargmodes => '{i,i,o,o,o,o,o,o,o,o,o}',
+ proargnames => '{pid, max_children, name, ident, parent, level, total_bytes, total_nblocks, free_bytes, free_chunks, used_bytes}',
prosrc => 'pg_get_backend_memory_contexts' },
+# print memory context of specified backend
+
# non-persistent series generator
{ oid => '1066', descr => 'non-persistent series generator',
proname => 'generate_series', prorows => '1000',
diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h
index 013850ac28..081822823c 100644
--- a/src/include/miscadmin.h
+++ b/src/include/miscadmin.h
@@ -84,6 +84,7 @@ extern PGDLLIMPORT volatile sig_atomic_t ProcDiePending;
extern PGDLLIMPORT volatile sig_atomic_t IdleInTransactionSessionTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t IdleSessionTimeoutPending;
extern PGDLLIMPORT volatile sig_atomic_t ProcSignalBarrierPending;
+extern PGDLLIMPORT volatile sig_atomic_t LogMemoryContextPending;
extern PGDLLIMPORT volatile sig_atomic_t ClientConnectionLost;
diff --git a/src/include/nodes/memnodes.h b/src/include/nodes/memnodes.h
index 9331ef80fd..87a8501c37 100644
--- a/src/include/nodes/memnodes.h
+++ b/src/include/nodes/memnodes.h
@@ -52,7 +52,8 @@ typedef struct MemoryContextCounters
*/
typedef void (*MemoryStatsPrintFunc) (MemoryContext context, void *passthru,
- const char *stats_string);
+ const char *stats_string,
+ bool is_dst_stderr);
typedef struct MemoryContextMethods
{
@@ -66,7 +67,8 @@ typedef struct MemoryContextMethods
bool (*is_empty) (MemoryContext context);
void (*stats) (MemoryContext context,
MemoryStatsPrintFunc printfunc, void *passthru,
- MemoryContextCounters *totals);
+ MemoryContextCounters *totals,
+ bool is_dst_stderr);
#ifdef MEMORY_CONTEXT_CHECKING
void (*check) (MemoryContext context);
#endif
diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h
index 4ae7dc33b8..1144536031 100644
--- a/src/include/storage/procsignal.h
+++ b/src/include/storage/procsignal.h
@@ -34,6 +34,8 @@ typedef enum
PROCSIG_PARALLEL_MESSAGE, /* message from cooperating parallel backend */
PROCSIG_WALSND_INIT_STOPPING, /* ask walsenders to prepare for shutdown */
PROCSIG_BARRIER, /* global barrier interrupt */
+ PROCSIG_LOG_MEMORY_CONTEXT, /* ask specified backend to log the
+ memory context */
/* Recovery conflict reasons */
PROCSIG_RECOVERY_CONFLICT_DATABASE,
@@ -69,6 +71,7 @@ extern int SendProcSignal(pid_t pid, ProcSignalReason reason,
extern uint64 EmitProcSignalBarrier(ProcSignalBarrierType type);
extern void WaitForProcSignalBarrier(uint64 generation);
extern void ProcessProcSignalBarrier(void);
+extern void ProcessLogMemoryContextInterrupt(void);
extern void procsignal_sigusr1_handler(SIGNAL_ARGS);
diff --git a/src/include/utils/memutils.h b/src/include/utils/memutils.h
index 36aae4e51c..9596aa7f14 100644
--- a/src/include/utils/memutils.h
+++ b/src/include/utils/memutils.h
@@ -46,6 +46,12 @@
#define AllocHugeSizeIsValid(size) ((Size) (size) <= MaxAllocHugeSize)
+typedef struct McxtLogData
+{
+ /* Max number of children to print per one parent context */
+ int maxChildrenPerContext;
+} McxtLogData;
+
/*
* Standard top-level memory contexts.
*
@@ -63,6 +69,8 @@ extern PGDLLIMPORT MemoryContext CurTransactionContext;
/* This is a transient link to the active portal's memory context: */
extern PGDLLIMPORT MemoryContext PortalContext;
+extern McxtLogData *mcxtLogData;
+
/* Backwards compatibility macro */
#define MemoryContextResetAndDeleteChildren(ctx) MemoryContextReset(ctx)
@@ -84,9 +92,12 @@ extern MemoryContext MemoryContextGetParent(MemoryContext context);
extern bool MemoryContextIsEmpty(MemoryContext context);
extern Size MemoryContextMemAllocated(MemoryContext context, bool recurse);
extern void MemoryContextStats(MemoryContext context);
-extern void MemoryContextStatsDetail(MemoryContext context, int max_children);
+extern void MemoryContextStatsDetail(MemoryContext context, int max_children,
+ bool is_dst_stderr);
extern void MemoryContextAllowInCriticalSection(MemoryContext context,
bool allow);
+extern void McxtLogShmemInit(void);
+extern Datum pg_log_backend_memory_contexts(int pid, int max_children);
#ifdef MEMORY_CONTEXT_CHECKING
extern void MemoryContextCheck(MemoryContext context);
diff --git a/src/test/modules/test_misc/t/002_print_memory_context_validation.pl b/src/test/modules/test_misc/t/002_print_memory_context_validation.pl
new file mode 100644
index 0000000000..056ced1da2
--- /dev/null
+++ b/src/test/modules/test_misc/t/002_print_memory_context_validation.pl
@@ -0,0 +1,90 @@
+use strict;
+use warnings;
+use PostgresNode;
+use TestLib;
+use Test::More tests => 4;
+
+# Set up node with logging collector
+my $node = get_new_node('primary');
+$node->init();
+$node->append_conf(
+ 'postgresql.conf', qq{
+ logging_collector = on
+ lc_messages = 'C'
+ });
+
+$node->start();
+
+# Verify that log output gets to the file
+$node->psql('postgres', 'select pg_get_backend_memory_contexts(pg_backend_pid(), 100)');
+
+# might need to retry if logging collector process is slow...
+my $max_attempts = 180 * 10;
+
+my $current_logfiles;
+for (my $attempts = 0; $attempts < $max_attempts; $attempts++)
+{
+ eval {
+ $current_logfiles = slurp_file($node->data_dir . '/current_logfiles');
+ };
+ last unless $@;
+ usleep(100_000);
+}
+die $@ if $@;
+
+note "current_logfiles = $current_logfiles";
+
+like(
+ $current_logfiles,
+ qr|^stderr log/postgresql-.*log$|,
+ 'current_logfiles is sane');
+
+my $lfname = $current_logfiles;
+$lfname =~ s/^stderr //;
+chomp $lfname;
+
+my $logfile;
+my $print_count;
+
+# Verify that the backtraces of the processes are logged into logfile.
+for (my $attempts = 0; $attempts < $max_attempts; $attempts++)
+{
+ $logfile = $node->data_dir . '/' . $lfname;
+ chomp $logfile;
+ print "file is $logfile";
+ open my $fh, '<', $logfile
+ or die "Could not open '$logfile' $!";
+ while (my $line = <$fh>)
+ {
+ chomp $line;
+ if ($line =~ m/Grand total: \d+ bytes in \d+ blocks/)
+ {
+ $print_count++;
+ }
+ }
+ last if $print_count == 1;
+ usleep(100_000);
+}
+
+is($print_count, 1, 'found expected memory context print in the log file');
+
+my $output;
+
+$node->psql('postgres', 'select pg_get_backend_memory_contexts(pg_backend_pid(), -1)',
+ stderr => \$output);
+
+like(
+ $output,
+ qr/The maximum number of children must be greater than or equal to 0/,
+ 'check if max_children check is working');
+
+$node->psql('postgres',
+ 'select pg_get_backend_memory_contexts(pid, 100) from pg_stat_activity where backend_type = \'checkpointer\'',
+ stderr => \$output);
+
+like(
+ $output,
+ qr/PID \d+ is not a PostgreSQL server process/,
+ 'check if PID check is working');
+
+$node->stop('fast');
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 9b12cc122a..5841e5cc12 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -1333,7 +1333,7 @@ pg_backend_memory_contexts| SELECT pg_get_backend_memory_contexts.name,
pg_get_backend_memory_contexts.free_bytes,
pg_get_backend_memory_contexts.free_chunks,
pg_get_backend_memory_contexts.used_bytes
- FROM pg_get_backend_memory_contexts() pg_get_backend_memory_contexts(name, ident, parent, level, total_bytes, total_nblocks, free_bytes, free_chunks, used_bytes);
+ FROM pg_get_backend_memory_contexts(0, 0) pg_get_backend_memory_contexts(name, ident, parent, level, total_bytes, total_nblocks, free_bytes, free_chunks, used_bytes);
pg_config| SELECT pg_config.name,
pg_config.setting
FROM pg_config() pg_config(name, setting);
