Hi, I just noticed that one regress log file for the pg_dump test is about 7MB long because it contains totally unnecessary dump files in the regress log output. This is because the tests run pg_dump of schema pg_catalog without redirecting the output anywhere, which means it goes to stdout, and then `prove` helpfully puts it in the regress log file. This is completely pointless. We can make the log file 1 MB instead by adding a --file parameter to that command, as 0001 attached.
We also have a couple of tests that leave rather large server logs behind that are perhaps not very useful. I would propose to run those servers with log_statement=off and avoid some of the bloat. 0002 does that. Another possible approach, instead of individually tweaking t/*.pl files to add 'log_statements=none', would be to set it that way in PostgreSQL::Test::Cluster globally. Right now we have log_statements=all there, but maybe that was not a great idea. I also noticed that an oauth test file contains a couple MBs of a gigantic string of just 'x'. I suppose that will compress well (with gzip at least, the 2 MB file becomes 8 kB). Still, it's kinda ridiculous and useless to bloat a 67 kB file to 2 MB that way. The problem here is that we print 'sending JSON response' and the verbatim response, included the 1 MB of "x" as '_pad_'. Can we supress this? 0003 does that by simply cutting the string short at 10k, which reduces the size of the log from 2 MB to some 86 kB. Maybe there are better ways to deal with this? Jacob? With these three patches, we go from having 62 MB bytes in files with "log" in their names under testrun/ (except those that have "catalog") to 30 MB. I think that's not a bad reduction. -- Álvaro Herrera PostgreSQL Developer — https://www.EnterpriseDB.com/ Are you not unsure you want to delete Firefox? [Not unsure] [Not not unsure] [Cancel] http://smylers.hates-software.com/2008/01/03/566e45b2.html
>From ae1568e9bb21b9ee6a6b24ef967c2bd14022233b Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=C3=81lvaro=20Herrera?= <[email protected]> Date: Tue, 25 Nov 2025 13:42:27 +0100 Subject: [PATCH 1/3] pg_dump tests: don't put dumps in stdout --- src/bin/pg_dump/t/002_pg_dump.pl | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl index 445a541abf6..a05212719d9 100644 --- a/src/bin/pg_dump/t/002_pg_dump.pl +++ b/src/bin/pg_dump/t/002_pg_dump.pl @@ -5019,7 +5019,12 @@ $node->command_fails_like( # Test dumping pg_catalog (for research -- cannot be reloaded) $node->command_ok( - [ 'pg_dump', '--port' => $port, '--schema' => 'pg_catalog' ], + [ + 'pg_dump', + '--port' => $port, + '--schema' => 'pg_catalog', + '--file' => 'pgdump_pgcatalog.dmp' + ], 'pg_dump: option -n pg_catalog'); ######################################### @@ -5029,7 +5034,8 @@ $node->command_ok( [ 'pg_dumpall', '--port' => $port, - '--exclude-database' => '"myhost.mydb"' + '--exclude-database' => '"myhost.mydb"', + '--file' => 'pgdumpall.dmp' ], 'pg_dumpall: option --exclude-database handles database names with embedded dots' ); -- 2.47.3
>From b10634f5f68e4a65708a0ce13fc0dd95a875bc4a Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=C3=81lvaro=20Herrera?= <[email protected]> Date: Tue, 25 Nov 2025 13:56:49 +0100 Subject: [PATCH 2/3] log_statement=off --- src/bin/pg_amcheck/t/003_check.pl | 5 ++++- src/bin/pg_dump/t/002_pg_dump.pl | 3 +++ src/bin/pgbench/t/001_pgbench_with_server.pl | 4 ++++ src/test/recovery/t/027_stream_regress.pl | 1 + src/test/subscription/t/035_conflicts.pl | 3 +++ 5 files changed, 15 insertions(+), 1 deletion(-) diff --git a/src/bin/pg_amcheck/t/003_check.pl b/src/bin/pg_amcheck/t/003_check.pl index 881854da254..63de75adf4f 100644 --- a/src/bin/pg_amcheck/t/003_check.pl +++ b/src/bin/pg_amcheck/t/003_check.pl @@ -121,7 +121,10 @@ sub perform_all_corruptions() # Test set-up $node = PostgreSQL::Test::Cluster->new('test'); $node->init(no_data_checksums => 1); -$node->append_conf('postgresql.conf', 'autovacuum=off'); +$node->append_conf('postgresql.conf', + qq{autovacuum=off +log_statement=none +}); $node->start; $port = $node->port; diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl index a05212719d9..db3fe1aa068 100644 --- a/src/bin/pg_dump/t/002_pg_dump.pl +++ b/src/bin/pg_dump/t/002_pg_dump.pl @@ -4839,6 +4839,9 @@ my %tests = ( my $node = PostgreSQL::Test::Cluster->new('main'); $node->init; +$node->append_conf( + 'postgresql.conf', + qq{log_statement=none}); $node->start; my $port = $node->port; diff --git a/src/bin/pgbench/t/001_pgbench_with_server.pl b/src/bin/pgbench/t/001_pgbench_with_server.pl index 581e9af7907..83b929642ec 100644 --- a/src/bin/pgbench/t/001_pgbench_with_server.pl +++ b/src/bin/pgbench/t/001_pgbench_with_server.pl @@ -42,6 +42,10 @@ my $node = PostgreSQL::Test::Cluster->new('main'); # Set to untranslated messages, to be able to compare program output with # expected strings. $node->init(extra => [ '--locale', 'C' ]); +$node->append_conf( + 'postgresql.conf', + qq{log_statement=none}); + $node->start; # tablespace for testing, because partitioned tables cannot use pg_default diff --git a/src/test/recovery/t/027_stream_regress.pl b/src/test/recovery/t/027_stream_regress.pl index 589c79d97d3..64b15f276e2 100644 --- a/src/test/recovery/t/027_stream_regress.pl +++ b/src/test/recovery/t/027_stream_regress.pl @@ -26,6 +26,7 @@ $node_primary->append_conf( qq{shared_preload_libraries = 'pg_stat_statements' pg_stat_statements.max = 50000 compute_query_id = 'regress' +log_statement=none }); # We'll stick with Cluster->new's small default shared_buffers, but since that diff --git a/src/test/subscription/t/035_conflicts.pl b/src/test/subscription/t/035_conflicts.pl index a94bde16f53..3ad6af2908e 100644 --- a/src/test/subscription/t/035_conflicts.pl +++ b/src/test/subscription/t/035_conflicts.pl @@ -14,6 +14,9 @@ use Test::More; # Create a publisher node my $node_publisher = PostgreSQL::Test::Cluster->new('publisher'); $node_publisher->init(allows_streaming => 'logical'); +$node_publisher->append_conf( + 'postgresql.conf', + qq{log_statements=none}); $node_publisher->start; # Create a subscriber node -- 2.47.3
>From 14a835cdaa6b0f8bad368bbf3f75f1fde74edb05 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=C3=81lvaro=20Herrera?= <[email protected]> Date: Tue, 25 Nov 2025 13:57:08 +0100 Subject: [PATCH 3/3] oauth_server: don't log huge strings --- src/test/modules/oauth_validator/t/oauth_server.py | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/src/test/modules/oauth_validator/t/oauth_server.py b/src/test/modules/oauth_validator/t/oauth_server.py index 0f8836aadf3..a4821a8e8b2 100755 --- a/src/test/modules/oauth_validator/t/oauth_server.py +++ b/src/test/modules/oauth_validator/t/oauth_server.py @@ -263,7 +263,8 @@ class OAuthHandler(http.server.BaseHTTPRequestHandler): self._response_code can be modified to send JSON error responses. """ resp = json.dumps(js).encode("ascii") - self.log_message("sending JSON response: %s", resp) + # avoid saving humongous strings to the log file + self.log_message("sending JSON response: %s", resp[0:10000]) self.send_response(self._response_code) self.send_header("Content-Type", self._content_type) -- 2.47.3
