On Tue, Nov 25, 2025 at 5:30 AM Álvaro Herrera <[email protected]> wrote: > I also noticed that an oauth test file contains a couple MBs of a > gigantic string of just 'x'. I suppose that will compress well (with > gzip at least, the 2 MB file becomes 8 kB). Still, it's kinda > ridiculous and useless to bloat a 67 kB file to 2 MB that way.
Sorry about that. > 0003 does that by simply cutting the string short at 10k, which reduces > the size of the log from 2 MB to some 86 kB. Maybe there are better > ways to deal with this? Jacob? Only thing I don't like about this is that the JSON you need for debugging might be after the useless padding. Attached is a patch that does things more surgically, tested against Python 3.6, and I'm running it through the CI now. Thanks, --Jacob
From 7f6b1b4d19ac19f62dc5525198bb3d047d21ae7e Mon Sep 17 00:00:00 2001 From: Jacob Champion <[email protected]> Date: Tue, 25 Nov 2025 10:07:29 -0800 Subject: [PATCH] WIP: truncate long JSON responses in logs MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Reported-by: Álvaro Herrera <[email protected]> Discussion: https://postgr.es/m/202511251218.zfs4nu2qnh2m%40alvherre.pgsql --- .../modules/oauth_validator/t/oauth_server.py | 22 ++++++++++++++++++- 1 file changed, 21 insertions(+), 1 deletion(-) diff --git a/src/test/modules/oauth_validator/t/oauth_server.py b/src/test/modules/oauth_validator/t/oauth_server.py index 0f8836aadf3..c70783ecbe4 100755 --- a/src/test/modules/oauth_validator/t/oauth_server.py +++ b/src/test/modules/oauth_validator/t/oauth_server.py @@ -257,13 +257,33 @@ class OAuthHandler(http.server.BaseHTTPRequestHandler): return token + def _log_response(self, js: JsonObject) -> None: + """ + Trims the response JSON, if necessary, and logs it for later debugging. + """ + # At the moment the biggest problem for tests is the _pad_ member, which + # is a megabyte in size, so truncate that to something more reasonable. + if "_pad_" in js: + pad = js["_pad_"] + + # Don't modify the original dict. + js = dict(js) + js["_pad_"] = pad[:64] + f"[...truncated from {len(pad)} bytes]" + + resp = json.dumps(js).encode("ascii") + self.log_message("sending JSON response: %s", resp) + + # If you've tripped this assertion, please truncate the new addition as + # above, or else come up with a new strategy. + assert len(resp) < 1024, "_log_response must be adjusted for new JSON" + def _send_json(self, js: JsonObject) -> None: """ Sends the provided JSON dict as an application/json response. self._response_code can be modified to send JSON error responses. """ resp = json.dumps(js).encode("ascii") - self.log_message("sending JSON response: %s", resp) + self._log_response(js) self.send_response(self._response_code) self.send_header("Content-Type", self._content_type) -- 2.34.1
