Re: Reducing output size of nodeToString

2024-03-22 Thread Peter Eisentraut
On 20.03.24 13:03, Matthias van de Meent wrote: On Wed, 20 Mar 2024 at 12:49, Peter Eisentraut wrote: On 19.03.24 17:13, Peter Eisentraut wrote: On 11.03.24 21:52, Matthias van de Meent wrote: * v7-0003-gen_node_support.pl-Mark-location-fields-as-type-.patch This looks sensible, but maybe

Re: Reducing output size of nodeToString

2024-03-20 Thread Matthias van de Meent
On Wed, 20 Mar 2024 at 12:49, Peter Eisentraut wrote: > > On 19.03.24 17:13, Peter Eisentraut wrote: > > On 11.03.24 21:52, Matthias van de Meent wrote: > >>> * v7-0003-gen_node_support.pl-Mark-location-fields-as-type-.patch > >>> > >>> This looks sensible, but maybe making Location a global type

Re: Reducing output size of nodeToString

2024-03-20 Thread Peter Eisentraut
On 19.03.24 17:13, Peter Eisentraut wrote: On 11.03.24 21:52, Matthias van de Meent wrote: * v7-0003-gen_node_support.pl-Mark-location-fields-as-type-.patch This looks sensible, but maybe making Location a global type is a bit much?  Maybe something more specific like ParseLocation, or

Re: Reducing output size of nodeToString

2024-03-19 Thread Matthias van de Meent
On Tue, 19 Mar 2024 at 17:13, Peter Eisentraut wrote: > > On 11.03.24 21:52, Matthias van de Meent wrote: > >> * v7-0003-gen_node_support.pl-Mark-location-fields-as-type-.patch > >> > >> This looks sensible, but maybe making Location a global type is a bit > >> much? Maybe something more

Re: Reducing output size of nodeToString

2024-03-19 Thread Peter Eisentraut
On 11.03.24 21:52, Matthias van de Meent wrote: * v7-0003-gen_node_support.pl-Mark-location-fields-as-type-.patch This looks sensible, but maybe making Location a global type is a bit much? Maybe something more specific like ParseLocation, or ParseLoc, to keep it under 12 characters. I've

Re: Reducing output size of nodeToString

2024-03-11 Thread Matthias van de Meent
On Mon, 11 Mar 2024 at 14:19, Peter Eisentraut wrote: > > On 22.02.24 16:07, Matthias van de Meent wrote: > > On Thu, 22 Feb 2024 at 13:37, Matthias van de Meent > > wrote: > >> > >> On Mon, 19 Feb 2024 at 14:19, Matthias van de Meent > >> wrote: > >>> Attached the updated version of the patch

Re: Reducing output size of nodeToString

2024-03-11 Thread Peter Eisentraut
On 22.02.24 16:07, Matthias van de Meent wrote: On Thu, 22 Feb 2024 at 13:37, Matthias van de Meent wrote: On Mon, 19 Feb 2024 at 14:19, Matthias van de Meent wrote: Attached the updated version of the patch on top of 5497daf3, which incorporates this last round of feedback. Now attached

Re: Reducing output size of nodeToString

2024-02-22 Thread Matthias van de Meent
On Thu, 22 Feb 2024 at 13:37, Matthias van de Meent wrote: > > On Mon, 19 Feb 2024 at 14:19, Matthias van de Meent > wrote: > > Attached the updated version of the patch on top of 5497daf3, which > > incorporates this last round of feedback. > > Now attached rebased on top of 93db6cbd to fix

Re: Reducing output size of nodeToString

2024-02-22 Thread Matthias van de Meent
On Mon, 19 Feb 2024 at 14:19, Matthias van de Meent wrote: > Attached the updated version of the patch on top of 5497daf3, which > incorporates this last round of feedback. Now attached rebased on top of 93db6cbd to fix conflicts with fbc93b8b and an issue in the previous patchset: I attached

Re: Reducing output size of nodeToString

2024-02-15 Thread Matthias van de Meent
On Thu, 15 Feb 2024 at 13:59, Peter Eisentraut wrote: > > Thanks, this patch set is a good way to incrementally work through these > changes. > > I have looked at > v4-0001-pg_node_tree-Omit-serialization-of-fields-with-de.patch today. > Here are my thoughts: > > I believe we had discussed

Re: Reducing output size of nodeToString

2024-02-15 Thread Peter Eisentraut
Thanks, this patch set is a good way to incrementally work through these changes. I have looked at v4-0001-pg_node_tree-Omit-serialization-of-fields-with-de.patch today. Here are my thoughts: I believe we had discussed offline to not omit enum fields with value 0 (WRITE_ENUM_FIELD). This

Re: Reducing output size of nodeToString

2024-02-12 Thread Matthias van de Meent
On Mon, 12 Feb 2024 at 20:32, Matthias van de Meent wrote: > > On Mon, 12 Feb 2024 at 19:03, Matthias van de Meent > wrote: > > Attached is patchset v2, which contains the improvements from these patches: > > Attached v3, which fixes an out-of-bounds read in pg_strtoken_next, > detected by asan,

Re: Reducing output size of nodeToString

2024-02-12 Thread Matthias van de Meent
On Mon, 12 Feb 2024 at 19:03, Matthias van de Meent wrote: > Attached is patchset v2, which contains the improvements from these patches: Attached v3, which fixes an out-of-bounds read in pg_strtoken_next, detected by asan, that was a likely cause of the problems in CFBot's FreeBSD regression

Re: Reducing output size of nodeToString

2024-02-12 Thread Matthias van de Meent
On Wed, 31 Jan 2024 at 18:47, Robert Haas wrote: > > On Wed, Jan 31, 2024 at 11:17 AM Matthias van de Meent > wrote: > > I was also thinking about smaller per-attribute expression storage, for > > index attribute expressions, table default expressions, and functions. > > Other than that, less

Re: Reducing output size of nodeToString

2024-01-31 Thread Robert Haas
On Wed, Jan 31, 2024 at 11:17 AM Matthias van de Meent wrote: > I was also thinking about smaller per-attribute expression storage, for index > attribute expressions, table default expressions, and functions. Other than > that, less memory overhead for the serialized form of these constructs

Re: Reducing output size of nodeToString

2024-01-31 Thread Matthias van de Meent
On Wed, 31 Jan 2024, 09:16 Peter Eisentraut, wrote: > On 30.01.24 12:26, Matthias van de Meent wrote: > >> Most of the other defaults I'm doubtful about. First, we are colliding > >> here between the goals of minimizing the storage size and making the > >> debug output more readable. > > I've

Re: Reducing output size of nodeToString

2024-01-31 Thread Peter Eisentraut
On 30.01.24 12:26, Matthias van de Meent wrote: Most of the other defaults I'm doubtful about. First, we are colliding here between the goals of minimizing the storage size and making the debug output more readable. I've never really wanted to make the output "more readable". The current one

Re: Reducing output size of nodeToString

2024-01-30 Thread Matthias van de Meent
On Tue, 9 Jan 2024, 09:23 Peter Eisentraut, wrote: > > On 04.01.24 00:23, Matthias van de Meent wrote: > > Something like the attached? It splits out into the following > > 0001: basic 'omit default values' > > /* Write an integer field (anything written as ":fldname %d") */ > -#define

Re: Reducing output size of nodeToString

2024-01-09 Thread Peter Eisentraut
On 04.01.24 00:23, Matthias van de Meent wrote: Something like the attached? It splits out into the following 0001: basic 'omit default values' /* Write an integer field (anything written as ":fldname %d") */ -#define WRITE_INT_FIELD(fldname) \ +#define WRITE_INT_FIELD_DIRECT(fldname) \

Re: Reducing output size of nodeToString

2024-01-03 Thread Matthias van de Meent
On Tue, 2 Jan 2024 at 11:30, Peter Eisentraut wrote: > > On 06.12.23 22:08, Matthias van de Meent wrote: > > PFA a patch that reduces the output size of nodeToString by 50%+ in > > most cases (measured on pg_rewrite), which on my system reduces the > > total size of pg_rewrite by 33% to 472KiB.

Re: Reducing output size of nodeToString

2024-01-03 Thread Matthias van de Meent
On Wed, 3 Jan 2024 at 03:02, David Rowley wrote: > > On Thu, 14 Dec 2023 at 19:21, Matthias van de Meent > wrote: > > > > On Thu, 7 Dec 2023 at 13:09, David Rowley wrote: > > > We could also easily serialize plans to binary format for copying to > > > parallel workers rather than converting

Re: Reducing output size of nodeToString

2024-01-02 Thread David Rowley
On Thu, 14 Dec 2023 at 19:21, Matthias van de Meent wrote: > > On Thu, 7 Dec 2023 at 13:09, David Rowley wrote: > > We could also easily serialize plans to binary format for copying to > > parallel workers rather than converting them to a text-based > > serialized format. It would also allow us

Re: Reducing output size of nodeToString

2024-01-02 Thread Peter Eisentraut
On 06.12.23 22:08, Matthias van de Meent wrote: PFA a patch that reduces the output size of nodeToString by 50%+ in most cases (measured on pg_rewrite), which on my system reduces the total size of pg_rewrite by 33% to 472KiB. This does keep the textual pg_node_tree format alive, but reduces its

Re: Reducing output size of nodeToString

2023-12-13 Thread Matthias van de Meent
On Thu, 7 Dec 2023 at 13:09, David Rowley wrote: > > On Thu, 7 Dec 2023 at 10:09, Matthias van de Meent > wrote: > > PFA a patch that reduces the output size of nodeToString by 50%+ in > > most cases (measured on pg_rewrite), which on my system reduces the > > total size of pg_rewrite by 33% to

Re: Reducing output size of nodeToString

2023-12-07 Thread David Rowley
On Thu, 7 Dec 2023 at 10:09, Matthias van de Meent wrote: > PFA a patch that reduces the output size of nodeToString by 50%+ in > most cases (measured on pg_rewrite), which on my system reduces the > total size of pg_rewrite by 33% to 472KiB. This does keep the textual > pg_node_tree format

Re: Reducing output size of nodeToString

2023-12-07 Thread Matthias van de Meent
On Thu, 7 Dec 2023 at 11:26, Peter Eisentraut wrote: > > On 06.12.23 22:08, Matthias van de Meent wrote: > > PFA a patch that reduces the output size of nodeToString by 50%+ in > > most cases (measured on pg_rewrite), which on my system reduces the > > total size of pg_rewrite by 33% to 472KiB.

Re: Reducing output size of nodeToString

2023-12-07 Thread Peter Eisentraut
On 06.12.23 22:08, Matthias van de Meent wrote: PFA a patch that reduces the output size of nodeToString by 50%+ in most cases (measured on pg_rewrite), which on my system reduces the total size of pg_rewrite by 33% to 472KiB. This does keep the textual pg_node_tree format alive, but reduces its

Reducing output size of nodeToString

2023-12-06 Thread Matthias van de Meent
Hi, PFA a patch that reduces the output size of nodeToString by 50%+ in most cases (measured on pg_rewrite), which on my system reduces the total size of pg_rewrite by 33% to 472KiB. This does keep the textual pg_node_tree format alive, but reduces its size signficantly. The basic techniques