Tom Lane wrote:
Yeah, I've just been doing that and some hand analysis too. What I get
(on a 64-bit machine) is that essentially all the space goes into
lists of A_Const lists: 320000000
lists of Const lists: 320000000
transformInsertRow extra lists: 144000000
I think we could safely list_free the input list in transformInsertRow
as your patch suggests, which would buy back the 144M part. But I don't
believe it's safe at all to free the raw_parser output --- the grammar
sometimes makes multiple links to the same subtree, eg in BETWEEN.
In any case the patch as proposed wouldn't catch all the detritus for
any case more complicated than a simple integer constant.
The way that the list memory usage works (again, 64-bit machine) is
sizeof(List) = 24
sizeof(ListCell) = 16
sizeof(A_Const) = 32
Each of these nodes will have 16 bytes palloc overhead, and the List
header will be rounded up to 32 bytes as well, so we have total space
for a 3-element integer list of
32+16 + (16+16 + 32+16) * 3
Add in 16+16 for the associated ListCell of the top list-of-lists,
and you come to 320 bytes per sublist. Const happens to also be
32 bytes so the transformed lists are the same size.
What if we built an array of A_Const nodes instead of a List? Maybe we
could use something akin to appendStringInfo()/enlargeStringInfo() to
build the array of nodes and enlarge it in chunks.
---------------------------(end of broadcast)---------------------------
TIP 5: don't forget to increase your free space map settings