On 04/06/2020 16:49, Vlad Khorsun wrote: >> >> I did some tests to see how many nodes were created and how much memory >> a statement pool uses: >> >> -- 39 nodes, 31728 bytes allocated >> select * from rdb$database; >> >> -- 109 nodes, 91904 bytes allocated >> select * from rdb$database >> union all >> select * from rdb$database >> union all >> select * from rdb$database >> union all >> select * from rdb$database >> union all >> select * from rdb$database; >> >> -- 163 nodes, 120496 bytes allocated >> execute block >> as >> declare n integer; >> begin >> select rdb$relation_id from rdb$database into n; >> -- select repeated 20 more times >> end! >> >> So multiplying number of nodes by 16 we have around 2% of waste in these >> tests. >
The total bytes allocated that I passed is less in releasae. The numbers I passed was in debug, so in percentage the waste is a bit more. > Can't say it is a big win - to save 2% of memory, sorry. > Really, it's not big numbers alone. But it's relative easy to do. > > It is doable of course, but if you want to really save memory (more > then 2%) > you could find more ways to do it. Sure. > For example - literal nodes. There is > a lot > of same literals used in many statements such as NULL, 0, 1, "" and so > on. And > there is no sense to allocate it again and again and again. > This is a good idea. I'll test it. > Next. To create compiled statement ready for execution from SQL > statement text, > engine: > a) generates DSQL tree > b) generates BLR code using DSQL tree > c) generates final compiled statement from BLR from (b). > > If I'm not mistaken, nodes created at step (a) are allocated from > statement pool > and not deleted until statement is destroyed. Is it possible to allocate > them > from separate pool (dsql scratch pool, probably ?) and to free it after > step (b) ? > In master this is already done. There is two pools in DSQL: scratch and statement. Scratch is destroyed after DSQL compilation of DML. Statement pool is necessary as it's the DSQL live structures. > Next. Many nodes have two set of fields: for DSQL processing and for > BLR (JRD) > processing. Obviously, DSQL fields is not used at all after step (b). > Splitting > such nodes by two could save some memory, I think. > Yes, we must find a good way to do it. > Also, many nodes is replaced when statement is parsed. We could try to > re-use > such nodes, probably. > I'm not sure many (or most) is lost, but sure, we can improve. > Next. Many optimizer internal structures are allocated from statement > pool and > then really deleted. But memory allocated for it is not reused and > wasted. Moving > such objects into separate temporary pool allows to reduce memory > allocation (not > usage!) by 7-10% in some tests. > This is something like was done in DSQL, and sure it must be done. > There is also inefficient memory allocation for a statement's impure > area I'm > investigating currently. It is about parameters, variables and probably > some other > nodes. > Also, our impure_value (which is used in most places) become bigger. These are all things we can improve, more Alex's MetaName dictionary, it seems we can improve a lot. Adriano Firebird-Devel mailing list, web interface at https://lists.sourceforge.net/lists/listinfo/firebird-devel
