On Tue, Jan 19, 2016 at 12:22 AM, Tomas Vondra <tomas.von...@2ndquadrant.com> wrote: > I dare to claim that if expanded objects require a separate memory context > per aggregate state (I don't see why would be the case), it's a dead end. > Not so long ago we've fixed exactly this issue in array_agg(), which > addressed a bunch of reported OOM issues and improved array_agg() > performance quite a bit. It'd be rather crazy introducing the same problem > to all aggregate functions.
Expanded objects require a separate memory context per Datum. I think I know how to rejigger things so that the overhead of that is minimized as much as possible, but it's still 200 bytes for the AllocSetContext + ~16 bytes for the name + 32 bytes for an AllocBlock + 48 bytes for an ExpandedObjectHeader. That's about 300 bytes of overhead per expanded datum, which means on this test case the theoretical minimum memory burn for this approach is about 300 MB (and in practice somewhat more). Ouch. The upshot seems to be that expanded objects aren't going to actually be useful in any situation where you might have very many of them, because the memory overhead is just awful. That's rather disappointing. Even if you could get rid of the requirement for a memory context per Datum - and it seems like you could if you added a method defined to reparent the object to a designated context, which looks like a totally reasonable innovation - somebody's probably going to say, not without some justification, that 48 bytes of ExpandedObjectHeader per object is an unaffordable luxury here. The structure has 8 bytes of unnecessary padding in it that we could elide easily enough, but after that there's not really much way to squeeze it down without hurting performance in some case cases where this mechanism is fast today. So I guess I'm going to agree that this approach is doomed. Rats. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
