On Sat, Jul 31, 2010 at 12:40 AM, Tom Lane <t...@sss.pgh.pa.us> wrote: > Another misbehavior that I noted while playing with Artur's example is > that, while GIN index build seems to adhere pretty well to whatever > maintenance_work_mem limit you give it in 8.4, it blows right by that > limit in 9.0 and HEAD --- I observed it happily eating something north > of 128MB with a maintenance_work_mem of 70MB. It looks to me like the > reason is the new "rbtree.c" code, which adds a whole lot of data > structure that's not being counted against the maintenance_work_mem > limit. > > Now the first question that might be asked is what we'd need to do to > rbtree.c's API to allow its memory consumption to be tracked. However, > I think there's a considerably more pressing question, which is what > is it that rbtree.c is doing that justifies a 2X bloat factor in GIN > index build --- which is pretty much what it's costing us, given the > observation above. We know that the amount of memory available for > the build has an enormous effect on build time. If we just do the > obvious thing of including the rbtree data structure in the > maintenance_work_mem calculation, what we're going to get is a very > substantial slowdown in build speed for the same maintenance_work_mem > setting, compared to the way 8.4 worked. > > So, I would like somebody to show cause why that whole module shouldn't > be ripped out and the code reverted to where it was in 8.4. My > recollection is that the argument for adding it was to speed things up > in corner cases, but what I think it's actually going to do is slow > things down in every case.
I've always been a bit suspicious of this code, too, even though I didn't think about the memory consumption issue. But see here: http://archives.postgresql.org/pgsql-hackers/2010-02/msg00307.php I think there may be a better way to avoid the pathological cases, but I'm not sure what it is. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
