Bertrand Petit <[EMAIL PROTECTED]> writes:And I just got another one, much simpler, that failed the same way with the same data set: UPDATE rimdb_atitles SET aka_title=convert(byte_title,charset,'UTF8');
[ where rimdb_atitles has an index on column "attribs varchar[]" ]
Uh-huh. Actually, any large insert or update on that table will run out of memory, I bet. The problem appears to be due to the newly-added support for indexing array columns --- array_cmp() leaks memory, which is verboten for index support operators.
Ugh.
I can think of a number of ways we might attack this, but none seem especially attractive ---
1. Have the index AMs create and switch into a special memory context for each call, rather than running in the main execution context. I am not sure this is workable at all, since the AMs tend to think they can create data structures that will live across calls (for example a btree lookup stack). It'd be the most general solution, if we could make it work.
This seems like a risky change at this point.
2. Modify the index AMs so that the comparison function FmgrInfo is preserved across a whole query. I think this requires changes to the index AM API (index_insert for instance has no provision for sharing data across multiple calls). Messy, and would likely mean an initdb. It would probably be the fastest answer though, since lookups wouldn't need to be done more than once per query.
This seems like a fairly big change this late in the game too.
3. Set up a long-lived cache internal to the array functions that can translate element type OID to the needed lookup data, and won't leak memory across repeated calls. This is not the fastest or most general solution, but it seems the most localized and safest fix.
I think I like #3 the best, but maybe that's because it's the one I think I understand the best ;-)
It seems to me that #3 is the least risky, and even if it isn't the best possible performance, this is the initial implementation of indexes on arrays, so it isn't like we're taking away something. Maybe solution #2 is better held as a performance enhancement for 7.5.
Do you want me to take a shot at this since I created the mess?
Joe
---------------------------(end of broadcast)--------------------------- TIP 5: Have you checked our extensive FAQ?
http://www.postgresql.org/docs/faqs/FAQ.html
