Bugs item #1593305, was opened at 2006-11-09 12:05 Message generated for change (Settings changed) made by boncz You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=482468&aid=1593305&group_id=56967
Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: PF/runtime Group: Pathfinder CVS Head >Status: Closed >Resolution: Wont Fix Priority: 3 Private: No Submitted By: Peter Boncz (boncz) Assigned to: Sjoerd Mullender (sjoerd) Summary: PF: perf issue: map_pid logging Initial Comment: map_pid is currently logged in full for each transaction. in the 10GB xmark, we map-pid has size 8000, which will times 1+8+8 bytes, thus 136KB per single update. given that map_pid is [void,oid], it would be better not log the head, and use the array-write functionality in the logger (that is faster and shorter than a log call for each bun). [ SIDE ISSUE: it would also be faster and shorter to use array-write for all other [oid,any] delta bats; splitting them by column ] additionally, given that the mapi_pid contains page numbers, we could convert it in te log to an [void,int] bat, or maybe even a [void,sht] bat, and dynamically cast upwards such datatypes during log recovery. In the 10GB case, this would reduce map_pid logging overhead to to 16KB per query. [ SIDE ISSUE AGAIN: array write save the log mode byte plus this upcasting will probably save additional space (on oids) on all other delta bats as well. An interesting idea would be to use the X100 compression interface for writing to the log, especially when all deltas are written with the array interface. ] But, for map_pid, it would be better to encode inserts and deletes, rather than log the entire thing: e.g. delta [oid,oid] head = insertpos, oid newtail where - insertpos is the original position after which the insert is done. - multiple identical insertpos entries indicate multipl tuples inserted at that pos - negative (higest bit set) insertpos are deletes ---------------------------------------------------------------------- >Comment By: Peter Boncz (boncz) Date: 2009-04-07 10:56 Message: will not be fixed anymore ---------------------------------------------------------------------- Comment By: Stefan Manegold (stmane) Date: 2007-06-04 20:38 Message: Logged In: YES user_id=572415 Originator: NO Peter, Sjoerd, is this one still an open issues after all the recent XQUF code changes/improvements? Stefan ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=482468&aid=1593305&group_id=56967 ------------------------------------------------------------------------------ This SF.net email is sponsored by: High Quality Requirements in a Collaborative Environment. Download a free trial of Rational Requirements Composer Now! http://p.sf.net/sfu/www-ibm-com _______________________________________________ Monetdb-bugs mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/monetdb-bugs
