https://issues.apache.org/SpamAssassin/show_bug.cgi?id=6793
--- Comment #1 from Mark Martinec <[email protected]> 2012-04-23 23:12:10 UTC --- > current version of sa-awl loads full database key list to memory before > showing any stats or performing maintenance. I believe it's obvious that > this behavior is undesirable and makes large databases impossible to handle I fully agree. Operations should not depend on being able to load a complete database into memory if the size of a database is potentially unlimited. > The patch below improves sa-awl scaling and responsiveness by scanning > database row-by-row basis instead of loading all keys to memory first. > Tested cleaning db with over 8 million rows. > For a cached db with 850K rows memory usage lowers from 1G to 6M, Great, thanks! > execution time is around 12% slower (26 vs. 29 sec), though. Despite Kevin's concern on the ML, I don't think this is worth introducing yet another option (and risking people using it without fully understanding the implications). One can make a program arbitrarily fast if correct and reliable operation is not mandated :) > I'm not a perl expert, please review. My only concern is deleting a key while the each() operator is active. Please see my alternative patch. -- Configure bugmail: https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are the assignee for the bug.
