Greetings, imap_update_summary is implemented in three or four steps:
o. Getting (all) the headers/uids o. Finding the ones that we must still fetch o. Fetching those (x) o. Writing out the summary The steps each consume memory and reuse some of the memory of the previous step. Pointers to that memory is stored in GPtrArray's like fetch_data and messages. In the code I have found no real reason to why this was done in separated loops (steps) rather than one step and at the end of the loop, free the data already. Especially for the third step (x), which seem to consume most memory while it's happening. The current implementation requires the data being received from the IMAP service to be kept in memory until the entire folder has been received and all steps done. This consumes more than one entire kilobyte per message. Multiply that with for example 5,000 headers and you'll get 5 MB memory consumption for fetching the new messages of a very small IMAP folder (in case no other messages had been received before you first started the procedure). Multiply that with 50,000 headers and you'll get 50 - 60 MB memory consumption for a not extremely big but relatively big IMAP folders. Which will be freed, yes, but nevertheless it's a slowly growing peak (the speed depends on the connection with your IMAP server) that only gets deallocated or when pressing cancel or when all messages are received (which can take a significant amount of time). The strange part is that if I measure the amount of bytes that I receive from the IMAP service; I measure far less bytes being transferred than bytes being consumed in memory. It not only stores all the received data, it also stores a lot more in memory (probably mostly 4 bytes pointers and GData stuff). I wonder whether there was a reason why it was implemented this way? If not, I'm planning to rewrite imap_update_summary in a different way. For example by immediately creating a CamelMessageInfo struct or burning it to the summary file instantly and freeing the GData created from the stream in-loop. -- Philip Van Hoof, software developer at x-tend home: me at pvanhoof dot be gnome: pvanhoof at gnome dot org work: vanhoof at x-tend dot be http://www.pvanhoof.be - http://www.x-tend.be _______________________________________________ Evolution-hackers mailing list [email protected] http://mail.gnome.org/mailman/listinfo/evolution-hackers
