Re: [Evolution-hackers] Moving the struct instance heap space to mmap

2006-09-11 Thread Philip Van Hoof
On Mon, 2006-09-11 at 12:07 +0200, Till Adam wrote: Excluding some people and mailing lists. Having attempted various ways of implementing on-disk and in-memory indexing, the last and current being mmap'd binary on-disk structures (with a lovely collection of data integrity, robustness and

Re: [Evolution-hackers] Moving the struct instance heap space to mmap

2006-09-08 Thread Eero Tamminen
Hi, ext Federico Mena Quintero wrote: gpointer gimme_ptr_to_subject_for_message_number (int n) { return pointers_to_message_summaries[n] + SUBJECT_OFFSET; } Functions that do nothing else besides single array or struct lookup are hopefully static inlines in some header, so that the

Re: [Evolution-hackers] Moving the struct instance heap space to mmap

2006-09-08 Thread Philip Van Hoof
On Thu, 2006-09-07 at 21:48 -0500, Federico Mena Quintero wrote: On Thu, 2006-09-07 at 21:14 +0200, Philip Van Hoof wrote: The *new*/*extra* idea is to create a second index file which contains the offsets to the pointers in the camel summary file. Then mmap also that file. Extra because

Re: [Evolution-hackers] Moving the struct instance heap space to mmap

2006-09-08 Thread Philip Van Hoof
On Fri, 2006-09-08 at 11:22 -0500, Federico Mena Quintero wrote: On Fri, 2006-09-08 at 10:37 +0200, Philip Van Hoof wrote: 1. What is the maximum length of those strings? What is the maximum length of one message header in the summary file? In the message info struct, can you use short ints

Re: [Evolution-hackers] Moving the struct instance heap space to mmap

2006-09-07 Thread Philip Van Hoof
On Thu, 2006-09-07 at 21:14 +0200, Philip Van Hoof wrote: typedef struct { CamelFolderSummary *fs; int nth; MemoryMessageInfo *m; } CamelMessageInfo; #define camel_message_info_from (x) \ ((x)-m ? (x)-m-from : (x)-fs-sstart+*((x)-fs-istart\

Re: [Evolution-hackers] Moving the struct instance heap space to mmap

2006-09-07 Thread Philip Van Hoof
On Thu, 2006-09-07 at 21:14 +0200, Philip Van Hoof wrote: To read message n, you would simply do something like: from = sstart + *(istart + (sizeof (int) * 4 * n) + 1) subject = sstart + *(istart + (sizeof (int) * 4 * n) + 2) to = sstart + *(istart + (sizeof (int) * 4 * n) + 3) flags =

Re: [Evolution-hackers] Moving the struct instance heap space to mmap

2006-09-07 Thread Federico Mena Quintero
On Thu, 2006-09-07 at 21:14 +0200, Philip Van Hoof wrote: The *new*/*extra* idea is to create a second index file which contains the offsets to the pointers in the camel summary file. Then mmap also that file. Extra because the idea will build on top of the existing Ummm, but this won't

Re: [Evolution-hackers] Moving the struct instance heap space to mmap

2006-09-07 Thread Michael Zucchi
Hmm, will it really reduce memory usage though? I'm not so sure. Remember that although the summary file contains all strings on disk, in memory they are actually unique-ized. For a lot of load-types (e.g. mailing lists), this saves a trememdous amount of memory - even after all the overhead of