Looks like the GByteArray is still being created. (gdb) break camel-mime-part-utils.c:82 Breakpoint 2 at 0xb6dd541e: file camel-mime-part-utils.c, line 82. (gdb) delete 1 (gdb) cont Continuing.
Breakpoint 2, simple_data_wrapper_construct_from_parser (dw=0xb3f02800, mp=0x827bcd0) at camel-mime-part-utils.c:82 82 if (buffer != NULL) { (gdb) print buffer $1 = (GByteArray *) 0x80e4dc0 (gdb) Breakpoint 1, camel_mime_parser_step (parser=0x827bcd0, databuffer=0xb4882f3c, datalength=0xb4882f40) at camel-mime-parser.c:610 610 struct _header_scan_state *s = _PRIVATE (parser); (gdb) bt #0 camel_mime_parser_step (parser=0x827bcd0, databuffer=0xb4882f3c, datalength=0xb4882f40) at camel-mime-parser.c:610 #1 0xb6dd5456 in simple_data_wrapper_construct_from_parser (dw=0xb3f02800, mp=0x827bcd0) at camel-mime-part-utils.c:81 #2 0xb6dd55e9 in camel_mime_part_construct_content_from_parser (dw=0x824e8c8, mp=0x827bcd0) at camel-mime-part-utils.c:127 #3 0xb6dd6ff1 in construct_from_parser (mime_part=0x824e8c8, mp=0x827bcd0) at camel-mime-part.c:968 #4 0xb6dd70af in camel_mime_part_construct_from_parser (mime_part=0x824e8c8, mp=0x827bcd0) at camel-mime-part.c:996 #5 0xb6de1aab in construct_from_parser (multipart=0x8246f80, mp=0x827bcd0) at camel-multipart.c:577 #6 0xb6de1bea in camel_multipart_construct_from_parser (multipart=0x8246f80, mp=0x827bcd0) at camel-multipart.c:609 #7 0xb6dd5681 in camel_mime_part_construct_content_from_parser (dw=0x8254570, mp=0x827bcd0) at camel-mime-part-utils.c:144 #8 0xb6dd6ff1 in construct_from_parser (mime_part=0x8254570, mp=0x827bcd0) at camel-mime-part.c:968 #9 0xb6dd1de4 in construct_from_parser (dw=0x8254570, mp=0x827bcd0) at camel-mime-message.c:597 #10 0xb6dd70af in camel_mime_part_construct_from_parser (mime_part=0x8254570, mp=0x827bcd0) at camel-mime-part.c:996 #11 0xb6dd7122 in construct_from_stream (dw=0x8254570, s=0x826ab10) at camel-mime-part.c:1012 #12 0xb6dc2f63 in camel_data_wrapper_construct_from_stream (data_wrapper=0x8254570, stream=0x826ab10) at camel-data-wrapper.c:270 #13 0xb60fbd97 in maildir_get_message (folder=0x80def28, uid=0x8269dd0 "1192085835.11467_1.evergrey", ........ [EMAIL PROTECTED]:~/Current/mailtests/md/spam1/cur$ ls -alh 1192085835.11467_1.evergrey\!2\,SH -rw-r--r-- 1 pvanhoof pvanhoof 401M 2008-01-27 13:28 1192085835.11467_1.evergrey!2,SH [EMAIL PROTECTED]:~/Current/mailtests/md/spam1/cur$ On Sat, 2008-01-26 at 23:22 -0500, Jeffrey Stedfast wrote: > Something like the attached patch might work, tho it is untested. > > If this doesn't work, then I suspect the problem is that the seek > position might get changed out from under the mime parser (assuming it > is using either a CamelStreamFs or an fd). > > Note that camel_stream_fs_new_with_fd[_and_bounds]() calls lseek() on > the fd passed in. > > >From the dup() man page: > > After a successful return from dup() or dup2(), the old and new file > descriptors may be used interchangeably. They refer to the same open > file description (see open(2)) and thus share file offset and file sta‐ > tus flags; for example, if the file offset is modified by using > lseek(2) on one of the descriptors, the offset is also changed for the > other. > > So my guess is that this will break the parser :( > > It might break in the stream case as well, you'd have to follow the code > paths a bit to know for sure. For instance, even if creating the > seekable substream doesn't perform an underlying seek on the original > stream, setting it in a data wrapper might call camel_stream_reset() > which /might/ do an lseek() on the source fs stream. > > Not an insurmountable problem to solve, but it does make things a little > more difficult and possibly touchy. > > Jeff > > > > On Sat, 2008-01-26 at 22:48 -0500, Jeffrey Stedfast wrote: > > On Sat, 2008-01-26 at 22:12 -0500, Jeffrey Stedfast wrote: > > > On Sat, 2008-01-26 at 13:44 +0100, Philip Van Hoof wrote: > > > > This is what happens if you try to open a truly large E-mail on a device > > > > that has not as much memory available: > > > > > > > > Is there something we can do about this? Can we change the MIME parsing > > > > algorithm to be less memory demanding for example? > > > > > > > > Note that GArray is not really very sparse with memory once you start > > > > having a really large array. Perhaps we can in stead change this to a > > > > normal pointer array of a fixed size (do we know the size before we > > > > start parsing, so that we can allocate an exact size in stead, perhaps?) > > > > > > eh, why would you change it to a GPtrArray? It doesn't hold pointers, it > > > holds message part content. > > > > > > Unfortunately we don't know the size ahead of time. > > > > > > I suppose you could use a custom byte array allocator so that you can > > > force it to grow by larger chunks or something, dunno. > > > > > > > > > The way GMime handles this is by not loading content into RAM, but > > > that may be harder to do with Camel, especially in the mbox case. > > > > er, I should probably explain this: > > > > - writing the code should be relatively easy to do, but in the mbox > > case, the mbox may end up getting expunged or rewritten for some other > > reason which may cause problems, not sure how that would work. > > > > I think in Maildir, as long as the fd remains open, the file won't > > actually disappear after an unlink() until the fd gets closed, so that > > might work out ok assuming you can spare the fd (which might be the > > other problem with Evolution?). > > > > Jeff > > > > > > _______________________________________________ > > Evolution-hackers mailing list > > Evolution-hackers@gnome.org > > http://mail.gnome.org/mailman/listinfo/evolution-hackers > _______________________________________________ > Evolution-hackers mailing list > Evolution-hackers@gnome.org > http://mail.gnome.org/mailman/listinfo/evolution-hackers -- Philip Van Hoof, freelance software developer home: me at pvanhoof dot be gnome: pvanhoof at gnome dot org http://pvanhoof.be/blog http://codeminded.be _______________________________________________ Evolution-hackers mailing list Evolution-hackers@gnome.org http://mail.gnome.org/mailman/listinfo/evolution-hackers