Hi, Thomas!
On Fri, 21 Feb 2003, Thomas Petersen wrote:
>Hi
>I'm using Bincimap in conjunction with php and I have noted that binc is very
>slow when I fetch an overview of the foldercontents with the imap_sort()
>function in php.
>With ethereal I can see that the request from php (or actually the c-client
>library) are as following:
>FETCH 1:20 (UID ENVELOPE BODY.PEEK[HEADER.FIELDS (Path Message-ID Content-MD5
>Content-Disposition Content-Language Content-Location Newsgroups Followup-To
>References)] INTERNALDATE RFC822.SIZE FLAGS)
>From bincimapd-fetch.cc I can tell that the fetch arguments RFC822.SIZE and
>BODY.PEEK causes binc to fully parse given messages even though only stuff
>from the header is returned (apart from the size maybe...).
The size, exactly ;) When fetching headers, only the header is parsed, but
the size must be derived properly.
>As an experiment I hacked binc up a bit (see the patch later in this e-mail).
>I changed bincimapd-fetch.cc so that it only calls message->parseOnlyHeader
>and parsers/mime/mime-parseonlyheader.cc so it sets the headerlength and size
>member variables.
>There might be a small problem with the size. With my change the size is
>simply the size of the file. The parseFull function calculates the size in an
>other way. As far as I can tell the difference is that in parseFull both CRLF
>and LF accounts for only one byte. With the filesize sollution CRLF gets
>counted as two bytes.
RFC822.SIZE means the exact size of a document fetched with BODY[], and we
can not use the size of the file here. We can, however, use the cache file
so that calculating the file size is done only once.
The header section would still have to be parsed, though this is a fairly
quick operation.
We could have a quick operation for RFC822.SIZE that only counts size in
bytes and does not actually parse the mime doc. I can do some tests here
to see if there is any noticable difference. But I think cacheing the size
in the cache file might be the best.
Thanks for providing a patch - you're setting a good example for the
others in this list. :)
Andy
>Here's the patch:
>
>diff -u -r bincimap-1.0.23/src/bincimapd-fetch.cc
>bincimap-1.0.23-patched/src/bincimapd-fetch.cc
>--- bincimap-1.0.23/src/bincimapd-fetch.cc 2003-02-08 14:49:13.000000000 +0100
>+++ bincimap-1.0.23-patched/src/bincimapd-fetch.cc 2003-02-20
>15:48:47.000000000 +0100
>@@ -498,8 +498,8 @@
> com << prefix;
> hasprinted = true;
>
>- if (!message->isAllParsed()) {
>- if (prevmessage != message || (prevmessage != NULL &&
>!prevmessage->isAllParsed())) {
>+ if (!message->isHeaderParsed()) {
>+ if (prevmessage != message || (prevmessage != NULL &&
>!prevmessage->isHeaderParsed())) {
> if (prevmessage != NULL) {
> prevmessage->close();
> prevmessage = NULL;
>@@ -509,8 +509,8 @@
>
> if ((fp = depot->open(message->getID())) == NULL)
> throw MessageException("error opening file");
>-
>- message->parseFull(fp);
>+
>+ message->parseOnlyHeader(fp);
> fclose(fp);
> fp = NULL;
> }
>@@ -579,23 +579,6 @@
>
> bool peek = (fatt.type == "BODY.PEEK");
>
>- if (!message->isAllParsed()) {
>- if (prevmessage != message || (prevmessage != NULL &&
>!prevmessage->isAllParsed())) {
>- if (prevmessage != NULL) {
>- prevmessage->close();
>- prevmessage = NULL;
>- }
>- prevmessage = message;
>- }
>-
>- if ((fp = depot->open(message->getID())) == NULL)
>- throw MessageException("error opening file");
>-
>- message->parseFull(fp);
>- fclose(fp);
>- fp = NULL;
>- }
>-
> com << fatt.toString();
>
> bool includeheaders = true;
>@@ -609,6 +592,42 @@
> else if (fatt.sectiontext == "TEXT" || fatt.sectiontext == "")
> bodyfetch = true;;
>
>+ if (bodyfetch || fatt.section != "") {
>+ if (!message->isAllParsed()) {
>+ if (prevmessage != message || (prevmessage != NULL &&
>!prevmessage->isAllParsed())) {
>+ if (prevmessage != NULL) {
>+ prevmessage->close();
>+ prevmessage = NULL;
>+ }
>+ prevmessage = message;
>+ }
>+
>+ if ((fp = depot->open(message->getID())) == NULL)
>+ throw MessageException("error opening file");
>+
>+ message->parseFull(fp);
>+ fclose(fp);
>+ fp = NULL;
>+ }
>+ } else {
>+ if (!message->isHeaderParsed()) {
>+ if (prevmessage != message || (prevmessage != NULL &&
>!prevmessage->isHeaderParsed())) {
>+ if (prevmessage != NULL) {
>+ prevmessage->close();
>+ prevmessage = NULL;
>+ }
>+ prevmessage = message;
>+ }
>+
>+ if ((fp = depot->open(message->getID())) == NULL)
>+ throw MessageException("error opening file");
>+
>+ message->parseOnlyHeader(fp);
>+ fclose(fp);
>+ fp = NULL;
>+ }
>+ }
>+
> const MimePart *mptr = message;
> string genpart;
>
>diff -u -r bincimap-1.0.23/src/parsers/mime/mime-parseonlyheader.cc
>bincimap-1.0.23-patched/src/parsers/mime/mime-parseonlyheader.cc
>--- bincimap-1.0.23/src/parsers/mime/mime-parseonlyheader.cc 2003-02-16
>17:28:59.000000000 +0100
>+++ bincimap-1.0.23-patched/src/parsers/mime/mime-parseonlyheader.cc
>2003-02-20 15:42:29.000000000 +0100
>@@ -50,6 +50,8 @@
> #include <stdio.h>
> #include <errno.h>
>
>+#include <sys/stat.h>
>+
> using namespace ::std;
> using namespace Binc::Util;
> using namespace Binc::io;
>@@ -91,7 +93,7 @@
> memset(cqueue, 0, sizeof(cqueue));
>
> headerstartoffset = bytenr;
>- // int headerstartoffsetcrlf = crlfnr;
>+ int headerstartoffsetcrlf = crlfnr;
>
> bool quit = false;
> int c = '\0';
>@@ -188,5 +190,11 @@
> h.add(name, content);
> }
>
>+ headerlength = crlfnr - headerstartoffsetcrlf;
>+
>+ struct stat attributes;
>+ fstat(fileno(fp), &attributes);
>+ size = attributes.st_size;
>+
> return 1;
> }
>
>
>--
>Thomas Petersen
>
>------------ Output from gpg ------------
>gpg: please see http://www.gnupg.org/faq.html for more information
>gpg: Signature made Fri 21 Feb 2003 10:15:09 AM CET using DSA key ID A8FBA173
>gpg: Can't check signature: public key not found
>
>
--
Andreas Aardal Hanssen | http://www.andreas.hanssen.name/gpg
Author of Binc IMAP | Nil desperandum
