[Imap-uw] Problem: mail_fetchtext() incomplete result
Hi all, I'm creating an IMAP backup app which basicaly fetches all mails and dumps them to a Maildir. I'm using UW's c-client as the IMAP interface API. I've been wrestling with a weird bug for some time now, and since I can't find a solution I'm turning to this list. When calling mail_fetchtext(), some messages return an incomplete string. I've only witnessed this on attachments, they're truncated to only a couple of bytes (like 60 bytes). If I use mail_fetchbody() for the section where the mail was truncated, I do get the complete string. It always happens on the same mails, and they are truncated to exactly the same amount, yet I can't find a common cause. Some mails are fetched succesfully with attachments of up to 8MB, while some problematic mails are less than that, so size is not the issue. It's not a NULL-character issue because the truncation also happens in base64 encoded strings. Any ideas what might be causing it and how I could fix it? I'd rather not use mail_fetchbody() because of the added overhead in glueing the parts together again afterwards. Thanks, David -- If automobiles had followed the same development cycle as the computer, a Rolls-Royce would today cost $100, get a million miles per gallon, and explode once a year, killing everyone inside. - Robert X. Cringely ___ Imap-uw mailing list Imap-uw@u.washington.edu http://mailman2.u.washington.edu/mailman/listinfo/imap-uw
Re: [Imap-uw] Problem: mail_fetchtext() incomplete result
Turn on debugging telemetry and you will see the IMAP protocol interactions via mm_dlog(). Is the data being truncated from the server? Note that mm_dlog() will not show the data in literals, but you will see the byte counts and that should indicate if you are getting truncated data. If the data is truncated from the server, that's where you need to persue your further investigations. If it's coming from the server OK, then there is some problem in how you are calling c-client. Are you aware that the buffer used by mail_fetchtext() is used by other functions? This means that you can only use the return data from a mail_fetch***() function until you call another mail_fetch() function. A common mistake is to call mail_fetchheader() and then mail_fetchtext() and expect to use both pointers afterwards. In any case, be sure to verify that the server isn't doing this, especially if it is not one of the standard good ones such as UW/Panda, Cyrus, or Dovecot. -- Mark -- http://panda.com/mrc Science does not emerge from voting, party politics, or public debate. Si vis pacem, para bellum. ___ Imap-uw mailing list Imap-uw@u.washington.edu http://mailman2.u.washington.edu/mailman/listinfo/imap-uw
Re: [Imap-uw] Problem: mail_fetchtext() incomplete result
Thanks Mark. I wasn't aware of the single buffer for all mail_fetch***() functions, but I'm converting the results to std::strings directly afterwards so it doesn't affect the app. Turning on debugging telemetry showed that the message is transmitted truncated by the server. This is the Gmail IMAP server. I guess I'll have to download the messages in parts and then glue them back together again. Thanks for your help! David On Sun, Jul 19, 2009 at 7:24 PM, Mark Crispinmrc...@panda.com wrote: Turn on debugging telemetry and you will see the IMAP protocol interactions via mm_dlog(). Is the data being truncated from the server? Note that mm_dlog() will not show the data in literals, but you will see the byte counts and that should indicate if you are getting truncated data. If the data is truncated from the server, that's where you need to persue your further investigations. If it's coming from the server OK, then there is some problem in how you are calling c-client. Are you aware that the buffer used by mail_fetchtext() is used by other functions? This means that you can only use the return data from a mail_fetch***() function until you call another mail_fetch() function. A common mistake is to call mail_fetchheader() and then mail_fetchtext() and expect to use both pointers afterwards. In any case, be sure to verify that the server isn't doing this, especially if it is not one of the standard good ones such as UW/Panda, Cyrus, or Dovecot. -- Mark -- http://panda.com/mrc Science does not emerge from voting, party politics, or public debate. Si vis pacem, para bellum. -- If automobiles had followed the same development cycle as the computer, a Rolls-Royce would today cost $100, get a million miles per gallon, and explode once a year, killing everyone inside. - Robert X. Cringely ___ Imap-uw mailing list Imap-uw@u.washington.edu http://mailman2.u.washington.edu/mailman/listinfo/imap-uw
Re: [Imap-uw] Problem: mail_fetchtext() incomplete result
Found an even easier solution: Reading the IMAP specification I noticed one can use mail_fetchbody() with an empty section specifier () to retrieve the whole mail, including headers. Downloading the message this way makes the Gmail IMAP server transmit the whole message, untruncated. Now I can even skip mail_fetchheader() and just use mail_fetchbody(). Two flies with one stone. David On Sun, Jul 19, 2009 at 8:20 PM, David Verhasseltda...@crowdway.com wrote: Thanks Mark. I wasn't aware of the single buffer for all mail_fetch***() functions, but I'm converting the results to std::strings directly afterwards so it doesn't affect the app. Turning on debugging telemetry showed that the message is transmitted truncated by the server. This is the Gmail IMAP server. I guess I'll have to download the messages in parts and then glue them back together again. Thanks for your help! David On Sun, Jul 19, 2009 at 7:24 PM, Mark Crispinmrc...@panda.com wrote: Turn on debugging telemetry and you will see the IMAP protocol interactions via mm_dlog(). Is the data being truncated from the server? Note that mm_dlog() will not show the data in literals, but you will see the byte counts and that should indicate if you are getting truncated data. If the data is truncated from the server, that's where you need to persue your further investigations. If it's coming from the server OK, then there is some problem in how you are calling c-client. Are you aware that the buffer used by mail_fetchtext() is used by other functions? This means that you can only use the return data from a mail_fetch***() function until you call another mail_fetch() function. A common mistake is to call mail_fetchheader() and then mail_fetchtext() and expect to use both pointers afterwards. In any case, be sure to verify that the server isn't doing this, especially if it is not one of the standard good ones such as UW/Panda, Cyrus, or Dovecot. -- Mark -- http://panda.com/mrc Science does not emerge from voting, party politics, or public debate. Si vis pacem, para bellum. -- If automobiles had followed the same development cycle as the computer, a Rolls-Royce would today cost $100, get a million miles per gallon, and explode once a year, killing everyone inside. - Robert X. Cringely -- If automobiles had followed the same development cycle as the computer, a Rolls-Royce would today cost $100, get a million miles per gallon, and explode once a year, killing everyone inside. - Robert X. Cringely ___ Imap-uw mailing list Imap-uw@u.washington.edu http://mailman2.u.washington.edu/mailman/listinfo/imap-uw
