Bug#981485: offlineimap3: crash when uploading locally created no-utf-8 encoded message

2021-02-19 Thread Johannes Stezenbach 
Hi Sudip,

On Fri, Feb 19, 2021 at 03:59:35PM +, Sudip Mukherjee wrote:
> On Mon, Feb 1, 2021 at 10:57 PM Sudip Mukherjee
>  wrote:
> >
> > Control: forwarded -1 https://github.com/OfflineIMAP/offlineimap3/issues/43
> 
> Will it be possible to test the change at
> https://github.com/OfflineIMAP/offlineimap3/commit/62490ff1835b22941d1775d1d4bb98d042df026b
> and confirm if it fixes your issue..

I had a look but read the correponding pull request before
testing, which concludes that this doesn't work
(corrupts the encoding when uploading to IMAP).
https://github.com/OfflineIMAP/offlineimap3/pull/53

My initial hot-fix patch has the same problem.


Thanks,
Johannes

Bug#981485: offlineimap3: crash when uploading locally created no-utf-8 encoded message

2021-02-19 Thread Sudip Mukherjee 
Hi Johannes,

On Mon, Feb 1, 2021 at 10:57 PM Sudip Mukherjee
 wrote:
>
> Control: forwarded -1 https://github.com/OfflineIMAP/offlineimap3/issues/43
> --
>



>
> Thanks for the details. An issue has been opened upstream about it.

Will it be possible to test the change at
https://github.com/OfflineIMAP/offlineimap3/commit/62490ff1835b22941d1775d1d4bb98d042df026b
and confirm if it fixes your issue..



-- 
Regards
Sudip

Bug#981485: offlineimap3: crash when uploading locally created no-utf-8 encoded message

2021-02-01 Thread Sudip Mukherjee 
Control: forwarded -1 https://github.com/OfflineIMAP/offlineimap3/issues/43
--

Hi Johannes,

On Sun, Jan 31, 2021 at 08:48:23PM +0100, Johannes Stezenbach wrote:
> Package: offlineimap3
> Version: 0.0~git20210105.00d395b+dfsg-2
> Severity: important
> 
> Dear Maintainer,
> 
> I sent a message composed in mutt, afterwards offlineimap3 crashed
> while trying to upload the sent message to IMAP server
> with message "UnicodeDecodeError: 'utf-8' codec can't
> decode byte 0xfc in position 317: invalid start byte".
> The message was composed in mutt and contained German umlauts.
> My system uses utf-8 encoding, however mutt has a send_charset
> option that defaults to “us-ascii:iso-8859-1:utf-8” and thus the
> message was encoded as "Content-Type: text/plain; charset=iso-8859-1".

Thanks for the details. An issue has been opened upstream about it.

--
Regards
Sudip

Bug#981485: offlineimap3: crash when uploading locally created no-utf-8 encoded message

2021-01-31 Thread Johannes Stezenbach 
Package: offlineimap3
Version: 0.0~git20210105.00d395b+dfsg-2
Severity: important

Dear Maintainer,

I sent a message composed in mutt, afterwards offlineimap3 crashed
while trying to upload the sent message to IMAP server
with message "UnicodeDecodeError: 'utf-8' codec can't
decode byte 0xfc in position 317: invalid start byte".
The message was composed in mutt and contained German umlauts.
My system uses utf-8 encoding, however mutt has a send_charset
option that defaults to “us-ascii:iso-8859-1:utf-8” and thus the
message was encoded as "Content-Type: text/plain; charset=iso-8859-1".

Full error message (for real):

OfflineIMAP 7.3.0
  Licensed under the GNU GPL v2 or any later version (with an OpenSSL exception)
imaplib2 v3.05, Python v3.9.1+, OpenSSL 1.1.1i  8 Dec 2020
Account sync foo:
 *** Processing account foo
 Establishing connection to mail.foo.com:993 (fooRemote)
Folder INBOX [acc: foo]:
 Syncing INBOX: IMAP -> Maildir
 Copy message UID -2 (1/2) fooLocal:INBOX -> fooRemote:INBOX
 ERROR: Copying message -2 [acc: foo]
  'utf-8' codec can't decode byte 0xfc in position 317: invalid start byte
 ERROR: while syncing INBOX [account foo]
  'utf-8' codec can't decode byte 0xfc in position 317: invalid start byte
 ERROR: ERROR in syncfolder for foo folder INBOX: Traceback (most recent call 
last):
  File "/usr/share/offlineimap3/offlineimap/accounts.py", line 666, in 
syncfolder
localfolder.syncmessagesto(remotefolder, statusfolder)
  File "/usr/share/offlineimap3/offlineimap/folder/Base.py", line 1186, in 
syncmessagesto
action(dstfolder, statusfolder)
  File "/usr/share/offlineimap3/offlineimap/folder/Base.py", line 1013, in 
__syncmessagesto_copy
self.copymessageto(uid, dstfolder, statusfolder, register=0)
  File "/usr/share/offlineimap3/offlineimap/folder/Base.py", line 902, in 
copymessageto
message = self.getmessage(uid)
  File "/usr/share/offlineimap3/offlineimap/folder/Maildir.py", line 262, in 
getmessage
retval = file.read()
  File "/usr/lib/python3.9/codecs.py", line 322, in decode
(result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xfc in position 317: 
invalid start byte

  'utf-8' codec can't decode byte 0xfc in position 317: invalid start byte
Account sync foo:
 *** Finished account 'foo' in 0:00
ERROR: Exceptions occurred during the run!
ERROR: Copying message -2 [acc: foo]
  'utf-8' codec can't decode byte 0xfc in position 317: invalid start byte

Traceback:
  File "/usr/share/offlineimap3/offlineimap/folder/Base.py", line 902, in 
copymessageto
message = self.getmessage(uid)
  File "/usr/share/offlineimap3/offlineimap/folder/Maildir.py", line 262, in 
getmessage
retval = file.read()
  File "/usr/lib/python3.9/codecs.py", line 322, in decode
(result, consumed) = self._buffer_decode(data, self.errors, final)

ERROR: while syncing INBOX [account foo]
  'utf-8' codec can't decode byte 0xfc in position 317: invalid start byte

Traceback:
  File "/usr/share/offlineimap3/offlineimap/folder/Base.py", line 1186, in 
syncmessagesto
action(dstfolder, statusfolder)
  File "/usr/share/offlineimap3/offlineimap/folder/Base.py", line 1013, in 
__syncmessagesto_copy
self.copymessageto(uid, dstfolder, statusfolder, register=0)
  File "/usr/share/offlineimap3/offlineimap/folder/Base.py", line 902, in 
copymessageto
message = self.getmessage(uid)
  File "/usr/share/offlineimap3/offlineimap/folder/Maildir.py", line 262, in 
getmessage
retval = file.read()
  File "/usr/lib/python3.9/codecs.py", line 322, in decode
(result, consumed) = self._buffer_decode(data, self.errors, final)

ERROR: ERROR in syncfolder for foo folder INBOX: Traceback (most recent call 
last):
  File "/usr/share/offlineimap3/offlineimap/accounts.py", line 666, in 
syncfolder
localfolder.syncmessagesto(remotefolder, statusfolder)
  File "/usr/share/offlineimap3/offlineimap/folder/Base.py", line 1186, in 
syncmessagesto
action(dstfolder, statusfolder)
  File "/usr/share/offlineimap3/offlineimap/folder/Base.py", line 1013, in 
__syncmessagesto_copy
self.copymessageto(uid, dstfolder, statusfolder, register=0)
  File "/usr/share/offlineimap3/offlineimap/folder/Base.py", line 902, in 
copymessageto
message = self.getmessage(uid)
  File "/usr/share/offlineimap3/offlineimap/folder/Maildir.py", line 262, in 
getmessage
retval = file.read()
  File "/usr/lib/python3.9/codecs.py", line 322, in decode
(result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xfc in position 317: 
invalid start byte

  'utf-8' codec can't decode byte 0xfc in position 317: invalid start byte


After debugging a bit I came up with this hotfix to get me out of
the situation without deleting the offending email:

--- offlineimap/folder/Maildir.py.orig  2021-01-05 09:55:15.0 +0100
+++ offlineimap/folder/Maildir.py   2021-01-31 20:25:39.570660781 +0100
@@