Re: Can mutt prefetch mail

[Yubin Ruan]

On Tue, May 16, 2017 at 02:51:50PM +0800, Feng Liu wrote:
> This bothers me too. I'd rather use Fastmail instead since I suffered
> from this problem. Or just change the Gmail UI to English. This made me
> feel Gmail is not as good as it suppose to be.
 
If you wish I can share my .offlineimaprc with you:

[general]
accounts = Gmail
# this is used for folder name decode/encode.
pythonfile = ~/.offlineimap.py

[Account Gmail]
localrepository = Gmail-Local
remoterepository = Gmail-Remote

[Repository Gmail-Local]
type = Maildir
localfolders = ~/Mail/Gmail
nametrans = lambda foldername: 
foldername.decode('utf-8').encode('imap4-utf-7')

[Repository Gmail-Remote]
type = Gmail
remoteuser = 
remotepass = 
ssl=true
# if you cannot find this file, install `OpenSSL'
sslcacertfile = /etc/ssl/certs/ca-certificates.crt
realdelete = no
maxconnections = 5
nametrans = lambda foldername: 
foldername.decode('imap4-utf-7').encode('utf-8')

This works find for Gmail. The .offlineimap.py, which handles the
decoding/encoding stuff, can be found at:
https://gist.github.com/gauteh/5402888

By the way, can Fastmail advoid this problem? Doesn't it use imap?

--
Yubin
 
> 在 2017年05月16日 16:14, Yubin Ruan 写道:
> > On Sun, May 14, 2017 at 09:33:46AM +1000, Cameron Simpson wrote:
> >> On 13May2017 17:32, Yubin Ruan  wrote:
> >>> I am wondering whether it is possible to tell mutt to prefetch mails 
> >>> folder by
> >>> folder so that I can read mails more quickly without waiting for the 
> >>> "Fetching
> >>> mails..." every time.
> >>>
> >>> I have set up mutt's cache, so it helps a little, but it is still slow 
> >>> because I
> >>> have to fetch the mails before mutt can cache them.
> >>>
> >>> I know there is something like `fetchmail', but, setting up it is tricky,
> >>> especially with a IMAP server (or, do you have good references?)
> >> The other common solution for IMAP mail accounts is offlineimap, which will
> >> mirror IMAP accounts to local storage.
> >>
> >> Because mutt is single threaded, all "prefetch" arrangements tend to store
> >> the mail locally. This is several advantages:
> >>
> >>  - you don't need to configure mutt to access a remote mailbox
> >>
> >>  - if you're offline all your mail is still there (as of the last update)
> >>
> >>  - access is _very_ fast, because it is local file access
> >>
> >>  - if you run a local mail system, you can reply to email even when 
> >> offline;
> >> it will queue locally on your machine until there is network access.
> >>I find this great for train trips. This also means you don't need mutt 
> >> to
> >> know SMTP settings; just deliver locally via the "sendmail" command.
> >>
> >> The advantage of offlineimap is that (by default) it mirrors your IMAP
> >> account, keeping a local set of mail folders matching upstream. This means:
> >>
> >>  - you can still access you imap account (eg through a phone or other 
> >> device)
> >>
> >>  - changes you make locally via mutt, such as deletions of moving messages
> >> ormarking them read etc, are pushed upstream to your IMAP account for
> >> you
> >>
> >> Offlineimap _is_ a little tricky to set up, but once running you can let it
> >> look after the mirroring in the background.
> > Yes offlineimap _is_ very tricky to setup. It cannot handle non-ascii 
> > characters
> > correctly. I have several folders on the Gamil server whose name is in 
> > Chinese.
> > I setup a `nametrans' in the .offlineimaprc:
> >
> > nametrans = lambda foldername: 
> > foldername.decode('imap4-utf-7').encode('utf-8')
> >
> > it works great in the first download, but will throw some encoding/decoding 
> > errors
> > in the sync afterwards:
> >
> > 'ascii' codec can't decode byte 0xe9 in position 8: ordinal not in 
> > range(128)
> >
> > I don't know why a software would only support ascii in the 21st century...
> >
> > --
> > Yubin
> 
> 

[WAS: Re: Can mutt prefetch mail] Your GPG signatur is wrong

[Michelle Konzack]

Hello Feng,

you should correct your GPG signature.

-- 
Michelle KonzackMiila ITSystems @ TDnet
GNU/Linux Developer 00372-54541400


signature.asc
Description: Digital signature

Re: Can mutt prefetch mail

[Feng Liu]

This bothers me too. I'd rather use Fastmail instead since I suffered
from this problem. Or just change the Gmail UI to English. This made me
feel Gmail is not as good as it suppose to be.


在 2017年05月16日 16:14, Yubin Ruan 写道:
> On Sun, May 14, 2017 at 09:33:46AM +1000, Cameron Simpson wrote:
>> On 13May2017 17:32, Yubin Ruan  wrote:
>>> I am wondering whether it is possible to tell mutt to prefetch mails folder 
>>> by
>>> folder so that I can read mails more quickly without waiting for the 
>>> "Fetching
>>> mails..." every time.
>>>
>>> I have set up mutt's cache, so it helps a little, but it is still slow 
>>> because I
>>> have to fetch the mails before mutt can cache them.
>>>
>>> I know there is something like `fetchmail', but, setting up it is tricky,
>>> especially with a IMAP server (or, do you have good references?)
>> The other common solution for IMAP mail accounts is offlineimap, which will
>> mirror IMAP accounts to local storage.
>>
>> Because mutt is single threaded, all "prefetch" arrangements tend to store
>> the mail locally. This is several advantages:
>>
>>  - you don't need to configure mutt to access a remote mailbox
>>
>>  - if you're offline all your mail is still there (as of the last update)
>>
>>  - access is _very_ fast, because it is local file access
>>
>>  - if you run a local mail system, you can reply to email even when offline;
>> it will queue locally on your machine until there is network access.
>>I find this great for train trips. This also means you don't need mutt to
>> know SMTP settings; just deliver locally via the "sendmail" command.
>>
>> The advantage of offlineimap is that (by default) it mirrors your IMAP
>> account, keeping a local set of mail folders matching upstream. This means:
>>
>>  - you can still access you imap account (eg through a phone or other device)
>>
>>  - changes you make locally via mutt, such as deletions of moving messages
>> ormarking them read etc, are pushed upstream to your IMAP account for
>> you
>>
>> Offlineimap _is_ a little tricky to set up, but once running you can let it
>> look after the mirroring in the background.
> Yes offlineimap _is_ very tricky to setup. It cannot handle non-ascii 
> characters
> correctly. I have several folders on the Gamil server whose name is in 
> Chinese.
> I setup a `nametrans' in the .offlineimaprc:
>
> nametrans = lambda foldername: 
> foldername.decode('imap4-utf-7').encode('utf-8')
>
> it works great in the first download, but will throw some encoding/decoding 
> errors
> in the sync afterwards:
>
> 'ascii' codec can't decode byte 0xe9 in position 8: ordinal not in 
> range(128)
>
> I don't know why a software would only support ascii in the 21st century...
>
> --
> Yubin




signature.asc
Description: OpenPGP digital signature

Re: Can mutt prefetch mail

[Yubin Ruan]

On Tue, May 16, 2017 at 12:42:32PM +1000, Cameron Simpson wrote:
> On 16May2017 16:14, Yubin Ruan  wrote:
> >On Sun, May 14, 2017 at 09:33:46AM +1000, Cameron Simpson wrote:
> >>The other common solution for IMAP mail accounts is offlineimap, which will
> >>mirror IMAP accounts to local storage.
> [...]
> >Yes offlineimap _is_ very tricky to setup. It cannot handle non-ascii
> >characters correctly. I have several folders on the Gamil server whose name 
> >is in Chinese.
> >I setup a `nametrans' in the .offlineimaprc:
> >   nametrans = lambda foldername: 
> > foldername.decode('imap4-utf-7').encode('utf-8')
> 
> You also have a reverse name trans, yes? Both are needed. And one should
> take care that they really are the reverse of each other.

Hmm...turns out that I have too many accounts set up so that I forget one of the
reverse namestrans.

> 
> >it works great in the first download, but will throw some encoding/decoding 
> >errors
> >in the sync afterwards:
> >   'ascii' codec can't decode byte 0xe9 in position 8: ordinal not in 
> > range(128)
> >
> >I don't know why a software would only support ascii in the 21st century...
> 
> The offlineimap authors are definitely not pure Western
> can-get-by-with-ASCII people, so I expect they're aware of this shortcoming.
> 
> Is imap4-utf-7 a known encoding? My local Python 3 says:
> 
>  % python3 -c 'print(repr(b"foo".decode("imap4-utf-7")))'
>  Traceback (most recent call last):
>File "", line 1, in 
>LookupError: unknown encoding: imap4-utf-7
> 
> It is possible that your exception above comes from another part of the
> code; what is the stack trace? It would also be useful to hack things to
> print the repr of the undecodable string: do you know it is a mailbox name?
> 
> IIRC offlineimap uses python 2, which is a bit vague in distinguishing
> strings and bytes.

Here is the encode/decode python script I use currently:
https://gist.github.com/gauteh/5402888

--
Yubin

Re: Can mutt prefetch mail

[Cameron Simpson]

On 16May2017 16:14, Yubin Ruan  wrote:

On Sun, May 14, 2017 at 09:33:46AM +1000, Cameron Simpson wrote:

The other common solution for IMAP mail accounts is offlineimap, which will
mirror IMAP accounts to local storage.

[...]
Yes offlineimap _is_ very tricky to setup. It cannot handle non-ascii 
characters correctly. I have several folders on the Gamil server whose name is in Chinese.

I setup a `nametrans' in the .offlineimaprc:
   nametrans = lambda foldername: 
foldername.decode('imap4-utf-7').encode('utf-8')


You also have a reverse name trans, yes? Both are needed. And one should take 
care that they really are the reverse of each other.



it works great in the first download, but will throw some encoding/decoding 
errors
in the sync afterwards:
   'ascii' codec can't decode byte 0xe9 in position 8: ordinal not in 
   range(128)


I don't know why a software would only support ascii in the 21st century...


The offlineimap authors are definitely not pure Western can-get-by-with-ASCII 
people, so I expect they're aware of this shortcoming.


Is imap4-utf-7 a known encoding? My local Python 3 says:

 % python3 -c 'print(repr(b"foo".decode("imap4-utf-7")))'
 Traceback (most recent call last):
   File "", line 1, in 
   LookupError: unknown encoding: imap4-utf-7

It is possible that your exception above comes from another part of the code; 
what is the stack trace? It would also be useful to hack things to print the 
repr of the undecodable string: do you know it is a mailbox name?


IIRC offlineimap uses python 2, which is a bit vague in distinguishing strings 
and bytes.


Cheers,
Cameron Simpson 

Re: Can mutt prefetch mail

[Yubin Ruan]

On Sun, May 14, 2017 at 09:33:46AM +1000, Cameron Simpson wrote:
> On 13May2017 17:32, Yubin Ruan  wrote:
> >I am wondering whether it is possible to tell mutt to prefetch mails folder 
> >by
> >folder so that I can read mails more quickly without waiting for the 
> >"Fetching
> >mails..." every time.
> >
> >I have set up mutt's cache, so it helps a little, but it is still slow 
> >because I
> >have to fetch the mails before mutt can cache them.
> >
> >I know there is something like `fetchmail', but, setting up it is tricky,
> >especially with a IMAP server (or, do you have good references?)
> 
> The other common solution for IMAP mail accounts is offlineimap, which will
> mirror IMAP accounts to local storage.
> 
> Because mutt is single threaded, all "prefetch" arrangements tend to store
> the mail locally. This is several advantages:
> 
>  - you don't need to configure mutt to access a remote mailbox
> 
>  - if you're offline all your mail is still there (as of the last update)
> 
>  - access is _very_ fast, because it is local file access
> 
>  - if you run a local mail system, you can reply to email even when offline;
> it will queue locally on your machine until there is network access.
>I find this great for train trips. This also means you don't need mutt to
> know SMTP settings; just deliver locally via the "sendmail" command.
> 
> The advantage of offlineimap is that (by default) it mirrors your IMAP
> account, keeping a local set of mail folders matching upstream. This means:
> 
>  - you can still access you imap account (eg through a phone or other device)
> 
>  - changes you make locally via mutt, such as deletions of moving messages
> ormarking them read etc, are pushed upstream to your IMAP account for
> you
> 
> Offlineimap _is_ a little tricky to set up, but once running you can let it
> look after the mirroring in the background.

Yes offlineimap _is_ very tricky to setup. It cannot handle non-ascii characters
correctly. I have several folders on the Gamil server whose name is in Chinese.
I setup a `nametrans' in the .offlineimaprc:

nametrans = lambda foldername: 
foldername.decode('imap4-utf-7').encode('utf-8')

it works great in the first download, but will throw some encoding/decoding 
errors
in the sync afterwards:

'ascii' codec can't decode byte 0xe9 in position 8: ordinal not in 
range(128)

I don't know why a software would only support ascii in the 21st century...

--
Yubin

Re: Can mutt prefetch mail

[Cameron Simpson]

On 13May2017 20:43, Ian Zimmerman  wrote:

On 2017-05-14 09:33, Cameron Simpson wrote:

The other common solution for IMAP mail accounts is offlineimap


After trying: direct IMAP access with mutt, offlineimap, maildirsync and
even git (no kidding!), I settled on unison for this purpose.  It works
great for me.  Of course unlike the IMAP aware solutions it assumes
maildir is the underlying store, and not mbox (or a set of mboxes).

The one slight disadvantage of unison (in my situation) is that it is
quite sensitive to version harmony; you really must have not only the
same major version of unison on both sides, but also compiled with the
same major version of Ocaml.  Which means you mostly won't be able to
use binary packages if operating systems differ.


I'm presuming Ian is speaking of this:

 https://www.cis.upenn.edu/~bcpierce/unison/

Looks like it would be effective if you have access to run it on both your own 
machine and the IMAP server.


Cheers,
Cameron Simpson 

Re: Can mutt prefetch mail

[Ian Zimmerman]

On 2017-05-14 09:33, Cameron Simpson wrote:

> The other common solution for IMAP mail accounts is offlineimap

After trying: direct IMAP access with mutt, offlineimap, maildirsync and
even git (no kidding!), I settled on unison for this purpose.  It works
great for me.  Of course unlike the IMAP aware solutions it assumes
maildir is the underlying store, and not mbox (or a set of mboxes).

The one slight disadvantage of unison (in my situation) is that it is
quite sensitive to version harmony; you really must have not only the
same major version of unison on both sides, but also compiled with the
same major version of Ocaml.  Which means you mostly won't be able to
use binary packages if operating systems differ.

-- 
Please *no* private Cc: on mailing lists and newsgroups
Personal signed mail: please _encrypt_ and sign
Don't clear-text sign:
http://primate.net/~itz/blog/the-problem-with-gpg-signatures.html

Re: Can mutt prefetch mail

[Cameron Simpson]

On 13May2017 17:32, Yubin Ruan  wrote:

I am wondering whether it is possible to tell mutt to prefetch mails folder by
folder so that I can read mails more quickly without waiting for the "Fetching
mails..." every time.

I have set up mutt's cache, so it helps a little, but it is still slow because I
have to fetch the mails before mutt can cache them.

I know there is something like `fetchmail', but, setting up it is tricky,
especially with a IMAP server (or, do you have good references?)


The other common solution for IMAP mail accounts is offlineimap, which will 
mirror IMAP accounts to local storage.


Because mutt is single threaded, all "prefetch" arrangements tend to store the 
mail locally. This is several advantages:


 - you don't need to configure mutt to access a remote mailbox

 - if you're offline all your mail is still there (as of the last update)

 - access is _very_ fast, because it is local file access

 - if you run a local mail system, you can reply to email even when offline; 
   it will queue locally on your machine until there is network access.
   I find this great for train trips. This also means you don't need mutt to 
   know SMTP settings; just deliver locally via the "sendmail" command.


The advantage of offlineimap is that (by default) it mirrors your IMAP account, 
keeping a local set of mail folders matching upstream. This means:


 - you can still access you imap account (eg through a phone or other device)

 - changes you make locally via mutt, such as deletions of moving messages or 
   marking them read etc, are pushed upstream to your IMAP account for you


Offlineimap _is_ a little tricky to set up, but once running you can let it 
look after the mirroring in the background.


Cheers,
Cameron Simpson 

All mail clients suck. This one just sucks less. - www.mutt.org

Re: Can mutt prefetch mail

[Georg Faerber]

On 17-05-13 11:36:16, Chris Green wrote:
> On Sat, May 13, 2017 at 05:32:43PM +0800, Yubin Ruan wrote:
> > I am wondering whether it is possible to tell mutt to prefetch mails
> > folder by folder so that I can read mails more quickly without
> > waiting for the "Fetching mails..." every time.
> > 
> My solution for that is to run a mailserver on my desktop machine
> (where I run mutt) and have my mail delivered by SMTP.  That may or
> may not be a practical solution for others.

I'm doing the same. Bonus: Outbound queue if there is no Internet
access.


signature.asc
Description: Digital signature

Re: Can mutt prefetch mail

[Chris Green]

On Sat, May 13, 2017 at 05:32:43PM +0800, Yubin Ruan wrote:
> Hi,
> 
> I am wondering whether it is possible to tell mutt to prefetch mails folder by
> folder so that I can read mails more quickly without waiting for the "Fetching
> mails..." every time.
> 
My solution for that is to run a mailserver on my desktop machine
(where I run mutt) and have my mail delivered by SMTP.  That may or
may not be a practical solution for others.

-- 
Chris Green

Re: Can mutt prefetch mail

[Patrick Shanahan]

* Yubin Ruan  [05-12-17 22:39]:
> Hi,
> 
> I am wondering whether it is possible to tell mutt to prefetch mails folder by
> folder so that I can read mails more quickly without waiting for the "Fetching
> mails..." every time.
> 
> I have set up mutt's cache, so it helps a little, but it is still slow 
> because I
> have to fetch the mails before mutt can cache them.
> 
> I know there is something like `fetchmail', but, setting up it is tricky,
> especially with a IMAP server (or, do you have good references?)

I used fetchmailconf to start and then altered the configuration to suit
me.



set logfile ""
set postmaster ""
set no bouncemail
set no spambounce
set daemon 

poll pop.mail.yahoo.com tracepolls with proto POP3 timeout 60
   user '@yahoo.com' there with password '' is
   '

Can mutt prefetch mail

[Yubin Ruan]

Hi,

I am wondering whether it is possible to tell mutt to prefetch mails folder by
folder so that I can read mails more quickly without waiting for the "Fetching
mails..." every time.

I have set up mutt's cache, so it helps a little, but it is still slow because I
have to fetch the mails before mutt can cache them.

I know there is something like `fetchmail', but, setting up it is tricky,
especially with a IMAP server (or, do you have good references?)

---
Yubin