Re: why does mutt take so long to load my python folder?

2022-11-19 Thread Cameron Simpson

On 19Nov2022 09:21, Kevin J. McCarthy  wrote:

On Sat, Nov 19, 2022 at 02:32:25PM +1100, Cameron Simpson wrote:
Maybe the docs should have a small mention around `~h` that it 
cannot utilise the header cache.


There is a note, "***)", next to ~b, ~B, ~h, ~M, and ~X that mentions 
they entail reading the whole message in and recommends not using them 
for index coloring.  <http://www.mutt.org/doc/manual/#patterns>


It is as you say. Never mind then.

Thanks for the help,
Cameron Simpson 


Re: why does mutt take so long to load my python folder?

2022-11-19 Thread Cameron Simpson

On 19Nov2022 09:16, Kevin J. McCarthy  wrote:

On Sat, Nov 19, 2022 at 02:15:05PM +1100, Cameron Simpson wrote:

On 18Nov2022 18:33, Kevin J. McCarthy  wrote:
The headers are colored as they are displayed, but also when any 
flags are updated.


If it's only colouring the visible index listing, that's fine with me.


No, it's not.  I will fix that.


Thanks!

BTW, is there a way to tell if the current message is a collapsed 
thread in the index or just a single message? There are times when 
that would be useful.


%M might work for you.


Nice. I was actually thinking of a pattern, so that I could adjust some 
macros depending on the state of the current row in the index.


Cheers,
Cameron Simpson 


Re: why does mutt take so long to load my python folder?

2022-11-18 Thread Cameron Simpson

On 18Nov2022 23:08, Kurt Hackenberg  wrote:

On Sat, Nov 19, 2022 at 02:25:50PM +1100, Cameron Simpson wrote:

from https://www.rfc-wiki.org/wiki/RFC2822. Is there a better RFC 2822
page? I used to use a nicely formatted tools.ietf.org URL, but that now
redirects to some RFC author editing ... thing.


Is this better? (For RFC 5322, successor to 2822.)
<https://www.rfc-editor.org/rfc/rfc5322.html>


Heaps better, thank you. Looks like the tools.ietf.org version used to.

I forgot about 5322, even though I was referring to it only some months 
ago. ObMWC: My brain is full.


Cheers,
Cameron Simpson 


Re: why does mutt take so long to load my python folder?

2022-11-18 Thread Cameron Simpson

On 19Nov2022 14:25, Cameron Simpson  wrote:

Anyway, I'll try plain `~f "cameron simnpson"` and see how it goes.


This is nice and fast:

color index white default  "((~P|(%f polyname ~f 'cameron simpson'))) | (~v 
~((~P|(%f polyname ~f 'cameron simpson'"

and does the same colouring I wanted.

Small further analysis: the reason commenting out my auto aliases sped 
things up was that it made `%f polyname` fail (unknown group) and 
therefore did not attempt the `~h` match.


Maybe the docs should have a small mention around `~h` that it cannot 
utilise the header cache.


Thank you,
Cameron Simpson 


Re: why does mutt take so long to load my python folder?

2022-11-18 Thread Cameron Simpson

On 18Nov2022 18:33, Kevin J. McCarthy  wrote:

On Sat, Nov 19, 2022 at 08:32:39AM +1100, Cameron Simpson wrote:
It sure looks like it is being applied to the entire folder 
contents, instead of only the lines being displayed. That is a 
massive performance hit for a big folder. Is that the case?


The headers are colored as they are displayed, [...]


I was asking this because loading the python folder in a very short 
window seemed just as slow as in a very tall window.


Cheers,
Cameron Simpson 


Re: why does mutt take so long to load my python folder?

2022-11-18 Thread Cameron Simpson

On 18Nov2022 18:48, Kevin J. McCarthy  wrote:

On Sat, Nov 19, 2022 at 01:16:47PM +1100, Cameron Simpson wrote:

On 19Nov2022 08:32, Cameron Simpson  wrote:

The issue is my index colouring, specificly this line:
 color index white default  "((~P|%f polyname ~h '^from:.*cameron simpson')) | (~v 
~((~P|%f polyname ~h '^from:.*cameron simpson')))"


I'm wondering if it is the `~h` pattern. That would search every 
header, which I presume exceeds the cache?


~h always opens up the message and scans header by header.  It doesn't 
use the header cache.


Aha. Well that'll do it.


BTW ~() can also be very expensive.


If you mean: it involves walking the thread trees, I accept that. The 
win of having most threads callapsed but ocrrectly coloured outwieghs 
that. My naive perusing of the mutt code suggests that the thread 
structure itself is in memory, yes?


Is there an efficient way to search the _text_ of the `From:` 
header?  Not match the from address part, but the comment string 
(which is where the "cameron simpson" resides)?


~f scans the display name part of the address in most cases, including 
for color commands.


If you mean an actual address comment field, e.g. (text), those are 
considered the display name if none otherwise exists.  But if there is 
also a display name, the parser discards the comment.


Terminology: I'm taking "display name" to be as from here:

mailbox = name-addr / addr-spec
name-addr = [display-name] angle-addr
angle-addr = [CFWS] "<" addr-spec ">" [CFWS] / obs-angle-addr
display-name = phrase

from https://www.rfc-wiki.org/wiki/RFC2822. Is there a better RFC 2822 
page? I used to use a nicely formatted tools.ietf.org URL, but that now 
redirects to some RFC author editing ... thing.


In this From: header body:

Cameron Simpson (a comment) 

would `~f "cameron simpson"` match it, and would `~f "comment"` _not_ 
match it?


An example actual header I'm trying to match is this one:

From: "Cameron Simpson via Discussions on Python.org" 


Anyway, I'll try plain `~f "cameron simnpson"` and see how it goes.

Thanks,
Cameron Simpson 


Re: why does mutt take so long to load my python folder?

2022-11-18 Thread Cameron Simpson

On 18Nov2022 18:33, Kevin J. McCarthy  wrote:

On Sat, Nov 19, 2022 at 08:32:39AM +1100, Cameron Simpson wrote:
It sure looks like it is being applied to the entire folder 
contents, instead of only the lines being displayed. That is a 
massive performance hit for a big folder. Is that the case?


The headers are colored as they are displayed, but also when any flags 
are updated.


If it's only colouring the visible index listing, that's fine with me.

Do you have a folder-hook doing anything related to that, or message 
scoring going on related to the $score_threshold_* settings?


I don't use scoring.

I've got this:

folder-hook . 'push 
"~P~NN~T"'

which takes any new messages from me and cleares the new flag, as I 
think I should already know their content :-)


This push was important because I used to uncollapse any thread 
containing new messages. I'm not doing that now though, but I colour new 
messages and threads-with-new anyway, so it still has value.


BTW, is there a way to tell if the current message is a collapsed thread 
in the index or just a single message? There are times when that would 
be useful.


Cheers,
Cameron Simpson 


Re: why does mutt take so long to load my python folder?

2022-11-18 Thread Cameron Simpson

On 19Nov2022 08:32, Cameron Simpson  wrote:

The issue is my index colouring, specificly this line:

   color index white default  "((~P|%f polyname ~h '^from:.*cameron simpson')) | (~v 
~((~P|%f polyname ~h '^from:.*cameron simpson')))"

This colours messages as white if they're from me or from a polyname 
and the "from:" header contains my name (mailing lists where the from 
address part is the list, not the author, but the descriptive name 
contains the author - increasingly common). Same colour for a collapsed 
thread.


I'm wondering if it is the `~h` pattern. That would search every header, 
which I presume exceeds the cache?


Is there an efficient way to search the _text_ of the `From:` header?  
Not match the from address part, but the comment string (which is where 
the "cameron simpson" resides)?


Cheers,
Cameron Simpson 


Re: why does mutt take so long to load my python folder?

2022-11-18 Thread Cameron Simpson

On 19Nov2022 07:55, Cameron Simpson  wrote:

On 18Nov2022 09:38, Kevin J. McCarthy  wrote:

 source ~/rc/mutt/aliases-auto
8407 aliases, nearly 2MB in size :-)
Yes, Mutt's internal structures used for aliases are not designed for 
that many.


It's not the number of aliases. I've just stripped the generation down 
to the 2 that I really use: htmlers and polyname, 85 lines and 5kB.


The issue is my index colouring, specificly this line:

color index white default  "((~P|%f polyname ~h '^from:.*cameron simpson')) | 
(~v ~((~P|%f polyname ~h '^from:.*cameron simpson')))"

This colours messages as white if they're from me or from a polyname and 
the "from:" header contains my name (mailing lists where the from 
address part is the list, not the author, but the descriptive name 
contains the author - increasingly common). Same colour for a collapsed 
thread.


I use this see threads in which I participate coloured in white so that 
they stand out.


This single line is the source of the slowness in the python folder 
load.


It sure looks like it is being applied to the entire folder contents, 
instead of only the lines being displayed. That is a massive performance 
hit for a big folder. Is that the case?


There are only 8 addresses in the "polyname" alias/group.

Cheers,
Cameron Simpson 


Re: why does mutt take so long to load my python folder?

2022-11-18 Thread Cameron Simpson

On 18Nov2022 09:38, Kevin J. McCarthy  wrote:

  source ~/rc/mutt/aliases-auto
8407 aliases, nearly 2MB in size :-)

Really, it is a bit excessive. Particularly since I think I don't 
ever use aliases anyway.


Yes, Mutt's internal structures used for aliases are not designed for 
that many.  It could lead to quite a slowdown in some circumstances.  
Do you have $reverse_alias set in your config?


Yes, but it doesn't seem to affect things in terms of loading the 
folder.


Cheers,
Cameron Simpson 


Re: why does mutt take so long to load my python folder?

2022-11-18 Thread Cameron Simpson

On 15Nov2022 20:10, Kevin J. McCarthy  wrote:
That `cur` message file dates from 2021-07-15 17:36:10 GMT (long ago) 
and the temporary file is indeed temporary - it vanishes. I speculate 
that mutt's... unpacking the source message in some way?


Seems likely, but off the top of my head I can't guess what's doing it.  
[...]


Thanks for the response. I've been experimenting tonight.

It isn't the sort. As you suggest, it is in a process after the sort.

Starting with an empty config except for a header cache dir setting, I 
notices a distinct slowdown of the scan at about 80%. Odd. So I moved 
the cache sideways and let it rebuild:


43869952 18 Nov 20:43 02a1cc9292e0222624346e5ff8742f34
73852160 18 Nov 20:09 02a1cc9292e0222624346e5ff8742f34-ASIDE

This brought a nice linear scan.

At this point the folder loads pretty promptly.

Turning on my normal settings including sort made no difference, still 
nice and fast. Even my macros to collapse/uncollapse certain patterns 
seem basicly free.


So I started adding back in my config files (I source a few). Here's the 
glaring one:


source ~/rc/mutt/aliases-auto

8407 aliases, nearly 2MB in size :-)

Really, it is a bit excessive. Particularly since I think I don't ever 
use aliases anyway.


With that commented out the python folder's still nice and fast.

What's weird to me is that the effect it has seems dependent on the 
folder I choose. Reading the file itself doesn't seem to slow; it seems 
to be some interaction with the folder.


Cheers,
Cameron Simpson 


why does mutt take so long to load my python folder?

2022-11-15 Thread Cameron Simpson
I'm using my python folder (a Maildir) because it is large enough to do 
a little live debugging on. Summary header:


c...@cskk.id.au reverse-threads/last-date 0% 1152M Msgs:140928 New:90812 
Del:0 Flag:608 =python

I'm using a header cache, and my sort is reverse-threads/last-date.

Command line:

mutt -f ~/mail/python -e fcc-hook . ~/mail/python; save-hook .  
~/mail/OLD/2022/python

(That's from ps; the `-e` argument is a single string of course.)

The stuff before the "Sorting mailbox..." phase seems ... fast enough.

During the "Sorting mailbox..." phase I see mutt reading message files 
I'd not expect it to need to touch, _and_ making some temporary files.  
Example `lsof` output:


mutt1162 cameron3r   REG1,5 8370   143651075 
/Users/cameron/mail/python/cur/1626370570.#7159M542304P658:2,S
mutt1162 cameron4u   REG1,50   599663041 
/Users/cameron/var/mutt/mutt-fleet2-lan-501-1162-13395134689776834598

That `cur` message file dates from 2021-07-15 17:36:10 GMT (long ago) 
and the temporary file is indeed temporary - it vanishes. I speculate 
that mutt's... unpacking the source message in some way?


I had expected that the sort would happen in memory using the header 
cache to obtain values. ISTR that the header cache only caches 
particular headers, but I can't see which in the docs and would have 
imagined it at least caches the headers used to sort messages.


Thoughts?

I append the output of `mutt -v` below.

Cheers,
Cameron Simpson 

Mutt 2.2.7 (2022-08-07)
Copyright (C) 1996-2022 Michael R. Elkins and others.
Mutt comes with ABSOLUTELY NO WARRANTY; for details type `mutt -vv'.
Mutt is free software, and you are welcome to redistribute it
under certain conditions; type `mutt -vv' for details.

System: Darwin 19.6.0 (x86_64)
ncurses: ncurses 5.7.20081102 (compiled with 5.7)
libiconv: 1.11
hcache backend: tokyocabinet 1.4.48

Compiler:
Apple clang version 12.0.0 (clang-1200.0.32.29)
Target: x86_64-apple-darwin19.6.0
Thread model: posix
InstalledDir: 
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin

Configure options: --disable-dependency-tracking --disable-warnings 
--prefix=/usr/local/Cellar/mutt/2.2.7 --enable-debug --enable-hcache 
--enable-imap --enable-pop --enable-sidebar --enable-smtp --with-gss 
--with-sasl --with-ssl=/usr/local/opt/openssl@1.1 --with-tokyocabinet 
--enable-gpgme CC=clang

Compilation CFLAGS: -g -O2

Compile options:
-DOMAIN
+DEBUG
-HOMESPOOL  +USE_SETGID  +USE_DOTLOCK  +DL_STANDALONE  +USE_FCNTL  -USE_FLOCK   
+USE_POP  +USE_IMAP  +USE_SMTP  
+USE_SSL_OPENSSL  -USE_SSL_GNUTLS  +USE_SASL  -USE_GSASL  +USE_GSS  +HAVE_GETADDRINFO  
+HAVE_REGCOMP  -USE_GNU_REGEX  
+HAVE_COLOR  +HAVE_START_COLOR  +HAVE_TYPEAHEAD  +HAVE_BKGDSET  
+HAVE_CURS_SET  +HAVE_META  +HAVE_RESIZETERM  +HAVE_FUTIMENS  
+CRYPT_BACKEND_CLASSIC_PGP  +CRYPT_BACKEND_CLASSIC_SMIME  +CRYPT_BACKEND_GPGME  
-EXACT_ADDRESS  -SUN_ATTACHMENT  
+ENABLE_NLS  -LOCALES_HACK  +HAVE_WC_FUNCS  +HAVE_LANGINFO_CODESET  +HAVE_LANGINFO_YESEXPR  
+HAVE_ICONV  -ICONV_NONTRANS  -HAVE_LIBIDN  -HAVE_LIBIDN2  +HAVE_GETSID  +USE_HCACHE  
+USE_SIDEBAR  -USE_COMPRESSED  -USE_INOTIFY  
-ISPELL

SENDMAIL="/usr/sbin/sendmail"
MAILPATH="/var/mail"
PKGDATADIR="/usr/local/Cellar/mutt/2.2.7/share/mutt"
SYSCONFDIR="/usr/local/Cellar/mutt/2.2.7/etc"
EXECSHELL="/bin/sh"
-MIXMASTER

To contact the developers, please mail to .
To report a bug, please contact the Mutt maintainers via gitlab:
https://gitlab.com/muttmua/mutt/issues



Re: Understanding message deletion model

2022-11-11 Thread Cameron Simpson

On 11Nov2022 15:38, Jason Franklin  wrote:

On Thu, Nov 10, 2022 at 07:18:26PM -0800, Kevin J. McCarthy wrote:

On Thu, Nov 10, 2022 at 08:47:44PM -0500, Jason Franklin wrote:
> Are "purging" and "deleting" not the same thing?

According to the IMAP protocol, they are not.  Deleting is marking the
message with the "deleted" flag, which normally doesn't automatically remove
it from the mailbox.  Purging (expunging) will remove messages marked as
"deleted" from the mailbox.


Very good to know.

It looks like I'm marking deleted with 'd', exiting without purging via
mutt, then my provider knows to purge the message anyway.


I'm using local email, not IMAP, but I map my "d" key to:



and I've got:

save-hook . archive-folder-name-here

so that the default save location (implicitly used in the macro) is the 
archive folder for my current folder.



It seems like an odd distinction to separate "purging" from "moving to
trash folder."


Persoanlly, I do not have a trash folder; I just "archive" messages. A 
bit like the GMail model is (or was? dunno any more).


Cheers,
Cameron Simpson 


Re: [ SOLVED ] Re: A bit off-topic: problems with sending to a Gmail user

2022-10-18 Thread Cameron Simpson

On 17Oct2022 21:19, Tim Chase  wrote:

I found that I had to both set my SPF for the sending-server (in
my case, my VPS is the sender, rather than outsourcing to a smart-host
elsewhere),


I tried that for us, but some places reject email from our cloud server, 
which is an AWS VM. So the home server runs outbound mail for some/all 
things through the ISP smart host.


And, regrettably, our phones kind of inherently use their telco/ISP 
smart host.  (Hmm, maybe wireguard would let us securely use the AWS 
host as a smart host).


Cheers,
Cameron Simpson 


Re: [ SOLVED ] Re: A bit off-topic: problems with sending to a Gmail user

2022-10-17 Thread Cameron Simpson

On 17Oct2022 09:48, Matthias Apitz  wrote:

El día sábado, marzo 12, 2022 a las 11:12:49a. m. +, Claus Assmann escribió:

On Fri, Mar 11, 2022, Stefan Hagen wrote:
> > > 550-5.7.26 This message does not have authentication 
> > > information or fails to


> Authenticated in this context means, you don't have SPF / DKIM / DMARC set up.

[,,,]


As more and more of my mails, also to friends could not reach them
@google.com, I studied the DNS record for SPF a bit and came up with
this single line in my DNS:

@ TXT v=spf1 ip4:178.254.4.101 include:unixare.de -all

and all is fine now.


Thank you!

I've just read up on this and have added an include for the ISP we often 
send through. I'll be interested to see if delivery to gmail.com 
improves for me - my own symptom is usually that they shunt my messages 
to spam/junk instead of bouncing them, so this may well not help me.


But I was unaware of "include" before now.

Cheers,
Cameron Simpson 


Re: Viewing HTML message with Firefox

2022-10-10 Thread Cameron Simpson

On 10Oct2022 17:00, Jan Eden via Mutt-users  wrote:

Firefox 105.0.3 on Ubuntu 22.04. It seems the switch to snap has many
disadvantages.


Yes and no. I like a bit of restraint. But it's good if the reasons are 
made apparent to the user. Example, on MacOS apps actually need to go 
through the OS access control if they try to access the Desktop folder.  
But at least I get an OS mediated popup saying "App X wants to access 
your Desktop folder". A bit ad hoc, but as a shared multipurpose area it 
is probaby a good thing.


It sounds like snap apps maybe run in a somewhat constrained container?

I've got $TMPDIR set to ~/tmp, so i probably haven't been bitten by this 
Firefox check, and I am a Firefox user.


Cheers,
Cameron Simpson 


Re: Viewing HTML message with Firefox

2022-10-09 Thread Cameron Simpson

On 09Oct2022 09:35, Jan Eden via Mutt-users  wrote:

Yes, I did use Chris Green's helper script, and it does work with
Epiphany, but not with Firefox.


Maybe it would prefer a "file:///path/to/file.html" URL?

Try some things by hand with the "open" command, see what works. You're 
aware that "open" (MacOS command) accepts a "-a appname" argument to 
select which application gets asked to open the target? You can do 
comparisons that way, or force a preference. [...] Um, your error log 
suggests you're on Linux. IIRC its "open" command is rather different.  
Anyway, experiment by hand rather than indirectly through the mailcap.


Cheers,
Cameron Simpson 


Re: Viewing HTML message with Firefox

2022-10-08 Thread Cameron Simpson

On 08Oct2022 21:55, Jan Eden via Mutt-users  wrote:
Thank you! This works well with Epiphany, but Firefox still fails 
(which

I do not care about, as long as I have a working browser option).


My recollection is that this is a timing issue. To run the viewer mutt 
goes:

- make a temp file containing the message or html
- run the mailcap command
- remove the temp file

Your command is "open", which tells firefox (your default browser) to 
open the file for viewing. That message takes little time. By the time 
firefox gets around to opening the file (probably after making the new 
browser window) the file has been removed by step 3 above.


This is why helper scripts like Chris Green's one take a copy of the 
file. They tell the browser to view the copy, which does not get 
removed.


As a hack, you might find it enough to include a short sleep in your 
command, enough to allow firefox to latch onto the message:


open %s && sleep 3

My personal process, which is by no means seamless, is a macro bond to 
my "V" key:


macro index,pager V "mail-open-attachments" "extract 
attachments to temp dir and open"
macro attach  V "qVv" "extract attachments to temp dir and open"

and the associated script is here:

https://github.com/cameron-simpson/css/blob/main/bin/mail-open-attachments

Like Chris' script, it makes a copy.

It unpacks everything in the message into a directory, then opens the 
Finder on that directory. Then I can use preview on the HTML or open it 
in Firefox. Etc etc for whatever other attachments there may have been.


Cheers,
Cameron Simpson 


Re: The way mutt handles long lines, seems odd/wrong to me

2022-10-01 Thread Cameron Simpson

On 01Oct2022 09:33, Chris Green  wrote:

As I said though it seems odd that only mutt suffers from the problem
(on my xubuntu systems anyway).  Presumably both 'less' and my editor
'vile' use ncurses too and they just wrap long lines if you tell them
to do so.


I'm pretty sure that less does not use curses (of whatever 
implementation).  Termcap/terminfo, sure.  But not curses, which 
maintains a model of the whole screen.


I have a (vague) recollection that some terminals keep track of whether 
a line was terminates with a newline or not, and that it affects how 
copy/paste was done, in that a newline causes a break in the string. But 
the memory is vague.


1) is your long string using the full width of the terminal? i.e. is 
there a character in the rightmost column?


2) if the text does go to the rightmost column, but you have a break in 
the copied text, what's in that break? eg if you paste into an editor, 
what gets inserted there?


Just curious, not sure this information would lead to a fix/workaround.

Cheers,
Cameron Simpson 


Re: Loading several intialisation files at startup

2022-09-14 Thread Cameron Simpson

On 13Sep2022 16:50, Sébastien Hinderer  wrote:
I would like to have a nice way to deal with several accounts in mutt. 
I am talking just about the .muttrc file aspect here.


My present configuration is that each account has its own muttrc file
which sources a common one.


I thought the common approach was what you next outline: a single muttrc 
which sources the appropriate account-specific muttrc.


My own muttrc starts like this:

source ~/private/rc/mutt/personal
source ~/rc/mutt/settings
source ~/rc/mutt/settings-forwarding
##source ~/rc/mutt/spam# spam scoring settings
source ~/rc/mutt/gpg
source ~/rc/mutt/aliases-auto
source ~/rc/mutt/lists

mailboxes +me +sh +mutt +python

auto_view text/html
mime_lookup application/octet-stream

mbox-hook +feeds +SAVED/feeds

source ~/rc/mutt/macros

and then has a bunch of arbitrary stuff I haven't subcategorised. The 
"private/rc/mutt/personal" file has stuff like this:


set my_cs='c...@cskk.id.au'
set my_CS='Cameron Simpson  (formerly c...@zip.com.au)'
set my_cs_gmail='cameron.simp...@gmail.com'
set my_CS_gmail="Cameron Simpson <$my_cs_gmail>"

set my_local_mail="$HOME/mail"

# email addresses in active use
alternates '^cs@(zip.com.au|zipworld.com.au|cskk.id.au)$' \
   '^cameron@(cskk.id.au|.)$' \
   ... etc etc ...

set my_account_email=UNKNOWN

# default
reply-hook . "my_hdr BCC: $my_cs"
reply-hook . 'set from="$NAME <$EMAIL>"'
reply-hook '~C @googlegroups.com' 'set from="$my_CS_gmail"'

folder-hook . 'set my_account_email=$my_cs; set folder=$my_local_mail'
# ... other patterns based on the folder path, usually for imap as 
# various identities ...

folder-hook 'imaps://' 'set folder=imaps://$imap_user@$my_imap_host/'
folder-hook . 'set status_format="$my_account_email %s/%S %P %l Msgs:%?M?%M/?%m%?n? 
New:%n?%?o? Del:%d?%?F? Flag:%F?%?t? Tag:%t?  f-%>-"'

account-hook 'imaps://cs@mail\.zip\.com\.au/' 'source "cs-mutt-imap-settings 
zipworld |"'
...

I really need to revisit account-hook, but the approach here is to 
autoset account specific things with folder-hooks and reply-hooks. You 
could source your account specific rc files on that basis or via macros, 
etc.


Cheers,
Cameron Simpson 


Re: [Mutt] Is linewrap dead?

2022-09-12 Thread Cameron Simpson

On 12Sep2022 21:07, Mihai Lazarescu  wrote:

On Monday, September 12, 2022 at 15:15:55 +, Nacho via Mutt-users wrote:
But in short, today it's just a matter of money and time to have your 
own

email system working perfectly, of course the cost have increased wildly in the
last few years and will keep doing so, but at least for me makes all the sense
to pay for it.


I tend to concur. I run just fine for several years now my own 
SMTP/IMAP servers + Let's Encrypt certificate on a cheap VPS 
(< $20/year).


It took some work to set it up, but that's it. Surely not a mass 
solution, yet feasible and stable.


Same here.

I had a stable email address for decades from an early ISP. Even after 
switching ISPs I kept the account there for the address, which had been 
advertised as an address-for-life. ~$3 every few months. They got 
bought, but the service was maintained. Then they got bought by a giant 
Oz telco who decided to drop it.


That drove me to run a VM for my current domain.

I spent months during the transition finding the lists and sites and 
updating things. I hope to never have to do that again.



Messages sent to Google accounts tick all green boxes.


Hah! Google seem to have a special filter for me; most email I send 
directly to a GMail account lands in that user's spam folder, even if 
they mark them as nonspam. It is not clear why. I did once maintain an 
adzapper :-(


Only Microsoft (outlook.com, hotmail.com) seem to filter the whole IP 
block, but I am too lazy to ask the provider to fix or change provider 
altogether.


I've been bitten by that kind of thing sometimes. I usually just run 
email for those domains via my ISP's SMTP service instead of the VM's 
SMTP service.


Cheers,
Cameron Simpson 


Re: Handling multiple From addresses

2022-09-06 Thread Cameron Simpson

On 06Sep2022 09:26, Jan Eden via Mutt-users  wrote:

Thank you! This is exactly what I needed. I tweaked your version a bit
to use the standard parameters for setting the required values:

set from="x...@eden.one"
set pgp_default_key = '0x20F7FHD80EC4C17294434A0220D37B3CE755FE8A'
set status_format="-%r x...@eden.one: %f [Msgs:%?M?%M/?%m%?n? New:%n?%?o? Old:%o?%?d? 
Del:%d?%?F? Flag:%F?%?t? Tag:%t?%?p? Post:%p?%?b? Inc:%b?%?l? 
%l?]---(%s/%S)-%>-(%P)---"
unmy_hdr *
my_hdr Bcc: t...@eden.one


I've got a slightly tweaked $status_format which begins:

set status_format="$my_account_email .."

because I've parameterised the email address, since it gets used in a 
few places.


Cheers,
Cameron Simpson 


Re: Is linewrap dead?

2022-09-05 Thread Cameron Simpson

On 05Sep2022 12:56, Kurt Hackenberg  wrote:

On Mon, Sep 05, 2022 at 07:40:54PM +1000, Cameron Simpson wrote:
It seems a little conceptually cleaner to have the editor do the 
whole job, rather than divide it between the editor and Mutt. But 
another complication is that you can edit a message more than once...


I think space stuffing is effectively an escape mechanism, like 
quoted printable; not something you see at authoring time. The RFC 
even says "(Note that space-stuffing is conceptually similar to 
dot-stuffing as specified in [SMTP].)", which is again a transport 
level escaping.


So we need the overt trailing-space stuff because that's an 
indicator of end-of-paragraph in f=f, but not the space stuffing, 
because that is only needed to transport the text.


Space-stuffing doesn't have to be shown to the user while editing, but 
it's part of format=flowed, and it's required, not optional. The 
paragraph before the one you quoted is this:


   On generation, any unquoted lines which start with ">", and any lines
   which start with a space or "From " MUST be space-stuffed.  Other
   lines MAY be space-stuffed as desired.

<https://www.rfc-editor.org/rfc/rfc3676.html#section-4.4>


Yes, it is required, just as dot stuffing in SMTP is required. I still 
consider it part of the transport/encoding layer _conceptually_. A bit 
like I can compose some nonASCII Unicode text and require 
quoted-printable UTF-8 behind the scenes I suppose.


That's why I consider it conceptually separate. I'm not arguing that 
there's no requirement for space stuffing. I'm arguing that there's no 
need or benefit to burden the authoring interface with it.


A bit like your long-lines<->format=flowed mode switch in your emacs 
mode: they're semanticly equivalent, but you could compose in long-lines 
mode and _send_ in format=flowed format as an entirely valid way to 
author a format=flowed message.


I'm not sure we're disagreeing here, except for the conceptual 
separation of the space-stuffing step.


Cheers,
Cameron Simpson 


Re: Is linewrap dead?

2022-09-05 Thread Cameron Simpson

On 05Sep2022 01:52, Kurt Hackenberg  wrote:

On Mon, Sep 05, 2022 at 08:36:54AM +1000, Cameron Simpson wrote:

But not space-stuffing, right?
Aye. I avoid lines commencing with a ">" just because they look quoted 
to my eye anyway, so that aside "live" space stuffing in authoring is 
something I'd find distracting.


Agreed, I wouldn't want space-stuffing maintained on the fly, while 
editing. It could be done at the end, when handing the message back to 
Mutt.


It seems a little conceptually cleaner to have the editor do the whole 
job, rather than divide it between the editor and Mutt. But another 
complication is that you can edit a message more than once...


I think space stuffing is effectively an escape mechanism, like quoted 
printable; not something you see at authoring time. The RFC even says 
"(Note that space-stuffing is conceptually similar to dot-stuffing as 
specified in [SMTP].)", which is again a transport level escaping.


So we need the overt trailing-space stuff because that's an indicator of 
end-of-paragraph in f=f, but not the space stuffing, because that is 
only needed to transport the text.


[...]
The second one works differently. It's a "major mode"; it has commands 
to convert both ways between format=flowed and one-line paragraphs, 
while you edit. The idea is to type with each paragraph being a single 
long line, and then "encode" that as format=flowed when you're done.  
That doesn't mess with existing mechanisms, and doesn't take much code. 
Emacs has something called "visual line mode" to make those long lines 
not ugly.


Ah, I was wondering about that, because I find the one-line-paragraph 
thing wearing with quite long lines.


That second one is attached, for anyone who's interested (Cameron, I 
suppose you probably don't use Emacs).


Indeed not. I took the Other Path long ago.


Don't encode/decode the message header section; that could
possibly break things.  It's best to tell Mutt

   set edit_headers = no

to avoid that possibility.


I like to edit headers, so my vim-flavoured mode trims all trailing 
spaces from the header lines at startup:


1,/^$/s/  *$//

That way they're "hard newlines" from the flowed point of view, and 
survive.


Cheers,
Cameron Simpson 


Re: Is linewrap dead?

2022-09-04 Thread Cameron Simpson

On 05Sep2022 08:37, raf via Mutt-users  wrote:

> I like your indenting of code blocks, but it seems to
> put an additional blank line after each code block.
> That might not be intentional.

Not intentional. I just wanted to keep the 4 space indent used to trigger a
code block for the same visual effect, since I use that instead of the
triple backticks usually. Pandoc tosses those spaces.


That's correct behaviour for markdown.


Doesn't mean I like it :-)


If you want the
output to be indented four spaces, the markdown source
needs to be indented eight spaces. But for email use,
it's a nice idea to do what you're doing.


I'd like the HTML to resemble the source MarkDown, visually.


If you want to prevent the extra line, the sed command
can be changed to remove theindenyt before the closing
line of the code block:

 sed '
 s/^\(\)\(.*<\/code><\/pre>\)$/\1\2/
 t post_pre_code
 /^/,/<\/code><\/pre>$/{
 s/^//
 s/^\(\)/\1/
 s/^\(<\/code><\/pre>\)/\1/
 }
 :post_pre_code
 '


Ah, right. Thanks.

Cheers,
Cameron Simpson 


Re: Is linewrap dead?

2022-09-04 Thread Cameron Simpson

On 05Sep2022 08:24, Cameron Simpson  wrote:

On 04Sep2022 11:33, Kurt Hackenberg  wrote:

But not space-stuffing, right?


I just reread https://www.rfc-editor.org/rfc/rfc3676#section-4.4 to 
refresh my brain. Yeah, I don't think I'd want that when writing a 
message.


Which I guess is why Mutt space-stuffs the format=flowed that it gets 
back from the editor.


Aye. I avoid lines commencing with a ">" just because they look quoted 
to my eye anyway, so that aside "live" space stuffing in authoring is 
something I'd find distracting.


Cheers,
Cameron Simpson 


Re: Is linewrap dead?

2022-09-04 Thread Cameron Simpson

On 04Sep2022 11:33, Kurt Hackenberg  wrote:

On 2022/09/04 06:37, Cameron Simpson wrote:

Vim can do 99% of it for you on the fly :-)


But not space-stuffing, right? Which I guess is why Mutt space-stuffs 
the format=flowed that it gets back from the editor.


I imagine it could be told to space stuff with a lot of work maybe? I 
was more thinking of the authoring side, where it word wraps and 
space-appends my input text as I type. As it's just done for me writing 
this.


Cheers,
Cameron Simpson 


Re: Is linewrap dead?

2022-09-04 Thread Cameron Simpson

On 04Sep2022 15:34, raf via Mutt-users  wrote:

On Sun, Sep 04, 2022 at 01:51:25PM +1000, Cameron Simpson  
wrote:

[...]
So I've revisited the manual and found the 
`$send_multipart_alternative`

option and its friend `$send_multipart_alternative_filter`. They work well!

So now I have a mechanism to send: `format=flowed` MarkDown with a aparallel
HTML alternative. The HTML should render in a "paragraphy" way for the HTML
people, and the MarkDown keeps me happy.

My default setting is now:

set send_multipart_alternative=no
set send_multipart_alternative_filter='echo text/html; echo; exec md2html'

which is the inactive form, and I've added:

message-hook . 'set send_multipart_alternative=no'
message-hook '%f htmlees' 'set send_multipart_alternative=no'


Oops. That should be =yes above.


Whoops indeed. I just sketched this out this morning. Thanks for the 
catch. fix applied.


The `md2html` script is my personal script, which wraps `pandoc` and 
post

munges the HTML to indent the code blocks, which `pandoc`'s HTML does not
please my eye. It's here:
https://github.com/cameron-simpson/css/blob/main/bin-cs/md2html
if anyone wants a starting point.


Thanks, Cameron. It's odd that there isn't an md2html
program out there already. I had to create one too
(using python's markdown module).


Ah, that might be easier to customise. I just tried a few markdown 
converters and ended up with pandoc.



I like your indenting of code blocks, but it seems to
put an additional blank line after each code block.
That might not be intentional.


Not intentional. I just wanted to keep the 4 space indent used to 
trigger a code block for the same visual effect, since I use that 
instead of the triple backticks usually. Pandoc tosses those spaces.


Pandoc's conversion is a bit clunky (but better than discount and the 
other tool I tried).



And it looks like it
doesn't do titles (which default to "-"). I guess that
doesn't matter for email use, except that pandoc
whinges about it on stderr.


Aye, annoying. There's no context for a title (eg the subject header), I 
may just have to provide something to shut it up. Or shift sideways to 
the Python module, particularly if that does a better job.



Do you have any advice for automating spaces at the end
of non-final paragraph lines for format=flowed in vim?


I use these settings:
https://github.com/cameron-simpson/css/blob/main/bin/vim-flowed
which autowraps and leaves trailing spaces automatically.


Perhaps I could just post-process messages with perl
in my mutt-editor wrapper script.


Vim can do 99% of it for you on the fly :-)

Cheers,
Cameron Simpson 


Re: Is linewrap dead?

2022-09-03 Thread Cameron Simpson

Well, this has been quite the read.

As a plain text person (aren't we all?) I find poor quality mail clients 
annoying, as shown by the motivating screenshot of a plain text hard 
folder message presenting on a narrow portrait mode mail reader.


There seem to two approaches available: `format=flowed` which works well 
if the mail reader supports it, and HTML which is a PITA to author for 
us.


Having just got my `format=flowed` stuff working again after being 
broken for a long time, and have been using some Discourse forums 
recently (via email) which accept MarkDown, I've been filled with 
enthusiasm for MarkDown email. And markdown's a decent source for basic 
HTML, since it's authoring overhead is low and it is very readable in 
its raw form.


So I've revisited the manual and found the `$send_multipart_alternative` 
option and its friend `$send_multipart_alternative_filter`. They work 
well!


So now I have a mechanism to send: `format=flowed` MarkDown with a 
aparallel HTML alternative. The HTML should render in a "paragraphy" way 
for the HTML people, and the MarkDown keeps me happy.


My default setting is now:

set send_multipart_alternative=no
set send_multipart_alternative_filter='echo text/html; echo; exec md2html'

which is the inactive form, and I've added:

message-hook . 'set send_multipart_alternative=no'
message-hook '%f htmlees' 'set send_multipart_alternative=no'

which will be turning it on for people in my (empty so far) "htmlees" 
mutt group. I use a similar pattern (`%htmlers`) for preserring HTML 
over plain text for certain messages.


The `md2html` script is my personal script, which wraps `pandoc` and 
post munges the HTML to indent the code blocks, which `pandoc`'s HTML 
does not please my eye. It's here:

https://github.com/cameron-simpson/css/blob/main/bin-cs/md2html
if anyone wants a starting point.

I'm probably going to bind a key to turn this mode on at some point.

Cheers,
Cameron Simpson 


Re: trouble with my_hdr and $format_flowed [SOLVED]

2022-09-02 Thread Cameron Simpson

On 02Sep2022 15:27, Cameron Simpson  wrote:

On 01Sep2022 20:18, Kevin J. McCarthy  wrote:

I'm not sure if it helps at this point, but I can report that I've
taken your template above and passed it to 'mutt -H' and the resulting
sent file has format=flowed in its content-type header.  :-/

Can you try directly doing this too, outside of your tmux-editing
framework?


Yes. I should have done that already. [...]
I'll be back after I've done some cleaner experiments.


So, a bare "mutt -H" here acts like yours, honouring the Content-Type 
header. This points the finger directly at my script, where I discovered 
a problem.


First up, it is necessary to hand apply a Content-Type header to the 
template file (as implied by the $text_flowed documentation).


Second, my script was:
- grabbing the template filename from the command line as $filename
- making the copy to be used by the tmux session's "mutt -H" as $tmpf
- patching the copy with a Content-Type header
- assigning filename=$tmpf after the copy was complete, for use in the 
  following code


During debugging I'd stuck that in a subshell, a habit when I turn on 
"set -x" in a script, thus:


blah blah ...
( set -x
  code beging traced
  ...
  ...
)

So the filename= assignment was lost, and the subsequent run-in-tmux 
code was using the unpatched no-Content-Type file.


Fixed and deployed.

This message should be format=flowed.

Cheers,
Cameron Simpson 


Re: trouble with my_hdr and $format_flowed

2022-09-01 Thread Cameron Simpson
On 01Sep2022 20:18, Kevin J. McCarthy  wrote:
>On Fri, Sep 02, 2022 at 08:28:01AM +1000, Cameron Simpson wrote:
>>and the file ~/var/mutt/mutt-fleet2-501-83365-16591433053586932493
>>passwd to "mutt -H" is already gone. Does mutt remove it?
>>
>>Ah, mutt might have move it to ~/var/mutt/compose/muttedit.94328, which
>>exists with the header above.
>>
>>Let's look at the Content-Type on this message...
>
>I'm not sure if it helps at this point, but I can report that I've 
>taken your template above and passed it to 'mutt -H' and the resulting 
>sent file has format=flowed in its content-type header.  :-/
>
>Can you try directly doing this too, outside of your tmux-editing 
>framework?

Yes. I should have done that already. I'm running Mutt 2.2.7 
(2022-08-07) BTW, not that I expect any of this to have recent changes.

I'll be back after I've done some cleaner experiments.

Thanks,
Cameron Simpson 


Re: trouble with my_hdr and $format_flowed

2022-09-01 Thread Cameron Simpson
On 02Sep2022 08:15, Cameron Simpson  wrote:
>Trying that with this reply.

Well, that tossed my Content-Type header and sent with:

Content-Type: text/plain; charset=us-ascii

I'm trying again here, for thoroughness. The source file starts like 
this:

Content-Type: text/plain; charset=utf-8; format=flowed
From: Cameron Simpson 
To: mutt-users@mutt.org
Cc:
Bcc: c...@cskk.id.au
Subject: Re: trouble with my_hdr and $format_flowed
Reply-To:
In-Reply-To: 

and the Content-Type is not presented in this current editor.

Interestingly, the process tree looks like this:

cameron 94359  1116 4299624  1220   \_ /bin/sh ~/bin/rmafter 
~/var/mutt/mutt-fleet2-501-83365-16591433053586932493 mutt -e set 
editor=vim-flowed -e unset signature -e set sidebar_visible=no -H 
~/var/mutt/mutt-fleet2-501-83365-16591433053586932493
cameron 94361 94359 4345688 11628 \_ mutt -e set editor=vim-flowed -e 
unset signature -e set sidebar_visible=no -H 
~/var/mutt/mutt-fleet2-501-83365-16591433053586932493
cameron 94383 94361 4403212 12656   \_ vim -c silent 1,/^$/s/ *$// -c 
set filetype=mail -c set formatoptions=walqj 
~/var/mutt/mutt-fleet2-501-94361-4463746866273406678

and the file ~/var/mutt/mutt-fleet2-501-83365-16591433053586932493 
passwd to "mutt -H" is already gone. Does mutt remove it?

Ah, mutt might have move it to ~/var/mutt/compose/muttedit.94328, which 
exists with the header above.

Let's look at the Content-Type on this message...

Cheers,
Cameron Simpson 


Re: trouble with my_hdr and $format_flowed

2022-09-01 Thread Cameron Simpson
On 01Sep2022 09:09, Kevin J. McCarthy  wrote:
>On Wed, Aug 31, 2022 at 06:45:08PM -0700, Kevin J. McCarthy wrote:
>>In my testing, a header 'Content-Type: text/plain; format=flowed' in 
>>the draft file does result in the sent email being format=flowed.
>
>My example wasn't great: you *should* add a charset parameter set to 
>your system charset (I'm assuming that's utf-8).

Trying that with this reply.

Cheers,
Cameron Simpson 


Re: trouble with my_hdr and $format_flowed

2022-08-31 Thread Cameron Simpson
On 01Sep2022 08:50, Cameron Simpson  wrote:
>>If you haven't played with it before, you might also look into
>>background editing and the contrib/bgedit-screen-tmux.sh script.  See
>><http://www.mutt.org/doc/manual/#bgedit>.
>
>I will; the background editing feature postdated my tmux games.

I don't think these things can do what I want.

The background editing feature seems to let you use an asynchronous 
editor (a GUI or a tmux/screen session with suitable 
wait-for-completion) and return immediately to mutt. Then have the 
primary (the only) mutt do the post-edit sending when the editor 
finishes. While that would let me, say, detach the tmux session and 
resume it elsewhere, it wouldn't let me quit the primary mutt.

My approach runs an entirely separate mutt to edit-then-send the 
message, and runs that as the "editor" in the primary mutt. I can finish 
in the tmux session and send, and it feels exactly like invoking vim 
directly from mutt. Or I can detach from the session, resume the primary 
mutt, do whatever, and _quit_ the primary mutt. The tmux session lurks 
around, ready for resumption later.

For example, as I type this my tmux listing contains:

 14 mutt-01sep2022-09_09-Re__trouble_with_my_hdr_and_format_flowed: 1 
windows (created Thu Sep  1 09:09:57 2022) (attached)

which is running "mutt -H -copy-of-the-original".

So I'm still left with the problem: how to invoke a "mutt -H" to send a 
format=flowed message. Still striking out with that so far.

Cheers,
Cameron Simpson 


Re: trouble with my_hdr and $format_flowed

2022-08-31 Thread Cameron Simpson
On 31Aug2022 15:37, Kevin J. McCarthy  wrote:
>On Thu, Sep 01, 2022 at 08:11:50AM +1000, Cameron Simpson wrote:
>>The essential problem is that I run a separate mutt instance using "mutt
>>-H" on a copy of my message template. The manual does say that
>>$text_flowed has no effect then, but I seem to have no other way to
>>apply the needed header - adding it by hand during composition is
>>ineffective.
>>
>>I _might_ do better adding it to the copy before starting the separate
>>mutt; I'll see if that works.
>
>I think this may work best for you, if the my_hdr approach doesn't work.

It seems to have no effect; the Content-Type eg from my reply just now 
to you is still not format=flowed. I did discover that an illegal 
Content-Type causes the "mutt -H" to abort.

>Content-Type and other MIME headers aren't included in the "edit 
>headers" generated headers, and they aren't parsed afterwards either.

Ok. I can see why that might be done, since in effect I'm editing a text 
attachment in some ways. I'm a little surprised one can't modify 
Content-Type though. That seems like something which would make sense in 
the "edit headers"; it feels like an "in the attachment" header. What if 
I wanted to send markdown? (I know there's a whole thread about this 
somewhere, to which I didn't pay enough attention at the time; I'll find 
it).

>If you haven't played with it before, you might also look into 
>background editing and the contrib/bgedit-screen-tmux.sh script.  See 
><http://www.mutt.org/doc/manual/#bgedit>.

I will; the background editing feature postdated my tmux games.

Cheers,
Cameron Simpson 


Re: trouble with my_hdr and $format_flowed

2022-08-31 Thread Cameron Simpson
On 31Aug2022 15:28, Kevin J. McCarthy  wrote:
>The parser for my_hdr is a bit different - it allows quote characters 
>in the header, so it doesn't treat them specially like elsewhere.  
>However, semicolons are still parsed normally.

Ah, right.

>I think this was a mistake, but would be a breaking change to fix now.

Aye.

>Anyway, a workaround would be to put a '\' in front of the semicolons:
>  my_hdr X-Foo: bar\; baz

Thank you!

Cheers,
Cameron Simpson 


Re: trouble with my_hdr and $format_flowed

2022-08-31 Thread Cameron Simpson
On 31Aug2022 18:06, David Lowry-Duda  wrote:
>>I have a pretty weird way of composing replies and new messages: I like
>>to be able to detach from the new message window. So I compose in a
>>separate tmux session.
>
>If you compose directly in mutt+vim without composing in a separate 
>tmux window, does it work? (I would assume so, as I use mutt+vim and 
>format flowed works fine).

Yes.

>I don't quite understand how your composing system works well enough to 
>have a more meaningful comment.

The essential problem is that I run a separate mutt instance using "mutt 
-H" on a copy of my message template. The manual does say that 
$text_flowed has no effect then, but I seem to have no other way to 
apply the needed header - adding it by hand during composition is 
ineffective.

I _might_ do better adding it to the copy before starting the separate 
mutt; I'll see if that works.

Cheers,
Cameron Simpson 


Re: trouble with my_hdr and $format_flowed

2022-08-31 Thread Cameron Simpson
I forgot the my_hdr issue. This line:

my_hdr "Content-Type: text/plain; charset=usascii; format-flowed"

results in this from from mutt at startup:

Error in /Users/cameron/rc/mutt/settings, line 129: charset=usascii: 
unknown command

So it appears that I can't use a semicolon in my_hdr?

Not that it would help me, since mutt seems to eat that header anyway :-(

Cheers,
Cameron Simpson 


Re: trouble with my_hdr and $format_flowed

2022-08-31 Thread Cameron Simpson
On 01Sep2022 07:58, Cameron Simpson  wrote:
>I've put a format=flowed into this message by hand, and I'll see if it
>survives.

It did not. The message sent by mutt discarded my Content-Type header 
and used:

Content-Type: text/plain; charset=us-ascii

So I've actually got a significant problem here, as modifying my wrapper 
script to provide the header isn't enough.

Cheers,
Cameron Simpson 


trouble with my_hdr and $format_flowed

2022-08-31 Thread Cameron Simpson
I've long had my mutt+vim setup to compose format=flowed email. There's 
just one problem: it wasn't working. So I'm digging into why.

I've got:

set text_flowed=yes

in my setup. And if I pare down enough mutt settings, my reply goes out 
with this header:

Content-Type: text/plain; charset=us-ascii; format=flowed

which is what I want (I've got send_charset="us-ascii:utf-8" so I 
presume if I put some nonascii in then mutt would bump that to "utf-8").

But without paring things down, I do not get that header.

I have a pretty weird way of composing replies and new messages: I like 
to be able to detach from the new message window. So I compose in a 
separate tmux session. That is done via a special script I use for 
$editor, which essentially goes:
- copy the message to a temp file
- start a tmux session running "mutt -H the-temp-file"
- quit

Because mutt silently drops back (and doesn't send) if the new message 
didn't get modified, this gives me a tmux session where I compose the 
message and send it, and then mutt silently discards my original, and 
I'm back in the index or pager as before. This is seamless.

So what's wrong?

It appears that mutt does not put "format=flowed" in the "Content-Type" 
header until message send time. The message I see in the editor before 
compose does not have the header.

I think the fix would be to presupply the "Content-Type" header. I also 
think such a fix might break the send_charset magic, unless mutt picks 
up an existing Content-Type header, parses it, and fills in the 
parameters if missing.

I've put a format=flowed into this message by hand, and I'll see if it 
survives.

Thoughts?
Cameron Simpson 


Re: macro does not work when mailbox opened read-only

2022-08-10 Thread Cameron Simpson
On 10Aug2022 11:10, todd zullinger  wrote:
>Fourhundred Thecat wrote:
>> When I delete my custom macro, and revert "q" to its default action,
>> then I get this message when exiting from mutt (case 2):
>>
>>   Purge 1 deleted message? ([yes]/no):
>>
>> When I create custom macro to sync automatically, I get the error on
>> read-only mutt, because it cannot be synced.
>>
>> So I need "q" that works in both cases.
>
>You may want to look at the $delete option.  It's a
>quad-option which defaults to ask-yes.
>
>http://www.mutt.org/doc/manual/#delete

Aye, what todd said.

As a comparison, my own settings have these:

set delete=yes
set delete_untag=yes

Cheers,
Cameron Simpson 


Re: Using "viu" to display attached images (SOLVED)

2022-06-27 Thread Cameron Simpson
On 22Jun2022 01:02, Andy Spiegl  wrote:
>Cool ideas.  I started with defining a macro like that:
>
> macro index r "\
> unset implicit_autoview\
> "
>
>Works!  But I don't know how to set it back afterwards, uhm.
>I tried unsuccessfully:
> macro index r "\
> unset implicit_autoview\
> \
> set implicit_autoview"

Does that not work? I would expect it to.

For settings which are available as variables the common practice is to 
stash the "current" value in a $my_something variable, change it, then 
set it back. Crude and only single level, but usually enough. Example 
from my muttrc where I turn off $wait_key to stop mutt pausing during an 
operation:

These 2 lines define a command string to save the current $wait_key then 
the beginning of the "set wait_key" command, then another string to 
reverse it:

set my_push_wait_key=':set my_prior_wait_key=$wait_key; set wait_key'
set my_pop_wait_key=':set wait_key=$my_prior_wait_key'

Then they get used in my rot13 toggle command string:

set my_toggle_rot13="$my_push_wait_key=noflag !  
MUTT_ROT13 -e flag MUTT_ROT13$my_pop_wait_key"

and then I've got some mac ros to run the toggle:

macro index \Cx "$my_toggle_rot13" 'toggle MUTT_ROT13'
macro pager \Cx "$my_toggle_rot13" 'toggle 
MUTT_ROT13'

The mechanics don't really matter, just the shuffle of the original into 
and out of $my_prior_wait_key. You should be able to do the same with 
$implicit_autoview.

Cheers,
Cameron Simpson 


Re: Using "viu" to display attached images

2022-06-20 Thread Cameron Simpson
On 20Jun2022 11:15, Kevin J. McCarthy  wrote:
>With $allow_ansi, Mutt will interpret ansi sequences _it understands_ 
>and convert them to curses calls.  I don't currently have viu available 
>to play with, but perhaps it's emitting sequences Mutt doesn't 
>understand.

How hard is it to extend this? Not asking you to do it, asking how 
painful it is. I'd like to add pass through for SIXEL support, which 
both Kitty and iTerm support (iTerm also has its own alternate image 
coding, but I switched to SIXEL recently and haven't looked back, 
because it is more portable and works well).

I've been using img2sixel (not yet in mutt); "viu" has a sixel mode, 
also.  I imagine it falls back to some rust flavour of libaa otherwise.

Cheers,
Cameron Simpson 


Re: Two questions regarding header display

2022-06-19 Thread Cameron Simpson
On 07Jun2022 09:56, raf  wrote:
>And I'm not sure I can do anything about it.

There are many things you can do. I see you've already shifted to just 
using "bold" etc in your color directives, but also:
- run a personal terminfo record without the color capabilities; 
  decompile the provided terminfo with untic, edit to remove the colours 
  (or change the colours to "mono" escape sequences, build new entry 
  with tic, set $TERMINFO to refer to it
- run mutt itself from a script or alias which sets $TERM just for mutt 
  i.e. overriding the $TERM provided by screen (which will be describing 
  the terminal capabilities of screen itself)
- switch from screen to tmux

Cheers,
Cameron Simpson 


Re: Mailcap selectively applying settings

2022-05-13 Thread Cameron Simpson
Slow to respond here.

Mutt's MIME handling stuff has a long section in the manual here:
http://www.mutt.org/doc/manual/#mimesupport
with the mailcap section commencing here:
http://www.mutt.org/doc/manual/#mailcap

In particular see the stuff about "copiousoutput" and the pager.

On 12May2022 15:38, Christopher Conforti  wrote:
>Hi list, I'm having a bit of a weird one, at least for me. I can't get mutt to
>read my entire mailcap file properly, it seems. For example, I have the
>following line in my mailcap:
>
>images/*;  eog %s

I'm surprised; image MIME types are normally "image/*", not "images/*".

>It handles this just fine; all image attachments are opened with eog. However,
>another line:
>
>text/html  lynx -dump %s | more
>
>is seemingly ignored, even when there are no other options given for handling
>HTML.

I was going to explain that this is interactive and thus not chosen. But 
I see that this is _exactly_ the example in the manual in the MIME 
section :-(

The section on Reading Mail has a part on the pager:
http://www.mutt.org/doc/manual/#pager-menu
which says that by default it uses its builtin pager. That means it 
cannot use an interactive programme unless you configure $pager:
http://www.mutt.org/doc/manual/#pager

But that is for the message as a whole, after conversion to plain text.

You're trying to configure how mutt presents HTML inline in the message 
body. Because that is paged (either with the builtin pager or via an 
external pager you specify with $pager) mutt cannot choose an 
interactive mailcap command. Therefore your command cannot be chosen.

Marking the mailcap entry with "copiousoutput" says that the command 
produces a lot of output, and should be paged. Since that works for you, 
this implies that when "more" isn't sending to a terminal it basicly 
acts like "cat". A quick experiment confirms this for me:

man mutt|more|less

pages the output with "less" - the "more" command has just passed the 
text through. So you get away with it.

Technically you should have 2 entries. Mine look lke this:

text/html; env DISPLAY= unhtml %s | less
text/html; exec 2>&1 && env DISPLAY= unhtml %s; copiousoutput

because the mailcap isn't a mutt-specific thing, and also since mutt 
doesn't use it just for the message display, but also for the attachment 
view action.

So the first entry there has mutt write an HTML component to a file, 
then call "unhtml" on that file and pipe the output through "less".

"unhtml" is just a shell script of my own which runs my current 
preferred HTML-to-plain-tetx converter. Presently it uses "lynx -stdin 
-dump" with a bunch of pre and post manking.

The second entry is for the pager: it has "copiousoutput" appended, and 
does not invoke a paging programme.

>What the heck is going on? Where can I learn how to fix it?

Give the MIME section of the manual a read - it has a lot of detail on 
what you can put in the mailcap entry, including an example of multiple 
entries for a type in the "3.3.2. Search Order" part.

Cheers,
Cameron Simpson 


Re: New to Mutt, unable to send messages in *any* attempted way

2022-05-10 Thread Cameron Simpson
On 10May2022 07:25, Sam Kuper  wrote:
>On Mon, May 09, 2022 at 11:01:20PM -0500, x...@trimaso.com.mx wrote:
>> ---To respond a received email in Mutt pager I hit 'r', and all the
>> rest. I only change the destination email address, and eventually
>> send. But even after successfully sent, the "responded" email in Mutt
>> pager is not marked with 'r'. Why?
>
>Maybe because in your examples above, you set the record variable to ""?
>
>I may be wrong, but: I think that in order for Mutt to know whether a
>message has been replied to, it checks the mailbox specified by the
>record variable - so if that variable is empty, Mutt has no way of
>checking.

I thought it just set a flag on the message.

I forget, is XTec using a local or IMAP mail folder?

Cheers,
Cameron Simpson 


Re: New to Mutt, unable to send messages in *any* attempted way

2022-05-08 Thread Cameron Simpson
On 07May2022 11:11, X Tec  wrote:
>On 2022-05-07 08:45:35, Cameron Simpson wrote:
>> Probably not? If you've been debugging the $smtp* variables (in
>> particular, having to debug the password) then it sounds like mutt is
>> actually sending directly with SMTP and _not_ using the local postfix.
>>
>> When you send with the local postfix, mutt just hands the message off to
>> the sendmail command and doesn't say anything about delivery - that is
>> sendmail's job.
>>
>I double-checked Mutt's current $sendmail variable, and it's certainly the 
>default "/usr/sbin/sendmail" command mentioned in the manual.
>The route exists, and it's by Postfix package in the system I use.

The manual says that setting $smtp_url overrides $sendmail. So your 
$sendmail setting is not in play.

>Also, I tried sending email to myself, and checked the full headers. It does 
>have the TLS thing you mention BTW, but on the other hand it has "User-Agent: 
>Mutt 1.10.1"...

These are not in conflict. Anything you send with mutt normally has a 
mutt user-agent header. Have a look at the received headers. They should 
trace the path from where you sent the email from through to how it was 
delivered.

>Also, Mutt does not say additional stuff or command line output (just exit 
>code '0') when sending email...
>I'm trying to use Mutt with external MTA/SMTP (Postfix in this case), as the 
>correct way intended, instead of Mutt's builtin SMTP.
>Am I failing? If so, why?

You're failing in this because you have $smtp_url defined. Comment out 
it and the other $smtp* variables.

When you send with $sendmail (the local system's mail system, postfix 
for you) the smtp settings come from the main.cf file, not from mutt.

>By the way, in Mutt default pager for reading emails, your words between 
>underscores '_' (_not_, _may_, etc...) are not being displayed... Why?

I think someone suggested turning off any colouring settings you have 
active. Possibly some colours match the terminal background colour.

Are the words invisible (gaps in the text) or missing (no gaps)?

>If sending email from the webmail, I'm sure it gets send from the email 
>address account/SMTP.

This is because the ISP itself is running the delivery, either by 
internal SMTP or its own sendmail equivalent.

>Same if sending from the popular "official" email clients (Outlook, 
>Thunderbird...)

That's because they're using SMTP, probably the the ISP SMTP service 
with a username/password (or better) authentication.

>But *not* sure when sending from Mutt...

According to the manual, if you set $smtp_url mutt will send with SMTP 
(or try) and send with $sendmail otherwise.

>And finally, what key to manually force to check for new mail, instead 
>of waiting or quitting and starting Mutt again?

The "$" key is usually bound to the  command, which updates the 
state of the current folder.

>Evidently just doing "any" activity in Mutt does not refresh/fetch new 
>email...

See http://www.mutt.org/doc/manual/#new-mail (particularly new mail 
Detection) and http://www.mutt.org/doc/manual/#mailboxes

Cheers,
Cameron Simpson 


Re: New to Mutt, unable to send messages in *any* attempted way

2022-05-06 Thread Cameron Simpson
On 06May2022 12:28, x...@trimaso.com.mx  wrote:
>Thanks again.
>
>As mentioned in first message, system has Postfix by default, with its 
>corresponding /usr/sbin/sendmail and /etc/postfix/main.cf .
>Since I don't have anything for $sendmail variable, it's defaulting 
>indeed for Postix sendmail.

Probably not? If you've been debugging the $smtp* variables (in 
particular, having to debug the password) then it sounds like mutt is 
actually sending directly with SMTP and _not_ using the local postfix.

When you send with the local postfix, mutt just hands the message off to 
the sendmail command and doesn't say anything about delivery - that is 
sendmail's job.

>On Thu, May 05, 2022 at 19:19:45 -0500, Cameron Simpson wrote:
>>Might be. It won't be a "real" Message-ID line as it looks like the log
>>tries to include the username of the user who sent the email.
>>
>>However, you're using SMTP to smtp.domain.tld, which means you're not
>>using the local mail system, which means that it should not be logging
>>locally for email you send using mutt.
>>
>Sorry... By "log" here I meant the default ~/sent one, default set for 
>$record variable

Ah, do this is a real message-id field then. Message ids are specified 
here: https://tools.ietf.org/rfcmarkup/5322#section-3.6.4

I'm not sure you can put a second "@" in a message id - my reading of 
the grammar says it basicly looks like "".

>>No, it is just what gets written into the heder line so that people 
>>know
>>what address to use for replies and citations.
>>
>Then how do I know the email is really being sent from  
>u...@domain.tld 's account/SMTP?

Hahahaha! You don't!

There are various headers which _may_ be added to messages and which a 
receiver _may_ consult to verify identity, but they're not appplied by 
default. (They inherently require extra setup, as you need to provide 
the extra identity in your config.)

There are headers which confirm that an ISP sent the message - that 
would tell the receiver that it can from _where_ it says it comes from, 
but not from _who_.

There are DNS records (SPF) which specify where messages for a domain 
may come _from_. These can be used by receiving systems to reject 
messages from other sources claiming to be from that domain. These are 
often loosely defined though, if present at all. Of course, you're then 
trusting the receiving system on this point when you collect your email.

>>Mutt has some debug flags. Try using the "-d 5" option.
>>
>Got: "DEBUG was not defined during compilation.  Ignored."

Oh, that is unfortunate. You might need to built it from source unless 
your vendor provides some kind of "debugging enabled" version of the 
mutt package.

>Any other ways to tell whether email is really being sent with STARTTLS?

The Received: headers of the message (when you get it back) record how 
the message was received (possibly there will be several of these). For 
example, this message of yours as received by me here has this header 
for the final hop:

Received: from smtp2.osuosl.org (smtp2.osuosl.org [140.211.166.133])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 
bits))
(No client certificate requested)
by mail.cskk.id.au (Postfix) with ESMTPS id 773F343999
for ; Fri,  6 May 2022 17:35:24 + (UTC)

The second line doesn't say that STARTTLS was used, just that TLS was 
used. That implies either a cleartext connect with a STARTTLS, or a 
TLS/SSL connection directly. Encrypted either way.

Of course, you need to trust the content of those headers.

>On Thu, May 05, 2022 at 23:06:15 -0500, Nathan Stratton Treadway wrote:
>>Some ISPs specifically block outgoing traffic on TCP port 25 (which is
>>the default port for "smtp:" traffic)
>>
>Which means, if not specifying port then Mutt (or Postfix?) takes port 
>25 as SMTP default...
>Wasn't/shouldn't port 587 the new default for SMTP?
>So Heirloom Mailx also takes port 25 as default?

Maybe? You could check its docs or help messages. You could get really 
funky and watch it send your email. here's me testing the "nc" command 
on a local Ubuntu system:

   [~]borg*> strace -e trace=network nc 127.0.0.1 25 Also, some more doubts aroused:
>
>---If adding the "-n" option to whole email sending command, it does 
>*nothing*, just immediately getting an exit status of '1'. What's 
>happening with this?

"man mutt" says that:

 -n Do not read the system-wide Muttrc configuration file.

If you're getting an exit 1, I'd expect some kind of error message as 
well.

>---Where in the manual can I find the meaning of each of the %Z 
>letters here http://www.mutt.org/doc/manual/#index-format ? I mean, 
>th

Re: New to Mutt, unable to send messages in *any* attempted way

2022-05-05 Thread Cameron Simpson
On 05May2022 15:11, x...@trimaso.com.mx  wrote:
>On Thu, May 05, 2022 at 12:07:44PM -0500, Kevin J. McCarthy wrote:
>>1) On the command line, the shell will expand shell variables inside
>>double quoted strings, before Mutt even sees it.
[...]
>First: had to delete/rename ~/.muttrc, because some previous settings 
>were perhaps conflicting...

Always good to get the config as simple as possible.

>This seemed to *finally* work (had to play with both single and double 
>quotes...):
>
>printf "%b\n" "$msg" | mutt -s "Test message" -e 'set 
>my_user="u...@domain.tld"; set my_url="smtp.domain.tld"; set 
>from="Send User "; set 
>smtp_url="smtp://$my_user:my_p4ss@$my_url:587"; set 
>ssl_starttls="yes"; set ssl_force_tls="yes"' recei...@domain.tld

This is why using a config file is often better. If you had:

set my_user="u...@domain.tld"
set my_url="smtp.domain.tld"
set from="Send User "
set smtp_url="smtp://$my_user:my_p4ss@$my_url:587"
set ssl_starttls="yes"
set ssl_force_tls="yes"

in some file "muttrc-test" you can then test with:

printf "%b\n" "$msg" | mutt -F muttrc-test -s "Test message" 
recei...@domain.tld

much more easily.

>Finally worked -seemingly-, but popped just many doubts:
>---There's a "sent email" log in local system, where sent emails are 
>logged. There's always this line:
>Message-ID: <  @ 
>mylocalpcu...@localhost.myisp.com >
>Is this correct?

Might be. It won't be a "real" Message-ID line as it looks like the log 
tries to include the username of the user who sent the email.

However, you're using SMTP to smtp.domain.tld, which means you're not 
using the local mail system, which means that it should not be logging 
locally for email you send using mutt.

Personally, I configure my local postfix install's main.cf (+etc) to 
send email from my laptop, and just have mutt deliver to the local 
sendmail. That gets me: local logging, local queue (good if offline), 
etc and a single place for the config (postfix). And anything else on my 
laptop can also send email. Such as cron etc.

>---Without "set from=" field, sender becomes 
>"mylocalpcu...@localhost.myisp.com"... What is this?

Probably defaults. In the good old days of shared standalone machines, 
localusername@machine_hostname would sometimes be a valid working 
address. And it is still all that can be guessed without any other 
config.

However, these days machine_hostname is almost never a valid email 
domain, and you want to configure the domain to be your usual ISP based 
domain. For example, my postfix main.cf has:

mydomain = cskk.id.au
myorigin = cskk.id.au

even though that domain is actually hosted on another machine, so that 
email from my laptop has a meaningful From: line. You're doing the same 
thing for mutt with "set from".

>Does the "from" field kind of guarantee that email is being *really* 
>sent from the u...@domain.tld address, and not from *local rig*?

No, it is just what gets written into the heder line so that people know 
what address to use for replies and citations.

>---In "set smtp_url" field, if using "$my_user:my_p4ss" notation, this 
>seems to override the "set smtp_pass" field completely?
>
>---Seemingly not needed smtp_authenticators... ??

Some ISPs allow unauthenticated email from "trusted" networks. For 
example, my home server accepts unauthenticated email from machines on 
my local LAN. An ISP _might_ accept it from the clients, but that seems 
more dubious.

>---Without smtp_url and smtp_pass fields, where does email go? 
>recei...@domain.tld doesn't receive it...

Probably delivered to the local mail system. If that can already deliver 
email, it will be doing the outbound SMTP for you.

>---Is email really being sent with STARTTLS, as wanted? How can I tell?

Mutt has some debug flags. Try using the "-d 5" option.

>---In "set from=" field, spaces between "Send User" and actual email 
>address... don't seem to matter?

The $from is for the "From:" header line, which accepts a full RFC2822 
email address which has a full name and username@hostname part, which 
can be written like:

Cameron Simpson 

which I prefer or:

c...@cskk.id.au (Cameron Simpson)

which is very common, and I find less pleasant to look at. Only the 
c...@cskk.id.au part matters for email delivery. (And the "From:" doesn't 
affect delivery for you, but it gets used by people replying, so it 
affects them.)

>>Are you able to send email via that account using other applications?
>
>Yes, I used to use Heirloom Mailx.

Re: vs. / vs.

2022-03-23 Thread Cameron Simpson
On 23Mar2022 15:01, Jon Brinkmann  wrote:
>I'm working to extract the original message and send it back through 
>procmail
>to be properly sorted into my mailboxes.  I've mapped this process to the
>'E' key:
>
>macro index E '5procmail;tail 
>procmail.log' 'extract & process a message that Microsoft Exchange 
>called undeliverable'
>
>It works great, with one problem:  doesn't pass the mailbox
>delimiting "From [email_address] [message_date]" to procmail, which sorts and
>writes what it received to a file that's not recognized as a mailbox by mutt.
>Is there some way to tell  to pass the delimiter line?  If not,
>should this be a code change request?

Well, the poblem here is that the From_ line is not part of the message 
itself, and won't be present in the attachment you have.

I'd be inventing one - it is broadly ignored by mail filters etc (again, 
not being part of the message), so you just need a valid one to work as 
the mbox delimiter line. Maybe something like this:

( echo "From nobody `date`"; cat ) | procmail

>FYI, it appears that  and  are identical.  At
>least they produce identical results.

The name distinction might be historic, one being from the top level 
(index) and one being from the attachment menu? Just guessing.

Cheers,
Cameron Simpson 


Re: Filter script to remove html, fullquotes and header lines

2022-03-20 Thread Cameron Simpson
On 20Mar2022 13:36, Martin Trautmann  wrote:
>do you know about any mutt script that would go from message to message 
>and
>
>1) remove a html part if a plain text part is given
>
>2) remove all trailing lines,
>   starting with a quote sign ">"
>   and at least e.g. 10 occurences
>
>  such as (^>[.*][\r\n]){9,} before the end of the message
>
>  Maybe I could append xzxzxzx to the end of the message first, delete 
>a fullquote up to there and remove xzxzxzx again?
>
>  Bonus: Do not remove fullquotes for messages without in-reply-to or 
>references headers.
>
>3) remove header lines which are longer than 5 lines
>
>I want to shrink the size of some mailboxes for archive purposes, 
>without throwing away too much.

I think you'll have to write your own.

At minimum you need a full mail message parser so that you are not 
filtering, say, base64 or QP content incorrectly. So something which 
scans a mailbox and for each message:
- decodes it completely
- applies your filters
- assembles the new message
and write this out to a new mailbox (so it isn't destructive and can be 
compared to the original - you don't want to accidentally shred your 
archive).

I'd do this in Python myself - it has a good email library and you can 
do all the things you describe fairly easily with it.

Cheers,
Cameron Simpson 


Re: Slightly OT: I'm looking for a tool to merge maildirs

2022-01-25 Thread Cameron Simpson
On 24Jan2022 08:55, Chris Green  wrote:
>On Mon, Jan 24, 2022 at 10:24:41AM +1100, Cameron Simpson wrote:
>> On 23Jan2022 10:46, Chris Green  wrote:
>> >This is a bit off topic for mutt specifically but it's about doing
>> >things to the mail I read using mutt, so it's not *very* OT.  :-)
>>
>> Had you considered a shell script which invokes mutt with -e to copy
>> messages tagged by criteria from a mail folder to your archive folder?
>>
>That's a good idea, thank you.  I have done things like this before
>(i.e. an mbox to maildir script using mutt), not very fast but who
>cares, it can take some hours for all I care.
>
>I think it may need a separate process to de-duplicate the destination
>but that's OK.

If you're merging from several folders to the archive, I expect so. Mutt 
can do that, BTW. Order by threads, tags duplicates (~=) and delete 
them.

My folder hooks start like this:

folder-hook . 'push 
"~=~T"'

to do that: tag duplicates, delete (conditional on anything being 
tagged), untag tagged messages (~T).

Cheers,
Cameron Simpson 


Re: Slightly OT: I'm looking for a tool to merge maildirs

2022-01-23 Thread Cameron Simpson
On 23Jan2022 10:46, Chris Green  wrote:
>This is a bit off topic for mutt specifically but it's about doing
>things to the mail I read using mutt, so it's not *very* OT.  :-)

Had you considered a shell script which invokes mutt with -e to copy 
messages tagged by criteria from a mail folder to your archive folder?

Cheers,
Cameron Simpson 


Re: NeoMutt Opinions

2021-12-22 Thread Cameron Simpson
On 22Dec2021 13:02, Lee K  wrote:
>I was wondering if anyone has an opinion on the neomutt project. I've
>been a long time mutt user, on and off for 15 years. Neomutt seems to be
>popular with a certain set of users these days. The configs seem pretty much
>compatible. I haven't seen any features that aren't in mutt, and it's sole
>virtue seems to be a more colorful site page.

I've not needed or wanted neomutt, myself. I believe it forked off 
during an extended period of low activity on mainline mutt. Kevin has 
merged a great many features that previously existed as optional patches 
and done a huge amount of additional work as well.

I'd stay with mutt unless there's some specific feature you eally want.

Cheers,
Cameron Simpson 


Re: pretty-print mutt emails

2021-11-30 Thread Cameron Simpson
On 24Nov2021 20:21, Jean Louis  wrote:
>When using grep, I think `grep -m1' should be used to find the first
>match, as I wonder what would be the result if words like "From:" or
>"Date: " are found in the body of the email, not only in the header.

I tend to use sed for this. A simple:

/^$/q

prevents falling through to the message body.

Cheers,
Cameron Simpson 


Re: pretty-print mutt emails

2021-11-22 Thread Cameron Simpson
On 22Nov2021 13:48, Globe Trotter via Mutt-users  wrote:
>/home/gt/bin/muttprint_print: /home/gt/bin/muttprint: /usr/bin/perl^M: 
>bad interpreter: No such file or directory

You've got carriage returns in the text of the script (the "^M" above).  
They are no considered whitespace (and in any case, IIRC, POSIX does not 
guarrentee to handle more than one argument after the command in a 
shebang ("#!") line).

You probably want to strip all the trailing carriage returns from this 
script.

Cheers,
Cameron Simpson 


Re: using mutt with postfix and localhost:25

2021-09-13 Thread Cameron Simpson
On 12Sep2021 15:17, Globe Trotter via Mutt-users  wrote:
>So, I do not know if I can do this, but I can send email by 
>specifically including the smtp with port 25. I dont really need it for 
>anything else. It is quite likely that I am not understanding the 
>import of your question.
>
>All I do is:
>
>sudo  ssh -L 25:mailhub:25 username@your_server
>
>And then, configuring 'localhost' as my smtp server on port 25 gets my 
>mail sent directly forward through the SSH link.

Ok, so you're running the ssh connection as root. That's usually a bad 
idea. The only reason you need to do that that I can see above is to 
bind to port 25 at the local end. I would recommend running  your tunnel 
like this:

ssh -L 1025:mailhub:25 username@your_server

and configuring your mutt smtp_url to use port 1025 instead of 25.  
Others have made this same suggestion.

IMO, it is also worth configuring your local postfix install, provided 
postfix is active locally. Particularly if this is your normal machine 
which you usually use - having a working local mail system is a good 
thing - lots of things send email, such as cron and at - they will all 
just start working if you set this up.

This means that you can compose and "send" (well, queue) email locally 
when offline or when the ssh tunnel is not up. It will go out next time 
the tunnel's up.  Then you just turn off the smtp_url setting and use 
the local sendmail command for delivery. That's what I do.

Most of postfix's configuration is in the file /etc/postfix/main.cf

In particular you want (taken from my own main.cf file here):

relayhost = 127.0.0.2:1025
mydomain = cskk.id.au
myorigin = cskk.id.au
mydestination = $myhostname, localhost.$mydomain, localhost
inet_interfaces = localhost
mynetworks_style = host

Obviously change:
- cskk.id.au to your own domain - you want the local email system to 
  send legitmate headers.
- relayhost to be your ssh tunnel: localhost:1025

Relayhost is the upstream smarthost which accepts your email. It is the 
equivalent of the smtp_url in the mutt config, but having postfix do 
that bit. It can of course be an ISP etc; for example one of the (many) 
commented out ones I've got here is:

##relayhost = mail.optusnet.com.au

which was there to deliver directly to a former ISP. My 127.0.0.2 is 
part of my local setup: I've got haproxy running there, to deliver to 
whatever upstream ISP is available - in my case the home server or an 
ssh tunnel to our cloud VM, both of which have their own postfix setups.

Cheers,
Cameron Simpson 


Re: No Messages in Hotmail Inbox

2021-08-13 Thread Cameron Simpson
On 12Aug2021 12:01, chalao.adda  wrote:
>Hello,
>
>I created my Hotmail account through mutt-wizard. Previously I created Gmail 
>and GMX accounts successfully. But this time I find no messages in Hotmail 
>Inbox, though I have thousands of emails in Hotmail. Here are the steps I did.
>
>gpg --full-generate-key
>pass init j**@hotmail.com
>mw -a j**@hotmail.com
>mw -y j**@hotmail.com
>Now when I run neomutt, I find no messages in Inbox.

I'm assuming mutt-wizard and "mw" are from here:

https://muttwizard.com/
https://github.com/LukeSmithxyz/mutt-wizard

and that "pass" is this:

https://www.passwordstore.org/

It seems to me that "mw -y" syncs all your email to local storage. Are 
you sure your neomutt is pointing at the local storage and not your IMAP 
mailbox?

Maybe it is only showing "new" messages? Just guessing vaguely here.

>Also when I run mbsync -a , it says,
>C: 3/3 B: 52/52 F: +2/2 *2/2 #0/0 N: +8/8 *0/0 #0/0an INBOX?

That seems like a similar symptom.

Cheers,
Cameron Simpson 


Re: Can index lines be wrapped?

2021-08-13 Thread Cameron Simpson
On 13Aug2021 08:17, Chris Green  wrote:
>On Fri, Aug 13, 2021 at 08:15:26AM +0200, Alexander Dahl wrote:
>> On Thu, Aug 12, 2021 at 08:30:11AM +0100, Chris Green wrote:
>> > I read my mail (via ssh) on quite a variety of devices, one is an
>> > Android phone running termux which has rather short lines which
>> > truncate much of the subject in index view.
>> >
>> > I could rearrange the index_format but that would change it for all my
>> > viewing devices and I don't really want to do that, and anyway I'd
>> > still lose some information.
>>
>> What about a macro or shortcut to toggle between different settings?
>>
>That's possible but it's just one more thing that one has to to
>manually which isn't really necessary.

I always invoke via a script (named "+") which has assorted 
figure-it-out logic before invoking mutt. You could sniff the terminal 
size and choose a narrower index_format for narrow terminals. (_And_ 
bind that to a keyboard toggle macro as well if you wanted.)

I've got a ttysize script which does this:

stty -a 


Re: Retrieve inbox contents and email body

2021-07-06 Thread Cameron Simpson
On 06Jul2021 18:19, Julius Hamilton  wrote:
>Thanks very much.
>I am now looking into using a Python IMAP library.

Note that imaplib from the Python standard library is only a low level 
wrapper for the protocol, not a fulling fledged "remote IMAP mailbox" 
tool (which you might expect to trweat it like a map or something).  
There are other IMAP modules in PyPI offering higher level access.

>However, I think there must be a way to just dump the contents of a Mutt
>page, and that could be an easier short-term solution.
>You mentioned there being a print function.

See the manual for "print", "print_command", "print_decode". The 
defaults are ok, except that you would want to set "print_command" to 
something like "cat" for the message to appear on stdout.

A bit of experimentation reveals that  is as good as 
, so something like this:

mutt -e 'set wait_key=no; push 
".cat"' -f YOUR_MBOX

seems to work. Open mailbox, tag every message (or whatever subset you 
want), pipe through "cat", quit.

This is all a bit of a hack - mutt is inherently an interactive 
programme and the above works my pushing keystrokes onto the input 
stream.

Cheers,
Cameron Simpson 


Re: Retrieve inbox contents and email body

2021-07-05 Thread Cameron Simpson
On 05Jul2021 19:19, Julius Hamilton  wrote:
>I would like to retrieve the contents of my inbox - the emails' 
>metadata,
>sender, subject, date - to stdout, rather than by opening an application.
>Can Mutt dump the inbox page? Does it have any way to return to stdout
>information about the inbox? Or should I use a different tool for this?

I would use a different tool. Personally I'd write a little Python 
programme to read you inbox (IMAP? Local mail folder?) and report.

>I would also like to retrieve the body of a specific email in the same way,
>to stdout. Do the emails have identifier tags by which they can be
>specified? Or, what would be a way to do this?

All messages have a Message-ID: header which has a unique value per 
message. Mutt's %i pattern operator matches message-ids. I suppose you 
could invoke mutt from the command line this instructions to open the 
message based on the message-id and "print" it, specifying the print 
action as something which writes output.

But again, if I'd already written the Python programme above (summarise 
ths inbox) it would probably be easier to extend that to write out the 
desired message.

More context? There are probably already tools to do things like what 
you suggest, even if they are not exact matches for what you ask for.

Cheers,
Cameron Simpson 


Re: {.quotelead} strings in markdown2html messages

2021-06-29 Thread Cameron Simpson
On 29Jun2021 20:36, Aitor Soroa  wrote:
>I am a big fan of the markdown2html script (in contrib), which allows 
>me to write messages in markdown format. The script converts the message into 
>html
>and mutt sends them as attachments.
>
>However, lately the script writes strange `{.quotelead}` strings, as can be
>seen in the recipe below. I'd rather get rid of those, but unfortunately
>i don't know how. Any help in this regard would be much appreciated!

Just looking at my local copy of this script (which don't use), it has a 
pretty obvious bug.

The string quotelead occurs in 3 places:

In some CSS included with the message:

.quotelead {
font-style: italic;
margin-bottom: -1em;
color: #999;
font-size: 80%;
}

In the generate_lines_with_context function, which has this little 
snippet:

for prev, cur, nxt in generate_lines_with_context(mdwn):
# The lead-in to a quote is a single line immediately preceding 
# the
# quote, and ending with ':'. Note that there could be multiple 
# of
# these:
if re.match(r'^.+:\s*$', cur) and nxt.startswith('>'):
ret.append(f'{{.quotelead}}{cur.strip()}')
# pandoc needs an empty line before the blockquote, so
# we enter one for the purpose of HTML rendition:
ret.append('')
continue

and in some post processing:

def _reformat_quotes(html):
'''
Earlier in the pipeline, we marked email quoting, using markers, which 
we
now need to turn into HTML classes, so that we can use CSS to style 
them.
'''
ret = html.replace('{.quotelead}', '')
ret = 
re.sub(r'\n((?:\n)*)(?:\{\.quote(\w+)\})',
 r'\n\g<1>', ret, 
flags=re.MULTILINE)
return ret

The issue is that it is looking for this:

{.quotelead}

to replace it with this:



to access the CSS, but the generated text just puts:

{.quotelead}stripped-line-of-text

into the message. So no match.

Try hacking the script to change this:

ret.append(f'{{.quotelead}}{cur.strip()}')
ret.append(f'{{.quotelead}}{cur.strip()}')

That might make for ugly HTML rendering, but would at least the litter 
will be gone.

Cheers,
Cameron Simpson 


Re: Reply from address with +

2021-06-27 Thread Cameron Simpson
On 25Jun2021 18:41, Chuck Martin  wrote:
>I don't think there's another way.  That's the way I do it, anyway, but
>I don't bother with the [^@], using .* instead.  I also have the whole
>thing in double quotes, and the backslashes are doubled for some reason.
>I don't remember why I had to do that, but it is probably necessary, or
>it wouldn't work for me (the doubled backslashes, I mean).

That will be the quotes. You want a single backslash in your regexp. In 
quotes some sequence \x means something (eg \n meaning a newline 
character and of course \\ to mean a backslash) so you need a double 
backslash in the quotes to get a single backslash in the post-quote 
value.

Cheers,
Cameron Simpson 


Re: Can Bounce command be used from the command line?

2021-06-07 Thread Cameron Simpson
On 08Jun2021 09:06, Cameron Simpson  wrote:
>The usual approach is to "push" the requisite keystrokes, which then 
>get played.

Example from one of my scripts:

mutt -f "$folder" -e "push '$pattern'"

I still recommend just using sendmail directly though.

I thought procmail had some directive to 'bounce" messages? Haven't used 
it for a while.

Cheers,
Cameron Simpson 


Re: Can Bounce command be used from the command line?

2021-06-07 Thread Cameron Simpson
On 07Jun2021 15:49, Jon LaBadie  wrote:
>Is there a way to use mutt from the command like to "Bounce" a
>received message.  I had hopes for the '-e "command"' option, but
>that is limited to configuration commands.

The usual approach is to "push" the requisite keystrokes, which then get 
played.

However, to "bounce" a message from the command line it is far more 
expedient to just go:

sendmail -oi addr addr addr ... < message

which is all mutt will be doing. Drops the same messages straight into 
the local mail system for deliver to the specified addrs.

Cheers,
Cameron Simpson 


Re: connection to dovecot times out about every ten minutes

2021-04-28 Thread Cameron Simpson
On 28Apr2021 17:23, Marco Fioretti  wrote:
>I honestly don't know if the old modem had an integrated router, and I
>have already disposed of it. What I am sure of is that I had NOT
>changed anything in its settings for many months, if not years, and
>everything was working without problems until a few weeks ago, when I
>posted here. Also, why would any modem come from the factory, or be
>remotely updated by an ISP in ways that interfere with an absolutely
>basic use case of hundreds millions of people, that is keeping one's
>email client connected to its IMAP server for hours?

These are good points. That the behaviour was new and the modem not 
pretty much invalidates my idea, unless a recent mutt IMAP change (or, I 
suppose, the upstream dovecot) made the connection more idle at a TCP 
level. But I'm grasping at straws.

>As for the new one, I cannot check it right now because I am not at
>home, but it is working, so whatever it does, it is OK.

Q: How many user support people does it take to change a light bulb?
A: We have an exact copy of the light bulb here and it seems to be
   working fine.  Can you tell me what kind of system you have?

Glad to hear this. Cheers,
Cameron Simpson 


Re: index color scheme

2021-04-26 Thread Cameron Simpson
On 26Apr2021 16:32, Jon LaBadie  wrote:
>I can't figure out the correct configuration for my desired
>3 color scheme for the index.
>
>I can get New messages one color and  Read messages a second.
>
>My third color would be for deleted messages.  It works fine
>for Read messages that are deleted, but has no affect on New
>messages.  They stay the same "New message" color when deleted.
>
>Probably I'm missing something basic and simple.

The color rules are applied in order. Can you show us your rules?

I would guess that your new message rule is after your deleted rule.

Cheers,
Cameron Simpson 


Re: connection to dovecot times out about every ten minutes

2021-04-26 Thread Cameron Simpson
On 26Apr2021 17:21, Marco Fioretti  wrote:
>update on this:
>to make a long story short
>1) I did run mutt with debug enabled , but could not recognize anything 
>useful
>2) I had the same problem with mutt from my laptop
>3) a few days ago I received a new modem from my ISP, as part of their
>network upgrade operations
>4) more or less in the same moment the problem I reported here
>disappeared. Now mutt stays connected even 24 hours without losing
>connection.
>
>I am NOT 100% sure that the problem disappeared AFTER the change of
>modem. That happened during a few chaotic days, both work- and
>family-wise, so I did not take notes. And modems may have nothing to
>do at all with the disconnections. But now the problem is not there
>anymore, I have no clue what may have happened, and if anybody can
>guess... thanks in advance.

_If_ the new modem is relevant, maybe the modem's internal firewqll 
rules are related. Anything which NATs (translates between your home LAN 
private address range to some external IP address used by the modem) 
must keep state for every connection crossing the modem.

There's no "idle detection" in TCP (without keepalives) or UCP so if 
some device on either side of the connection dies/crashes while the 
connection is _idle_ there's no indication at the modem that this has 
happenned - there's just no traffic for that connection, which is 
already the case.

So... stateful firewalls (eg your modem doing NAT) get bored, and 
usually have some setting to discard long-idle connections. I can 
imagine a "polite" device timing out such a TCP connection sending an 
RST (reset) packet in each direction just before discarding the state to 
inform the endpoints that the connection is gone (thus letting each end 
see this in a timely fashion, rather than just "next time they try to 
send traffic").

Maybe your previous modem's timeout for that was 10 minutes? And the new 
one is more generous (or even does not timeout connection states)?

Just guessing.

Cheers,
Cameron Simpson 


Re: HTML email?

2021-04-09 Thread Cameron Simpson
On 09Apr2021 22:41, Andy Spiegl  wrote:
>> So in fact I choose the alternative order per message:
>Great idea!  And thanks for pointing out the "group" command which was
>completely unknown to me so far.  Unfortunately I'm missing some good
>examples in the manual in order to completely understand its possibilites.
>
>Questions like
> - Do I have to repeat the -rx before every pattern?

No.

However, the manual's misleading:

4. Address Groups
Usage: group [ -group name ...] { -rx expr ... | -addr expr ... }

I'd read that as: "-rx expr ..." is one of the 2 group definition 
alternatives.  It looks like a group is either a set of addresses or a 
set of regexps.

However, the following text description then falsifies the implication:

The group command is used to directly add either addresses or
regular expressions to the specified group or groups. The
different categories of arguments to the group command can be
in any order.  The flags -rx and -addr specify what the following
strings (that cannot begin with a hyphen) should be interpreted
as: either a regular expression or an email address, respectively.

That says to me you can do this:

group -group group1 -group group2 -rx regexp1 regexp2 -addr addr1 addr2 -rx 
regexp3

which appends 5 definitions to "group1" and "group2". Or, on reflection, 
maybe:

group -group group1 group2 -rx regexp1 regexp2 -addr addr1 addr2 -rx regexp3

On that basis I'd have written the syntax like this:

Usage: group [ -group name ]... { -rx expr... | -addr expr... }...

Note the repetition bound to each optional thing: multiple "-group name"

But the above is somewhat conjecture - we need to read the source, alas.

> - Can I group mutt aliases with -addr or just addresses?

Not sure what you mean here.

I have no -rx groups, but _all_ my mutt alias definitions take the form:

alias -group 37signals 37signals \
37signals Billing , \
Jason Fried - 37signals Newsletter 

which defines _both_ an alias _and_ an address based group (which have 
the same name, hence the double "37signals" above - you can name these 
differently).

That's because all my groups are computed from my address db.

> - What is a good way to check/verify whether my group definition works?
>E.g. I defined the group "me":
> group -group me -rx me\..* -rx .*\.spiegl.*@ -rx .*\.andy.*@

Maybe write some index colouring rules based on the group membership?  
That seems the simplest way to test this stuff to me.

I think I'm going to have to write myself a macro to "reload all my 
settings" - I've had to muck around in the past trying stuff like this, 
and leaving/entering mutt every time I make a change is tedious.

>Hoping that it would also pick up all "me." aliases like: me.company1, 
>me.company2, ...

Looks like it should. But the regexps may need to match the whole 
address part, maybe you need:

spiegl\..*@.*

Or maybe not, if they're applied to the "whole address" eg "Andy Spiegl 
".

I confess I'm just guessing here.

But note this in the PATTERNS section:

Special attention has to be paid when using regular expressions 
inside of patterns. Specifically, Mutt's parser for these patterns 
will strip one level of backslash (“\”), which is normally used for 
quoting. If it is your intention to use a backslash in the regular 
expression, you will need to use two backslashes instead (“\\”).

So your regexps above need backslash doubling. Or maybe quoting :-)

>Do you happen to know where to find more about it? (my last resort 
>would be the source code)

The source, alas.

I don't use regexps for address matching (if I can help it); they're a 
lousy tool for matching addresses.

Cheers,
Cameron Simpson 


Re: Are there any good maildir manipulation utilities out there?

2021-04-08 Thread Cameron Simpson
On 08Apr2021 08:40, Chris Green  wrote:
>On Thu, Apr 08, 2021 at 08:43:48AM +1000, Cameron Simpson wrote:
>> It's also not particularly well suited
>> to Chris' requirement, which includes preserving the source tree shape
>> in the archiving process.
>>
>Yes, that was just one typical requirement.  The other major
>requirement is rather different.  I have (for example) lots of
>messages about orders from Screwfix which are currently spread around
>in various sub-directories of 'shopping'.  For suppliers like Screwfix
>from whom I buy a lot of rather mixed up sort of items it now makes
>sense to me to keep them all in a Screwfix directory.  So the
>requirement is to find all messages from Screwfix and put them in one
>directory.

Note that there's no fundamental reason you can't link the same message 
into many Maildirs. So link them all into a screwfix folder, and also 
where they naturally land courtesy of other filing rules.

Also, didn't you put some work into tagging messages. A search on 
tag=screwfix might go well.

>> I suspect Chris may need to roll his own. I'd imagine something like:
>>
>> find message paths using mairix \
>> | move message files sideways, making sure there's no conflicts
>>
>Yes, I think it may have to be a roll my own something like this.
>It's just that mairix doesn't provide a very good 'handle' to use.

I'm using notmuch. In particular, this:

notmuch search --output=files -- search terms ...

emits message pathnames. That gets you the folder structure with alittle 
path fiddling. So my "notmuch-search" script (aliased as "++") goes:

if [ $dothreads ]
then
  notmuch search --output=threads -- "$@" \
  | while read -r tid
do  notmuch search --output=files -- "$tid"
done
else
  notmuch search --output=files -- "$@"
fi \
| egrep '/(new|cur)/[^/]+$' \
| sort -u \
| xxargs arg1 -end "$mdir/new" set-x ln -i -s \

at the bottom:
- find the messages
- winnow some noise - probably unnecessary
- "sort -u" to remove duplicates
- make symlinks to all the messages in the search result maildir's "new" 
  subdir

Then it fires up mutt on the result maildir.

I would think you could shoehorn the above logic a bit to achieve your 
"keep the hierachy shape" thing.

Cheers,
Cameron Simpson 


Re: HTML email?

2021-04-08 Thread Cameron Simpson
On 07Apr2021 07:22, John Niendorf  wrote:
>How do you all deal with HTML email?

Composing HTML is a can of worms. I'd need to dig into the list archives 
- it has been discussed.

Displaying HTML uses 2 main settings:

The .mailcap entry for text/html with the "copiousoutput" flag. For 
example:

text/html; exec 2>&1 && env DISPLAY= unhtml %s; copiousoutput

"unhtml" is a personal script which invokes whatever I prefer to use to 
transcribe HTML as plain text. Currently it invokes:

lynx -stdin -dump

That way I don't have to hack my mailcap much, better to hack the script 
if I shift tools, eg to w3m.

The other setting is the alternative_order setting, which says which 
Content-Type to prefer of a multipart/alternative message. These usually 
have a text/plain and text/html part (though of course they course have 
other things, eg a text/markdown part). My default setting is:

alternative_order text/plain text/html

which prefers the plain text version, sidestepping the HTML altogether.

However, there are plenty of platforms which are HTML first and provide 
either very poor plaintext equivalents of empty ones, or ever just stuff 
the raw HTML into both parts. Absolutely rubbish quality of 
implementation, but there you go.

So in fact I choose the alternative order per message:

# alternative-order criteria
message-hook . 'unalternative_order *; alternative_order text/plain 
text/html'

message-hook '~h "X-Mailer: Apple Mail" ~X 1-' 'unalternative_order *; 
alternative_order text/html multipart/mixed text/plain'

message-hook '%f htmlers | ~f @no-re...@cc.yahoo-inc.com | ~f @outlook.com 
| ~f live.com | ~f @facebookmail.com' 'unalternative_order *; alternative_order 
text/html text/plain'

So far all messages I set up the default. Then for Apple Mail I put the 
HTML first because of the way Apple Mail packs attachments, which is 
weird. Then for an elite set of negligent idiots I put HTML first 
because I _know_ that they shift a plaintext version and the plaintext 
is always rubbish.

That last criterion is email from outlook.com, live.com, facebook.com, 
yahoo's PR/info people, and whomever I have explicitly added to my mutt 
"htmlers" group.

Cheers,
Cameron Simpson 


Re: Are there any good maildir manipulation utilities out there?

2021-04-07 Thread Cameron Simpson
On 07Apr2021 18:34, Kurt Hackenberg  wrote:
>On 2021/04/07 18:16, Kurt Hackenberg wrote:
>>On 2021/04/07 17:01, Chris Green wrote:
>>>I'm looking for a tool which will allow me to search through a large
>>>hierarchy of maildir messages and then provide actions to take on the
>>>matched messages.
>>
>>
>>fdm?
>><https://github.com/nicm/fdm/blob/master/MANUAL>
>>
>>Or some other filter-and-deliver program -- but not procmail, 
>>because it's a zombie, abandoned by its last maintainers, last 
>>release 20 years ago.
>
>
>Maybe Cameron Simpson's filer, if he publishes it.

It's got some dependencies on some stuff I need to weed out before it is 
easy for someone-not-me to use. It's also not particularly well suited 
to Chris' requirement, which includes preserving the source tree shape 
in the archiving process.

I suspect Chris may need to roll his own. I'd imagine something like:

find message paths using mairix \
| move message files sideways, making sure there's no conflicts

Cheers,
Cameron Simpson 


Re: my weekend project: a streaming POP3 fetcher, replacing fetchmail/getmail

2021-04-07 Thread Cameron Simpson
On 06Apr2021 23:12, Kurt Hackenberg  wrote:
>On Wed, Apr 07, 2021 at 09:43:36AM +1000, Cameron Simpson wrote:
>>My new tool streams the fetches: it issues RETRs for every message up
>>front at maximum network speed - fully buffered and with no waits. A
>>parallel worker thread collects the messages as they come in at full
>>speed (the upstream server likely also gets to fully buffer); it issues
>>DELEtes as each message is saved, also fully buffered.
>
>Slick.  Clearly the right way to handle that high latency.

Just tried it on the satellite link with an overnight load of messages, 
normally a 10 minute exercise with getmail (give or take). 411 messages, 
8.5 seconds.

>Have you ever tried the program fdm?  It fetches, filters, and
>delivers mail, like getmail and procmail combined.  I haven't tried
>it, but it looks interesting.

I have not.

My mail filer is decoupled from my fetcher: it monitors spool Maildirs 
(which also means I can refile a message just by saving it to a spool 
Maildir).  And it has its own syntax to my liking; other tools 
inherently will not :-)

And looking at the conf file, it seems that (like procmail, which I 
abandoned years ago) it matches using regexps. These are appalling for 
email addresses. When testing addresses, my filer does a correct address 
parse and compares the inner component (eg "c...@cskk.id.au" from "Cameron 
Simpson "), and can do set membership tests on that eg 
"is this address in this group?".

>This paragraph in its manual sounds like it might stream fetching like
>your program:
>
>"fdm tries to queue a number of mails simultaneously, so that older
>can be delivered while waiting for the server to provide the next. The
>maximum length of the queue for each account is set by the
>'queue-high' option (the default is two) and the maximum mail size
>accepted by the 'maximum-size' option (the default is 32 MB). In
>addition, the 'rewrite' action requires an additional temporary
>mail. Although fdm will fail rather than dropping mail if the disk
>becomes full, users should bear in mind the possibility and set the
>size of the temporary directory and the fdm options according to their
>needs."

I've been reading the manual. I think it does not. I think it actually 
allows the filing/saving to proceeed while requesting/fetching the next 
message. So a simple form of parallelism, but not one which reduces the 
fetch latency between requests.

>fdm is at github:
><https://github.com/nicm/fdm/>
>
>The paragraph quoted above is at about line 300 in the manual, which is here:
><https://github.com/nicm/fdm/blob/master/MANUAL>

Thanks. An interesting read.

Cheers,
Cameron Simpson 


my weekend project: a streaming POP3 fetcher, replacing fetchmail/getmail

2021-04-06 Thread Cameron Simpson
Like several here, I fetch email from my ISP mail spool(s) and file 
messages locally. If my laptop's been offline overnight there can be 
hundreds of messages to fetch when I wake it up. On a satellite link 
(geostationary) with a ping time of over 600ms this can be many minutes 
of tedium.

The raw bandwidth is fine and my filing process is pretty expeditious;
the root cause of that tedium is network latency and the synchronous 
behaviour of getmail. Its cycle is like this:

- RETRieve the message, collect and save
- DELEte the message
- repeat for all the messages, then QUIT to commit the deletes

Each of steps 1 and 2 above incurs over a second just in network 
latency. That scales up over hundreds of messages.

My new tool streams the fetches: it issues RETRs for every message up 
front at maximum network speed - fully buffered and with no waits. A 
parallel worker thread collects the messages as they come in at full 
speed (the upstream server likely also gets to fully buffer); it issues 
DELEtes as each message is saved, also fully buffered.

The code's here:

https://hg.sr.ht/~cameron-simpson/css/browse/lib/python/cs/pop3.py?rev=tip

The cs.pop3 modules on PyPI here:

https://pypi.org/project/cs.pop3/

and can be installed with:

pip install cs.pop3

which also provides a "pop3" command in your Python environment.

Typical use is:

pop3 dl mylo...@mail.cskk.id.au ~/var/mail/spool

specifying my internet mail spool and the local Maildir to receive the 
messages.

Cheers,
Cameron Simpson 

If you cannot, in the long run, tell everyone what you have been doing, 
your doing has been worthless.   - Erwin Schrodinger


Re: Automate standard replies

2021-03-30 Thread Cameron Simpson
On 30Mar2021 18:42, Andy Spiegl  wrote:
>> >Uhm, maybe compatibility issues between different bash versions?
>> I do not write for bash, I write for /bin/sh, which exists on every 
>> UNIX
>Sorry, I meant to write sh.

:-)

>> I can ship you a tarball of my out-of-my-homedir /opt/css tree, which has 
>> the lot.
>True.  Works now!  Even inside my zsh, it seems. :-)

My login shell is zsh also.

>Gotta go back to bed.  Covid-19 started to bug me since last night - 
>after 1 week with hardly any symptoms.

I'm sorry to hear that. Good luck.

Cheers,
Cameron Simpson 


Re: Automate standard replies

2021-03-29 Thread Cameron Simpson
On 29Mar2021 20:02, Andy Spiegl  wrote:
>> Well sure. Great desktop, BSD UNIX underneath. And, frankly, really 
>> nice hardware.
>A few years ago I would have fully agreed with you and only complained
>about the price.  But lately we're having so many problems with Apple
>devices and with every OS upgrade the trouble seems to get worse.

I've done fairly well then. Our apple devices are personal, not 
corporate, but still. What's happened?

>I wonder
>how much of the original BSD is left at all.

I think the kernel code at least is publicly available.

Apples have quite a few special purpose systems, but that's no worse 
than systemd and NetworkMangler etc etc spreading through the Linux 
systems. And at least most of the Apple daemons do have man pages, which 
is often more than can be said for the free software tools.

>I can't believe me writing
>that we are having almost no issues with the Windows 10 PCs and have to
>spend so much support time for Apple users.  Unfortunately there is only a
>minor number of Linux users in our department but they need hardly any
>attention from us. :-)

That's pretty user dependent, based on my time sysadmining in a former 
life.

>> It was written on Linux.
>Uhm, maybe compatibility issues between different bash versions?

I do not write for bash, I write for /bin/sh, which exists on every UNIX 
platform. I have, AFAIR, exactly _one_ bash script, which is my 
"readline" script, whose sole purpose is to prompt for an input using 
bash's readline support (file completion etc).

https://hg.sr.ht/~cameron-simpson/css/browse/bin/readline

Bash itself is nonportable, not universally present, and provides few 
features of use in scripting. By the time I want such things it is often 
time to step up a level to something like Python.

>I had to deal with several smaller issues.  Maybe because I only
>downloaded the referenced scripts and overlooked some.  That was most
>likely the problem.

I can ship you a tarball of my out-of-my-homedir /opt/css tree, which 
has the lot. That's the fast way.

Here:

https://www.dropbox.com/s/fa5fx70v0tw0xwn/css-20210330.tar.gz?dl=0

Typical install is to unpack in /opt, but you can put it anywhere.

>> For example, this is my current desktop.
>Cute. :-)
>I'm running FVWM window manager with 3x6 virtual desktops on two screens.
>A little bit more colorful than your setup but equally nerdy. :-)

Were I on X11 I'd be using FVWM as well. See here:


http://fvwm.sourceforge.net/screenshots/desktops/index.php?num=50=plain

Scan down the page for Cameron Simpson or Zen style. Looks like my 
writeup of the setup and design is offline though :-( An indefinite 
number of virtual desktops, many with names!

Cheers,
Cameron Simpson 


Re: Automate standard replies

2021-03-28 Thread Cameron Simpson
On 29Mar2021 10:10, Cameron Simpson  wrote:
>On 28Mar2021 14:04, Andy Spiegl  wrote:
>>> open some-directory-name
>>Oh, MacOS !  :-)
>
>Well sure. Great desktop, BSD UNIX underneath. And, frankly, really nice
>hardware.

Not to mention iterm, IMO the best terminal emulator I have ever used.

For example, this is my current desktop.

https://www.dropbox.com/s/dn0whcursdzfn23/desktop--2021-03-29.png?dl=0

That is:

- 2 iterm windows side by side, one filling each half of the screen

- the left one has 2 iterm panes: a big editor at the top, and a shallow 
  shell below it; those two panes are in the left of the two iterm tabs

- the editor window is vim, with 2 vim windows in it (my normal 
  arrangement because the vim focus is just a toggle instead of a 
  rotation among more windows)

- the right iterm window is _one_ pane, containing mutt in a tmux 
  session; this is the third of the 4 tabs in the right hand iterm 
  window

- because I'm replying to you, the reply compose is _also_ a tmux 
  session (which means I can detach!)

- the upper (mutt) tmux session has been split into 2 panes by my reply 
  macro - the upper has the mutt, the lower has the reply tmux session 
  (hence the double status line at the bottom - tmux inside tmux).
  Also notice the tmux session names at bottom left - these make the 
  tmux listing easy to recognise. Once this reply is sent, I'll have a 
  full height mutt index in front of me again.

- the iconic Mac menu bar is autohidden, as is the traditional Mac Dock; 
  they appear i I wave my mouse to the appropriate screen edge

- this is one of 10 virtual desktops

You may notice I have lots of pixels. That's a 16" Macbook pro laptop in 
native resolution mode.

Cheers,
Cameron Simpson 


Re: Automate standard replies

2021-03-28 Thread Cameron Simpson
On 28Mar2021 14:04, Andy Spiegl  wrote:
>> open some-directory-name
>Oh, MacOS !  :-)

Well sure. Great desktop, BSD UNIX underneath. And, frankly, really nice 
hardware.

>> Though I suppose that makes it part of my desktop environment. Which
>> might be what you intended.
>Yes.
>
>> Oh, me too. See the apphelper script :-)
>Nice.  Tried to get it to run on my Linux machine

It was written on Linux.

>but after downloading and adapting quite a few dependant scripts of yours
>I followed the 80% - 20% rule and gave up for now.

I know this isn't your priority, but ideally my scripts should not need 
adaption. Can you tell me what kind of changes you were trying?

The tissue paper effect means often the easy thing is the /opt/css 
install tarball, then ". /opt/css/env.sh" in your .profile (at the end, 
so nothing preempts your preferred stuff - it is all "additional").

apphelper runs of the standard mime files (mime.types and the other one, 
mailcap). The "ah" wrapper script (in bin-cs/) was how I usually used 
it.

Example mailcap entries:

image/png; ah %s -1 xv; gui
image/octet-stream; ah %s -1 xv; gui


>Thanks - I learned a lot today!

Excellent! Anything that's unclear, just ask.

Cheers,
Cameron Simpson 


Re: Automate standard replies

2021-03-27 Thread Cameron Simpson
On 28Mar2021 08:35, Cameron Simpson  wrote:
>On 27Mar2021 18:18, Andy Spiegl  wrote:
>>Just one piece is missing: "open" (used in mail-open-attachments)
>>Is that part of your windowmanager maybe?
>
>No, it's a standard MacOS command. It opens files (and various URL-like
>things) with their matching app.

Though I suppose that makes it part of my desktop environment. Which 
might be what you intended.

Cheers,
Cameron Simpson 


Re: Automate standard replies

2021-03-27 Thread Cameron Simpson
On 27Mar2021 18:18, Andy Spiegl  wrote:
>Just one piece is missing: "open" (used in mail-open-attachments)
>Is that part of your windowmanager maybe?

No, it's a standard MacOS command. It opens files (and various URL-like 
things) with their matching app.

open some-directory-name

pops up that directory in the Finder, the MacOS desktop file browser.  
There I can quickview attachements or open them in turn in their own 
appropriate apps. Sometimes that's the exasy thing for some messages.

Alas, the common Linux-side "open" command does something else.

>But no problem. I simply replaced it with:
> cd "$unpackdir" && xterm
>I'm a command line freak anyway. (c;

Oh, me too. See the apphelper script :-)

Cheers,
Cameron Simpson 


Re: Automate standard replies

2021-03-25 Thread Cameron Simpson
On 25Mar2021 23:23, Andy Spiegl  wrote:
>> Use  to send the raw message to a script of your own. 
>> Some
>> random macros of my own:
>...
>> macro index,pager V "mail-open-attachments" 
>> "extract attachments to temp dir and open"
>
>Cool idea!
>Would you mind sharing the "mail-open-attachments" with us?

The main trickiness of  is that you have the raw message - 
you need to extract the body part or whatever other bits you want.

mail-open-attachments is here:


https://hg.sr.ht/~cameron-simpson/css/browse/bin/mail-open-attachments?rev=tip

It just makes a scratch directory and unpacks the message with the 
"munpack" command, which is part of the "mpack" package:

https://web.archive.org/web/20190220145801/ftp.andrew.cmu.edu/pub/mpack/

The pops up the directory. I'm on a Mac so I do that with the "open" 
command, which pops up a Finder window for directory paths.

The script uses datecode and fixexts from the same place:

https://hg.sr.ht/~cameron-simpson/css/browse/bin/datecode?rev=tip
https://hg.sr.ht/~cameron-simpson/css/browse/bin/fixexts?rev=tip

datecode is just a convenience (which I use a lot) but fixexts is 
actually useful: it sniffs files and appends the corresponding extension 
(if the extension is missing), which helps file brwosers play nice, eg 
to open them with the correct application.

If you want to grab everything related and the tissue box effect is too 
large, I can ship you a tarball or you can clone the repo. The repo is 
Mercurial (hg), I've got a public git clone too if you've not got 
Mercurial.

Front page: https://hg.sr.ht/~cameron-simpson/css

Cheers,
Cameron Simpson 


Re: Automate standard replies

2021-03-21 Thread Cameron Simpson
On 21Mar2021 12:15, Francesco Ariis  wrote:
>how can I automate typing standard replies?
>
>Example: say I want to type «lgtm» (top post) as a reply to a message,
>as now I have to:
>- press `r` (reply)
>- press `i` (my editor is Vim)
>- «lgtm»
>- then  (exit insert mode)
>- ZZ (exit/save file)
>- finally `y` to send
>
>Is there a way to one-button this? I thought about a macro,
>but I suspect that wouldn’t work, since I am firing up an
>editor.
>Thanks in advance

If you're automating this, you don't want an editor do you?

Use  to send the raw message to a script of your own. Some 
random macros of my own:

#macro index,pager ,x "mailunpack -d ~/"
macro index,pager V "mail-open-attachments" "extract 
attachments to temp dir and open"
macro index,pager H "unhdr|a2urls|sort -u|less" "list URLs"

BTW, I've got "set autoedit=yes" and "set edit_headers=yes", drops you 
straight into an editor on reply etc.

Cheers,
Cameron Simpson 


Re: group and alias

2021-03-01 Thread Cameron Simpson
On 01Mar2021 09:45, Kevin J. McCarthy  wrote:
>On Mon, Mar 01, 2021 at 01:32:47AM -0500, Jon LaBadie wrote:
>>And where/how do you use the group rather than the alias?
>
>Groups are used in patterns.  See the modifiers starting with '%' in 
><http://www.mutt.org/doc/manual/#patterns>.

For example, the "%f htmlers" in the last line below:


# alternative-order criteria
message-hook . 'unalternative_order *; alternative_order text/plain 
text/html'
# Apple Mail embeds attachments in the HTML part instead of outside 
# the multipart/mixed
message-hook '~h "X-Mailer: Apple Mail" ~X 1-' 'unalternative_order *; 
alternative_order text/html multipart/mixed text/plain'
# senders who can't seem to master multipart/mixed, and send empty
# or useless text/plain sections
# or just badly badly formatted plain text, such as live.com etc
message-hook '%f htmlers | ~f @no-re...@cc.yahoo-inc.com | ~f @outlook.com 
| ~f live.com | ~f @facebookmail.com' 'unalternative_order *; alternative_order 
text/html text/plain'

Cheers,
Cameron Simpson 


Re: group and alias

2021-02-28 Thread Cameron Simpson
On 28Feb2021 19:00, Jon LaBadie  wrote:
>Trying to use the "group" facility.  Expected I could
>do something like:
>  group -group ABC -addr  -addr  -addr 
>
>And then email 3 people with
>
>  $ mutt ABC
>
>I am able to accomplish this with an alias:
>
>  alias ABC , , 
>
>Is this not an application for which "group" was intended?

Aliases probably predate groups, I'd expect. They can contain only 
addresses (including other aliases).

Groups can include regexps for matching, so they're arguably more a 
PATTERN thing than an alias is.

And I presume that recognising that often aliases and groups server the 
same purpose, alias has a -group options which also adds the addresses 
to a named group. I maintain my alias using a small otuside db, and 
autogenerate them. They all have the form:

alias -group group_name alias_name \
address, \
... etc ...

where in fact group_name and alias_name are the same name.

Cheers,
Cameron Simpson 


Re: Choose 'From:' address based on 'To: address

2021-02-25 Thread Cameron Simpson
On 25Feb2021 09:56, Will Yardley  wrote:
>On Thu, Feb 25, 2021 at 04:11:02PM +, Alexander Huemer wrote:
>> Is there a configuration option to choose which 'From:' email address
>> shall be used if a mail to a certain 'To:' address is composed? If a
>> 'To:' address isn't in the mapping, then the 'set from' configuration
>> setting should be the fallback.
>
>I do something like this:
>
>set use_from
>set from="lu...@example.net"
>send-hook . 'unmy_hdr From:'
>set reverse_name
>alternates '(.*@example\.net|someuser@example\.com)'

reverse_name and alternates: this is only effective for personal email, 
meaning email with one of your own addresses in the to/cc. Most mailing 
lists do not do that, so there's no way to "reverse" from the to/cc to 
the from of the reply.

You want Jon's send-hook based approach, setting from etc based on the 
target address of the message you're composing.

Cheers,
Cameron Simpson 


Re: not to set message id in outgoing email

2021-02-16 Thread Cameron Simpson
On 16Feb2021 09:37, Shehu Dikko  wrote:
>* Cameron Simpson [20210216 09:02:36]:
>>On 15Feb2021 21:53, Shehu Dikko  wrote:
>>>* Peng Yu [20210203 20:23:18]:
>>>>I want the first MTA add one, instead of using the one generated by
>>>>mutt.
>>>
>>>One way to get what you want is to use esmtp:
>>>
>>># muttrc
>>>set sendmail="/usr/bin/esmtp"
>>>
>>># esmtprc
>>>  [...]
>>>  message_id = disabled
>>>  [...]
>>
>>Does that actually strip the one from mutt, or just prevent esmtp from
>>_adding on if missing_. Peng Yu is trying to stop mutt adding a message
>>id. [...]
>
>As you can see from the User-Agent and Message-ID strings of this
>message, it does indeed strip the one from mutt.

Aha. Thank you. - Cameron Simpson 


Re: Marking as unread when moving mail to another folder

2021-02-16 Thread Cameron Simpson
On 16Feb2021 11:31, Sam Kuper  wrote:
>On Thu, Jul 16, 2020 at 10:57:13AM +0200, Kai Weber wrote:
>> I have a save-hook defined to move all mail into a certain folder.
>>
>> save-hook "~A"  +personal/Archive
>>
>> Very often I want to move the mail into the archive without opening the
>> mail at all but then the mail ends up as unread in the archive.
>>
>> I want to mark the mail as "read" when moving. How could I achieve this?
>> I guess a macro could do it? Should I get rid of the save-hook?

Does it matter if its unread in the archive? I guess to you it does.  Do 
you want to mark _everything_ in the archive as unread? Maybe it would 
be quicker with a folder-hook which matches your archive folder and 
marks everything unread as read on entry to the folder.

>This macro may do what you want.  It marks mail as "read" and moves it
>to the $record file.
>
>Note that it makes use of `<`, which is Mutt's built-in shortcut for the
>$record file.

Note: I missed this remark, thought it was a typo :-) So the OP might 
want that replaced with +personal/Archive.

># Unlike index mode, pager mode does not accept this macro without
># an initial ``. So we need two variants of this macro, one
># for each mode.
>macro index  ss"set 
> auto_tag=yesn<yset
>  auto_tag=noy"
>macro pager  ss  "set 
> auto_tag=yesn<yset
>  auto_tag=noy"
>
>If anyone reading this knows of a better solution, please share it.

Remarks: I usually spell the pager macro like this:

macro pager  ss  "ss"

which avoids a lot of repetition.

I rarely end macros with , it shows things down and I like 
to only sync when intended, via the $ keystroke.

I wouldn't fiddle with auto_tag, I'd just preceed things with 
, since you seem to have made this conditional on beging 
tagged anyway. In principle, fiddling an option requires unfiddling it 
at the end of the macro with the usual cumbersome

set $my_option=$option
set option=whatever
... do stuff
set option=$my_option

shuffle embedded in the macro.

If you set wait_key=no during the macro you can drop the "y" you have 
embedded after the save.

Cheers,
Cameron Simpson 


Re: Deleting old maildir messages, is what I'm doing OK?

2021-02-15 Thread Cameron Simpson
On 16Feb2021 09:05, Cameron Simpson  wrote:
>On 15Feb2021 17:07, Chris Green  wrote:
>>Does no one else ever delete mail messges? :-)
>
>Hmm:
>[~/mail]fleet2*1> ls -ldh spam*
>-rw-rw-r--  1 cameron  cameron   1.7G 16 Feb 08:43 spam
>-rw-rw-r--  1 cameron  cameron   609M 16 Feb 08:46 spam-definite

The other thing of note here is that these are mbox folders (like many 
of my less visited folders, for compactness). You could just move them 
sideways and remove "old" sideways folders? I guess that works for 
Maildir too, now that I think about it.

Cheers,
Cameron Simpson 


Re: Deleting old maildir messages, is what I'm doing OK?

2021-02-15 Thread Cameron Simpson
On 15Feb2021 17:07, Chris Green  wrote:
>Does no one else ever delete mail messges? :-)

Hmm:

[~/mail]fleet2*1> ls -ldh spam*
-rw-rw-r--  1 cameron  cameron   1.7G 16 Feb 08:43 spam
-rw-rw-r--  1 cameron  cameron   609M 16 Feb 08:46 spam-definite

Possibly not.

Can you tell mairix to not index the Junk folders?

Cheers,
Cameron Simpson 


Re: not to set message id in outgoing email

2021-02-15 Thread Cameron Simpson
On 15Feb2021 21:53, Shehu Dikko  wrote:
>* Peng Yu [20210203 20:23:18]:
>>I want the first MTA add one, instead of using the one generated by
>>mutt.
>
>One way to get what you want is to use esmtp:
>
># muttrc
>set sendmail="/usr/bin/esmtp"
>
># esmtprc
>identity = u...@ho.st
>  hostname = smtp.ho.st:587
>  username = u...@ho.st
>  password = password
>  starttls = enabled
>  message_id = disabled
>
># man esmtprc
>message_id
>
>  Whether to set the Message-ID field of the message before sending.
>  Normally the receiving MTA sets the Message-ID if missing, so you can
>  turn this off if your sending host does not have a fully qualified
>  domain name.
>
>  Allowed values are either enabled or disabled. It defaults to 
>  enabled.

Does that actually strip the one from mutt, or just prevent esmtp from 
_adding on if missing_. Peng Yu is trying to stop mutt adding a message 
id.

Maybe the objective can be achieved by stripping the mutt-provided 
message-id in the sendmail setting before it reaches the MTA?

Untested:

set sendmail="sed '1,/^\$/{/^Message-ID:/d'}' | /usr/bins/sendmail"

(The backslash if for mutt, not sed or the shell.)

Cheers,
Cameron Simpson 


Re: Quoting character when replying

2021-02-15 Thread Cameron Simpson
On 15Feb2021 05:56, Angel M Alganza  wrote:
>On Mon, Feb 15, 2021 at 03:40:16PM +1100, Cameron Simpson wrote:
>>Aye. I use format=flowed all the time; there's stuff in the wiki about
>>settings to go with it, for example to make composition easy.
>
>I see.  :-)  Oddly enough, your quotations appear with the leading >
>without the space, while mine appear with the space, despite me thinking
>for two years (!!!) that it wouldn't if I didn't add them by hand!  LOL

Hmm. Beginning to think this is presentation. What are your mutt flow 
settings? I've got this:

set reflow_text=yes
set reflow_wrap=-4
set text_flowed=yes

In mutt, my "Aye..." message above looks like this:

Content-Type: text/plain; charset=us-ascii

On 15Feb2021 05:20, Angel M Alganza  wrote:
>On Sun, Feb 14, 2021 at 07:36:34PM -0800, Kevin J. McCarthy wrote:
>>You can read a bit more about it at
>><http://www.mutt.org/doc/manual/#ff>, but you'll want to do more
>>research if you intend to compose format=flowed emails.
>
>Thank you, I'm going to have a look at this.  I'd like to keep 
>doing it,
>since it might help others to read my email better regardless of 
>their
>email client and display.

Aye. [...]

No spaces. The message source looks exactly the same. [... huh? ...] And 
it isn't in format-flowed. Grrr.

Yours _is_ flowed:

Content-Type: text/plain; charset=iso-8859-1; format=flowed

On Mon, Feb 15, 2021 at 03:40:16PM +1100, Cameron Simpson wrote:

>Aye. I use format=flowed all the time; there's stuff in the wiki 
>about settings to go with it, for example to make composition easy.

I see.  :-)  Oddly enough, your quotations appear with the leading >
without the space, while mine appear with the space, despite me 
thinking for two years (!!!) that it wouldn't if I didn't add them by hand! 
 
LOL

I've clearly mangled my composition setup.

Lemme dig into my composition setup...

Cheers,
Cameron Simpson 


Re: Quoting character when replying

2021-02-14 Thread Cameron Simpson
On 15Feb2021 05:20, Angel M Alganza  wrote:
>On Sun, Feb 14, 2021 at 07:36:34PM -0800, Kevin J. McCarthy wrote:
>>You can read a bit more about it at
>><http://www.mutt.org/doc/manual/#ff>, but you'll want to do more
>>research if you intend to compose format=flowed emails.
>
>Thank you, I'm going to have a look at this.  I'd like to keep doing it,
>since it might help others to read my email better regardless of their
>email client and display.

Aye. I use format=flowed all the time; there's stuff in the wiki about 
settings to go with it, for example to make composition easy.

The specification itself is here:

https://tools.ietf.org/rfcmarkup?doc=3676

I might stick in a PR to add that link to the manual.

It is quite conservations, to minimise the implementation burden. And 
that frugality is likely where the inflexibility about ">" comes from.

Cheers,
Cameron Simpson 


Re: Quoting character when replying

2021-02-14 Thread Cameron Simpson
On 14Feb2021 19:36, Kevin J. McCarthy  wrote:
>On Mon, Feb 15, 2021 at 03:55:35AM +0100, Angel M Alganza wrote:
>>At some point I must had done something that changed that behaviour 
>>to prepend it with just the greater sign (>) without the following 
>>space.
>
>It looks like you have enabled $text_flowed.  That overrides the value 
>of $indent_string.  You can read a bit more about it at 
><http://www.mutt.org/doc/manual/#ff>, but you'll want to do more 
>research if you intend to compose format=flowed emails.

While the format=flowed specification requires a plain ">" _in the 
message body text itself_, you might want to set 
$reflow_space_quotes=yes for presentation purposes.

Cheers,
Cameron Simpson 


Re: [unixbhas...@gmail.com: Notify-send pop up for specific mails]

2021-02-13 Thread Cameron Simpson
On 14Feb2021 06:48, Bhaskar Chowdhury  wrote:
>.but depending too much of other software makes it fragile and 
>prone to
>error and least to say it is complex.

Aye. But you asked. You can see from the opening comment of the alert 
script:

# Write a possibly-colourised string to the alert log.

that it has evolved a bit over time.

>Ordinary mortals(i.e me) who has less technical bend of mind, will run 
>away from this.

The core point here is the flow:
- getmail => folder
- mailfiler to file messages from the folder
- some messages run the alert script, which contains whatever 
  platform-specific popup incantation is required

All the rest has to do with my environment.

If you're using fetchmail+procmail, for example, you just need an 
"alert" rule at the right points in your rules.

>I do understand simplicity can not achieved without by digging and 
>getting into the rabbit
>hole...again but...how many are willing(including me, I am an truest sense lazy
>person) ...

You can use all my code of course. I'm well aware that even taking 
what's needed (==> implied, the box of tissues effect) and putting it 
into your own setup is not necessarily trivial.

>Not to deny, I do fall on fetchmail , procmail et al along with 
>mutt(the love for something do wonder) , but more to it ...nope...
>
>>The "desktop alert" part of that script is in the $to_desktop
>>if-statement at the bottom. Presently I'm on a Mac and use the
>>"terminal-notifier" command to issue a normal Mac Notification popup.
>>I'd be using whatever Linux desktop notification command line were
>>suitable were I on Linux.
>>
>notify-send is the "lightest" and "efficient" way of doing it on Linux.

Thanks. I'll look it up and stick it in. The whole point of the script 
is to remove platform "how do I alert" from everything else.

>Finally, I don't have the ability to create something native(think of 
>it as
>excuse not to put effort,if you like :) ) OR we don't want mutt to be 
>convoluted
>with some "airy-fairy" stuff.
>
>I am happy with it's limitation(partly because of my lack of understanding and
>as mentioned ability to extends it,being using it for long time 
>though).

Incremental change. Start with something basic. Modify it to better fit 
what you actually desire as its shortcomings become apparent. That's 
what I do. I've just been doing it a bit longer.

For example, your own "alert" script might just directly invoke 
notify-send and do nothing else. Mine started by writing coloured alerts 
to a text file, nothing else.

Then stick your own "alert" in your rules, and modify alert separately 
without needing to hack outside the script.

Cheers,
Cameron Simpson 


Re: [unixbhas...@gmail.com: Notify-send pop up for specific mails]

2021-02-13 Thread Cameron Simpson
On 13Feb2021 19:29, Bhaskar Chowdhury  wrote:
>On 07:40 Sat 13 Feb 2021, Cameron Simpson wrote:
>>On 12Feb2021 09:36, Kevin J. McCarthy  wrote:
>>>On Fri, Feb 12, 2021 at 09:59:38AM +0530, Bhaskar Chowdhury wrote:
>>>>The problem it pops up for every new mail arrive. [...]
>>>For more sophisticated notification, you'll need to use another tool.
>>
>>For xample, my own new mail desktop notifications happen from my mail
>>filer, not from mutt. This has the advantage that they still happen when
>>mutt is closed (if you can call that an advantage).
>>
>How?? Show us...share with the people ..

Ok

I collect my email with getmail, deliver to my local "+spool" mail 
folder, a Maildir (~/mail/spool).

I filter my messages using mailfer:

https://pypi.org/project/cs.app.mailfiler/

which monitors multiple Maildirs for new messages, and files them 
according to per-folder rules.

So it watches spool, spool-in, spool-out, spool-to-phone etc. "spool" 
winnows some spam and lets the rest through to spool-in. spool-in files 
to my inbox, various mailing list folders etc. Anything not matched 
lands in +UNKNOWN. It is mostly spam.  spool-out is what mutt's $record 
is set to - mutt saves sent mail there and mailfiler refiles it.  
spool-to-phone just has a rule to forward to my phone's email account.

So, alerts.

Mailfiler rules are usually quite simple, like this:

!me  Work  from:(ALERT)

which says to match messages from addresses in the group "alert" and do 
2 things:

- "me" save a copy in my +me folder, my "priority inbox"
- "!" run the $ALERT command with the message and save a copy to the 
  targets named in $ALERT_TARGETS

They're set like this:

ALERT=alert
ALERT_TARGETS="F,spool-to-phone"

So $ALERT runs my "alert" command, a script. Details below.
$ALERT_TARGETS says:

- "F" (F)lag the message (same as mutt's (F)lag command) so that it is 
  highlighted in mutt's index
- "spool-to-phone" save a copy in my +spool-to-phone folder, which 
  mailfiler also monitors (its rules say forward a copy to my phone's 
  email account)

The desktop popup comes from my "alert"r script:

https://hg.sr.ht/~cameron-simpson/css/browse/bin/alert?rev=tip

That will issue alerts to a variety of places depending on envvars and 
options particularly my dlog (a timestamped text log I use for reviewing 
things, since my invoicing system is a ghastly hack held together with 
string) and to the desktop.

The "desktop alert" part of that script is in the $to_desktop 
if-statement at the bottom. Presently I'm on a Mac and use the 
"terminal-notifier" command to issue a normal Mac Notification popup.  
I'd be using whatever Linux desktop notification command line were 
suitable were I on Linux.

I presume there _is_ a standard way to issue a popup alert on Linux 
systems these days? I used to just always run a permanent very short 
full-width terminal across the top of the screen tailing a log file 
myself, crude but effective. The same terminal also accepted commands if 
you typed them.

Anyway, that is how this is hooked into my email.

Suggestions for current Linux or other UNIX desktop popup command line 
tools welcomed.

Cheers,
Cameron Simpson 


Re: My experiences with Mutt to date: Suggestions for overcoming some issues

2021-02-13 Thread Cameron Simpson
On 13Feb2021 20:03, Sam Kuper  wrote:
>On Fri, Feb 12, 2021 at 01:30:04PM +0100, Angel M Alganza wrote:
>> On Mon, Feb 01, 2021 at 04:34:47PM -, Grant Edwards wrote:
>>> I maintained a local queueing MTA for many years, but after multiple
>>> screwups where mail wasn't getting sent (and I didn't find out in a
>>> timely manner) I switched to a non-queueing MTA (e.g. ssmtp) and
>>> later to mutt's SMTP support.
>>
>> I did the same myself for years and also switched to ssmtp.  But I
>> belive ssmtp was discontinued, and now I'm using msmtp:
>>
>>  https://marlam.de/msmtp/
>>
>> I couldn't be happier!
>
>A queue script is included with msmtp, so you can have the best of both
>worlds :)
>
>https://git.marlam.de/gitweb/?p=msmtp.git;a=blob_plain;f=scripts/msmtpq/README.msmtpq;hb=HEAD

Ah, I missed that. Nice.

Though I still like a local system MTA so that scripts and cron etc etc 
can also send email natively.

Cheers,
Cameron Simpson 


Re: My experiences with Mutt to date: Suggestions for overcoming some issues

2021-02-13 Thread Cameron Simpson
On 12Feb2021 23:25, boB Stepp  wrote:
>On 21/02/13 08:01AM, Cameron Simpson wrote:
>>I pull everything from my c...@cskk.id.au inbox, deleting it. But my 
>>mail
>>filing forwards a suite of messages to a separate account which is for
>>my phone. So anything important sends a copy back out to the cloud.
>>Well, a personal external server. Um, in the cloud :-(
>>
>>Likewise, any email I reply to sends the source message and the reply to
>>that account.
>>
>>In this way my phone has access to a copy of the critical stuff and also
>>any ongoing email discussions.
>
>So it seems you only concern yourself in keeping relatively current emails
>accessible to your phone?

Well I pretty much never purge it, so old stuff too.

>You haven't had occasion to need something from a
>while back to fetch on your phone that suddenly becomes needed?

Well, the phone account is meant to have "relevant" stuff rather than 
the dross comprising most email. So my filer forwards:
- everything it flags on arrival
- everything I send/reply
- the source message of anything I reply to

Cheers,
Cameron Simpson 


Re: My experiences with Mutt to date: Suggestions for overcoming some issues

2021-02-13 Thread Cameron Simpson
On 12Feb2021 23:20, boB Stepp  wrote:
>>Then tag messages in you inbox and ";s+folder" to move them in 
>>batches.
>>Here I mean mutt's (t)ag keystroke, an in memory flag. Emphemeral, used
>>entirely to do mutt things to an ad hoc batch of messages.
>
>Yeah, I just was playing around with this some earlier in the week.  I'm now
>doing this to tag and move emails into my Archive folder, which I have bound
>"S" for this purpose.  Which leads me to a question I have been meaning to
>ask.  This is what I am using right now:
>
>macro index,pager S "=Archive" "Send to
>Archive"
>
>If I don't add  then the email(s) sit there marked deleted
>(Funny that the same 'D' label is used here as for a real deletion.) until I 
>manually
>sync or it happens automatically.  I just want the mail to be _immediately_ 
>moved to my
>local Archive folder.  Is there a better, more direct way to do this without
>doing a sync operation?

The message will be copied to Archive immediately (you can check that by 
opening it in another window).  It is just the current folder where 
they're not yet removed until you sync; that lets you under the "delete" 
part of the "move" if you want.

If you wanted the sync immediate, include "" in the macro? It is a 
whole mail folder sync of course.

Personally, my (d)elete macro just archives :-)

Cheers,
Cameron Simpson 


Re: My experiences with Mutt to date: Suggestions for overcoming some issues

2021-02-13 Thread Cameron Simpson
On 12Feb2021 23:20, boB Stepp  wrote:
>Yeah, I think you have.  I have kept those earlier emails from you 
>nicely
>flagged in my inbox for reference.  Currently I am trying to see if I can get
>something I like working with notmuch.  I wish I could use notmuch's tagging
>capabilities from within Mutt in the sense of adding multiple tags to a single
>email or a set of tagged emails.  But the only example of an approach is to
>try to adapt the macro I cited earlier in this thread which deletes the inbox
>tag.  I see how to modify this to do *one* tag, but that would be hard-coded
>and not very flexible or useful.  I'm sure there must be a way to write a
>macro to allow for me to enter multiple tags for notmuch, but I don't know
>enough about Mutt macros to see a way forwards yet.

"notmuch tag" itself accepts multiple tags. Write a small shell script 
to prompt for the tags to change (in the same form as notmuch expects 
i.e. +tag and -tag) which then invokes notmuch.

If you mean this macro:

macro index  \
 "set my_old_pipe_decode=\$pipe_decode
  my_old_wait_key=\$wait_key nopipe_decode nowait_key\
   notmuch-mutt tag -- -inbox\
set pipe_decode=\$my_old_pipe_decode
 wait_key=\$my_old_wait_key" \
  "notmuch: remove message from inbox"

it:

- saves the pipe_decode and wait_key settings, and turns them off
- pipes the current message through notmuch-mutt to remove the 
  "inbox" tag
- restores the old pipe_decode and wait_key settings

The $my_blah= is a standard mutt hack because there's no "push temporary 
setting or value", and there are no settings called my_*. So people use 
$my_blah for "user variables".

Piping the whole message through notmuch lets notmuch-mutt get the 
message-id in order to know which item to tag or untag. The it passes 
"id:$mid" to notmuch, where $mid is the message id.

That's the mechanism.

To tag things interactively you have 2 issues:
- specifying the messages to tag
- specifying the tag changes

For the latter I'd have a shell script which prompted for that, then ran 
notmuch.

For the former, from inside mutt, I'd think there are 2 ways forward.  
With a single message you can just pipe it straight into the script, 
like the above macro does. With multiple messages I think I would tag 
them, and pipe all of them _separately_ through the script.

Now, there are some ways to make that more convenient, particularly 
these settings:

set auto_tag=yes
set pipe_split=yes

auto_tag=yes causes mutt to apply commands to all tagged messages (if 
any) or just the current message (if nothing tagged).

pipe_split=yes causes mutt to run separate pipes per message when it 
pipes multiple messages, instead of concatenating them and running one 
pipe. You want that for notmuch-mutt.

The notmuch-mutt script in the example above expects exactly one message 
on its input, _entirely_ to grab its message-id. So we want 
pipe_split-yes to invoke the script once for each message.

Regarding prompting, I'd write a script like this (untested):

#!/bin/sh
set -ue
echo -n "Enter tags to change (-tag, +tag): " >/dev/tty
read tags 


Re: My experiences with Mutt to date: Suggestions for overcoming some issues

2021-02-13 Thread Cameron Simpson
On 12Feb2021 22:53, boB Stepp  wrote:
>On 21/02/13 08:11AM, Cameron Simpson wrote:
>>I would guess your PC does not have a mail system installed. So cron
>>cannot deliver the cron job output by email.
>
>I was aware that a cron job could email its output, but I thought I would have
>to explicitly set that up.  Is this not the case?  In any event I do not know
>enough at this time to do this, so I haven't done so.  But if your hypothesis
>is correct and it is trying to email me, but finding no MTA to enable it to do
>so, why don't I *always* get this message?

Cron only sends email when the job's output is not empty. Silent jobs do 
not try to send email. Is that plausible?

>>One of the benefits of have a local email system is getting stuff from
>>cron et al. Also, you can use it for spooling - if mutt sends via the
>>local email system you can send when offline - it will just queue.
>>
>>>One concern based on earlier discussion in this thread.  I am now using
>>>msmtp as my MTA client.  What will happen if I send an email when, for
>>>whatever reason, Gmail connectivity is broken?  Will it get resent?
>>
>>I don't know. What does its manual say?
[...]
>reading the past few days!  My head is spinning chock full of half-understood
>information about multiple email software and topics!!
[...]
>As far as I have been able to tell from the man pages msmtp will attempt to
>send me a notification email in the event of delivery failure.  AFAICT, there
>is no explicit mention of msmtp trying to resend the email.  Also it is not
>clear to me if delivery failure includes not being able to connect to the
>Internet.  It did say it uses standard exit codes, but I did not see how it
>would act in this instance.

Looks to me like it does not queue:

% (echo To: c...@cskk.id.au; echo From: cs; echo Subject: msmtp 
test;echo;echo 1) | msmtp --host=bogushost.example.com --from=c...@cskk.id.au 
-t c...@cskk.id.au
msmtp: cannot locate host bogushost.example.com: nodename nor 
servname provided, or not known
msmtp: could not send mail

Caveat: I have no msmtp config - that's a bare command line test.

There's a package called "nullmailer" which can be used as your system 
MTA - it is intended for systems with a smarthost - upstream SMTP 
server, exactly what you msmtp setup will be using. But it has a queue.

You could install it without changing your mutt/msmtp setup at all. If 
it works for you, you could then switch to having mutt use the local 
sendmail command (the default) instead of msmtp.

Looks like its config lives in /usr/local/etc/nullmailer.

Source: https://github.com/bruceg/nullmailer

Install: https://github.com/bruceg/nullmailer/blob/master/INSTALL

Man page for nullmailer-send, which describes the config settings: 
https://github.com/bruceg/nullmailer/blob/master/doc/nullmailer-send.8

Disclaimer: I've not used it.

Cheers,
Cameron Simpson 


  1   2   3   4   5   6   7   8   >