Re: Are there any good maildir manipulation utilities out there?

[Kurt Hackenberg]

On 2021/04/08 03:34, Chris Green wrote:


fdm?


...

Ah, I maybe wasn't clear, I want to manage E-Mails which have been
delivered already.  All the maildir messages I want to move around are
already in ~/mail as maildirs.  I want to rearrange them.



I understand that. Fdm can do that. However, looks like you've found 
another way.

Re: Are there any good maildir manipulation utilities out there?

[Cameron Simpson]

On 08Apr2021 08:40, Chris Green  wrote:
>On Thu, Apr 08, 2021 at 08:43:48AM +1000, Cameron Simpson wrote:
>> It's also not particularly well suited
>> to Chris' requirement, which includes preserving the source tree shape
>> in the archiving process.
>>
>Yes, that was just one typical requirement.  The other major
>requirement is rather different.  I have (for example) lots of
>messages about orders from Screwfix which are currently spread around
>in various sub-directories of 'shopping'.  For suppliers like Screwfix
>from whom I buy a lot of rather mixed up sort of items it now makes
>sense to me to keep them all in a Screwfix directory.  So the
>requirement is to find all messages from Screwfix and put them in one
>directory.

Note that there's no fundamental reason you can't link the same message 
into many Maildirs. So link them all into a screwfix folder, and also 
where they naturally land courtesy of other filing rules.

Also, didn't you put some work into tagging messages. A search on 
tag=screwfix might go well.

>> I suspect Chris may need to roll his own. I'd imagine something like:
>>
>> find message paths using mairix \
>> | move message files sideways, making sure there's no conflicts
>>
>Yes, I think it may have to be a roll my own something like this.
>It's just that mairix doesn't provide a very good 'handle' to use.

I'm using notmuch. In particular, this:

notmuch search --output=files -- search terms ...

emits message pathnames. That gets you the folder structure with alittle 
path fiddling. So my "notmuch-search" script (aliased as "++") goes:

if [ $dothreads ]
then
  notmuch search --output=threads -- "$@" \
  | while read -r tid
do  notmuch search --output=files -- "$tid"
done
else
  notmuch search --output=files -- "$@"
fi \
| egrep '/(new|cur)/[^/]+$' \
| sort -u \
| xxargs arg1 -end "$mdir/new" set-x ln -i -s \

at the bottom:
- find the messages
- winnow some noise - probably unnecessary
- "sort -u" to remove duplicates
- make symlinks to all the messages in the search result maildir's "new" 
  subdir

Then it fires up mutt on the result maildir.

I would think you could shoehorn the above logic a bit to achieve your 
"keep the hierachy shape" thing.

Cheers,
Cameron Simpson 

Re: HTML email?

[Cameron Simpson]

On 07Apr2021 07:22, John Niendorf  wrote:
>How do you all deal with HTML email?

Composing HTML is a can of worms. I'd need to dig into the list archives 
- it has been discussed.

Displaying HTML uses 2 main settings:

The .mailcap entry for text/html with the "copiousoutput" flag. For 
example:

text/html; exec 2>&1 && env DISPLAY= unhtml %s; copiousoutput

"unhtml" is a personal script which invokes whatever I prefer to use to 
transcribe HTML as plain text. Currently it invokes:

lynx -stdin -dump

That way I don't have to hack my mailcap much, better to hack the script 
if I shift tools, eg to w3m.

The other setting is the alternative_order setting, which says which 
Content-Type to prefer of a multipart/alternative message. These usually 
have a text/plain and text/html part (though of course they course have 
other things, eg a text/markdown part). My default setting is:

alternative_order text/plain text/html

which prefers the plain text version, sidestepping the HTML altogether.

However, there are plenty of platforms which are HTML first and provide 
either very poor plaintext equivalents of empty ones, or ever just stuff 
the raw HTML into both parts. Absolutely rubbish quality of 
implementation, but there you go.

So in fact I choose the alternative order per message:

# alternative-order criteria
message-hook . 'unalternative_order *; alternative_order text/plain 
text/html'

message-hook '~h "X-Mailer: Apple Mail" ~X 1-' 'unalternative_order *; 
alternative_order text/html multipart/mixed text/plain'

message-hook '%f htmlers | ~f @no-re...@cc.yahoo-inc.com | ~f @outlook.com 
| ~f live.com | ~f @facebookmail.com' 'unalternative_order *; alternative_order 
text/html text/plain'

So far all messages I set up the default. Then for Apple Mail I put the 
HTML first because of the way Apple Mail packs attachments, which is 
weird. Then for an elite set of negligent idiots I put HTML first 
because I _know_ that they shift a plaintext version and the plaintext 
is always rubbish.

That last criterion is email from outlook.com, live.com, facebook.com, 
yahoo's PR/info people, and whomever I have explicitly added to my mutt 
"htmlers" group.

Cheers,
Cameron Simpson 

Re: Is there any way to customise sorting of directories in the index?

[Chris Green]

On Thu, Apr 08, 2021 at 11:31:34AM +0100, Chris Green wrote:
> On Thu, Apr 08, 2021 at 11:23:26AM +0100, Chris Green wrote:
> > I have a about 40 or so folders where I save mails I want to keep,
> > with names such as 'shopping', 'motorcycle' and so on.  By default
> > these are sorted alphabetically in the index which is not ideal.  I
> > save things in 'shopping' far more frequently than elsewhere so it
> > would be really handy if that was at the top or at least near the top. 
> > 
> > Is there any way to customise the sort order?  I suppose the ideal
> > would be 'most recently used' at the top but even reverse alphabetical
> > would improve things.
> > 
> > I *could* rename 'shopping' to 'Shopping' to fix that one but I'd
> > really like a more generally applicable solution.
> > 
> Ah, oops, I've just discovered the sort_browser setting which
> addresses this issue to some extent.  However some explanation of the
> settings might be useful:-
> 
> count - does this mean the count of messages in the directory,
> what about if it contains further subdirectories?
> 
> date - date of what?  If it's just the date of the directory it's
> not terribly useful!  
> 
> size - does it work or mean anything with maildir?
> 
> unread - what does this mean?
> 
Sorry, talking to myself here so much!

Anything other than alphabatical sorting is pretty useless with a long
list of directories, especially as it often puts .. somewhere other
than at the top.  If you have many directories at one level (my
top level directory has 32) finding anything in it other than by an
alphabetical sort is painful!  (I tried 'date' to see what it did).

I think my solution of changing shopping to Shopping may be the most
sensible one.

-- 
Chris Green

Re: Are there any good maildir manipulation utilities out there?

[Chris Green]

On Thu, Apr 08, 2021 at 09:55:52AM -0400, Dan Ritter wrote:
> Chris Green wrote: 
> > On Thu, Apr 08, 2021 at 08:43:48AM +1000, Cameron Simpson wrote:
> > > On 07Apr2021 18:34, Kurt Hackenberg  wrote:
> > > >On 2021/04/07 18:16, Kurt Hackenberg wrote:
> > > >>On 2021/04/07 17:01, Chris Green wrote:
> > > >>>I'm looking for a tool which will allow me to search through a large
> > > >>>hierarchy of maildir messages and then provide actions to take on the
> > > >>>matched messages.
> 
> ...
> 
> 
> > Yes, that was just one typical requirement.  The other major
> > requirement is rather different.  I have (for example) lots of
> > messages about orders from Screwfix which are currently spread around
> > in various sub-directories of 'shopping'.  For suppliers like Screwfix
> > from whom I buy a lot of rather mixed up sort of items it now makes
> > sense to me to keep them all in a Screwfix directory.  So the
> > requirement is to find all messages from Screwfix and put them in one
> > directory.
> > 
> > 
> > > I suspect Chris may need to roll his own. I'd imagine something like:
> > > 
> > > find message paths using mairix \
> > > | move message files sideways, making sure there's no conflicts
> > > 
> > Yes, I think it may have to be a roll my own something like this.
> > It's just that mairix doesn't provide a very good 'handle' to use.
> 
> Have you added a mairix search => maildir integration in your
> muttrc?
> 
> The Screwfix case would be:
> 
> macro index  "set my_cmd = 
> \`/usr/local/bin/mairixsearch\`push 
> \$my_cmd" "search messages" 
> 
> macro index  "~/results." 
> "display search results" 
> 
> F8 f screwfix
> F9
> # tag or untag or whatever until you've got the right ones
> ;s =screwfix
> 
> Would that work for you?
> 
Yes, on digging a bit further into this using mairix isn't too
difficult:-

Search with mairix producing a maildir full of symbolic links to
the hits.

Move the *target* of the symbolic links to the new directory as
required.

Delete the target of the symbolic links.

Job done!  Since there are already readlink and/or readpath commands
for finding the target of a symbolic link this is rather easy.

-- 
Chris Green

Re: Are there any good maildir manipulation utilities out there?

[Mark H. Wood]

On Wed, Apr 07, 2021 at 10:01:22PM +0100, Chris Green wrote:
> I'm looking for a tool which will allow me to search through a large
> hierarchy of maildir messages and then provide actions to take on the
> matched messages.
> 
> E.g. I might want to archive all messages older than a certain date
> but archive them into a similar hierarchy rather than to just a single
> archive directory.
> 
> Or I might want to search for all messages from a certain person (or
> business) and move them into a single folder.
> 
> I already use mairix for searching for messages but it doesn't offer
> the actions function I'm after.  

Have you looked at 'mu':

https://www.djcbsoftware.nl/code/mu/

-- 
Mark H. Wood
Lead Technology Analyst

University Library
Indiana University - Purdue University Indianapolis
755 W. Michigan Street
Indianapolis, IN 46202
317-274-0749
www.ulib.iupui.edu


signature.asc
Description: PGP signature

Re: Are there any good maildir manipulation utilities out there?

[Dan Ritter]

Chris Green wrote: 
> On Thu, Apr 08, 2021 at 08:43:48AM +1000, Cameron Simpson wrote:
> > On 07Apr2021 18:34, Kurt Hackenberg  wrote:
> > >On 2021/04/07 18:16, Kurt Hackenberg wrote:
> > >>On 2021/04/07 17:01, Chris Green wrote:
> > >>>I'm looking for a tool which will allow me to search through a large
> > >>>hierarchy of maildir messages and then provide actions to take on the
> > >>>matched messages.

...


> Yes, that was just one typical requirement.  The other major
> requirement is rather different.  I have (for example) lots of
> messages about orders from Screwfix which are currently spread around
> in various sub-directories of 'shopping'.  For suppliers like Screwfix
> from whom I buy a lot of rather mixed up sort of items it now makes
> sense to me to keep them all in a Screwfix directory.  So the
> requirement is to find all messages from Screwfix and put them in one
> directory.
> 
> 
> > I suspect Chris may need to roll his own. I'd imagine something like:
> > 
> > find message paths using mairix \
> > | move message files sideways, making sure there's no conflicts
> > 
> Yes, I think it may have to be a roll my own something like this.
> It's just that mairix doesn't provide a very good 'handle' to use.

Have you added a mairix search => maildir integration in your
muttrc?

The Screwfix case would be:

macro index  "set my_cmd = 
\`/usr/local/bin/mairixsearch\`push \$my_cmd" 
"search messages"

macro index  "~/results." 
"display search results"

F8 f screwfix
F9
# tag or untag or whatever until you've got the right ones
;s =screwfix

Would that work for you?

-dsr-

Re: HTML email?

[Dan Ciprus (dciprus)]

And I will open a can with worms: 

"deal" in what sense ? Just reading the html email or are you talking about 
replying to the HTML email with HTML email or maybe you're talking about keeping 
HTML replies intact and replying with plain txt ? There are several use cases 
which have workarounds all over the place. This mailing list has listed several 
of them in the past but unfortunately they're all just a workarounds not a real 
solutions to this multimedia nonsense which was brought to email world. 

Apologies for not replying with any suggestions, the only suggestion I have: 
look up the archive of this mailing list and you'll find plenty of ideas related 
to your question. 


Dan

On Wed, Apr 07, 2021 at 07:22:51AM -0600, John Niendorf wrote:

Hi Folks,

How do you all deal with HTML email?

Thank you,

John


--
Daniel Ciprus  .:|:.:|:.
CONSULTING ENGINEER.CUSTOMER DELIVERY   Cisco Systems Inc.
dcip...@cisco.com
tel: +1-703-484-0205
mob: +1-540-223-7098



signature.asc
Description: PGP signature

Re: Is there any way to customise sorting of directories in the index?

[Chris Green]

On Thu, Apr 08, 2021 at 11:23:26AM +0100, Chris Green wrote:
> I have a about 40 or so folders where I save mails I want to keep,
> with names such as 'shopping', 'motorcycle' and so on.  By default
> these are sorted alphabetically in the index which is not ideal.  I
> save things in 'shopping' far more frequently than elsewhere so it
> would be really handy if that was at the top or at least near the top. 
> 
> Is there any way to customise the sort order?  I suppose the ideal
> would be 'most recently used' at the top but even reverse alphabetical
> would improve things.
> 
> I *could* rename 'shopping' to 'Shopping' to fix that one but I'd
> really like a more generally applicable solution.
> 
Ah, oops, I've just discovered the sort_browser setting which
addresses this issue to some extent.  However some explanation of the
settings might be useful:-

count - does this mean the count of messages in the directory,
what about if it contains further subdirectories?

date - date of what?  If it's just the date of the directory it's
not terribly useful!  

size - does it work or mean anything with maildir?

unread - what does this mean?

-- 
Chris Green

Is there any way to customise sorting of directories in the index?

[Chris Green]

I have a about 40 or so folders where I save mails I want to keep,
with names such as 'shopping', 'motorcycle' and so on.  By default
these are sorted alphabetically in the index which is not ideal.  I
save things in 'shopping' far more frequently than elsewhere so it
would be really handy if that was at the top or at least near the top. 

Is there any way to customise the sort order?  I suppose the ideal
would be 'most recently used' at the top but even reverse alphabetical
would improve things.

I *could* rename 'shopping' to 'Shopping' to fix that one but I'd
really like a more generally applicable solution.

-- 
Chris Green

Re: HTML email?

[Rene Kita]

On Wed, Apr 07, 2021 at 07:22:51AM -0600, John Niendorf wrote:
> Hi Folks,
> 
> How do you all deal with HTML email?
I delete them.

Re: HTML email?

[lilydjwg]

On Wed, Apr 07, 2021 at 08:00:38AM -0600, John Niendorf wrote:
> Thank you - where do you put the python3 script and how do you let mutt know 
> it is there?

Put it somewhere in $PATH, or you can use an absolution path.

-- 
Best regards,
lilydjwg

Re: Are there any good maildir manipulation utilities out there?

[Chris Green]

On Wed, Apr 07, 2021 at 08:16:32PM -0700, N.J. Thomas wrote:
> * Chris Green  [2021-04-07 22:01:22+0100]:
> > I'm looking for a tool which will allow me to search through a large
> > hierarchy of maildir messages and then provide actions to take on the
> > matched messages.
> 
> You could do this pretty easily with Python, using the mailbox library.
> 
Yes, I guess so.  I already have a Python script (run from .forward)
which routes my incoming messages into appropriate places so I am
a bit familiar with Python's mail handling.

It just seems a pity not to use mairix (or other) to do the searching
bit.

-- 
Chris Green

Re: Are there any good maildir manipulation utilities out there?

[Chris Green]

On Thu, Apr 08, 2021 at 08:43:48AM +1000, Cameron Simpson wrote:
> On 07Apr2021 18:34, Kurt Hackenberg  wrote:
> >On 2021/04/07 18:16, Kurt Hackenberg wrote:
> >>On 2021/04/07 17:01, Chris Green wrote:
> >>>I'm looking for a tool which will allow me to search through a large
> >>>hierarchy of maildir messages and then provide actions to take on the
> >>>matched messages.
> >>
> >>
> >>fdm?
> >>
> >>
> >>Or some other filter-and-deliver program -- but not procmail, 
> >>because it's a zombie, abandoned by its last maintainers, last 
> >>release 20 years ago.
> >
> >
> >Maybe Cameron Simpson's filer, if he publishes it.
> 
> It's got some dependencies on some stuff I need to weed out before it is 
> easy for someone-not-me to use. It's also not particularly well suited 
> to Chris' requirement, which includes preserving the source tree shape 
> in the archiving process.
> 
Yes, that was just one typical requirement.  The other major
requirement is rather different.  I have (for example) lots of
messages about orders from Screwfix which are currently spread around
in various sub-directories of 'shopping'.  For suppliers like Screwfix
from whom I buy a lot of rather mixed up sort of items it now makes
sense to me to keep them all in a Screwfix directory.  So the
requirement is to find all messages from Screwfix and put them in one
directory.


> I suspect Chris may need to roll his own. I'd imagine something like:
> 
> find message paths using mairix \
> | move message files sideways, making sure there's no conflicts
> 
Yes, I think it may have to be a roll my own something like this.
It's just that mairix doesn't provide a very good 'handle' to use.

-- 
Chris Green

Re: Are there any good maildir manipulation utilities out there?

[Chris Green]

On Wed, Apr 07, 2021 at 06:16:31PM -0400, Kurt Hackenberg wrote:
> On 2021/04/07 17:01, Chris Green wrote:
> 
> > I'm looking for a tool which will allow me to search through a large
> > hierarchy of maildir messages and then provide actions to take on the
> > matched messages.
> 
> 
> fdm?
> 
> 
> Or some other filter-and-deliver program -- but not procmail, because it's a
> zombie, abandoned by its last maintainers, last release 20 years ago.

Ah, I maybe wasn't clear, I want to manage E-Mails which have been
delivered already.  All the maildir messages I want to move around are
already in ~/mail as maildirs.  I want to rearrange them.

-- 
Chris Green