Re: [OT-ish] Want command line program to selectively delete emails
On Mon, Aug 07, 2023 at 01:57:04AM +1000, Nemo Thorx wrote: > Quoting Ed Blackman from 04 Aug (a Friday in 2023) at 1434 hours... > > Any suggestions for a command line program to select emails for > > deletion based on command line options? I specifically want one that > > can remove emails that were received more than X days ago, but can > > also express "but don't delete if they're flagged". > > > > > I could call mutt (see, kinda OT!) with "-e" to tag, delete, sync, and > > quit, but that seems fiddly and heavyweight. > > But if it's being run from cron overnight or something, are you ever > going to notice? > > > I could give up on using the message headers to determine the message > > date, learn how filenames are constructed in Maildir (to read the > > flags) and use a shell script built around "find". > > I'm reasonably confident that all filenames (before the ':') are only > convention (I convert an nntp spool to maildir and name them according > to my own naming convention which everything (well, mutt and dovecot) > then honours), and mtime is unused by anything and/or not to be trusted. > > So honetsly, I'd suggest mutt's -e, since it gives you mutt's pattern > matching which is much more powerful than archivemail (admittedly, > I only used that for trivial stuff and only many years ago) and likely > more than anything else available. > > eg, instead of just "received more than X days ago and not flagged", > mutt can let you select/ignore whole threads based on those parameters, > and others alike. eg, I once had archiving setup on a folder-hook thus: > > folder-hook !(Maildir$|archive|search|mfolder|.r.) 'push > "t~m 600- !~(~m -600) !~(~r <32d) !~(~N) !~(~F) > !~D^.archivet"' > > To decode that: sort reverse threaded, select all messages after the > first 600, but not if it's part of a thread within the first 600, not if > it's in a thread with messages received in the last 32 days, not if it's > in a thread with messages marked 'New' or 'Flagged', and not if it's > a message already marked delete. Then save to the archive mailbox and > established my desired sorting and thread collapse visibility. > > Basically - I wanted a thread-preserving archiving mechanism, and mutt's > thread-aware pattern matching gave it to me. (I didn't include sync in > that setup, preferring at the time to let that occur with mailbox > closing) > > In a delete scenario as you're seeking, I'd suggest a multi-step process > which first archives threads you've contributed to - and only then > deleting old ignorable threads. > > Sure It's fiddly - which is your argument against using mutt. > > ...But it's powerful - which is my argument for using mutt! > > Aside - I used folder-hook before because I didn't know of the "-e" > option, and I stopped using it because I found I was often using mutt as > a mailbox debug tool, and this was just getting in the way. I can see > myself going down the rabbithole and reimplementing my old archiver now > (where "now" is "sometime in the next few months, maybe" :) Another very late reply to this thread, but I happen to have implemented almost exactly the concept you're describing. It's a python3 script that deletes whole threads that meet the criteria: - All messages are markes as read - No messages in the thread are unread - No messages in the thread are flagged (starred) - All messages in the thread are older than N days It's been several years since I've looked at this code (which in my mind is a good thing, since that means it hasn't broken), so definitely give it a read-over to see if it's suitable for your purposes. In my case, I run it directly on the mail server via a systemd service and timer. https://seangreenslade.com/source-code/scripts/list-thread-pruner/listthreadpruner-v0.1.py Happy to answer any questions about the code if anyone is curious or wants to use it. --Sean
Re: [OT-ish] Want command line program to selectively delete emails
Answering late: * Ed Blackman , 2023-08-04 14:34:44 Fri: > Any suggestions for a command line program to select emails for > deletion based on command line options? I specifically want one > that can remove emails that were received more than X days ago, but > can also express "but don't delete if they're flagged". [...] > > When I recently upgraded to Debian 12, I found that archivemail has > been removed from Debian because it's an unmaintained Python 2 > program. I went through the same realization some time ago. Checking my configuration, I see that I have replaced archivemail with mail-expire: https://manpages.debian.org/bookworm/mail-expire/mail-expire.1.en.html It does not seem to support your requirement to avoid deleting flagged messages, sadly, but this might be useful to others looking to move off archivemail. -- David Haguenauer
Re: [OT-ish] Want command line program to selectively delete emails
Would 'mu find' and 'mu remove' from mu/maildir-utils help? You'd have to do a bit of scripting around them. -- Mark H. Wood Lead Technology Analyst University Library Indiana University - Purdue University Indianapolis 755 W. Michigan Street Indianapolis, IN 46202 317-274-0749 www.ulib.iupui.edu signature.asc Description: PGP signature
Re: [OT-ish] Want command line program to selectively delete emails
On Sun, Aug 06, 2023 at 06:41:30PM +0100, Darac Marjal wrote: > On 04/08/2023 19:34, Ed Blackman wrote: > > I could probably port archivemail to Python3 with enough time. > > I had a go at rewriting archivemail into python3 (partly because I wanted to > pipe messages into spam learning as I archived things), but I've now settled Any chance you still have it? > on using imapfilter (https://github.com/lefcha/imapfilter) as a good > toolkit. You can basically use it as a Lua library for IMAP handling - that > is, imapfilter will do the heavy lifting of connecting, authenticating, > listing etc, you just need to provide it a filter file which tells it what > to do. (For example, you can say "Get all the messages in box 'Foo' and, for > each one if the received date is greater than X days and AND the message is > not flagged, then delete this message"). I'd prefer something that operated on a local mail store, but I am running Dovecot so I can access my mail folders from my phone Hmmm... -- Ed Blackman
Re: [OT-ish] Want command line program to selectively delete emails
On Mon, Aug 07, 2023 at 01:57:04AM +1000, Nemo Thorx wrote: > Quoting Ed Blackman from 04 Aug (a Friday in 2023) at 1434 hours... > > Any suggestions for a command line program to select emails for > > deletion based on command line options? I specifically want one that > > can remove emails that were received more than X days ago, but can > > also express "but don't delete if they're flagged". > > > > > I could call mutt (see, kinda OT!) with "-e" to tag, delete, sync, and > > quit, but that seems fiddly and heavyweight. > > But if it's being run from cron overnight or something, are you ever > going to notice? Fair enough for "heavyweight". I've saved your message in case I change my mind. -- Ed Blackman
Re: [OT-ish] Want command line program to selectively delete emails
On Fri, Aug 04, 2023 at 03:15:41PM -0400, Jon LaBadie wrote: > On Fri, Aug 04, 2023 at 02:34:44PM -0400, Ed Blackman wrote: > > I could probably port archivemail to Python3 with enough time. > > I make extensive, but very basic, use of archivemail. I'm concerned about > its going away. Does anyone know of an equivalent, maintained, > program? Shame if it is dropped. Though the last update was what, > 2011 I think, it has caused no problems, it just works. The issue is that the Python3 email API is much different from the Python2 API. Knowing that there's at least one other archivemail user that might be interested in a Python3 port, I've put some work in. Do *not* get your hopes up. After about 5 hours, I'm down to 36 errors and failures in the 99 item test suite, having started from about 68 errors and failures after a quick Py2->Py3 syntax port. I've also noted the absence of a lot of test cases. I'm not close enough for coverage to mean anything, but I'm guessing the test suite covered less than half the lines of the existing code. For now I'll keep working on it as I have time and interest, but I might drop it if I find something better. -- Ed Blackman
Re: [OT-ish] Want command line program to selectively delete emails
On 04/08/2023 19:34, Ed Blackman wrote: Any suggestions for a command line program to select emails for deletion based on command line options? I specifically want one that can remove emails that were received more than X days ago, but can also express "but don't delete if they're flagged". I had been using archivemail (https://manpages.debian.org/buster/archivemail/archivemail.1.en.html) from cron to delete emails in certain folders. It saves me from having to manually go in and purge folders for lists that I want to have available, but don't ever read old messages. It allows commands like: archivemail --quiet --delete --days=30 ~/mail/folder1/ archivemail --quiet --delete --days=30 --include-flagged ~/mail/folder2/ It supports reading Maildir, IMAP, mbox, MH, but the mailboxes I need to purge are all maildir. When I recently upgraded to Debian 12, I found that archivemail has been removed from Debian because it's an unmaintained Python 2 program. I could probably port archivemail to Python3 with enough time. I could compile Python2 from source, but don't want to. I could call mutt (see, kinda OT!) with "-e" to tag, delete, sync, and quit, but that seems fiddly and heavyweight. I could extract the interesting parts of archivemail into my own script (most of the script is about *archiving* email, which I'm not interested in). I could give up on using the message headers to determine the message date, learn how filenames are constructed in Maildir (to read the flags) and use a shell script built around "find". None of those options are great, so I thought I'd see if there's something that I could just install. I had a go at rewriting archivemail into python3 (partly because I wanted to pipe messages into spam learning as I archived things), but I've now settled on using imapfilter (https://github.com/lefcha/imapfilter) as a good toolkit. You can basically use it as a Lua library for IMAP handling - that is, imapfilter will do the heavy lifting of connecting, authenticating, listing etc, you just need to provide it a filter file which tells it what to do. (For example, you can say "Get all the messages in box 'Foo' and, for each one if the received date is greater than X days and AND the message is not flagged, then delete this message"). OpenPGP_signature Description: OpenPGP digital signature
Re: [OT-ish] Want command line program to selectively delete emails
Quoting Ed Blackman from 04 Aug (a Friday in 2023) at 1434 hours... > Any suggestions for a command line program to select emails for > deletion based on command line options? I specifically want one that > can remove emails that were received more than X days ago, but can > also express "but don't delete if they're flagged". > I could call mutt (see, kinda OT!) with "-e" to tag, delete, sync, and > quit, but that seems fiddly and heavyweight. But if it's being run from cron overnight or something, are you ever going to notice? > I could give up on using the message headers to determine the message > date, learn how filenames are constructed in Maildir (to read the > flags) and use a shell script built around "find". I'm reasonably confident that all filenames (before the ':') are only convention (I convert an nntp spool to maildir and name them according to my own naming convention which everything (well, mutt and dovecot) then honours), and mtime is unused by anything and/or not to be trusted. So honetsly, I'd suggest mutt's -e, since it gives you mutt's pattern matching which is much more powerful than archivemail (admittedly, I only used that for trivial stuff and only many years ago) and likely more than anything else available. eg, instead of just "received more than X days ago and not flagged", mutt can let you select/ignore whole threads based on those parameters, and others alike. eg, I once had archiving setup on a folder-hook thus: folder-hook !(Maildir$|archive|search|mfolder|.r.) 'push "t~m 600- !~(~m -600) !~(~r <32d) !~(~N) !~(~F) !~D^.archivet"' To decode that: sort reverse threaded, select all messages after the first 600, but not if it's part of a thread within the first 600, not if it's in a thread with messages received in the last 32 days, not if it's in a thread with messages marked 'New' or 'Flagged', and not if it's a message already marked delete. Then save to the archive mailbox and established my desired sorting and thread collapse visibility. Basically - I wanted a thread-preserving archiving mechanism, and mutt's thread-aware pattern matching gave it to me. (I didn't include sync in that setup, preferring at the time to let that occur with mailbox closing) In a delete scenario as you're seeking, I'd suggest a multi-step process which first archives threads you've contributed to - and only then deleting old ignorable threads. Sure It's fiddly - which is your argument against using mutt. ...But it's powerful - which is my argument for using mutt! Aside - I used folder-hook before because I didn't know of the "-e" option, and I stopped using it because I found I was often using mutt as a mailbox debug tool, and this was just getting in the way. I can see myself going down the rabbithole and reimplementing my old archiver now (where "now" is "sometime in the next few months, maybe" :) .../Nemo -- - - earth native
Re: [OT-ish] Want command line program to selectively delete emails
Ed Blackman wrote (Fri 2023-Aug-04 14:34:44 -0400): > ... remove emails that were received more than X days ago, but can also > express "but don't delete if they're flagged". .. > I could give up on using the message headers to determine the message date, > learn how filenames are constructed in Maildir (to read the flags) and use a > shell script built around "find". That's what I'd probably do. It might be simpler than expected: find ~/mail/folder1/cur/ -maxdepth 1 -type f -not -name '.*' \ -mtime +30 \ -not -name '*:2,*F*' \ -delete The conditions in the first line are generic, the other three lines should match your example: received more than 30 full days ago && not flagged => delete. If you also want to delete mails that "have not yet been seen by any mail application" [1], let "find" also look at new/. Entries in new/ never contain an info part (the stuff after the colon), so you can drop the check for an unset "F" flag if you need to invoke "find" separately for new/ anyway. [1] https://en.wikipedia.org/wiki/Maildir Cheers, Marcus -- Marcus C. Gottwald ·· @mcg:cheers.de
Re: [OT-ish] Want command line program to selectively delete emails
On Fri, Aug 04, 2023 at 02:34:44PM -0400, Ed Blackman wrote: Any suggestions for a command line program to select emails for deletion based on command line options? I specifically want one that can remove emails that were received more than X days ago, but can also express "but don't delete if they're flagged". I had been using archivemail (https://manpages.debian.org/buster/archivemail/archivemail.1.en.html) from cron to delete emails in certain folders. It saves me from having to manually go in and purge folders for lists that I want to have available, but don't ever read old messages. It allows commands like: archivemail --quiet --delete --days=30 ~/mail/folder1/ archivemail --quiet --delete --days=30 --include-flagged ~/mail/folder2/ It supports reading Maildir, IMAP, mbox, MH, but the mailboxes I need to purge are all maildir. When I recently upgraded to Debian 12, I found that archivemail has been removed from Debian because it's an unmaintained Python 2 program. I could probably port archivemail to Python3 with enough time. I make extensive, but very basic, use of archivemail. I'm concerned about its going away. Does anyone know of an equivalent, maintained, program? Shame if it is dropped. Though the last update was what, 2011 I think, it has caused no problems, it just works. Jon -- Jon H. LaBadie j...@labadie.us 154 Milkweed Dr (540) 868-8052 (H) Lake Frederick, VA 22630(703) 935-6720 (M)
