On 2018-02-23 (16:47 MST), Joseph Tam <jtam.h...@gmail.com> wrote: > > On Fri, 23 Feb 2018, @lbutlr wrote: > >> $ doveadm -f table fetch -u kremels 'hdr.message-id guid uid >> hdr.x-listname' mailbox "Archive" | sort| awk 'cnt[$1]++{if >> (cnt[$1]==2) print prev[$1]; print} {prev[$1]=$0}' |grep -E "[0-9] +$" >> |awk '{print "doveadm expunge -u kremels MAILBOX-GUID "$2" UID "$3}? > > I was unaware of the syntax "hdr.{header}" -- all the reference materials > I've seen only refers to "hdr" which returns the entire header block.
the error message from doveadm if you specify an invalid field is: Available fetch fields: hdr.<name> body.<section> binary.<section> user mailbox mailbox-guid seq uid guid flags modseq hdr body body.snippet text text.utf8 size.physical size.virtual date.received date.sent date.saved date.received.unixtime date.sent.unixtime date.saved.unixtime imap.envelope imap.body imap.bodystructure pop3.uidl pop3.order refcount storageid >> First, even after expunging a message and running doveadm index -u >> kremels ?Archive?, subsequent runs still show the same duplicate >> messages. > > I suspect client side caching. No, there is no client side involved. I am executing all of these these commands on the mail server. I expunge the messages, I index (or even force-resync) and the next search shows the same messages even though they are not in the Maildir anymore. > If {count} is what you expected, then dovecot has the correct information > and it's likely some client-side caching issue. I would have needed to check the count before doing this, and I did not. >> Second, what I really want to do is run this over ALL the mailboxes, >> except for Junk and Sent but if that is possible I can?t find the right >> syntax. > > You mean to remove duplicates from any 2 mailboxes, or remove duplicates > in mailboxes also found in Archive? I want to find any duplicates (based on msg ID) across all mailboxes, except Sent > doveadm -f table fetch -u kremels \ > 'hdr.message-id guid uid' \ > NOT mailbox Archive \ > NOT mailbox Junk \ > NOT mailbox Sent \ > | sort -b >list1 Aha! Didn't know you could use NOT mailbox. That probably solves my issue on that score. -- "It's unacceptable to think" - George W Bush 15/Sep/2006