On Wed, Jun 29, 2022 at 10:30 AM Eric Wong <[email protected]> wrote:
>
> Rob Herring <[email protected]> wrote:
> > Hi,
> >
> > I'm using lei with lore where I have 2 queries which overlap. Really,
> > one is a subset of the other. On those overlapping threads, I'm
> > finding that sometimes new messages are written to one mailbox and not
> > the other. (At least sometimes, the messages may be missing from all
> > mailboxes sometimes too. I'm not certain.) Using --remote-fudge-time
> > to force refetching seems to get the missing mails. I haven't found
> > anything strange in timestamps of the missing mails, but otherwise am
> > not sure how to debug this further. The queries are retrieving full
> > threads and the missing mails are in the threads, but not direct
> > matches to the queries. I realize that's not a lot of detail to go on.
> > Suggestions on debugging this further?
>
> Is this with 1.8 or 1.7?
Commit 68b53c888911 actually. So post 1.8.
> I forgot to note in the release notes, but there were some
> SQLite usage-related fixes which could avoid missing messages.
>
> You'll need "lei daemon-kill" after upgrading to 1.8 to ensure
> the new code is running.
It's possible I haven't done that since updating though I do vaguely
recall seeing something about needing to do that. Is there any way to
tell before I restart it?
> What might be interesting is to use the URLs lei prints and
> comparing the results w/o lei.
>
> I'll have to double-check if overlapping affects things, but it
> shouldn't; since the dedupe logic is per-output.
>
> Is this exclusively with HTTPS endpoints and writing to Maildirs
> (or something else?)
Yes. It's querying lore and writing to a maildir. Here's one of the queries:
[lei]
q = (dfn:drivers OR dfn:arch OR dfn:Documentation/* OR
dfn:include OR dfn:scripts) AND \
f:[email protected] AND rt:6.month.ago..
[lei "q"]
include = https://lore.kernel.org/all/
external = 1
local = 1
remote = 1
threads = 1
dedupe = mid
output = maildir:/home/rob/Mail/my-patches
>
> > It might be helpful if lei could print out message-ids of messages
> > written to mailboxes.
>
> That could get very noisy, especially as mailboxes are written
> in parallel.
Verbose mode already is. Maybe specifying what info you want to be
verbose would help. The network side is mostly uninteresting in this
case for example.
Is there any tool to list new messages in a maildir? I could do that
before and after. I've done the clearing the new flag in mutt between
runs, but that's not really ideal.
Rob
