David Bremner <[email protected]> writes:
Here's why it would freeze:
I have a post-new hook that runs a Python script. Depending on
whether the new email it is processing matches a rule I have,
it will fire off an email to the sender using the SMTP library
in Python.
I had recently upgraded my MTA (PostFix), and it had a backward
incompatible change that broke my config. I don't know why, but
I could still send emails via Emacs, but when I tried to send
them via Python, Postfix would log an error and it would not
send. The Python statement would freeze (I guess Postfix
doesn't return an appropriate response? Not sure why).
I have a cron job to run "notmuch new" 3 times an hour. Since
the hook was frozen, so was the notmuch new command. I had
quite a lot of "notmuch new" processes. I assume this meant the
DB was locked all this time for writing.
notmuch unlocks the database before running the hook, so I don't
understand how a hung hook results in a locked database. If it
happens again (or you're motivated to set up a testbed) I'd be
interested in the output of
Well, it results in a locked database because I have this in the
(Python) hook:
DATABASE = notmuch.Database(mode=notmuch.Database.MODE.READ_WRITE)
Soon after that I freeze the new messages. And at the end I thaw
them out. The hang occurs in between the two, I think.
Also, is this by chance a network file system? Because those
often break locking.
No - regular hard drive.
Now killing all those jobs did not fix the database. It was
still broken. And as we saw the second time round, it was
/really/ broken - it would not even open in read-only mode.
That seems like something the Xapian devs (in copy) might be
interested in fixing, if you could come up with a simple
reproducer.
I can think of two experiments:
1. Write a hook that opens the database as above, and then just
does nothing (e.g. while True). Let it run, say, for 24
hours. (Not sure if the "freeze" part is relevant.
2. Same as the above, but have a cron job that fires "notmuch new"
every 20 minutes. This will freeze on the database line above (all
except the first invocation which will be stuck at while True).
After a day of this, check if you can open the database in
READ_WRITE mode.
notmuch could be cleverer about timing out on trying to acquire
a lock. I suspect it's a bit delicate to get that right, and
I've been hoping the underlying primitives would get a bit more
flexible w.r.t. locking.
I agree having notmuch handle it is not ideal. I was originally
thinking there should be a default timeout that one can adjust as
needed. However, when someone does "notmuch new" to build a new
database, that can take several minutes. And others may have flows
very different from mine.
At the very least, we probably should know why the DB be clobbered
at all.
--
Don't take life so seriously. It won't last.
/\ /\ /\ /
/ \/ \ u e e n / \/ a w a z
>>>>>>[email protected]<<<<<<
anl
_______________________________________________
notmuch mailing list
[email protected]
https://notmuchmail.org/mailman/listinfo/notmuch
