Hi notmuch folks-- I just wanted to note a performance problem that i'm seeing with notmuch. I've attached a demonstration script ("crypto-performance-test"), which sets up a temporary GnuPG homedir and notmuch installation, adds 20 encrypted+signed messages in a thread (using indexed cleartext with stashed session IDs), and then tries to render the thread. It compares the performance of this rendering with manual extraction of the ciphertext, fed in a loop to manual invocations of gpg.
Problem Statement ----------------- On a long thread like this where each message is both signed and encrypted, "notmuch show" takes a long time to emit json ouput, even if all the session keys for the messages are stashed. I believe the bulk of the time spent rendering this is signature verification. I also note that if a comparable test is run with a large keyring, the duration increases as the keyring increases, but this test doesn't show that particular behavior. $ ./crypto-performance-test gpg: key 18523A27026D55EB marked as ultimately trusted Found 0 total files (that's not much mail). No new mail. gpg: checking the trustdb gpg: marginals needed: 3 completes needed: 1 trust model: pgp gpg: depth: 0 valid: 1 signed: 0 trust: 0-, 0q, 0n, 0m, 0f, 1u gpg: next trustdb check due at 2021-09-30 Processed 20 total files in 2s (8 files/sec.). Added 20 new messages to the database. performance of notmuch show: real 0m1.582s user 0m0.049s sys 0m0.042s performance of piped gpg: real 0m1.110s user 0m1.164s sys 0m0.200s performance of piped gpg --no-keyring: real 0m0.249s user 0m0.305s sys 0m0.175s $ note in particular the large delay for notmuch between wall-clock time ("real") and userspace ("user") and kernel ("sys") CPU time. Diagnosis --------- If i hack up gmime to disable signature verification (see id:87v9t9xw0l....@fifthhorseman.net over on gmime-devel-l...@gnome.org, which effectively adds the --no-keyring option to gpgme's gpg invocation), then i can reduce the kernel time and the overall delay: $ ./crypto-performance-test gpg: key B9F59627422A84DB marked as ultimately trusted Found 0 total files (that's not much mail). No new mail. gpg: checking the trustdb gpg: marginals needed: 3 completes needed: 1 trust model: pgp gpg: depth: 0 valid: 1 signed: 0 trust: 0-, 0q, 0n, 0m, 0f, 1u gpg: next trustdb check due at 2021-09-30 Processed 20 total files in 1s (11 files/sec.). Added 20 new messages to the database. performance of notmuch show: real 0m0.161s user 0m0.037s sys 0m0.017s performance of piped gpg: real 0m1.083s user 0m1.158s sys 0m0.176s performance of piped gpg --no-keyring: real 0m0.247s user 0m0.301s sys 0m0.177s $ But there even so, there is a real difference between the CPU time and the wall clock time. The system i've run these tests on is not starved for CPU or RAM, and it is not I/O bound. Even if there were disk troubles, the filesystem i'm running it on (where mktemp -d creates the working directory) is a tmpfs. My current guess is that bash's "time" built-in isn't tabulating all user+sys CPU time actually used by notmuch because gmime's gpgme invocations might be detaching its child processes entirely (i.e. double-fork, orphaning the grandchild). The gpg processes that appear during the "notmuch show" runs have parent pid 1. How to fix ---------- I think the right thing to do here is related to stashing signature verifications when they are first encountered, but i foresee trouble with that approach given notmuch's willingness to collapse messages that have the same message ID. That is: we want to stash the signature verification the first time we view it, and when displaying a message, we want to show the original signature verification (rather than computing it again), *as long as* the message has not changed since the previous verification. We can't do this at all right now until GMime gains the capability, but even if that happens i don't know how to do it safely. Any suggestion on how we ensure that a cached verification applies to all parts of the message being rendered via "notmuch show" ? --dkg
#!/bin/bash set -e workdir=$(mktemp -d) cleanup() { rm -rf "$workdir" } trap cleanup EXIT setup_gpg() { export GNUPGHOME="$workdir/g" mkdir -m 0700 -p "$GNUPGHOME" gpgargs=(--batch --pinentry-mode=loopback --passphrase '' --quiet --no-tty) gpg "${gpgargs[@]}" --quick-gen-key 'Test User <t...@example.org>' futuredefault } setup_notmuch() { mkdir -p "$workdir/m" export NOTMUCH_CONFIG="$workdir/notmuch-config" cat > "$NOTMUCH_CONFIG" <<EOF [database] path=$workdir/m [user] name=Test User primary_email=t...@example.org [new] tags=unread;inbox; EOF notmuch new } body() { cat <<EOF Content-Type: text/plain This is test message $1 EOF } encmsg() { cat <<EOF From: Test User <t...@example.org> To: Test User <t...@example.org> Date: $(date -R) Subject: Message $x Mime-Version: 1.0 Message-Id: <crypto-performance-$x...@example.org> References: <crypto-performa...@example.org> In-Reply-To: <crypto-performa...@example.org> Content-Type: multipart/encrypted; boundary="xxxx"; protocol="application/pgp-encrypted" --xxxx Content-Type: application/pgp-encrypted Version: 1 --xxxx Content-Type: application/octet-stream $(body $x | gpg --armor --sign -r t...@example.org --encrypt) --xxxx-- EOF } sk() { notmuch dump id:"crypto-performance-$1...@example.org" | grep session-key= \ | sed -r -e 's/.*session-key=([A-F0-9%a-f]+)*/\1/' -e 's/%3a/:/' } setup_gpg setup_notmuch for x in $(seq 1 20); do encmsg > "$workdir/m/$x.eml" done notmuch new --decrypt=true printf '\nperformance of notmuch show:' time notmuch show --format=json id:crypto-performanc...@example.org >/dev/null printf '\nperformance of piped gpg:' time for x in $(seq 1 20); do notmuch show --entire-thread=false --decrypt=false --format=raw --part=3 id:crypto-performance-$x...@example.org \ | gpg --quiet --batch --override-session-key $(sk $x) --decrypt >/dev/null 2>/dev/null done printf '\nperformance of piped gpg --no-keyring:' time for x in $(seq 1 20); do notmuch show --entire-thread=false --decrypt=false --format=raw --part=3 id:crypto-performance-$x...@example.org \ | gpg --quiet --batch --no-keyring --override-session-key $(sk $x) --decrypt >/dev/null 2>/dev/null || true done
signature.asc
Description: PGP signature
_______________________________________________ notmuch mailing list notmuch@notmuchmail.org https://notmuchmail.org/mailman/listinfo/notmuch