Re: Python3 cffi bindings
On Fri, Oct 16, 2020 at 9:19 AM Gaute Hope wrote: > On Wed, Oct 14, 2020 at 10:24 PM Floris Bruynooghe wrote: > >> Hi Gaute, >> >> On Thu 08 Oct 2020 at 10:13 +0200, Gaute Hope wrote: >> > I made another attempt at porting lieer to notmuch2, but I am missing >> the >> > get_directory method still. Any plans to look at it? >> >> Would indeed be good to add this sometime. I'm still curious to how you >> use it though to make sure we make a good API. I only found >> >> https://github.com/gauteh/lieer/blob/394d8c1a574fd57e63390e92a6e73363808ebac5/lieer/local.py#L280 >> and it seems you only use the `.path` attribute. Is this correct or did >> I miss anything? >> > > That is correct, as well as relying on an exception if the input directory > is not in the notmuch database. I also use `db.get_path()` to figure out > the relative path w.r.t to the database root, for use in `path:` queries ( > https://github.com/gauteh/lieer/blob/master/lieer/gmailieer.py#L315). > > Re-read the rest of the old thread, and I think you are right that I only need to use `db.get_path()` to do what I need. So that makes this issue less critical for me I guess. -- gaute ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: Python3 cffi bindings
On Wed, Oct 14, 2020 at 10:24 PM Floris Bruynooghe wrote: > Hi Gaute, > > On Thu 08 Oct 2020 at 10:13 +0200, Gaute Hope wrote: > > I made another attempt at porting lieer to notmuch2, but I am missing the > > get_directory method still. Any plans to look at it? > > Would indeed be good to add this sometime. I'm still curious to how you > use it though to make sure we make a good API. I only found > > https://github.com/gauteh/lieer/blob/394d8c1a574fd57e63390e92a6e73363808ebac5/lieer/local.py#L280 > and it seems you only use the `.path` attribute. Is this correct or did > I miss anything? > That is correct, as well as relying on an exception if the input directory is not in the notmuch database. I also use `db.get_path()` to figure out the relative path w.r.t to the database root, for use in `path:` queries ( https://github.com/gauteh/lieer/blob/master/lieer/gmailieer.py#L315). Regards, Gaute ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
Re: Python3 cffi bindings
Hi Floris and others, I made another attempt at porting lieer to notmuch2, but I am missing the get_directory method still. Any plans to look at it? Regards, Gaute On Sun, Nov 17, 2019 at 6:14 PM Floris Bruynooghe wrote: > Hi Gaute, > > Thanks for trying this out! > > On Mon 04 Nov 2019 at 11:27 +0100, Gaute Hope wrote: > > I just checked out the wip/cffi branch on git.notmuch.org with the > > purpose of porting Lieer (https://github.com/gauteh/lieer). There > > seems to be some missing functionality: `Database.get_directory()` > > specifically. > > Yeah, I didn't add that yet because I don't fully understand how it > should be used. Specifically I don't know where one might get a > pathname from to pass to .get_directory() and thus whether the API would > be cleaner to just return a reasonable directory object from whatever > location that might be. Maybe notmuch_database_get_path() is the only > entrypoint here and you can get further by listing files and directories > from it? But maybe people then use the filesystem directly to find a > directory and create the directories ad-hoc. > > I grepped lieer but I think you only use it in one place? And if I > understand it correctly you only do this to check if your mailstore/cwd > is inside the notmuch database. I.e. this is equivalent to checking if > your mailstore/cwd has notmuch2.Database.path as prefix which you could > easily do directly rather than using the FileError exception from > .get_directory(). > > So is anyone else aware of some code which uses db.get_directory() to > give an idea of how and why this is used? > > > I also ran into a couple of warning when building > > (included below). > > Thanks for pointing these out. I guess if the bindings are in the main > repo only the latest library version can be supported without any > further concerns. > > > By the way, it does not seem that the API is very far from the > > previous python API. If it is close enough, perhaps it is possible to > > get away with a bug version bump in the bindings rather than creating > > a new package. I understand the need for a new package, but it would > > be nice if we could avoid the future confusion of two python binding > > packages (if at all possible). > > While I'm glad to hear that you think a migration wouldn't be to painful > for you I am very weary of knowingly breaking APIs. I'd rather have > people have an easy migration rather than unexpected breakage after an > upgrade. > > > Cheers, > Floris > ___ notmuch mailing list -- notmuch@notmuchmail.org To unsubscribe send an email to notmuch-le...@notmuchmail.org
[lieer] Release v1.2
Greetings,
lieer v1.2 has been released:
https://github.com/gauteh/lieer
"Fast email-fetching and sending and two-way tag synchronization between
notmuch and GMail"
TL; DR: add the `-t` option to your config, and read the below
paragraph:
This release polishes the `sendmail`-emulation so that it may be used
with several classic MUAs. This also includes validating recipients
specified in the headers versus those specified as arguments (not
possible when using the GMail API, use the headers). So you probably
want to add the `-t` option in your MUA config and make you MUA
provide recipients in the header fields if it does not already.
Credits for contributions in this release:
* Gaute Hope
* Stefan Kangas
* Adrian Petrescu
* Florian Klink
Regards, Gaute
pgpQvLbjftYuK.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch
Re: [lieer] Release v1.1
On Fri, Apr 17, 2020 at 1:58 PM Thomas Schneider wrote: > Hi Gaute, > > Gaute Hope writes: > > > lieer v1.1 has been released with a bunch of minor fixes, and also the > > ability to (experimentally) send e-mail as a primitive sendmail stand-in. > > > > https://github.com/gauteh/lieer > > That’s nice to hear. I see the release on GitHub, but are you going to > upload it to PyPI as well? My Gentoo ebuild[0] gets the source from > there, so I noticed it’s not (yet) present on PyPI. > > Hi, I am not maintaining the PyPI package. Not sure exactly who it is at the moment, but I think the contact details are on the pypi page. Regards, Gaute ___ notmuch mailing list notmuch@notmuchmail.org https://notmuchmail.org/mailman/listinfo/notmuch
[lieer] Release v1.1
Hi,
lieer v1.1 has been released with a bunch of minor fixes, and also the
ability to (experimentally) send e-mail as a primitive sendmail stand-in.
https://github.com/gauteh/lieer
"Fast email-fetching and sending and two-way tag synchronization between
notmuch and GMail"
git shortlog:
* Gaute Hope
* Aurélien Ooms
* Radu Butoi
Regards, Gaute
pgpIzIBdOPvi9.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch
[lieer] Release v1.0
Hi, Lieer v1.0 has been released! Fast email-fetching and two-way tag synchronization between notmuch and GMail. https://github.com/gauteh/lieer Regards, Gaute ___ notmuch mailing list notmuch@notmuchmail.org https://notmuchmail.org/mailman/listinfo/notmuch
Re: Python3 cffi bindings
On Sun, Nov 17, 2019 at 6:14 PM Floris Bruynooghe wrote: > > Hi Gaute, > > Thanks for trying this out! > > On Mon 04 Nov 2019 at 11:27 +0100, Gaute Hope wrote: > > I just checked out the wip/cffi branch on git.notmuch.org with the > > purpose of porting Lieer (https://github.com/gauteh/lieer). There > > seems to be some missing functionality: `Database.get_directory()` > > specifically. > > Yeah, I didn't add that yet because I don't fully understand how it > should be used. Specifically I don't know where one might get a > pathname from to pass to .get_directory() and thus whether the API would > be cleaner to just return a reasonable directory object from whatever > location that might be. Maybe notmuch_database_get_path() is the only > entrypoint here and you can get further by listing files and directories > from it? But maybe people then use the filesystem directly to find a > directory and create the directories ad-hoc. If I understand correctly then these are the directories known to notmuch db, so may not correspond to filesystem. Lieer do not modify Directory objects directly, but others might. > I grepped lieer but I think you only use it in one place? And if I > understand it correctly you only do this to check if your mailstore/cwd > is inside the notmuch database. I.e. this is equivalent to checking if > your mailstore/cwd has notmuch2.Database.path as prefix which you could > easily do directly rather than using the FileError exception from > .get_directory(). Yes, I think that would work here. I need the path of the directory later (for the path:.. query). Seems that the current python API removes the leading path of the database for the argument to notmuch_database_get_directory(..) -- at least if the notmuch API docs are correct. I had some reported issues with symlinked directories and absolute paths, but I don't think any of that would be influenced by changes like these. Regards, Gaute ___ notmuch mailing list notmuch@notmuchmail.org https://notmuchmail.org/mailman/listinfo/notmuch
Re: Python3 cffi bindings
Hi,
I just checked out the wip/cffi branch on git.notmuch.org with the
purpose of porting Lieer (https://github.com/gauteh/lieer). There
seems to be some missing functionality: `Database.get_directory()`
specifically. I also ran into a couple of warning when building
(included below).
Thanks for your work.
By the way, it does not seem that the API is very far from the
previous python API. If it is close enough, perhaps it is possible to
get away with a bug version bump in the bindings rather than creating
a new package. I understand the need for a new package, but it would
be nice if we could avoid the future confusion of two python binding
packages (if at all possible).
Regards, Gaute
~/dev/notm/notmuch/bindings/python-cffi wip/cffi
notm ❯ python setup.py build
warning: no previously-included files found matching 'setup.pyc'
warning: no previously-included files matching 'yacctab.*' found under
directory 'tests'
warning: no previously-included files matching 'lextab.*' found under
directory 'tests'
warning: no previously-included files matching 'yacctab.*' found under
directory 'examples'
warning: no previously-included files matching 'lextab.*' found under
directory 'examples'
zip_safe flag not set; analyzing archive contents...
pycparser.ply.__pycache__.lex.cpython-37: module references __file__
pycparser.ply.__pycache__.lex.cpython-37: module MAY be using
inspect.getsourcefile
pycparser.ply.__pycache__.yacc.cpython-37: module references __file__
pycparser.ply.__pycache__.yacc.cpython-37: module MAY be using
inspect.getsourcefile
pycparser.ply.__pycache__.yacc.cpython-37: module MAY be using inspect.stack
pycparser.ply.__pycache__.ygen.cpython-37: module references __file__
Installed
/home/gauteh/dev/notm/notmuch/bindings/python-cffi/.eggs/pycparser-2.19-py3.7.egg
running build
running build_py
creating build
creating build/lib.linux-x86_64-3.7
creating build/lib.linux-x86_64-3.7/notdb
copying notdb/_message.py -> build/lib.linux-x86_64-3.7/notdb
copying notdb/__init__.py -> build/lib.linux-x86_64-3.7/notdb
copying notdb/_database.py -> build/lib.linux-x86_64-3.7/notdb
copying notdb/_errors.py -> build/lib.linux-x86_64-3.7/notdb
copying notdb/_tags.py -> build/lib.linux-x86_64-3.7/notdb
copying notdb/_thread.py -> build/lib.linux-x86_64-3.7/notdb
copying notdb/_query.py -> build/lib.linux-x86_64-3.7/notdb
copying notdb/_build.py -> build/lib.linux-x86_64-3.7/notdb
copying notdb/_base.py -> build/lib.linux-x86_64-3.7/notdb
running build_ext
generating cffi module 'build/temp.linux-x86_64-3.7/notdb._capi.c'
creating build/temp.linux-x86_64-3.7
building 'notdb._capi' extension
creating build/temp.linux-x86_64-3.7/build
creating build/temp.linux-x86_64-3.7/build/temp.linux-x86_64-3.7
gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3
-Wall -fPIC -I../../lib
-I/home/gauteh/.pyenv/versions/3.7.4/envs/notm/include
-I/home/gauteh/.pyenv/versions/3.7.4/include/python3.7m -c
build/temp.linux-x86_64-3.7/notdb._capi.c -o
build/temp.linux-x86_64-3.7/build/temp.linux-x86_64-3.7/notdb._capi.o
build/temp.linux-x86_64-3.7/notdb._capi.c: In function
‘_cffi_d_notmuch_database_add_message’:
build/temp.linux-x86_64-3.7/notdb._capi.c:980:3: warning:
‘notmuch_database_add_message’ is deprecated: function deprecated as
of libnotmuch 5.1 [-Wdeprecated-declarations]
return notmuch_database_add_message(x0, x1, x2);
^~
In file included from build/temp.linux-x86_64-3.7/notdb._capi.c:495:0:
../../lib/notmuch.h:637:1: note: declared here
notmuch_database_add_message (notmuch_database_t *database,
^~~~
build/temp.linux-x86_64-3.7/notdb._capi.c: In function
‘_cffi_f_notmuch_database_add_message’:
build/temp.linux-x86_64-3.7/notdb._capi.c:1033:3: warning:
‘notmuch_database_add_message’ is deprecated: function deprecated as
of libnotmuch 5.1 [-Wdeprecated-declarations]
{ result = notmuch_database_add_message(x0, x1, x2); }
^
In file included from build/temp.linux-x86_64-3.7/notdb._capi.c:495:0:
../../lib/notmuch.h:637:1: note: declared here
notmuch_database_add_message (notmuch_database_t *database,
^~~~
gcc -pthread -shared -L/home/gauteh/.pyenv/versions/3.7.4/lib
-L/home/gauteh/.pyenv/versions/3.7.4/lib
build/temp.linux-x86_64-3.7/build/temp.linux-x86_64-3.7/notdb._capi.o
-L../../lib -lnotmuch -o
build/lib.linux-x86_64-3.7/notdb/_capi.abi3.so
On Tue, Oct 8, 2019 at 11:03 PM Floris Bruynooghe wrote:
>
> Hi all,
>
> IIRC there was a thread in August about another attempt at bringing
> the CFFI-based bindings on board as a Python3-only version. I
> believe there was a desire to re-name things but my searching-fu is
> failing me and I can no longer find the email thread.
>
> Anyway, I found the code, checked things work, updated tests on new
> python versions, added a very basic intergration with the test
> framework and squashed the commits. Otherwise the attached patch
> is just a plain dump of the current state so interested people have
>
[astroid] release v0.15
Greetings, astroid v0.15 has been released! Astorid is: "A graphical threads-with-tags style, lightweight and fast, e-mail client for Notmuch" get it at: https://github.com/astroidmail/astroid Thanks to all contributors! See the git log for changes. Regards, Gaute ___ notmuch mailing list notmuch@notmuchmail.org https://notmuchmail.org/mailman/listinfo/notmuch
[lieer] Release v0.11
Greetings, a new release of lieer (earlier gmailieer) has been released! Lieer is a "Fast email-fetching and two-way tag synchronization between notmuch and GMail" get it at: https://github.com/gauteh/lieer The name change was necessary due to Googles new API rules. Regards, Gaute ___ notmuch mailing list notmuch@notmuchmail.org https://notmuchmail.org/mailman/listinfo/notmuch
gmailieer -> lieer
Hi, because of Googles new stricter rules [0] for public OAuth api keys gmailieer needs to be renamed to something non-google / non-gmail´y. So say hello to `lieer`. https://github.com/gauteh/lieer There will probably be a bunch of references that need to be updated. Hopefully this will not be a too big pain, the alternative would be to require all users to generate their own API key. And hopefully Google thinks this is Ok... Regards, Gaute [0] https://github.com/gauteh/lieer/issues/100 ___ notmuch mailing list notmuch@notmuchmail.org https://notmuchmail.org/mailman/listinfo/notmuch
Re: segfault using python bindings
David Bremner writes on November 11, 2018 21:16: [1]: There is some discussion in the list archives. See id:87lg5z74l3@tethera.net Regards, Gaute ___ notmuch mailing list notmuch@notmuchmail.org https://notmuchmail.org/mailman/listinfo/notmuch
[gmailieer] Gmailieer v0.10 released
Hi, Gmailieer v0.10 has been released! https://github.com/gauteh/gmailieer Gmailieer provides fast email-fetching and two-way tag synchronization between notmuch and GMail. Changes: * Remote tags/labels can now be ignored (special categories are now optionally ignored). * Various GMail quirks documented. Regards, Gaute ___ notmuch mailing list notmuch@notmuchmail.org https://notmuchmail.org/mailman/listinfo/notmuch
[astroid] Announcing Astroid v0.14
Greetings, Astroid v0.14 has been released! Astroid is a lightweight and fast graphical threads-with-tags email client for notmuch. Written in C++ using GTK+, WebKit2 and GMime. Astroid can be acquired, along with instructions for use, at: https://github.com/astroidmail/astroid once you get Astroid running press '?' to get a list of keybindings for the current context. For 'The Tour' check out: https://github.com/astroidmail/astroid/wiki ## Changes since v0.13 * Use cmark by default as markdown processor. * Respect $NOTMUCH_CONFIG and ignore configuration option if set. Otherwise fall back to option from configuration file. * We now have a man page! Generated either by scdoc or ronn. * Number of bugfixes. Gaute Hope (24): log: re-introduce log-level command line option to overrdide config git: only ignore build/ dirs, not build* tv: remove deactived syntax code readme: update demo image cmake: webkit2gtk >= 2.20 is required w2: fix jump to top / bottom readme: fix typos fix #535: close correct tab if Mode::close is called on non-active tab cmake: libhypocycloid should always be static test: bump plugins to GIR v0.2 fix #541: use 'cmark' rather than 'marked' as default markdown processor cmake: generate and install man pages using either scdoc or ronn" man: mention $NOTMUCH_CONFIG (93ef250b) history: man page fix #539: reference to LICENSE.md. fix #542 cf: do not show default startup queries when intentionally cleared. force saved_searches if no startup queries. cmake, man: add option to disable manpage generation -DDISABLE_DOCS=ON fix #548: correctly clear messages and focus state when loading new thread mw: remove modes when closing window tests: make gpg setup more verbose tests: quote cp src dir fix #550: use srcdoc attribute when setting message content in iframe tv, iframe: delay set iframe src in order to let body element be rendered first Release v0.14 Iain Buclaw (1): fix #501: Add hook to signal_delete_event, ensure proper destruction of window Johannes Löthberg (1): Use NOTMUCH_CONFIG if set, otherwise expand astroid.notmuch_config Timothée Floure (4): Add initial manual page astroid(1) Add the --log-level option to the manpage Rephrase the 'NOTE' (on software used with astroid) section of the manpage Mention the possibility to customize the thread-view in the manpage Tristan Cacqueray (1): cmake: fix scalable icon install path Yurii Rashkovskii (1): Problem: tests/run-test.sh might fail pgpOPKrLWHqWu.pgp Description: PGP signature ___ notmuch mailing list notmuch@notmuchmail.org https://notmuchmail.org/mailman/listinfo/notmuch
Re: [PATCH] introduce new.rename_tags for renamed (moved) messages
Michael J Gruber writes on September 21, 2018 10:22: Lastmod and (re)new tags serve different purposes, I would thank. The latter is a flag that can be cleared to signal that a certain job has been done (by whomever). Whereas, if one actor acts upon a lastmod change by modifying the db that will change the lastmod and (possibly) trigger another actor to act upon that. Different scenarios which look the same if you have one "actor" only. Agreed, very useful if all scenarios can be reliably caught! Maybe it was the "message changed without rename" that was problematic, since offlineimap would change the X-Keywords header without renaming the file. - gaute pgpEXchwqyctO.pgp Description: PGP signature ___ notmuch mailing list notmuch@notmuchmail.org https://notmuchmail.org/mailman/listinfo/notmuch
Re: [PATCH] introduce new.rename_tags for renamed (moved) messages
Michael J Gruber writes on July 6, 2018 16:04:
IMAP clients (such as webmail) use folders to mark messages as junk
etc., some even to mark messages as trash ("move to trash"). Such a
change is reported by notmuch as a rename; the message is not tagged
with new.tags since it is not new, so that there is no way to act upon a
rename.
Introduce new.rename_tags (default: not set) which are added by `notmuch
new` to renamed messages. This allows to act upon renames, e.g. to keep
the IMAP folder structure in sync with tags with a tool like `afew` or
homecooked scripts simply by filtering for this tag in the same ways as
one would filter for new messages using new.tags.
Hi,
think this would be very useful. I suggested something similar way back[0]
(no doubt bit-rotted by now), and seem to remember there were some
issues with cases where a rename would not be detected. Might be worth
checking out !
[0] id:1396800683-9164-1-git-send-email...@gaute.vetsj.com
BTW: My use-case was solved by using `lastmod:` queries in keywsync [1]
(later obsoleted by gmailieer[2]).
[1] https://github.com/gauteh/abunchoftags
[2] https://github.com/gauteh/gmailieer
Regards, Gaute
pgpPr3HwklXIs.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch
[astroid] Announcing Astroid v0.13 ( with WebKit2 ! )
Greetings, Astroid v0.13 has been released! Astroid is a lightweight and fast graphical threads-with-tags email client for notmuch. Written in C++ using GTK+, WebKit2 and GMime. Astroid can be acquired, along with instructions for use, at: https://github.com/astroidmail/astroid once you get Astroid running press '?' to get a list of keybindings for the current context. For 'The Tour' check out: https://github.com/astroidmail/astroid/wiki ## Changes since v0.12 * Move to WebKit2 We have now finally moved to WebKit2. This is a big change since WebKit2 required all DOM manipulation to be moved to a WebExtension running in a separate process. The IPC with the extension is implemented using Google ProtoBufs (>= 3.0) and Unix Domain Sockets. WebKit2 is actively maintained and consequently a long list of CVEs in the old WebKit no longer cause worries. Additionally, each part is shown in separate IFRAMEs so that essentially all received content is sandboxed (some metadata like addresses are escaped and shown in main page). This protects better against EFAIL type attacks (which I believe we were not vulnerable to anyway). Expect some instability while breaking in this new change! Important: Make sure you update your plugins! If they try to use the old WebKit GIO bindings with the new WebKit objects astroid will freeze. Part rendering: * Show parts in IFRAME * Toggle between parts without re-opening * The default part shown (HTML or TEXT) is now configurable * Optionally show HTML part if no other part Images and EFAIL: * Do not allow images in encrypted messages (#499) * Show CID (attached) images in HTML parts (on C-i) * Zoom messages with C-+ and C-- Syntax highlighting: * Syntax highlighting through JS has been replaced by a syntax highlight plugin based on Pygments. * Add filter_part plugin method, which can be used to modify the displayed HTML depending on either the text/plain or the text/html part. Misc: * Logging can be configured to go to syslog (or stdout, default). File logging is removed since we now have multiple processes and this would complicate writing to one file. * The configuration file is no longer written back by default if missing values are found. This also means that values undefined by the user will be updated if the default is changed. Various checks for merging older versions of the configuration file to the current version has been removed. The parser is more resilient and the old versions are old. Regards, Gaute pgpPc7Fb_M__O.pgp Description: PGP signature ___ notmuch mailing list notmuch@notmuchmail.org https://notmuchmail.org/mailman/listinfo/notmuch
Re: In some cases get_toplevel_messages() and _message_get_replies() does not return all messages
David Bremner writes on July 10, 2018 19:21: Gaute Hope writes: Hi, a while back I got a bug report for astroid about get_toplevel_messages() and _message_get_replies() not showing all messages [0]. When relying on these alone the user will occasionally miss emails. I do not have much more information, but I'll post this here in case someone else experiences this -- or it might already be fixed? Not sure it it's the same bug, but there was a bug related to reference loops that would have lost some top level messages. This bug was fixed in notmuch 0.26.2, so if the user hasn't tried with that or 0.27, they probably should. Thanks, I'll see if I can get a reply (as you probably noticed the issue is from 2016). I can probably change the code to throw an error on mismatching message count rather than looping through the messages. - gaute ___ notmuch mailing list notmuch@notmuchmail.org https://notmuchmail.org/mailman/listinfo/notmuch
In some cases get_toplevel_messages() and _message_get_replies() does not return all messages
Hi, a while back I got a bug report for astroid about get_toplevel_messages() and _message_get_replies() not showing all messages [0]. When relying on these alone the user will occasionally miss emails. I do not have much more information, but I'll post this here in case someone else experiences this -- or it might already be fixed? Regards, Gaute [0] https://github.com/astroidmail/astroid/pull/243 pgp1RMXnTniBa.pgp Description: PGP signature ___ notmuch mailing list notmuch@notmuchmail.org https://notmuchmail.org/mailman/listinfo/notmuch
[gmailieer] Gmailieer v0.9 released
Hi, Gmailieer v0.9 has been released! https://github.com/gauteh/gmailieer Gmailieer provides fast email-fetching and two-way tag synchronization between notmuch and GMail. Changes: * Fix docs and help (timeout 0 means min (forver, system-timeout)). * All output is ASCII, ensures working on all outputs (read non-ttys / pipes). Regards, Gaute pgpUJFFfQSxGq.pgp Description: PGP signature ___ notmuch mailing list notmuch@notmuchmail.org https://notmuchmail.org/mailman/listinfo/notmuch
[astroid] Announcing Astroid v0.12
Greetings, Astroid v0.12 has been released! Astroid is a lightweight and fast graphical threads-with-tags email client for notmuch. Written in C++ using GTK+, WebKit and gmime. Astroid can be acquired, along with usage and instructions for use, at: https://github.com/astroidmail/astroid once you get Astroid running press '?' to get a list of keybindings for the current context. For 'The Tour' check out: https://github.com/astroidmail/astroid/wiki ## Some of the changes since v0.11.1 * `mail.close_on_success` configures if page closes on successful send. * `terminal.font_description` configures font of embedded terminal. * `terminal.height` configures height of embedded terminal. * `mail.format_flowed` is off by default. * Make hooks undoable by providing a field for the inverse command. * Do not force throw-keyid (follow gpg config). * Allow GPG to be disabled with configuration option. Regards, Gaute pgpAkkA0TTvKO.pgp Description: PGP signature ___ notmuch mailing list notmuch@notmuchmail.org https://notmuchmail.org/mailman/listinfo/notmuch
[gmailieer] Gmailieer v0.8 released
Hi, Gmailieer v0.8 has been released! https://github.com/gauteh/gmailieer Gmailieer provides fast email-fetching and two-way tag synchronization between notmuch and GMail. Changes: * Allow custom tags to be ignored (when pushing), can be set with `gmi set`. * Bug fix in `gmi set`. Regards, Gaute ___ notmuch mailing list notmuch@notmuchmail.org https://notmuchmail.org/mailman/listinfo/notmuch
Re: Thread subqueries
man. 7. mai 2018 kl. 14:09 skrev David Bremner : > David Bremner writes: > > > This is the first non-WIP version of this series. It adds a small > > optimization (something like a 10% speedup on SSD), and some > > documentation and tests. > > pushed to master, with Jani's suggestions. Looking forward to test this! Great effort! Gaute ___ notmuch mailing list notmuch@notmuchmail.org https://notmuchmail.org/mailman/listinfo/notmuch
[gmaileer] Gmailieer v0.7 released
Hi, Gmailieer v0.7 has been released! https://github.com/gauteh/gmailieer Gmailieer provides fast email-fetching and two-way tag synchronization between notmuch and GMail. Changes: * Reload local repository if local changes have been made during sync. * Only allow one instance of gmi for each directory. * Ignore 'voicemail'. * Optionally disable progressbar with flag (-s) or when no installed tqdm (better support for non-tty output). * Increase batch size if several successful requests have been made. * Bug fixes. Regards, Gaute pgpp3Ao8nfCt2.pgp Description: PGP signature ___ notmuch mailing list notmuch@notmuchmail.org https://notmuchmail.org/mailman/listinfo/notmuch
Announcing Astroid v0.11.1
Gaute Hope writes on februar 4, 2018 11:46: Greetings, Astroid v0.11 has been released! Astroid is a lightweight and fast graphical threads-with-tags email client for notmuch. Written in C++ using GTK+, WebKit and gmime. Astroid can be acquired, along with usage and instructions for use, at: https://github.com/astroidmail/astroid Hi, Astroid v0.11.1 has been released, CMake now automatically uses the same GMime version as notmuch. Have fun - and please bug your distro to recompile packages against GMime 3 now that it is default. Regards, Gaute pgpk3fBEZNBS5.pgp Description: PGP signature ___ notmuch mailing list notmuch@notmuchmail.org https://notmuchmail.org/mailman/listinfo/notmuch
Re: Bcc, throw-keyids, and metadata hiding [was: Re: Announcing Astroid v0.11]
Daniel Kahn Gillmor writes on februar 5, 2018 9:33: On Mon 2018-02-05 08:33:36 +0100, Gaute Hope wrote: Yes; this seems like the ultimate approach to this problem, unless it will be possible for GPG to completely hide receivers - I am guessing this is inherently impossible? I'm not sure how gpg could do that -- the metadata leak of most recipients (To:, Cc:) is *outside* of the material that GnuPG handles, since GnuPG doesn't see the mesage headers when it's encrypting the body. Maybe i'm misunderstanding you though? I mean the recipient key list in the header of the encrypted packet [0][1]. I assume there must be a key list entry for each receiving key (even though it does not need to be accurate). It would be better to just remove the whole table of receiving keys, than setting each of them to 0. Regards, Gaute [0] https://www.ietf.org/rfc/rfc4880.txt [1] https://crypto.stackexchange.com/questions/10253/why-are-the-first-few-bytes-of-a-gpg-encryption-always-the-same ___ notmuch mailing list notmuch@notmuchmail.org https://notmuchmail.org/mailman/listinfo/notmuch
Re: Bcc, throw-keyids, and metadata hiding [was: Re: Announcing Astroid v0.11]
Daniel Kahn Gillmor writes on februar 5, 2018 2:21: On Sun 2018-02-04 16:18:02 -0500, Daniel Kahn Gillmor wrote: Well, i guess you could limit it to two copies total: one copy is to all Bcc'ed recipients, and one copy to all non-Bcc'ed recipients. you'd want to make sure that you got the same Message-ID on each generated copy, of course. That avoids even the count of the Bcc recipients going out to the non-bcc folks, too, which is a nice outcome. To avoid (b), you could do one copy of the message per Bcc'ed address, and never throw keyids at all. This isn't an extra metadata leak, because the bcc'ed person's e-mail address will be put in the SMTP envelope (and, likely, in Delivered-To or other equivalent headers appended by the MTA). So it's N + 1 copies of the message, where N is the number of Bcc'ed individuals. This also removes any leak of the number of Bcc'ed individuals from the Bcc'ed message. Yes; this seems like the ultimate approach to this problem, unless it will be possible for GPG to completely hide receivers - I am guessing this is inherently impossible? A couple of concerns, some of them UI: * What if one of the e-mails go through and not the other, especially from an UI perspective - how do you modify and re-send just that copy of the e-mail. * What if you want to reply-all to your own e-mail, in notmuch land the messages with the same ID will be joined together. An UI could do differently, but either way all the information about receivers you need is in different files. Realistically; I think the approach using optional/configurable - and if possible: custom `hidden-receivers` [0] - is much faster to implement + easier to get right. In other words, I would very much like to see a proper implementation of the multiple-messages approach, but unless someone else is able to help out, I will probably go for the simpler approach at first. Regards, Gaute [0] https://github.com/jstedfast/gmime/issues/45 pgp2WjSU3HO79.pgp Description: PGP signature ___ notmuch mailing list notmuch@notmuchmail.org https://notmuchmail.org/mailman/listinfo/notmuch
Re: umask/permissions on new files created via notmuch-insert(1) ?
Daniel Kahn Gillmor writes on februar 4, 2018 20:25: is there a reason that "notmuch insert" should be stricter than the umask? does this ring any bells for people? Are you asking why it is or why it should? If former; maybe because of line 230 in notmuch-insert.c ? Regards, Gaute pgpm8RbyQ5Nr_.pgp Description: PGP signature ___ notmuch mailing list notmuch@notmuchmail.org https://notmuchmail.org/mailman/listinfo/notmuch
Re: Announcing Astroid v0.11
Daniel Kahn Gillmor writes on februar 4, 2018 19:32: On Sun 2018-02-04 18:52:22 +0100, Gaute Hope wrote: This is done to hide Bcc-recipients. sure, but i'm wondering why you throw *all* keyids, instead of only the key-ids of the bcc'ed people? Because that is currently the only option when using GMime [0]. As you say, GnuPG must try all the secret keys; but many users use some sort of keyring to unlock their keys - in which case the hassle is limited to a bit extra time. I don't have any stats on this though! right, but the sender can't know whether this is the case or not, i think. fwiw, i do agree with you that the onus is ultimately on the recipient's MUA to fix this UI/UX disaster; but why force it on them in the case where it doesn't actually eliminate any metadata leakage? (i.e., when they're in To: or Cc: already, and not Bcc'ed) Agreed; it should be turned off (as per the spec in my previous email) when there are no Bcc-recipients. The best would of course be to send the e-mail seperately to each Bcc-recipient, but that feels like being overly careful / taking on the job of the MTA. Regards, Gaute [0] https://developer.gnome.org/gmime/stable/GMimeMultipartEncrypted.html#g-mime-multipart-encrypted-encrypt pgpojaMDtH9Gs.pgp Description: PGP signature ___ notmuch mailing list notmuch@notmuchmail.org https://notmuchmail.org/mailman/listinfo/notmuch
Re: Announcing Astroid v0.11
Daniel Kahn Gillmor writes on februar 4, 2018 16:37: On Sun 2018-02-04 11:46:20 +0100, Gaute Hope wrote: * Always throw key-id when sending (using GMime 3) Can you explain this choice? As someone who receives mail with a thrown key-id, and as someone who has multiple secret keys, the user experience of receiving encrypted mail like this is *terrible*. (terrible to the point of me wanting to ask people who do this for normal mail to just send me mail in the clear in the future :( ) In particular: GnuPG doesn't know which key to use, so it prompts me for passphrases for *all* of the secret keys i control, in succession. I understand the desire to reduce metadata leakage. But if you're wrapping the PGP/MIME application/pgp-encrypted part in an RFC822 message that contains a header with the person's e-mail address anyway, it's not clear that there has been a significant reduction of cleartext metadata. So this seems like a bad tradeoff for any case where the recipient is explicitly specified (i.e., not in Bcc:) Hi Daniel, This is done to hide Bcc-recipients. As you mention - and you know all this, but I'll list it for the sake of discussion - any Bcc-recipients would be listed as GPG recipients and would thus be exposed (which could be critical / embarassing). Of course the number of anonymous recipients does not match the visible receivers with Bcc-recipients. As a side-note it is theoretically possible to set the GPG-recipient-key to arbitrary values independent of the actual recipients. There has been some discussion on the issue on #astroid, and ideally we should make this optionable, default off (configurable) - with a warning before sending when Bcc-recipients are listed and keys are not thrown. (Note that this would also leak meta-data, since if keys are only thrown when there are Bcc-recipients you have another hint that there are Bcc-recipients, see above.) As you say, GnuPG must try all the secret keys; but many users use some sort of keyring to unlock their keys - in which case the hassle is limited to a bit extra time. I don't have any stats on this though! Regards, Gaute pgpvcIgmtx3pj.pgp Description: PGP signature ___ notmuch mailing list notmuch@notmuchmail.org https://notmuchmail.org/mailman/listinfo/notmuch
Announcing Astroid v0.11
Greetings, Astroid v0.11 has been released! Astroid is a lightweight and fast graphical threads-with-tags email client for notmuch. Written in C++ using GTK+, WebKit and gmime. Astroid can be acquired, along with usage and instructions for use, at: https://github.com/astroidmail/astroid once you get Astroid running press '?' to get a list of keybindings for the current context. For 'The Tour' check out: https://github.com/astroidmail/astroid/wiki ## Some of the changes since v0.10.2 * Move to the CMake build system. * Allow HTML parts to be composed using Markdown. * Use color to indicate whether a message is marked in thread-index. * Always throw key-id when sending (using GMime 3) * Bunch of minor stuff Regards, Gaute pgpmbTsieMSWn.pgp Description: PGP signature ___ notmuch mailing list notmuch@notmuchmail.org https://notmuchmail.org/mailman/listinfo/notmuch
gmailieer v0.6 released
Hi, Gmailieer v0.6 has been released! https://github.com/gauteh/gmailieer Gmailieer provides fast email-fetching and two-way tag synchronization between notmuch and GMail. Major changes: * Index emails in notmuch db by each received batch from GMail * Ignore 'Trash' tag * Verbose errors if manually modifying local maildir Regards, Gaute pgp_ELcSfPL_2.pgp Description: PGP signature ___ notmuch mailing list notmuch@notmuchmail.org https://notmuchmail.org/mailman/listinfo/notmuch
gmailieer v0.5 released
Hi, Gmailieer v0.5 has been released! The changes can be seen listed below, get the latest version here: https://github.com/gauteh/gmailieer Gmailieer provides fast email-fetching and two-way tag synchronization between notmuch and GMail. Changes: * 2017-11-20 5772de0 Gaute Hope local: ignore the `todo` tag, seems to be reserved. Regards, Gaute pgpXirYKvJv_8.pgp Description: PGP signature ___ notmuch mailing list notmuch@notmuchmail.org https://notmuchmail.org/mailman/listinfo/notmuch
Re: python bindings memory management
David Bremner writes on november 20, 2017 0:11: Gaute Hope writes: Hi, when writing this small script [0][1] using the notmuch python bindings I had to manually delete the python objects in the right order. Otherwise it segfaults when exiting, presumably because memory dependencies / cleanup isn't done correctly when the Python GC gets going. Essentially I have to delete the objects in the order of their dependency: Assuming this is python > 3.3, it sounds related to id:5694ca65.8010...@fastmail.net See in particular id:20160112185107.ga21...@odin.tremily.us Yep, probably the same thing. - Gaute ___ notmuch mailing list notmuch@notmuchmail.org https://notmuchmail.org/mailman/listinfo/notmuch
python bindings memory management
Hi,
when writing this small script [0][1] using the notmuch python bindings I
had to manually delete the python objects in the right order. Otherwise
it segfaults when exiting, presumably because memory dependencies /
cleanup isn't done correctly when the Python GC gets going. Essentially
I have to delete the objects in the order of their dependency:
```
db = notmuch.Database()
q = db.create_query ("some query")
q.set_sort (notmuch.Query.SORT.OLDEST_FIRST) # probably not necessary
msgs = q.search_messages ()
for m in msgs:
pass
# remove these to get segfault
del m
del msgs
del q
```
Regards, Gaute
[0] https://gist.github.com/gauteh/de546cb143ebae4a489a482a05ab4beb
[1] id:1507537566.y6qjbg4xoq.astr...@strange.none
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch
Re: bounce/forward not working due to CR at line end
Kai Harries writes on november 17, 2017 19:16: I am thinking about filing a ticket against gmailier because storing the emails with CRLF on a *nix is at least surprising. Hi, this was actually fixed in commit 71ce2b2 where line-endings are converted to \n if that is the OS line seperator. I am not modifying the file in any other way, but I think the DOS bomb should hopefully be set correctly already if on Linux [0]. https://github.com/gauteh/gmailieer/blob/master/lieer/local.py#L326 GMail API supplies the content of the message base64 encoded (the whole message not just any MIME parts) with CRLF line endings. Anyway, I realized I never made a release with that commit (I just did) so unless you were running 'master' you would have been missing out on that. Note that messages that were already downloaded before this change will not be converted by gmailieer, but you can do that youself. All gmailieer cares about is the file names. [0] https://en.wikipedia.org/wiki/Byte_order_mark#UTF-8 - gaute ___ notmuch mailing list notmuch@notmuchmail.org https://notmuchmail.org/mailman/listinfo/notmuch
gmailieer v0.4
Hi, I just release gmailieer v0.4! The changes can be seen listed below, get the latest version here: https://github.com/gauteh/gmailieer Regards, Gaute 2017-11-19 b7a9237 Gaute Hope release v0.4 (HEAD -> master, tag: v0.4, origin/master) 2017-11-17 5a9f3f7 Gaute Hope drop missing labels (if allowed) (origin/pr/51, origin/missing-gmail-label, missing-gmail-label) 2017-11-17 8beacba Gaute Hope add inverse option for unsetting replace slash with dot 2017-11-16 90bc1fb Giulio Eulisse add 'unknown' tag when there is a non-existing label 2017-11-17 28973a7 Gaute Hope avoid one extra dict lookup 2017-11-16 262c604 Gaute Hope reindent 2017-11-16 e1eedcc Gaute Hope pull: print gmail key for label as well 2017-11-14 18cc5fb Gaute Hope don't make backup on first write 2017-10-10 1eff3f6 Gaute Hope remote: copy backup, not move 2017-10-10 e3efa2c Gaute Hope local: store backup of .gmailieer.json every time 2017-10-10 ddb5337 Gaute Hope remote: accept empty message pages 2017-10-10 2bfab2b Gaute Hope pull, remote: fail fast on missing history field. (origin/history-again, history-again) 2017-10-10 2eb1fb7 Gaute Hope partial, remote: re-try to fetch history if missing history page. 2017-10-10 473c6aa Gaute Hope remote: GenericException -> Remote.GenericException 2017-10-07 c568745 Gaute Hope resp.code should be resp.status 2017-09-15 2901d76 Julian Andres Klode Do not pass self to self.__update_cache__() 2017-09-15 c050358 Julian Andres Klode Fix reference to gid that should be old_gid 2017-09-15 68a173c Gaute Hope factor out cache handling and clarify that we are operating on GIDs (origin/pr/44, origin/add-all, add-all) 2017-09-15 f66000c Gaute Hope local: add all files corresponding to message id to local cache 2017-09-15 b74da5f Gaute Hope get_content(): compact list filter (origin/pr/42, origin/also-new, also-new) 2017-09-15 fff8484 Gaute Hope local: add newly added messages to cache 2017-09-12 6d9cc52 Gaute Hope local: include messages stored in new in cache as well 2017-09-13 8ae3bc7 Gaute Hope remote: reduce default batch size to 50 (as encouraged by google) to avoid throttling. 2017-09-12 dc62fe6 Gaute Hope local: do not add 'T' to maildir-flags (same as notmuch) 2017-09-12 cb203e3 Gaute Hope local: don't remove message from cache during dry-run 2017-09-10 331cd0f Gaute Hope local: fix #41: check if message file is in root maildir, not just in cur/ 2017-09-08 71ce2b2 Gaute Hope local: store received mail with POSIX line endings depending on os.linesep pgp1wxRkbyzZj.pgp Description: PGP signature ___ notmuch mailing list notmuch@notmuchmail.org https://notmuchmail.org/mailman/listinfo/notmuch
Re: talloc_abort in notmuch_thread_get_tags () when db has been modified
David Bremner writes on november 3, 2017 13:18: Gaute Hope writes: Not sure if this is talloc_abort() anymore. Actually, at this point this seems to be caused by different GMime versions used for binary and notmuch library. Regards, Gaute OK, that sounds like not-a-notmuch-bug. Yes.. most definetely not. - gaute ___ notmuch mailing list notmuch@notmuchmail.org https://notmuchmail.org/mailman/listinfo/notmuch
Re: talloc_abort in notmuch_thread_get_tags () when db has been modified
Gaute Hope writes on november 3, 2017 11:45: David Bremner writes on februar 17, 2017 16:35: Gaute Hope writes: threads != NULL terminate called after throwing an instance of 'Xapian::DatabaseModifiedError' Yeah, that looks like a different problem. But it _should_ be something we can catch in libnotmuch. For reference; I'm getting several reports of this or similar error: * https://github.com/astroidmail/astroid/issues/414 * https://github.com/astroidmail/astroid/blob/master/tests/test_notmuch_standalone.cc Presently, I cannot reproduce it myself, but seems to be fairly consistent for the users this happens with. Not sure if this is talloc_abort() anymore. Actually, at this point this seems to be caused by different GMime versions used for binary and notmuch library. Regards, Gaute ___ notmuch mailing list notmuch@notmuchmail.org https://notmuchmail.org/mailman/listinfo/notmuch
Re: talloc_abort in notmuch_thread_get_tags () when db has been modified
David Bremner writes on februar 17, 2017 16:35: Gaute Hope writes: threads != NULL terminate called after throwing an instance of 'Xapian::DatabaseModifiedError' Yeah, that looks like a different problem. But it _should_ be something we can catch in libnotmuch. For reference; I'm getting several reports of this or similar error: * https://github.com/astroidmail/astroid/issues/414 * https://github.com/astroidmail/astroid/blob/master/tests/test_notmuch_standalone.cc Presently, I cannot reproduce it myself, but seems to be fairly consistent for the users this happens with. Not sure if this is talloc_abort() anymore. Regards, Gaute ___ notmuch mailing list notmuch@notmuchmail.org https://notmuchmail.org/mailman/listinfo/notmuch
Announcing Astroid v0.10.2
Astroid v0.10.2 has been released! * Fix build issues on Mac OS. * Gmime 2 used weird timezone for sent email. * Add Meson build system as alternative. Regards, Gaute Gaute Hope writes on september 21, 2017 12:34: Astroid v0.10 has been released! Astroid is a lightweight and fast graphical threads-with-tags email client for notmuch. Written in C++ using GTK+, WebKit and gmime. Astroid can be acquired, along with usage and instructions for use, at: https://github.com/astroidmail/astroid pgpFnyWK5u_e5.pgp Description: PGP signature ___ notmuch mailing list notmuch@notmuchmail.org https://notmuchmail.org/mailman/listinfo/notmuch
[github-thread] find and open corresponding github page for thread or message
Hi, this small script finds and opens the corresponding GitHub issue, comment, or pull-request. https://gist.github.com/gauteh/de546cb143ebae4a489a482a05ab4beb $ ./github-thread.py notmuch-thread-id [message-id] the first argument is always required. If a message id is specified, it first tries to scroll to that specific message. Note: This will work more reliably on newer issues since they contain a nice JSON packet. Requires: * Python >= 3.3 * Notmuch Python bindings Regards, Gaute pgpAt0CDPeYz4.pgp Description: PGP signature ___ notmuch mailing list notmuch@notmuchmail.org https://notmuchmail.org/mailman/listinfo/notmuch
Announcing Astroid v0.10.1
Gaute Hope writes on september 21, 2017 12:34: Greetings, Astroid v0.10 has been released! Astroid is a lightweight and fast graphical threads-with-tags email client for notmuch. Written in C++ using GTK+, WebKit and gmime. Astroid can be acquired, along with usage and instructions for use, at: https://github.com/astroidmail/astroid Astroid v0.10.1 has been released, this minor release fixes some compilation errors against different versions of notmuch and GMime. - gaute pgpVnBKykauJj.pgp Description: PGP signature ___ notmuch mailing list notmuch@notmuchmail.org https://notmuchmail.org/mailman/listinfo/notmuch
Announcing Astroid v0.10
Greetings, Astroid v0.10 has been released! Astroid is a lightweight and fast graphical threads-with-tags email client for notmuch. Written in C++ using GTK+, WebKit and gmime. Astroid can be acquired, along with usage and instructions for use, at: https://github.com/astroidmail/astroid once you get Astroid running press '?' to get a list of keybindings for the current context. For 'The Tour' check out: https://github.com/astroidmail/astroid/wiki P.S.: Make sure you use the same GMime for notmuch and astroid! ## Some of the changes since v0.9.1 * Support GMime 3 with backwards compatability with GMime 2. This orients us towards GMime 3, and support for GMime 2 will be dropped in the future. * Attach messages by mids with 'A' in new emails. * Colorful tags in tag editor and in searching * Tags in the tag editor is now separated by spaces (comma still works) * New configuration option 'signature_separate' determines if inline signatures will be separated from the message by a line containing '-- '. * Configurable query associated with account can be used to select account as From: when replying or forwarding e-mail. * Default clipboard target: CLIPBOARD since this is the only one accessible by keyboard in GTK. * Select target for yank operations with '"' * Track focus changes manually Regards, Gaute pgp5DV2PjuYgF.pgp Description: PGP signature ___ notmuch mailing list notmuch@notmuchmail.org https://notmuchmail.org/mailman/listinfo/notmuch
Re: [PATCH] python: Fix database.add_message
David Bremner writes on september 12, 2017 3:56: Dylan Baker writes: b10ce6bc23002d48916b1b2f375480e7540e3164 introduced a new index_file method as a replacement for add_message. It helpfully made add_message an alias for index_file, unfortunately it got the signature wrong by explicitly passing self as an argument. cc: Daniel Kahn Gillmor Thanks. We have another version that I was about to commit https://www.mail-archive.com/notmuch@notmuchmail.org/msg45038.html The only difference is that Gaute (more or less by request) added a deprecation warning. I don't really know how warnings in python work, does that sound sensible? Yup, also notice the missing `return`. Regards, Gaute ___ notmuch mailing list notmuch@notmuchmail.org https://notmuchmail.org/mailman/listinfo/notmuch
Re: [PATCH] WIP: add thread subqueries.
David Bremner writes on august 21, 2017 3:35:
David Bremner writes:
If you're interested in this feature, please test, let me know if it's
workable for your use case. The syntax here is thread:{$query}. The
usual caveats about whitespace apply. The running example is
thread:{from:me} and thread:{from:jian} and not thread:{from:dave}
I was playing around with this, and
'thread:"{from:bremner and date:2017}"' and thread:{from:sitaram}
is about 10x faster than without the date restriction. This is roughly
proportional to the number of threads matching the first subquery.
Unfortunately the date restriction needs to be inside the subquery with
the current design.
So just to clarify; to make true full thread searches there cannot be more than
one term in each thread:{} query? Otherwise, all terms in each thread: query
must be found in at least one message at the same time. So you could put the
date: restriction in a separate thread:{}, but it would be slower?
-gaute
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch
[PATCH v2] python: deprecated add_message calls index_file correctly and returns result
The deprecated Database.add_message now calls the new index_file with correct number of arguments (without an extra `self`), and returns the tuple from index_file - as it used to do before. This change also adds a DeprecationWarning to the function. --- bindings/python/notmuch/database.py | 6 +- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/bindings/python/notmuch/database.py b/bindings/python/notmuch/database.py index a2c025eb..1279804a 100644 --- a/bindings/python/notmuch/database.py +++ b/bindings/python/notmuch/database.py @@ -19,6 +19,7 @@ Copyright 2010 Sebastian Spaeth import os import codecs +import warnings from ctypes import c_char_p, c_void_p, c_uint, byref, POINTER from .compat import SafeConfigParser from .globals import ( @@ -471,7 +472,10 @@ class Database(object): def add_message(self, filename, sync_maildir_flags=False): """Deprecated alias for :meth:`index_file` """ -self.index_file(self, filename, sync_maildir_flags=sync_maildir_flags) +warnings.warn( +"This function is deprecated and will be removed in the future, use index_file.", DeprecationWarning) + +return self.index_file(filename, sync_maildir_flags=sync_maildir_flags) _remove_message = nmlib.notmuch_database_remove_message _remove_message.argtypes = [NotmuchDatabaseP, c_char_p] -- 2.14.1 ___ notmuch mailing list notmuch@notmuchmail.org https://notmuchmail.org/mailman/listinfo/notmuch
Re: [PATCH] python: deprecated add_message calls index_file correctly and returns result
Tomi Ollila writes on august 30, 2017 0:00: On Mon, Aug 28 2017, Gaute Hope wrote: The deprecated Database.add_message now calls the new index_file with correct number of arguments (without an extra `self`), and returns the tuple from index_file - as it used to do before. Instead of this change, should we just dump the whole function... ... although this looks good what message are we giving by committing changes to deprecated functions (unless this is not deprecated enough ;) Ok.. would that not be an argument against keeping any deprecated functions? Then n_d_add_message should perhaps be removed as well. It would be a good idea to use `warnings` and `DeprecationWarning` here. This deprecation-function wrapper is flawed though: the function signature is changed from the original, and it does not correctly call the new function. It is not valid python code. Original change: id:20170817231426.9627-2-...@fifthhorseman.net Regards, Gaute ___ notmuch mailing list notmuch@notmuchmail.org https://notmuchmail.org/mailman/listinfo/notmuch
[PATCH] python: deprecated add_message calls index_file correctly and returns result
The deprecated Database.add_message now calls the new index_file with correct number of arguments (without an extra `self`), and returns the tuple from index_file - as it used to do before. --- bindings/python/notmuch/database.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/bindings/python/notmuch/database.py b/bindings/python/notmuch/database.py index a2c025eb..6199f58b 100644 --- a/bindings/python/notmuch/database.py +++ b/bindings/python/notmuch/database.py @@ -471,7 +471,7 @@ class Database(object): def add_message(self, filename, sync_maildir_flags=False): """Deprecated alias for :meth:`index_file` """ -self.index_file(self, filename, sync_maildir_flags=sync_maildir_flags) +return self.index_file(filename, sync_maildir_flags=sync_maildir_flags) _remove_message = nmlib.notmuch_database_remove_message _remove_message.argtypes = [NotmuchDatabaseP, c_char_p] -- 2.14.1 ___ notmuch mailing list notmuch@notmuchmail.org https://notmuchmail.org/mailman/listinfo/notmuch
Re: [PATCH] python: open messages in binary mode
David Bremner writes on august 25, 2017 0:11: Florian Klink writes: To fix this, read file in binary mode and pass to email.message_from_binary_file(fp). Thanks for the patch, but notmuch is not (yet) python3 only. Apparently that function is only since python 3.2. I'm not sure if/when we'll drop python 2.7 support, but not without deprecating it for a few releases. Is there anyone still exclusively on Python 2.7? Perhaps the time is ripe for starting that process? Encoding compatability is an unholy mess to maintain for one Python distro. Is any of alot, afew, etc still on Python 2 only? Regards, Gaute ___ notmuch mailing list notmuch@notmuchmail.org https://notmuchmail.org/mailman/listinfo/notmuch
[Gmailieer] v0.2 released
Hi, this marks the release of version 0.2 of gmailieer - a fast fetch and two-way tag synchronization program for notmuch and GMail messages and tags: https://github.com/gauteh/gmailieer 'gmailieer' (or 'gmi') is a small program pulls email and labels (and changes to labels) from your GMail account. These are stored locally in a maildir with the labels synchronized with the tags in a notmuch database. The changes you make to tags in the notmuch database may be pushed back to your GMail account. 'gmailieer' is in use by several users daily, and has made almost a million request a day (peak) lately. Since the previous relase [0] we now * Do most push request in batch requests, together with exponential backoff and other error handling, this makes the synchronization much more robust. * Sync SPAM and TRASH messages as well (make sure to do a full gmi pull -f to get older SPAM and TRASH messages). * Made setup much easier since you do not need to create your own API key: $ gmi init your.acco...@gmail.com [... authorize ..] $ gmi pull to sync: $ gmi sync Regards, Gaute [0] id:1489401808.6fnnjm7u8s.astr...@strange.none pgp6sBI2sLNGs.pgp Description: PGP signature ___ notmuch mailing list notmuch@notmuchmail.org https://notmuchmail.org/mailman/listinfo/notmuch
Re: find threads where I and Jian participated but not Dave
Gaute Hope writes on juni 22, 2017 8:08: Daniel Kahn Gillmor writes on juni 21, 2017 23:30: On Wed 2017-06-21 13:04:53 -0700, Matt Armstrong wrote: For what it is worth, I've found this idea from Daniel intriguing and pretty useful in practice: "show me threads in which i've participated, where there are some messages flagged with 'inbox'" I implement it like this in my post-new hook: # All messages in threads in which I participate get tag:participated notmuch search --output=threads from:marmstrong | \ sed -e 's,^,+participated -- ,' | \ notmuch tag --batch cool, thx for the suggestion. the "notmuch search" part of the pipeline alone takes ~19s (wall time, and actual CPU time) for me though :/ It returns 30504 threads! how many threads do you get? Is there any reason why you do not filter on a tag 'new' as well? notmuch search --output=threads from:marmstrong and tag:new | \ sed -e 's,^,+participated -- ,' | \ notmuch tag --batch Nevermind, I get it - it might be possible to add a temporary tag new-tag to the whole thread first and not just new messages. That might be faster. As long as all sent messages get the new tag as well. pgpIkGDZ3Jptl.pgp Description: PGP signature ___ notmuch mailing list notmuch@notmuchmail.org https://notmuchmail.org/mailman/listinfo/notmuch
Re: find threads where I and Jian participated but not Dave
Daniel Kahn Gillmor writes on juni 21, 2017 23:30: On Wed 2017-06-21 13:04:53 -0700, Matt Armstrong wrote: For what it is worth, I've found this idea from Daniel intriguing and pretty useful in practice: "show me threads in which i've participated, where there are some messages flagged with 'inbox'" I implement it like this in my post-new hook: # All messages in threads in which I participate get tag:participated notmuch search --output=threads from:marmstrong | \ sed -e 's,^,+participated -- ,' | \ notmuch tag --batch cool, thx for the suggestion. the "notmuch search" part of the pipeline alone takes ~19s (wall time, and actual CPU time) for me though :/ It returns 30504 threads! how many threads do you get? Is there any reason why you do not filter on a tag 'new' as well? notmuch search --output=threads from:marmstrong and tag:new | \ sed -e 's,^,+participated -- ,' | \ notmuch tag --batch Regards, Gaute pgpUmguMAhigd.pgp Description: PGP signature ___ notmuch mailing list notmuch@notmuchmail.org https://notmuchmail.org/mailman/listinfo/notmuch
Re: find threads where I and Jian participated but not Dave
David Bremner writes on juni 15, 2017 22:20: Daniel Kahn Gillmor writes: One of my long-standing wishes is to be able to say "show me mails in my inbox from people who have replied to messages i've sent them". This could be re-framed as "show me threads in which i've participated, where there are some messages flagged with 'inbox'". but generating a huge list of all threads in which i've participated, just to be able to do an intersection operation with a (much smaller) list of all threads that have a message with the inbox flag seems like a pretty gross inefficiency. At the moment the best we could do is essentially the same algorithm, but in C instead of shell / python. Threads are not documents in the database, so they can't efficiently be searched for. Of course we could change that, but those kind of changes take a fair amount of effort, and some careful design work. There are probably multiple earlier references to this, but here's one: id:1471858269.x2m28lgosh.astroid@strange matching against the whole thread vs the individual messages would be very useful! Regards, Gaute ___ notmuch mailing list notmuch@notmuchmail.org https://notmuchmail.org/mailman/listinfo/notmuch
[gmailieer] [release v0.1] Fast fetch and two-way tag synchronization between notmuch and GMail
Hi, gmailieer now has its first release! 'gmailieer' (or 'gmi') is a small program that can pull email and labels from your GMail account and store them locally in a maildir with the labels synchronized with a notmuch database. The changes to tags in the notmuch database may be pushed back to your GMail account. Main new features are: * Exponential backoff on most remote requests making gmi more robust when synchronizing lots of changes. * An API key is included so you can get started right away without getting any keys from Google. Instructions and source code can be found here: https://github.com/gauteh/gmailieer Disclaimer: This is still experimental, but gmi does not have access to delete e-mail on your remote account - only fetch and change labels, so damage should be limited. Thanks to several contributors for bug-fixes and debugging! Regards, Gaute ___ notmuch mailing list notmuch@notmuchmail.org https://notmuchmail.org/mailman/listinfo/notmuch
[gmailieer] [release v0.1] Fast fetch and two-way tag synchronization between notmuch and GMail
Hi, gmailieer now has its first release! 'gmailieer' (or 'gmi') is a small program that can pull email and labels from your GMail account and store them locally in a maildir with the labels synchronized with a notmuch database. The changes to tags in the notmuch database may be pushed back to your GMail account. Main new features are: * Exponential backoff on most remote requests making gmi more robust when synchronizing lots of changes. * An API key is included so you can get started right away without getting any keys from Google. Instructions and source code can be found here: https://github.com/gauteh/gmailieer Disclaimer: This is still experimental, but gmi does not have access to delete e-mail on your remote account - only fetch and change labels, so damage should be limited. Thanks to several contributors for bug-fixes and debugging! Regards, Gaute pgpHm3cJHrObx.pgp Description: PGP signature ___ notmuch mailing list notmuch@notmuchmail.org https://notmuchmail.org/mailman/listinfo/notmuch
Announcing Astroid v0.9
Greetings, Astroid v0.9 has been released! Astroid is a lightweight and fast graphical threads-with-tags email client for notmuch. Written in C++ using GTK+, WebKit and gmime. Astroid can be acquired at: $ git clone https://github.com/astroidmail/astroid.git Usage and instructions: https://github.com/astroidmail/astroid#astroid-mua once you get Astroid running press '?' to get a list of keybindings for the current context. For 'The Tour' check out: https://github.com/astroidmail/astroid/wiki ( P.S: We now also have a homebrew tap for Mac OS users: https://github.com/astroidmail/homebrew-astroid !) ## Changes since v0.8 * Archive or close thread and then directly open next or next unread thread in originating thread-index. * Drop use of old deprecated non _st functions, use their new names. Will require 0.24 or newer. * No longer support pre-lastmod notmuches. * Catch missing content errors for messages, not fail. * Wait and block R/O database opening, not fail. * Defer changed_thread events on thread-index while still loading. * Do not update tags in thread-view when not ready. * Configuration option thread_view.expand_flagged determines whether flagged messages are expanded by default. * Better handle poll thread. Make sure to clean up connections here and in send_message. * Polling can be cancelled with C-c in main window. Regards, Gaute pgpVm2osPKhan.pgp Description: PGP signature ___ notmuch mailing list notmuch@notmuchmail.org https://notmuchmail.org/mailman/listinfo/notmuch
Re: [gmailieer] fast fetch and two-way tag synchronization between notmuch and GMail
Rafael Avila de Espindola writes on april 3, 2017 16:35: After a few issues with the initial sync this is working perfectly. Thanks a lot, it is a big improvement over mbsync. Great, thanks for the patches! This made inital sync much more robust. I have a non-user-api-key version working [0], but I need to figure out if it is safe first [1]. - Gaute [0] https://github.com/gauteh/gmailieer/pull/9 [1] http://stackoverflow.com/questions/43173367/is-it-safe-to-distribute-client-id-and-client-secret-for-google-api-for-an-insta pgpza2ufJQ1UL.pgp Description: PGP signature ___ notmuch mailing list notmuch@notmuchmail.org https://notmuchmail.org/mailman/listinfo/notmuch
Re: [gmailieer] fast fetch and two-way tag synchronization between notmuch and GMail
Rafael Espíndola writes on mars 27, 2017 15:23: $ gmi pull # does the initial pull, fetching all you messages - this # could take some time. It is currently running. Thank you so much. The only things I noticed so far: * running auth before init fails, but running init -c ... -a ... works * I had to name the auth file client_secret.json or "gmi pull" would fail trying to read it. Right, that should be the other way around. I'll update the README. There are two secret files. One is the API client_secret which can be specified with the '-c' option (also for pull), this one is only needed initially and can otherwise be stored somewhere else: When authenticating (either running 'auth' or some operation requiring authentication: pull/push) a local access token is retrieved, this is stored in: .credentials.gmailieer.json in the local mail repository. gmail. My previous setup was to use mbsync. It is really fast for fetching new emails (--pull-new), but with imap the only way to get flag changes (including read/unread) seems to be to ask for the flag of every message! Thanks! - gaute ___ notmuch mailing list notmuch@notmuchmail.org https://notmuchmail.org/mailman/listinfo/notmuch
Re: [gmailieer] fast fetch and two-way tag synchronization between notmuch and GMail
Rafael Avila de Espindola writes on mars 27, 2017 4:32: I currently have a mbsync + notmuch setup and I am just about to give this a try. One question is what the notmuch setup should look like. I assume I should create a new config file pointing to the directory where the emails will be written to. Do I have to run notmuch setup to create the db? Yes, it is a good idea to just back up your entire notmuch db and set up a new one to test gmailieer with. gmailieer respects $NOTMUCH_CONFIG and reads it to figure out which 'new' tags to add (no need to use inbox or unread here, that will only create confusion - I only use 'new' to do tagging). Assuming you want to set up a separate notmuch database to test you do the following: first: get a client secret API key for gmailieer [0], you only need this during setup or if gmailieer needs to be re-authenticated. so store it somewhere safe, say: ~/.secret/client_secret.json. ideally this should be unnecessary [1]. # setting up a separate notmuch db $ mkdir ~/test_mail/ # our new mail root $ cd ~/test_mail $ export NOTMUCH_CONFIG=$(pwd)/.notmuch-config $ mkdir mail # new mail database root $ notmuch setup * specify ~/test_mail/mail as mail database * specify only 'new' as new tag $ notmuch new # create db $ cd mail NOTE: As long as you are using a non-standard notmuch config location make sure that environment variable is set when using gmailieer. # setting up gmailieer $ mkdir gmail * this will be the gmailieer local repository. * it should be a sub-dir of the root mail db. * you can keep other mail (from other accounts) outside this gmailieer local repository. $ cd gmail $ gmi auth -c ~/.secret/client_secret.json * authenticate gmailieer, choose the correct account, say: charlie.r...@gmail.com $ gmi init -a charlie.r...@gmail.com # gmailieer should now be set up $ gmi pull # does the initial pull, fetching all you messages - this # could take some time. When you want to synchronize your account you do: $ gmi sync# which effectively does gmi push; gmi pull this will also update the notmuch database so it should not be necessary to run 'notmuch new' afterwards. Regards, Gaute [0] https://console.developers.google.com/flows/enableapi?apiid=gmail [1] https://github.com/gauteh/gmailieer/issues/1 ___ notmuch mailing list notmuch@notmuchmail.org https://notmuchmail.org/mailman/listinfo/notmuch
Announcing Astroid v0.8
Greetings,
Astroid v0.8 is now ready!
Astroid is a lightweight and fast graphical threads-with-tags email
client for notmuch. Written in C++ using GTK+, WebKit and gmime.
Astroid can be found at:
$ git clone https://github.com/astroidmail/astroid.git
## Usage and instructions
https://github.com/astroidmail/astroid#astroid-mua
once you get Astroid running press '?' to get a list of keybindings
for the current context.
and The Tour:
https://github.com/astroidmail/astroid/wiki
## Changes since v0.7
* New logo and icons by Elio Qoshi ( http://patreon.com/ura )
* Run hook for thread-view (Hugo Roy)
* Session-persistent folder history for attaching files and saving
attachments. default is configurable.
* Remove duplicates when reply-all'ing.
* Handle mailto when astroid is already running.
* `scons --propagate-environment` uses environment from build environment.
* Show pane-focus with blue bar at top, swap focus with M-space. The
keybinding is now located at 'pane.swap_focus'.
* Add optional drop-down terminal ('|') which has session-persistent
working directory. Will silently be disabled if VTE is missing.
* Delay sending of message with configured time, in order to be able to
cancel wrongly sent messages.
* New --start-polling, --stop-polling and --refresh LASTMOD options which
makes external polling mechanisms work better.
* All notmuch tags are added to message CSS with 'nm-' prefix and some
sanitation to allow thread-view customizing based on tag.
* Reply-To mailing list.
* Fix crypto and performance bugs among many others.
Regards, Gaute
pgpuEeKFeN_lG.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch
Re: [gmailieer] fast fetch and two-way tag synchronization between notmuch and GMail
Rafael Avila de Espindola writes on mars 14, 2017 16:51: That looks really interesting. What is the license? :-) Ay, GPL v3+ it is! - gaute ___ notmuch mailing list notmuch@notmuchmail.org https://notmuchmail.org/mailman/listinfo/notmuch
[gmailieer] fast fetch and two-way tag synchronization between notmuch and GMail
Hi, 'gmailieer' (or 'gmi') is a small program that can pull email and labels (and changes to labels) from your GMail account and store them locally in a maildir with the labels synchronized with a notmuch database. The changes to tags in the notmuch database may be pushed back remotely to your GMail account. The initial fetch of all emails takes some time, but synchronizing labels and tags, and checking for new messages, is usually done in 1-2 seconds. It requires the most recent notmuch, the python googleapi bindings and tqdm. Disclaimer: This is still experimental, but it does not have access to delete e-mail on your account - only fetch and change labels, so damage should be limited. Instructions and source code can be found here: https://github.com/gauteh/gmailieer Regards, Gaute pgp8j5fPpnYJH.pgp Description: PGP signature ___ notmuch mailing list notmuch@notmuchmail.org https://notmuchmail.org/mailman/listinfo/notmuch
[PATCH] bindings/python: test python Database.get_revision ()
Here is a test for Database.get_revision (), based on
the first part of T570-revision-tracking.sh.
---
test/T390-python.sh | 19 +++
1 file changed, 19 insertions(+)
diff --git a/test/T390-python.sh b/test/T390-python.sh
index 4726bc36..b27f0740 100755
--- a/test/T390-python.sh
+++ b/test/T390-python.sh
@@ -38,4 +38,23 @@ print (db.find_message_by_filename("i-dont-exist"))
EOF
test_expect_equal "$(cat OUTPUT)" "None"
+test_begin_subtest "get revision"
+test_python ${MAIL_DIR} <<'EOF'
+import notmuch
+db = notmuch.Database(mode=notmuch.Database.MODE.READ_ONLY)
+(revision, uuid) = db.get_revision()
+print ("%s\t%lu" % (uuid, revision))
+EOF
+notmuch_uuid_sanitize < OUTPUT > CLEAN
+cat <<'EOF' >EXPECTED
+UUID 53
+EOF
+test_expect_equal_file EXPECTED CLEAN
+
+grep '^[0-9a-f]' OUTPUT > INITIAL_OUTPUT
+
+test_begin_subtest "output of count matches test code"
+notmuch count --lastmod '*' | cut -f2-3 > OUTPUT
+test_expect_equal_file INITIAL_OUTPUT OUTPUT
+
test_done
--
2.11.1
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch
[PATCH] bindings/python: add bindings for notmuch_database_get_revision
Database.get_revision () returns a tuple with the current database
revision and the UUID string representing the database.
---
bindings/python/notmuch/database.py | 16
1 file changed, 16 insertions(+)
diff --git a/bindings/python/notmuch/database.py
b/bindings/python/notmuch/database.py
index 67fb1c41..8f918069 100644
--- a/bindings/python/notmuch/database.py
+++ b/bindings/python/notmuch/database.py
@@ -86,6 +86,11 @@ class Database(object):
_get_version.argtypes = [NotmuchDatabaseP]
_get_version.restype = c_uint
+"""notmuch_database_get_revision"""
+_get_revision = nmlib.notmuch_database_get_revision
+_get_revision.argtypes = [NotmuchDatabaseP, POINTER(c_char_p)]
+_get_revision.restype = c_uint
+
"""notmuch_database_open"""
_open = nmlib.notmuch_database_open
_open.argtypes = [c_char_p, c_uint, POINTER(NotmuchDatabaseP)]
@@ -261,6 +266,17 @@ class Database(object):
self._assert_db_is_initialized()
return Database._get_version(self._db)
+def get_revision (self):
+"""Returns the committed database revison and UUID
+
+:returns: (revison, uuid) The database revision as a positive integer
+and the UUID of the database.
+"""
+self._assert_db_is_initialized()
+uuid = c_char_p ()
+revision = Database._get_revision(self._db, byref (uuid))
+return (revision, uuid.value.decode ('utf-8'))
+
_needs_upgrade = nmlib.notmuch_database_needs_upgrade
_needs_upgrade.argtypes = [NotmuchDatabaseP]
_needs_upgrade.restype = bool
--
2.11.1
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch
Re: [RFC patch 2/2] lib: handle DatabaseModifiedError in _n_message_ensure_metadata
David Bremner writes on februar 24, 2017 3:49: David Bremner writes: The error handling here still needs work. The retry count should be handled in more sane way, and both running out of retries and an error return from notmuch_database_reopen should be handled. Probably the number of retries can be limited to 1, since afaict, reopen is not going to return any recoverable errors. Does _reopen block if the db is locked? ___ notmuch mailing list notmuch@notmuchmail.org https://notmuchmail.org/mailman/listinfo/notmuch
Re: add status value to _notmuch_message_ensure_metadata
David Bremner writes on februar 23, 2017 1:58: Gaute Hope writes: Gaute Hope writes on februar 20, 2017 10:27: David Bremner writes on februar 18, 2017 15:45: In id:1487339566.mz8acpov1j.astr...@strange.none , Gaute provided a traceback of an uncaught Xapian::DatabaseModifiedError. The fix for this is simple, but somewhat intrusive. [...] I haven't tested against Gaute's test case (needs more boost than I have handy). Alright then, attached is a non-boost version that takes a notmuch db path (absolute) as the first argument (no warranty). With the patches above this crashes in a predictable / preventable way, because notmuch_message_get_tags returns NULL. It isn't clear to me yet what the best API choice is here: minimize difference with the old API by returning NULL to indicate errors, or switch completely to the pattern of e.g. notmuch_query_search_messages_st. I suppose we could do something along the same lines and add new _st versions of the problematic functions Hi, Ideally if the error could be caught in `notmuch_threads_valid` or `notmuch_threads_get_thread` I think that would be the clearest, _st versions would be nice. As I mentioned in id:1487582192.57s86yczcg.astr...@strange.none it seems that at later arbitrary iterations (without re-loading the threads object) the functions return valid data (even `notmuch_thread_get_tags` does). Can this data be trusted? I feel like this should all be invalid at this point. Regards, Gaute ___ notmuch mailing list notmuch@notmuchmail.org https://notmuchmail.org/mailman/listinfo/notmuch
Re: add status value to _notmuch_message_ensure_metadata
Gaute Hope writes on februar 20, 2017 10:27:
David Bremner writes on februar 18, 2017 15:45:
In id:1487339566.mz8acpov1j.astr...@strange.none , Gaute provided a
traceback of an uncaught Xapian::DatabaseModifiedError. The fix for
this is simple, but somewhat intrusive.
[...]
I haven't tested against Gaute's test case (needs more boost than I
have handy).
Alright then, attached is a non-boost version that takes a notmuch db
path (absolute) as the first argument (no warranty).
- gaute
# include
# include
# include
using std::cout;
using std::endl;
int main (int argc, char **argv) {
if (argc < 2) {
std::cerr << "error: specify the path to the test notmuch database" << endl;
return 1;
}
std::string path_db;
path_db = argv[1];
cout << "using db: " << path_db << endl;
notmuch_database_t * nm_db;
notmuch_status_t s =
notmuch_database_open (
path_db.c_str(),
notmuch_database_mode_t::NOTMUCH_DATABASE_MODE_READ_ONLY,
&nm_db);
cout << "db: running test query.." << endl;
notmuch_query_t * q = notmuch_query_create (nm_db, "*");
unsigned int c;
notmuch_status_t st = notmuch_query_count_threads_st (q, &c); // destructive
notmuch_query_destroy (q);
q = notmuch_query_create (nm_db, "*");
cout << "query: " << notmuch_query_get_query_string (q) << ", approx: "
<< c << " threads." << endl;
notmuch_threads_t * threads;
notmuch_thread_t * thread;
st = notmuch_query_search_threads_st (q, &threads);
std::string thread_id;
int i = 0;
for (; notmuch_threads_valid (threads);
notmuch_threads_move_to_next (threads)) {
thread = notmuch_threads_get (threads);
i++;
if (i == 3)
thread_id = notmuch_thread_get_thread_id (thread);
notmuch_thread_destroy (thread);
if (i == 3) break;
}
cout << "thread id to change: " << thread_id << ", thread no: " << i << endl;
notmuch_query_destroy (q);
/* restart query */
cout << "restarting query.." << endl;
q = notmuch_query_create (nm_db, "*");
st = notmuch_query_search_threads_st (q, &threads);
i = 0;
int stop = 2;
cout << "moving to thread: " << stop << endl;
for ( ; notmuch_threads_valid (threads);
notmuch_threads_move_to_next (threads))
{
thread = notmuch_threads_get (threads);
notmuch_thread_get_thread_id (thread);
i++;
cout << "tags: ";
/* get tags */
notmuch_tags_t *tags;
const char *tag;
for (tags = notmuch_thread_get_tags (thread);
notmuch_tags_valid (tags);
notmuch_tags_move_to_next (tags))
{
tag = notmuch_tags_get (tags);
cout << tag << " ";
}
cout << endl;
notmuch_thread_destroy (thread);
if (i == stop) break;
}
/* now open a new db instance, modify the already loaded thread and
* continue loading the original query */
notmuch_database_t * nm_db2;
s = notmuch_database_open (
path_db.c_str(),
notmuch_database_mode_t::NOTMUCH_DATABASE_MODE_READ_WRITE,
&nm_db2);
char qry_s[256];
sprintf (qry_s, "thread:%s", thread_id.c_str ());
notmuch_query_t * q2 = notmuch_query_create (nm_db2, qry_s);
notmuch_threads_t * ts2;
notmuch_thread_t * t2;
st = notmuch_query_search_threads_st (q2, &ts2);
for ( ; notmuch_threads_valid (ts2);
notmuch_threads_move_to_next (ts2))
{
t2 = notmuch_threads_get (ts2);
std::string thread_id = notmuch_thread_get_thread_id (t2);
/* remove unread tag */
notmuch_messages_t * ms = notmuch_thread_get_messages (t2);
notmuch_message_t * m;
for (; notmuch_messages_valid (ms); notmuch_messages_move_to_next (ms)) {
m = notmuch_messages_get (ms);
st = notmuch_message_remove_tag (m, "unread");
notmuch_message_destroy (m);
}
notmuch_messages_destroy (ms);
notmuch_thread_destroy (t2);
break;
}
notmuch_query_destroy (q2);
notmuch_database_close (nm_db2);
/* re-add unread tag */
s = notmuch_database_open (
path_db.c_str(),
notmuch_database_mode_t::NOTMUCH_DATABASE_MODE_READ_WRITE,
&nm_db2);
q2 = notmuch_query_create (nm_db2, qry_s);
st = notmuch_query_search_threads_st (q2, &ts2);
for ( ; notmuch_threads_valid (ts2);
notmuch_threads_move_to_next (ts2))
{
t2 = notmuch_threads_get (ts2);
std::string thread_id = notmuch_thread_get_thread_id (t2);
/* remove unread tag */
notmuch_messages_t * ms = notmuch_thread_get_messages (t2);
notmuch_message_t * m;
for (; notmuch_messages_valid (ms); notmuch_messages_move_to_next (ms)) {
m = n
Re: add status value to _notmuch_message_ensure_metadata
David Bremner writes on februar 18, 2017 15:45:
In id:1487339566.mz8acpov1j.astr...@strange.none , Gaute provided a
traceback of an uncaught Xapian::DatabaseModifiedError. The fix for
this is simple, but somewhat intrusive.
[...]
I haven't tested against Gaute's test case (needs more boost than I
have handy).
Hi, thanks this seems to get rid of the hard crash, the output from my test
case is attached below.
I have not yet tried to use any new status returns, it seems that the
immediate result is that `notmuch_threads_get ()` returns a valid (but
empty?) notmuch_thread_t * which results in
`notmuch_thread_get_thread_id ()` returning NULL. Strangely, at some
seemingly arbitrary intervals the methods seem to work correctly and
return a valid `notmuch_thread_t` struct.
`threads` and `thread` remain not NULL.
```
$ test/test_notmuch_standalone
db: running test query..
query: *, approx: 10 threads.
thread id to change: 0002, thread no: 3
restarting query..
moving to thread: 2
tags: inbox unread
tags: inbox unread
continue loading..
threads != NULL
thread != NULL
loading: 2:
tags:
threads != NULL
thread != NULL
loading: 3: 0009
tags: inbox unread
threads != NULL
thread != NULL
loading: 4:
tags:
threads != NULL
thread != NULL
loading: 5:
tags:
threads != NULL
thread != NULL
loading: 6: 0004
tags: inbox unread
threads != NULL
thread != NULL
loading: 7: 0006
tags: inbox unread
threads != NULL
thread != NULL
loading: 8: 000a
tags: inbox signed unread
threads != NULL
thread != NULL
loading: 9: 0005
tags: inbox unread
```
with the following changes made to the test program:
```
diff --git a/test/test_notmuch_standalone.cc b/test/test_notmuch_standalone.cc
index 5ef0a00..4634986 100644
--- a/test/test_notmuch_standalone.cc
+++ b/test/test_notmuch_standalone.cc
@@ -189,8 +189,16 @@ int main () {
cout << "threads != NULL" << endl;
}
thread = notmuch_threads_get (threads);
+if (thread == NULL) {
+ cout << "thread == NULL" << endl;
+} else {
+ cout << "thread != NULL" << endl;
+}
+
cout << "loading: " << i;
-std::string tid = notmuch_thread_get_thread_id (thread);
+const char * cid = notmuch_thread_get_thread_id (thread);
+std::string tid = "";
+if (cid != NULL) tid = cid;
cout << ": " << tid << endl;
/* get tags */
```
- gaute
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch
Re: talloc_abort in notmuch_thread_get_tags () when db has been modified
David Bremner writes on februar 17, 2017 13:28: Gaute Hope writes: David Bremner writes on mars 7, 2016 13:01: Gaute Hope writes: Of course _why_ this error is happening could still be notmuch's fault. Can you reproduce the problem under valgrind? Hi again, For future reference: Attached is C++ test code that demonstrates the problem (at least on my setup). It is part of the astroid test suite. And did you try running this under valgrind? ``` $ valgrind test/test_notmuch_standalone ==9543== Memcheck, a memory error detector ==9543== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al. ==9543== Using Valgrind-3.12.0 and LibVEX; rerun with -h for copyright info ==9543== Command: test/test_notmuch_standalone ==9543== db: running test query.. query: *, approx: 10 threads. thread id to change: 0002, thread no: 3 restarting query.. moving to thread: 2 tags: unread tags: inbox continue loading.. threads != NULL terminate called after throwing an instance of 'Xapian::DatabaseModifiedError' ==9543== ==9543== Process terminating with default action of signal 6 (SIGABRT): dumping core ==9543==at 0xE46E04F: raise (in /usr/lib/libc-2.24.so) ==9543==by 0xE46F479: abort (in /usr/lib/libc-2.24.so) ==9543==by 0xD7494EC: __gnu_cxx::__verbose_terminate_handler() (vterminate.cc:95) ==9543==by 0xD7472A5: __cxxabiv1::__terminate(void (*)()) (eh_terminate.cc:47) ==9543==by 0xD7472F0: std::terminate() (eh_terminate.cc:57) ==9543==by 0xD747507: __cxa_throw (eh_throw.cc:87) ==9543==by 0xEEB987D: ??? (in /usr/lib/libxapian.so.30.2.0) ==9543==by 0xEEBC4A7: ??? (in /usr/lib/libxapian.so.30.2.0) ==9543==by 0xEEBEE14: ??? (in /usr/lib/libxapian.so.30.2.0) ==9543==by 0xEEBF0B7: ??? (in /usr/lib/libxapian.so.30.2.0) ==9543==by 0xEEBFF77: ??? (in /usr/lib/libxapian.so.30.2.0) ==9543==by 0xEE9539A: ??? (in /usr/lib/libxapian.so.30.2.0) ==9543== ==9543== HEAP SUMMARY: ==9543== in use at exit: 332,606 bytes in 1,171 blocks ==9543== total heap usage: 28,503 allocs, 27,332 frees, 3,835,392 bytes allocated ==9543== ==9543== LEAK SUMMARY: ==9543==definitely lost: 232 bytes in 1 blocks ==9543==indirectly lost: 285 bytes in 2 blocks ==9543== possibly lost: 8,577 bytes in 93 blocks ==9543==still reachable: 323,432 bytes in 1,074 blocks ==9543== of which reachable via heuristic: ==9543== newarray : 1,536 bytes in 16 blocks ==9543== suppressed: 0 bytes in 0 blocks ==9543== Rerun with --leak-check=full to see details of leaked memory ==9543== ==9543== For counts of detected and suppressed errors, rerun with: -v ==9543== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0) Aborted (core dumped) ``` The test-code must be adapted to your _test_ notmuch db. To pick up on this again, this issue started cropping up more frequently again, and I can't see a way currently to anticipate or recover from this from a user application of the notmuch library. There seems to be an XapianError, which may or may not be handled by notmuch. Previously you only reported a talloc error. Do you have a new stacktrace? Yeah - unsure if it is the same. ``` (gdb) r Starting program: /home/gaute/dev/mail/notm/astroid/test/test_notmuch_standalone [Thread debugging using libthread_db enabled] Using host libthread_db library "/usr/lib/libthread_db.so.1". db: running test query.. query: *, approx: 10 threads. thread id to change: 0002, thread no: 3 restarting query.. moving to thread: 2 tags: unread tags: inbox continue loading.. threads != NULL terminate called after throwing an instance of 'Xapian::DatabaseModifiedError' Program received signal SIGABRT, Aborted. 0x7fffee46b04f in raise () from /usr/lib/libc.so.6 (gdb) bt #0 0x7fffee46b04f in raise () at /usr/lib/libc.so.6 #1 0x7fffee46c47a in abort () at /usr/lib/libc.so.6 #2 0x7fffef2624ed in __gnu_cxx::__verbose_terminate_handler() () at /build/gcc-multilib/src/gcc/libstdc++-v3/libsupc++/vterminate.cc:95 #3 0x7fffef2602a6 in __cxxabiv1::__terminate(void (*)()) (handler=) at /build/gcc-multilib/src/gcc/libstdc++-v3/libsupc++/eh_terminate.cc:47 #4 0x7fffef2602f1 in std::terminate() () at /build/gcc-multilib/src/gcc/libstdc++-v3/libsupc++/eh_terminate.cc:57 #5 0x7fffef260508 in __cxxabiv1::__cxa_throw(void*, std::type_info*, void (*)(void*)) (obj=0xb0eff0, tinfo=0x7fffede100b8 , dest=0x7fffedaa7e30 ) at /build/gcc-multilib/src/gcc/libstdc++-v3/libsupc++/eh_throw.cc:87 #6 0x7fffedac587e in () at /usr/lib/libxapian.so.30 #7 0x7fffedac84a8 in () at /usr/lib/libxapian.so.30 #8 0x7fffedacae15 in () at /usr/lib/libxapian.so.30 #9 0x7fffedacb0b8 in () at /usr/lib/libxapian.so.30 #10 0x7fffedacbf78 in () at /usr/lib/libxapian.so.30 #11 0x7fffedaa139b in () at /usr/lib/libxapian.so.
Re: talloc_abort in notmuch_thread_get_tags () when db has been modified
David Bremner writes on mars 7, 2016 13:01:
Gaute Hope writes:
as far as I can see, there is _no_ way to catch this error without
completely crashing the application. I would have to isolate this code
in a separate process or trap SIGABRT (which is certainly messy).
I'm not sure what you expect libnotmuch to do here. There's a fatal
"should not happen" error in the memory allocator; it isn't really the
sort of thing one can recover from. It's also not in code we control.
Of course _why_ this error is happening could still be notmuch's
fault. Can you reproduce the problem under valgrind?
Hi again,
For future reference: Attached is C++ test code that demonstrates the problem
(at least on my setup). It is part of the astroid test suite.
The test-code must be adapted to your _test_ notmuch db.
To pick up on this again, this issue started cropping up more frequently
again, and I can't see a way currently to anticipate or recover from
this from a user application of the notmuch library. There seems to be
an XapianError, which may or may not be handled by notmuch.
Regards, Gaute
# include
# include
# include
namespace bfs = boost::filesystem;
using std::cout;
using std::endl;
int main () {
bfs::path path_db = bfs::absolute (bfs::path("./test/mail/test_mail"));
notmuch_database_t * nm_db;
notmuch_status_t s =
notmuch_database_open (
path_db.c_str(),
notmuch_database_mode_t::NOTMUCH_DATABASE_MODE_READ_ONLY,
&nm_db);
cout << "db: running test query.." << endl;
notmuch_query_t * q = notmuch_query_create (nm_db, "*");
unsigned int c;
notmuch_status_t st = notmuch_query_count_threads_st (q, &c); // destructive
notmuch_query_destroy (q);
q = notmuch_query_create (nm_db, "*");
cout << "query: " << notmuch_query_get_query_string (q) << ", approx: "
<< c << " threads." << endl;
notmuch_threads_t * threads;
notmuch_thread_t * thread;
st = notmuch_query_search_threads_st (q, &threads);
std::string thread_id;
int i = 0;
for (; notmuch_threads_valid (threads);
notmuch_threads_move_to_next (threads)) {
thread = notmuch_threads_get (threads);
i++;
if (i == 3)
thread_id = notmuch_thread_get_thread_id (thread);
notmuch_thread_destroy (thread);
if (i == 3) break;
}
cout << "thread id to change: " << thread_id << ", thread no: " << i << endl;
notmuch_query_destroy (q);
/* restart query */
cout << "restarting query.." << endl;
q = notmuch_query_create (nm_db, "*");
st = notmuch_query_search_threads_st (q, &threads);
i = 0;
int stop = 2;
cout << "moving to thread: " << stop << endl;
for ( ; notmuch_threads_valid (threads);
notmuch_threads_move_to_next (threads))
{
thread = notmuch_threads_get (threads);
notmuch_thread_get_thread_id (thread);
i++;
cout << "tags: ";
/* get tags */
notmuch_tags_t *tags;
const char *tag;
for (tags = notmuch_thread_get_tags (thread);
notmuch_tags_valid (tags);
notmuch_tags_move_to_next (tags))
{
tag = notmuch_tags_get (tags);
cout << tag << " ";
}
cout << endl;
notmuch_thread_destroy (thread);
if (i == stop) break;
}
/* now open a new db instance, modify the already loaded thread and
* continue loading the original query */
notmuch_database_t * nm_db2;
s = notmuch_database_open (
path_db.c_str(),
notmuch_database_mode_t::NOTMUCH_DATABASE_MODE_READ_WRITE,
&nm_db2);
char qry_s[256];
sprintf (qry_s, "thread:%s", thread_id.c_str ());
notmuch_query_t * q2 = notmuch_query_create (nm_db2, qry_s);
notmuch_threads_t * ts2;
notmuch_thread_t * t2;
st = notmuch_query_search_threads_st (q2, &ts2);
for ( ; notmuch_threads_valid (ts2);
notmuch_threads_move_to_next (ts2))
{
t2 = notmuch_threads_get (ts2);
std::string thread_id = notmuch_thread_get_thread_id (t2);
/* remove unread tag */
notmuch_messages_t * ms = notmuch_thread_get_messages (t2);
notmuch_message_t * m;
for (; notmuch_messages_valid (ms); notmuch_messages_move_to_next (ms)) {
m = notmuch_messages_get (ms);
st = notmuch_message_remove_tag (m, "unread");
notmuch_message_destroy (m);
}
notmuch_messages_destroy (ms);
notmuch_thread_destroy (t2);
break;
}
notmuch_query_destroy (q2);
notmuch_database_close (nm_db2);
/* re-add unread tag */
s = notmuch_database_open (
path_db.c_str(),
notmuch_database_mode_t::NOTMUCH_DATABASE_MODE_READ_WRITE,
&nm_db2);
q2 = notmuch_query_create (nm_db2, qry_
Announcing Astroid v0.7
Greetings, Astroid (https://github.com/astroidmail/astroid) v0.7 is now ready! Astroid is a lightweight and fast graphical threads-with-tags email client for notmuch. Written in C++ using GTK+, WebKit and gmime. Astroid can be found at: https://github.com/astroidmail/astroid : $ git clone https://github.com/astroidmail/astroid.git ## README with screenshot and information on building and usage: https://github.com/astroidmail/astroid#astroid-mua once you get Astroid running press '?' to get a list of keybindings for the current context. ## Tour and usage https://github.com/astroidmail/astroid/wiki ## Changes since v0.6 * Edit drafts using an external editor (optional). * Filter thread-index incrementally using 'C-f'. * Messages are marked unread after a delay (default 2 sec) in thread-view. They can be manually toggled with 'N'. * Use Tab and S-Tab to jump between unread messages in thread-view. * Yank raw or decoded message or element to primary clipboard with 'Y' or 'y', useful for applying patches. * Yank the Message-ID to the primary clipboard with C-y. * Make id:Message-ID urls clickable. * Make Message-ID customizable, and User-Agent optional. These may also be customized with a plugin. * Use Boost::log for logging. * Some hints about usage are shown (level of detail can be configured) * 'c' in thread-view composes message to sender (or receiver if sender is you). * Taeer Bar-Yam : Allow any key name defined by GDK to be entered. * Thread-view: '*' toggle 'flagged' on message, 'a' toggle inbox on whole thread. * Removed MathJax support, since it is not widely used. Can be enabled manually by adding sourcing the script in the thread-view theme file. * When replying, match encryption / signature level. * Cancel sending with Ctrl + C. * Open element with 'Space', also in editor. * Check for missing fields and for attachment words before sending. * Kim Ebert: Make sure all messages are loaded. * M. Dietrich, G. Hope: Fix buggy quoting of arbitrarily spaced '>'s. * Close thread-view after deleting draft. * Hugo Roy: Improve attachment, mime-message, and url display. Regards, Gaute pgpQEiZyhG7JX.pgp Description: PGP signature ___ notmuch mailing list notmuch@notmuchmail.org https://notmuchmail.org/mailman/listinfo/notmuch
sort by relevance?
Hi, while reading a bit about Xapian with regards to [0] I came across sorting by relevance? Is this currently possible? Especially in OR searches with many terms I think this would be very useful. Regards, Gaute [0] id:1471858269.x2m28lgosh.astroid@strange pgpCwFM0YXcoW.pgp Description: PGP signature ___ notmuch mailing list notmuch@notmuchmail.org https://notmuchmail.org/mailman/listinfo/notmuch
match on all messages in a thread - rather than messages invdividually
Hi, I often fail to find threads because there are no messages that match all the criteria, but the thread combined match the full criterias. Often I remember some bits about the whole thread, but they may not be in any one message - these are hard to find. Is it possible to search in a way that (through the API) such that the thread is matching, rather than the individual messages? How does muted thread work with regard to this? If I have one unmuted message in a thread that message matches the query. If a new message is added to a muted thread, is the muted tag applied to it? Regards, Gaute ___ notmuch mailing list notmuch@notmuchmail.org https://notmuchmail.org/mailman/listinfo/notmuch
Re: [PATCH v2] Omit User-Agent: header by default
Daniel Kahn Gillmor writes on august 9, 2016 1:35: The User-Agent: header can be fun and interesting, but it also leaks quite a bit of information about the user and their software stack. Is the message-id generated by gnus or notmuch-emacs? I could not find the relevant code. I noticed it has an *.fsf@* part as well as the, probably customizable, local FQDN. Regards, Gaute pgpgCYqK6GXxQ.pgp Description: PGP signature ___ notmuch mailing list notmuch@notmuchmail.org https://notmuchmail.org/mailman/listinfo/notmuch
Announcing Astroid v0.6
Greetings, Astroid (https://github.com/astroidmail/astroid) v0.6 is now ready! Astroid is a lightweight and fast graphical threads-with-tags mail user agent for notmuch. Written in C++ using GTK+, WebKit and gmime. Astroid can be found at: https://github.com/astroidmail/astroid : $ git clone https://github.com/astroidmail/astroid.git ## README with screenshot and information on building and using https://github.com/astroidmail/astroid#astroid-mua once you get Astroid running press '?' to get a list of keybindings for the current context. ## Tour and usage https://github.com/astroidmail/astroid/wiki ## Changes since v0.5 * Use any editor (general XEmbed support in edit-message window): Emacs works. * Plugins: Python and Lua plugins can now be made, parts of ThreadIndex and ThreadView are exposed allowing extesion of the user interface and advanced customization. Check out the plugins available at GitHub. * Crypto: GPG encryption and signing is added. Decryption and and signature status is shown in greater detail. * Repository moved to github.com/astroidmail. * Thread-view: Search email text using 'C-f' * Synchronize maildir flags if so configured in .notmuch-config. * Signatures: Include per-account signatures (either included or attached). * Saved searches: Save a search with 'C-S', then browse saved searches and stats with 'C-f'. You can also browse search history using Up and Down in the search-bar. * Colorful tags: Tag background color is calculated based on MD5 sum and normalized to within a configurable range. The same tag will therefore always have the same color. * Thread-index: load all threads in the background. * All actions requiring write-access run in the background and wait for any reads to finish. All read operations need to wait for any ongoing write-operations to finish. All write-operations are therefore asynchronous and non-blocking. Read-operations will only block on long write operations. * Tags can be modified on multiple marked threads with + and -. * Modify tags on single message, they are updated in thread-view and thread-index. * Keybindings modified to be more common #171. * scss: fix #82: make all variables global. * C-d / C-u / Page in thread-index: Page and move cursor in thread index. * Special keys like Tab and Home can now be used in user-keybindings. * M. Dietrich: clock_format=year option to use clock format from this or previous year setting in thread-index. * M. Dietrich, G. Hope: Background color in thread-index configurable. * M. Dietrich: Use correct reply-to address for mailinglist-messages. * Gravatar is configurable. * #103: Drafts are saved on force close or quit by default. * C-v: duplicate and refine query. * Mark thread moves to next by default. * ti: set cursor to first thread if we are at first thread when new addition is made. * build: --disable-libsass: do not depend on libsass, but use a sass compiler at build time (-scss-compiler=sassc). * Quote lines for reply and forward are formatted with Glib::DateTime::format(). Thanks to: Hugo Roy Franz Fellner M. Dietrich Max Mehl Regards, Gaute pgp6_ZRp_e96j.pgp Description: PGP signature ___ notmuch mailing list notmuch@notmuchmail.org https://notmuchmail.org/mailman/listinfo/notmuch
Re: [PATCH] create .mailmap file (for git shortlog/blame)
David Bremner writes on juli 22, 2016 14:47: It would potentially make sense to automagically update this file during the release process using the output from git shortlog. I use git-rank-contributors to maintain sup (written by William Morgan of sup fame, appropriately of this conversation). It used to live on gitorious, I have a fork here with some minor tweaks (like hard-coded aliases): https://github.com/gauteh/willgit Regards, Gaute pgpDqLHHZUQP3.pgp Description: PGP signature ___ notmuch mailing list notmuch@notmuchmail.org https://notmuchmail.org/mailman/listinfo/notmuch
Re: [PATCH] WIP: regexp matching in 'subject' and 'from'
Tomi Ollila writes on juni 11, 2016 19:09: On Sat, Jun 11 2016, Gaute Hope wrote: Maybe we could check if the search string contains a regexp and decide whether to pre-process it on the background of that? I think that would make the interface more user-friendly. You'd just always use search whether you decide that you need to put in some regexp or not. You probably wanted to suggest that the command line handling in notmuch goes through the search terms and potentially modify it before giving to xapian to chew for... I think this is deliberately avoided (*) -- this would get out of hands so easily (if we could decide syntax)... (*) there is some optmization done before feeding the query to xapian -- but that does not affect interface (i.e. it could be dropped and none of the users' expectations would be broken...) What one can do, is write ones own wrapper around notmuch. I have one that was written long before notmuch got date: searches (it mangles e.g 5h.. to 1234567890.. (**) and logs search and show queries (**) should change that to use date:... instead (i.e. date: queries w/o date: prefix). I "suggested" subject:/one's own subject re search w// slashes/ which one could pretty easily write to the wrapper... Yes, that is pretty much what I meant. So that the user only needs to know about 'search:', if it is 'search:foo' regular queryparser is used, if it is 'search:/^foo/' it is preprocessed using the regexp parser. Then the performance will remain the same for normal queries, but seamlessly switch to the heavier regexp'er if necessary. It could be done with a wrapper, but I am mainly using notmuch through the API and astroid - where it could also be implemented of course. -gaute ___ notmuch mailing list notmuch@notmuchmail.org https://notmuchmail.org/mailman/listinfo/notmuch
Re: [PATCH] WIP: regexp matching in 'subject' and 'from'
David Bremner writes on juni 10, 2016 13:09: Gaute Hope writes: Cool! Would it break a lot of things if you just replace the original prefix? It would change the matching behaviour. I guess there are people that like the current "sloppy" matching of from: and subject:. In my not-very-scientific tests, it is a factor of 5 to 10 times slower to do regexp search, which makes sense because it is effectively post processing the results from Xapian. At least on my system it seems fast enough to be usable interactively, but that is a pretty shocking performance regression. And I know there are people with more mail on slower systems. Maybe we could check if the search string contains a regexp and decide whether to pre-process it on the background of that? I think that would make the interface more user-friendly. You'd just always use search whether you decide that you need to put in some regexp or not. Could it be made to work on the message body? See Austin's previous reply for the details, but basically no; these "values" index in terms of whole strings, while the body is indexed by terms (roughly, words). In principle we could add a value slot for the body, but I think that would at least double the size of the database (maybe more). I would rather have double the db and be able wildcard beginning of terms. If it is not too much maintaining overhead it might be made optional? Regards, Gaute ___ notmuch mailing list notmuch@notmuchmail.org https://notmuchmail.org/mailman/listinfo/notmuch
Re: [PATCH] WIP: regexp matching in 'subject' and 'from'
David Bremner writes on juni 10, 2016 4:28: the idea is that you can run % notmuch search subject_re: % notmuch search from_re:' or % notmuch search subject:"your usual phrase search" % notmuch search from:"usual phrase search" This should also work with bindings, since it extends the query parser. This is trivial to extend for other value slots, but currently the only value slots are date, message_id, from, subject, and last_mod. Date is already searchable, and message_id is not obviously useful to regex match. --- This is more or less complete codewise, it fixes the know problems with the last version. Names of prefixes are debatable, and of course it needs doc and tests. I don't see any reason not to do this at the moment, since it's basically free; no new terms are added to the database. Cool! Would it break a lot of things if you just replace the original prefix? Could it be made to work on the message body? Regards, Gaute ___ notmuch mailing list notmuch@notmuchmail.org https://notmuchmail.org/mailman/listinfo/notmuch
Re: searching: '*analysis' vs 'reanalysis'
Austin Clements writes on juni 6, 2016 21:20: The experiment was specifically for regexp matching subject, but it should work for any header we store a literal copy of in the database. Does it work for terms in the body of the message? ___ notmuch mailing list notmuch@notmuchmail.org https://notmuchmail.org/mailman/listinfo/notmuch
Re: searching: '*analysis' vs 'reanalysis'
David Bremner writes on juni 6, 2016 14:42: Gaute Hope writes: Hi, I have an email with the word 'reanalysis' in the subject line and the email body. However, when I try to search for '*analysis' or 'analysis' I do not get any matches, should not '*analysis' at least match? We talked about this on IRC (the short answer is no), but is there some improvement you could suggest to the "Wildcards" section in notmuch-search-terms(7) ? Yes, thanks, not very important, but maybe add the sentence: It is not possible to use wildcards at the beginning of a term. after the current explanation to emphasize this limitation (possibly blaming Xapian to avoid futile requests). I think it is something many would expect (and want). The current description feels more like an example, and it is easy to make the assumption that it works for prefixing the terms as well - although, technically, nothing is promised in the original docs. -gaute ___ notmuch mailing list notmuch@notmuchmail.org https://notmuchmail.org/mailman/listinfo/notmuch
searching: '*analysis' vs 'reanalysis'
Hi, I have an email with the word 'reanalysis' in the subject line and the email body. However, when I try to search for '*analysis' or 'analysis' I do not get any matches, should not '*analysis' at least match? Regards, Gaute ___ notmuch mailing list notmuch@notmuchmail.org https://notmuchmail.org/mailman/listinfo/notmuch
Re: talloc_abort in notmuch_thread_get_tags () when db has been modified
Gaute Hope writes on January 18, 2016 13:45: David Bremner writes on January 18, 2016 13:25: The most likely cause of such a crash looks to me like nm_thread is NULL or corrupted when passed in to get_tags. It's used without checking as a talloc context, and that call to talloc never returns. Ok, I'll check some further. I am checking whether nm_thread is NULL though, [...] Hi, The stack trace that I get is as follows: ``` Stack trace of thread 15719: #0 0x7fc80cd9f2a8 raise (libc.so.6) #1 0x7fc80cda072a abort (libc.so.6) #2 0x7fc80c95889c n/a (libtalloc.so.2) #3 0x7fc80c95a02d talloc_named_const (libtalloc.so.2) #4 0x7fc814d674c5 _notmuch_string_list_create (libnotmuch.so.4) #5 0x7fc814d75f32 notmuch_thread_get_tags (libnotmuch.so.4) #6 0x004757cb _ZN7Astroid13NotmuchThread8get_tagsEP15_notmuch_thread (astroid) ``` this happens when: 1) start a long running query loading in the background 2) modify the db enough for the query to get invalidated. as far as I can see, there is _no_ way to catch this error without completely crashing the application. I would have to isolate this code in a separate process or trap SIGABRT (which is certainly messy). Best regards, Gaute ___ notmuch mailing list notmuch@notmuchmail.org https://notmuchmail.org/mailman/listinfo/notmuch
Re: need to call notmuch_threads_get (..) to actually move iterator
Gaute Hope writes on February 28, 2016 13:36: Gaute Hope writes on February 24, 2016 13:08: Hi, it seems to be necessary to actually call notmuch_threads_get (threads) to move the thread iterator from a query object, just calling notmuch_threads_move_to_next (..) is not enough: A test-case demonstrating this (for the 'astroid' code-base) is located here: https://github.com/gauteh/astroid/blob/ti-stateless-query/test/test_notmuch.cc#L66 At the moment, this test depends on the rest of the astroid test-suite, but it may be useful as a demonstration or skeleton for a notmuch test-case anyway. Disregard that, my test-case was erroneous. -Gaute ___ notmuch mailing list notmuch@notmuchmail.org https://notmuchmail.org/mailman/listinfo/notmuch
Re: need to call notmuch_threads_get (..) to actually move iterator
Gaute Hope writes on February 24, 2016 13:08: Hi, it seems to be necessary to actually call notmuch_threads_get (threads) to move the thread iterator from a query object, just calling notmuch_threads_move_to_next (..) is not enough: A test-case demonstrating this (for the 'astroid' code-base) is located here: https://github.com/gauteh/astroid/blob/ti-stateless-query/test/test_notmuch.cc#L66 At the moment, this test depends on the rest of the astroid test-suite, but it may be useful as a demonstration or skeleton for a notmuch test-case anyway. Regards, Gaute ___ notmuch mailing list notmuch@notmuchmail.org https://notmuchmail.org/mailman/listinfo/notmuch
Re: how do the different frontends deal with displaying large queries?
David Bremner writes on February 27, 2016 13:50: Gaute Hope writes: While loading the threads, if I make enough modifications to the database while the query is still loading (specifically 1: removing the unread tag from a thread, and 2: adding the unread tag to the same thread) apparently the query is invalidated in some way, and I get a hard crash in `notmuch_tags_get ()`. The thread in question has alreaday been loaded and is displayed. Do you deal with this issue in some way? I think we "deal with" it by not leaving the database open very long. The query is dumped as s-expr by a seperate process. Or perhaps we just aren't doing the same level of concurrent operations, I don't know. Neither do I, I load the threads in the background. But for a long query that takes a while. The database is only kept open for the duration it takes to load the threads in the query. How long is the db open? Would you notice if it crashed? I'm guessing the abort is triggered by an Xapian::ModifiedDatabase somewhere. I am only able to consistently reproduce the error if I change the tags of a thread not yet loaded, a (somewhat messy) test case is available here: https://github.com/gauteh/astroid/blob/ti-stateless-query/test/test_notmuch_standalone.cc If nothing else, it demonstrates the difficulty of handling these exceptions in functions that don't yet catch them, since they could happen along many of the internal steps. Regards, Gaute ___ notmuch mailing list notmuch@notmuchmail.org https://notmuchmail.org/mailman/listinfo/notmuch
Re: how do the different frontends deal with displaying large queries?
David Bremner writes on February 24, 2016 18:48: Gaute Hope writes: Hi, I am wondering how the different frontends deal with displaying large queries? Do you just display everything at the time? Or do you display a limited number and then show more on-demand? I only know the emacs front end, and I suspect the answer won't help you much. We start filling an emacs buffer asynchronously with the results, and display the top of the buffer with the first results. Results are appended to the end of the buffer, typically much faster than the user can scroll. Hi, I am testing out this way of loading threads, however, I run into an issue which I have been bugging you with before: While loading the threads, if I make enough modifications to the database while the query is still loading (specifically 1: removing the unread tag from a thread, and 2: adding the unread tag to the same thread) apparently the query is invalidated in some way, and I get a hard crash in `notmuch_tags_get ()`. The thread in question has alreaday been loaded and is displayed. Do you deal with this issue in some way? [ code in question: https://github.com/gauteh/astroid/pull/95 ] Regards, Gaute ___ notmuch mailing list notmuch@notmuchmail.org https://notmuchmail.org/mailman/listinfo/notmuch
Re: how do the different frontends deal with displaying large queries?
That makes sense of course, that could work. Thanks, Gaute Den onsdag 24. februar 2016 skrev David Bremner følgende: > Gaute Hope > writes: > > > Hi, > > > > I am wondering how the different frontends deal with displaying large > > queries? Do you just display everything at the time? Or do you display a > > limited number and then show more on-demand? > > > > I only know the emacs front end, and I suspect the answer won't help you > much. We start filling an emacs buffer asynchronously with the results, > and display the top of the buffer with the first results. Results are > appended to the end of the buffer, typically much faster than the user > can scroll. > > d > ___ notmuch mailing list notmuch@notmuchmail.org https://notmuchmail.org/mailman/listinfo/notmuch
need to call notmuch_threads_get (..) to actually move iterator
Hi,
it seems to be necessary to actually call notmuch_threads_get (threads)
to move the thread iterator from a query object, just calling
notmuch_threads_move_to_next (..) is not enough:
```
notmuch_query_t *query;
notmuch_threads_t *threads;
notmuch_thread_t *thread;
query = notmuch_query_create (database, query_string);
threads = notmuch_query_search_threads (query);
int i = 0;
for (;
notmuch_threads_valid (threads);
notmuch_threads_move_to_next (threads))
{
/*
* with this line commented out the iterator seems to remain in
* place, and if I below do another loop it will start from the
* beginning.
thread = notmuch_threads_get (threads);
notmuch_thread_destroy (thread);
*/
i++;
if (i > 100) break;
}
for (;
notmuch_threads_valid (threads);
notmuch_threads_move_to_next (threads))
{
/* the thread acquired here will be the first thread in the query.
* it should be the 101th. */
thread = notmuch_threads_get (threads);
notmuch_thread_destroy (thread);
}
notmuch_query_destroy (query);
```
It is quite slow to skip the threads in this way, might it be faster if
move_to_next works correctly?
Regards, Gaute
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch
how do the different frontends deal with displaying large queries?
Hi, I am wondering how the different frontends deal with displaying large queries? Do you just display everything at the time? Or do you display a limited number and then show more on-demand? In the on-demand case; how do you get to the same position? Keeping a query object alive for long might make it invalid when the index is changed. Showing a limited number and restarting the query from the same position might miss threads if something in the first batch has been removed. Regards, Gaute ___ notmuch mailing list notmuch@notmuchmail.org https://notmuchmail.org/mailman/listinfo/notmuch
Re: api-doc: return status for notmuch_query_search_threads_st ...
Gaute Hope writes on February 7, 2016 11:06:
Hi,
I am switching from the deprecated notmuch_query_search_threads to
notmuch_query_search_threads_st, but the api-documentation does not
specify what status values are returned and what they mean.
The same goes for notmuch_query_search_messages_st (...).
Indeed the typical usage section is also outdated (for both):
```
notmuch_query_t *query;
notmuch_messages_t *messages;
notmuch_message_t *message;
query = notmuch_query_create (database, query_string);
for (messages = notmuch_query_search_messages (query);
notmuch_messages_valid (messages);
notmuch_messages_move_to_next (messages))
{
message = notmuch_messages_get (messages);
notmuch_message_destroy (message);
}
notmuch_query_destroy (query);
```
-gaute
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch
api-doc: return status for notmuch_query_search_threads_st ...
Hi, I am switching from the deprecated notmuch_query_search_threads to notmuch_query_search_threads_st, but the api-documentation does not specify what status values are returned and what they mean. The same goes for notmuch_query_search_messages_st (...). Regards, Gaute ___ notmuch mailing list notmuch@notmuchmail.org https://notmuchmail.org/mailman/listinfo/notmuch
Announcing Astroid v0.5
Greetings, Astroid (https://github.com/gauteh/astroid) v0.5 is now ready! Astroid is a graphical threads-with-tags mail user agent for notmuch. Written in C++ using GTK+, WebKit and gmime. This release includes a lot of changes and improvements from several authors, including increased customizability and many new features. See below for more details and observe debug output for changed configuration options. Astroid can be found at: https://github.com/gauteh/astroid : $ git clone https://github.com/gauteh/astroid.git ## README with screenshot and information on building and using https://github.com/gauteh/astroid#astroid-mua once you get Astroid running press '?' to get a list of keybindings for the current context. ## Tour and usage There is now a tour of how to set up, customize and use astroid at: https://github.com/gauteh/astroid/wiki ## Changes since v0.4 * C-f: Toggle flat or indented view of messages. * C-i: Show remote images (cid-type images are not supported yet). * A-#: Page to # page. * Add gravatar support * M-p: Toggle auto poll. * C-j / C-k: Jump and scroll to next or previous element. * thread-index: C-Enter: open thread in new window. * Add popup menu with some actions for the thread-index. * Fix #23: Don't crash when message file is missing. * #25: Load html and scss from config/ui directory if present. * #25: Allow some parameters of the thread_index cells to be configured. * Move 'expand all' in thread view to C-e. * Save drafts with 's', discard them with 'D', edit them with 'E' from thread index or thread view. * Edit messages as new with 'E'. * Added a dinosaur. * Configurable keybindings in ~/.config/astroid/keybindings. * Show name of pre-defined searches in tab. * Tab: jump to next unread thread. * X: Force close tab. * C-s: Cycle through sorting the thread_index (default configurable) * Configurable sent-tags (per-account and global) * Allow messages to be sent as attachments. * Hugo Roy: Fix licences to be correct for borrowed stuff. * Franz Fellner: Fix some long standing annoying bugs. * Use ',' as separator for 'for_tags' in config, not ';'. * Hugo Roy: Refurbished thread-view theme and move to scss processing, your custom thread-view.css needs some attention. * Franz Fellner: Clean-up coding style and name-space pollution. * Hugo Roy: Made a tour of setting up astroid in the wiki. * Print message with C-P and print marked messages with t+p. * Run-hooks: Run user-defined shell hooks on keypress using keybinding configuration. * Franz Fellner: Integrate with notmuch config, read available options from there. You might have to update some of the affected configuration options. * Thread index: only use bold face on authors of unread messages. * Startup queries are invincible. * q: ask to quit, Q: quit right away. Regards, Gaute ___ notmuch mailing list notmuch@notmuchmail.org https://notmuchmail.org/mailman/listinfo/notmuch
Re: talloc_abort in notmuch_thread_get_tags () when db has been modified
David Bremner writes on January 18, 2016 13:25:
The most likely cause of such a crash looks to me like nm_thread is NULL
or corrupted when passed in to get_tags. It's used without checking as a
talloc context, and that call to talloc never returns.
Ok, I'll check some further. I am checking whether nm_thread is NULL
though, the preceding code is as follows
(astroid/src/modes/thread_index/thread_index.cc:258):
```
for (;
notmuch_threads_valid (threads);
notmuch_threads_move_to_next (threads)) {
notmuch_thread_t * thread;
thread = notmuch_threads_get (threads);
if (thread == NULL) {
log << error << "ti: error: could not get thread." << endl;
throw database_error ("ti: could not get thread (is NULL)");
}
/* test for revision discarded */
const char * ti = notmuch_thread_get_thread_id (thread);
if (ti == NULL) {
log << error << "ti: revision discarded, trying to reopen." << endl;
reopen_tries++;
refresh (all, current_thread + count, false);
return;
}
NotmuchThread *t = new NotmuchThread (thread); // get_tags is inside here
notmuch_thread_destroy (thread);
```
(note that there is a bit of code there trying to determine whether the
db is still valid, or needs to be re-opened)
- g
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch
talloc_abort in notmuch_thread_get_tags () when db has been modified
Hi,
a user of astroid [0] ran into a issue [1] (full trace at issue) where
reading a long query causes a talloc_abort in notmuch_thread_get_tags
(). 'notmuch new' is running at the same time, and most likely a thread
in the query has been modified since the query was done. Note that a
notmuch_thread_get_authors () call returns NULL without causing a full
crash. The code causing the crash is:
```
for (tags = notmuch_thread_get_tags (nm_thread);
notmuch_tags_valid (tags);
notmuch_tags_move_to_next (tags))
{
tag = notmuch_tags_get (tags); // tag belongs to tags
}
// or db.cc:508 in astroid/src.
```
while:
```
const char * auths = notmuch_thread_get_authors (nm_thread);
```
returns `NULL`, but does not crash.
Is there a way for me to handle this from the application side?
Admittedly I do keep query objects around for a while
(astroid/src/thread_index.cc:141), but in this case the issue would
probably occur anyway since it simply takes a long time to read the
query.
Regards, Gaute
[0] https://github.com/gauteh/astroid
[1] https://github.com/gauteh/astroid/issues/64
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch
Re: Question re undo
Den lørdag 21. november 2015 skrev Tomi Ollila følgende: > On Sat, Nov 21 2015, Bart Bunting > wrote: > > > Hi there, > > > > I've accidentally removed a bunch of tags in error and was dreaming > > there may be a way to fix it :). > > > > Is there any way to restore tags as they were at a certain point in > > time? > > some examination of lastmod: might help ? > > first run notmuch count --lastmod > > Then experiment with `notmuch search lastmod:` with "suitable" > values of num to figure out of the messages that have been affected... > > Tomi > > > Even if not now after i've done it, is there anything that could have > been > > done at the time to fix the error? > > probably not. > > You could dump the db at regular intervals, then you might be able to restore it. Gaute ___ notmuch mailing list notmuch@notmuchmail.org https://notmuchmail.org/mailman/listinfo/notmuch
Re: [PATCH 5/5] lib: Add "lastmod:" queries for filtering by last modification
David Bremner writes on August 20, 2015 8:22: David Bremner writes: From: Austin Clements The implementation is essentially the same as the date range search prior to Jani's fancy date parser. pushed the series. Those of you running master be prepared for a database update next time you run notmuch new. This particular update is relatively fast, it took about a minute for my 380k messages (on SSD). Great! -g ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
Announcing Astroid v0.4
Greetings, Astroid (https://github.com/gauteh/astroid) v0.4 is now ready! Astroid is a graphical threads-with-tags mail user agent for notmuch. Written in C++ using GTK+, WebKit and gmime. Astroid can be found at: https://github.com/gauteh/astroid : $ git clone https://github.com/gauteh/astroid.git ## README with screenshot and information on building and using https://github.com/gauteh/astroid#astroid-mua once you get Astroid running press '?' to get a list of keybindings for the current context. ## Changes since v0.3 * Mark threads with 't' or 'T' and apply actions to all marked threads with '+'. * Mark messages with 't' or 'T' in thread-view and apply actions to all marked messages with '+' (nice for patch saving). * Show attached icon in thread-view. * #18: Fix display of & and friends. * Configurable quote string for replying and forwarding. * Replying adds 'replied' tag to original. * Forwarding adds 'passed' tag to original. * Message is updated thread-index after being sent. * Show spinner in main window when polling. * Show status icon for message sending. * Indent messages according to reply-level in thread (configurable). Regards, Gaute
