Re: python CFFI bindings integration into notmuch build/test
Floris Bruynooghe writes: > > I still have a bunch of non-existing messages with weird unicode message > IDs in them, but no associated filename just an empty string. AFAIK > notmuch_database_remove_message() needs a filename though, I've tried > removing the empty file which does not fail but doesn't clear those > database entries. The messages are not ghosts either. Any idea how I > might be able to clear those bad messages from the database? In principle running "notmuch new" should do it, if there are no files on disk corresponding to those messages. If that doesn't work, you might have to use "xapian-delve" to figure out what terms are attached to those message documents. d ___ notmuch mailing list notmuch@notmuchmail.org https://notmuchmail.org/mailman/listinfo/notmuch
Re: python CFFI bindings integration into notmuch build/test
On Sat 16 Nov 2019 at 10:51 -0500, David Bremner wrote: > Floris Bruynooghe writes: >> Anyway, this looks good. Would you like some changes, e.g. the rename >> to notmuch2 or so as patches? What's the next step. > > If you could look at the rename that would be great. Patches on top of > wip/cffi, or a ref for me to pull both sound fine. > >> Kind of unrelated, but in my attempt to run the full notmuch test suite >> (cd tests; make test) I somehow ended up with lots of weird tags in my >> notmuch database and made the emacs UI pretty horrible - though it seems >> I haven't lost any email. I'm sure that was my fault somehow even >> though I was just trying to follow the readme. But if anyone has any >> hints how to recover from this that could save me some time :) > > Hmm. I've never encountered that, but maybe it has to do with killing > the environment? The test harness uses the environment variable > NOTMUCH_CONFIG to point to a test database, and if that was deleted, it > would look in ~/.notmuch-config. Yeah, I probably managed to make it use ~/.notmuch-config somehow. > Still that would really only be tags from your pytest tests, and you'd > probably recognize those? The were a bunch of weird characters that looked like unicode test-cases, the pytest tests don't try to test notmuch only the bindings so doesn't really make up weird strings IIRC. I recovered with something like "notmuch dump | cleanup_tags.py | notmuch restore", oh well. I still have a bunch of non-existing messages with weird unicode message IDs in them, but no associated filename just an empty string. AFAIK notmuch_database_remove_message() needs a filename though, I've tried removing the empty file which does not fail but doesn't clear those database entries. The messages are not ghosts either. Any idea how I might be able to clear those bad messages from the database? Cheers, Floris ___ notmuch mailing list notmuch@notmuchmail.org https://notmuchmail.org/mailman/listinfo/notmuch
Re: python CFFI bindings integration into notmuch build/test
Floris Bruynooghe writes: > Anyway, this looks good. Would you like some changes, e.g. the rename > to notmuch2 or so as patches? What's the next step. If you could look at the rename that would be great. Patches on top of wip/cffi, or a ref for me to pull both sound fine. > Kind of unrelated, but in my attempt to run the full notmuch test suite > (cd tests; make test) I somehow ended up with lots of weird tags in my > notmuch database and made the emacs UI pretty horrible - though it seems > I haven't lost any email. I'm sure that was my fault somehow even > though I was just trying to follow the readme. But if anyone has any > hints how to recover from this that could save me some time :) Hmm. I've never encountered that, but maybe it has to do with killing the environment? The test harness uses the environment variable NOTMUCH_CONFIG to point to a test database, and if that was deleted, it would look in ~/.notmuch-config. Still that would really only be tags from your pytest tests, and you'd probably recognize those? d ___ notmuch mailing list notmuch@notmuchmail.org https://notmuchmail.org/mailman/listinfo/notmuch
Re: python CFFI bindings integration into notmuch build/test
Tomi Ollila writes: > On Thu, Nov 14 2019, Floris Bruynooghe wrote: > >> Hi, >> >> Thanks for carrying on with this! >> >> I'm a little confused with how to follow this, is the current state of >> the code somewhere in a repo/branch where I can try things out and make >> changes from? > > In git://notmuchmail.org/git/notmuch David has ref origin/wip/cffi > which contains related changes -- You can fetch the code while waiting > for more collaboration instructions from David. > That should be just a normal branch. I see % git ls-remote origin | grep cffi 2a326a866ccd917f1d62bc7f2b475a0e2d26f6c8 refs/heads/wip/cffi Anyway, for now I'm leary of reposting the whole series since we're mainly discussing my mucking about, not Florian's largish patch. ___ notmuch mailing list notmuch@notmuchmail.org https://notmuchmail.org/mailman/listinfo/notmuch
Re: python CFFI bindings integration into notmuch build/test
On Thu 14 Nov 2019 at 23:24 +0100, Floris Bruynooghe wrote: > On Thu 14 Nov 2019 at 22:20 +0200, Tomi Ollila wrote: >> In git://notmuchmail.org/git/notmuch David has ref origin/wip/cffi >> which contains related changes -- You can fetch the code while waiting >> for more collaboration instructions from David. > > Aha, thanks! So I can run the tests using `make` and `cd tests; ./T-391-pytest-cffi.sh`. I can also run the tests still individually using something like: make set PATH (pwd) $PATH # I use fish pushd bindings/python-cffi pew workon notmuch # how I manage python virtualenvs pip install -e . pip install pytest pytest-cov pytest test/test_base.py And all works. I added a little function to the conftest.py to show which notmuch it's testing with: def pytest_report_header(): vers = subprocess.run(['notmuch', '--version'], capture_output=True, text=True) which = subprocess.run(['which', 'notmuch'], capture_output=True, text=True) return ['{} ({})'.format(vers.stdout.strip(), which.stdout.strip())] Maybe you find that useful too. Anyway, this looks good. Would you like some changes, e.g. the rename to notmuch2 or so as patches? What's the next step. Kind of unrelated, but in my attempt to run the full notmuch test suite (cd tests; make test) I somehow ended up with lots of weird tags in my notmuch database and made the emacs UI pretty horrible - though it seems I haven't lost any email. I'm sure that was my fault somehow even though I was just trying to follow the readme. But if anyone has any hints how to recover from this that could save me some time :) Cheers, Floris ___ notmuch mailing list notmuch@notmuchmail.org https://notmuchmail.org/mailman/listinfo/notmuch
Re: python CFFI bindings integration into notmuch build/test
On Thu 14 Nov 2019 at 22:20 +0200, Tomi Ollila wrote: > In git://notmuchmail.org/git/notmuch David has ref origin/wip/cffi > which contains related changes -- You can fetch the code while waiting > for more collaboration instructions from David. Aha, thanks! Floris ___ notmuch mailing list notmuch@notmuchmail.org https://notmuchmail.org/mailman/listinfo/notmuch
Re: python CFFI bindings integration into notmuch build/test
On Thu, Nov 14 2019, Floris Bruynooghe wrote: > Hi, > > Thanks for carrying on with this! > > I'm a little confused with how to follow this, is the current state of > the code somewhere in a repo/branch where I can try things out and make > changes from? In git://notmuchmail.org/git/notmuch David has ref origin/wip/cffi which contains related changes -- You can fetch the code while waiting for more collaboration instructions from David. > On Tue 05 Nov 2019 at 22:22 -0400, David Bremner wrote: > >> Tomi Ollila writes: >> >> >>> alternative: >>> >>>... >>>print('Invoking: {}'.format(' '.join(cmd))) >>> >>>def preexec_fn(): os.environ['NOTMUCH_CONFIG'] = str(cfg_fname) >>> >>>proc = subprocess.run(cmd, timeout=5, preexec_fn=preexec_fn) >>>... >>> >>> The unix fork ... (here preexec_fn called in child) ... exec model is >>> superior to any other alternative ! =D >> >> I don't consider myself a good judge of python style, so I'll defer to >> Floris on that one. > > I'd have gone with what David wrote to be fair, mostly because it's > pretty simple to see what's going on where for the preexec_fn I'd have > to look at the docs. Not sure if this has much to do with Python style > though, more with how much you like Unix tricks. Yes, the simple approach taken (by env copy) is fine. It is also good to know these powerful alternatives that may be useful elsewhere =D > > > Cheers, > Floris Tomi ___ notmuch mailing list notmuch@notmuchmail.org https://notmuchmail.org/mailman/listinfo/notmuch
Re: python CFFI bindings integration into notmuch build/test
Hi, Thanks for carrying on with this! I'm a little confused with how to follow this, is the current state of the code somewhere in a repo/branch where I can try things out and make changes from? On Tue 05 Nov 2019 at 22:22 -0400, David Bremner wrote: > Tomi Ollila writes: > > >> alternative: >> >>... >>print('Invoking: {}'.format(' '.join(cmd))) >> >>def preexec_fn(): os.environ['NOTMUCH_CONFIG'] = str(cfg_fname) >> >>proc = subprocess.run(cmd, timeout=5, preexec_fn=preexec_fn) >>... >> >> The unix fork ... (here preexec_fn called in child) ... exec model is >> superior to any other alternative ! =D > > I don't consider myself a good judge of python style, so I'll defer to > Floris on that one. I'd have gone with what David wrote to be fair, mostly because it's pretty simple to see what's going on where for the preexec_fn I'd have to look at the docs. Not sure if this has much to do with Python style though, more with how much you like Unix tricks. Cheers, Floris ___ notmuch mailing list notmuch@notmuchmail.org https://notmuchmail.org/mailman/listinfo/notmuch
Re: python CFFI bindings integration into notmuch build/test
Tomi Ollila writes: > alternative: > >... >print('Invoking: {}'.format(' '.join(cmd))) > >def preexec_fn(): os.environ['NOTMUCH_CONFIG'] = str(cfg_fname) > >proc = subprocess.run(cmd, timeout=5, preexec_fn=preexec_fn) >... > > The unix fork ... (here preexec_fn called in child) ... exec model is > superior to any other alternative ! =D I don't consider myself a good judge of python style, so I'll defer to Floris on that one. d ___ notmuch mailing list notmuch@notmuchmail.org https://notmuchmail.org/mailman/listinfo/notmuch
Re: python CFFI bindings integration into notmuch build/test
On Sun, Nov 03 2019, David Bremner wrote: > > @@ -32,10 +33,11 @@ def notmuch(maildir): > """ > cfg_fname = maildir.path / 'notmuch-config' > cmd = ['notmuch'] + list(args) > -print('Invoking: {}'.format(' '.join(cmd))) > +env = os.environ.copy() > +env['NOTMUCH_CONFIG'] = str(cfg_fname) > proc = subprocess.run(cmd, >timeout=5, > - env={'NOTMUCH_CONFIG': str(cfg_fname)}) > + env=env) Tried lambda (to have preexe_fn have cfg_fname in context), did not work, nested function also seems to hold context: alternative: ... print('Invoking: {}'.format(' '.join(cmd))) def preexec_fn(): os.environ['NOTMUCH_CONFIG'] = str(cfg_fname) proc = subprocess.run(cmd, timeout=5, preexec_fn=preexec_fn) ... The unix fork ... (here preexec_fn called in child) ... exec model is superior to any other alternative ! =D Tomi ___ notmuch mailing list notmuch@notmuchmail.org https://notmuchmail.org/mailman/listinfo/notmuch
Re: python CFFI bindings integration into notmuch build/test
David Bremner writes: > This series is on top of a slightly modified version [1] of > > id:20191008210312.20685-2-f...@devork.be > > If you want to try out the new bindings, you can try the branch wip/cffi in > the repo > https://git.notmuchmail.org/git/notmuch > > One current annoyance is that the version of pytest on travis is too > old for what is specified in tox.ini. > I fixed this by moving the travis env to bionic (2018.x). This involved dropping dkg's notmuch ppa. Daniel, do you happen to know if there is anything in that repo that we should still keep when building in ubuntu bionic? FWIW, the build succeeds without that PPA. d ___ notmuch mailing list notmuch@notmuchmail.org https://notmuchmail.org/mailman/listinfo/notmuch