[feature request] thread-based queries

[Patrick Totzke]

Hi everyone,

A recent discussion on alot's issue tracker[0]  made it clear that it might be 
useful to have a thread-based
matching for queries as an alternative to the current message based approach:

Consider a thread with two messages, one has a single tag 'flagged', the other 
a single tag 'unread'.
Now the thread doesn't match "tag:flagged AND tag:unread" as it doesn't contain 
any matching messages.
However, listing the tags of this thread accumulates to ['flagged', 'unread'].

As far as I know, there is no way one can tell notmuch to evaluate the query 
with threads as atomic entities
instead of messages so that the thread in the example above matches. I would 
like to see such a switch in 
libnotmuch and the python bindings, maybe as parameter to 
`notmuch_query_create` or as something similar to 
`notmuch_query_set_sort`.

I am aware that I can simulate the intended behaviour by rewriting the query 
myself, but I think
such a feature makes sense not only for alot users and should be implemented at 
a lower level.
Moreover, I wouldn't be surprised if doing this at a library level is less 
painful than higher up
as Xapian has to parse query formulae anyway and hence should offer some 
convenience methods for dealing with them.
Please correct me if I'm wrong here or have just overlooked previous 
discussions/solutions for this issue.

Best,
/p

[o]: https://github.com/pazz/alot/issues/270
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: [afew] announcing afew, an universal tagging solution with some fancy features

[Patrick Totzke]

Quoting Justus Winter (2011-12-21 08:53:23)
>Hey Pazz,
>
>Quoting Patrick Totzke (2011-12-19 19:17:12)
>>Also, should i not get some output when calling afew with -vv ?
>
>Yes you should ;). Here's an example run on my box:
Alright, I do get logging output with upstream master thanks.

FYI: I found the problem with afew not finding new mails on my box.
I had the following in my ~/.notmuch-config:


#   tagsA list (separated by ';') of the tags that will be
#   added to all messages incorporated by "notmuch new".
#
[new]
tags=new;

I suspect you don't split the config value and thus afew ended up with a false 
tag "new;".
its just cosmetics but nevertheless..

Thanks for afew! I translated my static filter rules and am very much enjoying 
the automagic list-id tags :)
/p

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[feature request] notmuch_thread_get_newest_subject

[Patrick Totzke]

Hi all,

There seems to be no way to get the "current" subject line of a thread.
I promised to relay my feature request to the list, so here it is..

This could be done using a parameter like "newest-or-original" for 
`notmuch_thread_get_subject` in threads.cc,
which would look up the newest message in the thread and return its subject.

Secondly, this should be usable from the python bindings.
I intend to use this in alot to display the newest subject for a thread by 
default.

cheers,
/p
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: For gmail how do you setup notmuch mail ?...

[Patrick Totzke]

Quoting Alex Botero-Lowry (2012-02-08 08:24:51)
>I'm using a similar setup.  An inconvenience with the setup is that I have
>various filters in gmail that filter, e.g., mailing list mails by skipping
>the gmail inbox and applying specific tags. ... 
>Ideally, I'd like a view in notmuch that shows the same messages as the
>gmail inbox.  Is there a way to accomplish this?
>- b

You could sync every subfolder *but* "All Mail" instead of just this folder:
Gmail tags are available as imap folders. You can then tune your tagging filters
to sort by folder: queries.
HTH,
/p
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: notmuch as a shared object aka library knigge

[Patrick Totzke]

Hi all,

Those of you with long enough backlog on the list to remember my rant 
(id:20110626202733.GA26837@brick)
can guess my opion on this matter but just to be sure..

I am not much of an expert on libnotmuch internals but am using the python 
bindings extensively.
It feels super-strange using a python module that possibly writes to stderr or 
any other descriptor
without me explicitly telling it to.
Also, if the library segfauls or calls exit, it essentially rips out the python 
interpreter underneath my code
without me being able to do any proper error handling.

I know that error handling on a library level is hard, juggling around with 
bare C, talloc and Xapian.
But i can only strongly encourage any rewrite that ends in the python bindings 
behaving more pythonic!

Cheers,
/p
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: Replacing my name/email with "me" (or similar) in author lists

[Patrick Totzke]

Quoting Daniel (2012-02-25 16:34:15)
>From what I understand, at least the Python bindings deliver primarily author
>names (not addresses),

To clarify, `notmuch.Thread.get_authors` returns a comma separated list of the 
realname parts of all From-headers that occur in messages of this thread.

> so it would seem more appropriate that this be done in the backend.
I agree. I personally think this is a nice feature to have in all
user interfaces to notmuch and therefore it makes sense to implement it once
in the lib. Also because ~/.notmuch-config already contains a list of my 
addresses
and therefore should have all information needed.

Implementing this feature for alot in python is easily doable, by constructing
the authors list from the messages From headers directly, comparing
the address part with all known own addresses.
We will do so if the response here is all too negative or nobody is brave 
enough to
step forward and touch the lib in this respect.

If I'm not mistaken, the lib extracts the authors name from the From-header
of a message, which is indexed. Somewhere around line 248 in thread.cc,
this string is added to a list of names.

Cheers,
/p
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: info about notmuch gui 'alot'

[Patrick Totzke]

Hi!

The config syntax changed recently, when I switched from configparser to 
configobj.
To clarify:

* the new config syntax is explained in the user docs at http://alot.rtfd.org.
  These are auto-generated at each commit and correspond to the `testing` 
branch,
  which is where the bleeding edge lives.

* both old and new config formats are (based on) INI:
  That means the *do* accept leading spaces, everything after a `#` is a 
comment.
  The new syntax makes use of nested sections, something configobj can parse
  and configparser cannot.

* both master and testing have switched to the new syntax, but only in testing
  is this actually documented in README/INSTALL.
  https://github.com/pazz/alot/blob/testing/INSTALL.md
  
This is due to me being lazy and that I'm planning to merge testing and master 
to make a next release soon anyway, so I
did not bother cherry picking that commit. But I'm happy to see that there are 
enough potential users
to be confused by this and will fix this inconsistency in the docs.

By the way: If you want to be on the safe side regarding the bindings, go for 
notmuch and its python bindings
from git master instead of potentially outdated versions provided by your 
distri.

Cheers,
/p
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: [alot] nottoomuch-addresses.sh config

[Patrick Totzke]

Quoting Philippe LeCavalier (2012-02-29 16:31:40)
>I had nottoomuch-addresses.sh working perfectly in emacs and I'm trying
>to get it going in alot. Here's[1] what I've got so far. One things that
>really confusing me is the brackets in the help file[2]. As soon as I
>put double brackets alot can't read the account. Am I misunderstanding
>something about those brackets?
the config uses a section called "account" and each subsection defines an 
account.
each account subsection may have an "abook" subsubsection to define the abook 
for this account.
here's what I use myself:

```
#.. global configs

[accounts]

[[gmail]]
realname = Patrick Totzke
address = patricktot...@gmail.com
aliases = patricktot...@googlemail.com,
sendmail_command = msmtp --account=gmail -t
draft_box = "maildir:///home/pazz/mail/gmail/[Google Mail].Drafts"
sent_box = "maildir:///home/pazz/mail/gmail/[Google Mail].Sent Mail"
[[[abook]]]
  command = nottoomuch-addresses.sh
  regexp = (\"(?P.+)\")?\s*<(?P.*@.+?)>

 [[uni]]
#...
```

> should I be creating a key binding to invoke the completion command?
No: this is used when you are prompted for recipients when composing a mail:
Hit `m`, select your account if you have more thn one, and when the prompt says
"To:" you 


>It would seem to me that I might need to specify where the heck the
>script lives?

In your $PATH.
alot hands this command string to `subprocess.Popen`, which accepts
command names to to look them up in $PATH or absolute paths to binaries
if i'm not mistaken.


hth,
/p
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: [alot] nottoomuch-addresses.sh config

[Patrick Totzke]

Sorry, forgot to finish this *caught*:

>> should I be creating a key binding to invoke the completion command?
>No: this is used when you are prompted for recipients when composing a mail:
>Hit `m`, select your account if you have more than one, and when the prompt 
>says
>"To:" you 

... use `tab` to complete names/addresses using the command from your config.

If you're using abook (http://abook.sourceforge.net/) consider using the
type "abook" address books directly instead of "shellcommand":
http://alot.readthedocs.org/en/latest/configuration/index.html#contacts-completion

best,
/p
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: [alot] howto multiple selections

[Patrick Totzke]

Hi!

Quoting Philippe LeCavalier (2012-03-01 15:45:12)
>Is it possible to select more than one msg in search-mode to retag?

No, not yet: https://github.com/pazz/alot/issues/116

Please have a look at the issue tracker for other feature requests
or if you're looking for inspiration on how you could help :D

Best,
/p


___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: [alot] mails tagged 'killed' reappearing

[Patrick Totzke]

Hi Phil,

Quoting Philippe LeCavalier (2012-03-03 18:37:32)
>I can't tell if it's _all_ killed mails but I can say for sure that some
>tagged mails reappear in the inbox despite them tagged 'killed' and the
>search mode filter is inbox AND NOT tag:killed.

I presume by reappear you mean that once you receive another mail of that 
thread,
the thread ends up in your inbox again?

You should be aware that tag "killed" is not special in any kind, nor is 
"inbox":
They are just strings attached to mails that make it easier for you to filter
your mailboxes.

The feature you might expect: threads tagged "killed" once will never pop up in 
your inbox
is not a feature of notmuch or alot; it *might* be a feature of your tagging 
script
if you decide to organize your mails like that.

I can highly recommend Justus' "afew" tagging script 
https://github.com/teythoon/afew,
which can be configured to provide above mentioned functionality among other 
things.
For other solutions have a look at http://notmuchmail.org/initial_tagging/.

Best,
/p
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: [alot] mails tagged 'killed' reappearing

[Patrick Totzke]

Quoting Philippe LeCavalier (2012-03-05 13:07:39)
>Quoting Patrick Totzke (2012-03-05 05:04:21)
>>I presume by reappear you mean that once you receive another mail of that 
>>thread,
>>the thread ends up in your inbox again?
>Yes.
>>
>>You should be aware that tag "killed" is not special in any kind, nor is 
>>"inbox":
>>They are just strings attached to mails that make it easier for you to filter
>>your mailboxes.
>
>Yeah I know what you mean. But the search mode where the killed threads
>reappear is filtered "inbox AND NOT tag:killed" which is why I do
>presume they shouldn't 'reappear'.

Right, the command that alot fires initially - if not stated otherwise in the 
config or command line arguments - is
"search tag:inbox AND NOT tag:killed", which opens a new search buffer for 
query "tag:inbox AND NOT tag:killed".

We might consider changing this, as it indeed suggest that inbox or killed tags 
are special in some sense.
I'm not sure if its a great idea to not fire any command initially by default. 
After all,
people don't start up theyr mail client and want to stare at a blank terminal.

Perhaps we could drop the "AND NOT tag:killed" here, I don't really have strong 
feelings about that.
To change the initial command edit the "initial_command" config option:
  http://alot.readthedocs.org/en/latest/configuration/index.html#initial-command

best,
/p
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: [alot] mails tagged 'killed' reappearing

[Patrick Totzke]

Quoting Philippe LeCavalier (2012-03-06 15:54:50)
>Quoting Patrick Totzke (2012-03-05 05:04:21)
>>I can highly recommend Justus' "afew" tagging script 
>>https://github.com/teythoon/afew,
>>which can be configured to provide above mentioned functionality among other 
>>things.
>>For other solutions have a look at http://notmuchmail.org/initial_tagging/.
>
>I can't seem to get dbacl in Fedora...Slowly starting to regret
>switching from Deb. Any ideas?

afaik you don't need dbacl for afew to run if you're not using this bayesian 
filtering foo (which i don't)
/p
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: [alot] reply from search mode

[Patrick Totzke]

Quoting Philippe LeCavalier (2012-03-06 16:00:16)
>How would I add a reply key binding to reply directly from the search
>mode. ie something like R. Forward and editnew would be nice as well.

This could indeed be useful, but I'm not convinced this is a great idea in 
general:
What would that command do if your selection is a thread with multiple 
leaf-mails?
reply to the first? ask the user?

cheers,
/p


___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[alot] announcing v0.3

[Patrick Totzke]

Hi everyone!

I have just released version 0.3 of my terminal GUI `alot`; You can get a 
tarball here [0].
This release features many bugfixes, a revised config syntax and a fairly 
complete user and API
manual [1].

Detailed usage updates since v0.21:
* revised config syntax!
* config file validation, better feedback on malformed configs
* themes read from separate files in their own (validated) syntax
* complete mailcap compatibility
* user manual
* direct addressbook type that parses `abook`s contacts
* completion for multiple recipients via AbooksCompleter
* completion for optional command parameter
* generate and set a Message-ID header when constructing mails
* add User-Agent header by default
* add sent and saved draft mails to the notmuch index and add custom tags
* guess file encodings with libmagic
* new thread mode command: "remove" to delete messages from the index
* new thread mode command: "editnew" e.g. to continue drafts (bound to 'n')
* new thread mode command: "togglesource" to display raw messages (bound to 'h')
* virtual "Tags" header for print and pipeto commands via --add_tags parameter
* much improved pipeto command in thread mode
* --spawn parameter for reply,forward,compose,editnew in thread mode
* --no-flush parameter for delayed flushing in tag,untag,toggletags commands
* make "signature as attachment" configurable; --omit_signature parameter for 
compose
* new envelope command: "save" to save as draft (bound to 'P')
* --no-refocus and --spawn parameter for edit in envelope mode
* header key completion for set/unset in envelope buffer
* "Me" substitution for ones own name/address in authors string
* new search mode command and search argument: "sort"
* renamed search mode command 'toggletag' to "toggletags"
* new search mode commands: "tag" and "untag"
* custom tagstring representation: hiding, substitution, colours, multi-matching

Thanks goes to those who contributed:
$git shortlog -sen 0.21..| tail -n +2
52  dtk 
34  Daniel 
15  Justus Winter <4win...@informatik.uni-hamburg.de>
 4  bjoernb 
 3  Stanislav Ochotnicky 
 2  Tom Prince ,
..everyone who send feedback and of course to all notmuch contributors.


After this release I will switch the git branching model [3], so expect the 
"testing" branch to
retire soon.

For the curious: The README.md lists some future goals together with their 
envisioned milestone
and pointers proof-of-concept implementations.

I could use some help from packagers. It should be relatively straight forward 
to package alot
but I haven't bothered to read up on the python packaging docs for 
$YOUR_DISTRI. A minor
complication could be that the docs should be generated at build time, which 
would introduce
some build-dependencies (mostly sphinx) that aren't runtime deps. The 
installation section in
the manual lists all dependencies, licence is GPL3+, let me know if you need 
any more info.

As always, don't hesitate to send feedback, bug reports, feature or pull 
requests via the
projects github page [2].

Cheers,
/p

[0]: https://github.com/pazz/alot/tarball/0.3
[1]: http://alot.rtfd.org
[2]: https://github.com/pazz/alot
[3]: http://blogpro.toutantic.net/2012/01/02/another-git-branching-model/
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: [RFC] Split notmuch_database_close into two functions

[Patrick Totzke]

Hi all,

I can confirm that this fixes a rather nasty core dump in a branch of alot
that closes and re-opens the database more frequently.
/p
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[RFC][alot] design decisions

[Patrick Totzke]

Hi all,

with alot 0.3 released, I started thinking seriously about proper™ MIME-display 
and gnupg
integration for alot. I have to make a few design decisions here and could 
really use
some informed opinions from more experienced UI developers. My question in 
particular is: 

 How do I best integrate calls to external mime-handlers into my MVC event 
driven code?

I mostly know how to implement the individual parts but I struggle coming up 
with a
consistent way of organizing it all without compromising my current layout.


Background
--

Model. Alot uses an abstraction layer over notmuch's thread, message and 
database objects
to make them less fragile and generally behave more OOP-y: You can e.g. 
directly call
message.add_tags(['foo','bar']) and so on. This is made possible by a 
"DatabaseManager"
that maintains centralized and queued access to the notmuch index and deals 
with notmuch
exceptions like a locked index [0].

View/Controler. We use an event-driven interface (urwid.TwistedEventLoop, [1]) 
that
handles user input and maintains a tree of widgets that knows how to render 
itself.
Twisted offers a neat concept of "Deferreds" to organize call- and errbacks 
that we
already make use of [2].  Alot has a "MessageWidget" that displays a single 
message.

We can easily access and parse email messages into MIME-trees (msg.get_email() 
returns a
email.message object; pythons email module allows extensive and RFC compliant 
dealing with
this [3]).

We already offer full compatibility with the mailcap protocol to constuct shell 
commands
of external MIME handlers.

We know how to call external commands asynchronously; 
`alot.helper.call_cmd_async` does this
and returns a Deferred to which we can connect callback functions.


Objective
-
We want to turn a email.message into nicely rendered text and display it as a 
widget.
Calls to renderers should be done asynchronously, so that displaying large 
threads
does not block the interface unnecessarily.
The text representation should be stored in a way that can be incrementally 
updated
for use with message parts decrypted at a later point.


Bad Solutions
-

1. Current solution: walk the MIME-tree depth-first, use blocking calls to 
handlers
and create a fixed "body" string that is displayed in the widget.
A silly and hackish solution that we want to get rid of for obvious reasons.

2. Use a "PartWidget" that is able to display a node in the MIME-tree:
Its constructor creates the representation in a RFC compliant way, that is,
it decodes 'text/plain' parts, stacks other widgets of the same kind on top of 
each other
for multipart parts or calls external renderer to get a text representation.

Cons:
 * widgets should not be clever and contain/construct any kind of additional 
info
 * its hard to update these widgets when a part gets decrypted; 
 * decrypted parts are not available outside the displaying widget, widgets get
   replaced completely when one rebuilds the buffer (refresh cmd)
Pros:
 * we can use Deferreds relatively painlessly in the ui (as opposed to the 
db-wrapper)

3. Accumulate rendered text in a tree structure *inside* message-objects and 
let that
massage update the widget that displays it. One could let the widget register
some update-callback with the widget that rebuilds the widgets content.
These callbacks are called after the external handler has finished and the 
message-
internal text-representation has been updated.

Pros: 
 * rendered text/decrypted message parts end up in the message and are 
available to process
   and redisplay at will
Cons:
 * the calls to MIME-handlers are dealt with as Deferreds which depend on the 
twisted event
   loop and indirectly on urwid. This is exactly the kind of tight coupling I'm 
trying to avoid:
   Messages should be independent of the interface.
 * Messages with multiple externally rendered parts will rebuild their widgets 
more than once
 * references to old widgets (replaced e.g. after a buffer rebuild) will be 
kept, which keeps
   them from being garbage-collected

I'm surely missing an easy and elegant solution here.
Any pointers or comments are much appreciated.
Thanks,
/p


[0]: http://alot.readthedocs.org/en/latest/api/database.html
[1]: http://excess.org/urwid/
[2]: http://twistedmatrix.com/documents/current/core/howto/defer.html
[3]: http://docs.python.org/library/email
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: Questions from a user new to notmuch

[Patrick Totzke]

Quoting Austin Clements (2012-03-22 22:39:07)
>> 2. I received a message that was addressed to a distribution group and
>>tried to reply.  Because the TO: address is not my address, notmuch
>>fails to guess the proper FROM: address to set.  Is there a way to
>>handle this use case?  This is with the emacs notmuch client.
>
>Not in general, since notmuch doesn't know where the mail was
>addressed to.  I believe some people have solved problems like this
>using Emacs hooks, but I don't know the details.

Does the notmuch CLI not use the 'Delivered-To' header for this?
/p
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: Passwordless multi-account SMTP

[Patrick Totzke]

Hi Jacek,

I am using msmtp for this: I keep my passwords in an encrypted
file and use a shellscript to decrypt and grep for passwords
in combination with msmtp's `passwordeval` option.
Have a look at

https://github.com/pazz/configs/blob/master/.msmtprc
https://github.com/pazz/scripts/blob/master/getpwd

I have a similar setup for offlineimap (configs also on github).
All this works quite nicely together with my ssh-agent (gnome-keyring-daemon)
which unlocks my gnupg keys upon login.
HTH,
/p


Quoting Jacek Generowicz (2012-04-03 16:29:07)
>Hello,
>
>I'm trying to use notmuch from Emacs. I have a number of SMTP accounts
>through which I would need to send mail. Suggested solutions I have
>found fall into 3 categories:
>
>+ Gnus posting styles. (I'd rather stay clear of gnus, at present.)
>
>+ Writing plaintext passwords in the MSMTP config files. (I'm rather
>  relutctant to leave plaintext passwords lying around.)
>
>+ Emacs' own SMTP library. (I don't understand how to get it to deal
>  with many accounts in a non-gnus based way. It also seems to be
>  undergoing an interface change in the transition from Emacs 23 to
>  24, and it was already confusing enough without that complication.)
>
>To further complicate matters, I'm looking for a solution which queues
>any messages which are sent while the machine is offline. (It seems
>that MSMTP and Emacs SMTP library do offer some help in those
>respects.)
>
>Hints gratefully received.
>
>Thank you.
>___
>notmuch mailing list
>notmuch@notmuchmail.org
>http://notmuchmail.org/mailman/listinfo/notmuch
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[alot] announcing v0.3.1

[Patrick Totzke]

Hi everyone!

I have just released alot v0.3.1; You can get a tarball here [0].
This is mainly a bugfix release, the most notable addition feature-wise is
the ability to sign outgoing messages via PGP/MIME. Cheers to Michael for this 
one!

Detailed usage updates since v0.3:
* use separate database for each write-queue entry when flushing
* fix behaviour of editor spawning
* fix opening of attachments in thread buffer
* fix pre_edit_translate hook
* fix opening of attachments without filename Content-Disposition parm
* clean up and complete theming (bindings help/envelope/mainframe body)
* fix datetime decoding issues
* fix abort commands on pre-hook exceptions
* fix correct default sendmail command to 'sendmail -t'
* use '> ' instead of '>' to quote in replies/fwds
* fix path completer wrt spaces in paths
* fix UI when no buffers are open
* fix issue with buffer type changing between flushes
* support multiple addresses per abook contact when using 'abook' completer
* smarter timestamp pretty printer
* new hook 'timestamp_format'
* merge multiple cc/to headers into one when displaying
* respect NOTMUCH_CONFIG env var
* smarter parsing of edited header values
* allow for singleton lists without trailing comma in config
* fix reverse-date sorted content in threadline if displayed
* emacs-style C-a and C-E in prompts
* added ability to sign outgoing mails via PGP/MIME

Thanks goes to those who contributed:
$ git shortlog -s -n 0.3.. |tail -n +2
24  Michael Stapelberg
 5  Jakob
..everyone who send feedback and of course to all notmuch contributors.

With David Bremners help I have started hacking together some metadata to build
a debian package. This can be found in branch 'debian' but I presume there are 
still
some things to be done on that (manpage generation..).

As always, don't hesitate to send feedback, bug reports, feature or pull 
requests via the
projects github page [1].

Cheers,
/p

[0]: https://github.com/pazz/alot/tarball/0.3.1
[1]: https://github.com/pazz/alot


signature.asc
Description: signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: [alot] announcing v0.3.1

[Patrick Totzke]

Quoting Jameson Graef Rollins (2012-05-17 16:42:53)
> On Thu, May 17 2012, Patrick Totzke  wrote:
> > With David Bremners help I have started hacking together some metadata to 
> > build
> > a debian package. This can be found in branch 'debian' but I presume there 
> > are still
> > some things to be done on that (manpage generation..).
> 
> Hi, Patrick.  I put together some Debian packaging a while back that I
> never did anything with:
> 
> git://finestructure.net/alot [debian]
> 
> It's a bit old, but it's mostly complete.  Hopefully you'll find it
> helpful.

Hey Jamie,

Ah yes, I forgot about this. Thanks for reminding me!
IIRC, I could not get this to "run through" on my machine but thats surely due
to my limited knowledge about debian packaging.
I'll have a look at this again and compare it with my stuff.
Cheers,
/p


signature.asc
Description: signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: Sending from multiple accounts

[Patrick Totzke]

try alot: multiple accounts [0] for free 
/p

[0] http://alot.readthedocs.org/en/latest/configuration/index.html#accounts

Quoting Michal Sojka (2012-05-23 11:29:27)
> Hi Doug,
> 
> Doug Penner  writes:
> > I am in the process of trying to migrate from alpine (which handles 
> > separate accounts *very* distinctly) to notmuch (which combines them). I 
> > have no problem with incoming e-mails being combined (actually looking 
> > forward to it) but am having a little trouble configuring outgoing e-mails 
> > to work correctly.
> >
> > I am currently using the vim interface (never like emacs, sorry) and have 
> > noticed that when replying, notmuch correctly uses the appropriate e-mail 
> > address (as defined in the To: or Cc: fields of the original), but does 
> > not have the ability to change the sender's "name".
> >
> > Ex: My "name" for my gmail account is "Doug Penner" (as you probably 
> > noticed with this on), but I also manage some other accounts where I do 
> > not go by this name. I manage a local organization's e-mail and need my 
> > sending name to be non-personal (ex: "This organization's Webmaster"). I 
> > can manually change the name before sending, but this gets very 
> > repetitive and annoying.
> >
> >
> > Is there any way to add names to the "alternate e-mails" setting in the 
> > .notmuch-config file? 
> 
> I don't think so. I use some elisp hooks within emacs interface for
> similar functionality but this wouldn't help in your case.
> 
> > I've tried switching to the standard "My Name"  and it then
> > completely fails to even detect that e-mail address as mine.
> 
> This should not be too hard to implement. There are gmime functions for
> email address manipulations which can be used to parse email addresses
> from .notmuch-config and to separate names from addresses.
> 
> Regards,
> -Michal
> ___
> notmuch mailing list
> notmuch@notmuchmail.org
> http://notmuchmail.org/mailman/listinfo/notmuch


signature.asc
Description: signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[alot] announcing v0.3.2

[Patrick Totzke]

Hi everyone!

I have just tagged alot v0.3.2; You can get a tarball here [0].
This version comes with redesigned theming features and supports
highlighting/retheming of thread lines in search mode based on
notmuch queries or thread tags.

I had to change the syntax of theme-files and custom tags-sections
of the config for this, so you'll have to revise your config.
The good news is that there are converter scripts included
to help you out with this:

* extra/tagsections_convert.py for your ~/.config/alot/config and 
* extra/theme_convert.py to update your custom theme files.

This release also features a colour picker (slightly modified version from [2])
that might turn out helpful when writing themes.
Check out extra/themes for some themes I made, and consider posting yours
for inclusion there..

Detailed usage updates since v0.3.1:
* fix bad GPG signatures for mails with attachments
* new theme-files + tags section syntax
* re-introduce "highlighting" of thread lines in search mode
* new global command "call" to directly call and bind python commands
* add new buffers to direct neighbourhood of current one
* fix sanitize --spawn for X11-less use
* add new hook 'touch_external_cmdlist'
* make statusline configurable
* fix update result count after tag operations in search mode
* add config options and hooks for reply/forward subject generation
* add config options and hook for quoting messages in replies/forwards
* allow True/False/None values for boolean command parameters
* new config option "attachment_prefix"
* various small fixes for libmagic, header encoding and decoding

Thanks goes to those who contributed:
$ git shortlog -s -n 0.3.1...|tail -n +2
10  Thomas Weißschuh
 3  Michael Stapelberg
 1  bjoernb

.. and everyone who send feedback!

As always, don't hesitate to file bug reports, feature or pull requests via the
projects github page [1].

Cheers,
/p

[0]: https://github.com/pazz/alot/tarball/0.3.2
[1]: https://github.com/pazz/alot
[2]: https://github.com/wardi/urwid/blob/master/examples/palette_test.py


signature.asc
Description: signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: [announce] Bower 0.4

[Patrick Totzke]

Quoting David Froger (2012-08-12 16:53:43)
> > > 3. Would it be possible to have the feature 'kill a thread' like Sup 
> > > have? (see
> > > http://sup.rubyforge.org/README.txt).
> > 
> > I haven't needed such a feature so I don't have a clear idea how it
> > should work.  There are some patches/ideas about muting threads using
> > notmuch generally around here.  I guess the simplest thing to do is hide
> > a thread if it has a certain tag, e.g. "muted" as suggested by notmuch
> > TODO file, unless you search for it explicitly.
> I  like this  solution,  I will  use it (adding 'and not tag:mutted' at the 
> end of my search).
> When  I'm not  interessed with  a long discussion on a mailling list,  I like 
> to  be able to 'kill'
> the thread so it does not reappear with every new mail.

I'm not sure if this actually solves your problem, because
notmuch's search is message, not thread based:
If you search for threads that match "is:inbox and not is:muted", you will see 
threads
that contain messages which are tagged with "inbox" and not with "muted".
This means if you just tag one (or all) msg in a thread "muted" and there is a 
new response,
this thread will contain such a matching message again and show up in your 
result list.

To make this work, you need to set up your tagging script (that you run after 
'notmuch new')
so that it updates those new messages accordingly. If a msg belongs to a thread 
that has 
muted messages, also tag the new one muted.
I can pretty much recommend Justus' tagging script afew: 
https://github.com/teythoon/afew
One of the filters that it supports out of the box is exactly the "kill 
threads" function
you want.

HTH,
/p


signature.asc
Description: signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[alot] announcing v0.3.3

[Patrick Totzke]

Good news everyone!

I have just tagged alot v0.3.3; You can get a tarball here [0].
This version comes with lots of bugfixes and minor ui tweaks, most notably the
ability to interpret command sequences and sequences of keypresses.

It is now possible to overwrite default bindings which obsoletes the assumption
that all bindings are given in the users config. Therefore, we don't write a
config stub on startup if no user config file exists.

Detailed usage updates since v0.3.2:
* interpret (semicolon separated) sequences of commands
* new input handling: allow for binding sequences of keypresses
* add ability to overwrite default bindings
* remove tempfiles (email drafts) as late as possible for better error recovery
* confirmation prompt when closing unsent envelopes
* prevent accidental double sendout of envelopes
* fix focus placement after tagcommand on last entry in search buffer
* new command 'buffer' that can directly jump to buffer with given number
* extra: sup theme
* fix tagstring sorting in taglist buffer
* update docs
* lots of internal cleanups
* search buffer theming fixes (alignment of threadline parts)
* fix help box theming
* comma-separate virtual "Tags" header added before printing mails
* fix pipeto command for interactive (foreground) shell commands
* handle possible errors occurring while saving mails
* indicate (yet uninterpreted) input queue in the status bar
* handle python exceptions that occur during 'call' command

Thanks goes to those who contributed:
$ git shortlog -s -n 0.3.2...|tail -n +2
 5  Kazuo Teramoto
 3  Simon Chopin

.. and everyone who send feedback!

As always, don't hesitate to file bug reports, feature or pull requests via the
projects github page [1].

Cheers,
/p

[0]: https://github.com/pazz/alot/tarball/0.3.3
[1]: https://github.com/pazz/alot


signature.asc
Description: signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: [alot] announcing v0.3.3

[Patrick Totzke]

Hi Sepp, welcome to alot :)

Quoting Sepp Tannhuber (2012-09-10 19:55:00)
> 1. As vi user I alway type gg/G to jump to the top/bottom. Is there a command 
> that I can
>    configure this way? I have not found a jump to top/bottom command in the 
> manual.
Short Answer:
This is not yet possible with the current design, no.
It is something i'd also like to see though and i have already a few ideas ho 
to do this.

Long version/dev info:
First of all, i only recently added the ability to interpret "gg" (or other 
multi-key strokes)
at all.. so thats done.
Now both for thread and search mode we use a toplevel `urwid.ListBox` widget 
iirc,
and those do not directly support "focus topmost line". But they do allow to
move the focus to some index and so implementing such a `MoveCommand` should 
not be hard.
One reason I have not bothered to implement this so far is that in the long run,
I want to have urwid Tree widgets as top level widgets in thread buffers.
This would also allow "move to next sibling/parent" and so on.

> 2. Mutt has a nice feature, the Attach: pseudo header. It can be used to 
> attach files from
>    within the editor. I use this small vi macro to attach files:
>      " mutt: insert attachment with ranger
> 
>      fun! RangerMuttAttach()
>      if filereadable('/tmp/chosendir')
>      silent !ranger --choosefiles=/tmp/chosenfiles --choosedir=/tmp/chosendir 
> "$(cat /tmp/chosendir)"
>      else
>      silent !ranger --choosefiles=/tmp/chosenfiles --choosedir=/tmp/chosendir
>      endif
>      if filereadable('/tmp/chosenfiles')
>      call append(3, map(readfile('/tmp/chosenfiles'), '"Attach: 
> ".escape(v:val," \\")'))
>      call system('rm /tmp/chosenfiles')
>      endif
>      redraw!
>      endfun
>      map  :call RangerMuttAttach()
>      imap  :call RangerMuttAttach()a
>    I would like to use the same with alot. Is it possible?

Nice idea!
So you can of course use this method to set an "Attach" header, but as of now
alot will not interpret and remove this when reading the edited text.
I guess one would also want that in the other direction if one really uses this 
feature:
When re-editing the message body, the editable headers part of the text should 
include
an Attach header line that lists the current attachments..

> 3. I have followed the instructions to use abook for contact completion. I 
> have used abook with mutt as well.
>    So I have put these two lines in my config:
>      [[[abook]]]
>        type = abook
>    But I did not find a way how to use it. Can anybody tell me?

It will be used for tabcompletion when you compose a mesage.
When you see the "to>" prompt, type in some prefix of a (realname part of a)
contact from abook and hit .

Cheers,
/p


signature.asc
Description: signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: [alot] announcing v0.3.3

[Patrick Totzke]

Quoting Sepp Tannhuber (2012-09-11 17:59:12)
> Hi Patrick,
> 
> thank you for answering and for this really good mail client!
pleasure :)

> > One reason I have not bothered to implement this so far is that in the long 
> > run,
> > I want to have urwid Tree widgets as top level widgets in thread buffers.
> > This would also allow "move to next sibling/parent" and so on.
> I did not know that it would be so complicated. But nice to hear that 
> development
> is going on.
Yea, trees in urwid are a bit messed up atm. alot's thread buffer uses its own
archaic widget layout, which is eventually to be replaced.

> > So you can of course use this method to set an "Attach" header, but as of 
> > now
> > alot will not interpret and remove this when reading the edited text.
> Exactly!

I think this will not so hard to do. Could you open up a feature request issue
on github for this?

> > When you see the "to>" prompt, type in some prefix of a (realname part of a)
> > contact from abook and hit .
> This was my assumption. But it is not working here. Obviously I make a 
> mistake. Here is my
> config:
> -- BEGIN ~/.config/alot/config --
> theme = solarized_dark
> editor_cmd = 'vim +/^$ "+normal j" "+:set textwidth=80" "+:set ft=mail"'
> 
> [accounts]
> 
>     [[work]]
>         realname = Joseph Tannhuber
>         address = jtannhu...@work.de
>         signature = ~/.signature
>         sendmail_command = msmtp --account=work -t
>         sent_box = maildir:///home/sepp/Maildir/Sent
>         draft_box = maildir:///home/sepp/Maildir/Drafts
>         [[[abook]]]
>             type = abook
> -- END --
> Is there anything missing?

No, that's what I use as well.
Note that this variant assumes your contact file is ~/.abook/addressbook. I'm 
not aware of format
issues for different abook versions so far. Also note that this will complete 
case-sensitive
strings of the format "Firstname Lastname ".

To debug this you can replay this introspect session and see how far you get.
Start alots python shell (pyshell command), then:

>>> from alot.settings import settings
>>> settings.get_addressbooks()
[, 
]
>>> settings.get_addressbooks()[0]

>>> A=settings.get_addressbooks()[0]
>>> A.get_contacts()
[... CONTACT TUPLES HERE...]

Hope this helps. Otherwise don't hesitate to open a bug report on github.

> By the way: Is it possible to set the X11 urgency hint flag when new mail 
> arrives? In principle
> one can use the visual bell in the terminal. Then alot must only invoke a 
> shell command:
>   echo -e "\b"

Well, alot does not pull your mails itself, so has no means of knowing when new 
mail arrives.
I personally use a mail indicator widget I wrote for my window manager that 
periodically
checks the notmuch index for new mail: 
http://awesome.naquadah.org/wiki/Notmuch_mail_integration

Alot can of course fire shell commands. What it is lacking at the moment is the 
ability to periodically
fire custom (alot-)commands. This feature would not be hard to implement, but 
so far I did not bother
to do so as I considered mail-fetching not part of alots responsibility.
But you're not the first one to want this, so I guess it'd be worth it.

Cheers,
/p


signature.asc
Description: signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: nbook: a notmuch based address book written in python

[Patrick Totzke]

Hey Suvayu, welcome to notmuch!

I hope you are aware that there are already a few search based abook tools
around for notmuch (listed in the wiki, albeit hidden in the emacs docs):
http://notmuchmail.org/emacstips/#index14h2
I personally use nottoomuch-addresses.sh, which apparently does some advanced
caching voodoo for speed.

But to your tool; practice test:
I wasn't able to use wildcards or simply prefixes of names. This is essential
if you want to use it for tabcompleting contacts in a MUA.
The time lookups take seems to depend on how many matches there are:

---
time nbook Suvayu
1 unique email addresses found for `Suvayu'
fatkasuvayu+li...@gmail.com Suvayu Ali

nbook Suvayu  0.04s user 0.01s system 95% cpu 0.050 total
---
time nbook Justus
...

nbook Justus  0.21s user 0.07s system 11% cpu 2.484 total
---
And If I look for my own name, this takes over a minute,
eventually dying. This could be an issue with libnotmuch though.
Possibly, your algorithm takes very long and then reads from an initially
opened Database object again, which was invalidated by concurrent writes of 
other processes..

---
[~] time nbook Patrick 

Error opening /home/pazz/mail/gmail/[Google Mail].All 
Mail/cur/1330682270_0.12958.megatron,U=8766,FMD5=66ff6a8bc18a8a3ac4b311daa93d358a:2,S:
 Too many open files
Traceback (most recent call last):
  File "/home/pazz/bin/nbook", line 167, in 
  File "/home/pazz/bin/nbook", line 71, in __init__
  File "/home/pazz/.local/lib/python2.7/site-packages/notmuch/message.py", line 
233, in get_header
notmuch.errors.NullPointerError
Error in sys.excepthook:
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/apport_python_hook.py", line 66, in 
apport_excepthook
ImportError: No module named fileutils

Original exception was:
Traceback (most recent call last):
  File "/home/pazz/bin/nbook", line 167, in 
  File "/home/pazz/bin/nbook", line 71, in __init__
  File "/home/pazz/.local/lib/python2.7/site-packages/notmuch/message.py", line 
233, in get_header
notmuch.errors.NullPointerError
nbook Patrick  3.20s user 5.47s system 12% cpu 1:11.65 total


Anyway, have fun hacking notmuch! If you are looking for a related project to 
bring in your python skills
I could think of one or two :D
Best,
/p
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: nbook: a notmuch based address book written in python

[Patrick Totzke]

Quoting Suvayu Ali (2012-10-08 10:34:29)
> Hi Patrick,
> 
> Sorry for the very late reply; I got distracted with some personal
> matters.
> 
> On Tue, Sep 25, 2012 at 11:44:57AM +0100, Patrick Totzke wrote:
> > Hey Suvayu, welcome to notmuch!
> > 
> > I hope you are aware that there are already a few search based abook tools
> > around for notmuch (listed in the wiki, albeit hidden in the emacs docs):
> > http://notmuchmail.org/emacstips/#index14h2
> > I personally use nottoomuch-addresses.sh, which apparently does some 
> > advanced
> > caching voodoo for speed.
> > 
> 
> I wasn't aware of either of them, thanks for pointing them out.  I'll
> take a look for inspiration and ideas.
> 
> > But to your tool; practice test:
> > I wasn't able to use wildcards or simply prefixes of names. This is 
> > essential
> > if you want to use it for tabcompleting contacts in a MUA.
> 
> Since the idea was inspired by the completion on the Gmail web
> interface, I already do a partial search so wildcards should not be
> necessary.

Not sure what you mean here: If I compose a mail using gmails web interface
and type a prefix of someone's name I will get this contect as a suggestion.
My point was that using your tool, I did not get a contact suggested
for all prefixes.

> > The time lookups take seems to depend on how many matches there are:
> > 
> > ---
> > time nbook Suvayu
> > 1 unique email addresses found for `Suvayu'
> > fatkasuvayu+li...@gmail.com Suvayu Ali
> > 
> > nbook Suvayu  0.04s user 0.01s system 95% cpu 0.050 total
> > ---
> > time nbook Justus
> > ...
> > 
> > nbook Justus  0.21s user 0.07s system 11% cpu 2.484 total
> > ---
> 
> Yes, I noticed this too when I searched for the more common names.  Not
> sure how to get around this though.

I think this is a conceptual problem with your algorithm:
You look up *all* messages and add a name to your result-list
if it matches. This means you go through some condidate
as often as you index contains mails from/to him.
What one really wants is to ask the database to do something like
  "SELECT name,email from RECIPIENTS_OR_SENDER"
where RECIPIENTS_OR_SENDER is some imaginary list that stores
a set of contacts.

Bottom line: One would have to change the layout of the underlying
database (not likely) or do regularly update some cache
and only work on that. This is what some of the mentioned tools do if i'm not 
mistaken.

> > And If I look for my own name, this takes over a minute,
> > eventually dying. This could be an issue with libnotmuch though.
> > Possibly, your algorithm takes very long and then reads from an initially
> > opened Database object again, which was invalidated by concurrent writes of 
> > other processes..
> > 
> > ---
> > [~] time nbook Patrick 
> > 
> > Error opening /home/pazz/mail/gmail/[Google Mail].All 
> > Mail/cur/1330682270_0.12958.megatron,U=8766,FMD5=66ff6a8bc18a8a3ac4b311daa93d358a:2,S:
> >  Too many open files
> > Traceback (most recent call last):
> >   File "/home/pazz/bin/nbook", line 167, in 
> >   File "/home/pazz/bin/nbook", line 71, in __init__
> >   File "/home/pazz/.local/lib/python2.7/site-packages/notmuch/message.py", 
> > line 233, in get_header
> > notmuch.errors.NullPointerError
> > Error in sys.excepthook:
> > Traceback (most recent call last):
> >   File "/usr/lib/python2.7/dist-packages/apport_python_hook.py", line 66, 
> > in apport_excepthook
> > ImportError: No module named fileutils
> > 
> > Original exception was:
> > Traceback (most recent call last):
> >   File "/home/pazz/bin/nbook", line 167, in 
> >   File "/home/pazz/bin/nbook", line 71, in __init__
> >   File "/home/pazz/.local/lib/python2.7/site-packages/notmuch/message.py", 
> > line 233, in get_header
> > notmuch.errors.NullPointerError
> > nbook Patrick  3.20s user 5.47s system 12% cpu 1:11.65 total
> > 
> > 
> 
> Yes someone else pointed this out too.  Again I'm not sure how to
> proceed here.  I had a quick look at this last week and it seemed to me
> the limitation comes from within the python bindings for notmuch.  Do
> you have any ideas?

As mentioned before, I think you invalidate the Database object concurrently
while your long-running algorithm goes through all messages.
Xapian doesn't handle concurrent access to the index like a

Re: Notmuch indexing takes too long

[Patrick Totzke]

Hi Ondrej,

Quoting Ondrej Jombik (2012-11-09 02:58:09)
> I am trying to move from mairix to some better solution. mairix has been
> working really well for me, but it had some limitations.
> 
> I decided to give a try to notmuch, but I has been suprised with
> estimated indexing time:
> 
>  Processed 4157 of 822462 files (10h 3m 42s remaining).
> 
> At the beggining when there was 11 hours estimate I simply thought that
> those numbers are wrong. But they are not. This is really going to take
> that long. I could not believe that you all wait that long to index your
> maildirs:)
> 
> Also if I understand correctly the concept, whenever I want to
> regenerate database, I have to move old away and create new one,
> correct? mairix worked that way that you run it and it updated its own
> database. Is it possible with notmuch as well? I simply do not want to
> wait 10+ hours for every database regeneration, thought I am willing to
> wait that long for initial indexing.

Notmuch does incremental idnex updates once the initial index is build.
The initial indexing takes a while, depending on the number of mails you have
but 10 *hours* seems a little off. For me its ~5mins on the hdd and <2 on a ssd.

> My last question is related to notmuch and Alpine. Alpine is
> a IMAP/Maildir e-mail client based on old PINE. I get used to that so
> much that I cannot move away (to Mutt, for example). Is there anyone who
> uses notmuch with Alpine and created some scripts and/or key-bindings?

You might want to have a look at notmuchfs: https://github.com/tsto/notmuchfs
Also, there are a few other MUAs based on notmuch:
Check out http://notmuchmail.org/frontends/

Have fun,
/p
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: gmail importer script

[Patrick Totzke]

Is this supposed to happen?

-
Logging in...
Selecting all mail...
Discovering local messages...
Receiving message list...
Traceback (most recent call last):
  File "./gmail-notmuch.py", line 175, in 
main()
  File "./gmail-notmuch.py", line 58, in main
new_messages = discover_new_messages(imap, old_messages)
  File "./gmail-notmuch.py", line 80, in discover_new_messages
typ, data = imap.fetch("1:*", "X-GM-MSGID")
  File "/usr/lib/python2.7/imaplib.py", line 443, in fetch
typ, dat = self._simple_command(name, message_set, message_parts)
  File "/usr/lib/python2.7/imaplib.py", line 1070, in _simple_command
return self._command_complete(name, self._command(name, *args))
  File "/usr/lib/python2.7/imaplib.py", line 825, in _command
', '.join(Commands[name])))
imaplib.error: command FETCH illegal in state AUTH, only allowed in states 
SELECTED

--


In [2]: imaplib.__version__
Out[2]: '2.58'

One thing you also might want to watch out for is
exceptions due to a locked notmuch index.
rock on,
/p
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: gmail importer script

[Patrick Totzke]

Quoting Jason A. Donenfeld (2012-12-07 13:49:46)
> Not sure what is causing this. My best guess is that your password was
> incorrect and that I'm not checking the login return value.

Yes, you're right, it was an incorrect passwd.

> One thing you also might want to watch out for is
> exceptions due to a locked notmuch index.
> 
> 
> How do I do this?

See e.g. here: https://github.com/pazz/alot/blob/master/alot/db/manager.py#L163
I'm sure "afew" (https://github.com/teythoon/afew) does a similar thing.


I have two new errors:

-
./gmail-notmuch.py -u patricktot...@gmail.com -p mypwd ~/mail/gmail/
Logging in...
Selecting all mail...
Error opening database at /home/pazz/mail/gmail/.notmuch: No such file or 
directory
Traceback (most recent call last):
  File "./gmail-notmuch.py", line 225, in 
main()
  File "./gmail-notmuch.py", line 56, in main
database = notmuch.Database(destination, False, 
notmuch.Database.MODE.READ_WRITE)
  File "/home/pazz/.local/lib/python2.7/site-packages/notmuch/database.py", 
line 154, in __init__
self.open(path, mode)
  File "/home/pazz/.local/lib/python2.7/site-packages/notmuch/database.py", 
line 214, in open
raise NotmuchError(status)
notmuch.errors.FileError
-

I also tried with maildir param ~/mail/gmail/\[Google\ Mail\].All\ Mail
as this is where my all-mail maildir is, and i also tried an absolute path, all 
the same result.

If I point it to the root of my notmuch directory I get:
-
[~/projects/gmail-notmuch] ./gmail-notmuch.py -u patricktot...@gmail.com -p 
mypwd /home/pazz/mail
Logging in...
Selecting all mail...
Traceback (most recent call last):
  File "./gmail-notmuch.py", line 225, in 
main()
  File "./gmail-notmuch.py", line 58, in main
messages = discover_messages(imap, total)
  File "./gmail-notmuch.py", line 104, in discover_messages
new_readline.progressbar = create_progressbar("Receiving message list", 
total).start()
  File "./gmail-notmuch.py", line 147, in create_progressbar
return ProgressBar(maxval=total, widgets=[text + ": ", SimpleProgress(), 
Bar(), Percentage(), " ", ETA(), " ", FileTransferSpeed(unit="emails")])
TypeError: __init__() got an unexpected keyword argument 'unit'

BTW: I had to change the hardcoded "[Gmail]" to "[Google Mail]" for it to get 
that far.


HTH,
/p


signature.asc
Description: signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: gmail importer script

[Patrick Totzke]

Quoting Jason A. Donenfeld (2012-12-08 17:04:48)
> On Sat, Dec 8, 2012 at 5:20 PM, Patrick Totzke  
> wrote:
> > I have two new errors:
> >
> > -
> > ./gmail-notmuch.py -u patricktot...@gmail.com -p mypwd ~/mail/gmail/
> >
> > I also tried with maildir param ~/mail/gmail/\[Google\ Mail\].All\ Mail
> > as this is where my all-mail maildir is, and i also tried an absolute path, 
> > all the same result.
> >
> > If I point it to the root of my notmuch directory I get:
> 
> Pointing it at the root directory is what you want. In fact, I'm
> removing the argument from the script and having it just get that from
> the config. So the correct invocation will be:
> 
> ./gmail-notmuch.py -u blabla -p password

Okay..
> 
> >   File "./gmail-notmuch.py", line 147, in create_progressbar
> > return ProgressBar(maxval=total, widgets=[text + ": ", 
> > SimpleProgress(), Bar(), Percentage(), " ", ETA(), " ", 
> > FileTransferSpeed(unit="emails")])
> > TypeError: __init__() got an unexpected keyword argument 'unit'
> 
> You're probably running with an old version of python-statusbar.

I just installed it from ubuntu 12.04's repository. the version was 2.2-2.
Installed from pip now, and it seems ok. you'll want to make more specific 
dependencies at some
point..

> >
> > BTW: I had to change the hardcoded "[Gmail]" to "[Google Mail]" for it to 
> > get that far.
> 
> Really? Are you in the UK? I'll have to do this dynamically using the
> XLIST imap extension.

Yea, but its a *really* old account I first got from google.de.
Reading this dynamically would def. help.


next try..
-

Traceback (most recent call last):
  File "./gmail-notmuch.py", line 246, in 
main()
  File "./gmail-notmuch.py", line 74, in main
new_messages = retag_old_messages(database, messages, destination)
  File "./gmail-notmuch.py", line 176, in retag_old_messages
progressbar.start()
  File "/usr/local/lib/python2.7/dist-packages/progressbar/__init__.py", line 
311, in start
self.update(0)
  File "/usr/local/lib/python2.7/dist-packages/progressbar/__init__.py", line 
283, in update
self.fd.write(self._format_line() + '\r')
  File "/usr/local/lib/python2.7/dist-packages/progressbar/__init__.py", line 
243, in _format_line
widgets = ''.join(self._format_widgets())
  File "/usr/local/lib/python2.7/dist-packages/progressbar/__init__.py", line 
223, in _format_widgets
widget = format_updatable(widget, self)
  File "/usr/local/lib/python2.7/dist-packages/progressbar/widgets.py", line 
38, in format_updatable
if hasattr(updatable, 'update'): return updatable.update(pbar)
  File "/usr/local/lib/python2.7/dist-packages/progressbar/widgets.py", line 
184, in update
return '%3d%%' % pbar.percentage()
  File "/usr/local/lib/python2.7/dist-packages/progressbar/__init__.py", line 
208, in percentage
return self.currval * 100.0 / self.maxval
ZeroDivisionError: float division by zero
--



Another feature request:
Could you make the progressbar stuff optinal? in the end one will want to run 
this via cronjob
and this shouldn't print anything to stdout.
Also, I can pretty much recommend the 'logging' module.

cheers,
/p


signature.asc
Description: signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: gmail importer script

[Patrick Totzke]

Quoting Jason A. Donenfeld (2012-12-09 23:13:12)
> On Sat, Dec 8, 2012 at 6:46 PM, Patrick Totzke  
> wrote:
> 
> ZeroDivisionError: float division by zero
> 
> 
> Fixed it! Sorry about all this... Jeeze Louise. 

thx

> 
> Another feature request:
> Could you make the progressbar stuff optinal? in the end one will want to
> run this via cronjob
> and this shouldn't print anything to stdout.
> Also, I can pretty much recommend the 'logging' module.
> 
> 
> Sure, but... >/dev/null doesn't cut it? I can add some more refined logging
> capabilities if you want.
It would, but its nicer not to load ressources you're not gonna use. Also, 
progressbar 2.2 seems to be the newest version you see on a reasonably new 
debian/ubuntu install.. 


Next exception.. do you have a bugtracker? (/me <3 github)

Logging in...
Selecting all mail...
Receiving message list: 24455 of 
24455||100% Time: 0:00:12   2.04 
kemails/s
Parsing message list and labels...
Searching for local messages...
Traceback (most recent call last):
  File "./gmail-notmuch.py", line 248, in 
main()
  File "./gmail-notmuch.py", line 75, in main
if len(new_messages) == 0:
TypeError: object of type 'NoneType' has no len()



/p
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: gmail importer script

[Patrick Totzke]

Quoting Jason A. Donenfeld (2012-12-11 07:06:22)
> On Mon, Dec 10, 2012 at 6:46 PM, Jason A. Donenfeld  wrote:
> 
> 
> 
> On Mon, Dec 10, 2012 at 10:41 AM, Patrick Totzke 
> wrote:
> 
> It would, but its nicer not to load ressources you're not gonna use.
> Also, progressbar 2.2 seems to be the newest version you see on a
> reasonably new debian/ubuntu install.. 
> 
> 
> I'll consider it.
> 
> 
> --silent or -s now silences stdout.

It seems to work, but i'm still waiting for the first run-through.

Downloading messages: 457 of 22831|

This takes ages. I hope it doesn't try to re-download all my messages everytime.

Another thing:
It seems that throughout its run, your script locks the notmuch database. This 
is particularly
annoying if you want to read/tag mails while waiting for a long sync-process. I 
think you only
really want to lock the index for a short time in the end.
Do you write to the index directly after a msg is downloaded?

/p
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: gmail importer script

[Patrick Totzke]

Quoting Jason A. Donenfeld (2012-12-11 09:12:56)
> It seems to work, but i'm still waiting for the first run-through.
> 
> 
> Downloading messages: 457 of 22831|
> 
> This takes ages. I hope it doesn't try to re-download all my messages
> everytime.
> 
> 
> Nope. It's very smart about not doing that.
>  

hmmk

I ended up killing the process after i saw that ETA was >3h. This broke my 
index!
afterwards, a `notmuch new` printed 

A Xapian exception occurred creating a directory: Expected block
485 to be level 1, not 0.

I wasnt able to properly read the index anymore and had to
restore an old index dump. not cool!

/p
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: notmuch python bindings corrupt db index (was: gmail importer script)

[Patrick Totzke]

Quoting Jason A. Donenfeld (2012-12-11 13:47:32)
> On Tue, Dec 11, 2012 at 11:04 AM, Patrick Totzke
>  wrote:
> > I ended up killing the process after i saw that ETA was >3h.
> 
> Gmail's bottleneck. Very unfortunate. Leave it going over night.

OK, i assumed i'd not have to download all messages because i already have them
on disk -- synced via offlineimap. but then i remembered that OI stores uses
its own naming scheme for the files in a maildir. so thats probably why your 
script
did not realize theyre already there. (maybe it should? it could look up the 
MIDs from notmuch)

> > This broke my index!
> >
> > afterwards, a `notmuch new` printed
> >
> > A Xapian exception occurred creating a directory: Expected block
> > 485 to be level 1, not 0.
> >
> > I wasnt able to properly read the index anymore and had to
> > restore an old index dump. not cool!
> 
> 
> Can you provide any more information about this? The script properly
> uses atomic transactions. Any corruption here is the result of a
> notmuch library bug, which should be investigated.

i am kind of reluctant to reproduce this again (on my work-desktop).
All i did was start your script, writing to the index in another terminal in 
between (using alot)
possibly my cronjob fired a "notmuch new; afew.." in between also.
when I killed your script the index war broken.

Can i turn on some global logging for notmuch to debug this?
Maybe libnotmuch had trouble with so many db-writes at a time?

/p




signature.asc
Description: signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: notmuch python bindings corrupt db index (was: gmail importer script)

[Patrick Totzke]

Quoting Jason A. Donenfeld (2012-12-13 14:32:53)
> On Wed, Dec 12, 2012 at 9:49 PM, Austin Clements  wrote:
> > There should be no way to corrupt the database at this level through
> > the Xapian API, which means nothing libnotmuch can do (much less users
> > of libnotmuch) should be able to corrupt the database.  If you can
> > reproduce the problem, it's probably a serious bug in Xapian, but it
> > could also have been a file system bug or even random file system
> > corruption.
> 
> Well that's... troubling.
> 
> Patrick: could you please backup and try to reproduce? Otherwise I'll
> assume this was a one-off situation.


I tried it again, concurrently triggered some modifications
and killed the process afer a while.
(my modifications were not flushed during the run of your script as
the index was locked)
The index seems to be OK. So I'm afraid I cannot reproduce this reliably.
/p


___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: gmail importer script

[Patrick Totzke]

Sorry, I'm misusing this thread as a bugtracker..


Traceback (most recent call last):4|   
| 10% ETA:  1:24:41   3.54  emails/s
  File "./gmail-notmuch.py", line 251, in 
main()
  File "./gmail-notmuch.py", line 89, in main
download_new_messages(imap, database, new_messages, destination)
  File "./gmail-notmuch.py", line 208, in download_new_messages
tag_message(database, dest, labels)
  File "./gmail-notmuch.py", line 168, in tag_message
raise e
notmuch.errors.FileNotEmailError



best,
/p
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: gmail importer script

[Patrick Totzke]

Quoting Jason A. Donenfeld (2012-12-15 08:22:46)
> On Thu, Dec 13, 2012 at 5:48 PM, Patrick Totzke 
> wrote:
> 
> 
> notmuch.errors.FileNotEmailError
> 
> 
> Delete the file it dies on and try again.

Well, thats not the point.. the script shouldn't die like this.
I think it's be better if the script caught that exception, deleted the file
and continued..

Thanks,
/p


signature.asc
Description: signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: gmail importer script

[Patrick Totzke]

Quoting Jason A. Donenfeld (2012-12-16 20:44:04)
> On Sat, Dec 15, 2012 at 11:41 AM, Patrick Totzke 
> wrote:
> 
> Well, thats not the point.. the script shouldn't die like this.
> I think it's be better if the script caught that exception, deleted the
> file
> and continued..
> 
> 
> Probably, but I suspect it's related to whatever mystery error you ran into
> before with the corruption.
> 
> Does deleting that file and trying again fix it? 

I don't know. How would I know which file to remove?
I just removed the newest files in new/cur/tmp
but this doesnt solve the issue.
What would really help here is some more informative error message!

I tried with the `-d` option but did not see any local path in the output.

> 
> Anyway, this is extremely stable for me and a few others at this point. I'm
> going to wait for other users to report errors. Alternatively, send me patches
> if you want things to happen.

Personally, I don't really care if 'things are happening', as I'm kind of OK 
with my current
offlineimap configuration and would seriously consider switching from OI
only if it meant an improvement in speed or stability.

I test this code because I believe that the notmuch ecosystem
would benefit from a working solution and that reviews and bug-reports
are important.

Of course, I don't expect any miracles in terms of speed at this early stage.
But the bug I reported above should be addressed at some point in order not to
scare off potential users.

Have you considered including your code in offlineimap?
I'm asking because OI already solved some of the problems you might face later 
on
if you want to continue working on this.

* multi-threaded downloads
* the ability to read passwords not as plaintext parameter..

best,
/p


signature.asc
Description: signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: gmail importer script

[Patrick Totzke]

BTW: I removed the whole maildir contents and restarted the downlowd over 
night. here's the result:

---
Traceback (most recent call last):2|
  | 26% ETA:  1:37:22   3.11  emails/s
  File "./gmail-notmuch.py", line 253, in 
main()
  File "./gmail-notmuch.py", line 89, in main
download_new_messages(imap, database, new_messages, destination)
  File "./gmail-notmuch.py", line 210, in download_new_messages
tag_message(database, dest, labels)
  File "./gmail-notmuch.py", line 168, in tag_message
raise e
notmuch.errors.FileNotEmailError
---

/p


signature.asc
Description: signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[alot] announcing v0.3.4

[Patrick Totzke]

Hi everyone!

I have just tagged alot v0.3.4; You can get a tarball here [0].
This minor version bump is mostly to flush out all the nice small features
I hacked together or received as pull request lately.

It comes with tons of ui tweaks, fixes and a very nice patch series
that allows to send PGP/MIME encrypted mails.

Not included, but here's a peek into alot's future for the curious:
I've been semi-busy, making alot's tree-widgets available for the wider
urwid audience [2] and am currently rewriting alot's thread buffers to make use 
of
this library [3]. This will make it easy™ to finally write a better mimeparser 
[4]
and display a messages MIME-tree as well as properly display nested 
encrypted/signed
mails..

More detailed usage updates since v0.3.3:
* extra: zsh completion file
* thread mode: add "tags" pseudo header to message display
* case insensitive matching in Addressbook completion
* compose: interpret "attach" pseudo header
* compose: set initial message tags
* envelope: completion for 'From'
* reply/forward: more flexible construction of "From" headers (hello plussing!)
* thread mode: added bounce command for direct redirection w/o an envelope 
buffer
* thread mode: more robust "pipeto" command
* add config option "prefer_plaintext"
* prevent multiple 'index locked' notifications
* kill some zombies! (#325)
* search mode: bulk tagging
* less annoying multi-key bindings
* add global "move" command for scriptable cursor movement
* support for encrypted outgoing mails using PGP/MIME

Thanks goes to those who contributed:
$ git shortlog -s -n 0.3.2...|tail -n +2

 30  Johannes Kulick
  9  William Erik Baxter
  5  Kazuo Teramoto
  4  Justus Winter
  3  Benjamin
  3  Simon Chopin
  2  Antoine Amarilli
  1  Johann Weging
  1  Michael Turquette
  1  bjoernb

.. and everyone who send feedback!

As always, don't hesitate to file bug reports, feature or pull requests via the
projects github page [1].

Cheers,
/p

[0]: https://github.com/pazz/alot/tarball/0.3.4
[1]: https://github.com/pazz/alot
[2]: https://github.com/pazz/urwidtrees-V2
[3]: https://github.com/pazz/alot/tree/0.3.3-feature-urwidtrees
[4]: https://github.com/pazz/alot/tree/0.3.3-mimeparser


signature.asc
Description: signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: afew filter rules

[Patrick Totzke]

Hey Reiner,

Quoting Rainer M Krug (2013-02-28 15:29:33)
> Based on the the filter, it seems that list_id is used to tag the emails - 
> correct?

Seems so, yea [0].

> But in
> addition, I have some lists which do not have list_id, but need to be 
> identified by their to
> address - how can I add these filters?

I use some custom filters like this in my setup [1].
Basically, you can add a custom filter to your filer chain by adding
a section with name "Filer.n" for increasing n..
Also check out afew's docs [2].

Cheers,
/p


[0]: 
https://github.com/teythoon/afew/blob/master/afew/filters/ListMailsFilter.py#L31
[1]: https://github.com/pazz/configs/blob/master/.config/afew/config
[2]: https://afew.readthedocs.org/en/latest/manual.html#customizing-filters


> Thanks,
> 
> Rainer
> 
> [1] https://github.com/teythoon/afew
> - -- 
> Rainer M. Krug, PhD (Conservation Ecology, SUN), MSc (Conservation Biology, 
> UCT), Dipl. Phys.
> (Germany)
> 
> Centre of Excellence for Invasion Biology
> Stellenbosch University
> South Africa
> 
> Tel :   +33 - (0)9 53 10 27 44
> Cell:   +33 - (0)6 85 62 59 98
> Fax :   +33 - (0)9 58 10 27 44
> 
> Fax (D):+49 - (0)3 21 21 25 22 44
> 
> email:  rai...@krugs.de
> 
> Skype:  RMkrug
> -BEGIN PGP SIGNATURE-
> Version: GnuPG v1.4.11 (GNU/Linux)
> Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
> 
> iQEcBAEBAgAGBQJRL3fcAAoJENvXNx4PUvmCO6oIALNAp5Z2SE6H2EqePpk0kwBc
> NCIK12IEyGzLya2BzCv87sucxyicCeXqBt5pNkGgsxK7pJxXaDQVaDtYZgguVfnB
> /IE1AgDCfgzUx+iPMS4knB9DaKiKHPIpuH1oL7PK3tbc332+zlWFxeeekJj9DvM9
> 53LZGr9oMQ74obX9vDnFcbHCNq/o2ALcNnnJlDRmTZFpFmSBxk6QxbWqlUZcRyGv
> R9CJEdx3+a1kQhLCwYN7CoX1LQx1y57fm2//qt4m7dn6ZWK1ueRneJRuVr/jobiy
> jKjRAI/fkT/HBk3l0eJ9fkAeOMZT5Ur/FXjipG8kUmwrnBojp6Eykk4NMIg+Mvo=
> =CoyQ
> -END PGP SIGNATURE-
> ___
> notmuch mailing list
> notmuch@notmuchmail.org
> http://notmuchmail.org/mailman/listinfo/notmuch


signature.asc
Description: signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: funcmail.py: generate notmuch-tags based on headers

[Patrick Totzke]

Could this be integrated into "afew" as a custom filter?
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: the future of notmuch-vim?

[Patrick Totzke]

Quoting guyzmo (2013-04-03 07:01:58)
> ...
> It may be nice and/or fun to use that kind  of  things  in  vim, but
> really, it's opposite to the philosophy of vim.
> ...
> Whereas you seem to have done a really good  job  integrating  it to
> vim,  I  personally  think  that  anything  that  makes  vim  an  IDE, a
> coffeemaker, or an Operating System is not worth the pain. So my opinion
> is to just drop vim-as-MUA script support.

Fullack! Thanks for spelling out my thoughts in great detail :D
I'm definitely for dropping support for all notmuch-vim scripts officially.
These scripts can live in their own repositories on github or alike
and we can link those in the wiki for completeness sake.
/p
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: the future of notmuch-vim?

[Patrick Totzke]

Quoting Felipe Contreras (2013-04-03 10:09:39)
> ...
> What you prefer is irrelevant; it's relevant only for you, that's why it's
> called a *preference*, the rest of us prefer different things.

Never mind preferences, I think originally, this thread was about dropping
*support* for the *original* vim plugin that lives in notmuch/contrib.
I think we can all agree that this is reasonable.

This should not keep you from hacking on your script, hosting
it whereever you like and support it.
For me, one of the charms of notmuch is exactly this extensibility.

> I've tried mutt-kz and alot, and I was utterly dissapointed by both.
> 
> See how snappy and fast notmuch-vim-ruby is[1], alot is much worst in these
> respects.

I always appreciate constructive feedback on alot..
regards,
/p


signature.asc
Description: signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: Xapers

[Patrick Totzke]

Quoting Jameson Graef Rollins (2013-04-15 18:18:00)
> On Mon, Apr 15 2013, Jameson Graef Rollins  wrote:
> > README and a screenshot of the curses ui here:
> >
> > http://finestructure.net/xapers/README
> > http://finestructure.net/xapers/screenshot.png
> 
> I maybe should have been a little clearer that all necessary
> instructions should be in the README, including the source repo:
> 
>   git://finestructure.net/xapers


Brilliant idea!

here's some early feedback:
* is there a way to batch import a bunch of pdfs? or better yet, a bibtex file 
with entries pointing
  to files like the ones produced by say jabref?
  There, entries have some line like this:
  file = {Boz2012.pdf:papers/Boz2012.pdf:PDF}

* got a bunch of traces:
  -
  $xapers import --source=ABK2011.pdf   
  1 ↵ master 
  Reading bibtex... done.
  Adding bibtex... 
  
  Traceback (most recent call last):
File "/home/pazz/.local/bin/xapers", line 145, in 
  docid = cli.add(query_string, infile=infile, source=source, tags=tags, 
prompt=prompt)
File "/home/pazz/.local/lib/python2.7/site-packages/xapers/cli/ui.py", line 
228, in add
  doc.add_bibtex(bibtex)
File "/home/pazz/.local/lib/python2.7/site-packages/xapers/documents.py", 
line 373, in add_bibtex
  self.bibentry = xapers.bibtex.Bibentry(bibtex)
File "/home/pazz/.local/lib/python2.7/site-packages/xapers/bibtex.py", line 
25, in __init__
  if os.path.exists(bibtex):
File "/usr/lib/python2.7/genericpath.py", line 18, in exists
  os.stat(path)
  TypeError: must be encoded string without NULL bytes, not str
  
  
  $ xapers show, then i moved around a bit

  
  Traceback (most recent call last):
File "/home/pazz/.local/bin/xapers", line 228, in 
  xapers.nci.UI(xroot, cmd=args)
File "/home/pazz/.local/lib/python2.7/site-packages/xapers/nci/ui.py", line 
73, in __init__
  self.mainloop.run()
File 
"/home/pazz/.local/lib/python2.7/site-packages/urwid-1.2.0_dev-py2.7-linux-x86_64.egg/urwid/main_loop.py",
 line 272, in run
  self.screen.run_wrapper(self._run)
File 
"/home/pazz/.local/lib/python2.7/site-packages/urwid-1.2.0_dev-py2.7-linux-x86_64.egg/urwid/raw_display.py",
 line 242, in run_wrapper
  return fn()
File 
"/home/pazz/.local/lib/python2.7/site-packages/urwid-1.2.0_dev-py2.7-linux-x86_64.egg/urwid/main_loop.py",
 line 337, in _run
  self.event_loop.run()
File 
"/home/pazz/.local/lib/python2.7/site-packages/urwid-1.2.0_dev-py2.7-linux-x86_64.egg/urwid/main_loop.py",
 line 708, in run
  self._loop()
File 
"/home/pazz/.local/lib/python2.7/site-packages/urwid-1.2.0_dev-py2.7-linux-x86_64.egg/urwid/main_loop.py",
 line 787, in _loop
  self._watch_files[fd]()
File 
"/home/pazz/.local/lib/python2.7/site-packages/urwid-1.2.0_dev-py2.7-linux-x86_64.egg/urwid/main_loop.py",
 line 388, in _update
  self.process_input(keys)
File 
"/home/pazz/.local/lib/python2.7/site-packages/urwid-1.2.0_dev-py2.7-linux-x86_64.egg/urwid/main_loop.py",
 line 488, in process_input
  k = self._topmost_widget.keypress(self.screen_size, k)
File 
"/home/pazz/.local/lib/python2.7/site-packages/urwid-1.2.0_dev-py2.7-linux-x86_64.egg/urwid/container.py",
 line 1120, in keypress
  return self.body.keypress( (maxcol, remaining), key )
File "/home/pazz/.local/lib/python2.7/site-packages/xapers/nci/search.py", 
line 304, in keypress
  eval(cmd)
File "", line 1, in 
File "/home/pazz/.local/lib/python2.7/site-packages/xapers/nci/search.py", 
line 155, in nextEntry
  self.listbox.set_focus(pos + 1)
File 
"/home/pazz/.local/lib/python2.7/site-packages/urwid-1.2.0_dev-py2.7-linux-x86_64.egg/urwid/listbox.py",
 line 567, in set_focus
  self.body.set_focus(position)
File 
"/home/pazz/.local/lib/python2.7/site-packages/urwid-1.2.0_dev-py2.7-linux-x86_64.egg/urwid/listbox.py",
 line 178, in set_focus
  raise IndexError, "No widget at position %s" % (position,)
  IndexError: No widget at position 1
  


* did you already set something up to report bugs? github issues, a maling list?

Can't wait until someone forks alot to use this. I've been waiting for someone 
(else)
to start a project like this for some time now :)

rock on,
/p



signature.asc
Description: signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[CLI] bug in notmuch setup

[Patrick Totzke]

Hi all,

As mentioned on IRC earlier, current master seems to have an issue with 
`notmuch setup`,
which fails to write ~/.notmuch-config if not already present.

Here are excerpts from the discussion from #notmuch:
--%<---
19:08:29  i have no notmuch config so far. when i run `notmuch` i get 
questioned about my name and so forth. 
then:
19:08:31  Error canonicalizing /home/pazz/.notmuch-config: No such file 
or directory
19:14:01  it could also be a bug. There was some change recently to 
avoid overwriting symlinks with files.
19:15:29  pazz: I think you found a bug. try "touch .notmuch-config" 
and run notmuch-setup again.
19:21:54  bremner: i'm a little confused because i thought i'd done that 
already. anyway: notmuch setup 
fails iff there is no ~/.notmuch-config present.
19:22:38  if there is an empty file notmuch setup runs ok, exit code is 0 
and the file gets overwritten
...
19:43:55  Error canonicalizing /home/too/.notmuch-config: No such file 
or directory
19:46:52  apparently realpath(3) fails in case there is no such file ;/
19:50:50  forgot to mention: i'm on 0.15.2+85~g44fe1f2d
19:52:55  yeah, the offending patch is +79, I think
19:53:25  the reason is obvious .. notmuch_config_save() executes 
filename = realpath
 (config->filename, NULL);
19:53:53  maybe just additionally checking for ENOENT could fix that...
19:57:20  sounds plausible. Have to double check we are not relying on 
some implicit re-allocation by 
   realpath.
19:58:41  true: free (filename) needs not to be done in case realpath() 
fails with ENOENT

-->%---

Best,
/p


signature.asc
Description: signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[alot] announcing v0.3.5

[Patrick Totzke]

Hi everyone!

I have just tagged alot v0.3.5; You can get a tarball here [0].

Most notably, this update comes with a rewrite of Thread buffers,
allowing among other things thread-tree based focus movements.
Alot now has full support for PGP/MIME encryption and signatures,
courtesy of Justus (teythoon). Apart from this, the new release comes with the
usual ton of fixes contributed by various people (see below).

I enabled the project wiki on github, so that people too shy to come forward
with pull requests for alot/extra/ can share their customizations,
in particular python-hooks [2].

Usage updates since v0.3.4:
* full support for PGP/MIME [de|en]cryption and signatures
* add missing "unattach" command in envelope buffer
* honour 'Mail-Followup-To' header and set if for selected mailinglists
* better handling of replies to self-sent messages
* make auto_remove_unread configurable
* rewrite thread buffer
* improved global move commands: first/last line, half-page up/down
* tree-based movement in threads (first/last reply, next/previous 
unfolded/sibling, parent)
* fold/unfold messages based on query string in thread mode
* respect mailcap commands that expect stdin
* Support different libmagic APIs
* new hooks called before/aftr buffer open/close/focus
* new global repeat command


Thanks goes to those who contributed:
$ git shortlog -s -n 0.3.2...|tail -n +2
11  Justus Winter
10  Sebastian Jeltsch
 6  josch
 4  Hamish Downer
 4  Yann Rouillard
 3  Lukas Zapletal
 2  Christopher Schwardt
 1  Benjamin
 1  Caio Prado
 1  Siim Põder
 1  bjoernb
 1  jakeogh

.. and everyone who send feedback!

As always, don't hesitate to file bug reports, feature or pull requests via the
projects github page [1].

Cheers,
/p

[0]: https://github.com/pazz/alot/tarball/0.3.5
[1]: https://github.com/pazz/alot
[2]: https://github.com/pazz/alot/wiki/Contrib-Hooks


signature.asc
Description: signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: proposal to remove contrib/notmuch-vim and contrib/notmuch-deliver

[Patrick Totzke]

+1 from me for getting rid of the vim plug in mainline notmuch.
Cheers,
/p


signature.asc
Description: signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: How about these: -- CR & NL char in rfc2047-encoded header...

[Patrick Totzke]

Quoting David Bremner (2013-10-27 12:48:24)
> tomi.oll...@iki.fi writes:
> 
> > In thread starting from id:08cb1dcd-c5db-4e33-8b09-7730cb3d5...@gmail.com
> > David wondered in id:87y58xv71x.fsf@zancas.localnet what filtering
> > is done by CLI and what by lib.
> >
> > But where should the "problem" presented in the Subject: header should
> > be handled. I did some hacks to emacs client to drop the ^J (newline)
> > characters from Subject: and From: headers but should these be filtered
> > in CLI (or in lib) instead?
> >
> 
> Eventually we decided this was a front-end isssue. The emacs front-end
> fixed this  with commit 0.16-111-ga7964c8


I agree that this should be an issue for the MUA.
FYI: alot also normalizes newlines and tabs in subject lines..
https://github.com/pazz/alot/blob/master/alot/db/utils.py#L369
I cannot reproduce the original issue with the escaped UTF8 header not being 
interpreted.
This works fine for me in alot.

Best,
/p


signature.asc
Description: signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: alot: can't read sent emails, after encryption

[Patrick Totzke]

Quoting Jameson Graef Rollins (2013-11-16 21:47:02)
> On Tue, Nov 12 2013, apman...@idaaas.com wrote:
> > I have recently switched to notmuch. Thank you for it!
> > I'm using "alot" as a frontend (thank you for it, too!). Everything
> > works smoothly, apart from one problem: with alot, I can't figure out how
> > to read encrypted emails I previously sent: they appear to be encrypted
> > using the addressee's key.
> >
> > Is there some way to store encrypted sent emails with my own public gpg
> > key?
> 
> What you really want is to tell gpg to always encrypt messages to your
> personal key as well, which will always make them viewable by you.  This
> way you don't have to worry about saving unencrypted versions of the
> message to disk, or there being two distinct versions of the message
> (one encrypted to the recipient and a different one encrypted to you).
> 
> See the "encrypt-to" gpg option [0].
> 
> jamie.
> 
> [0] 
> http://www.gnupg.org/documentation/manuals/gnupg-devel/GPG-Key-related-Options.html

Is this how notmuch emacs does it? I mean, is there some option to tell
emacs to always call gpg with --encrypt-to=me ?
I wonder if I need to change alot in any way or if one can simply globally 
configure
gnupg.. alot does not call the gpg binary but uses pygpgme.
cheers,
/p


signature.asc
Description: signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: alot: can't read sent emails, after encryption

[Patrick Totzke]

Quoting Alain-Pierre Manine (2013-11-18 08:38:33)
> Quoting Jameson Graef Rollins (2013-11-17 20:43:25)
> > On Sun, Nov 17 2013, Patrick Totzke  wrote:
> > > Quoting Jameson Graef Rollins (2013-11-16 21:47:02)
> > >> On Tue, Nov 12 2013, apman...@idaaas.com wrote:
> > >> > I have recently switched to notmuch. Thank you for it!
> > >> > I'm using "alot" as a frontend (thank you for it, too!). Everything
> > >> > works smoothly, apart from one problem: with alot, I can't figure out 
> > >> > how
> > >> > to read encrypted emails I previously sent: they appear to be encrypted
> > >> > using the addressee's key.
> > >> >
> > >> > Is there some way to store encrypted sent emails with my own public gpg
> > >> > key?
> > >> 
> > >> What you really want is to tell gpg to always encrypt messages to your
> > >> personal key as well, which will always make them viewable by you.  This
> > >> way you don't have to worry about saving unencrypted versions of the
> > >> message to disk, or there being two distinct versions of the message
> > >> (one encrypted to the recipient and a different one encrypted to you).
> > >> 
> > >> See the "encrypt-to" gpg option [0].
> > >> 
> > >> jamie.
> > >> 
> > >> [0] 
> > >> http://www.gnupg.org/documentation/manuals/gnupg-devel/GPG-Key-related-Options.html
> > >
> > > Is this how notmuch emacs does it? I mean, is there some option to tell
> > > emacs to always call gpg with --encrypt-to=me ?
> > > I wonder if I need to change alot in any way or if one can simply 
> > > globally configure
> > > gnupg.. alot does not call the gpg binary but uses pygpgme.
> > 
> > You do not need to change alot, just notmuch emacs also doesn't need to
> > do anything special to allow for this.  Just add an
> > 
> > encrypt-to 
> > 
> > line to your ~/.gnupg/gpg.conf, where  is your personal keyid.
> > Then all encrypted data is also encrypted to your personal key, making
> > it always viewable by you as well.  Then you can just open your
> > encrypted sent mail as you would any other encrypted mail.
> > 
> > jamie.
> 
> It's working! Thanks for the explanations.


Excelent, thanks Jamie.
I wasn't aware of this setting and assumed that the stuff in the
gnupg config file only applies to gpg itself and not, as in alot,
for calls through the gpgme library.
Thanks for testing this Alain-Pierre.
/p


signature.asc
Description: signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[alot] announcing v0.3.6

[Patrick Totzke]

Hi everyone!

I have just tagged alot v0.3.6; You can get a tarball here [0].

This version bump mostly reflects me catching up with the many fixes,
tweaks and some new features I received as pull requests.

Usage updates since v0.3.5:
* implement vim-style "move last" command (bound to G)
* fixes in reply/forwarding
* add option "--tags" to taglist command to display only a subset of the tags
* fix: safely interrumpt a command sequence
* use suffix ".eml" for temporary email files when editing
* interpret "compose mailto:foo@bar"; commands
* new "tomorrow" colour theme
* Add some Emacs keybindings for prompts

As usual, a quick hall of fame for recent contributors:
$ git shortlog -s -n 0.3.5...|tail -n +2
 4  Hamish Downer
 4  Yann Rouillard
 4  Your Name
 3  Stanislav Ochotnicky
 2  Cinghio Pinghio
 2  Simon Chopin
 1  Christian Geier
 1  Emmanuel Beffara
 1  Gonéri Le Bouder
 1  Martin Zimmermann
 1  Siim Põder

As you have surely noticed, alot is no longer actively developed and entered a
sluggish maintenance mode. This is of course due to real life catching up and
also because I don't see any immediate need for improvement. I don't plan to
abandon it as my main email client any time soon and therefore still
occasionally take the time to comment on and merge pull requests.
Let me know if you'd like to get involved in further developing/maintaining 
this project.

Nevertheless: There are still tons of good ideas for improvements floating
around as issues on github and some even have proof-of-concept implementations
(syntax highlighting for instance #272,#612).

Bug reports, feature or pull requests via the projects github page [1]
are of course always welcome.

Cheers,
/p

[0]: https://github.com/pazz/alot/tarball/0.3.6
[1]: https://github.com/pazz/alot


signature.asc
Description: signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: Looking for the perfect mail client

[Patrick Totzke]

Hi all,

Quoting Simon Chopin (2014-10-21 11:42:39)
> ...
>
> You can't show HTML content without a web browser or a conversion
> to ascii AFAIK (BTW, alot uses w3m, which is a web browser :-) ). So you need
> to choose between the two (or viewing raw HTML, of course)

Just for completenes's sake: alot might use w3m on your box because you told it
to :) It will happily use whichever html2text converter you set up in your
mailcap (see http://alot.readthedocs.org/en/latest/faq.html point 5).
This set up mirrors that of mutt and other console MUAs.

> > - which is fast
> 
> That last point is my main problem with alot. There are some algorithms
> within the widgets that seem unobviously slow to me.

I agree with you. Out of interest: do you have any specific examples/complaints?
There is a recent fix for some recursion issue within the tree widgets
(https://github.com/pazz/urwidtrees/pull/4) that should solve some related
issues at least.

Best wishes,
/p


signature.asc
Description: signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[alot] announcing v0.3.7

[Patrick Totzke]

Hi everyone!

I have just tagged alot v0.3.7; You can get a tarball here [0].

NOTE: THIS VERSION INTRODUCES A NEW DEPENDENCY: "urwidtrees",
a library of urwid-widgets for tree structures.
It grew out of the alot interface and is now a separate project [1].

Apart from this, the current version contains lots of small fixes and usability
improvements, mainly provided by users (see below).

Usage updates since v0.3.6:
* new config option: msg_summary_hides_threadwide_tags
* new config setting: thread_subject
* new hook: sanitize_attachment_filename
* new hook: exit()
* list replies (new command parameter and config setting: 
auto_replyto_mailinglist)
* new config setting for accounts: encrypt_by_default
* new config setting for accounts: alias_regexp
* new config setting for abooks: shellcommand_external_filtering
* switched to setuptools
* detached urwidtrees library into separate project (and new dependency)

As usual, a quick hall of fame for recent contributors:
$ git shortlog -s -n 0.3.5...|tail -n +2
 7  Lucas Hoffmann
 5  Martin Zimmermann
 3  Guillaume Seren
 3  Tomas Tomecek
 2  Andres MRM
 2  Christian Geier
 1  Anton Khirnov
 1  Anton Parkhomenko
 1  Cinghio Pinghio
 1  Ruben Pollan
 1  bjoernb
 1  josch
 1  vrs


New bug reports, feature or pull requests via the projects github page [2]
are always welcome.

Cheers,
/p

[0]: https://github.com/pazz/alot/tarball/0.3.7
[1]: https://github.com/pazz/urwidtrees
[2]: https://github.com/pazz/alot
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH] use notmuch_database_open_verbose

[Patrick Totzke]

... instead of the deprecated notmuch_database_open
when opening the database in notmuch.database.Database.open.
This prevents the library to print Xapian exceptions to stderr.
---
 bindings/python/notmuch/database.py | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/bindings/python/notmuch/database.py 
b/bindings/python/notmuch/database.py
index f304533..5400338 100644
--- a/bindings/python/notmuch/database.py
+++ b/bindings/python/notmuch/database.py
@@ -86,8 +86,8 @@ class Database(object):
 _get_version.argtypes = [NotmuchDatabaseP]
 _get_version.restype = c_uint
 
-"""notmuch_database_open"""
-_open = nmlib.notmuch_database_open
+"""notmuch_database_open_verbose"""
+_open = nmlib.notmuch_database_open_verbose
 _open.argtypes = [c_char_p, c_uint, POINTER(NotmuchDatabaseP)]
 _open.restype = c_uint
 
@@ -201,7 +201,7 @@ class Database(object):
 
 This function is used by __init__() and usually does not need
 to be called directly. It wraps the underlying
-*notmuch_database_open* function.
+*notmuch_database_open_verbose* function.
 
 :param status: Open the database in read-only or read-write mode
 :type status:  :attr:`MODE`
-- 
2.7.0

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH] use notmuch_database_open_verbose

[Patrick Totzke]

As far as I understand the bindings, the returned message is not needed
because the return
value of `Database._open` (line 212) contains information about the type of
error that occurred.
This is passed on two lines below if the open-call was unsuccessful.

The docstrings for this method are not ideal (the mentioned "status"
parameter is actually called "mode") but in terms of functionality it seems
ok.

On 12 March 2016 at 16:33, David Bremner  wrote:

> David Bremner  writes:
>
> >> -"""notmuch_database_open"""
> >> -_open = nmlib.notmuch_database_open
> >> +"""notmuch_database_open_verbose"""
> >> +_open = nmlib.notmuch_database_open_verbose
> >>  _open.argtypes = [c_char_p, c_uint, POINTER(NotmuchDatabaseP)]
> >>  _open.restype = c_uint
> >
> > I expect you probably want to add the new binding, rather than replacing
> > the old one. Otherwise updating the bindings will break all bindings
> > using scripts.
>
> Oh silly me. That's not actually exposed to the python API, so that's
> fine. On the other hand, don't you want to update the argument types and
> do something useful (throw an exception?) with the returned message?  As
> it stands you lose information.
>
> d
>
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[alot] announcing v0.4

[Patrick Totzke]

Hi everyone!

I have just tagged alot v0.4; You can get a tarball here [0].

I am happy to see a renewed interest in keeping this project alive.
Several people declared an interest in sharing some of the reponsibility
and pushing things forward (see [1] for the discussion).
For now, I will grant the following (github) users write access to the
repository: geier (py3 ports), lucc, dcbaker (testing) and dkg (gnupg).

Future pull requests will be required to pass the automatic QA tests [2],
receive at least one favourable review [3] and should be based against the
master branch.
I've decided to let the version number reflect the change in workflow.
We are now working towards version 0.5.

This version contains pushes mostly fixes and minor
usabilityimprovements (see below).

Usage updates since v0.3.7:
* signal: refresh current buffer on SIGUSR1
* signal: exit interface on SIGINT
* interpret C-g keybinding in prompts
* new config option:encrypt_by_default
* new config option: thread_authors_order_by
* indicate untrusted PGP signatures in thread view
* more flexible account selection for replies

As usual, a quick hall of fame for recent contributors:
$ git shortlog -s -n 0.3.7...|tail -n +2
 21 Lucas Hoffmann
 3  Micah Nordland
 3  Pol Van Aubel
 2  Guillaume Seren
 2  Johannes 'josch' Schauer
 2  Ruben Pollan
 2  Tony Cheneau
 2  bx
 1  Cody
 1  Dimitrios Semitsoglou-Tsiapos
 1  Emmanuel Beffara
 1  Jason Kölker
 1  Tommy Lindgren

New bug reports, feature or pull requests via the projects github page [4]
are always welcome.

Cheers,
P

[0]: https://github.com/pazz/alot/tarball/0.4
[1]: https://github.com/pazz/alot/issues/908
[2]: 
https://www.quantifiedcode.com/app/project/c5aaa4739c5b4f6eb75eaaf8c01da679?limit=20&offset=20&categories=correctness&tab=analysis
[3]: https://github.com/blog/2291-introducing-review-requests
[4]: https://github.com/pazz/alot


signature.asc
Description: signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[alot] announcing v0.4

[Patrick Totzke]

Hi everyone!

I have just tagged alot v0.4; You can get a tarball here [0].

I am happy to see a renewed interest in keeping this project alive.
Several people declared an interest in sharing some of the reponsibility
and pushing things forward (see [1] for the discussion).
For now, I will grant the following (github) users write access to the
repository: geier (py3 ports), lucc, dcbaker (testing) and dkg (gnupg).

Future pull requests will be required to pass the automatic QA tests [2],
receive at least one favourable review [3] and should be based against the
master branch.
I've decided to let the version number reflect the change in workflow.
We are now working towards version 0.5.

This version contains pushes mostly fixes and minor
usabilityimprovements (see below).

Usage updates since v0.3.7:
* signal: refresh current buffer on SIGUSR1
* signal: exit interface on SIGINT
* interpret C-g keybinding in prompts
* new config option:encrypt_by_default
* new config option: thread_authors_order_by
* indicate untrusted PGP signatures in thread view
* more flexible account selection for replies

As usual, a quick hall of fame for recent contributors:
$ git shortlog -s -n 0.3.7...|tail -n +2
 21  Lucas Hoffmann
 3  Micah Nordland
 3  Pol Van Aubel
 2  Guillaume Seren
 2  Johannes 'josch' Schauer
 2  Ruben Pollan
 2  Tony Cheneau
 2  bx
 1  Cody
 1  Dimitrios Semitsoglou-Tsiapos
 1  Emmanuel Beffara
 1  Jason Kölker
 1  Tommy Lindgren

New bug reports, feature or pull requests via the projects github page [4]
are always welcome.

Cheers,
P

[0]: https://github.com/pazz/alot/tarball/0.4
[1]: https://github.com/pazz/alot/issues/908
[2]: 
https://www.quantifiedcode.com/app/project/c5aaa4739c5b4f6eb75eaaf8c01da679?limit=20&offset=20&categories=correctness&tab=analysis
[3]: https://github.com/blog/2291-introducing-review-requests
[4]: https://github.com/pazz/alot


signature.asc
Description: signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

a python terminal gui?

[Patrick Totzke]

Hi all,

First of all, Thanks to everybody involved in the project. Using notmuch is
good fun and I totally agree that the spin-off/rewrite of sup is a great idea.
However, as a vim user I feel that the one thing that's truly missing is a 
maintainable, feature-rich commandline GUI. Let's change that.
I'm aware of existing proposed guis: vala-notmuch, notsomuch, I have also seen 
a minimal non-oop python gui that doesn't use the python bindings and whose 
name 
I forgot.

This is a proposal for a terminal gui, written in python.
Currently, it is more a sketch of a framework, so let me explain the key ideas 
here:

* I use python because I personally am heavily biased in the python vs. ruby 
discussion:
  I've been working with python for a while and have never used ruby anywhere 
else than 
  for my sup-mail config. I find rubys syntax highly unintuitive, had (and 
still have)
  problems to install/work with up-to-date ruby-gems on my ubuntu system.
  Also, I read somewhere™ that sup is based on a email library that is no 
longer maintained.
  I know these are only my personal, non representative and maybe uninformed 
impressions,
  but I have the feeling that I'm not alone here.
  On the other hand, cnotmuch and all other libs I (plan to) use in this project
  were easy_installed without any problems.

* I want a gui that looks and feels like sup, without the ugly parts.

* use libraries wherever possible. Therefore, instead of
  programming raw curses, I go for the urwid toolkit: It's a bit like gtk,
  you create a bunch of widgets and put them together in a tree-like structure
  and the library handles redrawing the screen and redirecting key presses etc.
  I'm aware of some arguments against urwid (PEP8 compatibility, slow py3k 
awareness),
  but consider the advantages:
   - no boring and error-prone ncurses hacking
   - possibly multiple frontends (curses, web)
   - correct display of unicode chars
   - a lot of ready-made widgets are available (textboxes, selectable lists). 
There's even
 an additional widget-lib called urwid-sat that defines widgets for 
password inputs etc.
   - ready made colour theming for mono/16/256 colours.
   
  one might also look around for a mutt/abook contact lib

You can find the code at here: https://github.com/pazz/notmuch-gui 
Obviously, there is a lot that's missing, but I think having this framework 
makes it easier
for anyone to contribute. For example, to be able to display mails,
one could code a urwid.Widget that can display a given email.message.Message.
That widget will then be used in a widget that displays a whole notmuch.thread
which we could put inside a new buffer class that extends my ng.buffer.Buffer
So far, there are only two modi (ng.buffer.Buffer subclasses):
a bufferlist and one to display thread search results.

I consider all of this to be a proof-of-concept, any part of this proposal
is up for discussion. I'd be interested in any feedback whatsoever, and happy
to delegate work to anybody willing to contribute. I'm aware that the code
is not pretty, probably not the sleekest oop design and non-conforming 
with a lot of conventions. Hit me :)

Cheers,
/pazz



signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: a python terminal gui?

[Patrick Totzke]

Hi Again,

Excerpts from Carl Worth's message of Fri May 20 23:00:23 +0100 2011:
> > However, as a vim user I feel that the one thing that's truly missing is a 
> > maintainable, feature-rich commandline GUI.
> 
> "... that's not implemented in emacs" you mean? ;-)
of course :D

> It's funny to me that you used the name "notmuch-gui" since I've always
> understood the 'G' in GUI to indicate "graphical", to distinguish from
> non-graphical terminal interfaces like you propose here.
I always intuitively understood "graphical" as anything visual.
But your right, http://en.wikipedia.org/wiki/Graphical_user_interface
suggests that a curses-like interface is not a GUI nor a CLI.
Anyhow, I agree that at some point I'll need a real name.

> I believe all new code should be doing "import notmuch" rather than
> "import cnotmuch", and if there is any package that installs bindings
> that work as "cnotmuch" we should get those updated right away.
Thanks for the clarification. I assume I'll always get the current
bindings distributed with notmuch itself.
A make target inside notmuch/bindings/python would indeed be helpful.

> > You can find the code at here: https://github.com/pazz/notmuch-gui 
> I was a bit surprised to see so many results in my initial view. I'd
> expect a default view to be tag:inbox rather than tag:unread. (I didn't
> realize I had over 100 thousand messages with the unread tag, but
> apparently I do.) Perhaps that's a sup default that I've long since
> forgotten?
Yep, its an artefact of my debugging behaviour. Having the initial
buffer show the inbox makes total sense and is also what sup does.

>   * The interface blocks for a while when searching for something (like
> "tag:unread" in my case) that returns a *lot* of results. Are you
> somehow sucking in all the results immediately rather than just what
> you need to dill the current screen?
That is surprising! I only fill the screen by iterating over an initial
part of the iterator returned by Query.search_threads()
I do a second query to count the messages by Query.count_messages(),
but I'd guess that this translates to some sort of "SELECT COUNT"
and should also be fast. Maybe there's some copying going on 
at lower levels?

>   * I was a bit surprised that 'i' and 'u' kept opening new buffers
> rather than switching to an existing buffer.
It shouldn't be like this, I agree. I'm just annoyed that sup treats
the inbox view different from other searches, in terms of keybindings.
But nevertheless one could have "I" point to a unique inbox search
buffer.

>   * The '/' key didn't seem to do anything for me, so I wasn't able to
> actually do any custom searches.
This is because it really is "\" :P that was a typo in the README.

> It looks like a fine start to me. Please let me know if there's anything
> I can do to help with this, (in terms of hosting the code or anything
> else).
Thanks, that's helpful. I guess it would make sense to place this under
notmuch/contrib at a later point if it gets usable.

Ah I have question regarding "toplevel" messages in threads:
How can it be that a mail that is not the one that started a thread
is contained in thread.get_toplevel_messages() ?
The only thing I can think of is that a user somehow forced two threads
to become one. Take this very thread for example. Why do I get
Mueen Nawaz's reply as a toplevel message? does this mean
that messages header has incorrect Reply-to headers set?

@Mueen Nawaz: It's a good idea to post this on the wiki. Will do.

Cheers,
/p


signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: a python terminal gui?

[Patrick Totzke]

Hi Again,
sorry if this comes as a repost..

Excerpts from Carl Worth's message of Fri May 20 23:00:23 +0100 2011:
> > However, as a vim user I feel that the one thing that's truly missing is a 
> > maintainable, feature-rich commandline GUI.
> 
> "... that's not implemented in emacs" you mean? ;-)
of course :D

> It's funny to me that you used the name "notmuch-gui" since I've always
> understood the 'G' in GUI to indicate "graphical", to distinguish from
> non-graphical terminal interfaces like you propose here.
I always intuitively understood "graphical" as anything visual.
But your right, http://en.wikipedia.org/wiki/Graphical_user_interface
suggests that a curses-like interface is not a GUI nor a CLI.
Anyhow, I agree that at some point I'll need a real name.

> I believe all new code should be doing "import notmuch" rather than
> "import cnotmuch", and if there is any package that installs bindings
> that work as "cnotmuch" we should get those updated right away.
Thanks for the clarification. I assume I'll always get the current
bindings distributed with notmuch itself.
A make target inside notmuch/bindings/python would indeed be helpful.

> > You can find the code at here: https://github.com/pazz/notmuch-gui 
> I was a bit surprised to see so many results in my initial view. I'd
> expect a default view to be tag:inbox rather than tag:unread. (I didn't
> realize I had over 100 thousand messages with the unread tag, but
> apparently I do.) Perhaps that's a sup default that I've long since
> forgotten?
Yep, its an artefact of my debugging behaviour. Having the initial
buffer show the inbox makes total sense and is also what sup does.

>   * The interface blocks for a while when searching for something (like
> "tag:unread" in my case) that returns a *lot* of results. Are you
> somehow sucking in all the results immediately rather than just what
> you need to dill the current screen?
That is surprising! I only fill the screen by iterating over an initial
part of the iterator returned by Query.search_threads()
I do a second query to count the messages by Query.count_messages(),
but I'd guess that this translates to some sort of "SELECT COUNT"
and should also be fast. Maybe there's some copying going on 
at lower levels?

>   * I was a bit surprised that 'i' and 'u' kept opening new buffers
> rather than switching to an existing buffer.
It shouldn't be like this, I agree. I'm just annoyed that sup treats
the inbox view different from other searches, in terms of keybindings.
But nevertheless one could have "I" point to a unique inbox search
buffer.

>   * The '/' key didn't seem to do anything for me, so I wasn't able to
> actually do any custom searches.
This is because it really is "\" :P that was a typo in the README.

> It looks like a fine start to me. Please let me know if there's anything
> I can do to help with this, (in terms of hosting the code or anything
> else).
Thanks, that's helpful. I guess it would make sense to place this under
notmuch/contrib at a later point if it gets usable.

Ah I have question regarding "toplevel" messages in threads:
How can it be that a mail that is not the one that started a thread
is contained in thread.get_toplevel_messages() ?
The only thing I can think of is that a user somehow forced two threads
to become one. Take this very thread for example. Why do I get
Mueen Nawaz's reply as a toplevel message? does this mean
that messages header has incorrect Reply-to headers set?

@Mueen Nawaz: It's a good idea to post this on the wiki. Will do.

Cheers,
/p


signature.asc
Description: Digital signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

one-time-iterators

[Patrick Totzke]

Hi!

I'm playing around with one-time iterators Threads in the python bindings
in combination with changes to the database.
Particularly, I'd like to iterate over a list of threads partially,
change the tags of a single thread and afterwards continue iterating.
Of course I get "Xapian DB-changed" exceptions.
As I see it, there are two possible solutions:
First, I iterate over all the threads and cache all the info i'd like to 
extract from each one (or just the id and re-query the info on demand).
The problem here is, that this list of thread-ids might be huge and
caching it will block my code.
Secondly, I could replace the iterator i'm using with a fresh one after 
db-changes. But then I'd need some magic that discards an initial
portion of the (fresh) iteration and/or updates the pre-db-change
partial iteration.

Wow. This reads really complicated. All I want to say is:
if I change tags in my search-results view, I get Xapian errors :)
The question: How do you solve this in the emacs code?
do you store all tids of a query? 

Thanks,
/p



signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: one-time-iterators

[Patrick Totzke]

hehe, did it again (dropping the list from cc). I need to stop
using sup :P thanks Austin.

Excerpts from Carl Worth's message of Thu May 26 18:20:21 +0100 2011:
> On Thu, 26 May 2011 09:31:19 +0100, Patrick Totzke 
>  wrote:
> > Wow. This reads really complicated. All I want to say is:
> > if I change tags in my search-results view, I get Xapian errors :)
> 
> Yes, that's frustrating. I wish that we had a more reliable interface at
> the notmuch library level. But I'm not entirely sure what would be the
> best way to do this.
Actually, I expected something like this. For this reason each sup instance 
locks its index.
At the moment I'm going for custom wrapper classes around notmuch.Thread
and notmuch.Messages that cache the result of the calls relevant for me.
But the real issue seems to be the iterator:
It takes an awful lot of time just to copy the thread ids of all threads from 
large a query result.

I tried the following in ipython:
 q=Database().create_query('*')
 time tids = [t.get_thread_id() for t in q.search_threads()]

which results in
CPU times: user 7.64 s, sys: 2.06 s, total: 9.70 s
Wall time: 9.84 s

It would really help if the Query object could return an iterator
of thread-ids that makes this copying unnecessary. Is it possible to
implement this? Or would this require the same amount of copying to happen
at a lower level?
I have not looked into the code for the bindings or the C code so far,
but I guess the Query.search_threads() translates to some 
"SELECT id,morestuff from threads"
where for me a "SELECT is from threads" would totally suffice. Copying 
(in the C code) only the ids so some list that yields an iterator should be 
faster.

> > The question: How do you solve this in the emacs code?
> > do you store all tids of a query? 
> 
> The emacs code does not use the notmuch library interface like your
> python bindings do. Instead, it uses the notmuch command-line tool, (and
> buffers up the text output by it). 
Ahh ok. Thanks for the explanation.

Excerpts from Austin Clements's message of Thu May 26 21:18:53 +0100 2011:
> I proposed a solution to this problem a while ago
> (id:"AANLkTi=kox8atjipkiarfvjehe6zt_jypoasmiiaw...@mail.gmail.com"),
> though I haven't tried implementing it yet.
Sorry, I wasn't on the list back then.

> Though, Patrick, that solution doesn't address your problem.  On the
> other hand, it's not clear to me what concurrent access semantics
> you're actually expecting.  I suspect you don't want the remaining
> iteration to reflect the changes, since your changes could equally
> well have affected earlier iteration results. 
That's right. 
> But if you want a
> consistent view of your query results, something's going to have to
> materialize that iterator, and it might as well be you (or Xapian
> would need more sophisticated concurrency control than it has).  But
> this shouldn't be expensive because all you need to materialize are
> the document ids; you shouldn't need to eagerly fetch the per-thread
> information.  
I thought so, but it seems that Query.search_threads() already
caches more than the id of each item. Which is as expected
because it is designed to return thread objects, not their ids.
As you can see above, this _is_ too expensive for me.

> Have you tried simply calling list() on your thread
> iterator to see how expensive it is?  My bet is that it's quite cheap,
> both memory-wise and CPU-wise.
Funny thing:
 q=Database().create_query('*')
 time tlist = list(q.search_threads())
raises a NotmuchError(STATUS.NOT_INITIALIZED) exception. For some reason
the list constructor must read mere than once from the iterator.
So this is not an option, but even if it worked, it would show
the same behaviour as my above test..

would it be very hard to implement a Query.search_thread_ids() ?
This name is a bit off because it had to be done on a lower level.
Cheers,
/p


signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: one-time-iterators

[Patrick Totzke]

Excerpts from Austin Clements's message of Thu May 26 22:43:02 +0100 2011:
> http://notmuch.198994.n3.nabble.com/notmuch-s-idea-of-concurrency-failing-an-invocation-tp2373468p2565731.html
ah, good old peterson :P thanks.

> > > Though, Patrick, that solution doesn't address your problem.  On the
> > > other hand, it's not clear to me what concurrent access semantics
> > > you're actually expecting.  I suspect you don't want the remaining
> > > iteration to reflect the changes, since your changes could equally
> > > well have affected earlier iteration results. 
> > That's right. 
> > > But if you want a
> > > consistent view of your query results, something's going to have to
> > > materialize that iterator, and it might as well be you (or Xapian
> > > would need more sophisticated concurrency control than it has).  But
> > > this shouldn't be expensive because all you need to materialize are
> > > the document ids; you shouldn't need to eagerly fetch the per-thread
> > > information.  
> > I thought so, but it seems that Query.search_threads() already
> > caches more than the id of each item. Which is as expected
> > because it is designed to return thread objects, not their ids.
> > As you can see above, this _is_ too expensive for me.
> 
> I'd forgotten that constructing threads on the C side was eager about
> the thread tags, author list and subject (which, without Istvan's
> proposed patch, even requires opening and parsing the message file).
> This is probably what's killing you.
> 
> Out of curiosity, what is your situation that you won't wind up paying
> the cost of this iteration one way or the other and that the latency
> of doing these tag changes matters?

I'm trying to implement a terminal interface for notmuch in python
that resembles sup.
For the search results view, i read an initial portion from a Threads iterator 
to fill my teminal window with threadline-widgets. Obviously, for a
large number of results I don't want to go through all of them.
The problem arises if you toggle a tag on the selected threadline and afterwards
continue to scroll down.

> > > Have you tried simply calling list() on your thread
> > > iterator to see how expensive it is?  My bet is that it's quite cheap,
> > > both memory-wise and CPU-wise.
> > Funny thing:
> >  q=Database().create_query('*')
> >  time tlist = list(q.search_threads())
> > raises a NotmuchError(STATUS.NOT_INITIALIZED) exception. For some reason
> > the list constructor must read mere than once from the iterator.
> > So this is not an option, but even if it worked, it would show
> > the same behaviour as my above test..
> 
> Interesting.  Looks like the Threads class implements __len__ and that
> its implementation exhausts the iterator.  Which isn't a great idea in
> itself, but it turns out that Python's implementation of list() calls
> __len__ if it's available (presumably to pre-size the list) before
> iterating over the object, so it exhausts the iterator before even
> using it.
> 
> That said, if list(q.search_threads()) did work, it wouldn't give you
> better performance than your experiment above.
> 
> > would it be very hard to implement a Query.search_thread_ids() ?
> > This name is a bit off because it had to be done on a lower level.
> 
> Lazily fetching the thread metadata on the C side would probably
> address your problem automatically.  But what are you doing that
> doesn't require any information about the threads you're manipulating?
Agreed. Unfortunately, there seems to be no way to get a list of thread
ids or a reliable iterator thereof by using the current python bindings.
It would be enough for me to have the ids because then I could
search for the few threads I actually need individually on demand.

Here is the branch in which I'm trying out these things. Sorry for the
messy code, its late :P
https://github.com/pazz/notmuch-gui/tree/toggletags

/p


signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: one-time-iterators

[Patrick Totzke]

Excerpts from Austin Clements's message of Fri May 27 03:41:44 +0100 2011:
> >> > > Have you tried simply calling list() on your thread
> >> > > iterator to see how expensive it is?  My bet is that it's quite cheap,
> >> > > both memory-wise and CPU-wise.
> >> > Funny thing:
> >> >  q=Database().create_query('*')
> >> >  time tlist = list(q.search_threads())
> >> > raises a NotmuchError(STATUS.NOT_INITIALIZED) exception. For some reason
> >> > the list constructor must read mere than once from the iterator.
> >> > So this is not an option, but even if it worked, it would show
> >> > the same behaviour as my above test..
> >>
> >> Interesting.  Looks like the Threads class implements __len__ and that
> >> its implementation exhausts the iterator.  Which isn't a great idea in
> >> itself, but it turns out that Python's implementation of list() calls
> >> __len__ if it's available (presumably to pre-size the list) before
> >> iterating over the object, so it exhausts the iterator before even
> >> using it.
> >>
> >> That said, if list(q.search_threads()) did work, it wouldn't give you
> >> better performance than your experiment above.
true. Nevertheless I think that list(q.search_threads())
should be equivalent to [t for t in q.search_threads()], which is
something to be fixed in the bindings. Should I file an issue somehow?
Or is enough to state this as a TODO here on the list?

> >> > would it be very hard to implement a Query.search_thread_ids() ?
> >> > This name is a bit off because it had to be done on a lower level.
> >>
> >> Lazily fetching the thread metadata on the C side would probably
> >> address your problem automatically.  But what are you doing that
> >> doesn't require any information about the threads you're manipulating?
> > Agreed. Unfortunately, there seems to be no way to get a list of thread
> > ids or a reliable iterator thereof by using the current python bindings.
> > It would be enough for me to have the ids because then I could
> > search for the few threads I actually need individually on demand.
> 
> There's no way to do that from the C API either, so don't feel left
> out.  ]:--8)  It seems to me that the right solution to your problem
> is to make thread information lazy (effectively, everything gathered
> in lib/thread.cc:_thread_add_message).  Then you could probably
> materialize that iterator cheaply. 
Alright. I'll put this on my mental notmuch wish list and 
hope that someone will have addressed this before I run out of
ideas how to improve my UI and have time to look at this myself.
For now, I go with the [t.get_thread_id for t in q.search_threads()]
approach to cache the thread ids myself and live with the fact that
this takes time for large result sets.

> In fact, it's probably worth
> trying a hack where you put dummy information in the thread object
> from _thread_add_message and see how long it takes just to walk the
> iterator (unfortunately I don't think profiling will help much here
> because much of your time is probably spent waiting for I/O).
I don't think I understand what you mean by dummy info in a thread
object.

> I don't think there would be any downside to doing this for eager
> consumers like the CLI.
one should think so, yes.
/p


signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: one-time-iterators

[Patrick Totzke]

Excerpts from Austin Clements's message of Fri May 27 20:29:24 +0100 2011:
> On Fri, May 27, 2011 at 2:04 PM, Patrick Totzke
>  wrote:
> > Excerpts from Austin Clements's message of Fri May 27 03:41:44 +0100 2011:
> >> >> > > Have you tried simply calling list() on your thread
> >> >> > > iterator to see how expensive it is?  My bet is that it's quite 
> >> >> > > cheap,
> >> >> > > both memory-wise and CPU-wise.
> >> >> > Funny thing:
> >> >> >  q=Database().create_query('*')
> >> >> >  time tlist = list(q.search_threads())
> >> >> > raises a NotmuchError(STATUS.NOT_INITIALIZED) exception. For some 
> >> >> > reason
> >> >> > the list constructor must read mere than once from the iterator.
> >> >> > So this is not an option, but even if it worked, it would show
> >> >> > the same behaviour as my above test..
> >> >>
> >> >> Interesting.  Looks like the Threads class implements __len__ and that
> >> >> its implementation exhausts the iterator.  Which isn't a great idea in
> >> >> itself, but it turns out that Python's implementation of list() calls
> >> >> __len__ if it's available (presumably to pre-size the list) before
> >> >> iterating over the object, so it exhausts the iterator before even
> >> >> using it.
> >> >>
> >> >> That said, if list(q.search_threads()) did work, it wouldn't give you
> >> >> better performance than your experiment above.
> > true. Nevertheless I think that list(q.search_threads())
> > should be equivalent to [t for t in q.search_threads()], which is
> > something to be fixed in the bindings. Should I file an issue somehow?
> > Or is enough to state this as a TODO here on the list?
> 
> Yes, they should be equivalent.
> 
> Sebastian was thinking about fixing the larger issue of generator
> exhaustion, which would address this, though the performance would
> depend on the cost of iterating twice.  This is why generators
> shouldn't support __len__.  Unfortunately, it's probably hard to get
> rid of at this point and I doubt there's a way to tell list() to
> overlook the presence of a __len__ method.
Why not simply removing support for __len__ in the Threads and Messages classes?

> >> >> > would it be very hard to implement a Query.search_thread_ids() ?
> >> >> > This name is a bit off because it had to be done on a lower level.
> >> >>
> >> >> Lazily fetching the thread metadata on the C side would probably
> >> >> address your problem automatically.  But what are you doing that
> >> >> doesn't require any information about the threads you're manipulating?
> >> > Agreed. Unfortunately, there seems to be no way to get a list of thread
> >> > ids or a reliable iterator thereof by using the current python bindings.
> >> > It would be enough for me to have the ids because then I could
> >> > search for the few threads I actually need individually on demand.
> >>
> >> There's no way to do that from the C API either, so don't feel left
> >> out.  ]:--8)  It seems to me that the right solution to your problem
> >> is to make thread information lazy (effectively, everything gathered
> >> in lib/thread.cc:_thread_add_message).  Then you could probably
> >> materialize that iterator cheaply.
> > Alright. I'll put this on my mental notmuch wish list and
> > hope that someone will have addressed this before I run out of
> > ideas how to improve my UI and have time to look at this myself.
> > For now, I go with the [t.get_thread_id for t in q.search_threads()]
> > approach to cache the thread ids myself and live with the fact that
> > this takes time for large result sets.
> >
> >> In fact, it's probably worth
> >> trying a hack where you put dummy information in the thread object
> >> from _thread_add_message and see how long it takes just to walk the
> >> iterator (unfortunately I don't think profiling will help much here
> >> because much of your time is probably spent waiting for I/O).
> > I don't think I understand what you mean by dummy info in a thread
> > object.
> 
> In _thread_add_message, rather than looking up the message's author,
> subject, etc, just hard-code some dummy values.  Performance-wise,
> this would simulate making 

[python] get all messages of a thread

[Patrick Totzke]

Hi!
I wonder how I would get all messages of a thread with the python
bindings. The doc says one can only use Thread.get_toplevel_messages()
and then must recursively call Message.get_replies().
But look:

 snip -
#/usr/bin/python
from notmuch import Database

def get_thread():
query = Database().create_query('thread:0c48')
return query.search_threads().next()

def replies(msg):
acc = []
r = msg.get_replies()
if r: #because we cant iterate on NoneType
for m in r:
acc.append(m)
acc += replies(m)
return acc

t=get_thread()

msgs = []
for m in t.get_toplevel_messages():
msgs.append(m)
msgs += replies(m)

print msgs
--- snap 
gives:

Traceback (most recent call last):
  File "replies.py", line 22, in 
msgs +=replies(m)
  File "replies.py", line 12, in replies
for m in r:
  File "/usr/local/lib/python2.7/dist-packages/notmuch/message.py", line 140, 
in next
raise NotmuchError(STATUS.NOT_INITIALIZED)
notmuch.globals.NotmuchError: Operation on uninitialized object impossible.
-
It seems that nobody needed this before. Even in bindings/python/notmuch.py
only Threads.get_toplevel_messages() gets called, and then a (undocumented)
Messages.print_messages is used (cf line 639, in show)

any suggestions?
I would rather not call the notmuch binary and parse its output..
Thanks!
/p


signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: notmuch crashing on misformated emails

[Patrick Totzke]

This actually sounds very familiar. I had this error before 
but had different problems at the time. Will try to isolate 
this when it happens again.
/p

Excerpts from Sebastian Spaeth's message of Thu Jun 02 15:40:43 +0100 2011:
> On Thu, 02 Jun 2011 09:16:33 +0200, Sebastian Spaeth  
> wrote:
> > in python:
> >   list(Threads())
> > 
> > leads to a program abortion (bombing out of python):
> > Internal error: Failed to read timestamp value from document. 
> > (lib/message.cc:731).
> > 
> > Can we not crash the app on misformated emails?
> 
> Just to follow up, I cannot reproduce this anymore. It works fine
> now. Sorry for the noise.
> 
> Sebastian


signature.asc
Description: PGP signature
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[python] segfaults at Message.get_date

[Patrick Totzke]

Hi all!
First off: Thanks Sebastian for your recent work on the python bindings.
It all makes a little bit more sense now without the __len__ and all.
As some of you might have read on the IRC channel, I'm facing really
strange behaviour with Message.get_date() which unfortunately,
I cannot reproduce reliably: I get it on one machine, not on the other,
despite using the same distribution (ubuntu natty - standard python
install), and notmuch and bindings from master on both boxes:

Unfortunately, as it segfaults, I don't get a stacktrace here.
The line that seems to cause the segfault is this one:
https://github.com/pazz/notmuch-gui/blob/master/alot/db.py#L181

To explain the circumstances in my code:
I crete these alot.Message objects by recursively calling
notmuch.Message.get_replies at line 153 in the same file.
The error occurs only at recursion depth 1, so to reproduce
one needs to open a thread with at least one reply.

Did anyone experience something similar? It would help if someone 
could try and run my code and report if it works for him/her:
 git clone https://github.com/pazz/notmuch-gui 
 python alot/init.py # to run directly 
 open a large enough thread by hitting enter

Just to be clear: I don't expect anybody to debug my broken code.
I think it might help to report segfaults, as it looks like it is 
a problem with the C-lib or the bindings.
Of course, I'd be grateful for any pointers (pun not intended *caught*)
or helpful hints to debug this.

thanks,
/patrick



signature.asc
Description: Digital signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: [python] segfaults at Message.get_date

[Patrick Totzke]

Thanks to amdragon's hint on how to get a stacktrace, here it is.
I startet gdb --args python, imported my stuff and ran it directly.
I can't really interpret this, but it seems as if its libxapians fault
doesn't it?
best,
/p


(gdb) bt
#0  0x006eb87d in Xapian::Document::Internal::get_value(unsigned int) const () 
from /usr/lib/sse2/libxapian.so.22
#1  0x006eb952 in Xapian::Document::get_value(unsigned int) const () from 
/usr/lib/sse2/libxapian.so.22
#2  0x00523963 in notmuch_message_get_date () from 
/usr/local/lib/libnotmuch.so.1
#3  0x0050c767 in ffi_call_SYSV () from 
/usr/lib/python2.7/lib-dynload/_ctypes.so
#4  0x0050c4bf in ffi_call () from /usr/lib/python2.7/lib-dynload/_ctypes.so
#5  0x00504e63 in _ctypes_callproc () from 
/usr/lib/python2.7/lib-dynload/_ctypes.so
#6  0x004fce6e in ?? () from /usr/lib/python2.7/lib-dynload/_ctypes.so
#7  0x080a464a in PyObject_Call ()
#8  0x080dad43 in PyEval_EvalFrameEx ()
#9  0x080dae8b in PyEval_EvalFrameEx ()
#10 0x080e11e7 in PyEval_EvalCodeEx ()
#11 0x08105a61 in ?? ()
#12 0x080a464a in PyObject_Call ()
#13 0x080a603f in ?? ()
#14 0x080a464a in PyObject_Call ()
#15 0x080da034 in PyEval_CallObjectWithKeywords ()
#16 0x080a6fff in PyInstance_New ()
#17 0x080a464a in PyObject_Call ()
#18 0x080dad43 in PyEval_EvalFrameEx ()
#19 0x080dae8b in PyEval_EvalFrameEx ()
#20 0x080dae8b in PyEval_EvalFrameEx ()
#21 0x080dae8b in PyEval_EvalFrameEx ()
#22 0x080dae8b in PyEval_EvalFrameEx ()
#23 0x080e11e7 in PyEval_EvalCodeEx ()
#24 0x08105a61 in ?? ()
#25 0x080a464a in PyObject_Call ()
#26 0x080a603f in ?? ()
#27 0x080a464a in PyObject_Call ()
#28 0x080da034 in PyEval_CallObjectWithKeywords ()
#29 0x080a6fff in PyInstance_New ()
#30 0x080a464a in PyObject_Call ()
#31 0x080dad43 in PyEval_EvalFrameEx ()
#32 0x080dae8b in PyEval_EvalFrameEx ()
#33 0x080dae8b in PyEval_EvalFrameEx ()
#34 0x080dae8b in PyEval_EvalFrameEx ()
#35 0x080dae8b in PyEval_EvalFrameEx ()
#36 0x080dae8b in PyEval_EvalFrameEx ()
#37 0x080dae8b in PyEval_EvalFrameEx ()
#38 0x080e16ee in PyEval_EvalCodeEx ()
#39 0x080dac2a in PyEval_EvalFrameEx ()
#40 0x080dae8b in PyEval_EvalFrameEx ()
#41 0x080dae8b in PyEval_EvalFrameEx ()
#42 0x080dae8b in PyEval_EvalFrameEx ()
#43 0x080e11e7 in PyEval_EvalCodeEx ()
#44 0x080dac2a in PyEval_EvalFrameEx ()
#45 0x080dae8b in PyEval_EvalFrameEx ()
#46 0x080e11e7 in PyEval_EvalCodeEx ()
#47 0x08105a61 in ?? ()
#48 0x080a464a in PyObject_Call ()
#49 0x080a603f in ?? ()
#50 0x080a464a in PyObject_Call ()
#51 0x080da034 in PyEval_CallObjectWithKeywords ()
#52 0x080a6fff in PyInstance_New ()
#53 0x080a464a in PyObject_Call ()
#54 0x080dad43 in PyEval_EvalFrameEx ()
#55 0x080dae8b in PyEval_EvalFrameEx ()
#56 0x080e11e7 in PyEval_EvalCodeEx ()
#57 0x0812c477 in PyEval_EvalCode ()
#58 0x0813c010 in ?? ()
#59 0x08070d09 in PyRun_InteractiveOneFlags ()
#60 0x08070e0b in PyRun_InteractiveLoopFlags ()
#61 0x0807124e in PyRun_AnyFileExFlags ()
#62 0x0805c069 in Py_Main ()
#63 0x0805b25b in main ()



signature.asc
Description: Digital signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

xapian exceptions not caught in python bindings?

[Patrick Totzke]

Hi all!
It's me again, with some strange python behaviour :/

When I iterate over threads or messages only partially,
then make changes to the index and continue the iteration,
then I don't necessarily get NotmuchError's but the underlying
libnotmuch seems to call terminate. I get the following two lines on
stderr:
  terminate called after throwing an instance of 'Xapian::DatabaseModifiedError'
  Aborted
then python aborts without an exception.
This happens on current master with current python bindings. To reproduce
use this to iterate over messages:

--%<---
#it_read.py
from notmuch import Query,Database
msgs = Database().create_query('*').search_messages()

i=0
for m in msgs:
i=i+1
if i%50==0:
raw_input()
readit = str(m)
-->%---

start it, then use this to change the index:

--%<---
#it_write.py 
from notmuch import Query,Database
db = Database(mode = Database.MODE.READ_WRITE)
msgs = Query(db,'*').search_messages()

i=0
for m in msgs:
i=i+1
if i%20==0:
m.add_tag('REMOVEME')
-->%---

you have to alternate a bit between hitting enter in the
read code and rerunning the write code.
Read will fail at some point.

The strange thing is that its not consistent:
I stumbled upon this while reading a list of threads from notmuch.threads on
demand. If reading from the iterator catches a NotmuchError, i create a new one
and update my list threads read so far. This does work, but: on every
NotmuchError, I get this on stderr:
 A Xapian exception occurred performing query: The revision being read has been 
discarded - you should call Xapian::Database::reopen() and retry the operation
 Query string was: thread:1876

After a couple of resets this does not happen again but the next time i read
from an outdated index, the whole UI terminates with the "Aborted" message
stated above.

To sum up, I think there are two bad things happening here:
1) should libnotmuch really print to stderr?
2) the python bindings are either not relaying all exceptions or libnotmuch is 
not relaying 
all xapian exceptions in order for the bindings to intercept them

cheers,
/p


signature.asc
Description: Digital signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[python] missing notmuch_database_needs_upgrade

[Patrick Totzke]

Hiya,
As mentioned shortly on the irc channel,
notmuch python bindings seem to be missing 'notmuch_database_needs_upgrade'
as Database().needs_upgrade() confirms.

This seems to be due to a typo..

index 5deb2a5..648f61a 100644
--- a/bindings/python/notmuch/database.py
+++ b/bindings/python/notmuch/database.py
@@ -195,7 +195,7 @@ class Database(object):
 # Raise a NotmuchError if not initialized
 self._verify_initialized_db()
 
-return notmuch_database_needs_upgrade(self._db) 
+return nmlib.notmuch_database_needs_upgrade(self._db)
 
 def upgrade(self):
 """Upgrades the current database

cheers,
/p


signature.asc
Description: Digital signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[python] querystrings as unicode literals?

[Patrick Totzke]

Hi!
Something strange goes on when I use unicode literals as querystrings:
Database().create_query(u'teststring') yields different results than
Database().create_query('teststring')..

Now it should not be a problem to decode the string to whatever encoding
is used by notmuch/xapian internally using 'teststring'.encode('utf8')
for example. But can I reliably expect all strings in the index to be valid 
utf8?

At any rate, I think this conversion should be made from inside the bindings.
A query should return the same results for querystrings as string- and unicode 
literals. 
Any thoughts?
/p



signature.asc
Description: Digital signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: Slowness (search opens every email file?)

[Patrick Totzke]

Hi Jason,
On Mon, Jul 11, 2011 at 03:07:21PM -0400, Jason Woofenden wrote:
> notmuch search tag:foo is slow!
> 
yes, i've just used the vim ui for the first time and i agreee, its sluggish,
searching for * takes a while.

> (when my e-mail files are not already in the disk cache)
> 
> I saw on my activity monitor applet that it was using mostly i/o,
> and started to wonder if it was opening every e-mail. I little work
> with strace and searching revealed that this command was opening
> many many e-mail files from my maildir(s). I spent a little while
> digging around in the notmuch source, and didn't see where it was
> opening the email files.

I cannot reproduce this. I'm no expert, but at least the output of 
 strace vim -c ":NotMuch" 2>log
does not contain any path that matches that of my maildir.
Also, I wout be surprised if all individual mails would be read,
because for this, a search for all messages feels too fast.

> 2) Do you agree that it should instead get this information from
> the database?
agreed. And If the mail files get read on every search for you,
something is definately going wrong there.

A quick browse through notmuch.vim tells me that 
1) it doesn't use notmuchs json output, i think it should, as iirc
this api is considered 'more stable' and is easier to parse than the 
default output. More importantly,
2) the output of notmuch is copied into a list. This will of course
be slow if your query matches a lot of messages. Could this be done
by asyncronously writing to the buffer somehow?

best,
/p


signature.asc
Description: Digital signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: Encodings

[Patrick Totzke]

Hi!

As discussed on irc, if notmuch stores header values in utf8,
its safe to decode them to unicode instances here.
best,
/p


On Mon, Jul 11, 2011 at 08:03:38AM -0700, Carl Worth wrote:
> On Mon, 11 Jul 2011 16:04:17 +0200, Sebastian Spaeth  
> wrote:
> > The answer is that things are very implicit. notmuch.h speaks of
> > strings but never mentions encodings
> 
> Much of this was intentional on my part.
> 
> For example, I intentionally avoided restrictions on what could be
> stored as a tag in the database, (other than the terminating character
> implied by "string" of course).
> 
> > So, can be document what encoding we are expected to pass in the various
> > APIs
> 
> Yes, let's clarify documentation wherever we need to.
> 
> > For some of the stuff we read directly from the files, eg
> > arbitrary headers, we can probably be least sure
> 
> The headers should be decoded to utf-8, (via
> g_mime_utils_header_decode_text), before being stored in the database.
> 
> > but are e.g. the returned tags always utf-8?
> 
> No. The tag data is returned exactly as the user presented it.
> 
> > I would love to make the python bindings use unicode() instances in
> > cases where we can be sure to actually receive utf-8 encoded strings.
> > 
> > Encodings make my brain hurt. Unfortunately one cannot simply ignore
> > them.
> 
> I think a lot of the pain here is due to some bad design decisions in
> python itself. Of course, my saying that doesn't make things any easier
> for you.
> 
> But do tell me what more we can do to clarify behavior or documentation.
> 
> -Carl
> 
> -- 
> carl.d.wo...@intel.com



> ___
> notmuch mailing list
> notmuch@notmuchmail.org
> http://notmuchmail.org/mailman/listinfo/notmuch

From 988a9832d714dfa0f91b2b1185a50acb4a6ca4b5 Mon Sep 17 00:00:00 2001
From: pazz 
Date: Tue, 12 Jul 2011 19:47:39 +0100
Subject: [PATCH 1/8] unicode return value for Message.get_header()

As discussed in IRC, notmuch recodes mailheaders to
utf-8, so we can safely decode them into unicode instances.
---
 bindings/python/notmuch/message.py |8 +---
 1 files changed, 5 insertions(+), 3 deletions(-)

diff --git a/bindings/python/notmuch/message.py b/bindings/python/notmuch/message.py
index 763d2c6..4a43a88 100644
--- a/bindings/python/notmuch/message.py
+++ b/bindings/python/notmuch/message.py
@@ -379,14 +379,16 @@ class Message(object):
 
 :param header: The name of the header to be retrieved.
It is not case-sensitive (TODO: confirm).
-:type header: str
-:returns: The header value as string
+:type header: str or unicode instance
+:returns: The header value as a unicode string
 :exception: :exc:`NotmuchError`
 
 * STATUS.NOT_INITIALIZED if the message 
   is not initialized.
 * STATUS.NULL_POINTER, if no header was found
 """
+if isinstance(header, unicode):
+header = header.encode('utf-8')
 if self._msg is None:
 raise NotmuchError(STATUS.NOT_INITIALIZED)
 
@@ -394,7 +396,7 @@ class Message(object):
 header = Message._get_header (self._msg, header)
 if header == None:
 raise NotmuchError(STATUS.NULL_POINTER)
-return header
+return header.decode('utf-8')
 
 def get_filename(self):
 """Returns the file path of the message file
-- 
1.7.4.1



signature.asc
Description: Digital signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: Encodings

[Patrick Totzke]

Hiya,

I noticed that commit 687366b920caa5de6ea0b66b70cf2a11e5399f7b
breaks things with Database.get_all_tags:

-->%-
AttributeErrorTraceback (most recent call last)

/home/pazz/projects/alot/ in ()

/usr/local/lib/python2.7/dist-packages/notmuch/tag.pyc in next(self)
 86 # No need to call nmlib.notmuch_tags_valid(self._tags);

 87 # Tags._get safely returns None, if there is no more valid tag.

---> 88 tag = Tags._get(self._tags).decode('utf-8')
 89 if tag is None:
 90 self._tags = None

AttributeError: 'NoneType' object has no attribute 'decode'
%<---

The reason is that the Tags.next() tries to decode before it tests if tag is 
None.
Now, we _could_ apply a patch like this one here:

-->%-
diff --git a/bindings/python/notmuch/tag.py b/bindings/python/notmuch/tag.py
index 65a9118..2ae670d 100644
--- a/bindings/python/notmuch/tag.py
+++ b/bindings/python/notmuch/tag.py
@@ -85,12 +85,12 @@ class Tags(object):
 raise NotmuchError(STATUS.NOT_INITIALIZED)
 # No need to call nmlib.notmuch_tags_valid(self._tags);
 # Tags._get safely returns None, if there is no more valid tag.
-tag = Tags._get(self._tags).decode('utf-8')
+tag = Tags._get(self._tags)
 if tag is None:
 self._tags = None
 raise StopIteration
 nmlib.notmuch_tags_move_to_next(self._tags)
-return tag
+return tag.decode('utf-8')
 
 def __nonzero__(self):
 """Implement bool(Tags) check that can be repeatedly used
---%<-

But as Carl sais, we cannot guarantee that a tag is utf8 encoded anyway.
So i'd suggest we just revore the commit in question.
best,
/p


signature.asc
Description: Digital signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: Notmuch mail notifier applet for Gnome?

[Patrick Totzke]

Hi Albin,

This is not an answer to your inquiry, but because I looked into something 
similar recently for my setup I'd thought I'd
share: I used xmonad before but switched to the awesome [0] tiling WM. I hacked 
my solution into their wiki, including a
screensot [1].
best,
/p



[0] http://awesome.naquadah.org/
[1] https://awesome.naquadah.org/wiki/Notmuch_mail_integration
On Wed, Jul 13, 2011 at 12:02:06AM +0200, Albin Stjerna wrote:
> Hi all
> 
> I've been using a simple notmuch count-based script together with xmobar in 
> xmonad to notify me of new mail. However, I'm thinking of switching to a 
> Gnome/xmonad-based combo, thus giving up xmobar. I've been looking for a 
> replacement to my mail checker that would work with gnome-panel, but so far I 
> haven't found any. I'm sure there has to be at least one Gnome user on the 
> notmuch mailing list, so I thought I'd ask for help.
> 
> How are you solving this? Basically anything that would display a 
> notification icon if a given shell command returned true would suffice, 
> though I'd prefer something that would also display a given text (read: a 
> short fom of the specified query and the mail count for it, something like: 
> »personal: 10« – possibly during mouse over).
> 
> Thanks in advance,
>   Albin
> ___
> notmuch mailing list
> notmuch@notmuchmail.org
> http://notmuchmail.org/mailman/listinfo/notmuch


signature.asc
Description: Digital signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: Encodings

[Patrick Totzke]

Hi Uwe,

On Wed, Jul 13, 2011 at 09:04:47AM +0200, Uwe Kleine-König wrote:
> > But as Carl sais, we cannot guarantee that a tag is utf8 encoded anyway.
> I think it would be right to enforce that tags are utf-8 encoded.
> Otherwise the users get strange results if they change their locale.

I agree that it would be very nice indeed if it was safe to assume
all tags are utf-8. But i also see that it's a bit of an effort
to ensure this as all UI's would have to explicitly recode
stuff that isn't utf-8.
It seems to be a conciously made design decision to allow
other encodings for tags, which is up for discussion f course.
All I'm saying is that the bindings should conform. And if it's 
not safe to assume utf-8 here, we shouldn't decode as such.

I'm unsure what happens in all the new get_part() parts of the api.
If there, all mimepart-text is also returned as utf-8, it would only
be consistant to bend tag encodings to utf-8 also. But I doubt thats the case.
Can anyone clarify this?
/Patrick


signature.asc
Description: Digital signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

date_relative in notmuch-search

[Patrick Totzke]

Hi,

I wodered why "notmuch search --format=json" doesn't provide the 
"date_relative" field for
results, as the show command does.

I'm not expert on notmuch internals but I got it working like this:

diff --git a/notmuch-search.c b/notmuch-search.c
index faccaf7..b1adc03 100644
--- a/notmuch-search.c
+++ b/notmuch-search.c
@@ -176,12 +176,14 @@ format_thread_json (const void *ctx,
 
 printf ("\"thread\": %s,\n"
"\"timestamp\": %ld,\n"
+   "\"date_relative\": \"%s\",\n"
"\"matched\": %d,\n"
"\"total\": %d,\n"
"\"authors\": %s,\n"
"\"subject\": %s,\n",
json_quote_str (ctx_quote, thread_id),
date,
+   notmuch_time_relative_date (ctx, date),

I'm unsure it it's better to use json_quote_str here but you tell me :)
Best,
/p


signature.asc
Description: Digital signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: date_relative in notmuch-search

[Patrick Totzke]

Hi Jason,

Thanks for the tip with git send-mail. I didn't really intend to
get this "patch" accepted. It was more that I feel bad asking for 
features all the time and as I'm not so comfortable with hacking
on the notmuch code I thought I'd ask if one could implement it like this.
Anyhow, attached you'll find a patch for the test and another for
the proposed change. Feel free to discard it :)
Best,
/p

On Fri, Jul 15, 2011 at 08:58:45AM -0700, Jameson Graef Rollins wrote:
> On Thu, 14 Jul 2011 23:22:58 +0100, Patrick Totzke 
>  wrote:
> > I wodered why "notmuch search --format=json" doesn't provide the 
> > "date_relative" field for
> > results, as the show command does.
> > 
> > I'm not expert on notmuch internals but I got it working like this:
> 
> Hi, Patrick.  I think this is a very reasonable patch.  However, there
> are a couple of things you need to do to get this patch accepted.
> 
> First of all, you should send the patch to the list in a format that
> will allow it to be applied directly to a git repo.  In particular, it
> needs to include a good commit log message.  Look at using git send-mail
> to send it to the list, which will format it correctly.
> 
> Also, I'm quite sure this patch will require a couple of changes to the
> test suite.  You should submit a patch that changes the test suite to
> reflect the desired changes *first*, followed by the patch that fixes
> the issue.
> 
> Thanks!
> 
> jamie.


From 9a0260e36e36af94bed695e713bbc1be0a069fce Mon Sep 17 00:00:00 2001
From: pazz 
Date: Fri, 15 Jul 2011 21:36:54 +0100
Subject: [PATCH 1/2] test: date_relative in notmuch search json output

expect the date_relative field for thread entries
in notmuch search's json output
---
 test/json |2 ++
 1 files changed, 2 insertions(+), 0 deletions(-)

diff --git a/test/json b/test/json
index 5a2544c..d54bf25 100755
--- a/test/json
+++ b/test/json
@@ -12,6 +12,7 @@ add_message "[subject]=\"json-search-subject\"" "[date]=\"Sat, 01 Jan 2000 12:00
 output=$(notmuch search --format=json "json-search-message" | notmuch_search_sanitize)
 test_expect_equal "$output" "[{\"thread\": \"XXX\",
 \"timestamp\": 946728000,
+\"date_relative\": \"2000-01-01\",
 \"matched\": 1,
 \"total\": 1,
 \"authors\": \"Notmuch Test Suite\",
@@ -28,6 +29,7 @@ add_message "[subject]=\"json-search-utf8-body-sübjéct\"" "[date]=\"Sat, 01 Ja
 output=$(notmuch search --format=json "jsön-search-méssage" | notmuch_search_sanitize)
 test_expect_equal "$output" "[{\"thread\": \"XXX\",
 \"timestamp\": 946728000,
+\"date_relative\": \"2000-01-01\",
 \"matched\": 1,
 \"total\": 1,
 \"authors\": \"Notmuch Test Suite\",
-- 
1.7.4.1

From 1a796b09eb4c6171ddaf5f847df56d1a0181cb7c Mon Sep 17 00:00:00 2001
From: pazz 
Date: Fri, 15 Jul 2011 21:41:02 +0100
Subject: [PATCH 2/2] json: date_relative for threads

include the date_relative field in the
json formated output of notmuch search
---
 notmuch-search.c |2 ++
 1 files changed, 2 insertions(+), 0 deletions(-)

diff --git a/notmuch-search.c b/notmuch-search.c
index faccaf7..b1adc03 100644
--- a/notmuch-search.c
+++ b/notmuch-search.c
@@ -176,12 +176,14 @@ format_thread_json (const void *ctx,
 
 printf ("\"thread\": %s,\n"
 	"\"timestamp\": %ld,\n"
+	"\"date_relative\": \"%s\",\n"
 	"\"matched\": %d,\n"
 	"\"total\": %d,\n"
 	"\"authors\": %s,\n"
 	"\"subject\": %s,\n",
 	json_quote_str (ctx_quote, thread_id),
 	date,
+	notmuch_time_relative_date (ctx, date),
 	matched,
 	total,
 	json_quote_str (ctx_quote, authors),
-- 
1.7.4.1



signature.asc
Description: Digital signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: xapian exceptions not caught in python bindings?

[Patrick Totzke]

Hi all,

I know this issue is not easy to fix and that it's partly a problem
with xapians bad error handling. I just want to state another problem we run
into with xapians exceptions in combination with the python bindings.
If you run the following snippet, you notice that not only do we get
xapian-garbage on stderr but we don't really get any exceptions at the
position where it would make sense:


from notmuch import Database
bad_querystring = "test AND"

# This should raise a NotmuchError and query should be NoneType
query = Database().create_query(bad_querystring)

# this prints to stderr but returns a Threads object.
threads = query.search_threads()

# this finally raises a NotmuchError(STATUS.NOT_INITIALIZED)
for t in threads: print t


Although the querystring is invalid syntactically, the query object gets 
instantiated and can of course not behave as it should later on.
From a user point of view it is really quite strange that only later
(if one iterates over threads) an exception is raised.

One solution that comes to mind here is simply bending sys.sdterr to something 
internal,
interpreting its output and raising the correct exceptions accordingly.
The downside is, this would have to happen every time the library gets called.
Also, a fix like this would be python specific, so everybody else would still 
get
garbage printed to stderr occasionally.

Thoughts anyone?
/p



On Sun, Jun 26, 2011 at 09:27:33PM +0100, Patrick Totzke wrote:
> Hi all!
> It's me again, with some strange python behaviour :/
> 
> When I iterate over threads or messages only partially,
> then make changes to the index and continue the iteration,
> then I don't necessarily get NotmuchError's but the underlying
> libnotmuch seems to call terminate. I get the following two lines on
> stderr:
>   terminate called after throwing an instance of 
> 'Xapian::DatabaseModifiedError'
>   Aborted
> then python aborts without an exception.
> This happens on current master with current python bindings. To reproduce
> use this to iterate over messages:
> 
> --%<---
> #it_read.py
> from notmuch import Query,Database
> msgs = Database().create_query('*').search_messages()
> 
> i=0
> for m in msgs:
> i=i+1
> if i%50==0:
> raw_input()
> readit = str(m)
> -->%---
> 
> start it, then use this to change the index:
> 
> --%<---
> #it_write.py 
> from notmuch import Query,Database
> db = Database(mode = Database.MODE.READ_WRITE)
> msgs = Query(db,'*').search_messages()
> 
> i=0
> for m in msgs:
> i=i+1
> if i%20==0:
> m.add_tag('REMOVEME')
> -->%---
> 
> you have to alternate a bit between hitting enter in the
> read code and rerunning the write code.
> Read will fail at some point.
> 
> The strange thing is that its not consistent:
> I stumbled upon this while reading a list of threads from notmuch.threads on
> demand. If reading from the iterator catches a NotmuchError, i create a new 
> one
> and update my list threads read so far. This does work, but: on every
> NotmuchError, I get this on stderr:
>  A Xapian exception occurred performing query: The revision being read has 
> been discarded - you should call Xapian::Database::reopen() and retry the 
> operation
>  Query string was: thread:1876
> 
> After a couple of resets this does not happen again but the next time i read
> from an outdated index, the whole UI terminates with the "Aborted" message
> stated above.
> 
> To sum up, I think there are two bad things happening here:
> 1) should libnotmuch really print to stderr?
> 2) the python bindings are either not relaying all exceptions or libnotmuch 
> is not relaying 
> all xapian exceptions in order for the bindings to intercept them
> 
> cheers,
> /p




signature.asc
Description: Digital signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: [Patch] tag.py: Bugfix to avoid decode() on a NoneType object

[Patrick Totzke]

Hi Michael,

I also fell over this a while ago. (cf. 
http://notmuch.198994.n3.nabble.com/Encodings-td3159281.html)
Your patch certainly fix the immediate error, but there is ore to the problem:
Tagstrings seem to be the only ones stored by notmuch as-is, so unlike
headers, they don't get converted to utf-8. The patch that lead to
this .decode('utf-8') was pushed a bit too hastily.
As Uwe mentiones in above cited thread, we could consider enforcing
tags to be utf-8..

best,
/p


On Fri, Jul 22, 2011 at 02:11:41PM +, Michael Heinrich wrote:
> Dear all,
> 
> with current head I get following error in my python scripts when I read the
> tags of a message:
> 
>   File "/home/heinrich/.local/lib/python2.6/site-packages/notmuch/tag.py", 
> line
> 88, in next
> tag = Tags._get(self._tags).decode('utf-8')
> 
> 
> Here is a patch:
> 
> diff --git a/bindings/python/notmuch/tag.py b/bindings/python/notmuch/tag.py
> index 65a9118..e9049fc 100644
> --- a/bindings/python/notmuch/tag.py
> +++ b/bindings/python/notmuch/tag.py
> @@ -85,10 +85,12 @@ class Tags(object):
>  raise NotmuchError(STATUS.NOT_INITIALIZED)
>  # No need to call nmlib.notmuch_tags_valid(self._tags);
>  # Tags._get safely returns None, if there is no more valid tag.
> -tag = Tags._get(self._tags).decode('utf-8')
> +tag = Tags._get(self._tags)
>  if tag is None:
>  self._tags = None
>  raise StopIteration
> +else:
> +tag = tag.decode('utf-8')
>  nmlib.notmuch_tags_move_to_next(self._tags)
>  return tag
>  
> 
> Michael.
> 
> ___
> notmuch mailing list
> notmuch@notmuchmail.org
> http://notmuchmail.org/mailman/listinfo/notmuch


signature.asc
Description: Digital signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: xapian exceptions not caught in python bindings?

[Patrick Totzke]

Hi all,

I hope the patch I send is correctly formated, I'm still fumbling with
git send-email and the --in-reply-to option.
Anyhow, forgive my language, of course I didn't mean to be condescending in any
way by calling these prints garbage! It's just that it's highly unusual and very
'non-pythonic' that a module directly prints to stderr instead of raising 
exceptions
and if you work directly with a curseslike interface on a terminal these
errormessages litter my screen.

The patch I send is a suggestion how to fix the behaviour described in my last 
post.
It introduces a ContextManager class that can be used to raise messages from 
stderr
as Xapian exceptions like this:

> from notmuch.globals import RaiseStderrErrors
> with RaiseStderrErrors():
>  do_stuff()

Now, if one executes: 

> from notmuch import Database
> bad_querystring = "test AND"
> query = Database().create_query(bad_querystring
---
one gets a nice Xapian exception
---
  File "syntax.py", line 4, in 
query = Database().create_query(bad_querystring)
  File "/usr/local/lib/python2.7/dist-packages/notmuch/database.py", line 432, 
in create_query
return Query(self, querystring)
  File "/usr/local/lib/python2.7/dist-packages/notmuch/database.py", line 514, 
in __init__
self.create(db, querystr)
  File "/usr/local/lib/python2.7/dist-packages/notmuch/database.py", line 547, 
in create
Query._count_messages(self._query)
  File "/usr/local/lib/python2.7/dist-packages/notmuch/globals.py", line 123, 
in __exit__
raise NotmuchError(STATUS.XAPIAN_EXCEPTION, message=err)
notmuch.globals.NotmuchError: A Xapian exception occurred: Syntax:  
AND 
Query string was: test AND


There are two problems with this suggestion however.
First, one needs to redirect sdterr to a tempfile: Using StringIO doesn't work 
since
just replacing sys.stderr with something else is not "low level" enough, the 
messages will
still get printed. An alternative is using os.dup2, which replaces the file 
descriptor
of stderr directly, but cStringIO objects dont have .fileno()..
see [0,1].
The second problem is, that creating a query object with a malformed 
querystring doesn't
print anything to stderr, only if you use that object errors get printed 
(notmuch/lib/query.cc).
Thus, I call count_messages() once after initialising a new 
notmuch.databas.Query object (at the end of  Query.create). This ensures that 
Query.__init__() raises an exception when given bad 
querystrings, but at the cost of triggering an unnecessary count_messages().

best,
/p




[0]: 
http://stackoverflow.com/questions/5903501/attributeerror-stringio-instance-has-no-attribute-fileno
[1]: 
http://code.activestate.com/recipes/577564-context-manager-for-low-level-redirection-of-stdou/


On Sun, Jul 17, 2011 at 04:51:41PM -0300, David Bremner wrote:
> On Sun, 17 Jul 2011 20:35:38 +0100, Patrick Totzke 
>  wrote:
> > If you run the following snippet, you notice that not only do we get
> > xapian-garbage on stderr but we don't really get any exceptions at the
> > position where it would make sense:
> 
> I wouldn't call that "xapian-garbage" since it is output from
> libnotmuch.
> 
> d


signature.asc
Description: Digital signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[PATCH] [python] decode headers from utf-8 to unicode

[Patrick Totzke]

From: patrick 

as mail headers are stored as utf-8 in the index,
it is safe to return them as unicode strings directly
---
 bindings/python/notmuch/message.py |4 ++--
 bindings/python/notmuch/thread.py  |4 ++--
 2 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/bindings/python/notmuch/message.py 
b/bindings/python/notmuch/message.py
index 435a05d..ae6ae1b 100644
--- a/bindings/python/notmuch/message.py
+++ b/bindings/python/notmuch/message.py
@@ -395,7 +395,7 @@ class Message(object):
 header = Message._get_header(self._msg, header)
 if header == None:
 raise NotmuchError(STATUS.NULL_POINTER)
-return header
+return header.decode('UTF-8')
 
 def get_filename(self):
 """Returns the file path of the message file
@@ -747,7 +747,7 @@ class Message(object):
 """A message() is represented by a 1-line summary"""
 msg = {}
 msg['from'] = self.get_header('from')
-msg['tags'] = str(self.get_tags())
+msg['tags'] = self.get_tags()
 msg['date'] = date.fromtimestamp(self.get_date())
 return "%(from)s (%(date)s) (%(tags)s)" % (msg)
 
diff --git a/bindings/python/notmuch/thread.py 
b/bindings/python/notmuch/thread.py
index 60f6c29..120f925 100644
--- a/bindings/python/notmuch/thread.py
+++ b/bindings/python/notmuch/thread.py
@@ -292,7 +292,7 @@ class Thread(object):
 """
 if self._thread is None:
 raise NotmuchError(STATUS.NOT_INITIALIZED)
-return Thread._get_authors(self._thread)
+return Thread._get_authors(self._thread).decode('UTF-8')
 
 def get_subject(self):
 """Returns the Subject of 'thread'
@@ -302,7 +302,7 @@ class Thread(object):
 """
 if self._thread is None:
 raise NotmuchError(STATUS.NOT_INITIALIZED)
-return Thread._get_subject(self._thread)
+return Thread._get_subject(self._thread).decode('UTF-8')
 
 def get_newest_date(self):
 """Returns time_t of the newest message date
-- 
1.7.4.1

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

gui release

[Patrick Totzke]

Good news everyone!

I am pleased to announce the first release of my terminal GUI named 'alot'.
You can get it here: https://github.com/pazz/alot
Install and usage infos are included.
As always, feedback of all kind is very welcome.

Cheers,
/p
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: gui release

[Patrick Totzke]

Whoops :)
Thanks! That is a leftover of todays merges. I did not notice
this as I hade the send.pyc file lying around..
/p


On 21:15pm, Mark Foxwell wrote:
>Patrick Totzke  wrote:
>> I am pleased to announce the first release of my terminal GUI named 'alot'.
>> You can get it here: https://github.com/pazz/alot
>> Install and usage infos are included.
>> As always, feedback of all kind is very welcome.
>
>Great! 
>
>I'm trying to package for Arch Linux [1]. Build completes ok, but when
>running alot I get the following error:
>
>Traceback (most recent call last):
>  File "/usr/bin/alot", line 19, in 
>from alot.init import main
>  File "/usr/lib/python2.7/site-packages/alot/init.py", line 24,
>in 
>from account import AccountManager
>  File "/usr/lib/python2.7/site-packages/alot/account.py",
>line 28, in 
>   from send import SendmailSender
>ImportError: No module named send
>
>I guess I'm missing a dependancy :S any ideas?
>
>-- 
>Mark
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[PATCH 2/2] [python] fix unsafe utf-8 decodings

[Patrick Totzke]

From: pazz 

This prevents unsafe calls to decode for return
value None in get_authors/get_subject
---
 bindings/python/notmuch/tag.py|4 +++-
 bindings/python/notmuch/thread.py |   10 --
 2 files changed, 11 insertions(+), 3 deletions(-)

diff --git a/bindings/python/notmuch/tag.py b/bindings/python/notmuch/tag.py
index d6abf28..9eb9fe2 100644
--- a/bindings/python/notmuch/tag.py
+++ b/bindings/python/notmuch/tag.py
@@ -86,7 +86,9 @@ class Tags(object):
 if not nmlib.notmuch_tags_valid(self._tags):
 self._tags = None
 raise StopIteration
-tag = Tags._get(self._tags).decode('utf-8')
+tag = Tags._get(self._tags)
+if tag:
+tag = tag.decode('UTF-8')
 nmlib.notmuch_tags_move_to_next(self._tags)
 return tag
 
diff --git a/bindings/python/notmuch/thread.py 
b/bindings/python/notmuch/thread.py
index 120f925..2a55bd9 100644
--- a/bindings/python/notmuch/thread.py
+++ b/bindings/python/notmuch/thread.py
@@ -292,7 +292,10 @@ class Thread(object):
 """
 if self._thread is None:
 raise NotmuchError(STATUS.NOT_INITIALIZED)
-return Thread._get_authors(self._thread).decode('UTF-8')
+authors = Thread._get_authors(self._thread)
+if authors:
+return authors.decode('UTF-8')
+return None
 
 def get_subject(self):
 """Returns the Subject of 'thread'
@@ -302,7 +305,10 @@ class Thread(object):
 """
 if self._thread is None:
 raise NotmuchError(STATUS.NOT_INITIALIZED)
-return Thread._get_subject(self._thread).decode('UTF-8')
+subject = Thread._get_subject(self._thread)
+if subject:
+return subject.decode('UTF-8')
+return None
 
 def get_newest_date(self):
 """Returns time_t of the newest message date
-- 
1.7.4.1

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH] Added --initial-index and --last-index to search/show

[Patrick Totzke]

Hi!
A very good idea indeed! This could become quite handy speeding up my interface.
One question: What do you (intend to) do to ensure that accumulated results
'add up well'? 
Consider the following: you query some 10 threads, the user tags one of them
so that it doesn't macht anymore and afterwards you list the next 100 threads.
After the tagging, the number of hits decreases, so if you query
from hit number 101 to 200, you will not return the 100th (now 99th) thread at 
all.

Ok, one could always list all threads up to the one one is interested in but 
that
would make refreshing the list slower and slower when you keep scrolling down..
best,
/p
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: emacs email appears empty

[Patrick Totzke]

Quoting David Bremner (2011-09-29 12:05:37)
>On Thu, 29 Sep 2011 11:21:21 +1000, Brian May  
>wrote:
>> Hello,
>> 
>> Have seen several cases now where the message appears blank in the
>> emacs interface. With nothing but the headers.
>> 
>> Pushing Shift+V shows the entire raw message. Nothing special, just a
>> plain text email without any attachments.
>
>Any chance you could share such a message with us?
>
>d

I have seen something like this using alot but thought i'd be an error in my 
code
(did not test with emacs). It occurs with mails send as google sholar alerts.
/p
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[python] can't find shared lib

[Patrick Totzke]

Hi everyone,
It seems we have a problem with ctypes and libnotmuch:
I manually removed everything notmuch related from my system and pulled the 
latest master branch from git.
After a successful make;sudo make install;
i installed the python bindings from notmuch/bindings/python
via `sudo python ./setup.py install`, which also succeeded. However, If I start 
using the bindings I get:

```...
  File "/usr/local/lib/python2.7/dist-packages/notmuch/__init__.py", line 55, 
in 
from notmuch.database import Database, Query
  File "/usr/local/lib/python2.7/dist-packages/notmuch/database.py", line 22, 
in 
from notmuch.globals import nmlib, STATUS, NotmuchError, Enum, _str
  File "/usr/local/lib/python2.7/dist-packages/notmuch/globals.py", line 28, in 

raise ImportError("Could not find shared 'notmuch' library.")
ImportError: Could not find shared 'notmuch' library.
```

Any ideas?
best,
/p
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: [python] can't find shared lib

[Patrick Totzke]

I fumbled around with resetting come commits, and it turns out that the
problem is in 9e976fc60c40e07cb62a7d656e86ee3101c1b2f5. Once resetted before 
that,
everything works nicely.

The strange thing is that with the current HEAD, so including this commit,
the CLI interface works, only the python bindings seem affected.

I forgot to mention, I'm on Ubuntu 11.04.
best,
/p

Quoting Patrick Totzke (2011-10-05 13:49:57)
>Hi everyone,
>It seems we have a problem with ctypes and libnotmuch:
>I manually removed everything notmuch related from my system and pulled the 
>latest master branch from git.
>After a successful make;sudo make install;
>i installed the python bindings from notmuch/bindings/python
>via `sudo python ./setup.py install`, which also succeeded. However, If I 
>start using the bindings I get:
>
>```...
>  File "/usr/local/lib/python2.7/dist-packages/notmuch/__init__.py", line 55, 
> in 
>from notmuch.database import Database, Query
>  File "/usr/local/lib/python2.7/dist-packages/notmuch/database.py", line 22, 
> in 
>from notmuch.globals import nmlib, STATUS, NotmuchError, Enum, _str
>  File "/usr/local/lib/python2.7/dist-packages/notmuch/globals.py", line 28, 
> in 
>raise ImportError("Could not find shared 'notmuch' library.")
>ImportError: Could not find shared 'notmuch' library.
>```
>
>Any ideas?
>best,
>/p
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: python-notmuch decoding error on a message

[Patrick Totzke]

Silly question: how do i get Antoine's msg stup into notmuch? i tried
using pythons mailbox lib to add this string to one of my mailboxes, which 
works fine.
but upon `notmuch new` I get something along the lines of "skipped non-mail 
file $myfile"..

back to the topic:
I find it hichgly suprising that this decode fails because one can easily do 
sth like:

```
>>>'=?utf-8?B?M+ht?= =?utf-8?Q?e?='.decode('UTF-8')
u'=?utf-8?B?M+ht?= =?utf-8?Q?e?='
```
So the actual string should not be the problem. Apparently,
the string as its stored in the index is not plain ascii anymore, which it was 
in the msg.
I thought notmuch stores exacctly what it gets?

Apart from this, I'd recommend replacing all decodes to unicode objects
by a subroutine that does the following:
If a global property notmuch.DEBUG is set to true: decode as is,
which will raise these exceptions upon errors
else: use .decode('UTF-8', errors='ignore').

In case the mail is not malformed, it will not contain any non-ascii symbols 
whatsoever,
so both ways should work. If you happen to deal with a malformed mail, you'd get
the problematic symbols omitted (beware of this when doing cryptostuff).

what do you think?
/p


Quoting David Bremner (2011-11-24 16:13:22)
>On Sun, 6 Nov 2011 23:15:54 +0100, Antoine Amarilli  
>wrote:
>> Hello,
>> 
>> The attached message makes python-notmuch crash when trying to access it (see
>> attached log).
>> 
>> I don't know if the encoding of Subject is valid or not, but it would 
>> probably
>> be better anyway to ignore decoding errors and return some approximation of
>> Subject instead of failing like this.
>> 
>
>I get a set of critical errors about forgetting to call g_type_init.
>
>We actually call g_type_init in the CLI now, thanks to 
>   
>   id:"1311625989-97755-1-git-send-email-aarone...@gmail.com"
>
>but it sounds like this probably needs to be called either in libnotmuch
>or in the bindings. 
>
>For what it is worth this message decodes fine in the CLI
>
>d
>___
>notmuch mailing list
>notmuch@notmuchmail.org
>http://notmuchmail.org/mailman/listinfo/notmuch

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[python] cleanup the bindings

[Patrick Totzke]

These patches clean up the bindings a bit.
I ran `pep8` and `pyflakes` over the code and fixed the parts it didn't like.
Lastly, I fixed some reStructuredText syntax errors that lead to warnings
while building the sphinx docs.
Best,
/p
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[PATCH 2/3] remove unused imports

[Patrick Totzke]

---
 bindings/python/notmuch/database.py |4 ++--
 bindings/python/notmuch/filename.py |2 +-
 bindings/python/notmuch/globals.py  |1 -
 bindings/python/notmuch/message.py  |3 +--
 bindings/python/notmuch/thread.py   |2 +-
 5 files changed, 5 insertions(+), 7 deletions(-)

diff --git a/bindings/python/notmuch/database.py 
b/bindings/python/notmuch/database.py
index ca17273..706e8c8 100644
--- a/bindings/python/notmuch/database.py
+++ b/bindings/python/notmuch/database.py
@@ -18,9 +18,9 @@ Copyright 2010 Sebastian Spaeth '
 """
 
 import os
-from ctypes import c_int, c_char_p, c_void_p, c_uint, c_long, byref, POINTER
+from ctypes import c_char_p, c_void_p, c_uint, c_long, byref, POINTER
 from notmuch.globals import (nmlib, STATUS, NotmuchError, NotInitializedError,
- NullPointerError, OutOfMemoryError, XapianError, Enum, _str,
+ NullPointerError, Enum, _str,
  NotmuchDatabaseP, NotmuchDirectoryP, NotmuchMessageP, NotmuchTagsP,
  NotmuchQueryP, NotmuchMessagesP, NotmuchThreadsP, NotmuchFilenamesP)
 from notmuch.thread import Threads
diff --git a/bindings/python/notmuch/filename.py 
b/bindings/python/notmuch/filename.py
index 077754e..6b332a9 100644
--- a/bindings/python/notmuch/filename.py
+++ b/bindings/python/notmuch/filename.py
@@ -18,7 +18,7 @@ Copyright 2010 Sebastian Spaeth '
 """
 from ctypes import c_char_p
 from notmuch.globals import (nmlib, STATUS, NotmuchError,
-NotmuchFilenamesP, NotmuchMessagesP, NotmuchMessageP)
+NotmuchFilenamesP, NotmuchMessageP)
 
 
 class Filenames(object):
diff --git a/bindings/python/notmuch/globals.py 
b/bindings/python/notmuch/globals.py
index f5c8d52..f69c73d 100644
--- a/bindings/python/notmuch/globals.py
+++ b/bindings/python/notmuch/globals.py
@@ -18,7 +18,6 @@ Copyright 2010 Sebastian Spaeth '
 """
 
 from ctypes import CDLL, c_char_p, c_int, Structure, POINTER
-from ctypes.util import find_library
 
 #-
 #package-global instance of the notmuch library
diff --git a/bindings/python/notmuch/message.py 
b/bindings/python/notmuch/message.py
index e0c7eda..2534742 100644
--- a/bindings/python/notmuch/message.py
+++ b/bindings/python/notmuch/message.py
@@ -19,7 +19,7 @@ Copyright 2010 Sebastian Spaeth '
 """
 
 
-from ctypes import c_char_p, c_void_p, c_long, c_uint, c_int
+from ctypes import c_char_p, c_long, c_uint, c_int
 from datetime import date
 from notmuch.globals import (nmlib, STATUS, NotmuchError, Enum, _str,
 NotmuchTagsP, NotmuchMessagesP, NotmuchMessageP, NotmuchFilenamesP)
@@ -27,7 +27,6 @@ from notmuch.tag import Tags
 from notmuch.filename import Filenames
 import sys
 import email
-import types
 try:
 import simplejson as json
 except ImportError:
diff --git a/bindings/python/notmuch/thread.py 
b/bindings/python/notmuch/thread.py
index dbd6c0f..c575c88 100644
--- a/bindings/python/notmuch/thread.py
+++ b/bindings/python/notmuch/thread.py
@@ -17,7 +17,7 @@ along with notmuch.  If not, see 
.
 Copyright 2010 Sebastian Spaeth '
 """
 
-from ctypes import c_char_p, c_void_p, c_long, c_int
+from ctypes import c_char_p, c_long, c_int
 from notmuch.globals import (nmlib, STATUS,
 NotmuchError, NotmuchThreadP, NotmuchThreadsP, NotmuchMessagesP,
 NotmuchTagsP,)
-- 
1.7.4.1

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[PATCH 1/3] pep8 fixes

[Patrick Totzke]

no changes to the code, only fixed stuff denounced by `pep8 *py`
---
 bindings/python/notmuch/__init__.py |2 +-
 bindings/python/notmuch/database.py |   28 +++--
 bindings/python/notmuch/globals.py  |   38 +++---
 bindings/python/notmuch/thread.py   |1 -
 4 files changed, 52 insertions(+), 17 deletions(-)

diff --git a/bindings/python/notmuch/__init__.py 
b/bindings/python/notmuch/__init__.py
index 539afed..f3ff987 100644
--- a/bindings/python/notmuch/__init__.py
+++ b/bindings/python/notmuch/__init__.py
@@ -69,7 +69,7 @@ from notmuch.globals import (
 TagTooLongError,
 UnbalancedFreezeThawError,
 UnbalancedAtomicError,
-NotInitializedError
+NotInitializedError,
 )
 from notmuch.version import __VERSION__
 __LICENSE__ = "GPL v3+"
diff --git a/bindings/python/notmuch/database.py 
b/bindings/python/notmuch/database.py
index 25b4b1b..ca17273 100644
--- a/bindings/python/notmuch/database.py
+++ b/bindings/python/notmuch/database.py
@@ -27,6 +27,7 @@ from notmuch.thread import Threads
 from notmuch.message import Messages, Message
 from notmuch.tag import Tags
 
+
 class Database(object):
 """The :class:`Database` is the highest-level object that notmuch
 provides. It references a notmuch database, and can be opened in
@@ -83,12 +84,14 @@ class Database(object):
 
 """ notmuch_database_find_message"""
 _find_message = nmlib.notmuch_database_find_message
-_find_message.argtypes = [NotmuchDatabaseP, c_char_p, 
POINTER(NotmuchMessageP)]
+_find_message.argtypes = [NotmuchDatabaseP, c_char_p,
+  POINTER(NotmuchMessageP)]
 _find_message.restype = c_uint
 
 """notmuch_database_find_message_by_filename"""
 _find_message_by_filename = nmlib.notmuch_database_find_message_by_filename
-_find_message_by_filename.argtypes = [NotmuchDatabaseP, c_char_p, 
POINTER(NotmuchMessageP)]
+_find_message_by_filename.argtypes = [NotmuchDatabaseP, c_char_p,
+  POINTER(NotmuchMessageP)]
 _find_message_by_filename.restype = c_uint
 
 """notmuch_database_get_all_tags"""
@@ -177,8 +180,8 @@ class Database(object):
 :param status: Open the database in read-only or read-write mode
 :type status:  :attr:`MODE`
 :returns: Nothing
-:exception: Raises :exc:`NotmuchError` in case
-of any failure (possibly after printing an error message on 
stderr).
+:exception: Raises :exc:`NotmuchError` in case of any failure
+(possibly after printing an error message on stderr).
 """
 res = Database._open(_str(path), mode)
 
@@ -293,12 +296,12 @@ class Database(object):
 (creating it if it does not exist(?))
 
 .. warning:: This call needs a writeable database in
-   :attr:`Database.MODE`.READ_WRITE mode. The underlying library will 
exit the
-   program if this method is used on a read-only database!
+   :attr:`Database.MODE`.READ_WRITE mode. The underlying library will
+   exit the program if this method is used on a read-only database!
 
 :param path: An unicode string containing the path relative to the path
-  of database (see :meth:`get_path`), or else should be an 
absolute path
-  with initial components that match the path of 'database'.
+  of database (see :meth:`get_path`), or else should be an absolute
+  path with initial components that match the path of 'database'.
 :returns: :class:`Directory` or raises an exception.
 :exception:
 :exc:`NotmuchError` with :attr:`STATUS`.FILE_ERROR
@@ -325,7 +328,8 @@ class Database(object):
 return Directory(_str(abs_dirpath), dir_p, self)
 
 _add_message = nmlib.notmuch_database_add_message
-_add_message.argtypes = [NotmuchDatabaseP, c_char_p, 
POINTER(NotmuchMessageP)]
+_add_message.argtypes = [NotmuchDatabaseP, c_char_p,
+ POINTER(NotmuchMessageP)]
 _add_message.restype = c_uint
 
 def add_message(self, filename, sync_maildir_flags=False):
@@ -490,7 +494,8 @@ class Database(object):
 """Returns :class:`Tags` with a list of all tags found in the database
 
 :returns: :class:`Tags`
-:execption: :exc:`NotmuchError` with :attr:`STATUS`.NULL_POINTER on 
error
+:execption: :exc:`NotmuchError` with :attr:`STATUS`.NULL_POINTER
+on error
 """
 self._assert_db_is_initialized()
 tags_p = Database._get_all_tags(self._db)
@@ -748,7 +753,8 @@ class Directory(object):
 _get_child_directories.restype = NotmuchFilenamesP
 
 def _assert_dir_is_initialized(self):
-"""Raises a NotmuchError(:attr:`STATUS`.NOT_INITIALIZED) if dir_p is 
None"""
+"""Raises a NotmuchError(:attr:`STATUS`.NOT_INITIALIZED)
+if dir_p is None"""
 if self._dir_p is None:
 

[PATCH 3/3] fix sphinx compile-time warnings

[Patrick Totzke]

no changes to the code, only makes compiling the docs smoother
as some rsT syntax errors were fixed
---
 bindings/python/docs/source/index.rst |   17 
 bindings/python/notmuch/database.py   |   67 ++--
 bindings/python/notmuch/message.py|   27 +++--
 bindings/python/notmuch/tag.py|   11 +++--
 4 files changed, 68 insertions(+), 54 deletions(-)

diff --git a/bindings/python/docs/source/index.rst 
b/bindings/python/docs/source/index.rst
index 73d2a3b..f7d3d60 100644
--- a/bindings/python/docs/source/index.rst
+++ b/bindings/python/docs/source/index.rst
@@ -138,10 +138,10 @@ More information on specific topics can be found on the 
following pages:
 
.. method:: __len__()
 
-   .. warning:: :meth:`__len__` was removed in version 0.6 as it exhausted
-   the iterator and broke list(Messages()). Use the
-   :meth:`Query.count_messages` function or use
-   `len(list(msgs))`.
+   .. warning::
+   
+  :meth:`__len__` was removed in version 0.6 as it exhausted the iterator 
and broke
+  list(Messages()). Use the :meth:`Query.count_messages` function or use 
`len(list(msgs))`.
 
 :class:`Message` -- A single message
 
@@ -205,10 +205,11 @@ More information on specific topics can be found on the 
following pages:
 
.. method:: __len__
 
-   .. warning:: :meth:`__len__` was removed in version 0.6 as it
-   exhausted the iterator and broke list(Tags()). Use
-   :meth:`len(list(msgs))` instead if you need to know the
-   number of tags.
+   .. warning::
+
+:meth:`__len__` was removed in version 0.6 as it exhausted the 
iterator and broke
+list(Tags()). Use :meth:`len(list(msgs))` instead if you need to 
know the number of
+tags.
 
.. automethod:: __str__
 
diff --git a/bindings/python/notmuch/database.py 
b/bindings/python/notmuch/database.py
index 706e8c8..471adaa 100644
--- a/bindings/python/notmuch/database.py
+++ b/bindings/python/notmuch/database.py
@@ -40,16 +40,19 @@ class Database(object):
 :exc:`XapianError` as the underlying database has been
 modified. Close and reopen the database to continue working with it.
 
-.. note:: Any function in this class can and will throw an
-   :exc:`NotInitializedError` if the database was not
-   intitialized properly.
-
-.. note:: Do remember that as soon as we tear down (e.g. via `del
-   db`) this object, all underlying derived objects such as
-   queries, threads, messages, tags etc will be freed by the
-   underlying library as well. Accessing these objects will lead
-   to segfaults and other unexpected behavior. See above for
-   more details.
+.. note::
+
+Any function in this class can and will throw an
+:exc:`NotInitializedError` if the database was not intitialized
+properly.
+
+.. note::
+
+Do remember that as soon as we tear down (e.g. via `del db`) this
+object, all underlying derived objects such as queries, threads,
+messages, tags etc will be freed by the underlying library as well.
+Accessing these objects will lead to segfaults and other unexpected
+behavior. See above for more details.
 """
 _std_db_path = None
 """Class attribute to cache user's default database"""
@@ -253,10 +256,8 @@ class Database(object):
 neither begin nor end necessarily flush modifications to disk.
 
 :returns: :attr:`STATUS`.SUCCESS or raises
-
-:exception: :exc:`NotmuchError`:
-:attr:`STATUS`.XAPIAN_EXCEPTION
-Xapian exception occurred; atomic section not entered.
+:exception: :exc:`NotmuchError`: :attr:`STATUS`.XAPIAN_EXCEPTION
+Xapian exception occurred; atomic section not entered.
 
 *Added in notmuch 0.9*"""
 self._assert_db_is_initialized()
@@ -295,9 +296,11 @@ class Database(object):
 """Returns a :class:`Directory` of path,
 (creating it if it does not exist(?))
 
-.. warning:: This call needs a writeable database in
-   :attr:`Database.MODE`.READ_WRITE mode. The underlying library will
-   exit the program if this method is used on a read-only database!
+.. warning::
+
+This call needs a writeable database in
+:attr:`Database.MODE`.READ_WRITE mode. The underlying library will
+exit the program if this method is used on a read-only database!
 
 :param path: An unicode string containing the path relative to the path
   of database (see :meth:`get_path`), or else should be an absolute
@@ -459,10 +462,11 @@ class Database(object):
 def find_message_by_filename(self, filename):
 """Find a message with the given filename
 
-.. warning:: This call needs a writeable database in
-   :attr:`Database.MODE`.READ_WRITE

[python] unicode strings

[Patrick Totzke]

Unicode handling fixes for the bindings:
 - use __unicode__ for string representations, __str__ falls back to this
 - less complicated __str__ for Thread and Message
 - use errors='ignore' parameter for str.decode(). This should fix the
   "UnicodeDecodeError: 'utf8' codec can't decode byte 0xe8 in position"
   exceptions people were seing.

Best,
/p
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch