Re: [notmuch] Quick thoughts on a notmuch daemon
I started to code something base on your idea of a notmuch daemon. You can find it on: git://gitorious.org/notsomuch/notmuch.git On the server branch. The idea is to use unix named sockets to intercomunicate between the daemon and the client. And threads on the server to handle every request. The implementation is no great, it's a fast hack. It can only handle one request per connection and breaks some times on concurrent request. But I hope helps to see the idea. I implemented both, daemon and client in the same binary. So you can still run as before: $ notmuch search inbox If the daemon is already running (so the socket is in MAILDIR_PATH/.notmuch/socket) it will connect to it and ask for the search. If is not running will fork creating it and send it the search. Up to now the comunication between daemon and client is with the same syntax of notmuch. But I think will be a nice idea to use JSON (or some other computer-friendly syntax) and convert it to human readable on the client. What do you think about that approach? Will it fit on what you imagined or is it to far? I'm not sure if that is adding to much complexity to notmuch or is a good idea. -- Rubén Pollán | jabber:mes...@jabber.org -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- Nos vamos a Croatan. signature.asc Description: Digital signature ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
Re: [notmuch] Quick thoughts on a notmuch daemon
On 14:27, Thu 03 Dec 09, Carl Worth wrote: A simple solution would be a notmuch daemon that can accept commands on stdin, (in basically the exact same form as the current notmuch command-line interface). If the daemon does the job of periodically incorporating new mail, then the only command necessary to solve (1) above would be the tag command. I like the idea. I didn't liked to fork for each command, so I started to play with the library for create a UI. But with a demon like that I guess will be nicer to use it than to call directly to the library. Why use stdin? Why not sockets? With them at could be possible to use several concurrent clients with the same server. (I really love moc for play music, and one of its greatest features is that) -- Rubén Pollán | jabber:mes...@jabber.org -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- No vamos a reivindicar nada, no vamos a pedir nada. Tomaremos, okuparemos. signature.asc Description: Digital signature ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
Re: [notmuch] Quick thoughts on a notmuch daemon
On Thu, 03 Dec 2009 14:27:05 -0800, Carl Worth cwo...@cworth.org wrote: A simple solution would be a notmuch daemon that can accept commands on stdin, (in basically the exact same form as the current notmuch command-line interface). If the daemon does the job of periodically incorporating new mail, then the only command necessary to solve (1) above would be the tag command. If you add a second pipe for notmuch to broadcast information about events (such as new mail being indexed) you could farm out most of the logic that will increasingly clutter up notmuch-new.c to an external daemon. Just the mailid and path would be enough for people to implement their own tagging based on directory, Maildir flags or (for all I care) Bayesian content filtering with relative ease. Just a thought -- Michiel Buddingh' ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
Re: [notmuch] Quick thoughts on a notmuch daemon
On Mon, 07 Dec 2009 15:02:03 -0400, David Bremner da...@tethera.net wrote: On Thu, 03 Dec 2009 14:27:05 -0800, Carl Worth cwo...@cworth.org wrote: It might be a bit blue sky, but if this daemon could (optionally) talk IMAP and translate tags into folders on the fly, this would be very useful for people who need imap access to their mail as well as from an custom notmuch client. For me, my IMAP needs are pretty much limited to my iPhone. I'm making do for now, but to make notmuch viable in the long term, what I'd need is: * notmuch shouldn't choke on mails I had in notmuch's database, and then marked read or deleted on my iPhone (which renames them in the maildir). This is coming with the moving/deleting patches. * notmuch should sync back read/unread state to maildir * notmuch should move read stuff out of my inbox. It would be acceptable if it moved everything into a designated archive folder unless it had the 'inbox' tag assigned, in which case it moved it there. Note that we have the moving/deleting patches, then this could even be done as a script and some searches. With this, my inbox would be usable from my iPhone. -- - Marten ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
Re: [notmuch] Quick thoughts on a notmuch daemon
Excerpts from Marten Veldthuis's message of Mon Dec 07 17:55:24 -0500 2009: On Mon, 07 Dec 2009 15:02:03 -0400, David Bremner da...@tethera.net wrote: On Thu, 03 Dec 2009 14:27:05 -0800, Carl Worth cwo...@cworth.org wrote: It might be a bit blue sky, but if this daemon could (optionally) talk IMAP and translate tags into folders on the fly, this would be very useful for people who need imap access to their mail as well as from an custom notmuch client. For me, my IMAP needs are pretty much limited to my iPhone. I'm making do for now, but to make notmuch viable in the long term, what I'd need is: * notmuch shouldn't choke on mails I had in notmuch's database, and then marked read or deleted on my iPhone (which renames them in the maildir). This is coming with the moving/deleting patches. * notmuch should sync back read/unread state to maildir * notmuch should move read stuff out of my inbox. It would be acceptable if it moved everything into a designated archive folder unless it had the 'inbox' tag assigned, in which case it moved it there. Note that we have the moving/deleting patches, then this could even be done as a script and some searches. With this, my inbox would be usable from my iPhone. I dont have an iPhone, but this would be a *killer* feature for me, and I suspect for others as well. I have two fundamental problems with label-state clients (sup, notmuch) that I am still trying to come to terms with, this bluesky feature would change that radically. The first issue is that I do not have unlimited IMAP storage space on the server, in fact I have a quota. I know, lots of people have effectively unlimited space,and others POP/fetchmail things so its not stored on the server...but lots of us still do have upper limits that we have to deal with and many of us do use IMAP, for good reasons. With label-state clients (sup, notmuch) you have to accept the fact that your mail is going to grow on the IMAP server indefinitely. This is not a good thing for those of us with quotas, but it is also a bad thing for some IMAP servers who choke to death on very large IMAP stores (or consume an awful lot of memory dealing with them). Or you have to setup kludgy archive operations to periodically deal with the disk space issue. The second mind-bending problem is that sometimes I actually do have to use mutt, sometimes webmail (for various reasons, but one important one in the early stages of a new email client is that notmuch just doesn't have support for certain operations such as inline/pgp-mime handling of emails, which means I need to open those emails in other clients that do support that). Other people likely are going to need to use things like iphones or blackberries. Indeed, using other clients is not an unreasonable thing for people to do. However, switching between notmuch and these clients is. Because the message state is stored in a DB which is only useful for notmuch, its a dreadful nightmare to do anything outside of the notmuch world. Most importantly, having all of your email state in a notmuch database format means that it can only be parsed by those tools, and is no longer in a standard format. I think it is great that 'notmuch restore' can deal with the sup database format, which may signal the dawn of a new label-state standardbut still the problem is the glue to the underlying maildir structure which provides a lot of useful information contained in reasonably well-defined ways is totally thrown out the window. I've switched mail clients enough over the past few years to know that switching itself is a major pain. I've settled on Maildir as my storage method of choice and in a way it is a 'client'. I can serve it via IMAP if I need, I can read it with different mail clients and maintain state across those mail clients, there are tools to manipulate and convert maildirs that have matured over the years. If I switch to an 'all-in-one' label-state mail client, like notmuch, I want to be sure that in 2 years from now if I happen to decide to change to something else (I'm hoping this wont EVER happen because notmuch is very promising, but I need to be honest based on past experiences here), then I'm going to want to make sure that all of my mail is marked as read, deleted mail is deleted and even though I was using labels for organizational purposes in notmuch, things are still organized in some way appropriate to folders on the filesystem. The way it is now, if I switch to notmuch and then try to switch back, my life is a total mess because all of the state is contained within the notmuch database, that is frightening for the long-term, but it also makes me very worried about simple corruption of that data store. If I lose that state, I'm totally screwed. At least in a maildir scenario, if I got corruption in one place it might cause me to lose one email, but if I get corruption in my notmuch database, I had better have a recent
Re: [notmuch] Quick thoughts on a notmuch daemon
On Mon, 07 Dec 2009 18:51:58 -0500, micah anderson mi...@riseup.net wrote: I've switched mail clients enough over the past few years to know that switching itself is a major pain. Absolutely. One concept in notmuch (compared to sup) is to (eventually) avoid people having to go through that pain by their current mail client becoming notmuch enabled. For me, I had always liked composing email in emacs, so I just have to add notmuch support to the existing emacs message-mode. Hopefully people working on other email interfaces will do similar things, (would be great to have Anjal and Thunderbird get some notmuch support for example). I definitely didn't like that with sup, to get all the global-search and tagging features one had to accept the curses UI as well. If I switch to an 'all-in-one' label-state mail client, like notmuch, I want to be sure that in 2 years from now if I happen to decide to change to something else (I'm hoping this wont EVER happen because notmuch is very promising, but I need to be honest based on past experiences here), then I'm going to want to make sure that all of my mail is marked as read, deleted mail is deleted and even though I was using labels for organizational purposes in notmuch, things are still organized in some way appropriate to folders on the filesystem. I appreciate your caution before making the commitment to a mail client. I think that's very wise. [*] And I think that for the case of I'm giving up on notmuch and want to switch to something else you can be quite assured that notmuch will allow you to take care of everything you need. At a system level, it's really an almost trivial matter to implement everything you describe above as small shell scripts based on notmuch search commands. And I can only imagine support for that kind of thing getting better all the time. The way it is now, if I switch to notmuch and then try to switch back, my life is a total mess because all of the state is contained within the notmuch database, that is frightening for the long-term, but it also makes me very worried about simple corruption of that data store. If I lose that state, I'm totally screwed. At least in a maildir scenario, if I got corruption in one place it might cause me to lose one email, but if I get corruption in my notmuch database, I had better have a recent backup or I am screwed. I highly recommend making very regular backups of the output of notmuch dump. It's a quick process to run, and the file it creates is not large. It's also nicely sorted so that if you are doing some kind of incremental backup, that should work well. I like to think that this concept of label-state, indexable storage-backed mail clients is the dawn of a new age, just as maildir was to mbox, but I'm still scared because there is no mb2md equivalent yet. I think we're really close to where you could write a notmuch2md script as a very tiny shell script: for tag in $(notmuch search --for=tags *); do notmuch search --for=messages --output=maildir:/some/dir/$tag tag:$tag done Being able to do that kind of scriptable output is pretty powerful. And we're not missing anything fundamental in the system to be able to support that, (just a little bit of support for the new command-line options like --for and --output). -Carl [*] I know I went through a similarly cautious evaluation when switching away from CVS. CVS made the transition away so painful that I was determined to choose a system that satisfied me on two criteria: 1. The system seemed well-designed enough that I could imagine it surviving forever. 2. That when realities exceeded my imagination, the system wouldn't make it hard for me to switch away. So far I'm still quite happy with git on both points. pgpAJYJXcOJTM.pgp Description: PGP signature ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
