notmuch's idea of concurrency / failing an invocation

2011-02-24 Thread Austin Clements
Now that I've split notmuch new up in to lots of small transactions, I think the database locking issue is quite approachable. Here's a proposed locking protocol where a notmuch operation that wants to modify the database blocks if there's another operation in progress (instead of immediately

notmuch's idea of concurrency / failing an invocation

2011-02-03 Thread micah anderson
On Thu, 27 Jan 2011 12:35:16 -0800, Jameson Rollins wrote: > On Thu, 27 Jan 2011 13:40:25 -0500, micah anderson > wrote: > > Due to my harddisk in my laptop being slow (5400RPM), my notmuch > > database growing, and perhaps some fragmentation somewhere, this has > > become *incredibly* annoying

Re: notmuch's idea of concurrency / failing an invocation

2011-02-03 Thread micah anderson
On Thu, 27 Jan 2011 12:35:16 -0800, Jameson Rollins jroll...@finestructure.net wrote: On Thu, 27 Jan 2011 13:40:25 -0500, micah anderson mi...@riseup.net wrote: Due to my harddisk in my laptop being slow (5400RPM), my notmuch database growing, and perhaps some fragmentation somewhere, this

notmuch's idea of concurrency / failing an invocation

2011-02-02 Thread Stewart Smith
On Sat, 29 Jan 2011 19:14:27 -0500, Daniel Kahn Gillmor wrote: > On 01/28/2011 08:05 PM, Stewart Smith wrote: > > I'm about at the point where I'm going to take my git mail store > > experiments and get them really to work (and everyone will have to use > > 'notmuch cat' or the like to access the

Re: notmuch's idea of concurrency / failing an invocation

2011-02-01 Thread Stewart Smith
On Sat, 29 Jan 2011 19:14:27 -0500, Daniel Kahn Gillmor d...@fifthhorseman.net wrote: On 01/28/2011 08:05 PM, Stewart Smith wrote: I'm about at the point where I'm going to take my git mail store experiments and get them really to work (and everyone will have to use 'notmuch cat' or the

notmuch's idea of concurrency / failing an invocation

2011-01-29 Thread Daniel Kahn Gillmor
On 01/28/2011 08:05 PM, Stewart Smith wrote: > I'm about at the point where I'm going to take my git mail store > experiments and get them really to work (and everyone will have to use > 'notmuch cat' or the like to access the messages) Would this hypothetical git-based mail store retain the

notmuch's idea of concurrency / failing an invocation

2011-01-29 Thread Mike Kelly
On Fri, 28 Jan 2011 11:57:34 -0500 Austin Clements wrote: > Yes, exactly. All of this. Unfortunately, Xapian doesn't expose the > ability to block on the lock (see the fcntl call in > backends/flint_lock.cc, which is hard-coded to the non-blocking > F_SETLK instead of F_SETLKW), so we'd either

notmuch's idea of concurrency / failing an invocation

2011-01-29 Thread Stewart Smith
On Thu, 27 Jan 2011 13:40:25 -0500, micah anderson wrote: > Due to my harddisk in my laptop being slow (5400RPM), my notmuch > database growing, and perhaps some fragmentation somewhere, this has > become *incredibly* annoying for me. I am checking email every 30 > minutes, and I'm nicing and

Re: notmuch's idea of concurrency / failing an invocation

2011-01-29 Thread Mike Kelly
On Fri, 28 Jan 2011 11:57:34 -0500 Austin Clements amdra...@mit.edu wrote: Yes, exactly. All of this. Unfortunately, Xapian doesn't expose the ability to block on the lock (see the fcntl call in backends/flint_lock.cc, which is hard-coded to the non-blocking F_SETLK instead of F_SETLKW), so

Re: notmuch's idea of concurrency / failing an invocation

2011-01-29 Thread Stewart Smith
On Thu, 27 Jan 2011 13:40:25 -0500, micah anderson mi...@riseup.net wrote: Due to my harddisk in my laptop being slow (5400RPM), my notmuch database growing, and perhaps some fragmentation somewhere, this has become *incredibly* annoying for me. I am checking email every 30 minutes, and I'm

Re: notmuch's idea of concurrency / failing an invocation

2011-01-29 Thread Daniel Kahn Gillmor
On 01/28/2011 08:05 PM, Stewart Smith wrote: I'm about at the point where I'm going to take my git mail store experiments and get them really to work (and everyone will have to use 'notmuch cat' or the like to access the messages) Would this hypothetical git-based mail store retain the

notmuch's idea of concurrency / failing an invocation

2011-01-28 Thread Mike Kelly
On Fri, 28 Jan 2011 10:45:19 +0100, Thomas Schwinge wrote: > > It would definitely be nice to avoid the complexity inherent in having a > > daemon, but how do you imagine "queue on a lock" to work? We don't have > > anything like that in place now. > > I suppose what he means is trying to get

notmuch's idea of concurrency / failing an invocation

2011-01-28 Thread Carl Worth
On Thu, 27 Jan 2011 17:20:21 -0500, Austin Clements wrote: > I'm looking into breaking notmuch new up into small transactions. It > wouldn't be much a leap from there to simply close and reopen the database > between transactions if another task wants to use it, which would release > the lock

notmuch's idea of concurrency / failing an invocation

2011-01-28 Thread Carl Worth
On Thu, 27 Jan 2011 19:20:00 +0100, Thomas Schwinge wrote: > Which is the original idea here? Is it that... There's no original idea yet. It's essentially an unsolved problem right now. > * each and every client should catch these kinds of errors, and retry, > or eventually give up at

notmuch's idea of concurrency / failing an invocation

2011-01-28 Thread Austin Clements
Actually, this is trivial to play with. Here's a stop-gap wrapper script for people having trouble with Xapian locking, #!/bin/bash NOTMUCH_BIN="/path/to/notmuch" MAIL_DIR="/path/to/mailroot" ( case "$1" in setup|help) ;;

notmuch's idea of concurrency / failing an invocation

2011-01-28 Thread Austin Clements
On Fri, Jan 28, 2011 at 10:36 AM, Mike Kelly wrote: > On Fri, 28 Jan 2011 10:45:19 +0100, Thomas Schwinge > wrote: > > > It would definitely be nice to avoid the complexity inherent in having > a > > > daemon, but how do you imagine "queue on a lock" to work? We don't have > > > anything like

notmuch's idea of concurrency / failing an invocation

2011-01-28 Thread Austin Clements
On Fri, Jan 28, 2011 at 4:45 AM, Thomas Schwinge wrote: > On Fri, 28 Jan 2011 15:10:01 +1000, Carl Worth wrote: > > On Thu, 27 Jan 2011 17:20:21 -0500, Austin Clements > wrote: > > > I'm looking into breaking notmuch new up into small transactions. It > > > wouldn't be much a leap from there

notmuch's idea of concurrency / failing an invocation

2011-01-28 Thread Thomas Schwinge
Hallo! On Fri, 28 Jan 2011 15:10:01 +1000, Carl Worth wrote: > On Thu, 27 Jan 2011 17:20:21 -0500, Austin Clements > wrote: > > I'm looking into breaking notmuch new up into small transactions. It > > wouldn't be much a leap from there to simply close and reopen the database > > between

Re: notmuch's idea of concurrency / failing an invocation

2011-01-28 Thread Austin Clements
On Fri, Jan 28, 2011 at 10:36 AM, Mike Kelly pi...@pioto.org wrote: On Fri, 28 Jan 2011 10:45:19 +0100, Thomas Schwinge tho...@schwinge.name wrote: It would definitely be nice to avoid the complexity inherent in having a daemon, but how do you imagine queue on a lock to work? We don't

Re: notmuch's idea of concurrency / failing an invocation

2011-01-28 Thread Austin Clements
Actually, this is trivial to play with. Here's a stop-gap wrapper script for people having trouble with Xapian locking, #!/bin/bash NOTMUCH_BIN=/path/to/notmuch MAIL_DIR=/path/to/mailroot ( case $1 in setup|help) ;; search|show|count|reply|dump|search-tags|part)

notmuch's idea of concurrency / failing an invocation

2011-01-27 Thread Thomas Schwinge
Hallo! Stepping away from the current code base -- what is notmuch's original idea of concurrency? That is, all of us probably know that one: A Xapian exception occurred opening database: Unable to get write lock on /home/thomas/Mail-schwinge.name-thomas/.notmuch/xapian: already

notmuch's idea of concurrency / failing an invocation

2011-01-27 Thread Austin Clements
I'm looking into breaking notmuch new up into small transactions. It wouldn't be much a leap from there to simply close and reopen the database between transactions if another task wants to use it, which would release the lock and let the queued notmuch task have the database for a bit. It seems

notmuch's idea of concurrency / failing an invocation

2011-01-27 Thread micah anderson
On Thu, 27 Jan 2011 19:20:00 +0100, Thomas Schwinge wrote: > Stepping away from the current code base -- what is notmuch's original > idea of concurrency? That is, all of us probably know that one: > > A Xapian exception occurred opening database: Unable to get write > lock on

notmuch's idea of concurrency / failing an invocation

2011-01-27 Thread Jameson Rollins
On Thu, 27 Jan 2011 13:40:25 -0500, micah anderson wrote: > Due to my harddisk in my laptop being slow (5400RPM), my notmuch > database growing, and perhaps some fragmentation somewhere, this has > become *incredibly* annoying for me. I am checking email every 30 > minutes, and I'm nicing and

notmuch's idea of concurrency / failing an invocation

2011-01-27 Thread Thomas Schwinge
Hallo! Stepping away from the current code base -- what is notmuch's original idea of concurrency? That is, all of us probably know that one: A Xapian exception occurred opening database: Unable to get write lock on /home/thomas/Mail-schwinge.name-thomas/.notmuch/xapian: already

Re: notmuch's idea of concurrency / failing an invocation

2011-01-27 Thread micah anderson
On Thu, 27 Jan 2011 19:20:00 +0100, Thomas Schwinge tho...@schwinge.name wrote: Stepping away from the current code base -- what is notmuch's original idea of concurrency? That is, all of us probably know that one: A Xapian exception occurred opening database: Unable to get write

Re: notmuch's idea of concurrency / failing an invocation

2011-01-27 Thread Jameson Rollins
On Thu, 27 Jan 2011 13:40:25 -0500, micah anderson mi...@riseup.net wrote: Due to my harddisk in my laptop being slow (5400RPM), my notmuch database growing, and perhaps some fragmentation somewhere, this has become *incredibly* annoying for me. I am checking email every 30 minutes, and I'm