segfault with xapian 1.3.1

2013-09-05 Thread Jason A. Donenfeld
As discussed on IRC, notmuch segfaults with 1.3.1 but not with 1.2.5 on notmuch-new: Program received signal SIGSEGV, Segmentation fault. 0x775a5367 in Xapian::TermIterator::skip_to(std::string const&) () from /usr/lib64/libxapian-1.3.so.1 (gdb) bt #0 0x775a5367 in

gmail importer script

2012-12-16 Thread Jason A. Donenfeld
On Sat, Dec 15, 2012 at 11:41 AM, Patrick Totzke wrote: > > Well, thats not the point.. the script shouldn't die like this. > I think it's be better if the script caught that exception, deleted the > file > and continued.. Probably, but I suspect it's related to whatever mystery error you ran

Re: gmail importer script

2012-12-16 Thread Jason A. Donenfeld
On Sat, Dec 15, 2012 at 11:41 AM, Patrick Totzke patricktot...@gmail.comwrote: Well, thats not the point.. the script shouldn't die like this. I think it's be better if the script caught that exception, deleted the file and continued.. Probably, but I suspect it's related to whatever

gmail importer script

2012-12-15 Thread Jason A. Donenfeld
On Thu, Dec 13, 2012 at 5:48 PM, Patrick Totzke wrote: > > notmuch.errors.FileNotEmailError Delete the file it dies on and try again. -- next part -- An HTML attachment was scrubbed... URL:

Re: gmail importer script

2012-12-15 Thread Jason A. Donenfeld
On Thu, Dec 13, 2012 at 5:48 PM, Patrick Totzke patricktot...@gmail.comwrote: notmuch.errors.FileNotEmailError Delete the file it dies on and try again. ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch

Re: notmuch python bindings corrupt db index (was: gmail importer script)

2012-12-15 Thread Jason A. Donenfeld
On Sat, Dec 15, 2012 at 7:18 AM, Austin Clements amdra...@mit.edu wrote: need to at least os.fsync before the os.link. Fixed, thanks for the suggestion. http://git.zx2c4.com/gmail-notmuch/commit/?id=3f9646058bfd91d7d0e2eda035521f97de92eabc ___

gmail importer script

2012-12-13 Thread Jason A. Donenfeld
\AllMail is the unlocalized generic version that maps to the localized version, as are all the other back slash situations.

gmail importer script

2012-12-13 Thread Jason A. Donenfeld
On Thu, Dec 13, 2012 at 3:35 PM, fREW Schmidt wrote: > I just mean a general programming technique. As you get information > about tags you hold them in memory and then sync it later and clear > your memory. Oh, yea, that's a great idea. I'll play with that. Good thinking.

notmuch python bindings corrupt db index (was: gmail importer script)

2012-12-13 Thread Jason A. Donenfeld
On Wed, Dec 12, 2012 at 10:26 PM, David Bremner wrote: > > One thing that Olly Betts mentioned is that there is a recently fixed > bug in xapian with respect to multiple threads in the same process The script's actually single threaded.

notmuch python bindings corrupt db index (was: gmail importer script)

2012-12-13 Thread Jason A. Donenfeld
On Wed, Dec 12, 2012 at 9:49 PM, Austin Clements wrote: > There should be no way to corrupt the database at this level through > the Xapian API, which means nothing libnotmuch can do (much less users > of libnotmuch) should be able to corrupt the database. If you can > reproduce the problem,

gmail importer script

2012-12-13 Thread Jason A. Donenfeld
Please don't reply offlist. This is message #2 that's been sent offlist. On Thu, Dec 13, 2012 at 3:22 PM, fREW Schmidt > One thing you might want to try is to only open the notmuch index for > writing at a specified sync time. How do I do this? What's the python function? I couldn't find

Re: notmuch python bindings corrupt db index (was: gmail importer script)

2012-12-13 Thread Jason A. Donenfeld
On Wed, Dec 12, 2012 at 10:26 PM, David Bremner da...@tethera.net wrote: One thing that Olly Betts mentioned is that there is a recently fixed bug in xapian with respect to multiple threads in the same process The script's actually single threaded.

Re: gmail importer script

2012-12-13 Thread Jason A. Donenfeld
On Thu, Dec 13, 2012 at 3:35 PM, fREW Schmidt fri...@gmail.com wrote: I just mean a general programming technique. As you get information about tags you hold them in memory and then sync it later and clear your memory. Oh, yea, that's a great idea. I'll play with that. Good thinking.

Re: gmail importer script

2012-12-13 Thread Jason A. Donenfeld
\AllMail is the unlocalized generic version that maps to the localized version, as are all the other back slash situations. ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch

notmuch python bindings corrupt db index (was: gmail importer script)

2012-12-12 Thread Jason A. Donenfeld
On Tue, Dec 11, 2012 at 7:26 PM, Patrick Totzke wrote: > OK, i assumed i'd not have to download all messages because i already have > them > on disk -- synced via offlineimap. but then i remembered that OI stores uses > its own naming scheme for the files in a maildir. so thats probably why

Re: notmuch python bindings corrupt db index (was: gmail importer script)

2012-12-12 Thread Jason A. Donenfeld
On Tue, Dec 11, 2012 at 7:26 PM, Patrick Totzke patricktot...@gmail.com wrote: OK, i assumed i'd not have to download all messages because i already have them on disk -- synced via offlineimap. but then i remembered that OI stores uses its own naming scheme for the files in a maildir. so

notmuch python bindings corrupt db index (was: gmail importer script)

2012-12-11 Thread Jason A. Donenfeld
On Tue, Dec 11, 2012 at 11:04 AM, Patrick Totzke wrote: > I ended up killing the process after i saw that ETA was >3h. Gmail's bottleneck. Very unfortunate. Leave it going over night. > This broke my index! > > afterwards, a `notmuch new` printed > > A Xapian exception occurred creating a

gmail importer script

2012-12-11 Thread Jason A. Donenfeld
It seems to work, but i'm still waiting for the first run-through. > > Downloading messages: 457 of 22831| > > This takes ages. I hope it doesn't try to re-download all my messages > everytime. > Nope. It's very smart about not doing that. > > Another thing: > It seems that throughout its run,

gmail importer script

2012-12-11 Thread Jason A. Donenfeld
On Mon, Dec 10, 2012 at 6:46 PM, Jason A. Donenfeld wrote: > > > On Mon, Dec 10, 2012 at 10:41 AM, Patrick Totzke gmail.com>wrote: >> >> It would, but its nicer not to load ressources you're not gonna use. >> Also, progressbar 2.2 seems to be the newest version

Re: gmail importer script

2012-12-11 Thread Jason A. Donenfeld
It seems to work, but i'm still waiting for the first run-through. Downloading messages: 457 of 22831| This takes ages. I hope it doesn't try to re-download all my messages everytime. Nope. It's very smart about not doing that. Another thing: It seems that throughout its run, your

notmuch python bindings corrupt db index (was: gmail importer script)

2012-12-11 Thread Jason A. Donenfeld
On Tue, Dec 11, 2012 at 11:04 AM, Patrick Totzke patricktot...@gmail.com wrote: I ended up killing the process after i saw that ETA was 3h. Gmail's bottleneck. Very unfortunate. Leave it going over night. This broke my index! afterwards, a `notmuch new` printed A Xapian exception occurred

gmail importer script

2012-12-10 Thread Jason A. Donenfeld
On Mon, Dec 10, 2012 at 10:41 AM, Patrick Totzke wrote: > > It would, but its nicer not to load ressources you're not gonna use. Also, > progressbar 2.2 seems to be the newest version you see on a reasonably new > debian/ubuntu install.. > I'll consider it. > > > Next exception.. do you have a

gmail importer script

2012-12-10 Thread Jason A. Donenfeld
On Sat, Dec 8, 2012 at 6:46 PM, Patrick Totzke wrote: > > ZeroDivisionError: float division by zero > Fixed it! Sorry about all this... Jeeze Louise. > > Another feature request: > Could you make the progressbar stuff optinal? in the end one will want to > run this via cronjob > and this

Re: gmail importer script

2012-12-10 Thread Jason A. Donenfeld
On Mon, Dec 10, 2012 at 10:41 AM, Patrick Totzke patricktot...@gmail.comwrote: It would, but its nicer not to load ressources you're not gonna use. Also, progressbar 2.2 seems to be the newest version you see on a reasonably new debian/ubuntu install.. /nitpick I'll consider it. Next

Re: gmail importer script

2012-12-10 Thread Jason A. Donenfeld
On Mon, Dec 10, 2012 at 6:46 PM, Jason A. Donenfeld ja...@zx2c4.com wrote: On Mon, Dec 10, 2012 at 10:41 AM, Patrick Totzke patricktot...@gmail.comwrote: It would, but its nicer not to load ressources you're not gonna use. Also, progressbar 2.2 seems to be the newest version you see

Re: gmail importer script

2012-12-09 Thread Jason A. Donenfeld
On Sat, Dec 8, 2012 at 6:46 PM, Patrick Totzke patricktot...@gmail.comwrote: ZeroDivisionError: float division by zero Fixed it! Sorry about all this... Jeeze Louise. Another feature request: Could you make the progressbar stuff optinal? in the end one will want to run this via cronjob

gmail importer script

2012-12-08 Thread Jason A. Donenfeld
On Sat, Dec 8, 2012 at 6:04 PM, Jason A. Donenfeld wrote: >> BTW: I had to change the hardcoded "[Gmail]" to "[Google Mail]" for it to >> get that far. > > Really? Are you in the UK? I'll have to do this dynamically using the > XLIST imap extension.

gmail importer script

2012-12-08 Thread Jason A. Donenfeld
On Sat, Dec 8, 2012 at 5:20 PM, Patrick Totzke wrote: > I have two new errors: > > - > ./gmail-notmuch.py -u patricktotzke at gmail.com -p mypwd ~/mail/gmail/ > > I also tried with maildir param ~/mail/gmail/\[Google\ Mail\].All\ Mail > as

Tags with spaces

2012-12-08 Thread Jason A. Donenfeld
Very helpful! Thanks folks for the explanations. Another thing to note is that the dump format doesn't add any quotes to the tags, so this is something of an issue I suppose.

The Gmail Importer Script: Complete

2012-12-08 Thread Jason A. Donenfeld
Hi everyone, My script to import messages from gmail into notmuch, including tag information, is now polished and feature complete. ** Watch a video of it here: ** https://www.youtube.com/watch?v=e-8EHIAr7wA

gmail importer script

2012-12-08 Thread Jason A. Donenfeld
> I actually prefer this approach, but I think it would be more useful to > leave the syncing of the emails to a different program, and then just > managing the labels / tags with your tool (which is notmuch territory). So > the workflow would be: > So, implementing the mail sync in this script

Tags with spaces

2012-12-08 Thread Jason A. Donenfeld
Do we support them? Are we going to? This doesn't work: notmuch search -tag:"This has spaces" Nor does: notmuch search "-tag:This has spaces"

Re: gmail importer script

2012-12-08 Thread Jason A. Donenfeld
I actually prefer this approach, but I think it would be more useful to leave the syncing of the emails to a different program, and then just managing the labels / tags with your tool (which is notmuch territory). So the workflow would be: So, implementing the mail sync in this script would,

The Gmail Importer Script: Complete

2012-12-08 Thread Jason A. Donenfeld
Hi everyone, My script to import messages from gmail into notmuch, including tag information, is now polished and feature complete. ** Watch a video of it here: ** https://www.youtube.com/watch?v=e-8EHIAr7wA

Re: Tags with spaces

2012-12-08 Thread Jason A. Donenfeld
Very helpful! Thanks folks for the explanations. Another thing to note is that the dump format doesn't add any quotes to the tags, so this is something of an issue I suppose. ___ notmuch mailing list notmuch@notmuchmail.org

Re: gmail importer script

2012-12-08 Thread Jason A. Donenfeld
On Sat, Dec 8, 2012 at 5:20 PM, Patrick Totzke patricktot...@gmail.com wrote: I have two new errors: - ./gmail-notmuch.py -u patricktot...@gmail.com -p mypwd ~/mail/gmail/ I also tried with maildir param ~/mail/gmail/\[Google\

Re: gmail importer script

2012-12-08 Thread Jason A. Donenfeld
On Sat, Dec 8, 2012 at 6:04 PM, Jason A. Donenfeld ja...@zx2c4.com wrote: BTW: I had to change the hardcoded [Gmail] to [Google Mail] for it to get that far. Really? Are you in the UK? I'll have to do this dynamically using the XLIST imap extension. Okay, I've updated it to dynamically find

gmail importer script

2012-12-07 Thread Jason A. Donenfeld
On Fri, Dec 7, 2012 at 2:57 PM, Rainer M Krug wrote: > > > > > > > 2) I am downloading with the patched notmuch (to add x-keywords > containing the labels) only > > the "All Mail" folder - does your script use this information (if > present) to tag the emails? > > > > > > What is the patched

gmail importer script

2012-12-07 Thread Jason A. Donenfeld
On Fri, Dec 7, 2012 at 2:48 PM, Jason A. Donenfeld wrote: > > > > >> 3) are label changes in gmail synched into the local emails? >> > > It pulls labels from gmail messages into notmuch, with the labels in tact, > once per each email. > Okay, it's way m

gmail importer script

2012-12-07 Thread Jason A. Donenfeld
On Fri, Dec 7, 2012 at 10:23 AM, Patrick Totzke wrote: > Is this supposed to happen? > Obviously not. > > - > Logging in... > Selecting all mail... > Discovering local messages... > Receiving message list... > Traceback (most recent call last): >

gmail importer script

2012-12-07 Thread Jason A. Donenfeld
On Fri, Dec 7, 2012 at 9:53 AM, Rainer M Krug wrote: > > Sounds interesting. > > I don't know much python, so excuse my questions: > > 1) are multiple downloads of an email done, if the mail has multiple > labels? > No. > 2) I am downloading with the patched notmuch (to add x-keywords

gmail importer script

2012-12-07 Thread Jason A. Donenfeld
Hi folks, I wrote a script that imports emails and tags from gmail. It's resumable and appears to be working reasonably well. I could use some experienced eyes looking at it, as my first exposure to notmuch was just a few hours ago. http://git.zx2c4.com/gmail-notmuch/tree/gmail-notmuch.py

Re: gmail importer script

2012-12-07 Thread Jason A. Donenfeld
On Fri, Dec 7, 2012 at 9:53 AM, Rainer M Krug r.m.k...@gmail.com wrote: Sounds interesting. I don't know much python, so excuse my questions: 1) are multiple downloads of an email done, if the mail has multiple labels? No. 2) I am downloading with the patched notmuch (to add x-keywords

Re: gmail importer script

2012-12-07 Thread Jason A. Donenfeld
On Fri, Dec 7, 2012 at 2:48 PM, Jason A. Donenfeld ja...@zx2c4.com wrote: 3) are label changes in gmail synched into the local emails? It pulls labels from gmail messages into notmuch, with the labels in tact, once per each email. Okay, it's way more robust now. Just updated it. Now

Re: gmail importer script

2012-12-07 Thread Jason A. Donenfeld
On Fri, Dec 7, 2012 at 2:57 PM, Rainer M Krug r.m.k...@gmail.com wrote: 2) I am downloading with the patched notmuch (to add x-keywords containing the labels) only the All Mail folder - does your script use this information (if present) to tag the emails? What is the patched

Tags with spaces

2012-12-07 Thread Jason A. Donenfeld
Do we support them? Are we going to? This doesn't work: notmuch search -tag:This has spaces Nor does: notmuch search -tag:This has spaces ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch

gmail importer script

2012-12-06 Thread Jason A. Donenfeld
Hi folks, I wrote a script that imports emails and tags from gmail. It's resumable and appears to be working reasonably well. I could use some experienced eyes looking at it, as my first exposure to notmuch was just a few hours ago. http://git.zx2c4.com/gmail-notmuch/tree/gmail-notmuch.py