Re: gmail importer script
BTW: I removed the whole maildir contents and restarted the downlowd over night. here's the result: --- Traceback (most recent call last):2| | 26% ETA: 1:37:22 3.11 emails/s File ./gmail-notmuch.py, line 253, in module main() File ./gmail-notmuch.py, line 89, in main download_new_messages(imap, database, new_messages, destination) File ./gmail-notmuch.py, line 210, in download_new_messages tag_message(database, dest, labels) File ./gmail-notmuch.py, line 168, in tag_message raise e notmuch.errors.FileNotEmailError --- /p signature.asc Description: signature ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
Re: gmail importer script
On Sat, Dec 15, 2012 at 11:41 AM, Patrick Totzke patricktot...@gmail.comwrote: Well, thats not the point.. the script shouldn't die like this. I think it's be better if the script caught that exception, deleted the file and continued.. Probably, but I suspect it's related to whatever mystery error you ran into before with the corruption. Does deleting that file and trying again fix it? Anyway, this is extremely stable for me and a few others at this point. I'm going to wait for other users to report errors. Alternatively, send me patches if you want things to happen. ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
Re: gmail importer script
Quoting Jason A. Donenfeld (2012-12-16 20:44:04) On Sat, Dec 15, 2012 at 11:41 AM, Patrick Totzke patricktot...@gmail.com wrote: Well, thats not the point.. the script shouldn't die like this. I think it's be better if the script caught that exception, deleted the file and continued.. Probably, but I suspect it's related to whatever mystery error you ran into before with the corruption. Does deleting that file and trying again fix it? I don't know. How would I know which file to remove? I just removed the newest files in new/cur/tmp but this doesnt solve the issue. What would really help here is some more informative error message! I tried with the `-d` option but did not see any local path in the output. Anyway, this is extremely stable for me and a few others at this point. I'm going to wait for other users to report errors. Alternatively, send me patches if you want things to happen. Personally, I don't really care if 'things are happening', as I'm kind of OK with my current offlineimap configuration and would seriously consider switching from OI only if it meant an improvement in speed or stability. I test this code because I believe that the notmuch ecosystem would benefit from a working solution and that reviews and bug-reports are important. Of course, I don't expect any miracles in terms of speed at this early stage. But the bug I reported above should be addressed at some point in order not to scare off potential users. Have you considered including your code in offlineimap? I'm asking because OI already solved some of the problems you might face later on if you want to continue working on this. * multi-threaded downloads * the ability to read passwords not as plaintext parameter.. best, /p signature.asc Description: signature ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
Re: gmail importer script
On Thu, Dec 13, 2012 at 5:48 PM, Patrick Totzke patricktot...@gmail.comwrote: notmuch.errors.FileNotEmailError Delete the file it dies on and try again. ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
Re: gmail importer script
Quoting Jason A. Donenfeld (2012-12-15 08:22:46) On Thu, Dec 13, 2012 at 5:48 PM, Patrick Totzke patricktot...@gmail.com wrote: notmuch.errors.FileNotEmailError Delete the file it dies on and try again. Well, thats not the point.. the script shouldn't die like this. I think it's be better if the script caught that exception, deleted the file and continued.. Thanks, /p signature.asc Description: signature ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
Re: gmail importer script
On Thu, Dec 13, 2012 at 03:30:19PM +0100, Jason A. Donenfeld wrote: Please don't reply offlist. This is message #2 that's been sent offlist. Oh I'm sorry, that was an accident. On Thu, Dec 13, 2012 at 3:22 PM, fREW Schmidt fri...@gmail.com One thing you might want to try is to only open the notmuch index for writing at a specified sync time. How do I do this? What's the python function? I couldn't find anything. Would be interested in doing this though if it exists. I just mean a general programming technique. As you get information about tags you hold them in memory and then sync it later and clear your memory. Also I was going to say you might want to catch SIGINT like The library already does this. -- fREW Schmidt http://blog.afoolishmanifesto.com pgpYfiABySGYw.pgp Description: PGP signature ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
Re: gmail importer script
On Thu, Dec 13, 2012 at 3:35 PM, fREW Schmidt fri...@gmail.com wrote: I just mean a general programming technique. As you get information about tags you hold them in memory and then sync it later and clear your memory. Oh, yea, that's a great idea. I'll play with that. Good thinking. ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
Re: gmail importer script
On Thu, Dec 13, 2012 at 03:28:30PM +0100, Jason A. Donenfeld wrote: On Thu, Dec 13, 2012 at 3:19 PM, fREW Schmidt fri...@gmail.com wrote: Okay, I've updated it to dynamically find the All Mail folder. I can't seem to find it now, but I read that Gmail actually localizes the name of various folders, so I think All Mail will actually be called something else for non-english speakers. It might be a good idea for you to at least make it a config option. A) Why are you replying off list? B) Did you totally miss my message directly above yours where I wrote Okay, I've updated it to dynamically find the All Mail folder? Right, I did see that and I did look at the code; all I'm saying is that Gmail may localize AllMail as I'm fairly sure they localize the other builtin labels (http://git.zx2c4.com/gmail-notmuch/tree/gmail-notmuch.py#n214) I could certainly be wrong though, I haven't looked at this at such a low level before. -- fREW Schmidt http://blog.afoolishmanifesto.com pgpqDMiLkFVHO.pgp Description: PGP signature ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
Re: gmail importer script
\AllMail is the unlocalized generic version that maps to the localized version, as are all the other back slash situations. ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
Re: gmail importer script
Sorry, I'm misusing this thread as a bugtracker.. Traceback (most recent call last):4| | 10% ETA: 1:24:41 3.54 emails/s File ./gmail-notmuch.py, line 251, in module main() File ./gmail-notmuch.py, line 89, in main download_new_messages(imap, database, new_messages, destination) File ./gmail-notmuch.py, line 208, in download_new_messages tag_message(database, dest, labels) File ./gmail-notmuch.py, line 168, in tag_message raise e notmuch.errors.FileNotEmailError best, /p ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
Re: gmail importer script
Quoting Jason A. Donenfeld (2012-12-11 07:06:22) On Mon, Dec 10, 2012 at 6:46 PM, Jason A. Donenfeld ja...@zx2c4.com wrote: On Mon, Dec 10, 2012 at 10:41 AM, Patrick Totzke patricktot...@gmail.com wrote: It would, but its nicer not to load ressources you're not gonna use. Also, progressbar 2.2 seems to be the newest version you see on a reasonably new debian/ubuntu install.. /nitpick I'll consider it. --silent or -s now silences stdout. It seems to work, but i'm still waiting for the first run-through. Downloading messages: 457 of 22831| This takes ages. I hope it doesn't try to re-download all my messages everytime. Another thing: It seems that throughout its run, your script locks the notmuch database. This is particularly annoying if you want to read/tag mails while waiting for a long sync-process. I think you only really want to lock the index for a short time in the end. Do you write to the index directly after a msg is downloaded? /p ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
Re: gmail importer script
It seems to work, but i'm still waiting for the first run-through. Downloading messages: 457 of 22831| This takes ages. I hope it doesn't try to re-download all my messages everytime. Nope. It's very smart about not doing that. Another thing: It seems that throughout its run, your script locks the notmuch database. This is particularly annoying if you want to read/tag mails while waiting for a long sync-process. I think you only really want to lock the index for a short time in the end. Do you write to the index directly after a msg is downloaded? I tried to solve this by closing and opening the database each time, but it actually reduced the performance considerably -- even slower than the gmail download. ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
Re: gmail importer script
Quoting Jason A. Donenfeld (2012-12-11 09:12:56) It seems to work, but i'm still waiting for the first run-through. Downloading messages: 457 of 22831| This takes ages. I hope it doesn't try to re-download all my messages everytime. Nope. It's very smart about not doing that. hmmk I ended up killing the process after i saw that ETA was 3h. This broke my index! afterwards, a `notmuch new` printed A Xapian exception occurred creating a directory: Expected block 485 to be level 1, not 0. I wasnt able to properly read the index anymore and had to restore an old index dump. not cool! /p ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
Re: gmail importer script
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 08/12/12 14:15, Jason A. Donenfeld wrote: I actually prefer this approach, but I think it would be more useful to leave the syncing of the emails to a different program, and then just managing the labels / tags with your tool (which is notmuch territory). So the workflow would be: So, implementing the mail sync in this script would, as I see it, kind of reinventing the wheel. You're misunderstanding me. Let me make it very clear what the script does. Actually why don't you just read the source code? Please? Well, Because I have never used python, and you know what your script is doing. I prefer to know what the script is doing, before I try it out and it changes up my local emails and my notmuch databases in unexpected ways, because I did not understand the source code properly? anyway, here: - It looks in gmail and asks it which messages gmail has OK. - It downloads those messages Here is my question - see below. - It applies gmail's labels to those messages as tags Perfect. Now my final question: after reading your info which is useful, I looked in more detail at the source code and it made now sense to me. My question is: If I only want to aply the tags and *not download* emails, can I simply remove the line 88: download_new_messages(imap, database, new_messages, destination) Is this correct? In this case, the script would ony tag y messages which are already downloaded? If this is the case, I would immediately start using the script. Rainer The end. Along the way it has some smart things to reduce redundant downloads. For more information, consult the source code. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://www.enigmail.net/ iEYEARECAAYFAlDFp5kACgkQoYgNqgF2egqfVACfWckdCl3SOmue6EEHF1cv6d44 IKgAmwV9cQH/VxCmPHmmU6z3akQ9pFZV =vZxS -END PGP SIGNATURE- ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
Re: gmail importer script
Quoting Jason A. Donenfeld (2012-12-09 23:13:12) On Sat, Dec 8, 2012 at 6:46 PM, Patrick Totzke patricktot...@gmail.com wrote: ZeroDivisionError: float division by zero Fixed it! Sorry about all this... Jeeze Louise. thx Another feature request: Could you make the progressbar stuff optinal? in the end one will want to run this via cronjob and this shouldn't print anything to stdout. Also, I can pretty much recommend the 'logging' module. Sure, but... /dev/null doesn't cut it? I can add some more refined logging capabilities if you want. It would, but its nicer not to load ressources you're not gonna use. Also, progressbar 2.2 seems to be the newest version you see on a reasonably new debian/ubuntu install.. /nitpick Next exception.. do you have a bugtracker? (/me 3 github) Logging in... Selecting all mail... Receiving message list: 24455 of 24455||100% Time: 0:00:12 2.04 kemails/s Parsing message list and labels... Searching for local messages... Traceback (most recent call last): File ./gmail-notmuch.py, line 248, in module main() File ./gmail-notmuch.py, line 75, in main if len(new_messages) == 0: TypeError: object of type 'NoneType' has no len() /p ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
Re: gmail importer script
On Mon, Dec 10, 2012 at 10:41 AM, Patrick Totzke patricktot...@gmail.comwrote: It would, but its nicer not to load ressources you're not gonna use. Also, progressbar 2.2 seems to be the newest version you see on a reasonably new debian/ubuntu install.. /nitpick I'll consider it. Next exception.. do you have a bugtracker? (/me 3 github) To hell with github. Sometimes I do use a bugtracker though. In light of me being an idiot, maybe I should get one for this script. But this script is so short, and these errors are really due to me being an idiot. Sit tight a little longer and things will begin to work. Sorry for the prior impartial fixes. Once this baby is running you will be quite delighted. TypeError: object of type 'NoneType' has no len() Fixed. http://git.zx2c4.com/gmail-notmuch/commit/?id=3bf8ce9698ec798a42bc2f92f596243fa8911bf8 ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
Re: gmail importer script
On 7 December 2012 16:32, Jason A. Donenfeld ja...@zx2c4.com wrote: On Fri, Dec 7, 2012 at 2:57 PM, Rainer M Krug r.m.k...@gmail.com wrote: 2) I am downloading with the patched notmuch (to add x-keywords containing the labels) only the All Mail folder - does your script use this information (if present) to tag the emails? What is the patched notmuch? What does that do? Sorry - should have been patched offlineimap ( https://github.com/aroig/offlineimap http://thread.gmane.org/gmane.mail.imap.offlineimap.general/5943/focus=5970and http://article.gmane.org/gmane.mail.imap.offlineimap.general/5970 ) It is synching the gmail labels into the x-keywords header as a comma separated list. These could then be used by notmuch to be added to the labels. It also does sync changes from the X-Keywords to gmail (see http://thread.gmane.org/gmane.mail.imap.offlineimap.general/5943/focus=5970for details) I see. I use the All Mail folder and get the tags using the X-GM-LABELS imap extension. I don't muck with the mail headers in the process, fortunately. I actually prefer this approach, but I think it would be more useful to leave the syncing of the emails to a different program, and then just managing the labels / tags with your tool (which is notmuch territory). So the workflow would be: Initial or download (only remote changes): 1) Download email (All Mail folder) with whatever you are using (offlinemail, ...) 2) call your script to index the downloaded emails (All Mail folder) with notmuch and get the labels and set / remove them in notmuch accordingly. Upload (only local changes): 1) upload emails using whatever (offlineimap, ...) 2) upload the changed tags and create the needed new labels (and possibly delete empty ons?) The tricky part will be the synching when changes on both sides were observed. The mail sync is implemented in e.g. offlineimap and works reilably. But I don't know how one could handle tag / label changes on both sides. So, implementing the mail sync in this script would, as I see it, kind of reinventing the wheel. One can leave it in, but I would like to have an option to leave the synching of the emails to a separate program. Cheers, Rainer -- NEW GERMAN FAX NUMBER!!! Rainer M. Krug, PhD (Conservation Ecology, SUN), MSc (Conservation Biology, UCT), Dipl. Phys. (Germany) Centre of Excellence for Invasion Biology Natural Sciences Building Office Suite 2039 Stellenbosch University Main Campus, Merriman Avenue Stellenbosch South Africa Cell: +27 - (0)83 9479 042 Fax:+27 - (0)86 516 2782 Fax:+49 - (0)321 2125 2244 email: rai...@krugs.de Skype: RMkrug Google: r.m.k...@gmail.com ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
Re: gmail importer script
On Mon, Dec 10, 2012 at 6:46 PM, Jason A. Donenfeld ja...@zx2c4.com wrote: On Mon, Dec 10, 2012 at 10:41 AM, Patrick Totzke patricktot...@gmail.comwrote: It would, but its nicer not to load ressources you're not gonna use. Also, progressbar 2.2 seems to be the newest version you see on a reasonably new debian/ubuntu install.. /nitpick I'll consider it. --silent or -s now silences stdout. ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
Re: gmail importer script
On Sat, Dec 8, 2012 at 6:46 PM, Patrick Totzke patricktot...@gmail.comwrote: ZeroDivisionError: float division by zero Fixed it! Sorry about all this... Jeeze Louise. Another feature request: Could you make the progressbar stuff optinal? in the end one will want to run this via cronjob and this shouldn't print anything to stdout. Also, I can pretty much recommend the 'logging' module. Sure, but... /dev/null doesn't cut it? I can add some more refined logging capabilities if you want. cheers, /p ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
Re: gmail importer script
I actually prefer this approach, but I think it would be more useful to leave the syncing of the emails to a different program, and then just managing the labels / tags with your tool (which is notmuch territory). So the workflow would be: So, implementing the mail sync in this script would, as I see it, kind of reinventing the wheel. You're misunderstanding me. Let me make it very clear what the script does. Actually why don't you just read the source code? Please? Well, anyway, here: - It looks in gmail and asks it which messages gmail has - It downloads those messages - It applies gmail's labels to those messages as tags The end. Along the way it has some smart things to reduce redundant downloads. For more information, consult the source code. ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
Re: gmail importer script
Quoting Jason A. Donenfeld (2012-12-07 13:49:46) Not sure what is causing this. My best guess is that your password was incorrect and that I'm not checking the login return value. Yes, you're right, it was an incorrect passwd. One thing you also might want to watch out for is exceptions due to a locked notmuch index. How do I do this? See e.g. here: https://github.com/pazz/alot/blob/master/alot/db/manager.py#L163 I'm sure afew (https://github.com/teythoon/afew) does a similar thing. I have two new errors: - ./gmail-notmuch.py -u patricktot...@gmail.com -p mypwd ~/mail/gmail/ Logging in... Selecting all mail... Error opening database at /home/pazz/mail/gmail/.notmuch: No such file or directory Traceback (most recent call last): File ./gmail-notmuch.py, line 225, in module main() File ./gmail-notmuch.py, line 56, in main database = notmuch.Database(destination, False, notmuch.Database.MODE.READ_WRITE) File /home/pazz/.local/lib/python2.7/site-packages/notmuch/database.py, line 154, in __init__ self.open(path, mode) File /home/pazz/.local/lib/python2.7/site-packages/notmuch/database.py, line 214, in open raise NotmuchError(status) notmuch.errors.FileError - I also tried with maildir param ~/mail/gmail/\[Google\ Mail\].All\ Mail as this is where my all-mail maildir is, and i also tried an absolute path, all the same result. If I point it to the root of my notmuch directory I get: - [~/projects/gmail-notmuch] ./gmail-notmuch.py -u patricktot...@gmail.com -p mypwd /home/pazz/mail Logging in... Selecting all mail... Traceback (most recent call last): File ./gmail-notmuch.py, line 225, in module main() File ./gmail-notmuch.py, line 58, in main messages = discover_messages(imap, total) File ./gmail-notmuch.py, line 104, in discover_messages new_readline.progressbar = create_progressbar(Receiving message list, total).start() File ./gmail-notmuch.py, line 147, in create_progressbar return ProgressBar(maxval=total, widgets=[text + : , SimpleProgress(), Bar(), Percentage(), , ETA(), , FileTransferSpeed(unit=emails)]) TypeError: __init__() got an unexpected keyword argument 'unit' BTW: I had to change the hardcoded [Gmail] to [Google Mail] for it to get that far. HTH, /p signature.asc Description: signature ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
Re: gmail importer script
On Sat, Dec 8, 2012 at 5:20 PM, Patrick Totzke patricktot...@gmail.com wrote: I have two new errors: - ./gmail-notmuch.py -u patricktot...@gmail.com -p mypwd ~/mail/gmail/ I also tried with maildir param ~/mail/gmail/\[Google\ Mail\].All\ Mail as this is where my all-mail maildir is, and i also tried an absolute path, all the same result. If I point it to the root of my notmuch directory I get: Pointing it at the root directory is what you want. In fact, I'm removing the argument from the script and having it just get that from the config. So the correct invocation will be: ./gmail-notmuch.py -u blabla -p password File ./gmail-notmuch.py, line 147, in create_progressbar return ProgressBar(maxval=total, widgets=[text + : , SimpleProgress(), Bar(), Percentage(), , ETA(), , FileTransferSpeed(unit=emails)]) TypeError: __init__() got an unexpected keyword argument 'unit' You're probably running with an old version of python-statusbar. BTW: I had to change the hardcoded [Gmail] to [Google Mail] for it to get that far. Really? Are you in the UK? I'll have to do this dynamically using the XLIST imap extension. ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
Re: gmail importer script
On Sat, Dec 8, 2012 at 6:04 PM, Jason A. Donenfeld ja...@zx2c4.com wrote: BTW: I had to change the hardcoded [Gmail] to [Google Mail] for it to get that far. Really? Are you in the UK? I'll have to do this dynamically using the XLIST imap extension. Okay, I've updated it to dynamically find the All Mail folder. ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
Re: gmail importer script
Quoting Jason A. Donenfeld (2012-12-08 17:04:48) On Sat, Dec 8, 2012 at 5:20 PM, Patrick Totzke patricktot...@gmail.com wrote: I have two new errors: - ./gmail-notmuch.py -u patricktot...@gmail.com -p mypwd ~/mail/gmail/ I also tried with maildir param ~/mail/gmail/\[Google\ Mail\].All\ Mail as this is where my all-mail maildir is, and i also tried an absolute path, all the same result. If I point it to the root of my notmuch directory I get: Pointing it at the root directory is what you want. In fact, I'm removing the argument from the script and having it just get that from the config. So the correct invocation will be: ./gmail-notmuch.py -u blabla -p password Okay.. File ./gmail-notmuch.py, line 147, in create_progressbar return ProgressBar(maxval=total, widgets=[text + : , SimpleProgress(), Bar(), Percentage(), , ETA(), , FileTransferSpeed(unit=emails)]) TypeError: __init__() got an unexpected keyword argument 'unit' You're probably running with an old version of python-statusbar. I just installed it from ubuntu 12.04's repository. the version was 2.2-2. Installed from pip now, and it seems ok. you'll want to make more specific dependencies at some point.. BTW: I had to change the hardcoded [Gmail] to [Google Mail] for it to get that far. Really? Are you in the UK? I'll have to do this dynamically using the XLIST imap extension. Yea, but its a *really* old account I first got from google.de. Reading this dynamically would def. help. next try.. - Traceback (most recent call last): File ./gmail-notmuch.py, line 246, in module main() File ./gmail-notmuch.py, line 74, in main new_messages = retag_old_messages(database, messages, destination) File ./gmail-notmuch.py, line 176, in retag_old_messages progressbar.start() File /usr/local/lib/python2.7/dist-packages/progressbar/__init__.py, line 311, in start self.update(0) File /usr/local/lib/python2.7/dist-packages/progressbar/__init__.py, line 283, in update self.fd.write(self._format_line() + '\r') File /usr/local/lib/python2.7/dist-packages/progressbar/__init__.py, line 243, in _format_line widgets = ''.join(self._format_widgets()) File /usr/local/lib/python2.7/dist-packages/progressbar/__init__.py, line 223, in _format_widgets widget = format_updatable(widget, self) File /usr/local/lib/python2.7/dist-packages/progressbar/widgets.py, line 38, in format_updatable if hasattr(updatable, 'update'): return updatable.update(pbar) File /usr/local/lib/python2.7/dist-packages/progressbar/widgets.py, line 184, in update return '%3d%%' % pbar.percentage() File /usr/local/lib/python2.7/dist-packages/progressbar/__init__.py, line 208, in percentage return self.currval * 100.0 / self.maxval ZeroDivisionError: float division by zero -- Another feature request: Could you make the progressbar stuff optinal? in the end one will want to run this via cronjob and this shouldn't print anything to stdout. Also, I can pretty much recommend the 'logging' module. cheers, /p signature.asc Description: signature ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
Re: gmail importer script
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 07/12/12 05:05, Kushal Kumaran wrote: Jason A. Donenfeld ja...@zx2c4.com writes: Hi folks, I wrote a script that imports emails and tags from gmail. It's resumable and appears to be working reasonably well. I could use some experienced eyes looking at it, as my first exposure to notmuch was just a few hours ago. http://git.zx2c4.com/gmail-notmuch/tree/gmail-notmuch.py Not specific to notmuch, but not every gmail user has a @gmail.com or @googlemail.com email address. Google Apps users can have arbitrary email addresses, and still use gmail's imap and web interface. I finished syncing my gmail locally (using offlineimap), and creating tags by hand not one day ago, or I would certainly have tried out this script. Sounds interesting. I don't know much python, so excuse my questions: 1) are multiple downloads of an email done, if the mail has multiple labels? 2) I am downloading with the patched notmuch (to add x-keywords containing the labels) only the All Mail folder - does your script use this information (if present) to tag the emails? 3) are label changes in gmail synched into the local emails? 4) I assume, local changes are not synced to gmail? 5) how are conflicts handled (local tag changes versus label changes on gmail)? Thanks, Rainer -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://www.enigmail.net/ iEYEARECAAYFAlDBrqAACgkQoYgNqgF2egqrEwCeOSr5TGZgAIaRvzHCl1Nv+DBk ko4AnRT0FMBwYNRGzSclXxZXzaH0Q23x =Sx0B -END PGP SIGNATURE- ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
Re: gmail importer script
Is this supposed to happen? - Logging in... Selecting all mail... Discovering local messages... Receiving message list... Traceback (most recent call last): File ./gmail-notmuch.py, line 175, in module main() File ./gmail-notmuch.py, line 58, in main new_messages = discover_new_messages(imap, old_messages) File ./gmail-notmuch.py, line 80, in discover_new_messages typ, data = imap.fetch(1:*, X-GM-MSGID) File /usr/lib/python2.7/imaplib.py, line 443, in fetch typ, dat = self._simple_command(name, message_set, message_parts) File /usr/lib/python2.7/imaplib.py, line 1070, in _simple_command return self._command_complete(name, self._command(name, *args)) File /usr/lib/python2.7/imaplib.py, line 825, in _command ', '.join(Commands[name]))) imaplib.error: command FETCH illegal in state AUTH, only allowed in states SELECTED -- In [2]: imaplib.__version__ Out[2]: '2.58' One thing you also might want to watch out for is exceptions due to a locked notmuch index. rock on, /p ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
Re: gmail importer script
On Fri, Dec 7, 2012 at 9:53 AM, Rainer M Krug r.m.k...@gmail.com wrote: Sounds interesting. I don't know much python, so excuse my questions: 1) are multiple downloads of an email done, if the mail has multiple labels? No. 2) I am downloading with the patched notmuch (to add x-keywords containing the labels) only the All Mail folder - does your script use this information (if present) to tag the emails? What is the patched notmuch? What does that do? 3) are label changes in gmail synched into the local emails? It pulls labels from gmail messages into notmuch, with the labels in tact, once per each email. 4) I assume, local changes are not synced to gmail? Correctamundo. 5) how are conflicts handled (local tag changes versus label changes on gmail)? They aren't. See 3. Thanks, Rainer -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://www.enigmail.net/ iEYEARECAAYFAlDBrqAACgkQoYgNqgF2egqrEwCeOSr5TGZgAIaRvzHCl1Nv+DBk ko4AnRT0FMBwYNRGzSclXxZXzaH0Q23x =Sx0B -END PGP SIGNATURE- ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
Re: gmail importer script
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 07/12/12 14:48, Jason A. Donenfeld wrote: On Fri, Dec 7, 2012 at 9:53 AM, Rainer M Krug r.m.k...@gmail.com mailto:r.m.k...@gmail.com wrote: Sounds interesting. I don't know much python, so excuse my questions: 1) are multiple downloads of an email done, if the mail has multiple labels? No. Good. 2) I am downloading with the patched notmuch (to add x-keywords containing the labels) only the All Mail folder - does your script use this information (if present) to tag the emails? What is the patched notmuch? What does that do? Sorry - should have been patched offlineimap ( https://github.com/aroig/offlineimap http://thread.gmane.org/gmane.mail.imap.offlineimap.general/5943/focus=5970 and http://article.gmane.org/gmane.mail.imap.offlineimap.general/5970 ) It is synching the gmail labels into the x-keywords header as a comma separated list. These could then be used by notmuch to be added to the labels. It also does sync changes from the X-Keywords to gmail (see http://thread.gmane.org/gmane.mail.imap.offlineimap.general/5943/focus=5970 for details) 3) are label changes in gmail synched into the local emails? It pulls labels from gmail messages into notmuch, with the labels in tact, once per each email. Good. 4) I assume, local changes are not synced to gmail? Correctamundo. OK. 5) how are conflicts handled (local tag changes versus label changes on gmail)? They aren't. See 3. Thanks, Rainer Thanks, Rainer -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://www.enigmail.net/ iEYEARECAAYFAlDBrqAACgkQoYgNqgF2egqrEwCeOSr5TGZgAIaRvzHCl1Nv+DBk ko4AnRT0FMBwYNRGzSclXxZXzaH0Q23x =Sx0B -END PGP SIGNATURE- ___ notmuch mailing list notmuch@notmuchmail.org mailto:notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch - -- Rainer M. Krug, PhD (Conservation Ecology, SUN), MSc (Conservation Biology, UCT), Dipl. Phys. (Germany) Centre of Excellence for Invasion Biology Stellenbosch University South Africa Tel : +33 - (0)9 53 10 27 44 Cell: +33 - (0)6 85 62 59 98 Fax : +33 - (0)9 58 10 27 44 Fax (D):+49 - (0)3 21 21 25 22 44 email: rai...@krugs.de Skype: RMkrug -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://www.enigmail.net/ iEYEARECAAYFAlDB9cMACgkQoYgNqgF2egrEkACeLD9p4CUW3EhliFQ44cigpZdF o8AAnA9M8tmw52W3m62QFJaFuoOLdto2 =6Zrk -END PGP SIGNATURE- ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
Re: gmail importer script
On Fri, Dec 7, 2012 at 2:48 PM, Jason A. Donenfeld ja...@zx2c4.com wrote: 3) are label changes in gmail synched into the local emails? It pulls labels from gmail messages into notmuch, with the labels in tact, once per each email. Okay, it's way more robust now. Just updated it. Now it will pull gmail labels to notmuch, each time, every time. So you can run this thing over and over, and it will always push gmail into notmuch. It doesn't yet sync the the other way yet, but at least this is now fairly robust. ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
Re: gmail importer script
On Fri, Dec 7, 2012 at 2:57 PM, Rainer M Krug r.m.k...@gmail.com wrote: 2) I am downloading with the patched notmuch (to add x-keywords containing the labels) only the All Mail folder - does your script use this information (if present) to tag the emails? What is the patched notmuch? What does that do? Sorry - should have been patched offlineimap ( https://github.com/aroig/offlineimap http://thread.gmane.org/gmane.mail.imap.offlineimap.general/5943/focus=5970and http://article.gmane.org/gmane.mail.imap.offlineimap.general/5970 ) It is synching the gmail labels into the x-keywords header as a comma separated list. These could then be used by notmuch to be added to the labels. It also does sync changes from the X-Keywords to gmail (see http://thread.gmane.org/gmane.mail.imap.offlineimap.general/5943/focus=5970for details) I see. I use the All Mail folder and get the tags using the X-GM-LABELS imap extension. I don't muck with the mail headers in the process, fortunately. ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
Re: gmail importer script
Jason A. Donenfeld ja...@zx2c4.com writes: Hi folks, I wrote a script that imports emails and tags from gmail. It's resumable and appears to be working reasonably well. I could use some experienced eyes looking at it, as my first exposure to notmuch was just a few hours ago. http://git.zx2c4.com/gmail-notmuch/tree/gmail-notmuch.py Not specific to notmuch, but not every gmail user has a @gmail.com or @googlemail.com email address. Google Apps users can have arbitrary email addresses, and still use gmail's imap and web interface. I finished syncing my gmail locally (using offlineimap), and creating tags by hand not one day ago, or I would certainly have tried out this script. -- regards, kushal ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
