hi, had to set up offlineimap. and cyrus. and prayer webmail. and exim4-lmtp. and exim4 authenticated smtp proxying...
...because i'm currently operating in china and it's a bitch. constant HTTP and especially HTTPS related timeouts (SSL handshake errors mostly)... maan it's insane. so, hello, out of necessity i'll be patching offlineimap so that i can actually like... communicate effectively by email. i'm currently in HK and will be in Taiwan next, so have decent connectivity for a while before i return to Zhuhai, i hope to have everything i need set up over the next four to five weeks. nicolas, thank you for agreeing an exception to the policy of not communicating publicly whilst i was exploring how to get in touch (and to send patches), i'm fine for the next 4-5 weeks and also have sync'd up (250,000 messages!!) really quick using the HK airport hotel's WIFI. what would have taken 5 days @ 1 message per 2-5 seconds was instead 4 messages / second and completed overnight. whew :) i have two patches i'll send separately then i want to work on a third. first: adding in authproxy config option. turns out that china doesn't interfere with imap.gmail.com (yay!) but it *does* completely balls up accounts.google.com which makes it impossible to use OAUTH2. so, i *only* need a proxy for oauth2 *not* IMAP, which means adding in a separate option. second: retries. god is it insane to be operating at 6-20k/sec with 20 to 80% packet loss and ping times that vary between 385ms and a whopping 40000ms. over openvpn when using tcp i've actually seen ping times of 200 *seconds*. staggering. anyway, SSL handshakes timeout, SSL connections return EOF on a constant and regular basis, the TCP stack gets its knickers in a twist also on a regular basis... simplest thing to do: instead of defaulting to 2 retries and terminating, allow an arbitrary number of retries otherwise if you just put offlineimap into a while loop it just totally wastes CPU cycles reloading the sqlite database. third: with 250,000 messages in the inbox, offlineimap.py ends up using 100% CPU for a good couple of minutes JUST loading the sqlite database. i've looked at the sqlite code: there's only one table, it's effectively a key-value store... i'm a huge fan of lmdb and especially python-lmdb... so i'll be adding that as an option. i can't possibly tolerate this laptop going into 100% CPU usage for up to 100 seconds at a time. lmdb is aawesome, i studied it whilst writing up the wikipedia page, and used it for a real-time network traffic parsing system to handle tens of thousands of packets per second... in python of all languages. anyway, thank you nicolas for explaining (in private email) about the DCO http://www.offlineimap.org/doc/dco.html i don't know if you need me to write explicitly here that i agree to it, but yes please take this as indication that i do indeed agree to it and certify that all contributions conform to the DCO. i'll be using git commit -s in future. ok patches to follow for 1 and 2 thanks to everyone who's contributed, for creating some extremely necessary software. l. -- -- -- http://crowdsupply.com/eoma68 eco-conscious computing _______________________________________________ OfflineIMAP-project mailing list: [email protected] http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/offlineimap-project OfflineIMAP homepages: - https://github.com/OfflineIMAP - http://offlineimap.org
