Re: [sup-devel] Adding backward synchronization for Maildir sources

2011-10-27 Thread Justin Mazzi
Damien Leone  fensalir.fr> writes:

> 
> Sup guys,
> 
> Sorry for having been so long.
> 
> What's new:
> 
> - I added an option in sup-sync-back-maildir to merge the :replied and
> :forwarded labels/flags so the first time you run the script you
> should not lose any information on the IMAP side;
> 
> - A way to detect if sup-sync-back-to-maildir should be executed in
> order to synchronize your index back to the Maildir source in order to
> prevent any loss of information. A magic file is written in the
> configuration directory everytime the script is executed and it is
> removed when the sync_back_to_maildir option is set from false to
> true;
> 
> - I fixed a case when messages were not properly updated when they
> were remotely modified.
> 
> I have pushed the last commits to the maildir-sync branch [0], it is
> rebased from master. I also have a branch rebased from next on my own
> repository if you'd rather a more up to date branch [1].
> 
> Please test and review! For more informations and a howto, read the
> message quoted below this mail.
> 
> [0] http://gitorious.org/sup/mainline/commits/maildir-sync
> [1] http://git.fensalir.fr/?p=dleone/sup.git

Hey,

The syncing is working pretty well for me except for one thing (with Gmail). 

When archiving a message in sup it doesn't remove the Inbox label 
from the message. Removing the Inbox label from the message is 
what makes the email disappear from other clients, like a 
smartphone.

- Justin Mazzi



___
Sup-devel mailing list
Sup-devel@rubyforge.org
http://rubyforge.org/mailman/listinfo/sup-devel


Re: [sup-devel] Adding backward synchronization for Maildir sources

2011-03-13 Thread Damien Leone
Sup guys,

Sorry for having been so long.

What's new:

- I added an option in sup-sync-back-maildir to merge the :replied and
:forwarded labels/flags so the first time you run the script you
should not lose any information on the IMAP side;

- A way to detect if sup-sync-back-to-maildir should be executed in
order to synchronize your index back to the Maildir source in order to
prevent any loss of information. A magic file is written in the
configuration directory everytime the script is executed and it is
removed when the sync_back_to_maildir option is set from false to
true;

- I fixed a case when messages were not properly updated when they
were remotely modified.

I have pushed the last commits to the maildir-sync branch [0], it is
rebased from master. I also have a branch rebased from next on my own
repository if you'd rather a more up to date branch [1].

Please test and review! For more informations and a howto, read the
message quoted below this mail.

[0] http://gitorious.org/sup/mainline/commits/maildir-sync
[1] http://git.fensalir.fr/?p=dleone/sup.git

Excerpts from Damien Leone's message of ven. juil. 09 18:44:01 +0200 2010:
> Sup guys,
> 
> I have been working on backward synchronization for Maildir sources
> last week and I need your help to test and review the code.
> 
> I pushed my commits (based on the 'next' branch) to my repo which you
> can find here [0] for the web interface and here [1] for the git
> address.
> 
> The following is related to Maildir sources ONLY.
> 
> So what does work and what does not (if you don't want to read this,
> scroll to the quick howto below):
> 
> - Sup will now stay synchronized with your remote sources by detecting
> if a message has been remotely updated (ie: you change a flag from
> another client), these updates will be applied to your local index and
> your thread-view-mode buffers should be properly refreshed;
> 
> - When a message is remotely deleted, it should now disappear from
> your index;
> 
> - There is a new configuration option called "sync_back_to_maildir"
> which is false by default. When true, this option updates your actual
> Maildir files when you change a label, this is done in real time
> before being saved to xapian. So for instance if you use offlineimap
> the changes should be synched back to your IMAP server by the next
> poll in Sup (assuming that your before-poll hook runs offlineimap);
> 
> - I wrote a "sup-sync-back-maildir" script that will synchronize all
> messages from your sources at once. This should be executed the first
> time BEFORE any polling, otherwise your IMAP server will be synched to
> your Sup index and if you did not used another client to mark your
> emails as read etc. (like me) and to update your IMAP server then you
> will lose all what you did in Sup (labels, etc.);
> 
> - I added two new hidden labels in Sup: 'replied' and 'forwarded',
> they are automatically added when replying, forwarding or bouncing a
> message, this is to bring a better Maildir support to Sup, this should
> be invisible for users;
> 
> - However you will probably lose all your remote 'replied' and
> 'forwarded' flags after your first backward synchronization to Maildir
> since Sup dropped these labels when it added your messages to xapian.
> 
> - Moving a message from a Maildir source to another is not (yet?)
> supported.
> 
> So, you should now be able to use multiple clients to handle your
> emails, for instance I can use gmail to mark messages as starred or
> so, it will be reflected in Sup. In the same way, if I mark a message
> as read in Sup it will appear as read in the gmail web interface.
> 
> I tested this code with my main account (2 GB and like 15 Maildir
> sources) it works fine so far.
> 
> Be warned that it is highly experimental, but if you use offlineimap
> it has a realdelete option so you *should* not lose any email, in the
> worst case it might mess your flags/labels up.
> 
> I tested it only with offlineimap working with a gmail account.
> 
> Quick howto:
> 
> 1. Close sup
> 2. Backup your emails and your xapian index or use another Sup session
> 3. Clone the branch [1]
> 4. IMPORTANT: run "bin/sup-sync-back-maildir" to synchronize the
>Maildirs you wish, check the help
> 5. Add ":sync_back_to_maildir: true" to your config.yaml
> 6. Run and use sup
> 
> Please test it and make reviews! :)
> 
> NOTE: if you want this to work, get sure that you are not calling
> offlineimap with the '-q' option (as suggested in the wiki) otherwise
> it will ignore the flag updates on IMAP server, so check your
> before-poll hook.
> 
> [0] http://git.fensalir.fr/?p=dleone/sup.git
> [1] $ git clone git://fensalir.fr/dleone/sup.git -b maildir-sync
> 

-- 
Damien Leone 

Web: http://dleone.fensalir.fr/
GPG: 0x82EB4DDF
___
Sup-devel mailing list
Sup-devel@rubyforge.org
http://rubyforge.org/mailman/listinfo/sup-devel


Re: [sup-devel] Adding backward synchronization for Maildir sources

2010-07-25 Thread Edward Z. Yang
Excerpts from Rich Lane's message of Sat Jul 24 17:30:16 -0400 2010:
> I've rebased this against master and merged it to branch maildir-sync in
> the official repo. This is a big change, so help with testing and code
> review would be much appreciated.

Hi Rich,

I've moved my Sup checkout onto this branch. *test test test test*
After a week of usage, I'd say the changes are pretty stable.

Cheers,
Edward
___
Sup-devel mailing list
Sup-devel@rubyforge.org
http://rubyforge.org/mailman/listinfo/sup-devel


Re: [sup-devel] Adding backward synchronization for Maildir sources

2010-07-24 Thread Rich Lane
I've rebased this against master and merged it to branch maildir-sync in
the official repo. This is a big change, so help with testing and code
review would be much appreciated.
___
Sup-devel mailing list
Sup-devel@rubyforge.org
http://rubyforge.org/mailman/listinfo/sup-devel


Re: [sup-devel] Adding backward synchronization for Maildir sources

2010-07-21 Thread Damien Leone
Sup Edward,

Excerpts from Edward Z. Yang's message of Wed Jul 21 02:44:38 +0200 2010:
> Excerpts from Damien Leone's message of Tue Jul 20 05:39:41 -0400 2010:
> In retrospect, I'm fairly certain the majority of crashes were from user 
> hooks.

Could you paste me these hooks please?

> My Sup is running off of f150276f62e263d146fa81ede645b5e90e06a8a1.  They don't
> appear to have the deleted flag, but I didn't look too closely.  It's 
> *possible*
> that a regular synchronization would have picked them up, if they accidentally
> got dropped from my Sup index but were still inside my Maildir.  I suggest not
> worrying too much about this unless someone else reproduces.

What is your setup? IMAP + offlineimap? Do deleted mails work fine
now?

Cheers,

--
Damien Leone 

Web: http://dleone.fensalir.fr/
GPG: 0x82EB4DDF


signature.asc
Description: PGP signature
___
Sup-devel mailing list
Sup-devel@rubyforge.org
http://rubyforge.org/mailman/listinfo/sup-devel


Re: [sup-devel] Adding backward synchronization for Maildir sources

2010-07-21 Thread Damien Leone
Excerpts from Damien Leone's message of Wed Jul 21 10:52:00 +0200 2010:
> What is your setup? IMAP + offlineimap?

My bad, you mentioned offlineimap a few mails ago.

--
Damien Leone 

Web: http://dleone.fensalir.fr/
GPG: 0x82EB4DDF


signature.asc
Description: PGP signature
___
Sup-devel mailing list
Sup-devel@rubyforge.org
http://rubyforge.org/mailman/listinfo/sup-devel


Re: [sup-devel] Adding backward synchronization for Maildir sources

2010-07-20 Thread Edward Z. Yang
Excerpts from Damien Leone's message of Tue Jul 20 05:39:41 -0400 2010:
> Excerpts from Edward Z. Yang's message of Mon Jul 19 00:46:33 +0200 2010:
> > It makes Sup very crashy and there's some strange behavior.  This is 
> > probably
> > the sixth time I'm composing this message (yeah, haven't learned my
> > lesson.) In more detail...
> 
> I'm impressed, I never experienced such crashes, it is quite stable
> here but I use only very basic hooks.

In retrospect, I'm fairly certain the majority of crashes were from user hooks.

> > * Backwards syncing appears to make old deleted mail show up again; I'm not
> >   sure if it's because touching the Maildir causes Sup to pick up on deleted
> >   tags again or something, but I had to go back and delete lots of deleted 
> > mail.
> 
> Hmm, strange, do you use the latest commits of the branch? I pushed a
> fix for deleted mails about 10 days ago.
> 
> The mails that show up again don't have the deleted flag at all?

My Sup is running off of f150276f62e263d146fa81ede645b5e90e06a8a1.  They don't
appear to have the deleted flag, but I didn't look too closely.  It's *possible*
that a regular synchronization would have picked them up, if they accidentally
got dropped from my Sup index but were still inside my Maildir.  I suggest not
worrying too much about this unless someone else reproduces.

> I did not test the code with hooks, what do you suggest for this one?

I mentioned this in the id and Nil message; basically, when iterating over all
messages messages might disappear, and some hooks need to check if a message is
Nil before doing anything to it.

> > * In a similar vein, while messages are going away, '@' will occasionally 
> > return
> >   no messages (pressing again usually fixes it).  Messages that are 
> > labelled inbox
> >   will show up in 'U', but not in the inbox view without a Sup restart.
> 
> I never use this keybinding but indeed I noticed that pressing '@' two
> times in a row was not showing anything.

It is possible that this is a bug in vanilla Sup.

Cheers,
Edward
___
Sup-devel mailing list
Sup-devel@rubyforge.org
http://rubyforge.org/mailman/listinfo/sup-devel


Re: [sup-devel] Adding backward synchronization for Maildir sources

2010-07-20 Thread Damien Leone
Hi,

Excerpts from Edward Z. Yang's message of Mon Jul 19 00:50:39 +0200 2010:
> I would recommend having some sort of flag like 'sync-back'd' programatically
> generated by sup-sync-back-maildir so that Sup will refuse to start if
> you turned on sync-backs without running this command.

Good idea, that might prevent some bad suprises.

> It seems to me that merging the Maildir flags and the Sup flags would
> let you preserve this, but for heavy Sup users this isn't a big deal.

True, and that could be done by sup-sync-back-maildir.

> I think this will be a pretty killer feature to have, because it means
> we can then assign "primary" tags and use those to have sync-backs move
> messages between Maildir folders (on the same account, I imagine), which
> would translate into a folder-sorted webmail view, which is about the
> best you can do without first-class label support in Maildir.  I know of
> at least one friend who this would be a deal-maker.

And that doesn't seem hard to implement, we should definitely consider
this feature, maybe after the primary code has been stabilized.

Cheers,

--
Damien Leone 

Web: http://dleone.fensalir.fr/
GPG: 0x82EB4DDF


signature.asc
Description: PGP signature
___
Sup-devel mailing list
Sup-devel@rubyforge.org
http://rubyforge.org/mailman/listinfo/sup-devel


Re: [sup-devel] Adding backward synchronization for Maildir sources

2010-07-20 Thread Damien Leone
Sup Edward,

Thank you for your feedback!

Excerpts from Edward Z. Yang's message of Mon Jul 19 00:46:33 +0200 2010:
> It makes Sup very crashy and there's some strange behavior.  This is probably
> the sixth time I'm composing this message (yeah, haven't learned my
> lesson.) In more detail...

I'm impressed, I never experienced such crashes, it is quite stable
here but I use only very basic hooks.

> * Backwards syncing appears to make old deleted mail show up again; I'm not
>   sure if it's because touching the Maildir causes Sup to pick up on deleted
>   tags again or something, but I had to go back and delete lots of deleted 
> mail.

Hmm, strange, do you use the latest commits of the branch? I pushed a
fix for deleted mails about 10 days ago.

The mails that show up again don't have the deleted flag at all?

> * When Sup is deleting messages, it is /extremely/ unsafe to iterate over
>   a list of all messages in the index and query them for info: the threading 
> means
>   that any given message might not exist anymore.  I had a hook that kept 
> triggering
>   this bug, and earlier on in the process drawing functions would also 
> frequently
>   Nil error.  This is probably under-tested codepaths in Sup that we should 
> track
>   down; unfortunately I didn't have the presence of mind to record traces 
> until
>   later (and the ones I got all look like my hooks fault).

I did not test the code with hooks, what do you suggest for this one?

> * In a similar vein, while messages are going away, '@' will occasionally 
> return
>   no messages (pressing again usually fixes it).  Messages that are labelled 
> inbox
>   will show up in 'U', but not in the inbox view without a Sup restart.

I never use this keybinding but indeed I noticed that pressing '@' two
times in a row was not showing anything.

I will try to track down these bugs.

Thanks again!
Cheers,

--
Damien Leone 

Web: http://dleone.fensalir.fr/
GPG: 0x82EB4DDF


signature.asc
Description: PGP signature
___
Sup-devel mailing list
Sup-devel@rubyforge.org
http://rubyforge.org/mailman/listinfo/sup-devel


Re: [sup-devel] Adding backward synchronization for Maildir sources

2010-07-18 Thread Edward Z. Yang
Here are some suggestions...

Excerpts from Damien Leone's message of Fri Jul 09 12:44:01 -0400 2010:
> - I wrote a "sup-sync-back-maildir" script that will synchronize all
> messages from your sources at once. This should be executed the first
> time BEFORE any polling, otherwise your IMAP server will be synched to
> your Sup index and if you did not used another client to mark your
> emails as read etc. (like me) and to update your IMAP server then you
> will lose all what you did in Sup (labels, etc.);

I would recommend having some sort of flag like 'sync-back'd' programatically
generated by sup-sync-back-maildir so that Sup will refuse to start if
you turned on sync-backs without running this command.

> - However you will probably lose all your remote 'replied' and
> 'forwarded' flags after your first backward synchronization to Maildir
> since Sup dropped these labels when it added your messages to xapian.

It seems to me that merging the Maildir flags and the Sup flags would
let you preserve this, but for heavy Sup users this isn't a big deal.

> - Moving a message from a Maildir source to another is not (yet?)
> supported.

I think this will be a pretty killer feature to have, because it means
we can then assign "primary" tags and use those to have sync-backs move
messages between Maildir folders (on the same account, I imagine), which
would translate into a folder-sorted webmail view, which is about the
best you can do without first-class label support in Maildir.  I know of
at least one friend who this would be a deal-maker.

Cheers,
Edward
___
Sup-devel mailing list
Sup-devel@rubyforge.org
http://rubyforge.org/mailman/listinfo/sup-devel


Re: [sup-devel] Adding backward synchronization for Maildir sources

2010-07-18 Thread Edward Z. Yang
Here is one trace that I managed to scrounge up

undefined method `content_width' for nil:NilClass
/home/ezyang/Dev/sup/lib/sup/modes/thread-index-mode.rb:953:in `from_width'
/home/ezyang/Dev/sup/lib/sup/modes/thread-index-mode.rb:870:in 
`text_for_thread_at'
/home/ezyang/Dev/sup/lib/sup/hook.rb:55:in `each_with_index'
/home/ezyang/Dev/sup/lib/sup/modes/thread-index-mode.rb:869:in `each'
/home/ezyang/Dev/sup/lib/sup/modes/thread-index-mode.rb:869:in `each_with_index'
/home/ezyang/Dev/sup/lib/sup/modes/thread-index-mode.rb:869:in 
`text_for_thread_at'
/home/ezyang/Dev/sup/lib/sup/modes/thread-index-mode.rb:811:in `regen_text'
/home/ezyang/Dev/sup/lib/sup/util.rb:423:in `map_with_index'
/home/ezyang/Dev/sup/lib/sup/hook.rb:55:in `each_with_index'
/home/ezyang/Dev/sup/lib/sup/util.rb:423:in `each'
/home/ezyang/Dev/sup/lib/sup/util.rb:423:in `each_with_index'
/home/ezyang/Dev/sup/lib/sup/util.rb:423:in `map_with_index'
/home/ezyang/Dev/sup/lib/sup/modes/thread-index-mode.rb:811:in `regen_text'
/home/ezyang/Dev/sup/lib/sup/modes/thread-index-mode.rb:262:in `update'
/home/ezyang/Dev/sup/lib/sup/modes/thread-index-mode.rb:217:in 
`handle_location_deleted_update'
/home/ezyang/Dev/sup/lib/sup/update.rb:26:in `send'
/home/ezyang/Dev/sup/lib/sup/update.rb:26:in `relay'
/home/ezyang/Dev/sup/lib/sup/update.rb:26:in `each'
/home/ezyang/Dev/sup/lib/sup/update.rb:26:in `relay'
/home/ezyang/Dev/sup/lib/sup/util.rb:588:in `send'
/home/ezyang/Dev/sup/lib/sup/util.rb:588:in `method_missing'
/home/ezyang/Dev/sup/lib/sup/poll.rb:197:in `poll_from'
/home/ezyang/Dev/sup/lib/sup/index.rb:258:in `each_message'
/home/ezyang/Dev/sup/lib/sup/index.rb:245:in `each_id'
/home/ezyang/Dev/sup/lib/sup/index.rb:245:in `each'
/home/ezyang/Dev/sup/lib/sup/index.rb:245:in `each_id'
/home/ezyang/Dev/sup/lib/sup/index.rb:257:in `each_message'
/home/ezyang/Dev/sup/lib/sup/util.rb:588:in `send'
/home/ezyang/Dev/sup/lib/sup/util.rb:588:in `method_missing'
[snip]

Cheers,
Edward
___
Sup-devel mailing list
Sup-devel@rubyforge.org
http://rubyforge.org/mailman/listinfo/sup-devel


Re: [sup-devel] Adding backward synchronization for Maildir sources

2010-07-18 Thread Edward Z. Yang
Sup Damien,

I cut over to your branch, plus one patch of my own making.

THE GOOD NEWS

It works!  Whereby, messages are marked read in my webmail client and
deleted messages go away.

THE BAD NEWS

It makes Sup very crashy and there's some strange behavior.  This is probably
the sixth time I'm composing this message (yeah, haven't learned my
lesson.) In more detail...

* You want the following OfflineImap patch, because otherwise the massive
  changes to labels that will surely be induced by backwards syncing will
  cause OfflineImap to CRAWL.

diff --git a/offlineimap/folder/LocalStatus.py 
b/offlineimap/folder/LocalStatus.py
index 157989d..7f4dc5b 100644
--- a/offlineimap/folder/LocalStatus.py
+++ b/offlineimap/folder/LocalStatus.py
@@ -33,7 +33,7 @@ class LocalStatusFolder(BaseFolder):
 self.messagelist = None
 self.repository = repository
 self.savelock = threading.Lock()
-self.doautosave = 1
+self.doautosave = 0
 self.accountname = accountname
 BaseFolder.__init__(self)

  Though, you probably want to undo this patch when you're done.  Autosaving
  means partial progress is saved.

* Backwards syncing appears to make old deleted mail show up again; I'm not
  sure if it's because touching the Maildir causes Sup to pick up on deleted
  tags again or something, but I had to go back and delete lots of deleted mail.

* When Sup is deleting messages, it is /extremely/ unsafe to iterate over
  a list of all messages in the index and query them for info: the threading 
means
  that any given message might not exist anymore.  I had a hook that kept 
triggering
  this bug, and earlier on in the process drawing functions would also 
frequently
  Nil error.  This is probably under-tested codepaths in Sup that we should 
track
  down; unfortunately I didn't have the presence of mind to record traces until
  later (and the ones I got all look like my hooks fault).

* In a similar vein, while messages are going away, '@' will occasionally return
  no messages (pressing again usually fixes it).  Messages that are labelled 
inbox
  will show up in 'U', but not in the inbox view without a Sup restart.

I haven't done a code review yet.

Cheers,
Edward
___
Sup-devel mailing list
Sup-devel@rubyforge.org
http://rubyforge.org/mailman/listinfo/sup-devel


Re: [sup-devel] Adding backward synchronization for Maildir sources

2010-07-18 Thread Tero Tilus
Damien Leone, 2010-07-17 15:55:
> So, any reviewer?

I haven't had the time yet.  I definitely will once the temperature
here in Finland drops below 20 degrees Celcius. ;)  For now I'm almost
totally off the net and on the beach.

> I thought backward synchronization was a wanted
> feature.

And you were _very_ much right!

> I often check my gmail web interface and everything is synched.

My setup (once I get there) will be mainly sup + occasional mutt (on
maildirs) and evolution (via dovecot).

-- 
Tero Tilus ## 050 3635 235 ## http://tero.tilus.net/
___
Sup-devel mailing list
Sup-devel@rubyforge.org
http://rubyforge.org/mailman/listinfo/sup-devel


Re: [sup-devel] Adding backward synchronization for Maildir sources

2010-07-17 Thread Damien Leone
Hi Ben,

Excerpts from Ben Walton's message of Sat Jul 17 19:05:44 +0200 2010:
> I believe it is (not by me personally as I'm not affected).  How does
> it play with the recent maildir branch merge from Rich?

It should play well since I based my branch on the 'next' branch which
already had the 'maildir' merge.

--
Damien Leone 

Web: http://dleone.fensalir.fr/
GPG: 0x82EB4DDF


signature.asc
Description: PGP signature
___
Sup-devel mailing list
Sup-devel@rubyforge.org
http://rubyforge.org/mailman/listinfo/sup-devel


Re: [sup-devel] Adding backward synchronization for Maildir sources

2010-07-17 Thread Ben Walton
Excerpts from Damien Leone's message of Sat Jul 17 08:55:56 -0400 2010:

> So, any reviewer? I thought backward synchronization was a wanted
> feature.

I believe it is (not by me personally as I'm not affected).  How does
it play with the recent maildir branch merge from Rich?

Thanks
-Ben
--
Ben Walton
Systems Programmer - CHASS
University of Toronto
C:416.407.5610 | W:416.978.4302

___
Sup-devel mailing list
Sup-devel@rubyforge.org
http://rubyforge.org/mailman/listinfo/sup-devel


Re: [sup-devel] Adding backward synchronization for Maildir sources

2010-07-17 Thread Damien Leone
So, any reviewer? I thought backward synchronization was a wanted
feature.

For information I have been testing it with my real account for more
than a week now without noticing any serious issue, I often check my
gmail web interface and everything is synched.

I had once a location trouble after having sent an email but I could
not reproduce and it has been resolved after a sup restart.

--
Damien Leone 

Web: http://dleone.fensalir.fr/
GPG: 0x82EB4DDF


signature.asc
Description: PGP signature
___
Sup-devel mailing list
Sup-devel@rubyforge.org
http://rubyforge.org/mailman/listinfo/sup-devel


Re: [sup-devel] Adding backward synchronization for Maildir sources

2010-07-09 Thread Edward Z. Yang
Excerpts from Damien Leone's message of Fri Jul 09 13:14:58 -0400 2010:
> > 1. Suppose we decide to test your branch, and run into some show-stopper
> >bugs.  What’s the procedure for backing out changes?
> 
> You should backup your Maildir directories and your xapian index, this
> is what I did and if something goes wrong I can still restore my
> backups and run offlineimap to resynch everything as it was before on
> the IMAP server.
> 
> But for my initial tests I used a test Maildir containing nothing
> important so maybe this is what you should do. ;)

I doubt a test Maildir will give you the sort of test pressure that you’re
looking for. Go go real datasets.

> > 2. What precisely are the semantics for handling Sup labels?
> 
> I followed the Maildir documentation [0]. When synching a message, Sup
> builds the flags string and relink the file to the disk, moving it
> from new/ to cur/ or vice versa if needed.
> 
> Related labels are: unread, replied, forwarded, deleted, draft (but it
> won't work for now because draft messages in Sup are handled
> separately), flagged (aka starred).

Great, thanks!  I’ll give it a spin this weekend.

Edward
___
Sup-devel mailing list
Sup-devel@rubyforge.org
http://rubyforge.org/mailman/listinfo/sup-devel


Re: [sup-devel] Adding backward synchronization for Maildir sources

2010-07-09 Thread Damien Leone
Hello Edward,

> 1. Suppose we decide to test your branch, and run into some show-stopper
>bugs.  What’s the procedure for backing out changes?

You should backup your Maildir directories and your xapian index, this
is what I did and if something goes wrong I can still restore my
backups and run offlineimap to resynch everything as it was before on
the IMAP server.

But for my initial tests I used a test Maildir containing nothing
important so maybe this is what you should do. ;)

> 2. What precisely are the semantics for handling Sup labels?

I followed the Maildir documentation [0]. When synching a message, Sup
builds the flags string and relink the file to the disk, moving it
from new/ to cur/ or vice versa if needed.

Related labels are: unread, replied, forwarded, deleted, draft (but it
won't work for now because draft messages in Sup are handled
separately), flagged (aka starred).

Every other labels are ignored because not supported.

[0] http://cr.yp.to/proto/maildir.html

--
Damien Leone 

Web: http://dleone.fensalir.fr/
GPG: 0x82EB4DDF


signature.asc
Description: PGP signature
___
Sup-devel mailing list
Sup-devel@rubyforge.org
http://rubyforge.org/mailman/listinfo/sup-devel


Re: [sup-devel] Adding backward synchronization for Maildir sources

2010-07-09 Thread Edward Z. Yang
Hey Damien,

This is great!  A few quick questions for you:

1. Suppose we decide to test your branch, and run into some show-stopper
   bugs.  What’s the procedure for backing out changes?

2. What precisely are the semantics for handling Sup labels?

Thanks,
Edward

Excerpts from Damien Leone's message of Fri Jul 09 12:44:01 -0400 2010:
> Sup guys,
> 
> I have been working on backward synchronization for Maildir sources
> last week and I need your help to test and review the code.
> 
> I pushed my commits (based on the 'next' branch) to my repo which you
> can find here [0] for the web interface and here [1] for the git
> address.
> 
> The following is related to Maildir sources ONLY.
> 
> So what does work and what does not (if you don't want to read this,
> scroll to the quick howto below):
> 
> - Sup will now stay synchronized with your remote sources by detecting
> if a message has been remotely updated (ie: you change a flag from
> another client), these updates will be applied to your local index and
> your thread-view-mode buffers should be properly refreshed;
> 
> - When a message is remotely deleted, it should now disappear from
> your index;
> 
> - There is a new configuration option called "sync_back_to_maildir"
> which is false by default. When true, this option updates your actual
> Maildir files when you change a label, this is done in real time
> before being saved to xapian. So for instance if you use offlineimap
> the changes should be synched back to your IMAP server by the next
> poll in Sup (assuming that your before-poll hook runs offlineimap);
> 
> - I wrote a "sup-sync-back-maildir" script that will synchronize all
> messages from your sources at once. This should be executed the first
> time BEFORE any polling, otherwise your IMAP server will be synched to
> your Sup index and if you did not used another client to mark your
> emails as read etc. (like me) and to update your IMAP server then you
> will lose all what you did in Sup (labels, etc.);
> 
> - I added two new hidden labels in Sup: 'replied' and 'forwarded',
> they are automatically added when replying, forwarding or bouncing a
> message, this is to bring a better Maildir support to Sup, this should
> be invisible for users;
> 
> - However you will probably lose all your remote 'replied' and
> 'forwarded' flags after your first backward synchronization to Maildir
> since Sup dropped these labels when it added your messages to xapian.
> 
> - Moving a message from a Maildir source to another is not (yet?)
> supported.
> 
> So, you should now be able to use multiple clients to handle your
> emails, for instance I can use gmail to mark messages as starred or
> so, it will be reflected in Sup. In the same way, if I mark a message
> as read in Sup it will appear as read in the gmail web interface.
> 
> I tested this code with my main account (2 GB and like 15 Maildir
> sources) it works fine so far.
> 
> Be warned that it is highly experimental, but if you use offlineimap
> it has a realdelete option so you *should* not lose any email, in the
> worst case it might mess your flags/labels up.
> 
> I tested it only with offlineimap working with a gmail account.
> 
> Quick howto:
> 
> 1. Close sup
> 2. Backup your emails and your xapian index or use another Sup session
> 3. Clone the branch [1]
> 4. IMPORTANT: run "bin/sup-sync-back-maildir" to synchronize the
>Maildirs you wish, check the help
> 5. Add ":sync_back_to_maildir: true" to your config.yaml
> 6. Run and use sup
> 
> Please test it and make reviews! :)
> 
> NOTE: if you want this to work, get sure that you are not calling
> offlineimap with the '-q' option (as suggested in the wiki) otherwise
> it will ignore the flag updates on IMAP server, so check your
> before-poll hook.
> 
> [0] http://git.fensalir.fr/?p=dleone/sup.git
> [1] $ git clone git://fensalir.fr/dleone/sup.git -b maildir-sync
> 
___
Sup-devel mailing list
Sup-devel@rubyforge.org
http://rubyforge.org/mailman/listinfo/sup-devel