Re: [Evolution-hackers] Moving from the single mbox file format for the local folders

2009-12-15 Thread Srinivasa Ragavan
Hello everyone,

On Wed, Dec 16, 2009 at 1:16 AM, Chenthill pchenth...@novell.com wrote:
 Hi fellow hackers!!
    I have been working for a while during last week on one the blockers
 in evolution - https://bugzilla.gnome.org/show_bug.cgi?id=550414 -
 'Folder and summary mismatch error'(old one -
 https://bugzilla.gnome.org/show_bug.cgi?id=213072). As a matter of fact
 we have been working as a team to get the blockers down. I have not been
 able to reproduce the issue or yet find the exact problematic area.

        The mismatch in the frompos index in the folder summary may be caused
 by either a threading issue or a crash while storing the indexes. I am
 still investigating it to find the real cause.

        Looking at other issues such as,
 https://bugzilla.gnome.org/show_bug.cgi?id=522433 - 'Fails opening mbox
 2GB', just got a thought if we could solve both the issues by,

 Approach #1,
 migrating local storage from mbox to maildir format. With maildir I have
 heard about two issues,

 * Not able to create subfolders under INBOX -
 https://bugzilla.gnome.org/show_bug.cgi?id=536240 .

 Approach #2,
 Migrate from a single mbox file per folder to mbox per email. Srini
 mentioned an advantage that this would avoid the file renames that
 maildir does. I think this is much like how other remote providers in
 evo store the email.

It should be rocket fast!. Expunge is just unlink one file. Change of
flags etc rewrites just that file when upsync happens. No rewrite 2gb
of a file, to expunge 10 mails. Startup/shutdown faster etc etc. I'm a
fan of this, what so ever! To overcome 100K mbox files in one folder,
distribute under multi-level subdirs and let summary know that.


 I thought of bring this in this list to gather more opinions to choose
 the right one. The approach #2 seems a better one as we are choosing a
 way for storing the messages internally in evo. Are we missing to see
 anything while we choose the second one ?

 One advantage which I see with #1 is that its a standard way.

You store everything as mbox only still. But one mbox file per mail,
distributed in multiple subdirs. Ofcourse you dont want 100K files in
one folder, which could move the bottleneck to a different place.
Distribute under multi-level subfolders or something like that.

-Srini
___
Evolution-hackers mailing list
Evolution-hackers@gnome.org
http://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] Moving from the single mbox file format for the local folders

2009-12-15 Thread Reid Thompson
On Wed, 2009-12-16 at 01:16 +0530, Chenthill wrote:

 
 Approach #1,
 migrating local storage from mbox to maildir format. With maildir I
 have
 heard about two issues,
 
 * Not able to create subfolders under INBOX -
 https://bugzilla.gnome.org/show_bug.cgi?id=536240 .
I hadn't noticed the above, so I guess it's a non-issue for me

What is the second issue?
 
 Approach #2,
 Migrate from a single mbox file per folder to mbox per email. Srini
 mentioned an advantage that this would avoid the file renames that
 maildir does. I think this is much like how other remote providers in
 evo store the email.
You still have a filename per email right?
What naming convention would be used?
 
 I thought of bring this in this list to gather more opinions to choose
 the right one. The approach #2 seems a better one as we are choosing a
 way for storing the messages internally in evo. Are we missing to see
 anything while we choose the second one ? 
 
 One advantage which I see with #1 is that its a standard way.
Would Evo provide a mechanism to migrate/convert a mailbox/folder with
this format to a mailbox/folder with a standard format?

I.E. Currently, dragging a folder in one format to an account in a
different format performs the proper migration to the new account's
folder format.

Or would it be up to the user to do something like

for file in `ls *mbox`
do
   movemail $file maildir:///tmp/maildr
done

if they wanted to migrate to standard format?
 
 Thanks, Chenthill.

___
Evolution-hackers mailing list
Evolution-hackers@gnome.org
http://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] Moving from the single mbox file format for the local folders

2009-12-15 Thread Chenthill
On Tue, 2009-12-15 at 15:09 -0500, Reid Thompson wrote:
 On Wed, 2009-12-16 at 01:16 +0530, Chenthill wrote:
 
  
  Approach #1,
  migrating local storage from mbox to maildir format. With maildir I
  have
  heard about two issues,
  
  * Not able to create subfolders under INBOX -
  https://bugzilla.gnome.org/show_bug.cgi?id=536240 .
 I hadn't noticed the above, so I guess it's a non-issue for me
 
 What is the second issue?
Sorry missed to mention it here, with maildir we would need to rename
files for unread/read flag changes which can be avoided in the later
approach.

  
  Approach #2,
  Migrate from a single mbox file per folder to mbox per email. Srini
  mentioned an advantage that this would avoid the file renames that
  maildir does. I think this is much like how other remote providers
 in
  evo store the email.
 You still have a filename per email right?
yes.
 What naming convention would be used?
We would be using the uid of the message . uid would be a 32 bit
unsigned integer.

  
  I thought of bring this in this list to gather more opinions to
 choose
  the right one. The approach #2 seems a better one as we are choosing
 a
  way for storing the messages internally in evo. Are we missing to
 see
  anything while we choose the second one ? 
  
  One advantage which I see with #1 is that its a standard way.
 Would Evo provide a mechanism to migrate/convert a mailbox/folder with
 this format to a mailbox/folder with a standard format?
Yes we would provide an UI option for exporting data. While going with
the second approach, I would prefer exporting in maildir format.

 I.E. Currently, dragging a folder in one format to an account in a
 different format performs the proper migration to the new account's
 folder format.
 
 Or would it be up to the user to do something like
 
 for file in `ls *mbox`
 do
movemail $file maildir:///tmp/maildr
 done
 
 if they wanted to migrate to standard format?
It would be the same behavior as before with the mbox-file-per-email if
we choose to do that.

- Chenthill.
  
  Thanks, Chenthill.
 


___
Evolution-hackers mailing list
Evolution-hackers@gnome.org
http://mail.gnome.org/mailman/listinfo/evolution-hackers