Re: [Evolution-hackers] Introduction and Questions

2007-06-08 Thread Philip Van Hoof
On Thu, 2007-06-07 at 09:25 -0400, Jeffrey Stedfast wrote:

 On Thu, 2007-06-07 at 10:56 +0300, Philip Van Hoof wrote:
 
  The best way is to ask for the ENVELOPE and the remaining info using the
  normal BODY.PEEK method.
 
 Have you actually ever tested this theory? or did you just pull this out
 of your proverbial?

I have. The ENVELOPE-way is slightly better because words like From:
and To: are not repeated each time. Although with COMPRESS and/or
support for deflate in the TLS/SSL library will probably compress that
difference away mostly.

Note I didn't test this with compression. I saw a difference of 5% to
10% depending on several factors in bandwidth saving. I usually test all
this with wireshark, which doesn't help a lot with compressed data. It
would be nice, indeed, to get some analysis of how data is being
compressed when compression by either COMPRESS or the SSL library is
being done.

Note that I haven't yet implemented COMPRESS in camel-lite, but I will
do this very soon (I have a CamelStreamGzip thingy ready for this).

On top of bandwidth, on pure speed a lot IMAP servers will return your
result faster if you only use ENVELOPE. That's because they optimise on
the fields that are in ENVELOPE (indexes, etc etc).

But then again, there's not enough in ENVELOPE to support all of
Evolution's features, so you'll still have to use BODY.PEEK which will
most likely slowdown everything back again.

 You should note that Thunderbird does not use ENVELOPE, probably because
 they found it was LESS efficient to fetch the ENVELOPE +
 BODY.PEEK[HEADER.FIELDS (REFERENCES CONTENT-TYPE)] on many existing IMAP
 servers in the wild.

Probably, indeed.

 Having myself tested this in the past, I don't recall it making much
 difference one way or the other.

On bandwidth, it does. Especially when you have a lot of messages in
some folders and the server is not compressing things (no COMPRESS nor
TLS/SSL compressing things for you).

 Sadly using ENVELOPE breaks on at least one proprietary IMAP server
 which occasionally likes to send back responses with NIL as each and
 every item, at least until a client has issued a BODY.PEEK fetch
 request.

Sounds like a bug in that server implementation.

   It should be possible to specify per folder
  which of the headers are to be fetched, to make it really efficient.
 
 how do you propose calculating the list of headers each folder should
 request? which headers would you eliminate conditionally?

I'd probably start with the references in case the user doesn't want his
headers-view to be threaded, and fetch only the sequence number plus the
references header the first time threaded is enabled on that folder and
after that (while its enabled) for each summary-item update too.

A lot mobile devices's screens are simply not suitable for threaded view
anyway: introduces way to much wasted white space. Why fetch this
information and waste the user's GPRS bandwidth and therefore often his
money too? (for a lot people, GPRS bandwidth costs money per downloaded
megabyte)

In case the From is not displayed, there's no real need to fetch it
until the first time the column is added. Same for the dates (date sent
and date received, usually only one of both is used by the user). 

Etc etc etc.
 
  The imap4 implementation seems to have an ENVELOPE parser, so I guess
  either it does it already or it will use ENVELOPE in future.
 
 it fetches:
 
 ENVELOPE BODY.PEEK[HEADER.FIELDS (Content-Type References In-Reply-To)]
 
 and optionally the mailing-list headers
 
 I believe it fetches IN-REPLY-TO (which is included in ENVELOPE)
 explicitly because some IMAP servers tried to be smart and parse the
 IN-REPLY-TO header for us and didn't handle embedded phrases which
 certain broken free-software mailers like to send.
 
  
  For a mobile device, that works over GPRS, you for example are usually
  not interested (not at all) in the mailinglist specific headers.
 
 this is mobile-device specific tho, when I was implementing imap4 and
 people started trying to use it, it was their #1 complaint (which is why
 imap4 conditionally requests the mlist headers... tho that flag may be
 hard-coded to TRUE now, I'm not sure)

But some people don't use it. Although people more often complain about
missing features and not so much about that their E-mail client is
wasting their money on GPRS bandwidth, I still believe it's important to
support low bandwidth modes too. Also for Evolution, as Evolution is
often used with laptops.

I do agree, though, that Evolution has a focus difference compared to
what I'm doing with Camel. In my opinion is this why splitting Camel
from Evolution might help: we could then more easily introduce features
for letting Camel switch from low bandwidth mode, to Evolution mode and
back.

Right now any change that isn't absolutely 100% super pro and for
Evolution and Evolution only, gets ignored by you guys at Novell.

Usually the explanation goes like this: Yeah but Evolution 

Re: [Evolution-hackers] Introduction and Questions

2007-06-08 Thread Jeffrey Stedfast
On Fri, 2007-06-08 at 14:39 +0300, Philip Van Hoof wrote:
 On Thu, 2007-06-07 at 09:25 -0400, Jeffrey Stedfast wrote:
 
  On Thu, 2007-06-07 at 10:56 +0300, Philip Van Hoof wrote:
  
   The best way is to ask for the ENVELOPE and the remaining info using the
   normal BODY.PEEK method.
  
  Have you actually ever tested this theory? or did you just pull this out
  of your proverbial?
 
 I have. The ENVELOPE-way is slightly better because words like From:
 and To: are not repeated each time. Although with COMPRESS and/or
 support for deflate in the TLS/SSL library will probably compress that
 difference away mostly.

ENVELOPE alone is no doubt as fast or faster (hopefully faster) than
explicit header fetching (I expect most servers probably cache ENVELOPE
info, whereas explicit header fetching would have to be done by
re-parsing the message) and (probably) less processor intensive
server-side.

 
 Note I didn't test this with compression. I saw a difference of 5% to
 10% depending on several factors in bandwidth saving. I usually test all
 this with wireshark, which doesn't help a lot with compressed data. It
 would be nice, indeed, to get some analysis of how data is being
 compressed when compression by either COMPRESS or the SSL library is
 being done.
 
 Note that I haven't yet implemented COMPRESS in camel-lite, but I will
 do this very soon (I have a CamelStreamGzip thingy ready for this).
 
 On top of bandwidth, on pure speed a lot IMAP servers will return your
 result faster if you only use ENVELOPE. That's because they optimise on
 the fields that are in ENVELOPE (indexes, etc etc).

nod, which is why imap4 used ENVELOPE. I later had to add explicit
fetching of other headers, though, to maintain feature compat with imap.

 
 But then again, there's not enough in ENVELOPE to support all of
 Evolution's features, so you'll still have to use BODY.PEEK which will
 most likely slowdown everything back again.

indeed, and it doesn't appear to be faster trying to 
use ENVELOPE + explicit header fetching, /probably/ because the server
still has to spend time parsing the actual message off disk, so the
performance gain of using ENVELOPE for the (typically? cached) summary
info is negated - possibly made even slower (server has to hit 2
code-paths instead of just the 1), but bandwidth is still slightly
decreased, possibly making up for the increased processing time.

 
  You should note that Thunderbird does not use ENVELOPE, probably because
  they found it was LESS efficient to fetch the ENVELOPE +
  BODY.PEEK[HEADER.FIELDS (REFERENCES CONTENT-TYPE)] on many existing IMAP
  servers in the wild.
 
 Probably, indeed.
 
  Having myself tested this in the past, I don't recall it making much
  difference one way or the other.
 
 On bandwidth, it does. Especially when you have a lot of messages in
 some folders and the server is not compressing things (no COMPRESS nor
 TLS/SSL compressing things for you).

ok, fair enough... not sure how much you really cut out tho, address
lists are likely more bandwidth intensive in the ENVELOPE response than
in raw format.

To: Jeffrey Stedfast [EMAIL PROTECTED], Miguel de Icaza [EMAIL 
PROTECTED]\r\n

ENVELOPE equiv:

((Jeffrey Stedfast NIL fejj novell.com) (Miguel de Icaza NIL miguel 
ximian.com))

You can see that the ENVELOPE response is more verbose, so the savings
really depends on individual messages.

You do save on the command echoing from the server, tho...


all in all, yea, probably you save some bandwidth. For most users,
ENVELOPE is probably a win as far as bytes-sent-across-the-wire.

The biggest win for ENVELOPE, in my opinion, is not bandwidth but server
processing (measured in both time and resources). Unfortunately, if you
have to explicitly request headers anyway, this is completely negated :(

 
  Sadly using ENVELOPE breaks on at least one proprietary IMAP server
  which occasionally likes to send back responses with NIL as each and
  every item, at least until a client has issued a BODY.PEEK fetch
  request.
 
 Sounds like a bug in that server implementation.

yes, it is.

 
It should be possible to specify per folder
   which of the headers are to be fetched, to make it really efficient.
  
  how do you propose calculating the list of headers each folder should
  request? which headers would you eliminate conditionally?
 
 I'd probably start with the references in case the user doesn't want his
 headers-view to be threaded, and fetch only the sequence number plus the
 references header the first time threaded is enabled on that folder and
 after that (while its enabled) for each summary-item update too.
 
 A lot mobile devices's screens are simply not suitable for threaded view
 anyway: introduces way to much wasted white space. Why fetch this
 information and waste the user's GPRS bandwidth and therefore often his
 money too? (for a lot people, GPRS bandwidth costs money per downloaded
 megabyte)
 
 In case the From is not displayed, there's no 

Re: [Evolution-hackers] Introduction and Questions

2007-06-08 Thread Philip Van Hoof
On Fri, 2007-06-08 at 09:48 -0400, Jeffrey Stedfast wrote:

 I think the laptop problem is solved with the basic headers feature,
 at least as far as collecting new summary info is concerned.
 
 Syncing flags is another story, and where the real
 slowness/user-frustration lies.
 
 I'm sure David Woodhouse would agree here.

ps. The condstore work in camel-lite is the most easy part to migrate.

  Right now any change that isn't absolutely 100% super pro and for
  Evolution and Evolution only, gets ignored by you guys at Novell.
 
 I would argue that not to be the case. I would instead argue that the
 reason many of your patches haven't been incorporated into Camel is
 that they are hackish and/or tailored explicitly for /your/ needs, and
 not for the good of the many.

I agree that a lot of the things are hackish, some of the changes are
this way because there are some design problems in Camel that make
certain things not possible without such hacks.

For example the fact that non-blocking read()s are not possible by the
default CamelStream things, and that none of the existing code really
copes very well with non-blocking reads.

The imap4 project is making things look better, though all in all it's
still much of the same (blocking and waiting for results, in stead of
letting the server do most the work and do pipelining).

 That's not the one I was thinking of, actually... I was more thinking
 of the BODY(STRUCTURE) stuff.

Camel-lite doesn't use BODYSTRUCTURE, it only uses body.peek[0] for
doing a partial retrieval. I'm, however, planning to let the
CamelMimePart late fetch not-yet retrieved parts when needed (in stead
of always downloading the full, or only the body part of the message).

That would conflict with IMAP servers that don't do mime parsing right.

 Then don't be so negative. I'd hardly say that imap4 has a majority of
 its design wrong.

 Try some constructive criticism for a change...

You are right, let's try to criticise constructive. My apologises for
the negative way of putting that I don't like how imap4 is designed.


-- 
Philip Van Hoof, software developer
home: me at pvanhoof dot be 
gnome: pvanhoof at gnome dot org 
http://www.pvanhoof.be/blog




___
Evolution-hackers mailing list
Evolution-hackers@gnome.org
http://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] Introduction and Questions

2007-06-08 Thread Jeffrey Stedfast
On Fri, 2007-06-08 at 12:23 -0700, Ross Boylan wrote:
 On Thu, 2007-06-07 at 09:25 -0400, Jeffrey Stedfast wrote:
  it's not possible to do better w/o dropping features like message
  threading.
  
  In fact, the above minimalizing of header fetching already breaks the
  quick context-menu vfolder on mailing-list and filter on
  mailing-list features as well as making vfoldering on mailing-list
  oodles slower (if it even still works after disabling the headers) 
 What about letting the server do the threading, if it can?

1. not all servers support it, so we need to implement threading
client-side anyway

2. not all servers do it /correctly/, so we'd need to have a lookup
table of server idents so we knew when to fallback (note that every mail
client I know of that tried to be smart and let the server do the
threading has lived to regret making this decision)

3. if threading behavior is different from other accounts, it will feel
weird to the user.

   I realize
 not all can, and probably all will seem not as smart as they could be
 (or just different from how threading would be done in evolution).  But
 otherwise, there's no way around getting lots of headers, and that is a
 huge hit with big folders.
 
 That's the theory; the practice seems uglier.  I tried mulberry, which
 is supposed to make very good use of IMAP.  In many ways it does, but
 its use of server-side threading doesn't work too well.  It seemed to
 get the response back quickly from the server, but then take minutes
 processing and parsing this response.  Furthermore, every time one
 scrolls the window, it does it all over again.
 
 The other problem with server-side threading is that it threads
 everything.  In principle it might make more sense to retrieve a window
 of headers and thread them.

actually, I think this is what Mulberry is doing - or at least that
would make sense if it has to re-fetch thread info when you scroll.

 
 The window might not include all items in the thread, but that seems a
 reasonable trade-off to me in return for decent performance.  One could
 do a more thorough job in the background and update when it finished.

but then you have items in the list moving around as more thread info
becomes available which would be annoying to users, imho.

Jeff


___
Evolution-hackers mailing list
Evolution-hackers@gnome.org
http://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] Introduction and Questions

2007-06-08 Thread Ross Boylan
On Fri, 2007-06-08 at 17:11 -0400, Jeffrey Stedfast wrote:
 On Fri, 2007-06-08 at 12:23 -0700, Ross Boylan wrote:
  On Thu, 2007-06-07 at 09:25 -0400, Jeffrey Stedfast wrote:
   it's not possible to do better w/o dropping features like message
   threading.
   
   In fact, the above minimalizing of header fetching already breaks the
   quick context-menu vfolder on mailing-list and filter on
   mailing-list features as well as making vfoldering on mailing-list
   oodles slower (if it even still works after disabling the headers) 
  What about letting the server do the threading, if it can?
 
 1. not all servers support it, so we need to implement threading
 client-side anyway
 
 2. not all servers do it /correctly/, so we'd need to have a lookup
 table of server idents so we knew when to fallback (note that every mail
 client I know of that tried to be smart and let the server do the
 threading has lived to regret making this decision)
The parenthetical remark is pretty interesting.
 
 3. if threading behavior is different from other accounts, it will feel
 weird to the user.
All good points.  I should explain I'm thinking of a mode that might
kick in only on large mailboxes, or as a user option or something like
that.  My main concern is that with very large folders most mail cients,
including evolution, become sluggish to the point of being unusable.
For many of the clients life is much uglier during the initial contact
with the folders than later.  For example, when I move a message into a
folder I haven't opened or operated on in evolution, I'm noticing long
delays (like  1 minute) while it does something (fetch headers and
create an index?  whatever it is uses some, but not all the CPU).

In this context an responsive but imperfect solution is better than an
unresponsive perfect one.

For example, if evo relies on the server's threads then it will
obviously inherit their behavior, warts and all.  In general I agree
with the point, made recently on this list, that it's desirable for
clients to shield users from server problems (and most users don't know
and don't care that the the real problem is with the server).  But if
doing so takes forever, some may be willing to give up on that.

Whether relaying on server side threading, which currently only operates
on the whole mailbox, can solve the performance problems is another
issue.
 
I realize
  not all can, and probably all will seem not as smart as they could be
  (or just different from how threading would be done in evolution).  But
  otherwise, there's no way around getting lots of headers, and that is a
  huge hit with big folders.
  
  That's the theory; the practice seems uglier.  I tried mulberry, which
  is supposed to make very good use of IMAP.  In many ways it does, but
  its use of server-side threading doesn't work too well.  It seemed to
  get the response back quickly from the server, but then take minutes
  processing and parsing this response.  Furthermore, every time one
  scrolls the window, it does it all over again.
  
  The other problem with server-side threading is that it threads
  everything.  In principle it might make more sense to retrieve a window
  of headers and thread them.
 
 actually, I think this is what Mulberry is doing - or at least that
 would make sense if it has to re-fetch thread info when you scroll.
My impression, based partly on what a (the?) Mulberry developer said, is
that Mulberry gets the entire threaded list from the server.  The time
seems to go into the post-processing, and apparently this gets redone
every time you scroll.  (The developer thought it was time parsing the
response string from the server, though I would think building up a
virtual message list window is a more likely culprit).

BTW, in case people aren't aware, Mulberry was recently open-sourced.
 
  
  The window might not include all items in the thread, but that seems a
  reasonable trade-off to me in return for decent performance.  One could
  do a more thorough job in the background and update when it finished.
 
 but then you have items in the list moving around as more thread info
 becomes available which would be annoying to users, imho.
Worst case the display of the results could be deferred until something
else, like a user hitting a scrollbar, causes a display update.  But I
don't see how this case differs from other things that might change the
display, such as the arrival of new mail.

Ross

___
Evolution-hackers mailing list
Evolution-hackers@gnome.org
http://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] Introduction and Questions

2007-06-08 Thread Jeffrey Stedfast
On Fri, 2007-06-08 at 15:13 -0700, Ross Boylan wrote:
 On Fri, 2007-06-08 at 17:11 -0400, Jeffrey Stedfast wrote:
  On Fri, 2007-06-08 at 12:23 -0700, Ross Boylan wrote:
   On Thu, 2007-06-07 at 09:25 -0400, Jeffrey Stedfast wrote:
it's not possible to do better w/o dropping features like message
threading.

In fact, the above minimalizing of header fetching already breaks the
quick context-menu vfolder on mailing-list and filter on
mailing-list features as well as making vfoldering on mailing-list
oodles slower (if it even still works after disabling the headers) 
   What about letting the server do the threading, if it can?
  
  1. not all servers support it, so we need to implement threading
  client-side anyway
  
  2. not all servers do it /correctly/, so we'd need to have a lookup
  table of server idents so we knew when to fallback (note that every mail
  client I know of that tried to be smart and let the server do the
  threading has lived to regret making this decision)
 The parenthetical remark is pretty interesting.

(Balsa is one such client, if you are interested)

  
  3. if threading behavior is different from other accounts, it will feel
  weird to the user.
 All good points.  I should explain I'm thinking of a mode that might
 kick in only on large mailboxes, or as a user option or something like
 that.  My main concern is that with very large folders most mail cients,
 including evolution, become sluggish to the point of being unusable.
 For many of the clients life is much uglier during the initial contact
 with the folders than later.  For example, when I move a message into a
 folder I haven't opened or operated on in evolution, I'm noticing long
 delays (like  1 minute) while it does something (fetch headers and
 create an index?  whatever it is uses some, but not all the CPU).

yea, it's creating a summary.

IMHO, the design needs to change such that a CamelFolder object need not
load the summary when instantiated. If that gets implemented, then
there's no need to hit the server to sync summary info before the object
gets returned to the caller who requested it, thus allowing appends w/o
this particular slowness.

This is all part of my camel architecture redesign ideas that I had
started implementing in libspruce as a testbed. I think Michael Zucchi
even started to implement this in the disk-summary branch (it basically
boiled down to ::open() and ::close() methods on a CamelFolder object)

 
 In this context an responsive but imperfect solution is better than an
 unresponsive perfect one.

I would argue that it's not a perfect solution if it's unresponsive :)

 
 For example, if evo relies on the server's threads then it will
 obviously inherit their behavior, warts and all.  In general I agree
 with the point, made recently on this list, that it's desirable for
 clients to shield users from server problems (and most users don't know
 and don't care that the the real problem is with the server).  But if
 doing so takes forever, some may be willing to give up on that.
 
 Whether relaying on server side threading, which currently only operates
 on the whole mailbox, can solve the performance problems is another
 issue.

it doesn't operate on the whole mailbox, it operates on a seqid or uid
set afaik.

been a while since I've read the specs



___
Evolution-hackers mailing list
Evolution-hackers@gnome.org
http://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] Introduction and Questions

2007-06-08 Thread Ross Boylan
On Fri, 2007-06-08 at 18:27 -0400, Jeffrey Stedfast wrote:
 On Fri, 2007-06-08 at 15:13 -0700, Ross Boylan wrote:
  On Fri, 2007-06-08 at 17:11 -0400, Jeffrey Stedfast wrote:
   On Fri, 2007-06-08 at 12:23 -0700, Ross Boylan wrote:
.
  All good points.  I should explain I'm thinking of a mode that might
  kick in only on large mailboxes, or as a user option or something like
  that.  My main concern is that with very large folders most mail cients,
  including evolution, become sluggish to the point of being unusable.
  For many of the clients life is much uglier during the initial contact
  with the folders than later.  For example, when I move a message into a
  folder I haven't opened or operated on in evolution, I'm noticing long
  delays (like  1 minute) while it does something (fetch headers and
  create an index?  whatever it is uses some, but not all the CPU).
 
 yea, it's creating a summary.
 
 IMHO, the design needs to change such that a CamelFolder object need not
 load the summary when instantiated. If that gets implemented, then
 there's no need to hit the server to sync summary info before the object
 gets returned to the caller who requested it, thus allowing appends w/o
 this particular slowness.
 
 This is all part of my camel architecture redesign ideas that I had
 started implementing in libspruce as a testbed. I think Michael Zucchi
 even started to implement this in the disk-summary branch (it basically
 boiled down to ::open() and ::close() methods on a CamelFolder object)
 
Why does it need to create a CamelFolder for the destination at all,
assuming I keep the focus on the source folder?

Actually, I guess I don't know if IMAP copy/delete is happening behind
the scenes, or if camel is creating the message from scratch in the
target folder.  The latter is a lot more work (though probably easier to
implement).

Second question: even if it creates a folder, does it need to stick
around for the folder creation to finish?  I think I remember seeing
that camel was single-threaded, relying on the client app to do
threading.  Would there be a way to multi-thread this somewhere (either
in camel or in the app)?  Obviously doing so would complicate things,
because at some point one might need to block (e.g., if I move a message
from folder A to B and then switch the UI to look at B).

  
  In this context an responsive but imperfect solution is better than an
  unresponsive perfect one.
 
 I would argue that it's not a perfect solution if it's unresponsive :)
For sure.
 
  
  For example, if evo relies on the server's threads then it will
  obviously inherit their behavior, warts and all.  In general I agree
  with the point, made recently on this list, that it's desirable for
  clients to shield users from server problems (and most users don't know
  and don't care that the the real problem is with the server).  But if
  doing so takes forever, some may be willing to give up on that.
  
  Whether relaying on server side threading, which currently only operates
  on the whole mailbox, can solve the performance problems is another
  issue.
 
 it doesn't operate on the whole mailbox, it operates on a seqid or uid
 set afaik.
 
 been a while since I've read the specs
 
I was mistaken about threading being only for the whole box.  I just
looked at
http://www.ietf.org/internet-drafts/draft-ietf-imapext-sort-18.txt and
see you can add additional search specifications to limit your thread
request.  Of course, clients might still ask for all the threads.

Though I've mostly focused on responsiveness, the index the whole
folder strategy also impacts bandwidth (as Philip pointed out) and
local disk useage.

Ross

___
Evolution-hackers mailing list
Evolution-hackers@gnome.org
http://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] Introduction and Questions

2007-06-08 Thread Jeffrey Stedfast
On Fri, 2007-06-08 at 16:02 -0700, Ross Boylan wrote:
 On Fri, 2007-06-08 at 18:27 -0400, Jeffrey Stedfast wrote:
  On Fri, 2007-06-08 at 15:13 -0700, Ross Boylan wrote:
   On Fri, 2007-06-08 at 17:11 -0400, Jeffrey Stedfast wrote:
On Fri, 2007-06-08 at 12:23 -0700, Ross Boylan wrote:
 .
   All good points.  I should explain I'm thinking of a mode that might
   kick in only on large mailboxes, or as a user option or something like
   that.  My main concern is that with very large folders most mail cients,
   including evolution, become sluggish to the point of being unusable.
   For many of the clients life is much uglier during the initial contact
   with the folders than later.  For example, when I move a message into a
   folder I haven't opened or operated on in evolution, I'm noticing long
   delays (like  1 minute) while it does something (fetch headers and
   create an index?  whatever it is uses some, but not all the CPU).
  
  yea, it's creating a summary.
  
  IMHO, the design needs to change such that a CamelFolder object need not
  load the summary when instantiated. If that gets implemented, then
  there's no need to hit the server to sync summary info before the object
  gets returned to the caller who requested it, thus allowing appends w/o
  this particular slowness.
  
  This is all part of my camel architecture redesign ideas that I had
  started implementing in libspruce as a testbed. I think Michael Zucchi
  even started to implement this in the disk-summary branch (it basically
  boiled down to ::open() and ::close() methods on a CamelFolder object)
  
 Why does it need to create a CamelFolder for the destination at all,
 assuming I keep the focus on the source folder?

because you need both a source and a destination folder to move the
message(s) to?

kinda hard to move messages to a folder if you don't know which folder
to move them to, don't you agree? :)

 
 Actually, I guess I don't know if IMAP copy/delete is happening behind
 the scenes, or if camel is creating the message from scratch in the
 target folder.  The latter is a lot more work (though probably easier to
 implement).

it uses imap COPY and mark-as-deleted

 
 Second question: even if it creates a folder, does it need to stick
 around for the folder creation to finish?  I think I remember seeing
 that camel was single-threaded

not true...

 , relying on the client app to do
 threading.  Would there be a way to multi-thread this somewhere (either
 in camel or in the app)?  Obviously doing so would complicate things,
 because at some point one might need to block (e.g., if I move a message
 from folder A to B and then switch the UI to look at B).

okay, I think you need to familiarize yourself with Camel's API before
we discuss anything further :)

 
   
   In this context an responsive but imperfect solution is better than an
   unresponsive perfect one.
  
  I would argue that it's not a perfect solution if it's unresponsive :)
 For sure.
  
   
   For example, if evo relies on the server's threads then it will
   obviously inherit their behavior, warts and all.  In general I agree
   with the point, made recently on this list, that it's desirable for
   clients to shield users from server problems (and most users don't know
   and don't care that the the real problem is with the server).  But if
   doing so takes forever, some may be willing to give up on that.
   
   Whether relaying on server side threading, which currently only operates
   on the whole mailbox, can solve the performance problems is another
   issue.
  
  it doesn't operate on the whole mailbox, it operates on a seqid or uid
  set afaik.
  
  been a while since I've read the specs
  
 I was mistaken about threading being only for the whole box.  I just
 looked at
 http://www.ietf.org/internet-drafts/draft-ietf-imapext-sort-18.txt and
 see you can add additional search specifications to limit your thread
 request.  Of course, clients might still ask for all the threads.

when you think about it, it's really the only way to do it. The only way
to figure out which UIDs to display in the message-list 'window' in
threaded mode depends on being able to calculate which rows to show. To
do this, you need the whole thread model since you aren't able to
request a partial thread-model based on view row indexes (which would be
ideal)

 
 Though I've mostly focused on responsiveness, the index the whole
 folder strategy also impacts bandwidth (as Philip pointed out) and
 local disk useage.

disk usage isn't that much, it's something like 2% of the size of the
mailbox.

also, unless you want to repeatedly request the summary info from the
server each session, then you need to cache it. You also need to cache
it for disconnected operation.

Mobile-device users might not care to have this, but a desktop client
requires it.

That said, Camel is pluggable... someone could easily write a
mobile-imap provider that works the way tnymail needs it to work.

Jeff

Re: [Evolution-hackers] Introduction and Questions

2007-06-08 Thread Ross Boylan
On Fri, 2007-06-08 at 20:22 -0400, Jeffrey Stedfast wrote:
  Why does it need to create a CamelFolder for the destination at all,
  assuming I keep the focus on the source folder?
 
 because you need both a source and a destination folder to move the
 message(s) to?
 
 kinda hard to move messages to a folder if you don't know which folder
 to move them to, don't you agree? :)
All you need for IMAP is the name of the target folder, not the fuller
representation that CamelFolder provides.  In particular, there's no
need to fetch the headers from the destination folder, index them, etc.

As you observe, I'm not too familiar with camel's internals: just a
quick look at the docs and a little debugging.  So I don't know if such
a minimalist approach would play very well in that framework.

Ross

___
Evolution-hackers mailing list
Evolution-hackers@gnome.org
http://mail.gnome.org/mailman/listinfo/evolution-hackers