Re: [Evolution-hackers] Introduction and Questions

2007-06-11 Thread Ross Boylan
I'm preserving the exchange for context; my responses are sprinkled
below.
On Sun, 2007-06-10 at 23:47 -0400, Jeffrey Stedfast wrote:
 On Sun, 2007-06-10 at 15:55 -0700, Ross Boylan wrote:
  On Fri, 2007-06-08 at 20:22 -0400, Jeffrey Stedfast wrote:

Second question: even if it creates a folder, does it need to stick
around for the folder creation to finish?  I think I remember seeing
that camel was single-threaded
   
   not true...
 
 I should have been clearer here:
 
 yes, it needs to wait for the folder object instantiation to finish (the
 function returns a pointer to the instantiated object, afterall).
And your proposal with open on that object is designed to decouple
creation of the object from creation of the summary, right?  When I said
folder creation I was thinking of the whole get headers and make index
process.
 
 not true that camel is single-threaded - it has a few of its own threads
 going - most notably is the thread for filtering incoming mail for
 providers like imap and maildir (they share the same thread).
 
   
, relying on the client app to do
threading.  Would there be a way to multi-thread this somewhere
   (either
in camel or in the app)?  Obviously doing so would complicate
   things,
because at some point one might need to block (e.g., if I move a
   message
from folder A to B and then switch the UI to look at B).
   
   okay, I think you need to familiarize yourself with Camel's API before
   we discuss anything further :)
   
  http://www.go-evolution.org/Mail_Threading begins
  The Camel API is blocking and synchronous
  although it then goes on to qualify this:
  Custom Threads
  Two tasks inside Camel use threads internally in order to implement a
  cancellable api ontop of a non-cancellable one.
  
  Further
  Mail-OPS
  mail-ops.h contains asynchronous versions of varous Camel api's
  It sounds as if mail-ops is outside of Camel, however.
 
 it is external to camel, yes.
 
  
  So it sounds as if Camel could (in principle) respond to a move request
  by issuing the appropriate IMAP command and then, starting a thread to
  do the other activities (indexing the target folder and deleting the the
  message from the  source folder) and return.
 
 it could, yes, but it'd need a way to report an error unless it waited
 for the operations to finish before returning. For moving mail, you
 typically want to know that it succeeded :)
 
 all of the current camel APIs are designed such that the caller expects
 that when the function returns, the operation has either succeeded or
 failed (where the failure would be set on the CamelException argument).
 
It would then block on
  operations that attempted to access the target folder until the other
  operations completed.
 
 yes, this is true... well, the way folders are implemented at this time
 anyway...
 
  
  I think this could be called a syncronous API, though perhaps that's a
  stretch.
 
 it is indeed a synchronous API :)
Syncronous, but it fails the you know if you've succeeded when the
function returns test.
 
  
  On the other hand, http://www.go-evolution.org/Camel.Operation does not
  sound like a bocking syncronous API at all, so maybe the statement
  quoted at the top is just obsolete?
 
 Camel code is written under the assumption that it is likely it is being
 used in a thread, so it has CamelOperation as a means to:
 
 1. report the progress
 2. cancel operations (typically i/o)
 
 what happens is Evolution has a thread-pool with a CamelOperation object
 for each of the threads the mailer creates so that it can cancel
 operations and/or get progress updates.
This part of the camel API is not syncronous, right?
 
  
  So, first of all I'm confused about the nature of Camel's API and
  operation as far as threading and syncronicity.  
  
  Second, I don't have a sense of whether its features are historical
  accidents (camel was implemented in a simple way and evo then used it as
  it was)
 
 evolution mailer was originally not multi-threaded if my recollection
 serves correctly, although around the time of the first 0.0 release
 threading was added (about 7 years ago).
 
 camel and evolution were developed together, so their designs evolved
 together.
Why was camel separated out at all?

Side note: when I first was trying to figure out what was going on, I
looked in the evo sources and discovered the code wasn't there, but was
in the evo-data-server.  This was confusing because, at least on debian,
the latter was described as providing calendar and address book
functions.  I think the Debian split mirrors the what I see in your
repository.  My current guess is that some of the code in
evolution-data-server (the directory) is really for evolution, and not
for the evo-data-server application.
 
   or the result of some deliberate design decisions.  Blocking
  syncronous operations are simpler to implement,
 
 yep
 
   to use,
 
 yep
 
   to debug,
 
 yep, altho with threads thrown 

Re: [Evolution-hackers] Introduction and Questions

2007-06-11 Thread Ross Boylan
On Mon, 2007-06-11 at 09:46 -0700, Ross Boylan wrote:
 Speaking of complexity, evolution/camel (and maybe Gnome?) do a kind
 of
 objects with C.  Why not just use C++?  Was the C++ toolchain too
 unreliable at the start, or was there some other reason? 
I didn't mean to exclude other alternatives.  The general question was
if you're doing objects anyway, why not use a language that already
supports them?
-- 
Ross Boylan  wk:  (415) 514-8146
185 Berry St #5700   [EMAIL PROTECTED]
Dept of Epidemiology and Biostatistics   fax: (415) 514-8150
University of California, San Francisco
San Francisco, CA 94107-1739 hm:  (415) 550-1062

___
Evolution-hackers mailing list
Evolution-hackers@gnome.org
http://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] Introduction and Questions

2007-06-11 Thread Jeffrey Stedfast
On Mon, 2007-06-11 at 09:46 -0700, Ross Boylan wrote:
 I'm preserving the exchange for context; my responses are sprinkled
 below.
 On Sun, 2007-06-10 at 23:47 -0400, Jeffrey Stedfast wrote:
  On Sun, 2007-06-10 at 15:55 -0700, Ross Boylan wrote:
   On Fri, 2007-06-08 at 20:22 -0400, Jeffrey Stedfast wrote:
 
 Second question: even if it creates a folder, does it need to stick
 around for the folder creation to finish?  I think I remember seeing
 that camel was single-threaded

not true...
  
  I should have been clearer here:
  
  yes, it needs to wait for the folder object instantiation to finish (the
  function returns a pointer to the instantiated object, afterall).
 And your proposal with open on that object is designed to decouple
 creation of the object from creation of the summary, right?

correct, the ::open() would load the summary from disk and (if the store
is network based), sync state with the server (find out about messages
which were deleted since last session, flag changes, new messages, etc)

   When I said
 folder creation I was thinking of the whole get headers and make index
 process.

ok

  
  not true that camel is single-threaded - it has a few of its own threads
  going - most notably is the thread for filtering incoming mail for
  providers like imap and maildir (they share the same thread).
  

 , relying on the client app to do
 threading.  Would there be a way to multi-thread this somewhere
(either
 in camel or in the app)?  Obviously doing so would complicate
things,
 because at some point one might need to block (e.g., if I move a
message
 from folder A to B and then switch the UI to look at B).

okay, I think you need to familiarize yourself with Camel's API before
we discuss anything further :)

   http://www.go-evolution.org/Mail_Threading begins
   The Camel API is blocking and synchronous
   although it then goes on to qualify this:
   Custom Threads
   Two tasks inside Camel use threads internally in order to implement a
   cancellable api ontop of a non-cancellable one.
   
   Further
   Mail-OPS
   mail-ops.h contains asynchronous versions of varous Camel api's
   It sounds as if mail-ops is outside of Camel, however.
  
  it is external to camel, yes.
  
   
   So it sounds as if Camel could (in principle) respond to a move request
   by issuing the appropriate IMAP command and then, starting a thread to
   do the other activities (indexing the target folder and deleting the the
   message from the  source folder) and return.
  
  it could, yes, but it'd need a way to report an error unless it waited
  for the operations to finish before returning. For moving mail, you
  typically want to know that it succeeded :)
  
  all of the current camel APIs are designed such that the caller expects
  that when the function returns, the operation has either succeeded or
  failed (where the failure would be set on the CamelException argument).
  
 It would then block on
   operations that attempted to access the target folder until the other
   operations completed.
  
  yes, this is true... well, the way folders are implemented at this time
  anyway...
  
   
   I think this could be called a syncronous API, though perhaps that's a
   stretch.
  
  it is indeed a synchronous API :)
 Syncronous, but it fails the you know if you've succeeded when the
 function returns test.

most of the camel APIs don't fail that test

  
   
   On the other hand, http://www.go-evolution.org/Camel.Operation does not
   sound like a bocking syncronous API at all, so maybe the statement
   quoted at the top is just obsolete?
  
  Camel code is written under the assumption that it is likely it is being
  used in a thread, so it has CamelOperation as a means to:
  
  1. report the progress
  2. cancel operations (typically i/o)
  
  what happens is Evolution has a thread-pool with a CamelOperation object
  for each of the threads the mailer creates so that it can cancel
  operations and/or get progress updates.
 This part of the camel API is not syncronous, right?

in so much as doing camel_operation_cancel() returning doesn't mean that
the other thread has stopped, correct. Basically it just means that
you've sent a command along a pipe to the thread to cancel at its
earliest convenience.

  
   
   So, first of all I'm confused about the nature of Camel's API and
   operation as far as threading and syncronicity.  
   
   Second, I don't have a sense of whether its features are historical
   accidents (camel was implemented in a simple way and evo then used it as
   it was)
  
  evolution mailer was originally not multi-threaded if my recollection
  serves correctly, although around the time of the first 0.0 release
  threading was added (about 7 years ago).
  
  camel and evolution were developed together, so their designs evolved
  together.
 Why was camel separated out at all?

good model/view split :) you 

Re: [Evolution-hackers] Introduction and Questions

2007-06-11 Thread Ross Boylan
On Mon, 2007-06-11 at 14:39 -0400, Jeffrey Stedfast wrote:
So it sounds as if Camel could (in principle) respond to a move
 request
by issuing the appropriate IMAP command and then, starting a
 thread to
do the other activities (indexing the target folder and deleting
 the the
message from the  source folder) and return.
   
   it could, yes, but it'd need a way to report an error unless it
 waited
   for the operations to finish before returning. For moving mail,
 you
   typically want to know that it succeeded :)
   
   all of the current camel APIs are designed such that the caller
 expects
   that when the function returns, the operation has either succeeded
 or
   failed (where the failure would be set on the CamelException
 argument).
   
  It would then block on
operations that attempted to access the target folder until the
 other
operations completed.
   
   yes, this is true... well, the way folders are implemented at this
 time
   anyway...
   

I think this could be called a syncronous API, though perhaps
 that's a
stretch.
   
   it is indeed a synchronous API :)
  Syncronous, but it fails the you know if you've succeeded when the
  function returns test.
 
 most of the camel APIs don't fail that test
 
This was in the context of something I was thinking of (So it sounds as
if Camel could ).  The feature/change I was contemplating sounds as
if it would be a bad idea because it violates the usual model for camel
calls.  Probably either a smaller change (keeping the
blocking/syncronous/useful return values in place) or a bigger one
(general asyncronicity) would be a better idea--at least in a perfect
world of infinite coding time.:)  In fact, your idea with folder open
preserves the expected features of the camel API.
-- 
Ross Boylan  wk:  (415) 514-8146
185 Berry St #5700   [EMAIL PROTECTED]
Dept of Epidemiology and Biostatistics   fax: (415) 514-8150
University of California, San Francisco
San Francisco, CA 94107-1739 hm:  (415) 550-1062

___
Evolution-hackers mailing list
Evolution-hackers@gnome.org
http://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] Introduction and Questions

2007-06-10 Thread Jeffrey Stedfast
On Sun, 2007-06-10 at 11:39 +0300, Philip Van Hoof wrote:
 On Sat, 2007-06-09 at 08:14 -0400, Jeffrey Stedfast wrote:
 
  server says:
  
  * 1 EXPUNGE
  
  camel-imap-summary does:
  
  g_ptr_array_remove_index (messages, seqid - 1);
 
 In imap_rescan, for example in case a message got removed by another
 E-mail client while this E-mail client was not online.
 

A hash table won't solve this... 

Jeff


___
Evolution-hackers mailing list
Evolution-hackers@gnome.org
http://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] Introduction and Questions

2007-06-10 Thread Ross Boylan
On Fri, 2007-06-08 at 20:22 -0400, Jeffrey Stedfast wrote:
  [Ross wrote]
  Second question: even if it creates a folder, does it need to stick
  around for the folder creation to finish?  I think I remember seeing
  that camel was single-threaded
 
 not true...
 
  , relying on the client app to do
  threading.  Would there be a way to multi-thread this somewhere
 (either
  in camel or in the app)?  Obviously doing so would complicate
 things,
  because at some point one might need to block (e.g., if I move a
 message
  from folder A to B and then switch the UI to look at B).
 
 okay, I think you need to familiarize yourself with Camel's API before
 we discuss anything further :)
 
http://www.go-evolution.org/Mail_Threading begins
The Camel API is blocking and synchronous
although it then goes on to qualify this:
Custom Threads
Two tasks inside Camel use threads internally in order to implement a
cancellable api ontop of a non-cancellable one.

Further
Mail-OPS
mail-ops.h contains asynchronous versions of varous Camel api's
It sounds as if mail-ops is outside of Camel, however.

So it sounds as if Camel could (in principle) respond to a move request
by issuing the appropriate IMAP command and then, starting a thread to
do the other activities (indexing the target folder and deleting the the
message from the  source folder) and return.  It would then block on
operations that attempted to access the target folder until the other
operations completed.

I think this could be called a syncronous API, though perhaps that's a
stretch.

On the other hand, http://www.go-evolution.org/Camel.Operation does not
sound like a bocking syncronous API at all, so maybe the statement
quoted at the top is just obsolete?

So, first of all I'm confused about the nature of Camel's API and
operation as far as threading and syncronicity.  

Second, I don't have a sense of whether its features are historical
accidents (camel was implemented in a simple way and evo then used it as
it was) or the result of some deliberate design decisions.  Blocking
syncronous operations are simpler to implement, to use, to debug, and to
understand, so they clearly have some advantages.   But it seems that
the entire application (evolution) does not have that character, so the
benefits of that simplicity end up lost anyway.
___
Evolution-hackers mailing list
Evolution-hackers@gnome.org
http://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] Introduction and Questions

2007-06-09 Thread Philip Van Hoof
On Fri, 2007-06-08 at 20:45 -0400, Jeffrey Stedfast wrote:
 On Fri, 2007-06-08 at 18:34 +0300, Philip Van Hoof wrote:
  The imap4 project is making things look better, though all in all it's
  still much of the same (blocking and waiting for results, in stead of
  letting the server do most the work and do pipelining).
 
 adding pipelining support would not be difficult to do, actually.

I noticed that, indeed. Last time I was reading it, I did notice a few
things in its code that would make it difficult.

All in all, the ideas behind the 'imap4' way of working (with the IMAP
server) are indeed a lot better than the 'imap' code.

Maybe it would be a good idea to bring features like condstore and idle
to it? If there are some more people interested in this, I would
probably help out too. Unlike the stuff that I have put in the 'imap'
code, this should be done in a proper non-hackish way then, imo.

What I disliked most about Camel's 'imap' code, though, is the fact that
the sequences have to correspond to the array indexes of the
CamelFolderSummary. It sounds like it would have been more easy if that
was a key in a hashtable.

For example if a message got expunged that had a sequence value in the
beginning of the folder, it right now more or less means that the entire
folder has to be re-synchronised. While the majority of the local data
can be perfectly corrected in stead too.

Luckily very few people often remove things in the beginning of their
mail folders (since those E-mails are usually the older emails).

I have some ideas on having blocks of mmap data for CamelFolderSummary,
that for example contain ~ 1000 items per block, and having reference
counting per block.

That way I could create a vfolder-like feature that keeps only the
blocks alive that contain summary items that matched the query. In stead
of having to keep all the memory of each folder in the vfolder alive.

I have a prototype of this somewhat working, although done extremely
hackish too. So at some point I'm going to rewrite this first.

Another benefit is that the blocks will never have to be rewritten:
removals are simply marked until  50% of the block is removed (and then
the entire block is rewritten), flag changes are stored in a different
file and on push-email events or a summary download, the latest headers
are kept in normal memory until there are enough of them to start a new
block.

Rewriting the summary file truly is a hit on performance on some
devices. On Flash it also introduces some level wearing. Which is why
it's another benefit.



-- 
Philip Van Hoof, software developer
home: me at pvanhoof dot be 
gnome: pvanhoof at gnome dot org 
http://www.pvanhoof.be/blog




___
Evolution-hackers mailing list
Evolution-hackers@gnome.org
http://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] Introduction and Questions

2007-06-09 Thread Jeffrey Stedfast
On Sat, 2007-06-09 at 13:00 +0300, Philip Van Hoof wrote:
 What I disliked most about Camel's 'imap' code, though, is the fact that
 the sequences have to correspond to the array indexes of the
 CamelFolderSummary. It sounds like it would have been more easy if that
 was a key in a hashtable.

...???

are you serious?

 
 For example if a message got expunged that had a sequence value in the
 beginning of the folder, it right now more or less means that the entire
 folder has to be re-synchronised. While the majority of the local data
 can be perfectly corrected in stead too.

huh? it doesn't re-fetch anything, it simply removes the item from the
array at the index given in the untagged EXPUNGE notification.

server says:

* 1 EXPUNGE

camel-imap-summary does:

g_ptr_array_remove_index (messages, seqid - 1);

(seqid in this case would be '1', but keep in mind that IMAP seqids
start at 1 while in c, array indexes start at 0)

How would a hash table simplify this? It'd only serve to complicate
things because you'd have to re-key the entire hash table after each
EXPUNGE notification. Not to mention it would consume a fair bit more
memory...

Jeff


___
Evolution-hackers mailing list
Evolution-hackers@gnome.org
http://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] Introduction and Questions

2007-06-08 Thread Philip Van Hoof
On Thu, 2007-06-07 at 09:25 -0400, Jeffrey Stedfast wrote:

 On Thu, 2007-06-07 at 10:56 +0300, Philip Van Hoof wrote:
 
  The best way is to ask for the ENVELOPE and the remaining info using the
  normal BODY.PEEK method.
 
 Have you actually ever tested this theory? or did you just pull this out
 of your proverbial?

I have. The ENVELOPE-way is slightly better because words like From:
and To: are not repeated each time. Although with COMPRESS and/or
support for deflate in the TLS/SSL library will probably compress that
difference away mostly.

Note I didn't test this with compression. I saw a difference of 5% to
10% depending on several factors in bandwidth saving. I usually test all
this with wireshark, which doesn't help a lot with compressed data. It
would be nice, indeed, to get some analysis of how data is being
compressed when compression by either COMPRESS or the SSL library is
being done.

Note that I haven't yet implemented COMPRESS in camel-lite, but I will
do this very soon (I have a CamelStreamGzip thingy ready for this).

On top of bandwidth, on pure speed a lot IMAP servers will return your
result faster if you only use ENVELOPE. That's because they optimise on
the fields that are in ENVELOPE (indexes, etc etc).

But then again, there's not enough in ENVELOPE to support all of
Evolution's features, so you'll still have to use BODY.PEEK which will
most likely slowdown everything back again.

 You should note that Thunderbird does not use ENVELOPE, probably because
 they found it was LESS efficient to fetch the ENVELOPE +
 BODY.PEEK[HEADER.FIELDS (REFERENCES CONTENT-TYPE)] on many existing IMAP
 servers in the wild.

Probably, indeed.

 Having myself tested this in the past, I don't recall it making much
 difference one way or the other.

On bandwidth, it does. Especially when you have a lot of messages in
some folders and the server is not compressing things (no COMPRESS nor
TLS/SSL compressing things for you).

 Sadly using ENVELOPE breaks on at least one proprietary IMAP server
 which occasionally likes to send back responses with NIL as each and
 every item, at least until a client has issued a BODY.PEEK fetch
 request.

Sounds like a bug in that server implementation.

   It should be possible to specify per folder
  which of the headers are to be fetched, to make it really efficient.
 
 how do you propose calculating the list of headers each folder should
 request? which headers would you eliminate conditionally?

I'd probably start with the references in case the user doesn't want his
headers-view to be threaded, and fetch only the sequence number plus the
references header the first time threaded is enabled on that folder and
after that (while its enabled) for each summary-item update too.

A lot mobile devices's screens are simply not suitable for threaded view
anyway: introduces way to much wasted white space. Why fetch this
information and waste the user's GPRS bandwidth and therefore often his
money too? (for a lot people, GPRS bandwidth costs money per downloaded
megabyte)

In case the From is not displayed, there's no real need to fetch it
until the first time the column is added. Same for the dates (date sent
and date received, usually only one of both is used by the user). 

Etc etc etc.
 
  The imap4 implementation seems to have an ENVELOPE parser, so I guess
  either it does it already or it will use ENVELOPE in future.
 
 it fetches:
 
 ENVELOPE BODY.PEEK[HEADER.FIELDS (Content-Type References In-Reply-To)]
 
 and optionally the mailing-list headers
 
 I believe it fetches IN-REPLY-TO (which is included in ENVELOPE)
 explicitly because some IMAP servers tried to be smart and parse the
 IN-REPLY-TO header for us and didn't handle embedded phrases which
 certain broken free-software mailers like to send.
 
  
  For a mobile device, that works over GPRS, you for example are usually
  not interested (not at all) in the mailinglist specific headers.
 
 this is mobile-device specific tho, when I was implementing imap4 and
 people started trying to use it, it was their #1 complaint (which is why
 imap4 conditionally requests the mlist headers... tho that flag may be
 hard-coded to TRUE now, I'm not sure)

But some people don't use it. Although people more often complain about
missing features and not so much about that their E-mail client is
wasting their money on GPRS bandwidth, I still believe it's important to
support low bandwidth modes too. Also for Evolution, as Evolution is
often used with laptops.

I do agree, though, that Evolution has a focus difference compared to
what I'm doing with Camel. In my opinion is this why splitting Camel
from Evolution might help: we could then more easily introduce features
for letting Camel switch from low bandwidth mode, to Evolution mode and
back.

Right now any change that isn't absolutely 100% super pro and for
Evolution and Evolution only, gets ignored by you guys at Novell.

Usually the explanation goes like this: Yeah but Evolution 

Re: [Evolution-hackers] Introduction and Questions

2007-06-08 Thread Jeffrey Stedfast
On Fri, 2007-06-08 at 14:39 +0300, Philip Van Hoof wrote:
 On Thu, 2007-06-07 at 09:25 -0400, Jeffrey Stedfast wrote:
 
  On Thu, 2007-06-07 at 10:56 +0300, Philip Van Hoof wrote:
  
   The best way is to ask for the ENVELOPE and the remaining info using the
   normal BODY.PEEK method.
  
  Have you actually ever tested this theory? or did you just pull this out
  of your proverbial?
 
 I have. The ENVELOPE-way is slightly better because words like From:
 and To: are not repeated each time. Although with COMPRESS and/or
 support for deflate in the TLS/SSL library will probably compress that
 difference away mostly.

ENVELOPE alone is no doubt as fast or faster (hopefully faster) than
explicit header fetching (I expect most servers probably cache ENVELOPE
info, whereas explicit header fetching would have to be done by
re-parsing the message) and (probably) less processor intensive
server-side.

 
 Note I didn't test this with compression. I saw a difference of 5% to
 10% depending on several factors in bandwidth saving. I usually test all
 this with wireshark, which doesn't help a lot with compressed data. It
 would be nice, indeed, to get some analysis of how data is being
 compressed when compression by either COMPRESS or the SSL library is
 being done.
 
 Note that I haven't yet implemented COMPRESS in camel-lite, but I will
 do this very soon (I have a CamelStreamGzip thingy ready for this).
 
 On top of bandwidth, on pure speed a lot IMAP servers will return your
 result faster if you only use ENVELOPE. That's because they optimise on
 the fields that are in ENVELOPE (indexes, etc etc).

nod, which is why imap4 used ENVELOPE. I later had to add explicit
fetching of other headers, though, to maintain feature compat with imap.

 
 But then again, there's not enough in ENVELOPE to support all of
 Evolution's features, so you'll still have to use BODY.PEEK which will
 most likely slowdown everything back again.

indeed, and it doesn't appear to be faster trying to 
use ENVELOPE + explicit header fetching, /probably/ because the server
still has to spend time parsing the actual message off disk, so the
performance gain of using ENVELOPE for the (typically? cached) summary
info is negated - possibly made even slower (server has to hit 2
code-paths instead of just the 1), but bandwidth is still slightly
decreased, possibly making up for the increased processing time.

 
  You should note that Thunderbird does not use ENVELOPE, probably because
  they found it was LESS efficient to fetch the ENVELOPE +
  BODY.PEEK[HEADER.FIELDS (REFERENCES CONTENT-TYPE)] on many existing IMAP
  servers in the wild.
 
 Probably, indeed.
 
  Having myself tested this in the past, I don't recall it making much
  difference one way or the other.
 
 On bandwidth, it does. Especially when you have a lot of messages in
 some folders and the server is not compressing things (no COMPRESS nor
 TLS/SSL compressing things for you).

ok, fair enough... not sure how much you really cut out tho, address
lists are likely more bandwidth intensive in the ENVELOPE response than
in raw format.

To: Jeffrey Stedfast [EMAIL PROTECTED], Miguel de Icaza [EMAIL 
PROTECTED]\r\n

ENVELOPE equiv:

((Jeffrey Stedfast NIL fejj novell.com) (Miguel de Icaza NIL miguel 
ximian.com))

You can see that the ENVELOPE response is more verbose, so the savings
really depends on individual messages.

You do save on the command echoing from the server, tho...


all in all, yea, probably you save some bandwidth. For most users,
ENVELOPE is probably a win as far as bytes-sent-across-the-wire.

The biggest win for ENVELOPE, in my opinion, is not bandwidth but server
processing (measured in both time and resources). Unfortunately, if you
have to explicitly request headers anyway, this is completely negated :(

 
  Sadly using ENVELOPE breaks on at least one proprietary IMAP server
  which occasionally likes to send back responses with NIL as each and
  every item, at least until a client has issued a BODY.PEEK fetch
  request.
 
 Sounds like a bug in that server implementation.

yes, it is.

 
It should be possible to specify per folder
   which of the headers are to be fetched, to make it really efficient.
  
  how do you propose calculating the list of headers each folder should
  request? which headers would you eliminate conditionally?
 
 I'd probably start with the references in case the user doesn't want his
 headers-view to be threaded, and fetch only the sequence number plus the
 references header the first time threaded is enabled on that folder and
 after that (while its enabled) for each summary-item update too.
 
 A lot mobile devices's screens are simply not suitable for threaded view
 anyway: introduces way to much wasted white space. Why fetch this
 information and waste the user's GPRS bandwidth and therefore often his
 money too? (for a lot people, GPRS bandwidth costs money per downloaded
 megabyte)
 
 In case the From is not displayed, there's no 

Re: [Evolution-hackers] Introduction and Questions

2007-06-08 Thread Philip Van Hoof
On Fri, 2007-06-08 at 09:48 -0400, Jeffrey Stedfast wrote:

 I think the laptop problem is solved with the basic headers feature,
 at least as far as collecting new summary info is concerned.
 
 Syncing flags is another story, and where the real
 slowness/user-frustration lies.
 
 I'm sure David Woodhouse would agree here.

ps. The condstore work in camel-lite is the most easy part to migrate.

  Right now any change that isn't absolutely 100% super pro and for
  Evolution and Evolution only, gets ignored by you guys at Novell.
 
 I would argue that not to be the case. I would instead argue that the
 reason many of your patches haven't been incorporated into Camel is
 that they are hackish and/or tailored explicitly for /your/ needs, and
 not for the good of the many.

I agree that a lot of the things are hackish, some of the changes are
this way because there are some design problems in Camel that make
certain things not possible without such hacks.

For example the fact that non-blocking read()s are not possible by the
default CamelStream things, and that none of the existing code really
copes very well with non-blocking reads.

The imap4 project is making things look better, though all in all it's
still much of the same (blocking and waiting for results, in stead of
letting the server do most the work and do pipelining).

 That's not the one I was thinking of, actually... I was more thinking
 of the BODY(STRUCTURE) stuff.

Camel-lite doesn't use BODYSTRUCTURE, it only uses body.peek[0] for
doing a partial retrieval. I'm, however, planning to let the
CamelMimePart late fetch not-yet retrieved parts when needed (in stead
of always downloading the full, or only the body part of the message).

That would conflict with IMAP servers that don't do mime parsing right.

 Then don't be so negative. I'd hardly say that imap4 has a majority of
 its design wrong.

 Try some constructive criticism for a change...

You are right, let's try to criticise constructive. My apologises for
the negative way of putting that I don't like how imap4 is designed.


-- 
Philip Van Hoof, software developer
home: me at pvanhoof dot be 
gnome: pvanhoof at gnome dot org 
http://www.pvanhoof.be/blog




___
Evolution-hackers mailing list
Evolution-hackers@gnome.org
http://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] Introduction and Questions

2007-06-08 Thread Jeffrey Stedfast
On Fri, 2007-06-08 at 12:23 -0700, Ross Boylan wrote:
 On Thu, 2007-06-07 at 09:25 -0400, Jeffrey Stedfast wrote:
  it's not possible to do better w/o dropping features like message
  threading.
  
  In fact, the above minimalizing of header fetching already breaks the
  quick context-menu vfolder on mailing-list and filter on
  mailing-list features as well as making vfoldering on mailing-list
  oodles slower (if it even still works after disabling the headers) 
 What about letting the server do the threading, if it can?

1. not all servers support it, so we need to implement threading
client-side anyway

2. not all servers do it /correctly/, so we'd need to have a lookup
table of server idents so we knew when to fallback (note that every mail
client I know of that tried to be smart and let the server do the
threading has lived to regret making this decision)

3. if threading behavior is different from other accounts, it will feel
weird to the user.

   I realize
 not all can, and probably all will seem not as smart as they could be
 (or just different from how threading would be done in evolution).  But
 otherwise, there's no way around getting lots of headers, and that is a
 huge hit with big folders.
 
 That's the theory; the practice seems uglier.  I tried mulberry, which
 is supposed to make very good use of IMAP.  In many ways it does, but
 its use of server-side threading doesn't work too well.  It seemed to
 get the response back quickly from the server, but then take minutes
 processing and parsing this response.  Furthermore, every time one
 scrolls the window, it does it all over again.
 
 The other problem with server-side threading is that it threads
 everything.  In principle it might make more sense to retrieve a window
 of headers and thread them.

actually, I think this is what Mulberry is doing - or at least that
would make sense if it has to re-fetch thread info when you scroll.

 
 The window might not include all items in the thread, but that seems a
 reasonable trade-off to me in return for decent performance.  One could
 do a more thorough job in the background and update when it finished.

but then you have items in the list moving around as more thread info
becomes available which would be annoying to users, imho.

Jeff


___
Evolution-hackers mailing list
Evolution-hackers@gnome.org
http://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] Introduction and Questions

2007-06-08 Thread Ross Boylan
On Fri, 2007-06-08 at 17:11 -0400, Jeffrey Stedfast wrote:
 On Fri, 2007-06-08 at 12:23 -0700, Ross Boylan wrote:
  On Thu, 2007-06-07 at 09:25 -0400, Jeffrey Stedfast wrote:
   it's not possible to do better w/o dropping features like message
   threading.
   
   In fact, the above minimalizing of header fetching already breaks the
   quick context-menu vfolder on mailing-list and filter on
   mailing-list features as well as making vfoldering on mailing-list
   oodles slower (if it even still works after disabling the headers) 
  What about letting the server do the threading, if it can?
 
 1. not all servers support it, so we need to implement threading
 client-side anyway
 
 2. not all servers do it /correctly/, so we'd need to have a lookup
 table of server idents so we knew when to fallback (note that every mail
 client I know of that tried to be smart and let the server do the
 threading has lived to regret making this decision)
The parenthetical remark is pretty interesting.
 
 3. if threading behavior is different from other accounts, it will feel
 weird to the user.
All good points.  I should explain I'm thinking of a mode that might
kick in only on large mailboxes, or as a user option or something like
that.  My main concern is that with very large folders most mail cients,
including evolution, become sluggish to the point of being unusable.
For many of the clients life is much uglier during the initial contact
with the folders than later.  For example, when I move a message into a
folder I haven't opened or operated on in evolution, I'm noticing long
delays (like  1 minute) while it does something (fetch headers and
create an index?  whatever it is uses some, but not all the CPU).

In this context an responsive but imperfect solution is better than an
unresponsive perfect one.

For example, if evo relies on the server's threads then it will
obviously inherit their behavior, warts and all.  In general I agree
with the point, made recently on this list, that it's desirable for
clients to shield users from server problems (and most users don't know
and don't care that the the real problem is with the server).  But if
doing so takes forever, some may be willing to give up on that.

Whether relaying on server side threading, which currently only operates
on the whole mailbox, can solve the performance problems is another
issue.
 
I realize
  not all can, and probably all will seem not as smart as they could be
  (or just different from how threading would be done in evolution).  But
  otherwise, there's no way around getting lots of headers, and that is a
  huge hit with big folders.
  
  That's the theory; the practice seems uglier.  I tried mulberry, which
  is supposed to make very good use of IMAP.  In many ways it does, but
  its use of server-side threading doesn't work too well.  It seemed to
  get the response back quickly from the server, but then take minutes
  processing and parsing this response.  Furthermore, every time one
  scrolls the window, it does it all over again.
  
  The other problem with server-side threading is that it threads
  everything.  In principle it might make more sense to retrieve a window
  of headers and thread them.
 
 actually, I think this is what Mulberry is doing - or at least that
 would make sense if it has to re-fetch thread info when you scroll.
My impression, based partly on what a (the?) Mulberry developer said, is
that Mulberry gets the entire threaded list from the server.  The time
seems to go into the post-processing, and apparently this gets redone
every time you scroll.  (The developer thought it was time parsing the
response string from the server, though I would think building up a
virtual message list window is a more likely culprit).

BTW, in case people aren't aware, Mulberry was recently open-sourced.
 
  
  The window might not include all items in the thread, but that seems a
  reasonable trade-off to me in return for decent performance.  One could
  do a more thorough job in the background and update when it finished.
 
 but then you have items in the list moving around as more thread info
 becomes available which would be annoying to users, imho.
Worst case the display of the results could be deferred until something
else, like a user hitting a scrollbar, causes a display update.  But I
don't see how this case differs from other things that might change the
display, such as the arrival of new mail.

Ross

___
Evolution-hackers mailing list
Evolution-hackers@gnome.org
http://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] Introduction and Questions

2007-06-08 Thread Jeffrey Stedfast
On Fri, 2007-06-08 at 15:13 -0700, Ross Boylan wrote:
 On Fri, 2007-06-08 at 17:11 -0400, Jeffrey Stedfast wrote:
  On Fri, 2007-06-08 at 12:23 -0700, Ross Boylan wrote:
   On Thu, 2007-06-07 at 09:25 -0400, Jeffrey Stedfast wrote:
it's not possible to do better w/o dropping features like message
threading.

In fact, the above minimalizing of header fetching already breaks the
quick context-menu vfolder on mailing-list and filter on
mailing-list features as well as making vfoldering on mailing-list
oodles slower (if it even still works after disabling the headers) 
   What about letting the server do the threading, if it can?
  
  1. not all servers support it, so we need to implement threading
  client-side anyway
  
  2. not all servers do it /correctly/, so we'd need to have a lookup
  table of server idents so we knew when to fallback (note that every mail
  client I know of that tried to be smart and let the server do the
  threading has lived to regret making this decision)
 The parenthetical remark is pretty interesting.

(Balsa is one such client, if you are interested)

  
  3. if threading behavior is different from other accounts, it will feel
  weird to the user.
 All good points.  I should explain I'm thinking of a mode that might
 kick in only on large mailboxes, or as a user option or something like
 that.  My main concern is that with very large folders most mail cients,
 including evolution, become sluggish to the point of being unusable.
 For many of the clients life is much uglier during the initial contact
 with the folders than later.  For example, when I move a message into a
 folder I haven't opened or operated on in evolution, I'm noticing long
 delays (like  1 minute) while it does something (fetch headers and
 create an index?  whatever it is uses some, but not all the CPU).

yea, it's creating a summary.

IMHO, the design needs to change such that a CamelFolder object need not
load the summary when instantiated. If that gets implemented, then
there's no need to hit the server to sync summary info before the object
gets returned to the caller who requested it, thus allowing appends w/o
this particular slowness.

This is all part of my camel architecture redesign ideas that I had
started implementing in libspruce as a testbed. I think Michael Zucchi
even started to implement this in the disk-summary branch (it basically
boiled down to ::open() and ::close() methods on a CamelFolder object)

 
 In this context an responsive but imperfect solution is better than an
 unresponsive perfect one.

I would argue that it's not a perfect solution if it's unresponsive :)

 
 For example, if evo relies on the server's threads then it will
 obviously inherit their behavior, warts and all.  In general I agree
 with the point, made recently on this list, that it's desirable for
 clients to shield users from server problems (and most users don't know
 and don't care that the the real problem is with the server).  But if
 doing so takes forever, some may be willing to give up on that.
 
 Whether relaying on server side threading, which currently only operates
 on the whole mailbox, can solve the performance problems is another
 issue.

it doesn't operate on the whole mailbox, it operates on a seqid or uid
set afaik.

been a while since I've read the specs



___
Evolution-hackers mailing list
Evolution-hackers@gnome.org
http://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] Introduction and Questions

2007-06-08 Thread Ross Boylan
On Fri, 2007-06-08 at 18:27 -0400, Jeffrey Stedfast wrote:
 On Fri, 2007-06-08 at 15:13 -0700, Ross Boylan wrote:
  On Fri, 2007-06-08 at 17:11 -0400, Jeffrey Stedfast wrote:
   On Fri, 2007-06-08 at 12:23 -0700, Ross Boylan wrote:
.
  All good points.  I should explain I'm thinking of a mode that might
  kick in only on large mailboxes, or as a user option or something like
  that.  My main concern is that with very large folders most mail cients,
  including evolution, become sluggish to the point of being unusable.
  For many of the clients life is much uglier during the initial contact
  with the folders than later.  For example, when I move a message into a
  folder I haven't opened or operated on in evolution, I'm noticing long
  delays (like  1 minute) while it does something (fetch headers and
  create an index?  whatever it is uses some, but not all the CPU).
 
 yea, it's creating a summary.
 
 IMHO, the design needs to change such that a CamelFolder object need not
 load the summary when instantiated. If that gets implemented, then
 there's no need to hit the server to sync summary info before the object
 gets returned to the caller who requested it, thus allowing appends w/o
 this particular slowness.
 
 This is all part of my camel architecture redesign ideas that I had
 started implementing in libspruce as a testbed. I think Michael Zucchi
 even started to implement this in the disk-summary branch (it basically
 boiled down to ::open() and ::close() methods on a CamelFolder object)
 
Why does it need to create a CamelFolder for the destination at all,
assuming I keep the focus on the source folder?

Actually, I guess I don't know if IMAP copy/delete is happening behind
the scenes, or if camel is creating the message from scratch in the
target folder.  The latter is a lot more work (though probably easier to
implement).

Second question: even if it creates a folder, does it need to stick
around for the folder creation to finish?  I think I remember seeing
that camel was single-threaded, relying on the client app to do
threading.  Would there be a way to multi-thread this somewhere (either
in camel or in the app)?  Obviously doing so would complicate things,
because at some point one might need to block (e.g., if I move a message
from folder A to B and then switch the UI to look at B).

  
  In this context an responsive but imperfect solution is better than an
  unresponsive perfect one.
 
 I would argue that it's not a perfect solution if it's unresponsive :)
For sure.
 
  
  For example, if evo relies on the server's threads then it will
  obviously inherit their behavior, warts and all.  In general I agree
  with the point, made recently on this list, that it's desirable for
  clients to shield users from server problems (and most users don't know
  and don't care that the the real problem is with the server).  But if
  doing so takes forever, some may be willing to give up on that.
  
  Whether relaying on server side threading, which currently only operates
  on the whole mailbox, can solve the performance problems is another
  issue.
 
 it doesn't operate on the whole mailbox, it operates on a seqid or uid
 set afaik.
 
 been a while since I've read the specs
 
I was mistaken about threading being only for the whole box.  I just
looked at
http://www.ietf.org/internet-drafts/draft-ietf-imapext-sort-18.txt and
see you can add additional search specifications to limit your thread
request.  Of course, clients might still ask for all the threads.

Though I've mostly focused on responsiveness, the index the whole
folder strategy also impacts bandwidth (as Philip pointed out) and
local disk useage.

Ross

___
Evolution-hackers mailing list
Evolution-hackers@gnome.org
http://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] Introduction and Questions

2007-06-08 Thread Jeffrey Stedfast
On Fri, 2007-06-08 at 16:02 -0700, Ross Boylan wrote:
 On Fri, 2007-06-08 at 18:27 -0400, Jeffrey Stedfast wrote:
  On Fri, 2007-06-08 at 15:13 -0700, Ross Boylan wrote:
   On Fri, 2007-06-08 at 17:11 -0400, Jeffrey Stedfast wrote:
On Fri, 2007-06-08 at 12:23 -0700, Ross Boylan wrote:
 .
   All good points.  I should explain I'm thinking of a mode that might
   kick in only on large mailboxes, or as a user option or something like
   that.  My main concern is that with very large folders most mail cients,
   including evolution, become sluggish to the point of being unusable.
   For many of the clients life is much uglier during the initial contact
   with the folders than later.  For example, when I move a message into a
   folder I haven't opened or operated on in evolution, I'm noticing long
   delays (like  1 minute) while it does something (fetch headers and
   create an index?  whatever it is uses some, but not all the CPU).
  
  yea, it's creating a summary.
  
  IMHO, the design needs to change such that a CamelFolder object need not
  load the summary when instantiated. If that gets implemented, then
  there's no need to hit the server to sync summary info before the object
  gets returned to the caller who requested it, thus allowing appends w/o
  this particular slowness.
  
  This is all part of my camel architecture redesign ideas that I had
  started implementing in libspruce as a testbed. I think Michael Zucchi
  even started to implement this in the disk-summary branch (it basically
  boiled down to ::open() and ::close() methods on a CamelFolder object)
  
 Why does it need to create a CamelFolder for the destination at all,
 assuming I keep the focus on the source folder?

because you need both a source and a destination folder to move the
message(s) to?

kinda hard to move messages to a folder if you don't know which folder
to move them to, don't you agree? :)

 
 Actually, I guess I don't know if IMAP copy/delete is happening behind
 the scenes, or if camel is creating the message from scratch in the
 target folder.  The latter is a lot more work (though probably easier to
 implement).

it uses imap COPY and mark-as-deleted

 
 Second question: even if it creates a folder, does it need to stick
 around for the folder creation to finish?  I think I remember seeing
 that camel was single-threaded

not true...

 , relying on the client app to do
 threading.  Would there be a way to multi-thread this somewhere (either
 in camel or in the app)?  Obviously doing so would complicate things,
 because at some point one might need to block (e.g., if I move a message
 from folder A to B and then switch the UI to look at B).

okay, I think you need to familiarize yourself with Camel's API before
we discuss anything further :)

 
   
   In this context an responsive but imperfect solution is better than an
   unresponsive perfect one.
  
  I would argue that it's not a perfect solution if it's unresponsive :)
 For sure.
  
   
   For example, if evo relies on the server's threads then it will
   obviously inherit their behavior, warts and all.  In general I agree
   with the point, made recently on this list, that it's desirable for
   clients to shield users from server problems (and most users don't know
   and don't care that the the real problem is with the server).  But if
   doing so takes forever, some may be willing to give up on that.
   
   Whether relaying on server side threading, which currently only operates
   on the whole mailbox, can solve the performance problems is another
   issue.
  
  it doesn't operate on the whole mailbox, it operates on a seqid or uid
  set afaik.
  
  been a while since I've read the specs
  
 I was mistaken about threading being only for the whole box.  I just
 looked at
 http://www.ietf.org/internet-drafts/draft-ietf-imapext-sort-18.txt and
 see you can add additional search specifications to limit your thread
 request.  Of course, clients might still ask for all the threads.

when you think about it, it's really the only way to do it. The only way
to figure out which UIDs to display in the message-list 'window' in
threaded mode depends on being able to calculate which rows to show. To
do this, you need the whole thread model since you aren't able to
request a partial thread-model based on view row indexes (which would be
ideal)

 
 Though I've mostly focused on responsiveness, the index the whole
 folder strategy also impacts bandwidth (as Philip pointed out) and
 local disk useage.

disk usage isn't that much, it's something like 2% of the size of the
mailbox.

also, unless you want to repeatedly request the summary info from the
server each session, then you need to cache it. You also need to cache
it for disconnected operation.

Mobile-device users might not care to have this, but a desktop client
requires it.

That said, Camel is pluggable... someone could easily write a
mobile-imap provider that works the way tnymail needs it to work.

Jeff

Re: [Evolution-hackers] Introduction and Questions

2007-06-08 Thread Ross Boylan
On Fri, 2007-06-08 at 20:22 -0400, Jeffrey Stedfast wrote:
  Why does it need to create a CamelFolder for the destination at all,
  assuming I keep the focus on the source folder?
 
 because you need both a source and a destination folder to move the
 message(s) to?
 
 kinda hard to move messages to a folder if you don't know which folder
 to move them to, don't you agree? :)
All you need for IMAP is the name of the target folder, not the fuller
representation that CamelFolder provides.  In particular, there's no
need to fetch the headers from the destination folder, index them, etc.

As you observe, I'm not too familiar with camel's internals: just a
quick look at the docs and a little debugging.  So I don't know if such
a minimalist approach would play very well in that framework.

Ross

___
Evolution-hackers mailing list
Evolution-hackers@gnome.org
http://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] Introduction and Questions

2007-06-07 Thread Gilles Dartiguelongue
Le jeudi 07 juin 2007 à 01:53 +0300, Philip Van Hoof a écrit :
 Without immediately writing to disk work, the download by itself will
 consume around 120 MB of RAM, and will most likely fail due to network
 timeouts and other such problems (it'll take a while, since Evolution
 fetches a ridiculous large amount of headers for each message for
 building its summary).
 
isn't the imap-features plugin's goal to reduce/customize the amount of
downloaded headers ? Or is it that it's still not enough ?
-- 
Gilles Dartiguelongue [EMAIL PROTECTED]
EE/CS last year student, ESIEE


signature.asc
Description: Ceci est une partie de message	numériquement signée
___
Evolution-hackers mailing list
Evolution-hackers@gnome.org
http://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] Introduction and Questions

2007-06-07 Thread Philip Van Hoof
On Thu, 2007-06-07 at 08:25 +0200, Gilles Dartiguelongue wrote:
 Le jeudi 07 juin 2007 à 01:53 +0300, Philip Van Hoof a écrit :
  Without immediately writing to disk work, the download by itself will
  consume around 120 MB of RAM, and will most likely fail due to network
  timeouts and other such problems (it'll take a while, since Evolution
  fetches a ridiculous large amount of headers for each message for
  building its summary).
  
 isn't the imap-features plugin's goal to reduce/customize the amount of
 downloaded headers ? Or is it that it's still not enough ?

It improves the situation by setting your url-string to have the
basic_headers option. In the imap code of Camel, it will then ask for
less headers (but still way too much).

A major improvement it certainly isn't.

The best way is to ask for the ENVELOPE and the remaining info using the
normal BODY.PEEK method. It should be possible to specify per folder
which of the headers are to be fetched, to make it really efficient.

The imap4 implementation seems to have an ENVELOPE parser, so I guess
either it does it already or it will use ENVELOPE in future.

For a mobile device, that works over GPRS, you for example are usually
not interested (not at all) in the mailinglist specific headers.

It would also be possible to do it in multiple passes. For example get a
modest list of headers, and after that get more and more.

In any case, none of the current Evolution code implements consuming the
CONDSTORE capabilities of some modern IMAP servers (like MBox and
Cyrus).

CONDSTORE is really going to make an enormous difference in bandwidth
consumption with large folders. That's because you only need to fetch
the flags of the changed messages to synchronise flags.

Note that Tinymail's camel-lite implements all the needed solutions to
this bandwidth consumption problem. And that its code is, although a lot
of work because the Evolution maintainers didn't seem interested at the
time it was happening, definitely portable to upstream.

Its implementation include solutions for the headers, it immediately
saves the headers to disk and unlike Evolution's Camel it can resume
partial summary downloads, it wont peek memory allocation during
downloading, it implements Push E-mail using IMAP IDLE and it implements
CONDSTORE.

Although I'm definitely not satisfied with any of either Camel nor
camel-lite's IMAP code. The thing is that I'd much prefer a client-side
implementation that does proper pipelining. For example Infotrope,
Telomer and Polymer does this on an experimental basis (Infotrope is the
library).

ps. In my opinion is also the imap4 project getting the majority of its
design wrong. Although it looks better than the imap one, it's not the
best way to use an IMAP server. If a project is going to write an IMAP
client implementation 'today': they better do it right. A lot is
changing in IMAP today (Lemonade, MORG, etc). It's important to get it
right this time (the vast majority of E-mail clients totally suck at how
they use the IMAP server, including Evolution indeed).


-- 
Philip Van Hoof, software developer
home: me at pvanhoof dot be 
gnome: pvanhoof at gnome dot org 
http://www.pvanhoof.be/blog




___
Evolution-hackers mailing list
Evolution-hackers@gnome.org
http://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] Introduction and Questions

2007-06-07 Thread Sankar P
On Thu, 2007-06-07 at 10:56 +0300, Philip Van Hoof wrote:
 On Thu, 2007-06-07 at 08:25 +0200, Gilles Dartiguelongue wrote:
  Le jeudi 07 juin 2007 à 01:53 +0300, Philip Van Hoof a écrit :
   Without immediately writing to disk work, the download by itself will
   consume around 120 MB of RAM, and will most likely fail due to network
   timeouts and other such problems (it'll take a while, since Evolution
   fetches a ridiculous large amount of headers for each message for
   building its summary).
   
  isn't the imap-features plugin's goal to reduce/customize the amount of
  downloaded headers ? Or is it that it's still not enough ?
 
 It improves the situation by setting your url-string to have the
 basic_headers option. In the imap code of Camel, it will then ask for
 less headers (but still way too much).

May be way too much for a mobile client; but not for a desktop email
client. Sure you dont want your desktop mail client to have threading or
show mail-size or have such basic things ?

 
 A major improvement it certainly isn't.

Try comparing the performance of fetching all headers and basic_headers
only. Some crude data which I collected a year back is at
http://psankar.blogspot.com/2006/05/imap-performance.html

 
 The best way is to ask for the ENVELOPE and the remaining info using the
 normal BODY.PEEK method. It should be possible to specify per folder
 which of the headers are to be fetched, to make it really efficient.

Do you really think any user on earth will really be interested in
configuring what headers to fetch on a folder-basis ? 

 
 The imap4 implementation seems to have an ENVELOPE parser, so I guess
 either it does it already or it will use ENVELOPE in future.
 
 For a mobile device, that works over GPRS, you for example are usually
 not interested (not at all) in the mailinglist specific headers.

None of the mailing list headers will be fetched if you have chosen the
basic_headers option.

 
 It would also be possible to do it in multiple passes. For example get a
 modest list of headers, and after that get more and more.
 
 In any case, none of the current Evolution code implements consuming the
 CONDSTORE capabilities of some modern IMAP servers (like MBox and
 Cyrus).
 
 CONDSTORE is really going to make an enormous difference in bandwidth
 consumption with large folders. That's because you only need to fetch
 the flags of the changed messages to synchronise flags.
 
 Note that Tinymail's camel-lite implements all the needed solutions to
 this bandwidth consumption problem. And that its code is, although a lot
 of work because the Evolution maintainers didn't seem interested at the
 time it was happening, definitely portable to upstream.
 
 Its implementation include solutions for the headers, it immediately
 saves the headers to disk and unlike Evolution's Camel it can resume
 partial summary downloads, it wont peek memory allocation during
 downloading, it implements Push E-mail using IMAP IDLE and it implements
 CONDSTORE.
 
 Although I'm definitely not satisfied with any of either Camel nor
 camel-lite's IMAP code. The thing is that I'd much prefer a client-side
 implementation that does proper pipelining. For example Infotrope,
 Telomer and Polymer does this on an experimental basis (Infotrope is the
 library).
 
 ps. In my opinion is also the imap4 project getting the majority of its
 design wrong. Although it looks better than the imap one, it's not the
 best way to use an IMAP server. If a project is going to write an IMAP
 client implementation 'today': they better do it right. A lot is
 changing in IMAP today (Lemonade, MORG, etc). It's important to get it
 right this time (the vast majority of E-mail clients totally suck at how
 they use the IMAP server, including Evolution indeed).

Such a sweeping generalization !?

As you mentioned, IMAP (like any other technology) grows at a very
high-speed. What you consider as the right client implementation today
may become obsolete tomorrow. When the initial IMAP provider was written
there may not be a standard spec. for the PUSH/IDLE etc.

No application can have an intelligent design that will remain forever
satisfying all new needs/changes. Application designs should just
evolve, adapting to the new changes.


IIRC, you had a branch on Evolution's Camel to work on whatever changes
you wanted to make, so that you don't have to wait for the delay due to
review/maintainer.  Would love to see some of the improvements that you
have done ported there. So that it becomes easy for the maintainer to
approve them and bring onto trunk.

 
 
-- 
Sankar

 Novell, Inc. 
Software for the Open Enterprise*
http://www.novell.com
___
Evolution-hackers mailing list
Evolution-hackers@gnome.org
http://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] Introduction and Questions

2007-06-07 Thread Philip Van Hoof
On Thu, 2007-06-07 at 03:05 -0600, Sankar P wrote:
 On Thu, 2007-06-07 at 10:56 +0300, Philip Van Hoof wrote:

  
  It improves the situation by setting your url-string to have the
  basic_headers option. In the imap code of Camel, it will then ask for
  less headers (but still way too much).
 
 May be way too much for a mobile client; but not for a desktop email
 client. Sure you dont want your desktop mail client to have threading or
 show mail-size or have such basic things ?

When I'm using Evolution on a laptop, I feel very mobile too.

btw. This is a quote that I have from Federico, I think (remember) he
said that to me last GUADEC. Although

Very often I'm indeed using my laptop at an airport or something, using
GPRS. Why isn't Evolution suitable for this use-case?

I agree, though, that Tinymail's camel-lite memory improvements are less
of a priority for Evolution. It's bandwidth work, though, is important
in my opinion. Same for the Push E-mail capabilities.

  A major improvement it certainly isn't.
 
 Try comparing the performance of fetching all headers and basic_headers
 only. Some crude data which I collected a year back is at
 http://psankar.blogspot.com/2006/05/imap-performance.html

Of course, you can look at the code to see what differs between the
basic_headers mode and the normal one. And that will obviously make a
big difference (the selection of the query is simply a lot smaller).

But if Evolution is causing 900% of its bandwidth to be unnecessary, and
the basic_headers mode saves 400% of that, there is still 500% of it
that is too much. Which is the situation with basic_headers.

The basic_headers option also doesn't implement condstore, which is
quite important to safe bandwidth (if supported by the IMAP server).

  The best way is to ask for the ENVELOPE and the remaining info using the
  normal BODY.PEEK method. It should be possible to specify per folder
  which of the headers are to be fetched, to make it really efficient.
 
 Do you really think any user on earth will really be interested in
 configuring what headers to fetch on a folder-basis ? 

The application should figure that list out automatically, obviously. 
 
  ps. In my opinion is also the imap4 project getting the majority of its
  design wrong. Although it looks better than the imap one, it's not the
  best way to use an IMAP server. If a project is going to write an IMAP
  client implementation 'today': they better do it right. A lot is
  changing in IMAP today (Lemonade, MORG, etc). It's important to get it
  right this time (the vast majority of E-mail clients totally suck at how
  they use the IMAP server, including Evolution indeed).
 
 Such a sweeping generalization !?

Maybe, but I've seen quite a lot of their code, and they are indeed very
badly programmed. Most of the custom Camel providers are in a very bad
shape too, by the way.

And ... camel-lite, camel upstream and its standard providers are also
in a bad shape :(

 As you mentioned, IMAP (like any other technology) grows at a very
 high-speed. What you consider as the right client implementation today
 may become obsolete tomorrow. When the initial IMAP provider was written
 there may not be a standard spec. for the PUSH/IDLE etc.

This is true. I agree.

 No application can have an intelligent design that will remain forever
 satisfying all new needs/changes. Application designs should just
 evolve, adapting to the new changes.

I also agree with this.

 IIRC, you had a branch on Evolution's Camel to work on whatever changes
 you wanted to make, so that you don't have to wait for the delay due to
 review/maintainer.  Would love to see some of the improvements that you
 have done ported there. So that it becomes easy for the maintainer to
 approve them and bring onto trunk.

I think the best way forward is what Matthew and I share an opinion on:

To split Camel away from Evolution, and start getting as much of
camel-lite back into that upstream version by making the new function-
ality more and more pluggable.

It would probably take a few releases to get things in upstream, though.

To do this adventure on my own, I have too much other priorities right
now. If I see somebody being interested, I will definitely help this
person and join his efforts as much as possible.


-- 
Philip Van Hoof, software developer
home: me at pvanhoof dot be 
gnome: pvanhoof at gnome dot org 
http://www.pvanhoof.be/blog




___
Evolution-hackers mailing list
Evolution-hackers@gnome.org
http://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] Introduction and Questions

2007-06-07 Thread Sankar P
On Thu, 2007-06-07 at 12:28 +0300, Philip Van Hoof wrote:
 On Thu, 2007-06-07 at 03:05 -0600, Sankar P wrote:
  On Thu, 2007-06-07 at 10:56 +0300, Philip Van Hoof wrote:
 
   
   It improves the situation by setting your url-string to have the
   basic_headers option. In the imap code of Camel, it will then ask for
   less headers (but still way too much).
  
  May be way too much for a mobile client; but not for a desktop email
  client. Sure you dont want your desktop mail client to have threading or
  show mail-size or have such basic things ?
 
 When I'm using Evolution on a laptop, I feel very mobile too.
 
 btw. This is a quote that I have from Federico, I think (remember) he
 said that to me last GUADEC. Although
 
 Very often I'm indeed using my laptop at an airport or something, using
 GPRS. Why isn't Evolution suitable for this use-case?

I spend more than 99% of my internet time in a non-GPRS connection. Just
because, I might want to check mail from an airport a few times in a
year, I don't want my mail client to be a minimalistic bare-bones
application all the time. I would rather use a web-based client at these
times. And if I want to use Evolution even in such scenario, then there
is an option to get basic_headers alone as well. (Such option was not
there in last GUADEC though)

 
 I agree, though, that Tinymail's camel-lite memory improvements are less
 of a priority for Evolution. It's bandwidth work, though, is important
 in my opinion. Same for the Push E-mail capabilities.
 
   A major improvement it certainly isn't.
  
  Try comparing the performance of fetching all headers and basic_headers
  only. Some crude data which I collected a year back is at
  http://psankar.blogspot.com/2006/05/imap-performance.html
 
 Of course, you can look at the code to see what differs between the
 basic_headers mode and the normal one. And that will obviously make a
 big difference (the selection of the query is simply a lot smaller).
 
 But if Evolution is causing 900% of its bandwidth to be unnecessary, and
 the basic_headers mode saves 400% of that, there is still 500% of it
 that is too much. Which is the situation with basic_headers.
 
 The basic_headers option also doesn't implement condstore, which is
 quite important to safe bandwidth (if supported by the IMAP server).

The band-width used by Evolution cannot be considered unnecessary, by
comparing with a specific style that is designed for mobiles and
intended to reduce bandwidth. 

It will be good to have CONDSTORE and ENVELOPE (and whatever-new) but it
is not something that makes a user to say, Damn. I need this and cant
use Evo unless I have it. But there are many other such things that
needs attention and resources.

There are more than 5500 bugs in Evolution and EDS in bgo alone and
among these there is just only one bug that complains about bandwidth.
This bug is filed by you and will eventually benefit the evolution
project once you complete the patch that you are attempting there and
being reviewed by Fejj. 

If Evolution (and hence Gnome as well) has to be successful in an
enterprise level it needs more things like: Better Exchange support,
Nicer and Richer UI, and more importantly Stability. So the Evolution
project's roadmap should be driven based on these, instead of a feature
that is not yet implemented by all the servers.

I am not denying that it is good to have all these bandwidth savings but
Feeping Creaturism(1) should be kept in check.

However, if you feel this is the biggest blocker for a mail-application,
patches are always welcomed :)

 
   The best way is to ask for the ENVELOPE and the remaining info using the
   normal BODY.PEEK method. It should be possible to specify per folder
   which of the headers are to be fetched, to make it really efficient.
  
  Do you really think any user on earth will really be interested in
  configuring what headers to fetch on a folder-basis ? 
 
 The application should figure that list out automatically, obviously. 

I still don't understand why you need to get different headers for
different folders.

  
   ps. In my opinion is also the imap4 project getting the majority of its
   design wrong. Although it looks better than the imap one, it's not the
   best way to use an IMAP server. If a project is going to write an IMAP
   client implementation 'today': they better do it right. A lot is
   changing in IMAP today (Lemonade, MORG, etc). It's important to get it
   right this time (the vast majority of E-mail clients totally suck at how
   they use the IMAP server, including Evolution indeed).
  
  Such a sweeping generalization !?
 
 Maybe, but I've seen quite a lot of their code, and they are indeed very
 badly programmed. Most of the custom Camel providers are in a very bad
 shape too, by the way.
 
 And ... camel-lite, camel upstream and its standard providers are also
 in a bad shape :(
 
  As you mentioned, IMAP (like any other technology) grows at a very
  high-speed. What you 

Re: [Evolution-hackers] Introduction and Questions

2007-06-07 Thread Ritesh Khadgaray
On Thu, 2007-06-07 at 04:56 -0600, Sankar P wrote:
 On Thu, 2007-06-07 at 12:28 +0300, Philip Van Hoof wrote:
  On Thu, 2007-06-07 at 03:05 -0600, Sankar P wrote:
   On Thu, 2007-06-07 at 10:56 +0300, Philip Van Hoof wrote:
  

It improves the situation by setting your url-string to have the
basic_headers option. In the imap code of Camel, it will then ask for
less headers (but still way too much).
   
   May be way too much for a mobile client; but not for a desktop email
   client. Sure you dont want your desktop mail client to have threading or
   show mail-size or have such basic things ?
  
  When I'm using Evolution on a laptop, I feel very mobile too.
  
  btw. This is a quote that I have from Federico, I think (remember) he
  said that to me last GUADEC. Although
  
  Very often I'm indeed using my laptop at an airport or something, using
  GPRS. Why isn't Evolution suitable for this use-case?
 
 I spend more than 99% of my internet time in a non-GPRS connection. Just
 because, I might want to check mail from an airport a few times in a
 year, I don't want my mail client to be a minimalistic bare-bones
 application all the time. I would rather use a web-based client at these
 times. And if I want to use Evolution even in such scenario, then there
 is an option to get basic_headers alone as well. (Such option was not
 there in last GUADEC though)

On Evoluion 2.10, i see a option for basic headers . 

Edit - preferences - mail account - 
select your imap account - edit - IMAP headers - Basic headers


 
  
  I agree, though, that Tinymail's camel-lite memory improvements are less
  of a priority for Evolution. It's bandwidth work, though, is important
  in my opinion. Same for the Push E-mail capabilities.
  
A major improvement it certainly isn't.
   
   Try comparing the performance of fetching all headers and basic_headers
   only. Some crude data which I collected a year back is at
   http://psankar.blogspot.com/2006/05/imap-performance.html
  
  Of course, you can look at the code to see what differs between the
  basic_headers mode and the normal one. And that will obviously make a
  big difference (the selection of the query is simply a lot smaller).
  
  But if Evolution is causing 900% of its bandwidth to be unnecessary, and
  the basic_headers mode saves 400% of that, there is still 500% of it
  that is too much. Which is the situation with basic_headers.
  
  The basic_headers option also doesn't implement condstore, which is
  quite important to safe bandwidth (if supported by the IMAP server).
 
 The band-width used by Evolution cannot be considered unnecessary, by
 comparing with a specific style that is designed for mobiles and
 intended to reduce bandwidth. 
 
 It will be good to have CONDSTORE and ENVELOPE (and whatever-new) but it
 is not something that makes a user to say, Damn. I need this and cant
 use Evo unless I have it. But there are many other such things that
 needs attention and resources.
 
 There are more than 5500 bugs in Evolution and EDS in bgo alone and
 among these there is just only one bug that complains about bandwidth.
 This bug is filed by you and will eventually benefit the evolution
 project once you complete the patch that you are attempting there and
 being reviewed by Fejj. 
 
 If Evolution (and hence Gnome as well) has to be successful in an
 enterprise level it needs more things like: Better Exchange support,
 Nicer and Richer UI, and more importantly Stability. So the Evolution
 project's roadmap should be driven based on these, instead of a feature
 that is not yet implemented by all the servers.
 
 I am not denying that it is good to have all these bandwidth savings but
 Feeping Creaturism(1) should be kept in check.
 
 However, if you feel this is the biggest blocker for a mail-application,
 patches are always welcomed :)
 
  
The best way is to ask for the ENVELOPE and the remaining info using the
normal BODY.PEEK method. It should be possible to specify per folder
which of the headers are to be fetched, to make it really efficient.
   
   Do you really think any user on earth will really be interested in
   configuring what headers to fetch on a folder-basis ? 
  
  The application should figure that list out automatically, obviously. 
 
 I still don't understand why you need to get different headers for
 different folders.
 
   
ps. In my opinion is also the imap4 project getting the majority of its
design wrong. Although it looks better than the imap one, it's not the
best way to use an IMAP server. If a project is going to write an IMAP
client implementation 'today': they better do it right. A lot is
changing in IMAP today (Lemonade, MORG, etc). It's important to get it
right this time (the vast majority of E-mail clients totally suck at how
they use the IMAP server, including Evolution indeed).
   
   Such a sweeping generalization !?
  
  Maybe, but I've seen quite a lot of their 

Re: [Evolution-hackers] Introduction and Questions

2007-06-07 Thread Jeffrey Stedfast
On Thu, 2007-06-07 at 10:56 +0300, Philip Van Hoof wrote:
 On Thu, 2007-06-07 at 08:25 +0200, Gilles Dartiguelongue wrote:
  Le jeudi 07 juin 2007 à 01:53 +0300, Philip Van Hoof a écrit :
   Without immediately writing to disk work, the download by itself will
   consume around 120 MB of RAM, and will most likely fail due to network
   timeouts and other such problems (it'll take a while, since Evolution
   fetches a ridiculous large amount of headers for each message for
   building its summary).
   
  isn't the imap-features plugin's goal to reduce/customize the amount of
  downloaded headers ? Or is it that it's still not enough ?
 
 It improves the situation by setting your url-string to have the
 basic_headers option. In the imap code of Camel, it will then ask for
 less headers (but still way too much).

if you set the option to basic headers, it will only fetch the minimal
set of headers required to show the message-list:

DATE FROM TO CC SUBJECT REFERENCES IN-REPLY-TO MESSAGE-ID MIME-VERSION
CONTENT-TYPE 

(actually, why is MIME-VERSION in there? it's useless)

 
 A major improvement it certainly isn't.

it's not possible to do better w/o dropping features like message
threading.

In fact, the above minimalizing of header fetching already breaks the
quick context-menu vfolder on mailing-list and filter on
mailing-list features as well as making vfoldering on mailing-list
oodles slower (if it even still works after disabling the headers)

 
 The best way is to ask for the ENVELOPE and the remaining info using the
 normal BODY.PEEK method.

Have you actually ever tested this theory? or did you just pull this out
of your proverbial?

You should note that Thunderbird does not use ENVELOPE, probably because
they found it was LESS efficient to fetch the ENVELOPE +
BODY.PEEK[HEADER.FIELDS (REFERENCES CONTENT-TYPE)] on many existing IMAP
servers in the wild.

Having myself tested this in the past, I don't recall it making much
difference one way or the other.

Sadly using ENVELOPE breaks on at least one proprietary IMAP server
which occasionally likes to send back responses with NIL as each and
every item, at least until a client has issued a BODY.PEEK fetch
request.

  It should be possible to specify per folder
 which of the headers are to be fetched, to make it really efficient.

how do you propose calculating the list of headers each folder should
request? which headers would you eliminate conditionally?

 
 The imap4 implementation seems to have an ENVELOPE parser, so I guess
 either it does it already or it will use ENVELOPE in future.

it fetches:

ENVELOPE BODY.PEEK[HEADER.FIELDS (Content-Type References In-Reply-To)]

and optionally the mailing-list headers

I believe it fetches IN-REPLY-TO (which is included in ENVELOPE)
explicitly because some IMAP servers tried to be smart and parse the
IN-REPLY-TO header for us and didn't handle embedded phrases which
certain broken free-software mailers like to send.

 
 For a mobile device, that works over GPRS, you for example are usually
 not interested (not at all) in the mailinglist specific headers.

this is mobile-device specific tho, when I was implementing imap4 and
people started trying to use it, it was their #1 complaint (which is why
imap4 conditionally requests the mlist headers... tho that flag may be
hard-coded to TRUE now, I'm not sure)

 
 It would also be possible to do it in multiple passes. For example get a
 modest list of headers, and after that get more and more.

but then you introduce a lot more complexity (e.g. scheduling such a
fetch)

slowness for the user (e.g. if he expects to be able to right-click and
select vfolder on mailing-list, he'd have to wait if the info had not
yet been fetched... or even just selecting a mailing-list vfolder)

keep in mind also that this is a one-time fetch, so while opening a
folder for the first time might be slow (if you have 1000's of
messages), future folder selections would take a much more reasonable
amount of time to open (especially with CONDSTORE support which reduces
the FLAGS sync requirements, which is where the real slowness is when
opening a folder for most users)

 
 In any case, none of the current Evolution code implements consuming the
 CONDSTORE capabilities of some modern IMAP servers (like MBox and
 Cyrus).
 
 CONDSTORE is really going to make an enormous difference in bandwidth
 consumption with large folders. That's because you only need to fetch
 the flags of the changed messages to synchronise flags.

yes, this would be nice to have... but as far as I'm aware, it is still
only a draft standard (e.g. it could change).

I would probably not be opposed to a CONDSTORE implementation making it
into upstream camel, tho.

 
 Note that Tinymail's camel-lite implements all the needed solutions to
 this bandwidth consumption problem. And that its code is, although a lot
 of work because the Evolution maintainers didn't seem interested at the
 time it was happening, definitely 

Re: [Evolution-hackers] Introduction and Questions

2007-06-06 Thread Philip Van Hoof
On Thu, 2007-05-31 at 14:10 -0400, Jeffrey Stedfast wrote:
 On Thu, 2007-05-31 at 07:58 -0700, Ross Boylan wrote:
  Hi.  I've been getting into the code of evolution recently, and am thinking 
  of 
  doing a bit more to see if I can get it working OK for my situation.  I 
  have 
  an IMAP mailbox which is very large, both in terms of folders (over 100) 
  and 
  messages (the largest folder has 300,000 messages; my INBOX has about 
  22,000).
 
 the largest INBOX I've ever used was about ~100,000 messages, so you may
 get to have some fun :)

Without CONDSTORE support, Evolution needs ridiculous amounts of both
time and bandwidth to synchronise such a folder's flags. When online,
this happens each time you click on a folder in Evolution.
 
Without immediately writing to disk work, the download by itself will
consume around 120 MB of RAM, and will most likely fail due to network
timeouts and other such problems (it'll take a while, since Evolution
fetches a ridiculous large amount of headers for each message for
building its summary).


-- 
Philip Van Hoof, software developer
home: me at pvanhoof dot be 
gnome: pvanhoof at gnome dot org 
http://www.pvanhoof.be/blog




___
Evolution-hackers mailing list
Evolution-hackers@gnome.org
http://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] Introduction and Questions

2007-06-01 Thread Ross Boylan
On Thursday 31 May 2007 16:09, Jeffrey Stedfast wrote:
 On Thu, 2007-05-31 at 17:18 -0400, Jeffrey Stedfast wrote:
 [snip]

   Single namespace.  It's all INBOX.folder.subfolder.
   The one wrinkle is that in some cases 'folder' exists in the namespace,
   but is not an actual box or folder (whatever the right term is) on the
   server: INBOX.folder.subfolder is a real folder; INBOX.folder is not.
 
  I guess just a bug

 I don't have a build environment anymore so I can't easily test this
 patch, but perhaps i will fix your missing folder troubles.

I'm not sure that the names that aren't folders problem is causing any of the 
problems I'm seeing.  While such names don't appear in the folder list on the 
left pane, I also have plenty of ordinary folders that aren't appearing.  And 
the ghost folders, along with their children, do show up in the Folder| 
subscription dialogue.

evolution seems to insist on showing only subscribed folders, even when I 
disable the show only subscribed folders option.

Since the ghost folders aren't folders, there is no way I can subscribe to 
them.  So if I wanted to see only subscribed folders, that might create a 
problem.

I was able to start up evo under the debugger on the system that's having 
problems, and this time it worked.  Yay!

I put future breakpoints on camel_store_supports_subscriptions, and (following 
the location of the patches you provided)  imap4_build_folder_info.  I also 
put one on  imap_build_folder_info (a wild guess).

Unfortunately none of these were hit when evo started up and was deciding what 
to show.  I did hit the camel_store_support_subscriptions when I opened the 
Folders | subscriptions dialogue.

I tried creating an IMAP type account.  This did hit imap_build_folder_info, 
and it did show all my folders (including ghosts).  Performance seems pretty 
good.

I'm going to stop for now, while I'm ahead.
___
Evolution-hackers mailing list
Evolution-hackers@gnome.org
http://mail.gnome.org/mailman/listinfo/evolution-hackers


[Evolution-hackers] Introduction and Questions

2007-05-31 Thread Ross Boylan
Hi.  I've been getting into the code of evolution recently, and am thinking of 
doing a bit more to see if I can get it working OK for my situation.  I have 
an IMAP mailbox which is very large, both in terms of folders (over 100) and 
messages (the largest folder has 300,000 messages; my INBOX has about 
22,000).

None of the email clients I've tried cope with this very well.  Since I've 
been using evo at work in a similar setup (cyrus server, though not quite as 
big), I thought it might be the best to try to tweak.  The problems I've had 
so far involve setup activities rather than core functionaility.  First, 
evolution couldn't create the account (solved); second I've had problems 
getting it to show all of my subfolders.

My narrow question is how to debug evolution.  When I launched evolution in 
gdb evolution (the GUI) came up, and then the debugger told me the process 
had exited (though the GUI was up and running).  I believe the initial 
process does some kind of activation of the real process and then exits.

I know I can attach to the second process, but I think the stuff I need to see 
happens at startup.  So how can I get a debugger on the startup process of 
the real evolution?

I tried to search the archive, but the search function seems broken: it 
searches everything, even though it says search this list only.

Also, evolution seems to have two presences on the web: a web site and a wiki.  
The developer stuff on the web site is old, and has no pointers to the newer 
stuff that I could find.  It would be good if it did.

I have some broader questions too, if anyone has any comments on them.

What version to start with?  I'm on Debian GNU/Linux, which currently has evo 
2.6.  I notice that's a bit dated (although I did see that a few months ago 
some of the Debian packagers were interested in making a more recent 
version).  I've been working from the Debian version.  Does that version, the 
last stable release (from evo, not Debian), or svn head make the most sense 
to work from?  (BTW, the one bug I fixed was one that was already fixed 
post-2.6).

Mission Impossible?  Am I likely to get anywhere without spending lots of 
time?  I'm a professional software developer, but I'm not familiar with 
GNOME, and this is clearly a complex application.

Mission Advisable?  If I get past the setup hurdles, is evolution likely to be 
able to handle the mail store I described?

How do I find out which of the imap store's code I'm actually using?

Thanks.
Ross Boylan
___
Evolution-hackers mailing list
Evolution-hackers@gnome.org
http://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] Introduction and Questions

2007-05-31 Thread Srinivasa Ragavan
Hi,

On Thu, 2007-05-31 at 07:58 -0700, Ross Boylan wrote:
 Hi.  I've been getting into the code of evolution recently, and am thinking 
 of 
 doing a bit more to see if I can get it working OK for my situation.  I have 
 an IMAP mailbox which is very large, both in terms of folders (over 100) and 
 messages (the largest folder has 300,000 messages; my INBOX has about 
 22,000).
 
 None of the email clients I've tried cope with this very well.  Since I've 
 been using evo at work in a similar setup (cyrus server, though not quite as 
 big), I thought it might be the best to try to tweak.  The problems I've had 
 so far involve setup activities rather than core functionaility.  First, 
 evolution couldn't create the account (solved); second I've had problems 
 getting it to show all of my subfolders.
 
 My narrow question is how to debug evolution.  When I launched evolution in 
 gdb evolution (the GUI) came up, and then the debugger told me the process 
 had exited (though the GUI was up and running).  I believe the initial 
 process does some kind of activation of the real process and then exits.

Hmm, doesn't it give you a point to do 'bt' to get the traces? 

 
 I know I can attach to the second process, but I think the stuff I need to 
 see 
 happens at startup.  So how can I get a debugger on the startup process of 
 the real evolution?
 
 I tried to search the archive, but the search function seems broken: it 
 searches everything, even though it says search this list only.
 
 Also, evolution seems to have two presences on the web: a web site and a 
 wiki.  
 The developer stuff on the web site is old, and has no pointers to the newer 
 stuff that I could find.  It would be good if it did.
 

We are in the process of revamping the wiki with more relevant in
information. We would also rebuild the site with latest informations.
But that would take a while. 

 I have some broader questions too, if anyone has any comments on them.
 
 What version to start with?  I'm on Debian GNU/Linux, which currently has evo 
 2.6.  I notice that's a bit dated (although I did see that a few months ago 
 some of the Debian packagers were interested in making a more recent 
 version).  I've been working from the Debian version.  Does that version, the 
 last stable release (from evo, not Debian), or svn head make the most sense 
 to work from?  (BTW, the one bug I fixed was one that was already fixed 
 post-2.6).

I would prefer, if you can try it with Evolution 2.10.2. It is the most
recent stable release.

-Srini.
 
 Mission Impossible?  Am I likely to get anywhere without spending lots of 
 time?  I'm a professional software developer, but I'm not familiar with 
 GNOME, and this is clearly a complex application.
 
 Mission Advisable?  If I get past the setup hurdles, is evolution likely to 
 be 
 able to handle the mail store I described?
 
 How do I find out which of the imap store's code I'm actually using?
 
 Thanks.
 Ross Boylan
 ___
 Evolution-hackers mailing list
 Evolution-hackers@gnome.org
 http://mail.gnome.org/mailman/listinfo/evolution-hackers

___
Evolution-hackers mailing list
Evolution-hackers@gnome.org
http://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] Introduction and Questions

2007-05-31 Thread Jeffrey Stedfast
On Thu, 2007-05-31 at 07:58 -0700, Ross Boylan wrote:
 Hi.  I've been getting into the code of evolution recently, and am thinking 
 of 
 doing a bit more to see if I can get it working OK for my situation.  I have 
 an IMAP mailbox which is very large, both in terms of folders (over 100) and 
 messages (the largest folder has 300,000 messages; my INBOX has about 
 22,000).

the largest INBOX I've ever used was about ~100,000 messages, so you may
get to have some fun :)

 
 None of the email clients I've tried cope with this very well.  Since I've 
 been using evo at work in a similar setup (cyrus server, though not quite as 
 big), I thought it might be the best to try to tweak.  The problems I've had 
 so far involve setup activities rather than core functionaility.  First, 
 evolution couldn't create the account (solved); second I've had problems 
 getting it to show all of my subfolders.

are they in different namespaces? Current evo IMAP provider doesn't
handle multiple namespaces :(

 
 My narrow question is how to debug evolution.  When I launched evolution in 
 gdb evolution (the GUI) came up, and then the debugger told me the process 
 had exited (though the GUI was up and running).  I believe the initial 
 process does some kind of activation of the real process and then exits.

sounds like you already had an instance of Evolution running (on another
virtual desktop perhaps?)

Evolution only allows one instance of itself to be running for the same
user account... if you try to start a second instance, it signals the
original instance to open a new window and then exits.

 
 I know I can attach to the second process, but I think the stuff I need to 
 see 
 happens at startup.  So how can I get a debugger on the startup process of 
 the real evolution?

should just be able to do gdb evolution

(Note: I haven't debugged evolution in over a year since I've moved onto
other projects, but that's how I always debugged it)

 
 I tried to search the archive, but the search function seems broken: it 
 searches everything, even though it says search this list only.
 
 Also, evolution seems to have two presences on the web: a web site and a 
 wiki.  
 The developer stuff on the web site is old, and has no pointers to the newer 
 stuff that I could find.  It would be good if it did.
 
 I have some broader questions too, if anyone has any comments on them.
 
 What version to start with?  I'm on Debian GNU/Linux, which currently has evo 
 2.6.  I notice that's a bit dated (although I did see that a few months ago 
 some of the Debian packagers were interested in making a more recent 
 version).  I've been working from the Debian version.  Does that version, the 
 last stable release (from evo, not Debian), or svn head make the most sense 
 to work from?  (BTW, the one bug I fixed was one that was already fixed 
 post-2.6).

probably best to start with 2.10(.2) (or svn if you are brave) so as to
avoid spending time fixing things that have already been fixed.

 
 Mission Impossible?  Am I likely to get anywhere without spending lots of 
 time?

I guess the answer to this is all relative... :)

   I'm a professional software developer, but I'm not familiar with 
 GNOME, and this is clearly a complex application.
 
 Mission Advisable?  If I get past the setup hurdles, is evolution likely to 
 be 
 able to handle the mail store I described?

I think it'll be possible, there's lots of improvement that can be made
to the current imap code :)

 
 How do I find out which of the imap store's code I'm actually using?

you are probably using the provider in
evolution-data-server/camel/providers/imap

- imap4 is a project I started to replace the current imap provider and
works fairly ok, but isn't quite complete (I forget all what it is
missing since it's been a few years since I actively hacked on it - I
think the main thing is cached search results?). This provider, like the
current imap backend, suffers from being synchronous, but it is far
better designed and much cleaner code to read. This backend also
supports multiplenamespaces (tho it'd be better if the Camel API
included multiple-namespace support, the way it works in imap4 is
because all folders are listed from a toplevel namespace rather than
from individual namespaces like it should).

- imapp is an old attempt at making a pipelined imap provider, tho it is
basically a dead-end at this point.

- imapx is yet another attempt at replacing the current imap backend but
depends on a lot of unfinished stuff in a development branch that has
been abandoned for about 2 years now I think (the guy that had been
working on that quit)


Jeff


___
Evolution-hackers mailing list
Evolution-hackers@gnome.org
http://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] Introduction and Questions

2007-05-31 Thread Ross Boylan
On Thu, 2007-05-31 at 23:14 +0530, Srinivasa Ragavan wrote:
 Hi,
 
 On Thu, 2007-05-31 at 07:58 -0700, Ross Boylan wrote:
  Hi.  I've been getting into the code of evolution recently, and am thinking 
  of 
  doing a bit more to see if I can get it working OK for my situation.  I 
  have 
  an IMAP mailbox which is very large, both in terms of folders (over 100) 
  and 
  messages (the largest folder has 300,000 messages; my INBOX has about 
  22,000).
  
  None of the email clients I've tried cope with this very well.  Since I've 
  been using evo at work in a similar setup (cyrus server, though not quite 
  as 
  big), I thought it might be the best to try to tweak.  The problems I've 
  had 
  so far involve setup activities rather than core functionaility.  First, 
  evolution couldn't create the account (solved); second I've had problems 
  getting it to show all of my subfolders.
  
  My narrow question is how to debug evolution.  When I launched evolution in 
  gdb evolution (the GUI) came up, and then the debugger told me the process 
  had exited (though the GUI was up and running).  I believe the initial 
  process does some kind of activation of the real process and then exits.
 
 Hmm, doesn't it give you a point to do 'bt' to get the traces? 
I don't think there's anything to trace, since the action is happening
in another process.
When I begin I am at the gdb command line.  I was able to insert a
breakpoint at what seemed like a good spot (this was a future
breakpoint, since the relevant library hadn't loaded).  Then when I did
'run' the GUI came up, and the program I was debugging exited.  There
was no error, and I didn't hit the breakpoint, so I don't think bt would
be meaningful.

Or perhaps I'm misunderstanding and it is only one thread that has
exited.  But I see in main.c a lot of IDL code, and I know GNOME apps
are supposed to be able to discover if they've already launched.  I
guessed (mostly by analogy with MS COM) that the initial program simply
requests a service, and that is either conveyed to an existing instance
of the app or a new  instance is created.
 
  
  I know I can attach to the second process, but I think the stuff I need to 
  see 
  happens at startup.  So how can I get a debugger on the startup process of 
  the real evolution?
  

 
-- 
Ross Boylan  wk:  (415) 514-8146
185 Berry St #5700   [EMAIL PROTECTED]
Dept of Epidemiology and Biostatistics   fax: (415) 514-8150
University of California, San Francisco
San Francisco, CA 94107-1739 hm:  (415) 550-1062

___
Evolution-hackers mailing list
Evolution-hackers@gnome.org
http://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] Introduction and Questions

2007-05-31 Thread Ross Boylan
On Thu, 2007-05-31 at 14:10 -0400, Jeffrey Stedfast wrote: 
 On Thu, 2007-05-31 at 07:58 -0700, Ross Boylan wrote:
  Hi.  I've been getting into the code of evolution recently, and am thinking 
  of 
  doing a bit more to see if I can get it working OK for my situation.  I 
  have 
  an IMAP mailbox which is very large, both in terms of folders (over 100) 
  and 
  messages (the largest folder has 300,000 messages; my INBOX has about 
  22,000).
 
 the largest INBOX I've ever used was about ~100,000 messages, so you may
 get to have some fun :)
 
  
  None of the email clients I've tried cope with this very well.  Since I've 
  been using evo at work in a similar setup (cyrus server, though not quite 
  as 
  big), I thought it might be the best to try to tweak.  The problems I've 
  had 
  so far involve setup activities rather than core functionaility.  First, 
  evolution couldn't create the account (solved); second I've had problems 
  getting it to show all of my subfolders.
 
 are they in different namespaces? Current evo IMAP provider doesn't
 handle multiple namespaces :(
Single namespace.  It's all INBOX.folder.subfolder.
The one wrinkle is that in some cases 'folder' exists in the namespace,
but is not an actual box or folder (whatever the right term is) on the
server: INBOX.folder.subfolder is a real folder; INBOX.folder is not.
 
  
  My narrow question is how to debug evolution.  When I launched evolution in 
  gdb evolution (the GUI) came up, and then the debugger told me the process 
  had exited (though the GUI was up and running).  I believe the initial 
  process does some kind of activation of the real process and then exits.
 
 sounds like you already had an instance of Evolution running (on another
 virtual desktop perhaps?)
I'm pretty sure I shut everything down.  Hmm, I just tried again and
this time I'm still in the debugger.

There was another instance of evolution running, but for a different
user.  That instance is in a funny state, because it is not visible in
the other user's screen.

Running under KDE in all cases.
 

 
  
  Mission Advisable?  If I get past the setup hurdles, is evolution likely to 
  be 
  able to handle the mail store I described?
 
 I think it'll be possible, there's lots of improvement that can be made
 to the current imap code :)
I'm hoping it will be tolerable without improvements :)  I think one of the 
main factors will be whether evo 
tries to get all the headers from each folder it operates on.  Even if it does, 
if it only does it once
and is otherwise responsive I'll be doing better than what I have now.

At work, I think my biggest folder has about 85k messages, and evo is working 
fine.
 
  
  How do I find out which of the imap store's code I'm actually using?
 
 you are probably using the provider in
 evolution-data-server/camel/providers/imap
Is the other not loaded, or does it depend on which option I picked for the 
server (I said IMAP4rev1).

Thanks to everyone for the quick replies.  Not having a functional email client 
is a big problem. 
I used to archive off my mail when it got to be too much.  I thought it would 
be more convenient to have it
consolidated under IMAP since, as a practical matter, once I archived the stuff 
it became invisible.  So far,
it's not so convenient!

So far, I have tried
evolution
kmail (using it for now)
thunderbird/icedove
mulberry
Apple's mail app
claws
mutt

-- 
Ross Boylan  wk:  (415) 514-8146
185 Berry St #5700   [EMAIL PROTECTED]
Dept of Epidemiology and Biostatistics   fax: (415) 514-8150
University of California, San Francisco
San Francisco, CA 94107-1739 hm:  (415) 550-1062

___
Evolution-hackers mailing list
Evolution-hackers@gnome.org
http://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] Introduction and Questions

2007-05-31 Thread Alfredo Matos
On Thu, 2007-05-31 at 14:10 -0400, Jeffrey Stedfast wrote:
 On Thu, 2007-05-31 at 07:58 -0700, Ross Boylan wrote:
  Hi.  I've been getting into the code of evolution recently, and am thinking 
  of 
  doing a bit more to see if I can get it working OK for my situation.  I 
  have 
  an IMAP mailbox which is very large, both in terms of folders (over 100) 
  and 
  messages (the largest folder has 300,000 messages; my INBOX has about 
  22,000).
 
 the largest INBOX I've ever used was about ~100,000 messages, so you may
 get to have some fun :)
 
  
  None of the email clients I've tried cope with this very well.  Since I've 
  been using evo at work in a similar setup (cyrus server, though not quite 
  as 
  big), I thought it might be the best to try to tweak.  The problems I've 
  had 
  so far involve setup activities rather than core functionaility.  First, 
  evolution couldn't create the account (solved); second I've had problems 
  getting it to show all of my subfolders.
 
 are they in different namespaces? Current evo IMAP provider doesn't
 handle multiple namespaces :(

Maybe it's time to plan this on the wiki ?

 
  
  My narrow question is how to debug evolution.  When I launched evolution in 
  gdb evolution (the GUI) came up, and then the debugger told me the process 
  had exited (though the GUI was up and running).  I believe the initial 
  process does some kind of activation of the real process and then exits.
 
 sounds like you already had an instance of Evolution running (on another
 virtual desktop perhaps?)
 
 Evolution only allows one instance of itself to be running for the same
 user account... if you try to start a second instance, it signals the
 original instance to open a new window and then exits.
 
  
  I know I can attach to the second process, but I think the stuff I need to 
  see 
  happens at startup.  So how can I get a debugger on the startup process of 
  the real evolution?
 
 should just be able to do gdb evolution

This is one of the reasons we independent instances across mail,
calendar, contacts, etc would be good. Easier debugging, isolation of
problems, performance tune up, all by component in a far easier way,
IMHO.

 
 (Note: I haven't debugged evolution in over a year since I've moved onto
 other projects, but that's how I always debugged it)
 
  
  I tried to search the archive, but the search function seems broken: it 
  searches everything, even though it says search this list only.
  
  Also, evolution seems to have two presences on the web: a web site and a 
  wiki.  
  The developer stuff on the web site is old, and has no pointers to the 
  newer 
  stuff that I could find.  It would be good if it did.
  
  I have some broader questions too, if anyone has any comments on them.
  
  What version to start with?  I'm on Debian GNU/Linux, which currently has 
  evo 
  2.6.  I notice that's a bit dated (although I did see that a few months ago 
  some of the Debian packagers were interested in making a more recent 
  version).  I've been working from the Debian version.  Does that version, 
  the 
  last stable release (from evo, not Debian), or svn head make the most sense 
  to work from?  (BTW, the one bug I fixed was one that was already fixed 
  post-2.6).
 
 probably best to start with 2.10(.2) (or svn if you are brave) so as to
 avoid spending time fixing things that have already been fixed.
 
  
  Mission Impossible?  Am I likely to get anywhere without spending lots of 
  time?
 
 I guess the answer to this is all relative... :)
 
I'm a professional software developer, but I'm not familiar with 
  GNOME, and this is clearly a complex application.
  
  Mission Advisable?  If I get past the setup hurdles, is evolution likely to 
  be 
  able to handle the mail store I described?
 
 I think it'll be possible, there's lots of improvement that can be made
 to the current imap code :)
 
  
  How do I find out which of the imap store's code I'm actually using?
 
 you are probably using the provider in
 evolution-data-server/camel/providers/imap
 
 - imap4 is a project I started to replace the current imap provider and
 works fairly ok, but isn't quite complete (I forget all what it is
 missing since it's been a few years since I actively hacked on it - I
 think the main thing is cached search results?). This provider, like the
 current imap backend, suffers from being synchronous, but it is far
 better designed and much cleaner code to read. This backend also
 supports multiplenamespaces (tho it'd be better if the Camel API
 included multiple-namespace support, the way it works in imap4 is
 because all folders are listed from a toplevel namespace rather than
 from individual namespaces like it should).
 
 - imapp is an old attempt at making a pipelined imap provider, tho it is
 basically a dead-end at this point.
 
 - imapx is yet another attempt at replacing the current imap backend but
 depends on a lot of unfinished stuff in a development branch 

Re: [Evolution-hackers] Introduction and Questions

2007-05-31 Thread Ross Boylan
On Thu, 2007-05-31 at 23:14 +0530, Srinivasa Ragavan wrote:
  [Ross] What version to start with? 
 
 I would prefer, if you can try it with Evolution 2.10.2. It is the
 most recent stable release.
Is EVOLUTION_DATA_SERVER_1_10_2 the corresponding tag to use for
evolution-data-server?
-- 
Ross Boylan  wk:  (415) 514-8146
185 Berry St #5700   [EMAIL PROTECTED]
Dept of Epidemiology and Biostatistics   fax: (415) 514-8150
University of California, San Francisco
San Francisco, CA 94107-1739 hm:  (415) 550-1062

___
Evolution-hackers mailing list
Evolution-hackers@gnome.org
http://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] Introduction and Questions

2007-05-31 Thread Srinivasa Ragavan
On Thu, 2007-05-31 at 12:22 -0700, Ross Boylan wrote:
 On Thu, 2007-05-31 at 23:14 +0530, Srinivasa Ragavan wrote:
   [Ross] What version to start with? 
  
  I would prefer, if you can try it with Evolution 2.10.2. It is the
  most recent stable release.
 Is EVOLUTION_DATA_SERVER_1_10_2 the corresponding tag to use for
 evolution-data-server?

Yes. You are right.

-Srini.

___
Evolution-hackers mailing list
Evolution-hackers@gnome.org
http://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] Introduction and Questions

2007-05-31 Thread Matthew Barnes
On Thu, 2007-05-31 at 07:58 -0700, Ross Boylan wrote:
 What version to start with?  I'm on Debian GNU/Linux, which currently has evo 
 2.6.  I notice that's a bit dated (although I did see that a few months ago 
 some of the Debian packagers were interested in making a more recent 
 version).  I've been working from the Debian version.  Does that version, the 
 last stable release (from evo, not Debian), or svn head make the most sense 
 to work from?  (BTW, the one bug I fixed was one that was already fixed 
 post-2.6).

FYI, Debian Unstable has Evolution 2.10.  Might be easier to grab at
least the 2.10 dependencies from there.  You'll need to upgrade gtkhtml
and likely also your GTK+ library stack to get 2.10 to build.

Matthew Barnes

___
Evolution-hackers mailing list
Evolution-hackers@gnome.org
http://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] Introduction and Questions

2007-05-31 Thread Jeffrey Stedfast
On Thu, 2007-05-31 at 13:38 -0700, Ross Boylan wrote:
 On Thu, 2007-05-31 at 16:10 -0400, Matthew Barnes wrote:
  On Thu, 2007-05-31 at 07:58 -0700, Ross Boylan wrote:
   What version to start with?  I'm on Debian GNU/Linux, which currently has 
   evo 
   2.6.  I notice that's a bit dated (although I did see that a few months 
   ago 
   some of the Debian packagers were interested in making a more recent 
   version).  I've been working from the Debian version.  Does that version, 
   the 
   last stable release (from evo, not Debian), or svn head make the most 
   sense 
   to work from?  (BTW, the one bug I fixed was one that was already fixed 
   post-2.6).
  
  FYI, Debian Unstable has Evolution 2.10.  Might be easier to grab at
  least the 2.10 dependencies from there.  You'll need to upgrade gtkhtml
  and likely also your GTK+ library stack to get 2.10 to build.
  
  Matthew Barnes
 If I update GTK+ is that going to break other apps that use that
 library?

no, gtk maintains binary compat with older versions of the same major
revision (2.x in this case)

Jeff


___
Evolution-hackers mailing list
Evolution-hackers@gnome.org
http://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] Introduction and Questions

2007-05-31 Thread Jeffrey Stedfast
On Thu, 2007-05-31 at 11:38 -0700, Ross Boylan wrote:
 On Thu, 2007-05-31 at 14:10 -0400, Jeffrey Stedfast wrote: 
  On Thu, 2007-05-31 at 07:58 -0700, Ross Boylan wrote:
   Hi.  I've been getting into the code of evolution recently, and am 
   thinking of 
   doing a bit more to see if I can get it working OK for my situation.  I 
   have 
   an IMAP mailbox which is very large, both in terms of folders (over 100) 
   and 
   messages (the largest folder has 300,000 messages; my INBOX has about 
   22,000).
  
  the largest INBOX I've ever used was about ~100,000 messages, so you may
  get to have some fun :)
  
   
   None of the email clients I've tried cope with this very well.  Since 
   I've 
   been using evo at work in a similar setup (cyrus server, though not quite 
   as 
   big), I thought it might be the best to try to tweak.  The problems I've 
   had 
   so far involve setup activities rather than core functionaility.  First, 
   evolution couldn't create the account (solved); second I've had problems 
   getting it to show all of my subfolders.
  
  are they in different namespaces? Current evo IMAP provider doesn't
  handle multiple namespaces :(
 Single namespace.  It's all INBOX.folder.subfolder.
 The one wrinkle is that in some cases 'folder' exists in the namespace,
 but is not an actual box or folder (whatever the right term is) on the
 server: INBOX.folder.subfolder is a real folder; INBOX.folder is not.

I guess just a bug

[snip]
   How do I find out which of the imap store's code I'm actually using?
  
  you are probably using the provider in
  evolution-data-server/camel/providers/imap
 Is the other not loaded, or does it depend on which option I picked for the 
 server (I said IMAP4rev1).
 

ah, in that case you chose imap4 and not imap

the imap provider is simply named IMAP in the drop-down menu in the
account editor.

Jeff

___
Evolution-hackers mailing list
Evolution-hackers@gnome.org
http://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] Introduction and Questions

2007-05-31 Thread Øystein Gisnås
2007/5/31, Matthew Barnes [EMAIL PROTECTED]:
 On Thu, 2007-05-31 at 07:58 -0700, Ross Boylan wrote:
  What version to start with?  I'm on Debian GNU/Linux, which currently has 
  evo
  2.6.  I notice that's a bit dated (although I did see that a few months ago
  some of the Debian packagers were interested in making a more recent
  version).  I've been working from the Debian version.  Does that version, 
  the
  last stable release (from evo, not Debian), or svn head make the most sense
  to work from?  (BTW, the one bug I fixed was one that was already fixed
  post-2.6).

 FYI, Debian Unstable has Evolution 2.10.  Might be easier to grab at
 least the 2.10 dependencies from there.  You'll need to upgrade gtkhtml
 and likely also your GTK+ library stack to get 2.10 to build.

In case you're on a Debian-based distribution and not pulling from
svn, I would recommend using pre-built packages, or even building the
packages yourself. 2.10.2 is in the archive, and I will do 2.11.2 this
weekend.

For my own development setup I use the 2.10.x packages plus custom
build from svn for the module I'm hacking on. e-d-s for example, I
install to /opt/evolution-data-server. Then I can start development
e-d-s with 'LD_LIBRARY_PATH=/opt/evolution-data-server/lib
/opt/evolution-data-server/libexec/evolution-data-server-1.12' and can
also start the stable e-d-s with
'/usr/lib/evolution/evolution-data-server-1.10'

Cheers,
Øystein
___
Evolution-hackers mailing list
Evolution-hackers@gnome.org
http://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] Introduction and Questions

2007-05-31 Thread Ross Boylan
On Thu, 2007-05-31 at 23:20 +0200, Øystein Gisnås wrote:
 2007/5/31, Matthew Barnes [EMAIL PROTECTED]:
  On Thu, 2007-05-31 at 07:58 -0700, Ross Boylan wrote:
   What version to start with?  I'm on Debian GNU/Linux, which currently has 
   evo
   2.6.  I notice that's a bit dated (although I did see that a few months 
   ago
   some of the Debian packagers were interested in making a more recent
   version).  I've been working from the Debian version.  Does that version, 
   the
   last stable release (from evo, not Debian), or svn head make the most 
   sense
   to work from?  (BTW, the one bug I fixed was one that was already fixed
   post-2.6).
 
  FYI, Debian Unstable has Evolution 2.10.  Might be easier to grab at
  least the 2.10 dependencies from there.  You'll need to upgrade gtkhtml
  and likely also your GTK+ library stack to get 2.10 to build.
 
 In case you're on a Debian-based distribution 
Yes; straight Debian.
 and not pulling from
 svn, I would recommend using pre-built packages, or even building the
 packages yourself. 2.10.2 is in the archive, and I will do 2.11.2 this
 weekend.
Terrific.  Is unstable the place to look, or experimental, or somewhere
else?
 
 For my own development setup I use the 2.10.x packages plus custom
 build from svn for the module I'm hacking on. e-d-s for example, I
 install to /opt/evolution-data-server. Then I can start development
 e-d-s with 'LD_LIBRARY_PATH=/opt/evolution-data-server/lib
 /opt/evolution-data-server/libexec/evolution-data-server-1.12' and can
 also start the stable e-d-s with
 '/usr/lib/evolution/evolution-data-server-1.10'
That's a great tip.  I built evo from Debian source (with one fix) and
it took 45 minutes.  I clearly need a shorter route to trying out
changes.

Thanks for packing evo, and for making the -dbg files available.

Ross
-- 
Ross Boylan  wk:  (415) 514-8146
185 Berry St #5700   [EMAIL PROTECTED]
Dept of Epidemiology and Biostatistics   fax: (415) 514-8150
University of California, San Francisco
San Francisco, CA 94107-1739 hm:  (415) 550-1062

___
Evolution-hackers mailing list
Evolution-hackers@gnome.org
http://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] Introduction and Questions

2007-05-31 Thread Matthew Barnes
On Thu, 2007-05-31 at 14:22 -0700, Ross Boylan wrote:
 That sounds as if no upgrade of other apps would be forced.
 Or did you mean the stuff the evolution and gtk+ depends on would all
 need to go to newer versions?  That's probably a lot of stuff, but it's
 not so bad.

Hard to predict.  You can always cancel apt-get.  :)


 Is there somewhere that has a useful introduction to
 bonobo-activation-server and related machinery?  The references from
 http://developer.gnome.org/doc/guides/platform-overview/platform-overview.html#bonobo-corba?
   None of those look very friendly. 

Not to my knowledge.  Bonobo itself isn't very friendly, and is on its
way out.

Matthew Barnes

___
Evolution-hackers mailing list
Evolution-hackers@gnome.org
http://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] Introduction and Questions

2007-05-31 Thread Øystein Gisnås
2007/5/31, Ross Boylan [EMAIL PROTECTED]:
 On Thu, 2007-05-31 at 23:20 +0200, Øystein Gisnås wrote:
  2007/5/31, Matthew Barnes [EMAIL PROTECTED]:
   On Thu, 2007-05-31 at 07:58 -0700, Ross Boylan wrote:
What version to start with?  I'm on Debian GNU/Linux, which currently 
has evo
2.6.  I notice that's a bit dated (although I did see that a few months 
ago
some of the Debian packagers were interested in making a more recent
version).  I've been working from the Debian version.  Does that 
version, the
last stable release (from evo, not Debian), or svn head make the most 
sense
to work from?  (BTW, the one bug I fixed was one that was already fixed
post-2.6).
  
   FYI, Debian Unstable has Evolution 2.10.  Might be easier to grab at
   least the 2.10 dependencies from there.  You'll need to upgrade gtkhtml
   and likely also your GTK+ library stack to get 2.10 to build.
 
  In case you're on a Debian-based distribution
 Yes; straight Debian.
  and not pulling from
  svn, I would recommend using pre-built packages, or even building the
  packages yourself. 2.10.2 is in the archive, and I will do 2.11.2 this
  weekend.
 Terrific.  Is unstable the place to look, or experimental, or somewhere
 else?

2.11.2 will end up in pkg-evolution's svn for sure. If we upload
binary packages, they will go to experimental.

  For my own development setup I use the 2.10.x packages plus custom
  build from svn for the module I'm hacking on. e-d-s for example, I
  install to /opt/evolution-data-server. Then I can start development
  e-d-s with 'LD_LIBRARY_PATH=/opt/evolution-data-server/lib
  /opt/evolution-data-server/libexec/evolution-data-server-1.12' and can
  also start the stable e-d-s with
  '/usr/lib/evolution/evolution-data-server-1.10'
 That's a great tip.  I built evo from Debian source (with one fix) and
 it took 45 minutes.  I clearly need a shorter route to trying out
 changes.

I don't think you get around the initial build; you can feature strip
it to reduce the time. Once you've built it. make will figure out what
has to be rebuilt. A make  make install when I edit files in only
one directory takes about 5 seconds on my system..

 Thanks for packing evo, and for making the -dbg files available.

You're welcome. Don't hesitate with questions on how you can use
packages or package scripts in your development setup..

Cheers,
Øystein
___
Evolution-hackers mailing list
Evolution-hackers@gnome.org
http://mail.gnome.org/mailman/listinfo/evolution-hackers