[Evolution-hackers] Memory usage

2015-03-24 Thread David Woodhouse
Evolution 3.12 used to take a week or two to reach 8 or 9 GiB of 
memory usage, at which point I'd have to kill it and start a new one.

Evolution 3.16 seems to manage to reach 10GiB overnight, and needs to 
be killed every morning to bring my machine out of swap death.

I don't really know where all the memory is going. Running with 
GOBJECT_DEBUG=instance-count doesn't really show a smoking gun; it 
looks something like http://david.woodhou.se/evo-statistics.png 
(sorry, don't know how to get a text version of that).

The count of WebKitDOMRange objects only ever seems to go up, so I 
suspect those are still leaking despite some other WebKit leak fixes 
recently. But surely that can't be all there is?

I've tried using valgrind (--tool=massif) to see where the memory 
allocations are. But I get wildly inconsistent results.

If I run with the default options, it doesn't seem to admit that much 
memory was allocated at all. It's been running a few hours, it's using 
11.1GiB (which is admittedly inflated by massif itself), but it only 
*admits* to under 100MiB:


  ntime(i) total(B)   useful-heap(B) extra-heap(B)stacks(B)

 33  2,954,848,107   89,532,360   77,000,59212,531,7680
 34  3,087,325,256   97,270,104   82,569,29314,700,8110
84.89% (82,569,293B) (heap allocation functions) malloc/new/new[], --alloc-fns, 
etc.

(http://david.woodhou.se/massif.out.9492)

If I add --pages-as-heap=true, that makes it ignore the normal 
allocation functions and just look at the pages which are mapped by 
mmap(). Which screws the statistics for normal allocations somewhat, 
because the *first* allocation in a new page gets 'credited' with 
taking the whole 4KiB, while malloc() calls which get other addresses 
in that same page don't show up at all. But this way, it *does* admit 
that gigabytes of memory are being allocated...


  ntime(i) total(B)   useful-heap(B) extra-heap(B)stacks(B)

 43 5,840,961,313,097   10,555,485,320   10,555,485,320 0   
 0
 44 5,946,777,533,186   10,488,375,240   10,488,375,240 0   
 0
 45 6,094,323,581,406   10,563,886,472   10,563,886,472 0   
 0
 46 6,288,117,422,939   10,685,658,696   10,685,658,696 0   
 0
 47 6,477,882,772,357   10,855,649,728   10,855,649,728 0   
 0
100.00% (10,855,649,728B) (page allocation syscalls) mmap/mremap/brk, 
--alloc-fns, etc.
-91.15% (9,894,895,616B) 0x38412FC3B9: mmap (syscall-template.S:81)

(http://david.woodhou.se/massif.out.1884)

In this case, I do see some interesting additional results, like the 
fact that WebKit is apparently allocating 1GiB of memory at startup:

| -09.98% (1,081,118,720B) 0x3829E16FA1: 
WTF::OSAllocator::reserveUncommitted(unsigned long, WTF::OSAllocator::Usage, 
bool, bool, bool) (OSAllocatorPosix.cpp:46)
| | -09.91% (1,073,750,016B) 0x3829B72416: 
JSC::ExecutableAllocator::initializeAllocator() (PageReservation.h:113)
| | | -09.91% (1,073,750,016B) 0x3829CC66B0: void 
std::__once_call_implstd::_Bind_simpleJSC::initializeThreading()::{lambda()
| | |   -09.91% (1,073,750,016B) 0x3841A0DACD: __pthread_once_slow 
(pthread_once.c:114)
| | | -09.91% (1,073,750,016B) 0x3829CC6723: JSC::initializeThreading() 
(gthr-default.h:699)
| | |   -09.91% (1,073,750,016B) 0x382BA67BD0: webkitInit 
(webkitglobals.cpp:540)
| | | -09.91% (1,073,750,016B) 0x382BA67DC7: 
webkit_get_default_session (webkitglobals.cpp:86)
| | |   -09.91% (1,073,750,016B) 0x382E412D49: shell_initable_init 
(e-shell.c:1396)
| | | -09.91% (1,073,750,016B) 0x3F8D260078: g_initable_new_valist 
(ginitable.c:228)
| | |   -09.91% (1,073,750,016B) 0x3F8D260134: g_initable_new 
(ginitable.c:146)
| | | -09.91% (1,073,750,016B) 0x40418F: main (main.c:388)

WTF indeed. But on further investigation I think that's only 
allocating the address space, and not really causing it to be backed 
with real pages until it's used.

But most of the other results with --pages-as-heap=yes look like 
they're through the normal allocation functions and *should* have 
showed up with --pages-as-heap=no. So why didn't they?

One possible explanation is that we are *massively* inefficient with 
our allocations (in glibc), and we manage to allocate one tiny object 
per page to the point where 100MiB of allocations actually get spread 
across 8GiB of pages. But surely that's unlikely?

Is there a decent way to reliably track *all* memory usage within a 
GNOME application? Am I doing something wrong?

Hm, I note that I'm up to 9 

Re: [Evolution-hackers] Move authentication of backends back to the client (3.13.90)

2015-03-16 Thread David Woodhouse
On Fri, 2015-02-27 at 20:14 +0900, Tristan Van Berkom wrote:
 
 [0]: A note on semantics of opening a client, we really should be
 deprecating the DRA specific semantics in favor of making DRA the
 default and just falling back on pure DBus in the case where DRA
 is unavailable, this was the original plan, and would make for
 one less 'technique' of opening the client, which is really just
 API bloat at this point, anyway.

Which back ends still don't support DRA? I fixed EWS but at that time
ISTR there were still some which weren't supporting it.

I'd quite like to merge evolution-pkcs11 at some point...

-- 
dwmw2



smime.p7s
Description: S/MIME cryptographic signature
___
evolution-hackers mailing list
evolution-hackers@gnome.org
To change your list options or unsubscribe, visit ...
https://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] Refactoring EWS GAL sqlitedb

2014-09-12 Thread David Woodhouse
On Thu, 2014-09-04 at 12:40 -0300, Tristan Van Berkom wrote:
 
 There is the useless overhead of constructing that vcard
 you dont use (the API is more about notifying about contact
 changes in running views), but it should be close enough I think.

EXPLAIN BEGIN
  STMT: SELECT summary.uid, summary.Rev, summary.bdata FROM 'folder_id' AS 
summary
STMT: EXPLAIN QUERY PLAN SELECT summary.uid, summary.Rev, summary.bdata FROM 
'folder_id' AS summary
  PLAN: SCAN TABLE folder_id AS summary
EXPLAIN END
TIME: 2004 ms

$ time sqlite3 contacts.db 'select uid,bdata from folder_id'  /dev/null

real0m0.314s
user0m0.173s
sys 0m0.137s


So that's 2 seconds instead of a third of a second. 

On my latest GAL update, that's a fifth of the time taken:

GAL update completed successfully in 10415825 µs. Added: 42, Changed: 1813, 
Unchanged 236260, Removed: 19

So it might be quite nice to ditch the generation of the EContacts. But
really, given that this update used to take tens of minutes, I'm not
going to worry too much :)

I'm actually more interested in the time it takes for autocompletion. 

-- 
David WoodhouseOpen Source Technology Centre
david.woodho...@intel.com  Intel Corporation


smime.p7s
Description: S/MIME cryptographic signature
___
evolution-hackers mailing list
evolution-hackers@gnome.org
To change your list options or unsubscribe, visit ...
https://mail.gnome.org/mailman/listinfo/evolution-hackers


[Evolution-hackers] String freeze break request for evolution-data-server (3.12)

2014-09-09 Thread David Woodhouse
I'd like to backport some fixes from Evolution master to 3.12 which fix
GSSAPI HTTP authentication and error reporting¹.

Previously, if we were unable to translate an error number into a string
we would end up with an untranslated message of the form 'Unknown error
code' or 'Unknown code %d' from libcom_err.

Now the code is fixed, we end up handling the lookup failure in EDS, and
report it slightly more coherently as
 _((Unknown GSSAPI mechanism code: %x))

So we have a new untranslated string... in place of a string which was
previously not only untranslated but untranslatable. In a case that
should hopefully never happen now that we're actually looking up the
error codes the right way, and where the message text wasn't giving the
user much information anyway (only the number might *possibly* be
helpful.

On the whole, I'm not going to lose sleep over it being untranslated :)

-- 
David WoodhouseOpen Source Technology Centre
david.woodho...@intel.com  Intel Corporation

¹ Specifically commits a523ac27, bd843434, f98caca8, 581c32aa.


smime.p7s
Description: S/MIME cryptographic signature
___
evolution-hackers mailing list
evolution-hackers@gnome.org
To change your list options or unsubscribe, visit ...
https://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] Refactoring EWS GAL sqlitedb

2014-09-04 Thread David Woodhouse
On Thu, 2014-09-04 at 07:07 +0200, Milan Crha wrote:
 Hi,
 I see. Do you compute the hash on your own, or it is provided by the 
 OAB? 

I'm computing it on my own. I'm just using the SHA1 of the binary record
in the OAB file, since that's easy enough to calculate. Then stick it in
the 'bdata' field in the sqlite.

The update process in my tree now goes as follows:

 - Build a GHashTable of uid - bdata (i.e. sha1) for each entry in the
   addressbook¹.

 - For each item in the new OAB:

   - look up the UID in the hash table to find the old SHA1 if it
 exists, and *remove* the UID from the hash table.

   - If the record existed and the SHA1 matches the record now bring
 processed, do nothing.

   - If the SHA1 doesn't match or the record didn't previously exist,
 insert it into the database.

 - When all records are processed, any UID *left* in the GHashTable is
   for an entry that no longer exists. Delete them. 

This works fairly nicely, although there are some optimisations I can
still do.

Firstly, I'd like a way to 'SELECT uid,bdata from folder_id'. Getting
the initial list of UIDs to build my hash table takes about 95ms.
Getting the bdata for each one individually after that takes another 7½
seconds. Tristan?

(I did try it with the e_book_sqlite_get_extra_data() call in the OAB
processing loop instead of storing them in the GHashTable values in
advance. That was actually slower; overall update time for a NULL update
went from 33 seconds to 38 seconds.

The majority of the time is still taken in the OAB binary processing. We
actually generate the full EContact for every record, *then* pass it
back up to do the SHA1 comparison and delete it unused for most records.
I should fix it to do the SHA1 comparison first.

-- 
dwmw2


smime.p7s
Description: S/MIME cryptographic signature
___
evolution-hackers mailing list
evolution-hackers@gnome.org
To change your list options or unsubscribe, visit ...
https://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] Refactoring EWS GAL sqlitedb

2014-09-03 Thread David Woodhouse
On Wed, 2014-09-03 at 14:18 +0200, Milan Crha wrote:
 Are you able to recognize affected contacts from the binary diff, or 
 from anything after the diff is applied? 

Well... theoretically perhaps. With a binary diff of course a lot of
data from the original file is preserved. So any contact contained
entirely within an unchanged part of the file should be unchanged...
although it *might* have changed its offset within the file¹.

It would be fairly non-trivial to hook into the low-level LZX
decompression code and get a list of '{un,}changed ranges' though.
Especially when we're applying more than one delta at a time.

-- 
dwmw2

¹ which matters if we're storing the file offset so that we can recreate
  the vcard on demand from the original.


smime.p7s
Description: S/MIME cryptographic signature
___
evolution-hackers mailing list
evolution-hackers@gnome.org
To change your list options or unsubscribe, visit ...
https://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] Refactoring EWS GAL sqlitedb

2014-09-03 Thread David Woodhouse
On Wed, 2014-09-03 at 18:22 +0200, Milan Crha wrote:
 then I'd go by the PidTagLastModified way, and rely on it in case it's 
 available. That way you'll update only offsets and changed contacts 
 (plus drop those removed). Having the PidTagLastModified easily 
 accessible from the table as such, without a need to parse the vCard, 
 will be a requirement.

I don't have PidTagLastModified. I have the list below. But I don't
think I need it. The sha1 of the binary record ought to suffice.

Total records is 238198 
prop 681f
prop 6804001e
prop 68010003
prop 6802001e
prop 8c98001e
prop 3003001e
prop 39fe001f
prop 3a19001f
prop 800f101f
prop 8c8e001f
prop 3a06001f
prop 8c8f001f
prop 3a11001f
prop 3a1f
prop 8c92001f
prop 3001001f
prop 8ce30003
prop 8ce20003
prop 8cb5000b
prop 806a0003
prop 8cac101f
prop 8c6d0102
prop 8cdd000b
prop 8ca3
prop 39050003
prop 39ff001e
prop 8006001e
prop 8c6a1102
prop 3a701102
prop 3a220102
prop 3004001f
prop 3a21001f
prop 3a2e001f
prop 3a1c001f
prop 3a23001f
prop 3a2f101f
prop 3a1b101f
prop 3a09001f
prop 8011001f
prop 8c90001f
prop 3a18001f
prop 3a30001f
prop 8c91001f
prop 3a16001f
prop 3a17001f
prop 3a26001f
prop 3a2a001f
prop 3a28001f
prop 3a27001f
prop 3a29001f
prop 3a0a001f
prop 3a08001f
prop 3a4b
prop ffe0003
prop 3903
prop 8009101e
prop 8008101e
prop 8cdb000d
prop 8cda000d
prop 8cd9000d
prop 8cd8000d
prop 8cc20102
prop 8c9e0102
prop 68051003


-- 
dwmw2


smime.p7s
Description: S/MIME cryptographic signature
___
evolution-hackers mailing list
evolution-hackers@gnome.org
To change your list options or unsubscribe, visit ...
https://mail.gnome.org/mailman/listinfo/evolution-hackers


[Evolution-hackers] [PATCH 3/6] test-book-client-custom-summary: Add 'exists' tests for email and X509 certificates

2014-08-26 Thread David Woodhouse
---
 .../client/test-book-client-custom-summary.c   | 25 ++
 tests/libebook/data/vcards/custom-7.vcf|  1 +
 2 files changed, 26 insertions(+)

diff --git a/tests/libebook/client/test-book-client-custom-summary.c 
b/tests/libebook/client/test-book-client-custom-summary.c
index ec3f59c..f17f56d 100644
--- a/tests/libebook/client/test-book-client-custom-summary.c
+++ b/tests/libebook/client/test-book-client-custom-summary.c
@@ -762,6 +762,30 @@ main (gint argc,
suites[i].custom,
FALSE);
 
+   add_client_test (
+   suites[i].prefix,
+   /Exists/Email,
+   suites[i].func,
+   e_book_query_field_exists (
+   E_CONTACT_EMAIL),
+   /* There are 13 contacts with email addresses */
+   13,
+   suites[i].direct,
+   suites[i].custom,
+   FALSE);
+
+   add_client_test (
+   suites[i].prefix,
+   /Exists/X509,
+   suites[i].func,
+   e_book_query_field_exists (
+   E_CONTACT_X509_CERT),
+   /* There is 1 contact with a cert listed */
+   1,
+   suites[i].direct,
+   suites[i].custom,
+   FALSE);
+
/*
 *PHONE NUMBERS  *
 */
diff --git a/tests/libebook/data/vcards/custom-7.vcf 
b/tests/libebook/data/vcards/custom-7.vcf
index 2141e36..dae210a 100644
--- a/tests/libebook/data/vcards/custom-7.vcf
+++ b/tests/libebook/data/vcards/custom-7.vcf
@@ -2,4 +2,5 @@ BEGIN:VCARD
 UID:custom-7
 FN:Purple Goose
 TEL;HOME:+49-89-7888 99
+KEY;ENCODING=b;TYPE=X509:AA==
 END:VCARD
-- 
1.9.3


-- 
David WoodhouseOpen Source Technology Centre
david.woodho...@intel.com  Intel Corporation


smime.p7s
Description: S/MIME cryptographic signature
___
evolution-hackers mailing list
evolution-hackers@gnome.org
To change your list options or unsubscribe, visit ...
https://mail.gnome.org/mailman/listinfo/evolution-hackers


[Evolution-hackers] [PATCH 5/6] EBookSqlite: Add support for storing x509Cert in a boolean summary field

2014-08-26 Thread David Woodhouse
 
(test-field_id),
+   EBSQL_STATUS_STR 
(context-status)));
+   }
break;
 
case BOOK_QUERY_EXISTS:
@@ -4545,11 +4570,15 @@ query_preflight_check (PreflightContext *context,
case E_BOOK_QUERY_REGEX_NORMAL:
 
/* All of these queries can only apply to string fields,
-* or fields which hold multiple strings 
+* or fields which hold multiple strings. Except for
+* BOOK_QUERY_EXISTS which can be used for the cert 
field.
 */
if (test-field) {
-   if (test-field-type != G_TYPE_STRING 
-   test-field-type != 
E_TYPE_CONTACT_ATTR_LIST) {
+   if (test-field-type == E_TYPE_CONTACT_CERT) {
+   if (field_test != BOOK_QUERY_EXISTS)
+   goto no_cert_content;
+   } else if (test-field-type != G_TYPE_STRING 
+  test-field-type != 
E_TYPE_CONTACT_ATTR_LIST) {
context-status = MAX (context-status, 
PREFLIGHT_INVALID);
EBSQL_NOTE (
PREFLIGHT,
@@ -5068,7 +5097,11 @@ field_test_query_exists (EBookSqlite *ebsql,
SummaryField *field = test-field;
 
ebsql_string_append_column (string, field, NULL);
-   ebsql_string_append_printf (string,  IS NOT NULL);
+
+   if (test-field-type == E_TYPE_CONTACT_CERT)
+   ebsql_string_append_printf (string,  IS NOT '0');
+   else
+   ebsql_string_append_printf (string,  IS NOT NULL);
 }
 
 /* Lookup table for field test generators per EBookQueryTest,
-- 
1.9.3


-- 
David WoodhouseOpen Source Technology Centre
david.woodho...@intel.com  Intel Corporation


smime.p7s
Description: S/MIME cryptographic signature
___
evolution-hackers mailing list
evolution-hackers@gnome.org
To change your list options or unsubscribe, visit ...
https://mail.gnome.org/mailman/listinfo/evolution-hackers


[Evolution-hackers] [PATCH 4/6] EBookSqlite: Support 'exists' queries from summary

2014-08-26 Thread David Woodhouse
---
 addressbook/libedata-book/e-book-sqlite.c | 7 ++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/addressbook/libedata-book/e-book-sqlite.c 
b/addressbook/libedata-book/e-book-sqlite.c
index 9131c09..13a4843 100644
--- a/addressbook/libedata-book/e-book-sqlite.c
+++ b/addressbook/libedata-book/e-book-sqlite.c
@@ -4233,7 +4233,12 @@ func_check (struct _ESExp *f,
 
query_type = GPOINTER_TO_UINT (data);
 
-   if (argc == 2 
+   if (argc == 1 
+   argv[0]-type == ESEXP_RES_STRING) {
+   query_name = argv[0]-value.string;
+
+   field_id = e_contact_field_id (query_name);
+   } else if (argc == 2 
argv[0]-type == ESEXP_RES_STRING 
argv[1]-type == ESEXP_RES_STRING) {
query_name = argv[0]-value.string;
-- 
1.9.3


-- 
David WoodhouseOpen Source Technology Centre
david.woodho...@intel.com  Intel Corporation


smime.p7s
Description: S/MIME cryptographic signature
___
evolution-hackers mailing list
evolution-hackers@gnome.org
To change your list options or unsubscribe, visit ...
https://mail.gnome.org/mailman/listinfo/evolution-hackers


[Evolution-hackers] EBookSqlite support for EWS

2014-08-22 Thread David Woodhouse
I accidentally¹ ported the EWS addressbook to EBookSqlite. Patch
(against 3.12.5) attached.

It ends up hitting this check and not actually doing any database
writes:

(evolution-addressbook-factory:19473): libedata-book-WARNING **: The
GCancellable passed to `e_book_sqlite_add_contacts' is not the same as
the cancel object passed to e_book_sqlite_lock()

I think the check is wrong, because we're not using e_book_sqlite_lock()
at all. Does it make sense to fix it thus: 

diff --git a/addressbook/libedata-book/e-book-sqlite.c 
b/addressbook/libedata-book/e-book-sqlite.c
index 0ae89cd..9e5cb78 100644
--- a/addressbook/libedata-book/e-book-sqlite.c
+++ b/addressbook/libedata-book/e-book-sqlite.c
@@ -209,7 +209,7 @@ ebsql_init_debug (void)
 #define EBSQL_LOCK_OR_RETURN(ebsql, cancellable, val) \
G_STMT_START { \
EBSQL_LOCK_MUTEX ((ebsql)-priv-lock); \
-   if (cancellable != NULL  \
+   if (cancellable != NULL  (ebsql)-priv-cancel  \
(ebsql)-priv-cancel != cancellable) { \
g_warning (The GCancellable passed to `%s'  \
   is not the same as the cancel object  \



-- 
dwmw2

¹ Seriously, it *was* an accident. I thought I needed to port to
  EBookSqlite to make cursors work, which I need for Yuuma's PKCS#11
  module. But EBookBackendSqliteDB can do cursors anyway, so I didn't
  need to do the conversion. Not that it's not worthwhile anyway...

diff --git a/src/addressbook/e-book-backend-ews.c b/src/addressbook/e-book-backend-ews.c
index 3315475..ec7bf38 100644
--- a/src/addressbook/e-book-backend-ews.c
+++ b/src/addressbook/e-book-backend-ews.c
@@ -73,7 +73,7 @@ struct _EBookBackendEwsPrivate {
 	gchar *oab_url;
 	gchar *folder_name;
 
-	EBookBackendSqliteDB *summary;
+	EBookSqlite *summary;
 
 	gboolean only_if_exists;
 	gboolean is_writable;
@@ -1371,7 +1371,7 @@ ews_create_contact_cb (GObject *object,
 
 		e_contact_set (create_contact-contact, E_CONTACT_UID, item_id-id);
 		e_contact_set (create_contact-contact, E_CONTACT_REV, item_id-change_key);
-		e_book_backend_sqlitedb_new_contact (ebews-priv-summary, ebews-priv-folder_id, create_contact-contact, TRUE, error);
+		e_book_sqlite_add_contact (ebews-priv-summary, create_contact-contact, NULL, TRUE, create_contact-cancellable, error);
 
 		if (error == NULL) {
 			GSList *contacts;
@@ -1510,6 +1510,7 @@ typedef struct {
 	EDataBook *book;
 	guint32 opid;
 	GSList *sl_ids;
+	GCancellable *cancellable;
 } EwsRemoveContact;
 
 static void
@@ -1529,7 +1530,7 @@ ews_book_remove_contact_cb (GObject *object,
 	g_return_if_fail (priv-summary != NULL);
 
 	if (!g_simple_async_result_propagate_error (simple, error))
-		deleted = e_book_backend_sqlitedb_remove_contacts (priv-summary, priv-folder_id, remove_contact-sl_ids, error);
+		deleted = e_book_sqlite_remove_contacts (priv-summary, remove_contact-sl_ids, remove_contact-cancellable, error);
 
 	if (deleted)
 		e_data_book_respond_remove_contacts (remove_contact-book, remove_contact-opid, EDB_ERROR (SUCCESS),  remove_contact-sl_ids);
@@ -1542,6 +1543,7 @@ ews_book_remove_contact_cb (GObject *object,
 	g_slist_free_full (remove_contact-sl_ids, g_free);
 	g_object_unref (remove_contact-ebews);
 	g_object_unref (remove_contact-book);
+	g_object_unref (remove_contact-cancellable);
 	g_free (remove_contact);
 	g_clear_error (error);
 }
@@ -1593,6 +1595,7 @@ e_book_backend_ews_remove_contacts (EBookBackend *backend,
 	remove_contact-book = g_object_ref (book);
 	remove_contact-opid = opid;
 	remove_contact-sl_ids = copy;
+	remove_contact-cancellable = g_object_ref(cancellable);
 
 	e_ews_connection_delete_items (
 		priv-cnc, EWS_PRIORITY_MEDIUM, (GSList *) id_list,
@@ -1647,12 +1650,11 @@ ews_modify_contact_cb (GObject *object,
 
 		id = e_contact_get (modify_contact-old_contact, E_CONTACT_UID);
 
-		e_book_backend_sqlitedb_remove_contact (priv-summary, priv-folder_id, id, error);
-		e_book_backend_sqlitedb_new_contact (
+		e_book_sqlite_remove_contact (ebews-priv-summary, id, modify_contact-cancellable, error);
+		e_book_sqlite_add_contact (
 ebews-priv-summary,
-ebews-priv-folder_id,
-modify_contact-new_contact,
-TRUE,
+modify_contact-new_contact, NULL,
+TRUE, modify_contact-cancellable,
 error);
 
 		if (error == NULL) {
@@ -1830,10 +1832,7 @@ e_book_backend_ews_modify_contacts (EBookBackend *backend,
 	id-id = e_contact_get (contact, E_CONTACT_UID);
 	id-change_key = e_contact_get (contact, E_CONTACT_REV);
 
-	old_contact = e_book_backend_sqlitedb_get_contact (
-		priv-summary, priv-folder_id,
-		id-id, NULL, NULL, error);
-	if (!old_contact) {
+	if (!e_book_sqlite_get_contact (priv-summary, id-id, TRUE, old_contact, error)) {
 		g_object_unref (contact);
 		e_data_book_respond_modify_contacts (book, opid, EDB_ERROR (NOT_SUPPORTED), NULL);
 		return;
@@ -1900,20 +1899,25 @@ e_book_backend_ews_get_contact_list (EBookBackend *backend,
 	GError *error = NULL;
 	

Re: [Evolution-hackers] EBookSqlite support for EWS

2014-08-22 Thread David Woodhouse
On Fri, 2014-08-22 at 17:04 -0500, David Woodhouse wrote:
 
 ¹ Seriously, it *was* an accident. I thought I needed to port to
   EBookSqlite to make cursors work, which I need for Yuuma's PKCS#11
   module. But EBookBackendSqliteDB can do cursors anyway, so I didn't
   need to do the conversion. Not that it's not worthwhile anyway...

Hm, actually I think maybe it *is* necessary. Sure, we have 
e_book_backend_sqlitedb_cursor_new() but we can't make an
EDataBookCursor out of that. Commit a8e5f5c0 ported
EDataBookCursorSqlite over to be based on EBookSqlite and now it doesn't
work for EBookBackendSqliteDB any more.

But that's OK; I've done the port to EBookSqlite now anyway. And as an
added bonus, as it migrates to the new database format I can add new
summary fields. With the new PKCS#11 module we're *really* going to want
e_book_query_field_exists(E_CONTACT_X509_CERT) to be fast.

Unfortunately, we only support boolean and string fields in the summary;
we can't just add E_CONTACT_X509_CERT to the list of summary fields.

Although actually, all I *wanted* was a boolean. I didn't really want
the whole of the cert data to be duplicated in the summary; I only
wanted a boolean indication that the cert was *present*...

Tristan, any ideas on how best to handle this?

-- 
dwmw2



smime.p7s
Description: S/MIME cryptographic signature
___
evolution-hackers mailing list
evolution-hackers@gnome.org
To change your list options or unsubscribe, visit ...
https://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] GSoC project

2014-03-04 Thread David Woodhouse
On Tue, 2014-03-04 at 00:55 -0500, Watson Sato wrote:
 Hi,
 
 My name is Watson, I'm new to the open source culture.
 
 I'm from Brazil, graduated in computer science. I've been working with
 Information Security for 4 years now, dealing with PKI, HSMs, tokens,
 PKCS#11, OpenSSL and stuff. I have always worked with closed source, I'd
 like to start working with open source software now, try to contribute.
 
 I got interested in the gsoc idea of implementing the small PKCS#11 module
 to handle certificates. I'm getting acquainted to Evolution source code.
 
 I tried to reproduce the related error described in Bug 704246, but I
 couldn't, certainly I'm missing something.

Hi, thanks for your email.

What you need to do to reproduce the error is create a contact in the
addressbook with a KEY: field, as shown in the example.

Do *not* add a certificate in the cert database for that person.

Then try sending an encrypted mail to them.

-- 
dwmw2


smime.p7s
Description: S/MIME cryptographic signature
___
evolution-hackers mailing list
evolution-hackers@gnome.org
To change your list options or unsubscribe, visit ...
https://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] Evolution EWS: Planning the 3.12

2013-08-30 Thread David Woodhouse
On Mon, 2013-08-26 at 08:27 +0200, Fabiano Fidencio wrote:
 People,
 
 I've been planning what I want to have implemented for 3.12, for
 evolution-ews.
 If you are interested, please, take a look into:
 http://blog.fidencio.org/2013/08/evolution-ews-planning-312.html
 
 Feel free to ping in #evolution or reply this message, if you want to
 discuss something about the planning.

Looks good; thanks. I'd have been happier if IRM support (bug 691103)
had been on it though :)

-- 
dwmw2



smime.p7s
Description: S/MIME cryptographic signature
___
evolution-hackers mailing list
evolution-hackers@gnome.org
To change your list options or unsubscribe, visit ...
https://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] Reconsidering our release cycle

2013-07-26 Thread David Woodhouse
On Fri, 2013-07-26 at 07:38 -0400, Matthew Barnes wrote:
 
 * I think our policy toward stable branch maintenance could be a little
   more relaxed during the first half of a release cycle in terms of what
   we allow in.  I don't know what that means exactly for us yet, but I
   have observed RHEL updates do sometimes include new features.  Maybe
   enterprise policies could be a rough guide until we find our footing.

Well, you lot haven't *actually* shouted at me very much for some of the
stuff I've added in stable branches. I get more pushback from the
translators if I have to add new strings, than I did for the whole of
the mspack and GAL rework which landed in 3.8 for example.

I'm happy enough with that :)

-- 
David WoodhouseOpen Source Technology Centre
david.woodho...@intel.com  Intel Corporation


smime.p7s
Description: S/MIME cryptographic signature
___
evolution-hackers mailing list
evolution-hackers@gnome.org
To change your list options or unsubscribe, visit ...
https://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] Reconsidering our release cycle

2013-07-24 Thread David Woodhouse

 On Tue, Jul 23, 2013 at 7:40 PM, Matthew Barnes mbar...@redhat.com wrote:
  Increasingly I'm feeling like the traditional 6-month release cycle is
  just too short for Evolution.  In terms of development, we have a pretty
  short window for landing major changes and allowing adequate time for
  testing before development freezes set in.
 
 I  like the idea very much and had similar plans before, but never
 went forward with it before.

I think I like the idea.

My concern is that it could also be longer before new features and fixes
actually make it into a release. For example, if we were on an annual
schedule and people were still using Evolution 3.6 today instead of
Evolution 3.8 we'd still be having to kill evolution-source-registry
after connecting to the VPN if we actually want to see our calendars.

And if a distribution ships a few weeks before a release, that now means
they can be shipping a version of Evolution which is a *year* old,
instead of only six months old.

Life would be a lot easier even with the existing release cycle if we
weren't so incestuously tied to the latest GNOME release (and if the
GNOME libraries didn't make compatibility so hard by deprecating things
so quickly). Obviously we'll have to fix that anyway, and I wonder how
much of the proposed benefit we could achieve by *only* fixing that,
while sticking with 6-monthly releases on the GNOME schedule?

For example, if users could use Evolution 3.8 on top of GNOME 3.6, and
upgrading didn't mean they have to rebuild their *whole* world with it,
the existing situation wouldn't be quite so bad anyway, would it? 

  But more importantly, our users seem to be constantly playing catch-up
  in terms of supported releases.  Because of the delay between upstream
  releases and distro releases, by the time users finally upgrade to a
  newer Evolution, more often than not they're upgrading to a version
  that's either nearing the tail end of its 6-month support window or is
  already unsupported.

But we've never *prevented* any distro packager from continuing to do
maintenance for older releases. I did it for 2.32 for a while, when
MeeGo was using that.

If distributions want to cherry-pick patches back into an unsupported
release that they're using, we should encourage them to do so.

  That's frustrating, both for users and for me as a developer, but we
  just don't have the manpower to support multiple stable releases and
  still get any kind of significant development work done.
 
  I'd like us to consider moving to a 12-month release cycle, which would
  sync up with GNOME's release schedule annually instead of semi-annually.
 
  Here's my initial proposal, if you guys are open to this:

Please take the above comments as just thinking aloud, rather than
objections. I'm more than happy to attempt what you propose.

On Tue, 2013-07-23 at 21:28 +0530, Srinivasa Ragavan wrote:
 The challenge will be to sync properly with the GNOME freezes during
 the second half of the cycle. It will be good to sync with that, so
 that when the product releases with GNOME release, there is doc /
 translation all ready.

Agreed.

 I really wouldn't want EDS to be part of this, if we ever want it to
 be a proper platform/core material. Just Evolution would be better fit
 for this model IMHO.

I don't think that makes sense. As Fabiano points out, Evo and EDS are
*very* closely tied. Even in the *stable* branch in 3.8.4 there are
fixes for EDS/EWS which require corresponding fixes in Evo.

Breaking the close version ties with the rest of GNOME makes sense, but
not between Evo and EDS.

-- 
dwmw2



smime.p7s
Description: S/MIME cryptographic signature
___
evolution-hackers mailing list
evolution-hackers@gnome.org
To change your list options or unsubscribe, visit ...
https://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] Reconsidering our release cycle

2013-07-24 Thread David Woodhouse
On Wed, 2013-07-24 at 02:54 +0200, Fabiano Fidêncio wrote:
 
 I also fully agree with your suggestion.
 
 As we have discussed, users are reporting bugs against 3.8.x now and
 they will need to wait at least 6 months before they get a fix in
 3.10.x. I mean, from the stability point of view it would be great if
 we have a larger window to work, test and correct our mistakes..

Am I the only one who fixes bugs and implements features on the
gnome-3-8 branch, then commits, cherry-picks that to master, and *then*
cherry-picks it back to gnome-3-8 (with the -x option) to make it look
like it happened in master first? 

-- 
dwmw2



smime.p7s
Description: S/MIME cryptographic signature
___
evolution-hackers mailing list
evolution-hackers@gnome.org
To change your list options or unsubscribe, visit ...
https://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] Placing of goto labels

2013-05-30 Thread David Woodhouse
On Thu, 2013-03-28 at 09:30 +0100, Milan Crha wrote:
 What's the usual way for goto labels in other projects?
 Would it make sense to write labels 'one-space indented'?

FWIW one-space indented is what emacs does by default, and is thus what
you'll usually see in code that I've written.

-- 
David WoodhouseOpen Source Technology Centre
david.woodho...@intel.com  Intel Corporation





smime.p7s
Description: S/MIME cryptographic signature
___
evolution-hackers mailing list
evolution-hackers@gnome.org
To change your list options or unsubscribe, visit ...
https://mail.gnome.org/mailman/listinfo/evolution-hackers


[Evolution-hackers] Backporting EWS GAL changes to 3.8

2013-05-30 Thread David Woodhouse

I've implemented a bunch of fixes to the offline addressbook handling in
evo-ews master, and I'd like to backport them to 3.8.

In fact I've done *all* my testing on the gnome-3-8 branch and haven't
tested master at all as I've been working. (Naughty dwmw2).

The code I'd like to push is at git:// or
http://git.infradead.org/users/dwmw2/evolution-ews.git

We had a local copy of the LZX decompression code, taken from libmspack
and then hacked up somewhat to support the variant that the EWS GAL
uses. I cleaned up that code and got it accepted into upstream
libmspack, and libmspack 0.4 has been released a week or two ago.

As part of that process, the libmspack maintainer spotted a number of
bugs in our changes. These have all been fixed in the 0.4 release.

I've fixed the build in master so that we either need to build with
libmspack 0.4 as an additional dependency (recommended), or use a
configure option to build with our internal version of the decompression
code instead. If you configure without an appropriate libmspack, the
resulting error will tell you about the --with-internal-lzx option.

I've fixed the most egregious of the bugs in our internal copy, but we'd
still be better off using the real libmspack. Our internal version
doesn't check CRC on its output, for one thing. And it's non-trivial to
add, because of the differences between the two code bases.

As well as the LZX fixes, I've also implemented incremental downloads.
My own company's addressbook was a 30M download for the full database,
and it changes every 12 hours. Now it's only a few hundred KiB each
time. As I have an an ADSL line where I pay for daytime bandwidth, I
consider this an important bug fix rather than purely an enhancement :)

So... I understand that we don't like forcing people to add new
dependencies or even to change their ./configure flags in a stable
release, but I think that backporting all this as-is is actually the
right thing to do. We should never have been shipping a hacked-up copy
of a third-party shared library *anyway*. I know it violates the Fedora
packaging guidelines, and probably for other distributions too?

Opinions? I know Milan wasn't happy about even requiring the
--with-internal-lzx option to preserve the (broken) status quo in 3.8,
but then disappeared from IRC before I could try to justify its
necessity. We *could* make it automatically do the right thing, using
libmspack if it's available or silently falling back to the internal
version. But some people hate configure scripts doing that, and I've
already changed master *not* to do it that way...

-- 
dwmw2



smime.p7s
Description: S/MIME cryptographic signature
___
evolution-hackers mailing list
evolution-hackers@gnome.org
To change your list options or unsubscribe, visit ...
https://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] Backporting EWS GAL changes to 3.8

2013-05-30 Thread David Woodhouse
On Thu, 2013-05-30 at 07:41 -0400, Matthew Barnes wrote:
 
 That's the right way.
 
 A two week-old release is far too new for a stable branch dependency,
 especially one we're introducing mid-stream.  It needs to be optional
 for now, but at the same time we do want packagers to be aware of it. 
 Aborting the configure script by default with a workaround message if
 libmspack 0.4 is not present accomplishes that.

(I know we care about more than just Fedora, but FWIW libmspack 0.4 is
in updates-testing for Fedora 19 and the Fedora packages of evo-ews
3.8.3 should be built to use it, if we do go ahead and push this.)

 We're giving special attention to the stable release this cycle, so I'm
 willing to bend the rules a bit.
 
 +1 from me.

Thanks. I'll wait to see if I've won Milan round, before doing so...

 I think we could remove the internal lzx copy as early as 3.11.

Note that I'm actually thinking of *disabling* the incremental-update
feature in the case where we're using the internal LZX code. It's all
very well not bothering to check CRCs when we're just doing a simple
decompression of the full file, but when we are applying multiple binary
deltas starting from a file we found in our local cache from last time,
we *really* ought to be checking the output more carefully.

I'm not really sure that affects the question I actually asked about
backporting to 3.8, either way; just reinforces the you *really* ought
to be using the proper library assertion. And it means I'd prefer to
remove the fallback before 3.10, rather than after it.

-- 
dwmw2


smime.p7s
Description: S/MIME cryptographic signature
___
evolution-hackers mailing list
evolution-hackers@gnome.org
To change your list options or unsubscribe, visit ...
https://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] Backporting EWS GAL changes to 3.8

2013-05-30 Thread David Woodhouse
On Thu, 2013-05-30 at 20:03 +0200, Milan Crha wrote:
 
 I expect, personally, if .2 release was building without any issue, that
 I just fire the build and it'll build the same flawlessly also in .3+.
 Thus I suggested on IRC to do the check on the stable version opposite,
 add an option to actually use the stock libmspack, instead of forcing
 not using it. Distro packagers sort of expect such (and bigger) changes
 when packaging new major version, thus that's all fine on master.

Yeah, I understand that and completely agree in the general case.

However, in this case the old build was *broken*. We really *should* be
using the proper libmspack. In this case I *want* to make people jump
through extra hoops if they want to continue to build this the broken
way, after I've gone to the trouble of fixing it :)

It's not as if adding --with-internal-lzx is *much* extra work, if
that's really what they decide to do.

 If you feel like me doing things unnecessary overcomplicated, then feel
 free to commit it as is. After all, it's only my opinion (in a good
 meaning of the words).

If it makes you feel better about the packaging work, I can go and file
a Fedora bug for the violation of the packaging guidelines; shipping a
hacked-up copy of parts of libmspack instead of using the real one? :)

When you package 3.8.3 for Fedora 19, you will *definitely* need to add
libmspack-devel as a BuildRequires, as part of the fix for that. If I
commit things as-is, that's *all* you'll have to do.

If I change things around as you suggest, you'll actually have to do
*more* work because you'll have to add the --without-internal-lzx option
to the configure line too :)

-- 
dwmw2



smime.p7s
Description: S/MIME cryptographic signature
___
evolution-hackers mailing list
evolution-hackers@gnome.org
To change your list options or unsubscribe, visit ...
https://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] Is it worth customizing the IMAP headers to fetch?

2012-12-15 Thread David Woodhouse
On Sat, 2012-12-15 at 13:32 -0500, Matthew Barnes wrote:
 Currently IMAPX just fetches all headers using RFC822.HEADER.
 
 Supporting the plugin would change this to use:
 
BODY.PEEK ([HEADERS | predefined-header-list [custom-header-list])
 
 depending on the plugin options selected (see attached screenshot).
 
 I'm skeptical about the real benefit of this, but I lack expertise in
 IMAP and particularly the IMAPX backend to make an informed judgment.

It's basically about download time/bandwidth. If you fetch *all* the
headers, that causes a fair amount of network traffic that isn't
strictly necessary.

 My intuition tells me the vast majority of users won't care or even
 understand what these options do, and those that do tinker with them
 probably won't notice any significant difference in download times.

My intuition is the same... although we could probably run a quick check
on our own mail stores and see the full size of *all* headers vs. the
size of the interesting headers that Evolution actually uses for
filtering, etc.

The interesting use case, perhaps, is the first run of Evolution on a
new mail store. That's when the full vs. targeted header fetch is going
to be most noticeable. 

But even if we conclude that the more specific fetch is worthwhile, I'm
still not convinced that a separate UI for *choosing* those headers is
at all sane. If the user sets up filters which *look* at a given header,
then we damn well ought to fetch that header automatically. The user
shouldn't be forced to go and find a separate plugin to make their
filter actually work. Or do I misunderstand how this works?

-- 
dwmw2



smime.p7s
Description: S/MIME cryptographic signature
___
evolution-hackers mailing list
evolution-hackers@gnome.org
To change your list options or unsubscribe, visit ...
https://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] Is it worth customizing the IMAP headers to fetch?

2012-12-15 Thread David Woodhouse
On Sat, 2012-12-15 at 17:15 -0500, Matthew Barnes wrote:
 
  But even if we conclude that the more specific fetch is worthwhile, I'm
  still not convinced that a separate UI for *choosing* those headers is
  at all sane. If the user sets up filters which *look* at a given header,
  then we damn well ought to fetch that header automatically. The user
  shouldn't be forced to go and find a separate plugin to make their
  filter actually work. Or do I misunderstand how this works?
 
 No you're right, I think that's the sole purpose of the Custom Headers
 section.

Note that there's a difference between filters which apply to new
messages (and hence should be able to operate on the currently-relevant
set of headers), and searches which apply to *old* messages too —
messages which may have been downloaded before the user ever *asked* us
to search on their X-Esoteric-Foo: header.

But that's why the IMAP SEARCH command exists... we should make better
use of it.

-- 
dwmw2



smime.p7s
Description: S/MIME cryptographic signature
___
evolution-hackers mailing list
evolution-hackers@gnome.org
To change your list options or unsubscribe, visit ...
https://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] Drop or limit ChangeLog files in tarball releases?

2012-11-20 Thread David Woodhouse
On Tue, 2012-11-20 at 17:00 +0100, Milan Crha wrote:
 Hi,
 I'm wondering about usability of git-generated ChangeLog files in
 tarball releases.

I'd say drop them. Using 'git log' is only one way of tracking
historical changes, and it's rarely useful all on its own.

This is especially true in a project where changes often aren't always
cleanly split into separate commits as is my personal preference, and
one commit may cover a *lot* of separate changes.

I often find myself using 'git blame', or 'git log -- $FILENAME', or if
I *do* use the 'git log' output it'll be coupled with 'git show $COMMIT'
to look at individual commits. So having the 'git log' output
pre-packaged in a text file doesn't really do much useful for me.

What would be *more* useful, perhaps, is publishing a tarball of a
'.git' directory which can be extracted into the same location as the
tarball, to turn it into a clean checkout of the source tree at the
appropriate tag.

-- 
dwmw2



smime.p7s
Description: S/MIME cryptographic signature
___
evolution-hackers mailing list
evolution-hackers@gnome.org
To change your list options or unsubscribe, visit ...
https://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] Return back folder remember for Open/Save dialog

2012-07-24 Thread David Woodhouse
On Mon, 2012-07-23 at 19:22 +0100, Philip Withnall wrote:
 That effort would better be put into improving GTK+ itself for
 everyone, rather than just fixing up Evolution. 

In the general case of course that's true, but where GTK+ is hell-bent
on making its widgets harder to use and actively *resists* all attempts
to improve it back to how it used to be, we end up having to do things
for ourselves.

Or are you suggesting that we could provide a new GTK+
GtkCrackFreeFileChooser widget available for everyone? I think that
would meet with the same resistance from the GTK+ folks.

-- 
dwmw2


smime.p7s
Description: S/MIME cryptographic signature
___
evolution-hackers mailing list
evolution-hackers@gnome.org
To change your list options or unsubscribe, visit ...
https://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] Return back folder remember for Open/Save dialog

2012-07-23 Thread David Woodhouse
On Mon, 2012-07-02 at 09:19 -0400, Matthew Barnes wrote:
 On Mon, 2012-07-02 at 14:47 +0200, Milan Crha wrote:
  Well, they do not do it right, and this kind of outsourcing proved to
  be problematic in the past, thus why to rely on something nonworking
  with not our fault, ask them to fix it explanation? I always thought
  we do software for people, not people for software.
 
 Consistency with other GTK+ applications is more important.
 
 If GtkFileChooser has chosen a bad policy, try and open a discussion
 with Federico about it.  Don't just circumvent the policy.  I would
 consider that a regression.

I strongly disagree that consistency with other Gtk+ applications is
more important than basic usability. The GtkFileChooser is just
stunningly broken — and by accepting its stupidity wholesale, that means
that Evolution has regressed in usability.

I agree with Milan that we should switch back to how we were doing
things before.

-- 
dwmw2


smime.p7s
Description: S/MIME cryptographic signature
___
evolution-hackers mailing list
evolution-hackers@gnome.org
To change your list options or unsubscribe, visit ...
https://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] WebKit port of the composer

2012-06-01 Thread David Woodhouse
On Fri, 2012-06-01 at 16:49 +0200, Daniel Vratil wrote:
 If you have any special wish (I bet Andre will come with many feature 
 requests 
 from bugzilla :P ) you'd like to have in the new composer, please share them 
 now. Regarding features, I want to make exact copy of GtkHTMLEditor and only 
 fix the most annoying GtkHTML issues, all crazy ideas will be deferred for 
 3.8 
 :) 

Fixing some of the brokenness of the To/Cc/Bcc headers in the composer
would be wonderful.

Try this in the current composer:
 1. Paste or enter this address into the To: header, exactly as follows:
Woodhouse, David david.woodho...@intel.com
 
 2. Click somewhere *outside* the To: header entry box.

 3. Realise that the name is stupidly backwards and contains a stupid
comma that shouldn't be in an RFC5322 display-name. (Yay Exchange)

 4. Go back to the To: header entry, and put quotes around the
display-name so it looks like
Woodhouse, David david.woodho...@intel.com

 5. Click somewhere outside the entry, again.

 6. Watch the address magically transform itself to nonsense:
Woodhouse, David david.woodho...@intel.com, David 
david.woodho...@intel.com


In the past when our message *display* also gratuitously screwed with
display-names to *remove* the quotes which were necessary to make them
correct, this used to happen quite a lot when addresses were cut and
pasted.

We should also be able to send a mail with the following headers:
  To: Some people I want to invite to my party : ; 
  Bcc: f...@bar.com

Currently I get an SMTP error when I try that, because it treats the
group in the To: header as a single address, and submits it in 
RCPT TO:Some people... party : ;

-- 
dwmw2


smime.p7s
Description: S/MIME cryptographic signature
___
evolution-hackers mailing list
evolution-hackers@gnome.org
To change your list options or unsubscribe, visit ...
https://mail.gnome.org/mailman/listinfo/evolution-hackers


[Evolution-hackers] Request: New function in evolution-data-server 3.4.3

2012-05-21 Thread David Woodhouse
I just installed Evolution 3.4 and found a massive performance
regression¹ when it notices that a bunch of messages got deleted on the
server (I archive mail quarterly so thousands of messages disappear at
once).

It was taking about 40ms to delete each one, with individual database
mangling for each message. By introducing a new method to do them in a
batch, I cut that down to about 50µs per message.

It's a simple new function camel_folder_summary_remove_uids() which
parallels the existing camel_folder_summary_remove_uid() function.

I'd like to add this to the 3.4.3 release of EDS. Since it's just a new
function, there should be no compatibility issues... but I believe I
need permission from the release team...

-- 
dwmw2

¹ https://bugzilla.gnome.org/show_bug.cgi?id=676403


smime.p7s
Description: S/MIME cryptographic signature
___
evolution-hackers mailing list
evolution-hackers@gnome.org
To change your list options or unsubscribe, visit ...
http://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] Did 3.4 just get really slow?

2012-05-19 Thread David Woodhouse
On Sat, 2012-05-19 at 11:25 +0200, Xavier Bestel wrote:
 I'm using Debian, and found Evo getting slower and slower (I've got
 something like 44K messages in my inbox).
 Now I just removed Tracker from my system, and it's snappy again.
 Happy me. 

Hm, that would affect Evolution's *own* performance? It was taking about
40ms for each call to imapx_expunge_uid_from_summary(), probably in
camel_folder_summary_remove_uid(). Most of the time was spent waiting on
a futex.

I got bored after a couple of hours and went to bed. Left it running
overnight to see what happened, and by the morning it had crashed:

camel-imapx:ERROR:camel-imapx-server.c:1373:imapx_untagged: code should not be 
reached
Aborted

That's an abort when it receives a FETCH response for a message it
didn't think it knew about. But it *had*. I'll do a better analysis,
post a log and file a bug later.

-- 
dwmw2


smime.p7s
Description: S/MIME cryptographic signature
___
evolution-hackers mailing list
evolution-hackers@gnome.org
To change your list options or unsubscribe, visit ...
http://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] Did 3.4 just get really slow?

2012-05-19 Thread David Woodhouse
On Sat, 2012-05-19 at 07:26 -0400, Adam Tauno Williams wrote:
 Once the index is up to date there is almost no way it noticeably
 effects performance unless something else it wrong. 

Heh. To effect performance¹ would be to make it go fast, right? :)

-- 
dwmw2

¹ as opposed to affect performance which I think it what you *meant*
  to say :)


smime.p7s
Description: S/MIME cryptographic signature
___
evolution-hackers mailing list
evolution-hackers@gnome.org
To change your list options or unsubscribe, visit ...
http://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] Rethinking account management

2012-04-30 Thread David Woodhouse
On Sun, 2012-04-29 at 19:50 -0400, Matthew Barnes wrote:
 It's been awhile since I've posted a progress update on this branch, but
 I just wanted to mention that as of this weekend I think I finally have
 Evolution pretty much wrapped up now and am moving on to the groupware
 modules starting with Evolution-EWS, since I'm also on the hook for
 integrating EWS with the new Exchange support in GNOME Online Accounts.

Yay, good work. Thanks.

When you're doing the groupware back ends, please don't forget the
evolution-activesync is in GNOME git too, now! :)

-- 
dwmw2


smime.p7s
Description: S/MIME cryptographic signature
___
evolution-hackers mailing list
evolution-hackers@gnome.org
To change your list options or unsubscribe, visit ...
http://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] Evolution's GTK+ requirement

2012-04-17 Thread David Woodhouse
On Tue, 2012-04-17 at 09:35 -0400, Matthew Barnes wrote:
 Sound okay?  Any objections? 

Sounds fine to me; the only problem I have is that you feel you have to
ask in the first place :)

Allowing users to build with the old version (of *anything* until
there's a *compelling* reason to break the build ought to be our default
behaviour anyway.

It's a constant PITA that you have to have the latest versions of
*everything*, when in practice most of those dependencies aren't really
true at all.

-- 
dwmw2


smime.p7s
Description: S/MIME cryptographic signature
___
evolution-hackers mailing list
evolution-hackers@gnome.org
To change your list options or unsubscribe, visit ...
http://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] PIM server synchronization and Evolution online/offline state

2012-04-13 Thread David Woodhouse
On Tue, 2012-04-03 at 12:05 -0400, Matthew Barnes wrote:
 On Tue, 2012-04-03 at 17:29 +0200, Christian Hilberg wrote:
  Seems to me that opening a connection in order to find out whether I could
  open a connection is more than evo-kolab would need. Unless the 
  service-available
  check would be really cheap, it seems to me that host-reachable would
  suffice. Once I actually try to connect and fail, I know that I cannot
  connect. Nothing lost. ;-) (What's more, if service-available was TRUE 
  half
  an hour ago, when the check was made, that does not automatically mean that
  it is still TRUE when I want to actually *use* the connection half an hour
  later - so, not sure whether a service-available check would help much).
 
 Perhaps then simply rename the online property to host-reachable to
 clarify it's meaning as a first step?  Online seems like too nebulous
 a term in this context anyway. 

Note that in the case of something like a company EWS connection, when
the user does something in Evolution that triggers that account to go
online, you'd actually want it to trigger a VPN connection so that it
*can* reach the host in question...

-- 
dwmw2


smime.p7s
Description: S/MIME cryptographic signature
___
evolution-hackers mailing list
evolution-hackers@gnome.org
To change your list options or unsubscribe, visit ...
http://mail.gnome.org/mailman/listinfo/evolution-hackers


[Evolution-hackers] ActiveSync support for Evolution

2012-03-30 Thread David Woodhouse
Since I've been using this in anger for my company email for the last
few months, and I haven't had cause to touch the code since the
beginning of December... I suppose it's about time I pulled my finger
out, called it a release, and tried to import it into GNOME. So...

 http://git.gnome.org/browse/evolution-activesync/
  git://git.gnome.org/evolution-activesync

For those who weren't aware: this is a basic implementation of the
ActiveSync protocol, in a standalone dæmon which is accessed by DBus.

Clients of this dæmon include a Camel provider which is included in the
git repository, and a SyncEvolution plugin for calendar/contacts
synchronisation, which is shipped with SyncEvolution itself.

We could add e-d-s back ends for direct access to calendar/contacts too.

The Camel provider supports Evolution 3.0 and 3.2; I've made a start at
updating it to 3.4 but won't finish it before I disappear for a week,
starting tomorrow. If any kind person feels like copying most of Milan's
commits f7dd1cd2 and 8cf26212 from evolution-ews, that would be much
appreciated. Otherwise I'll get to it when I get back.

-- 
dwmw2

___
evolution-hackers mailing list
evolution-hackers@gnome.org
To change your list options or unsubscribe, visit ...
http://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] [Evolution] EWS and the addressbook

2012-01-11 Thread David Woodhouse
On Tue, 2012-01-10 at 13:16 -0800, CLOSE Dave wrote:
 When I explicitly read a message, it takes about
 two seconds for the read attribute to change. Then when I view a
 different folder, the unread attribute returns. When I return to the
 folder with the message I already read, it initially shows unread,
 then changes to read after a few seconds. After the second time, the
 read attribute seems to finally stick. 

I see exactly the same behaviour with the ActiveSync back end; it's been
annoying me recently.

I'm reasonably sure I *didn't* see it in the past. When we first got
this working in EWS I was making changes over IMAP, switching to EWS and
ensuring they appeared, and vice versa. I'd definitely have noticed this
then. I'm fairly sure I'd have noticed it when we got it working in
ActiveSync, too.

So either we changed something in both EWS and ActiveSync back ends
simultaneously (which isn't unlikely; the camel code for ActiveSync is
closely based on EWS and we *have* copied changes over in both
directions), or perhaps something in evolution-data-server changed.

Changed to evo-hackers list... anyone got a clue?

Not sure what version(s) Dave is using; I'm seeing this with Evolution
3.2.2-1.fc16, EDS 3.2.2-2.fc16.

-- 
dwmw2


smime.p7s
Description: S/MIME cryptographic signature
___
evolution-hackers mailing list
evolution-hackers@gnome.org
To change your list options or unsubscribe, visit ...
http://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] [Evolution] EWS and the addressbook

2012-01-11 Thread David Woodhouse
On Wed, 2012-01-11 at 10:43 -0800, CLOSE Dave wrote:
 
 Just about exactly the same version:
 evolution-3.2.2-1.fc16.i686
 evolution-ews-3.2.2-1.fc16.i686 

By EDS I meant evolution-data-server; it wasn't a typo. I don't think
I'm running the packaged version of EWS yet (and in fact I'm mostly
using ActiveSync for my company mail at the moment anyway, not EWS).

Do you happen to remember when this started? F16 did have older versions
of evo and eds if you revert to those, does the problem go away?

I'd test this myself, but I'm kind of busy preparing to fly half-way
round the world tomorrow...

-- 
dwmw2


smime.p7s
Description: S/MIME cryptographic signature
___
evolution-hackers mailing list
evolution-hackers@gnome.org
To change your list options or unsubscribe, visit ...
http://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] [evolution-kolab] new mailing list

2011-12-06 Thread David Woodhouse
On Tue, 2011-12-06 at 17:03 +0100, Christian Hilberg wrote:
 the evolution-kolab plugin now has a mailing list, please see [1].
 Everyone who is interested in following the discussions there is
 invited to subscribe. I estimate this list to be a low-traffic one,
 at least for some time. :)

Hm, for EWS I deliberately *didn't* set up a new list; likewise
ActiveSync. I'd rather see it on evo-hackers.

-- 
dwmw2


smime.p7s
Description: S/MIME cryptographic signature
___
evolution-hackers mailing list
evolution-hackers@gnome.org
To change your list options or unsubscribe, visit ...
http://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] RFClue: Atomic folder updates

2011-12-05 Thread David Woodhouse
.


 Nonetheless, there are people whom are willing to connect to their
 exchange account from multiple machines. Did you try whether this sync
 can work in such environment, please?

Yes, I've been using EWS from both my laptop and my desktop for much of
the last year (until I switched focus to ActiveSync; now I run only on
one since I got a new laptop and haven't yet configured EWS on it).

-- 
David WoodhouseOpen Source Technology Centre
david.woodho...@intel.com  Intel Corporation




smime.p7s
Description: S/MIME cryptographic signature
___
evolution-hackers mailing list
evolution-hackers@gnome.org
To change your list options or unsubscribe, visit ...
http://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] RFClue: Atomic folder updates

2011-12-04 Thread David Woodhouse
On Sun, 2011-12-04 at 10:04 -0700, Sankar P wrote:
 My memory is very rusty and I have not seen Evolution sources in more
 than 3 years now. However, we had a similar situation when we working
 on a GroupWise backend and we used a trick to get the number of
 messages in a folder on startup of Evolution (or when the user
 explicitly presses the getmail button) and try to compare the mail
 counts between summary and server and update if they differ (and
 obviously, after fetching new items in this delta)
 
 I am not sure if Exchange server has such a count API. If so, it can
 be used without breaking the summary code much. 

We do something similar to that for IMAP — using the counts as a sanity
check and falling back to the old-style non-QRESYNC method of refetching
the flags for *every* message in the folder.

I don't believe we have fallback like that for EWS and ActiveSync. Those
didn't have sane delta-based operation 'tacked on' to an old protocol
which we can fall back to; they were designed to be delta-based from the
*start*, and the client is expected *not* to get itself out of sync.

Even if we could detect a problem in ActiveSync, we'd basically have to
refetch the entire folder to recover — and that would mean that message
IDs all change, so our cache has to be entirely blown away. I *think* we
could keep the cache in EWS and just refetch the message details, but I
still don't like it — it would be a horrid workaround for a bug, if we
ever have to use it.

When the server responds, it tells us here are a bunch of changes, and
your new 'bookmark' for the folder state is XXX. We should remember
that, atomically. If we crash, our cache on restart should reflect
either the state *before* the response, or the state *after* it. Never
something in between.

-- 
dwmw2


smime.p7s
Description: S/MIME cryptographic signature
___
evolution-hackers mailing list
evolution-hackers@gnome.org
To change your list options or unsubscribe, visit ...
http://mail.gnome.org/mailman/listinfo/evolution-hackers


[Evolution-hackers] RFClue: Atomic folder updates

2011-12-02 Thread David Woodhouse
We have at least three mail protocols now which are delta-based. That
is, you have a bookmark, 'sync key' or timestamp which represents what
the server *last* told you about a given folder. You say to the server
what changed since XXX, and get back a list of added/removed/changed
messages along with a *new* timestamp YYY.

It's a very efficient way to handle mailbox access, and it's used by at
least ActiveSync, EWS and IMAP+QRESYNC. In the Exchange protocols it's
called 'SyncState' or 'SyncKey', and in IMAP it's the HIGHESTMODSEQ.
(It's never *actually* a timestamp, since wall-clock time is a PITA. But
it's easy to *think* of it as as timestamp; the modification time on the
folder).

The problem is, we need to handle these updates *atomically*. If we
store the new timestamp before the changed messages, and we crash in the
middle of doing so, then we might miss out forever on the messages in
question. We'd restart, go to the server and say what happened since
YYY and we never get told *again* about the messages which came in
between time XXX and time YYY, that we didn't manage to fetch.

And if you do it the other way round and store the changed messages
first, and crash before you store the new timestamp, you get similar
issues (cf. bug 664637).

I'm not sure how to fix it. Looking at EWS first, since that's where we
noticed it, I pondered removing the camel_folder_summary_save_to_db()
call from the camel_ews_utils_sync_{created,updated,deleted}_items()
helper functions, so it happens just *once* at the end of the loop in
ews_refresh_info_sync() and commits all the changes at once. But just
deferring the camel_folder_summary bits isn't enough, is it? The
individual message_infos will have a lifetime of their own, and even
*internally*, camel_folder_summary_save_to_db() doesn't actually write
things out atomically using transactions in sqlite... does it?

Any suggestions or insight would be gratefully received...

-- 
dwmw2


smime.p7s
Description: S/MIME cryptographic signature
___
evolution-hackers mailing list
evolution-hackers@gnome.org
To change your list options or unsubscribe, visit ...
http://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] Camel Authentication Changes

2011-10-20 Thread David Woodhouse
On Thu, 2011-10-20 at 12:50 -0400, Matthew Barnes wrote:
 
 I think we could achieve this with multiple calls to
 camel_session_authenticate_sync() with a different mechanism name each
 time (kerberos, ntlm, login), and making sure the session logic
 knows when to loop and when to bail out if an auth attempt fails.
 
 It already kinda does that by looking up the CamelServiceAuthType struct
 associated with the mechanism name (if one was given) and checking its
 'need_password' flag.  If FALSE, it does ONE authentication attempt and
 bails on failure (instead of looping).  After which you could possibly
 call camel_session_authenticate_sync() again with a fallback mechanism
 name.
 
 Problem is when a mechanism name is NOT given, as is the case for most
 HTTP-based backends, or when 'need_password' is TRUE, it assumes you'll
 *will* need a password to authenticate and prompts if one is not found
 in the keyring (which I understand now to be a bad assumption).  That's
 the part that needs tweaking, for now. 

Yeah, the provider could just call camel_session_authenticate_sync() as
and when it actually *needs* a password (which looks fairly much to be
what imapx *is* doing, so I'm not really sure why it's demanding my
password unnecessarily).

The issue is the complexity. What happens is that the provider calls out
to camel_session_authenticate_sync(), which then calls back to the
provider to give it the information it wanted... but it provides it in a
*synchronous* method and waits for a result, so we can't just hand off
the password with a GAsyncResult to the code which called
camel_session_authenticate_sync() in the first place; we have to do
nasty things with threads and avoid deadlock while we make our
provider's -authenticate_sync() actually wait for the completion of the
function which *called* it.

Forget EWS for now; let's just look at it in imapx. This bit in
imapx_create_new_connection(), for example:

/* XXX As part of the connect operation the CamelIMAPXServer will
 * have to call camel_session_authenticate_sync(), but it has
 * no way to pass itself through in that call so the service
 * knows which CamelIMAPXServer is trying to authenticate.
 *
 * IMAPX is the only provider that does multiple connections
 * like this, so I didn't want to pollute the CamelSession and
 * CamelService authentication APIs with an extra argument.
 * Instead we do this little hack so the service knows which
 * CamelIMAPXServer to act on in its authenticate_sync() method.
 *
 * Because we're holding the CAMEL_SERVICE_REC_CONNECT_LOCK
 * (and our own CON_LOCK for that matter) we should not have
 * multiple IMAPX connections trying to authenticate at once,
 * so this should be thread-safe.
 */
imapx_store-authenticating_server = g_object_ref (conn);
success = camel_imapx_server_connect (conn, cancellable, error);
g_object_unref (imapx_store-authenticating_server);

Then in imapx_reconnect() you call camel_session_authenticate_sync()
which ends up calling *back* to imapx's imapx_authenticate_sync()
method... which uses that nastily-stashed 'server' object... which was
RIGHT THERE in the imapx_reconnect() function. We could have just
returned the password to imapx_reconnect() and all would have been well.

I'm unlikely to get much done before I disappear to Prague on Saturday,
and I'm on baby duty tomorrow because she's unwell and can't go to
nursery, but I'll try to put together a patch which sorts this out.
Maybe during next week.

-- 
dwmw2

___
evolution-hackers mailing list
evolution-hackers@gnome.org
To change your list options or unsubscribe, visit ...
http://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] Camel Authentication Changes

2011-10-19 Thread David Woodhouse
On Sat, 2011-10-15 at 11:18 -0400, Matthew Barnes wrote:
 Just a heads up that I've changed Camel's authentication API to factor
 out the password loop that each and every provider currently replicates
 for themselves.  Turns out they all have the same basic logic flow.
 
 These changes make password management more consistent and also make
 life easier for providers.  Now when a provider reaches the step where
 it needs to authenticate, it simply calls: 
 
 
   gboolean  camel_session_authenticate_sync  (CamelSession *session,
   CamelService *service,
   const gchar *mechanism,
   GCancellable *cancellable,
   GError **error);

This looks like a good idea, and cleaning up the various ways that the
providers were behaving differently in that authentication loop (and
calling directly out to the GUI) is a good thing.

I looked at those recently when implementing the -try_empty_password()
SASL methods for automatic NTLM auth, and I take my hat off to you for
finding a way through the mess.

Having said that, I do think we need to adjust this approach to deal
with authentication as a *reactive* thing, not *proactive*. It *used* to
be reactive, but you changed it as a side-effect of your improvements.

The point is that we authenticate ourselves *if* and *when* the server
asks us to. We cannot know in advance whether we're going to need a
password or not.

I tried this code, and the first thing it did was ask me for a password
for my IMAP connection, even though that IMAP account uses a custom
connection command 'ssh $mailhost /usr/libexec/dovecot/imap' so it gets
a *preauthenticated* connection.

(At other times, the custom command might have been something different
like  'ssh $bastionhost openssl s_client -connect $internalserver:993'
and the authentication *would* still have been required. You just can't
know until you connect.)

Even if the server asks for a password, with things like automatic NTLM
authentication you can't know whether the automatic challenge/response
will work. The old loop in the providers would handle that — it would
try first with the empty password, then it would only prompt the user
for a password if it really needed one. I think you've broken that in
master now, because session_authenticate_sync() will just return
immediate success if the SASL -try_empty_password() method indicates
that we can even *try* to authenticate without a user-provided password.

 The mechanism argument refers to the user's preferred SASL mechanism.
 This will usually be taken from url-authmech

That's fine at the moment, but the fact that we have to explicitly
choose *one* mechanism in advance ought to be considered a bug. In the
future, we ought to simply try *any* of the methods that the server
offers today, in some order of preference. So we could try Kerberos,
fall back to NTLM, etc. These changes seem to have made that harder to
fix.

  but it can also be NULL when not using SASL (such as for HTTP-based
 backends).

It looks like it'll always ask for a password for HTTP-based backends,
but those have similar options; they may authenticate with Kerberos, or
with automatic-NTLM, or with cookies... or they *may* actually decide
that they need a password today.

All of the above issues have a single solution — we need to change
things so that the password is requested *only* when we're actually
connected to the server and we have discovered that we actually need it.

I think the best way to handle it is to extend
camel_service_get_password(), which is already called from the providers
in the right place. If we make *it* trigger the password dialog, rather
than doing the dialog in advance, then I think we solve all the issues.

We'll also want to add a 'retrying' argument, for the caller to indicate
that the last password it received was not correct. That'll prompt for
actual user interaction, rather than just handing out the password from
gkr, for example.

 The CamelSession (or subclass) now handles all the looping.  So if the
 function returns TRUE, you're authenticated.  If it returns FALSE, you
 treat it like any other failed operation.  You don't loop.

Aside from the correctness issues, doing it internally can be a lot more
efficient. Imagine you're on a slow connection and each DNS lookup, TCP
connection establishment and SSL negotiation takes about 30 seconds.

Now, do you want to connect a first time and work out that your stored
password is invalid, then tear it all down and make a *new* connection,
only to find that the user fat-fingered the password and you need to try
a *third* time?

Or do you want to keep the same connection open and issue three
consecutive AUTH commands, looping *inside* the IMAP provider?

There are similar considerations when the password changes during a
session. TCP connections may break and 

Re: [Evolution-hackers] Camel Authentication Changes

2011-10-18 Thread David Woodhouse
On Tue, 2011-10-18 at 15:58 -0400, Matthew Barnes wrote:
 On Tue, 2011-10-18 at 21:29 +0200, Milan Crha wrote: 
  I'm currently trying to adapt evo-ews to current git master and this
  change doesn't make much sense there, at least for me, because on the
  first look there is used a libsoup for authentications, thus no such
  auth_loop or any same basic logic. I've a larger patch for the ews
  under go, which I will finish tomorrow, I only wanted to let you know
  about this and I hoped that you'll help with this particular thing.
 
 You might have to implement authenticate() and authenticate_finish()
 instead of authenticate_sync().
 
 Cache the password that authenticate() gives you, hand it to libsoup
 when it asks for it, and when libsoup responds with a status code,
 complete the GSimpleAsyncResult you created in authenticate().

Note that this also needs to cope with the case where the password
*becomes* invalid in the middle of an active session. The user should be
prompted for a new password (repeatedly until they get it right), and
then it should work for *all* of mail/cal/ebook access.

And if they hit *cancel* instead of entering a valid password, hopefully
it'll stop asking rather than popping up a dialog every few seconds
until they change to a VT and run 'killall evolution' :)

-- 
dwmw2


smime.p7s
Description: S/MIME cryptographic signature
___
evolution-hackers mailing list
evolution-hackers@gnome.org
To change your list options or unsubscribe, visit ...
http://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] evolution-ews: complete failure of git head calendar to work

2011-08-03 Thread David Woodhouse
On Mon, 2011-08-01 at 17:29 -0500, James Bottomley wrote:
 
 GDBus.Error:org.gtk.GDBus.UnmappedGError.Quark._e_2ddata_2dcal_2derror.Code113:
  Cannot open calendar: The EWS Id is in EwsLegacyId format which is not 
 supported by the Exchange version specified by your request. Please use the 
 ConvertId method to convert from EwsLegacyId to EwsId format.
 
 With the current git head, it just crashes in e_cal_backend_open().

I fixed that crash, but you still need to wipe your cache and start
again, as described in the mail that Pavel referred to.

-- 
dwmw2

___
evolution-hackers mailing list
evolution-hackers@gnome.org
To change your list options or unsubscribe, visit ...
http://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] [PATCH] evolution-ews: implement Exchange categories as evolution labels (read only)

2011-06-27 Thread David Woodhouse
On Mon, 2011-05-23 at 16:55 +0400, James Bottomley wrote:
 
 I kid you not.
 
 You can see this just by going to your own outlook web server (when you
 highlight a message, you'll see an empty tall oblong, click and it will
 bring up the categories).
 
 Exchange allows arbitrary labels (like we do), but the default ones are
 Blue, Green, Orange, Purple, Red and Yellow.  Evolution has label
 mappings for everything apart from Yellow.  I figured we should map
 default to default.

I don't really like mapping 'Red' to 'Important', etc.

Given that we support arbitrary labels on both sides, I think the best
solution is to synchronise the set of labels. If we see a label on an
Exchange message which isn't already known in Evolution, *add* it to
Evolution's list. And if we're trying to add a label to an Exchange item
which isn't already known on the Exchange side, add it there too.

-- 
dwmw2

___
evolution-hackers mailing list
evolution-hackers@gnome.org
To change your list options or unsubscribe, visit ...
http://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] Rethinking Camel settings

2011-06-13 Thread David Woodhouse
On Mon, 2011-06-13 at 11:05 -0400, Matthew Barnes wrote:
 
 
 That's definitely something to revisit.  I'd love to kill off those
 EPlugins.  For the time being, though, I have to limit my scope since
 I'm trying to finish off my account-mgmt branch as fast as possible.

Understood.

 If you have some further thoughts on making that more versatile, I'd
 suggest jotting down a few notes on a wiki page and then I can come back
 to it later in the year.  I assume such enhancements could also apply to
 the 'Sending E-mail' tab, even if there's less need for that.

My only thought would be to obsolete those plugins; anything they need,
we'd want to support.

On 'Sending E-mail', I would also like to be able to switch to SMTP
instead of using MAPI/EWS for sending email. Exchange is *so* broken,
and SMTP is often available...

-- 
dwmw2

___
evolution-hackers mailing list
evolution-hackers@gnome.org
To change your list options or unsubscribe, visit ...
http://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] [PATCH] evolution-ews: implement Exchange categories as evolution labels (read only)

2011-05-23 Thread David Woodhouse
On Mon, 2011-05-23 at 14:41 +0400, James Bottomley wrote:
 
 +
 +   /* this is a mapping from Exchange/Outlook categories to
 +* evolution labels based on the standard colours */
 +   const gchar *labels[] = {
 +   Red Category, $Labelimportant,
 +   Orange Category, $Labelwork,
 +   Green Category, $Labelpersonal,
 +   Blue Category, $Labeltodo,
 +   Purple Category, $Labellater,
 +   NULL, NULL
 +   }; 

That's the only reason I might *not* want to apply your patch
immediately, right?

Is there a correct way to handle the mapping? Does Exchange really
just store them by colour?

-- 
dwmw2

___
evolution-hackers mailing list
evolution-hackers@gnome.org
To change your list options or unsubscribe, visit ...
http://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] [PATCH] evolution-ews: implement Exchange categories as evolution labels (read only)

2011-05-23 Thread David Woodhouse
On Mon, 2011-05-23 at 17:08 +0400, James Bottomley wrote:
 
  could you use Gnome's bugzilla for patch reviews, please?
 
 Consider this more an RFC requiring feedback than a patch requiring
 review.

For Evo-EWS I'm much happier with patches in email anyway. I find it a
lot easier to review patches, and to apply them with 'git am', if they
are in email.

I'm going to get notified by email *anyway* — that's how I learn about
new bugs in bugzilla too. So it's just a question of whether all the
information I need is right there in the email when I first see it, or
whether I have to go grubbing around in a web browser to find it,
download it, and then play silly buggers to apply it and get the
authorship right.

In fact, James, if you're going to make a habit of sending patches for
Evo-EWS why not just get an account in GNOME git and apply them
directly? :)

http://live.gnome.org/NewAccounts

-- 
dwmw2

___
evolution-hackers mailing list
evolution-hackers@gnome.org
To change your list options or unsubscribe, visit ...
http://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] [PATCH] evolution-ews: implement Exchange categories as evolution labels (read only)

2011-05-23 Thread David Woodhouse
On Mon, 2011-05-23 at 17:46 +0400, James Bottomley wrote:
 I think you probably need me to get a feel for evolution development
 initially before you set me loose in the repo ...

Why? It's mostly my baby, and I'm a kernel hacker too... :)

 Incidentally, I presume you use bugzilla for bugs?  I just ran across
 one experimenting with Categories.  Apparently if I change the category
 on the outlook webserver, it causes us to delete the email (with or
 without my patch applied).

That'll be a bug that Chen introduced on Thursday. I've reverted those
commits for now. We're currently between Alpha and Beta releases; I
don't mind a little bit of 'churn', but I draw the line at data loss :)

Sorry about that.

Note that HEAD doesn't compile at the moment because of something
committed by Pavel Ocheretny pochere...@src.gnome.com last night.
I'll change that to a real, working email address and *POKE*.

-- 
dwmw2

___
evolution-hackers mailing list
evolution-hackers@gnome.org
To change your list options or unsubscribe, visit ...
http://mail.gnome.org/mailman/listinfo/evolution-hackers


[Evolution-hackers] [RFC] Make e-{addressbook, calendar}-factory automatically quit when superseded

2011-05-20 Thread David Woodhouse
I have lost count of the number of times I've been trying to debug
something in e-calendar-factory, but the factory I'm running in gdb is
*not* the one that's actually being used.

Sometimes I forget to kill the old factory. Sometimes I *do* kill it,
but Evolution, *while* complaining that calendars will never work again,
automatically starts a replacement (that it will never use).

This patch should allow replacement simply by running a new factory
process. This will upset Evolution, because its existing factory will go
away. But users should never be doing that anyway; this is only for
debugging. And it should make debugging a *lot* easier.

Comments?

diff --git a/addressbook/libedata-book/e-data-book-factory.c 
b/addressbook/libedata-book/e-data-book-factory.c
index b8aa145..4779d82 100644
--- a/addressbook/libedata-book/e-data-book-factory.c
+++ b/addressbook/libedata-book/e-data-book-factory.c
@@ -574,6 +574,8 @@ on_name_lost (GDBusConnection *connection,
}
 
g_mutex_unlock (factory-priv-connections_lock);
+   g_print (Superseded...\n);
+   g_main_loop_quit (loop);
 }
 
 #ifndef G_OS_WIN32
@@ -667,7 +669,7 @@ main (gint argc, gchar **argv)
 
owner_id = g_bus_own_name (G_BUS_TYPE_SESSION,
ADDRESS_BOOK_DBUS_SERVICE_NAME,
-   G_BUS_NAME_OWNER_FLAGS_NONE,
+   G_BUS_NAME_OWNER_FLAGS_REPLACE | 
G_BUS_NAME_OWNER_FLAGS_ALLOW_REPLACEMENT,
on_bus_acquired,
on_name_acquired,
on_name_lost,
diff --git a/calendar/libedata-cal/e-data-cal-factory.c 
b/calendar/libedata-cal/e-data-cal-factory.c
index 0b02e42..3af0ab6 100644
--- a/calendar/libedata-cal/e-data-cal-factory.c
+++ b/calendar/libedata-cal/e-data-cal-factory.c
@@ -891,6 +891,8 @@ on_name_lost (GDBusConnection *connection,
g_list_foreach (copy, remove_data_cal_cb, NULL);
g_list_free (copy);
}
+   g_print (Superseded...\n);
+   g_main_loop_quit (loop);
 }
 
 #ifndef G_OS_WIN32
@@ -992,7 +994,7 @@ main (gint argc, gchar **argv)
 
owner_id = g_bus_own_name (G_BUS_TYPE_SESSION,
CALENDAR_DBUS_SERVICE_NAME,
-   G_BUS_NAME_OWNER_FLAGS_NONE,
+   G_BUS_NAME_OWNER_FLAGS_REPLACE | 
G_BUS_NAME_OWNER_FLAGS_ALLOW_REPLACEMENT,
on_bus_acquired,
on_name_acquired,
on_name_lost,


-- 
dwmw2


___
evolution-hackers mailing list
evolution-hackers@gnome.org
To change your list options or unsubscribe, visit ...
http://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] 32 bit IDs in contact file backend

2011-05-17 Thread David Woodhouse
On Thu, 2011-04-28 at 15:16 +0200, Patrick Ohly wrote:
 As part of wrapping QtContacts around EDS [1] I ran into the same issue
 that Nokia already encountered in their Maemo 5 [2] backend: EDS uses
 strings as ID, QtContacts 32 bit integers.
 
 Nokia solved that by setting up an in-memory hash which maps the UID
 string to its CRC-16. I don't see any checks for the (possible) hash
 collisions. IMHO a proper solution has to keep a permanent mapping on
 disk, otherwise the 32 bit IDs won't be stable.
 
 Overall not a nice solution. My attempt to make it nicer at least in
 combination with the file backend (the main goal for QtContacts-EDS)
 focused on simplifying the problem instead: with 32 bit IDs in the
 storage, the mapping becomes easy. 

While I understand what you're saying when you say 'nicer', this seems
to be a fundamentally *wrong* approach.

You're suggesting that a user of the EDS API should rely on internal
implementation details of a single back end, which don't even apply to
other back ends.

Even if we *didn't* have immediate plans to use other back ends like EWS
with this setup, that would be entirely the wrong thing to do, surely? 

Or is this hack planned to be *extremely* limited for MeeGo 1.2, and
dropped in some way (perhaps by *fixing* the QtContacts API) by 1.3? In
which case, perhaps it really would make more sense to do it with a
persistent mapping in your wrapper?

-- 
David WoodhouseOpen Source Technology Centre
david.woodho...@intel.com  Intel Corporation

___
evolution-hackers mailing list
evolution-hackers@gnome.org
To change your list options or unsubscribe, visit ...
http://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] 32 bit IDs in contact file backend

2011-05-17 Thread David Woodhouse
On Tue, 2011-05-17 at 14:51 +0200, Patrick Ohly wrote:
 On Di, 2011-05-17 at 13:27 +0100, David Woodhouse wrote:
  On Tue, 2011-05-17 at 14:04 +0200, Patrick Ohly wrote:
   On Di, 2011-05-17 at 12:38 +0100, David Woodhouse wrote:
Even if we *didn't* have immediate plans to use other back ends like EWS
with this setup, that would be entirely the wrong thing to do, surely?
   
   I'm not so sure. We are pitching EDS as an alternative for other storage
   solutions that are highly optimized (= limited!) for specific use cases.
   What you are suggesting is that any attempt to add optimizations for a
   specific combination of app + EDS + backend is wrong and should be
   avoided. My feeling is that EDS will simply not be used at all unless
   such optimization are acceptable.
  
  [EDS upstream]
  
  I have no objection to an *optimisation*. You seemed to be describing a
  *fix*, not an optimisation.
  
  An *optimisation* allows things to work faster or more efficiently, when
  they were already working before.
  
  So if you expose an extra '32bit-numeric-uid' in your static
  capabilities for the back end, and the user can make use of that to
  operate more efficiently by bypassing the permanent uidstring-integer
  mapping, then I'm happy with that.
 
 That was the plan.

In that case I have no objection to the EDS part in principle.

  But *only* if it really is an optimisation, and designed such that
   the code still works (via the mapping) without it.
 
 I can't promise that the code will work without it right away because
 the mapping hasn't been implemented yet due to lack of time. See also:
 http://lists.meego.com/pipermail/meego-dev/2011-May/483078.html

That needs to be fixed; I've responded to that message.

-- 
dwmw2

___
evolution-hackers mailing list
evolution-hackers@gnome.org
To change your list options or unsubscribe, visit ...
http://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] [PATCH 1/2] e_cal_new_system_foo() should create corresponding source in GConf

2011-05-10 Thread David Woodhouse
On Tue, 2011-05-10 at 11:40 +0100, Dumez, Christophe wrote:
 
 I have tested the patch but it does not seem to help. I don't know
 what the reason is yet.

It may depend on earlier fixes? Can you show your patch?

 This e-mail and any attachments may contain confidential material for
 the sole use of the intended recipient(s). Any review or distribution
 by others is strictly prohibited. If you are not the intended
 recipient, please contact the sender and delete all copies.

Please don't post to public mailing lists with this unenforceable
nonsense. If you file a ticket with the helpdesk, they will remove it
for your outbound mail. (And if they refuse, get a name and contact me
in private).

-- 
dwmw2

___
evolution-hackers mailing list
evolution-hackers@gnome.org
To change your list options or unsubscribe, visit ...
http://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] [PATCH 1/2] e_cal_new_system_foo() should create corresponding source in GConf

2011-05-10 Thread David Woodhouse
On Tue, 2011-05-10 at 10:19 +0200, Patrick Ohly wrote:
 It seems that a similar problem exists in libebook if no address books
 were created already by Evolution. Chris is seeing such an issue with
 2.32.3 in MeeGo.

Oh, tits. I hate the fact that all this code is so *gratuitously*
separate. Because my patch ended up calling get_local_source() three
times (for cal/tasks/memos) I was tricked into thinking I'd already
covered everything; the fact that that was all still just *calendar* and
I hadn't handled the addressbook side had temporarily eluded my feeble
mind.

 We probably need to add the create GConf entry for local:system part
 to libebook in the gnome-2-32 branch. Is that something that is still
 of interest for Trunk, given that EClient API will obsolete it for
 3.2?

It's still relevant for now, so let's do it. We *could* make a case for
not bothering, and just fixing it in 3.0 and 2.32 instead. But our
policy is to backport only fixes from master, and I prefer not to set a
counter-precedent unless I absolutely have to. It doesn't *hurt* to fix
it in master first.

Do you have a patch?

-- 
David WoodhouseOpen Source Technology Centre
david.woodho...@intel.com  Intel Corporation

___
evolution-hackers mailing list
evolution-hackers@gnome.org
To change your list options or unsubscribe, visit ...
http://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] New 'eclient' branch in eds

2011-04-28 Thread David Woodhouse
On Thu, 2011-04-28 at 11:56 +0200, Patrick Ohly wrote:
 Are you saying that a soname version bump can and should be done in a
 backwards-incompatible way (= code compiled against older version no
 longer works with new lib)? That's a major pain for people trying to
 support multiple distros. Please avoid it whenever possible. 

As I understand it, an soname bump (from libfoo.so.5 to libfoo.so.6)
should *only* be used for backwards-incompatible changes.

If old clients can continue to use the newer library, then shouldn't it
be just a change of *minor* version from libfoo.so.5.2 to libfoo.so.5.3,
and the soname remains the same (libfoo.so.5).

-- 
dwmw2

___
evolution-hackers mailing list
evolution-hackers@gnome.org
To change your list options or unsubscribe, visit ...
http://mail.gnome.org/mailman/listinfo/evolution-hackers


[Evolution-hackers] Moving Evolution-EWS to GNOME infrastructure.

2011-04-21 Thread David Woodhouse
We should move EWS developnent to GNOME infrastructure.

I'll work out how to import the GIT repository and add a bugzilla
component for evolution-ews.

Pavel, Or and Mandy you will need to apply for a GNOME git account,
following the procedure at http://live.gnome.org/NewAccounts

-- 
David WoodhouseOpen Source Technology Centre
david.woodho...@intel.com  Intel Corporation

___
evolution-hackers mailing list
evolution-hackers@gnome.org
To change your list options or unsubscribe, visit ...
http://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] Moving Evolution-EWS to GNOME infrastructure.

2011-04-21 Thread David Woodhouse
On Thu, 2011-04-21 at 17:36 +0200, Milan Crha wrote:
 I understood that Chen wants this as a part of evolution-collab, where,
 as I understood it, will be added also evolution-exchange,
 evolution-mapi, groupwise bits, and one day, maybe also evolution-kolab.
 It may save you from bugzilla and new module procedure work, as Chen
 will do this all for you for free. 

Ah, are we still actually going ahead with that plan? I know it's been
discussed for some time...

-- 
David WoodhouseOpen Source Technology Centre
david.woodho...@intel.com  Intel Corporation

___
evolution-hackers mailing list
evolution-hackers@gnome.org
To change your list options or unsubscribe, visit ...
http://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] gnome-3-0 branch broken

2011-04-19 Thread David Woodhouse
On Tue, 2011-04-19 at 21:00 +0200, Denis Washington wrote:
 During today's round of jhbuilding I noticed that the gnome-3-0 branch 
 of e-d-s currently fails to build. The reason is a backport of a fix for 
 bug #645783 [1] which relies on the sealing of CamelService done in 
 master. I have attached a (hopefully correct) patch which fixes the problem.

Hey, at least it looked like real C code. Take a look at the same patch
backported to 2.32:
http://git.gnome.org/browse/evolution-data-server/commit/?h=gnome-2-32id=c25eab6d

This is now fixed in both 3.0 and 2.32; thanks for pointing it out.

-- 
David WoodhouseOpen Source Technology Centre
david.woodho...@intel.com  Intel Corporation


___
evolution-hackers mailing list
evolution-hackers@gnome.org
To change your list options or unsubscribe, visit ...
http://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] New 'eclient' branch in eds

2011-04-19 Thread David Woodhouse
On Tue, 2011-04-19 at 10:40 -0400, Matthew Barnes wrote:
 
 2) ECredentials is way too complex.  My intent for ECredentials was to
have a self-contained authentication handling interface to avoid all
the current nonsense with ECal and EBook where you have to always
remember to connect to authentication signals and then implement the
keyring lookup and user password prompting yourself.
 
I probably didn't spell this out very well initially, but what I had
in mind was a simple abstract interface to replace auth-required
signals.
 
struct _ECredentialsInterface {
GTypeInterface parent_interface;
 
void(*get_password)   (ECredentials *credentials,
   EClient *client,
   GCancellable *cancellable,
   GAsyncReadyCallback callback,
   gpointer user_data);
 
gchar * (get_password_finish) (ECredentials *credentials,
   GAsyncResult *result,
   GError **error);
};

Note the 'gboolean retrying' argument to the libsoup authenticate
signal handler. We probably want to have something similar in the above
API too, because that's what tells the UI that it needs to ditch the
existing remembered password and ask for a new one.

-- 
David WoodhouseOpen Source Technology Centre
david.woodho...@intel.com  Intel Corporation

___
evolution-hackers mailing list
evolution-hackers@gnome.org
To change your list options or unsubscribe, visit ...
http://mail.gnome.org/mailman/listinfo/evolution-hackers


[Evolution-hackers] Pre-alpha test builds of Evolution-EWS for Fedora / MeeGo

2011-04-17 Thread David Woodhouse
We can use these builds for doing pre-Alpha testing of EWS. Once it
passes our testing, we'll announce the Alpha release and then get on
with implementing full read/write calendar support.

Since the early QA has also shown up a bunch of evo/eds bugs, I've also
built evo/eds packages with the latest fixes from the gnome-2-32 branch
which will hopefully be in a 2.32.3 release soon.

Fedora 14 packages:
EWS: http://koji.fedoraproject.org/koji/taskinfo?taskID=3006315
EDS: http://koji.fedoraproject.org/koji/taskinfo?taskID=3006318
EVO: http://koji.fedoraproject.org/koji/taskinfo?taskID=3006329

MeeGo packages (for 1.1  Trunk):
http://build.meego.com/project/show?project=home%3Adwmw2%3Aevo

-- 
David WoodhouseOpen Source Technology Centre
david.woodho...@intel.com  Intel Corporation

___
evolution-hackers mailing list
evolution-hackers@gnome.org
To change your list options or unsubscribe, visit ...
http://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] Pre-alpha test builds of Evolution-EWS for Fedora / MeeGo

2011-04-17 Thread David Woodhouse
On Sun, 2011-04-17 at 09:16 +0100, David Woodhouse wrote:
 We can use these builds for doing pre-Alpha testing of EWS.

... which has already failed :)

Exchange puts some items in your inbox on which the 'message:From'
property is invalid. And rather than giving a warning but returning the
rest of the information when you try to fetch such items, Exchange
returns a *failure* for that fetch attempt.

Before, we were failing the whole request (with up to 100 items). So up
to 100 items were missing from the inbox, and we didn't notice. And
either we didn't get the error reported to the user, or we *really*
weren't paying attention. We need to fix things so that a fetch error
definitely *is* reported to the user.

With the code from last night, we skip that *single* item but then crash
later on a NULL pointer dereference when trying to process it.

I've committed a simple patch to fix the crash, but we still need to
stop fetching invalid properties, and make sure that we report fetch
errors properly.

  Once it
 passes our testing, we'll announce the Alpha release and then get on
 with implementing full read/write calendar support.
 
 Since the early QA has also shown up a bunch of evo/eds bugs, I've also
 built evo/eds packages with the latest fixes from the gnome-2-32 branch
 which will hopefully be in a 2.32.3 release soon.
 
 Fedora 14 packages:
 EWS: http://koji.fedoraproject.org/koji/taskinfo?taskID=3006315
 EDS: http://koji.fedoraproject.org/koji/taskinfo?taskID=3006318
 EVO: http://koji.fedoraproject.org/koji/taskinfo?taskID=3006329
 
 MeeGo packages (for 1.1  Trunk):
 http://build.meego.com/project/show?project=home%3Adwmw2%3Aevo
 


-- 
dwmw2

___
evolution-hackers mailing list
evolution-hackers@gnome.org
To change your list options or unsubscribe, visit ...
http://mail.gnome.org/mailman/listinfo/evolution-hackers


[Evolution-hackers] [PATCH 1/2] e_cal_new_system_foo() should create corresponding source in GConf

2011-04-15 Thread David Woodhouse
---
 calendar/libecal/e-cal.c |   56 +++--
 1 files changed, 53 insertions(+), 3 deletions(-)

diff --git a/calendar/libecal/e-cal.c b/calendar/libecal/e-cal.c
index 6835d45..69ff9c7 100644
--- a/calendar/libecal/e-cal.c
+++ b/calendar/libecal/e-cal.c
@@ -1023,6 +1023,56 @@ e_cal_new_from_uri (const gchar *uri, ECalSourceType 
type)
return cal;
 }
 
+static ECal *
+get_local_source (ECalSourceType type)
+{
+   ESourceList *sources;
+   ESourceGroup *on_this_computer;
+   GSList *local_sources, *iter;
+   gchar *source_uri = NULL;
+   ECal *ecal;
+
+   if (!e_cal_get_sources (sources, type, NULL)) {
+   g_warning (Could not get task sources from GConf!);
+   goto out;
+   }
+
+   on_this_computer = e_source_list_ensure_group (sources,
+  _(On This Computer),
+  local:, TRUE);
+   if (!on_this_computer) {
+   g_object_unref (sources);
+   goto out;
+   }
+
+   local_sources = e_source_group_peek_sources (on_this_computer);
+   for (iter = local_sources; !source_uri  iter != NULL; iter = 
iter-next) {
+   ESource *source = iter-data;
+   gchar *uri;
+
+   uri = e_source_get_uri (source);
+   if (g_strcmp0 (uri, local:system) == 0)
+   source_uri = uri;
+   else
+   g_free (uri);
+   }
+   if (!source_uri) {
+   ESource *source;
+   source = e_source_new (_(Personal), system);
+   e_source_set_color_spec (source, #BECEDD);
+   e_source_group_add_source (on_this_computer, source, -1);
+   g_object_unref (source);
+
+   if (!e_source_list_sync (sources, NULL))
+   g_warning (Cannot add system source to GConf!);
+   }
+   g_object_unref (on_this_computer);
+   g_object_unref (sources);
+ out:
+   ecal = e_cal_new_from_uri (source_uri?:local:system, type);
+   g_free (source_uri);
+   return ecal;
+}
 /**
  * e_cal_new_system_calendar:
  *
@@ -1036,7 +1086,7 @@ e_cal_new_from_uri (const gchar *uri, ECalSourceType type)
 ECal *
 e_cal_new_system_calendar (void)
 {
-   return e_cal_new_from_uri (local:system, E_CAL_SOURCE_TYPE_EVENT);
+   return get_local_source (E_CAL_SOURCE_TYPE_EVENT);
 }
 
 /**
@@ -1052,7 +1102,7 @@ e_cal_new_system_calendar (void)
 ECal *
 e_cal_new_system_tasks (void)
 {
-   return e_cal_new_from_uri (local:system, E_CAL_SOURCE_TYPE_TODO);
+   return get_local_source (E_CAL_SOURCE_TYPE_TODO);
 }
 
 /**
@@ -1068,7 +1118,7 @@ e_cal_new_system_tasks (void)
 ECal *
 e_cal_new_system_memos (void)
 {
-   return e_cal_new_from_uri (local:system, E_CAL_SOURCE_TYPE_JOURNAL);
+   return get_local_source (E_CAL_SOURCE_TYPE_JOURNAL);
 }
 
 /**
-- 
1.7.4.2



___
evolution-hackers mailing list
evolution-hackers@gnome.org
To change your list options or unsubscribe, visit ...
http://mail.gnome.org/mailman/listinfo/evolution-hackers


[Evolution-hackers] [PATCH 2/2] Fix comparison of URI in e_cal_new_from_uri().

2011-04-15 Thread David Woodhouse
If the URI only had a relative_uri property, it should build the full URI
for comparison rather than just skipping it.
---
 calendar/libecal/e-cal.c |   11 ++-
 1 files changed, 10 insertions(+), 1 deletions(-)

diff --git a/calendar/libecal/e-cal.c b/calendar/libecal/e-cal.c
index 69ff9c7..a60abfb 100644
--- a/calendar/libecal/e-cal.c
+++ b/calendar/libecal/e-cal.c
@@ -988,8 +988,17 @@ check_uri (ESource *source, gpointer uri)
g_return_val_if_fail (uri != NULL, FALSE);
 
suri = e_source_peek_absolute_uri (source);
+   if (suri)
+   return !g_ascii_strcasecmp (suri, uri);
+   else {
+   gboolean ret;
+   gchar *suri2;
 
-   return suri  g_ascii_strcasecmp (suri, uri) == 0;
+   suri2 = e_source_get_uri (source);
+   ret = !g_ascii_strcasecmp (suri2, uri);
+   g_free (suri2);
+   return ret;
+   }
 }
 
 /**
-- 
1.7.4.2


___
evolution-hackers mailing list
evolution-hackers@gnome.org
To change your list options or unsubscribe, visit ...
http://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] [PATCH 2/2] Fix comparison of URI in e_cal_new_from_uri().

2011-04-15 Thread David Woodhouse
On Fri, 2011-04-15 at 12:37 +0100, David Woodhouse wrote:
 If the URI only had a relative_uri property, it should build the full
 URI
 for comparison rather than just skipping it. 

We have the opposite problem in the file backend, in the hack that sets
the cache dir for the system source to system instead of the uid:

source_dir = e_source_peek_relative_uri (source);
if (!source_dir || !g_str_equal (source_dir, system))
source_dir = e_source_peek_uid (source);

But if we fix that, we risk changing the storage directory of existing
sources. So I suppose that has to remain as it was.

-- 
David WoodhouseOpen Source Technology Centre
david.woodho...@intel.com  Intel Corporation

___
evolution-hackers mailing list
evolution-hackers@gnome.org
To change your list options or unsubscribe, visit ...
http://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] Finding memory leaks in EWS with Valgrind

2011-04-13 Thread David Woodhouse
On Wed, 2011-04-13 at 07:54 +0200, Milan Crha wrote:
 I'm just wondering, how does this mail belong to evolution-hackers
 list, please?

http://mail.gnome.org/archives/evolution-hackers/2011-March/msg3.html

  I do not see anything general evolution-related in it, maybe apart of
 the above paragraph, but that's usually known to people whom are
 coping with valgrind on daily basis.

There was also the comment about running e-*-factory under valgrind as
well as just evolution itself.

But the main reason I sent it to the list was context; it was written
for the people working on the EWS back end. They are not *yet* using
valgrind regularly, and I had asked them to. I said I'd send an email
with instructions, and the evo-hackers list seemed like the best place
to send it.

-- 
dwmw2

___
evolution-hackers mailing list
evolution-hackers@gnome.org
To change your list options or unsubscribe, visit ...
http://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] Evolution - GSSAPI - Caldav implementation

2011-04-12 Thread David Woodhouse
On Mon, 2011-04-11 at 17:50 +0200, Jan Lange wrote:
 
 is there a way GSSAPI (negotiate) configure in the calendar plugin
 caldav?

Evolution's CalDAV support uses libsoup. Once libsoup can manage, it
should be relatively simple to make Evolution use it.

https://bugzilla.gnome.org/show_bug.cgi?id=587145

-- 
David WoodhouseOpen Source Technology Centre
david.woodho...@intel.com  Intel Corporation

___
evolution-hackers mailing list
evolution-hackers@gnome.org
To change your list options or unsubscribe, visit ...
http://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] Fedora builds with 2.32.2+ patches

2011-04-08 Thread David Woodhouse
On Fri, 2011-04-08 at 09:54 +0200, sean finney wrote:
 
  Sounds like you would be in a good position to do it though.
 
 Because I'm not a gnome dev, I (a) don't have push access, and (b)
 am a bit hesitant to go against Milan's wishes, since he's the dev
 who is primarily keeping things up for -mapi and has made his stance
 pretty clear.  I only brought it up because it seemed like there might
 be a change in that stance, and if so I'd be happy to share my
 currently unshared fixes in .32.

You're more than welcome to use git.infradead.org if you want. But even
if Milan sees the 2.32 branch as being dead and doesn't want to spend
any of his own time on it (and nobody can blame him for that), I would
hope that he wouldn't try to obstruct *you* if you feel you need to do
so.

 Then again, now that 3.0 is released I may try again to get something
 rolled together based on that since there are already a number of api
 breaks making backports difficult for .32, and it seems there are lots
 more in the pipe for 3.1.

Certainly, my point in maintaining fixes for 2.32 was *not* to
discourage people from upgrading. So if 3.0 is a viable option for you
then please do go ahead.

Having said that, the API breaks haven't *yet* reached the point where
we can't build evo-ews for 2.32 and 3.x from the *same* code base,
albeit with a number of icky ifdefs.

-- 
dwmw2

___
evolution-hackers mailing list
evolution-hackers@gnome.org
To change your list options or unsubscribe, visit ...
http://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] Fedora builds with 2.32.2+ patches

2011-04-08 Thread David Woodhouse
On Thu, 2011-04-07 at 15:08 +0200, Milan Crha wrote:
 Hmm, I still do not like the idea of adding things to gnome-2-32 branch,
 I thought I saw you telling that if anyone will complain then you'll not
 do that and I thought I complained to it, but I cannot find the email
 right now, so I cannot point to the logic error I see in such
 statement. :) 

Sorry, I do still owe you a reply to that. Shortly...

-- 
dwmw2

___
evolution-hackers mailing list
evolution-hackers@gnome.org
To change your list options or unsubscribe, visit ...
http://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] Maintenance fork for Evolution / EDS 2.32

2011-04-08 Thread David Woodhouse
On Mon, 2011-04-04 at 08:53 +0200, Milan Crha wrote:
 On Fri, 2011-04-01 at 20:07 +0100, David Woodhouse wrote:
  That's great; thanks. I'll do a little more testing on the patches
  I've cherry-picked into my trees, and then unless someone else has
  objected in the meantime I'll push them. 
 
   Hi,
 I objected against this many times, directly to you, on IRC, with no
 effect, obviously. If I recall correctly, the reason why release-team
 decreased releases is that distributions were *not* using .2 release.
 Which is just the opposite you are trying to convince us. If they are
 not using official releases, why should they use unofficial branch?

Fedora does. As Yves-Alexis said, Debian (and thus Ubuntu?) does.
And MeeGo does.

Even before we start looking at other distributions, that is *enough*
duplication of effort that it seems worthwhile to collaborate on a
single code base rather than each having their own 'fork' and
backporting fixes from HEAD for themselves.

 By the way, how would you look for a fix user reported to your
 distribution, as a distribution maintainer? 
 ...
 Note that I do not expect anyone looking into git branch for a
 particular fix, ...

That's all very true, for a *specific* fix for a bug that a user has
managed to report.

Having said that, if the distro *were* currently running 2.32.1, I'd
hope the 'workflow' you outline would *also* include checking in the
gnome-2-32 branch to see if the fix has *already* been backported and
tested there?

By collecting the backported patches into a central tree, we don't break
the workflow you describe — it stills works just as before, with an
extra 'shortcut' to a fully-backported-and-tested patch in some cases.

But I'm not thinking *just* about specific fixes for bugs that get
reported. Part of the benefit of a central tree is that if a bug gets
reported in *one* distribution and fixed there, the backported fix can
benefit users of *all* distributions.

Take the issue with ordering of modified recurrences, for example. How
many users would manage to actually track that down and make a coherent
bug report, and how many would just be inconvenienced by the fact that
changes sometimes don't show up right in Evolution, and put it down to
gremlins? I feel sure that the reason that bug went unreported for so
long was *not* because nobody actually *saw* it.

By putting that fix into a 2.32.3 release, we potentially get that fix
out to a large number of users of distributions who are stil on 2.32
(which includes Fedora, for the next 8 months or so).

I appreciate that you think I'm wasting my time. Perhaps I am; time will
tell. But my time is my own to waste. My main concern is to ensure that
I'm not wasting *your* time. And since you can choose to completely
ignore what we're doing in the gnome-2-32 branch, I think we're safe on
that front.

-- 
dwmw2

___
evolution-hackers mailing list
evolution-hackers@gnome.org
To change your list options or unsubscribe, visit ...
http://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] Calendar PIM objects with inlined attachments

2011-04-08 Thread David Woodhouse
On Fri, 2011-04-08 at 10:35 +0200, Milan Crha wrote:
 On Fri, 2011-04-08 at 10:04 +0200, Christian Hilberg wrote:
  We could use Evolution's file-link attachment mechanism by writing into 
  Evos 
  file cache from the backend and placing the file paths into the 
  ECalComponent 
  when reading calendar objects from the Kolab server, and read attachment 
  file 
  data from this same cache when creating a new object with attached files. 
  We 
  found a whole lot of issues with this approach (at least for Evo2.30), 
  which I 
  can detail if anyone is interested.
 
   Hi,
 CalDAV calendar plugin uses this approach, and it seems to work. It's
 surely better than passing rather-long attachments over the wire (D-Bus)
 in both ways. Check the CalDAV sources how it's done there, maybe you
 would be able to mimic that approach.

That's what we're doing in EWS, too. We store the attachments into the
backend's cache directory as obtained by e_cal_backend_get_cache_dir(),
and we set a file:// URL to point to them there. (If you have the MIME
Content-Type: for it, don't forget to put that into a FMTTYPE parameter
in the ical ATTACH property.)

I'm actually not *entirely* happy with that approach; I didn't really
want to fetch all the attachments in advance. I'd prefer to fetch them
on demand, instead.

I was toying with the idea of adding a new URL scheme to represent the
'location' of such attachments, and working out how to make that work in
Evolution. It would presumably need to translate into a DBus call to
evolution-data-server, and some method of passing the data back.

Perhaps the best option is to have a call to EDS which asks it to
'resolve' the attachment into a local file URL instead of a remote one?

For now, I've just stuck with the naïve approach of fetching them all in
advance, but I'd like to have a plan for how to fix that, when it
bubbles up to the top of my TODO list for EWS. And you may want
something similar for kolab?

-- 
dwmw2

___
evolution-hackers mailing list
evolution-hackers@gnome.org
To change your list options or unsubscribe, visit ...
http://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] Calendar PIM objects with inlined attachments

2011-04-08 Thread David Woodhouse
On Fri, 2011-04-08 at 12:17 +0100, David Woodhouse wrote:
 
 Perhaps the best option is to have a call to EDS which asks it to
 'resolve' the attachment into a local file URL instead of a remote
 one?

The more I think about this, the more I like it. Add an
X-EVOLUTION-UNRESOLVED-ATTACH: property with details like filename,
content-type and perhaps size, and sufficient information to actually
fetch it on demand. And add a ResolveAttachments() call for the UI to
call, which will turn it into a *real* attachment with a local file://
URL.

Does that seem reasonable?

(For now, though, I'll stick with just fetching the things up front)

-- 
David WoodhouseOpen Source Technology Centre
david.woodho...@intel.com  Intel Corporation

___
evolution-hackers mailing list
evolution-hackers@gnome.org
To change your list options or unsubscribe, visit ...
http://mail.gnome.org/mailman/listinfo/evolution-hackers


[Evolution-hackers] EWS MessageInfo leak wtf

2011-04-08 Thread David Woodhouse
==5510== 41 bytes in 1 blocks are possibly lost in loss record 8,473 of 17,762
==5510==at 0x4A05E46: malloc (vg_replace_malloc.c:195)
==5510==by 0x3B6A048B52: g_malloc (gmem.c:164)
==5510==by 0x3B6A06101D: g_strdup (gstrfuncs.c:102)
==5510==by 0x1693D7B1: ews_message_info_clone (camel-ews-summary.c:75)
==5510==by 0x1693E4FD: camel_ews_summary_add_message_info 
(camel-ews-summary.c:417)
==5510==by 0x16940ADD: camel_ews_utils_sync_created_items 
(camel-ews-utils.c:973)
==5510==by 0x16939819: sync_created_items (camel-ews-folder.c:660)
==5510==by 0x16939B41: ews_refresh_info_sync (camel-ews-folder.c:750)
==5510==by 0x4C78C99: camel_folder_refresh_info (camel-folder.c:1156)
==5510==by 0x33B6E7F0F7: mail_msg_proxy (mail-mt.c:469)
==5510==by 0x3B6A06BBC3: g_thread_pool_thread_proxy (gthreadpool.c:319)
==5510==by 0x3B6A069445: g_thread_create_proxy (gthread.c:1897)
==5510== 

Can't repeat it; don't know how it happened... except of course that it
obviously happened when SyncFolderItems returned some created items.

The frame in ews_message_info_clone() at camel_ews_summary:75 is setting
mi-change_key on the newly cloned MessageInfo:
http://git.infradead.org/evolution-ews.git/blob/cd1face:/src/camel/camel-ews-summary.c#l75

(I'm also *sure* it happened after commit 17bd5273 in which I fixed a
consistent leak of our mi-change_key by adding a message_info_free()
method to our class. Not only do I remember it that way, but the
timestamp of my valgrind log file is about four hours later than that
commit, and the *other* valgrind warnings that the -change_key leak
caused are noticeable in their absence.)

I'm confused by the use of camel_message_info_clone() in the first line
of camel_ews_summary_add_message_info():
http://git.infradead.org/evolution-ews.git/blob/cd1face:/src/camel/camel-ews-summary.c#l417
There is *one* caller of this function, and it's the one in the above
backtrace — camel_ews_utils_sync_create_items():
http://git.infradead.org/evolution-ews.git/blob/cd1face:/src/camel/camel-ews-utils.c#l973

And that *creates* a new MessageInfo of its own... which AFAICT never
gets freed, although valgrind doesn't seem to be complaining about that,
and I don't know why.

Removing the clone in camel_ews_summary_add_message_info() and just
using '(void *)info' seems to work just as well; I'm not really sure
what's going on here. And either way, I've never been able to repeat the
above warning.

Chen, was there a reason for the clone? And why isn't valgrind
complaining about it? Can anyone see something that would cause the
above complaint that it *did* make?

-- 
dwmw2

___
evolution-hackers mailing list
evolution-hackers@gnome.org
To change your list options or unsubscribe, visit ...
http://mail.gnome.org/mailman/listinfo/evolution-hackers


[Evolution-hackers] (no subject)

2011-04-07 Thread David Woodhouse
On Tue, 2011-04-05 at 16:31 +0530, Chenthill Palanisamy wrote:
 This would certainly help distributions which want to stay with
 Evolution 2.32 for a while.. My only concern here is, while
 cherry-picking patches how would we take care of the translations and
 documentation ? Are we adhering to the freezes in the gnome-2-32
 branch (am not calling this a latest stable branch) ?

I suspect we should apply the same policy that we do on an active
stable branch — string changes should be avoided, unless absolutely
necessary.

-- 
David WoodhouseOpen Source Technology Centre
david.woodho...@intel.com  Intel Corporation

___
evolution-hackers mailing list
evolution-hackers@gnome.org
To change your list options or unsubscribe, visit ...
http://mail.gnome.org/mailman/listinfo/evolution-hackers


[Evolution-hackers] Fedora builds with 2.32.2+ patches

2011-04-07 Thread David Woodhouse
I have built some Fedora 14 packages of Evo 2.32.2 with the latest
additional fixes, for testing.

The main reason for doing this is to fix bug 646164 with modified
instances of recurring appointments, since that's really getting on our
tits as we implement that part of the Evolution-EWS back end. But it
serves as a useful opportunity to test the other patches that we've
backported from HEAD, too.

EDS: http://koji.fedoraproject.org/koji/taskinfo?taskID=2981480
Evo: http://koji.fedoraproject.org/koji/taskinfo?taskID=2981538

This includes the 'proper' fix for GNOME bug #371705 (Calendar's day
view does not reset its IM context), instead of the preliminary
workaround that was previously present in the Fedora package. Is there
anyone who could test and confirm that it's working correctly in the
2.32 backport?

Once this passes muster, I'll push these patches (probably *without* the
NTLM bits, if you're looking closely at what I included) to the
gnome-2-32 branches and perhaps start doing a 'final call' for 2.32.3
candidate bugs/patches.

-- 
David WoodhouseOpen Source Technology Centre
david.woodho...@intel.com  Intel Corporation

___
evolution-hackers mailing list
evolution-hackers@gnome.org
To change your list options or unsubscribe, visit ...
http://mail.gnome.org/mailman/listinfo/evolution-hackers


[Evolution-hackers] [PATCH 2/3] Add asynchronous camel_sasl_try_empty_password()

2011-04-03 Thread David Woodhouse
---
 camel/camel-sasl.c |   49 +
 camel/camel-sasl.h |8 
 2 files changed, 57 insertions(+), 0 deletions(-)

diff --git a/camel/camel-sasl.c b/camel/camel-sasl.c
index 58a4a2b..90b64d6 100644
--- a/camel/camel-sasl.c
+++ b/camel/camel-sasl.c
@@ -288,6 +288,53 @@ sasl_challenge_finish (CamelSasl *sasl,
 }
 
 static void
+sasl_try_empty_password_thread (GSimpleAsyncResult *simple,
+   GObject *object,
+   GCancellable *cancellable)
+{
+   gboolean res;
+
+   res = camel_sasl_try_empty_password_sync (CAMEL_SASL (object), 
cancellable);
+
+   g_simple_async_result_set_op_res_gboolean (simple, res);
+}
+
+static void
+sasl_try_empty_password (CamelSasl *sasl,
+gint io_priority,
+GCancellable *cancellable,
+GAsyncReadyCallback callback,
+gpointer user_data)
+{
+   GSimpleAsyncResult *simple;
+
+   simple = g_simple_async_result_new (
+   G_OBJECT (sasl), callback, user_data, sasl_try_empty_password);
+
+   g_simple_async_result_run_in_thread (
+   simple, sasl_try_empty_password_thread, io_priority, 
cancellable);
+
+   g_object_unref (simple);
+}
+
+static gboolean
+sasl_try_empty_password_finish (CamelSasl *sasl,
+   GAsyncResult *result)
+{
+   GSimpleAsyncResult *simple;
+   gboolean res;
+
+   g_return_val_if_fail (
+   g_simple_async_result_is_valid (
+   result, G_OBJECT (sasl), sasl_try_empty_password), FALSE);
+
+   simple = G_SIMPLE_ASYNC_RESULT (result);
+   res = g_simple_async_result_get_op_res_gboolean (simple);
+
+   return res;
+}
+
+static void
 camel_sasl_class_init (CamelSaslClass *class)
 {
GObjectClass *object_class;
@@ -302,6 +349,8 @@ camel_sasl_class_init (CamelSaslClass *class)
 
class-challenge = sasl_challenge;
class-challenge_finish = sasl_challenge_finish;
+   class-try_empty_password = sasl_try_empty_password;
+   class-try_empty_password_finish = sasl_try_empty_password_finish;
 
g_object_class_install_property (
object_class,
diff --git a/camel/camel-sasl.h b/camel/camel-sasl.h
index eede608..4ef9b41 100644
--- a/camel/camel-sasl.h
+++ b/camel/camel-sasl.h
@@ -82,6 +82,14 @@ struct _CamelSaslClass {
GByteArray *(*challenge_finish) (CamelSasl *sasl,
 GAsyncResult *result,
 GError **error);
+   void(*try_empty_password)   (CamelSasl *sasl,
+gint io_priority,
+GCancellable *cancellable,
+GAsyncReadyCallback callback,
+gpointer user_data);
+   gboolean(*try_empty_password_finish)
+   (CamelSasl *sasl,
+GAsyncResult *result);
 };
 
 GType  camel_sasl_get_type (void);
-- 
1.7.4



___
evolution-hackers mailing list
evolution-hackers@gnome.org
To change your list options or unsubscribe, visit ...
http://mail.gnome.org/mailman/listinfo/evolution-hackers


[Evolution-hackers] [PATCH 1/3] Add camel_sasl_try_empty_password_sync() method.

2011-04-03 Thread David Woodhouse
This indicates that a SASL method with the need_password flag can be tried
without providing a password, for single-sign-on using system credentials.

This will be used by NTLM.
---
 camel/camel-sasl.c  |   25 ++
 camel/camel-sasl.h  |6 +++
 camel/providers/imap/camel-imap-store.c |   51 +++--
 camel/providers/imapx/camel-imapx-server.c  |   31 +++--
 camel/providers/smtp/camel-smtp-transport.c |   65 ++-
 5 files changed, 128 insertions(+), 50 deletions(-)

diff --git a/camel/camel-sasl.c b/camel/camel-sasl.c
index 02006c6..58a4a2b 100644
--- a/camel/camel-sasl.c
+++ b/camel/camel-sasl.c
@@ -419,6 +419,31 @@ camel_sasl_get_authenticated (CamelSasl *sasl)
 }
 
 /**
+ * camel_sasl_try_empty_password_sync:
+ * @sasl: a #CamelSasl object
+ *
+ * Returns: whether or not @sasl can attempt to authenticate without a
+ * password being provided by the caller. This will be %TRUE for an
+ * authentication method which can attempt to use single-sign-on
+ * credentials, but which can fall back to using a provided password
+ * so it still has the @need_password flag set in its description.
+ **/
+gboolean
+camel_sasl_try_empty_password_sync (CamelSasl *sasl, GCancellable *cancellable)
+{
+   CamelSaslClass *class;
+
+   g_return_val_if_fail (CAMEL_IS_SASL (sasl), FALSE);
+
+   class = CAMEL_SASL_GET_CLASS (sasl);
+
+   if (class-try_empty_password_sync == NULL)
+   return FALSE;
+
+   return class-try_empty_password_sync (sasl, cancellable);
+}
+
+/**
  * camel_sasl_set_authenticated:
  * @sasl: a #CamelSasl
  * @authenticated: whether we have successfully authenticated
diff --git a/camel/camel-sasl.h b/camel/camel-sasl.h
index 3bb3a84..eede608 100644
--- a/camel/camel-sasl.h
+++ b/camel/camel-sasl.h
@@ -64,6 +64,9 @@ struct _CamelSaslClass {
CamelObjectClass parent_class;
 
/* Synchronous I/O Methods */
+   gboolean(*try_empty_password_sync)
+   (CamelSasl *sasl,
+GCancellable *cancellable);
GByteArray *(*challenge_sync)   (CamelSasl *sasl,
 GByteArray *token,
 GCancellable *cancellable,
@@ -85,6 +88,9 @@ GType camel_sasl_get_type (void);
 CamelSasl *camel_sasl_new  (const gchar *service_name,
 const gchar *mechanism,
 CamelService *service);
+gboolean   camel_sasl_try_empty_password_sync
+   (CamelSasl *sasl,
+GCancellable *cancellable);
 gboolean   camel_sasl_get_authenticated(CamelSasl *sasl);
 void   camel_sasl_set_authenticated(CamelSasl *sasl,
 gboolean authenticated);
diff --git a/camel/providers/imap/camel-imap-store.c 
b/camel/providers/imap/camel-imap-store.c
index 521a185..9fa53ee 100644
--- a/camel/providers/imap/camel-imap-store.c
+++ b/camel/providers/imap/camel-imap-store.c
@@ -653,20 +653,23 @@ connect_to_server_wrapper (CamelService *service,
 
 static gboolean
 try_auth (CamelImapStore *store,
-  const gchar *mech,
+  CamelSasl *sasl,
   GCancellable *cancellable,
   GError **error)
 {
-   CamelSasl *sasl;
+   CamelService *service = CAMEL_SERVICE (store);
CamelImapResponse *response;
gchar *resp;
gchar *sasl_resp;
 
-   response = camel_imap_command (store, NULL, cancellable, error, 
AUTHENTICATE %s, mech);
-   if (!response)
+   response = camel_imap_command (store, NULL, cancellable, error,
+  AUTHENTICATE %s,
+  service-url-authmech);
+   if (!response) {
+   g_object_unref (sasl);
return FALSE;
+   }
 
-   sasl = camel_sasl_new (imap, mech, CAMEL_SERVICE (store));
while (!camel_sasl_get_authenticated (sasl)) {
resp = camel_imap_response_extract_continuation (store, 
response, error);
if (!resp)
@@ -718,6 +721,7 @@ imap_auth_loop (CamelService *service,
CamelSession *session = camel_service_get_session (service);
CamelServiceAuthType *authtype = NULL;
CamelImapResponse *response;
+   CamelSasl *sasl = NULL;
gchar *errbuf = NULL;
gboolean authenticated = FALSE;
const gchar *auth_domain;
@@ -754,12 +758,22 @@ imap_auth_loop (CamelService *service,
return FALSE;
}
 
-   if (!authtype-need_password) {
-   authenticated = try_auth (
-   store, authtype-authproto,
- 

[Evolution-hackers] [PATCH 3/3] Add support for NTLM single-sign-on using /usr/bin/ntlm_auth

2011-04-03 Thread David Woodhouse
There's a simple test version of ntlm_auth at
http://david.woodhou.se/ntlm_auth_v2.c
---
 camel/camel-sasl-ntlm.c |  134 +++
 1 files changed, 134 insertions(+), 0 deletions(-)

diff --git a/camel/camel-sasl-ntlm.c b/camel/camel-sasl-ntlm.c
index a24c837..61b9cc3 100644
--- a/camel/camel-sasl-ntlm.c
+++ b/camel/camel-sasl-ntlm.c
@@ -28,9 +28,14 @@
 #include glib/gi18n-lib.h
 
 #include camel-sasl-ntlm.h
+#include camel-stream-process.h
 
 struct _CamelSaslNTLMPrivate {
gint placeholder;  /* allow for future expansion */
+#ifndef G_OS_WIN32
+   CamelStream *helper_stream;
+   gchar *type1_msg;
+#endif
 };
 
 CamelServiceAuthType camel_sasl_ntlm_authtype = {
@@ -61,6 +66,8 @@ G_DEFINE_TYPE (CamelSaslNTLM, camel_sasl_ntlm, 
CAMEL_TYPE_SASL)
 #define NTLM_RESPONSE_HOST_OFFSET44
 #define NTLM_RESPONSE_FLAGS_OFFSET   60
 
+#define NTLM_AUTH_HELPER /usr/bin/ntlm_auth
+
 static void ntlm_calc_response   (const guchar key[21],
  const guchar plaintext[8],
  guchar results[24]);
@@ -661,6 +668,10 @@ sasl_ntlm_challenge_sync (CamelSasl *sasl,
   GCancellable *cancellable,
   GError **error)
 {
+#ifndef G_OS_WIN32
+   CamelSaslNTLM *ntlm = CAMEL_SASL_NTLM (sasl);
+   CamelSaslNTLMPrivate *priv = ntlm-priv;
+#endif
CamelService *service;
GByteArray *ret;
gchar *user;
@@ -671,6 +682,49 @@ sasl_ntlm_challenge_sync (CamelSasl *sasl,
 
ret = g_byte_array_new ();
 
+#ifndef G_OS_WIN32
+   if (priv-helper_stream  !service-url-passwd) {
+   guchar *data;
+   gsize length = 0;
+   char buf[1024];
+   gsize s = 0;
+   buf [0] = 0;
+
+   if (!token || !token-len) {
+   if (priv-type1_msg) {
+   data = g_base64_decode (priv-type1_msg, 
length);
+   g_byte_array_append (ret, data, length);
+   g_free (data);
+   g_free (priv-type1_msg);
+   priv-type1_msg = NULL;
+   }
+   return ret;
+   } else {
+   gchar *type2 = g_base64_encode (token-data, 
token-len);
+   if (camel_stream_printf (priv-helper_stream, TT %s\n,
+type2) = 0 
+  (s = camel_stream_read (priv-helper_stream, buf,
+  sizeof(buf), cancellable, NULL))  4 
+  buf[0] == 'K'  buf[1] == 'K'  buf[2] == ' ' 
+  buf[s-1] == '\n') {
+   data = g_base64_decode (buf + 3, length);
+   g_byte_array_append (ret, data, length);
+   g_free (data);
+   } else
+   g_warning (Didn't get valid response from 
ntlm_auth helper);
+
+   g_free (type2);
+   }
+   /* On failure, we just return an empty string. Setting the
+  GError would cause the providers to abort the whole
+  connection, and we want them to ask the user for a password
+  and continue. */
+   g_object_unref (priv-helper_stream);
+   priv-helper_stream = NULL;
+   return ret;
+   }
+#endif
+
if (!token || token-len  NTLM_CHALLENGE_NONCE_OFFSET + 8)
goto fail;
 
@@ -768,15 +822,95 @@ exit:
return ret;
 }
 
+static gboolean sasl_ntlm_try_empty_password_sync (CamelSasl *sasl, 
GCancellable *cancellable)
+{
+#ifndef G_OS_WIN32
+   CamelStream *stream = camel_stream_process_new ();
+   CamelService *service = camel_sasl_get_service (sasl);
+   CamelSaslNTLM *ntlm = CAMEL_SASL_NTLM (sasl);
+   CamelSaslNTLMPrivate *priv = ntlm-priv;
+   gchar *user;
+   gchar buf[1024];
+   gsize s;
+   gchar *command;
+   int ret;
+
+   if (access (NTLM_AUTH_HELPER, X_OK))
+   return FALSE;
+
+   user = strchr (service-url-user, '\\');
+   if (user) {
+   command = g_strdup_printf (%s --helper-protocol 
ntlmssp-client-1 
+  --use-cached-creds --username '%s' 
+  --domain '%.*s', NTLM_AUTH_HELPER,
+  user + 1, (int)(user - 
service-url-user), 
+  service-url-user);
+   } else {
+   command = g_strdup_printf (%s --helper-protocol 
ntlmssp-client-1 
+  --use-cached-creds --username '%s',
+  NTLM_AUTH_HELPER, 
service-url-user);
+   }
+   ret = 

Re: [Evolution-hackers] RFC: camel-sasl try empty password first

2011-04-02 Thread David Woodhouse
On Fri, 2011-04-01 at 19:30 -0400, Matthew Barnes wrote:
 
 I think the flags approach is fine.
 
 Long as you're breaking ABI (which is perfectly okay right now), you
 might consider combining the 'try_empty_password' and 'needs_password'
 booleans into a single CamelServiceAuthFlags enum to future-proof the
 ABI a little. 

Thanks for the feedback. In fact, on closer inspection of how I'd
actually implement this in the camel-sasl-ntlm code, I see I'd end up
with something like this in camel_sasl_ntlm_class_init() to override the
try_empty_password flag if the helper seems to be available:

#ifndef G_OS_WIN32
if (!access (NTLM_AUTH_HELPER, X_OK))
camel_sasl_ntlm_authtype.try_empty_password = TRUE;
#endif  

I don't like that much; I think I'd prefer to have a new method in the
CamelSaslClass, and a corresponding camel_sasl_can_empty_password()
method. That way, we can actually try spawning ntlm_auth at the right
time and see if it *currently* knows our password. Never any need to
restart Evolution to get the newly-installed auth helper to be noticed,
no false positives when it's *present* but doesn't know the password,
etc.

That makes it slightly more fun in the provider code, but not much. Will
look at that and post a revised RFC patch later today, when I have more
that one hand.

-- 
dwmw2

___
evolution-hackers mailing list
evolution-hackers@gnome.org
To change your list options or unsubscribe, visit ...
http://mail.gnome.org/mailman/listinfo/evolution-hackers


[Evolution-hackers] Maintenance fork for Evolution / EDS 2.32

2011-04-01 Thread David Woodhouse
Although the Evolution developers have moved on to better things and
consider Evolution 2.32 to be a dead end, there are distributions still
trying to ship this dead end. It is the latest stable release of
Evolution, after all.

Rather than all the distributors working separately to keep track of
patches and backport bug fixes from Evo HEAD, it would be useful if we
could make a git tree somewhere which collects such patches and reduces
our duplication of effort. Kind of like the -stable tree for the Linux
kernel.

To that end, I have created trees at:
http://git.infradead.org/evolution-data-server-2.32.git
 git://git.infradead.org/evolution-data-server-2.32.git

http://git.infradead.org/evolution-2.32.git
 git://git.infradead.org/evolution-2.32.git

We can define our operating policy as we go, but I'd be happy if we
could stick to *only* cherry-picking patches which have already been
applied to the master branch (with massaging to make them apply as
necessary). I've been using 'git cherry-pick -x' so that we can refer
back to the original commit easily.

I've backported a small number of fixes from HEAD that I think are
appropriate, or which are already being shipped in some distributions.

I'm more than happy to give accounts on git.infradead.org to
distribution maintainers, and look at other patches which distributions
are shipping.

I hope that eventually, we might be permitted to use the real
gnome-2-32 branch in GNOME git for this, rather than having to do it
elsewhere. If that branch is a dead end and would otherwise be unused,
then there's no harm in letting us use it as a more central location for
our collaboration, surely?

Presumably we'll do the same thing for Evolution 3.0 during the time
period when it is orphaned by upstream, but still being shipped by
distributions.

-- 
David WoodhouseOpen Source Technology Centre
david.woodho...@intel.com  Intel Corporation

___
evolution-hackers mailing list
evolution-hackers@gnome.org
To change your list options or unsubscribe, visit ...
http://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] Maintenance fork for Evolution / EDS 2.32

2011-04-01 Thread David Woodhouse
On Fri, 2011-04-01 at 13:28 -0400, Matthew Barnes wrote:
 On Fri, 2011-04-01 at 18:00 +0100, David Woodhouse wrote: 
  I hope that eventually, we might be permitted to use the real
  gnome-2-32 branch in GNOME git for this, rather than having to do it
  elsewhere. If that branch is a dead end and would otherwise be unused,
  then there's no harm in letting us use it as a more central location for
  our collaboration, surely?
 
 I'm fine with you using the real gnome-2-32 branch.  Would even be
 nice to see another formal 2.32 release if enough patches accumulate.

That's great; thanks. I'll do a little more testing on the patches I've
cherry-picked into my trees, and then unless someone else has objected
in the meantime I'll push them.

 Per GNOME's six-month release cycle, upstream maintainers have pretty
 much wrapped up 3.0 and are now focused on 3.1.

Although presumably there will be 3.01 and 3.02 releases so those
branches aren't *quite* as orphaned as 2.32 yet :)

-- 
David WoodhouseOpen Source Technology Centre
david.woodho...@intel.com  Intel Corporation

___
evolution-hackers mailing list
evolution-hackers@gnome.org
To change your list options or unsubscribe, visit ...
http://mail.gnome.org/mailman/listinfo/evolution-hackers


[Evolution-hackers] RFC: camel-sasl try empty password first

2011-04-01 Thread David Woodhouse
I'm working on single-sign-on support for NTLM, where we don't actually
*know* the password, but just delegate the whole challenge/response
thing to a helper program.

That helper program is /usr/bin/ntlm_auth; the only current
implementation is the Samba one which works when you've logged into the
system using your Windows password and pam_winbind, but we're working on
a less baroque version that works {like,with} gnome-keyring, and there's
a simple hack at http://david.woodhou.se/ntlm_auth_v2.c for testing.

So I need a way to indicate that the authentication mechanism should be
tried once *without* a password, and then if that fails we should try
providing a password.

Thus the patch below. Anyone got a better suggestion for how to handle
it? A patch to actually use this facility in the NTLM authenticator will
follow, of course...

One alternative approach might be to to stop letting the authenticators
look at service-url-passwd, and instead have an 'authenticate' signal
on the CamelSasl object, much the same way as libsoup does it.

But then I think we'd still have to have the users know somehow if that
authenticate signal had been called on the first attempt, to know when
to set the CAMEL_SESSION_PASSWORD_REPROMPT flag, etc. I suspect it would
be a lot of extra work for something that doesn't really end up looking
any prettier by the time it's working.

Hence the 'try_empty_password' flag, which is relatively simple.

Comments?

commit b1787a5f85cdc1e46fb6ccf0c57134fc34f4e12e
Author: David Woodhouse david.woodho...@intel.com
Date:   Fri Apr 1 23:25:05 2011 +0100

Add 'try_empty_password' flag to CamelServiceAuthType

NTLM will be use this, to support single-sign-on using /usr/bin/ntlm_auth

diff --git a/camel/camel-sasl-anonymous.c b/camel/camel-sasl-anonymous.c
index dbbb500..850b1ea 100644
--- a/camel/camel-sasl-anonymous.c
+++ b/camel/camel-sasl-anonymous.c
@@ -37,6 +37,7 @@ CamelServiceAuthType camel_sasl_anonymous_authtype = {
N_(This option will connect to the server using an anonymous login.),
 
ANONYMOUS,
+   FALSE,
FALSE
 };
 
diff --git a/camel/camel-sasl-cram-md5.c b/camel/camel-sasl-cram-md5.c
index ae4bb16..54f9daa 100644
--- a/camel/camel-sasl-cram-md5.c
+++ b/camel/camel-sasl-cram-md5.c
@@ -48,6 +48,7 @@ CamelServiceAuthType camel_sasl_cram_md5_authtype = {
   secure CRAM-MD5 password, if the server supports it.),
 
CRAM-MD5,
+   FALSE,
TRUE
 };
 
diff --git a/camel/camel-sasl-digest-md5.c b/camel/camel-sasl-digest-md5.c
index ae12f7b..384f216 100644
--- a/camel/camel-sasl-digest-md5.c
+++ b/camel/camel-sasl-digest-md5.c
@@ -61,6 +61,7 @@ CamelServiceAuthType camel_sasl_digest_md5_authtype = {
   secure DIGEST-MD5 password, if the server supports it.),
 
DIGEST-MD5,
+   FALSE,
TRUE
 };
 
diff --git a/camel/camel-sasl-gssapi.c b/camel/camel-sasl-gssapi.c
index 61b3404..832c128 100644
--- a/camel/camel-sasl-gssapi.c
+++ b/camel/camel-sasl-gssapi.c
@@ -93,6 +93,7 @@ CamelServiceAuthType camel_sasl_gssapi_authtype = {
   Kerberos 5 authentication.),
 
GSSAPI,
+   FALSE,
FALSE
 };
 
diff --git a/camel/camel-sasl-login.c b/camel/camel-sasl-login.c
index de3aba4..465a8f7 100644
--- a/camel/camel-sasl-login.c
+++ b/camel/camel-sasl-login.c
@@ -42,6 +42,7 @@ CamelServiceAuthType camel_sasl_login_authtype = {
   simple password.),
 
LOGIN,
+   FALSE,
TRUE
 };
 
diff --git a/camel/camel-sasl-ntlm.c b/camel/camel-sasl-ntlm.c
index 6d2313a..a285214 100644
--- a/camel/camel-sasl-ntlm.c
+++ b/camel/camel-sasl-ntlm.c
@@ -44,6 +44,7 @@ CamelServiceAuthType camel_sasl_ntlm_authtype = {
   NTLM / Secure Password Authentication.),
 
NTLM,
+   FALSE,
TRUE
 };
 
diff --git a/camel/camel-sasl-plain.c b/camel/camel-sasl-plain.c
index e27a6b9..5b1845d 100644
--- a/camel/camel-sasl-plain.c
+++ b/camel/camel-sasl-plain.c
@@ -46,6 +46,7 @@ CamelServiceAuthType camel_sasl_plain_authtype = {
   simple password.),
 
PLAIN,
+   FALSE,
TRUE
 };
 
diff --git a/camel/camel-sasl-popb4smtp.c b/camel/camel-sasl-popb4smtp.c
index 043291b..00ef7ff 100644
--- a/camel/camel-sasl-popb4smtp.c
+++ b/camel/camel-sasl-popb4smtp.c
@@ -49,6 +49,7 @@ CamelServiceAuthType camel_sasl_popb4smtp_authtype = {
 
POPB4SMTP,
FALSE,
+   FALSE,
 };
 
 /* last time the pop was accessed (through the auth method anyway), *time_t */
diff --git a/camel/camel-service.h b/camel/camel-service.h
index d1efa89..35cf1a9 100644
--- a/camel/camel-service.h
+++ b/camel/camel-service.h
@@ -137,6 +137,7 @@ typedef struct {
const gchar *description;
const gchar *authproto;
 
+   gboolean try_empty_password;
gboolean need_password;   /* needs a password to authenticate */
 } CamelServiceAuthType;
 
diff --git a/camel/providers/imap/camel-imap-store.c 
b/camel/providers/imap/camel-imap-store.c
index d241ee5

[Evolution-hackers] imapx last-minute fix for 3.0?

2011-03-20 Thread David Woodhouse
I've been working with Thomas to diagnose an imapx problem which turns
out to be a Domino IMAP server bug.

When we fetch a message chunk, the Domino server will lie to us about
the size of a literal that it's about to send us. It tells us a larger
size than it actually sends, so we 'eat' its command completion as if it
were part of the message chunk, then wait for ever for the end of the
literal data and the subsequent completion that's never coming.

My plan for fixing this is:
 - A configuration option for disabling multi-part fetch.

 - Ensure we have a sane timeout on incoming messages, so we time out in
   a reasonable amount of time in the above situation rather than
   hanging for ever.

 - When we *do* time out, look at the last line of the buffer that we
   *did* receive, to see if it looks like a command completion. If it
   does, then pop up an error message which advises the user to try
   disabling the multi-part fetches. Or better still, just *do* it.

This means new strings, which it's already late for, and it means code
changes, which it's damn close for since I haven't actually written them
yet. But still I suspect it's a good idea to get these into 3.0 since we
really want to be able to laugh and point at anyone who is still using
the old IMAP back end in 3.0. It should be deprecated, and IMAP+ should
be the default (is it?).

Thoughts?

-- 
dwmw2

___
evolution-hackers mailing list
evolution-hackers@gnome.org
To change your list options or unsubscribe, visit ...
http://mail.gnome.org/mailman/listinfo/evolution-hackers


[Evolution-hackers] String change request for Evolution

2011-03-20 Thread David Woodhouse
I've diagnosed a bug in Lotus Domino IMAP server that we need to work
around in Evolution's IMAP client:
https://bugzilla.gnome.org/show_bug.cgi?id=645332

The simple workaround is to disable multi-part fetches and just get the
whole message in one go. That code path *is* tested, because we were
doing it for small messages anyway. 

Doing it for large messages means that the connection is blocked while
the whole of the message is downloaded, so other, higher-priority user
requests can't be handled — and it means that the progress indication
doesn't work. That's why it was never an option before, but it *does*
suffice to work around the server bug, so it does seem to be necessary.

So I'd like permission to break the string freeze to add this option.

If anyone has improved suggestions for the English strings to use, that
would make me happy.

I wanted to do a positive option for Fetch messages in multiple parts,
to allow better responsiveness, which would default to TRUE. But when
we upgrade Evolution, any new option will be assumed to be FALSE in any
pre-existing account. So AFAICT it has to be a *negative* option, along
the lines of Fetch messages in one part.

-- 
David WoodhouseOpen Source Technology Centre
david.woodho...@intel.com  Intel Corporation

___
evolution-hackers mailing list
evolution-hackers@gnome.org
To change your list options or unsubscribe, visit ...
http://mail.gnome.org/mailman/listinfo/evolution-hackers


[Evolution-hackers] Remove e-soap-message, e-soap-response before 3.0?

2011-03-19 Thread David Woodhouse
I've just been reviewing the e-soap-{message,response} API, and I don't
like it much.

Some of our responses can be *huge* — the whole of a MIME message will
be base64-encoded and included in the XML response.

Yet we allow libsoup to gather the whole message into a SoupBuffer, and
then we parse that into an xmlDoc so we have *two* copies of it in
memory, at least for a while.

I don't want *any* copies of it in memory. So for a start I've changed
things to parse the response as it comes in, and we only have one copy:
http://git.infradead.org/evolution-ews.git/commitdiff/3e11452
... and I'll be working on using the SAX interface to libxml2
(optionally) so that we really can spool the MimeContent node to disk
as it comes in and not have *any* copies in memory.

We're going to have similar issues with outgoing requests, too. When we
call the Exchange server's CreateItem method, we'll need to include the
full MIME message in our *request*, and we'll want to stream that too.

Anyway, since nobody is actually *using* ESoapMessage and ESoapResponse
in 3.0, because the GroupWise code hasn't actually converted to it yet
and is still using its own private SoupSoapMessage/SoupSoapResponse, I
think we should rip it *out* of the 3.0 release, and put it back into
3.1 when it's finished. For now it can live in evolution-ews, which has
a copy anyway so that we can build that for 2.32.


Chen, what do you think? Could you rip it out before you do the release
on Monday? Matthew suggests not bothering to bump the libedataserver
soname, since that's just gonna cause havoc at the 11th hour over
something no one is using.

-- 
David WoodhouseOpen Source Technology Centre
david.woodho...@intel.com  Intel Corporation


___
evolution-hackers mailing list
evolution-hackers@gnome.org
To change your list options or unsubscribe, visit ...
http://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] Camel Manifesto (update)

2011-03-13 Thread David Woodhouse
On Thu, 2011-02-17 at 18:37 +, David Woodhouse wrote:
 I was told today that the GIO (and libsoup) async methods may not be
 called from a thread other than the one running the mainloop. I found a
 stupid race in libsoup¹ and filed a fix, but the bug was closed INVALID
 because apparently it's not *supposed* to be thread-safe.
 
 That's a serious issue for using it from Evolution; it means we have to
 jump through stupid hoops like using idle callbacks to call it from the
 main thread, just as we do for gconf. 

Ug, and now I've found that that workaround doesn't work either. If we
try to rename a folder, we end up blocking in the main thread, waiting
for the soup request that we deliberately put into an idle function so
that it could run in the main thread...

Thread 1 (Thread 0x77d93980 (LWP 9874)):
#0  pthread_cond_wait@@GLIBC_2.3.2 () at 
../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:162
#1  0x0034ff00ee2d in e_flag_wait (flag=0x2002220) at e-flag.c:130
#2  0x7fffe387e934 in ews_get_folder_info_sync (store=0x6e42a0 
[CamelEwsStore], top=0x1480880 asd, flags=1, error=0x0) at 
camel-ews-store.c:500
#3  0x0035020508ae in camel_store_get_folder_info (store=0x6e42a0 
[CamelEwsStore], top=0x1480880 asd, flags=1, error=0x0) at camel-store.c:1122
#4  0x003505460ce8 in folder_tree_cell_edited_cb (folder_tree=value 
optimized out, path_string=value optimized out, new_name=0x1ca6460 asd) at 
em-folder-tree.c:624

As before, I would prefer to *fix* the broken locking, so that we can
call the soup functions from any thread. But if I'm still not allowed to
do that, what's the best way to fix this deadlock?

-- 
dwmw2

___
evolution-hackers mailing list
evolution-hackers@gnome.org
To change your list options or unsubscribe, visit ...
http://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] Handy GObject debugging trick

2011-03-10 Thread David Woodhouse
On Wed, 2011-03-09 at 18:20 -0500, Matthew Barnes wrote:
 
 4. Read the stats to see how badly $MY_APPLICATION is leaking objects.

This is useful; thanks. Although we really *do* need to see it at
runtime. I played a little with Colin's LD_PRELOAD hack and Evolution:

CamelDataCache | created 214 | destroyed 0
CamelEwsFolder | created 30 | destroyed 0
CamelEwsSummary | created 30 | destroyed 0
CamelFolderSearch | created 232 | destroyed 0
CamelIMAPXFolder | created 183 | destroyed 0
CamelIMAPXSummary | created 183 | destroyed 0

Are we not *ever* closing a folder, after the UI has visited it once?

No wonder Evolution tends to grow until it's eating all the memory in my
system... :)

-- 
dwmw2

___
evolution-hackers mailing list
evolution-hackers@gnome.org
To change your list options or unsubscribe, visit ...
http://mail.gnome.org/mailman/listinfo/evolution-hackers


[Evolution-hackers] Cache encryption

2011-03-04 Thread David Woodhouse
I'm working on Enterprise use of Evolution, and one of the big
requirements is encryption of data at rest. The answer just encrypt the
whole of the user's home directory only puts them off for so long.

So I'm looking at implementing this directly in camel-data-cache,
e-cal-backend-cache, etc.

I'll probably do the encryption with a randomly-generated key, which
itself is stored locally, encrypted with a password. 

That way, changing the password doesn't involve re-encrypting the whole
of the store; you only need to re-encrypt the master key. It also means
that we can tie the password for the cache to the password for the
server; entering one will allow access to both.

Hopefully, the changes required to code that *uses* the cache
functionality should be fairly limited. Mostly it should be handled by
extra arguments to camel_data_cache_new(), e_cal_backend_cache_new(),
camel_db_open() and similar functions.

I'm hoping that it's reasonable to declare that *filenames* are not
sensitive, and that we only need to encrypt the *contents* of files.
Does that seem fair?

Any other comments on the approach?

-- 
David WoodhouseOpen Source Technology Centre
david.woodho...@intel.com  Intel Corporation

___
evolution-hackers mailing list
evolution-hackers@gnome.org
To change your list options or unsubscribe, visit ...
http://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] Cache encryption

2011-03-04 Thread David Woodhouse
On Fri, 2011-03-04 at 07:30 -0500, Matthew Barnes wrote:
 Got'cha.  Seems reasonable.  Thanks for humoring me. 

No problem; thanks for paying attention to my ramblings.

-- 
dwmw2

___
evolution-hackers mailing list
evolution-hackers@gnome.org
To change your list options or unsubscribe, visit ...
http://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] Cache encryption

2011-03-04 Thread David Woodhouse
On Fri, 2011-03-04 at 07:49 -0500, Jeffrey Stedfast wrote:
 Easiest way to implement this feature in Camel might be to implement a
 CamelMimeFilter or CamelStream that encrypts/decrypts the content as it
 reads/writes the data. Implementing it as a CamelStream might be the
 best approach as it would more easily allow seeking. 

Yeah, absolutely. I had looked at the camel_cache_cache_get() function
which returns a stream, and concluded that this made it nice and easy to
just slip in a stream class which does the encryption/decryption for us.

Thanks.

-- 
dwmw2

___
evolution-hackers mailing list
evolution-hackers@gnome.org
To change your list options or unsubscribe, visit ...
http://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] Cache encryption

2011-03-04 Thread David Woodhouse
On Fri, 2011-03-04 at 05:47 -0700, Sankar P wrote:
 
 Will it be not simpler if we can make Evolution use a custom location
 for cache, that the user/root can set ? 
 
 That way, we don't have to write (and more importantly maintain) yet
 another encryption/decryption library and instead just use a different
 folder for storing all secret/confidential data, which can be a custom
 mount point which runs on encrypted partition. 

I did look at that briefly. But I think it would have problems from both
the implementation and the user experience point of view.

With the encryption handled by e-d-s itself, it's all nice and
self-contained. When it needs a password it can *ask* for it. If it
wants to change the cache-encryption password to match the password for
the online account, then it can handle that too. It's all integrated and
works coherently.

If you start trying to co-ordinate it with an external thing like an
ecryptfs mount point, then there are a bunch of things you have to work
with which are essentially outside your control.

 From a distro point of view, libraries with security packages usually
 have extra maintenance overhead (Are you sure your package is not
 shipped to america-banned countries ? etc.)  So I believe it will be a
 better idea if the [en/de]cryption capable packages are less in
 number. 

I wasn't planning to do any of the actual crypto code directly in
Evolution; I was planning to use NSS for that. The additional
functionality would presumably live inside #ifdef CAMEL_HAVE_NSS, like a
lot of other code in e-d-s already does.

-- 
dwmw2

___
evolution-hackers mailing list
evolution-hackers@gnome.org
To change your list options or unsubscribe, visit ...
http://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] Cache encryption

2011-03-04 Thread David Woodhouse
On Fri, 2011-03-04 at 07:49 -0500, Jeffrey Stedfast wrote:
 Anyways, do you also plan on encrypting mbox/maildir files? Might make
 sense... 

Missed that bit; overtrimming my citations. I probably wouldn't do this
*myself* in the first round, but I'd hope that someone might add it
soon, and I might accidentally get drunk one night and do it anyway.
These things happen...

-- 
dwmw2

___
evolution-hackers mailing list
evolution-hackers@gnome.org
To change your list options or unsubscribe, visit ...
http://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] EWS: refresh your calendar cache

2011-03-02 Thread David Woodhouse
On Wed, 2011-03-02 at 08:11 -0500, Matthew Barnes wrote:
 
 Also, I'd encourage developers to post the occasional status update to
 the list.  Nothing formal, and at your own discretion.  

This is probably a good enough time for us to do that for evo-ews.

If you clone the tree from git://git.infradead.org/evolution-ews.git
then you should be able to build it on a stock Evolution 2.32 system
(for example Fedora 14) and use it against an Exchange 2007+ server.

It's working nicely in read-only mode and I've been doing that to read
all my company email for the last few weeks, and can view my company
calendar that way too.

There are two things I think need fixing before I'd really suggest that
people use it 'in anger', but which aren't *really* showstoppers:

 - Tell the server about messages having been read, so they don't appear
   as unread in IMAP/Outlook. And deletion too, I suppose.

 - Make it handle 'variance' of recurring meetings — for example if a
   single instance of a recurring meeting is deleted, or rescheduled.
   We don't show those exceptions yet.

It also doesn't let you send mail yet; I'm still using SMTP for that
(and IMAP for writing the Fcc to the mail store).

But for someone who wants to be an 'early adopter' and try it, it's
certainly feasible right now. Even if you just look at the things we
*think* we've done, and heckle when we've got things wrong, that's still
useful.

-- 
dwmw2

___
evolution-hackers mailing list
evolution-hackers@gnome.org
To change your list options or unsubscribe, visit ...
http://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] evolution-kolab: IMAPX RFC2086/RFC4314 (IMAP4 ACL) compliance

2010-08-25 Thread David Woodhouse
On Wed, 2010-08-25 at 10:29 +0200, Christian Hilberg wrote:
 Support for IMAP ACLs would be very useful when dealing with a Kolab server 
 (which uses Cyrus imapd with ACL support), so I'm interested in getting to 
 know about the current status of ACL support within IMAPX.

∄ ACL support in imapx.

-- 
dwmw2

___
evolution-hackers mailing list
evolution-hackers@gnome.org
To change your list options or unsubscribe, visit ...
http://mail.gnome.org/mailman/listinfo/evolution-hackers


[Evolution-hackers] Memory leaks

2010-08-23 Thread David Woodhouse
I got offended by Evolution growing to almost 4GiB and having to be
killed if I wanted to use my workstation for anything else, so I've
started running it in Valgrind.

I've filed bugs for the reports of 'definitely lost', and have fixed a
bunch of them.

But there are *lots* of reports of memory blocks being 'possibly lost',
too. Should we be worrying about those?

See http://valgrind.org/docs/manual/mc-manual.html#mc-manual.leaks for a
coherent description of what 'possibly lost' means -- basically it means
that the objects *might* be reachable, but not through direct pointers.
They're only reachable through pointers which point somewhere *inside*
the allocated block, and would only ever be freed if we do some kind of
pointer arithmetic magic to recalculate the allocated pointer and free
the block.

The Valgrind manual calls these 'interior-pointers', and says this of
them and the 'possibly lost' case:

This could just be a random value in memory that happens to
point into a block, and so you shouldn't consider this ok unless
you know you have interior-pointers.

I'm therefore tempted to go ahead and file bugs for *every* report of
'possibly lost' that Valgrind gives me when I run Evolution.

Even if some of them are false positives, I think we should find some
way to tell Valgrind about them -- either with a suppressions file, or
preferably by finding some annotation which lets Valgrind know that this
'interior-pointer' really *is* going to be followed.

Even if it means cleaning up some false positives, I think it would be
extremely useful to get to a point where valgrind runs cleanly and new
memory leaks can easily be detected.

What do you think?

-- 
David WoodhouseOpen Source Technology Centre
david.woodho...@intel.com  Intel Corporation

___
evolution-hackers mailing list
evolution-hackers@gnome.org
To change your list options or unsubscribe, visit ...
http://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] Memory leaks

2010-08-23 Thread David Woodhouse
On Mon, 2010-08-23 at 13:25 +0200, Milan Crha wrote: 
 On Mon, 2010-08-23 at 11:12 +0100, David Woodhouse wrote:
  I got offended by Evolution growing to almost 4GiB and having to be
  killed if I wanted to use my workstation for anything else, so I've
  started running it in Valgrind.
 
  I've filed bugs for the reports of 'definitely lost', and have fixed a
  bunch of them.
 
   Hi,
 that's pretty bad. What are the bug numbers/links, please?

https://bugzilla.gnome.org/showdependencytree.cgi?id=627707

I've grouped a bunch of different GtkHTML reports together. although I'm
not sure that was the right thing to do.

The fixes I mentioned are sitting in my working tree, and I'll commit
and push them today.

I'm now running in 'Massif', to see if that gives me anything
interesting.

  But there are *lots* of reports of memory blocks being 'possibly lost',
  too. Should we be worrying about those?
  
  ...
 
  I'm therefore tempted to go ahead and file bugs for *every* report of
  'possibly lost' that Valgrind gives me when I run Evolution.
 
 Please do not do that. Evolution has cache of CamelStore-s,
 CamelFolder-s and such, so almost anything from evolution can be pointed
 by some global/static variable. And indeed, these caches aren't freed by
 the application, but by the operating system itself.

If they're referenced by a global variable, surely valgrind would know
that they're reachable and would not complain about them?

And if *not*, can we find a way to teach Valgrind that they're
reachable, so that it can be useful for us and not have so many false
positives?

  Even if some of them are false positives, I think we should find some
  way to tell Valgrind about them -- either with a suppressions file, or
  preferably by finding some annotation which lets Valgrind know that this
  'interior-pointer' really *is* going to be followed.
  
  Even if it means cleaning up some false positives, I think it would be
  extremely useful to get to a point where valgrind runs cleanly and new
  memory leaks can easily be detected.
 
 I agree, though I'm afraid it's not possible (I didn't check valgrind
 yet, but I do not suppose it has any compiler flags/annotations in the
 code, because it uses the binary file itself, together with a debug
 information). I can be wrong here, though.

I had thought briefly about annotating the code, and perhaps extending
sparse to handle refcounts and 'ownership' of memory so that it can warn
at compile time if you write code which leaks.

But basically, what you seem to be saying is that Valgrind is
essentially useless for us. I'd be very disappointed if that really
turns out to be the case -- we should definitely talk to Julian about
that before we give up.


   Bye,
   Milan
 
 P.S.: I didn't CC you intentionally, because I know you are subscribed
 on this list, and because I do not want anyone CC'ing me on this list,
 because I'm subscribed to it too. ;)

I'm running Evo under Valgrind -- if I were to look into *all* of the 67
mailing list folders that have unread mail in them, I'd get nothing else
done today. I only looked back in this folder because I happened to
notice a reference to your reply on the IRC channel, half an hour later.

Conversations go slowly enough at times, with people being spread around
the world in different time zones, that it really doesn't help if they
get delayed by half and hour or more at a time because you drop the
participants from Cc.

Note that I also didn't Cc you intentionally, because I know (and at
this particular moment I remember) *your* preference and it would be
kind of rude to *deliberately* go against it.

-- 
dwmw2


___
evolution-hackers mailing list
evolution-hackers@gnome.org
To change your list options or unsubscribe, visit ...
http://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] Camel IMAPX RFC5464 compliance

2010-08-21 Thread David Woodhouse
Please don't drop me from Cc when replying to my messages.
See http://david.woodhou.se/reply-to-list.html

On Wed, 2010-08-18 at 10:30 +0200, Christian Hilberg wrote:
 Hi everyone,
 
 On Thursday 05 August 2010 David Woodhouse wrote:
  On Wed, 2010-08-04 at 12:22 +0200, Christian Hilberg wrote:
   Now, I would like to know how we should deal with the issue. We (the
   evolution-kolab developers) could patch the 2.30 version of IMAPX only to
   get things running. In this case, would our additions be pulled
   upstream?
  [...] 
  I would strongly recommend that you do it in the development branch
  first, then we can backport it to gnome-2-30.
  I've been backporting most IMAPX changes from master to the 2.30 branch;
  I see no particular reason why we shouldn't backport METADATA support
  too, as long as you're careful not to add new user-visible strings that
  would need translation.
 
 Okay, let's say, we will patch upstream IMAPX to support RFC5464. The patch 
 gets reviewed, and after being polished it will (hopefully :-) be accepted in 
 upstream.
 
 How long do you think it would take you to backport such a patch to 2.30, 
 assuming we heed to the aforementioned implementation recommendations?

Just to clarify -- I would expect you to do the backport yourself.

When I said I've been backporting most IMAPX changes from master to
2.30... that's because I'd been making most of them. I wasn't
volunteering to backport *your* changes too. :)

It shouldn't be that hard. Probably even less than a day -- I suspect
Chen was a little pessimistic in his estimate.

-- 
David WoodhouseOpen Source Technology Centre
david.woodho...@intel.com  Intel Corporation

___
evolution-hackers mailing list
evolution-hackers@gnome.org
To change your list options or unsubscribe, visit ...
http://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] Camel IMAPX RFC5464 compliance

2010-08-05 Thread David Woodhouse
On Wed, 2010-08-04 at 12:22 +0200, Christian Hilberg wrote:
 Now, I would like to know how we should deal with the issue. We (the 
 evolution-kolab developers) could patch the 2.30 version of IMAPX only to get 
 things running. In this case, would our additions be pulled upstream?
   As an alternative, would anyone like to implement RFC5464 in the current 
 upstream IMAPX so we could try and backport the changes into 2.30?

I would strongly recommend that you do it in the development branch
first, then we can backport it to gnome-2-30.

I've been backporting most IMAPX changes from master to the 2.30 branch;
I see no particular reason why we shouldn't backport METADATA support
too, as long as you're careful not to add new user-visible strings that
would need translation.

-- 
David WoodhouseOpen Source Technology Centre
david.woodho...@intel.com  Intel Corporation


___
evolution-hackers mailing list
evolution-hackers@gnome.org
To change your list options or unsubscribe, visit ...
http://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] imapx: Avoid running FETCH_NEW_MESSAGES and REFRESH_INFO jobs simultaneously

2010-07-12 Thread David Woodhouse
On Mon, 2010-07-12 at 11:32 +0530, chen wrote:
 
 I was just thinking whether we would need the logic to fetch new
 messages from refresh_info job at-all while scanning for changes. We
 fetch new messages always before scanning for changes, so ideally
 scan_changes should only sync the flags.
 
 At this point, fetching new messages from scan_changes would be used
 very rarely if there was any message deleted wrongly in cache 

Or if messages come in between the call to fetch_new_message() and the
later call to scan_changes().

And in fact there are a few other places that this can happen and we can
get flags for messages we've never heard about -- it happens in IDLE
too, or on NOOP, and in SELECT when we're using QRESYNC.

I was thinking of having a per-folder list of 'messages for which we
only have flags', and all those places can just append to it. And
something, somewhere, will check for it and trigger a fetch of the
headers. But it still won't care *why* we were told ;)

-- 
dwmw2

___
evolution-hackers mailing list
evolution-hackers@gnome.org
To change your list options or unsubscribe, visit ...
http://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] Heads Up: More Camel API breakage in 2.31

2010-07-11 Thread David Woodhouse
On Thu, 2010-07-08 at 11:58 -0400, Matthew Barnes wrote:
 I finally finished converting Camel's error handling code to GError.
 CamelException is no more.  I plan to push this later today after a bit
 more testing.  It will debut in 2.31.5 along with another libcamel
 soname increment.

I note that in some places you've elected not to propagate the error
pointer. For example in imapx_parse_status_info(): 

-imapx_parse_status_info (struct _CamelIMAPXStream *is, CamelException *ex)
+imapx_parse_status_info (struct _CamelIMAPXStream *is, GError **error)
-   sinfo-messages = camel_imapx_stream_number 
(is, ex);
+   sinfo-messages = camel_imapx_stream_number 
(is, NULL);
... etc.

Why is that? Don't we need to ensure that these subfunction calls return
errors properly? Previously, if they failed to parse the element they're
supposed to parse (a number in this case), they'd set the exception
which would eventually get noticed (although I would have thought the
old code _should_ have included !camel_exception_is_set() in the loop
conditions, as other loops did). But now, it just gets lost.

-- 
David WoodhouseOpen Source Technology Centre
david.woodho...@intel.com  Intel Corporation

___
evolution-hackers mailing list
evolution-hackers@gnome.org
To change your list options or unsubscribe, visit ...
http://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] imapx: Avoid running FETCH_NEW_MESSAGES and REFRESH_INFO jobs simultaneously

2010-07-11 Thread David Woodhouse
On Sun, 2010-07-11 at 15:11 +0100, David Woodhouse wrote:
 From 1d1b146e58f918f67ccff93c4fb5388429bf12e7 Mon Sep 17 00:00:00 2001
 From: David Woodhouse david.woodho...@intel.com
 Date: Sun, 11 Jul 2010 15:11:17 +0100
 Subject: [PATCH] imapx: Avoid running FETCH_NEW_MESSAGES and REFRESH_INFO 
 jobs simultaneously

 There are various places where we interpret FETCH results and use
 imapx_match_active_job to find the current job, which will behave badly
 if there are two jobs which could potentially be responsible for the FETCH.
 
 In particular, this was causing a problem when we triggered a fetch of new
 messages from select_done(), and that command was submitted at the same time
 as a refresh_info command to fetch all flags. The server (Dovecot) was
 returning all the untagged FETCH results before either completion line,
 and all the flags were getting assigned to the fetch_new_messages job,
 causing a bunch of 'g_array_append_vals: assertion `array' failed' warnings,
 and all messages to disappear because the refresh_info job didn't see them. 

I looked at fixing this by making the FETCH handling code work out which
job it should be looking at.. but that's hard because of the ways in
which a fetch_new_messages job might just be fetching flags, and a
refresh_info job might fetch new messages.

The real fix, I think, is to stop the FETCH handling code from caring
about the jobs at all. The untagged messages tell us about the state of
the mailbox, and we should just deal with that information as it comes
in, without worrying about _why_ the server is sending it.

For refresh_info to notice when messages are deleted, we could set a
'possibly expunged' flag in the message in our local store, then the
FETCH handler could clear that flag whenever it gets flags for a
message. The refresh_info logic would then be:
 - Set this flag on all messages
 - Issue UID FETCH 1:* (UID FLAGS)
 - Delete all messages without this flag set.

The idea would be to kill off _all_ use of job-u.refresh_info.infos and
similar fields. Where we've fetched message flags before the headers, we
could keep a list of those in the folder info, not the job. And that
way, _wherever_ those new flags come from (SELECT, IDLE, NOOP, FETCH,
etc.) they'll get handled consistently.

All use of imapx_match_active_job() without a uid parameter would then
be considered a bug.

Chen, what do you think?

-- 
David WoodhouseOpen Source Technology Centre
david.woodho...@intel.com  Intel Corporation

___
evolution-hackers mailing list
evolution-hackers@gnome.org
To change your list options or unsubscribe, visit ...
http://mail.gnome.org/mailman/listinfo/evolution-hackers


  1   2   >