[Evolution-hackers] help debugging a disk-space leak ...

2011-04-08 Thread Michael Meeks
Hi there,

I have (perhaps) an unusal setup - I get mail to an IMAP inbox, and
then filter much of it off to local storage, and move the rest manually
as I need to.

I have around 1000 mostly short, text only messages in my IMAP inbox;
and yet:

du -m 
/home/michael/.local/share/evolution/mail/groupwise/mme...@prv1-3.novell.com/folders/INBOX/cur
  | tail -n 1
2562
/home/michael/.local/share/evolution/mail/groupwise/mme...@prv1-3.novell.com/folders/INBOX/cur

2.5 Gb is a tad large for that.

Interestingly, the side-panel says 760 unread, 876 total, whereas the
right-click-properties says:

Unread messages  760
Total messages  2711

and of course, I've got synchronise for off-line use set.

So ... how should I go about debugging that ? I assume that blowing
away the entire folders directory there, and just re-building it from
the server is the easy/cheating hack-around the problem but ...

Thoughts appreciated.

Thanks,

Michael.

-- 
 michael.me...@novell.com  , Pseudo Engineer, itinerant idiot


___
evolution-hackers mailing list
evolution-hackers@gnome.org
To change your list options or unsubscribe, visit ...
http://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] Memory leaks

2010-09-08 Thread Michael Meeks

On Mon, 2010-08-23 at 16:32 +0200, Milan Crha wrote: 
 On Mon, 2010-08-23 at 13:10 +0100, David Woodhouse wrote:
  The fixes I mentioned are sitting in my working tree, and I'll commit
  and push them today.

This is awesome work - thanks ! :-)

  If they're referenced by a global variable, surely valgrind would know
  that they're reachable and would not complain about them?
 
 I do not know how it works precisely.

It should work nicely as David says.

 I just got a thought, what if some of those possibly lost come from
 GSlice? Then G_SLICE=always-malloc should help valgrind with it. At
 least a bit.

Right - except for this, which is a disaster area :-) I had forgotten
that it was critical to turn this on (which turns off the glib slab
allocator) before valgrinding myself ;-)

Hmm - I wonder how best to deal with that. I wonder if the glib guys
would accept a patch that incorporated /usr/include/valgrind.h's
RUNNING_ON_VALGRIND magic ;-) [ amazingly fun etc. ].

Thanks,

Michael.

-- 
 michael.me...@novell.com  , Pseudo Engineer, itinerant idiot



___
evolution-hackers mailing list
evolution-hackers@gnome.org
To change your list options or unsubscribe, visit ...
http://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] evolution-kolab: Lifting the veil

2010-07-08 Thread Michael Meeks
Hi Christian,

You write-up sounds really exciting :-)

On Thu, 2010-07-08 at 10:28 +0200, Christian Hilberg wrote:
  We will try to map as much of Kolab's functionality as possible onto 
 Evolution whithout changing Evolution itself (other than providing a plugin, 
 that is). Especially, we will not touch core Evolution or E-D-S.

It would be good to know what we are missing though - if there are new
and interesting things that can be added to the front-end.

 * targeting Linux for client and server *solely*. No specific attempts will
   be made to be portable to other platforms (like *BSD, Windows, ...).

That is quite normal; Fridrich loves to port stuff to Windows ;-) and
as you say using the right platform libs should make that easier.

 * targeting Evo/EDS 2.30 only (2.30.2 presently, we should be able to follow
   patchlevels). It is a customer requirement that the plugin be used with
   current stable versions of GNOME and we will not be able to stand API churns
   or move to GTK+-3 on our development platforms for several reasons, mostly
   due to development deadlines.

Unfortunate, but understandable :-)

 * providing DEB packages for Debian/GNU Linux only. RPM packages should not
   be too hard to generate from our debs (or by other means), but we will not
   be able to provide them.

Ah - each distro will package it their own way.

 Our project will be GPLv2 (or a comparable FLOSS license which will assure at 
 least GPLv2's freedom level ;-). There should not be any licensing issues 
 here. If there are doubts, please feel free to ask for clarification.

Soo - personally, I would love to see the code live inside e-d-s and/or
Evolution itself: that should make it easier to maintain longer term,
adapt for re-factoring, get translators involved etc.

Licensing wise, the e-d-s code is currently LGPLv2, and Evolution is
LGPLv2 or v3 (at your choice). I would recommend sticking with that, or
going LGPLv2+, rather than having a plain GPLv2.

  Our customer, for whom we will develop this solution, is also very
 specific about the project being hosted publicly and accessible by everyone.

Wonderful :-)

 If Evolution staff will be willing to host our project sources within
 Evo git repo, we'll happily transfer our stuff there as soon as we
 have a first preview ready.

Perhaps something to dicuss on #evolution (irc.gimp.net) - but it'd be
great to have you working in the same git repo IMHO [ not that it's my
decision of course - I defer to Matthew/Chen etc. ;-]

ATB,

Michael.

-- 
 michael.me...@novell.com  , Pseudo Engineer, itinerant idiot


___
evolution-hackers mailing list
evolution-hackers@gnome.org
To change your list options or unsubscribe, visit ...
http://mail.gnome.org/mailman/listinfo/evolution-hackers


[Evolution-hackers] alarm-notify ...

2010-04-23 Thread Michael Meeks
Hi guys,

I was reading calendar/gui/alarm-dialog/alarm-notify.c - trying (of
course) to work out why alarm delivery is apparently not working at all.

I just committed some code enabling dbus threading - without which you
can quite happily use dbus from multiple threads, only it will crash and
burn very horribly. That seemed to fix a nasty race I was getting
whereby no alarms showed up at all.

It might also help to undo:

static void
message_push (Message *msg)
{
/* This used be pushed through the thread pool. This fix is made to 
work-around
the crashers in dbus due to threading. The threading is not completely 
removed as
its better to have alarm daemon running in a thread rather than 
blocking main thread.
 This is the reason the creation of thread pool is commented out */
msg-func (msg);
}

This sort of thing.

I could also not see where the Repeat functionality (customize an
alarm, and select Repeat the alarm is setup - surely not on the e-d-s
side ? - do we still have it ?

It was also a surprise to me to see the default setting for create an
alarm 15 mins before each appointment to be off - is that a
feature ? :-) it was unexpected.

Finally - it seems some migration code went awry somewhere here, and
busted the use systray setting for the alarm daemon:

https://bugzilla.gnome.org/show_bug.cgi?id=616658

HTH,

Michael.

-- 
 michael.me...@novell.com  , Pseudo Engineer, itinerant idiot

___
evolution-hackers mailing list
evolution-hackers@gnome.org
To change your list options or unsubscribe, visit ...
http://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] edbus branch review ...

2010-03-16 Thread Michael Meeks

On Sat, 2010-02-27 at 09:37 +, Ross Burton wrote:
 On Fri, 2010-02-26 at 22:31 +, Michael Meeks wrote:
  Thoughts / flameage ? :-)
 
 My thoughts are woohoo and great work.  If I'm feeling brave on
 Sunday I'll start a clean build of eds and evo-edbus and dogfood it.

Sooo ... what with the code freezes, and the 'real' interactivity bug
fix being in dbus itself:

http://cgit.freedesktop.org/dbus/dbus/commit/?h=dbus-1.2id=ae95a0ff9651a774cd983436d7e656c62492659c

I suggest that we defer the GDBus integration until we have a GDBus IDL
compiler, and the code has been included into glib.

HTH,

Michael.

-- 
 michael.me...@novell.com  , Pseudo Engineer, itinerant idiot


___
Evolution-hackers mailing list
Evolution-hackers@gnome.org
http://mail.gnome.org/mailman/listinfo/evolution-hackers


[Evolution-hackers] looong blockup in the UI ...

2010-03-02 Thread Michael Meeks
Hi guys,

I just got an, oh - several second blocking of the UI of Evolution
inside:

Thread 1 (Thread 0xb5c1a740 (LWP 775)):
#0  0xe424 in __kernel_vsyscall ()
#1  0xb6d56aef in fsync () from /lib/libpthread.so.0
#2  0xb5fc0cbc in write_to_temp_file (err=value optimized out, 
dest_file=value optimized out, length=value optimized out, 
contents=value optimized out) at gfileutils.c:978
#3  IA__g_file_set_contents (err=value optimized out, dest_file=value 
optimized out, length=value optimized out, contents=value optimized out)
at gfileutils.c:1082
#4  0xb76c83e4 in shell_view_save_state (shell_view=value optimized out) at 
e-shell-view.c:235
#5  0xb76c8451 in shell_view_state_timeout_cb (shell_view=0x846cad0 
[EMailShellView]) at e-shell-view.c:249
#6  0xb5fd1b06 in g_timeout_dispatch (source=0xa63b1dd0, callback=0, 
user_data=0x846cad0) at gmain.c:3396
#7  0xb5fd1448 in g_main_dispatch (context=value optimized out) at 
gmain.c:1960
#8  IA__g_main_context_dispatch (context=value optimized out) at gmain.c:2513
#9  0xb5fd4bf3 in g_main_context_iterate (context=0x8074808, block=1, 
dispatch=1, self=0x807bf88) at gmain.c:2591
#10 0xb5fd50ba in IA__g_main_loop_run (loop=0x94b2380) at gmain.c:2799
#11 0xb675cc69 in IA__gtk_main () at gtkmain.c:1218
#12 0x0804ab37 in main (argc=134960160, argv=0x1) at main.c:611

This of course is down to the geniuses who decided that we need an
'fsync' in glib's g_file_set_contents (we don't).

Does anyone object if instead of g_file_set_contents we just use fopen,
fwrite, fclose ? any API method that does fsync is just a disaster-area
for latency on a loaded ext3 system; and it is -totally- un-necessary to
have some heavy-lifting transactional guarentees for an utter
corner-case, for a non-critical file anyway :-)

Thanks,

Michael.

-- 
 michael.me...@novell.com  , Pseudo Engineer, itinerant idiot


___
Evolution-hackers mailing list
Evolution-hackers@gnome.org
http://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] camel-folder-summary locking ...

2010-03-01 Thread Michael Meeks

On Fri, 2010-02-26 at 22:20 +, Michael Meeks wrote: 
   I just hit a nasty in camel-folder-summary; suggested patch attached,
 seemingly a simple problem of a re-enterancy hazard in the same thread
 with a ghashtable:

Ok; this was fixed by Chen's reversion of freeing the uid and NULL'ing
it (recently). However reading the code some more, I think we can do a
lot better, and more cleanly.

I attach the patch; discussed with Chen  Srini; I just committed to
gnome-2-28 and master.

HTH,

Michael.

-- 
 michael.me...@novell.com  , Pseudo Engineer, itinerant idiot


From c5a86b33ef7d10d25b227221efec8106d8c82ed9 Mon Sep 17 00:00:00 2001
From: Michael Meeks michael.me...@novell.com
Date: Mon, 1 Mar 2010 11:23:18 +
Subject: [PATCH] Fix unpleasant issue with (remove_item) more elegantly. Instead of
 taking an untaking the ref-lock thousands of times, and trying to
 free messages inside the loop (while cobbering their uid to encourage
 message_info_free not to do an unsave hash table removal, inside the
 g_hash_table_foreach_remove loop).
 Instead - we now just build a list very quickly of the messages we
 want to drop. As such, the hash table iteration will be ~instant,
 and we do not need to take / drop the lock each iteration, simplifying
 and perhaps accelerating the code. We then free the messages later
 before releasing the summary lock.

---
 camel/camel-folder-summary.c |   25 ++---
 1 files changed, 14 insertions(+), 11 deletions(-)

diff --git a/camel/camel-folder-summary.c b/camel/camel-folder-summary.c
index 85265ac..3218a88 100644
--- a/camel/camel-folder-summary.c
+++ b/camel/camel-folder-summary.c
@@ -778,21 +778,13 @@ cfs_count_dirty (CamelFolderSummary *s)
 
 /* FIXME[disk-summary] I should have a better LRU algorithm  */
 static gboolean
-remove_item (gchar *key, CamelMessageInfoBase *info, CamelFolderSummary *s)
+remove_item (gchar *key, CamelMessageInfoBase *info, GSList **to_free_list)
 {
 	d(printf(%d(%d)\t, info-refcount, info-dirty)); /* camel_message_info_dump (info); */
-	CAMEL_SUMMARY_LOCK(info-summary, ref_lock);
 	if (info-refcount == 1  !info-dirty  !(info-flags  CAMEL_MESSAGE_FOLDER_FLAGGED)) {
-		CAMEL_SUMMARY_UNLOCK(info-summary, ref_lock);
-		/* Hackit so that hashtable isn;t corrupted. */
-		/* FIXME: These uid strings are not yet freed. We should get this done soon. */
-		camel_pstring_free (info-uid);
-		info-uid = NULL;
-		/* Noone seems to need it. Why not free it then. */
-		camel_message_info_free (info);
+		*to_free_list = g_slist_prepend (*to_free_list, info);
 		return TRUE;
 	}
-	CAMEL_SUMMARY_UNLOCK(info-summary, ref_lock);
 	return FALSE;
 }
 
@@ -807,6 +799,7 @@ remove_cache (CamelSession *session, CamelSessionThreadMsg *msg)
 	struct _folder_summary_free_msg *m = (struct _folder_summary_free_msg *)msg;
 	CamelFolderSummary *s = m-summary;
 	CamelException ex;
+	GSList *to_free_list = NULL, *l;
 
 	CAMEL_DB_RELEASE_SQLITE_MEMORY;
 	camel_exception_init (ex);
@@ -819,7 +812,17 @@ remove_cache (CamelSession *session, CamelSessionThreadMsg *msg)
 	dd(printf(removing cache for  %s %d %p\n, s-folder ? s-folder-full_name : s-summary_path, g_hash_table_size (s-loaded_infos), (gpointer) s-loaded_infos));
 	/* FIXME[disk-summary] hack. fix it */
 	CAMEL_SUMMARY_LOCK (s, summary_lock);
-	g_hash_table_foreach_remove  (s-loaded_infos, (GHRFunc) remove_item, s);
+
+	CAMEL_SUMMARY_LOCK(s, ref_lock);
+	g_hash_table_foreach_remove  (s-loaded_infos, (GHRFunc) remove_item, to_free_list);
+	CAMEL_SUMMARY_UNLOCK(s, ref_lock);
+
+	/* Deferred freeing as _free function will try to remove
+	   entries from the hash_table in foreach_remove otherwise */
+	for (l = to_free_list; l; l = l-next)
+		camel_message_info_free (l-data);
+	g_slist_free (to_free_list);
+
 	CAMEL_SUMMARY_UNLOCK (s, summary_lock);
 	dd(printf(done .. now %d\n,g_hash_table_size (s-loaded_infos)));
 
-- 
1.6.0.2

___
Evolution-hackers mailing list
Evolution-hackers@gnome.org
http://mail.gnome.org/mailman/listinfo/evolution-hackers


[Evolution-hackers] camel-folder-summary locking ...

2010-02-26 Thread Michael Meeks
Hi guys,

I just hit a nasty in camel-folder-summary; suggested patch attached,
seemingly a simple problem of a re-enterancy hazard in the same thread
with a ghashtable:

Thread 121 (Thread 0xa7f37b70 (LWP 23631)):
#0  0xb68c13ef in g_hash_table_resize (hash_table=value optimized out) at 
ghash.c:424
#1  g_hash_table_maybe_resize (hash_table=value optimized out) at ghash.c:457
#2  g_hash_table_remove_internal (hash_table=value optimized out) at 
ghash.c:982
#3  0xb7c7fdba in message_info_free (s=0x8e7e438, info=0x90f89f0) at 
camel-folder-summary.c:3497
#4  0xb7c7fe75 in camel_message_info_free (o=0x90f89f0) at 
camel-folder-summary.c:4518
#5  0xb7c7ffce in remove_item (key=0x9142618 217647, info=0x90f89f0, 
s=0x8e7e438) at camel-folder-summary.c:788
#6  0xb68c0b2c in g_hash_table_foreach_remove_or_steal (hash_table=0x8f7d0c0 = 
{...}, func=0xb7c7ff40 remove_item, user_data=0x8e7e438, notify=1)
at ghash.c:1109
#7  0xb7c85356 in remove_cache (session=0x80ce680, msg=0xb093250) at 
camel-folder-summary.c:818
#8  0xb7ca0162 in session_thread_proxy (msg=0xb093250, session=0x80ce680) at 
camel-session.c:590
#9  0xb68fb036 in g_thread_pool_thread_proxy (data=0x8aa0cb8) at 
gthreadpool.c:265
---Type return to continue, or q return to quit---
#10 0xb68f99ef in g_thread_create_proxy (data=0xb0c1ee8) at gthread.c:635
#11 0xb764c725 in start_thread () from /lib/libpthread.so.0
#12 0xb680626e in clone () from /lib/libc.so.6

Having said that, the code looks a bit nervous (judging by the
comments) in this area. Review much appreciated etc. Reading through the
code there though, I too began to get a bit nervous trying to keep track
of what locks are held.

eg.
camel_folder_summary_migrate_infos
+ doesn't seem to take the summary_lock - why ?

likewise 'message_info_free'
+ called from camel_message_info_free
+ doesn't seem to take the lock ... - why ?
  if it is just assumed then:

+ most callers of camel_message_info_free *have* that lock,
+ except eg. camel_folder_summary_remove ...
+ or eg. camel_folder_summary_remove_index_fast
+ or summary_header_save, or message_info_load,
+ or msg_update_preview, 
+ or camel_folder_summary_migrate_infos

Soo ... if you read the GType / GObject code - which is similarly
twisted / complicated - then it uses name mangling to denote what lock
is held (personally, I rather like that). Thus you can see by reading
it:

do_foo()
{
take_lock();
do_foo_T();
release_lock();
}

is safe, and

do_foo_T() { do_baa(); }

is essentially not :-) (the '_T' suffix needs propagating). ORBit2 does
the same where things get sticky.

Failing that, the linux kernel seems in places to use comments of
the form:

 * The files-file_lock should be held on entry, and will be held on exit.

Clearly, something to reduce the barrier to entry here such that the
code is more readable would be good ;-)

HTH,

Michael.

-- 
 michael.me...@novell.com  , Pseudo Engineer, itinerant idiot
diff --git a/camel/camel-folder-summary.c b/camel/camel-folder-summary.c
index 85265ac..787f229 100644
--- a/camel/camel-folder-summary.c
+++ b/camel/camel-folder-summary.c
@@ -778,18 +778,15 @@ cfs_count_dirty (CamelFolderSummary *s)
 
 /* FIXME[disk-summary] I should have a better LRU algorithm  */
 static gboolean
-remove_item (gchar *key, CamelMessageInfoBase *info, CamelFolderSummary *s)
+remove_item (gchar *key, CamelMessageInfoBase *info, GSList **to_free_list)
 {
 	d(printf(%d(%d)\t, info-refcount, info-dirty)); /* camel_message_info_dump (info); */
 	CAMEL_SUMMARY_LOCK(info-summary, ref_lock);
 	if (info-refcount == 1  !info-dirty  !(info-flags  CAMEL_MESSAGE_FOLDER_FLAGGED)) {
 		CAMEL_SUMMARY_UNLOCK(info-summary, ref_lock);
-		/* Hackit so that hashtable isn;t corrupted. */
-		/* FIXME: These uid strings are not yet freed. We should get this done soon. */
-		camel_pstring_free (info-uid);
-		info-uid = NULL;
-		/* Noone seems to need it. Why not free it then. */
-		camel_message_info_free (info);
+
+		/* free the entry later */
+		*to_free_list = g_slist_prepend (*to_free_list, info);
 		return TRUE;
 	}
 	CAMEL_SUMMARY_UNLOCK(info-summary, ref_lock);
@@ -807,6 +804,7 @@ remove_cache (CamelSession *session, CamelSessionThreadMsg *msg)
 	struct _folder_summary_free_msg *m = (struct _folder_summary_free_msg *)msg;
 	CamelFolderSummary *s = m-summary;
 	CamelException ex;
+	GSList *to_free, *l;
 
 	CAMEL_DB_RELEASE_SQLITE_MEMORY;
 	camel_exception_init (ex);
@@ -819,7 +817,14 @@ remove_cache (CamelSession *session, CamelSessionThreadMsg *msg)
 	dd(printf(removing cache for  %s %d %p\n, s-folder ? s-folder-full_name : s-summary_path, g_hash_table_size (s-loaded_infos), (gpointer) 

[Evolution-hackers] edbus branch review ...

2010-02-26 Thread Michael Meeks
Hi guys,

I have a new fun branch (to review for merging)

mmeeks-gdbus-import

So I am wickedly piling up other misc. fixes in there (so they don't
mask other issues) - but I have the edbus code imported, and running -
and, indeed, it seems to block rather less nastily than the dbus-glib
variants[1]: which is all good.

It is essentially the work from origin/treitter-client-gdbus but adding
the 'refresh' method, and of course porting it all to the internal copy
of 'libedbus.so'.

This should let us stick with glib-2.28 (what I'm using), and yet get
the benefits of moving closer to what gdbus will be in the end (I hope)
- as well as the interactivity wins. I incorporate the fairly simple
perl script I used to do the substitution in the first instance.

Why am I telling you this ? - I'd love some review of course; I've been
playing with calendar  addressbook here for a bit; but would greatly
appreciate some more input. Particularly since I didn't write the
original gdbus port.

I'd also -really- like to get this in soonish;

Thoughts / flameage ? :-)

Thanks,

Michael.

[1] - which seems to like blocking the mainloop any time any other
thread does a synchronous call (not optimal).
-- 
 michael.me...@novell.com  , Pseudo Engineer, itinerant idiot

This is a version of David Zeuthan's gdbus code, intended for inclusion in glib.

Until it is included, it helps us to have a copy to work with included in Evolution.

Of course, some regexps are required to rename g_dbus - e_dbus etc. to avoid potential symbol conflicts.

The code is taken from:

git://anongit.freedesktop.org/~david/gdbus-standalone commit 7457cdc8863f08163c48de455e8cd7469b0fe876

With a number of fixes layered on top.

Also included is a renamed copy of gvariant from glib commit 1433655e23634446b990d1cf782b22ad6430496a

perl script to in-place re-write code using g_dbus:

#!/usr/bin/perl -pi.bak

s/G_VARIANT/E_VARIANT/g;
s/G_TYPE_VARIANT/E_TYPE_VARIANT/g;
s/GVariant/EVariant/g;
s/g_variant/e_variant/g;

s/g_bit_/e_bit_/g;
s/g_bus_/e_bus_/g;

s/GDBus/EDBus/g;
s/G_DBUS_/E_DBUS_/g;
s/G_TYPE_DBUS/E_TYPE_DBUS/g;
s/g_dbus_/e_dbus_/g;
___
Evolution-hackers mailing list
Evolution-hackers@gnome.org
http://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] Moving from the single mbox file format for the local folders

2009-12-16 Thread Michael Meeks

On Wed, 2009-12-16 at 01:16 +0530, Chenthill wrote:
 One advantage which I see with #1 is that its a standard way.

One thing about both approaches, is that they will consume more space;
eg. on my 'Sent' folder with 21k messages - on average (on ext3) we will
chew ~2k of space for each of these; which is ~40Mb - around 2%.

For my cvs commits mail archive, perhaps the worst case, of 350Mb,
(22334 mails) - this would also be around 43Mb - at ~12%.

That's not as bad as I was worried about; though of course there is
some overhead in terms of inodes and directory entries to worry about
that will crank up the overall size - but it doesn't seem horrible even
on ext3.

Of course - ext4 / btrfs will do a much better job here too; so - less
to worry about in future.

HTH,

Michael.

-- 
 michael.me...@novell.com  , Pseudo Engineer, itinerant idiot


___
Evolution-hackers mailing list
Evolution-hackers@gnome.org
http://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] Camel Manifesto

2009-11-30 Thread Michael Meeks
Hi Matthew,

On Fri, 2009-11-27 at 11:58 -0500, Matthew Barnes wrote:
 In fact the mail-to-eds effort is part of what's motivating all this.

Ah ! - ok :-) sounds good.

 To make the discussion more concrete, these are my current plans for
 Camel's extreme makeover.  The final API will hopefully feel like a
 natural extension of GIO.

So - I'm well up for that :-) reducing the incidence of wheel
re-invention, reducing LOC, increasing familiarity and reducing the
skill-set required to hack on the code are all excellent work; I'm
really eager to see the good results.

 Evolution will then be free to call Camel's async methods from the main
 thread.  I'll still have to deal with queuing up operations in order on
 the Evolution side, but everything will be cancellable (and actually
 cancel when you tell it to, especially when the network is flaky) and
 we'll hopefully have far fewer issues with frozen UIs.
 
 Does that sound like a more down-to-earth approach?

Yes - very sensible; of course, I love step-wise re-factoring, so we
don't end up with these massive branches that take forever to merge; ie.
merging just the GObjectification, then merging the GIO-ification [ or
whatever ]: I guess that could conflict with keeping Camel API stable,
but is that even a useful goal ?

The only thing that slightly concerns me, is that I am not certain
(knowing ~nothing about it) that the Camel API matches well what the
Evolution mailer uses, and/or what works in terms of IPC. When I last
did the analysis of a good place to cut here, the line went through some
of the Evo mailer code, as well as camel.

If the idea is to use GObject introspection to provide a useful 
usable D-BUS interface, I rather suspect that the result will be an
over-granular, and round-trippy disaster :-) nevermind exposing
lifecycle counting over the wire (which is a total disaster) [ or if not
that, then the GObject API will be somewhat unpleasantly chunky to use
in-process (if that is a goal) ].

As such, presumably it might be interesting to use the mail / d-bus
interfaces to provide the initial bulk of the asynchronicity, to clean
up the Evolution UI process. Then, in future - if people want to use the
backend API itself in an async fashion from other pieces of code, that
can be done as needed (?)

It just seems to me that since this is D-BUS and we have to implement
these awful in-lined object adaptors, we can at least use the
DBusMessage encapsulation to throw all the parameters to a slow
operation into it's own thread, before unpacking all them all. That
would contrast with the method of implementing all slow D-BUS methods
using async calls that then spawn threads - which would require
unpacking all arguments, and then carefully re-packing them into various
async closures.

But I speculate, as always :-)

HTH,

Michael.

-- 
 michael.me...@novell.com  , Pseudo Engineer, itinerant idiot

___
Evolution-hackers mailing list
Evolution-hackers@gnome.org
http://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] Camel Manifesto

2009-11-26 Thread Michael Meeks
Hi Matthew,

On Fri, 2009-11-20 at 10:46 -0500, Matthew Barnes wrote:
 There may be isolated cases internally to Camel where it can exploit
 parallelism in CPU-intensive tasks with threading or where threads are
 necessary for interacting with synchronous-only libraries, but it should
 be used sparingly and hidden behind a fully asynchronous API.

So - I'm well up for hiding complexity behind an asynchronous API in
general; that's a great goal. I guess there is also the mail-to-e-d-s
red herring to consider in the mix - that (potentially) adds a layer of
asynchronicity to the equation in the form of remote dbus calls; perhaps
worth considering that in parallel - though it would be cut at a
different place (potentially).

   It should not be central to the design of the entire mail application,
 as it is currently.  Basically I want the mail front-end in Evolution 3
 to be single threaded, or as close to that as possible.

Sounds reasonable.

 The first is what I think is a very insightful paper on the inherent
 problems with threads:
 http://www.eecs.berkeley.edu/Pubs/TechRpts/2006/EECS-2006-1.pdf

Sure - I read it carefully many moons hence; and it's good - though
IMHO it over-states the case somewhat, or at least - it seems to me that
sometimes the alternative is worse.

The problem IMHO comes when there is a multi-step process; eg.

DN lookup
socket connect
N way ssh handshake 

none of which the application cares about - really; the 'async' way is
to whack this all as atomised pieces of code, into some state machine:

switch (state) {
case DNS_CONNECTING:
if (not failed) {
state = DNS_CONNECTED;
pollstate = IO_OUT|ERR;
}
case DNS_CONNECTED:
send (lookup_msg);
pollstate = IO_IN|ERR;
state = DNS_WAIT_REPLY;
case DNS_WAIT_REPLY:
read () ...
if (!short_read)
state = SOCKET_NEW
else
continue in this state
case SOCKET_NEW:
...
case SOCKET_CONNECTED:
pollstate = IO_OUT|ERR;
case SOCKET_CAN_WRITE:
...
pollstate = IO_IN|ERR;
case SOCKET_WAIT_RESULT:
...
}

etc. etc. etc. This is basically what ORBit2 / linc does - although, of
course we get lazy when eg. Windows demands more round-trips, and we
probably do DNS synchronously (that stuff never worked well anyway), and
so on. It is not particularly awful - though, some tricks such as
checking for 'IO_IN' before HUP etc. to avoid loosing the end of a
message are worth not forgetting ;-)

Of course - as the number of the steps in the handshake grows the scope
for error and confusion grow - nevermind the debugging problem: when it
locks up, what went wrong ? :-) how do you even see the state of the
umpteen state machines that are ticking away behind the scenes ?

Of course - some large 'state' structure is required - replicating an
equivalent thread's stack (but on the heap), and that has to be
lifecycle managed and so on in a similar way to threads I guess - with
some extra function overhead.

The threaded version with async callback API I guess has the same
initial closure creation overhead; but then the code is fairly easy to
read:

host_addr = do_blocking_lookup (name);
if (cancelled || !host_addr)
goto emit_error;
fd = socket();
connect (host_addr);
if (cancelled || !connect_error)
goto emit_error;
write_request (fd);
read_reply (fd);
emit_success_callback () etc.

it is also rather easy to debug - as soon as anything fails - with
'bug-buddy' or other conventional debugging tools it is easy to see who
was causing the blocking / dead-locking, or what synchronous calls were
not responsive.

It is also far easier and clearer to re-use code via blocking calls (I
suspect) - than to cobble other sets of states into your state machine.

And of course, none of this is news to anyone I'm sure. Clearly though
- the simpler the locking, and the closer to clean  simple message
passing - the easier and safer the threading becomes.

Anyhow - to me at least ( thankfully shielded from the pain that is
suffered by users of camel ) it seems like a large influx of re-writing
everything as async is unlikely to give substantial reliability wins
(beyond those intrinsic to having a great hacker re-read, and test the
code).

But - since I'm not doing it, I can only write long  silly mails to
try to persuade :-)

ATB,

Michael.

-- 
 michael.me...@novell.com  , Pseudo Engineer, itinerant idiot

___
Evolution-hackers 

Re: [Evolution-hackers] Camel Manifesto

2009-11-20 Thread Michael Meeks
Hi Matthew,

On Wed, 2009-11-18 at 14:07 -0500, Matthew Barnes wrote:
 With work on Bonobo removal wrapping up, I've finally started taking a
 closer look at Camel (Evolution's mail storage and networking library)

Ah - another life-time of cleaning up, and polishing code: the goal
sounds really nice.

 This is a distant future goal and will have to happen gradually, but I
 would like Camel to shift to a single-threaded design where all file and
 network operations directly use or are derived from GIO's asynchronous
 file and stream APIs.

Hmm; you really propose to remove all threading from camel's
implementation ? or just from it's API ? a full removal might be
problematic.

While clearly threading, if done in an un-constrained way, has it's own
peculiar problems - there are a couple of obvious advantages:

First debugging - while both async and threaded programming loose
determinism due to event re-ordering - the debugger at least understands
threads - and can hopefully show you your state in a reasonably
follow-able way - you can step through slow/blocking calls with 'next',
'finish' etc.; I worry about a world packed with highly granular
asynchronous state-machines, all chained together - with no good way to
see what is happening, what state everything is in, and (of course) why
the app is serenely inactive in it's mainloop suddenly ;-)

Secondly - of course, a fully async mode potentially would loose any
(possible) benefit of parallelism that in theory threads can provide -
in this world of dual-core laptop CPUs, and even hyper-threaded Atoms.

OTOH - I'm sure you know what you're doing ;-)

ATB,

Michael.

-- 
 michael.me...@novell.com  , Pseudo Engineer, itinerant idiot

___
Evolution-hackers mailing list
Evolution-hackers@gnome.org
http://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] Status of the DBus port, future plans

2009-01-21 Thread Michael Meeks

On Tue, 2009-01-13 at 17:36 +, Chenthill wrote:
 On Tue, 2009-01-13 at 14:50 +, Ross Burton wrote:
  At present it contains a minimal port of the addressbook part of
  eds-dbus to a fairly current (~1 week old) EDS tree.  This mostly works
  and after a little cleanup should be ready for more testing. 

Nice.

  So, the gist of this rambling message is this: to merge the DBus port in
  the current state I'd need to add a context argument to all of the
  methods in e-cal-backend.  This will break API and I'll obviously be
  fixing the backends which are shipped as part of EDS at the same time.
  The good news is that the e-cal-backend-sync helper class effectively
  shields users of that from the change, so this affects less backends
  than you'd expect.

Sounds rather good.

  Any comments?

As long as you don't break the addressbook API, or ABI I'm happy
enough ;-) Those are the only bits OO.o uses, and by ABI I mean the
behaviour of libebook-1.2.so.9 and it's name / version [ configure.in
LIBEBOOK_CURRENT etc. ].

HTH,

Michael.

-- 
 michael.me...@novell.com  , Pseudo Engineer, itinerant idiot


___
Evolution-hackers mailing list
Evolution-hackers@gnome.org
http://mail.gnome.org/mailman/listinfo/evolution-hackers


[Evolution-hackers] camel-folder-summary.c - 64bit-ness ...

2009-01-05 Thread Michael Meeks
Hi guys,

I was just trying to reproduce some migration performance tests with my
mbox and summary data rsync'd from a 32bit machine to a 64bit machine.

Surprisingly this appears to crash immediately. Looking at the
camel-file-utils.c code I was surprised to see simultaneously an
apparent concern for network byte ordering:

camel_file_util_encode_fixed_int32 (FILE *out, gint32 value)
{
guint32 save;

save = g_htonl (value);
if (fwrite (save, sizeof (save), 1, out) != 1)
return -1;
return 0;
}

and also things like:

CFU_ENCODE_T(time_t)

that appear to generate data based on the sizeof the platform's time_t
- on my 64bit machine time_t is 8 bytes, on 32bit it is only 4.

Presumably this summary code is made obsolete by the new SQLite summary
code - and modulo some data as to what architecture a file was written
by it's perhaps less than obvious how to fix this. Also - why we're not
using fgetc_unlocked in these tight loops I don't know.

I guess I need an old evo. version to re-build all my summaries for
64bit now; or am I barking up the wrong tree ?

Thanks,

Michael.

-- 
 michael.me...@novell.com  , Pseudo Engineer, itinerant idiot

___
Evolution-hackers mailing list
Evolution-hackers@gnome.org
http://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] [Tracker] [Evolution] Beagle and Tracker, letting Evolution feed those beasts RDF triples instead

2008-12-15 Thread Michael Meeks
Hi Mikkel,

On Sat, 2008-12-13 at 00:18 +0100, Mikkel Kamstrup Erlandsen wrote:

 Is it that big a problem? I mean if you store 100,000 uris of avg.
 length 50 chars you will have a file about 5mb... One needs only keep
 an absolute minimum amount of metadata around. 

Well, true - it's not that much data I suppose - but without an agreed
lifecycle mechanism it grows without bound as you delete mail, and since
we have to do a lookup  prune for every new UUID we generate [ or
arguably emit ], there are potential size  performance issues prolly
worth addressing (in some simple way) in the design.

HTH,

Michael.

-- 
 michael.me...@novell.com  , Pseudo Engineer, itinerant idiot


___
Evolution-hackers mailing list
Evolution-hackers@gnome.org
http://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] [Evolution] Beagle and Tracker, letting Evolution feed those beasts RDF triples instead

2008-12-10 Thread Michael Meeks
Hi Philip,

On Wed, 2008-12-10 at 12:49 +0100, Philip Van Hoof wrote:
  What does the lifecycle for the data in that Unset store look like ?

 I think the LifeCycle is best described by this document:
 
 http://live.gnome.org/MetadataOnRemovableDevices

 It specifies a metadata cache format for removable devices in Turtle
 format. 

Not read that before; I just read it - and, as you say here is how
things are removed:

 For your information when reading the document: The removal of a
 resource as a special notation using blank resources  , and the
 removal of a predicate (of a field of a resource) uses the notation
 pfx:predicate .

Sure - so, that is fine - it's a representational detail of how
removals are stored. My concern is not that we can't represent removals
well - but that the life-cycle of that removal information is undefined.

Say eg. we install beagle, and tracker - but we never run beagle. Then
we have two parties that have registered an interest in changes. If we
run beagle only every year or so - we need to know all mails that were
deleted since a year ago. Unfortunately, perhaps we never run it again.
Does that mean we endlessly accumulate in some monster journal a huge
list of 'UnSets' ?

   For a cache it's important to know the modified timestamp so that
   you know whether your copy of the metadata is most recent, or the
   cache is about the resource is most recent.

Sure - I buy the timestamp thing; that's all great.

 - When a resource got deleted then the RDF store wants to know about
   this as soon as possible. Asynchronously (like if the RDF store,
   being a subscriber, joins the subscription after the deletion took
   place) this also counts: as soon as possible. Preferably immediately
   after the subscription.

Sure - so my problem is the life-cycle of the store of deletion
information: how long do we grow that list for, if people eg. turn off
the search client after finding it chews more resource than they had
hoped on their small machine :-)

   With IMAP there's a trick that you can do: you can assume that a hole
   in the UIDSET meant that some sort of deleting occurred.

Sounds interesting.

 I think, anyway, that it would make sense for Evolution to start doing
 two things in the CamelDB:

Agreed.

   * Log all deletions (just the UID should suffice), if the service
 reuses UIDs then upon effective reuse of the UID, this log's UID
 deletion should be removed from the log. Else you loose the E-mail
 at whoever depends on this log for knowing about effective
 deletions.

So there is at least some bound to the growth of the deleted UUID
log ;-) which is the size / likelyhood of re-use in the UUID space.

It's hard to think of solutions that are that satisfying; but - perhaps
something like cropping the deletion log-size at a percentage of stored
mail size, with some log overflow type message to flag that; or having
some arbitrary size bound on it, or more carefully disabling logging
when search services are disabled, or ... having only a single client,
or warning the user that they should run their search service some more,
or perhaps even coupling the indexing piece more closely to the mailer
itself somehow.

HTH,

Michael.

-- 
 [EMAIL PROTECTED]  , Pseudo Engineer, itinerant idiot

___
Evolution-hackers mailing list
Evolution-hackers@gnome.org
http://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] Getting rid of shipped libdb

2008-10-07 Thread Michael Meeks

On Mon, 2008-10-06 at 18:35 +0100, Rob Bradford wrote:
 Okay. Have you got these details? It would be good to see which of those
 still apply, etc..

Sure - the original rational here (AFAIR) is quite simple.

If you share the same .evolution across multiple machines, and the
version of libstupid is different: bad luck, you loose your contacts.

Basically all database-y things seem to love back-compatibility ( if
you're lucky ), but the idea of forward compatibility seems to
completely bypass them; the concept that the functionality is good
enough already, and doesn't require further file-format breakage is
apparently not present ;-

AFAIR the addressbook usages of libdb for the local addressbook were
annoying enough in previous instances that we moved to having an
internal version.

What I'd love to see instead, is a one-shot migration to a simple plain
text, authoritative file with the contacts and then (perhaps) optionally
a binary cache I guess. But for the volume of data there, presumably
slurping it and grokking it with some small / simple piece of code would
be rather more efficient. Ultimately contact searches are in response to
user-input, so we have loads of time to do work there.

HTH,

Michael.

-- 
 [EMAIL PROTECTED]  , Pseudo Engineer, itinerant idiot


___
Evolution-hackers mailing list
Evolution-hackers@gnome.org
http://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] Getting rid of shipped libdb

2008-10-07 Thread Michael Meeks
Hi Rob,

On Tue, 2008-10-07 at 16:49 +0100, Rob Bradford wrote:
 Hhh. But. The use case you outlined directly above about where this goes
 wrong also applies here: Oh. You ran e-d-s on a machine with a version
 that migrates it to Some Other Format (tm). You then add some contacts
 which go into the new format. Then you go back to your old machine.
 *Boing*. Same problem, your new contacts are missing :-( 

Yes true - but at least, this is a once-and-for-all fix ;-) and of
course there is no reason to say that we can't sync them to the old db
format too for a while.

 I would expect migrating from one version of GNOME and then back again
 is probably pretty problematic anyway...ultimately I think you need to
 draw the line at some point.

True - but having a problem at -every- version point, and across ~every
distribution such that I used SUSE and now I can't switch back to
FooBaked Linux ! is not good ;-)

 Somewhat orthogonally: Also, I wonder on the performance impact of the
 flat-file approach wrt, modification / deletion when dealing with ~1k
 contacts.

Fast enough I guess; I have ~3k real-life contacts, and that is ~1.5Mb
of addressbook.db [ which seems pretty much a flat vcard file when you
read it ;-].

It seems my pmap is struggling to work ATM, but it'd be interesting to
know how much of addressbook.db is 'hot' in e-d-s anyway.

HTH,

Michael.

-- 
 [EMAIL PROTECTED]  , Pseudo Engineer, itinerant idiot


___
Evolution-hackers mailing list
Evolution-hackers@gnome.org
http://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] [opensuse-gnome] Evolution 2.22 for Factory

2008-09-23 Thread Michael Meeks

On Sat, 2008-09-20 at 00:47 -0500, Hans Petter Jansson wrote:
 Wouldn't it be possible to use a different directory, e.g.
 mail/local-index/folders.db? That would avoid both problems.

That seems like a great idea to me; at least - if the 'hot' data set of
evolution is normally just the summary information, then keeping that in
a single directory [ ie. mangle the path names into file names ] would
make it rather more likely to be contiguous on disk too which might be
nice:

~/.evolution/mail/summaries/local-Inbox-suse-kernel.db # etc. ;-)

 I still think relocating folders.db is a better option, since it would
 work for everyone without necessitating an upgrade.

Sounds best to me, if we can do it that is.

Regards,

Michael.

-- 
 [EMAIL PROTECTED]  , Pseudo Engineer, itinerant idiot


___
Evolution-hackers mailing list
Evolution-hackers@gnome.org
http://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] Evolution: Taking forward...

2008-07-14 Thread Michael Meeks

On Sat, 2008-07-12 at 13:56 +0300, Felipe Contreras wrote:
 Could you consider talking with the samba guys to make libmapi LGPLv3?
 That would make things much easier, right?

Yep - would make life easier; OTOH I've spoken to them in the past
along these lines - but though they were initially interested some years
ago, sadly I think they are less interested today.

OTOH - it's always worth asking again I guess.

HTH,

Michael.

-- 
 [EMAIL PROTECTED]  , Pseudo Engineer, itinerant idiot

___
Evolution-hackers mailing list
Evolution-hackers@gnome.org
http://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] Evolution: Taking forward...

2008-07-11 Thread Michael Meeks
Hi Paul,

On Fri, 2008-07-11 at 15:30 +0200, Paul Bolle wrote:
  * Move Evolution licensing to  LGPL v2 and LGPL v3 to let us re-use
  the code more easily around the platform.
 
 Did you mean LGPLv2 _or_ LGPLv3 here?

Yes; it's dual licensed - which gives people rather a choice of
licenses, GPLv2, GPLv3, LGPLv2, LGPLv3 etc.

 Anyway, I am not familiar with libmapi, but it does indeed seem to be
 (what is commonly referred to as) a library that is licensed under the
 GPLv3. According to the FSF you can't release a project under the LGPLv2
 or LGPLv3 if you want to use a library under GPLv3. You'll have to
 convert your LGPLv2 or LGPLv3 code to the GPLv3 [1].

Sure - so, IANAL etc. however I don't see a problem with a LGPLv3 /
GPLv3 for the samba piece - I imagine, linking that in as a plugin makes
Evo, with the current structure as a whole GPLv3; but without it
potentially LGPLv2/GPLv2/LGPLv3 :-) AFAICS that gives some flexibility;
but of course it's possible there are yet further un-anticipated
problems ? :-) In the longer term, I'd prefer to see the samba piece
isolated into a separate process.

HTH,

Michael.

-- 
 [EMAIL PROTECTED]  , Pseudo Engineer, itinerant idiot


___
Evolution-hackers mailing list
Evolution-hackers@gnome.org
http://mail.gnome.org/mailman/listinfo/evolution-hackers


[Evolution-hackers] massive e-d-s memory leak persuit ...

2008-01-24 Thread Michael Meeks
Hi dudie,

So - I started to look at the e-d-s memory explosion situation quickly,
took a nice dump from gdb, ran strings on it and the heap has a ton of
strings around the place (as you would expect) - [ currently running at
only ~60Mb

strings /tmp/eds-heap | sort | uniq -c | sort -n 

gives me:

   1666 -CONTACT-UID
   1666 -NAME
   1736 ION-DEST-NAME
   1894 OLUTION-BOOK-URI
   2100 -EMAIL
   2184 ION-DEST-EMAIL
   2318 OLUTION-FILE-AS
   2506 OLUTION-LIST
   2992 ION-LIST
   3058 comp
   3321 OLUTION-DEST-EMAIL
   3329 OLUTION-DEST-CONTACT-UID
   3993 OLUTION-DEST-NAME
   4534 pwise://[EMAIL PROTECTED]/;Novell GroupWise Address Book
   5343 BEGIN:VCARD
   5372 ION-DEST-EMAIL
   5504 END:VCARD
   5505 VERSION:3.0
   6786 ION-DEST-NAME
   8606 para
  12739 ION-DEST-CONTACT-UID
  13642 OLUTION-DEST-CONTACT-UID
  18082 OLUTION-DEST-NAME
  19252 OLUTION-DEST-EMAIL
  21991 prop
  32508 ATTENDEE;CUTYPE=INDIVIDUAL;ROLE=REQ-PARTICIPANT;PARTSTAT=NEEDS-ACTION;
  40253 pA,


Where the first column is the count ... 32508 copies of that ATTENDEE
string seems a little excessive, as do the (apparently mangled?)
OLUTION-DEST-... strings.

Does that provide any insight wrt. code to audit for this huge leak ?
apparently it afflicts everything from SLED10-SP1 onwards. Also - in
general to reduce the (high) e-d-s memory usage, should we be using
GQuarks for some of these field names as we store them ?

Thanks,

Michael.

-- 
 [EMAIL PROTECTED]  , Pseudo Engineer, itinerant idiot


___
Evolution-hackers mailing list
Evolution-hackers@gnome.org
http://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] Synching Evolution/GNOME version

2007-10-04 Thread Michael Meeks
Hi Matthew,

On Thu, 2007-10-04 at 10:00 +0530, Srinivasa Ragavan wrote:
  And while we're at it, can we please drop the meaningless -1.2 suffix
  from the library names (e.g. libedataserver-1.2.so)?  As far as I can
  tell this is just an artifact from an age before the EDS sonames were
  properly versioned.

Cue, screaming ... please read:

http://lxr.go-oo.org/source/dba/connectivity/source/drivers/evoab2/EApi.cxx#046

and reconsider ;-) be aware that it takes weeks to months to get that
change up-stream, and months to get a new version of OO.o out, and all
the while OO.o will ~silently fail to work with people's new 'clean'
renamed library e-d-s :-)

AFAIR the name versioning was originally intended to ensure you could
compile  develop multiple versions of evo. on the same system.

 I don't think those application would be happy to do this. There are a
 few apps that use the .so directly without pkgconfig (iirc
 OpenOffice.org). 

Quite - we build our OO.o integration (which since it is up-stream has
to run on ~all existing systems - cf. the ISV problem ...), with
internal headers (to unwind the ABI breakage), and explicit dlopening 
hooking out of symbols etc.

 Unless there is a clear nod from the stake holders of those projects, I
 wouldn't be favor of doing this.

Thanks ! :-)

Of course, as long as the dlopens still work, I'm fine with renaming
the core library (to suit people's personal hygiene issues) as long as
we have a compat symlink in perpetuity ;-)

Thanks,

Michael.

-- 
 [EMAIL PROTECTED]  , Pseudo Engineer, itinerant idiot


___
Evolution-hackers mailing list
Evolution-hackers@gnome.org
http://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] More e-d-s ABI breakage ?

2006-08-11 Thread Michael Meeks
Hi Harish,

First - thanks for digging these changes out for me. But - no, I'm not
just interested in ebook (though for OO.o that is all), but I'm
-primarily- interested Evo. itself, in being able to use and test the
most recent version to help avoid regressions, and indeed ship it for
older platforms.

So - if you could do the same for the other e-d-s libraries, it'd be
great to see what changed there too.

On Fri, 2006-08-11 at 20:03 +0530, Harish Krishnaswamy wrote:
 The changes in question are as follows :
 http://cvs.gnome.org/viewcvs/evolution-data-server/addressbook/libebook/e-contact.h?r1=1.20r2=1.21

So - this changed the EContactPhoto structure - why ? surely that is
rather pointless. You could easily have added an EContactMimePhoto type
and added a synthetic back-compat field that would handle the old case
[ if it was of (whatever) mime type you expected ]. So - I see no
problem at all doing this compatibly whatsoever. Perhaps a few more
(~20) lines of code tops.

 http://cvs.gnome.org/viewcvs/evolution-data-server/addressbook/libebook/e-contact.h?r1=1.21r2=1.22

You converted a gpointer value to a 'const gpointer value' - I don't
see that that is particularly necessary, or likely to break the ABI of
anything unless it reflects some underlying lifecycle issue. Also the
enum insertions were (this time) added at the end of the enumeration, so
why should that break anything ? surely that's a compatible extension.

 This was not a trigger-happy change. 

I beg to differ.

 The patch for the first bug was submitted on 15 Aug 2005 (almost a year
 ago) and that for the second on 3 Mar 2004 ( more than 2 years ago) and
 had been in the back-burner only because we had wanted to preserve the
 ABI.

You realise you can *extend* the API/ABI without breaking the ABI
right ?

 The #313533 patch was vital for Ross Burton's dbus-based EDS and running
 EDS on devices (say Nokia 770) would not be possible w/o this change.

Nonsense - at least the link above has no API change that is necessary
for dbus or Nokia 770 support - unless I'm missing something huge; good
buzz-words though :-)

 The #259536 patch had been blocking a large number of users in Poland
 where Gadu Gadu is the defacto IM standard.

And doesn't break ABI anyway (AFAICS).

 IMO, a single break after two major releases against such gains is
 justifiable. These patches were reviewed and approved by the addressbook
 hackers before the bump. 

I'm sure they were reviewed ;-) I'm not debating that.

 Could these improvements have been absorbed w/o breaking the ABI and
 with not too much additional costs in the process?

Yes - clearly so.

 I do not know - nor did the patch authors or the reviewers. But if you
 could show us, I am all willing to learn.

Hokay - so - to get the new functionality you require changes on the
client side anyway, so we can ignore that as an issue. e-contact.c
*already* supports 'synthetic' properties [ that duplicate state in
other properties ]. We should have simply added a Photo2 structure [ or
some renamed thing ], changed the name of the property to reflect that,
and updated all callers that cared. [ NB. we had to update all callers
anyway, but with your change, since they using a type-unsafe gobject
property, unchanged callers will SEGV when they call this method ].
While this was being fixed we should have used a GObject for
EContactMimePhoto ( or whatever ) so we could compatibly extend it in
future.

 Can you also help me understand the severity of the break in OO.o ?

OO.o is one symptom of the huge system-wide problem that you cause for
every application developer foolhardy enough to link against
evolution-data-server. Every user, now who wants to eg. upgrade 'Ekiga'
to the latest version [ across this breakage ] will discover that going
from 1.1.x to 1.2.y he will have to upgrade some huge chunk of his
system - or simply give-up and not test/use the latest version. That is
a horrible idea for anyone - particularly when it's totally unnecessary.

 Why are the library files with SONAMEs hard-coded in a source file ?

Some applications exist that have to run on whatever is there ? - this
is called being an ISV :-) - Sun is an ISV for Linux, this means a world
of pain [ created by exactly this kind of breakage ]. ie. if OO.o is
going to run on NLD9, and SL10.0, and SL10.1, [ and now SL 10.2 ] how do
you recommend we integrate with evolution-data-server any other way ? -
we -have- to dlopen libraries since they may not be there, and we then
have to try to adapt to whatever breakage they introduced across
versions.

 Why would you not use (say) pkg-config instead ?

This is not what pkg-config files are for, and anyway:

$ rpm -qf /opt/gnome/lib/pkgconfig/libebook-1.2.pc 
evolution-data-server-devel-1.6.0-43.20

I'd love to see this change undone ASAP, before code-freeze for Gnome
2.16 is upon us; 

Re: [Evolution-hackers] More e-d-s ABI breakage ?

2006-08-11 Thread Michael Meeks
Hi Ross,

On Fri, 2006-08-11 at 17:32 +0100, Ross Burton wrote:
 The change was required for the 770 as ferrying the photos over the bus

Sure - I like the core change, but not the API/ABI detail :-)

 If it's required it might be possible to revert that change and
 introduce an alternative.  As you say, a photo2 property that returns
 a new type should be sufficient, I'll have a look later.

Sure, although of course it may be more complex than that; this was a
1st glance review.

 I have Grand Plans to write a replacement for EVCard and EContact
 though.  I've been adding new API to e-vcard to make it more usable, but
 that is just complicating the API for no gain.  Extending EContact is a
 pain as all of the data types are public structs with no padding.

Sounds lovely. From an OO.o perspective we want a thread-safe 'cursor /
iterator' API ;-) as in:

for (line = do_query_get_1st; line; line = line-getnext()) {
...
}

but I guess we handle all our async stuff via threads.

 EContact also manages to hit that sweet spot between
 complicated-but-powerful and easy-but-limited, so it's
 complicated-and-limited at once.

Heh ;-) I'm sure there are (no doubt) tons of problems with the API -
however, a drip, drip of incompatible breakage is no solution IMHO.
Clearly building your nice new API in parallel, deprecating the old API,
and [ in a few versions time ] doing a big bang switch to the new (by
now perfect, tested, GObject based, extensible etc. ;-) API seems a
better approach. Hopefully we can write an 'evoab3' connector to it
then.

All the best,

Michael.

-- 
 [EMAIL PROTECTED]  , Pseudo Engineer, itinerant idiot


___
Evolution-hackers mailing list
Evolution-hackers@gnome.org
http://mail.gnome.org/mailman/listinfo/evolution-hackers


Re: [Evolution-hackers] Something screwy when Evolution Shell invokes bonobo_activation_active_server_unregister()

2006-06-19 Thread Michael Meeks

On Mon, 2006-06-19 at 00:18 +0300, Tor Lillqvist wrote:
 I think I have a good guess now: The problem is that all sockets by
 default are inherited by child processes in Windows. (Like file
 descriptors in Unix.)

Ah - we had some wonderful b-a-s bugs for the few sockets we didn't
CLOEXEC in the past. Of course, we still allowed b-a-s clients to
inherit stdout/in/err (at least, until the g_spawn rework (?)) - but
that is sometimes useful.

HTH,

Michael.

-- 
 [EMAIL PROTECTED]  , Pseudo Engineer, itinerant idiot


___
Evolution-hackers mailing list
Evolution-hackers@gnome.org
http://mail.gnome.org/mailman/listinfo/evolution-hackers


[Evolution-hackers] Gender ...

2006-02-15 Thread michael meeks
So,

At the risk of offending the gender-confused; it would be -extremely-
useful to have a Gender boolean in the evolution addresbook [ of course,
perhaps there is  I'm just missing it but I dug into the code ].

The reason is OO.o will vary it's salutation on gender; ie.

Dear Mrs. Foo
Dear Mr. Foo

etc. - now one can argue whether this is broken etc. but there it
is ;-) the current ergonomics are rather built around this - and, you
can see that such a boolean internationalizes rather nicely.

So - the question is: can we have it ? currently the OO.o mail merge is
rather feeble without it.

Thanks,

Michael.

-- 
 [EMAIL PROTECTED]  , Pseudo Engineer, itinerant idiot

___
Evolution-hackers mailing list
Evolution-hackers@gnome.org
http://mail.gnome.org/mailman/listinfo/evolution-hackers


[Evolution-hackers] e-d-s pokeage ...

2006-02-14 Thread michael meeks
Hi there,

So - I was pleased to see that the libebook .so remains unchanged from
evo 2.4 to 2.6 - and I just did a little review to try to ensure that
this indeed reflects an unchanged ABI ;-)

It seems that is the case - which is great thanks - I need only to
update a comment in OO.o ;-) however, as with all reviews I happened on
some other things:

libebook:
+ e-name-western.h: urgh not good practice
+ do we really want to export this structure ?
  surely an opaque type  accessors is ~always 
  better ?

+ e-name-western-tables.h:
+ looks broken to me:
* if you include this header it will instantiate
  this large set of strings in each module / 
  shlib.
* all such lists of strings should prolly be
  defined 'const' so we put them in a shared
  linker section, to save memory.
* I imagine the header should have
  'extern const char *' and just the symbol names
+ why is this installed anyway ? are there 
  better ways to expose this ?

libedataserver/e-data-server-module.c
+ you add:

-   module-library = g_module_open (module-path, 0);
+   module-library = g_module_open (module-path, G_MODULE_BIND_LAZY);

in moving to 2.6 - which is some optimization; of course - it would be
rather better to use:

(G_MODULE_BIND_LAZY | G_MODULE_BIND_LOCAL)

instead. What that does is ensure that the plugin is not included in
the search path for symbols for other plugins - ie. subsequent plugins
as they link don't have to search this plugin for symbols. Assuming your
plugins don't depend on each other to provide symbols [ pretty broken
IMHO ] this not only accelerates linking, but prolly helps avoid various
potential tangled linking problems.

HTH  thanks for not breaking the ABI again :-)

Regards,

Michael.

-- 
 [EMAIL PROTECTED]  , Pseudo Engineer, itinerant idiot

___
Evolution-hackers mailing list
Evolution-hackers@gnome.org
http://mail.gnome.org/mailman/listinfo/evolution-hackers