[Evolution-hackers] ESourceGroup from ESource

[Scott Herscher]

How does one derive an ESourceGroup from an ESource?  If I call 
e_source_peek_group() with a valid ESource instance, and then immediately call 
E_IS_SOURCE_GROUP(), I am getting false.

Thank you in advance for any assistance.

Scott
___
Evolution-hackers mailing list
Evolution-hackers@gnome.org
http://mail.gnome.org/mailman/listinfo/evolution-hackers

Re: [Evolution-hackers] mmap patch

[Philip Van Hoof]

I'm just going to play open cards in my reply. No more contests, no more
"look how good I am". Just my true intentions, on a plate. I promised
Harish I was going to change that attitude. So I will.

I will also technically comment where necessary.

On Wed, 2006-07-19 at 16:29 -0400, Jeffrey Stedfast wrote:

> Well, I figured out why you're getting alignment problems... you're
> using int* pointers to decode stuff. You need to use a char*

- technical -

There's no data alignment problems in the latest patch. 

http://pvanhoof.be/files/evolution_data_server__mmap_summary.diff


> Secondly, your code is just really gross:
> 
> - Why do you have a filepos pointer? it's completely unnecessary.

- technical -

Because it's a hack. Initially I didn't want to change the internal API,
so I had no other options but to put it in the CamelFolderSummary
struct.

After federico's three points I , however, changed the internal API by
removing the FILE* from the methods that aren't using it anymore. A
better way would have been to pass the mmap addr position here, in stead
of the FILE*.

Note that the current patch isn't intended for HEAD. It is (for me)
mainly intended for testing purposes. 

If upstream is really interested, they will probably change the summary
file-format drastically anyway. You where the first to say that for mmap
it would be better to change that file format. I was the first to
acknowledge that. I guess we are in agreement here.


- open cards -

In fact, Jeff, if you want to know why I really really started the work:

I did it because I wanted to get things moving in Camel-land. Because
people, like maybe you, would be awaken and/or become passionate about
wanting to do the disk-summary branch so badly ... that they will just
start making it really happen.

It might surprise you, but in a way I was also happy that you said you
questioned this patch by mentioning the disk-summary branch. I hope I
inspired you to go for it.

In another way I was of course a little bit pissed. But it's passion
that moves things. Anger that makes passion. You fueled me to implement
the three points federico asked for. Without your fuel, I wouldn't have
done that.

I really believe in the concepts you and NotZed outlined with the disk
summary idea. Even a little bit more than I believe in my own mmap
ideas. The combination however ... a combination would be awesome.

I really think that a "just go for it"-team can make this happen. I
really believe that you, Jeffrey, could steer such a team.

I might often present myself as an asshole. My intention is to get
things moving. I want us to make things better. A lot better. Just
because we can. For me, that's definitely a reason why we should.

Evolution doesn't have to bleed to death (which is btw what a lot people
predict to happen). There's indeed a lot good stuff in it. Good stuff
that you guys made happen. I acknowledge that, if that is what you
wanted to hear from me. I too can read code myself and figure out who
the authors are. I too respect their work. I also respect the new guys.

> - Why do you use a ptr32 thing? That's gross. There are far cleaner ways
> of doing this... it's also the source of your alignment problems.

- technical -

All the "ptr32" variables are removed in the latest patch, they've been
replaced with the get_unaligned_u32 macro. Are you sure you checked the
latest patch?

If I grep for "ptr32" in the latest patch, there's no occurrences.

> - Why do you have ::needs_free and ::uid_needs_free? Please find another
> way to do this. This is a horrible hack.
> 
> Attached you'll find the beginnings of a much cleaner implementation of
> the mmap approach.

- open cards -

That's really great, you cleaning this up means you are considering the
patch. That is my intention.

It's good to know that you are at least looking at it.


- technical -

What matters for me (and also for tinymail) is that the data isn't
allocated by malloc if it doesn't need to be allocated. If it can be
read from by mmap, it's better for mobile devices to read it that way.

>From the beginning I was very uncertain that the patch would be fit for
daily Evolution usage. I turned out to just work.

A reason for the ugly pieces of that patch, is that uncertainty. 

- open cards -

Initially this was, indeed, a tryout. An experiment. A hack. I openly
admit that I had no idea if it would work with Evolution.

I was concerned people would look at me as somebody who's only intention
was to fork Camel for tinymail. Therefore I added four days of hacking
to the patch to make it work on Evolution.

I didn't have to do that, because after the second day .. it worked with
tinymail already. You can follow the dates in my blog. If I wanted to be
that forking asshole, I would have done a camel-lite (oh wait, I did).

Anyway .. the point that I'm trying to make is: I wouldn't have tried to
make it work with Evolution. Nobody would have cared a lot, and I
wouldn't have to write this open-card

[Evolution-hackers] mmap patch

[Jeffrey Stedfast]

Well, I figured out why you're getting alignment problems... you're
using int* pointers to decode stuff. You need to use a char*

Secondly, your code is just really gross:

- Why do you have a filepos pointer? it's completely unnecessary.

- Why do you use a ptr32 thing? That's gross. There are far cleaner ways
of doing this... it's also the source of your alignment problems.

- Why do you have ::needs_free and ::uid_needs_free? Please find another
way to do this. This is a horrible hack.

Attached you'll find the beginnings of a much cleaner implementation of
the mmap approach.



-- 
Jeffrey Stedfast
Evolution Hacker - Novell, Inc.
[EMAIL PROTECTED]  - www.novell.com
? camel-gpg-context.c.gpg
? camel-mime-tables.c
? camel_folder_summary_with_mmap_fixes11.txt
? const-charset-map.patch
? mmap.patch
? pgp-filter.patch
? pstring.patch
? providers/imap4/272058.patch
? providers/imap4/imap4-sockopt.patch
? providers/local/largefile-mbox.patch
? providers/pop3/pop3-folder.c
Index: camel-file-utils.c
===
RCS file: /cvs/gnome/evolution-data-server/camel/camel-file-utils.c,v
retrieving revision 1.17
diff -u -r1.17 camel-file-utils.c
--- camel-file-utils.c	2 Jun 2006 00:52:29 -	1.17
+++ camel-file-utils.c	19 Jul 2006 20:32:13 -
@@ -262,18 +262,18 @@
 int
 camel_file_util_encode_string (FILE *out, const char *str)
 {
-	register int len;
-
-	if (str == NULL)
-		return camel_file_util_encode_uint32 (out, 1);
+	size_t len;
 	
-	if ((len = strlen (str)) > 65536)
-		len = 65536;
+	if (str == NULL)
+		return camel_file_util_encode_size_t (out, 0);
 	
-	if (camel_file_util_encode_uint32 (out, len+1) == -1)
+	len = strlen (str);
+	if (camel_file_util_encode_size_t (out, len + 1) == -1)
 		return -1;
-	if (len == 0 || fwrite (str, len, 1, out) == 1)
+	
+	if (fwrite (str, len + 1, 1, out) == 1)
 		return 0;
+	
 	return -1;
 }
 
@@ -290,29 +290,317 @@
 int
 camel_file_util_decode_string (FILE *in, char **str)
 {
-	guint32 len;
-	register char *ret;
-
-	if (camel_file_util_decode_uint32 (in, &len) == -1) {
+	size_t len;
+	
+	if (camel_file_util_decode_size_t (in, &len) == -1) {
 		*str = NULL;
 		return -1;
 	}
-
-	len--;
-	if (len > 65536) {
+	
+	if (len == 0) {
+		*str = NULL;
+		return 0;
+	}
+	
+	if (!(*str = g_try_malloc (len))) {
 		*str = NULL;
 		return -1;
 	}
-
-	ret = g_malloc (len+1);
-	if (len > 0 && fread (ret, len, 1, in) != 1) {
-		g_free (ret);
+	
+	if (fread (*str, len, 1, in) != 1) {
+		g_free (*str);
 		*str = NULL;
 		return -1;
 	}
+	
+	return 0;
+}
+
+
+/**
+ * camel_mmap_util_encode_uint32:
+ * @out: file to output to
+ * @value: value to output
+ * 
+ * Utility function to save an uint32 to a file.
+ * 
+ * Return value: 0 on success, -1 on error.
+ **/
+int
+camel_mmap_util_encode_uint32 (char **out, guint32 value)
+{
+	char *outptr = *out;
+	unsigned char c;
+	int i;
+	
+	for (i = 28; i > 0; i -= 7) {
+		if (value >= (1 << i)) {
+			c = (value >> i) & 0x7f;
+			*outptr++ = c;
+		}
+	}
+	
+	c = value & 0x7f;
+	*outptr++ = c | 0x80;
+	
+	*out = outptr;
+	
+	return 0;
+}
+
+
+/**
+ * camel_mmap_util_decode_uint32:
+ * @in: file to read from
+ * @dest: pointer to a variable to store the value in
+ * 
+ * Retrieve an encoded uint32 from a file.
+ * 
+ * Return value: 0 on success, -1 on error.  @*dest will contain the
+ * decoded value.
+ **/
+int
+camel_mmap_util_decode_uint32 (const char **in, guint32 *dest)
+{
+	const char *inptr = *in;
+guint32 value = 0;
+	unsigned char v;
+	
+/* until we get the last byte, keep decoding 7 bits at a time */
+	while (((v = *inptr++) & 0x80) == 0) {
+		value |= v;
+		value <<= 7;
+	}
+	
+	*dest = value | (v & 0x7f);
+	*in = inptr;
+	
+return 0;
+}
+
+
+/**
+ * camel_mmap_util_encode_fixed_int32:
+ * @out: file to output to
+ * @value: value to output
+ * 
+ * Encode a gint32, performing no compression, but converting
+ * to network order.
+ * 
+ * Return value: 0 on success, -1 on error.
+ **/
+int
+camel_mmap_util_encode_fixed_int32 (char **out, gint32 value)
+{
+	gint32 save;
+	
+	save = GINT32_TO_LE (value);
+	memcpy (*out, &save, sizeof (gint32));
+	*out = *out + sizeof (gint32);
+	
+	return 0;
+}
+
+
+/**
+ * camel_mmap_util_decode_fixed_int32:
+ * @in: file to read from
+ * @dest: pointer to a variable to store the value in
+ * 
+ * Retrieve a gint32.
+ * 
+ * Return value: 0 on success, -1 on error.
+ **/
+int
+camel_mmap_util_decode_fixed_int32 (const char **in, gint32 *dest)
+{
+	const char *inptr = *in;
+	gint32 value = 0;
+	int i;
+	
+	for (i = 0; i < sizeof (guint32); i++, inptr++, value <<= 8)
+		value |= *inptr;
+	
+	*dest = GUINT32_FROM_LE (value);
+	*in = inptr;
+	
+	return 0;
+}
+
+#define MMAP_ENCODE_T(type)		\
+int	\
+camel_mmap_util_encode_##type(char **out, type value)			\
+{	\
+	type save;			\
+	\			
+	switch (sizeof (type)) {	\
+	case 4:\
+		save = GUINT32_TO_LE (value);\
+		break;			\
+	case 8:			

Re: [Evolution-hackers] Memory consumption and virtual machines

[Federico Mena Quintero]

On Wed, 2006-07-19 at 17:40 +0200, Philip Van Hoof wrote:

> > ... On the other hand, Philip, next time we meet in person I'll happily
> > buy you dinner :)
> 
> oh ... what about Boston? :)
> 
> I'll check with my daytime employer whether it's okay if I can visit the
> Summit. 

I don't know for sure whether I'll be at the Summit.  If not, next
GUADEC, maybe?

[Where do you live?  I'm in Boston this week and the next.]

  Federico

___
Evolution-hackers mailing list
Evolution-hackers@gnome.org
http://mail.gnome.org/mailman/listinfo/evolution-hackers

Re: [Evolution-hackers] Memory consumption and virtual machines

[Philip Van Hoof]

On Wed, 2006-07-19 at 09:53 -0500, Federico Mena Quintero wrote:
> On Wed, 2006-07-19 at 01:10 -0600, Veerapuram Varadhan wrote:
> > On Tue, 2006-07-18 at 23:05 +, Philip Van Hoof wrote:
> > > If Novell wants me to implement unit tests (or other tests) for this,
> > > I will ask for payment. 
> > 
> > I am afraid that you won't get paid as Camel already has a
> > neat-test-suite and can be used/extended, IMO. ;-)

Aaaah, damn :)

> ... On the other hand, Philip, next time we meet in person I'll happily
> buy you dinner :)

oh ... what about Boston? :)

I'll check with my daytime employer whether it's okay if I can visit the
Summit. 


-- 
Philip Van Hoof, software developer at x-tend 
home: me at pvanhoof dot be 
gnome: pvanhoof at gnome dot org 
work: vanhoof at x-tend dot be 
http://www.pvanhoof.be - http://www.x-tend.be

___
Evolution-hackers mailing list
Evolution-hackers@gnome.org
http://mail.gnome.org/mailman/listinfo/evolution-hackers

Re: [Evolution-hackers] libecal functions

[Teresa Thomas]

HiI have one last (hopefully) question about libecal. I want to add a VEVENT to an existing Evolution calendar. Hence, I created an iCalComponenent to be added and hope to use the e_cal_create_object(ECal* eCal,iCalComponent iCalcomp, char** uid, char** error) function.
Now, I understand that eCal is a calendar client. If I want to add my event to an Evolution calendar, what exactly should eCal contain? Is it some sort of an identity of a cal? If so, how can I obtain it.Thanks!
On 7/19/06, Teresa Thomas <[EMAIL PROTECTED]> wrote:
Thanks Mr Palanisamy!My Bad, I actually meant to ask about the function e_cal_component_new() in ECalComponent.h/c. Anyway, from what I understand it returns an  ECalComponent, which is an (easier to manipulate) wrapper around an iCalComponent.
On 7/19/06, chenthill palanisamy <
[EMAIL PROTECTED]> wrote:
On Tue, 2006-07-18 at 17:44 -0400, Teresa Thomas wrote:> Need some assistance in my quest to comprehend EDS.>> What exactly is the difference between e_cal_create_object() [in> ECalComponent] and e_cal_create_object() [in ECal ] ?
e_cal_create_object is used to create a appointment, task or a memoobject (ECalComponent) in the corresponding backend (file, http,groupwise etc).  There is no function named e_cal_create_object inECalComponent. There is a function e_cal_component_get_created which
gives the creation date of a ECalComponent.Please have a look at 
http://www.go-evolution.org/EDS_Architecture .- Chenthill.>



___
Evolution-hackers mailing list
Evolution-hackers@gnome.org
http://mail.gnome.org/mailman/listinfo/evolution-hackers

Re: [Evolution-hackers] libecal functions

[Teresa Thomas]

Thanks Mr Palanisamy!My Bad, I actually meant to ask about the function e_cal_component_new() in ECalComponent.h/c. Anyway, from what I understand it returns an  ECalComponent, which is an (easier to manipulate) wrapper around an iCalComponent.
On 7/19/06, chenthill palanisamy <[EMAIL PROTECTED]> wrote:
On Tue, 2006-07-18 at 17:44 -0400, Teresa Thomas wrote:> Need some assistance in my quest to comprehend EDS.>> What exactly is the difference between e_cal_create_object() [in> ECalComponent] and e_cal_create_object() [in ECal ] ?
e_cal_create_object is used to create a appointment, task or a memoobject (ECalComponent) in the corresponding backend (file, http,groupwise etc).  There is no function named e_cal_create_object inECalComponent. There is a function e_cal_component_get_created which
gives the creation date of a ECalComponent.Please have a look at http://www.go-evolution.org/EDS_Architecture .- Chenthill.>

___
Evolution-hackers mailing list
Evolution-hackers@gnome.org
http://mail.gnome.org/mailman/listinfo/evolution-hackers

Re: [Evolution-hackers] Memory consumption and virtual machines

[Federico Mena Quintero]

On Wed, 2006-07-19 at 01:10 -0600, Veerapuram Varadhan wrote:
> On Tue, 2006-07-18 at 23:05 +, Philip Van Hoof wrote:
> > If Novell wants me to implement unit tests (or other tests) for this,
> > I
> > will ask for payment. 
> 
> I am afraid that you won't get paid as Camel already has a
> neat-test-suite and can be used/extended, IMO. ;-)

... On the other hand, Philip, next time we meet in person I'll happily
buy you dinner :)

  Federico

___
Evolution-hackers mailing list
Evolution-hackers@gnome.org
http://mail.gnome.org/mailman/listinfo/evolution-hackers

Re: [Evolution-hackers] Memory consumption and virtual machines

[Philip Van Hoof]

I'm going to attempt to conclude this mini-thread that got extended to
other mailing lists.

On Wed, 2006-07-19 at 02:45 -0600, Veerapuram Varadhan wrote:

> I have created a branch exclusively for the "camel mmap summary" work,
> viz., "mmapped-camel-summary-branch" which will help Phillip to continue
> his research further.

I'm going to let the patch rest for a few days so that you can take a
good look at it and decide what needs improvement before it goes into
that branch.

> Phillip: Make sure you don't change the format-of-summary-file too
> drastically, the "\0" is acceptable though. 

We have the discuss this :). But lets do that in focused mailing lists
like evolution-hackers.
 
> Federico, I would like it to be maintained and well tested in the branch
> rather then rushing it up in HEAD.  IMO, a branch gives more freedom to
> do such research work than HEAD.  Correct me if I am wrong. :-)

I agree here. Bringing it to HEAD might be to early. Nevertheless would
I be indeed very pleased if a lot people test this on a lot config-
urations.


Thanks for considering the patch.


-- 
Philip Van Hoof, software developer at x-tend 
home: me at pvanhoof dot be 
gnome: pvanhoof at gnome dot org 
work: vanhoof at x-tend dot be 
http://www.pvanhoof.be - http://www.x-tend.be

___
Evolution-hackers mailing list
Evolution-hackers@gnome.org
http://mail.gnome.org/mailman/listinfo/evolution-hackers

Re: [Evolution-hackers] no-inst headers camel-private.h?

[Smartuser]

camel/providers like for example: groupwise uses CAMEL_SERVICE_LOCK which is 
declared in the camel-private.h file. So it doesn't seem private to me.

On Wednesday 19 July 2006 15:00, Harish Krishnaswamy wrote:
> On Wed, 2006-07-19 at 14:36 +0200, Smartuser wrote:
> > Hi,
> >
> > I've some question about the camel-private.h
> > Why is it a no-inst header according to the Makefile.am?
> > As far as i know almost every provider depends on it.
> > Is there a reason or is it just a mistake?
> >
> > Serjan Pruis
>
> As the name suggests, I am led to believe that it is a private header
> and intentionally not installable.
> The mistake could actually be that the providers include this directly.
>
> Harish
___
Evolution-hackers mailing list
Evolution-hackers@gnome.org
http://mail.gnome.org/mailman/listinfo/evolution-hackers

Re: [Evolution-hackers] no-inst headers camel-private.h?

[Harish Krishnaswamy]

On Wed, 2006-07-19 at 14:36 +0200, Smartuser wrote:
> Hi,
> 
> I've some question about the camel-private.h
> Why is it a no-inst header according to the Makefile.am?
> As far as i know almost every provider depends on it.
> Is there a reason or is it just a mistake?
> 
> Serjan Pruis

As the name suggests, I am led to believe that it is a private header
and intentionally not installable.
The mistake could actually be that the providers include this directly.

Harish

___
Evolution-hackers mailing list
Evolution-hackers@gnome.org
http://mail.gnome.org/mailman/listinfo/evolution-hackers

[Evolution-hackers] no-inst headers camel-private.h?

[Smartuser]

Hi,

I've some question about the camel-private.h
Why is it a no-inst header according to the Makefile.am?
As far as i know almost every provider depends on it.
Is there a reason or is it just a mistake?

Serjan Pruis
___
Evolution-hackers mailing list
Evolution-hackers@gnome.org
http://mail.gnome.org/mailman/listinfo/evolution-hackers

Re: [Evolution-hackers] Memory consumption and virtual machines

[Veerapuram Varadhan]

On Tue, 2006-07-18 at 18:32 +, Federico Mena Quintero wrote:
> 1. Branch evolution-data-server into HEAD (development, with Philip's
> patch), and the stable branch (without the patch).
> 
I have created a branch exclusively for the "camel mmap summary" work,
viz., "mmapped-camel-summary-branch" which will help Phillip to continue
his research further.

Phillip: Make sure you don't change the format-of-summary-file too
drastically, the "\0" is acceptable though. 

> 2. Make the patch *mandatory* in HEAD, so that it gets a good amount
> of
> testing.
> 
Federico, I would like it to be maintained and well tested in the branch
rather then rushing it up in HEAD.  IMO, a branch gives more freedom to
do such research work than HEAD.  Correct me if I am wrong. :-)

V. Varadhan

 Novell, Inc. 
Software for the Open Enterprise™
http://www.novell.com
___
Evolution-hackers mailing list
Evolution-hackers@gnome.org
http://mail.gnome.org/mailman/listinfo/evolution-hackers

Re: [Evolution-hackers] Memory consumption and virtual machines

[Dirk-Jan.Binnema]

Hi Philip,

>-Original Message-
>From: [EMAIL PROTECTED] 
>[mailto:[EMAIL PROTECTED] On Behalf Of ext 
>Philip Van Hoof

[]

>For you, attached and on a plate:
>
>o. The patch for evolution-data-server
>o. The patch for evolution-exchange
>
>
>Trying to get this upstream is, for me, saying thank you.
>
>Looking at the patch technically AND testing it (and if it 
>doesn't perform, giving me numbers that compare it with the 
>original implement-
>ation) is all I'm asking for.

Great work -- thanks! If you did some amazing things there; most
of all putting a bit of much needed focus on the performance 
issues in camel. The tinymail stuff makes it possible to use
libcamel on the 770; the mmap-stuff might/will enable us to
use insane amounts of headers as well.

Best wishes,
Dirk.
___
Evolution-hackers mailing list
Evolution-hackers@gnome.org
http://mail.gnome.org/mailman/listinfo/evolution-hackers

Re: [Evolution-hackers] Memory consumption and virtual machines

[Veerapuram Varadhan]

On Tue, 2006-07-18 at 23:05 +, Philip Van Hoof wrote:
> If Novell wants me to implement unit tests (or other tests) for this,
> I
> will ask for payment. 

I am afraid that you won't get paid as Camel already has a
neat-test-suite and can be used/extended, IMO. ;-)

V. Varadhan

 Novell, Inc. 
Software for the Open Enterprise™
http://www.novell.com
___
Evolution-hackers mailing list
Evolution-hackers@gnome.org
http://mail.gnome.org/mailman/listinfo/evolution-hackers

Re: [Evolution-hackers] Memory consumption and virtual machines

[Veerapuram Varadhan]

On Tue, 2006-07-18 at 18:32 +, Federico Mena Quintero wrote:
> Novell already has a bunch of LDTP stuff to test the Evo mailer from
> the
> user's viewpooint - run those tests on the patched version to see how
> well they work.  [Varadhan, those tests are already part of our QA
> process, aren't they?] 

Yes and Yes, they are part of our QA process and with the SoC thing, we
are getting quite a good number of automation scripts that are
compatible with Evolution 2.6. 

V. Varadhan

 Novell, Inc. 
Software for the Open Enterprise™
http://www.novell.com
___
Evolution-hackers mailing list
Evolution-hackers@gnome.org
http://mail.gnome.org/mailman/listinfo/evolution-hackers