On Jan 10, 2008 10:53 AM, Cedric BAIL <[EMAIL PROTECTED]> wrote:
>
> On Jan 10, 2008 2:38 PM, Gustavo Sverzut Barbieri <[EMAIL PROTECTED]> wrote:
> > On Jan 10, 2008 9:51 AM, Cedric BAIL <[EMAIL PROTECTED]> wrote:
> > > On Jan 7, 2008 2:44 PM, Gustavo Sverzut Barbieri <[EMAIL PROTECTED]>
> > > wrote:
> > > > On Jan 7, 2008 10:43 AM, Cedric BAIL <[EMAIL PROTECTED]> wrote:
> > > > > On Jan 7, 2008 5:13 AM, Gustavo Sverzut Barbieri <[EMAIL PROTECTED]>
> > > > > wrote:
> > > > > > On Jan 4, 2008 8:43 PM, Nathan Ingersoll <[EMAIL PROTECTED]> wrote:
> > > > > > > On Jan 4, 2008 4:02 PM, Cedric BAIL <[EMAIL PROTECTED]> wrote:
> > > > > > > > I am looking for a way to improve edje file load time. Right
> > > > > > > > now a
> > > > > > > > large amount of time is logically lost in
> > > > > > > > _eet_data_descriptor_decode.
> > > > > > > > So looking at its profile, most of the time is lost in
> > > > > > > > manipulating
> > > > > > > > string.
> > > > > > >
> > > > > > > Before getting into the details of your change ideas, could you
> > > > > > > explain the use case that you are profiling and the motivation
> > > > > > > behind
> > > > > > > it? For instance, are you seeing a slow load time when initially
> > > > > > > starting an application, or is this a case where various edje
> > > > > > > files
> > > > > > > are decoded over time, or accessing multiple data fields, etc?
> > > > >
> > > > > Sorry Nathan, I did reply directly to you and not to the mailing, it
> > > > > was a mistake.
> > > > >
> > > > > So I am profiling the time needed by an edje object to become
> > > > > visible/active to the end user after the edje_object_file_set. I was
> > > > > only testing with some quite complex edje file with many swallowed
> > > > > object. I just did test with multiple edje file at the same time and
> > > > > the profile result look the same.
> > > > >
> > > > > > Nathan points are valid, of course, but I'm really interested in
> > > > > > this
> > > > > > kind of stuff, because it's mostly "for free" if you can mmap it,
> > > > > > also
> > > > > > it will help on high memory pressure situations, since these regions
> > > > > > would be file-based and can be evicted and reloaded when app is used
> > > > > > again (not a situation I'd like, but would help on embedded systems
> > > > > > a
> > > > > > bit).
> > > > >
> > > > > Yes, it should help and reduce memory presure on embedded systems.
> > > > >
> > > > > > Cedric, something else related is trying to group these strings
> > > > > > near,
> > > > > > so when they're loaded they'd be on the same memory page, this would
> > > > > > help disk seeks and the issue I pointed before. Any knowledge of
> > > > > > this
> > > > > > area?
> > > > >
> > > > > Right now, string are repeated a lot in the eet file and they are at
> > > > > many different place. If you want to see this, just change the call to
> > > > > eet_data_write inside edje_cc_out.c to not compress data.
> > > > >
> > > > > So packing them together in one section and reducing them to only one
> > > > > occurence inside the eet file could be another improvement. But the
> > > > > way I see it (adding a new kind of section at the end of the eet file
> > > > > and refering to it) will break backward compatibility (eet file using
> > > > > this will not be loadable with previous libeet). It could solve some
> > > > > of the issue you are pointing.
> > > > >
> > > > > A side effect of this solution, we will not need to tell the
> > > > > application anything about the char* we are giving to it. Because all
> > > > > of them will be in a uncompressed section available as long as the eet
> > > > > file is referenced somewhere. This solution break eet file
> > > > > representation but not eet application.
> > > >
> > > > I'm not sure others will like, but I do like this idea, and given
> > > > raster's word that "after E is marked stable, it will have to be
> > > > stable, so break things before it's released", I really think we
> > > > should fix this now (and break compat) rather than staying with this
> > > > for some years more.
> > >
> > > I took a little bit more time studing the internal of eet and I think
> > > we could change the format to the following layout :
> > >
> > > /* NB: all int's are stored in network byte order on disk */
> > > /* file format: */
> > > int magic; /* magic number ie 0x1ee7ff42 */
> > > int num_string_entries; /* number of string entries to follow */
> > > int num_directory_entries; /* number of directory entries to follow */
> > > struct
> > > {
> > > int offset;
> > > int len;
> > > int hash;
> > > } string[num_string_entries];
> > > struct
> > > {
> > > int offset; /* bytes offset into file for data chunk */
> > > int flags; /* flags - for now 0 = uncompressed, 1 = compressed */
> > > int size; /* size of the data chunk */
> > > int data_size; /* size of the (uncompressed) data chunk */
> > > int name_index; /* index of the name in the string entries */
> > > } directory[num_directory_entries];
> > > /* first all null terminated string */
> > > /* and now startes the data stream... */
> > >
> > > This format will not be readable by older version of libeet, that's
> > > the reason behind changing it's magic value. I also put all the
> > > strings after the directory structure so their is more chance to have
> > > aligned data and during use they should have more chance to be in the
> > > same memory page.
> > >
> > > I think the change are quite small inside eet. And it should be
> > > possible to still be able to open old eet file if necessary, but this
> > > will increase the code complexity of the library.
> > >
> > > Before I start doing implementing this change, I would like to know
> > > more opinion on this subject. For information, follow the current eet
> > > file format :
> > >
> > > /* NB: all int's are stored in network byte order on disk */
> > > /* file format: */
> > > int magic; /* magic number ie 0x1ee7ff00 */
> > > int num_directory_entries; /* number of directory entries to follow */
> > > int bytes_directory_entries; /* bytes of directory entries to follow */
> > > struct
> > > {
> > > int offset; /* bytes offset into file for data chunk */
> > > int flags; /* flags - for now 0 = uncompressed, 1 = compressed */
> > > int size; /* size of the data chunk */
> > > int data_size; /* size of the (uncompressed) data chunk */
> > > int name_size; /* length in bytes of the name field */
> > > char name[name_size]; /* name string (variable length) */
> > > } directory[num_directory_entries];
> > > /* and now startes the data stream... */
> >
> > It's fine for me, just one minor thing: what was the use of
> > bytes_directory_entries? Is it ok to leave it out?
>
> In the current eet file format, string are inside the directory
> structure, so they don't have a fixed size and it's usefull during
> reading to know the size of all directory entries (It's not the direct
> result of some math, but it need the reading of all directory). With
> my proposal, I don't think we will need this information neither the
> size of all stored string.
I see, it makes sense.
--
Gustavo Sverzut Barbieri
--------------------------------------
Jabber: [EMAIL PROTECTED]
MSN: [EMAIL PROTECTED]
ICQ#: 17249123
Skype: gsbarbieri
Mobile: +55 (81) 9927 0010
-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace
_______________________________________________
enlightenment-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/enlightenment-devel
