Re: [Gegl-developer] PerfECT code repository
Hi, My excuses for the delay. I decided to test the project build and setup on a second platform and found some problems: - Arrays.copyOf is only java 1.6 and I want to keep the compatability with java 1.5 - servlet-api.jar mistakenly included in war file (only necessary at build-time) - ant build errors - etc... these have been fixed and I've created a wiki page with the instructions: http://code.google.com/p/perfekt/wiki/InstallationAndSetup You'll also need to read: http://code.google.com/p/perfekt/issues/detail?id=1 Regards, Henrik 2009/7/14 Martin Nordholts ense...@gmail.com: On 07/11/2009 08:14 AM, Henrik Akesson wrote: Hi, There are for the moment some discussion as to wheter the code I've produced should be in the main gimp project or in a separate one. However, at this point, I feel that it should be under some version control system, for ease of sharing and security. Thus, I've commited it to the following google code site: http://code.google.com/p/perfekt/ (the name perfect was already taken) Hi Henrik Could you give rough step-by-steps on how to run this please? BR, Martin ___ Gegl-developer mailing list Gegl-developer@lists.XCF.Berkeley.EDU https://lists.XCF.Berkeley.EDU/mailman/listinfo/gegl-developer ___ Gegl-developer mailing list Gegl-developer@lists.XCF.Berkeley.EDU https://lists.XCF.Berkeley.EDU/mailman/listinfo/gegl-developer
[Gegl-developer] PerfECT code repository
Hi, There are for the moment some discussion as to wheter the code I've produced should be in the main gimp project or in a separate one. However, at this point, I feel that it should be under some version control system, for ease of sharing and security. Thus, I've commited it to the following google code site: http://code.google.com/p/perfekt/ (the name perfect was already taken) I will of course move it, if necessary, according to the descision taken (when it is taken). Best regards, Henrik ___ Gegl-developer mailing list Gegl-developer@lists.XCF.Berkeley.EDU https://lists.XCF.Berkeley.EDU/mailman/listinfo/gegl-developer
Re: [Gegl-developer] Spawn a new project?
The only dependency there is between the visualisation tool and GEGL would be the output file format from callgrind. /Henrik 2009/7/7 Martin Nordholts ense...@gmail.com: On 07/07/2009 03:45 PM, Henrik Akesson wrote: Hi, the work I've done so far can be split into two parts: 1) gegl scripting - for the moment a ruby script that allows me to run tests of gegl and some test cases. 2) visualisation tool - a java web project that is made for displaying performance data. 1) should obviously be commited to the gegl project. 2) however, I believe should be a new open source project. How does the dependencies look here? I suppose GEGL would have a soft (optional) dependency on the visualization tool? And as for the VCS preference, git is my clear preference, it's a shame Google Code doesn't offer that. / Martin ___ Gegl-developer mailing list Gegl-developer@lists.XCF.Berkeley.EDU https://lists.XCF.Berkeley.EDU/mailman/listinfo/gegl-developer
[Gegl-developer] Spawn a new project?
Hi, the work I've done so far can be split into two parts: 1) gegl scripting - for the moment a ruby script that allows me to run tests of gegl and some test cases. 2) visualisation tool - a java web project that is made for displaying performance data. 1) should obviously be commited to the gegl project. 2) however, I believe should be a new open source project. For the moment, I've named the visualisation tool PerfECT, for Performance Evaluation and Control Tool. I propose to create a new open source project at code.google.com using a LGPL license. Do you have any preferences as to which version control system I should use? The google code hosting offers Mercurial and SVN. I'm leaning towards using SVN on the server and git-svn on my laptop, mainly because I already know both svn and git (well, git only a little bit). Regards, Henrik ___ Gegl-developer mailing list Gegl-developer@lists.XCF.Berkeley.EDU https://lists.XCF.Berkeley.EDU/mailman/listinfo/gegl-developer
[Gegl-developer] Perf tools evaluation and proposal
Hi, here's the promised evaluation of the current profiling tools and a proposal. Regards, Henrik TOOLS Valgrind - retained Oprofile - retained gprof, sprof - obsoleted by oprofile ltrace, ptrace - not capable of profiling dynamically loaded objects (dlopen-ed) GEGL instrumentation - same functionality is achieved by the below proposition. Further info: http://sites.google.com/site/computerresearcher/profiling-tools/ PART ONE - VALGRIND - Where the time is spent Current tools are capable of producing an abundance of profiling data. Callgrind (with Cachegrind activated) will produce 13 different measurements for every line of code that has been executed during the program run: 1 on how many times the line has been executed 8 on cache use 4 on branching Running callgrind on a command line gegl program that runs gaussian-blur produces profiling data for 149 different objects (89 of which are gegl operations that get loaded)!!! NOTE: A serious limitation of Valgrind is that it can only count events, it cannot tell you how much time things take (such as cache miss or execution time for an instruction). This is because it heavily modifies the code before running it, and therefore renders any time measurements useless. I propose to implement a tool that allows the user to (step 1) select the data he is interested in and (step 2) presents the results in a easy-to-understand way. Step 1 - SELECTION - user workflow: 1a) Select the libraries of interest 1b) Select the entry/exit function (normally the main function), i.e. only data measured inside this function (including calls to other functions) is displayed 1c) Hotpath elaboration. I.e. display and selection of the code execution path to display. (TBD) Step 2 - EVALUATION - workflow: 2a) Code annotation. I.e. display the above selected code with measurements 2b) Trend display Step 3 - MANAGEMENT - workflow: 3a) Adding new data (cmd line and web) 3b) Adding new evaluation scenarios (web) 3c) Listing data 3d) Deleting data 3e) Listing scenarios 3f) Deleting scenarios 3g) Adding scenarios PART TWO - OPROFILE - What the processor is doing An important limitation of the processor's performance counters is that from a choice of about 100, there are only 2-8, depending on processor model and mark, that can be used in the same moment. There has been some attempts by some researchers to multiplex them, but this requires modification of kernel code and thus is not accessible for the ordinary mortal developer. I propose to: a) By repeatedly running the test/performance case assemble all the important data into the database b) Possibly there will be a need to use statistics in order to have reliable data (i.e. determine which distribution and use this during data collection) TBD. c) Add groups of annotation data to the point 2a. Basically groups such as L1 Cache use, L2 Cache use, Vector extensions etc... Normally the data should be stored internally in the same format as for valgrind. This means that only an import tool needs to be written in order to add oprofile data. IMPLEMENTATION DETAILS I propose a web-based implementation using jRuby and Ruby on the Rails together with a sql database (could potentially be an object database = db4o). For the code annotation, I propose to use the existing subversion repository for retreiving the code to be annotated and GNU source highlight. ___ Gegl-developer mailing list Gegl-developer@lists.XCF.Berkeley.EDU https://lists.XCF.Berkeley.EDU/mailman/listinfo/gegl-developer
Re: [Gegl-developer] Perf tools evaluation and proposal
The main differences of my solution compared to KCachegrind are: I would use a database approach allowing to display performance history. I.e. in the case of a drop in performance, a developer would be able to localise it to a certain commit. Also useful for comparing the performance of two different solutions during development. The aggregation of counter, IMHO is very important for understanding what the processor is doing. It is a very difficult task, and just having data from 2-8 counters is like providing only a small window on a big complex image. I think that you could summarize kcachegrind as a tool that shows you all the data whereas the approach I propose is to show as little data as possible in order to localise performance issues and when found showing the full image of what the processor is doing. IMHO KCachegrind is not easy to use nor understand (ex: what does Distance 6-11 (6) mean? etc...) My solution is web based. I believe that this is enough of differences to justify a new tool. Is it to you? Henrik 2009/6/17 Boudewijn Rempt b...@valdyas.org: On Wed, 17 Jun 2009, Henrik Akesson wrote: I propose to implement a tool that allows the user to (step 1) select the data he is interested in and (step 2) presents the results in a easy-to-understand way. Doesn't kcachegrind provide all of this for you? Boudewijn ___ Gegl-developer mailing list Gegl-developer@lists.XCF.Berkeley.EDU https://lists.XCF.Berkeley.EDU/mailman/listinfo/gegl-developer
Re: [Gegl-developer] GEGL Perf tools - plan
AFAIK sysprof has a subset of the oprofile functionality, is very poorly documented and supports only x86 CPUs. This is why I don't plan on evaluating it. /Henrik 2009/6/4 Martin Nordholts ense...@gmail.com: Henrik Akesson wrote: This is the moment to let me know for anyone who wants me to include a certain tool in the survey. I'm currently at: - gprof, sprof - valgrind (cachegrind, callgrind) - ltrace, ptrace - oprofile Hi, Sysprof seems to be missing from that list Nice to hear you are making good progress anyway! / Martin ___ Gegl-developer mailing list Gegl-developer@lists.XCF.Berkeley.EDU https://lists.XCF.Berkeley.EDU/mailman/listinfo/gegl-developer ___ Gegl-developer mailing list Gegl-developer@lists.XCF.Berkeley.EDU https://lists.XCF.Berkeley.EDU/mailman/listinfo/gegl-developer
Re: [Gegl-developer] gegl cmd-line ppm patch
Martin, Well, I've got a git repo set up, even though I didn't start using it yet (will very soon). I'll have a look to see if I can set up my computer for public access at home (living in a shared appartment and don't have the passwd for the router). Would be good if you could add the patches I've sent manually and then in the future I'll make sure that you can do it the git way. Cheers, Henrik 2009/4/20 Martin Nordholts ense...@gmail.com: Henrik Akesson wrote: New patch... Hi Henrik Could we through IRC arrange me doing a git pull from you through e.g. a temporarily running git-daemon on your side? Would be nice to get a git workflow going for GEGL (and GIMP). Once I have pulled I can easily review, test and finally commit when things look OK. BR, Martin ___ Gegl-developer mailing list Gegl-developer@lists.XCF.Berkeley.EDU https://lists.XCF.Berkeley.EDU/mailman/listinfo/gegl-developer ___ Gegl-developer mailing list Gegl-developer@lists.XCF.Berkeley.EDU https://lists.XCF.Berkeley.EDU/mailman/listinfo/gegl-developer
Re: [Gegl-developer] two small patches
Now split (and no re-introduced commented code, sorry for that):
[gegl_buffer_rename.diff]
Patch from Henrik Akesson that renames pset/pget to
gegl_buffer_pixel_set and gegl_buffer_pixel_get in order to improve
readability of code and to conform to gegl coding standards.
* gegl/buffer/gegl-buffer-access.c
* gegl/buffer/gegl-buffer-private.h
[gegl_buffer_refactoring.diff]
Patch from Henrik Akesson that extracts duplicated code from
gegl_buffer_pixel_get and gegl_buffer_pixel_set into the function
gegl_buffer_in_abyss to improve readability of code and
maintainability.
* gegl/buffer/gegl-buffer-access.c
However for the get_rectangle_as_string functionality, I have to
reconsider given that the function returns a string that is used in
the GEGL_NOTE. You are right that g_string_free (string, FALSE)
doesn't free the actual character data, but it cannot as it is used in
the GEGL_NOTE function call. This means that the string has to be
freed later after the GEGL_NOTE call. In the end it just makes a
complicated code, so it's better not adding it.
Thanks,
Henrik
2009/3/24 Martin Nordholts ense...@gmail.com:
Henrik Akesson wrote:
Two patches attached. I've made the below comments in a more suitable
ChangeLog format:
[gegl_buffer.diff]
Patch from Henrik Akesson:
- Extracted duplicated code from pset/pget into a separate function
gegl_buffer_in_abyss to improve readability of code and maintainability.
- Renamed pset/pget to gegl_buffer_pixel_set and gegl_buffer_pixel_get
in order to improve readability of code and to be conformant to gegl
coding standards.
Hi,
Do you think you can split up that patch into two patches? One with the
renames and one with the new utility function. It also added g_debug()
statements that were commented away, please remove those. Thanks!
[gegl_utils.diff]
Patch from Henrik Akesson that adds a gegl_rectangle_as string
function for retreiving a rectangle as a debug string. Log statements
has been changed to use this said function as well as using GEGL_NOTE
(see gegl-debug.h).
That leaks memory because the data returned from g_string_free() needs
to be freed if FALSE is passed to the free_segment parameter (you pass
NULL (!)). Otherwise it is a nice change. Could you address the memory
leaks please?
BR,
Martin
Index: gegl/buffer/gegl-buffer-access.c
===
--- gegl/buffer/gegl-buffer-access.c (revision 2986)
+++ gegl/buffer/gegl-buffer-access.c (working copy)
@@ -50,11 +50,11 @@
#if 0
static inline void
-pset (GeglBuffer *buffer,
- gintx,
- ginty,
- const Babl *format,
- guchar *buf)
+gegl_buffer_pixel_set (GeglBuffer *buffer,
+ gintx,
+ ginty,
+ const Babl *format,
+ guchar *buf)
{
gint tile_width = buffer-tile_storage-tile_width;
gint tile_height = buffer-tile_storage-tile_width;
@@ -116,11 +116,11 @@
#endif
static inline void
-pset (GeglBuffer *buffer,
- gintx,
- ginty,
- const Babl *format,
- gpointerdata)
+gegl_buffer_set_pixel (GeglBuffer *buffer,
+ gintx,
+ ginty,
+ const Babl *format,
+ gpointerdata)
{
guchar *buf = data;
ginttile_width = buffer-tile_storage-tile_width;
@@ -200,11 +200,11 @@
}
static inline void
-pget (GeglBuffer *buffer,
- gintx,
- ginty,
- const Babl *format,
- gpointerdata)
+gegl_buffer_get_pixel (GeglBuffer *buffer,
+ gintx,
+ ginty,
+ const Babl *format,
+ gpointerdata)
{
guchar *buf = data;
ginttile_width = buffer-tile_storage-tile_width;
@@ -595,7 +595,7 @@
if (rect rect-width == 1 rect-height == 1) /* fast path */
{
- pset (buffer, rect-x, rect-y, format, src);
+ gegl_buffer_set_pixel (buffer, rect-x, rect-y, format, src);
}
else
{
@@ -949,7 +949,7 @@
rect-width == 1
rect-height == 1) /* fast path */
{
- pget (buffer, rect-x, rect-y, format, dest_buf);
+ gegl_buffer_get_pixel (buffer, rect-x, rect-y, format, dest_buf);
#if ENABLE_MP
g_static_rec_mutex_unlock (mutex);
#endif
@@ -1084,7 +1084,7 @@
/*#define USE_WORKING_SHORTCUT*/
#ifdef USE_WORKING_SHORTCUT
- pget (buffer, x, y, format, dest);
+ gegl_buffer_get_pixel (buffer, x, y, format, dest);
return;
#endif
Index: gegl/buffer/gegl-buffer-private.h
===
--- gegl/buffer/gegl-buffer-private.h (revision 2986)
+++ gegl/buffer/gegl-buffer-private.h (working copy)
@@ -47,8 +47,8 @@
GeglRectangle abyss;
- GeglTile *hot_tile; /* cached tile
[Gegl-developer] gegl/gegl documentation patch
Here's a patch for: - Documentation and debug statements added - Removed needs_visiting from the gegl-visitable interface and the implementing classes gegl-pad and gegl-node because it was not used (hardcoded to return TRUE). - Refactored gegl-eval-mgr field state to use an enumeration instead of integers for improving readability of the code. /Henrik gegldoc.diff Description: Binary data ___ Gegl-developer mailing list Gegl-developer@lists.XCF.Berkeley.EDU https://lists.XCF.Berkeley.EDU/mailman/listinfo/gegl-developer
[Gegl-developer] Small patch
Hi, here's a small patch on gegl-processor.c I've kept it very small in order to get some feedback on what I'm doing. The patch contains: - comments - refactoring of the render_rectangle function: extraction of code into a separate function get_band_size. No functional changes. - debug statements (well a few) I would especially like to get some feedback on the extraction (refactoring) of methods. There are some functions with about 300 lines, wich I wouldn't mind breaking up into smaller functions to make it easier to understand them. Thanks, Henrik gegl-processor.diff Description: Binary data ___ Gegl-developer mailing list Gegl-developer@lists.XCF.Berkeley.EDU https://lists.XCF.Berkeley.EDU/mailman/listinfo/gegl-developer
Re: [Gegl-developer] Small patch
Thank you for the comments, the patch has been updated accordingly. One question: sometimes I see some methods without the gegl_something_ prefix, does that signify something (like private method)? Second question: what does BLIT mean? Henrik 2009/3/12 Sven Neumann s...@gimp.org Hi, some comments on your patch: +#include glib.h #include glib-object.h That's redundant as glib-object.h already includes this file for you. +inline gint get_band_size (gint size); Splitting the code into smaller functions is cool. But please consider to name the functions according to the file it lives in. So this would become gegl_processor_get_band_size(). This makes it easier to interpret stack traces and to discuss the code as it is clear where the function is located. +inline gint +get_band_size ( gint size ) Please format this as get_band_size (gint size). Also you will want to declare this function as static. And as far as I can see it should also be labeled G_GNUC_CONST as it doesn't examine any values except its parameters, and has no effects except its return value. I wouldn't declare it inline though. The compiler can better decide if it makes sense to inline this function. I am not happy about the way you reformatted the code in the class_init method: - g_object_class_install_property (gobject_class, PROP_NODE, - g_param_spec_object (node, -GeglNode, -The GeglNode to process (will saturate the provider's cache if the provided node is a sink node), -GEGL_TYPE_NODE, - G_PARAM_WRITABLE | - G_PARAM_CONSTRUCT)); + g_object_class_install_property ( +gobject_class, +PROP_NODE, +g_param_spec_object ( node, +GeglNode, +The GeglNode to process (will saturate the provider's +cache if the provided node is a sink node), +GEGL_TYPE_NODE, +G_PARAM_WRITABLE | +G_PARAM_CONSTRUCT)); Can we please keep the formatting here? If you want to reduce the code to fewer columns, you could break the blurb into multiple lines. Other than that, the patch looks good to me. If you incorporate my suggestions I will take care of getting the next version into trunk. Nice to see some work being done here. Sven gegl-processor.diff Description: Binary data ___ Gegl-developer mailing list Gegl-developer@lists.XCF.Berkeley.EDU https://lists.XCF.Berkeley.EDU/mailman/listinfo/gegl-developer
[Gegl-developer] visitable_needs_visiting
The two implementations of GeglVisitable always returns TRUE on calls to visitable_needs_visiting (see gegl-node:957 and gegl-pad:319). It is used by gegl-visitor.c, but makes it more difficult to understand the code. Can I remove it? Henrik ___ Gegl-developer mailing list Gegl-developer@lists.XCF.Berkeley.EDU https://lists.XCF.Berkeley.EDU/mailman/listinfo/gegl-developer
[Gegl-developer] Gegl logging statements
Hi,
I tend to sprinkle the code with logging statements when learning/debugging.
glib has the appropriate functionality for managing the logs... Normally the
overhead of the logging statements (if turned off) is neglectable.
Should I leave my log statements in my patches (that will be mainly focused
on adding comments)?
Example statement:
g_debug ( {%s:%u} rectangle split to (%ux%u) and (%ux%u)\n,
__FILE__, __LINE__, dr-height, dr-width,
fragment-height, fragment-width);
Output:
** (lt-gegl:13884): DEBUG: {gegl-processor.c:460} rectangle split to
(675x496) and (675x256)
Thanks,
Henrik
___
Gegl-developer mailing list
Gegl-developer@lists.XCF.Berkeley.EDU
https://lists.XCF.Berkeley.EDU/mailman/listinfo/gegl-developer
[Gegl-developer] Documenting gegl
Hi all, I'm learning gegl internals for doing some work on parallelizing it (I've posted before on that subject). When I do it I normally write some documentation of my own (when it's missing), such as: http://computerresearcher.blogspot.com/search/label/Gegl Note that it's fragmented and still under construction. Instead of writing it on my blog I could contribute it. 1. Are you interested in that? If yes: 2. What is the appropriate format. Asciidoc? Note that there are/will be images, documentation for specific c files and general documentation. 3. It would need to be reviewed. I'm after all learning this stuff... How can it be done practically. I suppose that a separate svn branch for me with someone of you merging could be a solution. /Henrik ___ Gegl-developer mailing list Gegl-developer@lists.XCF.Berkeley.EDU https://lists.XCF.Berkeley.EDU/mailman/listinfo/gegl-developer
[Gegl-developer] Optimisations
Hi, This is the first time I write on this list, so I'll introduce myself briefly: I've been working/developing for the european space industry for 7 years, but have decided that I want to do SW research, which is why I'm currently doing a Master (to be followed by a PhD). During this MSc I'm specialising in parallelism. The second half of this master, (first half of 2009) is a research project. This will be on the subject of optimising image treatment algorithms using different parallel languages/addons such as : - CUDA - AMD Stream - OpenMP - MPI - the Barcelona Supercomputing Center's Cell/SMP Superscalar So, now to the point: I'd like to know if you would find it useful if I do and contribute this work on algoritms in GEGL? Would you have any restrictions on which technologies to use (availability of open source compiler/runtime, size etc...) ? If it interests you, could you point me towards the algorithms that needs performance? Regards, Henrik ___ Gegl-developer mailing list Gegl-developer@lists.XCF.Berkeley.EDU https://lists.XCF.Berkeley.EDU/mailman/listinfo/gegl-developer
