RE: [Gimp-developer] Gimp_image_delete and adding text layers
top isn't actually a very accurate way of profiling memory usage. The numbers you have shown so far can easily be explained by memory fragmentation and the fact that glibc allocates memory in pools. Smaller memory fragments are not returned to the operating system but are being kept for reuse in the application. Please run your script a lot more often and see if there's a significant increase in memory. Sven Jared Whiting [EMAIL PROTECTED] wrote: This script gets run 5000 times in a loop, and it's the only script accessing the gimp instance. ...I'm not sure how significant the increase is as I have to run the script quite a bit, but this is what I see after running several loops and using top after each loop: Start up: VIRT RES SHR S %CPU %MEM TIME+ COMMAND 17632 7448 13m S 0.0 1.4 0:01.14 gimp Loop 1: VIRT RES SHR S %CPU %MEM TIME+ COMMAND 21088 11m 13m S 0.0 2.2 11:45.59 gimp Loop 2: VIRT RES SHR S %CPU %MEM TIME+ COMMAND 25544 14m 13m S 0.0 2.9 23:28.40 gimp Loop 3: VIRT RES SHR S %CPU %MEM TIME+ COMMAND 27796 17m 13m S 0.0 3.5 35:11.54 gimp Loop 4: VIRT RES SHR S %CPU %MEM TIME+ COMMAND 31408 21m 13m S 0.0 4.2 46:54.84 gimp Loop 5: VIRT RES SHR S %CPU %MEM TIME+ COMMAND 35220 24m 13m S 0.0 4.8 58:35.12 gimp Loop 6: VIRT RES SHR S %CPU %MEM TIME+ COMMAND 38108 27m 13m S 0.0 5.5 70:11.92 gimp Loop 7: VIRT RES SHR S %CPU %MEM TIME+ COMMAND 42292 31m 13m S 0.0 6.2 81:51.41 gimp Thanks, Jared Is there any more information I can provide regarding my test script and the memory increase I observe with GIMP? Following your advice I ran my script more often, but I'm not sure if my response was what you were looking for. If I run a script that creates a new image with six text layers in a loop it results in about a 3000k increase in the amount of memory the GIMP process is holding on to for each 5000 times the script gets executed (as observed using top). There is only a single instance of the script being executed at any time. This behavior is similar to what I have observed in our more complicated script we use where small but permanent memory increases occur over time. I am open to the idea that there is no memory leak and that it's a problem with my script or that I'm providing unreliable/inadequate info regarding the issue so let me know if that's still the case. I mainly am looking for enough insight so I don't have to rely on having to restart GIMP for our image generation scripts unless it's absolutely necessary. Thanks, Jared ___ Gimp-developer mailing list Gimp-developer@lists.xcf.berkeley.edu http://lists.xcf.berkeley.edu/mailman/listinfo/gimp-developer
Re: [Gimp-developer] Gimp_image_delete and adding text layers
Hi, Jared Whiting [EMAIL PROTECTED] writes: Is there any more information I can provide regarding my test script and the memory increase I observe with GIMP? Following your advice I ran my script more often, but I'm not sure if my response was what you were looking for. If I run a script that creates a new image with six text layers in a loop it results in about a 3000k increase in the amount of memory the GIMP process is holding on to for each 5000 times the script gets executed (as observed using top). There is only a single instance of the script being executed at any time. 3MB for 5000 executions of your script doesn't look like we'd be leaking image tiles. We might leak some smaller structures that are allocated during the execution of the script. What you could do is run gimp in a memory profiler like valgrind (http://valgrind.org/). Use a command-line like valgrind --num-callers=24 --leak-check=yes --error-limit=no gimp You will need more memory than usually to run valgrind and you should also use a fast machine. Otherwise things will become incredibly slow. Start by running your script once, then quit gimp. valgrind should then leaked memory. A few hundred bytes of leaked memory are normal. These are being lost in libraries used by gimp and the memory lost here is constant and only allocated on initialization. Every leak in gimp should be reported though. If you don't find any leaks this way, we have to assume the problem is memory fragmentation. We could then try to figure out if there are ways to avoid this problem (for example by using GLib memchunks). Sven ___ Gimp-developer mailing list Gimp-developer@lists.xcf.berkeley.edu http://lists.xcf.berkeley.edu/mailman/listinfo/gimp-developer
RE: [Gimp-developer] Gimp_image_delete and adding text layers
top isn't actually a very accurate way of profiling memory usage. The numbers you have shown so far can easily be explained by memory fragmentation and the fact that glibc allocates memory in pools. Smaller memory fragments are not returned to the operating system but are being kept for reuse in the application. Please run your script a lot more often and see if there's a significant increase in memory. Sven I have a test script that does the following: Gimp::init; my $img = gimp_image_new(300, 200, 0); $img-gimp_image_undo_disable; my $layer = gimp_layer_new($img, 300,200,RGB, Layer 2, 100, NORMAL_MODE); gimp_image_add_layer($img, $layer, -1); my $text_layer = gimp_text_fontname($img,-1,0,0,, 0, 0, 14, 0, Arial Bold); gimp_image_merge_down($img, $text_layer, CLIP_TO_BOTTOM_LAYER); my $text_layer2 = gimp_text_fontname($img,-1,0,10,2, 0, 0, 14, 0, Arial Bold); gimp_image_merge_down($img, $text_layer2, CLIP_TO_BOTTOM_LAYER); my $text_layer3 = gimp_text_fontname($img,-1,0,20,3, 0, 0, 14, 0, Arial Bold); gimp_image_merge_down($img, $text_layer3, CLIP_TO_BOTTOM_LAYER); my $text_layer4 = gimp_text_fontname($img,-1,0,30,44, 0, 0, 14, 0, Arial Bold); gimp_image_merge_down($img, $text_layer4, CLIP_TO_BOTTOM_LAYER); my $text_layer5 = gimp_text_fontname($img,-1,0,40,555, 0, 0, 14, 0, Arial Bold); gimp_image_merge_down($img, $text_layer5, CLIP_TO_BOTTOM_LAYER); my $text_layer6 = gimp_text_fontname($img,-1,0,50,6, 0, 0, 14, 0, Arial Bold); gimp_image_merge_down($img, $text_layer6, CLIP_TO_BOTTOM_LAYER); $img-gimp_image_undo_enable; gimp_image_delete($img); Gimp::end; This script gets run 5000 times in a loop, and it's the only script accessing the gimp instance. I see no change in the perl-server and script fu processes, but I do see an increase for gimp. I'm not sure how significant the increase is as I have to run the script quite a bit, but this is what I see after running several loops and using top after each loop: Start up: VIRT RES SHR S %CPU %MEMTIME+ COMMAND 17632 7448 13m S 0.0 1.4 0:01.14 gimp Loop 1: VIRT RES SHR S %CPU %MEMTIME+ COMMAND 21088 11m 13m S 0.0 2.2 11:45.59 gimp Loop 2: VIRT RES SHR S %CPU %MEMTIME+ COMMAND 25544 14m 13m S 0.0 2.9 23:28.40 gimp Loop 3: VIRT RES SHR S %CPU %MEMTIME+ COMMAND 27796 17m 13m S 0.0 3.5 35:11.54 gimp Loop 4: VIRT RES SHR S %CPU %MEMTIME+ COMMAND 31408 21m 13m S 0.0 4.2 46:54.84 gimp Loop 5: VIRT RES SHR S %CPU %MEMTIME+ COMMAND 35220 24m 13m S 0.0 4.8 58:35.12 gimp Loop 6: VIRT RES SHR S %CPU %MEMTIME+ COMMAND 38108 27m 13m S 0.0 5.5 70:11.92 gimp Loop 7: VIRT RES SHR S %CPU %MEMTIME+ COMMAND 42292 31m 13m S 0.0 6.2 81:51.41 gimp Thanks, Jared ___ Gimp-developer mailing list Gimp-developer@lists.xcf.berkeley.edu http://lists.xcf.berkeley.edu/mailman/listinfo/gimp-developer
Re: [Gimp-developer] Gimp_image_delete and adding text layers
Hi, michael chang [EMAIL PROTECTED] writes: What about putting $img = undef; at the end -- does this change anything? Shouldn't really matter since gimp-perl runs in a separate process and the original claim was that gimp was leaking memory, not gimp-perl. Sven ___ Gimp-developer mailing list Gimp-developer@lists.xcf.berkeley.edu http://lists.xcf.berkeley.edu/mailman/listinfo/gimp-developer
RE: [Gimp-developer] Gimp_image_delete and adding text layers
How much memory increase do you see per image? Can you run this script a couple of times and show us memory usage before and after? Sven When I first start up GIMP: /usr/bin/gimp -d -i -c --batch '(extension-perl-server 1 0 0 )' I see the following using top (script-fu and Perl-Server processes never change so I'm not including those): VIRT RES SHR S %CPU %MEMTIME+ COMMAND 18440 7444 13m S 0.0 1.4 0:01.07 gimp Then I execute the code shown at the end of this message, and I see the following: VIRT RES SHR S %CPU %MEMTIME+ COMMAND 18648 8056 13m S 0.0 1.6 0:01.12 gimp Within the code I add an additional text layer and use gimp_merge_down, and see no change. I add seven text layers and still no change (I had originally thought adding a new text layer ALWAYS results in more memory being taken but this is not the case, so my initial description of the problem is misleading.) After running the script with the new text layers about 25 times in rapid succession I see the following: VIRT RES SHR S %CPU %MEMTIME+ COMMAND 18788 8196 13m S 0.0 1.6 0:06.23 gimp This behavior occurs with this sample pretty consistently. Not a huge increase, but the cgi I'm using in production does a lot more than this and can be generating a few thousand images in one day. Thanks, Jared ___ Gimp-developer mailing list Gimp-developer@lists.xcf.berkeley.edu http://lists.xcf.berkeley.edu/mailman/listinfo/gimp-developer
Re: [Gimp-developer] Gimp_image_delete and adding text layers
On 8/17/05, Jared Whiting [EMAIL PROTECTED] wrote: increase, but the cgi I'm using in production does a lot more than this and can be generating a few thousand images in one day. Are any of these concurrently, and if so how many; could this make a difference? -- ~Mike - Just my two cents - No man is an island, and no man is unable. ___ Gimp-developer mailing list Gimp-developer@lists.xcf.berkeley.edu http://lists.xcf.berkeley.edu/mailman/listinfo/gimp-developer
RE: [Gimp-developer] Gimp_image_delete and adding text layers
Are any of these concurrently, and if so how many; could this make a difference? ~Mike In production there can definitely be concurrent image generation, which might be playing a role in the issue I'm having there with increased memory. With the test code though I'm just running the script many times in sequence. Thanks, Jared ___ Gimp-developer mailing list Gimp-developer@lists.xcf.berkeley.edu http://lists.xcf.berkeley.edu/mailman/listinfo/gimp-developer
Re: [Gimp-developer] Gimp_image_delete and adding text layers
On 8/15/05, Jared Whiting [EMAIL PROTECTED] wrote: machine because in the process of creating images the GIMP keeps holding on to more and more memory. As an example, I can run the following $img-gimp_image_undo_enable; gimp_image_delete($img); Gimp::end; the GIMP instance grabs on to more memory and holds on to it permanently. Deleting the image and disabling undo do not result in the GIMP releasing memory for the new layer it created. I would prefer not What about putting $img = undef; at the end -- does this change anything? Also, in the Perl Docs, perlfaq3: = How can I free an array or hash so my program shrinks? (contributed by Michael Carman) You usually can't. Memory allocated to lexicals (i.e. my() variables) cannot be reclaimed or reused even if they go out of scope. It is reserved in case the variables come back into scope. Memory allocated to global variables can be reused (within your program) by using undef()ing and/or delete(). On most operating systems, memory allocated to a program can never be returned to the system. That's why long-running programs sometimes re- exec themselves. Some operating systems (notably, systems that use mmap(2) for allocating large chunks of memory) can reclaim memory that is no longer used, but on such systems, perl must be configured and compiled to use the OS's malloc, not perl's. In general, memory allocation and de-allocation isn't something you can or should be worrying about much in Perl. See also ``How can I make my Perl program take less memory?'' = How can I make my Perl program take less memory? When it comes to time-space tradeoffs, Perl nearly always prefers to throw memory at a problem. Scalars in Perl use more memory than strings in C, arrays take more than that, ... [I suggest you read the entirety of this question in the actual FAQ, it might be helpful.] = Hope this helps. -- ~Mike - Just my two cents - No man is an island, and no man is unable. ___ Gimp-developer mailing list Gimp-developer@lists.xcf.berkeley.edu http://lists.xcf.berkeley.edu/mailman/listinfo/gimp-developer
Re: [Gimp-developer] Gimp_image_delete and adding text layers
Hi, Jared Whiting [EMAIL PROTECTED] writes: I am currently using GIMP 2.8 for GNU/Linux, There is no such GIMP version (yet). What version are you really using? Sven ___ Gimp-developer mailing list Gimp-developer@lists.xcf.berkeley.edu http://lists.xcf.berkeley.edu/mailman/listinfo/gimp-developer
RE: [Gimp-developer] Gimp_image_delete and adding text layers
... meant to write GIMP 2.2.8 -Original Message- From: Sven Neumann [mailto:[EMAIL PROTECTED] On Behalf Of Sven Neumann Sent: Tuesday, August 16, 2005 2:09 PM To: Jared Whiting Cc: gimp-developer@lists.xcf.berkeley.edu Subject: Re: [Gimp-developer] Gimp_image_delete and adding text layers Hi, Jared Whiting [EMAIL PROTECTED] writes: I am currently using GIMP 2.8 for GNU/Linux, There is no such GIMP version (yet). What version are you really using? Sven ___ Gimp-developer mailing list Gimp-developer@lists.xcf.berkeley.edu http://lists.xcf.berkeley.edu/mailman/listinfo/gimp-developer
Re: [Gimp-developer] Gimp_image_delete and adding text layers
Hi, Jared Whiting [EMAIL PROTECTED] writes: ... meant to write GIMP 2.2.8 Ok, your script actually looks as if it does the right thing, at least on first sight. However, since we have run gimp-2.2 in a memory profiler which didn't show any memory leaks, I am not yet convinced that there's a bug in gimp. Might be gimp-perl or your script. Or gimp. We will have to find out... How much memory increase do you see per image? Can you run this script a couple of times and show us memory usage before and after? Sven ___ Gimp-developer mailing list Gimp-developer@lists.xcf.berkeley.edu http://lists.xcf.berkeley.edu/mailman/listinfo/gimp-developer
