Greetings!
I think that GCL does free up memory for loaded .o files to which no
live function still refers, at least when doing full gbc. Here is
sample with comments:
=============================================================================
/tmp/ll.l
=============================================================================
(defun foo (x) x)
(defun bar (x) (foo x))
=============================================================================
GCL (GNU Common Lisp) 2.6.7 CLtL1 Oct 29 2006 02:32:45
Source License: LGPL(gcl,gmp), GPL(unexec,bfd,xgcl)
Binary License: GPL due to GPL'ed components: (XGCL READLINE BFD UNEXEC)
Modifications of this banner must retain notice of a compatible license
Dedicated to the memory of W. Schelter
Use (help) to get some basic information on how to use GCL.
Temporary directory for compiler files set to /tmp/
>(compile-file "/tmp/ll.l")
Compiling /tmp/ll.l.
End of Pass 1.
End of Pass 2.
OPTIMIZE levels: Safety=0 (No runtime error checking), Space=0, Speed=3
Finished compiling /tmp/ll.l.
#p"/tmp/ll.o"
>(si::gbc t)
T
>(room)
211/211 47.6% CONS RATIO LONG-FLOAT COMPLEX STRUCTURE
4/28 64.4% FIXNUM SHORT-FLOAT CHARACTER RANDOM-STATE READTABLE
SPICE
76/404 85.3% SYMBOL STREAM
1/2 14.1% PACKAGE
2/38 30.8% ARRAY HASH-TABLE VECTOR BIT-VECTOR PATHNAME
CCLOSURE CLOSURE
20/32 92.2% STRING
11/27 91.5% CFUN BIGNUM
6/115 89.3% SFUN GFUN VFUN AFUN CFDATA
464/512 1 contiguous (120 blocks) ;; *** .o files
;; take up one block of
contiguous space, which may or may not
;; span multiple pages
13107 hole
5242 0.0% relocatable
331 pages for cells
19144 total pages
101481 pages available
10447 pages in heap but not gc'd + pages needed for gc marking
131072 maximum pages
>(load "/tmp/ll")
Loading /tmp/ll.o
start address -T 0x849be90 Finished loading /tmp/ll.o
228
>(si::gbc t)
T
>(room)
211/211 47.6% CONS RATIO LONG-FLOAT COMPLEX STRUCTURE
4/28 64.5% FIXNUM SHORT-FLOAT CHARACTER RANDOM-STATE READTABLE
SPICE
76/404 85.3% SYMBOL STREAM
1/2 14.1% PACKAGE
2/38 30.5% ARRAY HASH-TABLE VECTOR BIT-VECTOR PATHNAME
CCLOSURE CLOSURE
20/32 92.1% STRING
11/27 91.6% CFUN BIGNUM
6/115 89.4% SFUN GFUN VFUN AFUN CFDATA
464/512 2 contiguous (121 blocks) ;; *** block 121 is for ll.o
13107 hole
5242 0.0% relocatable
331 pages for cells
19144 total pages
101481 pages available
10447 pages in heap but not gc'd + pages needed for gc marking
131072 maximum pages
>(load "/tmp/ll.l") ;; *** This will remove function references to ll.o
Loading /tmp/ll.l
Finished loading /tmp/ll.l
T
>(si::gbc t)
T
>(room)
211/211 47.6% CONS RATIO LONG-FLOAT COMPLEX STRUCTURE
4/28 64.5% FIXNUM SHORT-FLOAT CHARACTER RANDOM-STATE READTABLE
SPICE
76/404 85.3% SYMBOL STREAM
1/2 14.1% PACKAGE
2/38 30.5% ARRAY HASH-TABLE VECTOR BIT-VECTOR PATHNAME
CCLOSURE CLOSURE
20/32 92.1% STRING
11/27 91.5% CFUN BIGNUM
6/115 89.3% SFUN GFUN VFUN AFUN CFDATA
464/512 3 contiguous (120 blocks) ;; *** block 121 now
available
13107 hole
5242 0.0% relocatable
331 pages for cells
19144 total pages
101481 pages available
10447 pages in heap but not gc'd + pages needed for gc marking
131072 maximum pages
>
=============================================================================
Now it is true that repeated allocations and deallocations of
contiguous space might lead to memory fragmentation, and hence greater
usage. This is also the slowest memory GCL has. It is managed as a
list of in-place free blocks sorted by available size.
SGC takes a given snapshot of the memory and marks it read only. If
subsequently written to, the page will be markes read-write. SGC will
then only mark and collect writable pages. So your old .o files, if
in the read-only set, will be there until a full GC is done. You saw
this effect with the cons usage you note. Unfortunately, the
contiguous blocks reported in sgc are the writable ones only, it
appears, so you cannot see what had been freed there unless you did a
(room) right after (si::sgc-on nil)
Matt Kaufmann <[EMAIL PROTECTED]> writes:
> Hi, Camm --
>
> We recently ran out of space in a GCL run (actually ACL2 3.2 built on
> top of GCL 2.6.7 on 32-bit linux), and I think I know why (and I think
> we have a workaround in this case). In ACL2, a typical "certify-book"
> operation goes like this:
>
> 1. Load a bunch of .o files and do a bunch of proofs and stuff.
> 2. Do an fmakunbound for each function defined in one of those .o
> files (among other things).
> 3. Load that same bunch of .o files again (and do other stuff).
>
> My understanding is that GCL doesn't free up memory for the .o files
> in Step 1. (If I'm wrong, then I'm surprised by what I'm seeing with
> (room).) Perhaps there's an easy way for you to free up such memory?
> I know almost nothing about GCL internals, so maybe what I'm about to
> say is off-base, but I can imagine this simple hack working:
>
> Remember the start address A when a .o file is loaded. If that file
> is loaded again later, then compare the current file contents to
> what is currently in memory starting at A. If these are the same,
> then don't bother loading the .o file.
>
> Sorry if that's a dumb idea, but anyhow I'm just looking for away to
> avoid running out of space in some cases.
>
I'd be interested if we could collect some evidence of possible memory
fragmentation.
> OR: Maybe I should just turn off sgc and run gbc after Step 2 above?
Yes.
> Or maybe it's worth automating that in GCL when space is getting low?
Perhaps. Right now, if the heap overruns the hole and requires adding
more pages to the core image, a 't_relocatable gc will be triggered,
which turns off sgc, does the gc, and turns sgc back on.
Looking at the below, you might get better mileage via judicious
toggling of si::*optimize-maximum-pages*, and perhaps a smaller
hole-size. These of course slow gc down but yield a more compat
image.
Please let me know if problems persist.
Take care,
> Here's an interesting log. Notice that the si::gbc doesn't do any
> good (because I had just done one), but if I turn off sgc and try
> again, then the percent of space used in CONS pages goes way down.
>
> ACL2>(room)
>
> 110074/110074 48.4% 231CONS RATIO LONG-FLOAT COMPLEX STRUCTURE
> 7382/7382 2.6% 33 FIXNUM SHORT-FLOAT CHARACTER RANDOM-STATE
> READTABLE SPICE
> 1616/2587 46.3% 6 SYMBOL STREAM
> 2/2 66.7% 1 PACKAGE
> 548/902 0.2% 2 ARRAY HASH-TABLE VECTOR BIT-VECTOR PATHNAME
> CCLOSURE CLOSURE
> 1677/2530 25.9% 5 STRING
> 12788/12788 0.1% 51 CFUN BIGNUM
> 79/157 85.4% SFUN GFUN VFUN AFUN CFDATA
>
> 4857/5544 2 contiguous (28 blocks)
> 28680 hole
> 27414 0.1% 46 relocatable
>
> 134166 pages for cells
> 195117 total pages
> 5516 pages available
> 61511 pages in heap but not gc'd + pages needed for gc marking
> 262144 maximum pages
>
> ACL2>(si::gbc t)
> [SGC for 511 CONTIGUOUS-BLOCKS pages..(138792 writable)..(T=143).GC
> finished]
>
> T
>
> ACL2>(room)
>
> 110074/110074 48.4% 231CONS RATIO LONG-FLOAT COMPLEX STRUCTURE
> 7382/7382 2.6% 33 FIXNUM SHORT-FLOAT CHARACTER RANDOM-STATE
> READTABLE SPICE
> 1616/2587 46.3% 6 SYMBOL STREAM
> 2/2 66.7% 1 PACKAGE
> 548/902 0.2% 2 ARRAY HASH-TABLE VECTOR BIT-VECTOR PATHNAME
> CCLOSURE CLOSURE
> 1677/2530 25.9% 5 STRING
> 12788/12788 0.1% 51 CFUN BIGNUM
> 79/157 85.4% SFUN GFUN VFUN AFUN CFDATA
>
> 4857/5544 3 contiguous (28 blocks)
> 28680 hole
> 27414 0.1% 46 relocatable
>
> 134166 pages for cells
> 195117 total pages
> 5516 pages available
> 61511 pages in heap but not gc'd + pages needed for gc marking
> 262144 maximum pages
>
> ACL2>(si::sgc-on nil)
> [SGC off]
> NIL
>
> ACL2>(si::gbc t)
> [GC for 4857 CONTIGUOUS-BLOCKS pages..(T=77).GC finished]
>
> T
>
> ACL2>(room)
>
> 110074/110074 16.7% 231CONS RATIO LONG-FLOAT COMPLEX STRUCTURE
> 7382/7382 1.3% 33 FIXNUM SHORT-FLOAT CHARACTER RANDOM-STATE
> READTABLE SPICE
> 1616/2587 46.3% 6 SYMBOL STREAM
> 2/2 66.7% 1 PACKAGE
> 548/902 0.1% 2 ARRAY HASH-TABLE VECTOR BIT-VECTOR PATHNAME
> CCLOSURE CLOSURE
> 1677/2530 19.7% 5 STRING
> 12788/12788 0.1% 51 CFUN BIGNUM
> 79/157 76.6% SFUN GFUN VFUN AFUN CFDATA
>
> 4857/5544 4 contiguous (2911 blocks)
> 52428 hole
> 27414 2.3% 46 relocatable
>
> 134166 pages for cells
> 218865 total pages
> 5516 pages available
> 37763 pages in heap but not gc'd + pages needed for gc marking
> 262144 maximum pages
>
> ACL2>
>
> Thanks --
> -- Matt
>
>
>
--
Camm Maguire [EMAIL PROTECTED]
==========================================================================
"The earth is but one country, and mankind its citizens." -- Baha'u'llah
_______________________________________________
Gcl-devel mailing list
Gcl-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/gcl-devel
