GC asserts and threads
Hi there, I'm debugging an issue here that cause the GC asserts to trigger, with the values compared being off by one. The problem disappears when I compile --without-threads. The program does not explicitly create threads Is there any code in GUILE that would create a thread (possibly leading to race conditions) when there is no explicit start-thread call in the code? The program (lilypond) does run through the regular GUILE boot procedure. -- Han-Wen Nienhuys - [EMAIL PROTECTED] - http://www.xs4all.nl/~hanwen
vm branch now uses vm repl by default
Hi, I've enabled the VM repl by default on the vm branch. Here's a brief annotated tour: $ ./pre-inst-guile Guile Scheme interpreter 0.5 on Guile 1.9.0 Copyright (C) 2001-2008 Free Software Foundation, Inc. Enter `,help' for help. A very pleasant introduction, no? And totally configurable with a nice programming interface. scheme@(guile-user) 'foo $1 = foo A normal repl. The `scheme' indicates the current language, and (guile-user) is the current module. The $1 = foo is from the history module. If you don't have this in your ~/.guile, you really really want it: (use-modules (ice-9 readline) (ice-9 history)) (activate-readline) Anyway, moving on: scheme@(guile-user) (lambda () (pk a #:bar)) $2 = #program b755ecf8 Entering in expressions actually compiles and executes them. In this case we compiled and loaded a thunk. Compiled procedures are programs, and print as such. scheme@(guile-user) (define a '(a . pair)) scheme@(guile-user) ($2) ;;; ((a . pair) #:bar) $3 = #:bar Procedures resolve toplevel bindings lazily, as in the interpreter, so you get letrec semantics in the repl. scheme@(guile-user) ,x $2 There is a wealth of meta-commands at the repl, commands that start with `,'. This command, `,x', is an abbreviation for `,disassemble'. Its output is this: Disassembly of #program b755ecf8: nargs = 0 nrest = 0 nlocs = 0 nexts = 0 The program has no arguments, no rest arguments, no local variables, and no external (lexically-bound) variables. Bytecode: 0(late-variable-ref 0) 2(late-variable-ref 1) 4(object-ref 2) ;; #:bar 6(tail-call 2) Objects: 0#variable b80057f0 value: #program b8005858 1#variable b7569af0 value: (a . pair) 2#:bar Late-variable-ref looks at a cell in the object vector. If it is a symbol, it is resolved relative to the module that was current when the program was made. The object cell is then replaced with the resulting resolved variable. Here we see that objects 0 and 1 were already resolved. Object 2 is just the constant, #:bar. All of the ref instructions push their values on the stack. Call instructions pop off arguments, if any, then call the program on the top of the stack. In this case it is a tail call. Sources: 8#(1 11 #f) Some instructions are annotated with source information. In this case, when the instruction pointer is at 8 (right after the tail-call -- one byte for tail-call and one for the number of arguments, 2), the original source was at line 1 and column 11 in an unnamed port (stdin in this case). scheme@(guile-user) ,option interp #t scheme@(guile-user) ,option trace #f interp #t Here we tell the repl that, given the option, we prefer to interpret rather than compile. Of course, if the current language doesn't support compilation, we always interpret. scheme@(guile-user) (lambda () (pk a #:bar)) $4 = #procedure #f () An interpreted procedure, like in olden times. Happy hacking! Andy -- http://wingolog.org/
Re: GC asserts and threads
On Tue 09 Sep 2008 07:58, Han-Wen Nienhuys [EMAIL PROTECTED] writes: Is there any code in GUILE that would create a thread (possibly leading to race conditions) when there is no explicit start-thread call in the code? The program (lilypond) does run through the regular GUILE boot procedure. Yep, when compiled with threads, guile spawns a separate thread to handle signals. Andy -- http://wingolog.org/
Re: development goals
Hi, On Mon 08 Sep 2008 12:16, [EMAIL PROTECTED] (Ludovic Courtès) writes: Han-Wen Nienhuys [EMAIL PROTECTED] writes: - Nobody has enough initiative to put a single strategic #ifdef in the code. To me, it looks like the strategic #if 0 was a way of admitting that we know our code is broken, we don't know why, and we don't want to investigate that ATM but we might eventually do that if we have time. So no, I didn't feel that happy with this. I was not happy with this either. - If someone finally does take initiative, it's only ok if it is perfect. It's great that you're working on this stuff! But it should have been pushed to a branch. The thing is, it really *does* need to be perfect. Maybe not at first push, but within a couple weeks of being merged to mainline. My fear was that since Guile hackers have so little time, in general, that you would push something broken in corner cases, then walk off -- not a concern specific to any one person. It's good that you are sticking around to iron out the bugs. Andy -- http://wingolog.org/
Re: [PATCH] Avoid `SCM_VALIDATE_LIST ()'
Hi Neil, Neil Jerram [EMAIL PROTECTED] writes: Snapshots as of now are up at http://www.ossau.uklinux.net/guile/snapshots, and I hope they will keep automagically appearing there. Looks good! I think you can already update the web page since the current link is broken anyway. Thanks! Ludo'.
Re: vm branch now uses vm repl by default
Hey! Andy Wingo [EMAIL PROTECTED] writes: I've enabled the VM repl by default on the vm branch. Here's a brief annotated tour: $ ./pre-inst-guile Guile Scheme interpreter 0.5 on Guile 1.9.0 Copyright (C) 2001-2008 Free Software Foundation, Inc. Enter `,help' for help. Cool! Late-variable-ref looks at a cell in the object vector. If it is a symbol, it is resolved relative to the module that was current when the program was made. The object cell is then replaced with the resulting resolved variable. Here we see that objects 0 and 1 were already resolved. Object 2 is just the constant, #:bar. Aaah, nice! So, previously, there was `variable-ref': -- Instruction: variable-ref Dereference the variable object which is on top of the stack and replace it by the value of the variable it represents. Now, there's also a vector associated with each closure to store references to global variables, right? Looks better! (Hint: the doc is outdated. :-)) scheme@(guile-user) ,option interp #t scheme@(guile-user) ,option trace #f interp#t Here we tell the repl that, given the option, we prefer to interpret rather than compile. Of course, if the current language doesn't support compilation, we always interpret. That means good old `CEVAL ()' is used, right? When that is the case, one can still use `{eval,debug}-options', right? It'd be nice if we could find a way to do something with the `current-reader' fluid at compilation time, like detecting top-level `(fluid-set! current-reader ...)' statements and use that to switch the compiler's reader (hacky...). Thanks for the good news! Ludo'.
Re: GC asserts and threads
Andy Wingo escreveu: On Tue 09 Sep 2008 07:58, Han-Wen Nienhuys [EMAIL PROTECTED] writes: Is there any code in GUILE that would create a thread (possibly leading to race conditions) when there is no explicit start-thread call in the code? The program (lilypond) does run through the regular GUILE boot procedure. Yep, when compiled with threads, guile spawns a separate thread to handle signals. but I am only seeing one $ guile guile (all-threads) (#thread 3086285552 (8208008)) Andy -- Han-Wen Nienhuys - [EMAIL PROTECTED] - http://www.xs4all.nl/~hanwen
Cleanup mark-during-GC debug checks.
Reviewers: hanwenn, Message: Hello guile devel, please go to http://codereview.appspot.com/4847 to review this patch. I hope you like it; thanks! Please review this at http://codereview.appspot.com/4847 Affected files: M libguile/__scm.h M libguile/gc-mark.c M libguile/gc.c M libguile/gc.h M libguile/inline.h
Getting rid of maintainer mode
Hello, Is anyone against getting rid of `AM_MAINTAINER_MODE'? If in doubt, see (info (automake) maintainer-mode). :-) Thanks, Ludo'.
Re: GC asserts and threads
2008/9/9 Han-Wen Nienhuys [EMAIL PROTECTED]: On Tue, Sep 9, 2008 at 4:00 AM, Andy Wingo [EMAIL PROTECTED] wrote: On Tue 09 Sep 2008 07:58, Han-Wen Nienhuys [EMAIL PROTECTED] writes: Is there any code in GUILE that would create a thread (possibly leading to race conditions) when there is no explicit start-thread call in the code? The program (lilypond) does run through the regular GUILE boot procedure. Yep, when compiled with threads, guile spawns a separate thread to handle signals. Where does that happen? Look for ensure_signal_delivery_thread in scmsigs.c. Neil
Re: Getting rid of maintainer mode
Ludovic Courtès escreveu: Hello, Is anyone against getting rid of `AM_MAINTAINER_MODE'? If in doubt, see (info (automake) maintainer-mode). :-) Getting rid of anything that starts with AM_ has my full support. -- Han-Wen Nienhuys - [EMAIL PROTECTED] - http://www.xs4all.nl/~hanwen
Re: Getting rid of maintainer mode
2008/9/9 Ludovic Courtès [EMAIL PROTECTED]: Hello, Is anyone against getting rid of `AM_MAINTAINER_MODE'? No, please go ahead. Neil
Re: Cleanup mark-during-GC debug checks.
Hello! [EMAIL PROTECTED] writes: Reviewers: hanwenn, Message: Hello guile devel, please go to http://codereview.appspot.com/4847 to review this patch. I hope you like it; thanks! A couple of notes: 1. I don't want to use a web interface to review code. Most free software projects use email in one form or another, which I find convenient. Having patches in-lined is optimal IMO. 2. I don't want to have a Google account. Thus, I'll comment on the patch here. * I'd name the macro `SCM_DEBUG_MARK_PHASE' rather, as it sounds mot idiomatic (but I'm not a native speaker). * Use static const char msg[] = Other than that, I'm OK to commit it. Thanks, Ludo'.
Re: [PATCH] Use Gnulib's `strftime'
Hi, [EMAIL PROTECTED] (Ludovic Courtès) writes: I'm planning to use Gnulib's `strftime' module on `master' to fix portability problems related to `strftime', aka. #24130 (https://savannah.gnu.org/bugs/?24130). The good thing is that `strftime' will now work the same regardless of the underlying libc. The source modification is attached. For a discussion of `nstrftime ()', see http://thread.gmane.org/gmane.comp.lib.gnulib.bugs/14380 . Please let me know if you think of a good reason to not do this, otherwise I'll commit it within a week or so. Applied. Thanks, Ludo'.
Re: GUILE_MAX_HEAP_SIZE
Hi, [EMAIL PROTECTED] (Ludovic Courtès) writes: Han-Wen Nienhuys [EMAIL PROTECTED] writes: Han-Wen Nienhuys escreveu: Ludovic Courtès escreveu: +/* + Classic MIT Hack, see e.g. http://www.tekpool.com/?cat=9 + */ +int scm_i_uint_bit_count(unsigned int u) (BTW, it'd make sense to use Gnulib's `count-one-bits' module, which is able to use GCC's `__builtin_popcount ()'.) Could you add the gnulib module? I'll do the rest. I asked for re-licensing: http://thread.gmane.org/gmane.comp.lib.gnulib.bugs/14349 Once that is done, all you need is to add it to `m4/gnulib-cache.m4', run gnulib-tool --update, commit the module changes and additions, and hack the thing. You can post the patches before committing, too. ;-) I just did it (patch attached). Thanks, Ludo'. From a8db4a59c898598cc55dd3bd86a6fd8618721d10 Mon Sep 17 00:00:00 2001 From: =?utf-8?q?Ludovic=20Court=C3=A8s?= [EMAIL PROTECTED] Date: Tue, 9 Sep 2008 22:46:04 +0200 Subject: [PATCH] Use Gnulib's `count-one-bits' as a replacement for `scm_i_uint_bit_count ()'. * libguile/gc-card.c: Include config.h and count-one-bits.h. (scm_i_uint_bit_count): Remove. (scm_i_card_marked_count): Use `count_one_bits_l ()' instead of `scm_i_uint_bit_count ()'. * libguile/gc-segment.c: Include config.h and count-one-bits.h. (scm_i_heap_segment_marked_count): Use `count_one_bits_l ()' instead of `scm_i_uint_bit_count ()'. * libguile/private-gc.h (scm_i_uint_bit_count): Remove. --- libguile/gc-card.c| 23 --- libguile/gc-segment.c | 10 -- libguile/private-gc.h |1 - 3 files changed, 16 insertions(+), 18 deletions(-) diff --git a/libguile/gc-card.c b/libguile/gc-card.c index 3511533..93e271a 100644 --- a/libguile/gc-card.c +++ b/libguile/gc-card.c @@ -1,4 +1,4 @@ -/* Copyright (C) 1995,1996,1997,1998,1999,2000,2001, 2002, 2004, 2005, 2006, 2007 Free Software Foundation, Inc. +/* Copyright (C) 1995,1996,1997,1998,1999,2000,2001, 2002, 2004, 2005, 2006, 2007, 2008 Free Software Foundation, Inc. * * This library is free software; you can redistribute it and/or * modify it under the terms of the GNU Lesser General Public @@ -15,8 +15,14 @@ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA */ +#if HAVE_CONFIG_H +# include config.h +#endif + #include assert.h #include stdio.h +#include count-one-bits.h + #include gmp.h #include libguile/_scm.h @@ -294,19 +300,6 @@ scm_i_init_card_freelist (scm_t_cell *card, SCM *free_list, } /* - Classic MIT Hack, see e.g. http://www.tekpool.com/?cat=9 - */ -int scm_i_uint_bit_count (unsigned int u) -{ - unsigned int u_count = u -- ((u 1) 0333) -- ((u 2) 0111); - return -((u_count + (u_count 3)) - 030707070707) % 63; -} - -/* Amount of cells marked in this cell, measured in 1-cells. */ int @@ -318,7 +311,7 @@ scm_i_card_marked_count (scm_t_cell *card, int span) int count = 0; while (bvec bvec_end) { - count += scm_i_uint_bit_count (*bvec); + count += count_one_bits_l (*bvec); bvec ++; } return count * span; diff --git a/libguile/gc-segment.c b/libguile/gc-segment.c index 4f7b6d5..f53ec96 100644 --- a/libguile/gc-segment.c +++ b/libguile/gc-segment.c @@ -1,4 +1,4 @@ -/* Copyright (C) 1995,1996,1997,1998,1999,2000,2001, 2002, 2006 Free Software Foundation, Inc. +/* Copyright (C) 1995,1996,1997,1998,1999,2000,2001, 2002, 2006, 2008 Free Software Foundation, Inc. * * This library is free software; you can redistribute it and/or * modify it under the terms of the GNU Lesser General Public @@ -15,10 +15,16 @@ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA */ +#if HAVE_CONFIG_H +# include config.h +#endif + #include assert.h #include stdio.h #include string.h +#include count-one-bits.h + #include libguile/_scm.h #include libguile/pairs.h #include libguile/gc.h @@ -109,7 +115,7 @@ scm_i_heap_segment_marked_count (scm_t_heap_segment *seg) int count = 0; while (bvec bvec_end) { - count += scm_i_uint_bit_count (*bvec); + count += count_one_bits_l (*bvec); bvec ++; } return count * seg-span; diff --git a/libguile/private-gc.h b/libguile/private-gc.h index d738665..93503ce 100644 --- a/libguile/private-gc.h +++ b/libguile/private-gc.h @@ -78,7 +78,6 @@ #define SCM_GC_IN_CARD_HEADERP(x) \ (scm_t_cell *) (x) SCM_GC_CELL_CARD (x) + SCM_GC_CARD_N_HEADER_CELLS -int scm_i_uint_bit_count (unsigned int u); int scm_getenv_int (const char *var, int def); -- 1.6.0
Re: vm branch now uses vm repl by default
Hello! Andy Wingo [EMAIL PROTECTED] writes: On Tue 09 Sep 2008 10:41, [EMAIL PROTECTED] (Ludovic Courtès) writes: So, previously, there was `variable-ref': There still is. It is used when a variable is bound immediately, when it is pushed on the stack by a link-now instruction. OK, I see. Now, there's also a vector associated with each closure to store references to global variables, right? Looks better! That's always been the case IIRC, only before it used to push and pop a bit more -- instead of (late-variable-ref 0) it would be (object-ref 0) (variable-ref) Oh, right. It'd be nice if we could find a way to do something with the `current-reader' fluid at compilation time, like detecting top-level `(fluid-set! current-reader ...)' statements and use that to switch the compiler's reader (hacky...). Perhaps, there is already a repl-reader fluid for readline's benefit. Note also that languages have readers as well, so that e.g. elisp can read differently from scheme. Right, but `current-reader' is a dynamic thing, which complicates the situation. A use case is the following: (define-module (foo)) (fluid-set! current-reader %my-favorite-reader) ;; use non-standard syntax extensions from now on I use it this way in Skribilo, but I may well be the only user, who knows. ;-) Anyway, if we are to handle this at all, we're probably going to have to pattern-match this in `translate.scm' and switch readers when we encounter it. Thanks, Ludo'.
Re: vm branch now uses vm repl by default
2008/9/9 Andy Wingo [EMAIL PROTECTED]: Sources: 8#(1 11 #f) Some instructions are annotated with source information. I guess source information is of interest for debugging, and I think you've observed previously elsewhere that the VM doesn't yet have much debugging support - by which I presume you mean something like the traps that the evaluator has. So I was just wondering if we actually _need_ any debugging support in the VM. If we can assume that the VM will always behave equivalently to the evaluator (except faster), then whenever a piece of code needs step-by-step debugging, the developer can drop back to using the evaluator in order to do that. Does that make sense? (Then another question is whether we can assume that the VM behaves equivalently to the evaluator. I wonder if there is some test methodology for partly proving this, by automatically running the evaluator in parallel with the VM, at least for code that doesn't have side effects? I'm sorry, this email has ended up a bit vague) Neil
Re: GUILE_MAX_HEAP_SIZE
Ludovic Courtès escreveu: Hi, [EMAIL PROTECTED] (Ludovic Courtès) writes: Han-Wen Nienhuys [EMAIL PROTECTED] writes: Han-Wen Nienhuys escreveu: Ludovic Courtès escreveu: +/* + Classic MIT Hack, see e.g. http://www.tekpool.com/?cat=9 + */ +int scm_i_uint_bit_count(unsigned int u) (BTW, it'd make sense to use Gnulib's `count-one-bits' module, which is able to use GCC's `__builtin_popcount ()'.) Could you add the gnulib module? I'll do the rest. I asked for re-licensing: http://thread.gmane.org/gmane.comp.lib.gnulib.bugs/14349 Once that is done, all you need is to add it to `m4/gnulib-cache.m4', run gnulib-tool --update, commit the module changes and additions, and hack the thing. You can post the patches before committing, too. ;-) I just did it (patch attached). Thanks. I'm confused though, commit 53f4876abcebf3f05d2a88bba3a898ddcda25a74 Merge: 69f2317... 242ebea... Author: Ludovic Courtès [EMAIL PROTECTED] Date: Tue Sep 9 22:03:42 2008 +0200 Merge branch 'master' into strftime-gnulib Conflicts: libguile/ChangeLog srfi/ChangeLog test-suite/ChangeLog I thought we were supposed to keep the history linear; did I miss something? -- Han-Wen Nienhuys - [EMAIL PROTECTED] - http://www.xs4all.nl/~hanwen
Re: i18n broken on mingw cross compile
Ludovic Courtès escreveu: Hi, Han-Wen Nienhuys [EMAIL PROTECTED] writes: i686-mingw32-gcc -mms-bitfields -DHAVE_CONFIG_H -I/home/lilydev/vc/gub/target/mingw/src/guile-1.9.git -I.. -I/home/lilydev/vc/gub/target/mingw/src/guile-1.9.git/lib -I../lib -Wall -Wmissing-prototypes -g -O2 -MT libguile_i18n_v_0_la-i18n.lo -MD -MP -MF .deps/libguile_i18n_v_0_la-i18n.Tpo -c /home/lilydev/vc/gub/target/mingw/src/guile-1.9.git/libguile/i18n.c -DDLL_EXPORT -DPIC -o .libs/libguile_i18n_v_0_la-i18n.o In file included from /home/lilydev/vc/gub/target/mingw/src/guile-1.9.git/libguile/i18n.c:296: /home/lilydev/vc/gub/target/mingw/src/guile-1.9.git/libguile/locale-categories.h: In function 'get_current_locale_settings': /home/lilydev/vc/gub/target/mingw/src/guile-1.9.git/libguile/locale-categories.h:24: error: 'LC_MESSAGES' undeclared (first use in this function) /home/lilydev/vc/gub/target/mingw/src/guile-1.9.git/libguile/locale-categories.h:24: error: (Each undeclared identifier is reported only once /home/lilydev/vc/gub/target/mingw/src/guile-1.9.git/libguile/locale-categories.h:24: error: for each function it appears in.) Can you try out the attached patch? Could you publish your changes through savannah as well? This makes checking the patch a lot easier. For example, you could push into a dev/ludo branch. -- Han-Wen Nienhuys - [EMAIL PROTECTED] - http://www.xs4all.nl/~hanwen
[PATCH] Revise GC asserts.
* libguile/gc.c (scm_i_gc): Change assert into deprecation warning. * libguile/private-gc.h (nil): introduce scm_i_last_marked_cell_count, as a private mechanism for maintaining cell counts. Previous versions incremented scm_cells_allocated in an inlined function, so loading dynamic objects of older GUILEs would break invariants. --- libguile/gc.c | 18 +- libguile/private-gc.h |5 +++-- 2 files changed, 16 insertions(+), 7 deletions(-) diff --git a/libguile/gc.c b/libguile/gc.c index f3ef585..0323f87 100644 --- a/libguile/gc.c +++ b/libguile/gc.c @@ -416,7 +416,7 @@ gc_end_stats () scm_gc_cells_allocated_acc += (double) scm_i_gc_sweep_stats.collected; - scm_gc_cells_marked_acc += (double) scm_cells_allocated; + scm_gc_cells_marked_acc += (double) scm_i_last_marked_cell_count; scm_gc_cells_marked_conservatively_acc += (double) scm_i_find_heap_calls; scm_gc_cells_swept_acc += (double) scm_i_gc_sweep_stats.swept; @@ -558,6 +558,8 @@ scm_check_deprecated_memory_return () scm_i_deprecated_memory_return = 0; } +long int scm_i_last_marked_cell_count; + /* Must be called while holding scm_i_sweep_mutex. This function is fairly long, but it touches various global @@ -603,11 +605,17 @@ scm_i_gc (const char *what) /* TODO(hanwen): figure out why the stats are off on x64_64. */ /* If this was not true, someone touched mark bits outside of the mark phase. */ - assert (scm_cells_allocated == scm_i_marked_count ()); + if (scm_i_last_marked_cell_count != scm_i_marked_count ()) +{ + static char msg[] = + The number of marked objects changed since the last GC. + Are you marking objects outside of the mark phase?; + scm_c_issue_deprecation_warning(msg); +} assert (scm_i_gc_sweep_stats.swept == (scm_i_master_freelist.heap_total_cells + scm_i_master_freelist2.heap_total_cells)); - assert (scm_i_gc_sweep_stats.collected + scm_cells_allocated + assert (scm_i_gc_sweep_stats.collected + scm_i_last_marked_cell_count == scm_i_gc_sweep_stats.swept); #endif /* SCM_DEBUG_CELL_ACCESSES */ @@ -617,8 +625,8 @@ scm_i_gc (const char *what) scm_mark_all (); scm_gc_mark_time_taken += (scm_c_get_internal_run_time () - t_before_gc); - scm_cells_allocated = scm_i_marked_count (); - + scm_i_last_marked_cell_count = scm_cells_allocated = scm_i_marked_count (); + /* Sweep TODO: the after_sweep hook should probably be moved to just before diff --git a/libguile/private-gc.h b/libguile/private-gc.h index 93503ce..f5331ab 100644 --- a/libguile/private-gc.h +++ b/libguile/private-gc.h @@ -273,8 +273,9 @@ SCM_INTERNAL void scm_i_sweep_all_segments (char const *reason, SCM_INTERNAL SCM scm_i_all_segments_statistics (SCM hashtab); SCM_INTERNAL unsigned long *scm_i_segment_table_info(int *size); -extern long int scm_i_deprecated_memory_return; -extern long int scm_i_find_heap_calls; +SCM_INTERNAL long int scm_i_deprecated_memory_return; +SCM_INTERNAL long int scm_i_find_heap_calls; +SCM_INTERNAL long int scm_i_last_marked_cell_count; /* global init funcs. -- 1.5.5.1