Fixing `flush-output-port` for R6RS transcoded ports

[Andreas Rottmann]

Hi!

As I reported in #39610 [0], `flush-output-port`, aka `force-output` is
no longer working for R6RS transcoded ports. It was still working in
Guile 2.0; the regression was introduced in Guile 2.2, via commit
"8399e7af5 ("Generic port facility provides buffering uniformly").

I'd be interested in fixing this, but I'm unclear how to tackle it. From
a glance at the current code, it seems to me that there is no mechanism
in place that can be used to achieve that, as the `flush` method for
port types is no more, and I see no way to pass the `flush` operation on
transcoded port to the port it is wrapping. My gut approach would be to
re-introduce the `flush` operation, but I'm not sure that would be
sensible and/or fit into the re-architected ports.

If someone, perhaps @wingo (being the author of the re-factoring), could
sketch an implementation strategy (or just affirm that re-introducing
`flush` is sensible), I would be grateful, and would be one step further
in resurrecting my R6RS Scheme code after almost a decade of abandon.

[0] https://debbugs.gnu.org/cgi/bugreport.cgi?bug=39610

Kind Regards, Rotty

Re: Reading data from a file descriptor

[Andreas Rottmann]

 writes:

> On Fri, Nov 13, 2015 at 10:51:58AM -0500, Mark H Weaver wrote:
>> Jan Synáček  writes:
>> 
>> > On Sun, Nov 8, 2015 at 12:49 AM, Andreas Rottmann 
>> > wrote:
>> >
>> > Also note that if there's no requirement to actually implement
>> > this in
>> > C, there's `fdes->inport' and `fdes->outport' on the Scheme level,
>> > so
>> > something like the following would be analogous to the C example
>> > code
>> > posted:
>> > 
>> > (import (ice-9 binary-ports))
>> > 
>> > (define (process-fd fd)
>> > (let ((port (fdes->inport fd)))
>> > (display "read: ")
>> > (display (get-bytevector-n port 100))
>> > (display "\n")))
>> > 
>> > (process-fd (acquire-valid-fd))
>> > 
>> >
>> > This is something very similar that I ended up with. Just instead of
>> > get-byte-vector, I used read-string!/partial.
>> 
>> I would advise against using 'read-string!/partial' or any of the
>> procedures in (ice-9 rw).  This is a vestigial module from Guile 1.8
>> when strings were arrays of bytes, which they no longer are.  We should
>> probably mark them as deprecated.
>> 
>> For one thing, when we switch to using UTF-8 as the internal string
>> encoding, it will not be possible to keep 'read-string!/partial'
>> efficient.  It will necessarily have to do an encoding conversion.
>> 
>> In Guile 2+, I would advise using byte vectors when working with binary
>> data.  Portions of these can be converted to strings with a given
>> encoding if desired.  I might be able to give better advice if I knew
>> more about what you are doing here.
>
> Mark,
>
> what Jan is after (and what I'd like to have too) is something
> akin to Unix read(2) with O_NONBLOCK: provide a buffer, request
> (up to) N bytes from the file (descriptor) and get an answer
> (with possibly less bytes).
>
> I tried that a while ago and was surprised that I had to resort
> to (character) strings, with all the downsides you mention. Something
> like that for byte vectors would be awesome. Either it exists (and
> neither Jan nor me have succeeded in finding it) or it doesn't.
>
The procedure with the closest semantics is R6RS
`get-bytevector-some`. While the R6RS says it will block if no data is
available, a quick look at Guile source code seems to indicate that it
probably works with non-blocking I/O -- I'd say it should return EOF if
called on a non-readable, non-blocking port, and otherwise not block,
and return the data available. This is all just from a quick
inspection, without running any actual code.

Regards, Rotty
-- 
Andreas Rottmann -- <http://rotty.xx.vu/>

Re: [PATCH] Add SRFI-25 implementation

[Andreas Rottmann]

l...@gnu.org (Ludovic Courtès) writes:

> Mark H Weaver  skribis:
>
>> l...@gnu.org (Ludovic Courtès) writes:
>>
>>> l...@gnu.org (Ludovic Courtès) skribis:
>>>
>>>> Andreas Rottmann  skribis:
>>>>
>>>>> Adds an implementation of SRFI 25 on top of Guile's native arrays. The
>>>>> implementation does not introduce a disjoint type; Guile arrays and
>>>>> SRFI-25 arrays can be used interchangably, though with different, partly
>>>>> conflicting APIs.
>>>>>
>>>>> [...]
>>>>
>>>> If there are no objections, you can push this patch within a day or two
>>>> to ‘stable-2.0’ (or I’ll do it if you happen to be unavailable.)
>>>
>>> Done in aaea5b2 with minor modifications in srfi-modules.texi to make it
>>> more consistent with the rest of the manual.
>>
>> This commit broke the build, because it forgot to add srfi-25.scm.
>>
>> For now, I reverted the commit instead of fixing it, because there are a
>> few other problems with srfi-25.scm which I had told Andreas about over
>> IRC but never posted about.  Sorry about that.  I guess maybe that's why
>> Andreas didn't respond more quickly to your earlier email.
>>
Yep, I didn't come around to address these issues, and assumed they were
communicated and taken into consideration when Ludo stated his intention
to apply my patch.

>> In a couple of places, it has code like this:
>>
>> + (cond ((array? i)
>> +(apply array-ref array (array->index-list i)))
>> +   ((vector? i)
>> +(apply array-ref array (vector->list i)))
>>
>> Since every vector is also considered an array, the second clause of the
>> 'cond' is dead code.
>>
>> Also, it would be good to avoid consing in common cases in 'array-ref'
>> and 'array-set!', by adding special cases for 1 and 2 dimensional array
>> accesses, at least when the indices are given as direct arguments.
>>
>> Finally, the following line should be put somewhere in the module:
>>
>> (cond-expand-provide (current-module) '(srfi-25))
>
> Too much asynchrony here. ;-)  I hope Andreas or you can look into it.
>
I hope I'll have time to look into it tomorrow, and post a patch based
on Ludo's adaptions, addressing the issues raised by Mark; thanks for
cleaning up the docs Ludo, BTW!

Regards, Rotty
-- 
Andreas Rottmann -- <http://rotty.xx.vu/>

Re: Reading data from a file descriptor

[Andreas Rottmann]

Artyom Poptsov  writes:

> Oh sorry, there was a mistake in my previous mail.  'scm_i_fdes_to_port'
> is an internal procedure, but 'scm_fdes_to_port' is defined as Guile API
> and should be available to Guile programs.  Here's its definition:
>
> SCM
> scm_fdes_to_port (int fdes, char *mode, SCM name)
>
> And the commentary that I quoted really applies to 'scm_fdes_to_port',
> not to 'scm_i_fdes_to_port'.
>
Also note that if there's no requirement to actually implement this in
C, there's `fdes->inport' and `fdes->outport' on the Scheme level, so
something like the following would be analogous to the C example code
posted:

(import (ice-9 binary-ports))

(define (process-fd fd)
  (let ((port (fdes->inport fd)))
(display "read: ")
(display (get-bytevector-n port 100))
(display "\n")))

(process-fd (acquire-valid-fd))

You could now just implement `acquire-valid-fd' in C and expose it to
Scheme, if that is even necessary. If you have the FD available via
e.g. an environment variable, `acquire-valid-fd' can be implemented in
Scheme as well.

Regards, Rotty
-- 
Andreas Rottmann -- <http://rotty.xx.vu/>

Re: [PATCH] Heed the reader settings implied by #!r6rs

[Andreas Rottmann]

Mark H Weaver  writes:

> Andreas Rottmann  writes:
>
>> When encountering the #!r6rs directive, apply the appropriate reader
>> settings to the port.
>>
>> * libguile/read.scm (read-string-as-list): New helper procedure.
>>   (scm_read_shebang): Set reader options implied by the R6RS syntax
>>   upon encountering the #!r6rs directive.
>> * test-suite/tests/reader.test (per-port-read-options): Add tests for
>>   the #!r6rs directive.
>
> Looks good to me.  Please push!
>
Thanks for the review! I've pushed it.

Regards, Rotty
-- 
Andreas Rottmann -- <http://rotty.xx.vu/>

[PATCH] Heed the reader settings implied by #!r6rs

[Andreas Rottmann]

When encountering the #!r6rs directive, apply the appropriate reader
settings to the port.

* libguile/read.scm (read-string-as-list): New helper procedure.
  (scm_read_shebang): Set reader options implied by the R6RS syntax
  upon encountering the #!r6rs directive.
* test-suite/tests/reader.test (per-port-read-options): Add tests for
  the #!r6rs directive.
---
 NEWS | 17 +++-
 libguile/read.c  | 40 +++--
 test-suite/tests/reader.test | 47 +---
 3 files changed, 94 insertions(+), 10 deletions(-)

diff --git a/NEWS b/NEWS
index 0292dcd..8ed1b8d 100644
--- a/NEWS
+++ b/NEWS
@@ -1,10 +1,25 @@
 Guile NEWS --- history of user-visible changes.
-Copyright (C) 1996-2014 Free Software Foundation, Inc.
+Copyright (C) 1996-2015 Free Software Foundation, Inc.
 See the end for copying conditions.
 
 Please send Guile bug reports to bug-gu...@gnu.org.
 
+Changes in 2.0.12 (since 2.0.11):
 
+* Notable changes
+
+** The #!r6rs directive now influences read syntax
+
+The #!r6rs directive now changes the per-port reader options to make
+Guile's reader conform more closely to the R6RS syntax. In particular:
+
+- It makes the reader case sensitive.
+- It disables the recognition of keyword syntax in conflict with the
+  R6RS (and R5RS).
+- It enables the `square-brackets', `hungry-eol-escapes' and
+  `r6rs-hex-escapes' reader options.
+
+
 Changes in 2.0.11 (since 2.0.10):
 
 This release fixes an embarrassing regression introduced in the C
diff --git a/libguile/read.c b/libguile/read.c
index 60a40d9..616529d 100644
--- a/libguile/read.c
+++ b/libguile/read.c
@@ -1,4 +1,4 @@
-/* Copyright (C) 1995-1997, 1999-2001, 2003, 2004, 2006-2012, 2014
+/* Copyright (C) 1995-1997, 1999-2001, 2003, 2004, 2006-2012, 2014, 2015
  *   Free Software Foundation, Inc.
  * 
  * This library is free software; you can redistribute it and/or
@@ -1421,6 +1421,12 @@ static void set_port_square_brackets_p (SCM port, 
scm_t_read_opts *opts,
 int value);
 static void set_port_curly_infix_p (SCM port, scm_t_read_opts *opts,
 int value);
+static void set_port_r6rs_hex_escapes_p (SCM port, scm_t_read_opts *opts,
+ int value);
+static void set_port_hungry_eol_escapes_p (SCM port, scm_t_read_opts *opts,
+   int value);
+static void set_port_keyword_style (SCM port, scm_t_read_opts *opts,
+enum t_keyword_style value);
 
 static SCM
 scm_read_shebang (scm_t_wchar chr, SCM port, scm_t_read_opts *opts)
@@ -1442,7 +1448,13 @@ scm_read_shebang (scm_t_wchar chr, SCM port, 
scm_t_read_opts *opts)
   scm_ungetc (c, port);
   name[i] = '\0';
   if (0 == strcmp ("r6rs", name))
-;  /* Silently ignore */
+{
+  set_port_case_insensitive_p (port, opts, 0);
+  set_port_r6rs_hex_escapes_p (port, opts, 1);
+  set_port_square_brackets_p (port, opts, 1);
+  set_port_keyword_style (port, opts, KEYWORD_STYLE_HASH_PREFIX);
+  set_port_hungry_eol_escapes_p (port, opts, 1);
+}
   else if (0 == strcmp ("fold-case", name))
 set_port_case_insensitive_p (port, opts, 1);
   else if (0 == strcmp ("no-fold-case", name))
@@ -2305,6 +2317,30 @@ set_port_curly_infix_p (SCM port, scm_t_read_opts *opts, 
int value)
   set_port_read_option (port, READ_OPTION_CURLY_INFIX_P, value);
 }
 
+/* Set OPTS and PORT's r6rs_hex_escapes_p option according to VALUE. */
+static void
+set_port_r6rs_hex_escapes_p (SCM port, scm_t_read_opts *opts, int value)
+{
+  value = !!value;
+  opts->r6rs_escapes_p = value;
+  set_port_read_option (port, READ_OPTION_R6RS_ESCAPES_P, value);
+}
+
+static void
+set_port_hungry_eol_escapes_p (SCM port, scm_t_read_opts *opts, int value)
+{
+  value = !!value;
+  opts->hungry_eol_escapes_p = value;
+  set_port_read_option (port, READ_OPTION_HUNGRY_EOL_ESCAPES_P, value);
+}
+
+static void
+set_port_keyword_style (SCM port, scm_t_read_opts *opts, enum t_keyword_style 
value)
+{
+  opts->keyword_style = value;
+  set_port_read_option (port, READ_OPTION_KEYWORD_STYLE, value);
+}
+
 /* Initialize OPTS based on PORT's read options and the global read
options. */
 static void
diff --git a/test-suite/tests/reader.test b/test-suite/tests/reader.test
index 9055e3b..cef95c6 100644
--- a/test-suite/tests/reader.test
+++ b/test-suite/tests/reader.test
@@ -1,6 +1,6 @@
  reader.test --- Reader test.-*- coding: iso-8859-1; mode: scheme -*-
 
- Copyright (C) 1999, 2001-2003, 2007-2011, 2014
+ Copyright (C) 1999, 2001-2003, 2007-2011, 2014, 2015
    Free Software Foundation, Inc.
 
  Jim Blandy 
@@ -61,6 +61,11 @@
 (lambda ()
   (read-options saved-options)
 
+(define (read-string-as-li

[PATCH] Add SRFI-25 implementation

[Andreas Rottmann]

Adds an implementation of SRFI 25 on top of Guile's native arrays. The
implementation does not introduce a disjoint type; Guile arrays and
SRFI-25 arrays can be used interchangably, though with different, partly
conflicting APIs.

* NEWS: Add preliminary, incomplete section on 2.0.12, noting the
  addition of SRFI-25.
* doc/ref/srfi-modules.texi (SRFI-25): New node.
* module/srfi/srfi-25.scm: New file.
* test-suite/tests/srfi-25.test: New file.
* module/Makefile.am:
* test-suite/Makefile.am: Add new files.
---
 NEWS  |  15 +-
 doc/ref/srfi-modules.texi | 240 +-
 module/Makefile.am|   1 +
 module/srfi/srfi-25.scm   | 159 +++
 test-suite/Makefile.am|   3 +-
 test-suite/tests/srfi-25.test | 461 ++
 6 files changed, 876 insertions(+), 3 deletions(-)
 create mode 100644 module/srfi/srfi-25.scm
 create mode 100644 test-suite/tests/srfi-25.test

diff --git a/NEWS b/NEWS
index 0292dcd..19c4e39 100644
--- a/NEWS
+++ b/NEWS
@@ -1,10 +1,23 @@
 Guile NEWS --- history of user-visible changes.
-Copyright (C) 1996-2014 Free Software Foundation, Inc.
+Copyright (C) 1996-2015 Free Software Foundation, Inc.
 See the end for copying conditions.
 
 Please send Guile bug reports to bug-gu...@gnu.org.
 
+Changes in 2.0.12 (since 2.0.11):
 
+* New interfaces
+
+** SRFI-25 (Multi-dimensional Array Primitives)
+
+Guile now includes SRFI-25, a core set of procedures for creating and
+manipulating multidimensional arrays. This functionality is already
+available in Guile, albeit with a different API, with its native array
+data type, but the inclusion of the SRFI is nevertheless useful for code
+intended to be portable across multiple implementations. See "SRFI-25" i
+the manual for details.
+
+
 Changes in 2.0.11 (since 2.0.10):
 
 This release fixes an embarrassing regression introduced in the C
diff --git a/doc/ref/srfi-modules.texi b/doc/ref/srfi-modules.texi
index d8ed8e1..53312e9 100644
--- a/doc/ref/srfi-modules.texi
+++ b/doc/ref/srfi-modules.texi
@@ -1,6 +1,6 @@
 @c -*-texinfo-*-
 @c This is part of the GNU Guile Reference Manual.
-@c Copyright (C) 1996, 1997, 2000-2004, 2006, 2007-2014
+@c Copyright (C) 1996, 1997, 2000-2004, 2006, 2007-2015
 @c   Free Software Foundation, Inc.
 @c See the file guile.texi for copying conditions.
 
@@ -36,6 +36,7 @@ get the relevant SRFI documents from the SRFI home page
 * SRFI-18:: Multithreading support
 * SRFI-19:: Time/Date library.
 * SRFI-23:: Error reporting
+* SRFI-25:: Multi-dimensional Array Primitives
 * SRFI-26:: Specializing parameters
 * SRFI-27:: Sources of Random Bits
 * SRFI-28:: Basic format strings.
@@ -3018,6 +3019,243 @@ locale.
 
 The SRFI-23 @code{error} procedure is always available.
 
+@node SRFI-25
+@subsection SRFI-25 - Multi-dimensional Array Primitives
+@cindex SRFI-25
+@cindex array, multi-dimensional
+@cindex multi-dimensional array
+
+Note that Guile's implementation of SRFI-25 does not introduce a
+disjoint type; Guile arrays (@pxref{Arrays}) and SRFI-25 arrays can be
+used interchangably, though with different, partly conflicting APIs. The
+SRFI-25 API can be used with,
+
+@example
+(use-modules (srfi srfi-25))
+@end example
+
+Note that this will override @code{make-array}, @code{array-ref} and
+@code{array-set!}. @ref{Using Guile Modules} describes how to control
+the import of identifiers, e.g., by importing the SRFI-25 variants with
+different names.
+
+This subsection is based on
+@uref{http://srfi.schemers.org/srfi-45/srfi-45.html, the specification
+of SRFI-45} written by Jussi Piitulainen.
+
+@c Copyright (C) Jussi Piitulainen (2001). All Rights Reserved.
+
+@c Permission is hereby granted, free of charge, to any person obtaining a
+@c copy of this software and associated documentation files (the
+@c "Software"), to deal in the Software without restriction, including
+@c without limitation the rights to use, copy, modify, merge, publish,
+@c distribute, sublicense, and/or sell copies of the Software, and to
+@c permit persons to whom the Software is furnished to do so, subject to
+@c the following conditions:
+
+@c The above copyright notice and this permission notice shall be included
+@c in all copies or substantial portions of the Software.
+
+@c THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS
+@c OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+@c MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+@c NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+@c LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+@c OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+@c WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+
+A core set of procedures for creating and manipulating hetero

Re: [PATCH 2/3] Add C++ extern "C" guards to internal headers.

[Andreas Rottmann]

Taahir Ahmed  writes:

> On Tuesday 14 July 2015 21:36:31 Andreas Rottmann wrote: 
>> Just a minor nitpick: it might make sense to add a pair of macros to
>> libguile/__scm.h for the '#ifdef __cplusplus' dance, so these
>> boilerplate additions become just two lines, e.g.:
>
> That's a good idea, and I've gone ahead and incorporated it.  I won't
> publish a new patchset here, because it doesn't really change anything.
>
> As an aside, have you received patch 3?  It hasn't shown up in the list
> archive, and I've sent it again, with no luck.  I'm unsure if it somehow
> got held for moderation.
>
No, it indeed hasn't reached me via the list. Is it perhaps very large?

Regards, Rotty
-- 
Andreas Rottmann -- <http://rotty.xx.vu/>

Re: [PATCH 2/3] Add C++ extern "C" guards to internal headers.

[Andreas Rottmann]

Taahir Ahmed  writes:

> ---
>  libguile/__scm.h   |  8 
>  [...]
>
> diff --git a/libguile/__scm.h b/libguile/__scm.h
> index 31e3952..1967069 100644
> --- a/libguile/__scm.h
> +++ b/libguile/__scm.h
> @@ -53,6 +53,10 @@
>  /* What did the configure script discover about the outside world?  */
>  #include "libguile/scmconfig.h"
>  
> +#ifdef __cplusplus
> +extern "C" {
> +#endif
> +
>
>  
>  
>  /* {Compiler hints}
> @@ -523,6 +527,10 @@ typedef long SCM_STACKITEM;
>  # define SCM_THREAD_LOCAL
>  #endif
>  
> +#ifdef __cplusplus
> +}
> +#endif
> +
>  #endif  /* SCM___SCM_H */
>
Just a minor nitpick: it might make sense to add a pair of macros to
libguile/__scm.h for the '#ifdef __cplusplus' dance, so these
boilerplate additions become just two lines, e.g.:

+ SCM_BEGIN_DECLS

...

+ SCM_END_DECLS

This is done in GLib, for example, and cuts down the noise a bit.

Regards, Rotty
-- 
Andreas Rottmann -- <http://rotty.xx.vu/>

Re: [PATCH] Split r6rs-ports.c according to module boundaries

[Andreas Rottmann]

Mark H Weaver  writes:

> Hi Andreas,
>
> This new patch has a very serious problem, which was not obvious at
> first glance.  We've made some significant improvements to r6rs-ports.c
> since you first introduced this patch, and this new patch would
> effectively undo all of them.  It's a good thing I took the time to
> check.  Please be more careful.
>  
Thanks for taking a look and sorry for being sloppy; I'd have expected
merge conflicts in this case when updating the patch; for some reason
topgit updated the patch without any conflicts.  Since you raise
concerns whether this patch really makes sense on a fundamental level,
let's figure out this question before going into specifics.

> I confess I'm not entirely clear on why this module is being split.  It
> seems to me that this whole thing is the R6RS ports API.  Can you help
> me understand the reason for this split?
>
I've split it to match the two Scheme modules that already existed:
(ice-9 binary-ports) and (rnrs io ports). The latter does a private
import (via '@@') of '%make-transcoded-port' defined in the former in
the implementation of 'transcoded-port'.

I'd like to get rid of that invocation of '@@', but perhaps splitting
the C code is misguided, and it would be better to instead move the
implementation of 'transcoded-port' to (ice-9 binary-ports), along with
the rest of the transcoder API, i.e. place the implementation of all
these identifiers in (ice-9 binary-ports):

  eol-style native-eol-style error-handling-mode make-transcoder
  transcoder-codec transcoder-eol-style transcoder-error-handling-mode
  native-transcoder latin-1-codec utf-8-codec utf-16-codec

Thoughts?
-- 
Andreas Rottmann -- <http://rotty.xx.vu/>

Re: Build failure (stable-2.0, "make distcheck")

[Andreas Rottmann]

l...@gnu.org (Ludovic Courtès) writes:

> Andreas Rottmann  skribis:
>
>> It is the case; guile.info is present, but it seems that it's not newer
>> that one of the sources it's built from:
>>
>> % tar --utc -tvf _build/guile-2.0.7.13-b5d83.tar.gz | grep -E 
>> '\.(info|texi)$' | awk '{ print $4 " " $5 " " $6 }' | sort | tail -n2
>> 2012-12-26 13:59 guile-2.0.7.13-b5d83/doc/ref/guile.info
>> 2012-12-26 13:59 guile-2.0.7.13-b5d83/doc/ref/standard-library.texi
>>
>> Probably standard-library.texi is created just a tiny bit earlier than
>> guile.info (in the same second).  That might be the reason the build
>> tries to run makeinfo inside the unpacked "dist" tarball?
>
> Can you unpack the tarball and check that hypothesis?
>
It turned out my hypothesis was wrong, the guile.info actually has a
newer timestamp. I've now run a build using "make distcheck --debug=v"
and attached the log for future reference. It seems that problem is the
following:

,
|Prerequisite `effective-version.texi' is newer than target 
`../../../doc/ref/guile.info'.
|   Must remake target `../../../doc/ref/guile.info'.
| make[5]: Entering directory 
`/home/rotty/src/guile/_build/guile-2.0.7.12-c23fb/_build/doc/ref'
| [... actual failure happens here ... ]
`

While "doc/ref/effective-version.texi", as contained in the tarball, is
indeed older than guile.info (A), it seems a build-directory version of
"effective-version.texi" is generated by configure (B, C), and that copy
is then used by make for calculating whether guile.info needs to be
(re-)built, as indicated by the above log snippet.

The below commands are executed inside the extracted copy of the
tarball, in which the build fails
(i.e. /home/rotty/src/guile/_build/guile-2.0.7.12-c23fb/):

(A) 

% ls --full-time doc/ref/guile.info doc/ref/effective-version.texi  
   
-r--r--r-- 1 rotty rotty27 2012-12-26 17:35:08.0 +0100 
doc/ref/effective-version.texi
-r--r--r-- 1 rotty rotty 29821 2012-12-26 17:35:16.0 +0100 
doc/ref/guile.info

(B)

% ls --full-time _build/doc/ref/effective-version.texi
-rw-r--r-- 1 rotty rotty 27 2013-01-05 16:32:37.150982801 +0100 
_build/doc/ref/effective-version.texi

(C)

,
| ...
| config.status: creating doc/ref/effective-version.texi
| ...
`

At least that's what I read out of the logs.  I just wonder why nobody
else seems to have run into this issue, which now looks like a genuine
buildsystem bug to me. Thoughts?

Kind Regards, Rotty
-- 
Andreas Rottmann -- <http://rotty.xx.vu/>

Re: Build failure (stable-2.0, "make distcheck")

[Andreas Rottmann]

l...@gnu.org (Ludovic Courtès) writes:

> Hi Andreas,
>
> Andreas Rottmann  skribis:
>
>> mkdir: cannot create directory '.am27744': Permission denied
>> ../../../doc/ref/guile.texi: No such file or directory
>> make[5]: *** [../../../doc/ref/guile.info] Error 1
>
> Normally guile.info is part of the tarball.  Can you check whether this
> is the case?  (Apparently you have ‘makeinfo’ installed so I don’t see
> why it wouldn’t be the case.)
>
It is the case; guile.info is present, but it seems that it's not newer
that one of the sources it's built from:

% tar --utc -tvf _build/guile-2.0.7.13-b5d83.tar.gz | grep -E '\.(info|texi)$' 
| awk '{ print $4 " " $5 " " $6 }' | sort | tail -n2
2012-12-26 13:59 guile-2.0.7.13-b5d83/doc/ref/guile.info
2012-12-26 13:59 guile-2.0.7.13-b5d83/doc/ref/standard-library.texi

Probably standard-library.texi is created just a tiny bit earlier than
guile.info (in the same second).  That might be the reason the build
tries to run makeinfo inside the unpacked "dist" tarball?

Regards, Rotty
-- 
Andreas Rottmann -- <http://rotty.xx.vu/>

Re: Fix reader options for R6RS `get-datum'

[Andreas Rottmann]

Mark H Weaver  writes:

> Andreas Rottmann  writes:
>
>> Mark H Weaver  writes:
>>
>>> Section 8.3 defines 'read' as follows:
>>>
>>>   Reads an external representation from textual-input-port and returns
>>>   the datum it represents. The read procedure operates in the same way
>>>   as get-datum, see section 8.2.9.
>>>
>>> I believe this last sentence clearly confirms my belief that 'read' and
>>> 'get-datum' should recognize the same syntax.
>>>
>> Well yes, R6RS `read' and R6RS `get-datum' need to understand the same
>> syntax, but I thought you were talking about Guile `read' and R6RS
>> `get-datum'.
>
> Ah, so you want R6RS 'read' to be different than Guile 'read'.
> I think this would be a mistake.
>
I think that's mandated by R6RS and the fact that Guile offers reader
options that plainly incompatible with the syntax described in R6RS.

> I'd like to allow coherent systems to be built from a mixture of R6RS
> code, R7RS code, native Guile code, etc.  With this in mind, I think it
> would be terribly confusing for users (and not particularly sensible)
> for the notation recognized by 'read' to depend upon whether the code
> that happens to call 'read' is in an R6RS library or a Guile module.
>
>
Strictly speaking, it's not whether the code is in a "Guile module" or
"R6RS library" (both are actually "Guile modules" in Guile's
implementation of R6RS), but whether the binding imported for `read' is
Guile's core `scm_read' or the one from `(rnrs io simple)'.

> For example, the code that calls 'read' when compiling source files
> happens to be in a Guile module.  What does that have to do with the
> language being read?  Nothing.
>
>> Yup, R6RS `read' needs to be implemented in terms of `get-datum', not
>> only because of reader options, but also because of the required
>> exception behavior.  This is how it's done already -- see
>> modules/rnrs/io/simple.scm.
>
> I thought we agreed on IRC that this is an unworkable approach to
> supporting R6RS exceptions in Guile.  That path leads to a future where
> there are two variants of every primitive procedure that might throw
> exceptions.  It also means duplicating every VM instruction that might
> throw exceptions.
>
Yeah, but until exception conversion in the `guard' (or `catch') is
implemented, `get-datum' & co. still need to adhere to the
specification, i.e. throw the exceptions mandated by R6RS in the
circustances described therin.  I don't think it is necessary to pull
the implementation strategy for exceptions into this discussion (even if
I mistakenly started with it ;-).  The issue of reader options is and
orthogonal, if related, one, IMO.

> Those facts alone would be bad enough, but it gets worse.  In a program
> composed of a mixture of R6RS and native Guile code, an R6RS exception
> handler should be able to properly catch an error that happened within
> native Guile code, and vice versa.  That won't work with this approach
> of throwing R6RS-style exceptions from within R6RS primitives and
> Guile-style exceptions within Guile primitives.
>
> IMO, to create a coherent system that allows mixing of code, we need a
> single unified exception system that is sufficiently fine-grained (and
> provides enough information) to satisfy the needs of both R6RS exception
> handlers and legacy Guile exception handlers.
>
> At any given time, there might be exception handlers installed by both
> Guile 'catch' and R6RS 'guard'.  The code that throws an exception has
> no way of knowing which kind of exception handler will catch it.
> Therefore, the conversion to native R6RS conditions needs to happen
> within the exception handler.
>
> Does that make sense?  I thought we discussed this on IRC and agreed on
> this general approach.
>
Yeah, we agreed that this is where we want to arrive at, but please
let's discuss reader options only in this thread.

>>> On the flip side, if someone has enabled SRFI-105 curly-infix
>>> expressions, or any other reader extension that does not conflict with
>>> standard R6RS notation, then both 'get-datum' and 'read' should honor
>>> that setting.
>>>
>>> Does that make sense?
>>>
>> It does, and I think this is also what my patch implements, if I
>> understood both the code and your words correctly :-).
>
> To make this more concrete, let's consider two of the reader options
> that you'd apparently like to override w

Re: Fix reader options for R6RS `get-datum'

[Andreas Rottmann]

Mark H Weaver  writes:

> Hi Andreas,
>
> Andreas Rottmann  writes:
>> Mark H Weaver  writes:
>>> Andreas Rottmann  writes:
>>>> This patch series addresses the problem that `get-datum' is using the
>>>> global reader options, even for those options that have to have fixed
>>>> values to make the reader behave in an R6RS-compatible way.
>>>
>>> I'm sorry to have not done so earlier, but I finally looked at the R6RS
>>> specification for 'get-datum', and I don't see anything to suggest that
>>> it should recognize a different notation than 'read' does.
>>>
>> I think it does.  But in the place where one would expect, namely the
>> docs on `get-datum' (8.2.6), it does not say it as clearly as one would
>> like:
>>
>>   Reads an external representation from textual-input-port and returns
>>   the datum it represents. The get-datum procedure returns the next
>>   datum that can be parsed from the given textual-input-port, updating
>>   textual-input-port to point exactly past the end of the external
>>   representation of the object.
>
> Section 8.3 defines 'read' as follows:
>
>   Reads an external representation from textual-input-port and returns
>   the datum it represents. The read procedure operates in the same way
>   as get-datum, see section 8.2.9.
>
> I believe this last sentence clearly confirms my belief that 'read' and
> 'get-datum' should recognize the same syntax.
>
Well yes, R6RS `read' and R6RS `get-datum' need to understand the same
syntax, but I thought you were talking about Guile `read' and R6RS
`get-datum'.

>>
>>> Maybe what we should have instead is a command-line option that sets
>>> some(?) of the global read options to conform with R6RS.
>>>
>>> What do you think?
>>>
>> I disagree quite strongly -- IMO, `get-datum' must, for every valid
>> datum, according to R6RS lexical syntax, return the Scheme data denoted
>> by that external representation.  By that rule, one must fix the values
>> of reader options which would otherwise lead to valid R6RS external
>> representations being read as a datum different from the one which would
>> result according to R6RS syntax.
>
> I agree that there should be an easy way to set the reader options so
> that all of the R6RS external representations are read as specified.
>
Yes, but this cannot be a program-global switch -- it is simply defined
behavior `get-datum', which must *not* be affected by any global
switches. An R6RS syntax commandline switch would make sense regardless,
for dealing with e.g. script syntax, but this is a completely orthogonal
issue, and I'd like to concentrate on the `get-datum' correctness issue
for now.

> IMO, this needs to be done not only for 'get-datum' but also for 'read',
> hence my suggestion to provide some way to set the global read options
> appropriately.
>
Yup, R6RS `read' needs to be implemented in terms of `get-datum', not
only because of reader options, but also because of the required
exception behavior.  This is how it's done already -- see
modules/rnrs/io/simple.scm.

> On the flip side, if someone has enabled SRFI-105 curly-infix
> expressions, or any other reader extension that does not conflict with
> standard R6RS notation, then both 'get-datum' and 'read' should honor
> that setting.
>
> Does that make sense?
>
It does, and I think this is also what my patch implements, if I
understood both the code and your words correctly :-).

Regards, Rotty
-- 
Andreas Rottmann -- <http://rotty.xx.vu/>

Re: Fix reader options for R6RS `get-datum'

[Andreas Rottmann]

Mark H Weaver  writes:

> Hi Andreas,
>
> Andreas Rottmann  writes:
>> This patch series addresses the problem that `get-datum' is using the
>> global reader options, even for those options that have to have fixed
>> values to make the reader behave in an R6RS-compatible way.
>
> I'm sorry to have not done so earlier, but I finally looked at the R6RS
> specification for 'get-datum', and I don't see anything to suggest that
> it should recognize a different notation than 'read' does.
>
I think it does.  But in the place where one would expect, namely the
docs on `get-datum' (8.2.6), it does not say it as clearly as one would
like:

  Reads an external representation from textual-input-port and returns
  the datum it represents. The get-datum procedure returns the next
  datum that can be parsed from the given textual-input-port, updating
  textual-input-port to point exactly past the end of the external
  representation of the object.

Note that "external representation", referenced twice in the above text,
appears to be defined at the beginning of Chapter 4:

  Syntactic data (also called external representations) double as a
  notation for objects, and Scheme’s (rnrs io ports (6)) library [...]
  provides the get-datum and put-datum procedures for reading and
  writing syntactic data, converting between their textual
  representation and the corresponding objects.

This leads me to conclude that `get-datum' should parse R6RS syntax, as
defined in R6RS 4.2.1.  My proposed changes get us further in that
direction, by making sure we set all the knobs currently available to
enlarge the syntactic subset of R6RS we parse correctly, but it does not
get us to the finishing line.

> If the user has enabled reader extensions for 'read', I don't see why
> 'get-datum' shouldn't honor those extensions as well.
>
Well, there's difference between upwardly-compatible extensions, and
ones that are incompatible with R6RS syntax, such as colon-prefix or
colon-suffix keywords.  Hash-colon keywords are ok, since they don't
conflict with the interpretation of legal R6RS code.

> Maybe what we should have instead is a command-line option that sets
> some(?) of the global read options to conform with R6RS.
>
> What do you think?
>
I disagree quite strongly -- IMO, `get-datum' must, for every valid
datum, according to R6RS lexical syntax, return the Scheme data denoted
by that external representation.  By that rule, one must fix the values
of reader options which would otherwise lead to valid R6RS external
representations being read as a datum different from the one which would
result according to R6RS syntax.

Accepting lexical syntax which is not defined by R6RS is another matter,
and I'm fine with allowing it.

Regards, Rotty
-- 
Andreas Rottmann -- <http://rotty.xx.vu/>

[PATCH 3/3] Make `get-datum' conform more closely to R6RS semantics

[Andreas Rottmann]

With Guile's default reader options, R6RS hex escape and EOL escape
behavior is missing.  This change enables the former via the
`r6rs-hex-escapes' option, and gets us closer to the latter by setting
`hungry-eol-escapes'.

* libguile/r6rs-ports.c (R6RS_READ_OPTION_MASK): New macro, defines
  which reader options need fixed values according to R6RS.
  (r6rs_read_options): New internal constant, defining the values of the
  relevant reader options.
  (scm_i_get_datum): New internal helper calling `scm_i_read' using the
  R6RS reader options.

* module/rnrs/io/ports.scm (get-datum): Call `%get-datum' instead of
  `read'.

* test-suite/tests/r6rs-ports.test ("8.2.9 Textual input")["get-datum"]:
  New tests.
---
 libguile/r6rs-ports.c|   29 
 module/rnrs/io/ports.scm |2 +-
 test-suite/tests/r6rs-ports.test |   45 ++
 3 files changed, 75 insertions(+), 1 deletion(-)

diff --git a/libguile/r6rs-ports.c b/libguile/r6rs-ports.c
index 19dea8d..973953a 100644
--- a/libguile/r6rs-ports.c
+++ b/libguile/r6rs-ports.c
@@ -174,6 +174,35 @@ SCM_DEFINE (scm_i_make_transcoded_port,
 }
 #undef FUNC_NAME
 
+#define R6RS_READ_OPTION_MASK \
+  ((1 << SCM_READ_OPTION_KEYWORD_STYLE)   \
+   | (1 << SCM_READ_OPTION_R6RS_ESCAPES_P)\
+   | (1 << SCM_READ_OPTION_CASE_INSENSITIVE_P)\
+   | (1 << SCM_READ_OPTION_SQUARE_BRACKETS_P) \
+   | (1 << SCM_READ_OPTION_HUNGRY_EOL_ESCAPES_P))
+
+static const scm_t_read_opts r6rs_read_options = {
+  SCM_KEYWORD_STYLE_HASH_PREFIX,
+  0, /* copy_source_p, not relevant */
+  0, /* record_positions_p, not relevant */
+  0, /* case_insensitive_p */
+  1, /* r6rs_escapes_p */
+  1, /* square_brackets_p */
+  1, /* hungry_eol_escapes_p */
+  0, /* curly_infix_p, compatible extension */
+  0, /* neoteric_p, not relevant */
+};
+
+SCM_DEFINE (scm_i_get_datum,
+"%get-datum", 1, 0, 0,
+(SCM port),
+"Read a datum in R6RS syntax from @var{port}")
+#define FUNC_NAME s_scm_i_get_datum
+{
+  return scm_i_read(port, &r6rs_read_options, R6RS_READ_OPTION_MASK);
+}
+#undef FUNC_NAME
+
 
 /* Initialization.  */
 
diff --git a/module/rnrs/io/ports.scm b/module/rnrs/io/ports.scm
index ee8c05a..b2828cd 100644
--- a/module/rnrs/io/ports.scm
+++ b/module/rnrs/io/ports.scm
@@ -458,7 +458,7 @@ return the characters accumulated in that port."
   (with-textual-input-conditions port (read-char port)))
 
 (define (get-datum port)
-  (with-textual-input-conditions port (read port)))
+  (with-textual-input-conditions port (%get-datum port)))
 
 (define (get-line port)
   (with-textual-input-conditions port (read-line port 'trim)))
diff --git a/test-suite/tests/r6rs-ports.test b/test-suite/tests/r6rs-ports.test
index ed49598..6a92987 100644
--- a/test-suite/tests/r6rs-ports.test
+++ b/test-suite/tests/r6rs-ports.test
@@ -707,6 +707,16 @@
(eq? (error-handling-mode replace)
 (transcoder-error-handling-mode t))
 
+;; FIXME: duplicated from reader.test
+(define (with-read-options opts thunk)
+  (let ((saved-options (read-options)))
+(dynamic-wind
+(lambda ()
+  (read-options opts))
+thunk
+(lambda ()
+  (read-options saved-options)
+
 (with-test-prefix "8.2.9  Textual input"
   
   (pass-if "get-string-n [short]"
@@ -724,6 +734,41 @@
   (s (string-copy "Isn't XXX great?")))
   (and (= 3 (get-string-n! port s 6 3))
(string=? s "Isn't GNU great?"
+  (with-test-prefix "get-datum"
+(let ((string->datum
+   (lambda (s)
+ ;; We should check against all possible permutations of
+ ;; read options, but we just enable (and leave disabled)
+ ;; the ones that each would break R6RS individually.
+ (with-read-options '(keywords prefix case-insensitive 
hungry-eol-escapes)
+   (lambda () (get-datum (open-input-string s)))
+  (pass-if "symbol"
+(eq? (string->datum "foo") 'foo))
+  (pass-if "symbol [starting with colon]"
+(eq? ':foo (string->datum ":foo")))
+  (pass-if "symbol ending with colon"
+(eq? 'foo: (string->datum "foo:")))
+  (pass-if "string"
+(string=? "foo" (string->datum "\"foo\"")))
+  (pass-if "string [with hex escapes]"
+(string=? "bar\nA" (string->datum "\"bar\\x0A;\\x41;\"")))
+  (pass-if "string [hungry EOL]"
+(string=? "bar baz" (string->datum "\"bar \\\n   baz\"")))
+  ;; FIXME: actually, R6RS demands an even more hungry EOL escape
+  ;; than the reader currently implements: also any whitespace
+  ;; between the backslash and the newline should vanish. Currently,
+  ;; the reader barfs on that.
+  (pass-if "string [hungry EOL, space also before newline]"
+(throw 'unresolved)
+(string=? "bar baz" (string->datum "\"bar \\  \n   baz\"")))
+  (pass-if "number [dec

[PATCH 2/3] Add internal API to specify reader options at reader invocation

[Andreas Rottmann]

* libguile/private-options.h: Introduce a new enum indexing the read
  options, and use its values as indices for scm_read_opts.
* libguile/read.c: Get rid of the bit field offsets #define-s, and
  instead use the enum values to caculate them.

* libguile/read.c (enum t_keyword_style, scm_t_read_opts): Moved to
  libguile/private-options.h, the names of the keyword style enum having
  been prefixed with "SCM_" (use sites adapted accordingly).  These have
  been moved out of the C file so that they can be used in other parts
  of libguile -- they are needed by the scm_i_read()'s interface.

* libguile/read.h (scm_i_read): New procedure, marked SCM_INTERNAL.
  (scm_t_read_opts): Opaque typedef for the struct defined in
  private-options.h.

* libguile/read.c (init_read_options): Add an additional argument
  `preset', to allow the caller to specify reader options that are
  preset and should not be changed.
  (scm_i_read): New internal procedure, allowing to fix (part of) the
  reader options.
---
 libguile/private-options.h |   51 
 libguile/read.c|  115 +++-
 libguile/read.h|6 ++-
 3 files changed, 95 insertions(+), 77 deletions(-)

diff --git a/libguile/private-options.h b/libguile/private-options.h
index ed0f314..0c85d0c 100644
--- a/libguile/private-options.h
+++ b/libguile/private-options.h
@@ -4,7 +4,7 @@
  * We put this in a private header, since layout of data structures
  * is an implementation detail that we want to hide.
  * 
- * Copyright (C) 2007, 2009, 2010, 2011 Free Software Foundation, Inc.
+ * Copyright (C) 2007, 2009, 2010, 2011, 2012 Free Software Foundation, Inc.
  * 
  * This library is free software; you can redistribute it and/or
  * modify it under the terms of the GNU Lesser General Public License
@@ -60,15 +60,46 @@ SCM_INTERNAL scm_t_option scm_print_opts[];
  */
 SCM_INTERNAL scm_t_option scm_read_opts[];
 
-#define SCM_COPY_SOURCE_P  scm_read_opts[0].val
-#define SCM_RECORD_POSITIONS_P scm_read_opts[1].val
-#define SCM_CASE_INSENSITIVE_P scm_read_opts[2].val
-#define SCM_KEYWORD_STYLE  scm_read_opts[3].val
-#define SCM_R6RS_ESCAPES_P scm_read_opts[4].val
-#define SCM_SQUARE_BRACKETS_P  scm_read_opts[5].val
-#define SCM_HUNGRY_EOL_ESCAPES_P scm_read_opts[6].val
-#define SCM_CURLY_INFIX_P  scm_read_opts[7].val
+enum
+  {
+SCM_READ_OPTION_COPY_SOURCE_P,
+SCM_READ_OPTION_RECORD_POSITIONS_P,
+SCM_READ_OPTION_CASE_INSENSITIVE_P,
+SCM_READ_OPTION_KEYWORD_STYLE,
+SCM_READ_OPTION_R6RS_ESCAPES_P,
+SCM_READ_OPTION_SQUARE_BRACKETS_P,
+SCM_READ_OPTION_HUNGRY_EOL_ESCAPES_P,
+SCM_READ_OPTION_CURLY_INFIX_P,
+SCM_N_READ_OPTIONS
+  };
 
-#define SCM_N_READ_OPTIONS 7
+typedef enum
+  {
+SCM_KEYWORD_STYLE_HASH_PREFIX,
+SCM_KEYWORD_STYLE_PREFIX,
+SCM_KEYWORD_STYLE_POSTFIX
+  } scm_t_keyword_style;
+
+struct scm_struct_read_opts
+{
+  scm_t_keyword_style keyword_style;
+  unsigned int copy_source_p: 1;
+  unsigned int record_positions_p   : 1;
+  unsigned int case_insensitive_p   : 1;
+  unsigned int r6rs_escapes_p   : 1;
+  unsigned int square_brackets_p: 1;
+  unsigned int hungry_eol_escapes_p : 1;
+  unsigned int curly_infix_p: 1;
+  unsigned int neoteric_p   : 1;
+};
+
+#define SCM_COPY_SOURCE_P  scm_read_opts[SCM_READ_OPTION_COPY_SOURCE_P].val
+#define SCM_RECORD_POSITIONS_P 
scm_read_opts[SCM_READ_OPTION_RECORD_POSITIONS_P].val
+#define SCM_CASE_INSENSITIVE_P 
scm_read_opts[SCM_READ_OPTION_CASE_INSENSITIVE_P].val
+#define SCM_KEYWORD_STYLE  scm_read_opts[SCM_READ_OPTION_KEYWORD_STYLE].val
+#define SCM_R6RS_ESCAPES_P 
scm_read_opts[SCM_READ_OPTION_R6RS_ESCAPES_P].val
+#define SCM_SQUARE_BRACKETS_P  
scm_read_opts[SCM_READ_OPTION_SQUARE_BRACKETS_P].val
+#define SCM_HUNGRY_EOL_ESCAPES_P 
scm_read_opts[SCM_READ_OPTION_HUNGRY_EOL_ESCAPES_P].val
+#define SCM_CURLY_INFIX_P  scm_read_opts[SCM_READ_OPTION_CURLY_INFIX_P].val
 
 #endif  /* PRIVATE_OPTIONS */ 
diff --git a/libguile/read.c b/libguile/read.c
index 222891b..754ea99 100644
--- a/libguile/read.c
+++ b/libguile/read.c
@@ -93,29 +93,6 @@ scm_t_option scm_read_opts[] =
from the global and per-port read options, and a pointer is passed
down to all helper functions. */
 
-enum t_keyword_style
-  {
-KEYWORD_STYLE_HASH_PREFIX,
-KEYWORD_STYLE_PREFIX,
-KEYWORD_STYLE_POSTFIX
-  };
-
-struct t_read_opts
-{
-  enum t_keyword_style keyword_style;
-  unsigned int copy_source_p: 1;
-  unsigned int record_positions_p   : 1;
-  unsigned int case_insensitive_p   : 1;
-  unsigned int r6rs_escapes_p   : 1;
-  unsigned int square_brackets_p: 1;
-  unsigned int hungry_eol_escapes_p : 1;
-  unsigned int curly_infix_p: 1;
-  unsigned int neoteric_p   : 1;
-};
-
-typedef struct t_read_opts scm_t_read_opts;
-
-
 /*
   Give meaningful error messages for errors
 
@@ -737,7 +714,7 @@ scm_read_mixed_case_symbol (scm_t_wchar ch

Fix reader options for R6RS `get-datum'

[Andreas Rottmann]

This patch series addresses the problem that `get-datum' is using the
global reader options, even for those options that have to have fixed
values to make the reader behave in an R6RS-compatible way.

* [PATCH 1/3] Split r6rs-ports.c according to module boundaries

  Needed by the last patch, since we want to add a private helper
  function for use in `(rnrs io ports)' there.

* [PATCH 2/3] Add internal API to specify reader options at reader

  This provides the basic mechanism, a.k.a. `scm_i_read()'.

* [PATCH 3/3] Make `get-datum' conform more closely to R6RS semantics
  
  And finally, this makes use of `scm_i_read()' to implement
  `get-datum'.

Re: thoughts on native code

[Andreas Rottmann]

Sjoerd van Leent Privé  writes:

> Hi Stefan,
>
> Just my idea about an assembler in Scheme. Sounds interesting. If it's
> done properly, it can be very promising to use scheme itself to
> directly emit machine instructions. This would also be interesting for
> meta compilation in the future (think of aiding GCC).
>
> So you are thinking about an assembler for x86? Perhaps I can help out
> on this one. I would like to do this part, as I haven't been able to
> aid on other parts besides voicing my ideas (anyways, I am on embedded
> development these days.)
>
> The only discussion is the syntax I believe, I mean, should it be AT&T
> like, Intel like, or leave this domain and do something new. I would
> go for instructions like this (using macros):
>
> (let ((target :x686))
>   (assemble target
>((mov long 100 EAX)
> (mov long 200 EBX)
> (add long EBX EAX
>
> Giving back the native machine code instructions. Perhaps special
> constructions can be made to return partially complete instructions
> (such as missing labels or calls to guile procedures...)
>
Regarding the assembler: I have a working AVR assembler [0] in my
"avrth" AVR Forth implementation [1]. That assembler also employs this
"partially complete instructions" idea you mentioned, having a symbol
resolution step. It also supports a simple evaluator for Assembly-time
expressions.  Maybe you find it interesting :-).

Here's an example snippet, from the Forth implementation's runtime:

(define-primitive-vocable vocabulary:primitive "1ms" (vm)
  (scheme
   (sleep-seconds 0.001))
  (assembly
   (ldi zl (lo8 (/ cpu-frequency 4000)))
   (ldi zh (hi8 (/ cpu-frequency 4000)))
   (sbiw zl 42) ;internal plus forth kernel overhead
   PFA_1MS1
   (sbiw zl 1)
   (brne PFA_1MS1)))

The assembler code is the `(assembly ...)' part, but above that you can
see the runable documentation, written in Scheme ;-).  You may also have
noticed the expressions used, i.e. `(lo8 ...)' and `(hi8 ...)' -- these
are evaluated at assembly-time by the `assembler-eval' as found in [0].

While I'm probably too time-starved to really help with Guile's
assembler, I'd be interested in having, and maybe working on (if time
permits), a Scheme-based assembler targeting ARM platforms, so I might
chime in this area at some point.

It should be fine to use my code (from a copyright view angle) as a
basis something to be incorporated into Guile (even though that file is
marked GPLv2, not GPLv2+), as long as you take care to eliminate the
actual AVR instruction generation code.  This part (i.e., the section
marked "Code emitters") is mostly originally transcribed from the
assembler [2] written in Forth included in AmForth [3], which is GPLv2
(only, unfortunatly).  However, I'm not even sure if the AmForth
copyright is still applicable to that part of my code, as the code has
been transformed substantially by transcription.  That might well be a
moot point, we probably don't want to target 8-bit microcontrollers as
Guile platforms anyway ;-).

[0] http://rotty.xx.vu/gitweb/?p=scheme/avrth.git;a=blob;f=assembler.sls;hb=HEAD
[1] http://rotty.xx.vu/gitweb/?p=scheme/avrth.git;a=summary
[2] 
http://amforth.svn.sourceforge.net/viewvc/amforth/trunk/lib/assembler.frt?revision=1301&view=markup
[3] http://amforth.sourceforge.net/

Regards, Rotty
-- 
Andreas Rottmann -- <http://rotty.xx.vu/>

Re: Adding to the end of the load path

[Andreas Rottmann]

l...@gnu.org (Ludovic Courtès) writes:

> Hi!
>
> Sorry for the delay.
>
> Andreas Rottmann  skribis:
>
>> Ian Price  writes:
>
> [...]
>
>>> Andreas Rottmann suggested something similar in February[1].
>>>
>> I've attached a patch implementing that suggestion, FWIW.
>>
>>> I don't have any concrete proposals better than what Andreas has
>>> suggested, but I felt I should make this post to the list, lest Mark and
>>> I forget.
>>>
>>
>> [...]
>>
>>> 1. http://lists.gnu.org/archive/html/guile-devel/2012-02/msg00038.html
>
> Like Mark, I’m not comfortable with changing the meaning of the empty
> string in the load path, and the behavior of ‘%parse-path’.
>
I agree to that -- there's quite a risk breaking existing setups this
way.

> I pretty much like Mark’s suggestion of using ‘...’ as a special marker,
> even though that’s a valid file name.
>
Well, there's a workaround -- specifying "./..." as an "escape sequence"
for "..." if you really need to have a three-dot relative directory in
the path.

> How would that work for you?
>
I would like the approach using separate _SUFFIX variables better, as it
doesn't have this special case.  While I don't think the special case
will be a problem for a user setting the environment variables
themselves, if you want to set them programatically, you now have to
consider treat "..." specially, escaping it like mentioned above, to be
general.  However, I can live with that, but maybe we can have it both
ways:

- Add the _SUFFIX environment variables, making it clear in the docs
  that they are supported only from Guile 2.0.7 onward.

- Additonally, add "..." as a special marker, but mention it is just
  provided to support Guile < 2.0.7, and should not be used in code that
  needs to depend on Guile 2.0.7 or newer for other reasons
  (e.g. reliance on another added feature or significant bugfix).

I'm not sure how the deprecation strategy is employed exactly, but we
could mark the "..." feature as deprecated right away, or at least in
master, and remove it in 2.2 or 2.4. This would also kind of lift the
"burden" from programs manipulating *_LOAD_PATH programatically, as they
can still be "general" wrt. _undeprecated_ features.  Opinions?

Regards, Rotty
-- 
Andreas Rottmann -- <http://rotty.xx.vu/>

Re: [PATCH] Add missing R6RS `open-file-input/output-port' procedure

[Andreas Rottmann]

Mark H Weaver  writes:

> Hi Andreas,
>
> This patch looks good to me!
>
Thanks for the quick review, also for the other patches! I've pushed
this one now.

Cheers, Rotty
-- 
Andreas Rottmann -- <http://rotty.xx.vu/>

[PATCH] Add missing R6RS `open-file-input/output-port' procedure

[Andreas Rottmann]

* module/rnrs/io/port.scm (r6rs-open): New internal helper procedure for
  opening files.
  (open-file-input-port, open-file-output-port): Make use of
  `r6rs-open'.
  (open-file-input/output-port): Implement in terms of `r6rs-open',
  add to exported identifiers list.

* module/rnrs.scm (open-file-input/output-port): Add to exported
  identifiers.

* test-suite/tests/r6rs-ports.test (test-input-file-opener): New
  procedure, collects several tests for opening file input ports.
  ("7.2.7 Input Ports"): Use `test-input-file-opener' for checking
  `open-file-input-port'.
  (test-output-file-opener): New procedure, collects several tests for
  opening file output ports.
  ("8.2.10 Output ports"): Use `test-output-file-opener' for checking
  `open-file-output-port'.
  ("8.2.13 Input/output ports"): New test prefix, making use of both
  `test-input-file-opener' and `test-output-file-opener' to check
  `open-file-input/output-port'.
---
 module/rnrs.scm  |2 +-
 module/rnrs/io/ports.scm |   70 +--
 test-suite/tests/r6rs-ports.test |   98 +-
 3 files changed, 100 insertions(+), 70 deletions(-)

diff --git a/module/rnrs.scm b/module/rnrs.scm
index 9fff820..a132c53 100644
--- a/module/rnrs.scm
+++ b/module/rnrs.scm
@@ -180,7 +180,7 @@
   call-with-bytevector-output-port
   call-with-string-output-port
   latin-1-codec utf-8-codec utf-16-codec
-  open-file-input-port open-file-output-port
+  open-file-input-port open-file-output-port 
open-file-input/output-port
   make-custom-textual-output-port
   call-with-string-output-port
  flush-output-port put-string
diff --git a/module/rnrs/io/ports.scm b/module/rnrs/io/ports.scm
index fddb491..7c17b0c 100644
--- a/module/rnrs/io/ports.scm
+++ b/module/rnrs/io/ports.scm
@@ -64,7 +64,10 @@
   call-with-string-output-port
   make-custom-textual-output-port
   flush-output-port
-   
+
+  ;; input/output ports
+  open-file-input/output-port
+
   ;; binary output
   put-u8 put-bytevector
 
@@ -305,19 +308,46 @@ read from/written to in @var{port}."
   (with-fluids ((%default-port-encoding "UTF-8"))
 (open-input-string str)))
 
-(define* (open-file-input-port filename
-   #:optional
-   (file-options (file-options))
-   (buffer-mode (buffer-mode block))
-   maybe-transcoder)
+(define (r6rs-open filename mode buffer-mode transcoder)
   (let ((port (with-i/o-filename-conditions filename
 (lambda ()
   (with-fluids ((%default-port-encoding #f))
-(open filename O_RDONLY))
-(cond (maybe-transcoder
-   (set-port-encoding! port (transcoder-codec maybe-transcoder
+(open filename mode))
+(cond (transcoder
+   (set-port-encoding! port (transcoder-codec transcoder
 port))
 
+(define (file-options->mode file-options base-mode)
+  (logior base-mode
+  (if (enum-set-member? 'no-create file-options)
+  0
+  O_CREAT)
+  (if (enum-set-member? 'no-truncate file-options)
+  0
+  O_TRUNC)
+  (if (enum-set-member? 'no-fail file-options)
+  0
+  O_EXCL)))
+
+(define* (open-file-input-port filename
+   #:optional
+   (file-options (file-options))
+   (buffer-mode (buffer-mode block))
+   transcoder)
+  "Return an input port for reading from @var{filename}."
+  (r6rs-open filename O_RDONLY buffer-mode transcoder))
+
+(define* (open-file-input/output-port filename
+  #:optional
+  (file-options (file-options))
+  (buffer-mode (buffer-mode block))
+  transcoder)
+  "Return a port for reading from and writing to @var{filename}."
+  (r6rs-open filename
+ (file-options->mode file-options O_RDWR)
+ buffer-mode
+ transcoder))
+
 (define (open-string-output-port)
   "Return two values: an output port that will collect characters written to it
 as a string, and a thunk to retrieve the characters associated with that port."
@@ -331,23 +361,11 @@ as a string, and a thunk to retrieve the characters 
associated with that port."
 (file-options (file-options))
 (buffer-mode (buffer-mode block))
 maybe-transcoder)
-  (let* ((flags (logior O_WRONLY
-(if (enum-set-member? 'no-create file-options)
-0
-O_CREAT)
-

Re: [PATCH] Fix `get-string-n!' &i/o-decoding exception behavior

[Andreas Rottmann]

Mark H Weaver  writes:

> Andreas Rottmann  writes:
>
>> Mark H Weaver  writes:
>>
>>> Why not leave the API as-is, and in the event of an error, just raise
>>> the proper R6RS exception from within 'scm_get_string_n_x'?
>>>
>> The problem here is that we have no easy way to raise R6RS exceptions
>> from C code, AFAICT.  It is certainly possible, but if it involves
>> convoluted code of doing imports of condition types and appropriate
>> constructors, then constructing a proper invocation, all in C, I'd
>> rather avoid it.
>
> It's not that bad.  In Guile 2.0 we have some convenient procedures for
> accessing arbitrary Scheme variables from C.
>
> Looking at your patch, I see that if '%get-string-n!' returned an error,
> then you did:
>
>   (raise (make-i/o-decoding-error port))
>
> This can be written in C as follows:
>
>   scm_call_1 (scm_c_public_ref ("rnrs exceptions", "raise"),
>   scm_call_1 (scm_c_public_ref ("rnrs io ports",
> "make-i/o-decoding-error")));
>
> Alternatively, you could write one or more private helper procedures in
> Scheme to raise R6RS exceptions, and call those private helpers from C
> using 'scm_c_private_ref' instead of 'scm_c_public_ref'.
>
> What do you think?
>
OK, I'll give this approach a try; I suspect it will indeed result in
the least churn.

Regards, Rotty
-- 
Andreas Rottmann -- <http://rotty.xx.vu/>

Re: Adding to the end of the load path

[Andreas Rottmann]

Mark H Weaver  writes:

> Hi Andreas,
>
> Andreas Rottmann  writes:
>
>> Ian Price  writes:
>>
>> From 1f72ddd3971a796a6f83a68ebedde7d7324c50b5 Mon Sep 17 00:00:00 2001
>> From: Andreas Rottmann 
>> Date: Thu, 8 Nov 2012 01:59:56 +0100
>> Subject: [PATCH] More flexible %load-path and %load-compiled-path handling
>>
>> ---
>>  libguile/load.c |   25 +
>>  1 file changed, 21 insertions(+), 4 deletions(-)
>>
>> diff --git a/libguile/load.c b/libguile/load.c
>> index af2ca45..a05cf76 100644
>> --- a/libguile/load.c
>> +++ b/libguile/load.c
>> @@ -226,7 +226,9 @@ SCM_DEFINE (scm_parse_path, "parse-path", 1, 1, 0,
>>  "Parse @var{path}, which is expected to be a colon-separated\n"
>>  "string, into a list and return the resulting list with\n"
>>  "@var{tail} appended. If @var{path} is @code{#f}, @var{tail}\n"
>> -"is returned.")
>> +"is returned.  In case an empty string occurs as an element\n"
>> +"of @var{path}, @var{tail} is inserted at that point instead 
>> of\n"
>> +"being append at the end of the list.")
>
> Note that 'parse-path' is documented in the manual, which would also
> have to be updated.  However, I'm really not comfortable with changing
> the behavior of 'parse-path'.  It is a documented general-purpose
> procedure which users may use for their own purposes, and I have
> recommended it as such.
>
Good point.

> If we decide to use this general approach, I'd want to leave
> 'parse-path' alone, make a new procedure that behaves differently, and
> then maybe change a few uses of 'parse-path' within Guile to use that
> new procedure.
>
> I'm also not comfortable with having the empty string be the special
> marker separating the prepend and append portions of the path.  As you
> noted in your earlier email, the empty string is currently interpreted
> as the current directory, and users may have come to rely on that.
>
> If we do this, I think the special marker should be a path component
> that is highly unlikely to occur in practice.  Some ideas off the top of
> my head are: "...", "*", or " ".  I think I like "..." best.
>
> What do you think?
>
Another idea would be to use an additional environment variable,
e.g. GUILE_LOAD_{COMPILED_,}_PATH_SUFFIX.  If we cannot use the empty
string as special token, I'd prefer introducing additional environment
variables.  IMO, introducing a "special token", for which there is no
"natural" equivalent is not the most beautiful design.  The empty string
is the only "special token", for which the escape sequence is even more
natural than itself: just use the single dot for the current directory,
instead of the empty string.  I however undertand it's an incompatible
change, so I suggest using additional environment variables.

What's also still missing is a corresponding command-line switch, if we
want to even provide one -- notably, GUILE_LOAD_COMPILED_PATH also has
no corresponding command-line switch.  Any opinion on whether we want
these command-line switches, and if so, suggestions on names?

Regards, Rotty
-- 
Andreas Rottmann -- <http://rotty.xx.vu/>

Re: [PATCH] Fix `get-string-n!' &i/o-decoding exception behavior

[Andreas Rottmann]

Mark H Weaver  writes:

> Hi Andreas,
>
> Thanks for the patch.  See below for my comments.
>
> Andreas Rottmann  writes:
>
>> diff --git a/libguile/ports.c b/libguile/ports.c
>> index 55808e2..b653af4 100644
>> --- a/libguile/ports.c
>> +++ b/libguile/ports.c
>> @@ -1392,12 +1392,10 @@ scm_t_wchar
>>  scm_getc (SCM port)
>>  #define FUNC_NAME "scm_getc"
>>  {
>> -  int err;
>> -  size_t len;
>> +  int err = 0;
>>scm_t_wchar codepoint;
>> -  char buf[SCM_MBCHAR_BUF_SIZE];
>>  
>> -  err = get_codepoint (port, &codepoint, buf, &len);
>> +  codepoint = scm_i_getc (port, &err);
>>if (SCM_UNLIKELY (err != 0))
>>  /* At this point PORT should point past the invalid encoding, as per
>> R6RS-lib Section 8.2.4.  */
>> @@ -1407,6 +1405,20 @@ scm_getc (SCM port)
>>  }
>>  #undef FUNC_NAME
>>  
>> +/* Read a codepoint from PORT and return it.  This version reports
>> +   errors via the ERROR argument instead of via exceptions. */
>> +scm_t_wchar
>> +scm_i_getc (SCM port, int *error)
>> +{
>> +  size_t len;
>> +  scm_t_wchar codepoint;
>> +  char buf[SCM_MBCHAR_BUF_SIZE];
>> +
>> +  *error = get_codepoint (port, &codepoint, buf, &len);
>> +
>> +  return codepoint;
>> +}
>
> Given how trivial 'scm_i_getc' is, I think I'd prefer to leave
> 'scm_getc' alone, to avoid the additional overhead of another non-static
> C function call, which has to be done via the procedure linkage table
> (PLT) when libguile is a shared library and is thus not entirely
> trivial.
>
Yup, that's fine with me.

>> diff --git a/libguile/r6rs-ports.c b/libguile/r6rs-ports.c
>> index e867429..bd10081 100644
>> --- a/libguile/r6rs-ports.c
>> +++ b/libguile/r6rs-ports.c
>> @@ -1242,18 +1242,17 @@ SCM_DEFINE (scm_i_make_transcoded_port,
>>  
>>  /* Textual I/O */
>>  
>> -SCM_DEFINE (scm_get_string_n_x,
>> -"get-string-n!", 4, 0, 0,
>> +SCM_DEFINE (scm_i_get_string_n_x,
>> +"%get-string-n!", 4, 0, 0,
>
> I'm a little bit nervous about this.  Although it is not documented in
> the manual, 'scm_get_string_n_x' is declared in r6rs-ports.h as SCM_API.
> I'm not sure it's safe to make that go away.
>
Oh, I missed that one; thanks.

> Why not leave the API as-is, and in the event of an error, just raise
> the proper R6RS exception from within 'scm_get_string_n_x'?
>
The problem here is that we have no easy way to raise R6RS exceptions
from C code, AFAICT.  It is certainly possible, but if it involves
convoluted code of doing imports of condition types and appropriate
constructors, then constructing a proper invocation, all in C, I'd
rather avoid it.  I think the tendency (in general, also in Guile's
implementation) was to do more things in Scheme, and less in C.

However, exceptions are a difficult topic; if we want efficient
(i.e. ones not requiring setting up exception converters on each call)
implementations of R6RS I/O procedures eventually, we'd either

(a) need to expose exception-less primitives (like I attempted with
scm_i_get_c and scm_i_get_string_n_x), and use those to implement
the actual exception-throwing procedures in Scheme, or

(b) if you really want this done (or doable) fully in C, I think we'd
first provide an API (at least an internal one), to make it possible
to easily raise R6RS conditions from C.

In that sense, it's unfortunate that r6rs-ports.h is public API at all,
since it effectively prevents us from using strategy (a), at least
without going through a deprecation phase. These functions are now all
broken wrt. exceptions -- do we want to fix this at the C level as well,
or rather deprecate them, and DTRT (only) for their Scheme counterparts?

>> diff --git a/module/ice-9/binary-ports.scm b/module/ice-9/binary-ports.scm
>> index c07900b..3f7b9e6 100644
>> --- a/module/ice-9/binary-ports.scm
>> +++ b/module/ice-9/binary-ports.scm
>> @@ -37,14 +37,14 @@
>>  get-bytevector-n!
>>  get-bytevector-some
>>  get-bytevector-all
>> -get-string-n!
>
> Users may have come to rely on this export from (ice-9 binary-ports).
> I don't think it's safe to remove it.
>
Hmm, yeah, that has bothered me a bit as well, but I forgot to
explicitly point it out making the patch potentially unsuitable for
stable.  Nevertheless, having textual I/O procedures in `(ice-9
binary-ports)' seems quite a bit strange; maybe leave it there (for
stable-2.0) but mark that binding as deprecated -- if users want R6RS
textual I/O, they know where to find it ;-). 

Also, I'm not sure if procedures in `(ice-9 ...)' can be expected to be
throw R6RS conditions at all...

Regards, Rotty
-- 
Andreas Rottmann -- <http://rotty.xx.vu/>

Re: [PATCH] Add ".guile.sls" and ".sls" to the default %load-extensions

[Andreas Rottmann]

l...@gnu.org (Ludovic Courtès) writes:

> Hi!
>
> Mark H Weaver  skribis:
>
>> One option would be to return to Aubrey Jaffer's model (used in SCM),
>> where a catalog of modules and their locations is maintained
>
> Hmm, I never really liked that.  And it’s quite an uncommon model.
>
>>> (I’m not sure if searching for a file name with no extension is
>>> intentional.  Ideas?)
>>
>> I looked into this.  Guile has searched for file names with no extension
>> since its initial import into VCS in 1996 (0f2d19dd).  Initially, names
>> with no extension were searched _before_ *.scm, and this logic was
>> hard-coded into the search procedure (try-module-autoload in
>> boot-9.scm).  Jim Blandy introduced the '%load-extensions' variable with
>> default value ("" ".scm") later in 1996 (26544b96), and Mikael Djurfeldt
>> reversed the order to (".scm" "") in 1999 (0a74e31d).  It has been that
>> way ever since.
>
> Woow, thanks for digging it up.  Now we should find out what these guys
> had in mind *before* Guile was first put under version control.  ;-)
>
> Anyway, I still don’t know what to do with .sls.  But the fact that R7
> may come up with its own extension makes me think that maybe we should
> just forget about that extension, as everyone will eventually do.   Hmm?
>
Putting my dorodango hat on, Guile currently provides all mechanisms to
make use of the .sls convention.  A potential --r6rs option seems like a
misnomer to me, due that it would implement an informal convention, not
something really specified in R6RS.

If such an option (maybe under another name) would be added, it would be
effectively just an abbreviation for the right -x command-line options.
Thus it is insignificant to dorodango, since it has to install a
launcher shell script for each program anyway, and whether that shell
script can become a few characters shorter doesn't matter.

What is (relatedly) *indeed* lacking is a way to make native libraries
take precedence to the third-party libraries installed by dorodango.
This would be addressed by making the $GUILE_LOAD_PATH (and
$GUILE_LOAD_COMPILED_PATH) mechanism to not only prepend, but also
append to the compiled-in load path, as implemented by the patch I
posted [0].

[0] http://lists.gnu.org/archive/html/guile-devel/2012-11/msg00056.html
-- 
Andreas Rottmann -- <http://rotty.xx.vu/>

Re: Adding to the end of the load path

[Andreas Rottmann]

Ian Price  writes:

> Hi guilers,
>
> On the #guile IRC channel, Mark Weaver has stressed to me that he thinks
> it would be a good idea for us to be able to add paths to the end of the
> load path, since it would allow us to, for example, prefer a built-in guile
> implementation over a guildhall provided one, when this exists. Right
> now, for the srfi[0] and guile-lib, I have handled this by not adding
> files to the package where they have been adopted by guile (e.g. statprof).
>
> Andreas Rottmann suggested something similar in February[1].
>
I've attached a patch implementing that suggestion, FWIW.

> I don't have any concrete proposals better than what Andreas has
> suggested, but I felt I should make this post to the list, lest Mark and
> I forget.
>

[...]

> 1. http://lists.gnu.org/archive/html/guile-devel/2012-02/msg00038.html

>From 1f72ddd3971a796a6f83a68ebedde7d7324c50b5 Mon Sep 17 00:00:00 2001
From: Andreas Rottmann 
Date: Thu, 8 Nov 2012 01:59:56 +0100
Subject: [PATCH] More flexible %load-path and %load-compiled-path handling

---
 libguile/load.c |   25 +
 1 file changed, 21 insertions(+), 4 deletions(-)

diff --git a/libguile/load.c b/libguile/load.c
index af2ca45..a05cf76 100644
--- a/libguile/load.c
+++ b/libguile/load.c
@@ -226,7 +226,9 @@ SCM_DEFINE (scm_parse_path, "parse-path", 1, 1, 0,
 	"Parse @var{path}, which is expected to be a colon-separated\n"
 	"string, into a list and return the resulting list with\n"
 	"@var{tail} appended. If @var{path} is @code{#f}, @var{tail}\n"
-	"is returned.")
+	"is returned.  In case an empty string occurs as an element\n"
+"of @var{path}, @var{tail} is inserted at that point instead of\n"
+"being append at the end of the list.")
 #define FUNC_NAME s_scm_parse_path
 {
 #ifdef __MINGW32__
@@ -237,9 +239,24 @@ SCM_DEFINE (scm_parse_path, "parse-path", 1, 1, 0,
   
   if (SCM_UNBNDP (tail))
 tail = SCM_EOL;
-  return (scm_is_false (path)
-	  ? tail
-	  : scm_append_x (scm_list_2 (scm_string_split (path, sep), tail)));
+  if (scm_is_false (path))
+return tail;
+  else
+{
+  SCM elements = scm_string_split (path, sep);
+  SCM seen = SCM_EOL;
+  SCM rest = elements;
+
+  for (; scm_is_pair (rest); rest = SCM_CDR (rest))
+{
+  if (scm_is_true (scm_string_null_p (SCM_CAR (rest
+return scm_append_x (scm_list_2 (scm_reverse_x (seen, tail),
+ SCM_CDR (rest)));
+  seen = scm_cons (SCM_CAR (rest), seen);
+}
+
+  return scm_append_x (scm_list_2 (elements, tail));
+}
 }
 #undef FUNC_NAME
 
-- 
1.7.10.4


Regards, Rotty
-- 
Andreas Rottmann -- <http://rotty.yi.org/>

[PATCH] Fix `get-string-n!' &i/o-decoding exception behavior

[Andreas Rottmann]

Previously, `get-string-n!' from `(rnrs io ports)' would not throw the
exception required by R6RS, and could not easily do so due to being
implemented entirely in C.

This change fixes this by introducing a corresponding internal C
function reporting errors by return value and reimplementing the
`get-string-n!' in Scheme on top of that.  Along with `get-string-n!',
`get-string-n' gets fixed, inheriting the correct behavior.

* libguile/ports.c (scm_i_getc): New function, a version of `scm_getc'
  not using exceptions.
  (scm_getc): Implemented using `scm_i_getc'.
* libguile/ports.h (scm_i_getc): Add prototype marked SCM_INTERNAL.

* libguile/r6rs-ports.c (scm_i_get_string_n_x): Exception-free version
  of `get-string-n!', making use of `scm_i_getc'.
  (scm_get_string_n_x): Removed, now implemented in Scheme.

* module/ice-9/binary-ports.scm (get-string-n!): Removed from export
  list, it doesn't fit the module module purpose anyway.
* module/rnrs/io/ports.scm (%get-string-n): Newly defined by internal
  reference to `(ice-9 binary-ports)'.
  (get-string-n!): Implemented in Scheme on top of `%get-string-n!'.

* test-suite/tests/r6rs-ports.test ("8.2.9 Textual
  input")["read-error"]: Activate commented-out exception-behavior tests
  of `get-string-n!'.
  ["decoding error"]: New test prefix with tests for `get-char',
  `get-string-n!' and `get-string-n' and `get-line'.
---
 libguile/ports.c |   20 
 libguile/ports.h |1 +
 libguile/r6rs-ports.c|   21 +++--
 module/ice-9/binary-ports.scm|6 +++---
 module/rnrs/io/ports.scm |   14 ++
 test-suite/tests/r6rs-ports.test |   24 
 6 files changed, 65 insertions(+), 21 deletions(-)

diff --git a/libguile/ports.c b/libguile/ports.c
index 55808e2..b653af4 100644
--- a/libguile/ports.c
+++ b/libguile/ports.c
@@ -1392,12 +1392,10 @@ scm_t_wchar
 scm_getc (SCM port)
 #define FUNC_NAME "scm_getc"
 {
-  int err;
-  size_t len;
+  int err = 0;
   scm_t_wchar codepoint;
-  char buf[SCM_MBCHAR_BUF_SIZE];
 
-  err = get_codepoint (port, &codepoint, buf, &len);
+  codepoint = scm_i_getc (port, &err);
   if (SCM_UNLIKELY (err != 0))
 /* At this point PORT should point past the invalid encoding, as per
R6RS-lib Section 8.2.4.  */
@@ -1407,6 +1405,20 @@ scm_getc (SCM port)
 }
 #undef FUNC_NAME
 
+/* Read a codepoint from PORT and return it.  This version reports
+   errors via the ERROR argument instead of via exceptions. */
+scm_t_wchar
+scm_i_getc (SCM port, int *error)
+{
+  size_t len;
+  scm_t_wchar codepoint;
+  char buf[SCM_MBCHAR_BUF_SIZE];
+
+  *error = get_codepoint (port, &codepoint, buf, &len);
+
+  return codepoint;
+}
+
 /* this should only be called when the read buffer is empty.  it
tries to refill the read buffer.  it returns the first char from
the port, which is either EOF or *(pt->read_pos).  */
diff --git a/libguile/ports.h b/libguile/ports.h
index d4d59b7..2f70056 100644
--- a/libguile/ports.h
+++ b/libguile/ports.h
@@ -281,6 +281,7 @@ SCM_API SCM scm_force_output (SCM port);
 SCM_API SCM scm_flush_all_ports (void);
 SCM_API SCM scm_read_char (SCM port);
 SCM_API scm_t_wchar scm_getc (SCM port);
+SCM_INTERNAL scm_t_wchar scm_i_getc (SCM port, int *error);
 SCM_API size_t scm_c_read (SCM port, void *buffer, size_t size);
 SCM_API void scm_c_write (SCM port, const void *buffer, size_t size);
 SCM_API void scm_lfwrite (const char *ptr, size_t size, SCM port);
diff --git a/libguile/r6rs-ports.c b/libguile/r6rs-ports.c
index e867429..bd10081 100644
--- a/libguile/r6rs-ports.c
+++ b/libguile/r6rs-ports.c
@@ -1242,18 +1242,17 @@ SCM_DEFINE (scm_i_make_transcoded_port,
 
 /* Textual I/O */
 
-SCM_DEFINE (scm_get_string_n_x,
-"get-string-n!", 4, 0, 0,
+SCM_DEFINE (scm_i_get_string_n_x,
+"%get-string-n!", 4, 0, 0,
 (SCM port, SCM str, SCM start, SCM count),
-"Read up to @var{count} characters from @var{port} into "
-"@var{str}, starting at @var{start}.  If no characters "
-"can be read before the end of file is encountered, the end "
-"of file object is returned.  Otherwise, the number of "
-"characters read is returned.")
-#define FUNC_NAME s_scm_get_string_n_x
+"Read up to @var{count} characters from @var{port} into @var{str}, 
"
+"starting at @var{start}. Returns the end of file object, a list "
+"containing an errno value, or the number of characters read.")
+#define FUNC_NAME s_scm_i_get_string_n_x
 {
   size_t c_start, c_count, c_len, c_end, j;
   scm_t_wchar c;
+  int err;
 
   SCM_VALIDATE_OPINPORT (1, port);
   SCM_VALIDATE_STRING (2, str);
@@ -1267,8 +1266,10 @@ SCM_DEFINE (scm_get_string_n_x,
 
   for (j = c_start; j < c_end; j++)
 {
-  c = scm_getc (port);
-  if (c == EOF)
+  c = scm_i_getc (port, &err);
+  if (SCM_UNLIKELY (err != 0))
+return scm_

Re: Release time!

[Andreas Rottmann]

Ian Price  writes:

> l...@gnu.org (Ludovic Courtès) writes:
>
>>> * Figure out a way to make Guildhall modules that will be overridden by
>>>   a matching module in core guile (if it exists).  This is important for
>>>   SRFIs.  Ian Price's Guildhall repository contains portable
>>>   implementions of several SRFIs that might become part of core Guile in
>>>   the future, and the core versions should take priority.
>>
>> Could guildhall use SRFI-0 to check whether a given SRFI is already
>> provided by the host’s Guile, and determine based on that whether to
>> install its own version?
>
> Well, maybe I could hack something that uses srfi-0, but it sounds kinda
> ugly, and liable to break if a guile upgrade changed the features it exported.
>
> Right now, a package can declare multiple 'provides' so that you can
> e.g. require srfi-1 and it would pull in the appropriate package. But as
> it stands, the provides are somewhat orthogonal to how the code gets
> installed.
>
> Andreas,
> Guildhall is a friendly fork of Dorodango, so what do you think about
> adding this sort of thing?
>
I've thought about this issue some time ago, but have not yet come to a
definite conclusion on how to handle this.  A rough idea is to add a
"proxy package" for the Scheme implementation of the destination
(i.e. installation target) to the `dorodango-support' bundle, which is
created upon initialization of the destination.

If that proxy package would "provide" (e.g. based on a feature check)
all SRFIs that are present in the target Scheme (aka "Guile" from now
on), that would bring us closer to the goal. If all SRFIs required by
the set of packages installed are indeed provided by Guile, no
additional package would be pulled in.  However, if a single package
provides multiple SRFIs, of which only some are provided by Guile, that
package will get installed to satisfy any requirements not met by the
proxy package, and will thus *override* any core SRFI implementation in
Guile that it provides as well.

However, there's a reasonable (I think) workaround: no longer `provide'
the SRFIs, but instead really use individual packages for them, so they
can be installed on an individual basis.  This is not as tedious as it
might sound: the pkg-list.scm file would have to list each SRFI, but the
collection of portable SRFIs can still be maintained in a single source
repository and dorodango bundle (i.e. unit of distribution).

Regards, Rotty
-- 
Andreas Rottmann -- <http://rotty.yi.org/>

Re: [PATCH] Make `get-datum' conform more closely to R6RS semantics

[Andreas Rottmann]

Mark H Weaver  writes:

> Hi Andreas,
>
> Andreas Rottmann  writes:
>> * module/rnrs/io/ports.scm (get-datum): Set reader options to be more
>>   compatible with R6RS syntax.
>>
>>   With Guile's default reader options, R6RS hex escape and EOL escape
>>   behavior is missing.  This change enables the former via the
>>   `r6rs-hex-escapes' option, and gets us closer to the latter by setting
>>   `hungry-eol-escapes'.
>>
>> * test-suite/tests/r6rs-ports.test ("8.2.9 Textual input")["get-datum"]:
>>   New tests.
>> ---
>>  module/rnrs/io/ports.scm |   13 +++--
>>  test-suite/tests/r6rs-ports.test |   28 
>>  2 files changed, 39 insertions(+), 2 deletions(-)
>>
>> [... patch elided ... ]

>
> The problem with the approach above is that it sets the read options
> globally, which is obviously a bad idea in a multithreaded program.
>
Oops, yes. I was under the impression that the read options are stored
in a fluid, but I didn't verify that assumption.

> Until very recently there was no other practical option, but now 'read'
> starts by building a private struct 'scm_t_read_opts' and passes it down
> to all the helper functions explicitly.  This was partly what enable
> per-port read options, which are now supported internally and accessible
> using reader directives such as #!fold-case, #!no-fold-case, and
> #!curly-infix.  It also means that it is now feasible to provide another
> 'read' procedure that accepts a set of read options explicitly.
>
> I've been avoiding adding a public API for this, because I feel that the
> current 'read-options' API is poorly-designed and I'd rather take the
> opportunity to come up with a clean design.
>
> For now, I suggest that we add 'get-datum' as a C function in read.c,
> which initializes the 'scm_t_read_opts' as needed for R6RS.
>
Yup, I've seen that change while skimming over recent changes in
stable-2.0, and have come to the conclusion that a solution that would
directly call `read' with an R6RS set of options would be better. My
patch has originated some way back -- I'll rework it along the lines of
your proposal.

> Also, while we're on the subject, now that we have per-port read
> options, perhaps #!r6rs ought to set some of them instead of being a
> no-op.  See 'scm_read_shebang' in read.c.
>
Yep, I think that's a good idea. This can also spare us from setting
them as part of an --r6rs command-line option, which I suggested in the
%load-extensions thread
<http://lists.gnu.org/archive/html/guile-devel/2012-11/msg00018.html>.

Regards, Rotty
-- 
Andreas Rottmann -- <http://rotty.yi.org/>

Re: [PATCH] Add ".guile.sls" and ".sls" to the default %load-extensions

[Andreas Rottmann]

l...@gnu.org (Ludovic Courtès) writes:

> Hi!
>
> Andreas Rottmann  skribis:
>
>> For performance reasons, it *might* make sense to not enable this
>> behavior by default, but provide a command-line switch; however,
>> enabling this behavior is *already* possible using command-line switches
>> ("-x .guile.sls -x .sls"), so I don't know...
>
> Right.  We’d endup stat’ing two more files for each module, right?
>
Indeed.

> How much of a hindrance is it to *not* have these in the default
> extension list?
>
Not a hindrance, just an annoyance, IMHO.  Maybe a reasonable compromise
would be to add an --r6rs switch, to add these extensions, as well as
turn on any R6RS-compatiblity reader options, so the script being
executed can use the R6RS syntax we already support, even when it is
disabled by default (cf. my latest patch). On the other hand, these
extensions are not part of R6RS and orthogonal to whether the script to
be run uses R6RS syntax incompatible with the default reader options.

Regards, Rotty
-- 
Andreas Rottmann -- <http://rotty.yi.org/>

[PATCH] Make `get-datum' conform more closely to R6RS semantics

[Andreas Rottmann]

* module/rnrs/io/ports.scm (get-datum): Set reader options to be more
  compatible with R6RS syntax.

  With Guile's default reader options, R6RS hex escape and EOL escape
  behavior is missing.  This change enables the former via the
  `r6rs-hex-escapes' option, and gets us closer to the latter by setting
  `hungry-eol-escapes'.

* test-suite/tests/r6rs-ports.test ("8.2.9 Textual input")["get-datum"]:
  New tests.
---
 module/rnrs/io/ports.scm |   13 +++--
 test-suite/tests/r6rs-ports.test |   28 
 2 files changed, 39 insertions(+), 2 deletions(-)

diff --git a/module/rnrs/io/ports.scm b/module/rnrs/io/ports.scm
index fddb491..6e6a66d 100644
--- a/module/rnrs/io/ports.scm
+++ b/module/rnrs/io/ports.scm
@@ -1,6 +1,6 @@
  ports.scm --- R6RS port API-*- coding: utf-8 -*-
 
-   Copyright (C) 2009, 2010, 2011 Free Software Foundation, Inc.
+   Copyright (C) 2009, 2010, 2011, 2012 Free Software Foundation, Inc.
 
  This library is free software; you can redistribute it and/or
  modify it under the terms of the GNU Lesser General Public
@@ -437,7 +437,16 @@ return the characters accumulated in that port."
   (with-textual-input-conditions port (read-char port)))
 
 (define (get-datum port)
-  (with-textual-input-conditions port (read port)))
+  (with-textual-input-conditions port
+(let ((saved-options (read-options)))
+  (dynamic-wind
+(lambda () (read-options '(positions
+   keywords #f
+   square-brackets
+   r6rs-hex-escapes
+   hungry-eol-escapes)))
+(lambda () (read port))
+(lambda () (read-options saved-options))
 
 (define (get-line port)
   (with-textual-input-conditions port (read-line port 'trim)))
diff --git a/test-suite/tests/r6rs-ports.test b/test-suite/tests/r6rs-ports.test
index 46da67f..6a7386e 100644
--- a/test-suite/tests/r6rs-ports.test
+++ b/test-suite/tests/r6rs-ports.test
@@ -719,6 +719,34 @@
   (and (= 3 (get-string-n! port s 6 3))
(string=? s "Isn't GNU great?"
 
+  (with-test-prefix "get-datum"
+(let ((string->datum (lambda (s) (get-datum (open-input-string s)
+  (pass-if "symbol"
+(eq? (string->datum "foo") 'foo))
+  (pass-if "symbol [starting with colon]"
+(eq? ':foo (string->datum ":foo")))
+  (pass-if "string"
+(string=? "foo" (string->datum "\"foo\"")))
+  (pass-if "string [with hex escapes]"
+(string=? "bar\nA" (string->datum "\"bar\\x0A;\\x41;\"")))
+  (pass-if "string [hungry EOL]"
+(string=? "bar baz" (string->datum "\"bar \\\n   baz\"")))
+  ;; FIXME: actually, R6RS demands an even more hungry EOL escape
+  ;; than the reader currently implements: also any whitespace
+  ;; between the backslash and the newline should vanish. Currently,
+  ;; the reader barfs on that.
+  (pass-if "string [hungry EOL, space also before newline]"
+(throw 'unresolved)
+(string=? "bar baz" (string->datum "\"bar \\  \n   baz\"")))
+  (pass-if "number [decimal]"
+(= (string->datum "42") 42))
+  (pass-if "number [hexadecimal]"
+(= (string->datum "#x2A") 42))
+  (pass-if "number [octal]"
+(= (string->datum "#o0777") 511))
+  (pass-if "number [binary]"
+(= (string->datum "#b101010") 42
+
   (with-test-prefix "read error"
 (pass-if-condition "get-char" i/o-read-error?
   (get-char (make-failing-port)))
-- 
1.7.10.4

Re: [PATCH] Add ".guile.sls" and ".sls" to the default %load-extensions

[Andreas Rottmann]

l...@gnu.org (Ludovic Courtès) writes:

> Hi,
>
> Mark H Weaver  skribis:
>
>> Any objections to adding ".guile.sls" and ".sls" to Guile's default
>> %load-extensions?
>
> OK for .sls, but why .guile.sls?
>
This is used for compatibility libraries, for instance:

   % ls spells/filesys/
   compat.guile.sls   compat.larceny.sls  compat.mzscheme.sls
   compat.ikarus.sls  compat.mosh.sls compat.ypsilon.sls

Only guile will have .guile.sls in its %load-extensions, so it will find
the correct library, in this case corresponding to the library name
`(spells filesys compat)'.

Other R6RS implementations, if the adhere to this convention, will only
consider "their" specific file, as they'd only look for
spells/filesys/compat.sls and
spells/filesys/compat..sls.  If there's a portable
implementation of that library, it will be under
spells/filesys/compat.sls, and be used (only) if there's no
implementation-specific file. So it's important that the ".guile.sls"
extension is considered *before* ".sls".

For performance reasons, it *might* make sense to not enable this
behavior by default, but provide a command-line switch; however,
enabling this behavior is *already* possible using command-line switches
("-x .guile.sls -x .sls"), so I don't know...

Regards, Rotty
-- 
Andreas Rottmann -- <http://rotty.yi.org/>

Re: Our temporary guildhall package repository down?

[Andreas Rottmann]

Nala Ginrut  writes:

> http://rotty.yi.org/doro/experimental is unavailable for me. And ijp
> said it's the same for him.
>
> But Rottmann's site is OK.
>
Yeah, seems I didn't copy the repository over when I moved the site onto
the new host.

> Well, if there's some trouble with his site to put guildhall package
> repository, I can provide a temporary repository from my web host.
>
I think there is a misunderstanding here: guildhall is a (friendly) fork
of dorodango; it's not yet clear if and how repositories would be shared
between dorodango (which targets R6RS implementations, primarily Guile
and Racket) and guildhall (which targets Guile only). 

> I think this may let us test guildhall till it really works for Guile
> at least.  Or we may have one more alternative repository.
>
I think guildhall definitly should have an separate repository (hosted
on official GNU resources), which perhaps might be used in combination
with dorodango's repository (or repositories). I'd also be happy to have
some directory on the (proposed) guildhall host to upload dorodango
(i.e. pure R6RS) packages to.

One thing that needs to be done IMO before a repository (with multiple
uploaders) is feasible, is a (secure) way to upload.  I wrote a script
some time ago based on the idea of an anonymous upload directory, which
packagers would upload packages along with detached GPG signatures.  If
signatures are valid and are made be a a group of people inside a GPG
keyring (i.e. allowed uploaders), these get moved into the actual
repository.  This is similiar (in principle) to how Debian's incoming
queue works.

The script is in `scripts/doro-update-archive' in the dorodango source
tree, but I've not recently checked if it still works, so YMMV.  It
should give you the gist of the idea, and some starting point if someone
wants to pick up this work.

What's also missing is an web-view of the repository.  I also have
started working on that, but likewise, it's unfinished, and much more so
than the upload script (which should be already usable, modulo bugs and
bitrot).  I've pushed the work already done to the `wip-web' branch.

If someone is interested on hacking on these two sub-projects, I'd be
happy to help along.

> Could anyone give me all the things of 'experimental' directory? I can
> upload them at once.
>
The stuff that used to be there is quite outdated, and it doesn't make
much sense IMO to put it anywhere.  For dorodango, I have in mind
finishing an outstanding feature (support akin to automake's DESTDIR
support), go over the documentation, and make an initial release.
However, my hacking time ATM is quite limited, so I cannot give any date
of when this will eventually happen.

Regards, Rotty
-- 
Andreas Rottmann -- <http://rotty.yi.org/>

Re: Psyntax security hole prevents secure sandboxing in Guile

[Andreas Rottmann]

Noah Lavine  writes:

>> Can you think of anything else that would need to be fixed, besides this
>> problem with forgeable syntax-objects?
>
> It depends how much of a sandbox you're thinking of, but I'd like to
> make sure that the untrusted code didn't go into an infinite loop,
> which means either putting it in a separate process or having a timer
> that would stop it after a deadline. Also you'd have to make sure that
> you didn't run any procedure returned by the untrusted code, for the
> same reason.
>
> Also, what if the untrusted code allocated a lot of memory? I suppose
> you could depend on that all being garbage-collected after it
> finished, but you'd have to be prepared to handle out-of-memory errors
> while it was running.
>
> It might be easiest to just put it in a separate process, although
> that would make communication harder.
>
Racket has a facility that achieves sandboxing (with the above property
of CPU and RAM usage bounds), i believe:

http://docs.racket-lang.org/reference/Sandboxed_Evaluation.html

Regards, Rotty
-- 
Andreas Rottmann -- <http://rotty.yi.org/>

Re: Comments on ‘wip-nio’

[Andreas Rottmann]

Nala Ginrut  writes:

> On Thu, Mar 22, 2012 at 6:39 AM, Ludovic Courtès  wrote:
>
>> Hello,
>>
>> I had a quick look at ‘wip-nio’, and here are initial comments.
>>
>>  • epoll is Linux-specific.  Any idea how a more multi-platform API
>>could be provided?  I guess libevent could be used, but we don’t
>>want to add one more dependency in 2.0.  How much is (ice-9 nio)
>>dependent on epoll vs. poll, for instance?
>>
>>
> Maybe add a module named "(ice-9 linux)"? And we may add more
> Linux-specific things.
> Also "(ice-9 bsd)" which provides "kqueue" or other interesting things.
> If users try to use BSD-specific modules under Linux, we may throw an
> exception. And functions like
> "under-bsd?" or "under-linux?" maybe helpful.
> Anyway, libevent for Guile is also a good idea. But I don't think libevent
> should integrate into Guile. We can write
> a standalone guile-libevent.
>
I've been looking a bit at C event loop libraries, and just wanted to
note that libev[0] seems like a viable alternative to libevent.

[0] http://software.schmorp.de/pkg/libev.html

Regards, Rotty
-- 
Andreas Rottmann -- <http://rotty.yi.org/>

Re: Non-stack-copying call-with-current-continuation?

[Andreas Rottmann]

David Kastrup  writes:

> Noah Lavine  writes:
>
>> Hello,
>>
>>> Sure, but things like gensym and make-prompt-tag (and (list '()) for
>>> creating an eq?-unique object) are artificial hygiene coming at a cost
>>> in symbol table and symbol generation time rather than "lexical"
>>> hygiene.  They need _extra_ work, whereas the
>>> call-with-current-continuation approach needed _less_ work.  Basically I
>>> want something like call-with-single-continuation that will only allow
>>> one return (and any dynwind out counts and should work if it is the
>>> first, so it is not exactly equivalent to using
>>> with-continuation-barrier) and come without the stack-copying cost of
>>> call-with-current-continuation.
>>
>> I agree that it's not pretty. We have hygienic macros so we don't have
>> to use gensym, after all. But I don't know of a better way.
>
> Well, to wrap this up: the manual (not current) states
>
> It is traditional in Scheme to implement exception systems using
> `call-with-current-continuation'.  Continuations (*note
> Continuations::) are such a powerful concept that any other control
> mechanism -- including `catch' and `throw' -- can be implemented in
> terms of them.
>
> [...]
>
>The more targeted mechanism provided by `catch' and `throw' does not
> need to save and restore the C stack because the `throw' always jumps
> to a location higher up the stack of the code that executes the
> `throw'.  Therefore Guile implements the `catch' and `throw' primitives
> independently of `call-with-current-continuation', in a way that takes
> advantage of this _upwards only_ nature of exceptions.
>
>
> I think that using something like "call-with-single-continuation" as the
> underlying primitive would make Guile quite more similar to
> "traditional" systems in the code base.  It would also provide a
> minimally-invasive tool for tuning existing code based on
> call-with-current-continuation in case that the stack copying semantics
> are _not_ required.  Definitely more Schemeish than stuff like, uh,
> prompts?
>
Just to throw my two cents in: Racket (and probably other Schemes)
provide this primitive under the name call-with-escape-continuation
(call/ec):

http://docs.racket-lang.org/reference/cont.html?q=call/ec#%28def._%28%28quote._~23~25kernel%29._call-with-escape-continuation%29%29

Regards, Rotty
-- 
Andreas Rottmann -- <http://rotty.yi.org/>

Re: Non-stack-copying call-with-current-continuation?

[Andreas Rottmann]

David Kastrup  writes:

> Noah Lavine  writes:
>
>> Hello,
>>
>>> Sure, but things like gensym and make-prompt-tag (and (list '()) for
>>> creating an eq?-unique object) are artificial hygiene coming at a cost
>>> in symbol table and symbol generation time rather than "lexical"
>>> hygiene.  They need _extra_ work, whereas the
>>> call-with-current-continuation approach needed _less_ work.  Basically I
>>> want something like call-with-single-continuation that will only allow
>>> one return (and any dynwind out counts and should work if it is the
>>> first, so it is not exactly equivalent to using
>>> with-continuation-barrier) and come without the stack-copying cost of
>>> call-with-current-continuation.
>>
>> I agree that it's not pretty. We have hygienic macros so we don't have
>> to use gensym, after all. But I don't know of a better way.
>
> Well, to wrap this up: the manual (not current) states
>
> It is traditional in Scheme to implement exception systems using
> `call-with-current-continuation'.  Continuations (*note
> Continuations::) are such a powerful concept that any other control
> mechanism -- including `catch' and `throw' -- can be implemented in
> terms of them.
>
> [...]
>
>The more targeted mechanism provided by `catch' and `throw' does not
> need to save and restore the C stack because the `throw' always jumps
> to a location higher up the stack of the code that executes the
> `throw'.  Therefore Guile implements the `catch' and `throw' primitives
> independently of `call-with-current-continuation', in a way that takes
> advantage of this _upwards only_ nature of exceptions.
>
>
> I think that using something like "call-with-single-continuation" as the
> underlying primitive would make Guile quite more similar to
> "traditional" systems in the code base.  It would also provide a
> minimally-invasive tool for tuning existing code based on
> call-with-current-continuation in case that the stack copying semantics
> are _not_ required.  Definitely more Schemeish than stuff like, uh,
> prompts?
>
Just to throw my two cents in: Racket (and probably other Schemes)
provide this primitive under the name call-with-escape-continuation
(call/ec):

http://docs.racket-lang.org/reference/cont.html?q=call/ec#%28def._%28%28quote._~23~25kernel%29._call-with-escape-continuation%29%29

Regards, Rotty
-- 
Andreas Rottmann -- <http://rotty.yi.org/>

Extending default %load-path via environment variable

[Andreas Rottmann]

Hi!

I just noticed that Guile apparently offers no way to _extend_ (as
opposed to completely override) its load path (or compiled load path)
via an environment variable.  What I'd like to have is something like
Racket, where setting PLTCOLLECTS to "/foo/bar::/qux/baz" means: Search
/foo/bar first, then the default path (whatever it may be), then
/qux/baz.

I'm not fixed to having this functionality exposed via an environment
variable, extending the set of options would be OK as well; but still,
there is an option missing to achieve the effect described above -- one
cannot add _after_ the default load path, just prepend to it.  Also,
besides the environment variable GUILE_LOAD_COMPILED_PATH, there seems
to be no possibility to influence %compiled-load-path, which hence also
lacks extensibility without knowing (or determining) the default value
beforehand.

Here is my proposal:

- Change `parse-path' to insert the default value if there is any empty
  element in the path.

- Add command-line options to
  - Append to the %load-path
  - Prepend and append to %load-compiled-path

Ideally, I'd like to add this functionality to stable-2.0, but it is an
incompatible change: ATM, an empty component is treated as "current
directory".  Adding the current directory could still be done in my
proposal by using "." (single dot).  Requiring an explicit dot would
also diminish the odds of unintentionally adding the current directory
via string concatenation in the shell: 

  GUILE_LOAD_PATH="$GUILE_LOAD_PATH:/my/directory"

The above code would currently add the current directory to the path if
$GUILE_LOAD_PATH is empty, while it would have the (probably) intended
meaning even under my proposal in any case.

If this change is deemded not suitable for stable-2.0, I'd like to add
(only) the command-line options in stable-2.0, and make the `parse-path'
changes in master only. Suggestions for option naming welcome!

WDYT?
-- 
Andreas Rottmann -- <http://rotty.yi.org/>

Re: Minimal Guile

[Andreas Rottmann]

Andy Wingo  writes:

> On Tue 03 Jan 2012 22:44, Mark H Weaver  writes:
>
>> If this single file is to be built only during the package build
>> process, and never modified after that, then it should be super-easy to
>> invent your own little homebrew format.
>
> Indeed.  It would be nice to use ELF, though.  I'd like to change the
> format of .go files to ELF in 2.2.  That would allow lots of things,
> like having multiple entry points (as for different modules).  It would
> also help to further separate debug information from code (helping
> performance), statically allocating data (lowering allocation,
> increasing cross-process sharing), and allowing for extensibility,
> including sections for natively-compiled code.
>
> But if you (or anyone) wants to work on something simpler, that's cool
> too :)  I do have a Scheme ELF parser around somewhere, and intend to
> get back to the "linker" sometime soon.
>
FWIW, I have written a simple ELF generator in R6RS, which can be found
here:

http://rotty.xx.vu/gitweb/?p=scheme/avrth.git;a=blob;f=elf.sls;hb=master

Regards, Rotty
-- 
Andreas Rottmann -- <http://rotty.yi.org/>

G-Wrap 1.9.14 released

[Andreas Rottmann]

Hi!

I'm pleased to announce G-Wrap 1.9.14; as always, it can be downloaded
from <http://www.nongnu.org/g-wrap/>.

This release adds support for the Guile 2.0.x branch, while staying
compatible with 1.8.x.

About G-Wrap:

  G-Wrap is a tool (and Guile library) for generating function wrappers
  for inter-language calls. It currently only supports generating Guile
  wrappers for C functions.

Changes since 1.9.13:

- Guile 2.0 is now supported.

- G-Wrap no longer ships SRFI 34 and SRFI 35 implementations, as
  these have been both provided by Guile since 1.8.3.

Regards, Rotty
-- 
Andreas Rottmann -- <http://rotty.yi.org/>

Re: guild hall update

[Andreas Rottmann]

Andy Wingo  writes:

> Hi all,
>
> Another status update on the construction of the guild hall.
>
> Since my last update, I moved all of the commands down from under the
> "guild hall" namespace, so that it's now "guild update", "guild
> install", etc.  I also hacked a bit on the `guild' executable itself,
> allowing "guild help FOO" to work, and generally making `guild' more
> pleasant to use.
>
> Unlike dorodango, 

[... changes elided ...].

> So those are the changes.  Now, the problems.  The biggest problem right
> now is that I don't know how it's supposed to work, by default.  Should
> I be able to "guild install foo" and then run guile, and import the
> (foo) module directly?  That's not how it currently works, because it
> installs to somewhere in ~/.local by default.  Should I be able to
> install to the configured --prefix ?  Should that be the default?  If we
> install to the --prefix, is it in a guild hall path or is it in Guile's
> site dir?  Should programs installed by the guild be in the default
> path?
>
> Also, many upstream dorodango R6RS packages still wouldn't work, because
> Guile doesn't do the .guile.sls / .sls thing by default.  I think Guile
> needs an --r6rs option that enables that and other R6RS options.
>
The way dorodango works (for programs) is that it installs a shell
script wrapper for the configured implementation, which sets up the
implementation-specific load-path and any options needed such as -x
.guile.sls -x .sls for Guile.  So for mere users, all they need to do is
place e.g. ~/.local/bin into $PATH.  I've tested that this works nicely
for Guile as well; in fact, I'm using a few small programs (and
dependent libraries) installed by dorodango, using Guile as
implementation, almost daily.

Scheme hackers OTOH are expected to modify their preferred
implementation's load-path appropriatly, and invoke their REPL of choice
with the correct incantation.  Of course, an --r6rs shorthand option for
Guile would help here.

> Sigil doesn't currently compile files that it installs.  It probably
> should.  It should probably recompile depending modules too, when
> updating a package.
>
Yeah, that's indeed a feature I'd really like to add to dorodango, but
I'm currently very short on personal hacking time.

Regards, Rotty
-- 
Andreas Rottmann -- <http://rotty.yi.org/>

Re: guildhall status

[Andreas Rottmann]

Andy Wingo  writes:

> Hi!
>
> An update on the guild-hall.
>
> On Fri 15 Jul 2011 13:55, Andy Wingo  writes:
>
>>   git clone git://gitorious.org/~wingo/dorodango/guildhall.git
>
> It is still here, with lots of activity, mostly administrative.  It can
> now be installed, has minimal docs (from dorodango), and passes
> distcheck.  I trimmed dependencies and otherwise folded things under the
> (guild ...) namespace.
>
> Also it is the (guild ...) namespace right now -- I am having trouble
> with metaphors.  It seems like the guild-hall package doesn't actually
> install the guild hall, more like it gives access to guild halls that
> might be on the internet.  Anyway, something to figure out.  I think
> that the right answer will come once I move the commands down from under
> "guild hall" -- so "guild update" instead of "guild hall update".
>
Yeah, maybe a "hall" is what dorodango calls (in its somewhat
old-fashioned way ;-)) a "repository".

>> Next up:
>>
>>   1) Check status of dorodango functionality.
>>   2) Fix things that don't work.
>>   3) Profit?
>>   4) Start thinking about hosting and accounts and UX and stuff.
>
> I still haven't made much progress on the first three, but as regards
> the last:
>
>> I will see if I can get work to sponsor a server that we can use, and
>> see if we can get it aliased to guildhall.gnu.org -- unless someone else
>> would like to provide the server.  It would be nice to have root on that
>> server, FWIW.  It could be a VM.
>
> Igalia did kindly offer, but Ludovic made the good point that we should
> try the FSF first, and they do seem to be able to host a VM for us, so
> we will probably use the FSF.
>
> But beyond hosting the bundles and available.scm, I am still not clear
> on what guildhall.gnu.org should offer.  It should probably have a
> facility for submitting packages, along with a GPG keyring.  Perhaps it
> should run tests on the whole guildhall archive every so often, for
> consistency and also for style perhaps.  Perhaps it should host
> documentation.  Who knows!
>
I've been hacking on two things in this general area:

- A Scheme script to allow updates to a repository, verifying uploaded
  packages (from a local directory) against a GPG keyring using detached
  signatures (using gpgv(1) for signature verification).

- A web interface somewhat similiar to <http://packages.debian.org>,
  based on SSAX and the ocelotl libsoup-based HTTPd.  Perhaps I will
  implement the ocelotl HTTPd API on top of Guile's webserver -- this
  would allow code to be shared quite seamlessly.

I can push both, if there's interest, but the web interface is in its
very early stages ATM.

Regards, Rotty
-- 
Andreas Rottmann -- <http://rotty.yi.org/>

Re: guildhall status

[Andreas Rottmann]

Andy Wingo  writes:

> Hi,
>
> I wanted to bootstrap the guildhall stuff, so last week I started poking
> at Andreas' excellent dorodango (http://www.nongnu.org/dorodango/).  I
> forked his repo on gitorious, removed a bunch of compat stuff for other
> implementations, imported dependencies into the archive, replaced the
> http stuff with Guile's (web) foo, and wired up a standard build system.
>
> The result is here:
>
>   https://gitorious.org/~wingo/dorodango/guildhall
>
> You can:
>
>   git clone git://gitorious.org/~wingo/dorodango/guildhall.git
>
> Then ./configure && make && make check.
>
> You will only be able to succeed there if you have (web client), which I
> have pushed to Guile's stable-2.0 branch.
>
> Wherever you see "doro" in the docs, replace it with "guild hall".
> Perhaps in the future we should drop the "hall", so "guild hall update"
> -> "guild update" or something.  Anyway, a thought for another time.
>
> So, the status:
>
>   1) Builds.
>   2) Passes make check.
>   3) Can update the available list.
>   4) Everything else is untested :-)
>
Cool!

> Next up:
>
>   1) Check status of dorodango functionality.
>   2) Fix things that don't work.
>   3) Profit?
>   4) Start thinking about hosting and accounts and UX and stuff.
>
> I will see if I can get work to sponsor a server that we can use, and
> see if we can get it aliased to guildhall.gnu.org -- unless someone else
> would like to provide the server.  It would be nice to have root on that
> server, FWIW.  It could be a VM.
>
> As far as relation with dorodango goes, we should do our best to keep
> the guildhall compatible with dorodango archives on the net.  We should
> also try hard to share code, but that is secondary.  Farther along I
> would like to rename (dorodango ...) in our source to (guildhall ...) or
> something so that we don't conflict with upstream.  I would also like to
> reduce the number of bundled dependencies, and for the ones that are
> left, include them under the (guildhall ...) namespace, making them
> effectively private.
>
+1; dorodango's dependencies are quite stable, so duplicating them in
the guidhall version should not be much of an issue.  As for reducing
their number, I've posted a mail with some ideas about how this could be
started off a while ago [0].  The thing that would be provide the most
benefit IMHO would be an interface to libzip, using Guile's dynamic FFI;
that would allow guildhall to get rid of industria as a dependency, and
speed up package installation quite a bit.  Also, this code could
eventually be folded back into dorodango, once spells' FFI is working on
Guile.  I'm in principle interested in working on a libzip interface,
but I'd rather get spells' FFI finished on Guile first.

>  That way you can also install dorodango on your
> machine, if you wish, and also install the wak- packages, industria,
> ocelotl, etc.
>
> I would really love for someone to take up this project.  I can help
> getting it to the minimally functional state.  Please let me know if you
> are interested.
>
Yeah, it would be cool if someone took care of this.  I can help with
any dorodango-related issues.

Regards, Rotty
-- 
Andreas Rottmann -- <http://rotty.yi.org/>

Re: no more *.la files in libgc-dev

[Andreas Rottmann]

David Pirotte  writes:

> Le Sun, 26 Jun 2011 22:15:23 +0200,
> Andreas Rottmann  a écrit :
>
> ...
>> The solution (other than just removing the offending .la files) would be to
>> re-build and re-install all software depending on libgc-dev, including
>> g-wrap, as to get rid of the stale references to libgc's .la file.
>> 
>> [0] http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=625187
>> 
>> Regards, Rotty
>
> Hello Andreas,
>
> Thanks your explanation. Though I just uninstalled rebuilt/installed g-wrap
> [v1.9.13-13-geeb1aae for guile-gnome-platform compatibility issues] and
> 'it' did build/install .la files: missing something?
>
No, that's expected: .la files are still built and installed by the
(upstream) G-Wrap, it's just that the corresponding Debian package
doesn't contain them anymore -- getting rid of .la files is a Debian
packaging thing, no change to upstream code or build system is made.

Regards, Rotty
-- 
Andreas Rottmann -- <http://rotty.yi.org/>

Re: How can I tell guile to shut up? ;)

[Andreas Rottmann]

Andy Wingo  writes:

> After some thinking, the base thing to do is just to add a warning port,
> and make warnings (non-fatal informative messages) write to that port.
> I have done this in the attached patches.  Any objections?
>
Great thing to do -- this adds quite some flexibility that can be put to
good use.  

One thing about the implementation, though: I don't quite like that
`%current-warning-port' (as already hinted at in the source) is intended
to be private (but isn't, but that's another, harder, issue), however
there is no way to redirect the warnings to another port for a dynamic
extent save for using that (assumably) private API.  How about
(especially since it looks like R7RS will include a subset of it in the
small language) moving SRFI 39 into the core and using that?

Regards, Rotty
-- 
Andreas Rottmann -- <http://rotty.yi.org/>

Re: no more *.la files in libgc-dev

[Andreas Rottmann]

David Pirotte  writes:

> Hello,
>
> Thinking it was a libgc-dev debian package problem I posted a mail ... and 
> got the
> following 2 answers [below].
>
> Once I manually removed all *.la files from my /usr/local/lib tree, I could 
> further
> try to compile guile-gnome-platform against gcc-4.6. 
>
> As the list of *.la files I removed does not only includes guile-gnome stuff 
> but
> also libguile, cairo and g-wrap, I am posting here instead of guile-gtk.
>
[Puts on Debian Maintainer hat] Debian indeed tries to get rid of .la
files wherever possible, as they cause spurious dependencies encoded in
the shared libraries, and subsequently, in Debian packages.  On the GNU
system, there is (somewhat ironically) no real advantage of having .la
files that I know of -- they are needed just for platforms with less
capable linkers, IIUC.

There thus is a coordinated effort going on to get rid of .la files
inside Debian, where first leaf packages (i.e. those where no package
build-depend on them, or more accurately, is built using the information
contained in their .la files) drop their .la files, making further
packages become leaf packages, who in turn can no drop .la files, and so
on.  If the process is followed correctly this should not cause any
issues internal to Debian, but there are occassional hiccups (see [0],
for an example where I was affected as a g-wrap maintainer).

For software installed "manually" (i.e. not via Debian packages), this
procedure may indeed cause issues, and my guess at what had happened in
your case is something like this: you built some software (for example
g-wrap) when libgc-dev still had its .la files installed by the Debian
package, leading to g-wrap's (to stay with the example) .la files
encoding the location of libgc's .la file.  When then libgc-dev dropped
the .la file, the .la files originally installed by g-wrap were broken,
thus breaking the build of any software depending on g-wrap.  The
solution (other than just removing the offending .la files) would be to
re-build and re-install all software depending on libgc-dev, including
g-wrap, as to get rid of the stale references to libgc's .la file.

[0] http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=625187

Regards, Rotty
-- 
Andreas Rottmann -- <http://rotty.yi.org/>

Re: [Guile-commits] GNU Guile branch, stable-2.0, updated. v2.0.1-40-g2252321

[Andreas Rottmann]

l...@gnu.org (Ludovic Courtès) writes:

> Hi Andreas,
>
> (Sorry for the lte reply, I'm just slowly catching up.)
>
> Andreas Rottmann  skribis:
>
>> Andreas Rottmann  writes:
>>
>>> l...@gnu.org (Ludovic Courtès) writes:
>>>
>>>> Hi Andreas,
>>>>
>>>> Thanks for taking care of this, and thanks for the great doc too!
>>>>
>>>> "Andreas Rottmann"  writes:
>>>>
>>>>> commit 2252321bb77fe83d98d5bcc9db1c76b914e9dd6a
>>>>> Author: Andreas Rottmann 
>>>>> Date:   Sat May 7 23:40:14 2011 +0200
>>>>>
>>>>> Make the R6RS simple I/O library use conditions
>>>>> 
>>>>> * module/rnrs/io/ports.scm (display): Implement as an
>>>>>   exception-converting wrapper around Guile's core display.
>>>>> * module/rnrs/io/simple.scm: Don't export Guile's corresponding core
>>>>>   procedures, but use `(rnrs io ports)' instead.  This way, we get the
>>>>>   conditions required by R6RS raised.
>>>>> 
>>>>> * doc/ref/r6rs.texi (rnrs io simple): Mention that these procedures 
>>>>> are
>>>>>   supposed to raise R6RS conditions.
>>>>
>>>> Could you add one or more test cases?
>>>>
>>> I've started to hack on this, aiming for at least providing tests of a
>>> "control sample" of exception-related behaviors in `(rnrs io simple)'
>>> and parts of `(rnrs io ports)'.  However, as this is already
>>> significantly more than one test, it may take a bit.  I could however
>>> push my existing work at any point, if that's needed for any reason.
>>>
>> I've now pushed a first patch (b6a66c2), in the process fixing two bugs.
>> These were not directly related to the simple I/O change you referenced,
>> but located in its base library `(rnrs io ports)'.
>
> +(define (with-i/o-port-error port make-primary-condition thunk)
> +  (with-throw-handler 'system-error
> +  thunk
> +(lambda args
> +  (let ((errno (system-error-errno args)))
> +(if (memv errno (list EIO EFBIG ENOSPC EPIPE))
> +(raise (condition (make-primary-condition)
> +  (make-i/o-port-error port)))
> +(apply throw args))
> +
> +(define-syntax with-textual-output-conditions
> +  (syntax-rules ()
> +((_ port body0 body ...)
> + (with-i/o-port-error port make-i/o-write-error
> +   (lambda () (with-i/o-encoding-error body0 body ...))
>
> [...]
>
>  (define (put-char port char)
> -  (with-i/o-encoding-error (write-char char port)))
> +  (with-textual-output-conditions port (write-char char port)))
>  
>  (define (put-datum port datum)
> -  (with-i/o-encoding-error (write datum port)))
> +  (with-textual-output-conditions port (write datum port)))
>
> I'm a bit concerned about the performance implications of the above
> change: ‘put-char’, ‘put-datum’, etc. now expand to something like
>
>  (with-i/o-port-error p xxx
>(lambda ()
>  (with-throw-handler 'decoding-error
>(lambda ()
>  (write-char c p))
>(lambda (key subr message errno port)
>  (raise (make-i/o-decoding-error port))
>
> So there are at least 4 additional function calls (2
> ‘with-throw-handler’ calls, and 2 anonymous closure calls.)
>
Yeah, this is bound to be slow, at least for something as basic as
`put-char' -- but correctness trumps performance, right? ;-).

> Did you do any measurements?  Would be nice to add micro-benchmarks
> under benchmark-suite/.
>
No, but I'm planning to add some benchmarks, and do some (basic)
performance work in this area; as my Guiling time is quite limited ATM,
don't hold your breath, though.

> One optimization would be to instead do something such that ‘put-char’
> would expand to something like:
>
>   (define (put-char p c)
> (with-throw-handler #t
>(lambda ()
>  (write-char c p))
>(lambda args
>  (case (car args)
>((decoding-error)
> (raise (make-i/o-decoding-error p)))
>((system-error)
> (if (memv (system-error-errno args)
>   `(,EIO ,EFBIG ,ENOSPC ,EPIPE))
> ...))
>(else
> ;; Not for us.
> (apply throw args))
>
> What do you think?
>
Yep, good idea.  I'll consider this when I have time to put some work
into it.

Regards, Rotty
-- 
Andreas Rottmann -- <http://rotty.yi.org/>

Re: [Guile-commits] GNU Guile branch, stable-2.0, updated. v2.0.1-40-g2252321

[Andreas Rottmann]

Andreas Rottmann  writes:

> l...@gnu.org (Ludovic Courtès) writes:
>
>> Hi Andreas,
>>
>> Thanks for taking care of this, and thanks for the great doc too!
>>
>> "Andreas Rottmann"  writes:
>>
>>> commit 2252321bb77fe83d98d5bcc9db1c76b914e9dd6a
>>> Author: Andreas Rottmann 
>>> Date:   Sat May 7 23:40:14 2011 +0200
>>>
>>> Make the R6RS simple I/O library use conditions
>>> 
>>> * module/rnrs/io/ports.scm (display): Implement as an
>>>   exception-converting wrapper around Guile's core display.
>>> * module/rnrs/io/simple.scm: Don't export Guile's corresponding core
>>>   procedures, but use `(rnrs io ports)' instead.  This way, we get the
>>>   conditions required by R6RS raised.
>>> 
>>> * doc/ref/r6rs.texi (rnrs io simple): Mention that these procedures are
>>>   supposed to raise R6RS conditions.
>>
>> Could you add one or more test cases?
>>
> I've started to hack on this, aiming for at least providing tests of a
> "control sample" of exception-related behaviors in `(rnrs io simple)'
> and parts of `(rnrs io ports)'.  However, as this is already
> significantly more than one test, it may take a bit.  I could however
> push my existing work at any point, if that's needed for any reason.
>
I've now pushed a first patch (b6a66c2), in the process fixing two bugs.
These were not directly related to the simple I/O change you referenced,
but located in its base library `(rnrs io ports)'.  There's still more to
come in that area:

- Tests for en/decoding error behavior.

- Fixing `get-string-n!' to raise proper conditions.  This also entails
  introducing an extension to scm_getc() which communicates errors via
  return value or output arguments, thus speeding up `get-char'.  I'll
  post a patch for review once I have something worth showing.

I think we are approaching completeness and correctness of our R6RS I/O
support -- besides Section 8.2.13 "Input/output ports" and the EOL
handling, I'm not aware of major stuff missing.

Regards, Rotty
-- 
Andreas Rottmann -- <http://rotty.yi.org/>

Re: [Guile-commits] GNU Guile branch, stable-2.0, updated. v2.0.1-40-g2252321

[Andreas Rottmann]

l...@gnu.org (Ludovic Courtès) writes:

> Hi Andreas,
>
> Thanks for taking care of this, and thanks for the great doc too!
>
> "Andreas Rottmann"  writes:
>
>> commit 2252321bb77fe83d98d5bcc9db1c76b914e9dd6a
>> Author: Andreas Rottmann 
>> Date:   Sat May 7 23:40:14 2011 +0200
>>
>> Make the R6RS simple I/O library use conditions
>> 
>> * module/rnrs/io/ports.scm (display): Implement as an
>>   exception-converting wrapper around Guile's core display.
>> * module/rnrs/io/simple.scm: Don't export Guile's corresponding core
>>   procedures, but use `(rnrs io ports)' instead.  This way, we get the
>>   conditions required by R6RS raised.
>> 
>> * doc/ref/r6rs.texi (rnrs io simple): Mention that these procedures are
>>   supposed to raise R6RS conditions.
>
> Could you add one or more test cases?
>
I've started to hack on this, aiming for at least providing tests of a
"control sample" of exception-related behaviors in `(rnrs io simple)'
and parts of `(rnrs io ports)'.  However, as this is already
significantly more than one test, it may take a bit.  I could however
push my existing work at any point, if that's needed for any reason.

Regards, Rotty
-- 
Andreas Rottmann -- <http://rotty.yi.org/>

[PATCH] Set the FD_CLOEXEC flag on the runtime's file descriptors

[Andreas Rottmann]

* libguile/_scm.h (scm_set_fd_cloexec): New convenience macro for
  setting the FD_CLOEXEC flah on platforms that support it; on other
  platforms it's a no-op.

* libguile/objcodes.c (scm_load_objcode): Mark the objectcode's FD as
  close-on-exec.
* libguile/scmsigs.c (start_signal_delivery_thread): Mark both ends of
  the signal delivery pipe as close-on-exec.
* libguile/threads.c (guilify_self_1): Likewise for the thread's
  sleep_pipe.
---
 libguile/_scm.h |   12 
 libguile/objcodes.c |2 ++
 libguile/scmsigs.c  |4 +++-
 libguile/threads.c  |3 +++
 4 files changed, 20 insertions(+), 1 deletions(-)

diff --git a/libguile/_scm.h b/libguile/_scm.h
index 2842130..8eff18f 100644
--- a/libguile/_scm.h
+++ b/libguile/_scm.h
@@ -62,6 +62,7 @@
 #endif
 
 #include 
+#include 
 #include 
 #include 
 #include "libguile/__scm.h"
@@ -132,6 +133,17 @@
 # define SCM_SYSCALL(line) line;
 #endif /* ndef SCM_SYSCALL */
 
+#if defined(HAVE_FCNTL) && defined(FD_CLOEXEC)
+#  define scm_set_fd_cloexec(fd)\
+  do  { \
+int old_flags = fcntl (fd, F_GETFD);\
+if (old_flags >= 0) \
+  fcntl (fd, F_SETFD, FD_CLOEXEC | old_flags);  \
+  } while (0)
+#else
+#  define scm_set_fd_cloexec(fd) fd
+#endif
+
 
 
 #ifndef min
diff --git a/libguile/objcodes.c b/libguile/objcodes.c
index 448bada..d6a38fc 100644
--- a/libguile/objcodes.c
+++ b/libguile/objcodes.c
@@ -299,6 +299,8 @@ SCM_DEFINE (scm_load_objcode, "load-objcode", 1, 0, 0,
   free (c_file);
   if (fd < 0) SCM_SYSERROR;
 
+  scm_set_fd_cloexec (fd);
+
   return make_objcode_from_file (fd);
 }
 #undef FUNC_NAME
diff --git a/libguile/scmsigs.c b/libguile/scmsigs.c
index 699a6de..e84390a 100644
--- a/libguile/scmsigs.c
+++ b/libguile/scmsigs.c
@@ -1,4 +1,4 @@
-/* Copyright (C) 1995,1996,1997,1998,1999,2000,2001, 2002, 2004, 2006, 2007, 
2008, 2009 Free Software Foundation, Inc.
+/* Copyright (C) 1995,1996,1997,1998,1999,2000,2001, 2002, 2004, 2006, 2007, 
2008, 2009, 2011 Free Software Foundation, Inc.
  *
  * This library is free software; you can redistribute it and/or
  * modify it under the terms of the GNU Lesser General Public License
@@ -190,6 +190,8 @@ start_signal_delivery_thread (void)
 
   if (pipe (signal_pipe) != 0)
 scm_syserror (NULL);
+  scm_set_fd_cloexec (signal_pipe[0]);
+  scm_set_fd_cloexec (signal_pipe[1]);
   signal_thread = scm_spawn_thread (signal_delivery_thread, NULL,
scm_handle_by_message,
"signal delivery thread");
diff --git a/libguile/threads.c b/libguile/threads.c
index f49696b..5d986e9 100644
--- a/libguile/threads.c
+++ b/libguile/threads.c
@@ -526,6 +526,9 @@ guilify_self_1 (struct GC_stack_base *base)
currently have type `void'.  */
 abort ();
 
+  scm_set_fd_cloexec (t.sleep_pipe[0]);
+  scm_set_fd_cloexec (t.sleep_pipe[1]);
+
   scm_i_pthread_mutex_init (&t.admin_mutex, NULL);
   t.current_mark_stack_ptr = NULL;
   t.current_mark_stack_limit = NULL;
-- 
1.7.5.1

[PATCH] Set the FD_CLOEXEC flag on the runtime's file descriptors

[Andreas Rottmann]

This is supposed to prevent Guile to leak internal file descriptors
across an exec* system call.  The Guile user has still to take care of
setting the CLOEXEC flag on all ports (e.g., using `port-for-each').

Linux's LVM tools are a nice test case for this, as they emit a warning
line for each leaked FD when invoked.

Re: g-wrap - fresh git clone - can't find libguile.h:

[Andreas Rottmann]

Andreas Rottmann  writes:

> David Pirotte  writes:
>
>> Le Tue, 03 May 2011 12:43:07 +0200,
>> Andy Wingo  a écrit :
>>
>>> On Mon 02 May 2011 20:57, David Pirotte  writes:
>>> 
>>> > After having just installed a guile fresh git clone, g-wrap [fresh git 
>>> > clone
>>> > too] won't make.
>>> 
>>> What platform are you on?  GNU/Linux?  From which distributor?
>>> 
>>> Andy
>>
>> Hello Andy,
>>
>>  x86_64 GNU/Linux
>>  Debian testing/unstable
>> 
> Im on the same platform (current Debian sid, amd64) as well.  I just
> added a workaround for a (probably unrelated) issue in g-wrap.  Could you
> update your checkout, and try again?  Also make sure ACLOCAL_FLAGS is
> set in a way so that the matching guile.m4 gets picked up; if you have
> installed Guile into /usr/local, that would be:
>
> export ACLOCAL_FLAGS="-I /usr/local/share/aclocal"
> ./autogen.sh && make
>
We have finally figured it out after some debugging via IRC:  David has
had set PKG_CONFIG=true when configuring Guile, which lead to an
installation with a (silently) broken guile-config script;
meta/guile-config.in contains:

(define %pkg-config-program "@PKG_CONFIG@")

The mayhem that resulted is left to the (hopefully vivid) imagination of
the reader ;-).

IMO, the following advice given in Guile's README is quite dangerous, as
can be seen from David's issue:

  - pkg-config

Guile's ./configure script uses pkg-config to discover the correct
compile and link options for libgc and libffi.  If you don't have
pkg-config installed, or you have a version of libgc that doesn't
provide a .pc file, you can work around this by setting some
variables as part of the configure command-line:

- PKG_CONFIG=true

- BDW_GC_CFLAGS=

- BDW_GC_LIBS=

Note that because you're bypassing all pkg-config checks, you will
also have to specify libffi flags as well:

- LIBFFI_CFLAGS=

- LIBFFI_LIBS=

Note the PKG_CONFIG=true setting.  This should at least come with a big
fat warning that the resulting installation of Guile will be broken
wrt. to building any software depending on Guile (unless the software in
question bypasses guile-config).  Alternatively, we could scrap that
section altogether and just say that pkg-config is required (which is in
fact true, with the current state of afairs).

Regards, Rotty
-- 
Andreas Rottmann -- <http://rotty.yi.org/>

Re: g-wrap - fresh git clone - can't find libguile.h:

[Andreas Rottmann]

David Pirotte  writes:

> Le Tue, 03 May 2011 12:43:07 +0200,
> Andy Wingo  a écrit :
>
>> On Mon 02 May 2011 20:57, David Pirotte  writes:
>> 
>> > After having just installed a guile fresh git clone, g-wrap [fresh git 
>> > clone
>> > too] won't make.
>> 
>> What platform are you on?  GNU/Linux?  From which distributor?
>> 
>> Andy
>
> Hello Andy,
>
>   x86_64 GNU/Linux
>   Debian testing/unstable
> 
Im on the same platform (current Debian sid, amd64) as well.  I just
added a workaround for a (probably unrelated) issue in g-wrap.  Could you
update your checkout, and try again?  Also make sure ACLOCAL_FLAGS is
set in a way so that the matching guile.m4 gets picked up; if you have
installed Guile into /usr/local, that would be:

export ACLOCAL_FLAGS="-I /usr/local/share/aclocal"
./autogen.sh && make

HTH, Rotty
-- 
Andreas Rottmann -- <http://rotty.yi.org/>

Re: binary-port?

[Andreas Rottmann]

l...@gnu.org (Ludovic Courtès) writes:

> Hi,
>
> Andreas Rottmann  writes:
>
>> - Ikarus and Ypsilon definitly have disjoint ports.
>>
>> - Racket natively has ports that will accept both binary and textual
>>   operations, but it's R6RS support wraps these ports so that the
>>   resulting R6RS ports are not compatible with Racket's native port
>>   operations, but do provide the binary/textual disjointness.
>
> So Racket really has 3 disjoint port types, right?  That looks
> inconvenient to me.
>
It is, at least if you don't deal with R6RS-mode code only.  It's a
similiar situation (although not as pervasively inconvinient) as the
mutable/immutable pair division.

>> These are the only implementations (besides Guile), that I'm familiar
>> with.  Ideally, Guile's R6RS support would provide real disjointness for
>> binary and textual ports -- I think the only artefact that hampers this
>> is that there's no way to distinguish Latin-1 encoded ports from "pure"
>> binary ports (as both have `port-encoding' being #f).
>
> And beyond that, you can put-bytevector on any port, read-char on any
> port, and so on.
>
> I wouldn’t want the “native” port type to be disjoint from the R6RS port
> types, notably because there’s no “native” equivalent to the R6RS binary
> I/O API, and also because it would hamper composition of R6RS and
> non-R6RS code.
>
Well, I'm not advocating making them disjoint in the sense that the
textual or binary operations are only possible on "matching" ports.
Allowing to mix binary and textual I/O on any port, is, IMHO, a fine and
reasonable implementation-specific extension that Guile provides.  What
I'm after is making `textual-port?' and `binary-port?' establish a
partition on the set of possible ports; i.e.

(textual-port? X) = (not (binary-port? X))

for any port X (or at least for any port obtainable via R6RS-specified
procedures).  For that to work, we somehow need to distinguish between
Latin-1 ports and "pure" binary ports.  Perhaps by adding a flag
indicating this to the port objects?  This flag would then be set by all
R6RS procedures specified to create binary ports, and would be checked
by `binary-port?' and `textual-port?'.  Additionally, we might want to
clear that flag when the port's encoding is changed to non-#f.  WDYT?

Regards, Rotty
-- 
Andreas Rottmann -- <http://rotty.yi.org/>

Re: binary-port?

[Andreas Rottmann]

l...@gnu.org (Ludovic Courtès) writes:

> Hi Andreas!
>
> Andreas Rottmann  writes:
>
>> l...@gnu.org (Ludovic Courtès) writes:
>
> [...]
>
>>> However, I’m wondering whether we should not just squarely do away with
>>> the binary/textual distinction, and just write:
>>>
>>>   (define (binary-port? p) #t)
>>>
>>> What do people with experience with pure R6RS code think?  Is the
>>> distinction actually used, and how?
>>>
>> I can only find one example in the code I wrote:
>> `copy-port', which works (with the probably obvious semantics), on both
>> binary and textual ports.  On Guile, when `binary-port?' would return #t
>> for all ports, `copy-port' would break, losing the transcoding effect
>> you'd get when you pass two textual ports of different encodings.
>
> Interesting.  Can you post a link to the code?
>
Sure, it's in `(spells ports)':

https://github.com/rotty/spells/blob/master/spells/ports.sls

> Anyway, that’s probably enough to keep the current semantics in 2.0.
>
>> With the current behavior, you still have to watch the order of your
>> port type checks, testing for `binary-port?' first, whereas on systems
>> following R6RS strictly, you'd get the same behavior regardless of
>> type check order.  I can live with the latter, but the former would be
>> unfortunate, IMHO.
>
> Do you know what Industria, Nausicaa, & co. do?
>
I don't have a copy of nausicaa lying around here, but in the R6RS code
I have on my machine, xitomatl/ports.sls is the only other occurance of
`binary-port?' (besides spells/ports.sls):

http://bazaar.launchpad.net/~derick-eddington/scheme-libraries/xitomatl/view/head:/ports.sls

It's use there is in `open-compound-input-port':

;; A compound input port is a custom port which represents the logical
;; concatenation of other input ports.  It starts out with an ordered
;; collection of input ports and reads from the first one until end of file
;; is reached, whereupon it reads from the second one, and so on, until end
;; of file is reached on the last of the contained input ports, and then
;; subsequent reads from the compound input port will return end of file.
;; After each component port is exhausted, it is closed.  Closing a compound
;; input port closes all remaining component ports.  get-position and
;; set-position! are not supported.

So, my impression is that while `binary-port?' and `textual-port?' are
not used widely, they do have their uses, and there is definitly code
out there that relies on the binary/textual distinction.

> Likewise, any idea which Schemes have disjoint binary/textual ports,
> and which don’t?
>

- Ikarus and Ypsilon definitly have disjoint ports.

- Racket natively has ports that will accept both binary and textual
  operations, but it's R6RS support wraps these ports so that the
  resulting R6RS ports are not compatible with Racket's native port
  operations, but do provide the binary/textual disjointness.

These are the only implementations (besides Guile), that I'm familiar
with.  Ideally, Guile's R6RS support would provide real disjointness for
binary and textual ports -- I think the only artefact that hampers this
is that there's no way to distinguish Latin-1 encoded ports from "pure"
binary ports (as both have `port-encoding' being #f).

Regards, Rotty
-- 
Andreas Rottmann -- <http://rotty.yi.org/>

Re: binary-port?

[Andreas Rottmann]

l...@gnu.org (Ludovic Courtès) writes:

> Hello,
>
> I just pushed 96128014bfaabe9e123c4f4928ce4c20427eaa53, which makes
> ‘binary-port?’ deterministic for ports intended to be binary.
>
Glad to see that!

> However, I’m wondering whether we should not just squarely do away with
> the binary/textual distinction, and just write:
>
>   (define (binary-port? p) #t)
>
> What do people with experience with pure R6RS code think?  Is the
> distinction actually used, and how?
>
I can only find one example in the code I wrote:
`copy-port', which works (with the probably obvious semantics), on both
binary and textual ports.  On Guile, when `binary-port?' would return #t
for all ports, `copy-port' would break, losing the transcoding effect
you'd get when you pass two textual ports of different encodings.  With
the current behavior, you still have to watch the order of your port
type checks, testing for `binary-port?' first, whereas on systems
following R6RS strictly, you'd get the same behavior regardless of type
check order.  I can live with the latter, but the former would be
unfortunate, IMHO.

Regards, Rotty
-- 
Andreas Rottmann -- <http://rotty.yi.org/>

Re: CPAN-type thing: specifications, wishes, thoughts?

[Andreas Rottmann]

Andreas Rottmann  writes:

>> I was thinking that most of this project could be written in Guile.
>>
> I think all of it can be written in Scheme, except for some low-level
> code interfacing with things like libzip (if ZIP is adopted as a package
> format).
>
Let me restate: *all* of it can/should be written in Scheme, as Guile
2.0 has a dynamic FFI which allows to build language bindings without
touching C code; however, there are some considerations when using the
dynamic FFI:

- You are essentially encoding the ABI of the library you are making a
  binding for into your Scheme code.  If the ABI changes, you need to
  adapt your Scheme code.

- Due to a bug in libltdl[0], it is currently not possible to specify
  the full path (including ABI designation, e.g. libzip.so.1 instead of
  libzip.so) when loading the shared library.  This means the shared
  library can change ABI without resulting in the Scheme code to fail in
  a clear way.

[0] http://lists.gnu.org/archive/html/bug-guile/2011-03/msg00138.html

Regards, Rotty
-- 
Andreas Rottmann -- <http://rotty.yi.org/>

Re: CPAN-type thing: specifications, wishes, thoughts?

[Andreas Rottmann]

CRLF0710  writes:

> Something like the Planets of Racket-lang? Should be able to install
> offline, though~
>
FWIW, dorodango supports local filesystem-based repositories, so you can
just mirror your favorite repository, and then install offline.

Regards, Rotty
-- 
Andreas Rottmann -- <http://rotty.yi.org/>

Re: CPAN-type thing: specifications, wishes, thoughts?

[Andreas Rottmann]

Noah Lavine  writes:

> Hello all,
>
> I'm afraid this email is coming much later in the planning process
> than it should, and quite possibly we won't be able to do any of this
> for SoC, and that's fine with me. But I was thinking about what we
> could do that would be a "killer feature" for Guile's CPAN - something
> that isn't already done by apt-get, dorodango, and every other package
> manager. One answer is that having a big collection of Guile packages
> *is* a killer feature, but we could have an apt repository of those if
> we wanted one.
>
> I think the answer is that the killer feature for a large repository
> of packages is having the ability to painlessly bundle a piece of
> software you've been writing and all of its dependencies in one
> easy-to-install thing for users. After all, it's easy when you're
> developing on your own machine to download a bunch of packages and use
> them to do whatever you need to do, but if you then want to send that
> to other people, you've got to collect the source code (or at least
> .go files) for all of them, put them in a folder, make sure the
> load-path will work out, and then most importantly, do all of that
> again every time a new version of one of your dependencies comes out.
> I think the feature that is missing is the ability to say "take my
> software and package it so that its only external dependency is
> POSIX", or something similar.
>
Well, IIRC, it's not a feature that "dorodango does not have" ;-):
dordango's package container format (referred to as a "bundle") can
contain multiple packages.  So it is possible to pack all your
dependencies in a single ZIP, and install them in one go using that
bundle.  There's a bit of text about that in the dorodango manual, but
here is how it works practically:

% doro show-bundle my-app.zip # to view packages in the bundle
% doro install --bundle=my-app-full.zip my-app # install my-app and all 
dependencies

> The implementation of such a thing isn't especially deep, but I bet
> there will be a lot of little things that need to be done just right
> for it to work.
>
Support for this has to be present in the conception of the
package/bundle format, so indeed it's important that taking this feature
into account from the beginning.

> I think this could be a part of a package manager that also does the
> other things Paul was listing.
>
Indeed!

Regards, Rotty
-- 
Andreas Rottmann -- <http://rotty.yi.org/>

Re: CPAN-type thing: specifications, wishes, thoughts?

[Andreas Rottmann]

Paul Raccuglia  writes:

[ Note that I'm writing here with my dorodango developer hat on, not so
  much as a potential co-mentor. ]

>
> My thoughts, specifically, were for the client to be fairly similar in
> function to apt-get. (from the user's perspective)
> The core commands would be:
>
> INSTALL package_name   -- queries a server (set in configuration
> files), gets an absolute link to the package, and a list of
> dependencies. Downloads the package, and then any dependencies, and
> takes care of unpacking, verifying, compiling, storing, etc. anything
> that needs to be done. The package would have to have meta-data
> attached in some fashion.
>
> REMOVE package_name -- just has to remove all the relevant files, and
> clean up any references to the package in the storage system. May want
> to check dependencies and alert the user.
>
> UPDATE [package_name] -- could be called to check all packages (by not
> specifying a package name) or to update a specific package. Warn the
> user of any dependencies that could be broken.
>
This, in slight variation, is all already implemented in Dorodango; you
may want to have a look at its manual[0].

[0] http://home.gna.org/dorodango/manual/

> My thought was that the package manager could, itself, be a package
> (but, hopefully, one that would be included by default).
>
Dorodango is a package; the way you install it works as follows: you
download a tarball containing dorodango and its dependencies. After
unpacking the tarball, you run 'setup', which sets up the load path for
the implementation and invokes it on the doro.sps Scheme program, which
then proceeds to install dorodango and all of its dependencies.

> I wouldn't imagine that the current code base would need to be
> modified a whole lot.
>
I think the plan was to develop the package manager as an external
project, and then integrate it into Guile proper, once its "ready".  I
imagine Guile core would not need to be modified at all, perhaps modulo
some missing features, should that become necessary.

> I was thinking that most of this project could be written in Guile.
>
I think all of it can be written in Scheme, except for some low-level
code interfacing with things like libzip (if ZIP is adopted as a package
format).

> Does that make sense?
>
> Some thoughts I heard were:
> using stowfs to handle storing the packages intelligently
> use dorodango to handle most of the acquisition
>
>From my (obviously strongly biased view), it would be preferable to
start with the current dorodango codebase, make sure it works well with
Guile (there should not be much work left), and work from there.

>
> For the server design: I was envisioning a website to navigate
> packages (like http://packages.debian.org/stable/). My thought is to
> do everything over HTTP (seems like that would work well with
> Dorodango). Doesn't seem like a hugely complicated issue?
>
I started on something like this (i.e. a web interface for dorodango
repositories), I'll put the little code I have online and notify you.

> Questions about the server:
> How would the central repository be maintained? Give trusted
> developers the ability to make changes to their personal packages at
> will?
>
This is indeed a good question; the way it works in Debian is that all
developers can make changes to any package (by doing a signed upload via
FTP), but there are social rules about when it is OK to upload a package
one is not the designated maintainer for; see Non-Maintainer Uploads
(NMU)[1]. There are also many collaboratively and team-maintained
packages where there's not a single maintainer.  Perhaps a similiar
strategy would work for Guile's repository as well.

[1] http://www.debian.org/doc/developers-reference/pkgs.html#nmu

> How will packages be nominated / selected for inclusion?
>
I'd imagine a discussion on guile-devel beforehand would work.

Regards, Rotty
-- 
Andreas Rottmann -- <http://rotty.yi.org/>

Re: [PATCH] Several tweaks for R6RS ports

[Andreas Rottmann]

Andy Wingo  writes:

> Hi Andreas,
>
> On Thu 24 Mar 2011 01:47, Andreas Rottmann  writes:
>
>> l...@gnu.org (Ludovic Courtès) writes:
>>
>>> Could you send a patch to update the manual?
>>>
>> I have started working on this, but I'm notorious (at least to myself)
>> for starving documentation-related work with interesting coding stuff
>> that comes along, so don't hold your breath ;-).
>
> Believe me, I understand.  However if you don't do it, it means someone
> else has to, or otherwise we go without docs.
>
> I can think of various kinds of patch economies out there.  If there
> existed people that just like to document other people's work, then we
> could indeed think of this arrangement as the most efficient.
>
> However until such a person comes along, the most fair thing seems to be
> for those that have the pleasure of interesting coding also handle their
> share of documentation.
>
> So, if you have time, we would appreciate docs on these pieces of Guile
> :)
>
I've put it firmly on my TODO list, but can't promise a timeframe, as
I'm a bit busy IRL (moving next week).  Since R6RS' text can be reused
in large parts, it shouldn't be too much work, however.

Regards, Rotty
-- 
Andreas Rottmann -- <http://rotty.yi.org/>

Re: define-inlinable

[Andreas Rottmann]

Andy Wingo  writes:

> Heya :)
>
> On Mon 11 Apr 2011 22:01, l...@gnu.org (Ludovic Courtès) writes:
>
>>> I don't know if this is important, as R6RS users probably have lots of
>>> other carnage to deal with, but it is strictly an ABI break.
>>
>> Well well, they’ll need to recompile.  My feeling is that it’s
>> acceptable, but I don’t have a strong opinion.
>
> In this case it probably is, but we don't really know what users there
> are.  But let's please keep this general issue in mind in the future.
>
Agreed.  I'm reluctant to using macros as a performance hack (e.g.,
define-inlinable) -- it just feels wrong, but sometimes performance
numbers are seductive...

Regards, Rotty
-- 
Andreas Rottmann -- <http://rotty.yi.org/>

Re: Hi! Interested in GSoC. Feedback on these ideas?

[Andreas Rottmann]

Paul Raccuglia  writes:

> Hi! I'm interested in working on guile as part of the Google Summer of Code.
>
[...]
> Currently I am thinking about:
>
> - A package manager (in the vein of apt-get)
> I know this is one that's come up a bit. I was thinking of writing a
> small web interface to browse packages, an aptitude style command-line
> method for downloading, verifying the hashes, checking dependencies,
> and then install the package(s) and an uninstall function as well. I
> saw the proposal of working with dorodango; I might look at it, but
> from what I saw of the code base, I thought it would make sense to try
> to write a GUILE specific package-manager.
>
Could you be more specific?  What exactly in the code base did make you
think so?

Thanks, Rotty
-- 
Andreas Rottmann -- <http://rotty.yi.org/>

Re: define-inlinable

[Andreas Rottmann]

l...@gnu.org (Ludovic Courtès) writes:

> Hello!
>
> Andreas Rottmann  writes:
>
>> Subject: Move `define-inlinable' into the default namespace
>>
>> +@node Inlinable Procedures
>> +@subsection Inlinable Procedures
>> +
>> +You can define an ``inlinable procedure'' by using
>
> Use @dfn{inlinable procedure} here.
>
Done.

>> +@code{define-inlinable} instead of @code{define}. An inlinable procedure
>
> I prefer two-spaces-after-period, but there’s no consensus.
>
I try to use two spaces in English text as well, but being a German
native speaker (where this is against typographical rules, AFAIK), I
slip sometimes.  Fixed.

>> +behaves the same as a regular procedure, but direct calls will result in
>> +the procedure body being inlined into the caller.
>> +
>> +Making a procedure inlinable eliminates the overhead of the call,
>
> How about:
>
>   Procedures defined with @code{define-inlinable} are @emph{always}
>   inlined, at all call sites.  This eliminates function call overhead at
>   the expense of an increase in code size.
>
Folded in, with the addition of using ".., at _direct_ call sites.".
There's no inlining happening when you use `apply', or rebind the
procedure with `let'.  Should this be made more explicit?

>> but at
>> +the same time means that the caller will not transparently use the new
>> +definition if the inline procedure is redefined.
>
>   ... redefined using @code{set!}.
>
I don't agree with that one: there are multiple ways a procedure can get
"redefined", `set!' being just one of them.  I was actually thinking
more of re-evaluating the procedure definition or something like
`geiser-compile-file', hence I left the text like it was, being more
vague.

>> Inlinable procedures
>> +will also not deal nicely with debugging and tracing.
>
> Instead of “not deal nicely”, what about something like:
>
>   It is not possible to trace an inlined procedures or install a
>   breakpoint in it (@pxref{Traps}).
>
Done.

>> Therefore, you
>> +should not make a procedure inlinable unless it demonstrably improves
>> +performance in a crucial way.
>> +
>> +In general, only small procedures should be considered for inlining, as
>> +making large procedures inlinable will probably result in an increase in
>> +code size.  Additionally, the elimination of the call overhead rarely
>> +matters for for large procedures.
>> +
>> +@deffn {Scheme Syntax} define-inlinable (name parameter ...) . body
>
> I’d write “body ...” instead of “. body”.  Besides being aesthetically
> nicer, the former matches a proper list whereas the latter matches a
> pair.
>
Agreed, and done.

Updated patch attached, is it OK to push this way?

From: Andreas Rottmann 
Subject: Move `define-inlinable' into the default namespace

* module/ice-9/boot-9.scm (define-inlineable): Moved here from SRFI-9.
* module/srfi/srfi-9 (define-inlinable): Removed here.

* doc/ref/api-procedures.texi (Inlinable Procedures): Add subsection
  about `define-inlinable'.

---
 doc/ref/api-procedures.texi |   29 -
 module/ice-9/boot-9.scm |   36 
 module/srfi/srfi-9.scm  |   32 
 3 files changed, 64 insertions(+), 33 deletions(-)

diff --git a/doc/ref/api-procedures.texi b/doc/ref/api-procedures.texi
index 02889c4..5c6d380 100644
--- a/doc/ref/api-procedures.texi
+++ b/doc/ref/api-procedures.texi
@@ -1,6 +1,6 @@
 @c -*-texinfo-*-
 @c This is part of the GNU Guile Reference Manual.
-@c Copyright (C)  1996, 1997, 2000, 2001, 2002, 2003, 2004, 2009, 2010
+@c Copyright (C)  1996, 1997, 2000, 2001, 2002, 2003, 2004, 2009, 2010, 2011
 @c   Free Software Foundation, Inc.
 @c See the file guile.texi for copying conditions.
 
@@ -16,6 +16,7 @@
 * Higher-Order Functions::  Function that take or return functions.
 * Procedure Properties::Procedure properties and meta-information.
 * Procedures with Setters:: Procedures with setters.
+* Inlinable Procedures::Procedures that can be inlined.
 @end menu
 
 
@@ -797,6 +798,32 @@ Return the setter of @var{proc}, which must be either a procedure with
 setter or an operator struct.
 @end deffn
 
+@node Inlinable Procedures
+@subsection Inlinable Procedures
+
+You can define an @dfn{inlinable procedure} by using
+@code{define-inlinable} instead of @code{define}.  An inlinable
+procedure behaves the same as a regular procedure, but direct calls will
+result in the procedure body being inlined into the caller.
+
+Procedures defined with @code{define-inlinable} are @emph{always}
+inlined, at all direct call sites.  This eliminates function call
+overhead at the expense of an incre

Re: [PATCH] Undeprecate read syntax for uniform complex vectors

[Andreas Rottmann]

Andy Wingo  writes:

> On Wed 06 Apr 2011 01:51, Mark H Weaver  writes:
>
>> FYI, I just pushed this fix.
>>
>> * libguile/read.c (scm_read_sharp): Move the "#c..." case outside of
>>   #if SCM_ENABLE_DEPRECATED, and to the same section which handles
>>   "#s...", "#u..." and "#f...".
>>   Thanks to Andreas Rottmann  for the bug report.
>
> Thanks!  What bug does it fix?  Sorry for being obtuse :)
>
A test suite failure in srfi-4.test when building with
--disable-deprecated, see also:

http://rotty.yi.org/irclogs/freenode/%23guile/2011-04-06/#e18

Regards, Rotty
-- 
Andreas Rottmann -- <http://rotty.yi.org/>

Re: GSoC 2011

[Andreas Rottmann]

ed in the .typelib file is complete
  enough to yield good and "schemely" bindings, there might still be
  some corners in the API that profit from some glossing over, and sbank
  provides such glossing where and when such cases are identified.

Regards, Rotty
-- 
Andreas Rottmann -- <http://rotty.yi.org/>

Re: [PATCH 3/3] Add `fixnum?' VM primitive

[Andreas Rottmann]

Andy Wingo  writes:

> Hi Andreas,
>
> I applied the first two, thanks.
>
> I am hesitant about this one for three reasons:
>
>  1) I don't want the compiler to import (rnrs arithmetic fixnums).
> Rather, if we were to do this, I would have that module register its
> primitives, as GOOPS does.
>
>  2) Something about this sounds wrong to me.  If fixnum? is important,
>
I don't hold the opinion that `fixnum?' per se is important; it's just
that to achieve better performance for R6RS' fixnum operations (while
still remaining within R6RS spec territory), it needs to be fast, due to
the abundance of calls to it.  If, for example, we'd make fixnum
operations VM primitives, we wouldn't need to care about `fixnum?'s
speed so much.

>  why not have it in Guile's default environment directly?
>
Well, this was what my original patch did [0], but Ludovic objected [1].

[0] http://lists.gnu.org/archive/html/guile-devel/2011-03/msg00223.html
[1] http://lists.gnu.org/archive/html/guile-devel/2011-03/msg00234.html

If you guys can reach an agreement that it should/could live in Guile's
default namespace, either named `fixnum?', or by some other name, I'll
happily re-do the patch accordingly.  My personal feeling is that having
an extension for `fixnum?' alone is clumsy, and I did this just due to
Ludovic's response.

>   Or some other non-R6RS namespace.
>
Suggestions?

>  What about, for example, `exact-integer?'?  And why for fixnums and
>  not flonums?
>
Well, the intention behind naming the files r6rs-arithmetic.{c,h} is
that similiar performance hacks for flonums could go in there as well,
but ATM I don't really care about flonum operations, so I didn't include
that in the patch.

>  3) Are there no alternatives, like exposing Guile's tags to Scheme
> directly?  (ice-9 tags) for example.  Then we could inline tc3?,
> tc8?, and tc16? ops.  Some other predicates would become
> unnecessary; char? for example.  Perhaps this is a bad idea though.
>
I don't like this idea.  It sounds like exposing what is a _really_
low-level implementation detail to Scheme for not a very good reason.

Granted, `fixnum?' exposes also an implementation detail, but one that
is probably common to all non-toy implementations of Scheme, and if
hardware architectures do not change fundamentally, will probably remain
to be so.  This is also my response to Ludovic's response that "this
fixnum thing is a breach in the numerical tower": Yes, it is, but one
that can potentially provide performance gains by having the user
explicitly stating (commonly found, in some areas) invariants about the
numerical range of certain expressions.  A compiler can then leverage
this additional information to give better performance on such code.
IMO, it's a performance hack, but a potentially useful one.

> Apologies for being crabby here after you've done all this work :)  It
> could well be that something close to this is the right thing.
>
Well, I've attached my Plan-B solution.  We can come back to the
`fixnum?' VM primitive when it's clear where its corresponding Scheme
binding should go.

From: Andreas Rottmann 
Subject: Move `define-inlinable' into the default namespace

* module/ice-9/boot-9.scm (define-inlineable): Moved here from SRFI-9.
* module/srfi/srfi-9 (define-inlinable): Removed here.

* doc/ref/api-procedures.texi (Inlinable Procedures): Add subsection
  about `define-inlinable'.

---
 doc/ref/api-procedures.texi |   27 ++-
 module/ice-9/boot-9.scm |   36 
 module/srfi/srfi-9.scm  |   32 
 3 files changed, 62 insertions(+), 33 deletions(-)

diff --git a/doc/ref/api-procedures.texi b/doc/ref/api-procedures.texi
index 02889c4..4b4870d 100644
--- a/doc/ref/api-procedures.texi
+++ b/doc/ref/api-procedures.texi
@@ -1,6 +1,6 @@
 @c -*-texinfo-*-
 @c This is part of the GNU Guile Reference Manual.
-@c Copyright (C)  1996, 1997, 2000, 2001, 2002, 2003, 2004, 2009, 2010
+@c Copyright (C)  1996, 1997, 2000, 2001, 2002, 2003, 2004, 2009, 2010, 2011
 @c   Free Software Foundation, Inc.
 @c See the file guile.texi for copying conditions.
 
@@ -16,6 +16,7 @@
 * Higher-Order Functions::  Function that take or return functions.
 * Procedure Properties::Procedure properties and meta-information.
 * Procedures with Setters:: Procedures with setters.
+* Inlinable Procedures::Procedures that can be inlined.
 @end menu
 
 
@@ -797,6 +798,30 @@ Return the setter of @var{proc}, which must be either a procedure with
 setter or an operator struct.
 @end deffn
 
+@node Inlinable Procedures
+@subsection Inlinable Procedures
+
+You can define an ``inlinable procedure'' by using
+@code{define-in

Re: Take some lowhanging fruit to speed up R6RS fixnum operations

[Andreas Rottmann]

Andy Wingo  writes:

> On Wed 30 Mar 2011 12:58, Andreas Rottmann  writes:
>
>> "unzip" times the unzipping of a not-so-large (675KiB) ZIP file using
>> Göran Weinholts ZIP implementation.
>
> How long does the standard unix utility take, for the record?
>
On a tmpfs, with the same ZIP file:

unzip ~/public_html/tmp/bundles/xitomatl-20110103.zip  0.07s user 0.03s system 
77% cpu 0.135 total

So there's quite some room for improvement.  Other data points: Ikarus
runs the benchmark in 0.55 seconds, Racket v5.1.0.2 takes 2.39 seconds.

For Dorodango, I intend to refactor ZIP support to allow for different
backends, selectable at runtime, and implement a libzip binding.  This
will get reasonable performance on implementations that don't have a
native-code compiler as fancy as Ikarus's.

Regards, Rotty
-- 
Andreas Rottmann -- <http://rotty.yi.org/>

[PATCH 3/3] Add `fixnum?' VM primitive

[Andreas Rottmann]

This primitive can be used to significantly speed up the operations in
`(rnrs arithmetic fixnums)'.

* libguile/r6rs-arithmetic.c: New file containing `fixnum?' procedure
  implementation as a new extension.
* libguile/r6rs-arithmetic.h: New file with prototypes for the above.
* libguile/Makefile.am: Add above files in relevant places.

* libguile/init.c (scm_i_init_guile): Register R6RS arithmetic
  extension.

* libguile/vm-i-scheme.c (fixnump): New VM primitive.
* module/language/tree-il/compile-glil.scm (*primcall-ops*): Add
  `fixnum?'.
* module/language/tree-il/primitives.scm
  (*interesting-primitive-names*, *effect-free-primitives*)
  (*effect+exception-free-primitives*): Add `fixnum?'.
---
 libguile/Makefile.am |4 ++
 libguile/init.c  |2 +
 libguile/numbers.c   |1 -
 libguile/numbers.h   |1 +
 libguile/r6rs-arithmetic.c   |   48 ++
 libguile/r6rs-arithmetic.h   |   30 ++
 libguile/vm-i-scheme.c   |8 -
 module/language/tree-il/compile-glil.scm |1 +
 module/language/tree-il/primitives.scm   |9 +++--
 module/rnrs/arithmetic/fixnums.scm   |   11 +++
 10 files changed, 103 insertions(+), 12 deletions(-)
 create mode 100644 libguile/r6rs-arithmetic.c
 create mode 100644 libguile/r6rs-arithmetic.h

diff --git a/libguile/Makefile.am b/libguile/Makefile.am
index ac27eb8..01a384d 100644
--- a/libguile/Makefile.am
+++ b/libguile/Makefile.am
@@ -179,6 +179,7 @@ libguile_@GUILE_EFFECTIVE_VERSION@_la_SOURCES = 
\
procs.c \
programs.c  \
promises.c  \
+   r6rs-arithmetic.c   \
r6rs-ports.c\
random.c\
rdelim.c\
@@ -275,6 +276,7 @@ DOT_X_FILES =   \
procprop.x  \
procs.x \
promises.x  \
+   r6rs-arithmetic.x   \
r6rs-ports.x\
random.x\
rdelim.x\
@@ -375,6 +377,7 @@ DOT_DOC_FILES = \
procprop.doc\
procs.doc   \
promises.doc\
+   r6rs-arithmetic.doc \
r6rs-ports.doc  \
random.doc  \
rdelim.doc  \
@@ -569,6 +572,7 @@ modinclude_HEADERS =\
programs.h  \
promises.h  \
pthread-threads.h   \
+   r6rs-arithmetic.h   \
r6rs-ports.h\
random.h\
rdelim.h\
diff --git a/libguile/init.c b/libguile/init.c
index 8b3b8cd..2c23b1e 100644
--- a/libguile/init.c
+++ b/libguile/init.c
@@ -100,6 +100,7 @@
 #include "libguile/programs.h"
 #include "libguile/promises.h"
 #include "libguile/array-map.h"
+#include "libguile/r6rs-arithmetic.h"
 #include "libguile/random.h"
 #include "libguile/rdelim.h"
 #include "libguile/read.h"
@@ -403,6 +404,7 @@ scm_i_init_guile (void *base)
   scm_bootstrap_programs ();
   scm_bootstrap_vm ();
   scm_register_r6rs_ports ();
+  scm_register_r6rs_arithmetic ();
   scm_register_foreign ();
   scm_register_srfi_1 ();
   scm_register_srfi_60 ();
diff --git a/libguile/numbers.c b/libguile/numbers.c
index 427e772..0a10030 100644
--- a/libguile/numbers.c
+++ b/libguile/numbers.c
@@ -6122,7 +6122,6 @@ SCM_DEFINE (scm_integer_p, "integer?", 1, 0, 0,
 }
 #undef FUNC_NAME
 
-
 SCM scm_i_num_eq_p (SCM, SCM, SCM);
 SCM_PRIMITIVE_GENERIC (scm_i_num_eq_p, "=", 0, 2, 1,
(SCM x, SCM y, SCM rest),
diff --git a/libguile/numbers.h b/libguile/numbers.h
index ab96981..fb97785 100644
--- a/libguile/numbers.h
+++ b/libguile/numbers.h
@@ -240,6 +240,7 @@ SCM_API SCM scm_complex_p (SCM x);
 SCM_API SCM scm_real_p (SCM x);
 SCM_API SCM scm_rational_p (SCM z);
 SCM_API SCM scm_integer_p (SCM x);
+SCM_API SCM scm_fixnum_p (SCM x);
 SCM_API SCM scm_inexact_p (SCM x);
 SCM_API SCM scm_num_eq_p (SCM x, SCM y);
 SCM_API SCM scm_less_p (SCM x, SCM y);
diff --git a/libguile/r6rs-arithmetic.c b/libguile/r6rs-arithmetic.c
new file mode 100644
index 000..b00f1f4
--- /dev/null
+++ b/libguile/r6rs-arithmetic.c
@@ -0,0 +1,48 @@
+/* Copyright (C) 2011 Free Software Foundation, Inc.
+ *
+ * This library is free software; you can redi

[PATCH 2/3] Several optimizations for R6RS fixnum arithmetic

[Andreas Rottmann]

* module/rnrs/arithmetic/fixnums.scm (assert-fixnum): Is now a
  macro.
  (assert-fixnums): New procedure checking a the elements of a list
  for fixnum-ness.  All callers applying `assert-fixnum' to a list
  now changed to use this procedure.

* module/rnrs/arithmetic/fixnums.scm (define-fxop*): New for defining
  n-ary inlinable special-casing the binary case using `case-lambda'.
  All applicable procedures redefined using this macro.

* module/rnrs/arithmetic/fixnums.scm: Alias all predicates to
  their non-fixnum counterparts.
---
 module/rnrs/arithmetic/fixnums.scm |   86 +---
 1 files changed, 41 insertions(+), 45 deletions(-)

diff --git a/module/rnrs/arithmetic/fixnums.scm 
b/module/rnrs/arithmetic/fixnums.scm
index befbe9d..03511ed 100644
--- a/module/rnrs/arithmetic/fixnums.scm
+++ b/module/rnrs/arithmetic/fixnums.scm
@@ -87,6 +87,7 @@
most-negative-fixnum)
  (ice-9 optargs)
  (rnrs base (6))
+ (rnrs control (6))
  (rnrs arithmetic bitwise (6))
  (rnrs conditions (6))
  (rnrs exceptions (6))
@@ -105,50 +106,45 @@
 (>= obj most-negative-fixnum) 
 (<= obj most-positive-fixnum)))
 
-  (define (assert-fixnum . args)
+  (define-syntax assert-fixnum
+(syntax-rules ()
+  ((_ arg ...)
+   (or (and (fixnum? arg) ...)
+  (raise (make-assertion-violation))
+
+  (define (assert-fixnums args)
 (or (for-all fixnum? args) (raise (make-assertion-violation
 
-  (define (fx=? fx1 fx2 . rst)
-(let ((args (cons* fx1 fx2 rst)))
-  (apply assert-fixnum args) 
-  (apply = args)))
-
-  (define (fx>? fx1 fx2 . rst)
-(let ((args (cons* fx1 fx2 rst))) 
-  (apply assert-fixnum args) 
-  (apply > args)))
-
-  (define (fx=? fx1 fx2 . rst)
-(let ((args (cons* fx1 fx2 rst)))
-  (apply assert-fixnum rst)
-  (apply >= args)))
-
-  (define (fx<=? fx1 fx2 . rst)
-(let ((args (cons* fx1 fx2 rst)))
-  (apply assert-fixnum rst)
-  (apply <= args)))
-  
-  (define (fxzero? fx) (assert-fixnum fx) (zero? fx))
-  (define (fxpositive? fx) (assert-fixnum fx) (positive? fx))
-  (define (fxnegative? fx) (assert-fixnum fx) (negative? fx))
-  (define (fxodd? fx) (assert-fixnum fx) (odd? fx))
-  (define (fxeven? fx) (assert-fixnum fx) (even? fx))
-
-  (define (fxmax fx1 fx2 . rst)
-(let ((args (cons* fx1 fx2 rst)))
-  (apply assert-fixnum args)
-  (apply max args)))
-
-  (define (fxmin fx1 fx2 . rst)
-(let ((args (cons* fx1 fx2 rst)))
-  (apply assert-fixnum args)
-  (apply min args)))
- 
+  (define-syntax define-fxop*
+(syntax-rules ()
+  ((_ name op)
+   (define name
+   (case-lambda
+ ((x y)
+  (assert-fixnum x y)
+  (op x y))
+ (args
+  (assert-fixnums args)
+   (apply op args)))
+
+  ;; All these predicates don't check their arguments for fixnum-ness,
+  ;; as this doesn't seem to be strictly required by R6RS.
+
+  (define fx=? =)
+  (define fx>? >)
+  (define fx=? >=)
+  (define fx<=? <=)
+
+  (define fxzero? zero?)
+  (define fxpositive? positive?)
+  (define fxnegative? negative?)
+  (define fxodd? odd?)
+  (define fxeven? even?)
+
+  (define-fxop* fxmax max)
+  (define-fxop* fxmin min)
+
   (define (fx+ fx1 fx2)
 (assert-fixnum fx1 fx2) 
 (let ((r (+ fx1 fx2))) 
@@ -219,9 +215,9 @@
   (values s0 s1)))
 
   (define (fxnot fx) (assert-fixnum fx) (lognot fx))
-  (define (fxand . args) (apply assert-fixnum args) (apply logand args))
-  (define (fxior . args) (apply assert-fixnum args) (apply logior args))
-  (define (fxxor . args) (apply assert-fixnum args) (apply logxor args))
+  (define-fxop* fxand logand)
+  (define-fxop* fxior logior)
+  (define-fxop* fxxor logxor)
 
   (define (fxif fx1 fx2 fx3) 
 (assert-fixnum fx1 fx2 fx3) 
-- 
1.7.4.1

[PATCH 1/3] Add a few benchmarks for R6RS fixnum arithmetic

[Andreas Rottmann]

* benchmark-suite/benchmarks/r6rs-arithmetic.bm: New file containing
  some benchmarks for R6RS fixnum operations.
* benchmark-suite/Makefile.am (SCM_BENCHMARKS): Add
  benchmarks/r6rs-arithmetic.
---
 benchmark-suite/Makefile.am   |1 +
 benchmark-suite/benchmarks/r6rs-arithmetic.bm |   35 +
 2 files changed, 36 insertions(+), 0 deletions(-)
 create mode 100644 benchmark-suite/benchmarks/r6rs-arithmetic.bm

diff --git a/benchmark-suite/Makefile.am b/benchmark-suite/Makefile.am
index bac1df3..f29743f 100644
--- a/benchmark-suite/Makefile.am
+++ b/benchmark-suite/Makefile.am
@@ -6,6 +6,7 @@ SCM_BENCHMARKS = benchmarks/0-reference.bm  \
  benchmarks/if.bm  \
  benchmarks/logand.bm  \
 benchmarks/ports.bm\
+benchmarks/r6rs-arithmetic.bm  \
 benchmarks/read.bm \
 benchmarks/srfi-1.bm   \
 benchmarks/srfi-13.bm  \
diff --git a/benchmark-suite/benchmarks/r6rs-arithmetic.bm 
b/benchmark-suite/benchmarks/r6rs-arithmetic.bm
new file mode 100644
index 000..4c9b8e6
--- /dev/null
+++ b/benchmark-suite/benchmarks/r6rs-arithmetic.bm
@@ -0,0 +1,35 @@
+;;; -*- mode: scheme; coding: utf-8; -*-
+;;; R6RS-specific arithmetic benchmarks
+;;;
+;;; Copyright (C) 2011 Free Software Foundation, Inc.
+;;;
+;;; This library is free software; you can redistribute it and/or
+;;; modify it under the terms of the GNU Lesser General Public
+;;; License as published by the Free Software Foundation; either
+;;; version 3 of the License, or (at your option) any later version.
+;;;
+;;; This library is distributed in the hope that it will be useful,
+;;; but WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+;;; Lesser General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU Lesser General Public
+;;; License along with this library. If not, see
+;;; .
+
+(define-module (benchmarks r6rs-arithmetic)
+  #:use-module (benchmark-suite lib)
+  #:use-module (rnrs arithmetic fixnums))
+
+
+(with-benchmark-prefix "fixnum"
+
+  (benchmark "fixnum? [yes]" 1e7
+(fixnum? 1))
+
+  (let ((n (+ most-positive-fixnum 100)))
+(benchmark "fixnum? [no]" 1e7
+  (fixnum? n)))
+
+  (benchmark "fxxor [2]" 1e7
+(fxxor 3 8)))
-- 
1.7.4.1

R6RS fixnum arithmetic optimizations

[Andreas Rottmann]

After getting no feedback on my last mail so far, here is a series of
proposed patches for the stable-2.0 branch.  In total, they speed up the
unzip benchmark by a factor of about 4.7, but it's still about 1.8 times
slower than what we'd get for using generic (non-fixnum) operations.

Re: Take some lowhanging fruit to speed up R6RS fixnum operations

[Andreas Rottmann]

[ Sorry for the incomplete mail I just sent; hit the wrong key combo ]

Andy Wingo  writes:

> On Wed 23 Mar 2011 00:20, Andreas Rottmann  writes:
>
>> In porting dorodango[0], I have noticed that Guile's R6RS fixnum
>> operations are quite slow; here's a patch to remedy that a bit.
>
> What is the state of things here?  I'm a bit lost in the discussion :)
>
> You said that decompressing the zip file takes 74 seconds with Guile as
> it is now.  What if you change to use Guile's arithmetic instead of
> fxops?  That would give is a good baseline so that we can know how much
> overhead the fxops have.
>
As promised, here are a bunch of numbers that will hopefully help to
determine the desired course of action:

| Codebase   | fixnum? | fxxor | unzip |
|+-+---+---|
| vanilla|5.66 | 24.69 |  65.9 |
| generic|5.62 |  1.92 |   7.7 |
| smart  |3.42 | 21.29 |  56.3 |
| smart + opt|3.42 |  6.99 |  21.9 |
| smart + opt-macros | 3.4 |  5.56 |  21.0 |
| vmop   |1.78 | 18.83 |  47.9 |
| vmop + opt |1.77 |  3.94 |  14.1 |
| vmop + opt-macros  |1.78 |   3.7 |  13.3 |

As you can see, in the "vanilla" case (i.e. current stable-2.0), it's
65.9 seconds -- I do not know where I got the 74 second number from,
perhaps it was from a profiling run.

The "generic" case is the one where all fixnum checks have been
eliminated (i.e. using the generic procedures, such as logxor), thus
allowing the compiler to emit VM primitives for most of the operations.

Now, with these baselines established, I implemented the following
optimizations, and benchmarked combinations of them as can be seen from
the above table:

- "smart" refers to having `fixnum?' implemented via `define-inline' and
  using `object-address'.

- "opt" refers to having many fixnum procedures (e.g. fx+, fxxor, ...)
  going through a case-lambda, thus optimizing the common, two-argument
  case not to cons.

- "opt-macros" refers to making the same procedures as in "opt" actually
  macros, expanding to a `(let ((x x-expr) (y y-expr)) (OP x y))' in the
  binary case.

- "vmop" means that the newly-introduced VM primitive `fixnum?' is used.

Regarding the columns, "fixnum?" is the time needed for 10 million
`fixnum?' checks, "fxxor" for 10 million binary `fxxor' operations, and
"unzip" times the unzipping of a not-so-large (675KiB) ZIP file using
Göran Weinholts ZIP implementation.

Now there are several questions:

- Can we put the `fixnum?' VM primitive into stable-2.0?  AFAIK, yes we
  can, but we will break things for users mixing 2.0.0 and 2.0.1 on the
  same machine (or sharing $HOME).  So do we want to?

- Should we complicate the implementation and take a potential code size
  hit by using the "opt-macros" hack, for a 5-6% gain in speed? Of
  course, that's just for the ZIP benchmark, but I think it gives a good
  estimate in what kind of ballpark we are in here.

  My preference would be to go with the simpler implementation ("opt"
  instead of "opt-macros").

- Should further improvements be considered?  I guess if we put the
  fixnum operations into the VM, we could get as fast as the "generic"
  case, and perhaps even a bit faster (i.e. about a 2x speedup on the
  ZIP benchmark compared to vmop + opt).  I'm not sure if that's worth
  it.

Regards, Rotty
-- 
Andreas Rottmann -- <http://rotty.yi.org/>

Re: Take some lowhanging fruit to speed up R6RS fixnum operations

[Andreas Rottmann]

Andy Wingo  writes:

> On Wed 23 Mar 2011 00:20, Andreas Rottmann  writes:
>
>> In porting dorodango[0], I have noticed that Guile's R6RS fixnum
>> operations are quite slow; here's a patch to remedy that a bit.
>
> What is the state of things here?  I'm a bit lost in the discussion :)
>
> You said that decompressing the zip file takes 74 seconds with Guile as
> it is now.  What if you change to use Guile's arithmetic instead of
> fxops?  That would give is a good baseline so that we can know how much
> overhead the fxops have.
>
OK, here are a bunch of numbers that will hopefully help to determine
the desired course of action:

| Codebase | fixnum? | fxxor | unzip |
|--+-+---+---|
| vanilla  |5.66 | 24.69 |  65.9 |
| generic  |5.62 |  1.92 |   7.7 |
| smart|3.42 | 21.29 |  56.3 |
| smart + opt|3.42 |  6.99 |  21.9 |
| smart + opt-macros | 3.4 |  5.56 |  21.0 |
| vmop |1.78 | 18.83 |  47.9 |
| vmop+opt |1.77 |  3.94 |  14.1 |
| vmop+opt-macros  |1.78 |   3.7 |  13.3 |

As you can see, in the "vanilla" case (i.e. current stable-2.0), it's
65.9 seconds -- I do not know where I got the 74 second number from,
perhaps it was from a profiling run.

The "generic" case is the one where all fixnum checks have been
eliminated, thus allowing the compiler to emit VM primitives for most of
the operations.

Now, with these baselines established, I implemented the following
optimizations, and benchmarked combinations of them as can be seen from
the above table.

-- 
Andreas Rottmann -- <http://rotty.yi.org/>

Re: Take some lowhanging fruit to speed up R6RS fixnum operations

[Andreas Rottmann]

Andy Wingo  writes:

> On Wed 23 Mar 2011 00:20, Andreas Rottmann  writes:
>
>> In porting dorodango[0], I have noticed that Guile's R6RS fixnum
>> operations are quite slow; here's a patch to remedy that a bit.
>
> What is the state of things here?  I'm a bit lost in the discussion :)
>
I'll post a summary mail regarding performace soonish; it's quite
tedious to set up branches for all the possibly interesting combinations
of optimizations, and then actually running the benchmarks.

> You said that decompressing the zip file takes 74 seconds with Guile as
> it is now.  What if you change to use Guile's arithmetic instead of
> fxops?  That would give is a good baseline so that we can know how much
> overhead the fxops have.
>
I'll put numbers for that into the benchmark; i.e. with a `(rnrs
arithmetic fixnum)' module just containing aliases.

Regards, Rotty
-- 
Andreas Rottmann -- <http://rotty.yi.org/>

Re: summer of code project: cpan

[Andreas Rottmann]

Andy Wingo  writes:

> Hello all,
>
> Now that GNU is in the Google SoC, I'd like to propose again a CPAN for
> Guile.  (It does needs a proper name, but that name doesn't have to
> correspond to the name of the command-line utility; see my other mail
> about "guido".)
>
> The proposal would be to start from dorodango, and to use stowfs
> locally.  We keep (largely) the dorodango network interfaces, but the
> implementation exists in a Guile-specific project, with a non-dorodango
> name (so as not to conflict with Andreas's more portable project).
>
Well, I believe Dorodango is quite modular, and large parts could be
re-used as-is.  Obviously, I'd like to avoid a "real" fork of the
codebase -- ideally code should be able to flow easily between Dorodango
and its Guile-specific cousin in both directions.  I am also open to
reconsider design decisions, should some of them not make sense for
Guile.

> Locally it uses something like stowfs and $XDG_DATA_DIRS, as noted in
> the previous SoC thread.
>
Support for the XDG Base Directory Specification would indeed be welcome
in Dorodango. Also, I am actually quite fond of how GNU Stow works;
adding support for that as an additional mode of operation would be nice
mini-project.

> The project can be developed outside of Guile initially, just
> integrating by defining "guido" commands.  If everything works it can be
> integrated within Guile itself.
>
One thing that speaks against directly reusing the dorodango codebase
(and later moving parts of it into Guile core) is that it comes with
some dependencies (from Dorodango's own package metadata):

  (srfi)  ; Not relevant on Guile, it provides the required
SRFIs in core
  (wak-foof-loop) ; Used all over the place
  (wak-fmt)   ; Ditto
  (wak-irregex)   ; Used in a few places; SRE syntax is exposed to the
  ; user, however
  (wak-parscheme) ; Used in only one place (minor UI code)
  (spells); That one's really prevasive: pathnames, filesystem,
  ; logging, ...
  (industria) ; Only used for handling ZIP, which is quite slow on
  ; Guile anyway.
  (ocelotl)   ; Weight-balanced trees, HTTP client, URIs.

> Andreas, what do you think about this?  If you are happy with this I
> would be most pleased to have you as a co-mentor.
>
I'd certainly be willing to answer all kinds of questions on the
Dorodango codebase and give advice as good as I can, regardless of what
direction the project takes.

I have a few ideas that would help bringing the codebase of Dorodango
"closer to Guile":

- Work on an SRE frontend to Guile's regexp engine.  I believe there
  might be some code out there doing that already (Guile-SCSH?).

- Introduce a Stow-like mode of operation.  This should be possible
  without abandoning the current ZIP support, and would allow for:

  - Using non-random-access archive files (e.g. tar.xz).

  - Using external programs to (completely) unpack a bundle, eliminating
the current performance issue introduced by Guile's relatively slow
number-crunching.

  - Eliminate the (hard) dependency on Industria.

So what remains as (quite) hard dependencies would be: foof-loop, fmt,
and spells; the others should not be too hard to eliminate in some way
or the other.

WDYT?

Regards, Rotty
-- 
Andreas Rottmann -- <http://rotty.yi.org/>

Re: CPAN for Guile

[Andreas Rottmann]

Andy Wingo  writes:

> On Mon 07 Mar 2011 21:13, Andreas Rottmann  writes:
>
>> I'm (very slowly) working on getting dorodango[0] to work on Guile.  In
>> principle, nothing in Dorodango is inherently tied to R6RS -- it should
>> very reasonably be possible to use it for Guile-specific packages as
>> well.
>>
>> [0] http://home.gna.org/dorodango/
>
> Looks very nice, and well thought-out.  Dorodango would be an excellent
> starting point.
>
> Alaric Snell-Pym gave us an overview of Chicken's system here:
> http://rotty.yi.org/irclogs/freenode/#guile/2011-02-10/#e212
>
> I agree with him that one of the most important aspects about
> CPAN-alikes is the social one: how the system enables the hack, and with
> what flavor.  For example right now Guile has a tight center, enforced
> by the stability needs of Guile, as well as the copyright assignment
> policy, then a wide-orbiting set of comet- and asteroid-like libraries.
> A good CPAN-alike would smooth this out into a gradient of more and less
> experimental hacks, to enable participation of folks with useful
> libraries, but which don't need to be in Guile proper.
>
> It would be good to also think about modules' test suites,
> documentation, code coverage, code analysis and such, and be opinionated
> about that.
>
Yeah, that are all areas I envisioned for dorodango, but have not yet
tackled.

> It would be interesting to have the ability to add on, as an additional
> source of code, repos with portable R6RS code.  
>
Indeed. Dorodango already has the capability to deal with multiple
repositories, and a Guile-specific version of Dorodango would probably
come with a default configuration that points to Guile's (default)
reprository; users would then add additional ones according to their
likes.

> I wonder how this would affect us socially though.
>
No idea :-).

Regards, Rotty
-- 
Andreas Rottmann -- <http://rotty.yi.org/>

Re: Fmt Module

[Andreas Rottmann]

Noah Lavine  writes:

> Hello,
>
>>> I think it would make sense to include ‘fmt’ in core Guile only if the
>>> API is reasonably stable and there are infrequent upstream releases, so
>>> we don’t quickly end up shipping an old incompatible version.
>>
>> Agreed, and I don't know if this is the case or not.
>>
>> I would add on another criteria, that we should avoid adding modules to
>> Guile unless they are to be used in Guile itself.  For everything else,
>> there is our CPAN that we need to make -- it will provide more freedom
>> both for module authors/packagers and for users.
>
> This makes sense, but I think there is an important distinction to
> make that we do not currently make. We should have a clear idea of
> what modules are used by Guile, so we can get a "minimal" Guile when
> we want it, for instance for embedding into other applications.
>
+1!  Racket has acquired a problem with that over the years, and there
are now efforts to rectify it [0].

[0] http://www.mail-archive.com/dev@racket-lang.org/msg02376.html

> However, it would be great if the standard install of Guile included
> lots of modules, such as htmlprag, because it is nice to have the
> batteries included when you are programming.
>
My opinion on that is that if it's really easy to install additional
libraries (which is not yet the case, unfortunatly!), there would be no
need to bundle additional stuff not needed by the core into Guile
releases.

Regards, Rotty
-- 
Andreas Rottmann -- <http://rotty.yi.org/>

Re: [PATCH] Take some lowhanging fruit to speed up R6RS fixnum operations

[Andreas Rottmann]

l...@gnu.org (Ludovic Courtès) writes:

> Hello,
>
> Andreas Rottmann  writes:
>
>> So, that's around 5% improvment (on the ZIP benchmark) for an IMHO
>> significantly more hackish implementation.  I'm not sure that's worth
>> it. WDYT?
>
> Was it with ‘fixnum?’ inline, or with the ‘fixnum?’ instruction?
>
The numbers from my previous mail were with the "clever" `fixnum?'
(i.e. the one that uses object-address and bit-twiddling), defined with
`define-inline'.  I have yet to produce numbers with `fixnum?' as VM
primitive and the more complicated, partially-inlinable fixnum
operators.  I'll keep you posted.

> It’s ironic that while R6RS fixnums are a performance hack, they end up
> being less efficient than unbounded integers in Guile.
>
Indeed.

> Do you know how other implementations deal with that?
>
I will do some research on that topic, investigating Racket, Ikarus and
Ypsilon -- these are the R6RS implementations, which I'm most familiar
with (besides Guile, obviously).

Regards, Rotty
-- 
Andreas Rottmann -- <http://rotty.yi.org/>

Re: Fmt Module

[Andreas Rottmann]

l...@gnu.org (Ludovic Courtès) writes:

> Hello,
>
> Andreas Rottmann  writes:
>
>> l...@gnu.org (Ludovic Courtès) writes:
>
>> However, even if I think based on your criteria nothing speaks against
>> including fmt in Guile, there is still the argument of code duplication:
>> if some external library (e.g., conjure) makes use of wak-fmt and
>> another chooses the version included in Guile, a third program/library
>> can't make use of both of these without ending up with two copies of the
>> `fmt' code loaded, incurring a load-time and memory usage overhead.
>> Obviously, there's also duplicated work involved in maintaining the
>> different adaptions of the `fmt' code.  
>
> I agree.
>
> I think there’s a tension between the interest of Guile, which is to
> provide a convenient way to access useful features, and the interests of
> implementation-neutral “platforms” like Wak.  For instance I find it
> important to have SXML, LALR, etc. usable out-of-the-box; it lowers the
> barrier to entry.
>
I certainly understand that.  Hopefully the CPAN effort for Guile will
bear some fruits and make it very easy to get additional libraries
installed.

> Besides it’s still unclear (to me) what the future of Wak and similar
> projects is.  I hope that it will take off, but I haven’t forgotten
> Snow, ScmPkg, etc. either.
>
Well, there's a (IMHO) important difference in that Wak packages are
based on R6RS, which specifies a module system.  This way, you don't
really need a package manager to install the code from Wak -- you can
reasonably install by copying or symlinking files.

>> I think the ideal solution
>
> I think there’s no ideal solution, not yet.  :-)
>
Indeed, that's why I used the subjunctive and even said that the
(hypothetical) ideal solution is not realistic yet :-p.

Regards, Rotty
-- 
Andreas Rottmann -- <http://rotty.yi.org/>

Re: [PATCH] Take some lowhanging fruit to speed up R6RS fixnum operations

[Andreas Rottmann]

Andreas Rottmann  writes:

> l...@gnu.org (Ludovic Courtès) writes:
>
>> Hi Andreas,
>>
>> I’m all for your suggestion.
>>
>> Andreas Rottmann  writes:
>>
>>> +  (define-syntax define-fxop*
>>> +(syntax-rules ()
>>> +  ((_ name op)
>>> +   (define name
>>> +(case-lambda
>>> +  ((x y)
>>> +   (assert-fixnum x y)
>>> +   (op x y))
>>> +  (args
>>> +   (assert-fixnums args)
>>> +   (apply op args)))
>>> +
>>> +  (define-fxop* fx=? =)
>>
>> How about making something like this (untested):
>>
>>   (define-syntax define-fxop*
>> (syntax-rules ()
>>   ((_ name op)
>>(define-syntax name
>>  (lambda (s)
>>(syntax-case s ()
>>  ((_ x y)
>>   #'(begin
>>   (assert-fixnum x y)
>>   (op x y)))
>>  ((_ args ...)
>>   #'(apply op args))
>>  (_ #'op)))
>>
>> This way, there’d be no procedure call involved and ‘=’, ‘+’, etc. would
>> use the corresponding instruction in the base case.
>>
> That's an excellent idea, and would probably boost performance quite a
> bit.
>
Sorry to reply to myself, but new evidence has been produced :-).

Using the following macro:

  (define-syntax define-fxop*
(lambda (stx)
  (define prefix (string->symbol "% "))
  (define (make-procedure-name name)
(datum->syntax name
   (symbol-append prefix (syntax->datum name)
  '-procedure)))
  (syntax-case stx ()
((_ name op)
 (with-syntax ((proc-name (make-procedure-name #'name)))
   #'(begin
   (define (proc-name . args)
 (assert-fixnums args)
 (apply op args))
   (define-syntax name
 (lambda (stx)
   (syntax-case stx ()
 ((_ x-expr y-expr)
  #'(let ((x x-expr)
  (y y-expr))
  (assert-fixnum x y)
  (op x y)))
 ((_ . args)
  #'(proc-name . args))
 (_
  (identifier? stx)
  #'proc-name))

It turns out that this does not have a great effect on the efficiency;
here are the numbers:

* stable-2.0 + rotty/wip-fixnum-speed (simple define-fxop*, clever fixnum?)

#+begin_example
% _build/meta/guile -e main -s benchmark-suite/guile-benchmark 
--benchmark-suite benchmarks r6rs-arithmetic.bm
;; running guile version 2.0.0.133-e47c9-dirty
;; calibrating the benchmarking framework...
;; framework time per iteration: 1.1444091796875e-7
("r6rs-arithmetic.bm: fixnum: fixnum? [yes]" 1000 user 3.42 benchmark 
2.2755908203125 bench/interp 2.2755908203125 gc 0.0)
("r6rs-arithmetic.bm: fixnum: fixnum? [no]" 1000 user 3.42 benchmark 
2.2755908203125 bench/interp 2.2755908203125 gc 0.0)
("r6rs-arithmetic.bm: fixnum: fxxor [2]" 1000 user 6.96 benchmark 
5.8155908203125 bench/interp 5.8155908203125 gc 0.0)

scheme@(guile-user)> ,time (run-benchmark)
clock utime stime cutime cstime gctime
23.81 23.76  0.00   0.00   0.00   0.00
#+end_example

* stable-2.0 + rotty/wip-fixnum-speed (macro-based define-fxop*, clever 
fixnum?)

#+begin_example
% _build/meta/guile -e main -s benchmark-suite/guile-benchmark 
--benchmark-suite benchmarks r6rs-arithmetic.bm
;; running guile version 2.0.0.133-e47c9-dirty
;; calibrating the benchmarking framework...
;; framework time per iteration: 1.52587890625e-7
("r6rs-arithmetic.bm: fixnum: fixnum? [yes]" 1000 user 3.4 benchmark 
1.87412109375 bench/interp 1.87412109375 gc 0.0)
("r6rs-arithmetic.bm: fixnum: fixnum? [no]" 1000 user 3.42 benchmark 
1.89412109375 bench/interp 1.89412109375 gc 0.0)
("r6rs-arithmetic.bm: fixnum: fxxor [2]" 1000 user 6.72 benchmark 
5.19412109375 bench/interp 5.19412109375 gc 0.0)

scheme@(guile-user)> ,time (run-benchmark)
clock utime stime cutime cstime gctime
22.76 22.73  0.00   0.00   0.00   0.00
#+end_example

So, that's around 5% improvment (on the ZIP benchmark) for an IMHO
significantly more hackish implementation.  I'm not sure that's worth
it. WDYT?

Regards, Rotty
-- 
Andreas Rottmann -- <http://rotty.yi.org/>

Re: [PATCH] Add `fixnum?' VM primitive

[Andreas Rottmann]

l...@gnu.org (Ludovic Courtès) writes:

> Hello!
>
> Andreas Rottmann  writes:
>
>> l...@gnu.org (Ludovic Courtès) writes:
>>
>>> Hi!
>>>
>>> Andreas Rottmann  writes:
>>>
>>>> +SCM_DEFINE (scm_fixnum_p, "fixnum?", 1, 0, 0,
>>>> +(SCM x),
>>>> +"Return @code{#t} if @var{x} is a fixnum, @code{#f} 
>>>> otherwise.")
>>>> +#define FUNC_NAME s_scm_fixnum_p
>>>> +{
>>>> +  return scm_from_bool (SCM_I_INUMP (x));
>>>> +}
>>>> +#undef FUNC_NAME
>>>
>>> For 2.0 I think you could go with this wonderful hack:
>>>
>>>   (define (fixnum? x)
>>> (not (= 0 (logand 2 (object-address x)
>>>
>>> (An inlinable variant thereof, as done in srfi-9.scm.)
>>>
>> Excellent.  Should we put `define-inlinable' in some common place to
>> avoid code duplication?  If so, where?
>
> That would be nice, if we are confident that it’s not broken any more. ;-)
> ISTR that Chez’ ‘define-integrable’ macro is hairier than that, so I
> wonder if our implementation is too naive.  Thoughts?
>
I'll have another look at that; if it turns out it's good, would `(ice-9
inline)' be a proper place for it?

>>> For ‘master’ your patch looks good to me modulo a few details.  Why did
>>> you need to renumber VM opcodes? 
>>>
>> I didn't need to do so, but the neat order elicited my instinct to keep
>> it ;-).  I'll revert that part; should I then put `fixnum?' at the end?
>
> Yes please.
>
> I guess we could even add the new opcode in stable-2.0 because Andy
> tweaked objcode version check to be smarter—i.e., 2.0.1 would be able to
> read objcodes produced by either 2.0.1 or 2.0.0.
>
> WDYT?
>
That would be nice indeed.

>>> Also, I’d prefer not to have ‘fixnum?’ in the default name space
>>> because:
>>>
>>>   1. In Guile parlance, it’d rather be ‘immediate-number?’ (info
>>>  "(guile) Immediate objects").
>>>
>>>   2. I think this fixnum thing is a breach in the numerical tower.
>>>
>> I'm fine with moving it out of the default namespace (I was actually
>> worried about that, too).  However, I vaguely remember the compiler not
>> outputting the opcode when I did not define `fixnum?' as a C-based
>> procedure in the default namespace, but I might be mistaken.  I would
>> appreciate pointer(s) of where to look to learn about how the compiler
>> decides if some procedure can be emitted as an opcode (but I think I
>> would be able to figure that out myself, but pointers would be nice
>> nevertheless ;-)).
>
> In (language tree-il primitives), there’s ‘add-interesting-primitive!’,
> which checks whether the primitive name is bound in the current module.
>
OK, I'll experiment with that.

> BTW why does the ZIP implementation in Industria keep calling ‘fixnum?’?
>
It's actually the implementation of `(rnrs arithmetic fixnums)' that
calls fixnum for almost every operation; also see my other mail [0],
where I talk about lifting that at least for some operations.

[0] http://lists.gnu.org/archive/html/guile-devel/2011-03/msg00235.html

Regards, Rotty
-- 
Andreas Rottmann -- <http://rotty.yi.org/>

Re: [PATCH] Add `fixnum?' VM primitive

[Andreas Rottmann]

l...@gnu.org (Ludovic Courtès) writes:

> Hi!
>
> Andreas Rottmann  writes:
>
>> +SCM_DEFINE (scm_fixnum_p, "fixnum?", 1, 0, 0,
>> +(SCM x),
>> +"Return @code{#t} if @var{x} is a fixnum, @code{#f} otherwise.")
>> +#define FUNC_NAME s_scm_fixnum_p
>> +{
>> +  return scm_from_bool (SCM_I_INUMP (x));
>> +}
>> +#undef FUNC_NAME
>
> For 2.0 I think you could go with this wonderful hack:
>
>   (define (fixnum? x)
> (not (= 0 (logand 2 (object-address x)
>
> (An inlinable variant thereof, as done in srfi-9.scm.)
>
Excellent.  Should we put `define-inlinable' in some common place to
avoid code duplication?  If so, where?

> For ‘master’ your patch looks good to me modulo a few details.  Why did
> you need to renumber VM opcodes? 
>
I didn't need to do so, but the neat order elicited my instinct to keep
it ;-).  I'll revert that part; should I then put `fixnum?' at the end?

> Also, I’d prefer not to have ‘fixnum?’ in the default name space
> because:
>
>   1. In Guile parlance, it’d rather be ‘immediate-number?’ (info
>  "(guile) Immediate objects").
>
>   2. I think this fixnum thing is a breach in the numerical tower.
>
I'm fine with moving it out of the default namespace (I was actually
worried about that, too).  However, I vaguely remember the compiler not
outputting the opcode when I did not define `fixnum?' as a C-based
procedure in the default namespace, but I might be mistaken.  I would
appreciate pointer(s) of where to look to learn about how the compiler
decides if some procedure can be emitted as an opcode (but I think I
would be able to figure that out myself, but pointers would be nice
nevertheless ;-)).

Regards, Rotty
-- 
Andreas Rottmann -- <http://rotty.yi.org/>

Re: [PATCH] Take some lowhanging fruit to speed up R6RS fixnum operations

[Andreas Rottmann]

l...@gnu.org (Ludovic Courtès) writes:

> Hi Andreas,
>
> I’m all for your suggestion.
>
> Andreas Rottmann  writes:
>
>> +  (define-syntax define-fxop*
>> +(syntax-rules ()
>> +  ((_ name op)
>> +   (define name
>> + (case-lambda
>> +   ((x y)
>> +(assert-fixnum x y)
>> +(op x y))
>> +   (args
>> +(assert-fixnums args)
>> +(apply op args)))
>> +
>> +  (define-fxop* fx=? =)
>
> How about making something like this (untested):
>
>   (define-syntax define-fxop*
> (syntax-rules ()
>   ((_ name op)
>(define-syntax name
>  (lambda (s)
>(syntax-case s ()
>  ((_ x y)
>   #'(begin
>   (assert-fixnum x y)
>   (op x y)))
>  ((_ args ...)
>   #'(apply op args))
>  (_ #'op)))
>
> This way, there’d be no procedure call involved and ‘=’, ‘+’, etc. would
> use the corresponding instruction in the base case.
>
That's an excellent idea, and would probably boost performance quite a
bit.  However, there's still the issue that the non-two-args cases don't
throw the exceptions required by R6RS.  Upon trying to interpret the
R6RS in a way that would lift that requirement, I still see no way to do
so in the general case, due to the following quite explicit language:

  Fixnum operations perform integer arithmetic on their fixnum
  arguments, but raise an exception with condition type
  &implementation-restriction if the result is not a fixnum.

So even if checking the arguments seems to be not explicitly required
(but might be implictly intended -- "fixnum arguments"), the result must
definitly be a fixnum, or the mentioned exception be thrown.

So for the sake of efficiency, while heeding what is explictly required,
I propose the following:

- Alias all fixnum operations that yield non-fixnums (e.g. fx?,
  ...) with the corresponding non-fixnum procedures.  IMHO, that seems
  to be within bounds wrt. R6RS; Section 5.4 "Argument checking"
  says:

  The implementation must check that the restrictions in the
  specification are indeed met, to the extent that it is reasonable,
  possible, and necessary to allow the specified operation to complete
  successfully.  

  One might argue that it is not reasonable (for efficiency reasons
  related to the way Guile is currently implemented) and certainly not
  necessary to check the arguments for fixnum-ness in these procedures.

- Otherwise, do the macro trick as above, but fall back on a procedure
  like in SRFI-9's `define-inlinable' for the non-binary case.

Thoughts?

Regards, Rotty
-- 
Andreas Rottmann -- <http://rotty.yi.org/>

Re: [PATCH] Several tweaks for R6RS ports

[Andreas Rottmann]

l...@gnu.org (Ludovic Courtès) writes:

> Hello Andreas,
>
> Thanks, I’ve applied all these.
>
Thanks.

> Could you send a patch to update the manual?
>
I have started working on this, but I'm notorious (at least to myself)
for starving documentation-related work with interesting coding stuff
that comes along, so don't hold your breath ;-).

Regards, Rotty
-- 
Andreas Rottmann -- <http://rotty.yi.org/>

Re: Fmt Module

[Andreas Rottmann]

l...@gnu.org (Ludovic Courtès) writes:

> Hello,
>
> Andreas Rottmann  writes:
>
>> Not that I have any say in that, but IMHO, it would be preferable to
>> keep external libraries maintained separatly from the Guile core;
>> however in this case, including it in the core might be justified by its
>> proposed use in the JIT compiler.
>
> When we do include external libs, we should strive to leave upstream
> files unmodified, as is done for (sxml ssax), (system base lalr),
> (ice-9 match), and others.
>
As I already mentioned, the Wak adaption of `fmt' does not modify
upstream files (except for the test suite); it is just adding library
definition files that include the upstream files.  As wak-fmt seems to
work nicely on Guile, I see no problem here.

> I think it would make sense to include ‘fmt’ in core Guile only if the
> API is reasonably stable and there are infrequent upstream releases, so
> we don’t quickly end up shipping an old incompatible version.
>
I think `fmt' qualifies these criteria.

However, even if I think based on your criteria nothing speaks against
including fmt in Guile, there is still the argument of code duplication:
if some external library (e.g., conjure) makes use of wak-fmt and
another chooses the version included in Guile, a third program/library
can't make use of both of these without ending up with two copies of the
`fmt' code loaded, incurring a load-time and memory usage overhead.
Obviously, there's also duplicated work involved in maintaining the
different adaptions of the `fmt' code.  

I think the ideal solution would be to move the R6RS library definitions
upstream, but I don't think that's realistic just yet, as (a) there's no
built-in `include' in R6RS (I think R7RS will fix that), so an `include'
implementation would either have to be shipped, again resulting in code
duplication with other libraries (as `include' is portably implementable
in R6RS), and (b) there's not yet a real, entrenched standard for naming
of R6RS library files (which again might be fixed by R7RS, and could in
the meantime be mitigated by a package manager such as dorodango,
although implementation-specific library<->file name mapping is not yet
implemented).

Even given all that, if some core component of Guile itself (e.g., a VM
code generator) wants to make use of `fmt', there's probably no way
around shipping a copy in core Guile itself.

Just my 0.02€
-- 
Andreas Rottmann -- <http://rotty.yi.org/>

fixnum? VM primitive, increasing fixnum operation speed

[Andreas Rottmann]

This is another piece of my attempt at getting more speed out of the
`(rnrs arithmetic fixnums)' library.  The idea is that this patch would
be applied to master. I think, at least in the current form, it is not
eligible for 2.0.x, as it messes up the VM opcode numbers, which breaks
all .go files, IIUC.

After this patch has been applied, the R6RS fixnums library can be
adapted to use the new primitive.  I have locally already tried this,
and on top of the previous patch ("Take some lowhanging fruit to speed
up R6RS fixnum operations"), this one speeds up the ZIP benchmark by
another factor of 1.75, yielding a runtime of 15 seconds.  The fxxor
benchmark added by the previous patch gets a ~3x speed boost.

Now, I'm still not entirely satisfied, but these two patches should
cover the lowest hanging fruits without changing the semantics of the
fixnum operations.  

For well-behaved programs, it would not be necessary to do the checks
for fixnum-ness at all, neither for the arguments nor for the return
value.  This would in effect alias the fixnum operations to their
non-fixnum counterparts, violating R6RS, which demands exceptions to be
thrown if arguments or return values exceed fixnum range.  I conjecture
that this would give another noticable boost, as it would eliminate at
least a procedure call and the `fixnum?' checks.

I'd argue that deviating from R6RS in this area is not desirable; after
all I think the intention of the R6RS fixnum library is to provide
specialized operations so that implementations can provide fast-path
versions of the more general procedures when it is known that fixnum
range suffices -- the fact that fixnum arithmetic is actually *slower*
than general arithmetic on Guile is not a given, but an artefact of the
way they are currently implemented.

Ideally (at least speed-wise), all of the fixnum operations should be VM
primitives.  I'm not sure if we have enough opcode space for that.  It
also raises the question of maintainablity (which probably can be kept
in check with appropriate C preprocessor trickery), and how to deal with
the exceptions these procedures are required to throw.  As we probably
don't want to tie the R6RS exception system directly into VM, I was
thinking of using a fluid containing a procedure that does the actual
exception throwing; that fluid could be set by some R6RS library.

Well, this is getting quite long already, so I'll stop for now.  I'm
looking forward to any responses!

,,rotty

[PATCH] Add `fixnum?' VM primitive

[Andreas Rottmann]

This primitive can be used to significantly speed up the operations in
`(rnrs arithmetic fixnums)'.

* libguile/numbers.c (scm_fixnum_p): New predicate.
* libguile/numbers.h: Add prototype.

* libguile/vm-i-scheme.c (fixnump): New VM primitive; renumbered
  subsequent operations.
* module/language/tree-il/compile-glil.scm (*primcall-ops*): Add
  `fixnum?'.
* module/language/tree-il/primitives.scm
  (*interesting-primitive-names*): Add `fixnum?'.
---
 libguile/numbers.c   |9 ++
 libguile/numbers.h   |1 +
 libguile/vm-i-scheme.c   |  148 +++--
 module/language/tree-il/compile-glil.scm |3 +-
 module/language/tree-il/primitives.scm   |4 +-
 5 files changed, 91 insertions(+), 74 deletions(-)

diff --git a/libguile/numbers.c b/libguile/numbers.c
index b8cfa5d..5d42fbf 100644
--- a/libguile/numbers.c
+++ b/libguile/numbers.c
@@ -6118,6 +6118,15 @@ SCM_DEFINE (scm_integer_p, "integer?", 1, 0, 0,
 }
 #undef FUNC_NAME
 
+SCM_DEFINE (scm_fixnum_p, "fixnum?", 1, 0, 0,
+(SCM x),
+"Return @code{#t} if @var{x} is a fixnum, @code{#f} otherwise.")
+#define FUNC_NAME s_scm_fixnum_p
+{
+  return scm_from_bool (SCM_I_INUMP (x));
+}
+#undef FUNC_NAME
+
 
 SCM scm_i_num_eq_p (SCM, SCM, SCM);
 SCM_PRIMITIVE_GENERIC (scm_i_num_eq_p, "=", 0, 2, 1,
diff --git a/libguile/numbers.h b/libguile/numbers.h
index ab96981..fb97785 100644
--- a/libguile/numbers.h
+++ b/libguile/numbers.h
@@ -240,6 +240,7 @@ SCM_API SCM scm_complex_p (SCM x);
 SCM_API SCM scm_real_p (SCM x);
 SCM_API SCM scm_rational_p (SCM z);
 SCM_API SCM scm_integer_p (SCM x);
+SCM_API SCM scm_fixnum_p (SCM x);
 SCM_API SCM scm_inexact_p (SCM x);
 SCM_API SCM scm_num_eq_p (SCM x, SCM y);
 SCM_API SCM scm_less_p (SCM x, SCM y);
diff --git a/libguile/vm-i-scheme.c b/libguile/vm-i-scheme.c
index 19b48c5..7d975aa 100644
--- a/libguile/vm-i-scheme.c
+++ b/libguile/vm-i-scheme.c
@@ -1,4 +1,4 @@
-/* Copyright (C) 2001, 2009, 2010 Free Software Foundation, Inc.
+/* Copyright (C) 2001, 2009, 2010, 2011 Free Software Foundation, Inc.
  * 
  * This library is free software; you can redistribute it and/or
  * modify it under the terms of the GNU Lesser General Public License
@@ -111,12 +111,18 @@ VM_DEFINE_FUNCTION (139, vectorp, "vector?", 1)
   RETURN (scm_from_bool (SCM_I_IS_VECTOR (x)));
 }
 
+VM_DEFINE_FUNCTION (140, fixnump, "fixnum?", 1)
+{
+  ARGS1 (x);
+  RETURN (scm_from_bool (SCM_I_INUMP (x)));
+}
+
 
 /*
  * Basic data
  */
 
-VM_DEFINE_FUNCTION (140, cons, "cons", 2)
+VM_DEFINE_FUNCTION (141, cons, "cons", 2)
 {
   ARGS2 (x, y);
   CONS (x, x, y);
@@ -130,21 +136,21 @@ VM_DEFINE_FUNCTION (140, cons, "cons", 2)
   goto vm_error_not_a_pair; \
 }
   
-VM_DEFINE_FUNCTION (141, car, "car", 1)
+VM_DEFINE_FUNCTION (142, car, "car", 1)
 {
   ARGS1 (x);
   VM_VALIDATE_CONS (x, "car");
   RETURN (SCM_CAR (x));
 }
 
-VM_DEFINE_FUNCTION (142, cdr, "cdr", 1)
+VM_DEFINE_FUNCTION (143, cdr, "cdr", 1)
 {
   ARGS1 (x);
   VM_VALIDATE_CONS (x, "cdr");
   RETURN (SCM_CDR (x));
 }
 
-VM_DEFINE_INSTRUCTION (143, set_car, "set-car!", 0, 2, 0)
+VM_DEFINE_INSTRUCTION (144, set_car, "set-car!", 0, 2, 0)
 {
   SCM x, y;
   POP (y);
@@ -154,7 +160,7 @@ VM_DEFINE_INSTRUCTION (143, set_car, "set-car!", 0, 2, 0)
   NEXT;
 }
 
-VM_DEFINE_INSTRUCTION (144, set_cdr, "set-cdr!", 0, 2, 0)
+VM_DEFINE_INSTRUCTION (145, set_cdr, "set-cdr!", 0, 2, 0)
 {
   SCM x, y;
   POP (y);
@@ -180,27 +186,27 @@ VM_DEFINE_INSTRUCTION (144, set_cdr, "set-cdr!", 0, 2, 0)
   RETURN (srel (x, y));\
 }
 
-VM_DEFINE_FUNCTION (145, ee, "ee?", 2)
+VM_DEFINE_FUNCTION (146, ee, "ee?", 2)
 {
   REL (==, scm_num_eq_p);
 }
 
-VM_DEFINE_FUNCTION (146, lt, "lt?", 2)
+VM_DEFINE_FUNCTION (147, lt, "lt?", 2)
 {
   REL (<, scm_less_p);
 }
 
-VM_DEFINE_FUNCTION (147, le, "le?", 2)
+VM_DEFINE_FUNCTION (148, le, "le?", 2)
 {
   REL (<=, scm_leq_p);
 }
 
-VM_DEFINE_FUNCTION (148, gt, "gt?", 2)
+VM_DEFINE_FUNCTION (149, gt, "gt?", 2)
 {
   REL (>, scm_gr_p);
 }
 
-VM_DEFINE_FUNCTION (149, ge, "ge?", 2)
+VM_DEFINE_FUNCTION (150, ge, "ge?", 2)
 {
   REL (>=, scm_geq_p);
 }
@@ -282,7 +288,7 @@ VM_DEFINE_FUNCTION (149, ge, "ge?", 2)
 #endif
 
 
-VM_DEFINE_FUNCTION (150, add, "add", 2)
+VM_DEFINE_FUNCTION (151, add, "add", 2)
 {
 #ifndef ASM_ADD
   FUNC2 (+, scm_sum);
@@ -294,7 +300,7 @@ VM_DEFINE_FUNCTION (150, add, "add", 2)
 #endif
 }
 
-VM_DEFINE_FUNCTION (151, add1, "add1", 1)
+VM_DEFINE_FUNCTION (152, add1, "add1", 1)
 {
   ARGS1 (x);
 
@@ -316,7 +322,7 @@ VM_DEFINE_FUNCTION (151, add1, "add1", 1)
   RETURN (scm_sum (x, SCM_I_MAKINUM (1)));
 }
 
-VM_DEFINE_FUNCTION (152, sub, "sub", 2)
+VM_DEFINE_FUNCTION (153, sub, "sub", 2)
 {
 #ifndef ASM_SUB
   FUNC2 (-, scm_difference);
@@ -328,7 +334,7 @@ VM_DEFINE_FUNCTION (152, sub, "sub", 2)
 #endif
 }
 
-VM_DEFINE_FUNCTION (153, sub1, "sub1", 1)
+VM_DEFINE_FUNCTION (154, sub1, "sub1", 1)
 {
   ARGS1 (x);
 
@@ -3

[PATCH] Take some lowhanging fruit to speed up R6RS fixnum operations

[Andreas Rottmann]

* module/rnrs/arithmetic/fixnums.scm (assert-fixnum): Is now a
  macro.
  (assert-fixnums): New procedure checking a the elements of a list
  for fixnum-ness.  All callers applying `assert-fixnum' to a list
  now changed to use this procedure.

* module/rnrs/arithmetic/fixnums.scm (define-fxop*): New macro for
  defining n-ary procedures special-casing the binary case via
  case-lambda.  All applicable procedures redefined using this
  macro.

* benchmark-suite/benchmarks/r6rs-arithmetic.bm: New file containing
  some benchmarks for R6RS fixnum operations.
---
 benchmark-suite/benchmarks/r6rs-arithmetic.bm |   35 +
 module/rnrs/arithmetic/fixnums.scm|   69 +++--
 2 files changed, 66 insertions(+), 38 deletions(-)
 create mode 100644 benchmark-suite/benchmarks/r6rs-arithmetic.bm

diff --git a/benchmark-suite/benchmarks/r6rs-arithmetic.bm 
b/benchmark-suite/benchmarks/r6rs-arithmetic.bm
new file mode 100644
index 000..4c9b8e6
--- /dev/null
+++ b/benchmark-suite/benchmarks/r6rs-arithmetic.bm
@@ -0,0 +1,35 @@
+;;; -*- mode: scheme; coding: utf-8; -*-
+;;; R6RS-specific arithmetic benchmarks
+;;;
+;;; Copyright (C) 2011 Free Software Foundation, Inc.
+;;;
+;;; This library is free software; you can redistribute it and/or
+;;; modify it under the terms of the GNU Lesser General Public
+;;; License as published by the Free Software Foundation; either
+;;; version 3 of the License, or (at your option) any later version.
+;;;
+;;; This library is distributed in the hope that it will be useful,
+;;; but WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+;;; Lesser General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU Lesser General Public
+;;; License along with this library. If not, see
+;;; .
+
+(define-module (benchmarks r6rs-arithmetic)
+  #:use-module (benchmark-suite lib)
+  #:use-module (rnrs arithmetic fixnums))
+
+
+(with-benchmark-prefix "fixnum"
+
+  (benchmark "fixnum? [yes]" 1e7
+(fixnum? 1))
+
+  (let ((n (+ most-positive-fixnum 100)))
+(benchmark "fixnum? [no]" 1e7
+  (fixnum? n)))
+
+  (benchmark "fxxor [2]" 1e7
+(fxxor 3 8)))
diff --git a/module/rnrs/arithmetic/fixnums.scm 
b/module/rnrs/arithmetic/fixnums.scm
index befbe9d..8c35dc6 100644
--- a/module/rnrs/arithmetic/fixnums.scm
+++ b/module/rnrs/arithmetic/fixnums.scm
@@ -87,6 +87,7 @@
most-negative-fixnum)
  (ice-9 optargs)
  (rnrs base (6))
+ (rnrs control (6))
  (rnrs arithmetic bitwise (6))
  (rnrs conditions (6))
  (rnrs exceptions (6))
@@ -105,50 +106,42 @@
 (>= obj most-negative-fixnum) 
 (<= obj most-positive-fixnum)))
 
-  (define (assert-fixnum . args)
+  (define-syntax assert-fixnum
+(syntax-rules ()
+  ((_ arg ...)
+   (or (and (fixnum? arg) ...)
+  (raise (make-assertion-violation))
+
+  (define (assert-fixnums args)
 (or (for-all fixnum? args) (raise (make-assertion-violation
 
-  (define (fx=? fx1 fx2 . rst)
-(let ((args (cons* fx1 fx2 rst)))
-  (apply assert-fixnum args) 
-  (apply = args)))
-
-  (define (fx>? fx1 fx2 . rst)
-(let ((args (cons* fx1 fx2 rst))) 
-  (apply assert-fixnum args) 
-  (apply > args)))
-
-  (define (fx=? fx1 fx2 . rst)
-(let ((args (cons* fx1 fx2 rst)))
-  (apply assert-fixnum rst)
-  (apply >= args)))
-
-  (define (fx<=? fx1 fx2 . rst)
-(let ((args (cons* fx1 fx2 rst)))
-  (apply assert-fixnum rst)
-  (apply <= args)))
-  
+  (define-syntax define-fxop*
+(syntax-rules ()
+  ((_ name op)
+   (define name
+(case-lambda
+  ((x y)
+   (assert-fixnum x y)
+   (op x y))
+  (args
+   (assert-fixnums args)
+   (apply op args)))
+
+  (define-fxop* fx=? =)
+  (define-fxop* fx>? >)
+  (define-fxop* fx=? >=)
+  (define-fxop* fx<=? <=)
+
   (define (fxzero? fx) (assert-fixnum fx) (zero? fx))
   (define (fxpositive? fx) (assert-fixnum fx) (positive? fx))
   (define (fxnegative? fx) (assert-fixnum fx) (negative? fx))
   (define (fxodd? fx) (assert-fixnum fx) (odd? fx))
   (define (fxeven? fx) (assert-fixnum fx) (even? fx))
 
-  (define (fxmax fx1 fx2 . rst)
-(let ((args (cons* fx1 fx2 rst)))
-  (apply assert-fixnum args)
-  (apply max args)))
+  (define-fxop* fxmax max)
+  (define-fxop* fxmin min)
 
-  (define (fxmin fx1 fx2 . rst)
-(let ((args (cons* fx1 fx2 rst)))
-  (apply assert-fixnum args)
-  (apply min args)))
- 
   (define (fx+ fx1 fx2)
 (assert-fixnum fx1 fx2) 
 (let ((r (+ fx1 fx2))) 
@@ -219,9 +212,9 @@
   (values s0 s1)))
 
   (define (fxnot fx) (assert-fixnum fx) (lognot fx))
-  (define (fxand . args) (apply assert-fixnum args) (apply logand args))
-  (define (fxior . args) (apply assert-fixnum args) (apply logior args))
-

Take some lowhanging fruit to speed up R6RS fixnum operations

[Andreas Rottmann]

In porting dorodango[0], I have noticed that Guile's R6RS fixnum
operations are quite slow; here's a patch to remedy that a bit.  The
patch contains a benchmark for `fxxor', which experiences a performance
improvement of >2 with the patch.

[0] http://home.gna.org/dorodango/

I also used real code to validate the performance impact, namely Göran
Weinholt's ZIP implementation from his industria collection[1]. There
the patch makes for an ~1.7x improvement for uncompressing a 675KiB
(compressed) ZIP file containing text files that yield good compression.

[1] https://code.launchpad.net/~weinholt/scheme-libraries/industria

However, it must be noted that while this patch improves things,
uncompressing the aforementioned ZIP file takes still 42 seconds on my
machine (down from 74 before).  I have another patch in the queue to
improve this by making `fixnum?' a VM primitive, but even with that,
performance is not stellar.

Re: Fmt Module

[Andreas Rottmann]

pand
- (chicken (use test) (load "fmt-chicken.scm"))
- (gauche
-  (use gauche.test)
-  (use text.fmt)
-  (define test-begin test-start)
-  (define orig-test (with-module gauche.test test))
-  (define-syntax test
+(define-syntax test
 (syntax-rules ()
-  ((test name expected expr)
-   (guard (e (else #f))
-  (orig-test name expected (lambda () expr
   ((test expected expr)
-   (test (let ((s (with-output-to-string (lambda () (write 'expr)
-   (substring s 0 (min 60 (string-length s
- expected expr)
- (else))
+ (test-equal expected expr
+
+;; pretty printing
+
+;; (define-macro (test-pretty str)
+;;   (let ((sexp (with-input-from-string str read)))
+;; `(test ,str (fmt #f (pretty ',sexp)
+
+(define-syntax test-pretty
+  (syntax-rules ()
+((test-pretty str)
+ (let ((sexp (call-with-string-input-port str read)))
+   (test str (fmt #f (pretty sexp)))
 
 (test-begin "fmt")
 
@@ -123,12 +122,10 @@
 (test "1.00" (fmt #f (fix 2 (num/fit 4 1
 (test "#.##" (fmt #f (fix 2 (num/fit 4 12.345
 
-;; (cond
-;;  ((feature? 'full-numeric-tower)
-;;   (test "1+2i" (fmt #f (string->number "1+2i")))
-;;   (test "1+2i" (fmt #f (num (string->number "1+2i"
-;;   (test "1.00+2.00i" (fmt #f (fix 2 (num (string->number "1+2i")
-;;   (test "3.14+2.00i" (fmt #f (fix 2 (num (string->number "3.14159+2i")))
+(test "1+2i" (fmt #f (string->number "1+2i")))
+(test "1+2i" (fmt #f (num (string->number "1+2i"
+(test "1.00+2.00i" (fmt #f (fix 2 (num (string->number "1+2i")
+(test "3.14+2.00i" (fmt #f (fix 2 (num (string->number "3.14159+2i")
 
 (test "3.9Ki" (fmt #f (num/si 3986)))
 (test "4k" (fmt #f (num/si 3986 1000)))
@@ -185,7 +182,7 @@
 (test "prefix: defgh" (fmt #f "prefix: " (fit/left 5 "abcdefgh")))
 (test "prefix: cdefg" (fmt #f "prefix: " (fit/both 5 "abcdefgh")))
 
-(test "abc\n123\n" (fmt #f (fmt-join/suffix (cut trim 3 <>) (string-split "abcdef\n123456\n" "\n") nl)))
+(test "abc\n123\n" (fmt #f (fmt-join (cut trim 3 <>) (string-split "abcdef\n123456\n" "\n") nl)))
 
 ;; utilities
 
@@ -224,18 +221,6 @@
  (wrt/unshared
   (let ((ones (list 1))) (set-cdr! ones ones) ones)
 
-;; pretty printing
-
-;; (define-macro (test-pretty str)
-;;   (let ((sexp (with-input-from-string str read)))
-;; `(test ,str (fmt #f (pretty ',sexp)
-
-(define-syntax test-pretty
-  (syntax-rules ()
-((test-pretty str)
- (let ((sexp (with-input-from-string str read)))
-   (test str (fmt #f (pretty sexp)))
-
 (test-pretty "(foo bar)\n")
 
 (test-pretty
@@ -422,22 +407,24 @@
 
 ;; misc extras
 
+#;
 (define (string-hide-passwords str)
   (string-substitute (regexp "(pass(?:w(?:or)?d)?\\s?[:=>]\\s+)\\S+" #t)
  "\\1**"
  str
  #t))
-
+#;
 (define hide-passwords
   (make-string-fmt-transformer string-hide-passwords))
 
+#;
 (define (string-mangle-email str)
   (string-substitute
(regexp "\\b([-+.\\w]+)@((?:[-+\\w]+\\.)+[a-z]{2,4})\\b" #t)
"\\1 _at_ \\2"
str
#t))
-
+#;
 (define mangle-email
   (make-string-fmt-transformer string-mangle-email))
 

Regards, Rotty
-- 
Andreas Rottmann -- <http://rotty.yi.org/>

Re: [PATCH] Several tweaks for R6RS ports

[Andreas Rottmann]

Andreas Rottmann  writes:

> Hi!
>
> Here's a few patches related to R6RS port support, in short:
>
> - Add missing `get-string-n!' and `get-string-n'
> - Fix a few missing exports
> - A bit of work on transcoder-related stuff
>
Attached is an updated version of the "rnrs-transcoders" patch; relative
to its predecessor, it just adds `textual-port?' to the exports in
`(rnrs)' and has a few tweaks to the ChangeLog entries.

From: Andreas Rottmann 
Subject: Enhance transcoder-related functionality of `(rnrs io ports)'

* module/rnrs/io/ports.scm (transcoder-eol-style)
  (transcoder-error-handling-mode): Export these.
  (textual-port?): Implement this procedure and export it.
* module/rnrs.scm: Export these here as well.

* module/rnrs/io/ports.scm (port-transcoder): Implement this procedure.
  (binary-port?): Treat only ports without an encoding as binary ports, 
  add docstring.

* module/rnrs/io/ports.scm: (standard-input-port, standard-output-port)
  (standard-error-port): Ensure these are created without an encoding.
  (eol-style): Add `none' as enumeration member.
  (native-eol-style): Switch to `none' from `lf'.

* test-suite/tests/r6rs-ports.test (7.2.7 Input ports)
  (8.2.10 Output ports): Test binary-ness of `standard-input-port',
  `standard-output-port' and `standard-error-port'.
  (8.2.6 Input and output ports): Add test for `port-transcoder'.

---
 module/rnrs.scm  |6 +++-
 module/rnrs/io/ports.scm |   42 +
 test-suite/tests/r6rs-ports.test |   32 ++--
 3 files changed, 66 insertions(+), 14 deletions(-)

diff --git a/module/rnrs.scm b/module/rnrs.scm
index 476a3ab..6fde880 100644
--- a/module/rnrs.scm
+++ b/module/rnrs.scm
@@ -162,12 +162,14 @@
 
 	  file-options buffer-mode buffer-mode?
 	  eol-style native-eol-style error-handling-mode
-	  make-transcoder transcoder-codec native-transcoder
+	  make-transcoder transcoder-codec transcoder-eol-style
+  transcoder-error-handling-mode native-transcoder
 	  latin-1-codec utf-8-codec utf-16-codec
 	  
 	  eof-object? port? input-port? output-port? eof-object port-eof?
 	  port-transcoder
-	  binary-port? transcoded-port port-position set-port-position!
+	  binary-port? textual-port? transcoded-port
+	  port-position set-port-position!
 	  port-has-port-position? port-has-set-port-position!?
   close-port call-with-port
 	  open-bytevector-input-port make-custom-binary-input-port get-u8 
diff --git a/module/rnrs/io/ports.scm b/module/rnrs/io/ports.scm
index b98811a..097c9d3 100644
--- a/module/rnrs/io/ports.scm
+++ b/module/rnrs/io/ports.scm
@@ -32,13 +32,14 @@
   ;; auxiliary types
   file-options buffer-mode buffer-mode?
   eol-style native-eol-style error-handling-mode
-  make-transcoder transcoder-codec native-transcoder
+  make-transcoder transcoder-codec transcoder-eol-style
+  transcoder-error-handling-mode native-transcoder
   latin-1-codec utf-8-codec utf-16-codec

   ;; input & output ports
   port? input-port? output-port?
   port-eof?
-  port-transcoder binary-port? transcoded-port
+  port-transcoder binary-port? textual-port? transcoded-port
   port-position set-port-position!
   port-has-port-position? port-has-set-port-position!?
   call-with-port close-port
@@ -128,11 +129,11 @@
   (enum-set-member? symbol (enum-set-universe (buffer-modes
 
 (define-enumeration eol-style
-  (lf cr crlf nel crnel ls)
+  (lf cr crlf nel crnel ls none)
   eol-styles)
 
 (define (native-eol-style)
-  (eol-style lf))
+  (eol-style none))
 
 (define-enumeration error-handling-mode
   (ignore raise replace)
@@ -189,10 +190,30 @@
 ;;;
 
 (define (port-transcoder port)
-  (error "port transcoders are not supported" port))
+  "Return the transcoder object associated with @var{port}, or @code{#f}
+if the port has no transcoder."
+  (cond ((port-encoding port)
+ => (lambda (encoding)
+  (make-transcoder
+   encoding
+   (native-eol-style)
+   (case (port-conversion-strategy port)
+ ((error) 'raise)
+ ((substitute) 'replace)
+ (else
+  (assertion-violation 'port-transcoder
+   "unsupported error handling mode"))
+(else
+ #f)))
 
 (define (binary-port? port)
-  ;; So far, we don't support transcoders other than the binary transcoder.
+  "Returns @code{#t} if @var{port} does not have an associated encoding,
+@code{#f} otherwise."
+  (not (port-encoding port)))
+
+(define (textual-port? port)
+  "Always returns @var{#t}, as all ports can be used for textual I/O in
+Guile."
   #t)
 
 (def

Re: [PATCH] Don't mix definitions and expressions in SRFI-9

[Andreas Rottmann]

l...@gnu.org (Ludovic Courtès) writes:

> Hi,
>
> Andy Wingo  writes:
>
>> On Sun 06 Mar 2011 23:26, l...@gnu.org (Ludovic Courtès) writes:
>>
>>> Andreas Rottmann  writes:
>>>
>>>> The expansion of `define-inlinable' contained an expression, which made
>>>> SRFI-9's `define-record-type' fail in non-toplevel contexts ("definition
>>>> used in expression context").
>>>
>>> SRFI-9 says “Record-type definitions may only occur at top-level”, and
>>> I’m inclined to stick to it.  If we diverge, then people could write
>>> code thinking it’s portable SRFI-9 code while it’s not.
>>
>> Does anyone actually care about this?  We provide many compatible
>> extensions to standard interfaces.  It seems like this would be an
>> "unnecessary restriction which makes `let-record-type' seem necessary".
>
> OK, I lost.  ;-)
>
> But, can we:
>
>   1. Document the extension.
>
>   2. Choose PROC-NAME such that -Wunused-toplevel won’t complain.
>  There’s a trick for this: if it contains white space, then
>  -Wunused-toplevel won’t complain; however, it has to be generated
>      deterministically because it can appear in other compilation units,
>  so we can’t use ‘generate-temporaries’ here.
>
I think the attached version of the patch takes your suggestions into
account.

From: Andreas Rottmann 
Subject: Don't mix definitions and expressions in SRFI-9

The expansion of `define-inlinable' contained an expression, which made
SRFI-9's `define-record-type' fail in non-toplevel contexts ("definition
used in expression context").

* module/srfi/srfi-9.scm (define-inlinable): Get rid of apparently
  useless expression in the expansion, so the expansion yields only
  definitions.  At the same time, use a space in the generated names to
  lessen the chances of name conflicts, also avoiding -Wunused-toplevel
  warnings.
* test-suite/tests/srfi-9.test (non-toplevel): New test verifying that
  `define-record-type' works in non-toplevel context as well.
* doc/ref/srfi-modules.texi (SRFI-9 - define-record-type): Add
  subsubsection noting that Guile does not enforce top-levelness.

---
 doc/ref/srfi-modules.texi|9 -
 module/srfi/srfi-9.scm   |8 +---
 test-suite/tests/srfi-9.test |   12 +++-
 3 files changed, 24 insertions(+), 5 deletions(-)

diff --git a/doc/ref/srfi-modules.texi b/doc/ref/srfi-modules.texi
index bda7cbb..eab8779 100644
--- a/doc/ref/srfi-modules.texi
+++ b/doc/ref/srfi-modules.texi
@@ -1,6 +1,6 @@
 @c -*-texinfo-*-
 @c This is part of the GNU Guile Reference Manual.
-@c Copyright (C)  1996, 1997, 2000, 2001, 2002, 2003, 2004, 2006, 2007, 2008, 2009, 2010
+@c Copyright (C)  1996, 1997, 2000, 2001, 2002, 2003, 2004, 2006, 2007, 2008, 2009, 2010, 2011
 @c   Free Software Foundation, Inc.
 @c See the file guile.texi for copying conditions.
 
@@ -1927,6 +1927,13 @@ The functions created by @code{define-record-type} are ordinary
 top-level @code{define}s.  They can be redefined or @code{set!} as
 desired, exported from a module, etc.
 
+@unnumberedsubsubsec Non-toplevel Record Definitions
+
+The SRFI-9 specification explicitly disallows record definitions in a
+non-toplevel context, such as inside @code{lambda} body or inside a
+@var{let} block.  However, Guile's implementation does not enforce that
+restriction.
+
 @unnumberedsubsubsec Custom Printers
 
 You may use @code{set-record-type-printer!} to customize the default printing
diff --git a/module/srfi/srfi-9.scm b/module/srfi/srfi-9.scm
index 80c3b60..fad570b 100644
--- a/module/srfi/srfi-9.scm
+++ b/module/srfi/srfi-9.scm
@@ -1,6 +1,6 @@
 ;;; srfi-9.scm --- define-record-type
 
-;; 	Copyright (C) 2001, 2002, 2006, 2009, 2010 Free Software Foundation, Inc.
+;; 	Copyright (C) 2001, 2002, 2006, 2009, 2010, 2011 Free Software Foundation, Inc.
 ;;
 ;; This library is free software; you can redistribute it and/or
 ;; modify it under the terms of the GNU Lesser General Public
@@ -69,9 +69,12 @@
   ;; the macro expansion, whereas references in non-call contexts refer to
   ;; the procedure.  Inspired by the `define-integrable' macro by Dybvig et al.
   (lambda (x)
+;; Use a space in the prefix to avoid potential -Wunused-toplevel
+;; warning
+(define prefix (string->symbol "% "))
 (define (make-procedure-name name)
   (datum->syntax name
- (symbol-append '% (syntax->datum name)
+ (symbol-append prefix (syntax->datum name)
 '-procedure)))
 
 (syntax-case x ()
@@ -81,7 +84,6 @@
  #`(begin
  (define (proc-name formals ...)
body ...)
- proc-name ;; unused
  (define-syntax name
  

CPAN for Guile

[Andreas Rottmann]

Andy Wingo  writes:

> Hey all,
>
> We need to get together a few Guile-proposed projects for SoC students
> to hack on.  How about we collect them here in this thread.  I'll kick
> it off with one idea:
>
>   * A CPAN for Guile.  My initial thoughts are that you want a
> combination of APT-like discovery and download of sources of
> software, and a stowfs-like local storage, allowing you to update a
> tree of symlinks in $XDG_DATA_DIRS/guile/2.0/ to add packages, roll
> back changes, etc.  See
> http://standards.freedesktop.org/basedir-spec/basedir-spec-latest.html
> for more on $XDG_DATA_DIRS.
>
> C extensions are outside the scope of this project.  IMO a CPAN for
> Guile should not compile C helper libraries.
>
I'm (very slowly) working on getting dorodango[0] to work on Guile.  In
principle, nothing in Dorodango is inherently tied to R6RS -- it should
very reasonably be possible to use it for Guile-specific packages as
well.

[0] http://home.gna.org/dorodango/

It currently doesn't use a symlinking approach like GNU Stow or scsh
packages, but installs real files.  However, it keeps track of all
installed files (like APT/dpkg) and allows upgrades, uninstalling, etc.
It even contains a full-blown dependency resolver, transcribed from
aptitude's C++ implementation (in the process shrinking ~9KLoC to ~3KLoC
;-)).

> You can start here by doing a survey of what other schemes do:
> chicken and racket being the obvious ones, and also perhaps taking a
> look at cabal and other CPAN-alikes developed in the last 10 years.
>
> You'd need to define a format for code -- tarballs or something.
> You need to think about signing too, and the possibility both of
> dumb servers and smart servers.  You'll need to write a web client
> in Guile, if one hasn't been written yet.  In short, a large project
> for an ambitious and careful hacker!
>
Well, I certainly don't want to stop anyone from coming up with
something entirely Guile-specific, but I want to point out that there's
already a good amount of code out there, and even some documentation
;-).  I'd be thrilled however, if some ambitious, careful hacker would
join me and help me enhance dorodango to work perfectly as Guile's
package manager!

Regards, Rotty
-- 
Andreas Rottmann -- <http://rotty.yi.org/>

Re: [PATCH] Don't mix definitions and expressions in SRFI-9

[Andreas Rottmann]

l...@gnu.org (Ludovic Courtès) writes:

> Hi Andreas,
>
> Andreas Rottmann  writes:
>
>> The expansion of `define-inlinable' contained an expression, which made
>> SRFI-9's `define-record-type' fail in non-toplevel contexts ("definition
>> used in expression context").
>
> SRFI-9 says “Record-type definitions may only occur at top-level”, and
> I’m inclined to stick to it.  If we diverge, then people could write
> code thinking it’s portable SRFI-9 code while it’s not.
>
I can certainly relate to that, and agree that it's a good principle,
however, see below.

> How about adding a ‘let-record-type’ or similar in (srfi srfi-9 gnu)?
>
The issue is not that I'm explictly writing code that uses
`define-record-type' in a non-toplevel context, but that I have a
testing framework (built upon Riastradh's trc-testing), which uses R6RS
`eval' to load testcases.  Since `eval' does not allow for the code to
be evaluated to be in a top-level context, I'm using this:

(eval `(let () ,@code-to-be-tested) the-environment-for-the-code)

Now `code-to-be-tested' can't contain any SRFI-9 record definitions if
the literal interpretation of SRFI-9 is used, even if that code would
run perfectly fine if entered at the REPL or being executed as an R6RS
script (modulo the import statement, but the test runner can work around
that).  This is certainly a special use case, but I think it is
reasonable one.

Regards, Rotty
-- 
Andreas Rottmann -- <http://rotty.yi.org/>

[PATCH] Don't mix definitions and expressions in SRFI-9

[Andreas Rottmann]

As always, see the patch header for details.

From: Andreas Rottmann 
Subject: Don't mix definitions and expressions in SRFI-9

The expansion of `define-inlinable' contained an expression, which made
SRFI-9's `define-record-type' fail in non-toplevel contexts ("definition
used in expression context").

* module/srfi/srfi-9.scm (define-inlinable): Get rid of apparently
  useless expression in the expansion, so the expansion yields only
  definitions.
* test-suite/tests/srfi-9.test (non-toplevel): New test verifying that
  `define-record-type' works in non-toplevel context as well.

---
 module/srfi/srfi-9.scm   |3 +--
 test-suite/tests/srfi-9.test |   12 +++-
 2 files changed, 12 insertions(+), 3 deletions(-)

diff --git a/module/srfi/srfi-9.scm b/module/srfi/srfi-9.scm
index 80c3b60..7c2b073 100644
--- a/module/srfi/srfi-9.scm
+++ b/module/srfi/srfi-9.scm
@@ -1,6 +1,6 @@
 ;;; srfi-9.scm --- define-record-type
 
-;; 	Copyright (C) 2001, 2002, 2006, 2009, 2010 Free Software Foundation, Inc.
+;; 	Copyright (C) 2001, 2002, 2006, 2009, 2010, 2011 Free Software Foundation, Inc.
 ;;
 ;; This library is free software; you can redistribute it and/or
 ;; modify it under the terms of the GNU Lesser General Public
@@ -81,7 +81,6 @@
  #`(begin
  (define (proc-name formals ...)
body ...)
- proc-name ;; unused
  (define-syntax name
(lambda (x)
  (syntax-case x ()
diff --git a/test-suite/tests/srfi-9.test b/test-suite/tests/srfi-9.test
index cf933a8..f8006c4 100644
--- a/test-suite/tests/srfi-9.test
+++ b/test-suite/tests/srfi-9.test
@@ -1,7 +1,7 @@
  srfi-9.test --- Test suite for Guile's SRFI-9 functions. -*- scheme -*-
  Martin Grabmueller, 2001-05-10
 
- Copyright (C) 2001, 2006, 2007, 2010 Free Software Foundation, Inc.
+ Copyright (C) 2001, 2006, 2007, 2010, 2011 Free Software Foundation, Inc.
  
  This library is free software; you can redistribute it and/or
  modify it under the terms of the GNU Lesser General Public
@@ -93,3 +93,13 @@
   ;; prior to guile 1.6.9 and 1.8.1 this wan't enforced
   (pass-if-exception "set-y! on bar" exception:wrong-type-arg
  (set-y! b 99)))
+
+(with-test-prefix "non-toplevel"
+
+  (define-record-type :frotz (make-frotz a b) frotz?
+(a frotz-a) (b frotz-b set-frotz-b!))
+
+  (pass-if "construction"
+(let ((frotz (make-frotz 1 2)))
+  (and (= (frotz-a frotz) 1)
+   (= (frotz-b frotz) 2)
-- 
tg: (d59dd06..) t/srfi-9-fix (depends on: stable-2.0)

Regards, Rotty
-- 
Andreas Rottmann -- <http://rotty.yi.org/>

[PATCH] Several tweaks for R6RS ports

[Andreas Rottmann]

Hi!

Here's a few patches related to R6RS port support, in short:

- Add missing `get-string-n!' and `get-string-n'
- Fix a few missing exports
- A bit of work on transcoder-related stuff

See the patch headers for details.  

The patches are attached in the order they should be applied, although
"get-string-n.diff" and the other two have been developed independently,
so applying them in this order will result in offsets and fuzzyness for
the later patches.  If there are issues with that, apply either
"get-string-n" or both of "rnrs-io-current-ports" and
"rnrs-transcoders", and come back to me, so I can regenerate the
remaining patches to make them apply cleanly.

From: Andreas Rottmann 
Subject: Add `get-string-n' and `get-string-n!' for R6RS ports

* libguile/r6rs-ports.c (scm_get_string_n_x): Implement `get-string-n!'
  in C for efficiency.
* libguile/r6rs-ports.h: Add prototype for this function.
* module/ice-9/binary-ports.scm: Export `get-string-n!'.

* module/rnrs/io/ports.scm (get-string-n): Implement based on
  `get-string-n!'.
  Export both `get-string-n!' and `get-string-n'.
* module/rnrs.scm: Also export these.

* test-suite/tests/r6rs-ports.test (8.2.9 Textual input): Add a few
  tests for `get-string-n' and `get-string-n!'.

---
 libguile/r6rs-ports.c|   42 +-
 libguile/r6rs-ports.h|3 +-
 module/ice-9/binary-ports.scm|1 +
 module/rnrs.scm  |3 +-
 module/rnrs/io/ports.scm |   16 -
 test-suite/tests/r6rs-ports.test |   18 
 6 files changed, 78 insertions(+), 5 deletions(-)

diff --git a/libguile/r6rs-ports.c b/libguile/r6rs-ports.c
index 8058ca0..1f72415 100644
--- a/libguile/r6rs-ports.c
+++ b/libguile/r6rs-ports.c
@@ -1,4 +1,4 @@
-/* Copyright (C) 2009, 2010 Free Software Foundation, Inc.
+/* Copyright (C) 2009, 2010, 2011 Free Software Foundation, Inc.
  *
  * This library is free software; you can redistribute it and/or
  * modify it under the terms of the GNU Lesser General Public License
@@ -1222,6 +1222,46 @@ SCM_DEFINE (scm_i_make_transcoded_port,
 #undef FUNC_NAME
 

+/* Textual I/O */
+
+SCM_DEFINE (scm_get_string_n_x,
+"get-string-n!", 4, 0, 0,
+(SCM port, SCM str, SCM start, SCM count),
+"Read up to @var{count} characters from @var{port} into "
+"@var{str}, starting at @var{start}.  If no characters "
+"can be read before the end of file is encountered, the end "
+"of file object is returned.  Otherwise, the number of "
+"characters read is returned.")
+#define FUNC_NAME s_scm_get_string_n_x
+{
+  size_t c_start, c_count, c_len, c_end, j;
+  scm_t_wchar c;
+
+  SCM_VALIDATE_OPINPORT (1, port);
+  SCM_VALIDATE_STRING (2, str);
+  c_len = scm_c_string_length (str);
+  c_start = scm_to_size_t (start);
+  c_count = scm_to_size_t (count);
+  c_end = c_start + c_count;
+
+  if (SCM_UNLIKELY (c_end > c_len))
+scm_out_of_range (FUNC_NAME, count);
+
+  for (j = c_start; j < c_end; j++)
+{
+  c = scm_getc (port);
+  if (c == EOF)
+{
+  size_t chars_read = j - c_start;
+  return chars_read == 0 ? SCM_EOF_VAL : scm_from_size_t (chars_read);
+}
+  scm_c_string_set_x (str, j, SCM_MAKE_CHAR (c));
+}
+  return count;
+}
+#undef FUNC_NAME
+
+
 /* Initialization.  */
 
 void
diff --git a/libguile/r6rs-ports.h b/libguile/r6rs-ports.h
index edde005..2ae3e76 100644
--- a/libguile/r6rs-ports.h
+++ b/libguile/r6rs-ports.h
@@ -1,7 +1,7 @@
 #ifndef SCM_R6RS_PORTS_H
 #define SCM_R6RS_PORTS_H
 
-/* Copyright (C) 2009, 2010 Free Software Foundation, Inc.
+/* Copyright (C) 2009, 2010, 2011 Free Software Foundation, Inc.
  *
  * This library is free software; you can redistribute it and/or
  * modify it under the terms of the GNU Lesser General Public License
@@ -38,6 +38,7 @@ SCM_API SCM scm_put_u8 (SCM, SCM);
 SCM_API SCM scm_put_bytevector (SCM, SCM, SCM, SCM);
 SCM_API SCM scm_open_bytevector_output_port (SCM);
 SCM_API SCM scm_make_custom_binary_output_port (SCM, SCM, SCM, SCM, SCM);
+SCM_API SCM scm_get_string_n_x (SCM, SCM, SCM, SCM);
 
 SCM_API void scm_init_r6rs_ports (void);
 SCM_INTERNAL void scm_register_r6rs_ports (void);
diff --git a/module/ice-9/binary-ports.scm b/module/ice-9/binary-ports.scm
index 63d09cf..c07900b 100644
--- a/module/ice-9/binary-ports.scm
+++ b/module/ice-9/binary-ports.scm
@@ -37,6 +37,7 @@
 get-bytevector-n!
 get-bytevector-some
 get-bytevector-all
+get-string-n!
 put-u8
 put-bytevector
 open-bytevector-output-port
diff --git a/module/rnrs.scm b/module/rnrs.scm
index 476a3ab..77090d0 100644
--- a/module/rnrs.scm
+++ b/module/rnrs.scm
@@ -182,7 +182,8 @@
   ma

Re: FFI on OS X?

[Andreas Rottmann]

l...@gnu.org (Ludovic Courtès) writes:

> Hi Andreas,
>
> Andreas Rottmann  writes:
>
>> Another related issue that has come up in IRC is versioning: If I
>> understand correctly, it is currently impossible to specify the version
>> of the shared object to be used (as one cannot even pass a full filename
>> to `dynamic-link').  This has two (IMHO) unacceptable implications:
>>
>> (1) On GNU/Linux, the .so symlink has to be installed for the FFI-using
>> code to work.  At least on Debian, this means that the -dev package
>> (which contains that symlink) has to be installed.  In turn, any
>> Guile application or library that would be packaged for Debian would
>> have to depend on the -dev package.  This is Wrong(tm).  There is
>> nothing inherent in a language binding for a given C library that
>> would require the presence of e.g. headers and the static library
>> (or library documentation, which is also often provided in the -dev
>> package) *at runtime*.
>
> I understand and I agree that this is a real problem, but this looks to
> me like a Debian-centric discussion.  I think ltdl’s file lookup rules
> should not be changed just to abide by the packaging rules of a distro.
>
This is not just Debian (and derivatives), I just used Debian as an
example sice it is what I know best.  Splitting out files needed only
for development is a common practice on at least Fedora[0], and
OpenSuSE[1], and I suspect most distributions that ship binary packages.

[0] http://fedoraproject.org/wiki/PackagingGuidelines#Devel_Packages
[1] http://en.opensuse.org/openSUSE:Shared_library_packaging_policy#Examples

>> Currently, guile uses lt_dlopenext(), which does not seem to provide a
>> way to specify ABI version information at all.  I'd propose extending
>> `dynamic-link' to allow for an optional second argument, similiar to
>> Racket's `ffi-lib'[0].  If that argument is provided, Guile would use
>> lt_dlopen() instead of lt_dlopenext(), passing it a shared library name
>> containing the specified ABI information.  Of course the mangling of the
>> library name and ABI version would depend on the platform, but on
>> GNU/Linux, it would work something like this:
>>
>> (dynamic-link "libSDL-1.2" '("0")) ;; calls: lt_dlopen("libSDL-1.2.so.0")
>>
>> [0]
>> http://docs.racket-lang.org/foreign/Loading_Foreign_Libraries.html?q=ffi-lib#(def._((lib._ffi/unsafe..rkt)._ffi-lib))
>
> Can you show actual uses of this of the VERSION argument?
>
> As I mentioned in another message, the SONAME and file name is a
> system-dependent thing.  So my impression is that it would be impossible
> to use it portably.
>
Yep, you are right (unfortunatly ;-)).  However, I think most
"interesting" libraries today use libtool, which uses a portable,
common-denominator mechanism to supply version information (see "7.2
Libtool's versioning system" in the libtool manual), centered around the
concept of an interface number represented as an integer.  So I guess it
might be possible to specify this libtool interface number and extend
libltdl to map this to the right system-specific magic number/name.

> IMO the Right Thing to do on GNU systems (and Solaris, at least) would
> be to allow ‘dynamic-func’ to use symbol versioning [0].  That would be
> finer-grain and would not rely on SONAME magic numbers.
>
Having symbol versioning support is certainly a good thing to have, but
it requires upstream supporting it -- not all libraries provide
versioned symbols.

Regards, Rotty
-- 
Andreas Rottmann -- <http://rotty.yi.org/>

Handling of the actively-maintained branches (master, stable-2.0)

[Andreas Rottmann]

Hi!

I just noticed that currently, master is mostly not being commited to,
and fixes accumulate in stable-2.0.  The exception to this being the
recent change that fixes popen.test (thanks for nailing that one,
Mark!), which has been applied to both branches.  

I wonder how this generally should be handled -- I think the most
appropriate way would be to commit any changes that can go into the
stable release into stable-2.0 (only), and then, at "convenient times"
(perhaps always before committing new, not-for-stable stuff to master)
merge stable-2.0 into master.  It probably doesn't matter, as git seems
to handle duplicate changes quite well (just tried, except for a
conflict in GUILE-VERSION, stable-2.0 merged cleanly into master, even
with the duplicate changeset).  Thoughts?

Regards, Rotty
-- 
Andreas Rottmann -- <http://rotty.yi.org/>