Re: bug#72913: Guile 3.0.10 fails to build for powerpc-darwin (3.0.9 built earlier)

[Ludovic Courtès]

Hi Sergey,

Sergey Fedorov  skribis:

> :info:build In language/cps/guile-vm.scm:
> :info:build 89:31  2 (target-symbol-hash _)
> :info:build 41:18  1 (jenkins-lookup3-hashword2
> "call-with-escape-continuation")
> :info:build In ice-9/boot-9.scm:
> :info:build   1676:22  0 (raise-exception _ #:continuable? _)
> :info:build ice-9/boot-9.scm:1676:22: In procedure raise-exception:
> :info:build Value out of range 0 to< 18446744073709551615: -512332661
> :info:build gmake[2]: *** [Makefile:2516: ice-9/control.go] Error 1

This affects all 32-bit platforms unfortunately:

  https://issues.guix.gnu.org/72215

Ludo’.

Re: [PATCH] Make get-bytevector-all suspendable.

[Ludovic Courtès]

Hi Maxime,

Maxime Devos  skribis:

> It currently is difficult to write a correct implementation of 
> get-bytevector-all in pure Scheme, because ‘get-bytevector-all’ needs to 
> return a _fresh_ bytevector and could return twice (e.g. in case of 
> system-async-mark + call-with-prompt shenanigans). I think the proposed 
> implementation is incorrect in this way.

Hmm I don’t see how it could return twice.  If an async runs while
‘get-bytevector-all’ is executed, it cannot cause ‘get-bytevector-all’
to abort to a prompt.  I think we’re fine, no?

> +(define (get-bytevector-all port)
> +  (define %initial-length 4096)
> +
> +  (let read-loop ((total 0)
> +  (result-length %initial-length)
> +  (result (make-bytevector %initial-length)))
>
> I sense a lack of tests for the test suite ...
> Also, to avoid avoidable allocations and copying, it would be nice if 
> %initial-length could be overridden (optional #:initial-length keyword 
> argument), in case the user has a better guess available.

Yeah, but since this interface is specified in R6RS, and since we’re
just porting the existing C implementation to Scheme, I’d rather not add
such a keyword argument.  We could provide an extension separately
though, as is done for (rnrs bytevectors gnu) for instance.

Regarding tests, you’re right: this is already covered.

I pushed this as 461ff313fa478d207a7668595e9d976a2ace9770 together with
a NEWS entry.

Thanks for your feedback!

Ludo’.

Re: [PATCH] add SRFI-119 / language/wisp to Guile? (new patch with more tests, squashed)

[Ludovic Courtès]

Hi Arne,

"Dr. Arne Babenhauserheide"  skribis:

> From 81e7cbbade4fd01e092a2c39a3af90e4abf7f684 Mon Sep 17 00:00:00 2001
> From: Arne Babenhauserheide 
> Date: Mon, 11 Mar 2024 06:34:52 +0100
> Subject: [PATCH] Add language/wisp, Wisp tests, and SRFI-119 documentation
>
> * doc/ref/srfi-modules.texi (srfi-119): add node
> * module/language/wisp.scm: New file.
> * module/language/wisp/spec.scm: New file.
> * test-suite/tests/srfi-119.test: New file.

I have the pleasure to inform you that I have finally pushed this!  :-)

Apologies for taking so long, and thank you for being patient.

Some of the suggestions I made earlier¹ were still not implemented
though:

  1. Using uninterned symbols rather than UUIDs.

  2. Using a record type for lines instead of tuples.

  3. Avoiding source properties.

I took the liberty to implement #1 in commit
27feb2bfd38087cf03989673da0fc74ed795307d.  Tests pass but please let me
know if you notice something wrong!

It’d be great if you could look at #2 and #3 along the lines of what I
suggested earlier.  This time we should be able to move forward more
quickly.  :-)

Thanks!

Ludo’.

¹ https://lists.gnu.org/archive/html/guile-devel/2023-08/msg9.html

Re: [PATCH] Make get-bytevector-all suspendable.

[Ludovic Courtès]

Hi Chris,

Christopher Baines  skribis:

> I'm looking at this since it's used in (web response)
> read-response-body.
>
> * module/ice-9/suspendable-ports.scm (get-bytevector-all): New
> procedure.
> (port-bindings): Add it.

Given that ‘get-bytevector-n!’ already has a variant in
suspendable-ports.scm, my preference would be to rewrite
‘get-bytevector-all’ in Scheme (patch attached).  That way, it would
naturally be suspendable.  (It’s also in line with the general strategy
of moving things to Scheme.)

I don’t expect significant performance difference compared to the C
implementation since that is dominated by allocations and I/O.

Thoughts?

Ludo’.

diff --git a/libguile/r6rs-ports.c b/libguile/r6rs-ports.c
index 7c51bf617..ffa1e1b2b 100644
--- a/libguile/r6rs-ports.c
+++ b/libguile/r6rs-ports.c
@@ -1,4 +1,4 @@
-/* Copyright 2009-2011,2013-2015,2018-2019,2023
+/* Copyright 2009-2011,2013-2015,2018-2019,2023,2024
  Free Software Foundation, Inc.
 
This file is part of Guile.
@@ -393,58 +393,23 @@ SCM_DEFINE (scm_get_bytevector_some_x, "get-bytevector-some!", 4, 0, 0,
 }
 #undef FUNC_NAME
 
-SCM_DEFINE (scm_get_bytevector_all, "get-bytevector-all", 1, 0, 0,
-	(SCM port),
-	"Read from @var{port}, blocking as necessary, until "
-	"the end-of-file is reached.  Return either "
-	"a new bytevector containing the data read or the "
-	"end-of-file object (if no data were available).")
-#define FUNC_NAME s_scm_get_bytevector_all
-{
-  SCM result;
-  size_t c_len, c_count;
-  size_t c_read, c_total;
-
-  SCM_VALIDATE_BINARY_INPUT_PORT (1, port);
+static SCM get_bytevector_all_var;
 
-  c_len = c_count = 4096;
-  result = scm_c_make_bytevector (c_count);
-  c_total = c_read = 0;
-
-  do
-{
-  if (c_read > c_len - c_total)
-	{
-	  /* Grow the bytevector.  */
-  SCM prev = result;
-
-  if (INT_ADD_OVERFLOW (c_len, c_len))
-scm_num_overflow (FUNC_NAME);
-
-  result = scm_c_make_bytevector (c_len * 2);
-  memcpy (SCM_BYTEVECTOR_CONTENTS (result),
-  SCM_BYTEVECTOR_CONTENTS (prev),
-  c_total);
-	  c_count = c_len;
-	  c_len *= 2;
-	}
-
-  /* `scm_c_read ()' blocks until C_COUNT bytes are available or EOF is
-	 reached.  */
-  c_read = scm_c_read_bytes (port, result, c_total, c_count);
-  c_total += c_read, c_count -= c_read;
-}
-  while (c_count == 0);
-
-  if (c_total == 0)
-return SCM_EOF_VAL;
+static void
+init_bytevector_io_vars (void)
+{
+  get_bytevector_all_var =
+scm_c_public_lookup ("ice-9 binary-port", "get-bytevector-all");
+}
 
-  if (c_len > c_total)
-return scm_c_shrink_bytevector (result, c_total);
+SCM
+scm_get_bytevector_all (SCM port)
+{
+  static scm_i_pthread_once_t once = SCM_I_PTHREAD_ONCE_INIT;
+  scm_i_pthread_once (&once, init_bytevector_io_vars);
 
-  return result;
+  return scm_call_1 (scm_variable_ref (get_bytevector_all_var), port);
 }
-#undef FUNC_NAME
 
 
 
diff --git a/module/ice-9/binary-ports.scm b/module/ice-9/binary-ports.scm
index b7eddc93d..864d9ef9a 100644
--- a/module/ice-9/binary-ports.scm
+++ b/module/ice-9/binary-ports.scm
@@ -1,5 +1,5 @@
 ;;; binary-ports.scm --- Binary IO on ports
-;;; Copyright (C) 2009-2011,2013,2016,2019,2021,2023 Free Software Foundation, Inc.
+;;; Copyright (C) 2009-2011,2013,2016,2019,2021,2023,2024 Free Software Foundation, Inc.
 ;;;
 ;;; This library is free software: you can redistribute it and/or modify
 ;;; it under the terms of the GNU Lesser General Public License as
@@ -27,6 +27,7 @@
 
 (define-module (ice-9 binary-ports)
   #:use-module (rnrs bytevectors)
+  #:autoload   (rnrs bytevectors gnu) (bytevector-slice)
   #:use-module (ice-9 match)
   #:use-module (ice-9 custom-ports)
   #:export (eof-object
@@ -180,3 +181,29 @@ bytevector composed of the bytes written into the port is returned."
 ;; FIXME: Instead default to current encoding, if
 ;; someone reads text from this port.
 #:encoding 'ISO-8859-1 #:conversion-strategy 'error))
+
+
+;;;
+;;; Binary input.
+;;;
+
+(define (get-bytevector-all port)
+  "Read from @var{port}, blocking as necessary, until
+the end-of-file is reached.  Return either a new bytevector containing
+the data read or the end-of-file object (if no data were available)."
+  (define initial-capacity 4096)
+
+  (let loop ((bv (make-bytevector initial-capacity))
+ (capacity initial-capacity)
+ (size 0))
+(match (get-bytevector-n! port bv size (- capacity size))
+  ((? eof-object?)
+   (bytevector-slice bv 0 size))
+  (read
+   (let ((size (+ read size)))
+ (if (= capacity size)
+ (let* ((capacity (* capacity 2))
+(new (make-bytevector capacity)))
+   (bytevector-copy! bv 0 new 0 size)
+   (loop new capacity size))
+ (loop bv capacity size)))

Re: bug#69921: [PATCH] eval-string: Fix setting port column

[Ludovic Courtès]

Jonas Hahnfeld  skribis:

> On Thu, 2024-03-14 at 17:11 +0600, Nikita Domnitskii wrote:
>> ---
>>  module/ice-9/eval-string.scm | 2 +-
>>  1 file changed, 1 insertion(+), 1 deletion(-)
>> 
>> diff --git a/module/ice-9/eval-string.scm b/module/ice-9/eval-string.scm
>> index ea0f1..9cac03632 100644
>> --- a/module/ice-9/eval-string.scm
>> +++ b/module/ice-9/eval-string.scm
>> @@ -81,7 +81,7 @@
>>(if line
>>(set-port-line! port line))
>>(if column
>> -  (set-port-column! port line))
>> +  (set-port-column! port column))
>>  
>>(if (or compile? (not (language-evaluator lang)))
>>((load-thunk-from-memory
>> 
>> 
>
> LGTM (wrong since 2011, it seems), but needs somebody with commit
> access to push.

This was pushed as 025bb024ae4b586dc721e3d2e3471fbea5f8cc81.

Closing!

Ludo’.

Re: bug#69315: [PATCH] Build system fixes

[Ludovic Courtès]

Hi Jonas,

Jonas Hahnfeld  skribis:

> From 428d1b17c5f664d3cb8da4cd5687bd47bdd87877 Mon Sep 17 00:00:00 2001
> From: Jonas Hahnfeld 
> Date: Thu, 22 Feb 2024 21:57:41 +0100
> Subject: [PATCH 1/2] build: Make sed invocation fully portable
>
> Commit 08041d216f attempted to make the "invocation compatible with
> BSD sed", but moving '-i' first does not solve the problem because
> it still requires to pass an argument. Instead just redirect the
> instantiated output into a temporary file and install that.
>
> * libguile/Makefile.am: Remove '-i' from INSTANTIATE.

[...]

> From ad01912391b449fcf547ac52ed468f9b572cb0ad Mon Sep 17 00:00:00 2001
> From: Jonas Hahnfeld 
> Date: Thu, 22 Feb 2024 22:10:06 +0100
> Subject: [PATCH 2/2] build: Fix cross-compilation in out-of-tree-builds
>
> gen-scmconfig.h is generated in libguile, not $(top_builddir).
>
> * libguile/Makefile.am: Add '-I.' when compiling gen-scmconfig.o.

Finally applied both, thanks!

Ludo’.

Re: bug#69314: [PATCH] Speed up stage0 bootstrap build using prebuilts

[Ludovic Courtès]

Hi Jonas,

Jonas Hahnfeld  skribis:

> On Thu, 2024-01-04 at 11:57 +0100, Jonas Hahnfeld wrote:
>> From 95f15821c535537c7ad4fdae1988855314d56ece Mon Sep 17 00:00:00 2001
>> From: Jonas Hahnfeld 
>> Date: Thu, 4 Jan 2024 11:44:55 +0100
>> Subject: [PATCH] Speed up stage0 bootstrap build using prebuilts
>> 
>> Use prebuilt bytecode of ice-9/eval.go and others for all of stage0,
>> it is optimized and evaluation is much faster. In my environment,
>> this speeds up the build of guile-3.0.9 from around 29 minutes to
>> only 19 minutes.
>> 
>> * meta/build-env.in: In stage0, prefer prebuilt bytecode over just
>> compiled stage0 files.
>> ---
>>  meta/build-env.in | 2 +-
>>  1 file changed, 1 insertion(+), 1 deletion(-)
>> 
>> diff --git a/meta/build-env.in b/meta/build-env.in
>> index bdc88ded4..446a536af 100644
>> --- a/meta/build-env.in
>> +++ b/meta/build-env.in
>> @@ -58,7 +58,7 @@ then
>>  fi
>>  export GUILE_LOAD_PATH
>>  case "$GUILE_BOOTSTRAP_STAGE" in
>> -stage0) 
>> GUILE_LOAD_COMPILED_PATH="${top_builddir}/stage0:${top_srcdir}/prebuilt/@SCM_PREBUILT_BINARIES@"
>>  ;;
>> +stage0) 
>> GUILE_LOAD_COMPILED_PATH="${top_srcdir}/prebuilt/@SCM_PREBUILT_BINARIES@:${top_builddir}/stage0"
>>  ;;

I don’t understand why changing the order would make a difference.
Surely if .go files are available under prebuilt/, they’ll be found,
even if that directory comes second?  Or am I missing something?

Thanks for the patch!

Ludo’.

The case for more macro-instructions?

[Ludovic Courtès]

Hi Andy and all,

Looking at the disassembly of -O1 code in a quest for more concise
bytecode¹ (a quest that’s not necessarily always relevant but probably
is at -O1), I noticed a few things:

  1. Code for free variable lookup, emitted by
 ‘emit-cached-toplevel-box’, is too large (~7 instructions per
 variable) for little in return.

  2. The ‘.data’ section is surprisingly large: for each symbol in the
 source, we end up in that section with a string, a stringbuf
 (pointing to contents in the ‘.rodata’ section), and a symbol.
 More on that below.

  3. ‘*lcm-page-size*’ is set to 64 KiB for the purposes of reducing the
 number of .go variants needed under prebuilt/.

 Should we default to sysconf(_SC_PAGESIZE) and use that common
 denominator only when building .go files under prebuilt/ (this
 requires adding a compiler flag to choose a different alignment)?

 (In the meantime, I changed the linker to create sparse files in
 commit 112b617f5921c67b4b2c45aae39f54cccd34d7ef.)

Regarding ‘.data’, look:

--8<---cut here---start->8---
$ echo sym > /tmp/t.scm
$ ./meta/uninstalled-env guild compile /tmp/t.scm -o /tmp/t.go
wrote `/tmp/t.go'
$ readelf -a /tmp/t.go |grep -A10 "^Section Headers"
readelf: Warning: [ 5]: Link field (0) should index a string section.
readelf: Warning: local symbol 0 found at index >= .symtab's sh_info value of 0
readelf: Warning: local symbol 1 found at index >= .symtab's sh_info value of 0
Section Headers:
  [Nr] Name  Type Address   Offset
   Size  EntSize  Flags  Link  Info  Align
  [ 0]   NULL   
        0 0 0
  [ 1] .guile.procprops  PROGBITS   00010588
        0 0 8
  [ 2] .rodata   PROGBITS 0158  0158
   0014     A   0 0 8
  [ 3] .data PROGBITS 0001  0001
   0058    WA   0 0 8
--8<---cut here---end--->8---

That’s 88 bytes for ‘.data’.

If ‘lookup-bound’ would take a string (instead of a symbol) like
‘lookup-bound-private’ and ‘lookup-bound-public’ do, we’d save
relocations and space.

Perhaps these instructions (or rather variants thereof) could even take
a raw UTF-8 buffer instead?

As for #1, I’m not sure what the best option is.  I initially thought
about adding a new macro-instruction, but then we’d lose on cache-hit
path, which is not good.

Thoughts?

Ludo’.

¹ https://issues.guix.gnu.org/70398

Re: [PATCH] Fixes for custom-ports

[Ludovic Courtès]

Hi,

Maxim Cournoyer  skribis:

>> PS: As far as I’m concerned, patches sent to bug-gu...@gnu.org are less
>> likely to be lost because I see them in M-x debbugs.
>>
>
> The contributing documentation mention to send patches to
> guile-devel at gnu dot org; should this guidance be changed?
>
> From the HACKING file:
>
> - If you have put together a change that meets the coding standards
> described below, we encourage you to submit it to Guile.  Post your
> patch to guile-devel@gnu.org.

Good idea; I’ve expanded that part, I hope it’s fine for everyone.

Thanks,
Ludo’.

Re: [PATCH] Fixes for custom-ports

[Ludovic Courtès]

Jonas Hahnfeld via "Developers list for Guile, the GNU extensibility
library"  skribis:

> From b5f1013ad969b6e4e35b36dc63798375ffbecda3 Mon Sep 17 00:00:00 2001
> From: Jonas Hahnfeld 
> Date: Tue, 24 Oct 2023 12:47:21 +0200
> Subject: [PATCH 1/2] Fix loading of custom-ports extension
>
> * module/ice-9/custom-ports.scm: Load extension also in expand and eval.

> From 78c97b8a49ba336516e954c6c62e4baa7f429f47 Mon Sep 17 00:00:00 2001
> From: Jonas Hahnfeld 
> Date: Tue, 24 Oct 2023 19:24:22 +0200
> Subject: [PATCH 2/2] Match on correct argument in make-custom-port
> 
> * module/ice-9/custom-ports.scm (make-custom-port): Match on correct
> argument for conversion strategy.

Applied both, thanks!

Ludo'.

PS: As far as I’m concerned, patches sent to bug-gu...@gnu.org are less
likely to be lost because I see them in M-x debbugs.

Re: [PATCH] Remove and ignore generated files

[Ludovic Courtès]

Hi Jonas,

Jonas Hahnfeld via "Developers list for Guile, the GNU extensibility
library"  skribis:

> running autogen.sh produces a number of generated files that are
> currently not properly ignored or even checked into git. Please take a
> look at the attached patches and apply if suitable.

This is finally (!) applied, thanks!

Ludo’.

Re: Guix Days & Declarative and Minimalistic Computing: CfP for FOSDEM 2024

[Ludovic Courtès]

Hey!

Pjotr Prins  skribis:

> Please spread among our small community. FOSDEM is great, so if you
> happen to want to come to Brussels you can also attend the Guix days
> before:
>
> => https://libreplanet.org/wiki/Group:Guix/FOSDEM2024

I added my name there; consider doing it too.  :-)

Looking forward to meeting all the wonderful people again!

Ludo’.

Re: [PATCHv3] Extensions for SRFI-171 (Transducers)

[Ludovic Courtès]

Hello,

Colin Woodbury  skribis:

> In that case, please hold on the patch. I am currently "rounding out"
> the "one true Transducer API" in several other Lisps, and am close to
> what I consider a representative set of the desirable primitives a
> developer would want out-of-the-box. One I'm done there, I will return
> to this and a few things that are currently missing.
>
> The other Lisps have also achieved a zip-like pattern (i.e. passing
> multiple collections to `transduce` at the same time, thus allowing
> functions like `map` to operate on multiple inputs).
>
> There is also the issue of SRFI-171's disambiguated `transduce-list`,
> `transduce-vector`, etc., since Scheme has no `defgeneric` by
> default. Guile _does_ however, so we could write a generic `transduce`
> in Guile's own extensions. In general though it would be nice if
> SRFI-XYZ "Transducers Redux" could depend on a standardized generics
> system, but alas.
>
> Anyway, thanks for getting back to me. I'll return to this effort in
> due time.

Alright, ping me when you feel ready!

Thanks,
Ludo’.

Re: [PATCH] add SRFI-119 / language/wisp to Guile? (new patch, squashed)

[Ludovic Courtès]

Hello Arne,

"Dr. Arne Babenhauserheide"  skribis:

> I did the changes for the review. It took a while (sorry for that) but
> it got cleaner as a result (thank you!)
>
> Firstoff: I’d like to assign my copyright to the FSF. I’ll sign the
> papers once I receive them. Also I have an Employer disclaimer of rights
> on paper for Emacs already, so that should not cause trouble.

OK.  I assumed you already emailed ass...@gnu.org the relevant form,
right?  Let us know how it goes.

> Changes:
>
> - [X] LGPL
> - [X] Please add the new files to the relevant ‘Makefile.am’ files.
> - [X] Note the changes in Makefiles in the commit
> - [X] Please capitalize “Scheme” and “Wisp” (in general we like to pay 
> attention to typography, spelling, and language in the manual.)
> - [X] s/(wisp)/, also referred to as @dfn{Wisp} (for ``Whitespace …'')/
> - [X] Two spaces after end-of-sentence periods, to facilitate navigation in 
> Emacs.
> - [X] indent “the usual way”
> - [X] comments always with ;; except for margin comments
> - [X] (read-enable 'curly-infix)
>   This needs to be:
> (eval-when (expand load eval)
>   (read-enable 'curly-infix))
> - [X] Please make them #:use-module clauses in the ‘define-module’ form
> - [X] I’d encourage following the usual naming convention, so
>   ‘in-indent?’, ‘in-comment?’, etc.
> - [X] use exception objects or SRFI-35 error conditions instead of throw with 
> symbols
> - [X] add a test for source location info

Awesome.

> A change I did not do:
>
> - [ ] +Use record-types for the lines+. Reason: I decided not to do
>   this because it currently needs to propagate the source properties
>   when retrieving the code, so this is not a good match for a record
>   type (it may become one with an annotated reader, but I’d like to
>   shift that to a later change:

What about having a ‘location’ field in that record type?  Would that
work for you?  Or, alternatively, add source properties just on the
relevant part of the list.

Having lots of ‘car’ and ‘cdr’ in the code to access the various
“fields” of the line hinders readability and prevents proper
type-checking and error-reporting.

As I wrote back in June, source properties are not ideal and explicitly
storing location info, as is done in syntax objects, is preferable:

  https://lists.gnu.org/archive/html/guile-devel/2023-06/msg8.html

Some comments:

> From 5857f8b961562d1ad2ae201401d5343233422eff Mon Sep 17 00:00:00 2001
> From: Arne Babenhauserheide 
> Date: Fri, 3 Feb 2023 22:20:04 +0100
> Subject: [PATCH] Add language/wisp, wisp tests, and srfi-119 documentation
>
> * doc/ref/srfi-modules.texi (srfi-119): add node
> * module/language/wisp.scm: New file.
> * module/language/wisp/spec.scm: New file.
> * test-suite/tests/srfi-119.test: New file.
> * am/bootstrap.am (SOURCES): add language/wisp.scm and language/wisp/spec.scm
> * test-suite/Makefile.am (SCM_TESTS): add tests/srfi-119.test

[...]

> +(define wisp-uuid "e749c73d-c826-47e2-a798-c16c13cb89dd")
> +;; define an intermediate dot replacement with UUID to avoid clashes.
> +(define repr-dot ; .
> +  (string->symbol (string-append "REPR-DOT-" wisp-uuid)))

As I wrote in June, please remove those UUIDs and use uninterned symbols
instead (which cannot be serialized but can be compared with ‘eq?’,
which is all we need.)

> +(define (match-charlist-to-repr charlist)
> +  (let
> +  ((chlist (reverse charlist)))
> +(cond
> + ((equal? chlist (list #\.))
> +  repr-dot)
> + ((equal? chlist (list #\'))
> +  repr-quote)

This would probably be more readable with ‘match’ instead of ‘cond’.

Also, as mentioned regarding the naming convention, please write
‘char-list’ or ‘chars’ rather than ‘chlist’.

> +(define (wisp-read port)
> +  "wrap read to catch list prefixes."
  ^
Capital letter.  Also maybe mention PORT and the return value.

> +  (cond
> +   ((or (< prefix-maxlen (length peeked)) (eof-object? (peek-char port)) 
> (equal? #\space (peek-char port)) (equal? #\newline (peek-char port)))
> +(if repr-symbol ; found a special symbol, return it.
> +repr-symbol
> +(let unpeek
> +((remaining peeked))

Please avoid long lines and write ‘let’ on a line as in:

  (let ((x 2))
…)

or:

  (let loop ((x 1))
…)

(This is the style commonly used elsewhere in Guile.)

> +(define (line-continues? line)
> +  (equal? repr-dot (car (line-code line
> +
> +(define (line-only-colon? line)
> +  (and
> +   (equal? ":" (car (line-code line)))
> +   (null? (cdr (line-code line)
> +
> +(define (line-empty-code? line)
> +  (null? (line-code line)))

I think this illustrates excessive use of lists, car, and cdr, and its
impact on readability.

> +(define (with-read-options opts thunk)
> +  (let ((saved-options (read-options)))
> +(dynamic-wind
> +(lambda ()
> +  (read-options opts))
> +thunk
> +(lambda ()
> +  (read-options saved-options)
> +
> +(define (wisp-

Re: [PATCHv3] Extensions for SRFI-171 (Transducers)

[Ludovic Courtès]

Hi Colin,

Colin Woodbury  skribis:

> Hi again, any thoughts on these SRFI-171 extensions?

Looong ago, before we waited for copyright assignment to be on file, I
made minor suggestions on the patch, which I think you haven’t responded
to:

  https://lists.gnu.org/archive/html/guile-devel/2023-01/msg00044.html

Could you take a look?

Apologies for the delays, we’re pioneering “slow dev” I guess.  :-)

Thanks,
Ludo’.

Re: [PATCH] doc: Use archived URL from Internet Archive for syntax-rules primer.

[Ludovic Courtès]

Hi,

Bruno Victal  skribis:

> * doc/ref/api-macros.texi (Syntax Rules): Use archived URL from
> Internet Archive for syntax-rules primer.

Finally applied, thanks!

Ludo’.

Re: Functional datatypes in Guile

[Ludovic Courtès]

Hello!

pukkamustard  skribis:

> I'm all for including SRFI-146 in Guile proper!
>
> If nobody else is up for it, I'll try and prepare some patches.

Yay, awesome!

Please make sure to add a section to the manual, tests, and then see

regarding copyright.

Thanks,
Ludo’.

Re: Functional datatypes in Guile

[Ludovic Courtès]

Hello,

pukkamustard  skribis:

> I've been using SRFI-146
> (https://srfi.schemers.org/srfi-146/srfi-146.html) for functional
> mappings. There's a Guile port:
> https://inqlab.net/git/guile-srfi-146.git/ (also in Guix -
> guile-srfi-146).

I’m late to the party, but I think it’d be nice to integrate this in
Guile proper, and possibly other implementations like fash/fector.

These data structures are crucial and the code is very much “write
once”, so cheap in terms of maintenance, so I’m all for getting them in
Guile.

WDYT?

Thanks,
Ludo’.

Re: [PATCH] add SRFI-119 / language/wisp to Guile? (new patch, squashed)

[Ludovic Courtès]

Hi!

"Dr. Arne Babenhauserheide"  skribis:

>>> Ludovic Courtès  writes:

[...]

>> Given the complexities in changing the way languages are handled (the
>> required discussions, as we’ve seen in the not yet resolved discussion),
>> would you be OK with keeping the question about adding support for
>> SRFI-119 to Guile separate from the general discussion about language
>> handling?
>>
>> Are there improvements needed besides the ones I did thanks to the
>> review by Maxime or is this good to go?
>
> I created a new (squashed) version of the patch to simplify reviewing:

It does, thank you!

Overall I’m confident the code has been battle-tested (I suppose there
are minimal changes compared to Wisp, right?), so I’ll comment mostly on
cosmetic issues.

> From c7a50f632293cf88f241d45d1fd52fa2f58ce772 Mon Sep 17 00:00:00 2001
> From: Arne Babenhauserheide 
> Date: Fri, 3 Feb 2023 22:20:04 +0100
> Subject: [PATCH] Add language/wisp, wisp tests, and srfi-119 documentation
>
> * doc/ref/srfi-modules.texi (srfi-119): add node
> * module/language/wisp.scm: New file.
> * module/language/wisp/spec.scm: New file.
> * test-suite/tests/srfi-119.test: New file.

Please add the new files to the relevant ‘Makefile.am’ files.

> +* SRFI-119::Wisp: simpler indentation-sensitive scheme.

Please capitalize “Scheme” (in general we like to pay attention to
typography, spelling, and language in the manual.)

> +@subsection SRFI-119 Wisp: simpler indentation-sensitive scheme.
> +@cindex SRFI-119
> +@cindex wisp

Same: “Scheme”, “Wisp”.

> +The languages shipped in Guile include SRFI-119 (wisp), an encoding of

s/(wisp)/, also referred to as @dfn{Wisp} (for ``Whitespace …'')/

> +Scheme that allows replacing parentheses with equivalent indentation and
> +inline colons. See

Two spaces after end-of-sentence periods, to facilitate navigation in
Emacs.

> +++ b/module/language/wisp.scm
> @@ -0,0 +1,761 @@
> +;;; Wisp
> +
> +;; Copyright (C) 2013, 2017, 2018, 2020 Free Software Foundation, Inc.
> +;; Copyright (C) 2014--2023 Arne Babenhauserheide.
> +;; Copyright (C) 2023 Maxime Devos 

I see you don’t have a copyright assignment on file for Guile.  If you
wish, you can assign copyright to the FSF:

  https://lists.gnu.org/archive/html/guile-devel/2022-10/msg8.html

Let us know what you’d like to do.

> +(define-module (language wisp)
> +   #:export (wisp-scheme-read-chunk wisp-scheme-read-all 
> +   wisp-scheme-read-file-chunk wisp-scheme-read-file
> +   wisp-scheme-read-string))

Please remove tabs from this file and indent it “the usual way”.  You
can do that by passing it through Emacs and using ‘M-x indent-region’,
or possibly using ‘guix style -f’.

> +; use curly-infix by default

Use two leading colons for comments, except for margin comments.

> +(read-enable 'curly-infix)

This needs to be:

  (eval-when (expand load eval)
(read-enable 'curly-infix))

> +(use-modules
> +  (srfi srfi-1)
> +  (srfi srfi-11); for let-values
> +  (ice-9 rw); for write-string/partial
> +  (ice-9 match))

Please make them #:use-module clauses in the ‘define-module’ form above.

> +;; Helper functions for the indent-and-symbols data structure: '((indent 
> token token ...) ...)
> +(define (line-indent line)
> + (car line))
> +
> +(define (line-real-indent line)
> + "Get the indentation without the comment-marker for unindented 
> lines (-1 is treated as 0)."
> + (let (( indent (line-indent line)))
> + (if (= -1 indent)
> +   0
> +   indent)))
> +
> +(define (line-code line)
> + (let ((code (cdr line)))
> + ; propagate source properties
> + (when (not (null? code))
> +(set-source-properties! code (source-properties line)))
> + code))

Instead of using lists or pairs for “lines”, I suggest defining a proper
record type, like:

  (define-record-type 
(input-line indent content)
input-line?
(indent  input-line-indent)
(content input-line-content))

This will make the code easier to read and type-clear.

> +; literal values I need
> +(define readcolon 
> +   (string->symbol ":"))
> +
> +(define wisp-uuid "e749c73d-c826-47e2-a798-c16c13cb89dd")
> +; define an intermediate dot replacement with UUID to avoid clashes.
> +(define repr-dot ; .
> +   (string->symbol (string-append "REPR-DOT-" wisp-uuid)))
> +
> +; allow using reader additions as the first element on a line to prefix the 
> list
> +(define repr-quote ; '
> +   (string->symbol (string-append "REPR-QUOTE-" wisp-uuid)))
> +(define repr-unqu

Re: (. wtf?)

[Ludovic Courtès]

HI,

Dmitry Alexandrov  skribis:

> On Fri, 05 May 2023 16:35 Ludovic Courtès  wrote:
>> (call-with-input-string "(. wtf?)" read)
>>
>> ⇒ wtf?
>>
>> #Guile #Scheme
>
> Hey!^W  Sorry...
>
> Dear Guile developer,
>
> your tweet made me deeply concerned.  Is it a sign that this behaviour is 
> going to be ‘fixed’ eventually?

I should clarify that it’s definitely a weird corner case, but not
something I’m worried about.

As I mentioned on Mastodon, Guile’s ‘read’ has always worked like that I
suppose; when he rewrite ‘read’ in Scheme, Andy took care to preserve
the exact same behavior, even for weird cases like this one.

We could remove support for it as it doesn’t bring anything, but I don’t
see any pressure to do so.

Ludo’.

Re: [PATCH] add language/wisp to Guile?

[Ludovic Courtès]

Hello!

Maxime Devos  skribis:

> Why should Wisp be a separate package when other SRFIs are made part
> of Guile?  Your point about maintenance and evolving applies equally
> to other SRFIs.

That’s a good point.  Making it available as (srfi srfi-119) would make
sense I guess.  I need to take a closer look…

>> Adding #lang support in Guile would be nice.  As discussed on IRC, it
>> can be experimented with in a WIP branch.
>
> Have you seen my messages on how the "#lang" construct is problematic
> for some languages, and how alternatives like "[comment delimiter] -*-
> stuff: scheme/ecmascript/... -*- [comment delimiter]" appear to be
> equally simple (*) and not have any downsides (**).
>
> (*) The port encoding detection supports "-*- coding: whatever -*-",
> presumably that functionality could be reused.
>
> (**) For compatibility with Racket, it's not like we couldn't
> implement both "#lang" and "-*- stuff: language -*-".

I haven’t seen your messages yet, I just wanted to express support of
the general idea.  For years, we have discussed #lang support; I know
Andy is enthusiastic about it, and while I was initially reluctant, I’ve
come to appreciate the idea.

What you point out is worth considering, but note that Guile already
supports #!r6rs for instance.

Thanks,
Ludo’.

Re: [PATCH 0/3] Add '-Wunused-module'

[Ludovic Courtès]

Hello!

Jan Nieuwenhuizen  skribis:

> It seems that only re-exporting a (non-macro) variable from an otherwise
> un used module gives a false positive `unused module' warning; it
> doesn't even say `possibly unused module'.  Not a big problem, but can
> anything be done about that?

I fixed that and pushed the result to ‘main’:

  e2ed33ef0 Remove unnecessary module imports.
  89c3bae3c Add -Wunused-module.
  821e0f9cd Add 'record-case' to '.dir-locals.el'.

Thanks again for your feedback!

Ludo’.

Re: [PATCH] Print backtraces for syntax errors too.

[Ludovic Courtès]

Maxime Devos  skribis:

> ;; Before:
> ;;  unknown file:#f:#f: syntax-stuff-twice: bad in subform
> # follow"> of # very hard to follow">
> ;; After:
> ;;  [the same thing]
> ;;
> ;; Looks like another patch is needed ...

What backtrace are you trying to get?

Getting a backtrace showing which macros are being expanded (similar to
what GCC does) would be great, but it’s much more work; changing this
one line in libguile won’t achieve that.

Ludo’.

Re: [PATCH] Print backtraces for syntax errors too.

[Ludovic Courtès]

Hi,

Maxime Devos  skribis:

> For complicated macros, especially macros that are used correctly but
> have a bug in their implementation somewhere and use 'syntax-case'
> or 'syntax-rules' multiple times, it can be very convenient to know
> _which_ syntax-case or syntax-rules raised the syntax-error.
>
> E.g., I'm currently debugging some changes to a (non-Guile) macro,
> and I don't know what to make of the following -- the '#:getter . datum-type'
> isn't even present in the original code anywhere:
>
> ice-9/boot-9.scm:1685:16: In procedure raise-exception:
> Syntax error:
> unknown location: source expression failed to match any pattern in form 
> (#:getter . datum-type)
> make: *** [Makefile:1333: gnu/gnunet/dht/client.go] Fout 1
>
> As such, partially revert the following commit that does not give a
> rationale on how backtraces for syntax errors aren't helpful.

Do you have a simple reproducer and a before/after comparison showing
what Guile prints?

Thanks,
Ludo’.

Re: [PATCH] add language/wisp to Guile?

[Ludovic Courtès]

Hi!

Sorry for the late reply.

FWIW, I think it might be best to keep Wisp as a separate package: that
allows it to evolve independently of Guile (and possibly more quickly
:-)), and it might simplify maintenance in some way.

Adding #lang support in Guile would be nice.  As discussed on IRC, it
can be experimented with in a WIP branch.

We’ll then have to discuss when to incorporate it.  My gut feeling is
that it may have to wait until the next stable series (3.2.x), as this
is quite a core change, but let’s see what people think.

Thanks!

Ludo’.

Re: [PATCH 0/3] Add '-Wunused-module'

[Ludovic Courtès]

Hey janneke!

Jan Nieuwenhuizen  skribis:

> Ludovic Courtès writes:
>
> Hello,
>
>> The new ‘-Wunused-module’ warning is enabled at ‘-W2’ only.  The main
>> reason for not enabling it at ‘-W1’ is that in the case of modules used
>> at macro-expansion time only, such as (srfi srfi-26), it cannot
>> determine whether a module is definitely unused.  In this case, the
>> compiler reports the module as “possibly unused”, and it is up to the
>> programmer to check that claim.
> [..]
>> Thoughts?
>
> The `possibly undefined' warnings about the srfi-9 gnu, srfi-26 and
> curried-definition modules are a bit unfortunate, but easy to check.
> It allowed me to trim quite some imports in Dezyne modules.
> Very nice!

Good!

> It seems that only re-exporting a (non-macro) variable from an otherwise
> un used module gives a false positive `unused module' warning; it
> doesn't even say `possibly unused module'.  Not a big problem, but can
> anything be done about that?

Oh, I hadn’t thought about that.  It should be possible to fix it, I’ll
take a look.

Thanks for testing!

Ludo’.

Re: [RFC,PATCH] Do not GC_INIT the Boehm GC if already initialized

[Ludovic Courtès]

Hi Maxime,

Maxime Devos  skribis:

>> [RFC,PATCH] Do not GC_INIT the Boehm GC if already initialized
> On 06-02-2023 19:34, Jose E. Marchesi wrote:
>> Hello Guile hackers.
>> We are in the process of integrating GNU poke[1] in GDB by mean of
>> libpoke.
>> Problem is, libpoke uses the Boehm GC, as guile does.  We are
>> working on
>> switching to an ad-hoc exact collector, but it will get some time.
>> So, in the interim, we may:
>> 1) Make both libguile and libpoke to do GC_INIT conditionally, only
>> if
>> no one else has initialized the collector before.  This is already in
>> poke master.  A suggested (untested!) patch for guile below. > [...]
>
> According to the Boehm GC documentation, this 'conditional
> initialisation' is unnecessary:
>
> /* Portable clients should call this at the program start-up.  More   */
> /* over, some platforms require this call to be done strictly from the*/
> /* primordial thread.  **Multiple invocations are harmless.** */
> #define GC_INIT() [...]
>
> (emphasis added).

The “Multiple invocations” bit isn’t in libgc 8.0.4.  Which version are
you looking at?

Thanks,
Ludo’.

[PATCH 2/3] Add 'record-case' to '.dir-locals.el'.

[Ludovic Courtès]

* module/language/tree-il/fix-letrec.scm (fix-letrec): Remove "Local
Variables" bit.
* .dir-locals.el (scheme-mode): Add 'record-case'.
---
 .dir-locals.el | 1 +
 module/language/tree-il/fix-letrec.scm | 4 
 2 files changed, 1 insertion(+), 4 deletions(-)

diff --git a/.dir-locals.el b/.dir-locals.el
index 90257e7bf..908670479 100644
--- a/.dir-locals.el
+++ b/.dir-locals.el
@@ -47,6 +47,7 @@
  (eval . (put '$letrec 'scheme-indent-function 3))
  (eval . (put '$kclause'scheme-indent-function 1))
  (eval . (put '$fun'scheme-indent-function 1))
+ (eval . (put 'record-case 'scheme-indent-function 1))
  (eval . (put 'syntax-parameterize 'scheme-indent-function 1
  (emacs-lisp-mode . ((indent-tabs-mode . nil)))
  (texinfo-mode. ((indent-tabs-mode . nil)
diff --git a/module/language/tree-il/fix-letrec.scm 
b/module/language/tree-il/fix-letrec.scm
index 2cd550ae9..12c1d500a 100644
--- a/module/language/tree-il/fix-letrec.scm
+++ b/module/language/tree-il/fix-letrec.scm
@@ -318,7 +318,3 @@
  
  (else x)))
  x)))
-
-;;; Local Variables:
-;;; eval: (put 'record-case 'scheme-indent-function 1)
-;;; End:
-- 
2.39.1

[PATCH 1/3] Add -Wunused-module.

[Ludovic Courtès]

* module/language/tree-il/analyze.scm (): New record type.
(unused-module-analysis): New variable.
(make-unused-module-analysis): New analysis.
(make-analyzer): Add it.
* module/system/base/message.scm (%warning-types): Add 'unused-module'.
* test-suite/tests/tree-il.test (%opts-w-unused-module): New variable.
("warnings")["unused-module"]: New test prefix.
* NEWS: Update.
---
 NEWS|  17 
 module/language/tree-il/analyze.scm | 138 ++-
 module/system/base/message.scm  |  11 ++-
 test-suite/tests/tree-il.test   | 141 +++-
 4 files changed, 304 insertions(+), 3 deletions(-)

diff --git a/NEWS b/NEWS
index 4313880c7..a0009406f 100644
--- a/NEWS
+++ b/NEWS
@@ -4,6 +4,23 @@ See the end for copying conditions.
 
 Please send Guile bug reports to bug-gu...@gnu.org.
 
+
+Changes in 3.0.10 (since 3.0.9)
+
+* New interfaces and functionality
+
+** New warning: unused-module
+
+This analysis, enabled at `-W2', issues warnings for modules that appear
+in a `use-modules' form or as a #:use-module clause of `define-module',
+and whose bindings are unused.  This is useful to trim the list of
+imports of a module.
+
+In some cases, the compiler cannot conclude whether a module is
+definitely unused---this is notably the case for modules that are only
+used at macro-expansion time, such as (srfi srfi-26).  In those cases,
+the compiler reports it as "possibly unused".
+
 
 Changes in 3.0.9 (since 3.0.8)
 
diff --git a/module/language/tree-il/analyze.scm 
b/module/language/tree-il/analyze.scm
index 7918b9ddd..ef68e2b9b 100644
--- a/module/language/tree-il/analyze.scm
+++ b/module/language/tree-il/analyze.scm
@@ -1,6 +1,6 @@
 ;;; Diagnostic warnings for Tree-IL
 
-;; Copyright (C) 2001,2008-2014,2016,2018-2022 Free Software Foundation, Inc.
+;; Copyright (C) 2001,2008-2014,2016,2018-2023 Free Software Foundation, Inc.
 
  This library is free software; you can redistribute it and/or
  modify it under the terms of the GNU Lesser General Public
@@ -334,6 +334,139 @@ given `tree-il' element."
 
  (make-reference-graph vlist-null vlist-null #f
 
+
+;;;
+;;; Unused module analysis.
+;;;
+
+;; Module uses and references to bindings of imported modules.
+(define-record-type 
+  (module-info location qualified-references
+   toplevel-references toplevel-definitions)
+  module-info?
+  (location  module-info-location);location vector | #f
+  (qualified-references  module-info-qualified-references) ;module name vhash
+  (toplevel-references   module-info-toplevel-references) ;list of symbols
+  (toplevel-definitions  module-info-toplevel-definitions)) ;symbol vhash
+
+(define unused-module-analysis
+  ;; Report unused modules in the given tree.
+  (make-tree-analysis
+   (lambda (x info env locs)
+ ;; Going down into X: extend INFO accordingly.
+ (match x
+   ((or ($  loc module name)
+($  loc module name))
+(let ((references (module-info-qualified-references info)))
+  (if (vhash-assoc module references)
+  info
+  (module-info (module-info-location info)
+   (vhash-cons module #t references)
+   (module-info-toplevel-references info)
+   (module-info-toplevel-definitions info)
+   ((or ($  loc module name)
+($  loc module name))
+(if (equal? module (module-name env))
+(let ((references (module-info-toplevel-references info)))
+  (module-info (module-info-location info)
+   (module-info-qualified-references info)
+   (cons x references)
+   (module-info-toplevel-definitions info)))
+(let ((references (module-info-qualified-references info)))
+  (module-info (module-info-location info)
+   (vhash-cons module #t references)
+   (module-info-toplevel-references info)
+   (module-info-toplevel-definitions info)
+   (($  loc module name)
+(module-info (module-info-location info)
+ (module-info-qualified-references info)
+ (module-info-toplevel-references info)
+ (vhash-consq name x
+  (module-info-toplevel-definitions info
+
+   ;; Record the approximate location of the module import.  We
+   ;; could parse the #:imports arguments to determine the location
+   ;; of each #:use-module but we'll leave that as an exercise for
+   ;; the reader.
+   (($  loc ($  _ '(guile) 'define-module*))
+(module-info loc
+ (module-info-qualified-references info)
+ (module-info-toplevel-references info)
+ (module-info-toplevel-definitions info)))
+   (($  loc ($  _ '(guil

[PATCH 3/3] Remove unnecessary module imports.

[Ludovic Courtès]

These were found with:

  make GUILE_WARNINGS='-W1 -Wunused-module'

* module/ice-9/copy-tree.scm:
* module/ice-9/eval-string.scm:
* module/ice-9/getopt-long.scm:
* module/ice-9/poll.scm:
* module/ice-9/popen.scm:
* module/ice-9/sandbox.scm:
* module/ice-9/threads.scm:
* module/sxml/apply-templates.scm:
* module/sxml/simple.scm:
* module/system/base/types.scm:
* module/system/repl/command.scm:
* module/system/repl/common.scm:
* module/system/repl/coop-server.scm:
* module/system/repl/debug.scm:
* module/system/repl/error-handling.scm:
* module/system/repl/repl.scm:
* module/system/repl/server.scm:
* module/system/vm/assembler.scm:
* module/system/vm/disassembler.scm:
* module/system/vm/dwarf.scm:
* module/system/vm/elf.scm:
* module/system/vm/frame.scm:
* module/system/vm/inspect.scm:
* module/system/vm/linker.scm:
* module/system/vm/program.scm:
* module/system/vm/trace.scm:
* module/system/vm/trap-state.scm:
* module/system/vm/traps.scm:
* module/system/xref.scm:
* module/texinfo/indexing.scm:
* module/texinfo/plain-text.scm:
* module/texinfo/reflection.scm:
* module/texinfo/string-utils.scm:
* module/web/client.scm:
* module/web/http.scm:
* module/web/request.scm:
* module/web/response.scm: Remove imports of unused modules.
---
 module/ice-9/copy-tree.scm| 1 -
 module/ice-9/eval-string.scm  | 1 -
 module/ice-9/getopt-long.scm  | 1 -
 module/ice-9/poll.scm | 1 -
 module/ice-9/popen.scm| 1 -
 module/ice-9/sandbox.scm  | 1 -
 module/ice-9/threads.scm  | 1 -
 module/sxml/apply-templates.scm   | 2 --
 module/sxml/simple.scm| 1 -
 module/system/base/types.scm  | 1 -
 module/system/repl/command.scm| 3 ---
 module/system/repl/common.scm | 2 --
 module/system/repl/coop-server.scm| 1 -
 module/system/repl/debug.scm  | 6 --
 module/system/repl/error-handling.scm | 1 -
 module/system/repl/repl.scm   | 4 
 module/system/repl/server.scm | 1 -
 module/system/vm/assembler.scm| 2 --
 module/system/vm/disassembler.scm | 2 --
 module/system/vm/dwarf.scm| 2 --
 module/system/vm/elf.scm  | 2 --
 module/system/vm/frame.scm| 2 --
 module/system/vm/inspect.scm  | 5 -
 module/system/vm/linker.scm   | 2 --
 module/system/vm/program.scm  | 1 -
 module/system/vm/trace.scm| 3 ---
 module/system/vm/trap-state.scm   | 1 -
 module/system/vm/traps.scm| 2 --
 module/system/xref.scm| 1 -
 module/texinfo/indexing.scm   | 1 -
 module/texinfo/plain-text.scm | 3 ---
 module/texinfo/reflection.scm | 2 --
 module/texinfo/string-utils.scm   | 2 --
 module/web/client.scm | 3 ---
 module/web/http.scm   | 2 --
 module/web/request.scm| 1 -
 module/web/response.scm   | 2 --
 37 files changed, 70 deletions(-)

diff --git a/module/ice-9/copy-tree.scm b/module/ice-9/copy-tree.scm
index e1d91ad9e..004167821 100644
--- a/module/ice-9/copy-tree.scm
+++ b/module/ice-9/copy-tree.scm
@@ -23,7 +23,6 @@
 
 
 (define-module (ice-9 copy-tree)
-  #:use-module (ice-9 match)
   #:use-module (srfi srfi-11)
   #:replace (copy-tree))
 
diff --git a/module/ice-9/eval-string.scm b/module/ice-9/eval-string.scm
index 789980938..ea0f1 100644
--- a/module/ice-9/eval-string.scm
+++ b/module/ice-9/eval-string.scm
@@ -21,7 +21,6 @@
 (define-module (ice-9 eval-string)
   #:use-module (system base compile)
   #:use-module (system base language)
-  #:use-module (system vm program)
   #:use-module (system vm loader)
   #:replace (eval-string))
 
diff --git a/module/ice-9/getopt-long.scm b/module/ice-9/getopt-long.scm
index 14eaf8e23..18b235390 100644
--- a/module/ice-9/getopt-long.scm
+++ b/module/ice-9/getopt-long.scm
@@ -161,7 +161,6 @@
   #:use-module (srfi srfi-9)
   #:use-module (ice-9 match)
   #:use-module (ice-9 regex)
-  #:use-module (ice-9 optargs)
   #:export (getopt-long option-ref))
 
 (define %program-name (make-fluid "guile"))
diff --git a/module/ice-9/poll.scm b/module/ice-9/poll.scm
index 57b5047ab..2688270ac 100644
--- a/module/ice-9/poll.scm
+++ b/module/ice-9/poll.scm
@@ -19,7 +19,6 @@
 
 (define-module (ice-9 poll)
   #:use-module (srfi srfi-9)
-  #:use-module (srfi srfi-9 gnu)
   #:use-module (rnrs bytevectors)
   #:export (make-empty-poll-set
 poll-set?
diff --git a/module/ice-9/popen.scm b/module/ice-9/popen.scm
index e638726a4..957cde0aa 100644
--- a/module/ice-9/popen.scm
+++ b/module/ice-9/popen.scm
@@ -19,7 +19,6 @@
  
 
 (define-module (ice-9 popen)
-  #:use-module (rnrs bytevectors)
   #:use-module (ice-9 binary-ports)
   #:use-module (ice-9 threads)
   #:use-module (srfi srfi-1)
diff --git a/module/ice-9/sandbox.scm b/module/ice-9/sandbox.scm
index fcfc57365..601485cce 100644
--- a/module/ice-9/sandbox.scm
+++ b/module/ice-9/sandbox.scm
@@ -21,7 +21,6 @@
 ;;; Code:

[PATCH 0/3] Add '-Wunused-module'

[Ludovic Courtès]

Hello Guilers!

Following a discussion started at the Guix Days and continued online¹,
it seems we could benefit from a compiler warning for unused modules,
and this is what this patch does.

The new ‘-Wunused-module’ warning is enabled at ‘-W2’ only.  The main
reason for not enabling it at ‘-W1’ is that in the case of modules used
at macro-expansion time only, such as (srfi srfi-26), it cannot
determine whether a module is definitely unused.  In this case, the
compiler reports the module as “possibly unused”, and it is up to the
programmer to check that claim.

Currently we cannot do any better because warnings operate at the
tree-il, after macro expansion, and there’s nothing indicating whether a
piece of code results from macro expansion.

There’s also the rare case of modules imported for their side effects,
not for their bindings, and that should be kept even if seemingly
“unused”.  One example might be the use of (rnrs bytevectors) in (rnrs
io ports).  But again, that’s super unusual.

Anyway, it’s pretty useful already and has allowed me to trim imports in
Guile modules.

Thoughts?

Ludo’.

¹ https://lists.gnu.org/archive/html/guix-devel/2023-02/msg00028.html

Ludovic Courtès (3):
  Add -Wunused-module.
  Add 'record-case' to '.dir-locals.el'.
  Remove unnecessary module imports.

 .dir-locals.el |   1 +
 NEWS   |  17 +++
 module/ice-9/copy-tree.scm |   1 -
 module/ice-9/eval-string.scm   |   1 -
 module/ice-9/getopt-long.scm   |   1 -
 module/ice-9/poll.scm  |   1 -
 module/ice-9/popen.scm |   1 -
 module/ice-9/sandbox.scm   |   1 -
 module/ice-9/threads.scm   |   1 -
 module/language/tree-il/analyze.scm| 138 +++-
 module/language/tree-il/fix-letrec.scm |   4 -
 module/sxml/apply-templates.scm|   2 -
 module/sxml/simple.scm |   1 -
 module/system/base/message.scm |  11 +-
 module/system/base/types.scm   |   1 -
 module/system/repl/command.scm |   3 -
 module/system/repl/common.scm  |   2 -
 module/system/repl/coop-server.scm |   1 -
 module/system/repl/debug.scm   |   6 --
 module/system/repl/error-handling.scm  |   1 -
 module/system/repl/repl.scm|   4 -
 module/system/repl/server.scm  |   1 -
 module/system/vm/assembler.scm |   2 -
 module/system/vm/disassembler.scm  |   2 -
 module/system/vm/dwarf.scm |   2 -
 module/system/vm/elf.scm   |   2 -
 module/system/vm/frame.scm |   2 -
 module/system/vm/inspect.scm   |   5 -
 module/system/vm/linker.scm|   2 -
 module/system/vm/program.scm   |   1 -
 module/system/vm/trace.scm |   3 -
 module/system/vm/trap-state.scm|   1 -
 module/system/vm/traps.scm |   2 -
 module/system/xref.scm |   1 -
 module/texinfo/indexing.scm|   1 -
 module/texinfo/plain-text.scm  |   3 -
 module/texinfo/reflection.scm  |   2 -
 module/texinfo/string-utils.scm|   2 -
 module/web/client.scm  |   3 -
 module/web/http.scm|   2 -
 module/web/request.scm |   1 -
 module/web/response.scm|   2 -
 test-suite/tests/tree-il.test  | 141 -
 43 files changed, 305 insertions(+), 77 deletions(-)


base-commit: 9d339ea1a95c3b2d04a88aa6b116f997349fc4f4
-- 
2.39.1

Re: Resurrecting top-notch continuous integration for Guile

[Ludovic Courtès]

Hi!

Ludovic Courtès  skribis:

> There’s still much tweaking we can do, but the basics are in place, and
> I find it pretty cool.  :-)  If there are no objections, I’ll merge this
> branch into “main”.  It’s going to be nice to have a dashboard to look
> at before the next release!

Merged in Guile’s ‘main’ branch as commit
32f33756d0fbbf28e848f087375f75c265d0a46c!

If Guix is your thing, you can now get started hacking on Guile with:

  guix shell

The continuous integration jobs are visible at:

  https://ci.guix.gnu.org/jobset/guile

Happy hacking!

Ludo’.

Resurrecting top-notch continuous integration for Guile

[Ludovic Courtès]

Hello Guilers!

(And hello Guix—that’s about an interesting use case. :-))

Back in 2009, together with Rob Vermaas of NixOS, we set up continuous
integration for Guile (among other GNU packages), using Nix coupled with
Hydra, its continuous integration (CI) system:

  https://lists.gnu.org/archive/html/guile-devel/2009-11/msg00084.html

The CI jobs still exist although they’ve been bitrotting for ~10 years:

  https://hydra.nixos.org/jobset/gnu/guile-2-0
  https://git.savannah.gnu.org/cgit/hydra-recipes.git/tree/guile/release.nix

Ten years ago is when I embarked on this side project *cough* called
Guix (Guix = Guile + Nix, see?).  To close the loop, it’s natural to
reinstate CI, but this time with Guix and Cuirass, its companion CI
tool—Guile code all the way down!  You can see these new CI jobs on the
build farm of the Guix project:

  https://ci.guix.gnu.org/jobset/guile

(Speaking of which, I hereby ask fellow Guix hackers to voice any
concerns they may have regarding this use of project resources.)

Code for these jobs is currently in a Guile branch:

  https://git.savannah.gnu.org/cgit/guile.git/log/?h=wip-cuirass-ci

This branch brings several things:

  1. The ‘guix.scm’ file at the top can be used to spawn a shell with a
 development environment for Guix:

   guix shell

 It can also be used to (cross-)build Guile, as in:

   guix build -f guix.scm --target=x86_64-w64-mingw32

  2. ‘build-aux/manifest.scm’ defines a Guix “manifest” for several
 Guile builds we may want to test: building with Clang,
 cross-compiling to MinGW and RISCV and whatnot, building with
 ‘--disable-threads’, etc.

  3. There’s a ‘.guix-channel’ file at the top, meaning that Guile can
 be used as a Guix “channel” providing additional packages.  You’d
 do that by having something as following in
 ~/.config/guix/channels.scm and then running ‘guix pull’:

   (cons (channel
  (name 'guile)
  (url "https://git.savannah.gnu.org/git/guile.git";)
  (branch "wip-cuirass-ci"))
 %default-channels)

 This is arguably the weirdest part, but that’s how we get Cuirass
 to consume Guile.

There’s still much tweaking we can do, but the basics are in place, and
I find it pretty cool.  :-)  If there are no objections, I’ll merge this
branch into “main”.  It’s going to be nice to have a dashboard to look
at before the next release!

Feedback welcome!

Ludo’.


signature.asc
Description: PGP signature

Re: Add internal definitions to derived forms

[Ludovic Courtès]

Hello!

Linus Björnstam  skribis:

> This commit adds internal definitions to derived conditional forms,
> with-fluids and and-let*. This means the bodies of when, unless and
> with-fluids, and the clause bodies of case and cond behave like a
> lambda body.
>
> There is no performance hit since guile optimizes a (let () ...)
> without internal definitions to a begin (i.e: no new lexical context
> is created).

Daniel pushed this as 764e3614b8c13de604399572a67d071621e9ca21 in
‘main’.  I had completely overlooked this thread but I wasn’t quite sure
about it, so I did not include it in 3.0.9.

The reason I’m hesitant is that, while I think it’s nice to be able to
have local ‘define’ in these contexts, I’m wary of diverging from R5RS
and R6RS.  Since it’s a one-way change (we won’t be able to revert it
once people rely on it), I thought we’d rather be careful.

What do people think?  Andy?

Thanks,
Ludo’.

GNU Guile 3.0.9 released

[Ludovic Courtès]

We are pleased to announce GNU Guile release 3.0.9, the latest in the
3.0 stable release series, corresponding to 138 commits by 27 people
since 3.0.8.

This release provides many bug fixes as well as new functionality,
including new bindings to POSIX interfaces.  See the ‘NEWS’ excerpt
below for details.

Compared to the previous stable series (2.2.x), Guile 3.0 adds support
for just-in-time native code generation, speeding up all Guile programs.


Guile is an implementation of the Scheme programming language, packaged
for use in a wide variety of environments.  In addition to implementing
the R5RS, R6RS, and R7RS Scheme standards, Guile includes full access to
POSIX system calls, networking support, multiple threads, dynamic
linking, a foreign function call interface, powerful string processing,
and HTTP client and server implementations.

Guile can run interactively, as a script interpreter, and as a Scheme
compiler to VM bytecode.  It is also packaged as a library so that
applications can easily incorporate a complete Scheme interpreter/VM.
An application can use Guile as an extension language, a clean and
powerful configuration language, or as multi-purpose "glue" to connect
primitives provided by the application.  It is easy to call Scheme code
from C code and vice versa.  Applications can add new functions, data
types, control structures, and even syntax to Guile, to create a
domain-specific language tailored to the task at hand.

Check out the web page for more info and resources:

  https://gnu.org/software/guile

Guile 3.0.9 can be installed in parallel with Guile 2.2.x; see
http://www.gnu.org/software/guile/manual/html_node/Parallel-Installations.html.

Here are the compressed sources:
  https://ftp.gnu.org/gnu/guile/guile-3.0.9.tar.gz   (9.3MB)
  https://ftp.gnu.org/gnu/guile/guile-3.0.9.tar.lz   (5.2MB)
  https://ftp.gnu.org/gnu/guile/guile-3.0.9.tar.xz   (5.5MB)

Here are the GPG detached signatures:
  https://ftp.gnu.org/gnu/guile/guile-3.0.9.tar.gz.sig
  https://ftp.gnu.org/gnu/guile/guile-3.0.9.tar.lz.sig
  https://ftp.gnu.org/gnu/guile/guile-3.0.9.tar.xz.sig

Use a mirror for higher download bandwidth:
  https://www.gnu.org/order/ftp.html

Here are the SHA1 and SHA256 checksums:

6ce38ec3fefc19aa08d4662e9b054f7018a72004  guile-3.0.9.tar.gz
GFJQea0poNRtFcdlgbXZHIcCMBv9ghZm0uHRNyYWKBE  guile-3.0.9.tar.gz
bcc02997587cdd03a831ee8d7153cad92629dc3f  guile-3.0.9.tar.lz
vA7go2D7E5GcFOtuJFMxmt8eyZgojJk4KbzxePtIzJo  guile-3.0.9.tar.lz
bf6af1aac320a56233d4d8c0fbeb2c0dca474eab  guile-3.0.9.tar.xz
GiYlrHKyNm6VeS8/51j9Lfd1tARKkKSpeHMm5mwNdQ0  guile-3.0.9.tar.xz

The SHA256 checksum is base64 encoded, instead of the
hexadecimal encoding that most checksum tools default to.

Use a .sig file to verify that the corresponding file (without the
.sig suffix) is intact.  First, be sure to download both the .sig file
and the corresponding tarball.  Then, run a command like this:

  gpg --verify guile-3.0.9.tar.gz.sig

The signature should match the fingerprint of the following key:

  pub   rsa4096 2014-08-11 [SC]
3CE4 6455 8A84 FDC6 9DB4  0CFB 090B 1199 3D9A EBB5
  uid   [ unknown] Ludovic Courtès 
  uid   [ unknown] Ludovic Courtès 
  uid   [ unknown] Ludovic Courtès (Inria) 

If that command fails because you don't have the required public key,
or that public key has expired, try the following commands to retrieve
or refresh it, and then rerun the 'gpg --verify' command.

  gpg --recv-keys 3CE464558A84FDC69DB40CFB090B11993D9AEBB5

As a last resort to find the key, you can try the official GNU
keyring:

  wget -q https://ftp.gnu.org/gnu/gnu-keyring.gpg
  gpg --keyring gnu-keyring.gpg --verify guile-3.0.9.tar.gz.sig


This release was bootstrapped with the following tools:
  Autoconf 2.71
  Automake 1.16.5
  Libtool 2.4.7
  Gnulib v0.1-5703-g356a414e8c
  Makeinfo 7.0.1


Changes in 3.0.9 (since 3.0.8)

* Notable changes

* New interfaces and functionality

** New `spawn' procedure to spawn child processes

The new `spawn' procedure creates a child processes executing the given
program.  It lets you control the environment variables of that process
and redirect its standard input, standard output, and standard error
streams.

Being implemented in terms of `posix_spawn', it is more portable, more
robust, and more efficient than the combination of `primitive-fork' and
`execl'.  See "Processes" in the manual for details, and see the 2019
paper entitled "A fork() in the road" (Andrew Baumann et al.) for
background information.

`system*', as well as the `open-pipe' and `pipeline' procedures of
(ice-9 popen) are now implemented in terms of `posix_spawn' as well,
which fixes bugs such as redirects: <https://bugs.gnu.org/52835>.

** `open-file' now supports an "e" flag for O_CLOEXEC

Until now, the high-level `open-file' facility did not provide a way to
pass O_CLOEXEC 

GNU Guile 3.0.9 released

[Ludovic Courtès]

We are pleased to announce GNU Guile release 3.0.9, the latest in the
3.0 stable release series, corresponding to 138 commits by 27 people
since 3.0.8.

This release provides many bug fixes as well as new functionality,
including new bindings to POSIX interfaces.  See the ‘NEWS’ excerpt
below for details.

Compared to the previous stable series (2.2.x), Guile 3.0 adds support
for just-in-time native code generation, speeding up all Guile programs.


Guile is an implementation of the Scheme programming language, packaged
for use in a wide variety of environments.  In addition to implementing
the R5RS, R6RS, and R7RS Scheme standards, Guile includes full access to
POSIX system calls, networking support, multiple threads, dynamic
linking, a foreign function call interface, powerful string processing,
and HTTP client and server implementations.

Guile can run interactively, as a script interpreter, and as a Scheme
compiler to VM bytecode.  It is also packaged as a library so that
applications can easily incorporate a complete Scheme interpreter/VM.
An application can use Guile as an extension language, a clean and
powerful configuration language, or as multi-purpose "glue" to connect
primitives provided by the application.  It is easy to call Scheme code
from C code and vice versa.  Applications can add new functions, data
types, control structures, and even syntax to Guile, to create a
domain-specific language tailored to the task at hand.

Check out the web page for more info and resources:

  https://gnu.org/software/guile

Guile 3.0.9 can be installed in parallel with Guile 2.2.x; see
http://www.gnu.org/software/guile/manual/html_node/Parallel-Installations.html.

Here are the compressed sources:
  https://ftp.gnu.org/gnu/guile/guile-3.0.9.tar.gz   (9.3MB)
  https://ftp.gnu.org/gnu/guile/guile-3.0.9.tar.lz   (5.2MB)
  https://ftp.gnu.org/gnu/guile/guile-3.0.9.tar.xz   (5.5MB)

Here are the GPG detached signatures:
  https://ftp.gnu.org/gnu/guile/guile-3.0.9.tar.gz.sig
  https://ftp.gnu.org/gnu/guile/guile-3.0.9.tar.lz.sig
  https://ftp.gnu.org/gnu/guile/guile-3.0.9.tar.xz.sig

Use a mirror for higher download bandwidth:
  https://www.gnu.org/order/ftp.html

Here are the SHA1 and SHA256 checksums:

6ce38ec3fefc19aa08d4662e9b054f7018a72004  guile-3.0.9.tar.gz
GFJQea0poNRtFcdlgbXZHIcCMBv9ghZm0uHRNyYWKBE  guile-3.0.9.tar.gz
bcc02997587cdd03a831ee8d7153cad92629dc3f  guile-3.0.9.tar.lz
vA7go2D7E5GcFOtuJFMxmt8eyZgojJk4KbzxePtIzJo  guile-3.0.9.tar.lz
bf6af1aac320a56233d4d8c0fbeb2c0dca474eab  guile-3.0.9.tar.xz
GiYlrHKyNm6VeS8/51j9Lfd1tARKkKSpeHMm5mwNdQ0  guile-3.0.9.tar.xz

The SHA256 checksum is base64 encoded, instead of the
hexadecimal encoding that most checksum tools default to.

Use a .sig file to verify that the corresponding file (without the
.sig suffix) is intact.  First, be sure to download both the .sig file
and the corresponding tarball.  Then, run a command like this:

  gpg --verify guile-3.0.9.tar.gz.sig

The signature should match the fingerprint of the following key:

  pub   rsa4096 2014-08-11 [SC]
3CE4 6455 8A84 FDC6 9DB4  0CFB 090B 1199 3D9A EBB5
  uid   [ unknown] Ludovic Courtès 
  uid   [ unknown] Ludovic Courtès 
  uid   [ unknown] Ludovic Courtès (Inria) 

If that command fails because you don't have the required public key,
or that public key has expired, try the following commands to retrieve
or refresh it, and then rerun the 'gpg --verify' command.

  gpg --recv-keys 3CE464558A84FDC69DB40CFB090B11993D9AEBB5

As a last resort to find the key, you can try the official GNU
keyring:

  wget -q https://ftp.gnu.org/gnu/gnu-keyring.gpg
  gpg --keyring gnu-keyring.gpg --verify guile-3.0.9.tar.gz.sig


This release was bootstrapped with the following tools:
  Autoconf 2.71
  Automake 1.16.5
  Libtool 2.4.7
  Gnulib v0.1-5703-g356a414e8c
  Makeinfo 7.0.1


Changes in 3.0.9 (since 3.0.8)

* Notable changes

* New interfaces and functionality

** New `spawn' procedure to spawn child processes

The new `spawn' procedure creates a child processes executing the given
program.  It lets you control the environment variables of that process
and redirect its standard input, standard output, and standard error
streams.

Being implemented in terms of `posix_spawn', it is more portable, more
robust, and more efficient than the combination of `primitive-fork' and
`execl'.  See "Processes" in the manual for details, and see the 2019
paper entitled "A fork() in the road" (Andrew Baumann et al.) for
background information.

`system*', as well as the `open-pipe' and `pipeline' procedures of
(ice-9 popen) are now implemented in terms of `posix_spawn' as well,
which fixes bugs such as redirects: <https://bugs.gnu.org/52835>.

** `open-file' now supports an "e" flag for O_CLOEXEC

Until now, the high-level `open-file' facility did not provide a way to
pass O_CLOEXEC 

Re: Add internal definitions to derived forms

[Ludovic Courtès]

Hi,

lloda  skribis:

> From 70ce7174a43ba17f2db0c3a6b7eeeb191a332663 Mon Sep 17 00:00:00 2001
> From: Daniel Llorens 
> Date: Tue, 24 Jan 2023 11:26:44 +0100
> Subject: [PATCH 2/2] Document multiple-value returns in forms taking a
>  let-expression body
>
> * doc/ref/api-binding.texi (Local Bindings): Document multiple-value
>   returns for let.
> * doc/ref/api-control.texi (begin): Document multiple-value returns for
>   begin.
>   (Conditionals): Document multiple-value returns and use 'body' in the
>   syntax description of when, unless, cond, case.
>   (Multiple values): Document multiple-value returns and use 'body' in
>   the syntax description of SRFI-8 receive.

LGTM, thanks!

Ludo’.

Re: GNU Guile 3.0.9rc1 available for testing!

[Ludovic Courtès]

Hi Greg,

Greg Troxel  skribis:

> Ludovic Courtès  writes:
>
>> So something like the patch below?
>>
>> Thanks,
>> Ludo’.
>>
>> diff --git a/libguile/posix.c b/libguile/posix.c
>> index 74c743119..0b1fe2637 100644
>> --- a/libguile/posix.c
>> +++ b/libguile/posix.c
>> @@ -105,8 +105,8 @@
>>  # else
>>  #  define W_EXITCODE(ret, sig)   ((ret) << 8 | (sig))
>>  # endif
>> -#endif
>>  verify (WEXITSTATUS (W_EXITCODE (127, 0)) == 127);
>> +#endif
>>  
>>  
>>  #include 
>
> I see you pushed that to master and I have done a full build from master
> with the pkgsrc JIT workarounds.  It mostly worked and I got this, which
> may be because I purged not-needed packages (but the guile build didn't
> complain at configure or check time, like it did for missing gperf which
> I put back).  Or maybe I just haven't gotten this far with 3.0.9-ish.
>
> CC   test_foreign_object_c-test-foreign-object-c.o
> CCLD test-foreign-object-c
>   ld: ../../libguile/.libs/libguile-3.0.so: warning: warning: tmpnam() 
> possibly used unsafely, use mkstemp() or mkdtemp()
>   ld: /tmp//ccOnwVqC.ltrans0.ltrans.o: in function `finalizer':
>   
> /home/n0/gdt/SOFTWARE/GUILE/guile/BUILD/test-suite/standalone/../../../test-suite/standalone/test-foreign-object-c.c:42:
>  undefined reference to `rpl_free'

I believe this is because these C files in test-suite/standalone end up
including Gnulib headers (like ), which really shouldn’t be
necessary, without linking against libgnu.a.

I believe the patch below fixes that.

[...]

> There are some issues for me to work through eventually, but for 3.0.9 I
> have arrived at being ok with you releasing the current state of git
> master (3 commits past rc1) as not having anything I can claim is a
> regression and not anything I can't work around as before.

Sounds good.

Thanks for your feedback!

Ludo’.

diff --git a/test-suite/standalone/Makefile.am b/test-suite/standalone/Makefile.am
index 547241afa..0b5d05ddb 100644
--- a/test-suite/standalone/Makefile.am
+++ b/test-suite/standalone/Makefile.am
@@ -1,7 +1,7 @@
 ## Process this file with automake to produce Makefile.in.
 ##
 ## Copyright 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010,
-##   2011, 2012, 2013, 2014, 2020, 2021, 2022 Free Software Foundation, Inc.
+##   2011, 2012, 2013, 2014, 2020, 2021, 2022, 2023 Free Software Foundation, Inc.
 ##
 ## This file is part of GUILE.
 ##
@@ -39,8 +39,7 @@ TESTS_ENVIRONMENT =		\
 
 ## Check for headers in $(srcdir) and build dir before $(CPPFLAGS), which
 ## may point us to an old, installed version of guile.
-AM_CPPFLAGS = -I$(top_srcdir) -I$(top_builddir) \
-	  -I$(top_srcdir)/lib -I$(top_builddir)/lib
+AM_CPPFLAGS = -I$(top_srcdir) -I$(top_builddir)
 
 test_cflags =	\
   -I$(top_srcdir)/test-suite/standalone -I.	\
@@ -55,7 +54,7 @@ LIBGUILE_LDADD =			\
 
 
 snarfcppopts =		\
-  -I$(top_srcdir) -I$(top_srcdir)/lib -I$(top_builddir)/lib -I$(top_builddir) \
+  -I$(top_srcdir) -I$(top_builddir)			\
   -I. $(DEFS) $(DEFAULT_INCLUDES) $(CPPFLAGS) $(CFLAGS)
 
 SUFFIXES = .x

Re: [PATCHv3] Extensions for SRFI-171 (Transducers)

[Ludovic Courtès]

Hi Colin,

Colin Woodbury  skribis:

> Sorry if I wasn't clear - I have alright assigned copyright to the FSF
> (about 1.5 years ago). Things should be good to go, unless I've
> misunderstood.

I checked the records on fencepost.gnu.org and AFAICS, you assigned
copyright for Emacs only (copyright assignments are for a single GNU
package), so you will need to do the paperwork for Guile this time.

Thanks in advance.  :-)

Ludo’.

Re: Add internal definitions to derived forms

[Ludovic Courtès]

Hi!

lloda  skribis:

>>> @lisp
>>> -(@var{test} @var{body} @dots{})
>>> +(@var{test} @var{body})
>> 
>> I think removing dots is incorrect here because it suggests, according
>> to the typographic conventions used in the document, that there can only
>> be a single expression.

[...]

> This was actually the main thing I wanted to fix in this patch. Linus' patch 
> had ‘body ...’ but that clearly means ‘zero or more bodies’, which doesn't 
> work because there's exactly one ‘body’. I.e. ‘body’ isn't an expression that 
> is tagged ‘body’, it's, well, a ‘body’.

Yeah, ‘body’ is a bit confusing here; in the example above, I’d have
written:

  (@var{test} @var{exp} @dots{})

because that’s what the “body” is: one or more expressions.

> The Scheme reports use one ‘’ and no dots in all these definitions. See 
> also the definition of let in the linked section ‘Local Bindings’, which 
> again uses ‘body’ and no dots. I hoped that section would count as definition 
> of ‘body’, and the section on ‘Internal Definitions’ explains precisely what 
> can go into ‘body’, so I linked to that as well. I see that isn't clear 
> enough. Maybe ‘body’ should be explicitly defined in one of these sections?

Damn it, I hadn’t realized this was a widespread convention, but yeah,
R5RS and parts of the Guile manual follow this convention.  So hmm, the
change you propose makes a lot of sense to me now.

So yeah overall I guess we should always write one of:

  (something @var{body})

or:

  (something @var{exp} @dots{})

Using @var{body} like you do in this patch is consistent with other
parts of the manual, so it LGTM.

Thanks,
Ludo’.

Re: [PATCHv3] Extensions for SRFI-171 (Transducers)

[Ludovic Courtès]

Hi Colin,

Colin Woodbury  skribis:

> Hi Ludovic, thanks for getting back to me! I've updated the patches as
> you've suggested. I think I've gotten the commit format correct this
> time.

Yup, it’s looks perfect!

> Also, I'll opt to assign copyright to the FSF, as I've already done so
> for Emacs (and signed the papers, etc.).

Alright.  In that case, could you send the form at

to ass...@gnu.org (as noted at the top).  If you don’t get a reply
within two weeks, please let Andy and/or myself know and we’ll ping
them.

I know this is frustrating but I’ll wait for their notification to push
the changes (on the bright side, it’s a one-time delay).

> Let there be transduction! Cheers,

Definitely.  :-)

Thanks!

Ludo’.

Re: Add internal definitions to derived forms

[Ludovic Courtès]

Hi Daniel,

Chiming in late in the discussion…

lloda  skribis:

> From 7aea299373f7370f31c9701035260ad412763724 Mon Sep 17 00:00:00 2001
> From: Daniel Llorens 
> Date: Thu, 19 Jan 2023 16:23:29 +0100
> Subject: [PATCH 2/2] Fix documentation for forms taking lambda-like bodies
>
> * doc/ref/api-control.texi (Conditionals): Use 'body' in the syntax
>   description of when, unless, cond, case.
> * doc/ref/api-binding.texi (Local Bindings): Normalize description of
>   body return values.
>   (Multiple values): Normalize use of 'body' and description of body
>   return values.

What about:
s/Fix documentation…/doc: Document multiple-value returns in let, cond, etc./
?

(That would clarify what’s being fixed.)

> +++ b/doc/ref/api-binding.texi
> @@ -128,6 +128,8 @@ expressions has a few properties which are well worth 
> knowing.
>  
>  The most basic local binding construct is @code{let}.
>  
> +@cindex body

That’s not a great index entry because there’s no context.  Maybe:

  @cindex body, of a @code{let} expression

?

> +with the special forms @code{when} and @code{unless}.  As an added
> +bonus, these forms take a @ref{Local Bindings,lambda-like body}, which can
> +contain @ref{Internal Definitions,internal definitions} and multiple 
> statements
> +to evaluate.

“Lambda-like body” is not defined; I guess it’s “lambda-like” in the
wrt. to local ‘define’, but it’s not “lambda-like” for the more crucial
aspect of defining a procedure, so I’d avoid that phrase.  WDYT?

Also, @ref in the middle of sentences may render poorly in Info (info
"(texinfo) @ref").  I’d suggest “(@pxref{Whatever})” at the end of the
sentence or proposition.

>  Each @code{cond}-clause must look like this:
>  
>  @lisp
> -(@var{test} @var{body} @dots{})
> +(@var{test} @var{body})

I think removing dots is incorrect here because it suggests, according
to the typographic conventions used in the document, that there can only
be a single expression.

>  @var{key} may be any expression, and the @var{clause}s must have the form
>  
>  @lisp
> -((@var{datum1} @dots{}) @var{body} @dots{})
> +((@var{datum1} @dots{}) @var{body})

Ditto.

>  and the last @var{clause} may have the form
>  
>  @lisp
> -(else @var{expr1} @var{body} @dots{})
> +(else @var{body})

Ditto.

> -@deffn {library syntax} receive formals expr body @dots{}
> +@deffn {library syntax} receive formals expr body

Likewise.

Otherwise LGTM; it’s certainly an improvement to have multiple-value
returns properly documented!

Thanks,
Ludo’.

Re: bug#60971: build failure of v3.0.9rc1 on mac os 12.6

[Ludovic Courtès]

lloda  skribis:

> lgtm, no other issues on mac os.

Awesome, pushed as 9b20ca275dba758a194073936cde7c95311bd51e.

Ludo’.

Re: GNU Guile 3.0.9rc1 available for testing!

[Ludovic Courtès]

Greg Troxel  skribis:

> My second approach is using the rc tarball in pkgsrc.  I had to patch
> out the verify call.  pkgsrc already works around PaX issues mostly, via
> paxctl on things that do jit, after build and before use, and by
>
>   --- libguile/loader.c.orig  2018-01-08 16:21:04.790894906 +
>   +++ libguile/loader.c
>   @@ -484,7 +484,7 @@ map_file_contents (int fd, size_t len, i
>  char *data;
>
>#ifdef HAVE_SYS_MMAN_H
>   -  data = mmap (NULL, len, PROT_READ, MAP_PRIVATE, fd, 0);
>   +  data = mmap (NULL, len, PROT_READ | PROT_WRITE, MAP_PRIVATE, fd, 0);
>  if (data == MAP_FAILED)
>SCM_SYSERROR;
>  *is_read_only = 1;
>
> because (something like, am fuzzy) mprotect (at least ours) can only
> reduce not add permissions.  I still don't understand why this isn't a
> more widespread issue.

My understanding is that the expectation of being able to later mprotect
that mapping to make it writable is a valid one per POSIX.  Quoth
:

  If PROT_WRITE is specified, the application shall ensure that it has
  opened the mapped objects in the specified address range with write
  permission, unless MAP_PRIVATE was specified in the original mapping,
  regardless of whether the file descriptors used to map the objects
  have since been closed.

Here the original file descriptors are O_RDONLY, but the mapping is
MAP_PRIVATE.

I’m not sure how to best address that.

Thoughts?

Ludo’.

Re: GNU Guile 3.0.9rc1 available for testing!

[Ludovic Courtès]

Hi,

Greg Troxel  skribis:

> and, in guile-readline/Makefile.am I see
>
>   if HAVE_READLINE
>
>   SOURCES += ice-9/readline.scm
>
> and I don't see an else with EXTRA_DIST.  I think it's a bug for make
> dist contents to vary with whether or not libs are available on the
> local system.

Oops, good catch.  I’ve fixed the machinery to require ‘HAVE_READLINE’
when running ‘make dist’ (that sounded more reliable than duplicating
the list of source files in ‘guile-readline/Makefile.am’).

Thanks,
Ludo’.

Re: GNU Guile 3.0.9rc1 available for testing!

[Ludovic Courtès]

Hola!

Aleix Conchillo Flaqué  skribis:

> Not because I worked on it :-), but missing JIT support on Apple Silicon
> chips in the NEWS, which makes Guile *fast* on that hardware.
>
> https://debbugs.gnu.org/cgi/bugreport.cgi?bug=44505

Oops, my bad; added to NEWS!

You confirm that building from the tarball still works, right?

Thanks,
Ludo’.

Re: GNU Guile 3.0.9rc1 available for testing!

[Ludovic Courtès]

Hi,

Greg Troxel  skribis:

> lloda  writes:
>
>> This looks like https://debbugs.gnu.org/60971 
>>  on mac os.
>
> Yes, it does.
>
> My quick reaction is that if the POSIX-required macros operation on
> system types that might be struct, then faking up ints for testing is
> unsound.
>
> Maybe only do verify if guile has to define macros, and don't try to
> test the OS?

So something like the patch below?

Thanks,
Ludo’.

diff --git a/libguile/posix.c b/libguile/posix.c
index 74c743119..0b1fe2637 100644
--- a/libguile/posix.c
+++ b/libguile/posix.c
@@ -105,8 +105,8 @@
 # else
 #  define W_EXITCODE(ret, sig)   ((ret) << 8 | (sig))
 # endif
-#endif
 verify (WEXITSTATUS (W_EXITCODE (127, 0)) == 127);
+#endif
 
 
 #include 

Re: GNU Guile 3.0.9rc1 available for testing!

[Ludovic Courtès]

Hi Jan,

Jan Nieuwenhuizen  skribis:

> Yeah, you need at least
>
> 76950b428 Support for x86_64-w64-mingw32.
>
> or something similar, or another approach for addressing the
> SIZEOF_LONG==4 issue on MinGW.

Oh sorry, I had completely overlooked that patch (and branch).

I’m hesitant about applying it at this late stage, also because I
haven’t thought about the possible choices and implications.  Maybe this
will have to wait until the next round (bah!)?

> And compile with --disable-jit, AFAIK nobody got that to work just
> yet.

OK.

Thanks for the heads-up!

Ludo’.

GNU Guile 3.0.9rc1 available for testing!

[Ludovic Courtès]

Hello Guilers!

(Cc’ing packagers I know; feel free to ping other packagers!)

I pushed the ‘v3.0.9rc1’ tag on the ‘main’ branch and uploaded the
tarballs produced by “make distcheck” from that tag:

  https://alpha.gnu.org/gnu/guile/guile-3.0.9rc1.tar.gz
  SHA256: f34dac704b388e16541dfca5305ee48da1e5bf8e43a899f1e7927cc7c888dc65

  https://alpha.gnu.org/gnu/guile/guile-3.0.9rc1.tar.lz
  SHA256: 25507afe7733ad72f14615e7b9d5f4c5cb5abcbaa880ef15edd083d7f333d6ee

  https://alpha.gnu.org/gnu/guile/guile-3.0.9rc1.tar.xz
  SHA256: 9590a56a31de60c11b673fbbc9d0166364e270e5f7c6890d3282272fcfb2e871

These files come with a detached OpenPGP signature (a ‘.sig’ file) made
with my key, 3CE464558A84FDC69DB40CFB090B11993D9AEBB5.

Please report any issues you may find.  I’m particularly interested in
portability regressions (NetBSD and other BSDs, macOS, MinGW, as well as
architectures other than x86_64).

The SONAME has been bumped to its value as it will appear in 3.0.9.

If nothing bad comes up, I propose publishing 3.0.9 in a week or so.

Thanks in advance!

Ludo’.


signature.asc
Description: PGP signature

Re: Reducing memory usage of the linker and assembler

[Ludovic Courtès]

Hello,

Ludovic Courtès  skribis:

> The branch replaces the ‘bv’ field of  with ‘size’ and
> ‘writer’, the latter being a procedure that takes a bytevector and
> writes to it.
>
> At this point, ‘link-elf’ still allocates one bytevector for each linker
> object.  Eventually we could rewrite those “writer” procedures to use
> binary I/O primitives instead of expecting a bytevector to write to, and
> that way we wouldn’t need those temporary bytevectors.  It’s a bit
> tedious to do though.

I’ve now merged this branch into ‘main’:

  3cd64feb2 * linker: Do not store entire ELF in memory when writing to a file.
  4ab71e1f0 * linker: Linker object writer takes a single argument.
  041f11b35 * linker, assembler: Avoid intermediate bytevectors.
  d0d974360 * linker: Separate effectful part of 'add-elf-objects'.
  d439a3f67 * assembler: Separate effectful part of 'link-docstrs'.
  13e2d5b66 * assembler: Separate effectful part of 'link-frame-maps'.
  dc0c4ccb1 * assembler: Separate effectful part of 'link-procprops'.
  c7f1522c6 * assembler: Separate effectful part of 'link-dynamic-section'.
  fc5eae5d0 * assembler: Separate effectful part of 'link-symtab'.
  15c4c4ceb * assembler: Separate 'process-relocs' from 'patch-relocs!'.

As discussed on IRC, I’d like to tag and upload 3.0.9rc1 hopefully
tomorrow to get some testing (in particular portability testing) and
release roughly a week later.

Thoughts?

Ludo’.

Re: [PATCH] Document R7RS bytevector functions

[Ludovic Courtès]

Hi Daniel,

lloda  skribis:

>> We should keep the manual in sync with docstrings in bytevectors.c.
>> 
>> Thus, my suggestion would be to not insert comments and footnotes about
>> R7RS in the existing sections, but instead to do that in the new section.
>
> Following this, I've removed the footnotes & references to R7RS in the main 
> section.
>
> Patch aside:
>
> I still think the links are useful, so I'd like to find a way to put them 
> back. I think someone who finds their way into the doc for r6rs 
> bytevector-copy should be told that there are in fact different versions of 
> this procedure. Just in general, I think there should be many more links in 
> the doc.

Yes, I agree with that, I’m just not sure about the initial proposal.

> From 547bd47887d7096e5568c31edb0b1661b1ac23ec Mon Sep 17 00:00:00 2001
> From: Daniel Llorens 
> Date: Sun, 15 Jan 2023 22:41:48 +0100
> Subject: [PATCH] Document R7RS functions related to bytevectors
>
> * doc/ref/api-data.texi (Bytevectors): Document R7RS bytevector,
>   bytevector-copy, bytevector-copy!, bytevector-append.
>   Fix typo in (r6:bytevector-copy), index need not be positive.
>   Fix typos in bytevector-length, bytevector=, bytevector-fill!.
> * doc/ref/api-io.texi (Binary I/O): Document R7RS
>   open-output-bytevector, write-u8, read-u8, peek-u8,
>   get-output-bytevector, open-input-bytevector, read-bytevector!,
>   read-bytevector, write-bytevector.

[...]

> -A @dfn{bytevector} is a raw bit string.  The @code{(rnrs bytevectors)}
> +A @dfn{bytevector} is a raw byte string.  The @code{(rnrs bytevectors)}
>  module provides the programming interface specified by the
>  @uref{http://www.r6rs.org/, Revised^6 Report on the Algorithmic Language
>  Scheme (R6RS)}.  It contains procedures to manipulate bytevectors and
  ^
Actually here we could insert something like:

  The R7RS also supports this interface with some additions and minor
  differences (@pxref{Bytevector Procedures in R7RS}).

> @@ -6799,6 +6804,86 @@ Return the length in bytes of bytevector @var{bv}.
>  Return a pointer to the contents of bytevector @var{bv}.
>  @end deftypefn
>  
> +@subsubheading Bytevector Procedures in R7RS

How about moving it at the bottom, after “Bytevectors as Uniform
Vectors”?

Also make sure to update the menu at the beginning of the “Bytevectors”
node (you can use ‘M-x texinfo-make-menu’ if you use Emacs).

OK for me with changes along these lines.

Thank you!

Ludo’.

Re: [PATCH] Add 'bytevector-slice'.

[Ludovic Courtès]

Hello!

I pushed the patch as commit e441c34f1666921f6b15597c1aa3a50596a129d7
with the following changes taking into account your comments:

  • adjusted copyright years;

  • removed comment about ‘SCM_F_BYTEVECTOR_CONTIGUOUS’ since it’s quite
clear from the discussion in this thread that this flag is
vestigial;

  • added an overflow check for ‘c_offset + c_size’ and a corresponding
test (really glad you reported this one!);

  • added another missing test that you mentioned.

I did not update the license header as you suggested, but I think we
should run a script on all the repo to homogenize those but it’s a bit
messy right now.

Thanks,
Ludo’.

Re: [PATCHv2] Extensions for SRFI-171 (Transducers)

[Ludovic Courtès]

Hi Colin and all!

These patches look like a nice addition.

First, you have the option of assigning your copyright for this
contribution (and future Guile contributions if you wish) to the FSF, or
you can choose not to:

  https://lists.gnu.org/archive/html/guile-devel/2022-10/msg8.html

Please take a look at the message above and let us know what you’d like
to do.  If you choose not to assign copyright, you’ll have to add
copyright lines for you (or whatever entity holds copyright on your
work) in the modified files.

Overall the changes LGTM; I have minor comments and suggestions:

Colin Woodbury  skribis:

> From 96856b184a507886db2c5c20323983ae125a6bdb Mon Sep 17 00:00:00 2001
> From: Colin Woodbury 
> Date: Mon, 19 Dec 2022 09:39:37 +0900
> Subject: [PATCH 1/4] srfi-171: add twindow and various reducers
>
> This adds a number of reduction primitives often seen in other languages
> to Guile's SRFI-171 extensions.
>
> Most critical may be `rfold`, which could be called the fundamental
> reducer, as it's likely that all other reducers could be defined in
> terms of it (though not all are). While `tfold` already exists in
> Guile's SRFI-171 extension as a transducer, folding is in essence a
> reduction. Also without a primative like `rlast` (also introduced here),
> the results of `tfold` are difficult to consume. This is avoided by
> providing `rfold` directly as a generalised means to collapse an entire
> transduction down into a single value (i.e. the whole point of
> reducers). `rfold` is also useful for the creation of ad-hoc reducers,
> as any 2-arg function can be passed to it to fold the stream of values.
>
> `rfirst`, `rlast`, and `rfind` are common idioms and so have been added.
>
> The equivalent of `rmax` and `rmin` are easy to write manually via
> `rfold`, but they have been provided here as a convenience in the same
> spirit as `rcons`.
>
> `rfor-each` also cannot be forgotten as a classic adaptation of its
> SRFI-1 cousin.
>
> Also added is `twindow`, handy for analysing groups of adjacent items.

[...]

> From 58e7ca2718a860ca2fb5692684d6d128a7c1ae75 Mon Sep 17 00:00:00 2001
> From: Colin Woodbury 
> Date: Tue, 20 Dec 2022 09:41:51 +0900
> Subject: [PATCH 2/4] doc: add new SRFI-171 reducers to the manual
>
> ---
>  doc/ref/srfi-modules.texi | 96 +--

[...]

> From 7b7538c61799fa0fa0e2fa18efba98b7de7da1ca Mon Sep 17 00:00:00 2001
> From: Colin Woodbury 
> Date: Wed, 21 Dec 2022 09:30:50 +0900
> Subject: [PATCH 3/4] srfi-171: add unit tests for new functions
>
> These tests mainly match the examples shown in the docs.
> ---
>  test-suite/tests/srfi-171.test | 66 ++

We’d squash these three commits together to provide a single
self-contained commit with code and the corresponding tests and doc.

The convention in Guile is for commit logs to follow the ChangeLog style
(see ‘git log’ for examples).  If you’re not sure how to do that, I can
do it on your behalf as a welcome present.  ;-)

> From 87a74d106f11680c4924befb664d7ef685c16b06 Mon Sep 17 00:00:00 2001
> From: Colin Woodbury 
> Date: Thu, 22 Dec 2022 20:32:33 +0900
> Subject: [PATCH 4/4] doc: added a guide for writing custom reducers
>
> The guide previously explained what reducers were, but not the specifics
> of how to write them yourself. This commits rectifies this.

Nice!

> +++ b/doc/ref/srfi-modules.texi
> @@ -5966,6 +5966,82 @@ Yield the maximum (or minimum) value of the 
> transduction, or the
>  @var{seed} value if there is none.
>  @end deffn
>  
> +@subheading Writing your own reducers
> +If you want to reduce some values via an ordinary function that you

Please capitalize section titles and leave a blank line below it (same
for the section that follows).

> +However, if you want more customized behaviour (such as early
> +termination and/or arbitrary manipulation of the input values) then
> +you're free to write a reducer manually. To do so, we need to write a

Normally we leave two spaces after end-of-sentence periods, to ease
navigation in Emacs and please Texinfo (info "(texinfo) Ending a
Sentence").

Could you send updated patches?

Thanks for your work!

Ludo’.

Re: [PATCH] Document R7RS bytevector functions

[Ludovic Courtès]

Hi Daniel,

lloda  skribis:

> Right now the manual just mentions (scheme base), but not the contents. This 
> patch at least makes sure that at least the bytevector-related R7RS functions 
> appear in the index.
>
> The patch documents a first group of purely bytevector functions and a second 
> group of binary I/O that are not elsewhere in Guile/R6RS or that exist but 
> have different definitions.

Nice.

> From 2b751615071aca023dac59db866fb09894fa2b57 Mon Sep 17 00:00:00 2001
> From: Daniel Llorens 
> Date: Fri, 13 Jan 2023 16:26:17 +0100
> Subject: [PATCH] Document R7RS functions related to bytevectors
>
> * doc/ref/api-data.texi: Document: bytevector bytevector-copy
>   bytevector-copy! bytevector-append
>   (r6:bytevector-copy): Index need not be positive (can be 0).
> * doc/ref/api-io.texi: Document: open-output-bytevector write-u8 read-u8 
> peek-u8
>   get-output-bytevector open-input-bytevector read-bytevector! read-bytevector
>   write-bytevector

Please specify the node names (see ‘git log’ for examples).

>  doc/ref/api-data.texi| 102 ++---
>  doc/ref/api-io.texi  | 121 +++
>  lib/iconv_open-aix.h |  68 +++---
>  lib/iconv_open-hpux.h|  92 ++---
>  lib/iconv_open-irix.h|  42 +++---
>  lib/iconv_open-osf.h |  80 +-
>  lib/iconv_open-solaris.h |  30 +-

I guess the lib/ changes are unintended.  :-)

> +++ b/doc/ref/api-data.texi
> @@ -6188,9 +6188,9 @@ thus created is determined implicitly by the number of 
> arguments given.
>  Return a newly allocated vector composed of the
>  given arguments.  Analogous to @code{list}.
>  
> -@lisp
> +@example
>  (vector 'a 'b 'c) @result{} #(a b c)
> -@end lisp
> +@end example

Please keep @lisp as this gives information that can then be exploited
for syntax highlighting (‘texinfo --html’ doesn’t take advantage of it
so far, but in Guix we have machinery to do that.)

> +@ref{R7RS Support,R7RS} defines another set of bytevector functions in
> +the module @code{(scheme base)}. These functions are also described in this
> +section.

Please leave two spaces after and end-of-sentence period, to ease
navigation in Emacs and to please Texinfo (info "(texinfo) Ending a
Sentence").

Maybe these two sentences should go to the intro of the new subsection
(see below).

> @@ -6726,9 +6730,10 @@ procedures and C functions.
>  @deffnx {C Function} scm_c_make_bytevector (size_t len)
>  Return a new bytevector of @var{len} bytes.  Optionally, if @var{fill}
>  is given, fill it with @var{fill}; @var{fill} must be in the range
> -[-128,255].
> +[-128,255].@footnote{This function is defined both by R6RS in @code{(rnrs 
> bytevectors)} and by @ref{R7RS Standard Libraries,R7RS} in @code{(scheme 
> base)}.}.
>  @end deffn
>  
> +@anchor{x-bytevector?}
>  @deffn {Scheme Procedure} bytevector? obj
>  @deffnx {C Function} scm_bytevector_p (obj)
>  Return true if @var{obj} is a bytevector.
> @@ -6757,26 +6762,32 @@ length and contents.
>  @deffnx {C Function} scm_bytevector_fill_x (bv, fill)
>  Fill positions [@var{start} ... @var{end}) of bytevector @var{bv} with
>  byte @var{fill}. @var{start} defaults to 0 and @var{end} defaults to the
> -length of @var{bv}.@footnote{R6RS defines @code{(bytevector-fill! bv
> +length of @var{bv}.@footnote{R6RS only defines @code{(bytevector-fill! bv
>  fill)}. Arguments @var{start} and @var{end} are a Guile extension
>  (cf. @ref{x-vector-fill!,@code{vector-fill!}},
>  @ref{x-string-fill!,@code{string-fill!}}).}
>  @end deffn
>  
> +@anchor{x-r6:bytevector-copy!}
>  @deffn {Scheme Procedure} bytevector-copy! source source-start target 
> target-start len
>  @deffnx {C Function} scm_bytevector_copy_x (source, source_start, target, 
> target_start, len)
>  Copy @var{len} bytes from @var{source} into @var{target}, starting
> -reading from @var{source-start} (a positive index within @var{source})
> +reading from @var{source-start} (an index index within @var{source})
>  and writing at @var{target-start}.
>  
>  It is permitted for the @var{source} and @var{target} regions to
>  overlap. In that case, copying takes place as if the source is first
>  copied into a temporary bytevector and then into the destination.
> +
> +See also @ref{x-r7:bytevector-copy!,the R7RS version}.
>  @end deffn
>  
> +@anchor{x-r6:bytevector-copy}
>  @deffn {Scheme Procedure} bytevector-copy bv
>  @deffnx {C Function} scm_bytevector_copy (bv)
>  Return a newly allocated copy of @var{bv}.
> +
> +See also @ref{x-r7:bytevector-copy,the R7RS version}.
>  @end deffn

We should keep the manual in sync with docstrings in bytevectors.c.

Thus, my suggestion would be to not insert comments and footnotes about
R7RS in the existing sections, but instead to do that in the new section.

> +@subsubheading Bytevector functions in R7RS

Please capitalize and use “procedure”:

  Bytevector Procedures in R7RS.

Here maybe you c

Re: [PATCH] Add 'bytevector-slice'.

[Ludovic Courtès]

Maxime Devos  skribis:

> On 13-01-2023 12:32, Ludovic Courtès wrote:
>>> IIUC, if you use bytevector-slice iteratively, say:
>>>
>>> (let loop ((bv some-initial-value)
>>> (n large-number))
>>>(if (> n 0)
>>>(loop (bytevector-slice bv 0 (bytevector-length bv))
>>>  (- n 1))
>>>bv))
>>>
>>> you will end up with a bytevector containing a reference to a
>>> bytevector containing a reference to ... containing a reference to the
>>> original reference, in a chain of length ≃ large-number.
>> The ‘parent’ word is here just so the backing memory isn’t reclaimed
>> while the slice is still alive.
>> Whether there’s a long chain of ‘parent’ links or not makes no
>> difference to GC performance nor to memory usage.
>
> This is false, the opposite holds: memory usage is at least linear in
> the length of the chain, because the objects in the chain are pairwise
> non-eq?: for a chain (O_1, O_2, ..., O_n) that is alive, each object
> O_i in the chain is kept alive (because that's what the 'parent' link
> is for).  Because bytevector-slice does a fresh allocation, there then
> are n different objects.  Hence, memory usage is at least
> 'SCM_BYTEVECTOR_HEADER_SIZE * sizeof(scm_t_bits) * n'.

Ah yes, that’s right, I misunderstood the comment.

In the example above, where we’re only dealing with slices, we could
“skip” the parent (i.e., have each slice’s parent point to the “root” of
the hierarchy), but I don’t think we can assume this to be the case
generally.

Ludo’.

Re: [PATCH] Add 'bytevector-slice'.

[Ludovic Courtès]

Hi,

Maxime Devos  skribis:

> The only thing missing for me, is a procedure
> 'bytevector-slice/read-only' and 'bytevector-slice/write-only', then I
> could throw the module implementing the wrapping in Scheme-GNUnet and
> the overhead incurred by wrapping away.

I did consider the read-only part, but it’s not really implementable
currently as SCM_F_BYTEVECTOR_IMMUTABLE flag is even ignored by
instructions, and fixing it is far beyond the scope of this patch:

  https://issues.guix.gnu.org/60779

> On 11-01-2023 16:00, Ludovic Courtès wrote:

[...]

>> +(use-modules (rnrs bytevectors)
>> + (rnrs bytevectors gnu))
>
> I thought that R6RS reserved (rnrs ...) to the RnRS process, so I
> would think that naming it (rnrs bytevectors gnu) would be
> standards-incompliant, though I cannot find in the specification, so
> maybe it isn't actually reserved.
>
> (SRFI looks a bit looser to me, so I find the (srfi ... gnu)
> acceptable, but (rnrs ... gnu) looks weird to me, I would propose
> (ice-9 bytevector-extensions) or such.).

I’ll stick to gnu.scm because that’s the convention we’ve used for
extensions so far, and “gnu” is arguably “reserved”.

>> +Copyright (C) 1996-1997, 2000-2005, 2009-2023 Free Software Foundation,
>
>
> Where does this year 2022 come from?  Does a previous version of this
> patch predate the new year?

I started working on it in December and my ‘write-file-hooks’ updated it.

>> +  c_offset = scm_to_size_t (offset);
>> +
>> +  if (SCM_UNBNDP (size))
>> +{
>> +  if (c_offset < SCM_BYTEVECTOR_LENGTH (bv))
>> +c_size = SCM_BYTEVECTOR_LENGTH (bv) - c_offset;
>> +  else
>> +c_size = 0; > +}
>> +  else
>> +c_size = scm_to_size_t (size);
>> +
>> +  if (c_offset + c_size > SCM_BYTEVECTOR_LENGTH (bv))
>> +scm_out_of_range (FUNC_NAME, offset);
>
>
> If offset=SIZE_MAX-1 and size=1, this will overflow to 0 and hence not
> trigger the error reporting.  This bounds check needs to be rewritten,
> with corresponding additional tests.

OK.

> IIUC, if you use bytevector-slice iteratively, say:
>
> (let loop ((bv some-initial-value)
>(n large-number))
>   (if (> n 0)
>   (loop (bytevector-slice bv 0 (bytevector-length bv))
> (- n 1))
>   bv))
>
> you will end up with a bytevector containing a reference to a
> bytevector containing a reference to ... containing a reference to the
> original reference, in a chain of length ≃ large-number.

The ‘parent’ word is here just so the backing memory isn’t reclaimed
while the slice is still alive.

Whether there’s a long chain of ‘parent’ links or not makes no
difference to GC performance nor to memory usage.

> Do you know what this 'parent' field even is for?  Going by some
> comments in 'libguile/bytevectors.c', it is for GC reasons, but going
> by the existence of the 'bytevector->pointer' + 'pointer->bytevector'
> trick which destroys 'parent' information, it seems unnecessary to me.

Like I wrote, it was introduced to keep backing memory alive, generally.
With ‘pointer->bytevector’, we want to make sure the original pointer
remains live as long as the bytevector is live (pointer objects can have
a finalizer, and that finalizer must not run while the bytevector is
live).

> Nowadays a https://.../ instead of a mail address, is more
> conventional and useful, I'd think.  Its even used in old files,
> e.g. libguile/r6rs-ports.c.

Noted.

> A test is missing for the case where the size is unaligned instead of
> the offset.

Noted.

I’ll come up with a second version taking this into account.

Thanks!

Ludo’.

Re: [EXT] [PATCH] Add 'bytevector-slice'.

[Ludovic Courtès]

lloda  skribis:

>> On 11 Jan 2023, at 18:37, Thompson, David  wrote:
>> 
>> On Wed, Jan 11, 2023 at 12:34 PM Ludovic Courtès  wrote:
>>> 
>>> What could be convenient though is ‘bytevector-copy’ (no bang), which
>>> would combine ‘make-bytevector’ + ‘bytevector-copy!’.
>> 
>> 'bytevector-copy' already exists, or do you mean some different
>> implementation of it?
>> 
>> - Dave
>
> The current bytevector-copy takes a single argument.

Right, what I had in mind is one that would take an offset and size; I
hadn’t realized the name was already taken.

Thanks,
Ludo’.

Re: [PATCH] Avoid 'frame-local-ref' errors when printing backtrace.

[Ludovic Courtès]

Hi,

Andrew Whatson  skribis:

> commit 164bdce6acf53796cb96ef1930a89c6caf84bc39
> Author: Andrew Whatson 
> Date:   Wed Jan 11 14:04:32 2023 +1000
>
> Test for 'frame-local-ref' errors when printing backtrace.
> 
> This test reproduces the error from , and
> passes with the workaround which was merged in commit
> c7fa78fc751eb336bcfafbb5ac59c460ee2c5d7a.
> 
> * test-suite/tests/eval.test ("avoid frame-local-ref out of range"): New
> test.

Applied, thanks!

Ludo’.

PS: Please use ‘git format-patch’ so the patch can be directly consumed
by ‘git am’.

The mysterious ‘SCM_F_BYTEVECTOR_CONTIGUOUS’

[Ludovic Courtès]

Ludovic Courtès  skribis:

> This is an updated version of the ‘bytevector-slice’ primitive I used in
> the linker/assembler patch series¹ that I think is ready to go.

While working on this, I noticed I might have to pay attention to
‘SCM_F_BYTEVECTOR_CONTIGUOUS’, as noted in the patch.

But it turns out that flag isn’t really used.  I found two places that
should add it and do not:

diff --git a/libguile/bytevectors.c b/libguile/bytevectors.c
index 6b920c88a..fd7fdad0b 100644
--- a/libguile/bytevectors.c
+++ b/libguile/bytevectors.c
@@ -274,7 +274,8 @@ make_bytevector_from_buffer (size_t len, void *contents,
 
   c_len = len * (scm_i_array_element_type_sizes[element_type] / 8);
 
-  SCM_SET_BYTEVECTOR_FLAGS (ret, element_type);
+  SCM_SET_BYTEVECTOR_FLAGS (ret,
+element_type | SCM_F_BYTEVECTOR_CONTIGUOUS);
   SCM_BYTEVECTOR_SET_LENGTH (ret, c_len);
   SCM_BYTEVECTOR_SET_CONTENTS (ret, contents);
   SCM_BYTEVECTOR_SET_PARENT (ret, SCM_BOOL_F);
diff --git a/module/system/repl/debug.scm b/module/system/repl/debug.scm
diff --git a/module/system/vm/assembler.scm b/module/system/vm/assembler.scm
index 165976363..61e0460ff 100644
--- a/module/system/vm/assembler.scm
+++ b/module/system/vm/assembler.scm
@@ -1857,8 +1857,9 @@ should be .data or .rodata), and return the resulting linker object.
   (define tc7-program #x45)
 
   (define tc7-bytevector #x4d)
-  ;; This flag is intended to be left-shifted by 7 bits.
+  ;; These flags are intended to be left-shifted by 7 bits.
   (define bytevector-immutable-flag #x200)
+  (define bytevector-contiguous-flag #x100)
 
   (define tc7-array #x5d)
 
@@ -2026,6 +2027,7 @@ should be .data or .rodata), and return the resulting linker object.
;; Bytevector immutable flag also shifted
;; left.
(ash (logior bytevector-immutable-flag
+bytevector-contiguous-flag
 (array-type-code obj))
 7)
   (case word-size

There are probably more.

Fundamentally, I’m not sure what this flag is supposed to mean.  AFAICS,
there’s no way to create a non-contiguous bytevector (or SRFI-4 vector).

This flag was added in 7ed54fd36d2e381aa46ef8a7d2fc13a6776b573a.  My
guess is that it was part of plan that wasn’t carried out in the end.

Andy, thoughts?  :-)

Thanks,
Ludo’.

Re: [PATCH] Add 'bytevector-slice'.

[Ludovic Courtès]

Hi,

Jean Abou Samra  skribis:

> What do you think about making it more similar to substrings?
> There the 'substring' procedure makes a copy-on-write substring,
> and you have substring/shared if you really want shared mutation.
> Would something like this be meaningful / feasible / useful
> for bytevectors?

I’m not convinced there’s a need for copy-on-write bytevectors, and it
would be hard to implement, and to implement
efficiently—bytevector-mutating instructions would have to check whether
they’re accessing a CoW bytevector and DTRT.

What could be convenient though is ‘bytevector-copy’ (no bang), which
would combine ‘make-bytevector’ + ‘bytevector-copy!’.

Ludo’.

[PATCH] Add 'bytevector-slice'.

[Ludovic Courtès]

-git a/test-suite/tests/bytevectors.test 
b/test-suite/tests/bytevectors.test
index 732aadb3e..dc4b32370 100644
--- a/test-suite/tests/bytevectors.test
+++ b/test-suite/tests/bytevectors.test
@@ -1,6 +1,6 @@
  bytevectors.test --- R6RS bytevectors. -*- mode: scheme; coding: utf-8; 
-*-
 
- Copyright (C) 2009-2015, 2018, 2021 Free Software Foundation, Inc.
+ Copyright (C) 2009-2015, 2018, 2021, 2023 Free Software Foundation, Inc.
 
  Ludovic Courtès
 
@@ -22,6 +22,7 @@
   #:use-module (test-suite lib)
   #:use-module (system base compile)
   #:use-module (rnrs bytevectors)
+  #:use-module (rnrs bytevectors gnu)
   #:use-module (srfi srfi-1)
   #:use-module (srfi srfi-4))
 
@@ -666,6 +667,56 @@
 exception:out-of-range
 (with-input-from-string "#vu8(0 256)" read)))
 
+
+(with-test-prefix "bytevector-slice"
+
+  (pass-if-exception "wrong size"
+  exception:out-of-range
+(let ((b #vu8(1 2 3)))
+  (bytevector-slice b 1 3)))
+
+  (pass-if-equal "slices"
+  (list #vu8(1 2) #vu8(2 3)
+#vu8(1)   #vu8(2) #vu8(3))
+(let ((b #vu8(1 2 3)))
+  (list (bytevector-slice b 0 2)
+(bytevector-slice b 1)
+(bytevector-slice b 0 1)
+(bytevector-slice b 1 1)
+(bytevector-slice b 2
+
+  (pass-if-exception "immutable flag preserved"
+  exception:wrong-type-arg
+(compile '(begin
+(use-modules (rnrs bytevectors)
+ (rnrs bytevectors gnu))
+
+;; The literal bytevector below is immutable.
+(let ((bv #vu8(1 2 3)))
+  (bytevector-u8-set! (bytevector-slice bv 1) 0 0)))
+
+ ;; Disable optimizations to invoke the full-blown
+ ;; 'scm_bytevector_u8_set_x' procedure, which checks for
+ ;; the SCM_F_BYTEVECTOR_IMMUTABLE flag.
+ #:optimization-level 0
+ #:to 'value))
+
+  (pass-if-equal "slice of f32vector"
+  '(8 2)
+(let* ((v #f32(1.1 1.2 3.14))
+   (s (bytevector-slice v 4)))
+  (and (= (f32vector-ref s 0)
+  (f32vector-ref v 1))
+   (list (bytevector-length s)
+ (f32vector-length s)
+
+  (pass-if-equal "unaligned slice of f32vector"
+  10
+(let* ((v #f32(1.1 1.2 3.14))
+   (s (bytevector-slice v 2)))
+  (and (not (f32vector? s))
+   (bytevector-length s)
+
 
 (with-test-prefix "Arrays"
 
-- 
2.38.1

Reducing memory usage of the linker and assembler

[Ludovic Courtès]

Hello Guilers!

The ‘wip-linker-assembler-memory-consumption’ branch I recently pushed
aims to reduce the allocation rate and memory consumption of the linker
and assembler, as well as the number of bytevector copies.

In 3.0.8, the linker and assembler keep roughly two copies of the final
ELF file in memory: each  contains a ‘bv’ field with its
wire representation, and then ‘link-elf’ allocates a bytevector to
contain the whole ELF file and copies each  bytevector
back into that one.  When you’re running ‘guild compile’, that big
bytevector is immediately passed to ‘put-bytevector’ to write it to
disk.

The branch replaces the ‘bv’ field of  with ‘size’ and
‘writer’, the latter being a procedure that takes a bytevector and
writes to it.

At this point, ‘link-elf’ still allocates one bytevector for each linker
object.  Eventually we could rewrite those “writer” procedures to use
binary I/O primitives instead of expecting a bytevector to write to, and
that way we wouldn’t need those temporary bytevectors.  It’s a bit
tedious to do though.

One big source of memory consumption that remains is the assembler: its
‘buf’ field contains an in-memory copy of the ‘.rtl-text’ section, which
is often a significant portion of the ELF file.  Addressing that would
be more difficult.

On the compilation at -O1 of a big file (‘gnu/packages/python-xyz.scm’
in Guix), I see a 5% reduction of the heap size as measured at the end
of the whole compilation process.  There are other factors before the
linking step that contribute to the peak heap size, which probably
explains why the impact is not higher.

The current set of commits looks like this:

  cdf9a0ac4 * DRAFT linker: Do not store entire ELF in memory when writing to a 
file.
  048651993 * linker: Linker object writer takes a single argument.
  2ac4e5a3d * linker, assembler: Avoid intermediate bytevectors.
  8c8d4bd37 * DRAFT Add 'bytevector-slice'.
  805275882 * linker: Separate effectful part of 'add-elf-objects'.
  8194e8a6e * assembler: Separate effectful part of 'link-docstrs'.
  0a5768add * assembler: Separate effectful part of 'link-frame-maps'.
  f6fb95db8 * assembler: Separate effectful part of 'link-procprops'.
  099cf8f30 * assembler: Separate effectful part of 'link-dynamic-section'.
  43dd2f184 * assembler: Separate effectful part of 'link-symtab'.
  ad4487091 * assembler: Separate 'process-relocs' from 'patch-relocs!'.

There’s no regression.  The main issue worth discussing is the last
commit, which changes ‘link-elf’ to optionally return a procedure
instead of a bytevector.  We also need to discuss ‘bytevector-slice’,
but that can be done separately.

Andy, what do you think of this approach?

Cheers,
Ludo’.

Re: [PATCH v2 00/14] Bindings to *at functions

[Ludovic Courtès]

Ludovic Courtès  skribis:

> I erroneously pushed a variant without copyright lines for yourself.  If
> you confirm that you are the copyright holder, I’ll just readd them as
> you had done.

Actually no, this is all fine.  I guess I just need some rest.  :-)

Ludo’.

Re: [PATCH v2 00/14] Bindings to *at functions

[Ludovic Courtès]

Hi,

Ludovic Courtès  skribis:

> Maxime Devos  skribis:
>
>> This is a v2 of
>> https://lists.gnu.org/archive/html/guile-devel/2021-03/msg0026.html,
>> with a lot more tests, a few less functions and more consistent 
>> documentation.
>> ‘rename-file-at’ has been modified to support #f as one of the two directory
>> arguments, denoting the current working directory.
>>
>> Maxime Devos (14):
>>   Allow file ports in ‘chdir’ when supported.
>>   Allow file ports in ‘readlink’.
>>   Allow file ports in ‘utime’.
>>   Define ‘symlinkat’ wrapper when supported.
>>   Define bindings to ‘mkdirat’ when the C function exists.
>>   Correct documentation of ‘mkdir’ w.r.t. the umask.
>>   Define AT_REMOVEDIR and others when available.
>>   Define a Scheme binding to ‘renameat’ when it exists.
>>   Define a Scheme binding to ‘fchmodat’ when it exists.
>>   Define a Scheme binding to ‘unlinkat’ when it exists.
>>   Define a Scheme binding to ‘fchownat’ when it exists.
>>   Define a Scheme binding to ‘fstatat’ when available.
>>   Define Scheme bindings to ‘openat’ when available.
>>   Update NEWS.
>
> I applied the whole series locally, skimmed over the patches, ran the
> tests, and it all LGTM.

And this is finally pushed as commit
793fb46a1e69fa2156805e4a97b340cf62e096a6!  \o/

I erroneously pushed a variant without copyright lines for yourself.  If
you confirm that you are the copyright holder, I’ll just readd them as
you had done.

If you choose to pursue your copyright assignment process despite the
delays and stumbling blocks, we can drop those lines again.

Thanks!

Ludo’.

Re: Packages depending on (guix build syscalls)

[Ludovic Courtès]

Oops, this was meant for guix-devel, apologies!

Re: Packages depending on (guix build syscalls)

[Ludovic Courtès]

Hi!

Maxime Devos  skribis:

> Additionally, to me it looks like a few of the things in (guix build
> syscalls) could be eventually moved in Guile itself, e.g. mktemp!,
> fdatasync and setxattr, which could eventually reduce the need of
> packages to import (guix build syscalls).

Yes, we should definitely add more of these bindings to Guile.

‘mkdtemp’, for example, was added in Guile 3.0.6, so we no longer need
to use our own variant.

Ludo’.

Packages depending on (guix build syscalls)

[Ludovic Courtès]

Hello Guix!

Quite a few packages depend on (guix build syscalls), starting from
‘ant-bootstrap’ (since commit cded3a759356ff66b7df668bcdbdfa0daf96f4c5
in 2018) up to GNOME-related packages such as ‘mutter’ (commit
d1c2fe248a7a326189fb7dcae64a59ece96251ba a few months ago).

It’s great that we can reuse this module in different contexts!  The
downside is that the module evolves quite often, because it’s a
foundation for Guix System and other things.  As a result, all these
packages get rebuilt every time we change it.

Maybe the only recommendation I would have is that we should make sure
we really need it before having a package deep down the graph depend on
it.  I wouldn’t want us to do ‘staging’ cycles when we need a change in
(guix build syscalls).

Thoughts?

Ludo’.

Re: [PATCH] Avoid 'frame-local-ref' errors when printing backtrace.

[Ludovic Courtès]

Hi,

Andrew Whatson  skribis:

> Ludovic Courtès  wrote:
>>
>> It would be great if you could add a simple test case though, so that
>> the bug doesn’t eventually come back to haunt us.
>>
>> Could you send an updated patch?
>
> Ah yes, getting this covered in a test is on my list.  I had trouble
> writing a reproducer previously, I think the bug might only occur in a
> nested compilation context (top-level error compiling a dependency
> module), but I'll have another go.

Awesome.

>> PS: BTW, it’ll be great to have more patches from you!  :-) To that end,
>> please check out the new Guile copyright policy and let us know what
>> option you’d like to choose:
>> <https://lists.gnu.org/archive/html/guile-devel/2022-10/msg8.html>.
>
> I have already assigned copyright of my work on Guile to the FSF,
> happy for that to remain.

Oh sorry I had overlooked that, perfect!

Thanks,
Ludo’.

Re: Send scheme procedure as callback to foreign thread.

[Ludovic Courtès]

Hi,

Zhu Zihao  skribis:

> In Guile FFI programming, we have procedure->pointer, which makes a
> Scheme procedure a foreign callback. And foreign library call use this
> callback.
>
> However, if this callback is called in a another foreign thread. The scheme
> context is not properly setup, It may cause crash or UB.
>
> IIUC, the procedure invoker "invoke_closure" defined in foreign.c
> doesn't use scm_with_guile. If we use scm_with_guile to make a foreign
> callback, would it be OK to execute Scheme procedure in multithreading 
> context?

Yes, it should be.

Could you come up with a minimal reproducer and maybe even a fix, now
that you likely found the solution?  :-)

Thanks in advance,
Ludo’.

PS: It may be best to use bug-guile to reduce the chances that the issue
gets lost.

Re: bug#58109: simple-format vs (ice-9 format) bug in 3.0.7?

[Ludovic Courtès]

Hi,

Jean Abou Samra  skribis:

> OK, understood. How about adding comments and documentation?

That’s a good idea.  Applied with minor tweaks and a commit log.

Thanks!

Ludo’.

Re: [PATCH v2] Add tests for warning locations.

[Ludovic Courtès]

Hi,

Andrew Whatson  skribis:

> These would have caught .
>
> * test-suite/Makefile.am (SCM_TESTS): Add sample code files.
> * test-suite/tests/tree-il.test ("warnings"): New tests.
> * test-suite/tests/tree-il/unbound-spaces.scm:
> * test-suite/tests/tree-il/unbound-tabs.scm:
> * test-suite/tests/tree-il/unused-variable.scm: Sample code for
> compilation warning tests.

The test looks great, thanks a lot!

One minor nitpick and then we’re ready to go:

> +   (with-test-prefix "location"
> + (define (test-file filename)
> +   (string-append
> +(dirname (current-filename)) "/" filename))
> +
> + (pass-if "unused variable"
> +   (let ((w (call-with-warnings
> + (lambda ()
> +   (compile-file (test-file "tree-il/unused-variable.scm")
> + #:opts %opts-w-unused
> + #:to 'cps)
> + (and (= (length w) 1)
> +  (number? (string-contains (car w) "unused variable `y'"))
> +  (number? (string-contains (car w) 
> "tree-il/unused-variable.scm:2:2")

Can we avoid the separate files and instead do something like:

  (call-with-input-string "  (foo)"
(lambda (port)
  (set-port-filename! port "test-error-location.scm")
  (read-and-compile port #:opts … #:to 'cps)))

?

Thanks,
Ludo’.

Re: [PATCH] Avoid 'frame-local-ref' errors when printing backtrace.

[Ludovic Courtès]

Hi Andrew,

Andrew Whatson  skribis:

> Workaround for .
>
> * module/system/vm/frame.scm (frame-call-representation): Treat a
> binding as "unspecified" if its slot exceeds 'frame-num-locals'.

Yay, great to see that fixed (or almost)!

It would be great if you could add a simple test case though, so that
the bug doesn’t eventually come back to haunt us.

Could you send an updated patch?

Thanks,
Ludo’.

PS: BTW, it’ll be great to have more patches from you!  :-) To that end,
please check out the new Guile copyright policy and let us know what
option you’d like to choose:
.

Re: [PATCH v3] Define SO_RCVTIMEO and SO_SNDTIMEO.

[Ludovic Courtès]

Hi,

Christopher Baines  skribis:

> These are important for reliable networking, since they prevent network
> operations from hanging indefinitely.
>
> * libguile/socket.c (scm_init_socket): Define SO_RCVTIMEO and
> SO_SNDTIMEO.
> (scm_getsockopt, scm_setsockopt): Include SO_RCVTIMEO and SO_SNDTIMEO in
> docstring and handle them.
> * doc/ref/posix.texi (Network Sockets and Communication): Document them.

Applied, thanks!

Ludo’.

Re: Modernizing autotools and assuming C89 C library

[Ludovic Courtès]

Hi Mike,

Mike Gran  skribis:

> On Thu, Sep 15, 2022 at 10:20:26PM -0700, Mike Gran wrote:
>> Hello Guile Devel,
>> 
>> I pushed a git branch: wip-modernize-autotools.
>> 
>> It removes some obsolete tests by presuming we at least have a
>> C89-compliant C library. It also uses LT_INIT for libtool, and removes
>> some macros that autoconf has deprecated.

Sounds good!

> I know patches like this are unlikely to spark enthusiasm
> or review because it is autotools, but, I think presuming a C89
> C library is a safe bet.

No no, I actually find it quite exciting!  :-)

Seriously, it’s good to see old cruft vanish.  I’ve looked at the
commits on that branch and they all LGTM.

Regarding , we’ve actually been using Gnulib’s headers, so most
likely the difference wasn’t relevant anymore.

So: LGTM!

It would be good to have some testing on OSes other than GNU/Linux
eventually (I’m looking at you, Greg ;-)).

Thanks,
Ludo’.

Re: guile.scheme.org

[Ludovic Courtès]

Hi,

Lassi Kortela  skribis:

> With the permission of the Guile maintainers, could we add
> guile.scheme.org redirecting to https://www.gnu.org/software/guile/
> (or any other URL of your choice)?

I’m (very) late to the party but that sounds like a good idea!

Thank you,
Ludo’.

Re: Make the pkgconfig file of Guile relocatable.

[Ludovic Courtès]

Hi,

Zhu Zihao  skribis:

> My idea is change the definition of sitedir, siteccachedir, extensiondir
> use the variable substition of pkg-config instead of autotools.
>
> For example
>
> sitedir=${datadir}/guile/site/@GUILE_EFFECTIVE_VERSION@
> siteccachedir=${libdir}/guile/@GUILE_EFFECTIVE_VERSION@/extensions

Sounds reasonable to me.  Would you like to submit a patch?

Thanks,
Ludo’.

Re: [PATCH] Added srfi-214: flexvectors

[Ludovic Courtès]

The following message is a courtesy copy of an article
that has been posted to gmane.lisp.guile.devel as well.

Hi Vijay,

Sorry for the looong delay!

Overall the patches look great to me.  Below are comments and
suggestions, mostly cosmetic.

Vijay Marupudi  skribis:

> From 42206dec4d5e9ae51665c6e98ef07715b89b12fe Mon Sep 17 00:00:00 2001
> From: Vijay Marupudi 
> Date: Tue, 18 Jan 2022 20:52:08 -0500
> Subject: [PATCH] Added srfi-214: flexvectors
>
> Included code, documentation, and tests

Please use the ChangeLog style for commit messages (see ‘git log’ for
examples); as a welcome gift, we can do it for you though.  :-)


[...]

> +@node Flexvectors
> +@subsection Flexvectors
> +@cindex flexvector

General comments:

  • please leave two spaces after end-of-sentence periods (a convention
eases navigation with Emacs);

  • use @dfn to decorate a term the first time it’s introduced, @code
for identifiers, etc.;

  • limit lines to ~80 columns.

> +Flexvectors are sometimes better known as a 
> @url{https://en.wikipedia.org/wiki/Dynamic_array,dynamic arrays}. This data
> +structure has a wide variety of names in different languages:

Maybe add a first sentence before this one, like: “The @code{(srfi
srfi-214)} module implements @dfn{flexvectors}, a data structure for
mutable vectors with adjustable size.”

> +@itemize @bullet{}

You can drop @bullet{}.

> +@item
> +JavaScript and Ruby call it an array
> +@item
> +Python calls it a list
> +@item
> +Java calls it an ArrayList (and, before that, it was called a Vector)

Add a semicolon at the end of the first two lines, a period for the last
one.  @code{ArrayList}.


[...]

> +++ b/module/srfi/srfi-214.scm
> @@ -0,0 +1,735 @@
> +;;; -*- mode: scheme; coding: utf-8; -*-
> +;;;
> +;;; Copyright (C) Adam Nelson (2020).
> +;;; Copyright (C) Vijay Marupudi (2022).

Nitpick: should be:

  Copyright (C) 2022 …

> +(define-module (srfi srfi-214))
> +
> +(use-modules ((scheme base)
> +  #:prefix r7:)
> + (scheme case-lambda)
> + (scheme write)
> + (srfi srfi-1)
> + (srfi srfi-9)
> + (srfi srfi-9 gnu)
> + (srfi srfi-11)
> + (rnrs io ports))
> +
> +
> +(export ; Constructors

Please use a single ‘define-module’ clause with #:use-module and
#:export lines (this is equivalent but that’s what we conventionally
use and it makes the interface clearer upfront).

Avoid using R6RS and R7RS modules as they pull in a whole lot of stuff.
For example, maybe you can use (ice-9 binary-ports) instead of (rnrs io
ports); maybe you don’t need (scheme …) at all because Guile most likely
has equivalent things built in.

> +make-flexvector flexvector
> +flexvector-unfold flexvector-unfold-right
> +flexvector-copy flexvector-reverse-copy
> +flexvector-append flexvector-concatenate flexvector-append-subvectors
> +

Don’t export  as it would expose implementation details (the
record interface).

> +(define (flexvector . xs)

In general we try to add docstrings for all public top-level
procedures.  Bonus points if you can do that.  :-)

The rest looks great to me!

Could you send an updated patch?

Thanks,
Ludo’.

Re: bug#22608: Module system thread unsafety and .go compilation

[Ludovic Courtès]

Hi!

Maxim Cournoyer  skribis:

> l...@gnu.org (Ludovic Courtès) writes:

[...]

>> For the record, these issues should be fixed in Guile 2.2.4:
>>
>> 533e3ff17 * Serialize accesses to submodule hash tables.
>> 46bcbfa56 * Module import obarrays are accessed in a critical section.
>> 761cf0fb8 * Make module autoloading thread-safe.
>>
>> ‘guix pull’ now defaults to 2.2.4, so we’ll see if indeed those crashes
>> disappear.
>
> I think we haven't seen these in the last 4 years!  We still have
> references to https://bugs.gnu.org/15602 in our code base though;
> although the upstream issue appears to have been fixed.  Could we remove
> the workarounds now?

The module thread-safety issue discussed here appears to be done.

However the workarounds for <https://bugs.gnu.org/15602> must remain:
that specific issue is still there.

Thanks,
Ludo’.

Re: Relaxing the copyright assignment policy

[Ludovic Courtès]

Hi Maxime,

Maxime Devos  skribis:

> On 06-10-2022 22:18, Ludovic Courtès wrote:
>> Hello Guilers!
>> [...]
>> New contributors are encouraged to assign copyright to the FSF by
>> emailing them one of the forms at
>> <https://git.savannah.gnu.org/cgit/gnulib.git/plain/doc/Copyright/>,
>> especially if they intend to contribute significantly going forward.
>> New contributors may instead choose to not assign copyright to the
>> FSF.
>
> What about the following third situation: the contributor doesn't mind
> assigning Guile-related copyright to the FSF, but the assignment
> process can take or is taking a long time or there are some
> disagreements with the assignment contract to resolve (which might not
> even be possible in the end), so
>
>   (1) first the contributor sends the patches without copyright
>   assignment
>   (2) then they are applied (after review, revisions, etc.)
>   (3) once/if the assignment completes, the copyright headers in Guile
>   are adjusted appropriately.
>
> -- this avoids the time delay, while at the same time hopefully
>eventually doing the assignment.
>
> Would such a situation be accepted? (It doesn't quite fit the two
> mentioned options.)

This is getting complicated.  :-)

A contributor could choose to not assign copyright at first for whatever
reason, and later change their mind and assign copyright for all past
changes.  In that case, all the contributed code would now be
FSF-copyrighted, and we’d remove the corresponding copyright lines.

Obviously we can only do that for *all* past changes and when there’s a
single copyright holder, otherwise it becomes hard to keep track of
things.

I encourage people to choose upfront though to reduce the burden on
committers.

Thanks,
Ludo’.

PS: We can discuss your own situation off-list if you want.

Relaxing the copyright assignment policy

[Ludovic Courtès]

Hello Guilers!

Until now, we required copyright for “legally significant¹”
contributions to Guile to be assigned to the Free Software Foundation
(FSF).  This requirement did not exist for code not initially written
for Guile—e.g., (ice-9 match), sxml-match, etc.  The purported advantage
of copyright assignment is described at
.

Your unreachable-but-nonetheless-friendly co-maintainers, Andy Wingo and
myself, came to the conclusion that the cost/benefit ratio of mandatory
copyright assignment is not good for Guile.  Like other projects such as
GCC², we have decided to relax that policy.

Nothing changes for contributors who have already assigned copyright on
future changes to Guile to the FSF.

New contributors are encouraged to assign copyright to the FSF by
emailing them one of the forms at
,
especially if they intend to contribute significantly going forward.

New contributors may instead choose to not assign copyright to the FSF.
In that case, copyright on the new code is held by the contributor
themself or, possibly, by their employer—contributors who choose this
option are expected to clarify which of these two applies to their
situation.  A copyright line for the new copyright holder is added to
files modified in a “legally significant” way³.

Guile remains under the same license, the GNU Lesser General Public
License, version 3 or any later version; contributions are expected
under the same terms.

We hope this to be one of the changes that will make it easier to
contribute to Guile.

Let us know if you have any questions, and enjoy the good hack!

Ludo’ & Andy.

¹ https://www.gnu.org/prep/maintain/maintain.html#Legally-Significant
² https://gcc.gnu.org/pipermail/gcc/2021-June/236182.html
³ https://www.gnu.org/prep/maintain/maintain.html#Copyright-Notices


signature.asc
Description: PGP signature

Re: bug#58109: simple-format vs (ice-9 format) bug in 3.0.7?

[Ludovic Courtès]

Hi,

Jean Abou Samra  skribis:

> Uh, at the end of module/ice-9/format.scm, there is
>
> ;; Thanks to Shuji Narazaki
> (module-set! the-root-module 'format format)
>
> which dates back to
>
> commit 14469b7c69feb0f2c5b8a093f19fe2a548b31c5b
> Author: Greg J. Badros 
> Date:   Thu Jan 20 20:58:30 2000 +

[...]

> This probably predates #:replace and could be removed now, right?

Yes, it could be removed, but probably not before the 4.0 series.

The ‘-Wformat’ warning introduced sometime in the 2.0 or 2.2 series
prepared for that removal by warning about simple-format/format
mismatches, but there’s probably still code out there that assumes
‘format’ is the full-blown ‘format’, even when (ice-9 format) is not
explicitly imported.

Thanks,
Ludo’.

Re: [PATCH] download: Do not wrap TLS port on GnuTLS >= 3.7.7.

[Ludovic Courtès]

Hi,

Maxime Devos  skribis:

> On 02-08-2022 09:59, Ludovic Courtès wrote:
>>>> +  (if (module-defined? (resolve-interface '(gnutls))
>>>> +   'set-session-record-port-close!) ;GnuTLS >= 
>>>> 3.7.7
>>> resolve-module (and presumably also sets #:ensure #t by default, which
>>> sometimes causes 'module not found' messages to be replaced by
>>> 'unbound variable', which I don't think is useful behaviour, can
>>> #:ensure be set to #false?
>> This is unnecessary: see the ‘load-gnutls’ mechanism there.  The idiom
>> above is already used in a couple of places.
>
> I have looked at the 'load-gnutls' procedure, but I do not see how it
> avoids the issue I mentioned (*).

[...]

> (*) The autoloading of gnutls in load-gnutls avoids compilation errors
> when gnutls is absent, but by the way it does it, it causes the module
> to be registered as 'it exists' even when it doesn't, so the
> information in the module system of Guix becomes incorrect.

I understand what you’re saying (I’m quite familiar with Guile’s module
system :-) and I do agree that #:ensure #t can lead to bad surprises),
but I don’t think this is correct:

--8<---cut here---start->8---
scheme@(guile-user)> (resolve-interface '(xxx))
ice-9/boot-9.scm:1685:16: In procedure raise-exception:
no code for module (xxx)

Entering a new prompt.  Type `,bt' for a backtrace or `,q' to continue.
scheme@(guile-user) [1]> ,q
scheme@(guile-user)> (resolve-module '(xxx) #f #:ensure #f)
$1 = #f
--8<---cut here---end--->8---

This is because ‘resolve-interface’ does (resolve-module … #:ensure #f).

Does that make sense?

Thanks,
Ludo’.

Re: [bug#56867] [PATCH] download: Do not wrap TLS port on GnuTLS >= 3.7.7.

[Ludovic Courtès]

Hi,

Thiago Jung Bauermann  skribis:

> Ludovic Courtès  writes:
>
>> First, I noticed that GnuTLS doesn’t implement ‘write_wait_fd’, only
>> ‘read_wait_fd’ (not sure how problematic that is):
>>
>> scheme@(guile-user)> ,use(web client)
>> scheme@(guile-user)> (define p (open-socket-for-uri "https://guix.gnu.org";))
>> scheme@(guile-user)> ((@@ (ice-9 suspendable-ports) wait-for-writable) p)
>> ice-9/boot-9.scm:1685:16: In procedure raise-exception:
>> In procedure write_wait_fd: unimplemented
>>
>> Entering a new prompt.  Type `,bt' for a backtrace or `,q' to continue.
>> scheme@(guile-user) [1]> ,q
>> scheme@(guile-user)> ,use(gnutls)
>> scheme@(guile-user)> (gnutls-version)
>> $1 = "3.7.7"
>> scheme@(guile-user)> ((@@ (ice-9 suspendable-ports) wait-for-readable) p)
>> $2 = 1
>
> This occasionally causes problems when fetching substitutes, as can be
> seen in bug #56005 (during substitution: write_wait_fd: unimplemented).

Oh, I have not seen it but it’s weird: (guix scripts substitute) doesn’t
use O_NONBLOCK sockets, so I don’t get how it can hit that.  Needs
investigation…

Thanks,
Ludo’.

Re: bug#56867: [PATCH] download: Do not wrap TLS port on GnuTLS >= 3.7.7.

[Ludovic Courtès]

Ludovic Courtès  skribis:

> The custom input/output port wrapping the TLS session record port would
> introduce overhead, and it would also prevent its uses in a non-blocking
> context--e.g., with Fibers.  The port close mechanism added in GnuTLS
> 3.7.7 allows us to get rid of that wrapper.
>
> * guix/build/download.scm (wrap-record-port-for-gnutls<3.7.7): New
> procedure, with code formerly in 'tls-wrap'.
> (tls-wrap): Check for 'set-session-record-port-close!' and use it when
> available; otherwise call 'wrap-record-port-for-gnutls<3.7.7'.

I synchronized Guile's copy of this code:

  317b06bf8 web: 'tls-wrap' retries handshake upon non-fatal errors.
  c01ca10b3 web: Do not wrap TLS port on GnuTLS >= 3.7.7.

I realized that’s not enough to make it possible to use non-blocking
ports though.

First, I noticed that GnuTLS doesn’t implement ‘write_wait_fd’, only
‘read_wait_fd’ (not sure how problematic that is):

--8<---cut here---start->8---
scheme@(guile-user)> ,use(web client)
scheme@(guile-user)> (define p (open-socket-for-uri "https://guix.gnu.org";))
scheme@(guile-user)> ((@@ (ice-9 suspendable-ports) wait-for-writable) p)
ice-9/boot-9.scm:1685:16: In procedure raise-exception:
In procedure write_wait_fd: unimplemented

Entering a new prompt.  Type `,bt' for a backtrace or `,q' to continue.
scheme@(guile-user) [1]> ,q
scheme@(guile-user)> ,use(gnutls)
scheme@(guile-user)> (gnutls-version)
$1 = "3.7.7"
scheme@(guile-user)> ((@@ (ice-9 suspendable-ports) wait-for-readable) p)
$2 = 1
--8<---cut here---end--->8---

Second, ‘open-socket-for-uri’ creates a blocking socket and uses that as
the backing file descriptor of the TLS session.

We’d need a way to pass flags for the ‘socket’ call made by
‘open-socket-for-uri’ so we can pass O_NONBLOCK, maybe as show below:

diff --git a/module/web/client.scm b/module/web/client.scm
index a08c4203c..9273a45ad 100644
--- a/module/web/client.scm
+++ b/module/web/client.scm
@@ -320,7 +320,8 @@ host name without trailing dot."
   (read-response port))
 
 (define* (open-socket-for-uri uri-or-string
-  #:key (verify-certificate? #t))
+  #:key (verify-certificate? #t)
+  (flags 0))
   "Return an open input/output port for a connection to URI-OR-STRING.
 When VERIFY-CERTIFICATE? is true, verify HTTPS server certificates."
   (define uri
@@ -373,10 +374,18 @@ When VERIFY-CERTIFICATE? is true, verify HTTPS server certificates."
 (when (and https? (current-https-proxy))
   (setup-http-tunnel s uri))
 
-(if https?
-(tls-wrap s (uri-host uri)
-  #:verify-certificate? verify-certificate?)
-s)))
+(let ((port (if https?
+(tls-wrap s (uri-host uri)
+  #:verify-certificate? verify-certificate?)
+s)))
+  (unless (zero? flags)
+;; FLAGS might contain O_NONBLOCK.  Thus, set it as a last step
+;; because 'handshake' otherwise throws an exception for
+;; GNUTLS_E_AGAIN.
+(let ((initial-flags (fcntl s F_GETFL)))
+  (fcntl s F_SETFL (logior initial-flags flags
+
+  port)))
 
 (define (extend-request r k v . additional)
   (let ((r (set-field r (request-headers)

… which lets us do that:

--8<---cut here---start->8---
scheme@(guile-user)> ,use(web client)
scheme@(guile-user)> (define p (open-socket-for-uri "https://guix.gnu.org"; 
#:flags O_NONBLOCK))
scheme@(guile-user)> (http-get "https://guix.gnu.org"; #:port p)
--8<---cut here---end--->8---

Thoughts?

Ludo’.

Re: bug#56867: [PATCH] download: Do not wrap TLS port on GnuTLS >= 3.7.7.

[Ludovic Courtès]

Ludovic Courtès  skribis:

> The custom input/output port wrapping the TLS session record port would
> introduce overhead, and it would also prevent its uses in a non-blocking
> context--e.g., with Fibers.  The port close mechanism added in GnuTLS
> 3.7.7 allows us to get rid of that wrapper.
>
> * guix/build/download.scm (wrap-record-port-for-gnutls<3.7.7): New
> procedure, with code formerly in 'tls-wrap'.
> (tls-wrap): Check for 'set-session-record-port-close!' and use it when
> available; otherwise call 'wrap-record-port-for-gnutls<3.7.7'.
> ---
>  guix/build/download.scm | 102 +---
>  1 file changed, 54 insertions(+), 48 deletions(-)

Pushed as Guix commit dd573ceea73295c7a872088ecd91e5f0fd74bf2b.

Ludo’.

Re: [PATCH] download: Do not wrap TLS port on GnuTLS >= 3.7.7.

[Ludovic Courtès]

Hi,

Maxime Devos  skribis:

> On 01-08-2022 11:07, Ludovic Courtès wrote:

[...]

>> +  (define (read! bv start count)
>> +(define read
>> +  (catch 'gnutls-error
>> +(lambda ()
>> +  (get-bytevector-n! record bv start count))
>> +(lambda (key err proc . rest)
>> +  ;; When responding to "Connection: close" requests, some servers
>> +  ;; close the connection abruptly after sending the response body,
>> +  ;; without doing a proper TLS connection termination.  Treat it as
>> +  ;; EOF.  This is fixed in GnuTLS 3.7.7.
>> +  (if (eq? err error/premature-termination)
>> +  the-eof-object
>> +  (apply throw key err proc rest)
>
> Objection: 'catch' makes the backtrace part happening inside the
> 'get-bytevector-n!' disappear, because it is unwinding, as has been
> noted a few times (in different contexts) by Attila Lendvai and me. 
> Maybe use 'guard' with an appropriate condition instead?

This code was already there and has just been moved around.  (It’s also
code that will no longer be used going forward.)

>> +  (if (module-defined? (resolve-interface '(gnutls))
>> +   'set-session-record-port-close!) ;GnuTLS >= 3.7.7
>
> resolve-module (and presumably also sets #:ensure #t by default, which
> sometimes causes 'module not found' messages to be replaced by
> 'unbound variable', which I don't think is useful behaviour, can
> #:ensure be set to #false?

This is unnecessary: see the ‘load-gnutls’ mechanism there.  The idiom
above is already used in a couple of places.

Thanks for your feedback!

Ludo’.

Re: bug#56867: [PATCH] download: Do not wrap TLS port on GnuTLS >= 3.7.7.

[Ludovic Courtès]

Ludovic Courtès  skribis:

> The custom input/output port wrapping the TLS session record port would
> introduce overhead, and it would also prevent its uses in a non-blocking
> context--e.g., with Fibers.  The port close mechanism added in GnuTLS
> 3.7.7 allows us to get rid of that wrapper.

And here’s the GnuTLS 3.7.7 package to test it; you need to make sure to
have 3.7.7 on your load path, for instance by running:

  ./pre-inst-env guix shell -D guix guile gnutls@3.7.7

Ludo’.

diff --git a/gnu/packages/tls.scm b/gnu/packages/tls.scm
index 1ee5400a9c..33c93b7a5b 100644
--- a/gnu/packages/tls.scm
+++ b/gnu/packages/tls.scm
@@ -329,6 +329,21 @@ (define-public gnutls
 (properties '((ftp-server . "ftp.gnutls.org")
   (ftp-directory . "/gcrypt/gnutls")
 
+(define-public gnutls-latest
+  (package
+(inherit gnutls)
+(version "3.7.7")
+(source (origin
+  (method url-fetch)
+  (uri (string-append "mirror://gnupg/gnutls/v"
+  (version-major+minor version)
+  "/gnutls-" version ".tar.xz"))
+  (patches (search-patches "gnutls-skip-trust-store-test.patch"
+   "gnutls-cross.patch"))
+  (sha256
+   (base32
+"01i1gl15k6qwvxmxx0by1mn9nlmcmym18wdpm7dn9awfsp8474dy"))
+
 (define-public gnutls/guile-2.0
   ;; GnuTLS for Guile 2.0.
   (package/inherit gnutls

[PATCH] download: Do not wrap TLS port on GnuTLS >= 3.7.7.

[Ludovic Courtès]

The custom input/output port wrapping the TLS session record port would
introduce overhead, and it would also prevent its uses in a non-blocking
context--e.g., with Fibers.  The port close mechanism added in GnuTLS
3.7.7 allows us to get rid of that wrapper.

* guix/build/download.scm (wrap-record-port-for-gnutls<3.7.7): New
procedure, with code formerly in 'tls-wrap'.
(tls-wrap): Check for 'set-session-record-port-close!' and use it when
available; otherwise call 'wrap-record-port-for-gnutls<3.7.7'.
---
 guix/build/download.scm | 102 +---
 1 file changed, 54 insertions(+), 48 deletions(-)

Hello!

I'll land a similar change in Guile's (web client) module afterwards
if there are no objections.

Ludo'.

diff --git a/guix/build/download.scm b/guix/build/download.scm
index 41583e8143..de094890b3 100644
--- a/guix/build/download.scm
+++ b/guix/build/download.scm
@@ -245,6 +245,54 @@ (define (print-tls-certificate-error port key args 
default-printer)
 (set-exception-printer! 'tls-certificate-error
 print-tls-certificate-error)
 
+(define (wrap-record-port-for-gnutls<3.7.7 record port)
+  "Return a port that wraps RECORD to ensure that closing it also closes PORT,
+the actual socket port, and its file descriptor.  Make sure it does not
+introduce extra buffering (custom ports are buffered by default as of Guile
+3.0.5).
+
+This wrapper is unnecessary with GnuTLS >= 3.7.7, which can automatically
+close SESSION's file descriptor when RECORD is closed."
+  (define (read! bv start count)
+(define read
+  (catch 'gnutls-error
+(lambda ()
+  (get-bytevector-n! record bv start count))
+(lambda (key err proc . rest)
+  ;; When responding to "Connection: close" requests, some servers
+  ;; close the connection abruptly after sending the response body,
+  ;; without doing a proper TLS connection termination.  Treat it as
+  ;; EOF.  This is fixed in GnuTLS 3.7.7.
+  (if (eq? err error/premature-termination)
+  the-eof-object
+  (apply throw key err proc rest)
+
+(if (eof-object? read)
+0
+read))
+  (define (write! bv start count)
+(put-bytevector record bv start count)
+(force-output record)
+count)
+  (define (get-position)
+(port-position record))
+  (define (set-position! new-position)
+(set-port-position! record new-position))
+  (define (close)
+(unless (port-closed? port)
+  (close-port port))
+(unless (port-closed? record)
+  (close-port record)))
+
+  (define (unbuffered port)
+(setvbuf port 'none)
+port)
+
+  (unbuffered
+   (make-custom-binary-input/output-port "gnutls wrapped port" read! write!
+ get-position set-position!
+ close)))
+
 (define* (tls-wrap port server #:key (verify-certificate? #t))
   "Return PORT wrapped in a TLS connection to SERVER.  SERVER must be a DNS
 host name without trailing dot."
@@ -317,55 +365,13 @@ (define (log level str)
   (apply throw args
 
 (let ((record (session-record-port session)))
-  (define (read! bv start count)
-(define read
-  (catch 'gnutls-error
-(lambda ()
-  (get-bytevector-n! record bv start count))
-(lambda (key err proc . rest)
-  ;; When responding to "Connection: close" requests, some
-  ;; servers close the connection abruptly after sending the
-  ;; response body, without doing a proper TLS connection
-  ;; termination.  Treat it as EOF.
-  (if (eq? err error/premature-termination)
-  the-eof-object
-  (apply throw key err proc rest)
-
-(if (eof-object? read)
-0
-read))
-  (define (write! bv start count)
-(put-bytevector record bv start count)
-(force-output record)
-count)
-  (define (get-position)
-(port-position record))
-  (define (set-position! new-position)
-(set-port-position! record new-position))
-  (define (close)
-(unless (port-closed? port)
-  (close-port port))
-(unless (port-closed? record)
-  (close-port record)))
-
-  (define (unbuffered port)
-(setvbuf port 'none)
-port)
-
   (setvbuf record 'block)
-
-  ;; Return a port that wraps RECORD to ensure that closing it also
-  ;; closes PORT, the actual socket port, and its file descriptor.
-  ;; Make sure it does not introduce extra buffering (custom ports
-  ;; are buffered by default as of Guile 3.0.5).
-  ;; XXX: This wrapper would be unnecessary if GnuTLS could
-  ;; automatically close SESSION's file descriptor when RECORD is
-  ;; closed, but that doesn't seem to be possible currently (as of
-  ;; 3.6.9).
-  (unbuffered
-   (make-custom-binar

Re: mmap for guile

[Ludovic Courtès]

Hi,

Maxime Devos  skribis:

> Ludovic Courtès schreef op ma 04-07-2022 om 12:09 [+0200]:
>> I don’t think the optimizer makes any such assumption, except for
>> literal bytevectors.
>
> It _does_ assume that bytevector lengths don't change:

[...]

> As can be seen in the above output, it first determines
> the length of the bytevector and then compares it again the index
> (for the index check) and then actually reads th byte
> so it assumes that the length doesn't change in-between -- it's not an
> atomic operation!

Oh right.

> Even if it didn't, being able to assume bytevector lengths don't change
> is important for optimising code like (begin (bytevector-ref bv 9000) 
> (bytevector-ref 8999) ...)),
> to optimise out many range checks, though I don't know if Guile currently
> does that.

Indeed.

Ludo’.

Re: mmap for guile

[Ludovic Courtès]

Hi,

Greg Troxel  skribis:

> Ludovic Courtès  writes:
>
>> Besides what Maxime points out, some more superficial issues:
>>
>>   • In documentation, please refer to the relevant glibc section instead
>> of “See man page” (info "(libc) Memory-mapped I/O").
>>
>>   • Please update doc/ref with a section on memory-mapped I/O.
>>
>>   • Make sure to follow the GNU coding in C: space before opening paren,
>> braces on a line of their own, etc.
>
> I have been meaning to try to build this under NetBSD, to check
> portability. I think the mmap code should by default rely only on what
> POSIX guarantees:
>   https://pubs.opengroup.org/onlinepubs/9699919799/functions/mmap.html

Agreed.

> As for referring to glibc, that reference only resolves on GNU/Linux
> systems, whereas any POSIX system ought to have an mmap man page, so it
> would be nice not to drop the man page ref, esp. as it grounds the
> implementation as being about the POSIX interface.

The manual can mention the man page too, sure, but note that the manual
already refers to glibc’s for all things libc.

Ludo’.

Re: [PATCH 1/2] web: Handle ending CRLF (\r\n) for chunked input and output ports.

[Ludovic Courtès]

Hi Chris,

Christopher Baines  skribis:

> The chunked transfer encoding specifies the chunked body ends with
> CRLF. This is in addition to the CRLF at the end of the last chunk, so
> there should be CRLF twice at the end of the chunked body:
>
>   https://datatracker.ietf.org/doc/html/rfc2616#section-3.6.1
>
> * module/web/http.scm (make-chunked-input-port): Read two extra bytes at
> the end of the chunked input.
> (make-chunked-output-port): Write the missing \r\n when closing the
> port.
> * test-suite/tests/web-http.test (chunked encoding): Add missing \r\n to
> test data.

[...]

> This port is of limited use if it cannot be used reliably. Rather than
> behaving as if the input has finished when it ends unexpectedly, instead
> raise an exception.
>
> * module/web/http.scm (make-chunked-input-port): Raise an exception on
> premature termination.
> (&chunked-input-ended-prematurely): New exception type.
> (chunked-input-ended-prematurely-error?): New procedure.
> * test-suite/tests/web-http.test (pass-if-named-exception): Rename to
> pass-if-named-exception.
> (pass-if-named-exception): New syntax.
> ("Exception on premature chunk end"): New test for this behaviour.

Applied, thanks!

Are there servers out there where you would hit this bug?  We were
“lucky” not to notice before I guess.

Ludo’.

Re: [PATCH v2 00/14] Bindings to *at functions

[Ludovic Courtès]

Hi Maxime,

First, apologies for the embarrassingly-long silence…

Maxime Devos  skribis:

> This is a v2 of
> https://lists.gnu.org/archive/html/guile-devel/2021-03/msg0026.html,
> with a lot more tests, a few less functions and more consistent documentation.
> ‘rename-file-at’ has been modified to support #f as one of the two directory
> arguments, denoting the current working directory.
>
> Maxime Devos (14):
>   Allow file ports in ‘chdir’ when supported.
>   Allow file ports in ‘readlink’.
>   Allow file ports in ‘utime’.
>   Define ‘symlinkat’ wrapper when supported.
>   Define bindings to ‘mkdirat’ when the C function exists.
>   Correct documentation of ‘mkdir’ w.r.t. the umask.
>   Define AT_REMOVEDIR and others when available.
>   Define a Scheme binding to ‘renameat’ when it exists.
>   Define a Scheme binding to ‘fchmodat’ when it exists.
>   Define a Scheme binding to ‘unlinkat’ when it exists.
>   Define a Scheme binding to ‘fchownat’ when it exists.
>   Define a Scheme binding to ‘fstatat’ when available.
>   Define Scheme bindings to ‘openat’ when available.
>   Update NEWS.

I applied the whole series locally, skimmed over the patches, ran the
tests, and it all LGTM.

I think the strategy to accept a string or a port where applicable
(utime, readlink) makes sense and is consistent with existing
interfaces; the new *at procedures look fine as well (there’s a naming
scheme discrepancy with ‘rename-file-at’ and ‘delete-file-at’, but I
think it’s fine: it’s just an evolution of the discrepancy that was
already there with ‘delete-file’ and ‘rename-file’.)

Copyright for Guile code is assigned to the FSF.  I’d like to offer you
to do the same if that’s an option for you (I’ll send you the details
off-list.)  This would add another delay, but hopefully a short one.

Thank you!

Ludo’.

Re: A lexical use-modules?

[Ludovic Courtès]

Hi,

Maxime Devos  skribis:

> I wondered if some kind of 'lexical use-modules' was possible, with
> sufficient macroology and module reflection, and it looks like it is:

I agree it would be useful.

Just yesterday wrote this (my goal here was to allow dynamic module
loading based on some condition);

--8<---cut here---start->8---
(define-syntax with-modules
  (syntax-rules ()
"Dynamically load the given MODULEs at run time, making the chosen
bindings available within the lexical scope of BODY."
((_ ((module #:select (bindings ...)) rest ...) body ...)
 (let* ((iface (resolve-interface 'module))
(bindings (module-ref iface 'bindings))
...)
   (with-modules (rest ...) body ...)))
((_ () body ...)
 (begin body ...
--8<---cut here---end--->8---

… which can be used like this:

--8<---cut here---start->8---
(with-modules (((fibers) #:select (spawn-fiber sleep))
   ((fibers channels)
#:select (make-channel put-message get-message)))
  (let ((channel (make-channel)))
(spawn-fiber
  …)))
--8<---cut here---end--->8---

Unlike ‘use-modules’, its clearly limited to the lexical scope of its
body.

IWBN to have a similar functionality in Guile proper, and it could
probably be made more efficient.

Ludo’.

Re: [PATCH] Test -flto on both compiler and linker

[Ludovic Courtès]

Hi Sergei,

Sergei Trofimovich  skribis:

> Before the change ./configure incorrectly enabled -flto on toolchains
> that support -flto on compiler side but don't support -flto on linker
> side. This caused incorrect type size detection on nixpkgs' Darwin:
>
>  configure:54594: checking size of size_t
>  configure:54600: clang -std=gnu11 -o conftest -g -O2 -flto   conftest.c  >&5
>  ld: warning: ignoring file 
> /private/tmp/nix-build-guile-3.0.8.drv-0/conftest-00e93d.o,
>building for macOS-x86_64 but attempting to link with file built
>for unknown-unsupported file format ( 0xDE 0xC0 0x17 0x0B 0x00 0x00 0x00 
> 0x00 0x14 0x00 0x00 0x00 0x80 0x1A 0x00 0x00 )
>  Undefined symbols for architecture x86_64:
>"_main", referenced from:
>   implicit entry/start for main executable
>  ld: symbol(s) not found for architecture x86_64
>  clang-11: error: linker command failed with exit code 1 (use -v to see 
> invocation)
>
> Taken from 
> https://github.com/NixOS/nixpkgs/pull/160051#issuecomment-1046105041
>
> The change makes sure -flto support tests basic support of just for
> object file generation but for linker as well.
>
> * configure.ac: use AC_LINK_IFELSE instead of AC_COMPILE_IFELSE.

Applied, thanks!

Ludo’.

Re: Maintenance and future of Guile

[Ludovic Courtès]

Hi,

Olivier Dion  skribis:

> On Fri, 17 Dec 2021, Ludovic Courtès  wrote:
>> Hi,
>>
>> Olivier Dion  skribis:
>>
>>> I would also like to contribute in some meaningful way.  In what way
>>> someone with none wizard knowledge of Scheme can contribute the most to the
>>> project?
>>
>> Triage of bugs and patches is always welcome I guess, and communicating
>> what needs to be applied/addressed first to whoever can actually commit
>> it.  That’s one possible way to help.
>
> Where can this be done?  I know that Guix is using debbugs, but do Guile
> does the same or is it all tracked on Savannah?

It’s happening on debbugs.gnu.org as well.  Debbugs this is easier to
work with via Emacs debbugs.el, which is nice if you already use Emacs
but otherwise unfortunate.

> Also, any way to help on the C side?

There isn’t much happening on the C side.  Maxime Devos submitted
patches adding bindings for openat(2) and friends that are unfortunately
still pending review.

Patches that add POSIX bindings and similar to libguile should in
general be relatively easy to review (though the openat(2) one may be
trickier because it’s a central functionality and we’d rather get the
interface right.)

Thanks,
Ludo’.

Re: Maintenance and future of Guile

[Ludovic Courtès]

Hi,

Olivier Dion  skribis:

> I would also like to contribute in some meaningful way.  In what way
> someone with none wizard knowledge of Scheme can contribute the most to the
> project?

Triage of bugs and patches is always welcome I guess, and communicating
what needs to be applied/addressed first to whoever can actually commit
it.  That’s one possible way to help.

Thanks,
Ludo’.

Re: Maintenance and future of Guile

[Ludovic Courtès]

Hi,

Blake Shaw  skribis:

> Ludovic Courtès  writes:
>
>> If someone is interested, please get in touch with us!
>>
>> While Andy focuses on major improvements to the compiler and VM with a
>> long-term vision, I think it would be great to also have people on the
>> maintainer team focusing on more day-to-day operations: incremental
>> improvements, bug fixes, etc.  I think we’re lucky that there’ve been a
>> number of talented contributors chiming in over the last couple of
>> years; let’s take advantage of this, at last!
>>
>
> Just want to chime in as a new Guile to say that while I'm still
> learning, I'm happy to work on any "chores" if there is guidance on how
> to do so. exploring guile has been half the fun so far.
>
> I know CHICKEN has a team of "janitors"; perhaps Guile could benefit
> from adopting a similar approach? 

Surely.  For my part, while I won’t pretend to do actual work myself, I
can dedicate time to mentor someone willing to step up.

Ludo’.

Re: Maintenance and future of Guile

[Ludovic Courtès]

Hello Guilers,

Jean Abou Samra  skribis:

> I see a number of similar cases on the mailing list.
>
> I understand the cost of reviewing and I know that
> nobody is entitled to anything in the free software
> world. However, I would like to voice the concern that
> Guile's maintenance is not scaling, and the project
> cannot attract new contributors if patches do not
> make it to the main branch. If the current maintainers
> need to drop their activity, it would be nice if
> they could share maintainership so that at least
> bug fixes can be applied.

I agree and I apologize for the unpleasant situation: it’s clear that I
haven’t been up to the task for several years already.  I’ll discuss
with Andy how I can formally leave the seat to someone with more energy,
whom I’ll be happy to help learn practices and conventions.

If someone is interested, please get in touch with us!

While Andy focuses on major improvements to the compiler and VM with a
long-term vision, I think it would be great to also have people on the
maintainer team focusing on more day-to-day operations: incremental
improvements, bug fixes, etc.  I think we’re lucky that there’ve been a
number of talented contributors chiming in over the last couple of
years; let’s take advantage of this, at last!

Thanks,
Ludo’.

Re: Guile 3 and wip-elisp/Emacs

[Ludovic Courtès]

Hello!

Christine Lemmer-Webber  skribis:

> I've pushed this as origin/wip-elisp-rebased.  I actually rebased it
> again, making some naming adjustments for myself and a couple of
> adjustments having talked to Robin.
>
> If nobody objects, I'd like to merge this into main.  Maintainers, if
> you have any objections, speak now or forever hold these commits!

I haven’t looked at the branch, but I think it’s great to see it live
and it’s great if it can be merged!

Some things to pay attention to before merging to ‘main’, since it
corresponds to the current 3.0 stable branch:

  • Make sure no backward incompatibilities are introduced in
preexisting modules;

  • Make sure the ABI of libguile-3.0.so and that of public modules
is unchanged, or is changed compatibly;

  • Make sure there are reasonable tests and doc so it can be maintained
(and used!) going forward;

  • Robin has a copyright assignment on file, so we should be fine
(whether we’ll keep doing copyright assignment for Guile is still
unclear, but we can discuss that separately).

I think we should also wait for a green light from Andy.

Thanks Gregg, Christine, Robin, and everyone involved!

Ludo’.

Re: new maintainers for guile libs

[Ludovic Courtès]

Hi Mike,

(Catching up on email…)

Mike Gran  skribis:

> Well 2021 has been a strange time for me as it has been for many.  At
> the beginning of 2021, I had the free time and the good health that
> allowed me to imagine being a real contributer again, but, stuff has
> changed.

I’m sad to read this; I hope you’ll recover soon and feel energized,
including to hack the good hack.

> So there are a few libraries in the Guile ecosystem I've worked on,
> and I'm hoping someone else might be interested in owning them
> henceforth.
>
> Roughly in order of maintainability, from simple to complex, these are
>
> - guile-aspell: a very simple binding to the Aspell spellcheck
>   library. This would be an easy task, since it is a Guile-only
>   binding with no C, and aspell rarely changes
> - guile-curl: a C-based but straight-forward binding the the cURL API.
> - guile-ncurses: a C-based binding for NCurses, libpanel, and libmenu.  It
>   is a bit convoluted bcause the memory model for ncurses it iself
>   quite strange.
>
> Then there a couple of very obscure and little used libraries that I could
> punt on, but, I don't really expect any interest
>
> - guile-plotutils: draw plots in Guile using the plotutils backend
> - zile-on-guile: a version of the zile editor, replacing its tiny LISP
>   interpreter with Guile.
>
> Guile-GI is a special case because at this point, I'm probably not the
> prinmary contributor. But it could use more contributers and could use
> a discussion about how to move it forward. I still hope to contribute
> as I can.

I won’t commit to anything (I’d do a poor job), but these are all useful
pieces of software and I hope someone picks them up!

Thanks,
Ludo’.